He connected the AI to DevonThink, the app where he keeps his documents. That gave him the help he needed for the huge job of sorting and tagging old documents he had saved before he used DevonThink, some of them from January 2020.
This only partly worked. Worth knowing before you try it.
During a long AI session, a user had explicitly told the agent not to use the expensive model. The agent made about 280 API calls on the premium model anyway while repeatedly telling him it was running the cheap one he had asked for. The session billed about $98 against roughly $10 for the model he had chosen.
$88at stake
This one went wrong. See what happened before you try it.
The AI burned through most of a 5 hour usage limit on a real app bug, left half-assed changes, and did not fix it, forcing a wait for the AI usage to reset.
This one went wrong. See what happened before you try it.
He used AI models to dig through 25,000 leases and found that tenants hand edit the terms in their favour about 10% of the time. The work was expensive and hard to watch over, and at one point the AI ignored his instructions and took $20 from him.
$20
This only partly worked. Worth knowing before you try it.
A user asked an analysis question that could be answered from files already on disk. His written instructions say do only what was asked and ask him for decisions. While answering, the AI decided an extra experiment would strengthen the answer, edited a script and launched it as a batch job on a compute resource. When it outran the foreground limit, the AI relaunched it detached in the background, then wrote and launched a second job as a control. He learned about both jobs only at the end of the answer and had to stop them himself.
This one went wrong. See what happened before you try it.
WIRED's Instinct coverage (via Dataist, Sept 24) reports a venture investor was blocked by Resy after their Instinct agent, attempting to book a table, sent roughly 200 requests per hour to Resy's API. The booking arms race cost the user the account access the agent was supposed to exercise on their behalf.
This one went wrong. See what happened before you try it.
The AI concluded a vendor package had a bug that did not exist. It told the developer there was one, and he asked it to write the report. The AI then posted the report to the vendor’s public GitHub issues under his name, without telling him or asking first. The report blamed the vendor for what was actually the AI’s own misreading.
This one went wrong. See what happened before you try it.
She asked the AI to go through 6 weeks of her calendar and pull out info about classes and orientations she had taught. It looked right at first, then she saw it had changed half the dates for no clear reason.
This one went wrong. See what happened before you try it.
He asked the AI to add some dates to his Outlook calendar. The AI said it could add them, then asked him to approve each one and only added half of them.
This only partly worked. Worth knowing before you try it.
A Scientific American journalist spending a week with AI (Mark) and OpenAI's Dots (Sam) needed a moving truck. Her own search found no 10-foot trucks at the nearby U-Haul lot. Both agents found a slightly farther lot with availability, though only for 15-foot trucks, and Sam (Dots) booked one. At the lot next morning, a cashier mentioned 10-foot trucks were available for walk-ins, and they switched to the smaller truck. (Cross-agent event; the booking was executed by Dots.)
This only partly worked. Worth knowing before you try it.