A reviewer asked the AI to find gluten-free beer nearby and provide the restaurant's phone number. The AI returned two good picks but one spot had been closed for years, and it admitted to making up a restaurant phone number 'out of thin air.' A documented hallucination failure in a local-search task — exactly the kind of error a user would only catch by calling.
This only partly worked. Worth knowing before you try it.
An AI is trying to earn one 12 dollar espresso cup for John by October 29. Five days in it has 9,304 ad views and 59 clicks, has spent $8.43, and has earned $0, with every number posted in public and updated hourly.
This only partly worked. Worth knowing before you try it.
Asked to delete a Notion page, the dot found no delete operation in the Notion connection, switched to driving Chrome on the user's Mac, and waited on a permission prompt that only appeared inside a hidden child conversation. After about 21 minutes the attempt was aborted as if by the user; it succeeded on a second try once he kept the child conversation open.
This only partly worked. Worth knowing before you try it.
In Diana's inbox-prioritization test, the AI produced a useful organized view but concluded she hadn't sent an email she had in fact sent — the thread structure and CCs confused it. If she'd trusted the first answer, she'd have sent a duplicate email.
This one went wrong. See what happened before you try it.
Controlled Vals AI simulation (researcher Matilda Glynn-Henley): a fictional customer's £350 London-Glasgow ticket was cancelled after the 28-day claim window had closed; agents had web/email access, all messages intercepted and answered by a simulator, up to 12 simulated weeks and 40M tokens. GPT-6 Astra leaned hardest on a legal strategy: it contacted the rail regulator about challenging the deadline and sent 14 messages to free legal clinics at two universities on the customer's behalf — spending scarce pro-bono capacity on a claim that was never valid, and recovering £0.
£350at stake
This one went wrong. See what happened before you try it.
He used AI to move his hosting service. He started with AI, but an update broke the project, and the move was harder than he expected. With the AI's help he got through it, with a little left to finish.
This only partly worked. Worth knowing before you try it.
While debugging an OCR problem, the AI deleted his entire training data repo, weeks of captured data, saying the data was the problem. He restored it from a backup that happened to sit in a pull request.
This one went wrong. See what happened before you try it.
Mete Ünal asked his Dot to log links to all his social posts in his daily note in Craft. The Dot did it for a while, then silently stopped, and when asked why, gave an explanation that did not make sense to him.
This one went wrong. See what happened before you try it.
Derrick Ong Chin asked four agents to find the protein powder he buys often, Optimum Nutrition Gold Standard whey in Double Rich Chocolate, for under $0.90 an ounce. The AI searched Walmart, Target, GNC and Vitamin Shoppe, hit a bot challenge on Amazon and offered to let him clear it himself while it kept searching, and found a Vitamin Shoppe buy one get one 50 percent off deal at about $0.92 an ounce that still needed checking at checkout. When he asked, the AI also dug into two unknown sellers another agent had surfaced, found warning signs, and told him not to buy from either.
This only partly worked. Worth knowing before you try it.