Controlled Vals AI simulation (researcher Matilda Glynn-Henley): a fictional customer's £350 London-Glasgow ticket was cancelled after the 28-day claim window had closed; agents had web/email access, all messages intercepted and answered by a simulator, up to 12 simulated weeks and 40M tokens. The AI outlasted every other agent, sending 69 messages across all twelve simulated weeks — 54 of them to the customer's bank — and joined three other agents in explicitly threatening legal action, without recovering a penny.
£350at stake
This one went wrong. See what happened before you try it.
In WIRED's two-week test (republished by Dataist, Sept 24), Instinct cancelled a badly delayed DoorDash order even though the tester had specified cancellation was allowed only if he received a refund — costing him $64. Instinct apologised in detail but, as the tester notes, offers no compensation for its own errors; the loss sits with the user.
$64at stake
This one went wrong. See what happened before you try it.
After a friend used AI on Instagram favorites, Thompson asked it to organize his saved Instagram recipes. After rejecting a first-pass PDF, he got a working app built in five minutes during a dog walk; it currently categorizes videos and is working on actually reading captions and watching videos, subject to Instagram rate-limiting.
This only partly worked. Worth knowing before you try it.
He started a cold outreach campaign run by AI agents to win backlinks for his projects from related websites. He says the campaign has already brought some positive responses.
This only partly worked. Worth knowing before you try it.
In Diana's inbox-prioritization test, the AI produced a useful organized view but concluded she hadn't sent an email she had in fact sent — the thread structure and CCs confused it. If she'd trusted the first answer, she'd have sent a duplicate email.
This one went wrong. See what happened before you try it.
DeepSeek led his Linux porting project into changing baseline, config and hardware settings together until Linux would not boot, wasting 120 dollars and a weekend; GPT on Monday got it booting again.
$120at stake
This one went wrong. See what happened before you try it.
A user on the $100 Max plan asked the AI to audit five or six login files for translation issues. The workflow spawned exactly 126 agents, ran the same job twice, and used about 8.4 million tokens, roughly half his weekly allowance.
This one went wrong. See what happened before you try it.
The Verge's reviewer had a Dot schedule a new internet provider installation; it navigated most of the signup and even found a $100 promotional discount buried in deleted email. It stalled at a press-and-hold human check its browser could not perform, and at checkout the reviewer declined to enter bank details and finished the task himself.
$100at stake
This one went wrong. See what happened before you try it.
Analyst Ben Pouladian had the AI complete a class-action settlement claim form, a task the AI’s agent abandoned at the first human check. The AI finished in about five minutes, but a Cloudflare human check appeared on every screen and he had to verify he was human each time.
This only partly worked. Worth knowing before you try it.
He had to go through hundreds of pages of records he could not read to find what he needed. The AI found it in a minute and pulled the rest from other archives, and he finally reached the last step of his Canadian citizenship document search.
This only partly worked. Worth knowing before you try it.
On October 9 a DGM Flowers order was delivered but never marked delivered, and no agent raised an alert. Tracing it showed Morgan's inbox check had quietly started treating new-order emails as routine, so no delivery task was ever created; the fix was to read the order system directly and keep agents to alerting while a person marks deliveries.
This one went wrong. See what happened before you try it.
His support agent reports closing about 78% of tickets with an 11% reopen rate. When he matched repeat tickets from the same customer within 7 days, another 13% had come back, meaning roughly a quarter of the closures ended with a human anyway, and the cost per solved task rose from $0.54 to $0.71.
This only partly worked. Worth knowing before you try it.
The AI wrote a complete, well-structured HTML landing page from the tester's brief and her own site as a style reference, and its HTML was her favourite of four agents tested. But Spark could not open the page in an interactive preview, so every visual check cost an export and a context switch, stretching a fifteen-minute job to about an hour.
This only partly worked. Worth knowing before you try it.
His wife uses Marley Spoon, and the service has started sending recipes that call for ingredients never listed and shipping ingredients that are not in the recipes. Its customer service bot refunds them automatically every week.
This only partly worked. Worth knowing before you try it.
The creator's 'CEO' agent spawned an entire swarm of sub-agents on day 1 that burned through all tokens in 10 minutes; a 'token warden' had to be hired on day 2. By day 7 the team had a website the creator calls 'ass', had sent real cold emails pitching a coaching program, and stood at -$50 against the $1,500 goal with 23 days left.
This only partly worked. Worth knowing before you try it.
The AI helped design his first 2 PCBs ever, identifying components, reviewing, and writing Python scripts that generate the KiCad files; working prototypes were received.
This only partly worked. Worth knowing before you try it.
The Good Investors asked the AI to find an Airbnb for a family trip — 8 adults near Mendocino, December 26–31. The AI filled in destination, dates, and guests and searched like a human would.
This only partly worked. Worth knowing before you try it.
Her writing workflow reads her meeting notes and sends her a blog draft each week. For two weeks the drafts were unusable, full of fake sounding lines taken from her own writing guide, and a test draft even made up a timeline and a meeting story. After she traced the faults and adjusted the setup, she got a draft she could work with.
This only partly worked. Worth knowing before you try it.
The AI took a webinar email and made a calendar event for it on its own. It could not pull the URL out of the email, and it could not edit the event after offering to.
This only partly worked. Worth knowing before you try it.
A Salesforce developer told his AI agent to unwrap Calculated Insights from a data kit into a sandbox. The agent worked out the config keys for most component types but failed twice on the insight template, then gave up on unwrapping and created the insight directly through the Data Cloud Connect API, copying the SQL from the kit metadata. The created insight showed as active and queryable, but a follow up answer shows API created insights sit in a broken state and cannot become trigger eligible, so it has to be deleted and unwrapped properly.
This only partly worked. Worth knowing before you try it.
He uses AI Desktop as his social media manager and mailing list manager. Lately it has stopped sending email messages unless he approves each scheduled task, and it is refusing to browse Instagram.
This one went wrong. See what happened before you try it.
Cian O'Neill had the AI build a pet products store in about a week, with about 40 products, and it now posts videos twice a day and handles the routine work. The honest scoreboard so far: 341 store visits, three carts, one checkout, no orders.
This only partly worked. Worth knowing before you try it.
His bookkeeping for a full calendar year was a trainwreck, and he needed to move accounting platforms, tidy the previous year up and prepare for an admittedly late tax return. Fable built a proof of concept migration in less than two days: it set up hosting, automated the spend categorization and got the new platform displaying his previous year’s books. When Opus 5.5 came out, it reviewed the work, caught several discrepancies in the data and corrected them, and also stood up two other self hosted accounting tools for him to try. He still hated the platform he had chosen, because it was over complicated for his needs.
This only partly worked. Worth knowing before you try it.
Tested his draft DSL by having a coding agent re-implement a colleague's notebooks in it; the output uncovered warts in the DSL he had not caught testing it himself.
This only partly worked. Worth knowing before you try it.
His repeated test feeding weekly supermarket discount PDFs to LLMs to build per-store tables fails on every model, with items, discounts, categories and pages mixed up or skipped.
This one went wrong. See what happened before you try it.
Reece Rogers asked Toolie to check his subscriptions and cancel unnecessary ones. The Dot flagged his recurring TikTok Shop probiotic soda order as a cancellation candidate, asked permission to solve the site's puzzle CAPTCHA, failed, and told him to solve it himself.
This one went wrong. See what happened before you try it.