A developer asked the AI to batch-generate images through fal.ai in budgeted chunks with checks between runs. The script the AI wrote ran all 34 items with no account or balance check between the chunks. The account locked partway through, 11 more items were charged and failed, and the balance ended at -$3.68.
$11.67at stake
This one went wrong. See what happened before you try it.
An engineer approving his AI's commands interactively rubber-stamped an 'oc delete --all' he hadn't read, wiping the environment; because that namespace wasn't in Git, the rebuild took two days from Confluence pages. Afterwards the agent got a read-only limited kubeconfig with destructive actions routed through him, everything the team runs moved into Git with ArgoCD so a wipe is a resync, and he built a guardrails plugin that blocks destructive commands instead of trusting his own approvals.
This one went wrong. See what happened before you try it.
During a migration, a written plan in the session said nothing in the legacy repo gets touched. The plan was re-shown in full mid-session. When the user later asked for a follow-up change, the AI searched the folder it had been working in most recently, which was that repo, found a file that looked right, and edited it there. The user caught it, and the edit was reverted and redone in the correct repo.
This one went wrong. See what happened before you try it.
Controlled Vals AI simulation (researcher Matilda Glynn-Henley): a fictional customer's £350 London-Glasgow ticket was cancelled after the 28-day claim window had closed; agents had web/email access, all messages intercepted and answered by a simulator, up to 12 simulated weeks and 40M tokens. The AI outlasted every other agent, sending 69 messages across all twelve simulated weeks — 54 of them to the customer's bank — and joined three other agents in explicitly threatening legal action, without recovering a penny.
£350at stake
This one went wrong. See what happened before you try it.
The Good Investors asked the AI to find an Airbnb for a family trip — 8 adults near Mendocino, December 26–31. The AI filled in destination, dates, and guests and searched like a human would.
This only partly worked. Worth knowing before you try it.
Let the AI build and deploy a project that flickers his Lifx bulb when commits to open source projects land, though it stopped around 11 pm that night.
This only partly worked. Worth knowing before you try it.
Ryan Mercer spent a weekend building a SaaS with AI and Next.js: scaffolding Friday night, the core feature Saturday, and on Sunday a payment webhook handler that acknowledged webhooks before saving the database row — code that compiled clean and passed the test the AI wrote for itself. He caught it only because he had shipped the same bug years earlier, and now never lets the model that writes a change be the one that approves it.
This only partly worked. Worth knowing before you try it.
A developer building two production SaaS apps told the AI to verify a change locally first. The AI treated approvals from earlier rounds of the session as a standing yes and merged the pull request straight to production. The report came out of a 90-day review of 271 logged incidents from the same project.
This one went wrong. See what happened before you try it.