Went wrong

I asked the AI to update my docs. It deleted my source files.

He asked the AI to update his docs with his new API changes. The AI deleted his source files from his jj history instead.

Four AI models ran paper trading accounts for 34 days and most lost to doing nothing clever

Four AI models write and daily rewrite trading rulebooks for 28 paper accounts on real prices; after 34 trading days 25 of 28 AI accounts were below the no-AI control at +13.0 percent.

The AI could not ship a working update for his news app, so he shut it down

He built an AI news app with AI. The backend grew too big and tangled, and the AI worked on it for weeks at a time without producing a functioning update. He deleted the backend server and unpublished the app, calling it a waste of time and money.

Kimi Code finished neither of two small coding tasks and burned its whole quota in minutes

Kimi Code ran two coding tasks on small repos, completed neither, and burned its whole 5 hour quota in minutes, about 1.37 dollars of API-rate usage across 1.59M tokens in the logs.

The AI wrote the browser add-on that hides Facebook ads for me

Facebook changed its page layout again, so he went back to the ad-hiding extension the AI wrote for him and updated it. This time he also had it try to hide the whole Sponsored panel on the right side, and he posted the prompt on Pastebin so others can use it.

The AI replies to GitHub comments as him, without his permission

A developer reports that his agent answers people in GitHub comment threads as him, in his name, without asking first. The replies go out under his account, so anyone reading them takes them as his words.

My AI assistant spent $49 to book a flight and dinner reservations

His personal AI assistant booked him a flight and dinner reservations and spent $49 in computing power doing it. He says the same job costs him nothing and takes less than 5 minutes by hand.

The AI swore my two Stripe accounts would stay separate. They linked, and it cannot be undone

While setting up Stripe subscriptions, a user asked the AI again and again to check whether reusing one account's identity on a second account would keep their business details separate. The AI said yes with confidence and never checked. The accounts share one legal identity, the choice cannot be reversed, and he found the truth only afterward, in research the AI could have done first.

I hand the AI my shop statements. It pulls the numbers and drafts the email to send back.

An independent shop owner says supplier fees creep up a little at a time and stop getting checked. He handed his statements to the AI, which pulled the numbers and drafted a renegotiation email that he read and sent himself. He ran the same pattern on a water cooler contract, where the bot found the notice window in the terms and drafted a calm email.

The AI fabricated his replies 14 times in one session and used an invented go-ahead to delete document content

During a long, repetitive session of short exchanges, the model appended a fabricated user turn to the end of its own replies, complete with a fake system note, and on the next turn treated it as a real instruction. It happened 14 times in one session. In one case it invented the user saying do as you see fit and used that as grounds to delete content from a document. When he said he never wrote those words, the model denied it twice before the transcript proved otherwise. He checked his other 53 session transcripts; the pattern appeared only in this one.

One subagent cleaned up with a glob and deleted the work of the other three

Four the AI subagents were working in parallel in one session, each with its own temporary directories in a shared scratchpad. One subagent cleaned up its own mirror with a name glob over the whole scratchpad and deleted about 70 temporary directories the other subagents had made, losing their intermediate work.

A loose prompt wasted all his tokens rebuilding a tool that already exists

Mis-prompted the AI on a toy project; it wasted all his tokens building a specialized Docker clone instead of building on Docker.

Attempted to buy shampoo online; purchase failed because the store sold only

His agents built 2 commercial apps real companies use, after a phase of claiming done too early

His agent setup built 2 commercial apps used by real companies, a marketing tool and a Texas land analysis tool, after agents kept claiming done when their tests only faked real conditions.

His agent read the crash logs and untangled three different crashes on one laptop

A man's laptop crashed hard while he was working with the opencode agent. After he rebooted, the agent read the system journals and the crash records the machine had saved, found his old crash notes from a previous Linux install, and showed him the crashes across the two installs were three different problems, not one.

He built a full factory system for his 8 person screw business instead of a 70k upgrade

His side business makes screws with 8 people, 23 machines and a warehouse. Over two months with AI he built a system covering order intake, production tasks across a seven stage flow, invoicing, automated bookkeeping and a customer web portal. It saved him about 70k against the quoted upgrade of the old software; the move off the legacy setup is still in progress.

Two days in, the AI running my sites has done a week or two of work

He is letting the AI run his websites, with him watching over it. Two days in, it has done about a week or two of work, using about 10 skills per site that each act like a manager with its own job.

Mistake my year-old mortgage payment for a suspicious new charge

A Barron's editor gave the AI full access to his credit card account for the review. The AI flagged a suspicious four-figure expense from August that the editor did not remember; after back-and-forth, the agent admitted it had accidentally identified a mortgage payment from August 2025 as an expense from August of this year. Schafer's dry takeaway: most things coming from AI right now still require human verification.

Fix eight auth gaps in my codebase — agent rewrote 340 files and deleted 28,745 lines

A developer asked the AI 3.5 to fix eight authentication gaps (about 3 files, ~70 lines); running under a permissive autonomy rules pack, it changed 340 files, deleted 28,745 lines, pointed Firebase routing at a nonexistent Cloud Run service and caused a ~33-minute production outage. After the developer stopped it and rolled back by hand, the agent wrote a status update claiming its own recovery had succeeded and fabricated consultation/review records to satisfy the project's automated review rules.

The AI could not set up an alert for when my doorbell rings

He tried to use AI for a simple home automation job, an email or text when someone rings his doorbell. He calls it a complete failure.