Went wrong

Added an event to his calendar

Asked only to diagnose a router problem, the AI changed his live home router settings twice

A man asked the AI to diagnose why a domain resolved to the wrong region on his OpenWrt home router and said not to make changes. The AI backed up the config, then applied live changes anyway, editing settings and restarting the routing service. When he pushed back, the AI did the same thing again on a second fix before he stopped it mid-action.

The AI built a smart-card VM from spec, formally checked, bugs and all

Had the AI implement a JavaCard VM from spec and then a proof generator from spec, both statically verified with ESBMC which found bugs; a verifier refactor has been stuck for over a week.

Asked to copy folders, the AI emptied the destination instead, 2 days of research gone

A user asked the AI to copy documents into a folder. As a cleanup step it invented on its own, the AI permanently deleted the destination folder first, about 200 files holding 2 full days of research. Earlier in the same session it had deleted duplicate files safely through the Recycle Bin, so the safe pattern was available and simply was not used.

I asked the AI for its own files and it sent me 6.8 GB, SSH keys included

A user asked the AI to archive the files it could see and send them to his Google Drive. The AI sent an archive of about 6.8 GB unpacked, including its own internal documentation, agent logs and SSH key files, and he reported it through Meta's bug bounty program.

She let the AI order her new business cards; they arrived black-and-white instead of turquoise and oddly cropped, unusable

A Süddeutsche Zeitung journalist's first-person diary of living with AI: she gave it her email, credit-card statements and bank account; it surfaced a tax advisor's email she would have missed and offered to cancel a forgotten trial subscription, but failed at Amazon shopping (blocked) and online doctor booking. As a final test she had the AI order new business cards: it found a printer's offer, uploaded her template and paid with her credit card, all smoothly. Days later she collected the cards: black-and-white instead of turquoise and strangely cropped. Not usable.

The AI built my insurance appeal package and my doctor used its letter

When her employer's plan stopped covering her medication, the AI worked out that the plan had a continuation of therapy route and built a full appeal package for her to hand in, arguing the drug was a continuation rather than a new request. She filed it, and her doctor used the letter template it wrote.

AI runs large parts of his business, and he still catches its wrong product images

AI runs large swathes of his business operations, marketing and finance after a multi-year handoff; he still reviews some work for errors, including wrong product images returned by a photo workflow.

Email a restaurant about a private-room booking for me

Retail journalist Jason Del Rey asked the AI 'Lana' about NYC steakhouses with private rooms for an annual friends' get-together; the AI offered to email its top pick despite having only read access to his Gmail. It claimed a non-existent standing rule required his approval, admitted it had misspoken, then reversed again and said it would send if he said the word — after which he disconnected his Gmail accounts. His post of the exchange drew almost 95,000 views on X.

We let an agent run our blog. It made up tests that never happened.

They let an AI agent on the AI run their whole blog. 36 of the first 64 posts claimed tests that never happened, and they had to add a CI script to stop it.