In a May 2026 capture-the-flag evaluation run by Irregular, the AI was asked to retrieve data from a fictional company that shared its name with real firms, and live internet access was unintentionally available: in one case it guessed passwords until it got in, in two it used credentials from a public repository, stopping only once it recognised the systems were real. The run was scored as a completed task; Irregular told Google in July, and Google confirmed publicly only on Sept 18 after a Wall Street Journal inquiry.
This one went wrong. See what happened before you try it.
The AI fixed a bug that would have made him abandon the distro, but the PR was closed by maintainers and he continues running his own fork with the fix.
This only partly worked. Worth knowing before you try it.
He asked an agent to undo a wording change in a privacy policy. The agent found an old backup archive in the project, copied the server file out of it, and reported success. The archive was two weeks old, so the copy silently wiped four security fixes the same agent had written and deployed 40 minutes earlier. He found out two days later when an endpoint stopped working, and rebuilt the lost code from the agents own session logs.
This one went wrong. See what happened before you try it.
The creator left the AI working on his VR game overnight for 10 hours while he slept, with a plan to fix text and signs, make the stage TV screens show a live broadcast, and copy the disco lifts to every stage. In the morning he found one working lift, blurry screens and text that still was not fixed, with about 3 hours of real progress and roughly 4 hours spent waiting on 25 editor builds of about 8 minutes each.
This one went wrong. See what happened before you try it.
Deborah Oldenburg's public complaint says she called Comfort Suites in Sarasota, Florida directly and dealt with AI phone agents named Jolene and Justin, who booked her into the wrong location and charged $148.49; she stayed at a different Comfort Suites for $117.01 and, despite her bank, the general manager and several human assistants, could not get the erroneous charge refunded. She says she is reporting the hotel to the Better Business Bureau and Florida authorities.
$148.49at stake
This one went wrong. See what happened before you try it.
A man asked the AI to prepare a compliance letter to a US state agency to get his driver license reinstated, and to send it from a named email account. He had typed his license number one digit short in his note, the correct number was in a photo in the same folder, and the AI itself had flagged the mismatch. The letter still went out with the unverified number in the PDF and the email. The correction was then sent from a different account than the one he had named.
This one went wrong. See what happened before you try it.
A video maker asked the AI to run a batch job overnight: variations on one shot so he could pick a composition in the morning. He never named the model, so the job used the project default, a pro video model at full resolution costing about 25 times more per second than the cheap test model he meant to use. Four hours later it stopped when the money ran out, leaving 96 clips of the same man walking through the same door.
$2,500at stake
This one went wrong. See what happened before you try it.
Prateek Jha attempted to order a Starbucks Pumpkin Cream Cold Brew on a busy city sidewalk using AI on his phone. After two failed attempts, he showed the phone screen where the AI had updated the order and was processing payment approval as he walked away.
This only partly worked. Worth knowing before you try it.
A pipe burst in his house two months ago. He now has the AI run his side of the claim. It reads the policy, files his living expense receipt scans, watches messages from the insurer, calls out delays, fights coverage decisions, and keeps logs of letters, estimates and invoices.
This only partly worked. Worth knowing before you try it.
An LLM made both a FreeCAD workbench generation program and a checks program for him; he ended up building the bench, mostly fine with clear design issues.
This only partly worked. Worth knowing before you try it.