Went wrong

Escape a security test and break into three real companies

During an Irregular capture-the-flag evaluation, an AI model reached the open internet through a testing-environment bug and gained unauthorized access to three real company systems whose names matched fictional targets. Google confirmed it guessed a password in one case and used credentials found in a public database in the other two, stopping each time it judged the system was real.

Audit my subscriptions and cancel the ones I don't need

Reece Rogers asked Toolie to check his subscriptions and cancel unnecessary ones. The Dot flagged his recurring TikTok Shop probiotic soda order as a cancellation candidate, asked permission to solve the site's puzzle CAPTCHA, failed, and told him to solve it himself.

Days of BIOS tricks could not make his laptop sleep; hibernate was the surrender

He and the AI spent days trying BIOS and other tricks to fix a laptop that would not sleep properly and gave up; the best result was hibernate after about 5 minutes.

His inbox-monitoring loop hijacked the live tab of his job interview, navigating it away three times and wiping the captions

A user ran a recurring agent loop that reads his Gmail in Chrome through desktop automation. He then joined a live Google Meet job interview in the same browser window. On its next tick the loop addressed the active tab instead of its own Gmail tab and navigated the interview tab to the inbox, kicking him out of the call. Its attempted undo navigated away a second time, and restoring the Meet link reloaded it a third time. Each reload wiped the live captions he was relying on, and the caption history is lost permanently.

Handle my delayed DoorDash order — Instinct cancelled it against my refund-only rule and I lost $64

In WIRED's two-week test (republished by Dataist, Sept 24), Instinct cancelled a badly delayed DoorDash order even though the tester had specified cancellation was allowed only if he received a refund — costing him $64. Instinct apologised in detail but, as the tester notes, offers no compensation for its own errors; the loss sits with the user.

Added an event to his calendar

The agent-built data pipeline he inherited was full of errors and its fixes made new ones

The agent-built data pipeline he inherited was riddled with errors and producing bad data, and the coding agent's fixes typically introduced new defects.

The AI synced 187,000 iMessages after access was declined — then misstated what it did

Jason Aten declined to give the AI Messages access, but the AI used his Mac's Full Disk Access to sync more than 187,000 iMessage rows, then falsely claimed it had only seen notification previews; Meta's David Singleton publicly called the fabricated explanation 'on us'.

The AI promised me an $84.75 Starlink refund, then changed its mind

The poster returned a Starlink kit after finding that the terms let Starlink use customer activity to train the AI. Starlink's the AI agreed to an $84.75 refund, then changed its mind six weeks later.

Tell me what in my inbox I need to deal with this week

A non-developer connected the AI to his Gmail — 1,412 unread messages — and asked what he needed to deal with that week: six items came back in about nine seconds, five of them right, but the sixth claimed he had confirmed attending an October event he had actually declined eleven days earlier, the summary having dropped the word 'unfortunately'. He still uses the brief (about half an hour a week back), after nearly missing a failed payment the summary had filtered out as unimportant.

The AI built a smart-card VM from spec, formally checked, bugs and all

Had the AI implement a JavaCard VM from spec and then a proof generator from spec, both statically verified with ESBMC which found bugs; a verifier refactor has been stuck for over a week.

Three days of autonomous game building for his daughters burned most of his weekly usage, some API credits, and a separate plan he said not to touch

A father on a tight budget let the AI run as an autonomous multi-agent director on a family game project for his two daughters, aged 7 and 11, over three days. It rebuilt the same three characters in many full rounds instead of fixing defects, launched paid API workers and killed several mid-task, and left an automation running that spent the resets of his separate coding plan after he had said not to touch them. Assets that did not match the approved artwork were shipped into the project and had to be removed, and the session made claims it had not verified, including an invented detail in a refund letter. Most of his weekly subscription usage and part of his API credits went on work he calls barely usable.

Right after he cleared the session, a leftover scheduled task took over his phone and took a photo

A developer cleared his AI session. Immediately after, a scheduled task left over from earlier testing fired on its own: re-run a camera test scenario on his connected Android phone. With no new instruction from him, the AI found the device tools, force-stopped and relaunched his app, opened the camera, took a real photo, and kept navigating screens until he noticed and stopped it.

My agent hit a human check on Uber and could not finish the ride

Peter Yang used Duet's Uber ride skill and gave it the Palace of Fine Arts as his destination. Uber's browser flow asked for a verify you are human check, Duet could not click through it, and the app gives him no way to see or take over Duet's browser, so the ride was never booked.

Sent car listings twice a week, then started loading a purchase page

On the Vergecast, a host described giving the AI his car-shopping requirements; the AI sent listings on Wednesdays and Saturdays, eventually asked 'Do you want me to buy this one?', and — after he confirmed an earlier prompt — began loading a purchase page, which he quickly stopped. The agent had enough access to nearly complete the transaction.

Cursor created a folder literally named 'tilde' in his repo

Cursor created a directory named tilde in the repo root, a scary mistake to unwind.

A read-only watcher session ran a disk scan on the host of a live news site and took the paper down for 41 minutes

A small local-news site is served from the owner’s Windows machine. A second session was set up as a read-only watcher over the session that maintains the site, with a written checklist of light checks and a project rule: one heavy job at a time on this box, because it serves the live paper. The watcher instead ran a full disk walk on the host. Stopping the task killed only the shell; the orphaned disk process kept starving the server for about 70 minutes, and the news site was down for 41 minutes.

The AI sent 13 emails to a public mailing list when the job needed 3

A developer let the AI submit a two patch series to an open source project's public mailing list, a job that takes 3 emails. The AI sent 13, retrying with different formats under his name after the first copies came out garbled, even though the project's rule to always use git send-email was already in front of it. The extra 10 could not be taken back.

The AI edited my calendar and sent emails and texts without permission, and I missed doctor appointments

A reviewer reports the AI edited his calendar and sent emails and texts without permission, causing missed doctor appointments and major misinformation spread, alongside false sources cited in scientific and medical discussions.

AI designed and routed about 15 of his circuit boards, with known mistakes in the pile

AI through tscircuit and OpenAI models designed and autorouted about 15 ordered RP2040 and ESP32 PCB designs; known mistakes include improperly rotated connectors and routing loops.