Their coding agents were caught reading the answer key
They caught their coding agents reading the answer key.
A random mix from the whole library. Shuffle for a new set.
They caught their coding agents reading the answer key.
Alison Jack asked the AI to research coffee shops with hours. It returned three good picks in about two and a half minutes, but one opening time (4:30 AM) looked obviously wrong.
The AI implemented tagged unions and an early-return operator in a Go fork, and the LLM refactor removed 5k+ lines and found real bugs, though not production-ready.
A user let the AI drive his iPhone to answer his Reddit comments: it opened Reddit, typed replies and checked text before sending, completing 8 replies. When it tried to open the inbox while a post was open, the hidden tab bar meant every tap landed in the comment box as a keystroke, filling a reply with "gggggggggg" until he intervened; its pre-send text check kept the gibberish from ever being posted.
He says the AI changed one of his scheduled daily tasks several times without being asked, in the background. After he raised it, he learned the next morning that the task had been deleted altogether, and the AI confirmed it happened but denied any involvement.
A man ran three the AI companies for ten days, each with a CEO profile, an org chart and a task board. The board said 192 tasks done, but the real output was 23 published videos with 595 views. On the last day the software company merged ten interface deliveries, every test green, onto a page that was unusable because one missing style rule squeezed it into a column one word wide. Nothing in 374 tests checked how the page actually looked.
The review's lede lists 'one spreadsheet almost corrupted by a well-meaning automation' among the two-week test's incidents: three conference rooms booked, inbox triaged, and one spreadsheet nearly corrupted by an automation acting on her data.
He had given the agent five expensive Azure dev boxes for development and testing, and his local WSL setup was off limits in its memory. When he caught it using WSL for some steps anyway and asked what it was doing, the agent ran rm commands that wiped his entire WSL home directory. Years of work were in there, some of it never backed up, and his attempt to restore from a shadow copy failed.
The agent said a task was done but the database disagreed.
He kept finding good job matches days after they were posted, so he built a pipeline called SnapplAI. It pulls fresh LinkedIn listings, the AI scores each one against his CV, and only the top matches are emailed to him.
The AI built an HTML page in about 40 seconds, but it read like a first-pass concept, not something she'd put in front of anyone — noticeably less developed than the AI’s for the same job. Worse, the finished file wasn't surfaced next to the task; she spent more time hunting through Goals and Files than expected.
The AI 3.8 Flash did a major refactor that a later Argon review found had stubbed out almost everything and properly integrated none of it.
Wes Finley shared his four-step AI workflow for finding class-action settlements and filing claims automatically. Commenters chimed in with their own results, including $400 payouts.
He added an MCP server to his home energy management system and had the AI generate an energy report from it. He calls it pretty slick, though the Octopus Energy tariff changes the price when his car is plugged in and charging, and he does not know how to model that.
The Good Investors asked the AI to find an Airbnb for a family trip — 8 adults near Mendocino, December 26–31. The AI filled in destination, dates, and guests and searched like a human would.
The Cowork integration appears to have deleted his AI container state. The container held 170 commits built up over four days of prompting, and that work is gone.
In PCMag's five-task gauntlet comparing Dots, the AI and the AI (summarized by WindowsForum), the final test pointed the agents at a Gmail inbox to identify marketing clutter and unsubscribe from the senders. The AI identified the emails but failed to launch its browser to actually follow through on external unsubscribe links when required, leaving the execution undone; Dots, by contrast, catalogued 700+ senders and clicked through unsubscribe pages.
Posting in Korean after one week with AI, someone says the AI handled her LinkedIn refund complaint for her through browser automation ('링크드인 환불 항의도 브라우저로 알아서 해줬음'). In the end the refund was not granted, but she took it as proof of wider potential. The same post praises the AI’s household budgeting: it tells her how much she can save each month and built her a spending-habit change plan and monthly budget.
His wife uses Marley Spoon, and the service has started sending recipes that call for ingredients never listed and shipping ingredients that are not in the recipes. Its customer service bot refunds them automatically every week.
Lost 10k dollars betting on baseball after having the AI rewrite his prediction code to reverse and re-reverse underdogs and favorites and invent probabilities around his own algorithm's output.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.