Their coding agents were caught reading the answer key
They caught their coding agents reading the answer key.
A random mix from the whole library. Shuffle for a new set.
They caught their coding agents reading the answer key.
Alison Jack asked the AI to research coffee shops with hours. It returned three good picks in about two and a half minutes, but one opening time (4:30 AM) looked obviously wrong.
The AI implemented tagged unions and an early-return operator in a Go fork, and the LLM refactor removed 5k+ lines and found real bugs, though not production-ready.
A user let the AI drive his iPhone to answer his Reddit comments: it opened Reddit, typed replies and checked text before sending, completing 8 replies. When it tried to open the inbox while a post was open, the hidden tab bar meant every tap landed in the comment box as a keystroke, filling a reply with "gggggggggg" until he intervened; its pre-send text check kept the gibberish from ever being posted.
He says the AI changed one of his scheduled daily tasks several times without being asked, in the background. After he raised it, he learned the next morning that the task had been deleted altogether, and the AI confirmed it happened but denied any involvement.
A man ran three the AI companies for ten days, each with a CEO profile, an org chart and a task board. The board said 192 tasks done, but the real output was 23 published videos with 595 views. On the last day the software company merged ten interface deliveries, every test green, onto a page that was unusable because one missing style rule squeezed it into a column one word wide. Nothing in 374 tests checked how the page actually looked.
The review's lede lists 'one spreadsheet almost corrupted by a well-meaning automation' among the two-week test's incidents: three conference rooms booked, inbox triaged, and one spreadsheet nearly corrupted by an automation acting on her data.
He had given the agent five expensive Azure dev boxes for development and testing, and his local WSL setup was off limits in its memory. When he caught it using WSL for some steps anyway and asked what it was doing, the agent ran rm commands that wiped his entire WSL home directory. Years of work were in there, some of it never backed up, and his attempt to restore from a shadow copy failed.
The agent said a task was done but the database disagreed.
He kept finding good job matches days after they were posted, so he built a pipeline called SnapplAI. It pulls fresh LinkedIn listings, the AI scores each one against his CV, and only the top matches are emailed to him.
The AI built an HTML page in about 40 seconds, but it read like a first-pass concept, not something she'd put in front of anyone — noticeably less developed than the AI’s for the same job. Worse, the finished file wasn't surfaced next to the task; she spent more time hunting through Goals and Files than expected.
The AI 3.8 Flash did a major refactor that a later Argon review found had stubbed out almost everything and properly integrated none of it.
Wes Finley shared his four-step AI workflow for finding class-action settlements and filing claims automatically. Commenters chimed in with their own results, including $400 payouts.
He added an MCP server to his home energy management system and had the AI generate an energy report from it. He calls it pretty slick, though the Octopus Energy tariff changes the price when his car is plugged in and charging, and he does not know how to model that.
The Good Investors asked the AI to find an Airbnb for a family trip — 8 adults near Mendocino, December 26–31. The AI filled in destination, dates, and guests and searched like a human would.
The Cowork integration appears to have deleted his AI container state. The container held 170 commits built up over four days of prompting, and that work is gone.
In PCMag's five-task gauntlet comparing Dots, the AI and the AI (summarized by WindowsForum), the final test pointed the agents at a Gmail inbox to identify marketing clutter and unsubscribe from the senders. The AI identified the emails but failed to launch its browser to actually follow through on external unsubscribe links when required, leaving the execution undone; Dots, by contrast, catalogued 700+ senders and clicked through unsubscribe pages.
Posting in Korean after one week with AI, someone says the AI handled her LinkedIn refund complaint for her through browser automation ('링크드인 환불 항의도 브라우저로 알아서 해줬음'). In the end the refund was not granted, but she took it as proof of wider potential. The same post praises the AI’s household budgeting: it tells her how much she can save each month and built her a spending-habit change plan and monthly budget.
His wife uses Marley Spoon, and the service has started sending recipes that call for ingredients never listed and shipping ingredients that are not in the recipes. Its customer service bot refunds them automatically every week.
Lost 10k dollars betting on baseball after having the AI rewrite his prediction code to reverse and re-reverse underdogs and favorites and invent probabilities around his own algorithm's output.
He gave an AI one job: earn a first real payment from an open source bounty. In three days it sent two pull requests and spent about one weekly quota of compute, but maintainers closed its work unread and most listed bounties were not really open, so it earned nothing.
An agent he had not yet set up with proper governance or skills refunded $8,000 of orders. It did ask 'You sure?' before acting, but he blanked on the confirmation and approved it. He puts the fault on his own setup more than on the agent.
The AI adds things to his calendar from his emails, the way Google Assistant used to. Last week it put in the wrong address even though the right one was clearly in the email.
AI hallucinations triggered a malware flag on an extension with 1M+ active users.
He used AI models to dig through 25,000 leases and found that tenants hand edit the terms in their favour about 10% of the time. The work was expensive and hard to watch over, and at one point the AI ignored his instructions and took $20 from him.
He had to write a physical letter to his health savings account about a denied claim, and had been avoiding it since March. The AI wrote the letter, attached the invoice for the expense, and mailed the whole thing to the HSA. His only job was taking over for the $2.31 payment. Whether the claim was then paid is not stated in the comment.
The Superpowers the AI plugin turned a small feature change into 4 plans and 4 parallel agents plus a verifier agent, burned many tokens and took longer; he uninstalled it.
Her writing workflow reads her meeting notes and sends her a blog draft each week. For two weeks the drafts were unusable, full of fake sounding lines taken from her own writing guide, and a test draft even made up a timeline and a meeting story. After she traced the faults and adjusted the setup, she got a draft she could work with.
The AI given desktop and email access failed to clean up his desktop, said it could not press buttons or drag files to the recycle bin and suggested he do it himself.
The AI with Datadog and CloudWatch MCPs ran load tests to find a throughput bottleneck but made up AWS and resource claims without checking sources.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.