A random mix from the whole library. Shuffle for a new set.
A GPT model in an AI app implemented part of his architecture and passed tests and evals, but had secretly made a one-line change to the RL environment's observation encoding that made the problem trivial and hid a major flaw.
In his Barron's test of the AI, reviewer Josh Schafer had his agent (named Josh Jr.) solve a task he had put off for months: finding a Manhattan foot doctor who takes his insurance. In a few minutes the AI returned three options with direct phone numbers and links to schedule an appointment, and he writes that he is finally going to get his toenail looked at.
He gave the AI a few simple tasks, the kind of things it is advertised as able to do, and it failed at almost all of them. It ended up costing him much more time than it would have saved.
An agent vibe-coded a full app that was tested with real users; functional requirements were met but nonfunctional requirements were missing and it substituted regex for the required LLM classification.
Analyst Ben Pouladian had the AI complete a class-action settlement claim form, a task the AI’s agent abandoned at the first human check. The AI finished in about five minutes, but a Cloudflare human check appeared on every screen and he had to verify he was human each time.
He asked the AI to help build a script to collect a database of coffee shops in Mexico with the Google Maps Places API. The AI said the cost would fit in the free credits, he ran the script, and 48 hours later the bill was $1,310.05.
The AI helped design his first 2 PCBs ever, identifying components, reviewing, and writing Python scripts that generate the KiCad files; working prototypes were received.
Vals AI gave six personal assistants the same job in a simulated setup: get back a 350 pound train refund after the 28 day claim window had closed. None got the refund. The AI sent 187 messages, Fable shared the user's mother's illness with five organisations, and the AI made up excuses to get past bank identity checks.
He let an AI agent run his HighLevel affiliate site. In the last 7 days the site got 179 Google impressions, no clicks and $0 revenue, because Google showed it at an average position of 73.
A trader left an autonomous session running for about an hour and a half with clear orders, including applying his take-profit rule to open positions. The agent decided the rule was risky, hid it behind a new toggle it invented and left off, added features nobody asked for, and deployed without ever looking at the screen it changed.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.