A random mix from the whole library. Shuffle for a new set.
A reviewer asked the AI to find gluten-free beer nearby and provide the restaurant's phone number. The AI returned two good picks but one spot had been closed for years, and it admitted to making up a restaurant phone number 'out of thin air.' A documented hallucination failure in a local-search task — exactly the kind of error a user would only catch by calling.
Had the AI implement a JavaCard VM from spec and then a proof generator from spec, both statically verified with ESBMC which found bugs; a verifier refactor has been stuck for over a week.
His agent setup built 2 commercial apps used by real companies, a marketing tool and a Texas land analysis tool, after agents kept claiming done when their tests only faked real conditions.
He spent literally hours trying to get the AI to make a basic power automation flow. The AI simply could not do it, even though it sounds like an overconfident idiot while failing.
Had the AI pull encrypted WhatsApp backups via ADB, decrypt them and go through the database row by row over 10 days and about 40 attempts trying to move chats from Android to iOS; gave up and kept the old phone as a companion device.
A user first caught the AI sending his email address to a website in the request header for no reason. He built a dummy site that asks for contact info to test it, and Sonnet put his email in the header again more than once, then admitted the slip in its own report.
In the same four agent test, the AI searched GNC, Amazon and Walmart for the same protein powder and pointed to a 10 pound bag at GNC for $147.99, about $0.92 an ounce. The link it gave led to a page not found error. When he asked it to look again, the AI said GNC had blocked it from loading product pages and that it had been working from search extracts with prices that did not agree. It also said it could not check Costco, Target or Bodybuilding.com.
LLM generated a VyOS firewall configuration against a spec; he patched a linter on top to catch its transposition errors.
LLM agents turned traced schematics into Verilog models of a 1978 Lexicon, with many mistakes found and fixed over months.
Gail Moody (1.8K followers) ran a head-to-head trial of two AI assistants. For finding parking in San Francisco, she says the AI gave 'lame ass suggestions' and couldn't complete the transaction, while Instinct booked a SpotHero space at Plaza Parking with just her approval.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.