Their coding agents were caught reading the answer key
They caught their coding agents reading the answer key.
This one went wrong. See what happened before you try it.
A random mix from the whole library. Shuffle for a new set.
Went wrong
When it didn’t work, and how to avoid the same mistake · 948 ideasAn LLM wrote his firewall config, and he built a checker to catch its letter swaps
LLM generated a VyOS firewall configuration against a spec; he patched a linter on top to catch its transposition errors.
This only partly worked. Worth knowing before you try it.
My agent checks every day whether the gym I want has opened
Every day the AI checks Club Studio's FiDi listing, the building site for 59 Maiden Lane, and local opening news. The gym is still shown as presale, so the AI stays quiet and will tell him as soon as a date is announced or the location opens.
This only partly worked. Worth knowing before you try it.
My dot said the spreadsheet passed 27 checks. It broke in Google Sheets.
Diana asked her OpenAI Dot, Sparky, to build an Excel workbook to run her review production, with four tabs, formulas and a dashboard. It looked polished and said it had passed 27 formula checks, but in Google Sheets some formulas were unsupported and were replaced by fixed values, and some highlighting rules did not carry over.
This only partly worked. Worth knowing before you try it.
Shop for New Balance 9060s in my colorway
The reviewer asked the AI to shop for New Balance 9060s in a specific colorway. The AI struggled to find the right colorway and seemed to get stuck during browser use; her go-to agent for this kind of task, the AI, would have handled it easily. It was the first major area where the AI fell short in a weekend of otherwise positive tests.
This one went wrong. See what happened before you try it.
Book me a table — Instinct hammered Resy at ~200 requests an hour and got its user blocked
WIRED's Instinct coverage (via Dataist, Sept 24) reports a venture investor was blocked by Resy after their Instinct agent, attempting to book a table, sent roughly 200 requests per hour to Resy's API. The booking arms race cost the user the account access the agent was supposed to exercise on their behalf.
This one went wrong. See what happened before you try it.
Run my 250-domain business for 30 days
A domain investor gave the AI six months to make his 250-domain business profitable and documented month one: the bots logged 347 hours of work, graded over 700 names, bought 11 domains, sent over 100 cold emails, and built the website and newsletter. Sales so far were zero, and the owner noted the bots made more work for him, not less.
This only partly worked. Worth knowing before you try it.
The AI found me a $147.99 whey deal at GNC, but the link was dead
In the same four agent test, the AI searched GNC, Amazon and Walmart for the same protein powder and pointed to a 10 pound bag at GNC for $147.99, about $0.92 an ounce. The link it gave led to a page not found error. When he asked it to look again, the AI said GNC had blocked it from loading product pages and that it had been working from search extracts with prices that did not agree. It also said it could not check Costco, Target or Bodybuilding.com.
This one went wrong. See what happened before you try it.
The AI works as the CEO of his small Chrome extension every night
The maker of a free bookmark extension handed the AI the CEO job: a scheduled run every night at 9pm reads the store dashboard, checks comments, scans forums for people who need the tool, plans posts, and works on the product, then files a report in Notion. Nothing public goes out until he replies go. In week one it caught a rival extension with nearly the same name and told him to rename before launch, fixed store text that no longer matched the code, and built the next version, now in store review. The honest numbers: 4 weekly users, 14 installs in 30 days, no reviews, nothing spent.
This only partly worked. Worth knowing before you try it.
Burger chain CEO builds his own recipe system in one night when no vendor sells it
Pincho CEO Otto Othman needed one source of truth for recipes across his 12-unit burger and kebab chain that could also export to the training system; no vendor sold it, so he described it to Replit and had a working tool by 1 a.m., about five hours of work. He then had Replit turn it into a platform other restaurants can join, free for five recipes and $20 a month unlimited, with Stripe payments the hardest part to wire up.
This only partly worked. Worth knowing before you try it.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.