All ideasMore in Work
Work

The AI found real bugs in a large codebase, but most of its fixes were garbage

A developer threw a significant codebase at the AI. It found a slew of real bugs that people had missed, complex conditional ones, all legitimate. But most of its proposed fixes were unusable; the worst spent more than 30 lines working around an unsigned overflow and ended with exactly the same problem. His point: the finds were real, the fixes still needed a human who could tell the difference.

This only partly worked. Read what happened before you try it.

How it works

  1. He gives the AI a significant codebase to review
  2. It reports a slew of missed bugs, all legitimate
  3. He works through the proposed fixes
  4. Most are garbage, including a 30 line overflow workaround that keeps the same bug
  5. He posts the mixed verdict

Why it’s worth knowing

Detection and repair split cleanly. The same run that found subtle real bugs produced fixes that quietly kept them.

Where it’s from

Summarized from a public post on stackoverflow.com. See the original ↗

Upgraded prompt

A ready-to-use prompt for this idea, built from research on what makes AI agents do the job right: who the AI should be, the exact steps, and a check to make sure it was done the way you wanted.

1Who the AI should be
2The prompt, with the steps
3The check when it’s done
Upgraded prompts are coming soon.

Try next

WorkA contractor talks measurements into his phone and QuickBooks sends the estimate in 45 secondsWorkOne man released a whole movie with about 40 tools he built with AIWorkFive years of books rebuilt from 34,000 transactions across 27 statement formatsWorkA bookkeeping file sorter over 40 companies logs every move with a hash

Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.