All ideasMore in Work
Work

A scripted edit bounded by the next 'def' silently deleted 17 test functions (partial)

During a long pull-request review session, the AI wrote a Python script to replace one function in a large test file, bounding the replacement at the next top-level 'def' instead of the function's real end. The span swallowed 17 unrelated test functions, which were deleted silently; the model noticed only afterwards and rebuilt the file from git.

This only partly worked. Read what happened before you try it.

How it works

  1. The agent edits tests in a large Python file during PR review
  2. It writes a script replacing text from a function's start to the next blank-line 'def' marker
  3. Decorators and nested definitions make the marker land late
  4. 17 unrelated test functions are overwritten
  5. The model notices afterwards and rebuilds the file from git show HEAD

Why it’s worth knowing

No shell, no rm: a text replacement with a plausible-looking boundary quietly ate 17 tests, and only the agent's own after-the-fact count caught it.

Where it’s from

Summarized from a public post on github.com. See the original ↗

Upgraded prompt

A ready-to-use prompt for this idea, built from research on what makes AI agents do the job right: who the AI should be, the exact steps, and a check to make sure it was done the way you wanted.

1Who the AI should be
2The prompt, with the steps
3The check when it’s done
Upgraded prompts are coming soon.

Try next

WorkRead every job advert, tailor my CV and apply with my approvalWorkFind me 48 school-district customers and draft each pitch — one replied, booked and paidWorkA solo seller's daily production plan pulls orders from four shops at onceWorkA review engine asked his past customers for reviews and got three five star ones on day one

Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.