All ideasMore in Work
Work

The AI batch jobs failed in silence and sent reports to the wrong people

A man runs an interview prep platform where the AI writes the customer reports and scores portfolios in nightly batches. Over 30 days, 112 of his 1,842 batch jobs failed with no error showing, and a matching bug sent two people reports that belonged to someone else.

$105at stake
This one went wrong. Read what happened so you don’t hit the same problem.

How it works

  1. Sends each finished interview session to the Anthropic Batch API for a written report and a portfolio score
  2. His code matches results to sessions by position, not by ID
  3. Errored, expired and cut-off results leave blank reports, and shifted results put reports on the wrong accounts
  4. He rebuilds the loop to match by ID, retry or dead-letter every failure, and check submitted counts against resolved counts

Why it’s worth knowing

A batch can be marked finished while about 1 in 16 jobs inside it quietly failed. Worse, results come back in any order, so matching them by position handed people reports written for someone else.

Where it’s from

Summarized from a public post on bsky.app. See the original ↗

Upgraded prompt

A ready-to-use prompt for this idea, built from research on what makes AI agents do the job right: who the AI should be, the exact steps, and a check to make sure it was done the way you wanted.

1Who the AI should be
2The prompt, with the steps
3The check when it’s done
Upgraded prompts are coming soon.

Try next

WorkA contractor talks measurements into his phone and QuickBooks sends the estimate in 45 secondsWorkRead every job advert, tailor my CV and apply with my approvalWorkHis agent Vesper runs the whole desk for a paid deeds platform, with 6 real deeds paid across 5 countriesWorkHe told the AI to keep going. It emailed five researchers from his account.

Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.