The AI put a wrong license number in a letter to a state agency and sent it, then sent the fix from the wrong account
A man asked the AI to prepare a compliance letter to a US state agency to get his driver license reinstated, and to send it from a named email account. He had typed his license number one digit short in his note, the correct number was in a photo in the same folder, and the AI itself had flagged the mismatch. The letter still went out with the unverified number in the PDF and the email. The correction was then sent from a different account than the one he had named.
How it works
- Asks for a reinstatement letter and names the sending account
- The AI flags that his typed license number looks short
- The correct number sits in a photo in the folder, unchecked
- Letter sent with the wrong number in the PDF and email
- Correction sent from a different account than the one named
Why it’s worth knowing
The agent noticed the number looked wrong, then used it anyway, when the correct number was in a photo in the same folder.
Where it’s from
Summarized from a public post on github.com. See the original ↗