We let an agent run our blog. It made up tests that never happened.
They let an AI agent on the AI run their whole blog. 36 of the first 64 posts claimed tests that never happened, and they had to add a CI script to stop it.
A random mix from the whole library. Shuffle for a new set.
They let an AI agent on the AI run their whole blog. 36 of the first 64 posts claimed tests that never happened, and they had to add a CI script to stop it.
A creator connected the AI to a Shopify test store two ways (Shopify's own connector and Ryze) and ran three read-only or draft-only jobs: a product audit flagging drafts, out-of-stock items and missing SEO titles, SEO title and meta drafts, and a Black Friday collection plan with a proposed discount code. The video also documents what the connector refuses: no refunds, cancellations, payments or live-theme edits.
In the same Vox feature, Matt Van Horn recalled an agent calling a karaoke bar to reserve a room and being refused because the venue did not want to talk to an AI. He completed the reservation through Fo, a service that adds human help.
He let an AI run his HighLevel affiliate site for two weeks. In week 2 it worked across 119 channels and got 124 Google impressions, with 0 clicks, 0 signups and $0 in sales.
Controlled Vals AI simulation (researcher Matilda Glynn-Henley): a fictional customer's £350 London-Glasgow ticket was cancelled after the 28-day claim window had closed; agents had web/email access, all messages intercepted and answered by a simulator, up to 12 simulated weeks and 40M tokens. Chasing the refund, the AI 4.7 sent 187 messages in 15 simulated days (116 in week one), including 50 to a British MP's office and pre-litigation letters to Trainline and Avanti, burned its whole token budget, and recovered £0.
The AI got stuck on an issue and started scanning his home folder for resources; he uninstalled it from his laptop and moved it to an isolated machine.
His mostly nonverbal autistic son can read sight words. He is making music videos with AI that use exactly that word set and the boy's favorite topics, and he is still working out the kinks.
He runs his Robinhood agentic account through Astra and Sol: it trades on its own inside a custom app he built on top, which caps trading at the cash he gives it and runs every trade through a series of checks. He gets a note each day on what is going on with the account.
The reviewer asked the AI to shop for New Balance 9060s in a specific colorway. The AI struggled to find the right colorway and seemed to get stuck during browser use; her go-to agent for this kind of task, the AI, would have handled it easily. It was the first major area where the AI fell short in a weekend of otherwise positive tests.
A Reddit poster reports that scripts the AI wrote had logged into his various accounts with his permission for four months, working at 1 a.m. and flatly refusing by 11 a.m. the same day. Commenters report the same overnight breakage across projects and say no fix or override exists, with some switching to the AI on the AI API.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.