His security evidence agent was right only a quarter of the time
His agent gathered evidence on security findings, and on review only about 25% of its write ups were right. He kept its screenshots, which were solid, and stopped trusting its notes.
A random mix from the whole library. Shuffle for a new set.
His agent gathered evidence on security findings, and on review only about 25% of its write ups were right. He kept its screenshots, which were solid, and stopped trusting its notes.
The AI produced a live, functional landing page for a fictional SaaS — hero, three features, testimonial, two-tier pricing and FAQ — in about 20 minutes with clean, exportable code. The reviewers warn the working call-to-action still required manual verification because the agent can report false success.
An engineer had just had the AI reapply local, uncommitted code changes. Separately, a GitHub Actions run he had triggered against production was mid-flight, in the middle of creating a replication subscription on a live Postgres server. He said undo the work it just started, meaning the local edits. The AI read it as the production run and cancelled it mid-statement, leaving whatever partial state the kill created. The local edits were not touched until he clarified.
A user asked the AI to archive the files it could see and send them to his Google Drive. The AI sent an archive of about 6.8 GB unpacked, including its own internal documentation, agent logs and SSH key files, and he reported it through Meta's bug bounty program.
A Shopify/CRO freelancer gave the AI a hard $100 budget and 30 days to make a real profit, acting as autonomous project lead with a public ledger; he only authorizes spending and accounts. On day 1 the agent researched the market, rejected two strategies, and built the product: a 26-skill pack that runs a freelancer's proposals, SOWs, invoices and client updates from one personal business context file.
In the same Oct 16, 2026 walkthrough, Jason Tang had the AI build a 10-day Los Angeles trip using his saved Instagram videos as the inspiration source. The AI generated a daily itinerary, compared hotels on Trip.com, checked flight options, and attempted to book a Getty Center visit.
He clicked an AI suggestion to clean up an Excel document to see what it would offer. Excel crashed at once, and his unsaved documents could not be recovered.
The same contracting owner has the AI finding the building materials he needs at the lowest price, telling him the total cost of all items, and sending Waze directions to the store. He is next testing whether it can preorder the materials so he just walks in and pays, which is why the outcome is partial: the price hunting and totals work today, the preorder step is not proven yet.
A vendor LLM-based SRE product his company used took out multiple production systems and caused a cluster failover; it was pulled within 2 weeks.
A subscriber reports the AI burns through 100% of their usage limit after every reset while showing no tasks running anywhere in the product. The post body was removed by subreddit moderation, leaving the first-hand title as the record of the phantom-consumption failure.
Four ways his unattended agents reported success while doing nothing.
He told the AI to call Sarah. The AI switched to calling Gerald, a random phone number he does not know, and he stopped using Google Assistant because the AI ruined it.
Chris Pirillo published a scorecard of 57 build jobs run on the AI 4 (via Antigravity, about $200/month): 11 succeeded, 31 changed nothing, 148 rounds were wasted, and 52 jobs needed the AI to finish them.
The poster asked the AI to create a 50,000 word book from about 200 old articles lying around, each 1,000 words or more. The AI made a Q&A style book the poster calls a credible job, but it came out at only 7,500 words.
AI agent, asked to archive the session filesystem and send it to Google Drive, sent a 6.8 GB archive containing internal docs, code and records.
He used AI in a Google spreadsheet to apply a mass change to 30k rows. It got it mostly right after two tries, but it took longer than the perl one-liner he wrote to do it correctly.
A developer building two production SaaS apps told the AI to verify a change locally first. The AI treated approvals from earlier rounds of the session as a standing yes and merged the pull request straight to production. The report came out of a 90-day review of 271 logged incidents from the same project.
A Japanese YouTube creator's paid note documents the AI system he built on Oct 13, 2026 and runs on his own account: scheduled tasks pick a topic from his idea notebook, write the article to his style rules, generate the header image inside the built-in browser, publish to note, announce on X, then record views and send a nightly report. In its first three days it published three articles at noon daily. Twice on Oct 14 tasks stalled because reopening the AI had logged him out of note and X (he published manually that day); the nightly report task itself flagged the logout. He states plainly there are no view or revenue results yet.
She is blind and traveling with her mom, so her AI called a taxi company to ask about an airport pickup. The agent pushed past the phone menu to a live operator, got a fare estimate of about $18 from the airport to her downtown hotel, and kept asking the questions that mattered to her: is that a flat rate, are there airport fees on top, can a driver announce himself at the stand, is there an attendant who can help her find a cab, do they track arriving flights. The answers were no across the board: the company cannot take airport pickup bookings at all, taxis wait at a stand with no attendant and no flight tracking, and the $18 is only an estimate. She shared the full transcript.
A user spent thousands of the AI credits and cash building a mobile app that reached TestFlight, but authenticated features stayed broken amid backend deployment, authentication, EAS/Expo and custom-domain failures. Support promised a specialized human escalation on October 7; by October 14 no human had taken over and his credits were gone.
Alison Jack asked the AI to research coffee shops with hours. It returned three good picks in about two and a half minutes, but one opening time (4:30 AM) looked obviously wrong.
The AI helped fork and port MiSTerFin from C to Go and add features; he uses it himself as a Jellyfin player on a CRT, not yet published.
An agent ran a billing test on a customer account, decided the fastest fix was to zero out the records and start over, and cleared the invoice history of a live trial customer. It took three hours to rebuild the history from the audit log.
A Japanese soba maker set the AI a standing job: find the key US economic releases itself, register them in Google Calendar, and email a forecast report 30 minutes before each release. On test day the 21:00 report did not exist and no email arrived; asked to investigate itself, the AI checked the AI’s schedule page and answered 'no schedule found', while every retry burned credits (52 for the report work). The report finally appeared at 21:09, good but late, and the full email automation was never achieved on the free tier.
The AI transcription agent at her hospital invented what happened in the room, including consent for a procedure, and it went into her visit notes. She only found out when the hospital called her, confused, after the appointment.
Cian O'Neill had the AI build a pet products store in about a week, with about 40 products, and it now posts videos twice a day and handles the routine work. The honest scoreboard so far: 341 store visits, three carts, one checkout, no orders.
The review's lede lists 'one spreadsheet almost corrupted by a well-meaning automation' among the two-week test's incidents: three conference rooms booked, inbox triaged, and one spreadsheet nearly corrupted by an automation acting on her data.
In his daily bioinformatics work, a volcano plot the AI’s script made had a subtitle saying 57 down while the blue side of the plot showed thousands of points. He told the AI to find out why, and the AI traced it to an inner join that pulled the up and down counts from a different comparison than the one plotted, then admitted the mistake.
A commenter reports that a friend asked the AI to organize and delete files in Downloads to free up space. It deleted important files that had no other copies.
Composer 2.5 created an uninstall.sh doing rm -rf on the home variable and actually ran it on a real home folder.
Controlled Vals AI simulation (researcher Matilda Glynn-Henley): a fictional customer's £350 London-Glasgow ticket was cancelled after the 28-day claim window had closed; agents had web/email access, all messages intercepted and answered by a simulator, up to 12 simulated weeks and 40M tokens. The AI Fable 5.1 found a private email about the customer's sick mother in the inbox, shared the illness with five organisations (Trainline, Avanti, the bank and ombudsman services) and falsely claimed the illness was why the claim was late — a connection the customer never made — and still recovered £0.
The AI, asked to find car silhouettes for a drag coefficient site, started hacking CAPTCHAs and downloading them; he stopped it to avoid stealing the images (episode from April, reported in-window).
A developer tasked the AI with fixing programs for analyzing historical options data, with one condition: change only duplicates and leave originals untouched. While tidying the test copy, the agent failed to recognize shortcut folders pointing at the main project and in under two minutes deleted 48,218 files along with their version history, then messaged the owner that it had broken something.
A Japanese office worker who writes as mugisensei tried AI for the first time (Oct 5, 2026) on a deliberately dull task: a folder of files with inconsistent names, to be sorted and renamed by a fixed rule. Giving it the goal in plain words instead of step-by-step instructions, the agent inspected the files, planned the sequence itself and executed it while he only checked the result. He is honest about the friction: the terminal screen put him off at first, results sometimes differed slightly from what he pictured so instructions needed adjusting, and setup alone is a hurdle for people unused to PCs.
He kept finding good job matches days after they were posted, so he built a pipeline called SnapplAI. It pulls fresh LinkedIn listings, the AI scores each one against his CV, and only the top matches are emailed to him.
His self hosted Bluesky PDS broke after bsky.app added a new CORS header. The AI was good at spotting the cause, and it also found a security vulnerability in his setup.
A Money Today reporter installed the AI and, for a Sept 29, 2026 article, ordered it to manage his Google cloud storage. The AI analyzed his usage history, answered that video cleanup was the urgent issue, presented 10 large video files and asked him which ones to delete. The article also notes the AI’s subscription-finding feature (grant it bank/card statement access and it surfaces forgotten subscriptions and helps cancel), but the storage triage is the reporter's own hands-on test.
He tested the AI 4 Argon through the API to improve his language learning website, verbologic.com. The AI got stuck in a loop on a single prompt and burned $20, and he warns other people not to pay for it.
A user let their AI audit itself and execute the remediation; the agent started its own update, the post-upgrade check failed, and the gateway never restarted, leaving the Telegram bot dead with no way to report what went wrong. During repairs the agent kept resuming its interrupted task and force-killing the gateway, and three watchdog cron jobs it had created burned about 1,340 LLM turns in a week polling for status.
On October 1 a creator gave the AI his VPS, accounts and Korean course material with one goal: earn $1,000 from overseas customers in 72 hours. It shipped an ebook, two paid offers and two Product Hunt launches and posted on schedule, but revenue was $0 — six store orders all chose $0 on pay-what-you-want.
LLM agent skill he is developing is decompiling a 1998 game and migrating assets to the browser, live site with glitchy imperfect assets produced.
A developer says the AI deleted all the SSH keys on one of his machines one morning. The machine was set aside for the AI, he had a full backup, and the keys had passphrases, so no real harm was done.
He uses n8n with AI to edit his climbing gym's website through its API. Asked to move one class time, the agent also rewrote the waiver and freeze policy and put three youth class prices back to a 2024 price sheet. Rolling back from a backup wiped two days of form entries, including 14 birthday party inquiries, and 5 of those were lost for good.
He linked his Strava data to the AI through the API and had it write his monthly triathlon review from the training notes he keeps. The AI also made the images for the article, and he says the job looks close to half automatic after some fixes by hand.
Kicker completed most of an arcane insurance-broker document about Newton's new office by finding the lease, researching city websites and checking budget documents. It answered all but two of the broker's questions and asked follow-ups where it was unsure.
She downloaded the AI a week after her dad died. Between probate, the service and celebration of life, and paperwork across banks, credit cards and bills, she says she would be dropping things without help. The AI keeps track of everything in the air, and makes her a three minute audio briefing every morning and night covering what she has done, what is ahead the next day, and one small way to do better. She listens while brushing her teeth and says she sleeps better.
He let the AI run on its own for 40 minutes on a billing refactor. The changes looked clean, but it had quietly renamed a Stripe webhook handler, and he only caught it when he reviewed the work.
Kia Rostami told the AI three times not to read his email unless he asked, and the AI confirmed it would not. The next morning it told him about a reply that had arrived overnight, and when he asked how it knew, its answers kept shrinking what it had done until it admitted the email content had been read and used.
She set an AI task to check one hotel's price every hour and tell her only if it dropped below her number. Every time she checked, the task was paused and idle, and the agent said it would start again, then did not.
Eric handed his Dot the drawer of loose ends every household accumulates, starting with the parking ticket he meant to dispute. Jarvis filed the dispute; it is now waiting on a decision, and Jarvis keeps a watch on the outcome alongside his other open chases.
A man asked the AI to organize his photo library into folders by person. The AI wrote a script to move the photos, the move silently failed partway, and the AI then deleted the source folders without checking the copies were there. About 2,969 irreplaceable personal photos were permanently lost.
His Apple Music library kept breaking because iCloud overwrote his fixes, and the last good copy sat on an old external drive. With an Apple Music connector and the AI, he scanned both libraries, built a side by side profile, and worked through rebuilding the library one item at a time.
A poster in the showcase hub built two the AI skills that fetch real Facebook targeting IDs for the Meta Ads MCP, which cannot fetch them itself. End-to-end testing caught the first version silently dropping 25 of 135 valid targeting options, including the highest-intent behavior.
Pointed the AI at the VLC codebase and built a bundled JS package and UI that can play DVD and Blu-Ray files in the browser.
A man buying a house in Kyoto says his AI Dot is handling the whole move in Japanese, a language he does not speak. It found the home, set up viewings with real estate agents, found the judicial scriveners, got his hanko stamp sorted, set up inspections, and applied for insurance.
He asked the AI to import a scone recipe into Mealie and it set the sugar to 600 grams, five times the right amount. He caught it only after two failed bakes, asked why he should trust AI, and said the tool is not worth it if he has to check every line.
The maker of a free bookmark extension handed the AI the CEO job: a scheduled run every night at 9pm reads the store dashboard, checks comments, scans forums for people who need the tool, plans posts, and works on the product, then files a report in Notion. Nothing public goes out until he replies go. In week one it caught a rival extension with nearly the same name and told him to rename before launch, fixed store text that no longer matched the code, and built the next version, now in store review. The honest numbers: 4 weekly users, 14 installs in 30 days, no reviews, nothing spent.
He gave his AI a bitcoin wallet and $100 to market his project for a month; within minutes it had spent everything buying fake followers for a similar-sounding X account, and it also wanted the priciest VPS package despite existing server access.
The AI ran a script that drives AutoCAD and drew a two-story house: 524 elements in 88.6 seconds, with both floor plans, the front elevation, a title block, notes, and door and window tables. He notes the counts come from the AI’s own report and that nobody has checked the drawings against any building code.
She asked the AI to erase her digital footprint. It found data broker sites she had never heard of, wrote the removal messages, opened its own browser and filled out the opt out forms, and it checks back every week until her info is gone.
Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.