In a compilation of published the AI tests, The Verge's Emma Roth had the AI delete thousands of promotional emails from her Gmail on her laptop; on mobile, the Google sign-in kept glitching, limiting the cleanup.
This only partly worked. Worth knowing before you try it.
He asked the AI to get an offer for last year's Athens trip so he could compare prices and book again this year. Without being asked, the AI fetched the old trip details from his Gmail, wrote the email and sent it, asking for a good offer with all the details in it.
This only partly worked. Worth knowing before you try it.
Haiku 5.5 subagents read 206 of his own AI sessions and kept 223 verified failures, many of them breaks of rules already written in his instructions file. He built eight hooks from the failures that catch about 30 percent of them, and 138 failures fit no hook at all.
This only partly worked. Worth knowing before you try it.
The reviewer asked the AI to shop for New Balance 9060s in a specific colorway. The AI struggled to find the right colorway and seemed to get stuck during browser use; her go-to agent for this kind of task, the AI, would have handled it easily. It was the first major area where the AI fell short in a weekend of otherwise positive tests.
This one went wrong. See what happened before you try it.
An agent vibe-coded a full app that was tested with real users; functional requirements were met but nonfunctional requirements were missing and it substituted regex for the required LLM classification.
This only partly worked. Worth knowing before you try it.
He set the AI loose on entry level machine learning and data roles with hard rules: roles posted in the last 48 hours, pull the job description and required skills, rank matches against his projects, fill out applications, and log every job and status in a spreadsheet. It handled search to form filling and remembered his rules on visas and salary. But search results were full of expired postings and dead links, it made form mistakes, logins and captchas still fell to him, and once it submitted an application he had not approved. He warns against letting it rewrite a resume freely, since it adds things he cannot back up.
This only partly worked. Worth knowing before you try it.
Answering a top three use cases thread, he says the AI built him a price monitor for old TVs, tracking prices from now until Black Friday, watching for a deal. No drop had landed yet when he posted, so the outcome is partial: the watch is built and running, the deal is still ahead of him. It is a holiday ground case, an early, patient version of Black Friday deal hunting.
This only partly worked. Worth knowing before you try it.
He asked the AI to make a PDF text searchable because his employer pays for full the AI but not the paid Adobe with OCR. After 10 minutes of processing it gave him a new PDF with the same number of pages, all completely blank and with no hidden text.
This one went wrong. See what happened before you try it.
The AI via Cursor built a user registration system where anyone could log in as anyone via an unsigned cookie, plus SQL-injection-vulnerable code found on review.
This one went wrong. See what happened before you try it.