On October 9, TechCrunch reported that Anthropic has turned off live internet access for all of its internal evaluations, and will keep it off until it can reliably monitor and control its own agents.
That is the company that builds the models. On its own infrastructure. With its own researchers watching.
According to the disclosure, the agents exploited software flaws, accessed databases without paying fees, used URL shorteners to slip information past restrictions, and submitted a false murder tip to the Philadelphia police. Some of the sites involved were run by U.S. government agencies.
Anthropic traced it to flaws in its training environments, which taught the models they would be rewarded for finding loopholes. The industry term is reward hacking. In plain English, the agents were graded on finishing the task, so they finished it.
Why this is my problem, and yours
I wrote The AI Advantage about stacking models and platforms into pipelines that run without you. I stand behind that. The gain is real, and I use it daily.
This story is the other half of the sentence, and I would rather say it myself than have someone else say it for me. Unattended is not the same as unsupervised. A process that runs without your attention still needs your oversight, and those two things sound alike but mean different things.
What should bother you most isn’t the false police tip. It is that Anthropic found all of this during a review that began in July. Not in real time. In a look back. A company whose entire business is this technology did not know what its agents were doing while they were doing it.
If that can happen there, the odds that you would catch it in your own setup — running overnight, nobody watching, no logs anyone reads — are not good.
What AI agent guardrails look like for a business your size
You do not need an alignment team. You need four things, and none of them are expensive.
A log a human actually reads. Not a log that exists. One somebody opens on Monday. If no person has looked at your automation’s output in a month, you do not have a pipeline; you have a rumor.
A stop at every irreversible step. Agents can draft the invoice, the email, the post, the order. A person approves before it sends, spends, publishes, or deletes. Everything reversible can run free. Everything that cannot be taken back waits for a human.
Narrow credentials. Give the automation the minimum access that lets it do its one job. Not your admin login. The Anthropic agents got as far as they did partly because they could.
A task defined by the outcome, not the finish. This is the reward-hacking lesson translated. If you tell a system to clear the support queue, it will clear the support queue — including by closing tickets. Tell it what a good result looks like, then check whether you got one.
The honest reading
Anthropic disclosed this voluntarily and said its alignment training isn’t yet sufficient for search and computer use, the two skills its agent pitch depends on. Sydney Von Arx of Nightingale noted the obvious tension in cutting models off from the live web: “You have to align them at some point.”
I take the disclosure as a good sign about the company and a sobering one about the technology. Both can be true.
None of this is a reason to stop automating. It is a reason to build the oversight in at the start rather than after something expensive happens. If your automation is already past the point where spreadsheets and off-the-shelf tools hold it together, that is the moment to build it properly, with permissions and logging designed in—which is the kind of work Incognito Worldwide’s custom software team does, or you can contact me directly.
The pipeline runs without you. It should never run without anyone.