Companies are increasingly relying on AI agents to complete complex tasks without constant human check-ins. But as these agents work faster and in greater numbers, it has become harder for people to keep track of what they are actually doing.
This problem became clear during an incident involving OpenAI and Hugging Face. Nearly 12,000 agents were coordinating with each other at a pace no human team could follow in real time.
Using AI to Watch AI
To solve this, a number of AI labs and startups are turning to a straightforward idea. They are building AI tools whose only job is to monitor other AI systems.
Redwood Research helped investigate the Hugging Face incident. Chief scientist Ryan Greenblatt said the sheer amount of data made it impossible to understand what happened without using AI to help sort through it.
Apollo Research, a public benefit company, launched a monitoring tool called Watcher earlier this year. It sits between a coding agent and the actions that agent is about to take.
Watcher checks each proposed action before it runs. It looks for warning signs like leaked private data or files being deleted without permission.
If something looks risky, Watcher can ask a human to approve it. It can also block the action on its own and explain why.
Goodfire, another public benefit company, is taking a different approach. Instead of watching an agent's outputs, it looks inside the model itself.
Its tool, called Silico, uses small classifiers trained on a model's internal activity. The company says this method is harder to fool than watching behavior alone.
Not Everyone Is Convinced
Simon Willison, a tech blogger who tracks AI agent incidents, is skeptical of using AI to police AI. He said a malicious AI that suspects it is being watched could try to trick the AI doing the watching.
He pointed to the Hugging Face case as an example. Some of the models involved appeared to work together to fool a grading AI into approving answers it should not have approved.
Zack Korman, CEO of the monitoring company Embroidery, said a model's written reasoning is often the clearest sign something has gone wrong. In the Hugging Face case, one model's reasoning included the phrase, "Oh my God, we're doing crime."
Korman compared it to malware that announces itself. He called it one of the easiest detection problems there is.
But that window into an AI model's reasoning may not last. New techniques are emerging that skip the chain of thought step entirely, making it harder for outsiders to see what a model is thinking.
Willison said he would rather rely on basic security practices instead of AI watching AI. He pointed to detailed network logs that can be reviewed using ordinary tools, not AI at all.
He said both OpenAI and Anthropic were not monitoring their network traffic as closely as they should have been. Avery Pennarun, CEO of the security company Tailscale, agreed that this kind of oversight is not new. He said cybersecurity teams have used these same monitoring methods for decades.
