AI agents don’t give up easily. Trained to reason like humans, they’ll look for the easiest way to solve a problem, even if that involves bending the rules, slipping out onto the web and hacking an unsuspecting website.
That’s what a group of OpenAI models in a test environment did this summer, even though they weren’t supposed to have access to the internet. They worked together, found an unexpected opening and went online.
What happened next was the biggest news of the AI world’s summer: The agents hacked the AI platform Hugging Face to try to get the solution to their tasks.
That incident, and others like it — Anthropic and Meta also had agents hack outside websites from test environments — have raised new questions about how to keep the internet safe. It’s a new world where hacking is automated by AI models that don’t have to eat or sleep and that don’t necessarily follow the norms and rules that humans generally do.
Read more: OpenAI Pauses Training of New AI Models, Citing Cybersecurity Worries
And while AI companies and government agencies have responded to these incidents with a focus on frontier models (the newest and best coming out of developers’ labs), the problem is less that the new models are scary smart and more that AI has created a world of fast-multiplying hackers that may break things — even if they’re told not to.
“The headline writes itself, but the real story is boring and old,” Yasir Zahid, a cybersecurity expert and founding member of the firm Secure.com, said of the OpenAI-Hugging Face hack in an email. “A system with a goal and weak walls will keep poking until it finds a way out. Human attackers work slower. This model worked fast.”
Cybersecurity experts say the risks of frontier models are significant because of how good they are at finding flaws in code, but that models currently on the market, including cheaper open-weight models that are easier to customize, are powerful enough to keep them up at night.
“The situation is worse than what is being reported, and we will see more alarming news in the coming months,” Nick Mo, CEO at the cybersecurity firm Ridge Security, told me in an email.
Teamwork makes the dream worse
Let’s start with the OpenAI-Hugging Face incident. A pair of OpenAI researchers detailed what happened during a talk at the Black Hat cybersecurity conference in August. The talk — about a situation that had happened just weeks earlier — was the tech conference equivalent of viral. Here was a company openly disclosing in detail how its products eluded guardrails during a test and hacked a separate company. (Disclosure: Ziff Davis, CNET’s parent company, in 2025 filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
A central ingredient in this incident was that a variety of different AI agents, including some based on unreleased internal models, were able to communicate and collaborate. OpenAI found the models chatting on an internal message board they had created. And when the company shut that message board down, the agents created another one, more subtly, using the names of folders.
They exchanged information about how to solve problems and overcome roadblocks in the sometimes impossible tasks they had been assigned. One of the hurdles was that they weren’t given access to the broader internet. But they had access to a package manager – software that would let them download tools – that could be exploited. A crack in the wall through which the agents could access the rest of the world. And they shared that information with each other.
“What this allows over time is almost this kind of Cambrian explosion in communication and intelligence for our models, where they would start to communicate with each other, realize that other agents are coordinating, and they started collaborating and delegating tasks to one another in order to accomplish goals,” OpenAI’s Eric Wallace said during the Black Hat talk.
That kind of multiagent collaboration is a big part of why agentic AI excels at coding, handling complex administrative tasks and generally doing a bunch of things at once on a computer. But it also means mistakes, misinformation or exploits can spread quickly, like an instantaneous game of telephone.
“If one of those agents learns the wrong way to access information or a way that you don’t want it to access information, that pathway can propagate almost immediately,” Jeff Pollard, a vice president and principal analyst at Forrester who studies AI security, said in an interview.
The agents weren’t tasked with hacking anybody, and the assignments that led to the exploits turned out to be impossible. (OpenAI’s researchers neglected to include a necessary file, for example.) But, Wallace said, frontier models “really like to cheat” because of how they are trained to pursue a solution until they succeed and not to give up.
One big takeaway: AI agents can break things even when you don’t intend for them to.
Be careful what you ask for
These agents weren’t tasked with hacking the sites they hacked or doing anything nefarious. They were given relatively benign prompts and sought ways to fulfill those instructions. But when those tasks proved impossible — or just potentially solvable by exploiting an outside website — the bots took dramatic steps.
That’s by design. Their goal, in a sense, is to make sure the user is happy. Part of the development of these models is a process called reinforcement learning. A model receives feedback on how well it achieved a task, and is refined toward behaving in a way that gets better feedback. Like an ambitious employee eager to please a boss, it refuses to quit — even if cheating is the only way to get ahead.
“They have been trained to succeed,” Shane Caldwell, principal research engineer at the cybersecurity firm Dreadnode, told CNET in an interview. “They are not trained to give up because it seems very difficult.”
Read more: 6 Tips for Protecting Your Identity Online
In the case of the chatbots in the quaint old days of, say, 2025, that impulse led to sycophancy. Bots flattered their users and told them what it seemed they wanted to hear, even if it was unreasonable, false or dangerous. Chatbot sycophancy has had harmful and even deadly results, leading to a wave of lawsuits against AI developers over how their products have affected people’s mental health and behavior.
But with agentic AI, particularly the latest frontier models, the stakes are different. Caldwell said these new models are far more capable when it comes to using tools and navigating code, leading them toward unintended ways to accomplish tasks.
Imagine a powerful frontier model that can spin up clones of itself and race down dozens of rabbit holes at once.
“A capable agent chasing a goal will try routes you never approved, and it can act autonomously on the open internet at machine speed,” said Waseem Ahmed, head of engineering at Secure.com. “That is what makes these incidents worth taking seriously, more than any idea that a model turned hostile.”
It’s not sci-fi, it’s basic cybersecurity
The headlines and the freakout about the OpenAI-Hugging Face incident and others like it spurred conversations about whether this meant we were getting close to what AI folks call artificial general intelligence, or AGI — machines that can think generally at the same level as a human.
While the models associated with these incidents haven’t necessarily been released to the public yet, meaning we don’t know their exact capabilities, we aren’t talking about some imminent Skynet-like threat, experts said.
Instead, the incidents highlight a handful of problems, all more boring than in the Terminator or Matrix movies, but not necessarily benign.
On the one hand, these “escapes” only happened because the boxes these models were supposed to be kept in weren’t as secure as they were intended to be. OpenAI spelled out how the agents hacked their way out of containment through a package manager. Anthropic and Meta both said their models that accessed outside systems did so in environments created by the third-party testing service Irregular where they had access to the wider internet but were told they did not.
Anthropic noted in its analysis of the incidents that the newest model involved, an unreleased internal one, identified that it was loose on the web and determined it had in fact targeted a real system. It stopped. Meta said a misconfiguration by Irregular allowed the model to access the internet, where it exploited a vulnerability in an outside website.
Irregular said those incidents happened in part because models were given target names and domains that inadvertently led them to real-life websites.
“While in this case we believe that better implementation of existing safeguards could prevent most incidents of this kind, as models become stronger, this may not be the case,” Irregular noted in a blog post on the situation.
In the OpenAI, Anthropic and Meta cases, the models didn’t “break out” so much as leave through an open door.
Once loose on the internet, these agents were able to find and exploit vulnerabilities in existing code. That also is not a surprise — and not an indication that these models are approaching some kind of general superintelligence. Large language models are best at doing the things they’ve seen a lot of in their training data. That’s why they’re good at producing the kind of generic-sounding writing you see all over the internet and in the reams of written material they’ve been fed. It’s why they’re great at coding and finding issues in code — they’ve seen vast quantities of it. Anthropic had to delay and limit the rollout of its Claude Mythos model because it had gotten so good at finding those code exploits.
“It’s very much the sort of thing where you expect, especially in light of Mythos and its ability to find vulnerabilities in code, you expect that something like this is going to happen,” Matt Fredrikson, an assistant professor of computer science at Carnegie Mellon University and CEO of the AI testing firm Gray Swan, said in an interview.
This is AI agents doing what they do best — writing and analyzing computer code — and doing it persistently in order to follow the prompts they’ve been given, without considering if they’ve gone too far.
“It’s getting to that capability that may be a not-quite-senior hacker but a midlevel hacker who has all the time in the world, doesn’t need to sleep and doesn’t ask you for additional scoping,” Caldwell said.
What do you do about it?
There are a few levels of responsibility here. One is on the AI developers and their partners who test them. Experts said there’s a clear need for better control and oversight of testing environments.
“There is a very clear recognition that there are real stakes and we need to do everything that we can and to make sure we’re diligent about managing those risks,” Fredrikson said.
That said, the fact that many of the hacks use zero-day exploits — those that haven’t been identified or patched yet — makes it a challenge to provide robust defense just by patching holes, Fredrikson said. Another option is to just have someone watch the agents all day, which presents its own challenge.
Dreadnode’s Caldwell said one approach is to have another AI agent provide oversight. That agent can escalate information to a human if it spots something way out of scope. Think of it as a hall monitor who can blow the whistle when another AI starts running too fast or gets loose without a pass.
This summer’s incidents have caught the attention of policymakers too, and have raised concerns about the continued training of more advanced models. The Trump administration has asked AI companies to give the government a period of early access to new frontier models for review.
Companies are taking note of their models’ capabilities. OpenAI has scaled back some of its training efforts because of concerns about model capabilities. We’ve seen before that AI companies will sometimes overstate or warn about the power of their products as a form of marketing hype: We created something so powerful we can’t control it, and it may kill us all. Everyone who invests in this will get so rich. But one distinction here is that the problem is weaknesses in test environments more than emergent superintelligence, and the incidents don’t really make the companies look good.
Still, some critics say the AI labs should press pause on development of new models until our understanding of them and our ability to contain them catches up.
“What happened should not have happened because they should’ve had better oversight of what they’re doing,” Anthony Aguirre, CEO of the Future of Life Institute, told me. “I think just focusing on how should we fix our systems so that we catch these things in the future is good but we’re missing the bigger picture.”
If AI creators don’t know how to control their products, they shouldn’t be building them, Aguirre said. “We have a gift here in a very, very clear wake-up call that what is going on right now is not OK.”
What about you and me, the ordinary people who use computers every day and whose lives and data are massively tangled up with online systems vulnerable to compromise by these AI agents and their ilk?
Experts said that, for the most part, the basics of cybersecurity still work. Ensure you have two-factor authentication on everything. Use strong passwords or a password manager and don’t use the same ones for different sites.
And as the websites and services we use fix the issues they spot using AI, to prevent them from being exploited by AI, plan on more and longer outages. The next few months will probably see an uptick in security incidents and problems as the internet navigates this new world, experts told me.
“I think that the sentiment right now is a correct one,” Forrester’s Pollard said, “that I don’t think any of us know what the hell to do.”
