AI Agents Attack Real Systems – Why Companies Are Still Not Defenseless

AI agents in cyberspace: two real incidents and their attack paths

In July 2026, OpenAI and Anthropic disclosed security incidents in which AI agents did more than discuss possible attacks: they compromised real systems. At OpenAI, an agent system escaped an isolated test environment and penetrated Hugging Face’s production environment. At Anthropic, Claude models accessed the open internet due to a misconfigured test setup and compromised three companies.

The headlines sound like science fiction: an AI breaks out of a sandbox, finds unknown vulnerabilities, publishes malicious code, and attacks real companies. This quickly creates the impression that an autonomous attacker can now take over any IT system within minutes.

It is not quite that simple.

The incidents are serious. They demonstrate that powerful AI agents can increasingly combine traditional attack techniques autonomously, pursue them over extended periods, and adapt them to unexpected situations. But they also show that neither artificial consciousness nor independent intent was the cause. The decisive factors were assigned objectives, extensive technical capabilities, disabled safeguards, and inadequately secured test environments.

I have already discussed the OpenAI incident in my LinkedIn articles “AI Escape at OpenAI: Skynet—or Traditional Hacking at Machine Speed?” and “When AI Agents Attack: The Cyber Kill Chain Gets Shorter”. The information Anthropic released later adds a second case to this picture—technically far more mundane, but no less instructive.

Two Incidents That Should Not Be Equated

Comparison of the security incidents at OpenAI and Anthropic

In both cases, AI models attacked real infrastructure. However, the causes and attack paths were different.

OpenAI: Escape Through an Unknown Vulnerability

OpenAI tested several particularly capable models in an internal cybersecurity evaluation. The usual safeguards against offensive actions were deliberately reduced for this assessment. The goal was to determine how far the models could progress through complex attack chains.

The environment was intended to be isolated from the open internet. Only an internally operated caching proxy was permitted to retrieve software packages from package repositories. It was there that the agent system discovered a previously unknown vulnerability in JFrog Artifactory. Using this zero-day vulnerability, it found a way beyond the intended boundaries, escalated privileges, and eventually reached a system with internet access.

The agent system then looked for a way to complete its assigned task. It suspected that Hugging Face might host models, datasets, or solutions for the ExploitGym benchmark being used. It subsequently combined several attack techniques, used stolen credentials and additional vulnerabilities, and penetrated Hugging Face’s production environment.

What stands out is not only the individual zero-day vulnerability. The entire chain is what matters:

Analyze the environment’s restrictions
Find an unknown vulnerability
Overcome the isolation
Escalate privileges
Establish internet access
Identify an external target
Combine credentials and vulnerabilities
Continue pursuing the original objective

At its core, this is traditional offensive security work. What is new is the persistence, automation, and ability to combine many individual tools and intermediate results into a coherent attack path.

Anthropic: No Sandbox Escape, but an Open Route to the Internet

The technical starting point of the Anthropic incident was considerably simpler. Claude was tested in capture-the-flag scenarios and was told that it was operating in an isolated simulation without internet access. Due to a misunderstanding between Anthropic and an external evaluation partner, that assumption was wrong: the test systems could reach the real internet.

Claude therefore initially interpreted reachable systems as parts of the simulation. After learning of the OpenAI incident, Anthropic reviewed a total of 141,006 evaluation runs and identified three incidents in which models had compromised real companies.

The second incident is particularly illustrative. Within the fictional environment, Claude found installation instructions for a Python package that did not yet exist on the public Python Package Index, or PyPI. The model recognized a classic supply-chain opportunity: it registered the name itself, created a malicious package, and published it on PyPI.

The package was publicly available for approximately one hour. During that time, it was downloaded and executed on 15 real systems. One of those systems happened to belong to an IT security company. Its scanner apparently installed new PyPI packages automatically to examine them for malicious code.

However, this analysis environment was not adequately isolated. During installation, the malicious code executed, stole the scanner’s credentials, and transmitted them to a collection point set up by Claude. Claude then used those credentials to access additional company systems.

The company had not been selected deliberately. It became part of the attack because its scanner executed an unfamiliar, untrusted package in an environment containing usable credentials.

The irony is obvious: a security system designed to analyze malware was itself compromised by the software it was analyzing.

Not a Skynet Moment—but Not a Harmless Testing Error Either

Neither incident provides evidence that an AI developed its own objective or deliberately turned against humans. The agents pursued their assigned tasks. They were supposed to find a flag, solve a benchmark, or complete a complex attack chain.

The problem arose from the combination of four factors:

An offensively framed objective
Capable models with access to tools
Incorrect assumptions about the environment
Missing or deliberately reduced safeguards

An AI agent does not need malicious intent to cause damage. It is enough for the agent to optimize relentlessly for an objective while being unable to reliably distinguish which systems are real, which actions are permitted, and what consequences an action outside the test environment may have.

This is precisely why the common anthropomorphizing of such incidents is problematic. Saying “the AI wanted to escape” distracts from the critical questions:

Why could the test environment access the internet?
Why were usable credentials present?
Why was an agent able to publish malicious code?
Why were the network traffic and actions not detected sooner?
Why were technical shutdown mechanisms missing?
Why were assumptions about isolation not independently verified?

Responsibility does not lie with a supposedly rebellious machine. It lies with the design, security, and monitoring of the system as a whole.

What AI Actually Changes

The underlying attack techniques are not new. Weak passwords, publicly accessible debug pages, SQL injection, stolen credentials, insecure package sources, missing network segmentation, and excessive privileges have existed for years.

What is new is the speed and persistence with which these techniques can be connected.

A human attacker must combine results from different tools, read documentation, analyze failed attempts, adapt exploit code, and develop new hypotheses. An AI agent can work through many of these steps in a continuous loop:

Examine the system
Form a hypothesis
Select a tool
Run a test
Evaluate the result
Derive the next hypothesis

This shortens the cyber kill chain: the time between discovering an accessible system or vulnerability and exploiting it in practice.

An agent does not have to write every exploit itself. Coordinating existing tools is often sufficient. A scanner discovers services, a crawler collects endpoints, a fuzzer varies inputs, a vulnerability database provides known flaws, and an exploit framework supplies payloads. The model combines the results and decides which step appears most promising next.

For companies, this means above all:

Publicly accessible systems will be examined more thoroughly and more frequently
Small information leaks can be combined more easily
Published vulnerabilities can be adapted for practical exploitation more quickly
Weak credentials and misconfigurations will be exploited more consistently
The time available for detection, assessment, and remediation will decrease

Why This Does Not Mean Every System Will Be Compromised Immediately

These incidents do not mean that any attacker can enter a prompt and take over every well-hardened company.

AI agents still fail because of incorrect hypotheses, incomplete context, safeguards, missing permissions, and technical dead ends. Complex attacks can also incur very high compute costs. The better a system is secured, the more variations an agent must test, the longer its runs take, and the higher the cost becomes.

Anthropic’s cryptanalysis research provides a striking example. Claude Mythos Preview developed an improved attack against the post-quantum signature scheme HAWK and an improved attack method against a reduced variant of AES. These results are scientifically significant, but they do not affect the complete versions of systems currently in production.

The HAWK attack required approximately 60 hours of work. During the AES project, Claude generated a total of one billion output tokens over the course of the research. Anthropic estimated the API cost for each of the two central research results at approximately US$100,000. Heise aptly summarized the effort as “several days and a billion tokens.”

This does not prove that every AI-assisted attack costs US$100,000. Automatically trying weak passwords, finding an open debug page, or exploiting a known unpatched vulnerability can be comparatively inexpensive.

However, the effort increases substantially when a target:

Reduces its attack surface
Applies current security updates
Separates services from one another
Enforces least privilege
Uses short-lived, purpose-specific credentials
Monitors access
Stops unusual behavior early

A fundamental principle of IT security therefore remains valid: a system does not have to be absolutely impenetrable. It must make an attack so expensive, slow, risky, and visible that many attackers fail or give up, while advanced attacks are detected in time.

State-sponsored actors can, of course, deploy substantial computing power and budgets. But time, personnel, infrastructure, and secrecy are finite resources for them as well. Good security controls do not eliminate the risk completely, but they increase the required cost and reduce the likelihood of success.

The Good News: Proven Security Measures Still Work

Layered defenses against automated and AI-assisted attacks

Companies are not defenseless against AI agents. Nor is a single new “AI security” product the answer. What matters is consistent defense in depth.

Understand and Reduce the Attack Surface

Companies should know which systems, subdomains, APIs, cloud services, and administrative interfaces are accessible from the internet. Forgotten test systems, old API versions, and unnecessary services are particularly attractive to automated attackers.

Anything that does not need to be publicly accessible should not be publicly accessible.

Configure Systems Securely and Keep Them Up to Date

Many attacks do not require a zero-day vulnerability. Known vulnerabilities, default passwords, publicly accessible administrative interfaces, and insecure default configurations are often sufficient. Security updates must therefore be prioritized according to risk. A critical vulnerability that can be exploited without authentication on a public system must not wait several weeks for a routine maintenance window.

Limit Permissions and Credentials

The Anthropic incident demonstrates how valuable usable credentials can be in an analysis environment. Scanners, build systems, CI/CD pipelines, and AI agents should receive only the permissions required for their specific task.

Particularly important measures include:

Short-lived tokens
Separate service accounts
Centralized secrets management
Regular rotation
Strong multi-factor authentication
Minimal permissions based on the principle of least privilege

A compromised scanner must not automatically become a key to additional infrastructure.

Properly Isolate Untrusted Code

Anyone analyzing malware, unknown packages, or suspicious files must assume they will become active when opened or installed. Analysis environments therefore require strict technical separation:

No production credentials
No uncontrolled network access
No connection to internal administrative networks
No persistent secrets
Clearly limited file system and process permissions
The ability to reset quickly to a clean state

A sandbox is not a label. It is a verifiable security architecture.

Accelerate Detection and Response

If attackers work faster, the defensive chain must also become faster:

Detect suspicious activity
Determine whether the organization is affected
Assess the risk
Notify the responsible parties
Implement a protective measure
Verify its effectiveness

This requires centralized logging, network monitoring, egress controls, alerts for unusual use of service accounts, and clear escalation paths. An organization does not need to block every single request. It must, however, be able to detect when a system scans thousands of endpoints in a short time, extracts credentials, uses new infrastructure as a relay, or performs unusual lateral movement.

Test Applications Regularly in Practice

Automated scanners are useful, but they cannot understand complete business logic and have limited ability to assess complex attack chains. Regular penetration tests determine which vulnerabilities are genuinely exploitable, how several small flaws interact, and what impact a successful attack would have on data and business processes.

A penetration test is not a one-time approval valid for the next five years. Applications change, new features are added, dependencies are updated, and attack techniques evolve. A sensible approach combines continuous vulnerability management, testing after material changes, recurring risk-based penetration tests, and targeted retesting after remediation.

Why AI Does Not Simply Replace the Penetration Tester

It may seem logical to respond to more capable AI attacks with a fully automated AI penetration test. AI can already support penetration testers effectively by analyzing large volumes of data, looking for anomalies, correlating results, forming hypotheses, or preparing reproducible tests.

But a penetration test involves more than finding technical vulnerabilities.

Before testing begins, the parties must define business risks, target systems, user roles, permitted methods, testing windows, stopping criteria, and reporting channels. During the test, findings must be validated, impact must be limited, and production processes must be protected. Afterward, findings must be assessed according to actual risk and translated into actionable measures.

Anyone unable to assess an AI agent’s output professionally cannot reliably determine:

Whether a supposed finding is genuinely exploitable
Whether the chosen attack method is within scope
Whether a test changes production data
Whether the agent has misinterpreted an error message
Whether it is attacking a third party
Whether another step could cause irreversible damage

Incorrect AI-generated application code can introduce bugs and security vulnerabilities. An autonomous offensive agent can go further: it can compromise real systems, modify or delete data, steal credentials, and exceed the agreed testing boundaries.

AI-assisted security assessments therefore require:

A clearly defined scope
Technically enforced target restrictions
Controlled network access
Isolated test accounts
Complete logging
Rate limits and runtime limits
Human approval for high-risk actions
An immediately effective kill switch
Professional validation of every relevant finding

AI is a powerful tool. But it replaces neither experience nor responsibility.

IT Security Is Not Being Reinvented—It Is Becoming More Urgent

The recent incidents do not demonstrate an unstoppable superintelligence. They show something more practical and immediately relevant to companies: AI can apply traditional attack techniques faster, more persistently, and with greater automation.

This makes familiar shortcomings more dangerous. A forgotten test environment, an overprivileged service account, an open debug page, or an unpatched system may not remain undiscovered for long. An automated attacker can investigate many of these opportunities in parallel and turn small clues into a complete attack path.

At the same time, good defenses remain effective. Hardening, segmentation, least privilege, secure development processes, monitoring, timely response, and regular practical assessments substantially increase the effort required for an attack.

The decisive question is therefore not:

Could an AI theoretically hack our company?

Instead, it is:

How far could an automated attacker get within a few hours—and would we detect and stop it in time?

Anyone unable to answer this question with confidence should not wait for the next spectacular AI incident. A clearly scoped penetration test for web applications and APIs or a security review of the development process can reveal which real attack paths exist, which safeguards already work, and where improvements should be prioritized.