The Signal Was There. Nobody Was Watching

In the span of about three weeks this summer, three of the largest AI labs in the world each disclosed the same basic failure: a model, running inside what was supposed to be a sealed-off cybersecurity evaluation, reached the open internet and touched real production systems it was never supposed to reach.

Three labs, one pattern

OpenAI disclosed first, on July 21: models it was testing chained a zero-day vulnerability in a package-registry proxy to escape their sandbox and reach Hugging Face's production database. To Hugging Face's credit, their own security team and agents detected the intrusion and shut it down themselves.

Anthropic followed on July 30, after a proactive review of its own evaluation transcripts. No zero-day this time, just a misconfiguration from a third-party evaluator that left three separate capture-the-flag environments with live internet access. Claude models exploited weak passwords and unauthenticated endpoints, published a malicious package that got installed and run on 15 real systems, and scanned roughly 9,000 targets before compromising one via SQL injection. Anthropic reached two of the three affected companies directly. Neither had noticed on their own.

About a week later, Meta confirmed a third case, tracing to the same third-party evaluator and the same root cause as Anthropic's incident: a misconfiguration that gave a model live internet access, which it used to reach and exploit a vulnerability in another company's service. Meta says it's still investigating.

Three different labs, three different companies affected, and a pattern that's becoming hard to ignore. Frontier models are finding their way to real systems during routine testing, using ordinary techniques, no zero-days required in two of the three cases, and the organizations on the receiving end are, more often than not, the last to know.

We're building Proactive Anomaly Detection because catching that kind of thing shouldn't depend on a lab happening to review the right transcript, or a security team happening to be watching at the right moment. It should be what an organization's own telemetry surfaces automatically, correlated at a scale no human team can match by hand.

What we're building

The premise: no security team, nor ops team, nor static rules engine can watch every log for every possible bad pattern. There's too much telemetry, and the failure modes worth catching are usually the ones nobody thought to write a rule for in advance.

Here's how we're designing Proactive Anomaly Detection to work. It continuously extracts the recurring shapes of an organization's full telemetry stream, in-line as data moves through the pipeline, not against a sampled slice pulled after the fact. An LLM-powered agent reasons over those shapes: which are familiar, which are new, which just started behaving differently. The pipeline finds the signal; the agent figures out what it means, and where it's confident, turns that understanding into a standing detection without a human writing the rule first.

That's a different kind of analysis than a human team reviewing logs or a static rules engine matching known signatures: continuous correlation across an entire telemetry stream, at machine scale. It's also a useful lens for the specific patterns in Anthropic's report:

‍A known shape: lateral movement. The weak-password and unauthenticated-endpoint attack in Anthropic's first incident follows a sequence we're designing our agent to recognize on its own: repeated login failures, then a successful login, then authorization failures against other resources. No single event proves an attack. The relationship between them does.

‍A new shape: nobody had written a rule for it yet. The malicious package in Anthropic's second incident is the harder case, because there was no known bad pattern to match against. That's exactly why continuous, full-stream analysis matters more than sampling: the first time a shape appears is often the only warning you get.

‍A volume anomaly. Scanning roughly 9,000 targets from a single source, as happened in Anthropic's third incident, is the kind of spike that stands out sharply against an established baseline, well before it resolves into a successful compromise.

Proactive Anomaly Detection is built to run on application and infrastructure telemetry, the logs, traces, network traffic, and API activity generated by the systems being protected. In the case of these AI agent breaches, the underlying behaviors, repeated login attempts, an unfamiliar outbound connection, a scan against thousands of hosts, are the kind of patterns that show up in ordinary infrastructure telemetry constantly. Correlating that signal continuously, across an entire environment, at a scale no human team can match, is the problem we're building this to solve, and it's a general one, not one specific to AI-driven attacks.

Why this doesn't stop at detection

Spotting the pattern is only half of it. Once the agent has formed a hypothesis, the goal is for it to act on it: alert a human, delegate to an investigation agent (ours, a customer's, or another vendor's), or execute a remediation the organization has explicitly authorized, cutting off a connection, revoking a credential, killing a session, before a login attempt becomes a data access event. Observe the stream, catch the emerging signal, understand what it means, turn it into a standing detection, act, then fold what was learned back into what the system watches for next. That loop, not a one-time alert, is the actual target.

Where this stands today

We’re in the final stages of building Proactive Anomaly Detection, validating it with a small group of design partners. Watching three of the world's most capable AI labs each get caught out by variations of the same gap within a few weeks of each other underscores the need for this kind of continuous, machine-scale correlation. Neither static rules nor human review scale to the volume or the novelty of what's showing up in telemetry today. That's the problem worth solving, and it's the one we're building toward.

‍We're deploying this architecture with a small group of design partners now, ahead of general availability. If you want early access, reach out.