Jad Naous on The a16z Show: Why AI Can Write Code But Can't Keep It Running

Why AI SRE Is Harder Than AI Coding

I sat down with Martin Casado, a general partner at a16z, to talk about whether AI can run the systems it writes code for. My answer was no, not yet, because the complexity and the randomness that exists in large production environments is hard for an AI to reason about on its own without help. BUT… I did say it can reason about the steps to troubleshoot, understand and build context that can help it do that, and look at the telemetry data that’s passing by and use it as part of those steps. This is what Grepr is building, a harness for AI agents to be able to reason about telemetry and to help it operate a production environment.

Why a lot of AI SRE tools don't work

Martin said the anecdotal reports he gets on AI SRE products have not been great. A lot of those systems start from a lagging indicator. A metric spiked or an alert fired, so agents have to go back through a very complex system and try to figure out why. Backtracking into a root cause in a chaotic environment is hard.

At Grepr, we’re tackling it the other way around. Going forward from something that might look like an anomaly or change that could lead to a problem is more tractable. And if we can do it early enough, we can actually prevent a percentage of potential problems rather than taking a downtime hit in the first place. Our current trials show that this indeed works because we start much earlier in the causal chains of incidents.

The state space problem in AI for operations

In our conversation, Martin argued that every bit you add to a program doubles the number of states it can get into. I agree it's a much tougher problem than code. Martin restated my view this way: the universe of things that can break is heavy-tailed, but the set of steps used to debug it is manageable, and you can teach a model those steps, just as humans operate on runbooks too.

The question is can an AI understand what a root cause means? Ultimately, most incidents are caused by some kind of change somewhere. If we can use telemetry to find all changes that are candidates to problems, we don’t have to backtrack from a broken SLO to the change that caused it. And that makes it easier for an AI to find issues and fix them.

Why This Matters and Where We’re Taking It

The stakes keep climbing. For more than 90% of mid-size and large enterprises, a single hour of downtime now costs over $300,000 (ITIC, 2024 Hourly Cost of Downtime Report). And the incidents that cause it are rarely the ones you already alert on. They are the ones nobody knew to look for: a dependency that starts failing in a new way, a field renamed upstream, a failure that never throws an error.  That is the problem we are building Grepr to solve. Instead of waiting for an alert, Grepr watches your live telemetry, investigates every new or changed pattern the moment it appears, and surfaces what matters with the reasoning attached, across reliability, performance, and security. We have already caught issues this way in our own production and in customer environments, problems that no alert would have fired on. AI can write the code. Keeping it running is the harder problem, and this is the problem we are focused on.

‍

‍