One Query, Four Engines: How Grepr Translates Log Queries

A query like -service:web can get translated in Grepr four times: into Java compiled into a streaming job, into Flink SQL for backfills, and into Trino and Athena SQL for the data lake. All four have to agree on exactly which logs match, including the ones with no service tag at all.

This post covers the layer that makes them agree, sql-babel, and the intermediate language at its center, Grepr SQL. It also covers some strange queries we've had to support, and a feature on the way that keeps lake queries fast.

The problem

Grepr sits between your agents and your observability vendor. Logs flow through a Flink streaming job that reduces them, and everything lands in an Iceberg data lake on S3.

The queries come from elsewhere. Teams arrive with thousands of Datadog and New Relic queries behind their dashboards and alerts, and nobody wants to rewrite them. So we accept Datadog search, New Relic search, NRQL and our own Grepr SQL, and run each one wherever it's needed:

If an imported alert query matches a log in the stream but not in the lake, the pipeline and search show different numbers, and users stop trusting both.

Grepr SQL: one language in the middle

With four inputs and four outputs, writing every translator directly means sixteen translators, and sixteen slightly different ideas of what NOT means. Instead, every input is parsed into one intermediate language, and every output is written from it.

Grepr SQL is Apache Calcite's SQL over a logical logs table: a message, a severity, timestamps, a tags map and a parsed attributes document. A few Grepr functions cover ideas the input languages need and standard SQL lacks. Calcite validates and optimizes the plan, and one writer per target spells it out.

The rule that keeps this sane: the parser owns meaning, the writer owns spelling. NRQL's LIKE, for example, ignores case, so the NRQL parser emits ILIKE, and no writer ever needs to know the query came from New Relic. Grepr SQL is also user-facing: Log Explorer's Analyze mode takes Grepr SQL expressions directly.

One query, three spellings

Here is -service:web ("logs not from the web service") as it leaves our writers, copied from our test snapshots and trimmed to the filter.

-- Grepr SQL
ARRAY_CONTAINS_IGNORE_CASE(`tags`['service'], 'web') IS NOT TRUE

// Compiled Java, run once per event in the stream
final Collection cse_0 = (Collection) event.getTags().get("service");
return !(cse_0 == null ? false : stringCollectionContainsIgnoreCase(cse_0, "web"));

-- Trino / Athena
(COALESCE("t_service" = 'Vweb', FALSE)
  OR ("t_service" IN ('M', 'U') OR "t_service" IS NULL)
      AND ANY_MATCH(ELEMENT_AT("tags", 'service'), "value" -> LOWER("value") = LOWER('web')))
IS NULL
OR NOT
(COALESCE("t_service" = 'Vweb', FALSE)
  OR ("t_service" IN ('M', 'U') OR "t_service" IS NULL)
      AND ANY_MATCH(ELEMENT_AT("tags", 'service'), "value" -> LOWER("value") = LOWER('web')))

Three details matter:

• Case. Datadog lowercases tags at ingest, so service:web finds service:Web. We keep tags as received, so we ignore case at query time instead.
• Negation is IS NOT TRUE, not NOT. A log with no service tag makes the check NULL, and NOT NULL is still NULL, so SQL would drop the row. Datadog keeps it, so we do too. Trino writes IS NOT TRUE out longhand as IS NULL OR NOT, which is why its expression appears twice.
• t_service is an index column. Iceberg can't see inside a map, so a filter on tags['service'] alone can't skip files. Common tags also get their own lowercased column, which Iceberg's per-file statistics can use. When that column holds a single value it answers the query on its own. Otherwise the query falls back to the map, which stays the source of truth.

War stories: match the vendor, not the spec

When a customer connects Grepr, we import every query behind their pre-existing vendor dashboards and monitors, often thousands of them. Some of those queries are ones nobody has looked at in a year. Others drive alerts that wake people up at night. Users can add any of them into a Grepr pipeline as an “exception” to avoid touching them, and we translate them so that each one returns exactly what it returns in the vendor language, even when the query itself looks wrong.

Queries that shouldn't parse, but do. One day in July, 294 parser errors in our logs traced back to nine queries from live customer dashboards and monitors. A textbook Lucene grammar rejected every one. Datadog ran all but one:

We changed our grammar to match. The odd one out had an unbalanced parenthesis. Datadog rejects it too (its widget was already broken there), so we left it failing.

Quirks count too. Datadog lowercases tags when it stores them, but it doesn't lowercase your search. So env:prod finds logs sent as env:PROD, but env:PROD finds nothing, because no stored tag has an uppercase letter. Grepr stores tags as they were sent, so we could make the uppercase search work for languages that care about it. For Datadog queries, we deliberately don't: a dashboard that showed zero in Datadog should show zero in Grepr, not a new number nobody can explain.

NULL drops rows silently. Our first -error compiled to NOT REGEXP_LIKE(message, ...). On a log with no message that expression is NULL, so the row vanished. Nothing failed; the counts were just slightly low. That's where the IS NOT TRUE pattern came from.

Correct can still be expensive. A dotted key like @http.status_code can be stored nested, as a literal "http.status_code" key, or both. Our first lake query covered every layout with a recursive CTE. Athena unrolled it into twelve table scans, so it billed 12× the bytes. A per-row walk with Trino's REDUCE gives the same answers for the cost of one scan.

Keeping the engines honest

In every story above, the generated SQL looked right and an engine returned the wrong rows. So our tests check the rows that come back, on real engines:

• Write each case once. A Datadog test lives in the Datadog suite: a query, a few log rows, and which rows should match. Every engine's test class inherits every suite.
• Run it for real. Each case runs as a compiled Java class, on a Flink MiniCluster, on Trino in a container, and nightly on Athena.
• Gaps must announce when they close. An engine that can't handle a case yet lists it as an exclusion. Excluded cases still run, and the build fails if one starts passing, so the lists only get shorter.

Coming soon: Attribute and Tags Autopromotion (aka Shredding)

Columns like t_service cover tags, but most interesting filters are on attributes: @http.status_code:500, @duration:>1000. Attributes are stored as one JSON column today. Iceberg keeps no statistics inside it, so an attribute filter can't skip a single file or Parquet chunk and has to parse the JSON of every remaining row.

Autopromotion fixes this without anyone picking columns:

• The writer promotes paths as it sees them. As the streaming job writes logs to the lake, it gives each attribute path it encounters its own column, and the table evolves in place with no pipeline restart.
• Numbers get a numeric column too, so range filters like @duration:>1000 can skip files, not just equality filters.
• A per-dataset cap bounds it. Include and exclude lists override the automatic choice.
• Each column handles flattened paths with an encoding from dotted path to common column format accepted across potential storage layers.

Queries don't change. Users still write @http.status_code:500. sql-babel adds a condition on the promoted column so Iceberg can skip files, and reads each row's value from that column instead of the JSON. Files written before a path was promoted have NULL in that column, so the query falls back to the JSON for them. Results stay the same, and queries get faster as the table learns the shape of the data.

What it buys you

Moving to Grepr shouldn't mean rewriting your observability. Grepr reads the queries behind your Datadog or New Relic dashboards and alerts, and uses each one twice. In the pipeline, it becomes a compiled predicate, i.e. an “exception”, to pass through data that powers existing alerts unreduced. In the data lake, the same query runs on Trino or Athena and returns the logs it would have returned via a vendor query.

Both rest on one promise: the stream, the lake and the tool you came from all agree. That holds for every query you bring, including the ones that only parse because a vendor's grammar is forgiving.

We plan to add support to many more languages over time, including PromQL and piped query languages, and we will be supporting hot stores for the storage layer, like Clickhouse, in the future.

‍

FAQ

Why does Grepr translate a single query into four different engines?

‍A query has to run wherever logs live. Grepr compiles it to a Java predicate to decide per event in the streaming job, to Flink SQL for backfills that replay stored logs, and to Trino and Athena SQL for search and analysis over the data lake. If the same query returned different results in the stream and the lake, the numbers would disagree and users would stop trusting both.

What are Grepr SQL and sql-babel?

‍sql-babel is the translation layer. Rather than writing a translator for every input-output pair, Grepr parses every input language into one intermediate language, Grepr SQL, and writes each engine's output from that. Grepr SQL is Apache Calcite's SQL over a logical logs table, with a few Grepr functions for ideas the input languages need that standard SQL lacks.

Can I keep my Datadog and New Relic queries when I move to Grepr?

‍Yes. Grepr imports the queries behind your existing dashboards, monitors, and alerts and accepts Datadog search, New Relic search, NRQL, and Grepr SQL. You do not rewrite them. Each one is translated so it returns exactly what it returned in the vendor language.

Why does Grepr match vendor behavior even when a query looks malformed?

‍Because a dashboard that showed a certain number in Datadog should show the same number in Grepr. Vendor grammars are forgiving in specific ways, so some real customer queries only parse because of that. Grepr matches the vendor's actual behavior rather than a textbook grammar, including case handling, so the stream, the lake, and the original tool all agree.

Why does translated SQL use IS NOT TRUE instead of NOT?

‍Because of NULL handling. A log with no matching tag makes the check NULL, and NOT NULL is still NULL, so standard SQL would silently drop the row. Datadog keeps that row, so Grepr does too, which is why negations translate to IS NOT TRUE.

What is attribute and tag autopromotion (shredding)?

‍Attributes are stored as one JSON column today, which Iceberg can't index, so an attribute filter has to parse every remaining row. Autopromotion gives each attribute path its own column as the streaming job writes logs, so filters like @http.status_code:500 or @duration:>1000 can skip files. Your queries don't change, and they get faster as the table learns the shape of the data.

‍