New Relic bills scale with log ingest volume. The problem is that most log data has low analytical value: health checks, debug output, connection lifecycle events, and other repetitive messages that obscure the signals you actually care about. Grepr addresses this by applying machine learning-based log reduction upstream of New Relic, summarizing high-volume patterns while preserving unique events and anomalies.

This walkthrough covers the integration architecture and configuration steps for routing logs through Grepr using Fluent Bit.

View our New Relic Documentation here.

Architecture Overview

The integration inserts Grepr as a processing layer in your existing log pipeline:

Application → Fluent Bit → Grepr → New Relic

Fluent Bit continues to use the New Relic output plugin. The only configuration change is the endpoint URL, which points to Grepr's ingestion API instead of New Relic's. Grepr authenticates to New Relic using the same license key your collectors already use, then forwards processed logs to your New Relic account.

Prerequisites

• Fluent Bit or Fluentd configured with the New Relic output plugin
• Your New Relic license key (the same key used by your existing collectors)
• A Grepr account with API access

Optional: To enable automatic exception parsing (recommended), you will also need your New Relic account ID and a user key. These credentials are required because this feature parses queries from your existing dashboards and alerts, then excludes matching log patterns from reduction.

Step 1: Configure the New Relic Integration

In the Grepr UI, navigate to Integrations and click Add new under Observability Vendors. Select New Relic and provide:

• A descriptive name for the integration
• Your New Relic license key
• (Optional) Your account ID and user key for exception parsing

The license key authenticates both the inbound connection from your collectors and the outbound connection to New Relic. Use the same key your Fluent Bit instances currently use.

Exception Parsing

When you provide an account ID and user key, Grepr periodically syncs with the New Relic API to pull query definitions from your dashboards and alerts. It parses these queries, extracts the log patterns they reference, and adds them to an exception list. Logs matching these patterns bypass reduction entirely, ensuring your existing monitoring and alerting remain unaffected.

By default, Grepr checks for dashboard activity within the last 28 days. You can adjust this window in the integration settings. To view the current exception list, click the cloud sync icon on the Integrations page.

Step 2: Create a Pipeline

From the Grepr homepage, create a new pipeline. Under Sources, add your New Relic integration. This generates an ingestion endpoint specific to your pipeline:

https://<integration-id>-<org-id>.ingest.grepr.ai/nr/log/v1

Copy this URL. You will use it in the Fluent Bit configuration.

Step 3: Update Fluent Bit Configuration

Modify the OUTPUT section in your fluent-bit.conf to point at the Grepr endpoint:

[OUTPUT]
    Name       newrelic
    Match      *
    licenseKey ${NR_LICENSE_KEY}
    endpoint   https://<integration-id>-<org-id>.ingest.grepr.ai/nr/log/v1

Replace the endpoint placeholder with your actual ingestion URL. The license key remains unchanged.

Parallel Shipping for Validation

During initial rollout, you may want to ship logs to both New Relic directly and through Grepr for comparison. To configure this, add a second OUTPUT block:

[OUTPUT]
    Name       newrelic
    Match      *
    licenseKey ${NR_LICENSE_KEY}

[OUTPUT]
    Name       newrelic
    Match      *
    licenseKey ${NR_LICENSE_KEY}
    endpoint   https://<integration-id>-<org-id>.ingest.grepr.ai/nr/log/v1

This allows side-by-side comparison of filtered versus unfiltered data in New Relic before cutting over completely.

Step 4: Configure the Sink

On the pipeline overview page, add a New Relic sink using your integration. This completes the pipeline: logs flow from Fluent Bit to Grepr for processing, then to New Relic for storage and analysis.

How Log Reduction Works

Grepr's ML analyzes incoming log streams and identifies high-frequency patterns. Rather than dropping these logs, it generates summary records that preserve metadata (counts, time ranges, sample messages) while reducing volume. The original logs are retained in low-cost object storage.

Key behaviors:

Pattern detection is automatic. The ML identifies repetitive content based on message structure and frequency. No manual rule configuration required.

Unique events pass through unmodified. Errors, exceptions, and anomalies that deviate from established patterns are forwarded to New Relic without summarization.

Exception parsing protects monitored patterns. Logs referenced by your dashboards and alerts are excluded from reduction.

Full fidelity is retained. Because Grepr stores all of your raw log data, you can search your logs, including the events that were summarized, using a familiar query syntax.  If you need the raw data for incident investigation, submit a backfill job to restore specific time ranges to New Relic. Backfill jobs may be triggered manually or automatically via a webhook from a New Relic alert notification.

Verifying the Integration

After the ML has had time to baseline your log traffic, typically a few hours, depending on volume, check your New Relic dashboard. You should see summary entries replacing high-volume repetitive logs, with unique events and exceptions flowing through normally.

The Grepr dashboard provides metrics on reduction effectiveness, including volume comparisons and pattern breakdowns.

Backfill and Historical Access

Grepr stores all ingested logs in object storage, including those that were summarized.

When you need historical data in New Relic for deeper analysis or correlation with other telemetry, submit a backfill request. Grepr retrieves the specified logs from storage and forwards them to your New Relic account. Backfill jobs may be triggered manually or automatically via a webhook from a New Relic alert notification.

Multi-Vendor Pipelines

While this guide focuses on New Relic, Grepr pipelines support heterogeneous configurations. You can source logs from one vendor's agents and sink to a different platform, which is useful during observability platform migrations or for organizations running mixed tooling.

Limitations

The New Relic integration supports only HTTP and HTTPS connections. TCP transport is not supported.

Next Steps

To evaluate Grepr with your production log data, start a free trial at grepr.ai.
‍

‍