Snippets · Pipelines

Route logs to different backends by attribute (routing connector)

Different teams, different backends, one collector. The routing connector reads an attribute on every batch of logs - here the resource attribute team - and hands it to the pipeline whose condition matches; default_pipelines takes everything no route claims. Each target pipeline has its own exporter, so the two backends never see each other's data. The table below shows which pipeline each record reached when CI ran this config.

Included steps

Receive OTLP
Route by resource attribute (routing connector)
Batch per backend
Export each route to its own backend

OpenTelemetry Collector

OpenTelemetry Collector YAML

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  batch:
    timeout: 1s

connectors:
  routing:
    # Logs no route matches go here. Without it they are dropped, silently.
    default_pipelines: [logs/default]
    error_mode: ignore
    table:
      - context: resource
        condition: attributes["team"] == "payments"
        pipelines: [logs/payments]

exporters:
  otlphttp/payments:
    endpoint: https://payments-logs.example.internal:4318
    sending_queue:
      enabled: true
  otlphttp/default:
    endpoint: https://central-logs.example.internal:4318
    sending_queue:
      enabled: true

service:
  pipelines:
    logs/in:
      receivers: [otlp]
      processors: [memory_limiter]
      exporters: [routing]
    logs/payments:
      receivers: [routing]
      processors: [batch]
      exporters: [otlphttp/payments]
    logs/default:
      receivers: [routing]
      processors: [batch]
      exporters: [otlphttp/default]

LinkMesh

In LinkMesh this is routing, not a component: two routes on the same collector. The first route's input filter is resource.attributes["team"] == "payments" and its destination is the payments backend; the second route has an empty filter - a catch-all - and sends to the central backend. Routes are evaluated in priority order and, by default, the first match wins, so the catch-all must have the lower priority.

Filter records on a route

Download config.yamlValidated with otelcol-contrib v0.161.0 on 2026-10-26

Complete config files, each checked with the collector's own validate command (otelcol-contrib v0.161.0).

Before and after

Example telemetry in, and what this exact config sent out when we ran it in otelcol-contrib v0.161.0. CI re-runs it on every change and fails if the output differs. Changed values are highlighted.

3 log records in, 3 log records out

Before
body
card authorisation declined
severity
ERROR
resource.service.name
checkout
resource.team
payments
After
pipeline
logs/payments
body
card authorisation declined
severity
ERROR
resource.service.name
checkout
resource.team
payments
Before
body
product page rendered
severity
INFO
resource.service.name
catalog
resource.team
shop
After
pipeline
logs/default
body
product page rendered
severity
INFO
resource.service.name
catalog
resource.team
shop
Before
body
export took 42s
severity
WARN
resource.service.name
batch-export
After
pipeline
logs/default
body
export took 42s
severity
WARN
resource.service.name
batch-export

The gotcha

default_pipelines is what keeps unmatched data. In our test, with the default_pipelines line removed and nothing else changed, the two of three records without team: payments were dropped - no error, no warning in the collector log, and the logs/default pipeline still defined but empty.

Manage this across a fleet

LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).

Install LinkMesh →See all snippets