Snippets · Sources

Parse key=value log lines

Many Go services and network appliances log key=value pairs. The key_value_parser operator splits them into attributes; quoted values may contain spaces and even the delimiter itself.

OpenTelemetry Collector

OpenTelemetry Collector YAML

extensions:
  file_storage:
    directory: /var/lib/otelcol/storage
    create_directory: true

receivers:
  filelog:
    include: [/var/log/myapp/*.log]
    start_at: end
    storage: file_storage
    operators:
      # key=value pairs separated by spaces; quoted values may contain spaces.
      - type: key_value_parser
        delimiter: "="
        pair_delimiter: " "
        parse_from: body
        parse_to: attributes
        on_error: send

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  batch:
    timeout: 5s

exporters:
  otlphttp:
    endpoint: https://your-backend:4318

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [filelog]
      processors: [memory_limiter, batch]
      exporters: [otlphttp]

LinkMesh

Library › Processors › Key-Value Log Parser

LinkMesh adds these operators to the source's receiver (filelog/input_src_00000501):

Pushed to the collector

operators:
  - parse_from: body
    type: key_value_parser

The same, as an API call (see automation recipes):

API call

curl -X POST "$LINKMESH/pipelines" \
  -H "Authorization: Bearer $LMSAT" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Key-Value Log Parser",
  "processors": [
    {
      "config": {},
      "enabled": true,
      "order": 1,
      "templateSlug": "kv-log-parser"
    }
  ]
}'

The rest of the collector config above (memory_limiter, batch, otlphttp) is not part of this template: in LinkMesh those are separate sources, processors and destinations on the route.

Generated from LinkMesh's template catalogue by linkmesh-server snippet-render 1.464.3, not written by hand.

Download config.yamlValidated with otelcol-contrib v0.161.0 on 2026-10-26

Complete config files, each checked with the collector's own validate command (otelcol-contrib v0.161.0). Replace https://your-backend:4318 with your own OTLP endpoint before you run it.

Before and after

Example telemetry in, and what this exact config sent out when we ran it in otelcol-contrib v0.161.0. CI re-runs it on every change and fails if the output differs. Changed values are highlighted.

2 log records in, 2 log records out

Before
body
level=info msg="user signed in" user=u-4711 duration_ms=12
After
body
level=info msg="user signed in" user=u-4711 duration_ms=12
attributes.log.file.name
input.log
attributes.level
info
attributes.msg
user signed in
attributes.user
u-4711
attributes.duration_ms
12
Before
body
level=error msg="payment provider timeout" order=1002 note="a=b"
After
body
level=error msg="payment provider timeout" order=1002 note="a=b"
attributes.level
error
attributes.msg
payment provider timeout
attributes.log.file.name
input.log
attributes.order
1002
attributes.note
a=b

The gotcha

Every value comes out as a string. In our test duration_ms=12 arrived as the string "12", where the same field from a JSON log is the number 12 - so a numeric filter or a sum in the backend sees text. Convert the fields you compute on (a transform with Int() or Double()) after parsing.

Manage this across a fleet

LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).

Doing this in LinkMesh: Sources

Install LinkMesh →See all snippets