Snippets · Sources
Parse Apache access logs into semantic conventions
Apache httpd's combined log format carries client, user, time, request line, status, bytes, referer and user agent in one line. This config parses it on the filelog receiver and maps each field to its OpenTelemetry semantic-convention name, with the status code and body size as integers and the record timestamp taken from the line.
- Signals
- Logs
- Category
- CollectionParsing
- Source
- Apache access log
- Destination
- OTLP/HTTP endpoint
OpenTelemetry Collector
extensions:
file_storage:
directory: /var/lib/otelcol/storage
create_directory: true
receivers:
filelog:
include: [/var/log/apache2/access.log]
start_at: end
storage: file_storage
operators:
# Combined log format: client, user, time, request line, status, bytes,
# referer, user agent. The request line is split into method, path,
# query and protocol version.
- type: regex_parser
regex: '^(?P<client_address>\S+) \S+ (?P<user_name>\S+) \[(?P<time>[^\]]+)\] "(?P<method>[A-Z]+) (?P<path>[^ ?"]*)(?:\?(?P<query>[^ "]*))? HTTP/(?P<protocol_version>[0-9.]+)" (?P<status>\d{3}) (?P<size>\d+|-) "(?P<referer>[^"]*)" "(?P<user_agent>[^"]*)"'
timestamp:
parse_from: attributes.time
layout: '%d/%b/%Y:%H:%M:%S %z'
on_error: send
processors:
memory_limiter:
check_interval: 1s
limit_percentage: 80
spike_limit_percentage: 25
# Maps the parsed fields to OpenTelemetry semantic convention names and
# types (status code and body size as integers).
transform/semconv:
error_mode: ignore
log_statements:
- context: log
# Only lines the regex parsed: an unparsed line keeps its body and
# gets none of these attributes.
conditions:
- attributes["method"] != nil
statements:
- set(attributes["client.address"], attributes["client_address"])
- set(attributes["user.name"], attributes["user_name"]) where attributes["user_name"] != "-"
- set(attributes["http.request.method"], attributes["method"])
- set(attributes["url.path"], attributes["path"])
- set(attributes["url.query"], attributes["query"]) where attributes["query"] != nil and attributes["query"] != ""
- set(attributes["network.protocol.name"], "http")
- set(attributes["network.protocol.version"], attributes["protocol_version"])
- set(attributes["http.response.status_code"], Int(attributes["status"]))
- set(attributes["http.response.body.size"], Int(attributes["size"])) where attributes["size"] != "-"
- set(attributes["http.request.header.referer"], [attributes["referer"]]) where attributes["referer"] != "-"
- set(attributes["user_agent.original"], attributes["user_agent"])
- delete_matching_keys(attributes, "^(client_address|user_name|time|method|path|query|protocol_version|status|size|referer|user_agent)$")
batch:
timeout: 5s
exporters:
otlphttp:
endpoint: https://your-backend:4318
service:
extensions: [file_storage]
pipelines:
logs:
receivers: [filelog]
processors: [memory_limiter, transform/semconv, batch]
exporters: [otlphttp]
LinkMesh
LinkMesh has no built-in template for this job. Add it as a custom source in raw mode, with filelog as the component type and this body - the same lines as under filelog in the collector config:
include: [/var/log/apache2/access.log]
start_at: end
storage: file_storage
operators:
- type: regex_parser
regex: '^(?P<client_address>\S+) \S+ (?P<user_name>\S+) \[(?P<time>[^\]]+)\] "(?P<method>[A-Z]+) (?P<path>[^ ?"]*)(?:\?(?P<query>[^ "]*))? HTTP/(?P<protocol_version>[0-9.]+)" (?P<status>\d{3}) (?P<size>\d+|-) "(?P<referer>[^"]*)" "(?P<user_agent>[^"]*)"'
timestamp:
parse_from: attributes.time
layout: '%d/%b/%Y:%H:%M:%S %z'
on_error: send
How custom components are validated and rolled out: create a custom source.
Download config.yamlValidated with otelcol-contrib v0.161.0 on 2026-10-26
Complete config files, each checked with the collector's own validate command (otelcol-contrib v0.161.0). Replace https://your-backend:4318 with your own OTLP endpoint before you run it.
Before and after
Example telemetry in, and what this exact config sent out when we ran it in otelcol-contrib v0.161.0. CI re-runs it on every change and fails if the output differs. Changed values are highlighted.
2 log records in, 2 log records out
Before
- body
- 203.0.113.7 - - [22/Oct/2026:10:15:02 +0200] "GET /index.html HTTP/1.1" 200 2326 "https://www.example.ch/" "Mozilla/5.0 (Macintosh) Safari/605.1.15"
After
- body
- 203.0.113.7 - - [22/Oct/2026:10:15:02 +0200] "GET /index.html HTTP/1.1" 200 2326 "https://www.example.ch/" "Mozilla/5.0 (Macintosh) Safari/605.1.15"
- timestamp
- 2026-10-22T08:15:02.000Z
- attributes.log.file.name
- input.log
- attributes.client.address
- 203.0.113.7
- attributes.http.request.method
- GET
- attributes.url.path
- /index.html
- attributes.network.protocol.name
- http
- attributes.network.protocol.version
- 1.1
- attributes.http.response.status_code
- 200
- attributes.http.response.body.size
- 2326
- attributes.http.request.header.referer
- ["https://www.example.ch/"]
- attributes.user_agent.original
- Mozilla/5.0 (Macintosh) Safari/605.1.15
Before
- body
- 198.51.100.23 - - [22/Oct/2026:10:15:05 +0200] "GET /logo.png HTTP/1.1" 304 - "https://www.example.ch/index.html" "Mozilla/5.0 (Macintosh) Safari/605.1.15"
After
- body
- 198.51.100.23 - - [22/Oct/2026:10:15:05 +0200] "GET /logo.png HTTP/1.1" 304 - "https://www.example.ch/index.html" "Mozilla/5.0 (Macintosh) Safari/605.1.15"
- timestamp
- 2026-10-22T08:15:05.000Z
- attributes.log.file.name
- input.log
- attributes.client.address
- 198.51.100.23
- attributes.http.request.method
- GET
- attributes.url.path
- /logo.png
- attributes.network.protocol.name
- http
- attributes.network.protocol.version
- 1.1
- attributes.http.response.status_code
- 304
- attributes.http.request.header.referer
- ["https://www.example.ch/index.html"]
- attributes.user_agent.original
- Mozilla/5.0 (Macintosh) Safari/605.1.15
The gotcha
Apache's %b writes a dash, not 0, when no body was sent - every 304 and most HEAD requests. Int("-") fails: in our test, without the guard on that statement, the collector logged a "failed to execute statement" warning for the 304 line (and would for every one on a busy server), and the size attribute was simply missing. The where clause skips the dash instead.
Manage this across a fleet
LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).
Doing this in LinkMesh: Sources
Related snippets
- Parse nginx access logs into semantic conventionsTurn nginx combined-format access log lines into OpenTelemetry semantic-convention attributes, with typed status codes.
- Parse IIS W3C access logs into semantic conventionsEvery IIS request as a structured log: method, path, query, status, sizes, time taken and user agent, with the request's own UTC timestamp.
- nginx access logs to Elasticsearch, parsed into semantic conventionsTail nginx, parse every request into semantic-convention fields with typed status codes, and index it in Elasticsearch.
- Parse JSON log lines into attributesTurn each JSON log line into attributes you can filter and group by, keeping the original line as the body.