LinkMesh

Search docs, blog and changelog

ENDE

Snippets · Sources

Parse IIS W3C access logs into semantic conventions

IIS writes one W3C line per request, space-separated, with "-" for empty fields, "+" for spaces in the user agent and a UTC timestamp. This config tails the site log files, drops the # header lines, parses the default IIS 10 fields, takes the timestamp from the line and maps the fields to OpenTelemetry semantic conventions with integer status codes and sizes. The before/after below is the collector's output for a real IIS 10 log, written on a freshly provisioned GitLab SaaS Windows Server runner.

Signals
Logs
Category
CollectionParsing
Source
IIS W3C log files
Components
filelogbatchmemory_limitertransformotlphttp
Destination
OTLP/HTTP endpoint

OpenTelemetry Collector

OpenTelemetry Collector YAML
extensions:
  file_storage:
    directory: C:\ProgramData\otelcol\storage
    create_directory: true

receivers:
  filelog:
    include: ['C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log']
    start_at: end
    storage: file_storage
    operators:
      # W3C header lines (#Software, #Version, #Date, #Fields) describe the
      # file; they are not requests.
      - type: filter
        expr: 'body matches "^#"'
      # The default W3C fields of IIS 10, as its own #Fields line lists them:
      # date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username
      # c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus
      # sc-win32-status sc-bytes cs-bytes time-taken. "-" means empty.
      - type: regex_parser
        regex: '^(?P<date>\S+) (?P<time>\S+) (?P<server_address>\S+) (?P<method>\S+) (?P<path>\S+) (?P<query>\S+) (?P<server_port>\d+) (?P<user_name>\S+) (?P<client_address>\S+) (?P<user_agent>\S+) (?P<referer>\S+) (?P<status>\d{3}) (?P<substatus>\d+) (?P<win32_status>\d+) (?P<response_size>\d+) (?P<request_size>\d+) (?P<time_taken>\d+)$'
        on_error: send
      # IIS writes these timestamps in UTC.
      - type: add
        field: attributes.timestamp
        value: EXPR(attributes.date + " " + attributes.time)
        if: 'attributes.date != nil'
      - type: time_parser
        parse_from: attributes.timestamp
        layout: '%Y-%m-%d %H:%M:%S'
        location: UTC
        if: 'attributes.timestamp != nil'

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  transform/semconv:
    error_mode: ignore
    log_statements:
      - context: log
        conditions:
          - attributes["method"] != nil
        statements:
          - set(attributes["http.request.method"], attributes["method"])
          - set(attributes["url.path"], attributes["path"])
          - set(attributes["url.query"], attributes["query"]) where attributes["query"] != "-"
          - set(attributes["server.address"], attributes["server_address"])
          - set(attributes["server.port"], Int(attributes["server_port"]))
          - set(attributes["client.address"], attributes["client_address"])
          - set(attributes["user.name"], attributes["user_name"]) where attributes["user_name"] != "-"
          # IIS writes spaces in the user agent as "+".
          - set(attributes["user_agent.original"], attributes["user_agent"]) where attributes["user_agent"] != "-"
          - replace_pattern(attributes["user_agent.original"], "\\+", " ") where attributes["user_agent.original"] != nil
          - set(attributes["http.response.status_code"], Int(attributes["status"]))
          - set(attributes["iis.substatus"], Int(attributes["substatus"]))
          - set(attributes["iis.win32_status"], Int(attributes["win32_status"]))
          - set(attributes["http.response.size"], Int(attributes["response_size"]))
          - set(attributes["http.request.size"], Int(attributes["request_size"]))
          - set(attributes["iis.time_taken_ms"], Int(attributes["time_taken"]))
          - delete_matching_keys(attributes, "^(date|time|timestamp|server_address|method|path|query|server_port|user_name|client_address|user_agent|referer|status|substatus|win32_status|response_size|request_size|time_taken)$")
  batch:
    timeout: 5s

exporters:
  otlphttp:
    endpoint: https://your-backend:4318

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [filelog]
      processors: [memory_limiter, transform/semconv, batch]
      exporters: [otlphttp]

LinkMesh

LinkMesh has no built-in template for this job. Add it as a custom source in raw mode, with filelog as the component type and this body - the same lines as under filelog in the collector config:

Custom source body (filelog)
include: ['C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log']
start_at: end
storage: file_storage
operators:
  - type: filter
    expr: 'body matches "^#"'
  - type: regex_parser
    regex: '^(?P<date>\S+) (?P<time>\S+) (?P<server_address>\S+) (?P<method>\S+) (?P<path>\S+) (?P<query>\S+) (?P<server_port>\d+) (?P<user_name>\S+) (?P<client_address>\S+) (?P<user_agent>\S+) (?P<referer>\S+) (?P<status>\d{3}) (?P<substatus>\d+) (?P<win32_status>\d+) (?P<response_size>\d+) (?P<request_size>\d+) (?P<time_taken>\d+)$'
    on_error: send
  - type: add
    field: attributes.timestamp
    value: EXPR(attributes.date + " " + attributes.time)
    if: 'attributes.date != nil'
  - type: time_parser
    parse_from: attributes.timestamp
    layout: '%Y-%m-%d %H:%M:%S'
    location: UTC
    if: 'attributes.timestamp != nil'

How custom components are validated and rolled out: create a custom source.

Download config.yamlValidated on Windows with otelcol-contrib v0.161.0 on 2026-10-26

Complete config files, each checked with the collector's own validate command (otelcol-contrib v0.161.0). Replace https://your-backend:4318 with your own OTLP endpoint before you run it.

Before and after

Example telemetry in, and what this exact config sent out when we ran it in otelcol-contrib v0.161.0. CI re-runs it on every change and fails if the output differs. Changed values are highlighted.

7 log records in, 3 log records out

  1. Before

    body
    #Software: Microsoft Internet Information Services 10.0

    After

    Dropped

  2. Before

    body
    #Version: 1.0

    After

    Dropped

  3. Before

    body
    #Date: 2026-10-23 09:05:29

    After

    Dropped

  4. Before

    body
    #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

    After

    Dropped

  5. Before

    body
    2026-10-23 09:05:29 ::1 GET / - 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 200 0 0 928 154 360

    After

    Dropped

  6. Before

    body
    2026-10-23 09:05:29 ::1 GET /missing-page id=42 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 404 0 2 5027 148 37

    After

    Dropped

  7. Before

    body
    2026-10-23 09:05:29 ::1 GET /iisstart.png - 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 200 0 0 99937 142 140

    After

    Dropped

  8. Before

    (not in the input)

    After

    body
    2026-10-23 09:05:29 ::1 GET / - 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 200 0 0 928 154 360
    timestamp
    2026-10-23T09:05:29.000Z
    attributes.log.file.name
    input.log
    attributes.http.request.method
    GET
    attributes.url.path
    /
    attributes.server.address
    ::1
    attributes.server.port
    80
    attributes.client.address
    ::1
    attributes.user_agent.original
    Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.20348.5499
    attributes.http.response.status_code
    200
    attributes.iis.substatus
    0
    attributes.iis.win32_status
    0
    attributes.http.response.size
    928
    attributes.http.request.size
    154
    attributes.iis.time_taken_ms
    360
  9. Before

    (not in the input)

    After

    body
    2026-10-23 09:05:29 ::1 GET /missing-page id=42 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 404 0 2 5027 148 37
    timestamp
    2026-10-23T09:05:29.000Z
    attributes.log.file.name
    input.log
    attributes.http.request.method
    GET
    attributes.url.path
    /missing-page
    attributes.url.query
    id=42
    attributes.server.address
    ::1
    attributes.server.port
    80
    attributes.client.address
    ::1
    attributes.user_agent.original
    Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.20348.5499
    attributes.http.response.status_code
    404
    attributes.iis.substatus
    0
    attributes.iis.win32_status
    2
    attributes.http.response.size
    5027
    attributes.http.request.size
    148
    attributes.iis.time_taken_ms
    37
  10. Before

    (not in the input)

    After

    body
    2026-10-23 09:05:29 ::1 GET /iisstart.png - 80 - ::1 Mozilla/5.0+(Windows+NT;+Windows+NT+10.0;+en-US)+WindowsPowerShell/5.1.20348.5499 - 200 0 0 99937 142 140
    timestamp
    2026-10-23T09:05:29.000Z
    attributes.log.file.name
    input.log
    attributes.http.request.method
    GET
    attributes.url.path
    /iisstart.png
    attributes.server.address
    ::1
    attributes.server.port
    80
    attributes.client.address
    ::1
    attributes.user_agent.original
    Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.20348.5499
    attributes.http.response.status_code
    200
    attributes.iis.substatus
    0
    attributes.iis.win32_status
    0
    attributes.http.response.size
    99937
    attributes.http.request.size
    142
    attributes.iis.time_taken_ms
    140

The gotcha

Match the fields to your #Fields line, not to a remembered default. IIS 10 logs sc-bytes and cs-bytes before time-taken; in our test a regex for the shorter, older field list parsed none of the three request lines - no error, every line passed through unparsed, because on_error: send keeps them. The fields are configurable per site, so read the #Fields line of your own log first.

Manage this across a fleet

LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).