Snippets · Pipelines

Prometheus counters to a delta-only backend (cumulativetodelta)

Prometheus counters are cumulative; Dynatrace's OTLP metrics ingest accepts delta only. The prometheus receiver scrapes the application, cumulativetodelta converts the counters whose name ends in _total (the receiver keeps the suffix: in our scrape the counter arrived as orders_placed_total) and leaves gauges as they are, and the result goes to Dynatrace over OTLP.

Included steps

OpenTelemetry Collector

OpenTelemetry Collector YAML

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: checkout
          scrape_interval: 30s
          static_configs:
            - targets: [checkout.example.internal:9464]

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  # Prometheus counters are cumulative; Dynatrace's OTLP metrics ingest
  # accepts delta only.
  cumulativetodelta:
    include:
      match_type: regexp
      metrics: [".*_total$"]
  batch:
    timeout: 5s

exporters:
  otlphttp/dynatrace:
    endpoint: https://abc12345.live.dynatrace.com/api/v2/otlp
    headers:
      Authorization: Api-Token ${env:DT_API_TOKEN}
    sending_queue:
      enabled: true

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: [memory_limiter, cumulativetodelta, batch]
      exporters: [otlphttp/dynatrace]

LinkMesh

LinkMesh has no built-in template for this job. Add it as a custom processor in raw mode, with cumulativetodelta as the component type and this body - the same lines as under cumulativetodelta in the collector config:

Custom processor body (cumulativetodelta)

include:
  match_type: regexp
  metrics: [".*_total$"]

How custom components are validated and rolled out: create a custom processor.

Download config.yamlValidated with otelcol-contrib v0.161.0 on 2026-10-26

Complete config files, each checked with the collector's own validate command (otelcol-contrib v0.161.0).

Before and after

Example telemetry in, and what this exact config sent out when we ran it in otelcol-contrib v0.161.0. CI re-runs it on every change and fails if the output differs. Changed values are highlighted.

3 metrics in, 2 metrics out (3 data points in, 2 out)

Before
metric
orders_placed_total
point 1
5
resource.service.name
checkout
After
Dropped
Before
metric
orders_placed_total
point 1
8
resource.service.name
checkout
After
Dropped
Before
metric
orders_placed_total
point 1
15
resource.service.name
checkout
After
Dropped
Before
(not in the input)
After
metric
orders_placed_total
point 1
3
resource.service.name
checkout
Before
(not in the input)
After
metric
orders_placed_total
point 1
7
resource.service.name
checkout

The gotcha

The first reading of a series is only a baseline when the series started before the collector did - so every collector restart costs one reading per series, while an application restart does not. In our test the counter readings 5, 8 and 15 of a series older than the collector came out as 3 and 7; after the application restarted (a new start time, later than the collector's), its first reading 2 came out as 2.

Manage this across a fleet

LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).

Install LinkMesh →See all snippets