LinkMesh

Search docs, blog and changelog

ENDE

Snippets · Processors

memory_limiter and batch, in the right order

A collector that runs out of memory under a traffic spike, and one that floods its backend with tiny requests. memory_limiter and batch fix one each.

Signals
Logs
Category
Reliability
Source
OTLP
Destination
OTLP/HTTP endpoint
OpenTelemetry Collector YAML
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  batch:
    timeout: 5s
    send_batch_size: 8192
    send_batch_max_size: 10000

exporters:
  otlphttp:
    endpoint: https://your-backend:4318

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp]
Grafana Alloy
// stability: generally-available — runs on a default Alloy install.

otelcol.receiver.otlp "default" {
	grpc {
		endpoint = "0.0.0.0:4317"
	}

	output {
		logs = [otelcol.processor.memory_limiter.default.input]
	}
}

otelcol.processor.memory_limiter "default" {
	check_interval         = "1s"
	limit_percentage       = 80
	spike_limit_percentage = 25

	output {
		logs = [otelcol.processor.batch.default.input]
	}
}

otelcol.processor.batch "default" {
	timeout             = "5s"
	send_batch_size     = 8192
	send_batch_max_size = 10000

	output {
		logs = [otelcol.exporter.otlphttp.default.input]
	}
}

otelcol.exporter.otlphttp "default" {
	client {
		endpoint = "https://your-backend:4318"
	}
}

Complete config files, each checked with the tool's own validate command (otelcol-contrib v0.161.0, Grafana Alloy v1.19.2). Replace https://your-backend:4318 with your own OTLP endpoint before you run it.

The gotcha

memory_limiter goes first and batch goes last. Put batch in front and the data is already held in memory by the time the limiter checks, so the limiter cannot protect anything.

Manage this across a fleet

LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).