Snippets · Processors
memory_limiter and batch, in the right order
A collector that runs out of memory under a traffic spike, and one that floods its backend with tiny requests. memory_limiter and batch fix one each.
- Signals
- Logs
- Category
- Reliability
- Source
- OTLP
- Destination
- OTLP/HTTP endpoint
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
processors:
memory_limiter:
check_interval: 1s
limit_percentage: 80
spike_limit_percentage: 25
batch:
timeout: 5s
send_batch_size: 8192
send_batch_max_size: 10000
exporters:
otlphttp:
endpoint: https://your-backend:4318
service:
pipelines:
logs:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [otlphttp]
// stability: generally-available — runs on a default Alloy install.
otelcol.receiver.otlp "default" {
grpc {
endpoint = "0.0.0.0:4317"
}
output {
logs = [otelcol.processor.memory_limiter.default.input]
}
}
otelcol.processor.memory_limiter "default" {
check_interval = "1s"
limit_percentage = 80
spike_limit_percentage = 25
output {
logs = [otelcol.processor.batch.default.input]
}
}
otelcol.processor.batch "default" {
timeout = "5s"
send_batch_size = 8192
send_batch_max_size = 10000
output {
logs = [otelcol.exporter.otlphttp.default.input]
}
}
otelcol.exporter.otlphttp "default" {
client {
endpoint = "https://your-backend:4318"
}
}
Complete config files, each checked with the tool's own validate command (otelcol-contrib v0.161.0, Grafana Alloy v1.19.2). Replace https://your-backend:4318 with your own OTLP endpoint before you run it.
The gotcha
memory_limiter goes first and batch goes last. Put batch in front and the data is already held in memory by the time the limiter checks, so the limiter cannot protect anything.
Manage this across a fleet
LinkMesh manages collector configuration across a fleet from one self-hosted control plane, so a change like this is made once rather than host by host. The first 25 Collectors are free after a no-card registration in the OpenSight Customer Portal (5 without one).
Related snippets
- Tail a log fileShip an application log file to your backend, and keep your place in it across collector restarts.
- Receive OTLP over gRPC and HTTPThe front door: accept OTLP from SDKs and other collectors, on both transports, for all three signals.
- Receive syslog (RFC 5424 and RFC 3164)Accept modern and legacy syslog on one collector, from network gear and hosts that speak nothing else.
- Prove data is flowing with the debug exporterThe first thing to reach for when nothing arrives: print what the collector actually receives.