Add a collector extension
An OpenTelemetry Collector’s extensions: block holds everything that isn’t
a receiver, processor or exporter: authenticators, disk storage for queues,
health checks, forwarders, observers. LinkMesh generates the collector’s
whole config, so you don’t paste an extensions: block anywhere. You
declare each extension instead:
- You give a label (your short handle for it), the extension’s
type (
oauth2client,file_storage,health_check…) and its settings as YAML. You can declare any type your collector contains. - LinkMesh gives it a component name, adds it under
extensions:, and lists it inservice.extensions. - Your components reference it as
${ext:<label>}in a custom body, or a destination picks it in its Authentication extension field. LinkMesh fills in the name it chose.
The reason is that the collector loads its config as one document. An
extension with a name nothing points to, a reference to a name that doesn’t
exist, or an extension that is defined but not listed in service.extensions
doesn’t fail on its own. The collector rejects the entire config, and
every source and destination on it stops. When LinkMesh does the naming and
the wiring, those mistakes can’t happen.
What the collector receives
Section titled “What the collector receives”You declare an extension labelled health of type health_check, with these
settings:
endpoint: 0.0.0.0:13133
path: /health
The collector receives:
extensions:
health_check/ext_4b1e09c2:
endpoint: "0.0.0.0:13133"
path: "/health"
service:
extensions: ["health_check/ext_4b1e09c2"]
The part after ext_ comes from the extension’s ID. It stays the same for
the extension’s whole life. The names LinkMesh gives its own extensions
(for example the storage behind Durable Delivery) never start with ext_,
so yours can’t collide with them.
Declare it
Section titled “Declare it”-
Open the collector (Collectors, then click it) or the collector group (Collector Groups), and go to its Extensions tab. Click Add extension.
-
Label — lowercase letters, digits,
-and_, up to 32 characters, starting with a letter or digit. It must be unique on each collector, counting the collector’s own extensions and its groups’ extensions together. The form shows the token it gives you, such as${ext:idp}. -
Type — the extension type exactly as the collector names it, with no
/namepart. The picker offers the extension types this collector contains, and says where that list came from. See The inventory check. -
Settings (YAML) — what goes under the extension’s entry. Leave out the
extensions:key and thetype/name:line. LinkMesh adds both, and a body that includes either is refused. Some extensions, such aspprofandzpages, run on their defaults and need no settings at all. -
Click Add extension. LinkMesh re-applies the config of every collector the extension applies to.
An extension declared on a group applies to every current member and to every collector that joins later. On a member’s own Extensions tab it is marked from group — edit it on the group.
After you save, the settings are hidden. Reveal shows them, and Replace clears the field so you can enter new settings.
Through the API it is POST /api/v1/collectors/{collectorId}/extensions
(or /api/v1/collector-groups/{groupId}/extensions) with
{"label": "…", "type": "…", "body": "…"}. The response carries the
componentId LinkMesh chose, and a notices list when the type couldn’t
be checked. PUT and DELETE on …/extensions/{extensionId} edit and
remove it.
The inventory check
Section titled “The inventory check”A collector can only load an extension that is compiled into its binary. So LinkMesh checks the type against the collector’s component inventory when you save. It uses the first of these that it has:
| Inventory | Where it comes from | Shown as |
|---|---|---|
| Reported | The collector sent the list of its components when it connected | reported by collector |
| Known list | LinkMesh ships the component list for each otelcol-contrib version it supports, and uses it for a collector that hasn’t reported one | known list for otelcol-contrib 0.153.0 (its type and version) |
| Unknown | The collector hasn’t reported its components, and LinkMesh has no list for its exact version | unknown |
What happens to the save:
- The type is in the inventory — it is saved.
- The type is not in the inventory — the save is refused. The message names the collector, its version, which inventory was used and the missing type.
- The inventory is unknown — it is saved, with a Not checked notice. “Not checked” means just that: nothing confirmed that the binary contains the type, and nothing ruled it out either. If the binary doesn’t have it, the collector rejects its whole config. The picker accepts any type you type in, for this reason.
On a group, the save checks every member. The picker marks a type that some members lack (Missing on: …), and the save is refused while any member lacks it. A collector that joins a group is checked against the group’s extensions too, and the join is refused if it lacks one of them or already declares an extension with the same label.
The check runs again before every config push, against the collector’s current inventory. That covers the cases a save can’t see: a collector whose binary was replaced by a build without the extension, or a group member running a different build from the others. See When a collector goes incompatible.
When a collector goes incompatible
Section titled “When a collector goes incompatible”Before LinkMesh sends a config to a collector, it checks every declared extension in that config against the collector’s inventory, the same way the save does. If one is missing:
-
The config is not sent. A collector rejects its whole config over one component it doesn’t have, so sending it would stop every source and destination on the collector.
-
The last good config keeps running. The collector goes on with the config it last applied, and keeps delivering telemetry with it. Changes you make in the meantime don’t reach it.
This holds as long as that last config doesn’t use the missing extension itself. If a collector’s binary is replaced while the config it runs already contains the extension, the collector can’t load its own config. LinkMesh still marks it incompatible and sends nothing new, but the collector reports an error until it runs a build with the extension again, or you remove the extension.
-
The collector is marked incompatible. Its detail page shows an Incompatible chip next to its name and a red notice on every tab, naming the missing extension: its label, its type, its component name, and whether it was declared on the collector or on its group. The Extensions tab shows the same notice above the extension list, and the Events tab records
collector_incompatible. -
Only that collector is affected. In a group, each member is checked on its own. The members that have the extension get the new config as usual.
An unknown inventory is not a reason to withhold a config: the push goes ahead, as the save does.
Right after a collector connects with a changed binary, LinkMesh waits up to a minute for it to report its new component list, and checks against that rather than the old one.
The mark clears by itself, and the new config is sent, as soon as the check passes. That happens when:
- the collector reports an inventory that contains the extension (after you go back to a build that has it), or
- you remove the extension, or change its type to one the collector has.
The Events tab then records collector_compatible.
Reference it with ${ext:<label>}
Section titled “Reference it with ${ext:<label>}”In the body of a custom source or destination,
write ${ext:<label>} wherever the component expects an extension name:
auth:
authenticator: ${ext:idp}
sending_queue:
storage: ${ext:queue}
LinkMesh replaces each token with the extension’s name when it generates the collector’s config. The rules:
- The token is the whole value.
${ext:idp}works.prefix-${ext:idp}, or a token used as a key, is refused. In a flow list, quote it:watch_observers: ["${ext:k8s}"]. - Extension settings take a token, never a name.
auth.authenticator, anystoragekey andwatch_observersmust hold${ext:…}tokens. Anywhere else in the body, a value shaped like an extension name (<extension type>/<name>, such asoauth2client/idp) is refused too. LinkMesh chooses the names itself, so a name you type would match nothing. - The label must be declared where the body runs. Attaching the source or destination is refused if the label isn’t declared on any of the collectors you attach it to. On a group where only some members have the label, the members without it don’t get that source or destination. Their config is still sent, without it, and the apply reports a warning naming the label.
- An
auth.authenticatortoken must name an authenticator. The extension it resolves to must be able to authenticate that component — see Which extensions can authenticate. Attaching a body whose token resolves to anything else is refused withextension_not_authenticator. - A referenced extension can’t be removed or relabelled. Removing or
relabelling it is refused, and the message lists the sources and
destinations that still reference it — by token, or through their
Authentication extension field. Remove the reference first. Changing
the type of an extension used as an authenticator to a type that can’t
authenticate that component is refused the same way, with
extension_not_authenticator.
The Raw YAML editor has a Reference an extension picker that inserts the token at the cursor. Before you save, the editor flags a label that isn’t declared anywhere, a token inside a longer string, and a malformed label.
An extension’s own settings can’t reference another declared extension. Tokens work in custom source and destination bodies. A destination built from the form (not a custom one) takes an authenticator through its Authentication extension field instead — see Authenticate a destination through an extension.
Authenticate a destination through an extension
Section titled “Authenticate a destination through an extension”A destination you create from the form — OTLP gRPC, OTLP HTTP, Loki, Grafana Cloud, Prometheus RW, Datadog, Splunk HEC or Elasticsearch — has an Authentication extension field. Pick a declared label there, and the destination’s exporter authenticates through that extension:
exporters:
otlphttp/output_dst_1c9e0d4a:
endpoint: "https://otlp.example.com"
auth:
authenticator: "oauth2client/ext_4b1e09c2"
The rules:
- One authenticator per destination. A destination with basic_auth
credentials (or, for Grafana Cloud, an instance ID and API token) can’t
also have an authentication extension: the exporter would use one and
silently ignore the other. The field is unavailable while those
credentials are set, and a save that sets both is refused with
auth_extension_credentials_conflict. Clear one of them. - The label must be declared where the destination runs — the same rule
as for a token. Attaching the destination is refused with
extension_reference_unresolvedif no collector you attach it to has the label. On a group, members without it don’t get the destination, and the apply reports a warning. - The extension must be an authenticator for this destination. The field
offers only declared extensions whose type can authenticate the
destination’s exporter — see
Which extensions can authenticate.
Attaching the destination, or pointing an attached one at such a label, is
refused with
extension_not_authenticator, naming the label and its type. - Kafka, File and Debug destinations don’t have the field. Their exporters have no authenticator setting.
- Custom destinations don’t have it either. Their body names the
authenticator itself:
auth: {authenticator: ${ext:<label>}}.
Through the API, the field is auth_extension in the destination’s
defaultConfig (or in an activation’s configOverrides), holding the label.
GET /api/v1/extension-labels?authenticatorFor=<destination type> lists the
labels the field may name for that destination type.
Which extensions can authenticate
Section titled “Which extensions can authenticate”An exporter’s or receiver’s auth.authenticator accepts only an extension of
a type that can authenticate that component. Anything else — a
health_check, a file_storage, an http_forwarder — passes the
collector’s config check and then stops the whole collector when it starts,
so LinkMesh refuses it. There is no override: a type not listed here is
refused, even if your collector build contains it. Use a custom destination
without an authenticator, and set the credentials in its body, for anything
else.
| Where | Permitted extension types |
|---|---|
OTLP gRPC destination (and custom otlp, otelarrow, stef, loadbalancing, coralogix bodies) |
asapclient, basicauth, bearertokenauth, googleclientauth, headers_setter, oauth2client |
| Every other destination with an authenticator (OTLP HTTP, Loki, Grafana Cloud, Prometheus RW, Datadog, Splunk HEC, Elasticsearch, and custom bodies of other exporters) | the six above, plus azure_auth, sigv4auth, sumologic |
A custom source body (a receiver’s auth.authenticator) |
azure_auth, basicauth, bearertokenauth, oidc |
azure_auth, sigv4auth and sumologic authenticate HTTP requests only,
so a gRPC exporter can’t use them. oidc validates incoming requests, so it
works only on a receiver. For basicauth on a destination, set
client_auth in its settings; on a source, htpasswd.
A destination saved with an authenticator that isn’t permitted is not sent to any collector: the collector gets the rest of its config without that destination, and the apply reports a warning naming the destination, the extension and its type. Point the destination at a permitted extension, or clear the field, and apply again.
Examples
Section titled “Examples”All four run on otelcol-contrib as LinkMesh ships it. Replace the hosts, ports and paths with your own.
OAuth2 client credentials on an OTLP/HTTP destination
Section titled “OAuth2 client credentials on an OTLP/HTTP destination”Declare the authenticator on the collector:
-
Label
idp, Typeoauth2client -
Settings:
client_id: linkmesh client_secret: ${env:IDP_CLIENT_SECRET} token_url: https://idp.example.com/oauth2/token scopes: ["api.write"]
Then create the destination. From the form:
- Destinations, then New Destination. Keep Guided mode and pick
Exporter Type
OTLP HTTP. - Set Endpoint to
https://otlp.example.com. Leave basic_auth empty. - In Authentication Extension, pick
idp, and create the destination.
Or, as a custom destination, with exporter type otlphttp and this
body:
endpoint: https://otlp.example.com
auth:
authenticator: ${ext:idp}
Either way, the exporter fetches a token from token_url and sends it as a bearer
token. A wrong secret fails this exporter only. The collector’s other
destinations keep delivering, and the collector’s log shows the
authentication error against this one exporter. See
Keep secrets out of LinkMesh for setting
IDP_CLIENT_SECRET on the host.
A custom destination with a disk-backed queue
Section titled “A custom destination with a disk-backed queue”Declare the storage:
-
Label
queue, Typefile_storage -
Settings:
directory: /var/lib/otelcol/queue create_directory: true
and point the exporter’s queue at it:
endpoint: https://otlp.example.com
sending_queue:
storage: ${ext:queue}
Anything waiting in the queue is kept on disk and delivered after the collector restarts. The directory must be writable by the user the collector runs as.
The simpler way to get the same result is the Durable Delivery setting on the destination itself, with no extension to declare. See Resume after restart. Use one or the other on a destination, not both.
http_forwarder on its own
Section titled “http_forwarder on its own”An extension doesn’t have to be referenced by anything. http_forwarder
accepts HTTP on one address and forwards it to another:
-
Label
forward, Typehttp_forwarder -
Settings:
ingress: endpoint: 0.0.0.0:6060 egress: endpoint: http://upstream.example.com:4318
LinkMesh doesn’t check the listen port against the ports your sources use. Pick one that nothing else on the collector binds. If the port is taken, the collector fails to start.
health_check
Section titled “health_check”-
Label
health, Typehealth_check -
Settings:
endpoint: 0.0.0.0:13133 path: /health
GET http://<collector-host>:13133/health then returns 200 while the
collector is running. Point a load balancer or a Kubernetes probe at it.
Which collectors take one
Section titled “Which collectors take one”| Runtime | Declared extensions |
|---|---|
| otelcol-contrib (OpAMP) | Any extension type in the collector’s inventory, as described above |
| Grafana Alloy | None. Every type is refused |
Grafana Alloy has no generic extension block. Its authenticators and storage are components with their own settings, written in River rather than YAML, and an OpenTelemetry extension body doesn’t translate into River. So a declaration on an Alloy collector is refused, not converted or dropped. The message ends:
declared extensions render on the otelcol runtime only — Grafana Alloy has
no generic extension block, and an OpenTelemetry extension body does not
translate into River. Declare it on an otelcol collector instead
On an Alloy collector the Extensions tab says so and offers no Add extension button. A group with any Alloy member can’t take a declared extension, and an Alloy collector can’t join a group that declares one. Likewise, a destination with an Authentication extension can’t be attached to an Alloy collector, and an Alloy collector can’t join a group that has such a destination.
Keep secrets out of LinkMesh
Section titled “Keep secrets out of LinkMesh”Write credentials as ${env:NAME}. The collector reads the variable from
its own environment when it starts, so the value never leaves the host.
LinkMesh stores and shows ${env:NAME} exactly as you wrote it.
A credential written in clear is not stored in clear. When you save, every
setting whose key looks like a credential is moved into LinkMesh’s
secrets vault, and the stored settings hold
a ${secret:…} reference instead. Keys containing secret, password or
token count, among others. An endpoint address such as token_url is not
a credential and is stored as you wrote it, unless the URL itself carries one
(a user and password, or a query string); then it is moved into the vault too.
If the vault is disabled, the save is refused rather than storing the value
in clear.
Where the collector’s environment is set
Section titled “Where the collector’s environment is set”The supervisor starts the collector, and the collector inherits the supervisor’s environment. So set the variable on the supervisor.
Linux (install script or manual install). The supervisor runs as the
systemd unit opamp-supervisor. Add a drop-in rather than editing the unit
file, because re-running the installer rewrites the unit:
sudo systemctl edit opamp-supervisor
[Service]
Environment="IDP_CLIENT_SECRET=…"
Or, to keep the value out of the unit, use EnvironmentFile=/etc/otelcol-supervisor/secrets.env
and make that file readable by root only. Then restart:
sudo systemctl restart opamp-supervisor
Kubernetes (DaemonSet). Add the variable to the supervisor
container’s env:, taken from a Kubernetes Secret:
env:
- name: IDP_CLIENT_SECRET
valueFrom:
secretKeyRef: { name: idp-credentials, key: client-secret }
The pods pick it up when they restart. See Install on Kubernetes for the full manifest.
A variable that isn’t set resolves to an empty string. For an authenticator that usually means the exporter fails to authenticate, not that the collector fails to start. Set it before you reference it.
Related
Section titled “Related”- Create a custom source or destination — the bodies that reference extensions
- Create a custom processor — the same escape hatch, for processors
- Resume after restart — Durable Delivery, the one-switch disk queue
- Collector groups — how group settings reach members
- Config not applying after save — when a change doesn’t reach the collector