LinkMesh is in Early Access. Releases in this phase carry frequent and numerous breaking changes, so read every upgrade note below before you upgrade — a small version bump is not a routine one. This release makes the Prometheus, OTLP, syslog, file and Kubernetes sources properly configurable and secure, and lets data survive a collector restart. Settings that were silently accepted and then discarded are now honoured or refused at the write — hence 21 breaking changes.
Breaking changes
This release contains breaking changes. Read the upgrade notes below before upgrading.
Read the marker, not the version number. Version numbers do not carry a semver major warning: a breaking change can ship in a minor release. Rely on the breaking-change marker, not on the version number.
An unlicensed server now enrols at most 5 collectors (was 25) and refuses API automation — service-account tokens, config-as-code and Git-settings writes return 403 license_automation_required.
Written by us
Before you upgrade
WHO IS AFFECTED. Only installs running with NO licence file at /etc/linkmesh/license.key. A paid licence (tier managed, standards or mixed) is unaffected in every respect: same cap, and API automation continues to work.
WHAT CHANGES, in two independent parts.
- The unlicensed collector cap drops from 25 to 5. Enforcement is forward-only, so an existing unlicensed install already running 6 to 25 collectors KEEPS them and they keep delivering telemetry — LinkMesh is not in the data path. What it cannot do is enrol another one: the next enrolment is refused with 403 license_cap_reached until the fleet is back under the cap or a licence is applied.
- API automation now requires a paid licence. With no licence file, or with a free tier "community" envelope, these return 403 with the machine-readable code license_automation_required:
- creating, updating or deleting a service account, and minting or rotating its tokens;
- ANY mutating request (POST/PUT/PATCH/DELETE) authenticated with a service-account token, on any path;
- config-as-code commit, publish, rollback and discard;
- writes to /settings/git.
Reads are never affected — GET stays open on every path, for every principal including a service-account token, so existing monitoring and inventory scripts keep working. The web UI is likewise unaffected: creating collectors, editing pipelines, sources, destinations and processors, and uploading a licence all continue to work on every tier. No product feature is paywalled. Agent and OpAMP enrolment are unchanged and still take the collector cap, not this gate.
BEFORE YOU UPGRADE.
If you run unlicensed with more than 5 collectors, or you drive this server from Terraform, Ansible, a CI pipeline, a GitOps webhook or any script using an lmsat_ token, register at https://portal.opensight.ch first. Registration issues a signed 25-collector Community licence at no cost; upload it at Settings -> License (or linkmesh-server license apply) and the cap returns to 25. Note that the free Community licence deliberately does NOT restore API automation — that requires Enterprise (https://linkmesh.io/pricing/).
If neither applies to you, no action is needed.
HOW TO CHECK YOUR OWN SERVER, before or after upgrading:
linkmesh-server license info
reports the cap and an "API automation" line, and GET /api/v1/license returns the same as maxCollectors and canAutomate. Settings -> License shows both in the UI.
AIR-GAPPED INSTALLS. Nothing about this reaches the network: linkmesh-server still phones home to nobody, and licence verification stays a local Ed25519 signature check. Registration happens on the portal from any machine; the artifact is a .key file you carry in.
ALSO FIXED, and it matters for the step above: on the container image, /etc/linkmesh was owned by root while the server runs as uid 1000, so PUT /api/v1/license returned HTTP 500 "License verified but apply failed" after the envelope had already verified correctly. Licence upload therefore could not succeed on any container install before 1.362.0. If you previously tried to apply a licence in a container and hit that error, the file was never written — retry the upload on this version. Kubernetes installs that mount the licence as a read-only Secret at that path were never affected.
An OTLP source now listens on exactly the protocols its config names — a gRPC-template source stops opening port 4318, an HTTP-template source stops opening 4317.
Written by us
Before you upgrade
WHO IS AFFECTED. Anyone whose senders reach an OTLP source on a port the source was never configured for. Before this release an OTLP source opened both 4317 (gRPC) and 4318 (HTTP) regardless of what its config said, so a sender could be pointed at the undeclared port and it worked by accident. On the next config apply after upgrading, that traffic stops being received.
BEFORE YOU UPGRADE, in this order.
- List your OTLP sources and note which were created from the "OTLP gRPC" or "OTLP HTTP" template, plus the OTLP placeholder source that a freshly enrolled collector gets — that placeholder now opens gRPC only.
- For each one, check what your senders actually target. If anything sends to 4318 on a gRPC-template source (or 4317 on an HTTP-template source), you are relying on the old behaviour.
- Set BOTH endpoints on any source that must keep accepting both protocols: fill in gRPC Endpoint AND HTTP Endpoint on the source, or on that collector's activation override if only one collector needs both. Do this before upgrading, not after — the change takes effect on the next config push.
- Where you would rather move the senders, repoint them at the declared port instead. Either is fine; doing neither drops that telemetry silently at the sender's end.
AFTERWARDS. Check the source's status view: a source that is listening and receiving reports data; one whose senders have gone elsewhere will show no data rather than an error, because nothing is failing — the packets simply arrive at a closed port.
Create and update requests carrying a field the API does not recognise are now refused with 422 instead of silently dropping it — sources, pipelines, routes, collector groups, user groups, roles, claim mappings, processor templates, alert rules, Git settings, enrolment tokens and pipeline dry-run.
Written by us
Before you upgrade
WHO IS AFFECTED. API clients only. The web UI never sends unknown fields. If you drive LinkMesh from Terraform, Ansible, a script or a CI job, read on.
WHAT CHANGED. An unrecognised field used to be dropped without a word and the request answered 2xx. Any client relying on that was already broken — it believed it had configured something the server never stored. The request is now refused with 422 and the error names the offending field.
BEFORE YOU UPGRADE.
- Replay your automation against a staging server on this version, or diff the payloads your client sends against the API schema at /api/v1/openapi.json. A 422 naming a field is the whole diagnosis.
- Look first at config-shaped keys on sources and destinations. A setting that belongs inside defaultConfig but was sent at the top level is the most common case, and the error says so by name.
- Two specific renames are refused by name rather than ignored: configYaml on PATCH /pipelines/{id}, and kind on PATCH /collector-groups/{id}. Remove them.
- Note one status-code change: unknown-field refusals on destinations move from 400 to 422. A client that branches on 400 needs to accept 422 as well.
WHAT DOES NOT CHANGE. Reads are untouched. Stored records are untouched — nothing is rejected until something writes to it.
A route created through POST /api/v1/routes without an isFinal field is now final (first-match-wins), where it was previously non-final whenever the request also carried an inputFilter.
Written by us
Before you upgrade
Affects API clients that create routes. Before upgrading, find any automation that creates a route with an inputFilter and no explicit isFinal, and relies on the event continuing on to lower-priority routes. Set isFinal: false explicitly on those calls. Routes already stored are not changed by this release — if you want existing non-final routes to stay non-final, nothing to do; if you want them corrected, PATCH them. Conversely, duplicate delivery that was being absorbed downstream stops at the next route you create.
POST /api/v1/routes now creates a route enabled when the request omits the enabled field, so a route starts forwarding as soon as it is created.
Written by us
Before you upgrade
Affects API clients that create routes ahead of time and rely on them staying inert until a separate PATCH enables them. That pattern now forwards telemetry immediately, which can mean vendor ingest cost earlier than you expected. Before upgrading, add enabled: false explicitly to those create calls — it is honoured unchanged. Routes created through the UI are unaffected, and routes already stored are untouched.
GET /api/v1/entity-health now returns an object {status, lastDataAt?} per entity instead of a bare status string, and the vocabulary gains "stopped" and "unknown".
Written by us
Before you upgrade
Affects external consumers of GET /api/v1/entity-health only; the bundled UI is updated. A client that reads each map value as a string will now read an object. Before upgrading, change those readers to take .status, and accept the two new values "stopped" and "unknown" alongside ok, erroring and no_data. The optional lastDataAt field is new and can be ignored.
A Kubernetes collector-group member that stops reporting for longer than fleet.reapAfter (default 24h) is now removed from the group and releases its licence seat.
Written by us
Before you upgrade
Affects Kubernetes collector groups only. VM collectors, VM groups, and collectors that are merely offline are untouched. A reaped node rejoins automatically when it comes back — but it re-takes the licence cap check to do so, so if your fleet is at the cap, a node returning from a long outage can be refused enrolment. Before upgrading: if you run near your collector cap, either leave headroom or raise the licence; if you would rather prune group members by hand as before, set fleet.reapAfter to a negative duration, which disables reaping entirely.
Prometheus scrape limits and metric_relabel_configs that were stored but silently ignored now take effect, so an affected source can start capping scrapes or dropping metrics that were previously forwarded.
Written by us
Before you upgrade
This is the one item in this release that can REDUCE the telemetry you receive without any configuration change on your side. Before upgrading, review every Prometheus source whose stored config carries any of: sample_limit, target_limit, label_limit, label_name_length_limit, label_value_length_limit, body_size_limit, metric_relabel_configs. Those settings had no effect on either runtime until now. If a value was set experimentally, or copied from elsewhere, clear the field (or set a limit to 0) to turn it off before the next config push. Sources authored as a hand-written config: document are unaffected — that path already passed through verbatim.
Saving a Prometheus source whose targets entries are not plain host:port is now refused with 422 invalid_scrape_target instead of accepted.
Written by us
Before you upgrade
A scheme-prefixed target such as "https://prom.corp:9090" was accepted before and then bricked the collector on apply, so this refusal replaces a silent failure. It applies to the source and to a source activation's config overrides. Before upgrading, rewrite any such target as a plain address ("prom.corp:9090") and express the rest in the dedicated fields: select https in the new Scheme field, and put any path in Metrics Path. Stored sources are not rejected until something writes to them, but a collector running one was already failing.
A source config carrying a key LinkMesh cannot render on the Grafana Alloy runtime is now refused with 422 unrenderable_on_alloy instead of being stored and silently dropped — including the hand-authored Prometheus config: document.
Written by us
Before you upgrade
The hand-authored config: document on a Prometheus source used to pass through to the otelcol receiver verbatim and vanish entirely on Alloy. A source carrying one can no longer be saved or activated. Everything that document was the only route to is now a typed field that renders identically on both runtimes: scheme, basic_auth, bearer_token, tls_config, proxy_url, no_proxy, proxy_from_environment, the six cardinality caps, and metric_relabel_configs. Before upgrading, find your Prometheus sources that use a raw config: block and rewrite each setting into the matching field. The 422 names the offending key and the field to use instead, so a save attempt is a usable checklist. Stored configs are not rejected until something writes to them.
The Docker Stats source type is retired — it is gone from the Receiver Type list, the API refuses creating one, and the config importer no longer recognises a docker_stats receiver.
Written by us
Before you upgrade
Sources already using Docker Stats keep collecting, keep their metrics pipeline and stay editable. Only creating new ones stops. No action is required to upgrade. For new container metrics, use a container-aware Host Metrics source with root_path set, or Kubernetes Node Metrics (kubeletstats). If you import collector configs, strip any docker_stats receiver from them first — the importer will no longer recognise it.
A source config carrying a Kubernetes setting the collector cannot load is now refused at the API write with 422 invalid_k8s_source_config instead of stored.
Written by us
Before you upgrade
Because defaultConfig is a free-form map, this can reject a record that saved before. The refused values are: an extra_metadata_labels entry other than container.id or k8s.volume.type; a k8s_api_config.auth_type outside none, serviceAccount, kubeConfig and tls; a distribution other than kubernetes or openshift; an unparseable metadata_collection_interval; an exclude_watch_type entry that is not a Kubernetes watch verb; and namespace on a k8s_cluster source. Before upgrading, check any Kubernetes source you authored through the API. Fix the value the 422 names; for namespace, move it to namespaces as a list.
A TCP Log or UDP Log source config carrying a setting that receiver does not have is now refused at the write instead of being stored and silently ignored.
Written by us
Before you upgrade
Only API-authored configs can hold these keys — neither template ever offered them, and they never did anything. On UDP Log the refused keys are tls, max_log_size and multiline; on TCP Log it is async. Refused on the same terms and previously storable by the same route: an encoding the collector cannot build, both multiline patterns at once, an uncompilable framing pattern, and a non-positive async counter. Before upgrading, remove those keys from any API-authored raw-log source. The next write that carries the config is refused until the key is gone.
A File Tail source config carrying a key LinkMesh cannot render on both runtimes is now refused with 422 invalid_filelog_config instead of accepted.
Written by us
Before you upgrade
Affects API callers only — no key in this set is reachable from the UI, and every one of them was previously written into the OpAMP config and silently dropped from the Alloy one. Refused: operators (the source's own pipeline generates the receiver's operator chain), delete_after_read, attributes, resource, fingerprint_size, force_flush_period, retry_on_failure, the raw storage argument (use the checkpoint toggle, which wires the storage extension for you), and any key that is not a File Tail field at all. The refusal names the key and says what to use instead. Existing stored configs are not rejected until something writes to them.
Three operator-visible changes to journald and http_check sources: a multi-unit journald source now collects ALL units on Alloy (it silently collected only the first), unrenderable journald fields are refused, and http_check's endpoint list moves from endpoints to targets.
Written by us
Before you upgrade
- LOG VOLUME. A journald source listing more than one systemd unit previously collected only the first on the Grafana Alloy runtime and discarded the rest without a word. It now collects all of them, so such a source will receive MORE log volume after this release. Before upgrading, review your multi-unit journald sources and confirm you want every unit, and that your destination can take the extra volume.
- REFUSED JOURNALD FIELDS. A journald config carrying grep, files, namespace, start_at, all, merge, convert_message_bytes, root_path, journalctl_path or retry_on_failure is now refused at the API write instead of being passed through to the OpenTelemetry Collector — Alloy's loki.source.journal has no equivalent for any of them, so the source behaved differently depending on which runtime ran it. Use units, identifiers, matches, dmesg and directory instead.
- HTTP_CHECK RENAME. An http_check source's endpoint list moves from
endpointstotargets, so that a target can carry its own headers, TLS and method. Stored sources are rewritten at startup and the generator still reads the old key, so nothing stops collecting — but an API write carryingendpointsis refused and names the new key. Update your automation. tcp_check is unaffected and keepsendpoints.
A key other than endpoint inside an OTLP source's nested protocols.grpc / protocols.http block is now refused with 422 invalid_otlp_bounds instead of being accepted and dropped.
Written by us
Before you upgrade
Reachable only by API or a hand-authored config — the form has never written that shape. A config using the nested block was stored with a 201 and then dropped by both config generators without a word, so it already did nothing. It is refused rather than left to be rediscovered because it is the shape upstream's own documentation shows. Before upgrading, move any such setting into the top-level grpc / http block this release adds, which is the working spelling and now carries the size limits, keepalive, CORS and custom URL paths.
Destination queue settings (durable_queue, queue_storage_directory, queue_size, block_on_overflow) are now validated instead of stored and ignored, and refused with 422 invalid_durable_queue where they cannot be honoured.
Written by us
Before you upgrade
A client that previously sent any of these keys believed it had configured durability and had not; the write now says so. The refusal fires when the destination type cannot carry a sending queue on both runtimes (Loki, Prometheus Remote Write, File), when a queue directory is relative, when the queue depth is not a positive number, or when a companion setting is given with durable delivery switched off. Explicitly falsy values — durable_queue: false, an empty queue_size — are treated as silence and still accepted, so ordinary edits are unaffected. Before upgrading, check any automation that sets these on a Loki, Prometheus Remote Write or File destination: those combinations will now be refused, and they never worked. Also note that a NEW destination is durable by default from this release.
A Kubernetes Node Metrics (kubeletstats) source activated on a Grafana Alloy collector is no longer rendered — it emitted a component Alloy does not have, which made that collector's entire config unparseable.
Written by us
Before you upgrade
This trades one loss for a larger recovery: the affected collector's other sources and destinations start working again, and its node metrics stop. Before upgrading, list any Kubernetes Node Metrics sources activated on Alloy collectors. Move them to an OpAMP/otelcol collector, which collects them unchanged. Alloy collectors that never ran a Node Metrics source are unaffected.
The Kubernetes (OpAMP/otelcol) install command now also creates a ServiceAccount, ClusterRole and ClusterRoleBinding, so applying it requires permission to create cluster-scoped RBAC.
Written by us
Before you upgrade
Existing deployed collectors are unaffected until they are reinstalled. Before your next Kubernetes install, check that the credentials running kubectl apply can create a ClusterRole and ClusterRoleBinding — an apply that used to succeed will fail without them. This is the cost of a real fix: the collector the old manifest produced could not run a Kubernetes source at all, because it had no permission to read the API server. If your cluster policy forbids cluster-scoped RBAC from the installing identity, have a cluster administrator apply the RBAC half once, then run the install.
Kubernetes Pod Logs records now carry log.file.path and the k8s.namespace.name / k8s.pod.name / k8s.container.name / k8s.pod.uid / k8s.container.restart_count attributes, where before they carried none of them.
Written by us
Before you upgrade
Nothing stops working and no configuration changes, but the delivered telemetry is not what it was. Before upgrading: review any pipeline, dashboard or alert rule written against the attribute-less records these sources used to produce — they will now see a different record shape. If a destination maps attributes to index labels, Loki in particular, it gains those labels, which changes stream cardinality and can affect your ingest cost and existing queries. Adjust queries and any relabelling before the next config push rather than after.
A hostmetrics source config carrying a per-scraper setting LinkMesh does not render on both runtimes is now refused with 422 invalid_hostmetrics_config instead of being accepted.
Written by us
Before you upgrade
Reachable only by API or a hand-authored config — the form never offered these, and every existing source generates byte-identical config. Such a setting reached otelcol and vanished on Alloy, so a config using one already meant two different things depending on the runtime. Refused keys: filesystem include_mount_points and include_fs_types (use the exclude filters instead), load.cpu_average, per-metric metrics.<name>.enabled, initial_delay, mute_process_user_error, mute_process_cgroup_error, exclude_devices. Before upgrading, remove them from any API-authored hostmetrics config. The scraper NAME list is unchanged, and this release also adds root_path so a containerised collector reports the host rather than itself.
Deprecations
The Docker Stats source type is deprecated: existing sources keep collecting and stay editable, but no new one can be created.
Stops working in a future release, not yet fixed.
Docker Stats never had a configuration form and is gone from the Receiver Type list; the API refuses creating a source with it, and the config importer no longer recognises a docker_stats receiver. Existing Docker Stats sources are unaffected by this release — they keep collecting, keep their metrics pipeline and remain editable. Migrate them when convenient: use a container-aware Host Metrics source with root_path set, or a Kubernetes Node Metrics (kubeletstats) source in a cluster. No end-of-life version is fixed yet; this note will be updated when one is.
Security updates
No security updates in this release.
Fixes
- S-4506Editing a Source or a Destination now pushes the change to the collectors running it. An edit previously sat unapplied until something else triggered a config push.
- S-4507 S-4530A new route is created enabled and final, instead of being silently inert or fanning one event through to several routes' destinations.
- S-4423 S-4511A Prometheus source no longer emits an invalid receiver on either runtime, and the Alloy builder now renders the source config it was given or refuses the save, instead of storing keys it silently drops.
- S-4515An OTLP source opens only the protocols its configuration names, rather than always opening both 4317 and 4318.
- S-4605 S-4606Kubernetes sources load on Grafana Alloy again, and a Kubernetes Node Metrics source no longer makes an Alloy collector's entire configuration unparseable.
- S-4601Kubernetes Pod Logs carry Kubernetes metadata again — namespace, pod, container, pod UID, restart count and the log file path.
- S-4600 S-4609 S-4624The shipped Kubernetes manifest starts, includes the RBAC the Kubernetes sources need, and gives the collector a writable temporary directory so a File source can checkpoint.
- S-4513Source status says what it actually knows — running, stopped, erroring or unknown, with the time data was last seen — instead of reporting "No data" for every uncertain case.
- S-4592A collector that crashes reports its real reason through OpAMP health, instead of a generic failure.
- S-4518A syslog source parses the RFC its senders actually speak.
- S-4492 S-4494 S-4495Collector group membership stays true to the fleet: deleting a collector removes it from its group, deleting a group takes its configuration tree with it, and Kubernetes group members that are gone are reconciled away and release their licence seat.
- S-4523Every Receiver Type the New Source dropdown offers now has a configuration form behind it; the Docker Stats type, which had none, is retired for new sources.
- S-4386A create or update request carrying a field the API does not recognise is refused and the field named, instead of being silently discarded while the request reports success.
- S-4128Sidebar navigation icons render as real glyphs in brand blue instead of placeholder question marks.
Features and improvements
- S-4508 S-4509 S-4510 S-4512Prometheus scraping is now properly configurable: HTTPS with a Scheme field and a separate Metrics Path, basic auth, bearer token, a private CA, a corporate proxy, and per-scrape cardinality caps with metric filtering.
- S-4516 S-4526An OTLP listener can be secured and bounded: TLS and mutual TLS, request size limits, keepalive, CORS, and a custom URL path.
- S-4519 S-4522The syslog TCP and raw TCP log listeners can run behind TLS, and both raw log sources now control framing, encoding and maximum log size.
- S-4520 S-4521 S-4610Data survives a collector restart: File Tail and Kubernetes Pod Logs resume where they left off instead of re-reading or losing what was written, destinations keep a persistent sending queue with a chosen answer to a full queue, and a new destination is durable by default.
- S-4524Kubernetes sources can be scoped and authenticated — namespaces, API authentication mode, cluster distribution, metadata collection interval and watched resource types.
- S-4525 S-4517Host Metrics scrapers are configurable rather than only switched on or off, including root_path so a containerised collector reports the host it runs on instead of its own container.
- S-4527An http_check target carries its own headers, TLS settings and HTTP method, and a journald source can be scoped by unit, identifier and match.
- S-4493A collector managed by a group shows a compact "managed by" tag, and the tabs the group owns are locked so a per-collector edit cannot silently diverge from it.
- S-4501The unlicensed free tier is 5 collectors and no API automation. Registering at portal.opensight.ch issues a free signed Community licence that restores the 25-collector cap; API automation requires Enterprise.
21 further changes — build and test work, internal refactoring, and changes with no effect you could observe — were reviewed and deliberately left out.
8 other dependency updates in shipped components, with no security relevance.