Onboard Kubernetes, layer by layer

LinkMesh sets up a Kubernetes cluster with opinionated defaults. Cluster-wide data is collected once. Records carry the workload that produced them, minus the attributes that only add noise or leak secrets. Every record says which cluster it belongs to and, once you switch on the ownership mapping, which service. This page goes through the cluster one layer at a time. For each layer it covers what LinkMesh does, what it leaves to your backend or to the platform, and how to switch each default off.

It assumes a collector fleet is already running. If not, start with Deploy a collector fleet on Kubernetes. For background on what to monitor at each layer, independent of any tool, see the Kubernetes Monitoring Guide on opensight.ch.

Every setting on this page lives on the group’s Kubernetes tab (Groups → your group → Kubernetes), except label promotion, which is a setting of the destination. You can also set the group’s settings through PATCH /api/v1/collector-groups/{id}.

What LinkMesh does and what it leaves to others

LinkMesh does	Left to your backend	Left to the platform
Configures upstream collectors: which receivers run where, with which defaults	Alert rules	The control plane itself, and on managed Kubernetes its log and audit exports
Collects cluster-wide data exactly once	Dashboards	Monitoring stacks the platform already runs (OpenShift, Rancher, Kubermatic)
Enriches records with Kubernetes metadata and drops churning or sensitive attributes	Storage and retention	Node exporters, kube-state-metrics and similar agents
Stamps cluster, service and tenant identity, without overwriting what an application sets

LinkMesh routes and configures telemetry. It does not store or evaluate it, so alerting and dashboards belong in the backend you route to. It also ships no collector distribution and no extra exporters. Where the platform already runs kube-state-metrics or node-exporter, scrape those with a Prometheus source instead of running a second copy.

The two halves of a cluster: node group and cluster collector

A cluster in LinkMesh is two collector groups:

• The node group is the DaemonSet fleet, with one collector per node. It collects what each node can see for itself: pod logs, node and pod metrics from the kubelet, host metrics and OTLP from applications on that node.
• The cluster collector is a single-replica Deployment. It reads the Kubernetes API for the data that exists once per cluster: cluster-object metrics and Kubernetes events. On a DaemonSet that data would be collected once per node.

To add the cluster collector, open the node group’s Kubernetes tab, choose Cluster setup in the menu on the left, and click Add cluster collector. LinkMesh creates the second group, pairs it with the node group, and shows a manifest that already contains a reusable enrollment token. Before you apply the manifest, check that kubectl points at the right cluster. LinkMesh pairs the two groups when it creates them. It does not compare cluster identities at runtime, so a manifest applied to the wrong context pairs the wrong cluster. Then activate Cluster metrics and Cluster events on the cluster collector. Clicking the button again shows the same pair and its manifest. It does not create a second pair.

Three properties of the cluster collector:

• It runs otelcol-contrib. The two cluster receivers exist only in otelcol-contrib.
• It is one replica, and it keeps its identity. The manifest runs replicas: 1 with strategy: Recreate and a collector id that LinkMesh assigns. A replacement pod reports as the same collector. The group admits one member. A second collector that tries to join while the first is still registered is refused. Do not scale the Deployment: two replicas would run under one id.
• Its RBAC is read-only. Its ServiceAccount gets get, list and watch on the objects the cluster receivers read. It has no write verb, no access to Secrets or ConfigMaps, and no cluster-admin.

Cluster metrics and Cluster events are refused on a node group, whatever the group’s member count. On a one-node cluster the data would look correct at first, then start duplicating when a second node joins. If one of those sources is already active on a node group, it keeps running. Move it to the cluster collector so it is collected once.

The settings below (remove list, cluster name, ownership, tenant) belong to the cluster. You set them once, on the node group. The cluster collector’s Kubernetes tab shows them read-only, with a link back. It reads them from the node group, so node and cluster telemetry never disagree.

Per-layer sources

Layer	Source	Runs on
Hardware and kernel	Host Metrics, with the node’s root filesystem mounted (see Host metrics: report the node, not the pod)	node group
Nodes and kubelet	Kubernetes Node Metrics (kubeletstats)	node group
Control plane	Not collected by LinkMesh. See Platform notes	—
Cluster state	Kubernetes Cluster Metrics	cluster collector
Events	Kubernetes Events	cluster collector
Workload logs	Kubernetes Pod Logs	node group
Application traces, metrics and logs	OTLP, sent to the collector on the sender’s node	node group

The Overview section of the Kubernetes tab activates them in one click, with the Collect action on each source that is not collected yet. On the node group it offers Node metrics; the two cluster-wide sources are collected on the cluster collector. On the cluster collector it offers Cluster metrics and Cluster events.

Capacity defaults on cluster metrics

A new Kubernetes Cluster Metrics source starts with these settings. The upstream receiver ships with all of them off:

Field	New source starts with	To switch off
Allocatable Types To Report	cpu, memory, pods, ephemeral-storage	Remove lines. Leave it empty to report none.
Node Conditions To Report	Ready, MemoryPressure, DiskPressure, PIDPressure, NetworkUnavailable	Remove lines. Leave it empty to report Ready alone.
Optional Metrics	k8s.service.endpoint.count, k8s.service.load_balancer.ingress.count, k8s.container.status.reason, k8s.pod.status_reason, k8s.persistentvolumeclaim.status.phase	Remove a line to stop collecting that metric.

With these defaults you can compare requests with what each node can hold, see memory, disk and PID pressure next to readiness, and find a Service without ready endpoints, a LoadBalancer without an address, a crash-looping container or a claim stuck in Pending. Optional Metrics accepts only those five names. An unknown metric name would make the collector fail to load its config.

A Service that selects no pod reports no series

k8s.service.endpoint.count with condition=ready drops to 0 when a Service’s endpoints are not ready. A Service whose selector matches no pod at all reports no series. An alert on == 0 alone misses it, so also alert on the missing series.

These are the values a source starts with. A source you already have keeps its own settings, and nothing about it changes.

Events: Warning only by default, and read the count

A new Kubernetes Events source starts with Field Selector set to type=Warning. That drops the routine Normal events (Scheduled, Pulled, Started), which make up most of the event volume and are rarely what anyone looks for. To receive Normal events too, clear the field.

A repeated event is one event with a rising count, not a new event each time. A BackOff that keeps firing arrives as ADDED with count: 1, then as MODIFIED with 2, 3, 4 on the same event object. The count is in the record body. To measure how often something happened, read the count. Counting records gives the wrong number.

Enrichment

On a Kubernetes node group, LinkMesh adds a managed k8sattributes processor to Kubernetes Pod Logs, Kubernetes Node Metrics and OTLP sources. On the cluster collector, it adds the same processor to Kubernetes Cluster Metrics. It adds:

• Where the record ran: k8s.namespace.name, k8s.pod.name, k8s.node.name and the container image name and tag.
• The full owner chain: k8s.deployment.name, k8s.statefulset.name, k8s.daemonset.name, k8s.replicaset.name, k8s.job.name, k8s.cronjob.name.
• The recommended app labels: app.kubernetes.io/name, /instance, /version, /part-of and /component, written under the semantic-convention name k8s.pod.label.<key> (for example k8s.pod.label.app.kubernetes.io/name).

Each node collector caches only the pods on its own node, so a large cluster does not cost every collector memory for every pod. The cluster collector caches every pod in the cluster, because its cluster metrics cover pods on every node. That cost falls on its one replica.

On cluster metrics, enrichment applies to the pod and container metrics, such as k8s.pod.phase and k8s.container.restarts. Each one gets the name of the workload that owns its pod. Static pods such as etcd or kube-apiserver have no owner workload, so they get no owner name.

OTLP from applications is matched to its pod first by k8s.pod.uid or k8s.pod.ip, if the application sets them, then by the connection’s source address. Matching by connection works only when the application sends to the collector on its own node, through a Service with internalTrafficPolicy: Local. A hostPort does not work: the CNI rewrites the source address of same-node traffic to a hostPort. OTLP sent to the cluster collector is not enriched. It is one pod on one node, so it would label some senders and miss the rest.

Enrichment has no off switch. It only adds metadata. What you can switch off is what happens next: the remove list and the ownership mapping.

Remove list

After enrichment, every pipeline on a Kubernetes group drops these resource attributes:

process.pid, process.parent_pid, process.executable.path, process.command_line, process.command_args, process.owner, process.runtime.version, process.runtime.description, host.ip, host.mac, k8s.pod.start_time, k8s.pod.uid, container.image.id, container.image.repo_digests, os.description, os.build_id

They change on every rollout (k8s.pod.uid, k8s.pod.start_time, container.image.id), they multiply series without adding information (process.pid, host.ip), or they can carry secrets. process.command_line includes every flag a process was started with, tokens included. Enrichment still uses k8s.pod.uid to find the pod. The order is fixed: match the pod, enrich, then remove. The uid is used but never exported.

On the otelcol runtime, the Cluster metrics source also leaves out the OTLP entity reference it would otherwise put on each pod resource. That reference identifies the pod by k8s.pod.uid, so it would point at an attribute the export no longer has. Pod resources carry k8s.pod.name, k8s.namespace.name and k8s.node.name instead. With the remove list off, pod resources keep the reference.

The remove list applies to every pipeline on the cluster, OTLP from applications included. The processor-chain preview of a pipeline on a Kubernetes group shows it as a managed step, so you see what the export will contain.

To switch it off: on the node group’s Kubernetes tab, in the Data quality section, open Attribute remove list and clear Drop these attributes on every pipeline of this cluster. Through the API, set managedRemoveList: false on the node group. The setting covers both halves of the cluster.

Cluster identity and ownership

The ownership settings decide which cluster, which service and which tenant each record belongs to. Every value they stamp is insert-only. A value set by an SDK, by OTEL_RESOURCE_ATTRIBUTES, by a pod annotation or by upstream resource detection is never overwritten. The mapping fills gaps. It does not correct applications.

To change them, open the node group’s Kubernetes tab. In the Overview section, the stack view states what each bracket’s telemetry is reported as; click Edit ownership to open the Cluster ownership dialog.

Cluster name

Every record from both halves of the cluster carries k8s.cluster.name. It defaults to the node group’s name, and you can edit it in Cluster name. Some backends index it: Grafana Cloud Loki’s OTLP path, for example, makes it a stream label. To switch it off: empty the field (through the API, clusterName: "").

Platform or workload

The mapping classifies every resource as platform or workload:

1. On the receivers that describe the cluster (node metrics, cluster metrics, events, host metrics, pod logs), a resource with no pod and no namespace is platform: a node, the kubelet, the cluster state.
2. A resource whose namespace matches the Platform namespaces list is platform.
3. Everything else is workload.

On OTLP and on Prometheus scrapes, only rule 2 applies. A resource without pod metadata there means “not enriched”, not “a node”. Events are classified by their namespace, so a node’s events, which live in default, count as workload.

The default Platform namespaces list is kube-system, kube-public, kube-node-lease, linkmesh, ingress-nginx, cert-manager, calico-system, tigera-operator, gmp-system, gmp-public, gke-managed-*, amazon-cloudwatch and monitoring. Entries are globs. Edit the list to match your cluster, for example add openshift-* on OpenShift or cattle-* on Rancher.

Platform resources get the Platform service namespace you configure (for example managed-kubernetes) as service.namespace. LinkMesh never invents a default. Their service.name is the component:

Platform resource	service.name
Kubelet metrics about a node	kubelet
Host metrics	node
Cluster metrics about a node or another object outside a namespace	cluster-state
Events in a platform namespace	cluster-events
Pod in a platform namespace	its owner workload (coredns, kube-proxy), else its container

A static pod such as etcd-<node> has no owner workload. It gets the platform service.namespace and no service.name.

The workload precedence chain

For workload resources, each attribute takes the first value it finds, in this order:

Step	service.namespace	service.name
1. Pod annotation	resource.opentelemetry.io/service.namespace	resource.opentelemetry.io/service.name
2. Label keys (opt-in)	the keys you list, on the pod, then on its Deployment or StatefulSet	the keys you list, on the pod
3. Namespace map	the first namespace-glob = service namespace line that matches	—
4. Fallback	k8s.namespace.name	the owner workload (Deployment, StatefulSet, DaemonSet, CronJob, Job), then the container name

The annotation always wins. The ReplicaSet is skipped as an owner because its name changes on every rollout. With steps 2 and 3 empty, the result is the one the OpenTelemetry semantic conventions describe.

• Label keys are off by default. Tick Use Kubernetes labels for service.namespace and service.name and the fields are pre-filled with app.kubernetes.io/part-of for service.namespace, and app.kubernetes.io/instance, app.kubernetes.io/name for service.name. Change them to the labels your teams actually set. Reading a Deployment’s or StatefulSet’s labels needs get, list and watch on apps/deployments and apps/statefulsets. The install commands on the Add Collector page grant them. If you installed with your own manifest, add them.
• Namespace map is empty by default. Add one namespace-glob = service namespace line per rule, for example shop-* = webshop. The first match wins.

Cluster metrics about an object in a workload namespace are reported under cluster-state, not under the workload. This covers a Deployment’s replica counts, a ReplicaSet, a StatefulSet, a Job and the namespace itself. Such a record has no pod, so the pod annotation and the pod labels cannot reach it. If the chain ran on it, the record would fall back to the Deployment’s name, and a workload whose pods are named by an annotation or a label would appear under two services. The record’s service.namespace comes from the namespace map, then k8s.namespace.name. It keeps k8s.deployment.name and the other object names, so filter on those to find one workload’s replica counts. Cluster metrics about a pod or a container follow the chain like every other pod record.

Both scopes switch on together, when you set a Platform service namespace. Without one, the mapping is off and no record gains a service.* from LinkMesh. To switch it off: clear Map platform telemetry to this service namespace, or empty the platform service namespace.

Tenant

Tenant value stamps a tenant attribute on every record of the cluster. It is off until you set a value. The key defaults to linkmesh.tenant.id. The semantic conventions have no general tenant attribute, and their only tenant.id means an Azure AD tenant. Set Tenant attribute key to the key your backend already uses, for example customer. The key cannot be a service.* key or k8s.cluster.name. To switch it off: empty the value.

Worked example: a provider running managed Kubernetes

A hosting provider runs Kubernetes clusters for its customers. Its operations team owns the platform as the business service managed-kubernetes. The customer Acme runs a webshop in the namespaces shop-eu and shop-us, and a CRM in crm. On Acme’s node group the provider sets:

Field	Value
Cluster name	acme-prod-eu
Platform service namespace	managed-kubernetes
Platform namespaces	the default list
Tenant attribute key / value	customer / acme
Namespace map	shop-* = webshop

Every record then carries k8s.cluster.name=acme-prod-eu and customer=acme, and the records split like this:

Record	service.namespace	service.name	Why
Kubelet metrics of a node	managed-kubernetes	kubelet	platform, no pod
A coredns pod’s logs in kube-system	managed-kubernetes	coredns	platform namespace, owner name
Cluster state of a node	managed-kubernetes	cluster-state	platform, no pod
Logs of Deployment checkout in shop-eu	webshop	checkout	namespace map, owner name
Replica counts of Deployment checkout in shop-eu	webshop	cluster-state	object in a workload namespace, namespace map
OTLP from checkout, whose SDK sets service.namespace=payments	payments	as the SDK sets it	the SDK’s value is kept
Logs of StatefulSet crm-db in crm	crm	crm-db	namespace fallback, owner name

The provider’s own dashboards filter on service.namespace=managed-kubernetes and see only the platform. Acme’s teams see their own services. Nobody has to add a label to a single workload.

Check enrichment and ownership

The Enrichment and ownership check shows what the records from this cluster actually carry: whether enrichment found their pods, whether the ownership mapping gave them the service.namespace you expect, and whether the remove list is doing its job. It lives on the node group’s Kubernetes tab: choose Data quality in the menu on the left and open the card. Click Run check.

It needs the linkmesh-agent on the nodes together with linkmesh-agent-capture.yaml; see Install on Kubernetes. Running it needs the capture:run permission. On a cluster collector group the card points you to the node group, because the check samples what the node collectors enrich.

What it samples

• One node collector: the first connected otelcol collector of the node group whose node runs a linkmesh-agent that supports the check.
• Logs, metrics and traces, taken where they enter the group’s first enabled route.
• At most 20 seconds and 500 records per signal. The report says when a signal reached its 500-record bound.
• Only counts leave the cluster. No record is sent to LinkMesh or stored.
• The collector restarts twice: once when the temporary capture configuration is applied, once when it is removed. Right after a restart the first records can pass through unenriched while the collector’s pod cache warms up.

What the numbers mean

Row	What it counts
Pod records with a workload owner	Records with k8s.pod.name that also carry a Deployment, StatefulSet, DaemonSet, Job or CronJob name.
ReplicaSet pods with their Deployment name	Records with k8s.replicaset.name that also carry k8s.deployment.name.
Records with service.name	All counted records that carry service.name.
Records still carrying k8s.pod.uid	Should be 0 while the remove list is on.
Workload records that fell back to the Kubernetes namespace	Shown only with the ownership mapping on: workload records whose service.namespace is just their Kubernetes namespace, because nothing in the chain matched them.

Below the table, Resulting service.namespace values lists every value the sample carried, marked platform or workload, with its share. A value that came from the namespace name is marked as such. Records without any service.namespace are listed as (none).

A share under 95% counts as low and produces a likely cause.

Likely causes

• Pods carry no workload metadata at all. The collector’s ServiceAccount cannot read pods (get, list, watch on pods), or the records carry nothing the processor can match to a pod (k8s.pod.uid or the pod IP). The collector’s log shows forbidden in the first case.
• Some pods carry it and some do not. The processor passes records through unenriched until its pod cache is ready after a start, and that takes longer when one of its reads is forbidden. Check the collector’s log for forbidden and its ClusterRole against the install manifests.
• Pods without an owner. Bare pods and static pods such as kube-apiserver have no controller. This is information, not a fault.
• ReplicaSet pods without a Deployment name. The ReplicaSets are not managed by a Deployment, or the collector cannot read replicasets in the apps group.
• No owner labels. The workload chain reads Deployment or StatefulSet labels, and no record carries one. The collector cannot read deployments, statefulsets and replicasets, or no workload sets the label keys the chain names.
• Ownership mapping is off. service.namespace and service.name then come only from SDKs, OTEL_RESOURCE_ATTRIBUTES or pod annotations.
• Workloads fell back to their namespace. No annotation, label key or namespace map entry matched them. Add a map entry or set the label.
• Records without service.namespace although the mapping is on. They come from a source the mapping does not reach, or the workload chain for service.namespace is off.
• Records without service.name. The workload chain for service.name is off, or the records have no owner and no container name to fall back to.
• k8s.pod.uid still present. With the remove list on, those records reach the route without passing a managed source pipeline. With it off, this is expected.

When it cannot check

The card says Cannot be checked on this install and names the reason:

• Live Capture is not available on this LinkMesh server.
• No collector of the group is connected.
• No linkmesh-agent with the capture Service runs on the collectors’ nodes. Install the agent DaemonSet together with linkmesh-agent-capture.yaml.
• The linkmesh-agent on the node is too old. Upgrade the agent DaemonSet.
• The group has no enabled route, or no source sends into a route yet.
• No record reached the agent in 20 seconds. Either the sources sent nothing, or the collector could not reach the agent on its node: look for capture exporter errors such as connection refused or 401 in the collector’s log.

A check can also fail outright: another capture is already running on that node, the licence is read-only so the temporary configuration cannot be pushed, or the collector went offline. Wait and run it again.

Labels in Prometheus backends

Enrichment puts the workload on resource attributes. A Prometheus backend keeps those on the target_info series, not on every metric, so a query by Deployment needs a join.

A Prometheus Remote Write destination can promote a selected set to labels on every series instead. Turn on Promote Selected Resource Attributes To Labels. With Promoted Attributes left empty, it promotes k8s.cluster.name, k8s.namespace.name, k8s.deployment.name, k8s.statefulset.name, k8s.daemonset.name, k8s.node.name, service.namespace and service.name. Everything else stays on target_info.

It is off by default; leave it off to keep the destination’s config unchanged. OTLP and Grafana Cloud destinations have no such switch: the backend decides which attributes become labels.

Platform detection

LinkMesh reads the platform the cluster runs on from the collectors’ own self-telemetry and, where the LinkMesh agent runs in the cluster, from the agent’s cluster inventory. It does not read or change customer records for this. GET /api/v1/collector-groups/{id} returns platform on Kubernetes groups, with platformDetail (reportingCollectors, evidence, cloudProvider, reportedAt, source, monitoringStack, and inventory and conflict when the agent reported):

platform	When
openshift	OpenShift, on a cloud or on-premises
gke, gke-autopilot	Google Kubernetes Engine, Standard or Autopilot
aks	Azure Kubernetes Service. Self-managed Kubernetes on Azure VMs reads the same.
eks	Amazon EKS, including Fargate. Needs the agent.
k3s, rke2	K3s and RKE2. Needs the agent.
kubeadm	A cluster set up with kubeadm, kind included. Needs the agent.
unknown	Anything else, with cloudProvider set when a cloud was seen. Without the agent, EKS and self-managed Kubernetes on EC2 read unknown with cloudProvider: aws.

Both halves of the cluster count. An unknown member never outvotes a member that names a platform. Off a cloud, detection adds up to about 8 seconds to each collector start. otelcol-contrib collectors report only when their manifest sets config_apply_timeout: 30s on the supervisor, as the current install commands do. Until then the group reads unknown with the evidence “not reported yet”.

With the agent in the cluster

The agent reports the API server’s version, a fixed set of platform labels from up to 50 nodes, and the kubeadm.kubernetes.io/ annotations on kube-system pods. No other node label or annotation leaves the cluster. It reports them when it connects and every 30 minutes. Reading the node labels needs get and list on nodes, which the agent’s install grants. Without them, the agent still names EKS, K3s, RKE2 and kubeadm from the version and the annotations.

• The collectors’ answer comes first. The agent’s answer is used only where the collectors say unknown. Then source is agent-inventory. When both name a platform and they differ, the collectors’ answer is shown and conflict names both answers. The stack view shows the conflict under How we know.
• A plain API server version names nothing. kubeadm, Talos and MicroK8s all report one. kubeadm is named only from the annotations kubeadm writes.

Monitoring stack

monitoringStack says whether the cluster already runs kube-state-metrics, node-exporter or OpenShift monitoring. The stack view shows it next to the platform:

status	Meaning
detected	Found, with the namespace and workload of each component
none-found	The agent lists the cluster’s workloads and none of them is one of the three
not-determinable	No agent runs in the cluster, so nothing in LinkMesh can see its workloads

Where one is found, you can scrape it with a Prometheus source instead of collecting the same series a second time.

Platform notes

Managed control planes

On EKS, GKE and AKS the provider runs the control plane, and no collector in the cluster can scrape it. Its logs and audit events go to the provider’s own logging service, and you switch them on there:

• EKS: enable control-plane logging for the types you need (api, audit, authenticator, controllerManager, scheduler). They go to CloudWatch Logs, in the log group /aws/eks/<cluster>/cluster:

  aws eks update-cluster-config --name <cluster> \
    --logging '{"clusterLogging":[{"types":["api","audit","authenticator"],"enabled":true}]}'

• GKE: API server, scheduler and controller-manager logs are optional Cloud Logging components. Admin Activity audit logs are always on. Data Access audit logs are configured in the project’s IAM audit settings.

  gcloud container clusters update <cluster> \
    --logging=SYSTEM,WORKLOAD,API_SERVER,SCHEDULER,CONTROLLER_MANAGER

• AKS: create a diagnostic setting on the cluster resource for the categories you need (kube-apiserver, kube-audit-admin, kube-audit, kube-controller-manager, kube-scheduler). kube-audit-admin leaves out get and list requests and is much smaller than kube-audit.

  az monitor diagnostic-settings create --name control-plane \
    --resource <aks-resource-id> --workspace <log-analytics-workspace-id> \
    --logs '[{"category":"kube-audit-admin","enabled":true},{"category":"kube-apiserver","enabled":true}]'

Self-managed control planes

On kubeadm, RKE2 and similar clusters, the API server, scheduler, controller manager and etcd run as static pods in kube-system. Their logs are ordinary pod logs on the control-plane nodes, and the install manifests tolerate the control-plane taint, so a node collector runs there and Kubernetes Pod Logs collects them. If your control-plane nodes carry a different taint, add it as described under Production hardening. With the ownership mapping on, these logs go to the platform service namespace, because kube-system is in the default platform list.

The API server’s own metrics endpoint listens on 127.0.0.1 on kubeadm by default. LinkMesh does not scrape the control plane.

Audit policy basics

On a self-managed cluster you choose what the audit log records, with the policy file passed to the API server (--audit-policy-file). The API server writes the log to a file on the control-plane node (--audit-log-path). A File Tail source can read it, provided the collector on that node mounts the directory. A useful starting policy:

apiVersion: audit.k8s.io/v1
kind: Policy
omitStages: [RequestReceived]
rules:
  # Secrets, ConfigMaps and token reviews: who touched what, never the content.
  - level: Metadata
    resources:
      - group: ""
        resources: [secrets, configmaps]
      - group: authentication.k8s.io
        resources: [tokenreviews]
  # Routine reads are noise.
  - level: None
    verbs: [get, list, watch]
  # Everything else: who did what to which object.
  - level: Metadata

Log Secrets at Metadata level, never at Request or RequestResponse. Those levels write the Secret’s content into the audit log, and from there into every system that receives it.

Clusters with their own monitoring stack

• OpenShift ships its own monitoring stack (Prometheus in openshift-monitoring) and its own logging. Use what the platform already collects instead of scraping it again. Install with the OpenShift platform variant and the stack view marks cluster state, capacity and host metrics Provided by OpenShift instead of offering to collect them. Add openshift-* to the platform namespaces so the platform’s own pods are classified as platform.
• Rancher installs its monitoring into cattle-monitoring-system. Add cattle-* to the platform namespaces, and scrape its kube-state-metrics and node-exporter with a Prometheus source rather than running a second copy.
• Kubermatic (KKP) runs user-cluster monitoring from the seed cluster. Treat it like any existing stack: collect what it does not, and do not duplicate what it does.

GKE Autopilot and EKS Fargate

Both refuse what the standard node collectors mount from the node. Pick the GKE Autopilot or EKS Fargate platform in the Add Collector wizard (see Pick your platform):

• GKE Autopilot refuses writable host directories. Its variant mounts nothing from the node and keeps read positions and queued data in the pod, so they are lost when the pod is replaced. Platform detection names Autopilot gke-autopilot and the wizard pre-selects it.
• EKS Fargate runs no DaemonSets. Its variant is one collector pod, and a Fargate profile must select the linkmesh namespace. Kubernetes attributes are added only for pods on the collector’s own node, which on Fargate is the collector itself. Fargate cannot be detected, so pick it by hand. The cluster collector is a plain Deployment with no host directory and runs on Fargate unchanged.

On both, pod logs, node and kubelet metrics, host metrics and the system journal are not available, and the stack view marks them so rather than offering them.

Defaults and opt-outs at a glance

Default	Applies to	To switch it off
Cluster metrics and events run on the cluster collector, never on a node group	Kubernetes groups	No opt-out.
Allocatable types, five node conditions, five optional metrics	new Kubernetes Cluster Metrics sources	Remove lines from the fields.
type=Warning field selector	new Kubernetes Events sources	Clear Field Selector.
Managed k8sattributes enrichment	pod logs, node metrics and OTLP on node groups; cluster metrics on the cluster collector	No opt-out. It only adds attributes.
Remove list	every pipeline of the cluster	Attribute remove list checkbox, or managedRemoveList: false
k8s.cluster.name = node group’s name	every record of the cluster	Empty Cluster name, or clusterName: ""
Ownership mapping	off until a platform service namespace is set	Clear the checkbox, or empty the platform service namespace.
Label keys in the workload chain	off	They are off unless you tick the label opt-in.
Tenant attribute	off until a value is set	Empty Tenant value.
Label promotion	off	Leave Promote Selected Resource Attributes To Labels off on the Prometheus Remote Write destination.

Next steps

• Deploy a collector fleet on Kubernetes: install the node group.
• Preview a processor chain: see the managed steps on real records before they reach a collector.
• Concepts → Collector group: group kinds and membership.

Ready-to-use collector configs

Complete OpenTelemetry Collector configs for the same jobs, validated against the collector and with the mistake that usually breaks each one:

• Kubernetes pod, namespace and deployment on every record
• Collect Kubernetes pod logs (DaemonSet)
• Collect node, pod and container metrics from the kubelet
• Collect Kubernetes cluster metrics (one replica)
• Collect Kubernetes events as logs
• Scrape cAdvisor container metrics from the kubelet
• Scrape pods by prometheus.io annotations
• Kubernetes pod logs to Loki, enriched with deployment and cluster