k8seventsreceiver

README

Kubernetes Events Receiver

The Kubernetes Events Receiver collects events from the Kubernetes API server. It collects all the new or updated events that come in.

Status
Stability	alpha: logs
Distributions	contrib, k8s
Issues
Code coverage
Code Owners	@dmitryax, @TylerHelmuth, @ChrsMark

Currently this receiver supports authentication via service accounts only. See example for more information.

Configuration

The following settings are optional:

• auth_type (default = serviceAccount): Determines how to authenticate to the K8s API server. This can be one of none (for no auth), serviceAccount (to use the standard service account token provided to the agent pod), or kubeConfig to use credentials from ~/.kube/config.
• namespaces (default = all): An array of namespaces to collect events from. This receiver will continuously watch all the namespaces mentioned in the array for new events.
• kube_api_qps (default = 5): Maximum number of queries per second to the Kubernetes API. Increase this if you see client-side throttling warnings in the collector logs.
• kube_api_burst (default = 10): Maximum burst size for requests to the Kubernetes API. Increase this alongside kube_api_qps if you see client-side throttling warnings.
• k8s_leader_elector (default: none): if specified, will enable Leader Election by using k8sleaderelector extension
• storage (default: none): specifies the storage extension to use for persisting the latest resourceVersion. When configured, the receiver persists the resourceVersion after processing each watch event. On restart, the receiver resumes from the persisted resourceVersion, preventing duplicate events.

  Important storage considerations:

  • Local or node-pinned volumes (hostPath, local PV): the collector pod becomes tied to a specific node. If that node fails or the pod is rescheduled elsewhere, the persisted data will not be accessible and persistence will not work correctly.
  • Network-attached volumes (ReadWriteMany): the volume is accessible from any node, so the collector pod can be freely rescheduled or fail over to a different node while still resuming from the correct resourceVersion. This is the recommended approach, especially when used with k8s_leader_elector.
  • Block volumes (ReadWriteOnce): supported for single-replica deployments where restarts are graceful. Not recommended with leader election across multiple nodes, as Kubernetes may take 30–90 seconds to detach and reattach the volume after a node failure.

• dedup_interval (default = 0): Throttles MODIFIED watch events per Event UID. See Event Deduplication.

Examples:

  k8s_events:
    auth_type: kubeConfig
    storage: file_storage
    k8s_leader_elector: k8s_leader_elector
    namespaces: [default, my_namespace]

The full list of settings exposed for this receiver are documented in config.go with detailed sample configurations in testdata/config.yaml.

Event Deduplication

dedup_interval is opt-in (default 0 preserves existing behavior). It throttles MODIFIED watch notifications per Event UID, useful when recurring events (CrashLoopBackOff, failing readiness probes, etc.) inflate log volume without adding new information.

Recurrence count is preserved on each emitted record via the k8s.event.count attribute.

Configuration

dedup_interval	Behavior
0 (default)	No throttling — emit every MODIFIED
Positive (e.g. 5m)	Emit the first MODIFIED per UID, then drop until the interval elapses
Negative (e.g. -1s)	Drop all MODIFIED events

ADDED events are always emitted and DELETED watch events are excluded. The dedup cache only tracks UIDs that have produced a MODIFIED, so single-fire Events do not consume cache space.

k8s_events:
  dedup_interval: 5m   # MODIFIED at most once per 5 min per Event UID

Dedup state retention

Per-UID dedup state lives in an in-memory cache so it does not grow unbounded. The entry TTL is derived automatically as dedup_interval + 5m and is not configurable. The TTL is reset on every emit, so an entry persists for as long as that UID is being actively emitted (at most once per dedup_interval); once emissions stop, the entry is evicted after the TTL elapses. The 5m buffer keeps state alive slightly longer than the throttle window so a UID that is still being throttled is never evicted mid-window.

Note: Dedup state is in-memory and is not persisted. On a collector restart or a leader election change, the cache starts empty, so the first MODIFIED seen afterward for each still-recurring Event UID is re-emitted before throttling resumes — throttling is best-effort across restarts.

Internal telemetry

When dedup_interval is set, the counter otelcol.k8s.events.modified.filtered reports how many MODIFIED watch notifications were dropped. See documentation.md for the metric definition.

Example

Here is an example deployment of the collector that sets up this receiver along with the OTLP Exporter.

Follow the below sections to setup various Kubernetes resources required for the deployment.

Configuration

Create a ConfigMap with the config for otelcontribcol. Replace OTLP_ENDPOINT with valid value.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
data:
  config.yaml: |
    extensions:
      file_storage:
        directory: /var/lib/otelcol/storage

    receivers:
      k8s_events:
        auth_type: serviceAccount
        storage: file_storage
        namespaces: [default, my_namespace]

    exporters:
      otlp_grpc:
        endpoint: <OTLP_ENDPOINT>
        tls:
          insecure: true

    service:
      extensions: [file_storage]
      pipelines:
        logs:
          receivers: [k8s_events]
          exporters: [otlp_grpc]
EOF

Service Account

Create a service account that the collector should use.

<<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: otelcontribcol
  name: otelcontribcol
EOF

RBAC

Use the below commands to create a ClusterRole with required permissions and a ClusterRoleBinding to grant the role to the service account created above. Alternatively, a namespace-scoped Role and RoleBinding can be used when the receiver is configured to watch specific namespaces via the namespaces option.

<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
rules:
- apiGroups: [""]
  resources: ["events"]
  verbs: ["get", "list", "watch"]
EOF

<<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: otelcontribcol
subjects:
- kind: ServiceAccount
  name: otelcontribcol
  namespace: default
EOF

Deployment

Create a Deployment to deploy the collector.

<<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: otelcontribcol-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: otelcontribcol
  labels:
    app: otelcontribcol
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otelcontribcol
  template:
    metadata:
      labels:
        app: otelcontribcol
    spec:
      serviceAccountName: otelcontribcol
      containers:
      - name: otelcontribcol
        # This image is created by running `make docker-otelcontribcol`.
        # If you are not building the collector locally, specify a published image: `otel/opentelemetry-collector-contrib`
        image: otelcontribcol:latest
        args: ["--config", "/etc/config/config.yaml"]
        volumeMounts:
        - name: config
          mountPath: /etc/config
        - name: storage
          mountPath: /var/lib/otelcol/storage
        imagePullPolicy: IfNotPresent
      volumes:
        - name: config
          configMap:
            name: otelcontribcol
        - name: storage
          persistentVolumeClaim:
            claimName: otelcontribcol-storage
EOF

Conversion Example

The following K8s Event

{
    "apiVersion": "v1",
    "count": 4,
    "eventTime": null,
    "firstTimestamp": "2025-09-08T06:20:41Z",
    "involvedObject": {
        "apiVersion": "v1",
        "fieldPath": "spec.containers{bad}",
        "kind": "Pod",
        "name": "bad-pod",
        "namespace": "default",
        "resourceVersion": "11326",
        "uid": "2c2a4312-12b4-4ef0-bba4-5295add58417"
    },
    "kind": "Event",
    "lastTimestamp": "2025-09-08T06:22:13Z",
    "message": "Pulling image \"nonexistentrepo/nonexistimage\"",
    "metadata": {
        "creationTimestamp": "2025-09-08T06:20:41Z",
        "name": "bad-pod.18633a5aeb89ba21",
        "namespace": "default",
        "resourceVersion": "11520",
        "uid": "86bc5e70-a921-4fbc-8b64-fa1316289423"
    },
    "reason": "Pulling",
    "reportingComponent": "kubelet",
    "reportingInstance": "kind-control-plane",
    "source": {
        "component": "kubelet",
        "host": "kind-control-plane"
    },
    "type": "Normal"
}

will be converted to the following log

{
  "SchemaURL": "",
  "ResourceAttributes": {
    "k8s.node.name": "kind-control-plane",
    "k8s.object.kind": "Pod",
    "k8s.object.name": "bad-pod",
    "k8s.object.uid": "2c2a4312-12b4-4ef0-bba4-5295add58417",
    "k8s.object.fieldpath": "spec.containers{bad}",
    "k8s.object.api_version": "v1",
    "k8s.object.resource_version": "11326"
  },
  "ScopeLogs": [
    {
      "SchemaURL": "",
      "InstrumentationScope": {},
      "LogRecords": [
        {
          "ObservedTimestamp": "1970-01-01T00:00:00Z",
          "Timestamp": "2025-09-08T06:22:13Z",
          "SeverityText": "Normal",
          "SeverityNumber": 9,
          "Body": "Pulling image \"nonexistentrepo/nonexistimage\"",
          "Attributes": {
            "k8s.event.reason": "Pulling",
            "k8s.event.action": "",
            "k8s.event.start_time": "2025-09-08T06:20:41Z",
            "k8s.event.name": "bad-pod.18633a5aeb89ba21",
            "k8s.event.uid": "86bc5e70-a921-4fbc-8b64-fa1316289423",
            "k8s.namespace.name": "default",
            "k8s.event.reporting_controller": "kubelet",
            "k8s.event.reporting_instance": "kind-control-plane",
            "k8s.event.count": 4
          },
          "TraceID": "",
          "SpanID": "",
          "Flags": 0
        }
      ]
    }
  ]
}

Documentation

Index

• func NewFactory() receiver.Factory
• type Config
  • func (cfg *Config) Validate() error

Functions

func NewFactory

func NewFactory() receiver.Factory

NewFactory creates a factory for k8s_cluster receiver.

Types

type Config

type Config struct {
	k8sconfig.APIConfig `mapstructure:",squash"`

	// List of ‘namespaces’ to collect events from.
	Namespaces []string `mapstructure:"namespaces"`

	// Storage is the ID of the storage extension to use for resource version persistence.
	// When set, the receiver will persist the latest resource version and resume from it on restart,
	// preventing duplicate events. Only valid for watch mode (which is the only mode this receiver uses).
	Storage *component.ID `mapstructure:"storage"`

	K8sLeaderElector *component.ID `mapstructure:"k8s_leader_elector"`

	// DedupInterval controls throttling of MODIFIED watch events per event UID.
	//   positive: emit MODIFIED only after interval has elapsed since the last emit for that UID
	//   0 (default): no throttling — current behavior, backward compatible
	//   negative: drop all MODIFIED events
	DedupInterval time.Duration `mapstructure:"dedup_interval"`
	// contains filtered or unexported fields
}

Config defines configuration for kubernetes events receiver.

func (*Config) Validate

func (cfg *Config) Validate() error