If you are running applications that are not using OpenTelemetry in your Kubernetes cluster, you can still collect the events from your cluster.
In this guide, you will learn how to use an OpenTelemetry Collector to get additional insight into your data by sending your Kubernetes Events to Honeycomb using OpenTelemetry’s Kubernetes Objects Receiver.
Before beginning this guide, you should have:
honeycomb
.Enable the Kubernetes Objects Receiver to collect events by adding the kubernetesEvents
preset in the values file for your OpenTelemetry Deployment-mode Collector. Place it near the top of the values file under the config
section:
presets:
kubernetesEvents:
enabled: true
By default, the kubernetesEvents
preset in the OpenTelemetry Helm chart will configure the Collector to pull all of the events from the Kubernetes cluster and export them as logs.
Honeycomb users can derive a great deal of value from these logs–when they are structured appropriately. To transform the bodies of these logs into structured content that is easily queried in Honeycomb, use the OpenTelemetry Transform Processor to parse the event data.
To force the event data to conform to the standard k8s.*
attribute naming that all other telemetry types use:
Add the following to the values file for your OpenTelemetry Deployment-mode Collector. Place it near the top of the values file under the config
section:
processors:
transform/events:
error_mode: ignore
log_statements:
- context: log
statements:
- set(attributes["watch-type"], body["type"]) where IsMap(body) and body["type"] != nil
- merge_maps(attributes, body, "upsert") where IsMap(body) and body["object"] == nil
- merge_maps(attributes, body["object"], "upsert") where IsMap(body) and body["object"] != nil
- merge_maps(attributes, attributes[ "metadata"], "upsert") where IsMap(attributes[ "metadata"])
# Maps the name of the resource to the right k8s.* attribute
- set(attributes["k8s.pod.name"], attributes["regarding"] ["name"]) where attributes["regarding"]["kind"] == "Pod"
- set(attributes["k8s.node.name"], attributes["regarding"]["name"]) where attributes["regarding"]["kind"] == "Node"
- set(attributes["k8s.job.name"], attributes["regarding"]["name"]) where attributes["regarding"]["kind"] == "Job"
- set(attributes["k8s.cronjob.name"], attributes["regarding"]["name"]) where attributes["regarding"]["kind"] == "CronJob"
- set(attributes["k8s.namespace.name"], attributes["regarding"]["namespace"]) where attributes["regarding"]["kind"] == "Pod" or attributes["regarding"]["kind"] == "Job" or attributes["regarding"]["kind"] == "CronJob"
# Converts event types to Otel log Severities
- set(severity_text, attributes["type"]) where attributes["type"] == "Normal" or attributes["type"] == "Warning"
- set(severity_number, SEVERITY_NUMBER_INFO) where attributes["type"] == "Normal"
- set(severity_number, SEVERITY_NUMBER_WARN) where attributes["type"] == "Warning"
Under the config
section, update your pipelines
section to match the following:
service:
pipelines:
metrics:
exporters: [ otlp/k8s-metrics ]
logs:
processors: [ memory_limiter, transform/events, batch ]
exporters: [ otlp/k8s-logs ]