Add Automatic Instrumentation (Low Code)

Once you have a telemetry pipeline in place, you can gather metrics, logs, and traces from your deployed applications by adding automatic instrumentation to your applications. You can do this with very little code by installing the OpenTelemetry Operator for Kubernetes in your cluster. Once installed, the Operator will monitor when pods are created and check whether it should sideload automatic instrumentation packages. We consider this approach to be “low-code” because it requires you to access and modify the Kubernetes deployment manifests for your application. This approach also pushes automatic instrumentation into your code base as the application starts, so that trace data can be gathered. Although this is a very minimal amount of change overall and does not require you to directly modify any application code, we differentiate this from “no-code” because it does require that you modify how your applications are deployed on Kubernetes. Supported languages include:

.NET
Java
Node.js
Python
Go

Before You Begin

Before beginning this guide, you should have:

Created a running Kubernetes cluster.
Created a namespace named honeycomb.
Deployed an OpenTelemetry Collector in DaemonSet mode, listening on the Node IP.

Step 1: Install the Operator

Install the OpenTelemetry Operator Pods into your cluster using a Helm chart:

helm install \
--set admissionWebhooks.certManager.enabled=false \
--set admissionWebhooks.autoGenerateCert.enabled=true \
--set manager.collectorImage.repository="ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-k8s" \
--namespace honeycomb \
--create-namespace \
  opentelemetry-operator open-telemetry/opentelemetry-operator

In this example, we want the Operator to add automatic instrumentation to a Go application, so we must include the manager.featureGates configuration line to the command. If you are working with a different programming language, you can omit this line.

In this example, we use an automatically generated self-signed certificate by setting admissionWebhooks.certManager.enabled to false and admissionWebhooks.autoGenerateCert.enabled to true. We do not recommend that you use this configuration in a production environment, but setting up a cert-manager is outside the scope of this document.

Step 2: Verify the Operator Installation

Check that the Operator is installed by using the kubectl command to see if the pod is running:

kubectl get pods --namespace honeycomb

This command should return something like:

NAME                                                             READY   STATUS    RESTARTS   AGE
opentelemetry-operator-567bc4ff75-p287g                          2/2     Running   0          21s

The result should contain one pod, with two containers, running under a name containing the prefix opentelemetry-operator.

After verifying that the pod is running correctly, you can monitor the work the operator does by tailing the logs with the kubectl command:

kubectl logs -n honeycomb -f opentelemetry-operator-567bc4ff75-p287g

This command should return something like:

{"level":"info","ts":"2023-09-22T15:21:27Z","msg":"Starting Controller","controller":"opentelemetrycollector","controllerGroup":"opentelemetry.io","controllerKind":"OpenTelemetryCollector"}
{"level":"info","ts":"2023-09-22T15:21:27Z","logger":"instrumentation-resource","msg":"default","name":"otel-autoinstrumentation"}
{"level":"info","ts":"2023-09-22T15:21:27Z","logger":"collector-upgrade","msg":"no instances to upgrade"}
{"level":"info","ts":"2023-09-22T15:21:27Z","logger":"instrumentation-resource","msg":"validate update","name":"otel-autoinstrumentation"}
{"level":"info","ts":"2023-09-22T15:21:27Z","msg":"Starting workers","controller":"opentelemetrycollector","controllerGroup":"opentelemetry.io","controllerKind":"OpenTelemetryCollector","worker count":1}
{"level":"info","ts":"2023-09-22T15:34:24Z","logger":"instrumentation-resource","msg":"default","name":"otel-autoinstrumentation"}
{"level":"info","ts":"2023-09-22T15:34:24Z","logger":"instrumentation-resource","msg":"validate update","name":"otel-autoinstrumentation"}

Step 3: Configure Automatic Instrumentation

Deploy an Instrumentation manifest to your Kubernetes cluster, which will enable the OpenTelemetry Operator to automatically instrument your services:

kubectl apply --namespace honeycomb -f https://docs.honeycomb.io/_assets/code-samples/kubernetes/values-files/otel-autoinstrumentation.yaml

View the Manifest File

Download

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: otel-autoinstrumentation
spec:
  exporter:
    endpoint: http://$(OTEL_NODE_IP):4317
  propagators:
    - tracecontext
    - baggage
    - b3
  sampler:
    type: parentbased_traceidratio
    argument: "1"
  python:
    env:
      # Required if endpoint is set to 4317.
      # Python autoinstrumentation uses http/proto by default
      # so data must be sent to 4318 instead of 4317.
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://$(OTEL_NODE_IP):4318
  dotnet:
    env:
      # Required if endpoint is set to 4317.
      # Dotnet autoinstrumentation uses http/proto by default
      # See https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/blob/888e2cd216c77d12e56b54ee91dafbc4e7452a52/docs/config.md\#otlp
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://$(OTEL_NODE_IP):4318
  java:
    image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:2.10.0
    env:
      # Required if endpoint is set to 4317.
      # Java autoinstrumentation agent v2.0+ uses http/proto by default
      # See https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/9bbfe7fe4e3f65cb698d6d2320ac87372d5d572f/javaagent-tooling/src/main/java/io/opentelemetry/javaagent/tooling/config/OtlpProtocolPropertiesSupplier.java#L19
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://$(OTEL_NODE_IP):4318
  nodejs:
    env:
      # Optional.
      # We recommend disabling fs automatic instrumentation because
      # it can be noisy and expensive during startup
      - name: OTEL_NODE_DISABLED_INSTRUMENTATIONS
        value: fs

This manifest instructs the Operator to perform the following actions whenever you add annotations to your pods:

Inject instrumentation into your applications
Configure the export to send data to the node’s IP address

Step 4: Add Annotations to Your Application’s Kubernetes Manifest

In the deployment manifest for your application, add the required annotations. These will vary according to programming language. For example, our Go application requires the following annotations. Note where the annotations are located in the manifest:

apiVersion: apps/v1

kind: Deployment
metadata:
  name: my-go-app
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: my-go-app
      annotations:
        # Added Annotations for auto-instrumentation go here - points at the namespace where the instrumentation
        #  manifest is installed and the name of meta.name field in the instrumentation manifest
        instrumentation.opentelemetry.io/inject-go: "honeycomb/otel-autoinstrumentation"
        # Go Requires an additional annotation to the path of the binary on the container
        instrumentation.opentelemetry.io/otel-go-auto-target-exe: "/path/to/my-go-app"
    spec:
      serviceAccountName: my-go-app

Each supported programming language requires different annotations:

.NET
Java
Node.js
Python
Go

annotations:
  instrumentation.opentelemetry.io/inject-dotnet: "honeycomb/otel-autoinstrumentation"

annotations:
  instrumentation.opentelemetry.io/inject-java: "honeycomb/otel-autoinstrumentation"

annotations:
  instrumentation.opentelemetry.io/inject-nodejs: "honeycomb/otel-autoinstrumentation"

annotations:
  instrumentation.opentelemetry.io/inject-python: "honeycomb/otel-autoinstrumentation"

Go requires an extra annotation that tells the Operator where to find the Go binary of your application on the container.

annotations:
  instrumentation.opentelemetry.io/inject-go: "honeycomb/otel-autoinstrumentation"
  instrumentation.opentelemetry.io/otel-go-auto-target-exe: "/app/frontend-service"

Possible values for the annotation include:

"true" - inject an Instrumentation Kubernetes Custom Resource (CR) instance from the current namespace
"my-instrumentation" - inject a specific Instrumentation CR instance from the current namespace
"my-other-namespace/my-instrumentation" - inject a specific Instrumentation CR instance from another namespace
"false" - do not inject an Instrumentation CR instance

Explore Your Data In Honeycomb

The next time requests are made to your application, data should start flowing to Honeycomb. Once your applications are sending data, you can explore trace data and gain insights from the Kubernetes attributes that have been attached by the Kubernetes Attributes Processor in your Collector.

If several requests have been made to your application and after several minutes, you still do not see any data, reach out for help in our Pollinators Community Slack. Pro/Enterprise users can visit Honeycomb Support or email support@honeycomb.io.

Instrument Applications

Instrument Infrastructure

CI/CD Observability

Manage Pipelines

Shape Data

Control Data Volume

Use Cases

Add Automatic Instrumentation (Low Code)

Before You Begin

Step 1: Install the Operator

Step 2: Verify the Operator Installation

Step 3: Configure Automatic Instrumentation

Step 4: Add Annotations to Your Application’s Kubernetes Manifest

Explore Your Data In Honeycomb

Additional Resources

​Before You Begin

​Step 1: Install the Operator

​Step 2: Verify the Operator Installation

​Step 3: Configure Automatic Instrumentation

​Step 4: Add Annotations to Your Application’s Kubernetes Manifest

​Explore Your Data In Honeycomb

​Additional Resources

Before You Begin

Step 1: Install the Operator

Step 2: Verify the Operator Installation

Step 3: Configure Automatic Instrumentation

Step 4: Add Annotations to Your Application’s Kubernetes Manifest

Explore Your Data In Honeycomb

Additional Resources