OpenTelemetry for Python | Honeycomb

We use cookies or similar technologies to personalize your online experience & tailor marketing to you. Many of our product features require cookies to function properly.

Read our privacy policy I accept cookies from this site

OpenTelemetry for Python

This guide will help you add OpenTelemetry to your web service, show you how to add additional custom attributes to that instrumentation, and ensure that instrumentation data is being sent to Honeycomb.

Requirements 

These instructions will explain how to set up manual instrumentation for a service written in Python. In order to follow along, you will need:

  • Python 3.6 or higher
  • An app written in Python
  • A Honeycomb API Key, which you can find on your Team Settings page. If you do not have an API key yet, sign up for a free Honeycomb account.

Adding Instrumentation 

Install Packages 

This is a quick start guide for Python with Flask.

Install these packages to instrument a Flask app with OpenTelemetry:

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install opentelemetry-exporter-otlp-proto-http
pip install opentelemetry-instrumentation-flask
pip install opentelemetry-instrumentation-requests

Initialize 

Create a new file, tracing.py. This file will create and initialize a tracer and Flask instrumentation to send data to Honeycomb:

# tracing.py

from opentelemetry import trace
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Initialize tracing and an exporter that can send data to Honeycomb
provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)

# Initialize automatic instrumentation with Flask
app = flask.Flask(__name__)
FlaskInstrumentor().instrument_app(app)
RequestsInstrumentor().instrument()

Configure and Run 

If using the dataset-only data model, refer to the Honeycomb Classic tab for instructions. Not sure? Learn more about Honeycomb versus Honeycomb Classic.

Configure OpenTelemetry to send events to Honeycomb using environment variables.

The header x-honeycomb-team is your API key. The header x-honeycomb-dataset is the name of a Dataset that you need to specify. A Dataset is a bucket where data gets stored in Honeycomb.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io/"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key,x-honeycomb-dataset=your-dataset"
export OTEL_SERVICE_NAME="your-service-name"
python app.py tracing.py

Configure OpenTelemetry to send events to Honeycomb using environment variables.

The header x-honeycomb-team is your API key. Your service name will be used as the Service Dataset in Honeycomb, which is where data is stored. The service name is specified by OTEL_SERVICE_NAME.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io/"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
export OTEL_SERVICE_NAME="your-service-name"
python app.py tracing.py

gRPC Instead of HTTP 

To use gRPC instead of HTTP, install the package, replace the import, and update the endpoint:

pip install opentelemetry-exporter-otlp-proto-grpc
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

# The rest of your code...
export OTEL_EXPORTER_OTLP_ENDPOINT=api.honeycomb.io:443

Creating Spans 

Now we have a tracer configured, we can create spans to describe what is happening in your application. For example, this could be a HTTP handler, a long running operation, or a database fetch. Spans are created in a parent-child pattern, so each time you create a new span, the current active span is used as its parent.

from opentelemetry import trace

tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("http-handler"):
    with tracer.start_as_current_span("my-cool-function"):
        # do something

Adding Attributes to Spans 

It is often beneficial to add context to a currently executing span in a trace. For example, you may have an application or service that handles users and you want to associate the user with the span when querying your dataset in Honeycomb. In order to do this, get the current span from the context and set an attribute with the user ID:

from opentelemetry import trace

...

span = trace.get_current_span()
span.set_attribute("user.id", user.id())

Sampling 

To control how many spans are being sent to Honeycomb, you can configure a sampler to send a portion of your traffic. Configuration of sampling can be done with environment variables:

export OTEL_TRACES_SAMPLER="traceidratio"
# set to a float representation of the desired ratio
export OTEL_TRACES_SAMPLER_ARG=0.5
# set to the denominator of the desired ratio
export OTEL_RESOURCE_ATTRIBUTES="SampleRate=2"

You can configure the OpenTelemetry SDK to sample the data it generates. Honeycomb re-weights sampled data, so it is recommended that you set a resource attribute containing the sample rate.

In the example above, our goal is to keep approximately half (1/2) of the data volume. The resource attribute contains the denominator (2), while the OpenTelemetry sampler argument contains the decimal value (0.5).

If you have multiple services that communicate with each other, it is important that they have the same sampling configuration. Otherwise, each service might make a different sampling decision, resulting in incomplete or broken traces. You can sample using a standalone proxy as an alternative, like Honeycomb Refinery, or when you have more robust sampling needs.

Distributed Trace Propagation 

When a service calls another service, you want to ensure that the relevant trace information is propagated from one service to the other. This allows Honeycomb to connect the two services in a trace.

Distributed tracing enables you to trace and visualize interactions between multiple instrumented services. For example, your users may interact with a front-end API service, which talks to two internal APIs to fulfill their request. In order to have traces connect spans for all these services, it is necessary to propagate trace context between these services, usually by using an HTTP header.

Both the sending and receiving service must use the same propagation format, and both services must be configured to send data to the same Honeycomb dataset.

Troubleshooting 

The OpenTelemetry Python SDK typically shows errors in the console when applicable. If no errors appear but your data is not in Honeycomb as expected, use a ConsoleSpanExporter to print your spans to the console. This will help confirm whether your app is being instrumented with the data you expect.

Import the ConsoleSpanExporter and temporarily replace the OTLPSpanExporter:

from opentelemetry.sdk.trace.export import (SimpleSpanProcessor, ConsoleSpanExporter)

trace.get_tracer_provider().add_span_processor(
    SimpleSpanProcessor(ConsoleSpanExporter())
)

Keep in mind that printing to the console is not recommended for production and should only be used for debugging purposes.

No Traces for a Service 

The service name is a required configuration value. If it is unspecified, all trace data will be sent to a default dataset called unknown_service.

OTLP Protobuf Definitions 

Honeycomb supports receiving telemetry data via OpenTelemetry’s native protocol, OTLP, over gRPC and HTTP/protobuf. Currently supported versions of OTLP protobuf definitions are 0.7.0 through 0.16.0 for traces, and 0.7.0 through 0.11.0 for metrics.

If the protobuf version in use by the SDK does not match a supported version by Honeycomb, a different version of the SDK may need to be used. If the SDK’s protobuf version is newer than this range, temporarily use an SDK with a supported version until the newer version is supported. If the SDK’s protobuf version is older than this range, upgrade the SDK to a version with the supported protobuf definitions. If using an added dependency on a proto library, ensure the version of protobuf definitions matches the supported version of the SDK.

Receiving 464 Errors 

You may receive a 464 error response from the Honeycomb API when sending telemetry using gRPC and HTTP1. The gRPC format depends on using HTTP2 and any request over HTTP1 will be rejected by the Honeycomb servers.

Did you find what you were looking for?