Send Data with the OpenTelemetry Python SDK | Honeycomb

Send Data with the OpenTelemetry Python SDK

Use the OpenTelemetry Python SDK to instrument Python applications in a standard, vendor-agnostic, and future-proof way and send telemetry data to Honeycomb.

In this guide, we will walk you through instrumenting with OpenTelemetry for Python, which will include adding automatic instrumentation to your application.

Before You Begin 

Before you can set up automatic instrumentation for your Python application, you will need to do a few things.

Prepare Your Development Environment 

To complete the required steps, you will need:

  • A working Python environment
  • An application written in Python

Get Your Honeycomb API Key 

To send data to Honeycomb, you’ll need to sign up for a free Honeycomb account and create a Honeycomb Ingest API Key. To get started, you can create a key that you expect to swap out when you deploy to production. Name it something helpful, perhaps noting that it’s a getting started key. Make note of your API key; for security reasons, you will not be able to see the key again, and you will need it later!

Tip
For setup, make sure you check the “Can create datasets” checkbox so that your data will show up in Honeycomb. Later, when you replace this key with a permanent one, you can uncheck that box.

If you want to use an API key you previously stored in a secure location, you can also look up details for Honeycomb API Keys any time in your Environment Settings, and use them to retrieve keys from your storage location.

Add Automatic Instrumentation 

Automatic instrumentation is provided by an agent, a command line tool opentelemetry-instrument that is used to run your application. Add additional instrumentation to your application manually using the included OpenTelemetry Python SDK.

Acquire Dependencies 

To add instrumentation, you should install required OpenTelemetry packages and instrumentation libraries.

  1. Install the opentelemetry-distro and exporter packages:

    python -m pip install opentelemetry-distro \
        opentelemetry-exporter-otlp
    
  2. Install instrumentation libraries for the packages used by your application. We recommend using the opentelemetry-bootstrap tool that comes with the OpenTelemetry SDK to scan your application packages and print out a list of available instrumentation libraries. You should then add these libraries to your requirements.txt file:

    opentelemetry-bootstrap >> requirements.txt
    pip install -r requirements.txt
    

    If you do not use a requirements.txt file, you can install the libraries directly in your current environment:

    opentelemetry-bootstrap --action=install
    
  1. Install the opentelemetry-distro and exporter packages:

    poetry add opentelemetry-distro \
        opentelemetry-exporter-otlp
    
  2. Install instrumentation libraries for the packages used by your application. We recommend using the opentelemetry-bootstrap tool that comes with the OpenTelemetry SDK to scan your application packages to get a list of available libraries.

    poetry run opentelemetry-bootstrap
    

    The tool does not support installing packages directly when using Poetry, so you must install them manually. For example, to install the Flask instrumentation library:

    poetry add opentelemetry-instrumentation-flask
    

Configure and Run 

Once you have acquired the necessary dependencies, you can configure your SDK to send events to Honeycomb, and then run your application to see traces.

  1. Configure OpenTelemetry to send events to Honeycomb using environment variables:

    export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io" # US instance
    #export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.eu1.honeycomb.io" # EU instance
    export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
    export OTEL_SERVICE_NAME="your-service-name"
    
    Variable Description
    OTEL_EXPORTER_OTLP_ENDPOINT Honeycomb endpoint to which you want to send your data.
    OTEL_EXPORTER_OTLP_HEADERS Header containing x-honeycomb-team=, plus your API Key generated in Honeycomb.
    OTEL_SERVICE_NAME Service name. When you send data, Honeycomb creates a dataset in which to store your data and uses this as the name. Can be any string.
    Note

    If you use Honeycomb Classic, you must also specify the Dataset using the HONEYCOMB_TRACES_DATASET environment variable.

    export HONEYCOMB_TRACES_DATASET=my-traces
    

    To learn more about configuration options, visit Agent Configuration in the OpenTelemetry documentation.

  2. To see traces for your application, run your application using the OpenTelemetry Python automatic instrumentation tool opentelemetry-instrument, which configures the OpenTelemetry SDK:

    opentelemetry-instrument python myapp.py
    
    poetry run opentelemetry-instrument python myapp.py
    

Sampling 

You can configure the OpenTelemetry SDK to sample the data it generates. Honeycomb weights sampled data based on sample rate, so you must set a resource attribute containing the sample rate.

Use a TraceIdRatioBased sampler, with a ratio expressed as 1/N. Then, also create a resource attribute called SampleRate with the value of N. This allows Honeycomb to reweigh scalar values, like counts, so that they are accurate even with sampled data.

In the example below, our goal is to keep approximately half (1/2) of the data volume. The resource attribute contains the denominator (2), while the OpenTelemetry sampler argument contains the decimal value (0.5).

export OTEL_TRACES_SAMPLER="traceidratio"
export OTEL_TRACES_SAMPLER_ARG=0.5
export OTEL_RESOURCE_ATTRIBUTES="SampleRate=2"

The value of SampleRate must be a positive integer.

Choosing between gRPC and HTTP 

Most OpenTelemetry SDKs have an option to export telemetry as OTLP either over gRPC or HTTP/protobuf, with some also offering HTTP/JSON. If you are trying to choose between gRPC and HTTP, keep in mind:

  • Some SDKs default to using gRPC, and it may be easiest to start with the default option.
  • Some firewall policies are not set up to handle gRPC and require using HTTP.
  • gRPC may improve performance, but its long-lived connections may cause problems with load balancing, especially when using Refinery.

gRPC default export uses port 4317, whereas HTTP default export uses port 4318.

Troubleshooting 

To explore common issues when sending data, visit Common Issues with Sending Data in Honeycomb.