Send Data with the OpenTelemetry .NET SDK | Honeycomb

Send Data with the OpenTelemetry .NET SDK

Use the OpenTelemetry .NET SDK to instrument .NET applications in a standard, vendor-agnostic, and future-proof way and send telemetry data to Honeycomb.

In this guide, we will walk you through instrumenting with OpenTelemetry for .NET, which will include adding automatic instrumentation to your application.

Before You Begin 

Before you can set up automatic instrumentation for your .NET application, you will need to do a few things.

Prepare Your Development Environment 

To complete the required steps, you will need:

  • A working .NET environment
  • An application written in .NET

Get Your Honeycomb API Key 

To send data to Honeycomb, you’ll need to sign up for a free Honeycomb account and create a Honeycomb Ingest API Key. To get started, you can create a key that you expect to swap out when you deploy to production. Name it something helpful, perhaps noting that it’s a getting started key. Make note of your API key; for security reasons, you will not be able to see the key again, and you will need it later!

Tip
For setup, make sure you check the “Can create datasets” checkbox so that your data will show up in Honeycomb. Later, when you replace this key with a permanent one, you can uncheck that box.

If you want to use an API key you previously stored in a secure location, you can also look up details for Honeycomb API Keys any time in your Environment Settings, and use them to retrieve keys from your storage location.

Add Automatic Instrumentation 

Configure 

You must configure your SDK to send events to Honeycomb, and then run your application to see traces.

When using OpenTelemetry for .NET, the following configuration properties are required:

OtlpExporterOptions property /
Environment Variable
Value
Endpoint
OTEL_EXPORTER_OTLP_ENDPOINT
https://api.honeycomb.io (US instance)
https://api.eu1.honeycomb.io (EU instance)
Headers
OTEL_EXPORTER_OTLP_HEADERS
x-honeycomb-team=HONEYCOMB_API_KEY
ServiceName
OTEL_SERVICE_NAME
your-service-name

You must set the service name in either the TracerProvider configuration code or as an environment variable.

The code below configures a tracer provider that can export traces to Honeycomb.

services.AddOpenTelemetry().WithTracing(builder => builder
    .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName))
    .AddAspNetCoreInstrumentation()
    .AddHttpClientInstrumentation()
    .AddOtlpExporter(option =>
    {
        option.Endpoint = new Uri("https://api.honeycomb.io"); // US instance
        //option.Endpoint = new Uri("https://api.eu1.honeycomb.io"); // EU instance
        option.Headers = $"x-honeycomb-team={honeycombApiKey}";
    }));

To learn more about OpenTelemetry configuration options, visit the OpenTelemetry for .NET GitHub repository.

Sampling 

You can configure the OpenTelemetry SDK to sample the data it generates. Honeycomb weights sampled data based on sample rate, so you must set a resource attribute containing the sample rate.

Use a TraceIdRatioBased sampler, with a ratio expressed as 1/N. Then, also create a resource attribute called SampleRate with the value of N. This allows Honeycomb to reweigh scalar values, like counts, so that they are accurate even with sampled data.

In the example below, our goal is to keep approximately half (1/2) of the data volume. The resource attribute contains the denominator (2), while the OpenTelemetry sampler argument contains the decimal value (0.5).

services.AddOpenTelemetry().WithTracing((builder) => builder
    .SetResourceBuilder(ResourceBuilder.CreateDefault()
        .AddService(serviceName)
        // IMPORTANT: add a SampleRate of 2 as a resource attribute
        .AddAttributes(new[] { new KeyValuePair<string, object>("SampleRate", 2) })
    )
    .SetSampler(new TraceIdRatioBasedSampler(0.5)) // sampler
    .AddAspNetCoreInstrumentation()
    .AddHttpClientInstrumentation()
    .AddOtlpExporter(option =>
    {
        option.Endpoint = new Uri("https://api.honeycomb.io"); // US instance
        //option.Endpoint = new Uri("https://api.eu1.honeycomb.io"); // EU instance
        option.Headers = $"x-honeycomb-team={honeycombApiKey}";
    }));

Using HTTP Instead of gRPC 

By default, the OpenTelemetry .NET SDK uses the gRPC protocol to send telemetry data. To use HTTP instead of gRPC for traces, update the protocol in the OtlpExporter:

services.AddOpenTelemetry().WithTracing(builder => builder
    .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName))
    .AddAspNetCoreInstrumentation()
    .AddHttpClientInstrumentation()
    .AddOtlpExporter(option =>
    {
        option.Endpoint = new Uri("https://api.honeycomb.io/v1/traces"); // US instance
        //option.Endpoint = new Uri("https://api.eu1.honeycomb.io/v1/traces"); // EU instance
        option.Headers = $"x-honeycomb-team={honeycombApiKey}";
        option.Protocol = OtlpExportProtocol.HttpProtobuf;
    }));

Note that the endpoint URL is different for HTTP/protobuf than for gRPC in that it specifies the signal type (/v1/traces).

If the application is running on .NET Core 3.x, also add the following at application startup:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
Note

If you are sending data directly to Honeycomb, you can find Endpoint configuration options and OpenTelemetry Headers in the OpenTelemetry for .NET chart.

If you are using an OpenTelemetry Collector, specify the endpoint of the collector, and add the headers to the collector configuration file.

Endpoint URLs for OTLP/HTTP 

When using the OTEL_EXPORTER_OTLP_ENDPOINT environment variable with an SDK and an HTTP exporter, the final path of the endpoint is modified by the SDK to represent the specific signal being sent.

For example, when exporting trace data, the endpoint is updated to append v1/traces. When exporting metrics data, the endpoint is updated to append v1/metrics. So, if you were to set the OTEL_EXPORTER_OTLP_ENDPOINT to https://api.honeycomb.io, traces would be sent to https://api.honeycomb.io/v1/traces and metrics would be sent to https://api.honeycomb.io/v1/metrics.

The same modification is not necessary for gRPC.

export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io # US instance
#export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.eu1.honeycomb.io # EU instance

If the desired outcome is to send data to a different endpoint depending on the signal, use OTEL_EXPORTER_OTLP_<SIGNAL>_ENDPOINT instead of the more generic OTEL_EXPORTER_OTLP_ENDPOINT.

When using a signal-specific environment variable, these paths must be appended manually. Set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT for traces, appending the endpoint with v1/traces, and OTEL_EXPORTER_OTLP_METRICS_ENDPOINT for metrics, appending the endpoint with v1/metrics.

Send both traces and metrics to Honeycomb using this method by setting the following variables:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.honeycomb.io/v1/traces # US instance
#export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.eu1.honeycomb.io/v1/traces # EU instance

export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://api.honeycomb.io/v1/metrics # US instance
#export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://api.eu1.honeycomb.io/v1/metrics # EU instance

More details about endpoints and signals can be found in the OpenTelemetry Specification.

URL Query String Parameter Value Redaction 

The Instrumentation.AspNetCore-1.8.1 and Instrumentation.Http-1.8.1 instrumentation packages redact query string parameter values by default. For example, a query string parameter of key=value would be added as a span attribute with a name of url.query and a value of key=Redacted.

You can disable this redaction by setting the environment variable OTEL_DOTNET_EXPERIMENTAL_HTTPCLIENT_DISABLE_URL_QUERY_REDACTION to true.

Add Multi-Span Attributes 

Sometimes you want to add the same attribute to many spans within the same trace. This attribute may include variables calculated during your program, or other useful values for correlation or debugging purposes.

To add this attribute to multiple spans, leverage the OpenTelemetry concept of baggage. Baggage allows you to add a key with a value as an attribute to every subsequent child span of the current application context.

First, install the OpenTelemetry.Extensions package with the .NET CLI:

dotnet add package OpenTelemetry.Extensions --prerelease

Then, when configuring the OpenTelemetry SDK tracer provider, add the BaggageActivityProcessor:

services.AddOpenTelemetry().WithTracing(builder => builder
    .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName))
    .AddBaggageActivityProcessor()
    .AddOtlpExporter(option =>
    {
        option.Endpoint = new Uri("https://api.honeycomb.io"); // US instance
        //option.Endpoint = new Uri("https://api.eu1.honeycomb.io"); // EU instance
        option.Headers = $"x-honeycomb-team={honeycombApiKey}";
    }));

Finally, add a baggage entry for the current trace and replace key and value with your desired key-value pair:

Baggage.Current.SetBaggage("key", "value");

Note: Any Baggage attributes that you set in your application will be attached to outgoing network requests as a header. If your service communicates to a third party API, do NOT put sensitive information in the Baggage attributes.

Troubleshooting 

To explore common issues when sending data, visit Common Issues with Sending Data in Honeycomb.