1. Docs
2. Send Data to Honeycomb
3. OpenTelemetry for Ruby

OpenTelemetry for Ruby

Table of Contents

OpenTelemetry for Ruby can be used to add automatic and manual instrumentation to your applications and have trace data sent to Honeycomb. Automatic instrumentation is enabled by adding instrumentation packages. Manual instrumentation can be added using the OpenTelemetry API.

Requirements 

These instructions will explain how to set up automatic and manual instrumentation for a Ruby service. In order to follow along, you will need:

• Ruby 2.5 or higher
• A service written in Ruby
• A Honeycomb API Key. You can find your API key in your Environment Settings. If you do not have an API key yet, sign up for a free Honeycomb account.

Automatic Instrumentation 

Acquire Dependencies 

Add these gems to your Gemfile:

gem 'opentelemetry-sdk'
gem 'opentelemetry-exporter-otlp'
gem 'opentelemetry-instrumentation-all'

Including opentelemetry-instrumentation-all provides instrumentation for Rails, Sinatra, several HTTP libraries, and more.

Initialize 

Initialize OpenTelemetry early in your application lifecycle. For Rails applications, we recommend that you use a Rails initializer. For other Ruby services, initialize as early as possible in the startup process.

# config/initializers/opentelemetry.rb
require 'opentelemetry/sdk'
require 'opentelemetry/exporter/otlp'
require 'opentelemetry/instrumentation/all'

OpenTelemetry::SDK.configure do |c|
    c.use_all() # enables all instrumentation!
end

Configure 

Use environment variables to configure OpenTelemetry to send events to Honeycomb:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
export OTEL_SERVICE_NAME="your-service-name"

export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key,x-honeycomb-dataset=your-dataset"

Run 

Run your Ruby application.

In Honeycomb’s UI, you should now see your application’s incoming requests and outgoing HTTP calls generate traces.

Adding Manual Instrumentation 

Automatic instrumentation is the easiest way to get started with instrumenting your code, but in order to get the most insight into your system, you should add manual instrumentation where appropriate. To do this, use the OpenTelemetry SDK to access the currently executing span and add attributes to it, and/or to create new spans.

To add manual instrumentation, you need to add the OpenTelemetry SDK gem to your Gemfile:

gem 'opentelemetry-sdk'

Acquiring a Tracer 

To create spans, you need to get a Tracer.

require "opentelemetry/sdk"

tracer = OpenTelemetry.tracer_provider.tracer("tracer.name.here")

When you create a Tracer, OpenTelemetry requires you to give it a name as a string. This string is the only required parameter.

When traces are sent to Honeycomb, the name of the Tracer is turned into the library.name field, which can be used to show all spans created from a particular tracer.

In general, pick a name that matches the appropriate scope for your traces. If you have one tracer for each service, then use the service name. If you have multiple tracers that live in different “layers” of your application, then use the name that corresponds to that “layer”.

The library.name field is also used with traces created from instrumentation libraries.

Adding Attributes to Spans 

It is often beneficial to add information to a currently executing span in a trace. For example, you may have an application or service that handles users, and you want to associate the user with the span when querying your service in Honeycomb. In order to do this, get the current span from the context and set an attribute with the user ID:

# somewhere within the service, the SDK has been required and configured
require 'opentelemetry/sdk'
OpenTelemetry::SDK.configure do ... end

# ...

def handle_user(user)
  current_span = OpenTelemetry::Trace.current_span
  current_span.set_attribute("user.id", user.id)
end

This will add a user.id field to the current span so that you can use the field in WHERE, GROUP BY, or ORDER clauses in the Honeycomb query builder.

Creating New Spans 

Automatic instrumentation can show the shape of requests to your system, but only you know the really important parts. In order to get the full picture of what is happening, you will have to add manual instrumentation and create some custom spans. To do this, grab the tracer from the OpenTelemetry API:

# somewhere within the service, the SDK has been required and configured
require 'opentelemetry/sdk'
OpenTelemetry::SDK.configure do ... end

# ...

def run_query
  tracer = OpenTelemetry.tracer_provider.tracer('my-tracer')
  tracer.in_span("expensive-query") do |span|
    # ... cool stuff
    span.set_attribute('coolness', 100)
  end
end

Sampling 

You can configure the OpenTelemetry SDK to sample the data it generates. Honeycomb weights sampled data based on sample rate, so you must set a resource attribute containing the sample rate.

Use a TraceIdRatioBased sampler, with a ratio expressed as 1/N. Then, also create a resource attribute called SampleRate with the value of N. This allows Honeycomb to reweigh scalar values, like counts, so that they are accurate even with sampled data.

In the example below, our goal is to keep approximately half (1/2) of the data volume. The resource attribute contains the denominator (2), while the OpenTelemetry sampler argument contains the decimal value (0.5).

export OTEL_TRACES_SAMPLER="traceidratio"
export OTEL_TRACES_SAMPLER_ARG=0.5
export OTEL_RESOURCE_ATTRIBUTES="SampleRate=2"

If you have multiple services that communicate with each other, it is important that they have the same sampling configuration. Otherwise, each service might make a different sampling decision, resulting in incomplete or broken traces. You can sample using a standalone proxy as an alternative, like Honeycomb Refinery, or when you have more robust sampling needs.

Distributed Trace Propagation 

When a service calls another service, you want to ensure that the relevant trace information is propagated from one service to the other. This allows Honeycomb to connect the two services in a trace.

Distributed tracing enables you to trace and visualize interactions between multiple instrumented services. For example, your users may interact with a front-end API service, which talks to two internal APIs to fulfill their request. In order to have traces connect spans for all these services, it is necessary to propagate trace context between these services, usually by using an HTTP header.

Both the sending and receiving service must use the same propagation format, and both services must be configured to send data to the same Honeycomb environment.

Endpoint URLs for OTLP/HTTP 

When using the OTEL_EXPORTER_OTLP_ENDPOINT environment variable with an SDK and an HTTP exporter, the final path of the endpoint is actually modified by the SDK to represent the specific signal being sent.

For example, when exporting trace data, the endpoint is updated to append v1/traces. When exporting metrics data, the endpoint is updated to append v1/metrics. The same modification is not necessary for gRPC.

By setting OTEL_EXPORTER_OTLP_ENDPOINT to https://api.honeycomb.io, traces are sent to https://api.honeycomb.io/v1/traces and metrics to https://api.honeycomb.io/v1/metrics.

export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io

If the desired outcome is to send data to a different endpoint depending on the signal, use OTEL_EXPORTER_OTLP_<SIGNAL>_ENDPOINT instead of the more generic OTEL_EXPORTER_OTLP_ENDPOINT.

When using a signal-specific environment variable, these paths must be appended manually. Set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT for traces, appending the endpoint with v1/traces, and OTEL_EXPORTER_OTLP_METRICS_ENDPOINT for metrics, appending the endpoint with v1/metrics.

Send both traces and metrics to Honeycomb using this method by setting the following variables:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.honeycomb.io/v1/traces
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://api.honeycomb.io/v1/metrics

More details about endpoints and signals can be found in the OpenTelemetry Specification.

Troubleshooting 

If no errors appear in the console but your data is not in Honeycomb as expected, use a ConsoleSpanExporter to print your spans to the console. This will help confirm whether your app is being instrumented with the data you expect.

As shown in the OpenTelemetry SDK for Ruby, configuration options can be set via environment variables or programmatically.

To use an environment variable to print to the console, set the variable before calling configure:

require 'opentelemetry/sdk'
require 'opentelemetry/exporter/otlp'
require 'opentelemetry/instrumentation/all'

ENV['OTEL_TRACES_EXPORTER'] = 'console' # for debugging
OpenTelemetry::SDK.configure do |c|
  c.use_all()
end

Alternatively, add the ConsoleSpanExporter to your configuration:

 OpenTelemetry::SDK.configure do |c|
   c.add_span_processor(
     OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new(
       OpenTelemetry::SDK::Trace::Export::ConsoleSpanExporter.new
     )
   )
...
end

Keep in mind that printing to the console is not recommended for production and should only be used for debugging purposes.

No Traces for a Service 

The service name is a required configuration value. If it is unspecified, all trace data will be sent to a default dataset called unknown_service.

My Traces have Duplicate Spans 

If a This trace has multiple spans sharing the same non-null span ID error appears in Honeycomb, it is likely that your application is not instrumented correctly and is sending the same trace to Honeycomb more than once.

One possible misconfiguration is initializing OpenTelemetry more than once. Make sure to only initialize OpenTelemetry once when the application starts, and then use the Tracing API throughout the application runtime to add manual instrumentation.

Apple Mac M1 Protobuf Library Errors 

The OpenTelemetry libraries may produce an error on Apple Mac M1 computers.

/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- google/protobuf_c (LoadError)

Rubygems, by default, installs this gem with native extensions. Reinstallation of the gem without these extensions may remove this error.

First, uninstall the native version of the google-protobuf library.

gem uninstall google-protobuf

Then, install the google-protobuf library for the ruby platform.

gem install google-protobuf --platform=ruby

OTLP Protobuf Definitions 

Honeycomb supports receiving telemetry data via OpenTelemetry’s native protocol, OTLP, over gRPC, HTTP/protobuf, and HTTP/JSON. The minimum supported versions of OTLP protobuf definitions are 0.7.0 for traces and metrics.

If the protobuf version in use by the SDK does not match a supported version by Honeycomb, a different version of the SDK may need to be used. If the SDK’s protobuf version is older than the minimum supported version, and telemetry is not appearing as expected in Honeycomb, upgrade the SDK to a version with the supported protobuf definitions. If using an added dependency on a proto library, ensure the version of protobuf definitions matches the supported version of the SDK.

Receiving 464 Errors 

You may receive a 464 error response from the Honeycomb API when sending telemetry using gRPC and HTTP1. The gRPC format depends on using HTTP2 and any request over HTTP1 will be rejected by the Honeycomb servers.

Did you find what you were looking for?

Yes No

---

If you have questions, join our Pollinators Community Slack!
Pro/Enterprise users can visit Honeycomb Support or email support@honeycomb.io.

Submit