Send Data with Honeycomb Distribution for Node.js

Warning

While the Honeycomb distributions of the OpenTelemetry SDKs are not yet deprecated, they are in maintenance.

Honeycomb provides the Honeycomb OpenTelemetry Distribution for Node.js to help you instrument your applications and send telemetry data to Honeycomb as quickly and easily as possible. Under the hood, the Honeycomb Distribution uses OpenTelemetry for JavaScript, so advanced users or those who have already instrumented their applications with OpenTelemetry do not need to use this Distribution.

The Honeycomb Distribution reads variables you provide and translates them to variables understood by the upstream OpenTelemetry SDK. For example, the Honeycomb Distribution automatically configures exporters to send telemetry data to Honeycomb’s API (if you are using our US instance, api.honeycomb.io, or if you are using our EU instance, api.eu1.honeycomb.io). If you want to send data to Honeycomb using OpenTelemetry without the Honeycomb Distribution, you will need to configure your implementation to match variables expected by OpenTelemetry.

In this guide, we explain how to set up automatic and custom, or manual, instrumentation for a service written in Node.js. If you prefer learning by example, we provide several examples of applications configured to send OpenTelemetry data to Honeycomb using the Honeycomb OpenTelemetry Distribution for Node.js.

Before You Begin 

Before you can set up instrumentation for your Node.js application, you will need to do a few things.

Prepare Your Development Environment 

To complete the required steps, you will need:

  • A working Node.js environment with Node.js 14 or higher
  • An application written in Node.js

Get Your Honeycomb API Key 

To send data to Honeycomb, you’ll need to sign up for a free Honeycomb account and create a Honeycomb Ingest API Key. To get started, you can create a key that you expect to swap out when you deploy to production. Name it something helpful, perhaps noting that it’s a getting started key. Make note of your API key; for security reasons, you will not be able to see the key again, and you will need it later!

Tip
For setup, make sure you check the “Can create datasets” checkbox so that your data will show up in Honeycomb. Later, when you replace this key with a permanent one, you can uncheck that box.

If you want to use an API key you previously stored in a secure location, you can also look up details for Honeycomb API Keys any time in your Environment Settings, and use them to retrieve keys from your storage location.

Add Automatic Instrumentation 

Automatic instrumentation is enabled by adding instrumentation packages. Add custom, or manual, instrumentation using the OpenTelemetry API.

Acquire Dependencies 

Open your terminal, navigate to the location of your project on your drive, and install OpenTelemetry’s automatic instrumentation meta package and Honeycomb’s OpenTelemetry Node.js distribution package:

npm install --save \
    @opentelemetry/auto-instrumentations-node \
    @honeycombio/opentelemetry-node
Module Description
auto-instrumentations-node OpenTelemetry’s meta package that provides a way to add automatic instrumentation to any Node application to capture telemetry data from a number of popular libraries and frameworks, like express, dns, http, and more.
opentelemetry-node Honeycomb’s Node.js distribution package that streamlines configuration and allows you to instrument as quickly and easily as possible.

Alternatively, install individual instrumentation packages.

Open your terminal, navigate to the location of your project on your drive, and install OpenTelemetry’s automatic instrumentation meta package and Honeycomb’s OpenTelemetry Node.js distribution package:

yarn add \
    @opentelemetry/auto-instrumentations-node \
    @honeycombio/opentelemetry-node
Module Description
auto-instrumentations-node OpenTelemetry’s meta package that provides a way to add automatic instrumentation to any Node application to capture telemetry data from a number of popular libraries and frameworks, like express, dns, http, and more.
opentelemetry-node Honeycomb’s Node.js distribution package that streamlines configuration and allows you to instrument as quickly and easily as possible.

Alternatively, install individual instrumentation packages.

If using TypeScript, install ts-node to run the code:

yarn add --dev ts-node

Initialize 

Create an initialization file, commonly known as the tracing.ts file:

// Example filename: tracing.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { HoneycombSDK } from '@honeycombio/opentelemetry-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

// Uses environment variables named HONEYCOMB_API_KEY and OTEL_SERVICE_NAME
const sdk: NodeSDK = new HoneycombSDK({
  instrumentations: [    
    getNodeAutoInstrumentations({
      // We recommend disabling fs automatic instrumentation because 
      // it can be noisy and expensive during startup
      '@opentelemetry/instrumentation-fs': {
        enabled: false,
      },
    }),
  ],
});

sdk.start();

Create an initialization file, commonly known as the tracing.js file:

// Example filename: tracing.js
'use strict';

const { HoneycombSDK } = require('@honeycombio/opentelemetry-node');
const {
  getNodeAutoInstrumentations,
} = require('@opentelemetry/auto-instrumentations-node');

// Uses environment variables named HONEYCOMB_API_KEY and OTEL_SERVICE_NAME
const sdk = new HoneycombSDK({
  instrumentations: [getNodeAutoInstrumentations({
      // We recommend disabling fs automatic instrumentation because
      // it can be noisy and expensive during startup
      '@opentelemetry/instrumentation-fs': {
        enabled: false,
      },
  })]
});

sdk.start()

Configure 

Use environment variables to configure the Honeycomb OpenTelemetry Node.js distribution package:

export HONEYCOMB_API_ENDPOINT="https://api.honeycomb.io:443" # US instance
#export HONEYCOMB_API_ENDPOINT="https://api.eu1.honeycomb.io:443" # EU instance
export OTEL_SERVICE_NAME="your-service-name"
export HONEYCOMB_API_KEY="your-api-key"
Variable Description
HONEYCOMB_API_ENDPOINT Honeycomb endpoint to which you want to send your data.
OTEL_SERVICE_NAME Service name. When you send data, Honeycomb creates a dataset in which to store your data and uses this as the name. Can be any string.
HONEYCOMB_API_KEY Your API Key generated in Honeycomb. Learn how to find your Honeycomb API Key.
Note

If you use Honeycomb Classic, you must also specify the Dataset using the x-honeycomb-dataset header.

export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key,x-honeycomb-dataset=your-dataset"
Note
If you are sending data directly to Honeycomb, you must configure the API key and service name. If you are using an OpenTelemetry Collector, configure your API key at the Collector level instead.

To set configuration in code, instead of environment variables, add to the HoneycombSDK:

const sdk = new HoneycombSDK({
  apiKey: "your-api-key",
  serviceName: "your-service-name",
  instrumentations: [getNodeAutoInstrumentations()]
})

Advanced Configuration 

Explore all configuration options for the Honeycomb Distribution:

Environment Variable Default Value Description
HONEYCOMB_API_KEY None [required – see note below] Your Honeycomb API key
OTEL_SERVICE_NAME unknown_service [required – see note below] service.name attribute, where all trace data is sent
HONEYCOMB_TRACES_APIKEY Value of HONEYCOMB_API_KEY Your Honeycomb API key for sending traces
HONEYCOMB_METRICS_APIKEY Value of HONEYCOMB_API_KEY Your Honeycomb API key for sending metrics
HONEYCOMB_METRICS_DATASET None Honeycomb dataset where metrics will be sent
HONEYCOMB_API_ENDPOINT api.honeycomb.io:443 (US instance)
api.eu1.honeycomb.io:443 (EU instance)
Honeycomb ingest endpoint
HONEYCOMB_TRACES_ENDPOINT Value of HONEYCOMB_API_ENDPOINT Honeycomb ingest endpoint for traces (defaults to the value of HONEYCOMB_API_ENDPOINT)
HONEYCOMB_METRICS_ENDPOINT Value of HONEYCOMB_API_ENDPOINT Honeycomb ingest endpoint for metrics (defaults to the value of HONEYCOMB_API_ENDPOINT)
SAMPLE_RATE 1 (retain all data) Sample rate for the deterministic sampler. Must be a positive integer.
OTEL_METRICS_ENABLED false Enable metrics export (metrics dataset must be configured as well)
HONEYCOMB_ENABLE_LOCAL_VISUALIZATIONS false Enable local visualizations
DEBUG false Enable debug mode
Note
If you are sending data directly to Honeycomb, you must configure the API key and service name. If you are using an OpenTelemetry Collector, configure your API key at the Collector level instead.

Run 

Run the Node.js app and include the initialization file you created:

ts-node -r ./tracing.ts YOUR_APPLICATION_NAME.ts

Be sure to replace YOUR_APPLICATION_NAME with the name of your application’s main file.

Alternatively, you can import the initialization file as the first step in your application lifecycle.

In Honeycomb’s UI, you should now see your application’s incoming requests and outgoing HTTP calls generate traces.

Run the Node.js app and include the initialization file you created:

node -r ./tracing.js YOUR_APPLICATION_NAME.js

Be sure to replace YOUR_APPLICATION_NAME with the name of your application’s main file.

Alternatively, you can import the initialization file as the first step in your application lifecycle.

In Honeycomb’s UI, you should now see your application’s incoming requests and outgoing HTTP calls generate traces.

Add Custom Instrumentation 

Automatic instrumentation is the easiest way to get started with instrumenting your code. To get additional insight into your system, you should also add custom, or manual, instrumentation where appropriate.

To learn more about custom, or manual, instrumentation, visit the comprehensive set of topics covered by Manual Instrumentation for JavaScript in OpenTelemetry’s documentation.

Acquire Dependencies 

To start adding custom instrumentation, ensure that the OpenTelemetry API package exists as a direct dependency in your project. This package provides access to the high-level instrumentation APIs, which gives the ability to retrieve the current span to enrich with additional attributes, to create new spans, and to generate metrics.

npm install --save @opentelemetry/api
yarn add @opentelemetry/api

Add Attributes to Spans 

Adding attributes to a currently executing span in a trace can be useful. For example, you may have an application or service that handles users, and you want to associate the user with the span when querying your service in Honeycomb. To do this, get the current span from the context and set an attribute with the user ID:

import opentelemetry from '@opentelemetry/api';

function handleUser(user: User) {
  let activeSpan = opentelemetry.trace.getActiveSpan();
  activeSpan.setAttribute("user.id", user.getId());
}
const opentelemetry = require("@opentelemetry/api");

function handleUser(user) {
  let activeSpan = opentelemetry.trace.getActiveSpan();
  activeSpan.setAttribute("user.id", user.getId());
}

This will add a user.id field to the current span, so you can use the field in WHERE, GROUP BY or ORDER clauses in the Honeycomb query builder.

Initialize a Tracer 

To create spans, you need to initialize a Tracer.

import opentelemetry from '@opentelemetry/api';

const tracer = opentelemetry.trace.getTracer("tracer.name.here");
const opentelemetry = require("@opentelemetry/api");

const tracer = opentelemetry.trace.getTracer("tracer.name.here");

When you create a Tracer, OpenTelemetry requires you to give it a name as a string. This string is the only required parameter.

When traces are sent to Honeycomb, the name of the Tracer is turned into the library.name field, which can be used to show all spans created from a particular tracer.

In general, pick a name that matches the appropriate scope for your traces. If you have one tracer for each service, then use the service name. If you have multiple tracers that live in different “layers” of your application, then use the name that corresponds to that “layer”.

The library.name field is also used with traces created from instrumentation libraries.

You can then use this tracer to create custom spans.

Create New Spans 

Automatic instrumentation can show the shape of requests to your system, but only you know the truly important parts. To get the full picture of what is happening, you must add custom, or manual, instrumentation and create some custom spans. To do this, grab the tracer from the OpenTelemetry API:

import opentelemetry from '@opentelemetry/api';

const tracer = opentelemetry.trace.getTracer("my-service-tracer");

function runQuery() {
  tracer.startActiveSpan("expensive-query", (span) => {
    // ... do cool stuff
    span.end();
  });
}
const opentelemetry = require("@opentelemetry/api");

const tracer = opentelemetry.trace.getTracer("my-service-tracer");

function runQuery() {
  tracer.startActiveSpan("expensive-query", (span) => {
    // ... do cool stuff
    span.end();
  });
}

Add Multi-Span Attributes 

Sometimes you want to add the same attribute to many spans within the same trace. This attribute may include variables calculated during your program, or other useful values for correlation or debugging purposes.

To add this attribute, leverage the OpenTelemetry concept of baggage. Baggage allows you to add a key with a value as an attribute to every subsequent child span within the current application context.

import {
  Context,
  context,
  propagation,
} from '@opentelemetry/api';

tracer.startActiveSpan('main', (span) => {
  span.setAttribute('key', 'value'); // add to current span

  // new context based on current, with key/values added to baggage
  const ctx: Context = propagation.setBaggage(
    context.active(),
    propagation.createBaggage({ 'key': { value: 'value' } })
  );

  // within the new context, do some work and baggage will be
  // applied as attributes on child spans
  context.with(ctx, () => {
    tracer.startActiveSpan('childSpan', (childSpan) => {
      doTheWork();
      childSpan.end();
    });
  });

  span.end();
});
tracer.startActiveSpan('main', (span) => {
  span.setAttribute('key', 'value'); // add to current span

  // new context based on current, with key/values added to baggage
  const ctx = propagation.setBaggage(
    context.active(),
    propagation.createBaggage({ 'key': { value: 'value' } })
  );

  // within the new context, do some work and baggage will be
  // applied as attributes on child spans
  context.with(ctx, () => {
    tracer.startActiveSpan('childSpan', (childSpan) => {
      doTheWork();
      childSpan.end();
    });
  });

  span.end();
});

Note: Any Baggage attributes that you set in your application will be attached to outgoing network requests as a header. If your service communicates to a third party API, do NOT put sensitive information in the Baggage attributes.

Sending Data to a Collector or Other Endpoint 

You can send data to an OpenTelemetry Collector instance or another endpoint by specifying a different endpoint:

export HONEYCOMB_API_ENDPOINT="http://<your-collector-endpoint>"

Alternatively, if you prefer to configure this in code, you can configure your endpoint with the HoneycombSDK:

const sdk = new HoneycombSDK({
  apiKey: "your-api-key",
  serviceName: "your-service-name",
  instrumentations: [getNodeAutoInstrumentations()],
  endpoint: "http://<your-collector-endpoint>",
})

The Honeycomb SDK will automatically append the endpoint with the appropriate traces-specific path of v1/traces.

Sampling 

Deterministic head sampling can be used with the Honeycomb SDK, with or without manual instrumentation.

The SDK will read these variables and expect an integer that represents the sample rate you would like to apply. For example, a value of 5 means that one out of every five traces will be sent to Honeycomb.

To add sampling to the SDK, set the SAMPLE_RATE environment variable:

export SAMPLE_RATE=5

To set the sample rate in code instead of an environment variable, add sampleRate to the HoneycombSDK:

const sdk = new HoneycombSDK({
  apiKey: "your-api-key",
  serviceName: "your-service-name",
  instrumentations: [getNodeAutoInstrumentations()],
  sampleRate: 5,
})

The value of your sample rate must be a positive integer.

If you have multiple services that communicate with each other, it is important that they have the same sampling configuration. Otherwise, each service might make a different sampling decision, resulting in incomplete or broken traces. You can sample using a standalone proxy as an alternative, like Honeycomb Refinery, or when you have more robust sampling needs.

Distributed Trace Propagation 

When a service calls another service, you want to ensure that the relevant trace information is propagated from one service to the other. This allows Honeycomb to connect the two services in a trace.

Distributed tracing enables you to trace and visualize interactions between multiple instrumented services. For example, your users may interact with a front-end API service, which talks to two internal APIs to fulfill their request. In order to have traces connect spans for all these services, it is necessary to propagate trace context between these services, usually by using an HTTP header.

Both the sending and receiving service must use the same propagation format, and both services must be configured to send data to the same Honeycomb environment.

Note: Any Baggage attributes that you set in your application will be attached to outgoing network requests as a header. If your service communicates to a third party API, do NOT put sensitive information in the Baggage attributes.

Trace context propagation is done by sending and parsing headers that conform to the W3C Trace Context specification.

By default, the Honeycomb’s OpenTelemetry Distribution for Node.js uses the W3C trace context format.

If you opt to use a different trace context specification than W3C, ensure that both the sending and receiving service are using the same propagation format, and that both services are configured to send data to the same Honeycomb environment.

Visualizing Traces Locally 

Honeycomb’s OpenTelemetry Distribution for Node.js can create a link to a trace visualization in the Honeycomb UI for local traces. Local visualizations enables a faster feedback cycle when adding, modifying, or verifying instrumentation.

To enable local visualizations:

  1. Set the HONEYCOMB_ENABLE_LOCAL_VISUALIZATIONS environment variable to true:

    export HONEYCOMB_ENABLE_LOCAL_VISUALIZATIONS=true
    
  2. Run your application:

    node -r ./tracing.js APPLICATION_MAIN_FILE.js
    
    Note

    To set the local visualizations option in code instead of an environment variable, add localVisualizations to the HoneycombSDK:

    const sdk = new HoneycombSDK({
      apiKey: "your-api-key",
      serviceName: "your-service-name",
      instrumentations: [getNodeAutoInstrumentations()],
      localVisualizations: true,
    })
    

    The output displays the name of the root span and a link to Honeycomb that shows its trace. For example:

    Trace for root-span-name
    Honeycomb link: <link to Honeycomb trace>
    
  3. Select the link to view the trace in detail within the Honeycomb UI.

Note
In production, disable local visualization. Local visualization creates additional overhead to create the link to a trace in Honeycomb and print it to the console.

Using gRPC instead of HTTP/protobuf 

By default, the Honeycomb OpenTelemetry SDK uses the HTTP protocol (http/protobuf) to send telemetry data. To use gRPC instead of HTTP/protobuf, update the protocol to grpc:

export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
Tip
If using the signal-specific OTEL_EXPORTER_OTLP_<SIGNAL>_ENDPOINT environment variable, you must append the endpoint with the appropriate signal path. For example, if sending traces, append the endpoint with v1/traces. If sending metrics, append the endpoint with v1/metrics.

Alternatively, if you prefer to configure this protocol in code, you can configure it with the HoneycombSDK:

const sdk = new HoneycombSDK({
  apiKey: "your-api-key",
  serviceName: "your-service-name",
  instrumentations: [getNodeAutoInstrumentations()],
  protocol: OtlpProtocolKind.Grpc,
})
const sdk = new HoneycombSDK({
  apiKey: "your-api-key",
  serviceName: "your-service-name",
  instrumentations: [getNodeAutoInstrumentations()],
  protocol: "grpc",
})

Troubleshooting 

To explore common issues when sending data, visit Common Issues with Sending Data in Honeycomb.