Beeline for Node.js Reference

Beelines have reached End of Life and are now archived.

Adding new instrumentation? We recommend that you use OpenTelemetry for Node.js instead.
Already using a Beeline? We recommend that you migrate to OpenTelemetry.
Concerned about mixing Beelines and OpenTelemetry? Learn more about Mixing Beelines and OpenTelemetry.

The Node.js Beeline provides instant, per-request visibility for your Express application. It automatically instruments many common Node.js packages with traces and events to capture useful information from your application. The Node.js Beeline allows you to slice and dice requests by endpoint, status, or even User ID, with zero custom instrumentation required. It creates traces automatically out of the box. While this is a great way to get general insights about your app as quickly as possible, as you forge ahead on your observability journey, you may find you would like to add new events or traces to add more details specific to your app. The Node.js Beeline provides simple interfaces for adding both. If you would like to see more options in the Node.js Beeline, please file an issue or vote up an already filed one! If you prefer more control over your application’s instrumentation, the Node.js Beeline has an API of its own for adding to traces.

Requirements

Node.js 10+
A Honeycomb API key

You can find your API key in your Environment Settings. If you do not have a API key yet, sign up for a free Honeycomb account.

Quick Installation

If using the dataset-only data model, refer to the Honeycomb Classic tab for instructions. Not sure? Learn more about Honeycomb versus Honeycomb Classic.

Honeycomb
Honeycomb Classic

If you have a NodeJS Express or Fastify app, you can get request-level instrumentation for those frameworks and other supported packages you use, automatically.

Install the Node.js Beeline package using npm:
```
npm install --save honeycomb-beeline
```

Add the following code to the top of your app.js. Important: It must be before any require or import statements for other packages.

require("honeycomb-beeline")({
  writeKey: "YOUR_API_KEY",
  // The name of your app/service is a good choice
  serviceName: "my-node-service"
});

If you have a NodeJS Express or Fastify app, you can get request-level instrumentation for those frameworks and other supported packages you use, automatically.

Install the Node.js Beeline package using npm:
```
npm install --save honeycomb-beeline
```

Add the following code to the top of your app.js. Important: It must be before any require or import statements for other packages.

require("honeycomb-beeline")({
  writeKey: "YOUR_API_KEY",
  // The name of your app is a good choice to start with
  dataset: "my-node-distributed-app",
  serviceName: "my-node-service"
});

Adding Context to Events

The packaged instrumentations send context to Honeycomb about requests and queries, but they can not automatically capture all context that you might want. Additional fields are an excellent way to add detail to your events. Try putting a timer around a section of code, add adding per-user information, or details about what it took to craft a response. You can add fields when and where you need to, or for some events but not others. (Error handlers are a good example of this.) Here is an example of adding a custom field to the currently-active span:

const beeline = require("honeycomb-beeline")();

const calcBigNum = num => {
  beeline.addContext({ oldNum: num });
  // ... do thing with num
  beeline.addContext({ newNum: num });
};

Note that beeline.addContext only adds fields to the current span. To add fields to all downstream spans, use beeline.addTraceContext instead:

const beeline = require("honeycomb-beeline")();

const handleInput = () => {
  beeline.addTraceContext({ userId: userId });
};

Note that beeline.addTraceContext will prefix your field name with app., so that all downstream spans will be populated with an app.userId field. These additional fields are your opportunity to add important and detailed context to your instrumentation. Put a timer around a section of code, add per-user information, include details about what it took to craft a response, and so on. It is expected that some fields will only be present on some requests. Error handlers are a great example of this; they will obviously only exist when an error has occurred. It is common practice to add in these fields along the way as they are processed in different levels of middleware. For example, if you have an authentication middleware, it would add a field with the authenticated user’s ID and name as soon as it resolves them. Later on in the call stack, you might add additional fields describing what the user is trying to achieve with this specific HTTP request. See the Adding Context to Spans section for more information on the full API available to you.

Adding Spans To a Trace Or Starting New Traces

We encourage people to think about instrumentation in terms of “units of work”. As your program grows, what constitutes a unit of work is likely to be portions of your overall service rather than an entire run. Spans are a way of breaking up a single external action (say, an HTTP request) into several smaller units in order to gain more insight into your service. Together, many spans make a trace, which you can visualize traces within the Honeycomb query builder. Express and Fastify instrumentations automatically start and finish traces for incoming requests. If you are using a different framework, choose to disable those automatic instrumentations. If you have code that runs outside the request handlers, you will need to manage the trace lifecycle manually. beeline.startTrace() starts a new local trace and initializes the async context propagation machinery. You must have an active trace for the tracing API to work correctly. If you call startSpan when you are not currently in a trace, an Error will be thrown. The instrumentations, which must operate in both trace/non-trace environments, handle this by checking beeline.traceActive() and only creating spans if they’re within a trace. This method also creates a root span for the trace (using beeline.startSpan below), and adds metadataContext as its initial context. This root span is installed as the current span.

You must call startTrace outside of an async function. Other beeline calls will work inside async functions.

Below is an example of starting a trace:

let trace = beeline.startTrace({
  field1: value1,
  field2: value2
});

To start a new span in the existing trace, call beeline.startSpan();. This returns a reference to the span which can then be used in finishSpan like this:

let span = beeline.startSpan({
  field1: value1,
  field2: value2
});
let value = calculateSomeValue();
beeline.addTraceContext({
  field3: value
})
beeline.finishSpan(span);

If you are doing something synchronously (for example, looping, or using a synchronous node API) you can use withSpan to wrap this operation. Since it returns the return value of fn, it can be used in an expression context. Here is an example:

let sum = beeline.withSpan(
  {
    task: "calculating the sum"
  },
  () => {
    let s = 0;
    for (let i of bigArray) {
      s += i;
    }
    return s;
  }
);

After you have added all the spans you need to your trace, call beeline.finishTrace(trace); to send the trace’s root span, and complete the necessary teardown. Below is an example of finishing a trace:

beeline.finishTrace(trace);

Some traces can be expressed as a single function, for example, if you are doing something synchronously (maybe in a script). In this case you can use, withTrace() as seen below:

beeline.withTrace(
  {
    task: "writing a file",
    filePath
  },
  () => fs.writeFileSync(filePath, fileContents)
);

// Another example of withTrace, in an expression context:
console.log(
  `the answer is ${beeline.withTrace(
    {
      task: "computing fibonacci number",
      n
    },
    () => computeFib(n)
  )}`
);

Asynchronous Spans

Each span in a trace, except the first (root) span, has a parent. Every span must finish before its parent, otherwise you will see incomplete or broken traces in Honeycomb. If you have asynchronous code, where a span might finish after its parent, use beeline.startAsyncSpan() to address this situation. This function takes a callback, which will be called with the new span. When using beeline.startAsyncSpan(), make sure to finish the new span within the callback. For example:

beeline.startAsyncSpan({
  task: "writing a file",
  filePath,
}, span => {
  fs.writeFile(filePath, fileContents, err => {
    if (err) {
      beeline.addTraceContext({ fileError: err.toString() });
    }
    beeline.finishSpan(span);
  });
});

When using async/await syntax, for example:

await beeline.startAsyncSpan({
  task: "writing a file",
  filePath,
}, async (span) => {
  try {
    await fs.writeFile(filePath, fileContents);
  } catch (e) {
    beeline.addTraceContext({ fileError: err.toString() });
  } finally {
    beeline.finishSpan(span);
  }
});

Instrumented Packages

The following is a list of packages for which we have added instrumentation. Some add context to events, while others propagate context so the Beeline can create events in downstream packages. The source for each individual instrumented package can be found in the lib/instrumentation folder on GitHub.

`bluebird`

Instrumented only for context propagation

`mpromise`

Instrumented only for context propagation

`express`

Adds columns with prefix request. Configuration Options:

Name	Type
`express.userContext`	Array<string>\|Function<(request) => Object>

express.userContext If the value of this option is an array, it is assumed to be an array of string field names of req.user. If a request has req.user, the named fields are extracted and added to events with column names of express.user.$fieldName. For example: If req.user is an object { id: 1, username: "toshok" } and your config settings include express: { userContext: ["username"] }, the following will be included in the express event sent to honeycomb:

`request.user.username`
`toshok`

If the value of this option is a function, it will be called on every request and passed the request as the sole argument. All key-values in the returned object will be added to the event. If the function returns a falsy value, no columns will be added. To replicate the above Array-based behavior, you could use the following configuration: express: { userContext: (req) => req.user && { username: req.user.username } } This function is not limited to using the request object, and can pull info from anywhere to enrich the data sent about the user.

`http`

Adds columns with prefix http.

`https`

Adds columns with prefix https.

`mysql2`

Adds columns with prefix db.

`pg`

Adds columns with prefix db.

`sequelize`

Instrumented only for context propagation

`mongoose`

Instrumented only for context propagation

`mongodb`

Adds columns with prefix db. Configuration options:

Name	Type
`mongodb.includeDocuments`	boolean

mongodb.includeDocuments If true, documents in the api will be JSON serialized and included in the events sent to honeycomb.

`react-dom/server`

Adds columns with prefix react.

`child_process`

Instrumented only for context propagation

Optional Configuration

The additional optional configuration in the code example above is where you can add global settings (Honeycomb credentials and service name) or per-instrumentation settings:

{
    writeKey:    "YOUR_API_KEY",
    serviceName: "your-service-name"
    $instrumentationName: {
        /* instrumentation specific settings */
    }
}

You may also specify writeKey and serviceName by setting HONEYCOMB_WRITEKEY and SERVICE_NAME in your environment. To add custom instrumentation settings, specify them in your configuration object as a key/value pair using the name of the instrumentation as the key. For example, to add configuration options for express, your configuration object might look like:

{
    writeKey: "YOUR_API_KEY",
    serviceName: "your-service-name"
    express: {
    /* express-specific settings */
    }
}

See the Instrumented packages section below for available configuration options for each package.

Customizing Instrumented Packages

If you want to disable automatic instrumentation for whatever reason, for either an individual package or all packages, you can pass enabledInstrumentations when configuring the Beeline. It should be an array of package names to automatically instrument. For instance, if you want to enable the beeline instrumentation only for the http and https packages:

require("honeycomb-beeline")({
  enabledInstrumentations: ["http", "https"]
  /* ... additional configuration ... */
});

The Beeline also exports getInstrumentations, which returns a list of all enabled instrumentation. You can use this to filter out specific instrumentations you want to disable. If you want to enable all instrumentation except mongodb:

const beeline = require("honeycomb-beeline");
beeline({
  enabledInstrumentations: beeline
    .getInstrumentations()
    .filter(i => i !== "mongodb")
  /* ... additional configuration ... */
});

Finally, to disable all automatic instrumentation, pass an empty array as in:

require("honeycomb-beeline")({
  enabledInstrumentations: []
  /* ... additional configuration ... */
});

Augmenting Or Scrubbing Spans

If you have some transformations you would like to make on every span before it leaves the process for Honeycomb, the presendHook is your opportunity to make these changes. Examples are scrubbing sensitive data (for example, you may want to ensure specific fields are dropped or hashed) or augmenting data (like making out-of-band translations between an ID and a more human readable version). Similar to the samplerHook discussed below, you pass the presendHook a function that will be called on every span with the span as an argument. The function is free to mutate the span passed in and those mutations will be what finally gets sent to Honeycomb. As an example, say we have some HTTP requests that provide sensitive values which have been added to a span. This code will examine all spans just before they are sent to Honeycomb and remove the sensitive values.

const beeline = require("honeycomb-beeline");
beeline({
  presendHook: function(ev) {
    // either:
    ev.data.scrubMe = undefined;
    // or:
    // delete ev.data.scrubMe
  }
});
beeline.finishTrace(beeline.startTrace({ name: "foo", scrubMe: "sensitive data" }));

Proxy Configuration

If the environment variables HTTPS_PROXY or https_proxy are set, the Beeline will pick up and configure itself to use the proxy for all event traffic to Honeycomb.

Sampling Events

We have built-in support for sampling in the Beeline. Simply set the sampleRate variable to your beeline configuration. This sends 1/n of all events, so a sample rate of 5 would send 20% of your events:

require("honeycomb-beeline")({
  writeKey:    "YOUR_API_KEY",
  serviceName: "your-service-name",
  sampleRate: 5 // This will send 1/5 (20%) of events
  /* ... additional optional configuration ... */
});

The value of sampleRate must be a positive integer. Sampling by default will apply the same sampling decision to all spans in a trace, so adding sampling will not break your traces. Either all spans in a trace will be sent, or no spans in the trace will be sent. If sampling across multiple Beeline-instrumented services, set the same sample rate in all beeline configurations to avoid breaking traces.

Customizing Sampling Logic

The samplerHook configuration option is available to customize the logic used for deterministic sampling. To avoid breaking traces with the custom samplerHook option, ensure that sampling logic is applied to data that all spans within a trace will have (such as trace.trace_id). If needed, you may promote span fields to trace-level (making them available on all spans within a trace) by calling addTraceContext() at the beginning of a trace, for instance: beeline.addTraceContext({ request_route: '/x/alive' }). A custom samplerHook must return an object with this structure:

{
  shouldSample: boolean, // false will drop the event, true will keep it
  sampleRate: number // optional, will be reported to honeycomb
}

For example, assume you have instrumented an HTTP server. You would like to keep all requests to login, skip all health checks, and sample the rest at a default rate. You could define a sampler function like so:

const beeline = require("honeycomb-beeline");
const createHash = require("crypto").createHash;

beeline({
    writeKey:  "YOUR_API_KEY",
  serviceName: "your-service-name",
    samplerHook: samplerHook,
});

// deterministicSampler function based on https://github.com/honeycombio/beeline-nodejs/blob/main/lib/deterministic_sampler.js
function deterministicSampler(traceId, sampleRate) {
  const MAX_UINT32 = Math.pow(2, 32) - 1;
  const sum = createHash("sha1")
      .update(traceId)
      .digest();
  const upperBound = (MAX_UINT32 / sampleRate) >>> 0;
  return sum.readUInt32BE(0) <= upperBound;
}

function samplerHook(data) {
  // default sample rate to 10
  let sampleRate = 10;

  // default sampling decision to deterministic based on trace_id
  let shouldSample = deterministicSampler(data["trace.trace_id"], sampleRate);

  // drop all health checks requests
  if (data["request.path"] === "/x/alive") {
      shouldSample = false;
      sampleRate = 0;
  }
  return {
      shouldSample,
      sampleRate
  }
}

app.get('/x/alive', function(req, res) {
  res.send("I'm alive");
});

Defining a sampling hook overrides the default deterministic sampling behavior for trace IDs. For head-based sampling behavior across a more complicated trace than the health check example above, you must ensure sampling decisions are based on a value all spans will have (such as trace_id or a custom trace context as we did above). Otherwise, you will get incomplete traces.

Distributed Trace Propagation

When a service calls another service, you want to ensure that the relevant trace information is propagated from one service to the other. This allows Honeycomb to connect the two services in a trace. Distributed tracing enables you to trace and visualize interactions between multiple instrumented services. For example, your users may interact with a front-end API service, which talks to two internal APIs to fulfill their request. In order to have traces connect spans for all these services, it is necessary to propagate trace context between these services, usually by using an HTTP header. Both the sending and receiving service must use the same propagation format, and both services must be configured to send data to the same Honeycomb environment. Automatic instrumentation supports trace propagation automatically, as long as your services are using the Honeycomb beeline, and an instrumented component to send and receive requests (express and https).

Interoperability With OpenTelemetry

Trace context propagation with OpenTelemetry is done by sending and parsing headers that conform to the W3C Trace Context specification. To get Beelines and OpenTelemetry instrumentation to interoperate, you will need to use W3C headers. The Beeline includes marshal and unmarshal functions that can generate and parse W3C Trace Context headers. Honeycomb Beelines default to using a Honeycomb-specific header format on outgoing requests, but can automatically detect incoming W3C headers and parse them appropriately. In mixed environments where some services are using OpenTelemetry and some are using Beeline, W3C header propagation should be used. To propagate trace context, a parser hook and propagation hook are needed. The parser hook is responsible for reading the trace propagation context out of incoming HTTP requests from upstream services. The propagation hook is responsible for returning the set of headers to add to outbound HTTP requests to propagate the trace propagation context to downstream services.

Older versions of Honeycomb Beelines required HTTP parsing hooks to properly parse incoming W3C headers. Current versions of Honeycomb Beelines can automatically detect incoming W3C headers and parse them appropriately. Check the release notes for your Beeline version to confirm whether an upgraded version is needed.

To specify that a service should propagate W3C Trace Context Headers with outgoing requests, you must specify a propagation hook in the beeline configuration. An httpTraceParserHook is a function that takes an HTTP request as an argument and returns a Honeycomb trace context object. The HTTP request is provided to the function so that the author can make decisions about whether to trust the incoming headers based on information contained in the request (for example, perhaps you do not want to accept headers from the public internet). An httpTracePropagationHook is a function that takes a Honeycomb trace context object as an argument and returns a map of name, value pairs representing serialized headers. This example adds these hooks to parse and propagate W3C headers:

const beeline = require('honeycomb-beeline');

beeline({
    writeKey:    'YOUR_API_KEY',
    serviceName: 'your-service-name',
    httpTraceParserHook: beeline.w3c.httpTraceParserHook,
    httpTracePropagationHook: beeline.w3c.httpTracePropagationHook
});

Troubleshooting The Beeline

No Traces for a Service

The service name is a required configuration value. If it is unspecified, all trace data will be sent to a default dataset called unknown_service.

The Events I Am Generating Do Not Contain The Content I Expect

Enable debug output (sent to STDOUT) by setting DEBUG=honeycomb-beeline:* in your environment. This will print the JSON representation of events to STDOUT as well as other debug level details and errors. This lets you quickly see what is getting sent and allows you to modify your code accordingly.

DEBUG=honeycomb-beeline:*

My Traces Are Showing Missing Root Spans

There can be a number of reasons for missing root spans. One potential reason could be that there is an upstream service, load balancer, or other proxy propagating W3C trace headers as part of your distributed trace. Since beelines accept both Honeycomb and W3C headers, that service propagating a W3C header will cause “missing span” gaps in your trace if the service is not also configured to send telemetry to Honeycomb. The solution is to either instrument that service and configure it to send telemetry to Honeycomb, or to specify in the downstream service’s beeline configuration that only Honeycomb propagation headers should be parsed. To override undesired W3C trace header propagation behavior, configure the Beeline to use an httpTraceParserHook:

const beeline = require("honeycomb-beeline");

beeline({
    writeKey:    "YOUR_API_KEY",
    serviceName: "your-service-name",
    httpTraceParserHook: beeline.honeycomb.httpTraceParserHook,
});

The above configuration will solely use the Honeycomb format when parsing incoming trace headers. See Distributed Trace Propagation for more details.

Example Event

Here is a sample event created by the Node.js Beeline:

{
  "Timestamp": "2018-03-20T00:47:25.339Z",
  "request.base_url": "",
  "request.fresh": false,
  "request.host": "localhost",
  "request.http_version": "HTTP/1.1",
  "request.remote_addr": "127.0.0.1",
  "request.method": "POST",
  "request.original_url": "/checkValid",
  "request.path": "/checkValid",
  "request.scheme": "http",
  "request.query": "{}",
  "request.secure": false,
  "request.url": "/checkValid",
  "request.xhr": true,
  "response.status_code": "200",
  "meta.instrumentation_count": 4,
  "meta.instrumentations": "[\"child_process\",\"express\",\"http\",\"https\"]",
  "meta.type": "express",
  "meta.version": "4.16.3",
  "meta.beeline_version": "1.0.2",
  "meta.node_version": "v9.10.0",
  "totals.mysql2.count": 2,
  "totals.mysql2.duration_ms": 13.291,
  "totals.mysql2.query.count": 2,
  "totals.mysql2.query.duration_ms": 13.291,
  "trace.trace_id": "11ad83a2-ca8d-4918-9db2-27524456d9f7",
  "trace.span_id": "4a3892ba-0936-46e1-8e17-31b887326027",
  "name": "request",
  "service_name": "express",
  "duration_ms": 15.229326
}

Queries To Try

Try these examples to get started querying your app’s behavior.

Which Endpoints Are The Slowest?

Group By url
Visualize the P99 of duration_ms values
Where meta.type == express
Order by P99(duration_ms) in descending (DESC) order

Where Is My App Spending The Most Time?

Group By meta.type
Visualize the P99 of duration_ms values
Order by P99(duration_ms) in descending (DESC) order

Which Users Are Using The Endpoint I Would Like To Deprecate?

Group By user.email
Visualize the overall COUNT
Where url == your endpoint url

Which XHR Endpoints Take The Longest?

Group By url
Visualize the P99 of duration_ms values
Where meta.type == express and xhr == true
Order by P99(duration_ms) in descending (DESC) order

Contributions

Bug fixes and other changes to Beelines are gladly accepted. Please open issues or a pull request with your change via GitHub. Remember to add your name to the CONTRIBUTORS file! All contributions will be released under the Apache License 2.0.

​Requirements

​Quick Installation

​Adding Context to Events

​Adding Spans To a Trace Or Starting New Traces

​Asynchronous Spans

​Instrumented Packages

​bluebird

​mpromise

​express

​http

​https

​mysql2

​pg

​sequelize

​mongoose

​mongodb

​react-dom/server

​child_process

​Optional Configuration

​Customizing Instrumented Packages

​Augmenting Or Scrubbing Spans

​Proxy Configuration

​Sampling Events

​Customizing Sampling Logic

​Distributed Trace Propagation

​Interoperability With OpenTelemetry

​Troubleshooting The Beeline

​No Traces for a Service

​The Events I Am Generating Do Not Contain The Content I Expect

​My Traces Are Showing Missing Root Spans

​Example Event

​Queries To Try

​Which Endpoints Are The Slowest?

​Where Is My App Spending The Most Time?

​Which Users Are Using The Endpoint I Would Like To Deprecate?

​Which XHR Endpoints Take The Longest?

​Contributions

Requirements

Quick Installation

Adding Context to Events

Adding Spans To a Trace Or Starting New Traces

Asynchronous Spans

Instrumented Packages

`bluebird`

`mpromise`

`express`

`http`

`https`

`mysql2`

`pg`

`sequelize`

`mongoose`

`mongodb`

`react-dom/server`

`child_process`

Optional Configuration

Customizing Instrumented Packages

Augmenting Or Scrubbing Spans

Proxy Configuration

Sampling Events

Customizing Sampling Logic

Distributed Trace Propagation

Interoperability With OpenTelemetry

Troubleshooting The Beeline

No Traces for a Service

The Events I Am Generating Do Not Contain The Content I Expect

My Traces Are Showing Missing Root Spans

Example Event

Queries To Try

Which Endpoints Are The Slowest?

Where Is My App Spending The Most Time?

Which Users Are Using The Endpoint I Would Like To Deprecate?

Which XHR Endpoints Take The Longest?

Contributions