We use cookies or similar technologies to personalize your online experience and tailor marketing to you. Many of our product features require cookies to function properly. Your use of this site and online product constitutes your consent to these personalization technologies. Read our Privacy Policy to find out more.

X

Beeline for Node.js

The Node.js Beeline provides instant, per-request visibility for your Express application. It automatically instruments many common Node.js packages to capture useful information from your application.

The Node.js Beeline allows you to slice and dice requests by endpoint, status, or even User ID, with zero custom instrumentation required. It also supports tracing out of the box. While this is a great way to get general insights about your app as quickly as possible, as you forge ahead on your observability journey, you may find you’d like to add new events or traces to add more details specific to your app. The Node.js Beeline provides simple interfaces for adding both.

If you’d like to see more options in the Node.js Beeline, please file an issue or vote up one already filed! You can also contact us at support@honeycomb.io.

If you prefer more control over your application’s instrumentation, the Node.js Beeline has an API of its own for adding to traces.

Requirements

You can find your API key on your Team Settings page. If you don’t have a API key yet, sign up for a Honeycomb trial.

Quick installation

If you’ve got a NodeJS express app, you can get request-level instrumentation of Express and other packages you use, magically.

  1. Install the Node.js Beeline package using npm:
   npm install --save honeycomb-beeline
  1. Add the following code to the top of your app.js.

Important: It must be before any require or import statements for other packages.

   require("honeycomb-beeline")({
     writeKey: "YOUR_API_KEY",
     dataset: "my-node-distributed-app",
     serviceName: "my-node-service",
     /* ... additional optional configuration ... */
   });

You are currently logged in to the team, so we have populated the write key here to the first write key for that team.

Adding context to events

The packaged instrumentations send context to Honeycomb about requests and queries, but they can’t automatically capture all context that you might want. Additional fields are an excellent way to add detail to your events. Try putting a timer around a section of code, add adding per- user information, or details about what it took to craft a response. You can add fields when and where you need to, or for some events but not others. (Error handlers are a good example of this.) To add additional fields, use the customContext interface.

Below is an example of adding a custom field. It adds a new field app.extra with value val:

var honeycombBeeline = require("honeycomb-beeline")();

/* your custom code */

honeycombBeeline.customContext.add("extra", val);

These additional fields are your opportunity to add important and detailed context to your instrumentation. Put a timer around a section of code, add per- user information, include details about what it took to craft a response, and so on. It is expected that some fields will only be present on some requests. Error handlers are a great example of this; they will obviously only exist when an error has occurred.

It is common practice to add in these fields along the way as they are processed in different levels of middleware. For example, if you have an authentication middleware, it would add a field with the authenticated user’s ID and name as soon as it resolves them. Later on in the call stack, you might add additional fields describing what the user is trying to achieve with this specific HTTP request.

You can add custom context several ways, including with callbacks or promises. See Custom Context Propagation for more detail on these methods.

Adding spans to a trace or starting new traces

We encourage people to think about instrumentation in terms of “units of work”. As your program grows, what constitutes a unit of work is likely to be portions of your overall service rather than an entire run. Spans are a way of breaking up a single external action (say, an HTTP request) into several smaller units in order to gain more insight into your service. Together, many spans make a trace, which you can visualize traces within the Honeycomb query builder.

If you’re using Express, you may not ever have to use any of the trace API.

You can read more about Traces and Spans in the Node.JS Github Repo

beeline.startTrace() starts a new local trace and initializes the async context propagation machinery. You must have an active trace for the tracing API to work correctly. If you call startSpan when you aren’t currently in a trace, an Error will be thrown. The instrumentations (which must operate in both trace/non-trace environments) handle this by checking beeline.traceActive() and only creating spans if they’re within a trace.

This method also creates a root span for the trace (using beeline.startSpan below), and adds metadataContext as its initial context. This root span is installed as the current span.

Below is an example of starting a trace:

let trace = beeline.startTrace({
  field1: value1,
  field2: value2
});

To start a new span in the existing trace, call beeline.startSpan();. This returns a reference to the span which can then be used in finishSpan like this:

let span = beeline.startSpan({
  task: "writing a file",
  filePath
});
fs.writeFile(filePath, fileContents, err => {
  beeline.customContext.add("fileError", err.toString());
  beeline.finishSpan(span);
});

If you’re doing something synchronously (for example, looping, or using a synchronous node api) you can use withSpan to wrap this operation. Since it returns the return value of fn, it can be used in an expression context.

Here’s an example:

let sum = beeline.withSpan(
  {
    task: "calculating the sum"
  },
  () => {
    let s = 0;
    for (let i of bigArray) {
      s += i;
    }
    return s;
  }
);

After you’ve added all the spans you need to your trace, call beeline.finishTrace(trace); to send the trace’s root span, and complete the necessary teardown. Below is an example of finishing a trace:

beeline.finishTrace(trace);

Some traces can be expressed as a single function, for example, if you’re doing something synchronously (maybe in a script). In this case you can use, withTrace() as seen below:

beeline.withTrace(
  {
    task: "writing a file",
    filePath
  },
  () => fs.writeFileSync(filePath, fileContents)
);

// Another example of withTrace, in an expression context:
console.log(
  `the answer is ${beeline.withTrace(
    {
      task: "computing fibonacci number",
      n
    },
    () => computeFib(n)
  )}`
);

Instrumented packages

The following is a list of packages we’ve added instrumentation for. Some add context to events, while others propagate context so the Beeline can create events in downstream packages.

The source for each individual instrumented package can be found in the lib/instrumentation folder on GitHub.

(if you’d like to see anything more here, please file an issue or 👍 one already filed!)

bluebird

Instrumented only for context propagation

mpromise

Instrumented only for context propagation

express

Adds columns with prefix request.

Configuration Options:

Name Type
express.userContext Array<string>|Function<(request) => Object>

express.userContext

If the value of this option is an array, it’s assumed to be an array of string field names of req.user. If a request has req.user, the named fields are extracted and added to events with column names of express.user.$fieldName.

For example:

If req.user is an object { id: 1, username: "toshok" } and your config settings include express: { userContext: ["username"] }, the following will be included in the express event sent to honeycomb:

request.user.username
toshok

If the value of this option is a function, it will be called on every request and passed the request as the sole argument. All key-values in the returned object will be added to the event. If the function returns a falsey value, no columns will be added. To replicate the above Array-based behavior, you could use the following config: express: { userContext: (req) => req.user && { username: req.user.username } }

This function isn’t limited to using the request object, and can pull info from anywhere to enrich the data sent about the user.

http

Adds columns with prefix http.

https

Adds columns with prefix https.

mysql2

Adds columns with prefix db.

sequelize

Instrumented only for context propagation

mongoose

Instrumented only for context propagation

mongodb

Adds columns with prefix db.

Configuration options:

Name Type
mongodb.includeDocuments boolean

mongodb.includeDocuments

If true, documents in the api will be JSON serialized and included in the events sent to honeycomb.

react-dom/server

Adds columns with prefix react.

child_process

Instrumented only for context propagation

Optional configuration

The additional optional configuration in the code example above is where you can add global settings (Honeycomb credentials and dataset name) or per-instrumentation settings:

{
    writeKey: "YOUR_API_KEY",
    dataset: "my-dataset-name"
    $instrumentationName: {
        /* instrumentation specific settings */
    }
}

dataset is optional. If you do not specify a dataset, it will default to nodejs.

You may also specify writeKey and dataset by setting HONEYCOMB_WRITEKEY and HONEYCOMB_DATASET in your environment.

To add custom instrumentation settings, specify them in your config object as a key/value pair using the name of the instrumentation as the key. For example, to add configuration options for express, your config object might look like:

{
    writeKey: "YOUR_API_KEY",
    dataset: "my-dataset-name",
    express: {
    /* express-specific settings */
    }
}

See the Instrumented packages section below for available configuration options for each package.

Customizing instrumented packages

If you want to disable automatic instrumentation for whatever reason, for either an individual package or all packages, you can pass enabledInstrumentation when configuring the beeline. It should be an array of package names to automatically instrument. For instance, if you want to enable the beeline instrumentation only for the http and https packages:

require("honeycomb-beeline")({
  enabledInstrumentation: ["http", "https"],
  /* ... additional configuration ... */
});

The beeline also exports getInstrumentations, which returns a list of all enabled instrumentation. You can use this to filter out specific instrumentations you want to disable. If you want to enable all instrumentation except mongodb:

const beeline = require("honeycomb-beeline");
beeline({
  enabledInstrumentation: beeline.getInstrumentations().filter(i => i !== "mongodb")
  /* ... additional configuration ... */
});

Finally, to disable all automatic instrumentation, pass an empty array as in:

require("honeycomb-beeline")({
  enabledInstrumentation: [],
  /* ... additional configuration ... */
});

Proxy configuration

If the environment variables HTTPS_PROXY or https_proxy are set, the Beeline will pick up and configure itself to use the proxy for all event traffic to Honeycomb.

Sampling events

We have built-in support for sampling in the Beeline. Simply set the sampleRate variable to your config object in your app.js:

require("honeycomb-beeline")({
  writeKey: "YOUR_API_KEY",
  dataset: "my-dataset-name",
  sampleRate: 2
  /* ... additional optional configuration ... */
});

This will give you a sample rate of 1 in 2 events.

Troubleshooting the Beeline

There are two general approaches to finding out what’s wrong when the Node.js Beeline isn’t doing what you expect.

“The events I’m generating don’t contain the content I expect”

Enable debug output (sent to STDOUT) by setting DEBUG=honeycomb-beeline:* in your environment. This will print the JSON representation of events to the terminal instead of sending them to Honeycomb. This lets you quickly see what’s getting sent and allows you to modify your code accordingly.

$ DEBUG=honeycomb-beeline:*

Example event

Here is a sample event created by the Node.js Beeline:

{
  "Timestamp": "2018-03-20T00:47:25.339Z",
  "request.base_url": "",
  "request.fresh": false,
  "request.host": "localhost",
  "request.http_version": "HTTP/1.1",
  "request.remote_addr": "127.0.0.1",
  "request.method": "POST",
  "request.original_url": "/checkValid",
  "request.path": "/checkValid",
  "request.scheme": "http",
  "request.query": "{}",
  "request.secure": false,
  "request.url": "/checkValid",
  "request.xhr": true,
  "response.status_code": "200",
  "meta.instrumentation_count": 4,
  "meta.instrumentations": "[\"child_process\",\"express\",\"http\",\"https\"]",
  "meta.type": "express",
  "meta.version": "4.16.3",
  "meta.beeline_version": "1.0.2",
  "meta.node_version": "v9.10.0",
  "totals.mysql2.count": 2,
  "totals.mysql2.duration_ms": 13.291,
  "totals.mysql2.query.count": 2,
  "totals.mysql2.query.duration_ms": 13.291,
  "trace.trace_id": "11ad83a2-ca8d-4918-9db2-27524456d9f7",
  "trace.span_id": "4a3892ba-0936-46e1-8e17-31b887326027",
  "name": "request",
  "service_name": "express",
  "duration_ms": 15.229326
}

Queries to try

Here are some examples to get you started querying your app’s behavior:

Which endpoints are the slowest?

Where is my app spending the most time?

Which users are using the endpoint I’d like to deprecate?

Which XHR endpoints take the longest?

Contributions

Features, bug fixes and other changes to Beelines are gladly accepted. Please open issues or a pull request with your change via GitHub. Remember to add your name to the CONTRIBUTORS file!

All contributions will be released under the Apache License 2.0.