We use cookies or similar technologies to personalize your online experience and tailor marketing to you. Many of our product features require cookies to function properly. Your use of this site and online product constitutes your consent to these personalization technologies. Read our Privacy Policy to find out more.

X

Getting started with Rails + Honeycomb

This Guide is about our deprecated Rails integration. If you have a Rails 4, Rails 5, or Rack app, use our newer Ruby Beeline. We now recommend the Rails integration only for Rails 3 apps.

We hear you’ve got a Rails application you’d like to learn more about! Webapps like yours run the Internet, and we’re here to help you use Honeycomb to observe your Rails application. This guide will use the RailsBridge Bridge Troll application as an example (code here and site here), but you’re welcome to follow along in your own application instead. You’ll need Ruby 2.2 or newer and Rails 3.0 or newer, and a Honeycomb API key - sign up for a trial to get one!

Here are some example questions that this guide will aim to help you answer:

By the end of this guide, if you’ve followed along (and you should!), we should have:

Many thanks to the fine folks of RailsBridge for producing such a high-quality public Rails app for the community to learn from!

Preface: What has been staring us in the face already?

If you’re a Rails developer, you already know how wonderful Rails is for getting things to Just Work. And if you’ve spent any time looking at the default server output, you know how many timers are embedded in Rails itself— and how much useful information gets output to the (unstructured—what a shame!) server logs.

Example Rails log output

As you read through this guide, consider the following: What other sorts of data do you see in development all the time that might be helpful for debugging your application in production?

Installation

Using the honeycomb-rails gem it’s easy to configure your Rails app to send all that great metadata to Honeycomb. The Rails team built some instrumentation “hooks” into Rails itself, so that developers can be notified when certain events happen inside their application. honeycomb-rails takes advantage of some of these to capture events about what your Rails app is doing, and to forward them on to Honeycomb!

To get started, add the following to your app’s Gemfile:

gem 'honeycomb-rails'

That will configure Rails to send events to Honeycomb as your app processes HTTP requests and queries the database. We’ll be able to isolate specific controllers, actions, and formats; track successes and failures via HTTP status code; and drill down into how much time we’re spending in the database. Cool!

Note that libhoney, the library used to send events to Honeycomb, does all of its HTTP transmission on a background thread, so you can trust honeycomb-rails to have minimal impact on your application’s performance.

However, we still need to configure some credentials so we know which Honeycomb account and datasets to send events to. Add an initializer (we’ve named ours config/initializers/honeycomb.rb). See the RubyDoc for more Config options.

# config/initializers/honeycomb.rb

HoneycombRails.configure do |conf|
  conf.writekey = 'YOUR_API_KEY'
  conf.dataset = 'rails-dataset'
  conf.db_dataset = 'db-dataset'
end

You are currently logged in to the team, so we have populated the write key here to the first write key for that team.

After clicking around a bit in our local Bridge Troll application to trigger some HTTP requests, we can start seeing data in Honeycomb: below, we’re exploring which controllers and actions are being requested the most and how long those requests are taking.

Count and P95(duration_ms) of controller/action pairs

Next: understand individual user behavior

For apps that use Devise for authentication, honeycomb-rails automatically adds information about the current user to the event sent for each request. This means we can analyze individual users’ Rails requests from inside Honeycomb! We can figure out which users are triggering the slowest requests, or even just see traffic patterns with a given email address.

For example, it’s trivial to filter down to a single user’s traffic:

Zoom into a single user's traffic trivially

Honeycomb Tip: Capturing user information in events is a classic example of a “high cardinality” field— meaning, there are tons of possible unique values for the field.

Including user IDs or email addresses can sometimes cause problems in other data systems— but Honeycomb makes it easy to filter your app’s requests by a given email address while also answering high-level questions (e.g. “Which endpoints are the slowest for this particular user?”).

In addition to any user information, honeycomb-rails also captures any flash messages that were set for the current response. They can be helpful if we ever have to debug an error or strange user interaction in the future. Breaking events down by flash_notice or flash_error can give you a unique insight into how your users have been experiencing your system.

Breaking events down by flash message can give you a unique insight into how your users experience your system

Honeycomb Tip: Adding fields like this won’t slow Honeycomb queries down, because Honeycomb’s query engine only pulls in the fields actively being queried over.

Recording application-specific information

The automatic instrumentation that honeycomb-rails provides out of the box has gotten us pretty far— we can answer questions around user behavior, performance anomalies, etc— but if we want to really understand the detailed interactions in our application, we’ll want the ability to capture things specific to individual controllers and actions.

Add useful fields for (most of the) whole controller

To record information in the scope of a particular request being processed, honeycomb-rails adds a honeycomb_metadata method to your controllers. It returns a hash, whose contents will be added to the event that we’re sending to Honeycomb. So recording information about a request is as simple as adding data to the hash.

As an example, we’ll instrument a classic Rails controller, Bridge Troll’s ChaptersController, which manages listing, displaying, updating, and (sometimes) destroying RailsBridge chapters.

Most actions on this controller work with a single chapter, so it’ll be helpful to track which chapter we’re operating on. The chapter is stored in the @chapter instance variable, set by the assign_chapter method. So we can extend assign_chapter to also record the chapter id in the honeycomb_metadata:

# app/controllers/chapters_controller.rb

def assign_chapter
  @chapter = Chapter.find(params[:id])

  # Add this line:
  honeycomb_metadata[:chapter_id] = @chapter.id
end

Because assign_chapter is set to run as a before_action hook for the actions that work with a single chapter, we can be sure that we’ll record the chapter for all those actions, without having to instrument each action individually.

Add useful fields for one important action on a controller

There are often specific things in specific actions that are worth tracking. In our ChaptersController#index, for example, it might be interesting to know how many chapters are being returned to the user.

# app/controllers/chapters_controller.rb
def index
  skip_authorization
  @chapters = Chapter.all.includes(:organization)

  # Add this line:
  honeycomb_metadata[:num_chapters] = @chapters.size
end

Honeycomb Tip: This is an example of a field that contributes to “sparse data”: sometimes not all fields exist on all payloads in a Honeycomb dataset, and that’s okay! Honeycomb handles these extra fields with grace, ignoring them if applicable (e.g. if you’re calculating the AVG(num_chapters), the query engine will ignore Honeycomb events without a num_chapters value set.).

Now, when everything is saved and we reload ChaptersController#index, the page loads and… our Honeycomb dataset now knows about num_events on requests for this controller and action!

As another example, we can add num_events to track the number of events returned for a given Chapter in ChaptersController#show:

# app/controllers/chapters_controller.rb
def show
  skip_authorization
  @chapter_events = (
    @chapter.events.includes(:organizers, :location).published_or_visible_to(current_user) + @chapter.external_events
  ).sort_by(&:ends_at)

  # Add this line:
  honeycomb_metadata[:num_events] = @chapter_events.size

  # ... Keep on keepin' on
end

Then (see the video below!) we can start with a graph of the COUNT and AVERAGE(num_events) of requests going to individual controller/actions, then adjust the query to break down by chapter_id instead (including only the controller/actions that contribute towards num_events). This lets us iterate on how we want to break down our app’s traffic, while always making sure we have access to the raw events.

Thinking about instrumentation

Understanding database performance

To go beyond just controller activity, honeycomb-rails also sends events about SQL queries run via ActiveRecord events to describe how your application is interacting with the database. These events log the normalized SQL query, separating out the shape of the query (SELECT * FROM users WHERE users.id = ? from the parameters ["id", 1]).

A quick note on where we’re sending our data. Our events representing HTTP requests are currently going to a dataset in Honeycomb called “yourapp” (configured in config/initializers/honeycomb.rb above). The events representing SQL queries are instead going to a dataset called “yourapp_db”.

Honeycomb Tip: We recommend separating events into different Datasets when two events aren’t comparable in their frequency or their scope.

In the "yourapp" dataset, one inbound HTTP request maps to a Honeycomb event—this makes that dataset easy to reason about and analyze. Because a single Rails request may trigger many ActiveRecord queries, and because ActiveRecord queries have little in common with the controller or action that invoked them, it feels more natural to keep them in separate datasets.

Capture traces alongside Rails requests

Honeycomb’s power lies in being able to switch from high-level analytics, to raw data, and back, as we iterate towards an insight—and what better way to zoom into some detail of execution than with a deep trace of what the Rails application is doing?

Instantiate your own libhoney instance

At the moment, honeycomb-rails doesn’t understand the concept of a trace—but because tracing instrumentation is simply a set of extra metadata Honeycomb uses to infer a hierarchy out of a set of events, we can easily manage this capture ourselves.

First, add libhoney to your app:

# Gemfile
gem "libhoney"

Then, make it available to your application by modifying our initializer. Note the change to the HoneycombRails.configure block in order to use the provided libhoney instance.

# config/initializers/honeycomb.rb
require 'libhoney'

key = 'YOUR_API_KEY'
dataset = "rails#{Rails.env.production? '-prod' : ''}"

$libhoney = Libhoney::Client.new(
  writekey: key,
  dataset: dataset,
  user_agent_addition: HoneycombRails::USER_AGENT_SUFFIX,
)

HoneycombRails.configure do |conf|
  conf.writekey = key
  conf.dataset = dataset
  conf.db_dataset = dataset + "-db"
  conf.client = $libhoney
end

Populate tracing headers on Rails requests

This tracing model will consider Rails HTTP requests to be the root span, or the top of a trace. Not all requests may have children or an extended trace, but we’ll hang our traces off of an initial HTTP request to start with.

To achieve this, capture common attributes by overriding append_info_to_payload in your application_controller.rb.

  # app/controllers/application_controller.rb
  def append_info_to_payload(payload)
    super(payload)
    honeycomb_metadata["trace.trace_id"] = request.request_id
    honeycomb_metadata["trace.span_id"] = request.request_id
    honeycomb_metadata[:service_name] = "rails"
    honeycomb_metadata[:name] = self.class.name
  end

Thread identifiers through while tracing

Let’s say that one of our controllers calls out synchronously to some third-party service, and this is the first path we want to trace. First, let’s make sure that we pass the request.request_id to the next span as both the trace ID and parent ID.

  # app/controllers/users_controller.rb
  foo = SynchronousWork.do_work(request.request_id)

Once we have our trace ID and initial parent ID available available for the next unit of work, we can go about creating our trace (aka, a series of events with pointers creating a hierarchy). We’ve used some syntactic sugar (with_span) to help us with propagating the relevant identifiers via thread locals.

(Note that, soon, honeycomb-rails will expose this support in a more streamlined manner.)

  # app/work/synchronous_work.rb
  def do_work(request_id)
    Thread.current[:request_id] = request_id
    Thread.current[:span_id] = request_id
    with_span("do_work", other_metadata_key: other_metadata_value) do
      # ... do work

      # `with_span` can be nested as deeply as needed to capture interesting
      # paths of execution.
      with_span("http_get", url: url) do
        HTTParty.get(url).body
      end
    end
  end

Bonus: If you’re not on Rails

If you’re looking for a Ruby-based solution but aren’t on Rails, we’ve still got you covered! This approach of mapping individual requests to Honeycomb events is also useful for folks to get started with analyzing their system.

Our Beeline for Ruby captures a number of the same things that honeycomb-rails captures: request path, request duration, and status code for Sinatra or other Rack apps.

Contributions

Features, bug fixes and other changes to our Rails integration are gladly accepted. Please open issues or a pull request with your change via GitHub. Remember to add your name to the CONTRIBUTORS file!

All contributions will be released under the Apache License 2.0.