1. Docs
  2. Investigate
  3. Detect Anomalies

Detect Anomalies

Honeycomb’s BubbleUp feature detects anomalies and explains how a subset of data differs from other data. With BubbleUp, you visually select a set of data points from a heatmap, compare it to the remaining data, and then investigate this comparison with visual charts. Values that contrast provide answers or additional fields to explore.

Use BubbleUp to identify:

  • the cause of a slow API (example scenario)
  • which service caused a slowdown or error
  • issues across distributed systems

Access BubbleUp 

Choose your method of accessing BubbleUp:

  1. In Query Builder, VISUALIZE a Heatmap with HEATMAP(<variable>).

  2. Select Run Query.

    Query Builder with VISUALIZE heatmap with the Run Query button This creates a heatmap below the Query Builder.

  3. Identify data that stands apart from other data. BubbleUp works based on a selection you make in a heatmap.

  4. In the heatmap, draw a box around the data to define the selection. A menu appears.

    In BubbleUp with a selected area of a heatmap

  5. Choose Detect Anomalies (BubbleUp)

  6. View the BubbleUp charts below the heatmap. This creates a comparison between the selection and the area not selected, or the baseline. These comparisons are represented as charts.

In a query, VISUALIZE clauses generate line charts, except for VISUALIZE HEATMAP(<variable>).

  1. In Query Builder, create a query with at least one VISUALIZE clause and one GROUP BY clause.

  2. Select Run Query. This query creates at least one visualization below the Query Builder. Each result, or line on the chart, represents a grouped field.

  3. In any non-Heatmap chart, identify data to investigate further.

    View of Query Results chart before BubbleUp action

  4. Select the result, or line, to investigate further. A menu appears.

  5. Choose Detect Anomalies (BubbleUp) The ability to view and select this menu option requires two or more results, or lines on the chart.

    Highlighted result in chart from selecting Investigate with BubbleUp in the menu. This action performs a BubbleUp query that graphs the selected result's value in yellow and all the other values (the baseline) in blue.

  6. View the BubbleUp charts below the visualization(s). This creates a comparison between the selection and all other results, or the baseline. These comparisons are represented as charts.

  1. In Query Builder, GROUP BY a value and select Run Query.

  2. Within Overview, view a summary of grouped fields below any VISUALIZE displays.

  3. Hover over the row representing a grouped field and select the grouped field’s ellipsis (…) menu.

    Results table hover over one value shows ellipsis An Actions menu appears.

  4. Select BubbleUp: Compare <value> to all other events in the Actions menu.

    Results table where one value's context menu is opened and BubbleUp is highlighted

    This action performs a BubbleUp query that graphs the selected row’s value in yellow and all the other values, the baseline, in blue.

  5. View the BubbleUp charts below the graph.

    These charts represent the selection compared to the baseline across all of the fields.

    Results of a BubbleUp Query with charts where app.user.email = <user@example.com>

Interacting with BubbleUp Charts 

A dataset or environment has many fields. BubbleUp represents each field with a chart.

The charts divide into two groups:

  • Dimensions contain fields with categorical or ordinal values
  • Measures contain fields with numeric values

Each chart categorizes the data into two groups:

  • Selection, rendered in yellow on the right side of a value, contains the points in the area selected in the heatmap.
  • Baseline, rendered in blue on the left side of a value, contains all the points outside of the area selected.
BubbleUp dimensions chart example with annotations

The title of the chart is the name of the field.

In the upper right corner, the two donut charts display a ratio of how often the field is found in the data. A field and its values may not be populated in a dataset or environment.

Hover over the title and donut charts to display a tooltip with the full field name and a percentage of how often the field appears in the Selection (yellow) and Baseline (blue).

The bar chart displays each value in bar form, which represents its frequency in the Selection or in the Baseline. The height of each bar is proportional to the number of times the value occurs in the results of the query. The bar chart displays a maximum of seventy-five values, a subset of both Selection and Baseline values.

Hover over the bar for a value, to display a tooltip with its full value name and a percentage of how often the value appears in the Selection and Baseline.

BubbleUp dimension bar chart value tooltip when hovered

A field may contain largely unique values. These are sometimes referred to as nominal data. The tooltips for nominal columns show the exact number of occurrences instead of a percentage in the Selection or the Baseline. For example, events that capture a span in a trace maintain a unique ID for the span in the trace.span_id column.

BubbleUp dimension with nominal values that show a value occurred 1 time in the baseline and 0 times in the selection

Click on a value, or a pair of bars, to display an action menu to take further actions. For charts in the Dimensions section, this actions menu appears:

BubbleUp dimension bar chart value action menu when clicked
Group by Field
Adds a GROUP BY clause and re-runs the query. Select Results to view a summary of grouped fields below the heatmap.
Show only where field is value
Adds a WHERE clause to filter with <field> = <value> and re-runs the query.
Show only where field is not value
Adds a WHERE clause to filter with <field> != <value> and re-runs the query.
Copy field name
Copies the field name to your OS clipboard.

For charts in the Measures section, this actions menu appears:

BubbleUp measures histogram chart value action menu when clicked
Show only where field less than
Adds a WHERE clause to filter with <field> < <value> and re-runs the query.
Show only where field greater than
Adds a WHERE clause to filter with <field> > <value> and re-runs the query.

Use BubbleUp Permalinks to share BubbleUp results with others.

  1. Create a BubbleUp result.

  2. Copy the URL link in your web browser.

  3. Share the URL with other users to view the same BubbleUp result.

    Copy link to BubbleUp result directly from browser's URL field

BubbleUp Filter 

Filter your BubbleUp results with keywords to find specific fields and their charts faster.

  1. Create a BubbleUp result.
  2. Locate the BubbleUp Filter search box above the BubbleUp Charts.
  3. Enter a keyword into the search box.
  4. The BubbleUp Charts dynamically update to show charts that contains relevant field names.

In the example below, the word “trace” in BubbleUp Filter search box results in four results with field names that start with “trace”.

Search BubbleUp results based on keywords

BubbleUp Example Scenario: Slow API 

An operations team responsible for API performance receives an alert that their system is handling some requests slowly.

With Honeycomb, the investigating user generates a HEATMAP visualization that shows the statistical distribution of duration_ms of an application’s requests over the selected time period.

Query results of HEATMAP(duration_ms) over a two hour time period

The investigating user discovers a group of events with a high duration_ms. With BubbleUp, the user draws a box on the heatmap to define the selection. The selection, depicted below, defines an area ranging two hours on the x-axis and from near 800ms to 1400ms on the y-axis.

BubbleUp selection made from outstanding data

With the selection made, BubbleUp renders charts for each field below the heatmap. The user notes several fields with similar and different values.

The fields app.platform and app.build_id have fairly similar proportions. The bars for each value are very close in height.

BubbleUp field chart shows similar values between selection and baseline

The fields app.user_id, app.endpoint, and name have very different proportions. The bars for some of the values in these fields have drastically different heights compared to the bars for the other values in these fields.

BubbleUp field chart shows different values between selection and baseline

The user hovers over the tallest selection bar in the app.endpoint chart.

BubbleUp app.endpoint chart hovered over the tallest endpoint

The user finds that the /api/v2/tickets/export endpoint is responsible for the slow requests. From this information, the team explores the endpoint’s implementation and its external dependencies for issues during that time frame. The user formulates new queries to explore the external instrumentation.

Tips and Tricks 

  • Define a selection of data that is separate from the remaining data. BubbleUp works best when the data in the selection contrasts the data in the baseline. It is far more difficult to tell how a dataset is different when the selection is inside the main cluster of data, as in the following screenshot: Heatmap with BubbleUp selection inside a sea of data, like Jupiter's eye

  • Define a smaller time range to find data separated from the baseline. Bigger time ranges may display greater variance across the data. An isolated time range helps reduce unrelated variation in your data that accumulate over time.

  • Add a filter, or WHERE clause to your query to explore the selected data further. A field’s action menu quickly provides these filters. Other times, it can be valuable to pursue a breakdown to understand better how fields vary from each other.

On this page: