Trace datasets are exactly the same as any other dataset in Honeycomb: you can break down, filter and aggregate by any field.
To view traces associated with your query, select the “Traces” tab. This will show you the slowest traces that match the query, calculated by root span duration.
The “Traces” tab will show a maximum of 10 sample traces. If you need to access a larger list of traces that match your query, you can click the “Raw Data” tab and click any trace ID to view the trace that includes that event.
Alternatively, add a Break Down by
traceId to your query. (Use
if you’re using a Beeline.) Click on a
the results table to see its trace diagram.
If you want to see specific traces that correspond to a feature on the graph, such as a latency spike, you can select a region on the graph, and the list of trace summaries will update:
Click on one of the
traceId links to see the full waterfall diagram for that
The first row is the root span, the parent of the entire trace. Below it are each of the spans in this trace. The bars on the right display how much time each operation took to execute. This type of graph is also called a waterfall diagram, as it shows how the order of operations cascade across the total execution time.
To see more detail about an individual span, click on a row to view that span’s raw data in the right sidebar:
To display additional columns, open the column selection menu by clicking the plus icon in the header. Check the boxes for the columns you’d like to include. To hide a column, click on the column name and select “Hide column” from the menu. To collapse or expand a portion of the trace, click on the arrow on the left side of a span row.
Here is an example of investigating a problem using tracing:
A few users are reporting slow API calls. The initial investigation shows one endpoint is periodically slow. One user is making a lot of calls, and that is slowing down responses for everyone.
Selecting a representative trace shows the full execution of the API request, helping us understand why these API requests are slow.
The first part seems normal, but
ticketBackend is making more calls than
mysql. Clicking on one of the
mysql spans shows more information
about it in the sidebar, including the specific database query.
The information from this trace points to a possible next step. Is the user running a script gone wrong? Or maybe this endpoint needs tighter rate limiting. Traces show a level of detail that can be hard to see otherwise.