If you need to troubleshoot your visualizations, explore these solutions to common issues.
Troubleshoot visualizations in Honeycomb Home.
If your dataset is not a tracing dataset (that is, it does not include trace.trace_id or equivalent), then the chart visualization will default to “Total Events”, which will count all events that Honeycomb receives.
Both “Total Error Events” visualizations rely on a source field being mapped to the Error dataset field.
To ensure both visualizations are populated, when you configure your visualizations, map a source field to the Error dataset field, or send a source field named error to Honeycomb.
Both “95th Percentile Latency” visualizations rely on a source field being mapped to the Duration dataset field.
To ensure both visualizations are populated, when you configure your visualizations, map a source field to the Duration dataset field, or send a source field named duration_ms to Honeycomb.
Troubleshoot visualizations in Board Templates.
Visualizations in board templates depend on specific fields being available in your data. Add additional insights and eliminate missing queries by configuring or sending the required fields. Sending the necessary data populates the templates with more queries. You can also choose to select different fields to populate each query within the Setup view. For more information, refer to our available templates and their required fields.
Troubleshoot visualizations in Service Map.
An empty service map can happen for two reasons:
Service Map generates automatically from trace data sent to Honeycomb and, with the exception of timestamp, is based on these defined Tracing fields:
If any of this data is missing or undefined, the map will not display.
Confirm that your Dataset fields in Dataset Definitions are defined with Field name values.
Service Map may take up to a few minutes to generate after you start sending data to Honeycomb. Check back in a few minutes to confirm if the Service Map has generated.
First, confirm that traces include the service name.
Service Map generates services from trace data that includes the service name.
To confirm, run a VISUALIZE COUNT WHERE service.name = <service name> query in Query Builder where service.name is the service name field.
The query will return a count of events associated with the service and helps to validate whether the Service exists in that time range.
If the query result returns 0, then it means that no requests were made to that service and Service Map is accurate.
If COUNT returns a result greater than zero, it may not be visible on the map because:
Recently sent requests to this service may take a few minutes to process and display on the map
The service is instrumented with component: proxy or net.component: proxy.
Thus, Honeycomb treats the service as a Gateway, which visually transforms the service into a Gateway square on an edge as opposed to a Service node.
Confirm by running the following query to see if your service is instrumented with gateway attributes:
COUNTservice.name = <service name>component:proxy, net.component:proxySecond, use the time picker in the top right to select a larger range of time. Service Map automatically samples traces to represent higher volume services. This means that Service Map may not represent services or edges with very low traffic in a selected time range. Expand the date range to view a larger sample, and the missing service may appear.
Service Map generates based on trace data sent to Honeycomb. If traces only include one service, the singular service displays as one disconnected node. Send traces that include more than one service to visualize edge connections between services on your map.
Follow the instrumentation guide to flag gateways, so Service Map can represent them differently.
Use Services or Filter Traces dropdowns to narrow down your map to a specific set of services. After narrowing down to a smaller diagram, hover over a specific service on the map and select Isolate to display all dependencies, or its incoming and outgoing services, for the selected service.
When filter parameters are applied to the Service Map, the results return a maximum of 10,000 traces. These traces are then displayed on the Service Map when using Filters and/or selectively displayed when using the Service Filter.
Because of the limit applied, there is a possibility that very low volume traces are not represented in the sample returned.
This difference is expected.
Service Map calculates the p95 duration for services and edges differently from the p95(duration_ms) query in Query Builder.
Service Map latency calculations include synchronous spans, which communicate between services. Query Builder latency calculations includes both synchronous and asynchronous, or internal, spans.
When instrumenting your gateways and they still do not appear on your map, it is possible that your gateway requests do not occur between known services. Service Map only displays gateways on an edge, which is formed when one service send requests to another service. If an edge does not exist, either because a service does not send requests to another service or if the service sends requests to another service through multiple gateways, then gateways may not display on your map.
If either of these scenarios are common in your architecture, join #discuss-service-map in our Pollinators Community Slack and let us know.
Filters find traces with at least one span that matches all filters you have entered.
For example, if the filters of app.cart.items > 5 AND app.user.currency = USD are set in Filter Traces, then it displays traces that have at least one span that matches both those filters.
If the fields used in your filters exist across different spans in a trace, a result is not returned.
If possible, try propagating context throughout your traces to enable filters to return matching results.