Service Map in Honeycomb visually represents how traffic flows through your system in any given environment. It displays known services, service dependencies, and communication between services. Interact with the Service Map to examine and investigate smaller sets of services in detail.
Service Map generates a view of:
p95 duration of request response time for each service and for service to service communication.
p95 stands for 95th percentile.
p95 duration indicates that 5% of the data sample have durations higher than that threshold and 95% of the data sample have durations lower than that threshold.
Use cases for Service Map includes, but are not limited to:
Service Map is an Enterprise feature that is only available for teams sending data to new environments. Enterprise Honeycomb Classic teams are encouraged to migrate to Environments in order to use Service Map.
Access Service Map from the left navigation bar:
Service Map is built with distributed tracing data that you send to Honeycomb.
Spans within traces must contain several key pieces of data to construct the Service Map:
These fields can be configured in each dataset’s Definitions.
At the top, the Service Map summarizes how many services are displayed.
A label indicates when the Service Map last regenerated. Use the time picker to modify the selected timespan. Use a preset time range or a custom time range.
Navigate your selection history with the left and right arrows.
Use your mouse or trackpad to magnify and view parts of the map in detail. Use the Recenter button in the top right corner to reset the map display to an overview magnification level.
Select the Legend icon in the bottom left corner to translate symbol meanings.
Select a service to trigger a hover box with service name and
p95 duration information.
Select Isolate in the hover box to activate Isolate Mode.
Hovering on a service also highlights that service’s dependencies within the Service Map.
The right side panel displays details about services and lists a sample of related traces.
Use the collapsible right side panel to:
Each Service is represented by a circular node.
The size of the service represent the relative volume of requests that the service receives compared to other services in your environment.
Services in purple have the highest number of dependencies, or combined incoming and outgoing services.
The service’s labels display the
service.name value and its
Disconnected services display separately from the main set of connected services. Disconnected services are services that do not communicate to or reference other services.
Select a service to populate its details in the right side panel.
An Edge, or line, represent communication between two services.
The thickness of the edge represents the relative volume of requests between the two services compared to other services.
When it appears, the edge’s label displays its
Select an edge to populate its details in the right side panel.
The right side panel displays details about services, lists a sample of related traces, and provides the ability to filter and highlight.
The panel’s title indicates if it summarizes the Overall Service Map, a specific service by its name, or a selected edge.
The panel displays an Overall view when the map initially loads, or when no service or edge is selected.
All services are displayed in an alphabetized list with their
Disconnected appears in the Services list with the total of disconnected services present.
Disconnected services are services that do not communicate to or reference other services.
When a Service is selected, the side panel displays:
p95 Latency: The
p95 duration for this service to respond to requests
p95 duration between the selected service and the incoming service
p95 duration between the selected service and the outgoing service
When an Edge is selected, the side panel displays:
p95 Latency: The
p95 duration for the receiving service to respond to the requesting service
In the right side panel, use Search services to enter terms and search for a specific service. As you type, Honeycomb updates the list of displayed services.
In the right side panel, Service Map provides sample traces that correspond to the Service Map. When selecting a service or edge, the right side panel updates to display a list of five traces that contain the selected service(s).
Select See More for a longer list of traces in Query Builder and the ability to explore related traces in detail.
For a service, See More creates a query with a filter where
service.name = <service>.
For an edge, See More creates a query with a filter where
<service 1> calls <service 2> = true.
This unique filter includes a custom-created Honeycomb derived column, specific for this query, to isolate traces where the requesting service (
<service 1>) calls the receiving service (
In the Services dropdown, enter one or more service names to modify the Service Map display and show only the entered service(s). As you type, Honeycomb autocomplete prompts with selectable service names.
In the Filter Traces dropdown, enter fields to modify the Service Map display and show only services with traces that contain at least one span that matches the entered criteria.
To define a filter, enter a field name, operator, and field value. As you enter terms, Honeycomb autocomplete prompts with selectable field options.
Use the environment-wide fields that appear as a prompt to start defining a filter.
For example, use the
Route preset to filter for all routes that contains
The following environment-wide preset filters are available and based on each dataset’s dataset definitions:
Isolate Mode focuses the map to display a single service and its immediate dependents. To activate, hover over the target service in the map and select Isolate in the hover box.
Then, “Isolate Mode” appears at the top of the Service Map display to indicate its status. The map updates to show only the target service, the Incoming Services that send requests to the target service, and Outgoing Services that the target service sends request to.
Select Show full map at the top left to leave Isolate Mode and return to the overall map.
Use Gateways to illustrate which services communicate to another through a known gateway.
Learn more about instrumenting gateways. When toggled on, any Gateway appears as a blue square on an edge.
Hover over the blue square to see the Gateway’s name and its
Use Entry Services to highlight which services are in the root span of any trace.
Toggle Entry Services in the top right corner of the Service Map to modify the Service Map display. When activated, any Entry Service appears with an additional dashed circle around its circular node.
In order for Service Map to be generated, at minimum, teams need to send tracing datasets to specific environments. The following defined fields for spans in traces are required for a map to be automatically generated from your tracing datasets:
To draw a meaningful map, the traces should span multiple services.
By default, Service Map treats
service.name in spans as a distinct service.
For infrastructure with gateways, this treatment could be problematic as it could obscure true service to service relationships.
If multiple services communicate through this gateway, the visual may appear as if some services speak exclusively to the gateway, or as if some services only receive traffic from the gateway.
For example, if Service
A communicates to Service
B through a gateway
foo, the automatically generated Service Map will show Service
A communicating to
foo communicating to Service
To ensure the relationship is not hidden by this representation, users can adjust their instrumentation so Honeycomb can represent gateways accurately.
No semantic convention exists for designating services on spans as gateways. We recommend the following methods for enabling Gateway visuals on Service Map. These recommendations are subject to change, pending the creation of an OpenTelemetry standard for Gateways.
To flag services as gateways and thus change its representation on the map:
This option automatically instruments
component.proxy to each span.
No additional manual instrumentation is required.
We recommend manually instrumenting
net.component: proxy for your gateway service(s) with one of two options:
meshConfig to include
- key: "net.component"
We recommend using Service Map to understand overall request traffic, as Service Map automatically samples traces to represent higher volume services. This means that Service Map may not represent services or edges with very low traffic in a selected time range.
Use Services Map in the following situations:
Quickly onboard engineers into a large complex architecture. Ask and identify:
During debugging or Root Cause Analysis for an issue with one or more services, ask and identify:
When validating system observability, or instrumentation quality assurance, ask and identify:
First, confirm that your Dataset fields in Dataset Definitions are defined with Field name values. Service Map generates automatically from trace data sent to Honeycomb, and with the exception of timestamp, is based on these defined Tracing fields:
If any of this data is missing or undefined, the map will not display.
Second, the Service Map may take up to a few minutes to generate after starting to send data to Honeycomb. Check back in a few minutes to confirm if the Service Map has generated.
First, confirm that traces include the service name.
Service Map generates services from trace data that includes the service name.
To confirm, run a VISUALIZE
service.name = <service name> query in Query Builder where
service.name is the service name field.
The query will return a count of events associated with the service and helps to validate whether the Service exists in that time range.
If the query result returns
0, then it means that no requests were made to that service and Service Map is accurate.
If COUNT returns a result greater than zero, it may not be visible on the map because:
Recently sent requests to this service may take a few minutes to process and display on the map
The service is instrumented with
component: proxy or
Thus, Honeycomb treats the service as a Gateway, which visually transforms the service into a Gateway square on an edge as opposed to a Service node.
Confirm by running the following query to see if your service is instrumented with gateway attributes:
service.name = <service name>
Second, use the time picker in the top right to select a larger range of time. Service Map automatically samples traces to represent higher volume services. This means that Service Map may not represent services or edges with very low traffic in a selected time range. Expand the date range to view a larger sample, and the missing service may appear.
Service Map generates based on trace data sent to Honeycomb. If traces only include one service, the singular service displays as one disconnected node. Send traces that include more than one service to visualize edge connections between services on your map.
Follow the instrumentation guide to flag gateways, so Service Map can represent them differently.
Use Services or Filter Traces dropdowns to narrow down your map to a specific set of services. After narrowing down to a smaller diagram, hover over a specific service on the map and select Isolate to display all dependencies, or its incoming and outgoing services, for the selected service.
When filter parameters are applied to the Service Map, the results return a maximum of 10,000 traces. These traces are then displayed on the Service Map when using Filters and/or selectively displayed when using the Service Filter.
Because of the limit applied, there is a possibility that very low volume traces are not represented in the sample returned.
This difference is expected.
Service Map calculates the
p95 duration for services and edges differently from the
p95(duration_ms) query in Query Builder.
Service Map latency calculations include synchronous spans, which communicate between services. Query Builder latency calculations includes both synchronous and asynchronous, or internal, spans.
When instrumenting your gateways and they still do not appear on your map, it is possible that your gateway requests do not occur between known services. Service Map only displays gateways on an edge, which is formed when one service send requests to another service. If an edge does not exist, either because a service does not send requests to another service or if the service sends requests to another service through multiple gateways, then gateways may not display on your map.
If either of these scenarios are common in your architecture, join #discuss-service-map in our Pollinators Community Slack and let us know.
Filters find traces with at least one span that matches all filters you have entered.
For example, if the filters of
app.cart.items > 5 AND
app.user.currency = USD are set in Filter Traces, then it displays traces that have at least one span that matches both those filters.
If the fields used in your filters exist across different spans in a trace, a result is not returned.
If possible, try propagating context throughout your traces to enable filters to return matching results.