Visualize Data Flow

Ent

This feature is available as part of the Honeycomb Enterprise plan.

This functionality is available only for teams using Honeycomb’s current data model. If you use Honeycomb Classic, we recommend migrating to Honeycomb Environments, so you can take advantage of its expanded data model and future product updates.

Honeycomb’s Service Map visualizes how traffic flows through your system in any given environment. It displays known services, service dependencies, and communication between services. Interact with the Service Map to examine and investigate smaller sets of services in detail.

What Service Map shows

Service Map generates a view of:

Services in your architecture and the request volume that each service receives
Which services communicate with each other and how often
p95 duration for requests to each service and for service-to-service calls

p95 stands for 95th percentile. A p95 duration means 95% of the requests in the data sample completed faster than that threshold, and 5% took longer.

Use cases

Service Map supports a few common investigation patterns:

Onboard engineers into a large complex architecture. Use the map to answer:
- What services exist in the system?
- What are the busiest services?
- Which services have the most dependencies?
- How often do two services communicate, relative to other service pairs?
- What are the slowest or fastest services in the system?
- What does this map look like for specific requests?
Debug an issue or run a root cause analysis. Use the map to answer:
- What downstream services are impacted?
- What upstream services may be causing this?
- What sample traces can you inspect to find the source of the problem?
Validate system observability and instrumentation quality. Use the map to answer:
- Are services instrumented correctly?
- Is instrumentation missing for part of the architecture?
- Are there unexpected instrumented components?

Want to see a working example of a Service Map at Honeycomb? Check out this interactive demo that requires no setup!

How Service Map works

Service Map derives its visualizations from the trace data that you send to Honeycomb, rather than from a live query over every event in your datasets.

The pipeline

When Honeycomb receives eligible spans, it works through a five-step process:

Identifies spans that include the required tracing fields.
Groups those spans into traces.
Walks each trace’s parent-child span tree.
Records cross-service calls as dependencies.
Uses those dependencies to draw the map for a selected time range.

In practice: if span A has service.name = frontend and a child span has service.name = checkout, Service Map draws an edge frontend → checkout.

Services and edges

A service is a node identified by the value of your dataset’s Service name definition. With OpenTelemetry, that’s typically service.name. An edge represents at least one observed call from a parent service to a child service in the selected time range. Services that call other services, or are called by other services, appear connected. Services that only communicate with themselves don’t create edges. Edge direction is caller → callee:

Parent span’s service is the caller
Child span’s service is the callee

Honeycomb derives an edge by walking each eligible trace:

Find each span’s parent using trace.parent_id.
Resolve the parent span’s service and the child span’s service.
If those services differ, record a dependency: parent service → child service.
Skip same-service parent/child pairs; intra-service calls don’t create edges.
Collapse known gateway/sidecar hops so the edge connects the real services on either side.

What doesn’t create an edge

Some spans and traces never produce an edge, even though they are present in your data. Knowing these exclusions up front saves time when a map looks incomplete and the cause turns out to be expected behavior, not missing instrumentation.

Spans missing required fields
Parent and child with the same service.name
Orphaned children whose parent span never arrived in the same trace
Span events and links (annotation types that aren’t full spans)
Traces excluded by Service Map sampling

External and entry traffic

Root spans (spans with no parent) can create inbound dependencies from outside your instrumented services. In the UI, these often appear related to Entry Points.

Gateways

By default, Service Map treats service.name in spans as a distinct service. For infrastructure with gateways, this can obscure true service to service relationships. If multiple services communicate through a gateway, the visual may appear as if some services speak exclusively to the gateway, or as if some services only receive traffic from the gateway. To ensure the relationship is not hidden in Service Map, you can adjust your instrumentation so Honeycomb can represent gateways accurately.

Sampling

Service Map is a sampled, dependency-oriented view of your system that is optimized for understanding call relationships. Service Map samples traces when generating dependencies. High-volume paths are represented more reliably than rare paths, so services with very low traffic in the selected time range may not appear even though they exist.

Large environments and high service cardinality

Service Map nodes are always keyed by service name. If your instrumentation registers thousands of distinct service.name values, for example one name per microservice, gRPC server, worker, or Kafka consumer, the map becomes hard to use. Current constraints:

The UI renders at most 300 services. Beyond that, it asks you to refine the map with filters instead.
Dependency queries default to 10,000 edges, with a maximum of 64,000. The UI also limits nodes to 5,000.

We recommend that you follow these practices to keep the map usable as your service count grows:

Filter first. Use service selection, isolate mode, and trace filters to inspect a subset of the system.
Choose intentional service names. Reserve service.name for the identity you want on the map. Put more granular identity in other attributes, such as k8s.deployment.name, k8s.pod.name, rpc.service, messaging.destination, or a custom app.component field.
Model journeys explicitly. Add a stable critical user journey/flow attribute on spans in the request path so you can filter the UI or query traces for that journey.
Build custom views when needed. For grouping by namespace, binary, or another label, query traces directly and reconstruct caller/callee relationships yourself.

Instrumenting for Service Map

Service Map requires tracing datasets in an Environment. To draw a meaningful map, the traces should span multiple services.

Required fields

To enter the Service Map pipeline, a span should include the fields listed in the table below (or equivalent fields mapped via Dataset Definitions). Most come from your instrumentation.

Recommended fields for richer filtering

Although these fields aren’t required to generate a service map, including them improves filtering and investigation in the UI:

Role	Typical field names	Description
Span Name	`name`	Operation the span represents, for example a function name or a request handler. Lets you filter by operation.
Route	`http.route`, `request_path`	URL path or endpoint pattern the span handled, for example `/api/v1/users/:id`. Lets you filter by endpoint/path.
Error	`error`, error-related fields	Boolean or string set on the span when the operation it represents failed. Lets you filter to failing paths.
Status Code	`http.status_code`, `http.response.status_code`	HTTP response code returned for the request, for example `200` or `500`. Lets you filter by response code.
User	`user.id`	Unique identifier of the end user associated with the span, set by your instrumentation. Lets you filter to a user journey.

Best practices

Follow these practices when instrumenting new services to keep Service Map accurate and easy to read as your architecture grows.

Use OpenTelemetry and propagate context across service boundaries so parent span IDs survive the hop.
Keep service.name stable and meaningful for dependency views.
Put high-cardinality instance identity in other attributes.
Add journey attributes early in the request path if you investigate by product flow.
Verify Dataset Definitions after changing field names.

Instrumenting for gateways

Honeycomb treats a service as a gateway when its spans include proxy instrumentation, commonly net.component = proxy (also recognized as component = proxy in some setups). Spans classified this way are collapsed on the map rather than shown as ordinary service nodes; they typically appear as gateway indicators on edges. You can optionally distinguish the deployment role with net.component.deployment = gateway or net.component.deployment = sidecar. If net.component is proxy and no deployment value is set, Honeycomb treats the hop as a gateway.

If a service “disappears” from the map but still has trace volume, check whether it is being classified as a gateway.

Without gateway instrumentation, a gateway hop appears as its own node. For example, if service A calls service B through a gateway named foo, an uninstrumented map shows A calling foo, and foo calling B, rather than the direct relationship between A and B. Without gateway instrumentation: With gateway instrumentation:

Istio service meshes and gateways

Istio automatically instruments component.proxy on each span. No additional manual instrumentation is required.

Other meshes and gateways

For other service meshes and gateways, instrument net.component: proxy on your gateway service spans:

OpenTelemetry Collector
Mesh trace configuration

When sending telemetry through an OpenTelemetry Collector, use the Attributes Processor to add the attribute to the pipeline that carries the mesh’s spans:

processors:
  batch:
  attributes:
    actions:
      - key: "net.component"
        value: "proxy"
        action: insert

From the service mesh or gateway, modify the meshConfig to include custom_tags:

spec:
  meshConfig:
    enableTracing: true
    defaultConfig:
      tracing:
        custom_tags:
          net.component:
            literal:
              value: proxy
        zipkin:
          address: otel-collector.default:9411

Troubleshooting

If you experience difficulties when working with Service Map, explore these solutions to common issues.

My map is empty

Confirm that your Dataset fields in Dataset Definitions are defined with Field name values. Service Map generates automatically from trace data sent to Honeycomb and, with the exception of timestamp, is based on these defined Tracing fields:

Role	OpenTelemetry/Honeycomb field names	Description
Service Name	`service.name`, `serviceName`, `service_name`	Name of the instrumented service. Becomes the map node identity.
Span ID	`trace.span_id`, `id`	Unique identifier of the span.
Timestamp	`time` (not mapped via Dataset Definitions)	Start time of the span. Places the dependency in the selected time range. With OpenTelemetry, your SDK sets this automatically. If a span arrives without a timestamp, Honeycomb defaults to the time of arrival.
Span Duration	`duration_ms`, `durationMs`	Amount of time in milliseconds that the span took to complete. Used for latency display. With OpenTelemetry, Honeycomb usually populates `duration_ms` from the span’s start and end timestamps during ingest. If you send spans through another path, confirm a duration field is present and mapped.
Parent Span ID	`trace.parent_id`, `parentId`	Unique identifier of the span’s parent span or the call location the current span was called from. Any child span must include a parent ID to be mapped properly; a span with no parent ID is treated as a root span rather than dropped.
Trace ID	`trace.trace_id`, `traceId`	Unique identifier of the trace that the span belongs to. Groups spans into one trace.

If any of this data is missing or undefined, the map will not display.

Check back in a few minutes to confirm if the Service Map has generated. Service Map may take up to a few minutes to generate after you start sending data to Honeycomb.

My map contains only one disconnected service

Make sure you are sending traces that include more than one service. Service Map generates based on trace data sent to Honeycomb and visualize edge connections between services. If your trace includes only one service, that singular service will display as one disconnected node.

My map contains multiple services, but one service is disconnected

Disconnected services have little or no cross-service dependency in the selected window. Common causes:

Cause	What to check
Incomplete traces	Child spans reference a `trace.parent_id` whose parent span never arrived
Missing parent IDs	Non-root spans lack `trace.parent_id`, so Honeycomb can’t join the call chain
Same-service-only traffic	Parent and child share the same `service.name`, so no edge is created
Sampling	Quiet paths weren’t selected into the dependency sample
Gateway collapse	Intermediate services are folded into gateway hops
Narrow time range	Cross-service calls happened outside the selected window

The diagrams below illustrate the most common cause: an incomplete trace. In a complete trace, each span’s trace.parent_id resolves to a real parent span, so Honeycomb can walk the full chain and record the cross-service edges. In an incomplete trace, a child span’s trace.parent_id points to a parent span that never arrived. Honeycomb can’t join that child to its parent, so the child appears disconnected, an island, instead of connected to the rest of the trace. Complete trace: Incomplete trace, missing parent:

Validating parent-child linkage in your traces

In Query Builder or your own scripts, investigate:

Required fields present. Count spans missing service.name, trace.trace_id, trace.span_id, or duration_ms.
Orphaned children. Spans with trace.parent_id set where no span in the same trace.trace_id has that ID as trace.span_id.
Cross-service children. For services that should be connected, confirm child spans exist whose parent span has a different service.name.
Roots that never call out. Root spans with no different-service descendants appear as disconnected or entry-only nodes.

A service is missing from my map

Confirm the service has spans in the time range. In Query Builder, run:

VISUALIZE COUNT
WHERE service.name = <service name>

If the returned count is 0, the map is correctly omitting it for that window. If the returned count is greater than zero:

Wait a few minutes for recently ingested traffic to appear
Confirm Dataset Definitions for service name, trace ID, span ID, parent span ID, and duration
Expand the time range to view a larger sample; sampling may hide low-volume services
Check whether the service is classified as a gateway (net.component: proxy or component: proxy attributes). In Query Builder, run:
VISUALIZE COUNT WHERE service.name = <service name> GROUP BY `component:proxy`, `net.component:proxy`
Confirm the span’s timestamp reflects when the call actually happened; clock skew or delayed export can push a span outside the selected time range even though the service has current trace volume

My instrumented gateways do not appear on my map

If you instrumented gateways and they still do not appear on your map, your gateway requests may not occur between known services. Service Map only displays gateways on an edge, which is formed when one service send requests to another service. If an edge does not exist, either because a service does not send requests to another service or because the service sends requests to another service through multiple gateways, then gateways may not display on your map. If either of these scenarios are common in your architecture, join #discuss-service-map in our Pollinators Community Slack and let us know.

All services point to the gateway service

Make sure you have instrumented for gateways, so Service Map can represent them appropriately.

The map shows too many services

Apply service filters, isolate a service, or filter traces to a journey attribute. If you routinely exceed hundreds of distinct service names, revisit how service.name is assigned in instrumentation.

My map is dense and hard to read

Narrow your map to a specific set of services using the Services or Filter Traces dropdowns. After narrowing down to a smaller diagram, hover over a specific service on the map and select Isolate to display all dependencies (incoming and outgoing services) for the selected service.

When I apply filters, an expected service or path does not display

When filter parameters are applied to the Service Map, the results return a maximum of 10,000 traces. These traces are then displayed on the Service Map when using Filters and/or selectively displayed when using the Service Filter. Because of the applied limit, sometimes very low volume traces may not be represented in the returned sample.

When I filter my map, a “No Service Map data found” error appears, but I found a trace that matches the filter set

Filters find traces with at least one span that matches all filters you have entered. For example, if the filters of app.cart.items > 5 AND app.user.currency = USD are set in Filter Traces, then the map displays traces that have at least one span that matches both filters. If the fields used in your filters exist across different spans in a trace, a result is not returned. If possible, try propagating context throughout your traces to enable filters to return matching results.

Sample traces are empty when I select a service

The right panel’s sample traces for a service are based on dependencies where that service appears as the callee (child) of an edge. You may see a node with no sample traces when:

The service only appears as a caller in the selected window; it calls others, but nothing calls it in the sampled dependencies
Active UI filters exclude all matching traces
Sampling or time range left no child-side dependencies for that service

Try:

Clearing trace filters
Widening the time range
Selecting an edge into or out of the service and inspecting edge sample traces
Using Isolate on the service to inspect neighbors

p95 latency for services and edges in Service Map do not match p95(duration_ms) in Query Builder

This difference is expected. Service Map calculates the p95 duration for services and edges differently from the p95(duration_ms) query in Query Builder. Service Map latency calculations include synchronous spans, which communicate between services. Query Builder latency calculations includes both synchronous and asynchronous, or internal, spans.

Investigate with AI

Observe

Query

Collaborate

Diagnose

What Service Map shows

Use cases

How Service Map works

The pipeline

Services and edges

What doesn’t create an edge

External and entry traffic

Gateways

Sampling

Large environments and high service cardinality

Instrumenting for Service Map

Required fields

Recommended fields for richer filtering

Best practices

Instrumenting for gateways

Istio service meshes and gateways

Other meshes and gateways

Troubleshooting

My map is empty

My map contains only one disconnected service

My map contains multiple services, but one service is disconnected

Validating parent-child linkage in your traces

A service is missing from my map

My instrumented gateways do not appear on my map

All services point to the gateway service

The map shows too many services

My map is dense and hard to read

When I apply filters, an expected service or path does not display

When I filter my map, a “No Service Map data found” error appears, but I found a trace that matches the filter set

Sample traces are empty when I select a service

p95 latency for services and edges in Service Map do not match p95(duration_ms) in Query Builder

​What Service Map shows

​Use cases

​How Service Map works

​The pipeline

​Services and edges

​What doesn’t create an edge

​External and entry traffic

​Gateways

​Sampling

​Large environments and high service cardinality

​Instrumenting for Service Map

​Required fields

​Recommended fields for richer filtering

​Best practices

​Instrumenting for gateways

​Istio service meshes and gateways

​Other meshes and gateways

​Troubleshooting

​My map is empty

​My map contains only one disconnected service

​My map contains multiple services, but one service is disconnected

​Validating parent-child linkage in your traces

​A service is missing from my map

​My instrumented gateways do not appear on my map

​All services point to the gateway service

​The map shows too many services

​My map is dense and hard to read

​When I apply filters, an expected service or path does not display

​When I filter my map, a “No Service Map data found” error appears, but I found a trace that matches the filter set

​Sample traces are empty when I select a service

​p95 latency for services and edges in Service Map do not match p95(duration_ms) in Query Builder

What Service Map shows

Use cases

How Service Map works

The pipeline

Services and edges

What doesn’t create an edge

External and entry traffic

Gateways

Sampling

Large environments and high service cardinality

Instrumenting for Service Map

Required fields

Recommended fields for richer filtering

Best practices

Instrumenting for gateways

Istio service meshes and gateways

Other meshes and gateways

Troubleshooting

My map is empty

My map contains only one disconnected service

My map contains multiple services, but one service is disconnected

Validating parent-child linkage in your traces

A service is missing from my map

My instrumented gateways do not appear on my map

All services point to the gateway service

The map shows too many services

My map is dense and hard to read

When I apply filters, an expected service or path does not display

When I filter my map, a “No Service Map data found” error appears, but I found a trace that matches the filter set

Sample traces are empty when I select a service

p95 latency for services and edges in Service Map do not match p95(duration_ms) in Query Builder