> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Send Data with the Honeycomb Distribution for Java

> Reference documentation for the Honeycomb Distribution for Java, a deprecated wrapper for the OpenTelemetry Java Agent. Migrate to the OpenTelemetry Java Agent instead.

<Warning>
  Honeycomb OpenTelemetry Distributions have reached [End of Life](/troubleshoot/product-lifecycle/release-stages/#end-of-life) and are now archived.

  * Adding new instrumentation? We recommend that you use the [OpenTelemetry Java Agent](/send-data/java/opentelemetry-agent/) instead.
  * Already using a Honeycomb distribution? We recommend that you [migrate to OpenTelemetry](/troubleshoot/product-lifecycle/recommended-migrations/migrate-from-honeycomb-distributions/).
</Warning>

Honeycomb provides the [Honeycomb OpenTelemetry Distribution for Java](https://github.com/honeycombio/honeycomb-opentelemetry-java) to help you instrument your applications and send telemetry data to Honeycomb as quickly and easily as possible.
Under the hood, the Honeycomb Distribution uses [OpenTelemetry for Java](https://opentelemetry.io/docs/languages/java/), so advanced users or those who have already instrumented their applications with OpenTelemetry do not need to use this Distribution.

The Honeycomb Distribution reads variables you provide and translates them to variables understood by upstream OpenTelemetry.
For example, the Honeycomb Distribution automatically converts the system property `honeycomb.traces.endpoint` to the OpenTelemetry system property `otel.exporter.otlp.traces.endpoint`.
If you want to [send data to Honeycomb using OpenTelemetry without the Honeycomb Distribution](/send-data/java/opentelemetry-agent/), you will need to configure your implementation to match variables expected by OpenTelemetry.

In this guide, we explain how to set up automatic and custom, or manual, instrumentation for a service written in Java.
If you prefer learning by example, we provide [several examples](https://github.com/honeycombio/honeycomb-opentelemetry-java/tree/main/examples/) of applications configured to send OpenTelemetry data to Honeycomb using the Honeycomb OpenTelemetry Distribution for Java.

## Before You Begin

Before you can set up instrumentation for your Java application, you will need to do a few things.

### Prepare Your Development Environment

To complete the required steps, you will need:

* A working Java environment with Java version 8 or higher
* An application written in Java

### Get Your Honeycomb API Key

To send data to Honeycomb, you'll need to [sign up for a free Honeycomb account](https://ui.honeycomb.io/signup) and [create a Honeycomb Ingest API Key](/configure/environments/manage-api-keys/#create-api-key).
To get started, you can create a key that you expect to swap out when you deploy to production.
Name it something helpful, perhaps noting that it's a getting started key.
Make note of your API key; for security reasons, you will not be able to see the key again, and you will need it later!

<Tip>
  For setup, make sure you check the "Can create datasets" checkbox so that your data will show up in Honeycomb. Later, when you replace this key with a permanent one, you can uncheck that box.
</Tip>

If you want to use an API key you previously stored in a secure location, you can also [look up details for Honeycomb API Keys](/configure/environments/manage-api-keys/#find-api-keys) any time in your Environment Settings, and use them to retrieve keys from your storage location.

## Add Automatic Instrumentation

Automatic instrumentation is handled with a Java Agent that runs alongside your application.
Additional manual instrumentation can be added using the OpenTelemetry API, which is available when using our SDK as a dependency.

If you do not want to use automatic instrumentation, you will need to use the [SDK Builder](#using-the-honeycomb-sdk-builder) to instantiate an `OpenTelemetryConfiguration` instance.
The Builder is not needed if you are using the Agent.

### Acquire Dependencies

The OpenTelemetry Java Agent supports many [Java libraries and frameworks](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries---frameworks).

The automatic instrumentation agent for Honeycomb OpenTelemetry Java will automatically generate trace data from your application.
The agent is packaged as a JAR file and is run alongside your app.

To use the automatic instrumentation agent, you must first download it:

```shell theme={}
curl -LO https://github.com/{honeycomb_opentelemetry_java.repo_name}/releases/download/v{honeycomb_opentelemetry_java.version}/honeycomb-opentelemetry-javaagent-{honeycomb_opentelemetry_java.version}.jar
```

### Configure

Choose from **one** of the following configuration options:

<Tabs>
  <Tab title="Environment variables">
    Use environment variables:

    ```ini theme={}
    # US instance
    HONEYCOMB_API_ENDPOINT=https://api.honeycomb.io:443
    # EU instance
    #HONEYCOMB_API_ENDPOINT=https://api.eu1.honeycomb.io:443
    SERVICE_NAME=my-favorite-service
    HONEYCOMB_API_KEY=my-api-key
    HONEYCOMB_METRICS_DATASET=my-metrics # optional
    ```
  </Tab>

  <Tab title="System properties">
    Use system properties:

    ```ini theme={}
    # US instance
    honeycomb.api.endpoint=https://api.honeycomb.io:443
    # EU instance
    #honeycomb.api.endpoint=https://api.eu1.honeycomb.io:443
    sample.rate=2
    service.name=my-favorite-service
    honeycomb.api.key=my-api-key
    honeycomb.metrics.dataset=my-metrics
    ```

    And set an environment variable with a path to your properties file:

    ```ini theme={}
    HONEYCOMB_CONFIG_FILE=/path/to/properties/file
    ```
  </Tab>
</Tabs>

<Note>
  If you use [Honeycomb Classic](/troubleshoot/product-lifecycle/recommended-migrations/#migrate-from-honeycomb-classic-to-honeycomb-environments), you must also specify the Dataset using the `HONEYCOMB_TRACES_DATASET` environment variable or `honeycomb.traces.dataset` system property.

  ```ini theme={}
  # environment variable
  HONEYCOMB_TRACES_DATASET=my-traces
  # system property
  honeycomb.traces.dataset=my-traces
  ```
</Note>

#### Advanced Configuration

You can configure the Honeycomb Agent using system properties or environment variables.
If both are specified, system properties will take precedence over environment variables.

Explore all configuration options for the Honeycomb Agent:

| System Property /<br /> Environment Variable                    | Default Value                                                                                      | Description                                                                                           |
| --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `honeycomb.api.key` <br />`HONEYCOMB_API_KEY`                   | None                                                                                               | **\[required -- see note below]** Your Honeycomb API key                                              |
| `service.name` <br />`SERVICE_NAME`                             | None                                                                                               | **\[required -- see note below]** `service.name` attribute, where all trace data is sent              |
| `honeycomb.traces.apikey` <br />`HONEYCOMB_TRACES_APIKEY`       | Value of `honeycomb.api.key`                                                                       | Your Honeycomb API key for sending traces                                                             |
| `honeycomb.metrics.apikey` <br />`HONEYCOMB_METRICS_APIKEY`     | Value of `honeycomb.api.key`                                                                       | Your Honeycomb API key for sending metrics                                                            |
| `honeycomb.metrics.dataset` <br />`HONEYCOMB_METRICS_DATASET`   | None                                                                                               | Honeycomb dataset where metrics will be sent (metrics will not be exported if this is not configured) |
| `honeycomb.api.endpoint` <br />`HONEYCOMB_API_ENDPOINT`         | `https://api.honeycomb.io:443` (US instance)<br />`https://api.eu1.honeycomb.io:443` (EU instance) | Honeycomb ingest endpoint.                                                                            |
| `honeycomb.traces.endpoint` <br />`HONEYCOMB_TRACES_ENDPOINT`   | Value of `honeycomb.api.endpoint`                                                                  | Honeycomb ingest endpoint for traces (defaults to the value of `HONEYCOMB_API_ENDPOINT`).             |
| `honeycomb.metrics.endpoint` <br />`HONEYCOMB_METRICS_ENDPOINT` | Value of `honeycomb.api.endpoint`                                                                  | Honeycomb ingest endpoint for metrics (defaults to the value of `HONEYCOMB_API_ENDPOINT`).            |
| `sample.rate` <br />`SAMPLE_RATE`                               | `1` (retain all data)                                                                              | Sample rate for the deterministic sampler. Must be a positive integer.                                |

To learn more about agent configuration, visit [Java Agent Configuration](https://opentelemetry.io/docs/languages/java/automatic/agent-config/) in the OpenTelemetry documentation, which contains details about configuring and suppressing specific instrumentations.

<Note>
  If you are sending data to Honeycomb directly, you must configure API key and service name.
  If you are using an [OpenTelemetry Collector](/send-data/opentelemetry/collector/), you can configure your API key in the collector instead.
</Note>

### Run

Run your application with the automatic instrumentation agent as a sidecar:

```shell theme={}
java -javaagent:honeycomb-opentelemetry-javaagent-{honeycomb_opentelemetry_java.version}.jar -jar /path/to/myapp.jar
```

You can also include configuration values with an invocation of your application:

```shell theme={}
java \
-Dhoneycomb.config.file=/path/to/properties/file \
-javaagent:honeycomb-opentelemetry-javaagent-{honeycomb_opentelemetry_java.version}.jar -jar /path/to/myapp.jar
```

For more configuration options, visit our documentation on the [OpenTelemetry Java Agent](/send-data/java/opentelemetry-agent/).

In Honeycomb's UI, you should now see your application's incoming requests and outgoing HTTP calls generate traces.

## Add Custom Instrumentation

Automatic instrumentation is the easiest way to get started with instrumenting your code.
To get additional insight into your system, you should also add custom, or manual, instrumentation where appropriate.
You can use manual instrumentation whether you are using the Agent or the Builder.

To learn more about custom, or manual, instrumentation, visit the comprehensive set of topics covered by [Manual Instrumentation for Java](https://opentelemetry.io/docs/languages/java/manual/), including [the Annotations API](https://opentelemetry.io/docs/languages/java/automatic/annotations/), in OpenTelemetry's documentation.

### Add Dependencies

To add custom instrumentation, add the Honeycomb OpenTelemetry SDK as a dependency in your project.
This provides access to the OpenTelemetry APIs, which lets you access the currently executing span and add attributes to it, and/or to create new spans.

<Tabs>
  <Tab title="Gradle">
    ```groovy theme={}
    dependencies {
      implementation('io.honeycomb:honeycomb-opentelemetry-sdk:{honeycomb_opentelemetry_java.version}')
    }
    ```
  </Tab>

  <Tab title="Maven">
    ```xml theme={}
    <dependency>
      <groupId>io.honeycomb</groupId>
      <artifactId>honeycomb-opentelemetry-sdk</artifactId>
      <version>{honeycomb_opentelemetry_java.version}</version>
    </dependency>
    ```
  </Tab>
</Tabs>

### Add Attributes to Spans

Adding attributes to a currently executing span in a trace can be useful.
For example, you may have an application or service that handles users and you want to associate the user with the span when querying your service in Honeycomb.
To do this, get the current span from the context and set an attribute with the user ID.

In your code, import `io.opentelemetry.api.trace.Span` to get access to the span:

```java theme={}
import io.opentelemetry.api.trace.Span;

...

Span span = Span.current();
span.setAttribute("user.id", user.getId());
```

This will add a `user.id` field to the current span so that you can use the field in `WHERE`, `GROUP BY` or `ORDER` clauses in the Honeycomb query builder.

### Acquire a Tracer

To create spans, you need to get a `Tracer`.

```java theme={}
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.trace.Tracer;

//...

Tracer tracer = GlobalOpenTelemetry.getTracer("tracer.name.here");
```

When you create a `Tracer`, OpenTelemetry requires you to give it a name as a string.
This string is the only required parameter.

When traces are sent to Honeycomb, the name of the `Tracer` is turned into the `library.name` field, which can be used to show all spans created from a particular tracer.

In general, pick a name that matches the appropriate scope for your traces.
If you have one tracer for each service, then use the service name.
If you have multiple tracers that live in different "layers" of your application, then use the name that corresponds to that "layer".

The `library.name` field is also used with traces created from instrumentation libraries.

### Create New Spans

Automatic instrumentation can show the shape of requests to your system, but only you know the really important parts.
To get the full picture of what's happening, you will have to add custom, or manual, instrumentation and create some custom spans.
To do this, create or re-use `Tracer` registered by the Agent and start a span.

In your code, import `io.opentelemetry.api.GlobalOpenTelemetry`, `io.opentelemetry.api.trace.Span`, and `io.opentelemetry.api.trace.Tracer`:

```java theme={}
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;

...

Tracer tracer = GlobalOpenTelemetry.getTracer("my-service");
Span span = tracer.spanBuilder("expensive-query").startSpan();

// ... do cool stuff

span.end();
```

#### Creating Spans Around Methods

You can also use the annotation `@WithSpan` to wrap the execution of a method with a span.
The span will be automatically closed once the method has completed.
Unless explicitly specified, the span will be named `className.methodName`.
To override the name of the span, add a name in parentheses as an argument.

In your code, import `io.opentelemetry.instrumentation.annotations.WithSpan` to allow usage of this annotation:

```java theme={}
import io.opentelemetry.instrumentation.annotations.WithSpan;

...

@WithSpan("importantSpan")
public String getImportantInfo() {
    return importantInfo;
}
```

### Add Multi-Span Attributes

Sometimes you want to add the same attribute to many spans within the same trace.
This attribute may include variables calculated during your program, or other useful values for correlation or debugging purposes.

To add this attribute, leverage the OpenTelemetry concept of [baggage](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/overview.md#baggage-signal).
Baggage allows you to add a `key` with a `value` as an attribute to every subsequent child span of the current application context.

In your code, import `io.opentelemetry.api.baggage.Baggage` to allow use of the `Baggage` class:

```java theme={}
import io.opentelemetry.api.baggage.Baggage;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.context.Scope;

...

try (final Scope ignored = Baggage.current()
    .toBuilder()
    .put("app.username", name)
    .build()
    .makeCurrent()
) {
    // all subsequently created span in this block will have the `app.username` attribute
}
```

<Note>
  Any Baggage attributes that you set in your application will be attached to outgoing network requests as a header. If your service communicates to a third party API, do **NOT** put sensitive information in the Baggage attributes.
</Note>

## Sampling

You can use [deterministic head sampling](/manage-data-volume/sample/) with the Java Agent, with or without custom instrumentation.

The agent will read these variables and expect an integer that represents the sample rate you would like to apply.
For example, a value of `5` means that one out of every five traces will be sent to Honeycomb.

To add sampling to the agent, specify the `SAMPLE_RATE` environment variable or the `-Dsample.rate` system property when invoking the Java Virtual Machine (JVM), as shown in [Advanced Configuration](#advanced-configuration).

The value of your sample rate must be a positive integer.

If you have multiple services that communicate with each other, it is important that they have the same sampling configuration.
Otherwise, each service might make a different sampling decision, resulting in incomplete or broken traces.
You can sample using a standalone proxy as an alternative, like [Honeycomb Refinery](/manage-data-volume/sample/honeycomb-refinery/), or when you have more robust sampling needs.

## Metrics Collection

The JVM emits metrics at regular intervals that can be sent automatically to a Honeycomb dataset.
These metrics will show up in data as `runtime.jvm.gc.count`, `runtime.jvm.gc.time`, `runtime.jvm.memory.area`, and so on.
If an instrumentation library also supports emitting metrics, then those metrics will be sent as well.

Metrics are currently disabled by default.
To enable metrics, set a `HONEYCOMB_METRICS_DATASET` environment variable.
Optionally, set a specific `HONEYCOMB_METRICS_APIKEY` or `HONEYCOMB_METRICS_ENDPOINT` (if different from the traces API key and endpoint), as shown in [Advanced Configuration](#advanced-configuration).

<Tip>
  Metrics sent by the Java automatic instrumentation will consume a significant portion of a team's event quota, especially if on the Free tier.
  For JVM metrics, we recommend either:

  * [suppressing specific automatic instrumentation as needed](https://opentelemetry.io/docs/languages/java/automatic/configuration/#suppressing-specific-auto-instrumentation)
  * configuring an [OpenTelemetry Collector](/send-data/opentelemetry/collector/) as part of your pipeline that sends data to Honeycomb.
    With the OpenTelemetry Collector, you can filter metrics and aggregate them into longer intervals, which reduces the quantity of reports to Honeycomb
</Tip>

## Distributed Trace Propagation

When a service calls another service, you want to ensure that the relevant trace information is propagated from one service to the other.
This allows Honeycomb to connect the two services in a trace.

Distributed tracing enables you to trace and visualize interactions between multiple instrumented services.
For example, your users may interact with a front-end API service, which talks to two internal APIs to fulfill their request.
In order to have traces connect spans for all these services, it is necessary to propagate trace context between these services, usually by using an HTTP header.

Both the sending and receiving service must use the same propagation format, and both services must be configured to send data to the same Honeycomb environment.

## Using the Honeycomb SDK Builder

If you do not wish to use the automatic instrumentation agent, and/or wish to do more advanced instrumentation like adding other span processors or extensions, you can use the Honeycomb SDK to instantiate an `OpenTelemetryConfiguration` instance, which provides access to a `Tracer` and the rest of the OpenTelemetry API.

You should configure this as early as possible in the entry point of your application.
This builder is not required if the agent is in use.

Aside from requiring this builder, [setting up manual instrumentation](#add-custom-instrumentation) will be the same as when using the agent.

The `OpenTelemetryConfiguration` builder provides convenient methods for configuration:

```java theme={}
import io.honeycomb.opentelemetry.OpenTelemetryConfiguration;
import io.honeycomb.opentelemetry.sdk.trace.samplers.DeterministicTraceSampler; // optional
import io.honeycomb.opentelemetry.sdk.trace.spanprocessors.BaggageSpanProcessor; // optional

public OpenTelemetry honeycomb() {
  return OpenTelemetryConfiguration.builder()
    .setApiKey("your-api-key")
    .setServiceName("your-service-name")
    .setOtlpProtocol("http/protobuf") // optional - use HTTP instead of gRPC to transmit telemetry
    .setSampler(new DeterministicTraceSampler(2)) // optional - sends 1/2 of all traces generated
    .addSpanProcessor(new BaggageSpanProcessor()) // optional
    .buildAndRegisterGlobal();
}
```

When using the Honeycomb SDK builder, specify a span processor and/or a sampler with a sample rate as an argument to the constructor.

The `BaggageSpanProcessor` is available as part of the Honeycomb SDK; it is what allows for [multi-span attributes](#add-multi-span-attributes).
The `DeterministicTraceSampler` is available as part of the Honeycomb SDK that samples based on a common field, given a [sample rate](#sampling).

The `setSampler` method accepts any instance of a class that implements the [`io.opentelemetry.sdk.trace.samplers.Sampler`](https://javadoc.io/doc/io.opentelemetry/opentelemetry-sdk-trace/latest/io/opentelemetry/sdk/trace/samplers/Sampler.html) interface.
Look at the [provided deterministic sampler](https://github.com/honeycombio/honeycomb-opentelemetry-java/blob/main/common/src/main/java/io/honeycomb/opentelemetry/sdk/trace/samplers/DeterministicTraceSampler.java) as an example of how to create your own custom sampler, if needed.

## Including the Java Agent as Part of Build Process

Another option for using the agent is to include it when running Maven targets or Gradle tasks.
For example, here is a possible configuration to attach the Java agent when running a Spring Boot app:

<Tabs>
  <Tab title="Gradle">
    ```groovy theme={}
    bootRun {
        environment = ["HONEYCOMB_API_KEY": System.getenv("HONEYCOMB_API_KEY"),
                       "OTEL_SERVICE_NAME": System.getenv("OTEL_SERVICE_NAME")]
        jvmArgs = ["-javaagent:honeycomb-opentelemetry-javaagent-{honeycomb_opentelemetry_java.version}.jar"]
    }
    ```
  </Tab>

  <Tab title="Maven">
    ```xml theme={}
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <configuration>
                        <environmentVariables>
                            <OTEL_SERVICE_NAME>${env.OTEL_SERVICE_NAME}</OTEL_SERVICE_NAME>
                            <HONEYCOMB_API_KEY>${env.HONEYCOMB_API_KEY}</HONEYCOMB_API_KEY>
                        </environmentVariables>
                        <agents>
                            <agent>
                                honeycomb-opentelemetry-javaagent-{honeycomb_opentelemetry_java.version}.jar
                            </agent>
                        </agents>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    ```
  </Tab>
</Tabs>

Be sure to set the environment variables for `HONEYCOMB_API_KEY` and `OTEL_SERVICE_NAME` as described in the [Configure step](#configure) above.

## Using HTTP/protobuf instead of gRPC

By default, the Honeycomb OpenTelemetry SDK uses the gRPC protocol to send telemetry data.
To use HTTP instead of gRPC, update the protocol using one of the configuration methods:

* System property: `-Dotel.exporter.otlp.protocol=http/protobuf`
* Environment variable: `export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf`

  <Tip>
    If using the signal-specific `OTEL_EXPORTER_OTLP_<SIGNAL>_ENDPOINT` environment variable, you must append the endpoint with the appropriate signal path.
    For example, if sending traces, append the endpoint with `v1/traces`.
    If sending metrics, append the endpoint with `v1/metrics`.
  </Tip>

## Troubleshooting

To explore common issues when sending data, visit [Common Issues with Sending Data in Honeycomb](/troubleshoot/common-issues/sending-data/#opentelemetry-sdks-and-honeycomb-distributions).