> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Common Issues with Honeycomb Refinery

> Troubleshoot common Refinery problems, including configuration errors, sampling rule issues, cluster connectivity, and data not reaching Honeycomb as expected.

Troubleshoot common issues with [Honeycomb Refinery](/manage-data-volume/sample/honeycomb-refinery/).

<Tip>
  To ask questions and learn more, visit our [Support Knowledge Base](https://support.honeycomb.io/) or [join our Pollinators Community](/troubleshoot/community/).
</Tip>

## Debug Logs

The default logging level is `warn`.
The `debug` level emits too much data to be used in production, but contains excellent information in a pre-production environment.
Setting the logging level to `debug` during initial configuration will help you understand what is working and what is not, but when traffic volumes increase it should be set to `warn`.

## Restarts

Refinery does not yet buffer traces or sampling decisions to disk.
When you restart the process, all in-flight traces will be flushed and sent upstream to Honeycomb, but you will lose the record of past trace decisions.
When started back up, Refinery will start with a clean slate.

## Configuration

Configuration file formats (TOML and YAML) can be confusing to read and write.
There is an option to check the loaded configuration by using one of the `/query` endpoints from the command line, from a server that can access a Refinery host.

The `/query` endpoints are protected and can be enabled by specifying `QueryAuthToken` in the configuration file or specifying `REFINERY_QUERY_AUTH_TOKEN` in Refinery's environment.
All requests to any `/query` endpoint must include the header `X-Honeycomb-Refinery-Query` set to the value of the specified token.

### Checking All Rules

Retrieve the entire rules configuration in the desired format from Refinery:

```shell theme={}
curl --get $REFINERY_HOST/query/allrules/$FORMAT --header "x-honeycomb-refinery-query: my-local-token"
```

where:

* `$REFINERY_HOST` should be the url of your refinery.
* `$FORMAT` can be one of `json`, `yaml`, or `toml`.

### Checking Rules for a Given Context

Retrieve the rule set that Refinery uses for the environment (or dataset in Classic Mode) defined in the variable `$ENVIRON`:

```shell theme={}
curl --get $REFINERY_HOST/query/rules/$FORMAT/$ENVIRON --header "x-honeycomb-refinery-query: my-local-token"
```

where:

* `$REFINERY_HOST` should be the URL of your refinery.
* `$FORMAT` can be one of `json`, `yaml`, or `toml`.
* `$ENVIRON` is the name of the environment (or dataset, in Classic Mode).

The response contains a map of the sampler type to its rule set.

### Checking Configuration Metadata

Retrieve information about the configurations currently in use, including the timestamp when the configuration was last loaded:

```bash theme={}
curl --include --get $REFINERY_HOST/query/configmetadata --header "x-honeycomb-refinery-query: my-local-token"
```

where:

* `$REFINERY_HOST` should be the URL of your refinery.

The response contains a JSON blob of information about Refinery's configurations.
It will look something like this:

```json theme={}
[
  {
    "type": "config",
    "id": "tools/loadtest/config.yaml",
    "hash": "1047bb6140b487ecdb0745f3335b6bc3",
    "loaded_at": "2022-11-08T22:24:18-05:00"
  },
  {
    "type": "rules",
    "id": "tools/loadtest/rules.yaml",
    "hash": "2d88389e1ff6530fba53466973e591e0",
    "loaded_at": "2022-11-08T22:24:18-05:00"
  }
]
```

For file-based configurations (the only type currently supported), the `hash` value is identical to the value generated by the `md5sum` command available in major operating systems.

## Sampling Decisions

Refinery can send telemetry that includes information that can help debug the sampling decisions that are made.
To enable it, in the configuration file, set `AddRuleReasonToTrace` to `true`.
This will cause traces that are sent to Honeycomb to include a field `meta.refinery.reason`.
This field will contain text indicating which rule was evaluated that caused the trace to be included.

## Rules Datatypes

The rules comparisons in Refinery's Rules-Based Sampler take the datatype of the fields into account.
In particular, a rule that compares `status_code` to `200` (an integer) will fail if the status code is actually `"200"` (a string), and vice-versa.
In a mixed environment where either datatype may be included in the telemetry, you should create a separate rule for each case.

This situation can be hard to diagnose, because Honeycomb's backend converts all the values of a given field to the datatype specified in the [dataset schema](/reference/honeycomb-ui/manage-data/datasets/dataset-settings/).
Inspection of the data in Honeycomb will not give any indication that this has happened.
If you see rules that appear to not execute when they should have, please consider this possibility of incorrect datatype.

## Health Checks

Use health check API endpoints to determine if an instance is bootstrapped.
The Refinery cluster machines respond to two different health check endpoints via HTTP:

### `/alive`

This `/alive` API call will return a `200` JSON response.
It does not perform any checks beyond the web server's response to requests.

```json theme={}
{
  "source": "refinery",
  "alive": "yes"
}
```

### `/x/alive`

This `/x/alive` API call will return a `200` JSON response that has been proxied from the Honeycomb API.
This can be used to determine if the instance is able to communicate with Honeycomb.

```json theme={}
{
  "alive": "yes"
}
```

### gRPC Health Check

If gRPC is configured, Refinery also responds to [a standard gRPC Health Probe](https://github.com/grpc/grpc/blob/592bfb9c0040fba382e5dc106e2ebbf0c96465fb/doc/health-checking.md).