> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Upgrading Refinery
> Upgrade Refinery across major versions to access new features and improvements. Use the configuration conversion tool to update your existing config files.
This guide walks you through upgrading Refinery across major versions to access new features and improvements.
## Upgrading to Refinery 3.0
Refinery 3.0 introduces significant improvements and includes breaking changes that require updating your configuration.
If you're upgrading from Refinery 1.x, first [upgrade to Refinery 2.x](#upgrading-to-refinery-20) using the [Configuration Conversion Tool](#configuration-conversion-tool) to convert your file format, then follow these Refinery 3.0 upgrade instructions.
### What's New in Refinery 3.0
Upgrading your Refinery instance to 3.0 provides:
* More efficient CPU and memory utilization
* Enhanced cluster stability and reliability
### Update Your Configuration
Update your Refinery configuration before upgrading to version 3.0.
Your cluster will fail to start or behave as intended if you upgrade without updating your configuration first.
#### Options to Remove
Refinery 3.0 eliminates several configuration options that are now handled automatically or replaced with new features.
##### Trace Redistribution
Refinery 3.0 no longer supports redistributing in-progress traces during scaling events.
Remove these options:
* `DisableRedistribution`
* `RedistributionDelay`
##### Outgoing Buffer Sizes
Refinery 3.0 automatically manages upstream and peer queue sizes.
Remove these options:
* `PeerBufferSize`
* `UpstreamBufferSize`
##### Trace Locality Mode (Experimental)
Refinery 3.0 removes the Trace Locality experimental feature.
Remove these options:
* `TraceLocalityMode`
* `MaxDropDecisionBatchSize`
* `DropDecisionSendInterval`
* `MaxKeptDecisionBatchSize`
* `KeptDecisionSendInterval`
#### Options to Replace
Some Refinery 2.x settings have been renamed or redesigned in version 3.0.
##### CacheCapacity
Replace `CacheCapacity` with queue size options:
1. Remove `CacheCapacity` from your configuration.
2. Add `IncomingQueueSize` and `PeerQueueSize` (if they aren't already present).
3. Set both queue sizes to 3 times your previous `CacheCapacity` value.
For example, if `CacheCapacity` was `1000`, set both `IncomingQueueSize` and `PeerQueueSize` to `3000`.
##### LegacyMetrics
Replace `LegacyMetrics` with `OTelMetrics`:
1. Replace `LegacyMetrics` with `OTelMetrics` in your configuration.
2. Keep the same configuration options.
Both metrics types use identical settings.
Refinery deprecated `LegacyMetrics` in version 2.0 and removes it in version 3.0.
Some metric names will change slightly after this update.
### Update Your Rules
In your rules, replace any field condition using `trace.parent_id` (or a name listed in configuration under [ParentNames](/manage-data-volume/sample/honeycomb-refinery/configure/#parentnames)) with the `has-root-span` operator.
#### Before
```yaml theme={}
Rules:
...
- Name: only when root span present
...
Conditions:
- Field: trace.parent_id
Operator: not-exists
```
#### After
```yaml theme={}
Rules:
...
- Name: only when root span present
...
Conditions:
- Operator: has-root-span
Value: true
```
### Perform the Upgrade
After updating your configuration, upgrade your Refinery instances using your usual deployment method (such as Helm or Kubernetes).
Deployment steps vary by environment, so follow your standard process to complete the upgrade.
## Upgrading to Refinery 2.0
Refinery 2.0 introduced a new YAML-based configuration format and included breaking changes that require migrating your configuration.
Use the [Configuration Conversion Tool](#configuration-conversion-tool) to convert your Refinery 1.x files (JSON, TOML, and YAML format) to the Refinery 2.x YAML format and preserve your settings.
### What's New in Refinery 2.0
Upgrading to Refinery 2.0 or later provides:
* Improved `.yaml`-based configuration with better default values
* New samplers: `EMAThroughputSampler` and `WindowedThroughputSampler`
* Built-in configuration file validation
* More accurate span counting (instead of trace counting) for dynamic samplers, making it easer to right-size sampling based on event volume
* Improved metrics, including OpenTelemetry output
* Support for emitting multiple metrics sources
### Migrate Your Configuration
To migrate from Refinery 1.x to 2.x:
1. Gather this information before you start:
* Whether you use Helm with Kubernetes to manage your Refinery instance
* The file paths for your existing Refinery config and rules files
* Any custom comments in your existing configuration files that you want to preserve
2. [Install the Configuration Conversion Tool](#installing-the-conversion-tool).
3. [Convert your configuration files](#converting-configuration-files) using one of these methods:
* Convert Helm values file (if you use Kubernetes with Helm)
* Convert config and rules files (if you don't use Helm)
4. Review the converted files and add any custom comments you want to preserve.
The Configuration Conversion Tool does not migrate custom comments from your existing files.
Save any important comments separately and add them to your new files manually.
For detailed information about the tool's capabilities, visit [Configuration Conversion Tool](#configuration-conversion-tool).
### Perform the Upgrade
Once you’ve converted and reviewed your configuration files, upgrade Refinery using your existing deployment tool or process (for example, Helm, Kubernetes, or a custom deployment script).
Follow your environment’s standard procedure to complete the upgrade.
## Configuration Conversion Tool
The Configuration Conversion Tool is a command line tool that helps you migrate Refinery configurations, validate configuration files, and generate local documentation.
While primarily used for translating from Refinery 1.x to 2.x file format, you can also use it independently for validation and documentation tasks.
### What It Does
The Configuration Conversion Tool:
* Converts Refinery 1.x configuration files (JSON, TOML, or YAML) to the Refinery 2.x YAML format.
* Validates Refinery configuration files automatically during conversion.
* Generates local documentation in Markdown format for offline reference.
Refinery 2.x and 3.x use the same YAML configuration format.
The conversion tool is only needed when upgrading from Refinery 1.x to 2.x or later.
### How It Works
The tool maps variables from their Refinery 1.x locations to the appropriate Refinery 2.x locations.
Any non-default values carry over to the new format if still applicable.
Example: If your Refinery 1.x `config` file specifies `MetricsAPIKey` in the `HoneycombMetrics` section, the tool creates a Refinery 2.x `config.yaml` file with that key listed as `APIKey` in the `LegacyMetrics` section.
The generated Refinery 2.x `config.yaml` files include helpful comments describing each configuration variable and its possible values.
The tool doesn't migrate custom comments from your existing configuration files.
Save any important comments separately and add them to your new files manually.
### Installing the Conversion Tool
The configuration conversion tool is available as pre-built binaries for Linux, macOS, and Windows.
* **If you use Helm with Kubernetes:** The configuration conversion tool installs automatically with Refinery 2.0 and later.
* **If you don't use Helm:** Download the configuration conversion tool from the [Releases page](https://github.com/honeycombio/refinery/releases) in the Refinery GitHub repository.
Find the single binary file for your operating system listed alongside each Refinery release, starting with Refinery 2.0.
### Converting Configuration Files
Follow the instructions for your installation method:
Convert your Refinery Helm chart to the Refinery 2.x format:
```shell theme={}
convert helm --input /path/to/values.yaml --output /path/to/values.yaml
```
Command structure:
* `convert`: Command used to convert the specified input file
* `helm`: Indicates the conversion template to apply
* `--input /path/to/values.yaml`: Required path to your existing Refinery 1.x `values.yaml` source file.
Adjust the file type and path as needed.
* `--output /path/to/values.yaml`: Required path that designates where the tool will save your new Refinery 2.x `values.yaml` destination file
If you don't use Helm, convert your Refinery `config` and `rules` files separately.
#### Convert Your Config File
Convert your Refinery `config` file to the Refinery 2.x format:
```shell theme={}
convert config --input /path/to/config.toml --output /path/to/config.yaml
```
Command structure:
* `convert`: Command used to convert the specified input file
* `config`: Indicates the conversion template to apply
* `--input /path/to/config.toml`: Required path to your existing Refinery 1.x config source file.
Adjust the file type and path as needed.
* `--output /path/to/config.yaml`: Required path that designates where the tool will save your new Refinery 2.x config destination file
The tool automatically detects the input file type based on the file extension.
Override this by adding the [`--type` option](#command-line-options).
#### Convert Your Rules File
Convert your Refinery `rules` file to the Refinery 2.x format:
```shell theme={}
convert rules --input /path/to/rules.toml --output /path/to/rules.yaml
```
Command structure:
* `convert`: Command used to convert the specified input file
* `rules`: Indicates the conversion template to apply
* `--input /path/to/config.toml`: Required path to your existing Refinery 1.x rules source file.
Adjust the file type and path as needed.
* `--output /path/to/config.yaml`: Required path that designates where the tool will save your new Refinery 2.x rules destination file
The tool automatically detects the input file type based on the extension.
Override this by adding the [`--type` option](#command-line-options).
### Troubleshooting Conversion Errors
The conversion tool automatically validates your Refinery file during conversion.
If the tool finds issues, it stops and displays descriptive error messages.
For example, the following output could appear when multiple errors are encountered for the `rules` file:
```bash theme={}
Validation Errors in rules file:
Within sampler dataset1: field DynamicSampler.UseTraceLength must be a bool
Within sampler dataset1: field DynamicSampler.ClearFrequency (1) must be a valid duration like '3m30s' or '100ms'
Within sampler dataset1: unknown field DynamicSampler.SamplerRate; did you mean SampleRate?
Within sampler dataset1: field DynamicSampler.FieldList must be a string array but contains non-string 10
Within sampler dataset1: the group DynamicSampler is missing its required field SampleRate
```
To continue with migration:
1. Fix the errors in your Refinery 1.x file.
2. Run the conversion tool again.
For help resolving specific errors, visit the Refinery documentation for [configuration (`config.yaml`)](/manage-data-volume/sample/honeycomb-refinery/configure/) and [sampling (`rules.yaml`)](/manage-data-volume/sample/honeycomb-refinery/sampling-methods/).
### Command Line Options
The tool supports these command line options:
`-i`,`--input`
: Path to the Refinery 1.x file to convert (default: `config.toml`)
`-o`, `--output`
: Path where the tool saves the converted Refinery 2.x file (default: `stdout`)
`-t Y`, `--type=Y`
: Indicates that the tool should treat the input file as YAML format.
Use when the file extension doesn't indicate format.
`-t T`, `--type=T`
: Indicates that the tool should treat the input file as TOML format.
Use when the file extension doesn't indicate format.
`-t J`, `--type=J`
: Indicates that the tool should load the input file as JSON format.
Use when the file extension doesn't indicate format.
`-h`, `--help`
: Display the help message.
### Validating Configuration Files
Manually validate your Refinery configuration files, such as in a Continuous Integration (CI) process.
(Refinery automatically validates files during the `convert` command.)
To validate your Refinery `config` file against the Refinery 2.x format, use:
```shell theme={}
convert validate config --input config.yaml
```
To validate your Refinery `rules` file against the Refinery 2.x format, use:
```shell theme={}
convert validate rules --input rules.yaml
```
Command structure:
* `convert validate`: Command used to validate the specified input file
* `config` or `rules`: Indicates the appropriate validation template
* `--input`: Required path to the file you want to validate
### Generating Local Documentation
Generate local Refinery documentation in Markdown format for offline reference.
To generate documentation for `config.yaml`, use:
```shell theme={}
convert doc config
```
To generate documentation for `rules.yaml`, use:
```shell theme={}
convert doc rules
```
To learn more, visit Refinery documentation for [configuration (`config.yaml`)](/manage-data-volume/sample/honeycomb-refinery/configure/) and [sampling (`rules.yaml`)](/manage-data-volume/sample/honeycomb-refinery/sampling-methods/).
## Get Help
Join `#discuss-refinery` in the Pollinators Community Slack to ask questions and learn more.
For Pro and Enterprise users, contact Support via [support.honeycomb.io](https://support.honeycomb.io/), or email at [support@honeycomb.io](mailto:support@honeycomb.io)