Skip to main content
This guide walks you through upgrading Refinery across major versions to access new features and improvements.

​
Upgrading to Refinery 3.0

Refinery 3.0 introduces significant improvements and includes breaking changes that require updating your configuration.
If you’re upgrading from Refinery 1.x, first upgrade to Refinery 2.x using the Configuration Conversion Tool to convert your file format, then follow these Refinery 3.0 upgrade instructions.

​
What’s New in Refinery 3.0

Upgrading your Refinery instance to 3.0 provides:
  • More efficient CPU and memory utilization
  • Enhanced cluster stability and reliability

​
Update Your Configuration

Update your Refinery configuration before upgrading to version 3.0. Your cluster will fail to start or behave as intended if you upgrade without updating your configuration first.

​
Options to Remove

Refinery 3.0 eliminates several configuration options that are now handled automatically or replaced with new features.
Trace Redistribution
Refinery 3.0 no longer supports redistributing in-progress traces during scaling events. Remove these options:
  • DisableRedistribution
  • RedistributionDelay
Outgoing Buffer Sizes
Refinery 3.0 automatically manages upstream and peer queue sizes. Remove these options:
  • PeerBufferSize
  • UpstreamBufferSize
Trace Locality Mode (Experimental)
Refinery 3.0 removes the Trace Locality experimental feature. Remove these options:
  • TraceLocalityMode
  • MaxDropDecisionBatchSize
  • DropDecisionSendInterval
  • MaxKeptDecisionBatchSize
  • KeptDecisionSendInterval

​
Options to Replace

Some Refinery 2.x settings have been renamed or redesigned in version 3.0.
CacheCapacity
Replace CacheCapacity with queue size options:
  1. Remove CacheCapacity from your configuration.
  2. Add IncomingQueueSize and PeerQueueSize (if they aren’t already present).
  3. Set both queue sizes to 3 times your previous CacheCapacity value. For example, if CacheCapacity was 1000, set both IncomingQueueSize and PeerQueueSize to 3000.
LegacyMetrics
Replace LegacyMetrics with OTelMetrics:
  1. Replace LegacyMetrics with OTelMetrics in your configuration.
  2. Keep the same configuration options. Both metrics types use identical settings.
Refinery deprecated LegacyMetrics in version 2.0 and removes it in version 3.0. Some metric names will change slightly after this update.

​
Update Your Rules

In your rules, replace any field condition using trace.parent_id (or a name listed in configuration under ParentNames) with the has-root-span operator.

​
Before

Rules:
  ...
  - Name: only when root span present
    ...
    Conditions:
      - Field: trace.parent_id
        Operator: not-exists

​
After

Rules:
  ...
  - Name: only when root span present
    ...
    Conditions:
      - Operator: has-root-span
        Value: true

​
Perform the Upgrade

After updating your configuration, upgrade your Refinery instances using your usual deployment method (such as Helm or Kubernetes). Deployment steps vary by environment, so follow your standard process to complete the upgrade.

​
Upgrading to Refinery 2.0

Refinery 2.0 introduced a new YAML-based configuration format and included breaking changes that require migrating your configuration. Use the Configuration Conversion Tool to convert your Refinery 1.x files (JSON, TOML, and YAML format) to the Refinery 2.x YAML format and preserve your settings.

​
What’s New in Refinery 2.0

Upgrading to Refinery 2.0 or later provides:
  • Improved .yaml-based configuration with better default values
  • New samplers: EMAThroughputSampler and WindowedThroughputSampler
  • Built-in configuration file validation
  • More accurate span counting (instead of trace counting) for dynamic samplers, making it easer to right-size sampling based on event volume
  • Improved metrics, including OpenTelemetry output
  • Support for emitting multiple metrics sources

​
Migrate Your Configuration

To migrate from Refinery 1.x to 2.x:
  1. Gather this information before you start:
    • Whether you use Helm with Kubernetes to manage your Refinery instance
    • The file paths for your existing Refinery config and rules files
    • Any custom comments in your existing configuration files that you want to preserve
  2. Install the Configuration Conversion Tool.
  3. Convert your configuration files using one of these methods:
    • Convert Helm values file (if you use Kubernetes with Helm)
    • Convert config and rules files (if you don’t use Helm)
  4. Review the converted files and add any custom comments you want to preserve.
    The Configuration Conversion Tool does not migrate custom comments from your existing files. Save any important comments separately and add them to your new files manually.
For detailed information about the tool’s capabilities, visit Configuration Conversion Tool.

​
Perform the Upgrade

Once you’ve converted and reviewed your configuration files, upgrade Refinery using your existing deployment tool or process (for example, Helm, Kubernetes, or a custom deployment script). Follow your environment’s standard procedure to complete the upgrade.

​
Configuration Conversion Tool

The Configuration Conversion Tool is a command line tool that helps you migrate Refinery configurations, validate configuration files, and generate local documentation. While primarily used for translating from Refinery 1.x to 2.x file format, you can also use it independently for validation and documentation tasks.

​
What It Does

The Configuration Conversion Tool:
  • Converts Refinery 1.x configuration files (JSON, TOML, or YAML) to the Refinery 2.x YAML format.
  • Validates Refinery configuration files automatically during conversion.
  • Generates local documentation in Markdown format for offline reference.
Refinery 2.x and 3.x use the same YAML configuration format. The conversion tool is only needed when upgrading from Refinery 1.x to 2.x or later.

​
How It Works

The tool maps variables from their Refinery 1.x locations to the appropriate Refinery 2.x locations. Any non-default values carry over to the new format if still applicable. Example: If your Refinery 1.x config file specifies MetricsAPIKey in the HoneycombMetrics section, the tool creates a Refinery 2.x config.yaml file with that key listed as APIKey in the LegacyMetrics section. The generated Refinery 2.x config.yaml files include helpful comments describing each configuration variable and its possible values.
The tool doesn’t migrate custom comments from your existing configuration files. Save any important comments separately and add them to your new files manually.

​
Installing the Conversion Tool

The configuration conversion tool is available as pre-built binaries for Linux, macOS, and Windows.
  • If you use Helm with Kubernetes: The configuration conversion tool installs automatically with Refinery 2.0 and later.
  • If you don’t use Helm: Download the configuration conversion tool from the Releases page in the Refinery GitHub repository. Find the single binary file for your operating system listed alongside each Refinery release, starting with Refinery 2.0.

​
Converting Configuration Files

Follow the instructions for your installation method:
Convert your Refinery Helm chart to the Refinery 2.x format:
convert helm --input /path/to/values.yaml --output /path/to/values.yaml
Command structure:
  • convert: Command used to convert the specified input file
  • helm: Indicates the conversion template to apply
  • --input /path/to/values.yaml: Required path to your existing Refinery 1.x values.yaml source file. Adjust the file type and path as needed.
  • --output /path/to/values.yaml: Required path that designates where the tool will save your new Refinery 2.x values.yaml destination file

​
Troubleshooting Conversion Errors

The conversion tool automatically validates your Refinery file during conversion. If the tool finds issues, it stops and displays descriptive error messages. For example, the following output could appear when multiple errors are encountered for the rules file: 
Validation Errors in rules file:
  Within sampler dataset1: field DynamicSampler.UseTraceLength must be a bool
  Within sampler dataset1: field DynamicSampler.ClearFrequency (1) must be a valid duration like '3m30s' or '100ms'
  Within sampler dataset1: unknown field DynamicSampler.SamplerRate; did you mean SampleRate?
  Within sampler dataset1: field DynamicSampler.FieldList must be a string array but contains non-string 10
  Within sampler dataset1: the group DynamicSampler is missing its required field SampleRate
To continue with migration:
  1. Fix the errors in your Refinery 1.x file.
  2. Run the conversion tool again.
For help resolving specific errors, visit the Refinery documentation for configuration (config.yaml) and sampling (rules.yaml).

​
Command Line Options

The tool supports these command line options: -i,--input : Path to the Refinery 1.x file to convert (default: config.toml) -o, --output : Path where the tool saves the converted Refinery 2.x file (default: stdout) -t Y, --type=Y : Indicates that the tool should treat the input file as YAML format. Use when the file extension doesn’t indicate format. -t T, --type=T : Indicates that the tool should treat the input file as TOML format. Use when the file extension doesn’t indicate format. -t J, --type=J : Indicates that the tool should load the input file as JSON format. Use when the file extension doesn’t indicate format. -h, --help : Display the help message.

​
Validating Configuration Files

Manually validate your Refinery configuration files, such as in a Continuous Integration (CI) process. (Refinery automatically validates files during the convert command.) To validate your Refinery config file against the Refinery 2.x format, use: 
convert validate config --input config.yaml
To validate your Refinery rules file against the Refinery 2.x format, use: 
convert validate rules --input rules.yaml
Command structure:
  • convert validate: Command used to validate the specified input file
  • config or rules: Indicates the appropriate validation template
  • --input: Required path to the file you want to validate

​
Generating Local Documentation

Generate local Refinery documentation in Markdown format for offline reference. To generate documentation for config.yaml, use: 
convert doc config
To generate documentation for rules.yaml, use: 
convert doc rules
To learn more, visit Refinery documentation for configuration (config.yaml) and sampling (rules.yaml).

​
Get Help

Join #discuss-refinery in the Pollinators Community Slack to ask questions and learn more. For Pro and Enterprise users, contact Support via support.honeycomb.io, or email at support@honeycomb.io