Starting with Refinery 2.0, there is a breaking change that requires migration to take advantage of new and improved Refinery features. Use our configuration conversion tool to convert your existing configuration and to preserve important settings.
Benefits of upgrading your Refinery instance to 2.0 or later include:
.yaml-based configuration with better default valuesEMAThroughputSampler and WindowedThroughputSampler
Our configuration conversion tool is a command line tool.
The tool maps the variables in your existing Refinery 1.x config and rules files and translates them to the .yaml format.
Any non-default values from their Refinery 1.x location are copied to the appropriate Refinery 2.x location if still applicable.
For example, if the HoneycombMetrics section in the Refinery 1.x config file specified MetricsAPIKey, then the Refinery 2.x config.yaml file will list that key as the APIKey name in the LegacyMetrics section.
Supported Refinery 1.x file formats for this tool are .json, .toml, and .yaml.
The new Refinery 2.x config.yaml files also includes comments that describe each configuration variable and its possible values.
.yaml files.In addition, the configuration conversion tool has advanced features and command line options.
Before installing the configuration conversion tool:
config and rules filesHaving this information identified ahead of time helps to ensure a smoother migration process.
The configuration conversion tool is available as built binaries for Linux, macOS, and Windows.
If using Helm with Kubernetes to install and upgrade Refinery, the configuration conversion tool is automatically available with Refinery 2.0 and later.
If not using Helm, download the configuration conversion tool from the Releases page of the Refinery GitHub repository. It is available as a single binary file for each operating system alongside each Refinery release, starting with Refinery 2.0.
After installation (if necessary), run the Configuration Conversion Tool for your installation option:
For Helm users, convert your Refinery Helm chart to the Refinery 2.0 and later format by using:
convert helm --input /path/to/values.yaml --output /path/to/values.yaml
where:
convert is the command to convert the designated --input filehelm is the conversion template to apply--input is a required command line option to designate the file path of your source file/path/to/values.yaml is the file path of your existing Refinery 1.x values.yaml file--output is a required command line option to designate the file path of your new Refinery 2.x values.yaml destination file/path/to/values.yaml is the file path of the designated --output destination fileIf not a Helm user, use the following two commands to convert your Refinery config and rules files.
To convert your Refinery config file to the Refinery 2.x format, use:
convert config --input /path/to/config.toml --output /path/to/config.yaml
where:
convert is the command to convert the designated --input fileconfig is the conversion template to apply--input is a required command line option to designate the file path of your source file/path/to/config.toml is the file path of your existing Refinery 1.x --input source file. Your migration situation may involve a different file type or path--output is a required command line option to designate the file path of the new Refinery 2.x .yaml destination file/path/to/config.yaml is the file path of the designated --output destination file--type option. To convert your Refinery rules file to the Refinery 2.x format, use:
convert rules --input /path/to/rules.toml --output /path/to/rules.yaml
where:
convert is the command to convert the designated --input filerules is the conversion template to apply--input is a required command line option to designate the file path of your source file/path/to/rules.toml is the file path of your existing Refinery 1.x --input source file. Your migration situation may involve a different file type or path--output is a required command line option to designate the file path of your .yaml destination file/path/to/rules.yaml is the file path of the designated --output destination file--type option. As a part of the configuration conversion process, the Refinery file is automatically validated. If one or more issues are encountered, the conversion process stops and descriptive error message(s) appear in the command line output.
For example, the following output appears with multiple error messages for the rules file:
Validation Errors in rules file:
Within sampler dataset1: field DynamicSampler.UseTraceLength must be a bool
Within sampler dataset1: field DynamicSampler.ClearFrequency (1) must be a valid duration like '3m30s' or '100ms'
Within sampler dataset1: unknown field DynamicSampler.SamplerRate; did you mean SampleRate?
Within sampler dataset1: field DynamicSampler.FieldList must be a string array but contains non-string 10
Within sampler dataset1: the group DynamicSampler is missing its required field SampleRate
To continue with migration, correct the errors found in your Refinery 1.x file and re-run the configuration conversion tool.
In troubleshooting, reference our online Refinery documentation for configuration (config.yaml) and sampling (rules.yaml).
The configuration conversion tool features additional command line options and tool features as documented below.
The Refinery configuration conversion tool has the following command line options:
-i,--inputconfig.toml)-o, --outputstdout)-t Y, --type=Y-t T, --type=T-t J, --type=J-h, --help In addition to file conversion, the configuration conversion tool also features the ability to manually validate and generate local documentation.
Use the following commands to manually validate the Refinery configuration files, such as manual validation in a Continuous Integration (CI) process.
Refinery automatically validates as a part of the convert command.
To validate your Refinery config file against the Refinery 2.x format, use:
convert validate config
To validate your Refinery rules file against the Refinery 2.x format, use:
convert validate rules
For example, this command validates the config.yaml config file:
convert validate config --input config.yaml
where:
convert validate is the command to validate the designated --input fileconfig is the validation template to apply--input is a required command line option to designate the file path of your source fileconfig.yaml is the file path of the designated --input source file To generate local Refinery documentation in markdown (.md) form for config.yaml, use:
convert doc config
To generate local Refinery documentation in markdown (.md) form for rules.yaml, use:
convert doc rules
Also, refer to our online Refinery documentation for configuration (config.yaml) and sampling (rules.yaml).
Join the #discuss-refinery in our Pollinators Community Slack to ask questions and learn more.
For Pro and Enterprise users, contact Support via support.honeycomb.io, or email at support@honeycomb.io