HashiCorp Consul is a service networking solution that enables teams to manage secure network connectivity between services, and across on-prem and multi-cloud environments and runtimes. Consul offers service discovery, service mesh, traffic management, and automated updates to network infrastructure devices.
Configure Consul to send cluster metrics to Honeycomb with an OpenTelemetry Collector.
The Consul agent’s metrics endpoint supports Prometheus-formatted metrics. As with other services which expose such an endpoint, use an OpenTelemetry Collector to scrape this endpoint and get these metrics into Honeycomb.
Refer to the Consul documentation for a list of Key Metrics, as well as a full Metrics Reference.
Prometheus metrics are not enabled by default.
Therefore, to enable, set the prometheus_retention_time value to at least twice the scrape interval of your OpenTelemetry Collector.
The HashiCorp documentation also suggests setting disable_hostname to avoid having hostname-prefixed metrics.
A suggested agent configuration can be created as /etc/consul.d/metrics.hcl on each Consul agent, as follows:
telemetry {
disable_hostname = true
prometheus_retention_time = "72h"
}
Scraping the Consul agent’s Prometheus metrics endpoint requires configuring a OpenTelemetry Collector with a pipeline that starts with a Prometheus receiver and ends with an OTLP exporter. Depending on your chosen method of Consul deployment, the resource detection processor may be helpful to further enrich the OTLP Metrics being sent to Honeycomb.
The following example OpenTelemetry Collector configuration uses the system resource detector processor:
receivers:
prometheus:
config:
scrape_configs:
- job_name: consul-metrics
scrape_interval: 60s
metrics_path: /v1/agent/metrics
static_configs:
- targets:
- localhost:8500
processors:
batch:
resourcedetection/os:
detectors:
- system
system:
hostname_sources:
- os
exporters:
otlp/metrics:
endpoint: api.honeycomb.io:443 # US instance
#endpoint: api.eu1.honeycomb.io:443 # EU instance
headers:
"x-honeycomb-team": "YOUR_API_KEY"
"x-honeycomb-dataset": "consul-metrics"
service:
pipelines:
metrics:
receivers:
- prometheus
processors:
- resourcedetection/os
- batch
exporters:
- otlp/metrics