Send Metrics from AWS CloudWatch | Honeycomb

Send Metrics from AWS CloudWatch

AWS CloudWatch provides Metric Streams, which forward all metrics captured by Amazon during normal use of AWS offerings to third-party destinations, including Honeycomb. Note that AWS charges its customers to use this API, please refer to Amazon for specifics on associated egress costs.

How It Works 

Metrics stored in AWS CloudWatch can be streamed to other systems using AWS Kinesis Data Firehose. Honeycomb provides an endpoint that is compatible with CloudWatch Metric Streams, and will store the data it receives in a dataset for easy querying.

An architecture diagram depicting AWS Cloudwatch with Metric Streams pointed at AWS Kinesis. Kinesis is subsequently forwarding data to Honeycomb over Data Firehose configured to format data with OpenTelemetry Line Protocol (OTLP).

Terraform Module 

Honeycomb provides a Terraform module to automate configuration of CloudWatch Metric Streams to Honeycomb. Refer to the module’s README for instructions on its use.

Alternatively, read below to learn how to manually configure CloudWatch using the AWS Console.

Configuration Using the AWS Console 

  1. Create a new Kinesis Data Firehose Delivery Stream. The stream’s Source should be Direct PUT and Destination set to Honeycomb with the following details:

    • Honeycomb Kinesis endpoint:
      • US instance: https://api.honeycomb.io/1/kinesis_events/your_dataset_name
      • EU instance: https://api.eu1.honeycomb.io/1/kinesis_events/your_dataset_name
    • API Key: Enter a Honeycomb API key capable of sending events.
    • Content Encoding: Set to GZIP.
  2. Create a new CloudWatch Metric Stream.

    1. Open the CloudWatch AWS console at https://console.aws.amazon.com/cloudwatch/.
    2. In the left navigation pane, choose MetricsStreams.
    3. Click the Create metric stream button.
    4. Choose the CloudWatch metric namespaces to include in the metric stream.
    5. Choose Select an existing Firehose owned by your account, and select the Firehose Delivery Stream you created earlier.
    6. Specify an Output Format of OpenTelemetry 1.0.
    7. Optionally, specify a name for this metric stream under Metric Stream Name.
    8. Choose Create metric stream.

NOTE: Honeycomb supports receiving metrics via the OTLP format. Metrics sent through CloudWatch must have the specified output format of OpenTelemetry to be properly processed by Honeycomb.