Skip to main content
Ent+
This feature is available as an add-on for the Honeycomb Enterprise plan. Please contact your Honeycomb account team for details.

​
Prerequisites

While installing AWS CLI is not required in order to collect logs using the AWS Cloudwatch source type, it still provides an easier means of authentication. With AWS Cloudwatch, users are required to provide some form of authentication. This can be either profile credentials or environment variables that provide access keys for user accounts. Setting these credentials up can prove tedious and confusing. Luckily, AWS CLI can generate the user’s profile/credentials with the aws configure command. Here is the AWS CLI Getting Started Guide, which outlines a few prerequisites. Some prerequisites:
  1. Creating an IAM user account
    • Required Permissions
      • logs:GetLogEvents
      • logs:DescribeLogGroups
      • logs:DescribeLogStreams
    • The user does not require console access
  2. Create an access key ID and secret access key
In the AWS CLI getting started guide, it will instruct you to install it for your current user or all users.
The observIQ OTEL Collector runs as root by default, meaning the AWS CLI and credentials should be installed under the collector system’s root account.

​
Credentials

​
Credential and Config Files

AWS Authentication utilizes a user profile specified in the user’s home directory at .aws/credentials. Each profile’s credentials should include, at minimum, the profile name, access key, and secret access key.
In addition to the credentials file, there is also a .aws/config. This includes less sensitive configuration options such as the region, output format, etc. A typical entry in the config file should look as such.
More information on AWS Configuration and Credentials

​
Environment Variables

Alternatively, AWS Environment variables can be specified to override a credentials file.
You can modify the collector’s environment variables by configuring a Systemd override.
Run sudo systemctl edit observiq-otel-collector and add your access key, secret key, and region.
After making that change, reload Systemd and restart the collector service.

​
Setup

  1. Once logged in, Select the Configs tab at the top of the Honeycomb Telemetry Pipeline Home page.
  2. Select a pre-existing config or create a new one.
  3. Add a new source and select AWS Cloudwatch.
  4. After configuring credentials using the AWS CLI on the collector system, using the default values in the source form should enable the collector to collect logs from Cloudwatch. If credentials were configured using environment variables, you will need to leave the Profile field blank
  5. Click save configuration.
  6. Add the destination type of your choosing.
  7. Apply configuration to the desired agent.
  8. Voila. Logs should be collecting

​
Supported Platforms

​
Configuration Table

* required field

​
Discovery Type

​
Default Settings

When starting with an AWS Cloudwatch set to its default values, you should see log collection from all log groups with no filtering of log streams. The default polling interval for Cloudwatch is 1 minute, so there may be a delay before seeing any logs coming through.

​
AutoDiscover

When using Discovery Type AutoDiscover, there are some optional parameters that can be added to / filter the amount of logs collected.
  • limit: limits the number of discovered log groups(default = 50).
  • prefix: Prefix for log groups to limit the number of log groups discovered
    • prefix: /aws/eks/
    • If omitted, all log groups up to the limit will be collected.
  • names: A list of full log stream names to filter the discovered log groups to collect from.
    • names: [kube-apiserver-ea9c831555adca1815ae04b87661klasdj]
  • prefixes: A list of log stream prefixes to filter the discovered log groups to collect from.
    • prefixes: [kube-api-controller]

​
Named

This Discovery Type filters logs by listing only the desired log groups to collect from and omitting any other log groups. When selecting this Discovery Type, at least one log group is required otherwise no logs would be collected. When listing log groups the ID field of each log group instance should match the full name of the group. Additionally, Named also provides prefixes and names parameters for each listed log group that filters out the listed log streams. These parameters should be listed underneath each log group’s IDs as they are unique to each individual log group.