> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# How Honeycomb's AWS Integrations Work

> Find out how Honeycomb's AWS integrations collect logs and metrics, which AWS services use which collection methods, and how the data flows to Honeycomb.

export const PermissionsTable = ({children}) => {
  return <div className="permissions-table">
      {children}
    </div>;
};

This section describes the mechanics of Honeycomb AWS Integrations that collect logs and metrics.
It describes various collection methods and documents which AWS services use those methods.

For more details, visit the Honeycomb AWS Integrations GitHub repository for [Terraform](https://github.com/honeycombio/terraform-aws-integrations) or [CloudFormation](https://github.com/honeycombio/cloudformation-integrations).

## AWS CloudWatch Logs

Many AWS services publish logs to CloudWatch Logs.
By default, most AWS services send unstructured logs to CloudWatch.
Because Honeycomb is designed to work best with [structured events](/get-started/basics/observability/concepts/events-metrics-logs/), Honeycomb AWS Integrations convert some of those CloudWatch log streams into structured data.

Deploy the Honeycomb CloudWatch Logs integration with either [Terraform](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/cloudwatch-logs) or [CloudFormation](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create?stackName=cloudwatch-logs\&templateURL=https://honeycomb-builds.s3.amazonaws.com/cloudformation-templates/latest/cloudwatch-logs.yml).

### How AWS CloudWatch Logs Integrations Work

AWS CloudWatch provides [Subscription Filters](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html), which can be configured to forward all logs in a log group to a Kinesis Firehose destination that can then stream the logs to our Honeycomb Kinesis endpoint.
AWS services listed in the unstructured data column of the table below use a workflow, as shown by the diagram below, to send AWS CloudWatch logs to Honeycomb.

```mermaid actions={false} theme={}
flowchart LR
  D("Kinesis Data Transformation Lambda") <--> B
  A("AWS CloudWatch Log Group") --> B("AWS Kinesis Data Firehose") --> C("Honeycomb")
```

1. AWS CloudWatch log groups point at AWS Kinesis Firehose.
2. Firehose forwards data to Honeycomb.

Kinesis Data Firehose can invoke Lambda functions to transform incoming source data before delivering the transformed data to its destination.
Supported AWS services listed in the structured data column of the table below are either JSON-formatted by default or use an additional step in their workflow, as shown by the diagram above, to structure their log data before sending it to Honeycomb.

### Supported AWS Services Through CloudWatch Logs

Honeycomb AWS Integrations are designed to work with **all** services that logs to CloudWatch.

For a full list of possible data sources, refer to AWS documentation for [services that publish logs to CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/aws-services-sending-logs.html).

This list is a selection of supported AWS services that publish logs through CloudWatch Logs and the type of log data Honeycomb can collect from them.

<PermissionsTable>
  | AWS Services supported through CloudWatch Logs | Structured | Unstructured |
  | ---------------------------------------------- | ---------- | ------------ |
  | Amazon API Gateway                             |            | ✔            |
  | Amazon Aurora                                  |            | ✔            |
  | AWS CloudHSM                                   |            | ✔            |
  | AWS CloudWatch                                 | ✔          |              |
  | AWS CodeBuild                                  |            | ✔            |
  | AWS CodeDeploy                                 |            | ✔            |
  | Amazon Cognito                                 |            | ✔            |
  | AWS Elastic Beanstalk                          |            | ✔            |
  | Amazon Elastic Container Service (ECS)         | ✔          |              |
  | Amazon Elastic Kubernetes Service (EKS)        | ✔          |              |
  | Elastic Load Balancing (ELB)                   | ✔          |              |
  | Amazon ElastiCache                             |            | ✔            |
  | AWS Fargate                                    |            |              |
  | Amazon Keyspaces                               |            | ✔            |
  | AWS Lambda                                     | ✔          |              |
  | Amazon Managed Streaming for Kafka             |            | ✔            |
  | Amazon MQ                                      |            | ✔            |
  | AWS Network Firewall                           |            | ✔            |
  | AWS OpsWorks                                   |            | ✔            |
  | Amazon Relational Database Service (RDS)       | ✔          |              |
  | Amazon Route 53                                |            | ✔            |
  | Amazon Simple Notification Service (SNS)       | ✔          |              |
  | Amazon Simple Storage Service (S3)             | ✔          |              |
  | AWS Step Functions                             |            | ✔            |
  | Amazon Virtual Private Cloud (VPC)             | ✔          |              |
</PermissionsTable>

### Working With Unstructured AWS Logs

Each field in a structured log (or event) is queryable with Honeycomb.
This makes working with structured AWS log data similar to working with other types of data in Honeycomb.

Unstructured logs are received by Honeycomb as events with a verbose `message` attribute.
All unstructured log data is contained within this single attribute.
For tips on working with unstructured AWS logs, refer to [Work with Your AWS Data](/investigate/debug/aws-data-in-honeycomb/).

### Advanced Use

The CloudWatch Logs integration is a great way to get started and gain insights across a variety of AWS services.
If you want to customize the structure of the logs, you might want to write your own data transformation Lambda to be used with the Kinesis Data Firehose.
Source code for our RDS data transformations can be viewed in the [Agentless Integrations GitHub repository](https://github.com/honeycombio/agentless-integrations-for-aws/tree/main/rds-mysql-kfh-transform) as an example.

## AWS CloudWatch Metrics

Honeycomb AWS Integrations support all AWS services that send metrics to CloudWatch.

AWS CloudWatch provides [Metric Streams](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html), which forward all metrics captured by Amazon during normal use of AWS offerings to third-party destinations, including Honeycomb.

Deploy the Honeycomb Cloudwatch Metrics integration with either [Terraform](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/cloudwatch-metrics) or [CloudFormation](https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/new?stackName=cloudwatch-metrics\&templateURL=https://honeycomb-builds.s3.amazonaws.com/cloudformation-templates/latest/cloudwatch-metrics.yml).

<Note>
  AWS charges its customers to use the Cloudwatch Metrics API.
  Please refer to Amazon for specifics on associated egress costs.
</Note>

### How AWS CloudWatch Metrics Integrations work

Metrics stored in AWS CloudWatch can be streamed to other systems using AWS Kinesis Data Firehose.

Honeycomb provides an endpoint that is compatible with CloudWatch Metric Streams, and stores the data it receives in a dataset for easy querying.

```mermaid actions={false} theme={}
   flowchart LR
   A("AWS Cloudwatch") -->|"Metrics Streams"| B("AWS Kinesis") -->|"Data Firehose (OTLP)"| C("Honeycomb")
```

1. AWS Cloudwatch with Metric Streams pointed at AWS Kinesis.
2. Kinesis forwards data to Honeycomb over Data Firehose, which is configured to format data with OpenTelemetry Line Protocol (OTLP).

### Supported AWS Services Through CloudWatch Metrics

For a full list of possible data sources, refer to AWS documentation for services that [publish metrics to CloudWatch Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html).

## Amazon S3 Logs

Honeycomb provides an agentless integration for ingesting [S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html) logfiles and sending them to Honeycomb as structured events.

Deploy the Honeycomb S3 Logs integration with either [Terraform](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/s3-logfile) or [CloudFormation](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create?stackName=s3-logfile\&templateURL=https://honeycomb-builds.s3.amazonaws.com/cloudformation-templates/latest/s3-logfile.yml).

### How Amazon S3 Logs Integrations Work

AWS services listed in the the table below use a workflow, as shown by the diagram below, to send Amazon S3 logs to Honeycomb.
The integration uses a [Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) function, which is subscribed to `PutObject` events on your bucket.
The Lambda source code is available [on GitHub](https://github.com/honeycombio/agentless-integrations-for-aws/blob/main/s3-handler/main.go).

```mermaid actions={false} theme={}
   flowchart LR
   A("S3 Bucket with Logs") --> B("Lambda") --> C("Honeycomb")
```

1. S3 logs point at a Lambda.
2. Lambda delivers structured logs to Honeycomb.

### Supported AWS Services Through Amazon S3 Logs

This is a complete list of AWS services logging to S3 that are supported by Honeycomb AWS Integrations.

<PermissionsTable>
  | AWS Services supported via S3 Logs       | Structured | Unstructured |
  | ---------------------------------------- | ---------- | ------------ |
  | AWS CloudFront                           | ✔          |              |
  | AWS CloudTrail                           | ✔          |              |
  | Elastic Load Balancing (ELB) Access Logs | ✔          |              |
  | Amazon S3 Access Logs                    | ✔          |              |
  | Amazon Virtual Private Cloud (VPC)       | ✔          |              |
</PermissionsTable>