> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Send Data from AWS via HashiCorp Terraform

> Set up Honeycomb's AWS integrations using HashiCorp Terraform as your IaC method to provision the resources needed to send data to Honeycomb.

Honeycomb AWS Integrations can be set up with [HashiCorp Terraform](https://developer.hashicorp.com/terraform/intro) as your preferred Infrastructure-as-Code (IaC) method and deployment process.

<Note>
  Honeycomb AWS Integrations utilize AWS CloudWatch, Amazon Kinesis, and AWS Lambda to send data to Honeycomb.
  Refer to [How AWS Integrations Work](/integrations/aws/how-aws-integrations-work/) to reference which AWS services use which methods.
  Note that standard AWS charges apply.
  Please refer to Amazon for specifics on associated egress costs.
</Note>

## HashiCorp Terraform

Honeycomb provides a [Terraform module](https://github.com/honeycombio/terraform-aws-integrations) to automate configuration of various AWS services to Honeycomb.
Each integration has an independently deployable submodule that the top-level module deploys all together with the ability to turn functionality on and off as needed.

Supported Terraform Integrations:

* CloudWatch Logs
* CloudWatch Metrics
* RDS Logs
* Amazon S3 Bucket Logs

### Terraform Setup

Implement all of the Terraform submodules at once, or choose among the available Terraform submodules.

<Tabs>
  <Tab title="All">
    To configure for **all** supported Terraform integrations:

    1. Add the minimal Terraform configuration, which includes the required fields for all supported Terraform integrations:

       ```hcl theme={}
       module "honeycomb-aws-integrations" {
         source = "honeycombio/integrations/aws"

         # aws cloudwatch logs integration
         cloudwatch_log_groups = [module.log_group.cloudwatch_log_group_name] // CloudWatch Log Group names to stream to Honeycomb.

         # aws rds logs integration
         enable_rds_logs  = true
         rds_db_name      = var.db_name
         rds_db_engine    = "mysql"
         rds_db_log_types = ["slowquery"] // valid types include general, slowquery, error, and audit (audit will be unstructured)

         # aws metrics integration - pro/enterprise Honeycomb teams only
         # enable_cloudwatch_metrics = true

         # s3 logfile - alb access logs
         s3_bucket_arn  = var.s3_bucket_arn
         s3_parser_type = "alb" // valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval

         #honeycomb
         honeycomb_api_key = var.honeycomb_api_key             // Honeycomb API key.
         honeycomb_dataset = "terraform-aws-integrations-test" // Your Honeycomb dataset name that will receive the logs.

         # Users generally do not need to set this, but it may be necessary when working with a proxy like Honeycomb's Refinery.
         honeycomb_api_host = var.honeycomb_api_host
       }
       ```

    2. Set the `TF_VAR_HONEYCOMB_API_KEY` environment variable to your team's Honeycomb API Key.

       ```shell theme={}
       export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
       ```

    3. Set the environment variables with your AWS credentials.

       ```shell theme={}
       export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
       export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
       export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
       ```

       For more details and options, visit [Terraform documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).

    4. Run `terraform plan/apply` in sequence.

    For more configuration options, refer to [USAGE.md](https://github.com/honeycombio/terraform-aws-integrations/blob/main/USAGE.md).
  </Tab>

  <Tab title="CloudWatch Logs">
    To configure for **CloudWatch Logs**:

    1. Add the minimal Terraform configuration, which includes the required fields:

       ```hcl theme={}
       module "honeycomb-aws-cloudwatch-logs-integration" {
         source = "honeycombio/integrations/aws//modules/cloudwatch-logs"

         name = var.cloudwatch_logs_integration_name // A name for the Integration.

         #aws cloudwatch integration
         cloudwatch_log_groups = ["/aws/lambda/S3LambdaHandler-test"] // CloudWatch Log Group names to stream to Honeycomb.
         s3_failure_bucket_arn        = var.s3_bucket_name
         // S3 bucket ARN that will store any logs that failed to be sent to Honeycomb.

         #honeycomb
         honeycomb_api_key      = var.HONEYCOMB_API_KEY // Honeycomb API key.
         honeycomb_dataset_name = "cloudwatch-logs" // Your Honeycomb dataset name that will receive the logs.
       }
       ```

    2. Set the `TF_VAR_HONEYCOMB_API_KEY` environment variable to your team's Honeycomb API Key.

       ```shell theme={}
       export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
       ```

    3. Set the environment variables with your AWS credentials.

       ```shell theme={}
       export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
       export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
       export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
       ```

       For more details and options, visit [Terraform documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).

    4. Run `terraform plan/apply` in sequence.

    For more configuration options, refer to [USAGE.md](https://github.com/honeycombio/terraform-aws-integrations/blob/main/USAGE.md).
  </Tab>

  <Tab title="CloudWatch Metrics">
    To configure for **CloudWatch Metrics**:

    1. Add the minimal Terraform configuration, which includes the required fields:

       ```hcl theme={}
       module "honeycomb-aws-cloudwatch-metrics-integration" {
         source = "honeycombio/integrations/aws//modules/cloudwatch-metrics"

         name = var.cloudwatch_metrics_integration_name // A name for the Integration.

         honeycomb_api_key      = var.HONEYCOMB_API_KEY // Honeycomb API key.
         honeycomb_dataset_name = "cloudwatch-metrics" // Your Honeycomb dataset name that will receive the metrics.

         s3_failure_bucket_arn = var.s3_bucket_arn // A S3 bucket that will store any metrics that failed to be sent to Honeycomb.
       }
       ```

    2. Set the `TF_VAR_HONEYCOMB_API_KEY` environment variable to your team's Honeycomb API Key.

       ```shell theme={}
       export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
       ```

    3. Set the environment variables with your AWS credentials.

       ```shell theme={}
       export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
       export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
       export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
       ```

       For more details and options, visit [Terraform documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).

    4. Run `terraform plan/apply` in sequence.

    For more configuration options, refer to [USAGE.md](https://github.com/honeycombio/terraform-aws-integrations/blob/main/USAGE.md).
  </Tab>

  <Tab title="RDS Logs">
    To configure for **RDS Logs**:

    1. Add the minimal Terraform configuration, which includes the required fields:

       ```hcl theme={}
       module "honeycomb-aws-rds-logs-integration" {
         source = "honeycombio/integrations/aws//modules/rds-logs"

         name                   = "rds-logs-integration"
         db_engine              = "mysql"
         db_name                = "mysql-db-name"
         db_log_types           = ["slowquery"]
         honeycomb_api_key      = var.honeycomb_api_key // Your Honeycomb team's API key
         honeycomb_dataset_name = "rds-mysql-logs"

         s3_failure_bucket_arn = var.s3_bucket_arn      // The full ARN of the bucket storing Kinesis Firehose failure logs.
       }
       ```

    2. Set the `TF_VAR_HONEYCOMB_API_KEY` environment variable to your team's Honeycomb API Key.

       ```shell theme={}
       export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
       ```

    3. Set the environment variables with your AWS credentials.

       ```shell theme={}
       export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
       export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
       export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
       ```

       For more details and options, visit [Terraform documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).

    4. Run `terraform plan/apply` in sequence.

    For more configuration options, refer to [USAGE.md](https://github.com/honeycombio/terraform-aws-integrations/blob/main/USAGE.md).
  </Tab>

  <Tab title="Amazon S3 Bucket Logs">
    To configure for **Amazon S3 Logs from a bucket**:

    1. Add the minimal Terraform configuration, which includes the required fields:

       ```hcl theme={}
       module "logs_from_a_bucket_integrations" {
         source = "honeycombio/integrations/aws//modules/s3-logfile"
         name   = var.logs_integration_name

         parser_type   = var.parser_type // valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval
         s3_bucket_arn = var.s3_bucket_arn     // The full ARN of the bucket storing the logs.


         honeycomb_api_key      = var.honeycomb_api_key // Your Honeycomb team's API key.
         honeycomb_dataset_name = "alb-logs" // Your Honeycomb dataset name that will receive the metrics.
       }
       ```

    2. Set the `TF_VAR_HONEYCOMB_API_KEY` environment variable to your team's Honeycomb API Key.

       ```shell theme={}
       export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
       ```

    3. Set the environment variables with your AWS credentials.

       ```shell theme={}
       export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
       export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
       export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
       ```

       For more details and options, visit [Terraform documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).

    4. Run `terraform plan/apply` in sequence.

    For more configuration options, refer to [USAGE.md](https://github.com/honeycombio/terraform-aws-integrations/blob/main/USAGE.md).
  </Tab>
</Tabs>

## Troubleshooting

### Kinesis Firehose

Kinesis Firehose sends failed data to the configured S3 bucket.
Depending on the stage of the flow when the error occurred, the error log ends up in a different subdirectory.
For example, if our HTTP endpoint sends a non-`2xx` error code back, the error log will be located in the `http-endpoint-failed` directory with the exact status code and error message.

### RDS

A pre-requisite to using both the [RDS logs Terraform module](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/rds-logs) and the [RDS logs CloudFormation stack](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create?stackName=rds-logs\&templateURL=https://honeycomb-builds.s3.amazonaws.com/cloudformation-templates/latest/rds-logs.yml) is enabling RDS log exports to CloudWatch.
Refer to [Publishing database logs to Amazon CloudWatch](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Procedural.UploadtoCloudWatch.html) for instructions.
Once enabled, the module can scoop logs from those CloudWatch log groups and begin streaming them to Honeycomb.

<Note>
  When enabling log exports for RDS Postgresql, ensure the `log_statement` attribute on the parameter group is not set to `ALL`.
</Note>

Currently, structured logs are supported for the following:

* MySQL general logs
* MySQL slow query logs
* MySQL error logs
* Postgresql slow query logs

For any RDS log types not listed above, the integration will still deliver them to Honeycomb but they will be unstructured logs.