Honeycomb AWS Integrations can be set up with HashiCorp Terraform as your preferred Infrastructure-as-Code (IaC) method and deployment process.
Honeycomb provides a Terraform module to automate configuration of various AWS services to Honeycomb. Each integration has an independently deployable submodule that the top-level module deploys all together with the ability to turn functionality on and off as needed.
Supported Terraform Integrations:
Implement all of the Terraform submodules at once, or choose among the available Terraform submodules.
To configure for all supported Terraform integrations, add the minimal Terraform configuration, which includes the required fields for all supported Terraform integrations.
module "honeycomb-aws-integrations" {
source = "honeycombio/integrations/aws"
# aws cloudwatch logs integration
cloudwatch_log_groups = [module.log_group.cloudwatch_log_group_name] // CloudWatch Log Group names to stream to Honeycomb.
# aws rds logs integration
enable_rds_logs = true
rds_db_name = var.db_name
rds_db_engine = "mysql"
rds_db_log_types = ["slowquery"] // valid types include general, slowquery, error, and audit (audit will be unstructured)
# aws metrics integration - pro/enterprise Honeycomb teams only
# enable_cloudwatch_metrics = true
# s3 logfile - alb access logs
s3_bucket_arn = var.s3_bucket_arn
s3_parser_type = "alb" // valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval
#honeycomb
honeycomb_api_key = var.honeycomb_api_key // Honeycomb API key.
honeycomb_dataset = "terraform-aws-integrations-test" // Your Honeycomb dataset name that will receive the logs.
# Users generally do not need to set this, but it may be necessary when working with a proxy like Honeycomb's Refinery.
honeycomb_api_host = var.honeycomb_api_host
}
Set the TF_VAR_HONEYCOMB_API_KEY environment variable to your team’s Honeycomb API Key.
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
Set the environment variables with your AWS credentials.
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
Please refer to Terraform documentation for more details and options.
Now, you can run terraform plan/apply in sequence.
For more configuration options, refer to USAGE.md.
To configure for CloudWatch Logs, add the minimal Terraform configuration, which includes the required fields:
module "honeycomb-aws-cloudwatch-logs-integration" {
source = "honeycombio/integrations/aws//modules/cloudwatch-logs"
name = var.cloudwatch_logs_integration_name // A name for the Integration.
#aws cloudwatch integration
cloudwatch_log_groups = ["/aws/lambda/S3LambdaHandler-test"] // CloudWatch Log Group names to stream to Honeycomb.
s3_failure_bucket_arn = var.s3_bucket_name
// S3 bucket ARN that will store any logs that failed to be sent to Honeycomb.
#honeycomb
honeycomb_api_key = var.HONEYCOMB_API_KEY // Honeycomb API key.
honeycomb_dataset_name = "cloudwatch-logs" // Your Honeycomb dataset name that will receive the logs.
}
Set the TF_VAR_HONEYCOMB_API_KEY environment variable to your team’s Honeycomb API Key.
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
Set the environment variables with your AWS credentials.
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
Please refer to Terraform documentation for more details and options.
Now you can run terraform plan/apply in sequence.
For more configuration options, refer to USAGE.md.
To configure for CloudWatch Metrics, add the minimal Terraform configuration, which includes the required fields:
module "honeycomb-aws-cloudwatch-metrics-integration" {
source = "honeycombio/integrations/aws//modules/cloudwatch-metrics"
name = var.cloudwatch_metrics_integration_name // A name for the Integration.
honeycomb_api_key = var.HONEYCOMB_API_KEY // Honeycomb API key.
honeycomb_dataset_name = "cloudwatch-metrics" // Your Honeycomb dataset name that will receive the metrics.
s3_failure_bucket_arn = var.s3_bucket_arn // A S3 bucket that will store any metrics that failed to be sent to Honeycomb.
}
Set the TF_VAR_HONEYCOMB_API_KEY environment variable to your team’s Honeycomb API Key.
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
Set the environment variables with your AWS credentials.
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
Please refer to Terraform documentation for more details and options.
Now you can run terraform plan/apply in sequence.
For more configuration options, refer to USAGE.md.
To configure for RDS Logs, add the minimal Terraform configuration, which includes the required fields:
module "honeycomb-aws-rds-logs-integration" {
source = "honeycombio/integrations/aws//modules/rds-logs"
name = "rds-logs-integration"
db_engine = "mysql"
db_name = "mysql-db-name"
db_log_types = ["slowquery"]
honeycomb_api_key = var.honeycomb_api_key // Your Honeycomb team's API key
honeycomb_dataset_name = "rds-mysql-logs"
s3_failure_bucket_arn = var.s3_bucket_arn // The full ARN of the bucket storing Kinesis Firehose failure logs.
}
Set the TF_VAR_HONEYCOMB_API_KEY environment variable to your team’s Honeycomb API Key.
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
Set the environment variables with your AWS credentials.
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
Please refer to Terraform documentation for more details and options.
Now you can run terraform plan/apply in sequence.
For more configuration options, refer to USAGE.md.
To configure for Amazon S3 Logs from a bucket, add the minimal Terraform configuration, which includes the required fields:
module "logs_from_a_bucket_integrations" {
source = "honeycombio/integrations/aws//modules/s3-logfile"
name = var.logs_integration_name
parser_type = var.parser_type // valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval
s3_bucket_arn = var.s3_bucket_arn // The full ARN of the bucket storing the logs.
honeycomb_api_key = var.honeycomb_api_key // Your Honeycomb team's API key.
honeycomb_dataset_name = "alb-logs" // Your Honeycomb dataset name that will receive the metrics.
}
Set the TF_VAR_HONEYCOMB_API_KEY environment variable to your team’s Honeycomb API Key.
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
Set the environment variables with your AWS credentials.
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
Please refer to Terraform documentation for more details and options.
Now you can run terraform plan/apply in sequence.
For more configuration options, refer to USAGE.md.
Kinesis Firehose sends failed data to the configured S3 bucket.
Depending on the stage of the flow when the error occurred, the error log ends up in a different subdirectory.
For example, if our HTTP endpoint sends a non-2xx error code back, the error log will be located in the http-endpoint-failed directory with the exact status code and error message.
A pre-requisite to using both the RDS logs Terraform module and the RDS logs CloudFormation stack is enabling RDS log exports to CloudWatch. Refer to Publishing database logs to Amazon CloudWatch for instructions. Once enabled, the module can scoop logs from those CloudWatch log groups and begin streaming them to Honeycomb.
log_statement attribute on the parameter group is not set to ALL.Currently, structured logs are supported for the following:
For any RDS log types not listed above, the integration will still deliver them to Honeycomb but they will be unstructured logs.