Set Up an Amazon S3 Archive with Terraform


Use Terraform to create the IAM policy and role that Honeycomb uses to access your S3 archive for trace and log rehydration.

Note
This feature is available as an add-on for the Honeycomb Enterprise plan. Please contact your Honeycomb account team for details.

To enable trace and log rehydration, you will need an AWS IAM role that includes:

  • Permissions to list and retrieve objects from your S3 bucket.
  • A trust policy that allows the Honeycomb IAM role to assume this role when interacting with your archive.

This guide walks you through creating these resources using Terraform. If you prefer, you can use the AWS Management Console instead.

After you create the IAM policy and role, share the details with your Honeycomb account team, so they can complete your S3 archive setup.

Create an IAM Policy 

First, create an IAM policy that grants permissions to list and retrieve objects from your Amazon S3 bucket. Use the configuration that matches your encryption method:

Important
Replace the placeholder ARNs in the resources argument with the actual ARNs for your bucket.
data "aws_iam_policy_document" "s3_bucket_access" {
  statement {
    effect = "Allow"
    actions = [
      "s3:GetObject",
      "s3:ListBucket",
      "s3:GetBucketLocation"
    ]

    resources = [
      "arn:aws:s3:::<bucket name>/*",
      "arn:aws:s3:::<bucket name>"
    ]
  }
}

resource "aws_iam_policy" "s3_bucket_access" {
  name   = "${var.env}_s3_bucket_access"
  policy = data.aws_iam_policy_document.s3_bucket_access.json
}
Important
Replace the placeholder ARNs in the resources arguments with the actual ARNs for your bucket and AWS KMS key. For the KMS key, specify the full key rather than an alias.
data "aws_iam_policy_document" "s3_bucket_access" {
  statement {
    effect = "Allow"
    actions = [
      "s3:GetObject",
      "s3:ListBucket",
      "s3:GetBucketLocation",
    ]

    resources = [
      "arn:aws:s3:::<bucket name>/*",
      "arn:aws:s3:::<bucket name>"
    ]
  }
  
  statement {
    effect = "Allow"
    actions = [
      "kms:Decrypt",
    ]
    
    resources = [
      "arn:aws:kms:<region>:<acct#>:key/<UUID of key>",
    ]
  }
}

resource "aws_iam_policy" "s3_bucket_access" {
  name   = "${var.env}_s3_bucket_access"
  policy = data.aws_iam_policy_document.s3_bucket_access.json
}

Create the IAM Role 

Next, create an IAM role that can access your Amazon S3 bucket, and authorize Honeycomb to assume that role when interacting with your bucket:

Important

Replace the honeycomb_role_arn placeholder with the appropriate ARN, depending on your team’s Honeycomb instance:

  • US: "arn:aws:iam::702835727665:role/production-eks-bulk-ingest-role"
  • EU: "arn:aws:iam::919259170365:role/production-eu1-eks-bulk-ingest-role"
data "aws_iam_policy_document" "hny_assume_role_policy" {
  statement {
    effect = "Allow"
    actions = [
      "sts:AssumeRole"
    ]
    principals {
      type = "AWS"
      identifiers = [honeycomb_role_arn]
    }
  }
}

resource "aws_iam_role" "hny_s3_bucket_access_role" {
  name               = "${var.env}-hny-s3-bucket-access-role"
  assume_role_policy = data.aws_iam_policy_document.hny_assume_role_policy.json
}

Attach the Policy to the Role 

Attach the "s3_bucket_access" policy to your new IAM role:

resource "aws_iam_role_policy_attachment" "s3_bucket_access" {
  role       = aws_iam_role.s3_bucket_access_role.name
  policy_arn = aws_iam_policy.s3_bucket_access.arn
}
KMS Users: Check Your Key Policy
If the data in your S3 bucket is encrypted with an AWS KMS encryption key, ensure that the corresponding KMS Key Policy allows this IAM Role to use the key to perform kms:Decrypt actions. If the IAM Role does not have permission to decrypt using the KMS key, rehydration will fail.

Share with Your Honeycomb Account Team 

To complete your setup, share the following information with your Honeycomb account team:

  • The name of your Amazon S3 bucket

  • The ARN of the IAM role that will interact with your bucket

    Tip

    To locate the IAM Role’s ARN:

    1. In the IAM service’s navigation pane in the console, choose Roles.
    2. Search for the name of the IAM role you created, then choose the role from the results.
    3. In the Summary section, locate ARN, and use the copy icon to copy the ARN value to your clipboard.
  • s3_prefix from your OpenTelemetry Collector exporter configuration, if configured

  • s3_partition_format from your OpenTelemetry Collector exporter configuration, if configured

  • indexed_fields from your OpenTelemetry Collector exporter configuration, if you configured custom indexed fields