Skip to main content
Ent+ Beta
This feature is available as an add-on for the Honeycomb Enterprise plan. This feature is in beta. Please contact your Honeycomb account team for details.
If you have already configured an Amazon S3 bucket as an archive for unsampled OpenTelemetry data, you can use the Send to Enhance S3 Archive component to export trace and log data from your telemetry pipeline into that archive. This component writes your trace and log data to S3 while simultaneously building field-based indexes, making rehydration faster and more cost-effective by letting you query only the data you need.

​
Adding the Component to Your Pipeline

To add the Send to Enhance S3 Archive component to your pipeline:
  1. Select Manage Data () from the navigation menu, and choose Pipelines.
  2. Select the pipeline you want to modify.
  3. Select Edit as draft. To create a new draft instead, open the Drafts view and select New draft.
  4. On the Pipeline Builder canvas, expand the Send () section, then add the Send to Enhance S3 Archive output to your configuration by selecting Add.
  5. Select the Send to Enhance S3 Archive component to configure its properties:
    Example of the Send to Enhance S3 Archive component and its available properties on the Pipeline Builder canvas
    PropertyDescription
    BucketAmazon S3 bucket in which to store your data.
    RegionAWS region for your bucket. Default: us-east-1.
    Prefix(Optional) Virtual directory path for organizing files in your bucket. Example: traces-logs-directory.
    Partition Format(Optional) Partition format to use when writing files to S3. Default: "year=%Y/month=%m/day=%d/hour=%H/minute=%M" (minute-level resolution).
    TimeoutMaximum time allowed for each send attempt. Default: 30s.
    MarshalerEncoding format for telemetry data. Default: otlp_protobuf. Options: otlp_protobuf (smaller files, better performance), otlp_json (human-readable output).
    Batch TimeoutMaximum time to wait before sending a batch to S3, even if it isn’t full. Default: 30s Examples: 45000ms, 1m.
    Batch SizeMaximum number of items (spans or log records) per batch. Default: 500000.
    Queue SizeMaximum number of items (spans or log records) that the export queue can buffer before blocking new data. Default: 500000.
    Indexed Fields(Optional) Additional fields to index beyond the defaults (trace.trace_id, service.name, session.id). When you run a query that requires rehydration, you can use these indexes to locate and retrieve only the relevant subset of your archived data. Choose high-cardinality fields that you frequently query, such as user.id, customer.id, or environment. For best performance, we recommend no more than 5 custom indexes.
    Share your indexed fields with your Honeycomb account team when you share your S3 bucket information and IAM role ARN. This ensures that rehydration is configured correctly.
  6. Connect the Send to Enhance S3 Archive component to your the appropriate receiver or processor by dragging the input/output connectors between components.
  7. Select Save Draft.