If you have already configured an Amazon S3 bucket as an archive for unsampled OpenTelemetry data, you can use the Send to Enhance S3 Archive component to export trace and log data from your telemetry pipeline into that archive.
This component writes your trace and log data to S3 while simultaneously building field-based indexes, making rehydration faster and more cost-effective by letting you query only the data you need.
To add the Send to Enhance S3 Archive component to your pipeline:
Select Manage Data (
) from the navigation menu, and choose Pipelines.
Select the pipeline you want to modify.
Select Edit as draft. To create a new draft instead, open the Drafts view and select New draft.
On the Pipeline Builder canvas, expand the Send (
) section, then add the Send to Enhance S3 Archive output to your configuration by selecting Add.
Select the Send to Enhance S3 Archive component to configure its properties:
| Property | Description |
|---|---|
| Bucket | Amazon S3 bucket in which to store your data. |
| Region | AWS region for your bucket. Default: us-east-1. |
| Prefix | (Optional) Virtual directory path for organizing files in your bucket. Example: traces-logs-directory. |
| Partition Format | (Optional) Partition format to use when writing files to S3. Default: "year=%Y/month=%m/day=%d/hour=%H/minute=%M" (minute-level resolution). |
| Timeout | Maximum time allowed for each send attempt. Default: 30s. |
| Marshaler | Encoding format for telemetry data. Default: otlp_protobuf. Options: otlp_protobuf (smaller files, better performance), otlp_json (human-readable output). |
| Batch Timeout | Maximum time to wait before sending a batch to S3, even if it isn’t full. Default: 30s Examples: 45000ms, 1m. |
| Batch Size | Maximum number of items (spans or log records) per batch. Default: 500000. |
| Queue Size | Maximum number of items (spans or log records) that the export queue can buffer before blocking new data. Default: 500000. |
| Indexed Fields | (Optional) Additional fields to index beyond the defaults (trace.trace_id, service.name, session.id). When you run a query that requires rehydration, you can use these indexes to locate and retrieve only the relevant subset of your archived data. Choose high-cardinality fields that you frequently query, such as user.id, customer.id, or environment. For best performance, we recommend no more than 5 custom indexes. |
Connect the Send to Enhance S3 Archive component to your the appropriate receiver or processor by dragging the input/output connectors between components.
Select Save Draft.
