Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt

Use this file to discover all available pages before exploring further.

The OpenTelemetry Collector drainprocessor applies the Drain online log parsing algorithm to each log record and attaches the derived template as an attribute. A template captures the invariant skeleton of a log message, with variable parts replaced by <*> wildcards. For example, the messages user 1234 logged in from 10.0.0.1 and user 5678 logged in from 10.0.0.2 both reduce to user <*> logged in from <*>. Deriving templates at the collector means a stable template attribute, defaulting to log.record.template, lands on every log record in your Honeycomb dataset. You can GROUP BY it to find dominant patterns, build SLOs or Triggers against specific templates, and feed template-based conditions into Refinery sampling rules.
The drain processor is at alpha stability for logs and is distributed in the OpenTelemetry Collector Contrib and Kubernetes builds. The processor operates on log records only; it does not affect traces or metrics. Configuration options and behavior may change between releases. Pin a specific collector version, test in a non-production environment, and read the upstream release notes before upgrading.

​
When to use the drain processor

The drain processor is the right choice when you need to discover and stabilize log patterns at the collector rather than define them manually. Use it when:
  • You have high-volume or noisy logs and want to identify the dominant message patterns without manually defining them.
  • You want a stable template attribute on every event so that SLOs, Triggers, and Refinery rules can reference it directly.
  • You want to make log volume reduction decisions (sampling, dropping, or routing) at the collector based on pattern rather than message content.
Reach for an alternative when:
  • You only need to template logs for a single ad-hoc query. A Calculated Field using regex or string functions is enough and doesn’t require collector changes. Calculated Fields are per-query and can’t be referenced by SLOs, Triggers, or Refinery.
  • You already know the exact patterns and just need to tag them. The transform processor with OTTL is more precise than a learned algorithm.
  • You need a stable, generally-available processor and can’t adopt alpha components.

​
How drain works

Drain incrementally builds a fixed-depth parse tree of tokens seen across log messages. Each leaf holds a small set of templates; an incoming message walks the tree to the matching leaf and is merged with the closest template if a sufficient fraction of tokens match. This approach is fast, runs online without retraining, and converges to a stable set of templates after a short warmup. The two parameters that most affect output quality are tree_depth and merge_threshold. To learn more, refer to the Tune the Algorithm section.

​
Configuration reference

This table covers the most common fields. For the authoritative option list at the version of the collector you run, refer to the upstream drainprocessor README.
FieldTypeRequiredDefaultPurpose
tree_depthintNo4Depth of the parse tree. Minimum is 3. Higher values increase precision but use more memory.
merge_thresholdfloatNo0.4Fraction of tokens that must match an existing template (range 0.0–1.0). Lower values merge more aggressively into fewer templates.
max_node_childrenintNo100Maximum children per internal tree node.
max_clustersintNo0Maximum number of templates retained. 0 means unlimited. When the cap is hit the least-recently-used template is evicted.
extra_delimiters[]stringNo[]Additional characters to treat as token delimiters, in addition to whitespace.
body_fieldstringNo""When the log body is a map, the key whose value should be templated. Empty means template the stringified body.
template_attributestringNolog.record.templateAttribute name written to each log record.
seed_templates[]stringNo[]Pre-loaded templates the processor starts with, using <*> for wildcards.
seed_logs[]stringNo[]Example log lines used to seed templates at startup.
warmup_min_clustersintNo0Minimum templates that must exist before the processor begins annotating records. Useful to avoid noisy templates during a cold start.
storagestringNoin-memoryID of a storage extension for persisting templates across collector restarts.
save_intervaldurationNo0sHow often to snapshot template state to the storage extension. 0s disables periodic snapshots.

​
Basic configuration

Wire the processor into your logs pipeline. Spans and metric pipelines should not include it. 
processors:
  drain:
    tree_depth: 4
    merge_threshold: 0.4
    max_clusters: 500

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [drain, batch]
      exporters: [otlp/honeycomb]
After deployment, every log record arriving at Honeycomb has a log.record.template attribute set to its derived template. To start with a curated set of templates rather than learning from scratch, seed the processor: 
processors:
  drain:
    seed_templates:
      - "user <*> logged in from <*>"
      - "request <*> completed in <*> ms"
    warmup_min_clusters: 20
warmup_min_clusters holds back annotation until the processor has learned the specified number of templates, preventing a noisy first few minutes of data.

​
Tune the algorithm

The defaults work well for general application logs but produce poor results for very high-volumne logs with many distinct patterns, or very low-volume logs where the algorithm has little data to learn from. Two parameters do most of the work:
  • tree_depth: Controls how many leading tokens the tree splits on. Increase it (5–6) for logs where the first few tokens carry less signal, such as logs prefixed with a long timestamp or request ID. Decrease it (3) for short, structured logs.
  • merge_threshold: Controls how readily new messages merge into existing templates. Lower values (0.2–0.3) produce fewer, broader templates. Higher values (0.5–0.7) produce more, narrower templates. If you see hundreds of near-duplicate templates, lower the threshold.
max_clusters is a memory ceiling, not a quality control. Set it to roughly twice the number of stable templates you expect; LRU eviction handles the rest. For most services this is in the low hundreds. If a single source emits structured logs as a map, set body_field to the key that holds the human-readable message so the processor templates the right value.

​
Use templates in Honeycomb

Once log records carry log.record.template, you can use it like any other attribute:
  • Find dominant patterns: Run a query over your logs dataset with COUNT grouped by log.record.template, ordered descending. The top rows are your noisiest patterns.
  • Build SLOs and Triggers: Reference log.record.template = "<specific template>" in the SLI or Trigger condition. A stable template attribute makes these conditions resilient to changes in variable values that would otherwise require regex.
  • Drive Refinery sampling: Configure Refinery rules to drop or downsample high-volume templates while keeping a representative sample of each pattern.
  • Identify reduction candidates: A small number of templates accounting for a large fraction of log volume usually means easy reductions are available, either by dropping at the collector, by pairing with the log deduplication processor downstream, or by addressing the source.

​
Reduce volume by pairing with the log deduplication processor

A common and effective pattern is to put the logdedupprocessor immediately downstream of the drain processor. The drain processor attaches a stable template attribute to every record; the log deduplication processor then collapses records that share the same template within a time window into a single record annotated with a count. For high-volume, repetitive logs, this can reduce egress to Honeycomb by an order of magnitude or more while preserving the information needed to reason about pattern frequency. By default, the log deduplication processor matches on the full body, attributes, severity, and resource, which prevents two records with the same template but different variable values from collapsing. To deduplicate by template, restrict matching to the template attribute and any resource attributes you care about with include_fields. 
processors:
  drain:
    tree_depth: 4
    merge_threshold: 0.4

  log_dedup:
    interval: 60s
    log_count_attribute: log.record.count
    include_fields:
      - attributes["log.record.template"]
      - resource.attributes["service.name"]

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [drain, log_dedup, batch]
      exporters: [otlp/honeycomb]
Order matters. The drain processor must run before the log deduplication processor so the template attribute exists when deduplication evaluates its match fields. In Honeycomb, deduplicated records carry the count attribute (renamed to log.record.count in the example above) and first_observed_timestamp/last_observed_timestamp attributes that the log deduplication processor adds. Use SUM(log.record.count) rather than COUNT when summing volume across templates.
The log deduplication processor is also at alpha stability for logs. The same upgrade caveats apply.

​
Persist templates across restarts

By default, the processor’s learned templates are in-memory and are lost on collector restart. After a restart, the processor relearns templates and may produce different output until the algorithm converges again. To persist templates, configure a storage extension and reference it from the processor: 
extensions:
  file_storage:
    directory: /var/lib/otelcol/drain

processors:
  drain:
    storage: file_storage
    save_interval: 5m

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [otlp]
      processors: [drain, batch]
      exporters: [otlp/honeycomb]
save_interval controls how often state is snapshotted. Shorter intervals reduce data loss on crash at the cost of more disk activity.

​
Limitations

Keep these constraints in mind when planning your pipeline configuration:
  • The processor operates on logs only. Span and metric pipelines should not include it.
  • Drain is an online clustering algorithm; output is deterministic only when seeded with seed_templates or seed_logs. Two collectors without shared state will produce slightly different template inventories.
  • Without a storage extension, templates do not survive collector restarts.
  • Alpha stability means option names and defaults may change between collector releases. Validate after every upgrade.
These resources cover the tools and approaches that work alongside or instead of the drain processor: