1. Docs
  2. Send Data to Honeycomb
  3. Honeycomb Telemetry Pipeline
  4. Honeycomb Telemetry Pipeline Sources
  5. File

File [Ent]

Note
This feature is available as an add-on for the Honeycomb Enterprise plan. Please contact your Honeycomb account team for details.
Warning
This source offers a delete_after_read option that can be hazardous. When this option is combined with file globbing, it will delete every file that matches the globbing pattern. Use with caution and care.

Supported Platforms 

Platform Metrics Logs Traces
Linux ✓
Windows ✓
macOS ✓

Configuration Table 

Parameter Type Default Description
file_path* strings File or directory paths to tail for logs.
exclude_file_path strings "" File or directory paths to exclude.
log_type string “file” A friendly name that will be added to each log entry as an attribute.
parse_format enum none Method to use when parsing. Valid values are none, json, and regex. When regex is selected, ‘Regex Pattern’ must be set.
regex_pattern string The regex pattern that is used when parsing log entries.
multiline_line_start_pattern string Regex pattern that matches the beginning of a log entry for handling multiline logs.
multiline_line_end_pattern string Regex pattern that matches the end of a log entry, useful for terminating parsing of multiline logs.
parse_timestamp bool false Whether to parse the timestamp from the log entry.
timestamp_field string timestamp The field containing the timestamp in the log entry.
parse_timestamp_format enum ISO8601 The format of the timestamp in the log entry. Choose a common format, or specify a custom format. Options include “ISO8601”, “RFC3339”, “Epoch”, and “Manual”.
epoch_timestamp_format enum s The layout of the epoch-based timestamp. It’s required when parse_timestamp_format is set to “Epoch”. Options include “s”, “ms”, “us”, “ns”, “s.ms”, “s.us”, “s.ns”.
manual_timestamp_format string ‘%Y-%m-%dT%H:%M:%S.%f%z’ The strptime layout of the timestamp. It’s used when parse_timestamp_format is set to “Manual”.
timezone timezone UTC The timezone to use if the Timestamp Format doesn’t include a timezone. Otherwise, the timezone in the Timestamp Format will be respected. NOTE: This is also required to parse timezone abbreviations due to their ambiguity.
parse_severity bool false Whether to parse severity from the log entry.
severity_field string severity The field containing the severity in the log entry.
encoding enum utf-8 The encoding of the file being read. Valid values are nop, utf-8, utf-16le, utf-16be, ascii, and big5.
include_file_name_attribute bool true Whether to add the file name as the attribute log.file.name.
include_file_path_attribute bool false Whether to add the file path as the attribute log.file.path.
include_file_name_resolved bool false Whether to add the file name after symlinks resolution as the attribute log.file.name_resolved.
include_file_path_resolved bool false Whether to add the file path after symlinks resolution as the attribute log.file.path_resolved.
delete_after_read bool false Whether to delete the file(s) after reading. Only valid in combination start_at: beginning.
offset_storage_dir string $OIQ_OTEL_COLLECTOR_HOME/storage The directory where the offset storage file will be created. It is okay if multiple receivers use the same directory. By default, the observIQ Distro for OpenTelemetry Collector sets $OIQ_OTEL_COLLECTOR_HOME in its runtime.
poll_interval int 200 The duration of time in milliseconds between filesystem polls.
max_concurrent_files int 1024 The maximum number of log files from which logs will be read concurrently. If the number of files matched exceeds this number, then files will be processed in batches.
parse_to string body The field that the log will be parsed to. Some exporters handle logs favorably when parsed to attributes over body and vice versa.
start_at enum end Start reading the file from the ‘beginning’ or ’end’.
On this page: