If you have a number of old log files that you’d like to load into Honeycomb, use
honeytail with the
Note: honeytail does not unzip log files, so you’ll need to do this before backfilling.
Here’s an example
honeytail invocation to pull in multiple existing logs and as much as the current log as possible.
honeytail \ -c /etc/honeytail/honeytail.conf \ --file=/var/log/app/myapp.log.* \ --file=/var/log/app/myapp.log \ --backfill
Let’s break down the various parts of this command.
--parser=json: For the purposes of this example, all logs are already JSON formatted. Take a look at the timestamp section of the JSON connector to make sure your historical logs have their times interpreted correctly.
--file=/var/log/app/myapp.log.*: Honeycomb understands file globs and will ingest all of the files in series.
--file=/var/log/app/myapp.log: Specify the
--file(or its short form,
-f) as many times as necessary to include additional files that don’t match a glob. Ingest as much of the current file as exists.
--backfill: This flag tells
honeytailto read the specified files in their entirety, stop when finished reading, and to respond to rate limited responses (HTTP 429) by slowing down the rate at which it sends events.
Honeytail will read all the content in all the old logs and then stop. When it finishes, you’re ready to send new log lines. By default,
honeytail will keep track of its progress through a file, and if interrupted, will pick back up where it left off. By launching honeytail pointing at the main app log, it will find the state file it created while reading in the backlog and start up where it left off.
Here’s the second
honeytail invocation, where it will tail the current log file and send in recent entries:
honeytail \ --writekey=YOUR_API_KEY \ --parser=json \ --dataset='My App' \ --file=/var/log/app/myapp.log
You are currently logged in to the
so we have populated the write key here to the first write key for that team.
You can find more detail about
honeytail options on the Honeytail page.