> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Send JSON Logs

> Ingest and backfill JSON log files into Honeycomb using Honeytail. Learn how Honeycomb handles nested JSON structures.

export const honeytail = {
  "version": "1.10.0",
  "deb_amd64": {
    "sha256": "3db441215f97eaed068aa0531c986cf5405957e3e8e26b22c16b571091caf917",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail_1.10.0_amd64.deb"
  },
  "deb_arm64": {
    "sha256": "4220756e5a941cde6a484cb4cfde184eb189aaf29170df301a874eb143e960ed",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail_1.10.0_arm64.deb"
  },
  "rpm": {
    "sha256": "b23215a9301b20b2e2262a0823c9e761e8b57e1a62fd5cec35f697fce41fa863",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail-1.10.0-1.x86_64.rpm"
  },
  "bin_linux_amd64": {
    "sha256": "c9cc7dd1aa2b12afeb30b089061870f3407d2df0119e7c2807fec648b603e2d5",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail-linux-amd64"
  },
  "bin_linux_arm64": {
    "sha256": "1dd37227788548c4ed44592554e3c90e374c4d796c444dde9f372db8618bc7fa",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail-linux-arm64"
  },
  "bin_darwin_amd64": {
    "sha256": "9a3da0f48fe21b1e610ac6b63130dfb8118a9a0ec16abae13350edba02d85e4d",
    "url": "https://honeycomb.io/download/honeytail/v1.10.0/honeytail-darwin-amd64"
  },
  "bin_name": "honeytail"
};

export const SchemaTable = ({children}) => {
  return <div className="schema-table">
      {children}
    </div>;
};

Learn how to use Honeytail to send JSON logs directly to Honeycomb.

## Data Expectations

Honeycomb expects data with a flat structure.
By default, any structure deeper than top level keys will be serialized and a string representation of the content will be used in the field.
However, Honeycomb can automatically unpack nested JSON objects and flatten them into unique columns.
This is a per-dataset setting, and it is off by default.
You must be a team owner to change this setting.

If you enable this setting, nested objects will be flattened with new fields and field names created based on the keys.
For example, `{"outer": {"inner": 42}}` would become a field `outer.inner` with a value of 42.

To tell Honeycomb to automatically unpack JSON objects:

1. In the left navigation menu, select **Manage Data**.
2. In the list, locate and select **Datasets** and select the dataset you want to configure.
3. Select the **Schema** view.
4. Enable the **Automatically unpack non-OpenTelemetry nested JSON** toggle.
5. Select your preferred **Maximum unpacking depth** for your data.

Changes to this setting take effect within 60 seconds.

<Note>
  If your objects are deeply-nested, unpacking may result in a very large number of columns in Honeycomb. Consider unpacking only to the level of columns you will find useful. Any objects nested more deeply than the depth you select here will be converted to strings under the last unpacked column.
  In particular, if nested structures in your data can be created/added by your users (for example, HTTP headers), consider not unpacking them to that level.
</Note>

## Installation

Download and install the latest `honeytail` by running:

<Tabs>
  <Tab title="deb-amd64">
    Download the `honeytail_1.10.0_amd64.deb` package.

    ```shell theme={}
    wget -q https://honeycomb.io/download/honeytail/v1.10.0/honeytail_1.10.0_amd64.deb
    ```

    Verify the package.

    ```shell theme={}
    echo '3db441215f97eaed068aa0531c986cf5405957e3e8e26b22c16b571091caf917  honeytail_1.10.0_amd64.deb' | sha256sum -c
    ```

    Install the package.

    ```shell theme={}
    sudo dpkg -i honeytail_1.10.0_amd64.deb
    ```

    The packages install `honeytail`, its config file `/etc/honeytail/honeytail.conf`,
    and some start scripts.
    Build `honeytail` from source if you need it in an unpackaged form or for ad-hoc use.
  </Tab>

  <Tab title="deb-arm64">
    Download the `honeytail_1.10.0_arm64.deb` package.

    ```shell theme={}
    wget -q https://honeycomb.io/download/honeytail/v1.10.0/honeytail_1.10.0_arm64.deb
    ```

    Verify the package.

    ```shell theme={}
    echo '4220756e5a941cde6a484cb4cfde184eb189aaf29170df301a874eb143e960ed  honeytail_1.10.0_arm64.deb' | sha256sum -c
    ```

    Install the package.

    ```shell theme={}
    sudo dpkg -i honeytail_1.10.0_arm64.deb
    ```

    The packages install `honeytail`, its config file `/etc/honeytail/honeytail.conf`,
    and some start scripts.
    Build `honeytail` from source if you need it in an unpackaged form or for ad-hoc use.
  </Tab>

  <Tab title="rpm">
    Download the `honeytail_1.10.0-1.x86_64.rpm` package.

    ```shell theme={}
    wget -q https://honeycomb.io/download/honeytail/v1.10.0/honeytail_1.10.0-1.x86_64.rpm
    ```

    Verify the package.

    ```shell theme={}
    echo 'b23215a9301b20b2e2262a0823c9e761e8b57e1a62fd5cec35f697fce41fa863  honeytail_1.10.0-1.x86_64.rpm' | sha256sum -c
    ```

    Install the package.

    ```shell theme={}
    sudo rpm -i honeytail_1.10.0-1.x86_64.rpm
    ```

    The packages install `honeytail`, its config file `/etc/honeytail/honeytail.conf`,
    and some start scripts.
    Build `honeytail` from source if you need it in an unpackaged form or for ad-hoc use.
  </Tab>

  <Tab title="bin-linux-amd64">
    Download the 1.10.0 binary.

    ```shell theme={}
    wget -q -O honeytail https://honeycomb.io/download/honeytail/v1.10.0/honeytail-linux-amd64
    ```

    Verify the binary.

    ```shell theme={}
    echo 'c9cc7dd1aa2b12afeb30b089061870f3407d2df0119e7c2807fec648b603e2d5  honeytail' | shasum -a 256 -c
    ```

    Set the permissions to allow execution.

    ```shell theme={}
    chmod 755 ./honeytail
    ```
  </Tab>

  <Tab title="bin-linux-arm64">
    Download the 1.10.0 binary.

    ```shell theme={}
    wget -q -O honeytail https://honeycomb.io/download/honeytail/v1.10.0/honeytail-linux-arm64
    ```

    Verify the binary.

    ```shell theme={}
    echo '1dd37227788548c4ed44592554e3c90e374c4d796c444dde9f372db8618bc7fa  honeytail' | shasum -a 256 -c
    ```

    Set the permissions to allow execution.

    ```shell theme={}
    chmod 755 ./honeytail
    ```
  </Tab>

  <Tab title="bin-darwin-amd64">
    Download the 1.10.0 binary.

    ```shell theme={}
    wget -q -O honeytail https://honeycomb.io/download/honeytail/v1.10.0/honeytail-darwin-amd64
    ```

    Verify the binary.

    ```shell theme={}
    echo '9a3da0f48fe21b1e610ac6b63130dfb8118a9a0ec16abae13350edba02d85e4d  honeytail' | shasum -a 256 -c
    ```

    Set the permissions to allow execution.

    ```shell theme={}
    chmod 755 ./honeytail
    ```
  </Tab>

  <Tab title="source">
    Clone the [Honeytail](https://github.com/honeycombio/honeytail) repository.

    ```shell theme={}
    git clone https://github.com/honeycombio/honeytail
    ```

    Install from source.

    ```shell theme={}
    cd honeytail; go install
    ```
  </Tab>
</Tabs>

You should modify the config file and uncomment and set:

* `ParserName` to `json`
* `WriteKey` to your API key, available from [the account page](https://ui.honeycomb.io/account)
* `LogFiles` to the path for the log file you want to ingest, or `-` for stdin
* `Dataset` to the name of the dataset you wish to create with this log file.

## Launch the Agent

Start up a `honeytail` process using `upstart` or `systemd` or by launching the process by hand.
This will tail the log file specified in the config and leave the process running as a daemon.

<Tabs>
  <Tab title="upstart">
    ```shell theme={}
    sudo initctl start honeytail
    ```
  </Tab>

  <Tab title="systemd">
    ```shell theme={}
    sudo systemctl start honeytail
    ```
  </Tab>

  <Tab title="manual">
    ```shell theme={}
    honeytail -c /etc/honeytail/honeytail.conf
    ```
  </Tab>
</Tabs>

## Backfilling Archived Logs

To backfill existing data, run `honeytail` with `--backfill` the first time:

```shell theme={}
honeytail -c /etc/honeytail/honeytail.conf \
  --file /var/log/myapp/log12.json \
  --backfill
```

This command can also be used at any point to backfill from older, rotated log files.
You can read more about our [backfill behavior here](/send-data/logs/structured/honeytail/).

<Note>
  If you have chosen to backfill from old JSON logs, do not forget to transition into the default streaming behavior to stream live logs to Honeycomb!
</Note>

## Timestamp Parsing

Honeycomb expects all events to contain a timestamp field; if one is not provided, the server will associate the current time of ingest with the given payload.

By default, we look for a few candidate fields based on name (`"timestamp"`, `"time"`) and handle the following time formats:

* RFC3339 (`2006-01-02T15:04:05Z07:00`)
* RFC3339 with nanoseconds (`2006-01-02T15:04:05.999999999Z07:00`)
* Unix string representation (`Mon Jan 2 15:04:05 MST 2006`)
* Ruby string representation (`Mon Jan 02 15:04:05 -0700 2006`)
* Golang string representation (`2006-01-02 15:04:05.999999999 -0700 MST`)

If your timestamps are not correctly handled by the above formats,
use the `--json.timefield` and `--json.format` flags
to help `honeytail` understand where and how to extract the event's timestamp.

For example, given a JSON log file with events like the following:

```javascript theme={}
{"color":"orange","size":3,"server_time":"Aug 12 2016, 15:12:06 -0800"}
{"color":"blue","server_time":"Sep 01 2016, 06:10:32 -0800","size":4}
```

The command to consume those log lines (while retaining the `"server_time"` field as the event's timestamp)
would look something like:

```shell theme={}
honeytail --writekey=YOUR_API_KEY --dataset="API Server Logs" --parser=json \
  --file=/var/log/api_server.log \
  --json.timefield="server_time" --json.format="%b %d %Y, %k:%M:%S %z"
```

The `--json.timefield="server_time"` argument tells `honeytail` to consider the `"server_time"` value
to be the canonical timestamp for the events in the specified file.

The `--json.format` argument specifies the timestamp format to be used while parsing.
(It understands common [`strftime`](https://www.strfti.me/) formats.)

Ultimately, the above command would would produce events with the fields
(note the times below are represented in UTC; Honeycomb parses time zone information if provided).

<SchemaTable>
  | time                 | color  | size |
  | :------------------- | :----- | :--- |
  | 2016-08-12T23:12:06Z | orange | 3    |
  | 2016-09-01T14:10:32Z | blue   | 4    |
</SchemaTable>