> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeycomb.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Filter and Send Logstash Data

> Fork your Logstash pipeline to send a copy of all log traffic to Honeycomb using Logstash's flexible plugin architecture and HTTP output plugin.

Thanks to [Logstash](https://www.elastic.co/downloads/logstash)'s flexible plugin architecture, you can send a copy of all the traffic that Logstash is processing to Honeycomb.
This topic explains how to use Logstash plugins to convert incoming log data into events and then send them to Honeycomb.

<Tip>
  If your system uses logspout as a log router for Docker containers, you can send logs to Honeycomb with one of the [logspout third-party modules](https://github.com/gliderlabs/logspout#third-party-modules) that integrates with [logstash](/integrations/logs/logstash/) or [fluentd](/integrations/logs/fluentd/).
</Tip>

## Data Format Requirements

<Tip>
  Honeycomb is at its best when the events you send are broad and capture lots of information about a given process or transaction.
</Tip>

To process the log data coming into Logstash into Honeycomb events, you can use Logstash filter plugins.
These filter plugins transform the data into top-level keys based on the original source of the data.
We have found these to be especially useful:

* [grok](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) matches regular expressions and has configs for many common patterns (such as the apache, nginx, or haproxy log format).
* [json](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) matches JSON-encoded strings and breaks them up in to individual fields.
* [kv](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html) matches `key=value` patterns and breaks them out into individual fields.

To add and configure filter plugins, refer to [Working with Filter Plugins](https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html) on the Logstash documentation site.

### Example: Using Logstash Filter Plugins to Process Haproxy Logs for Honeycomb Ingestion

Let us say you are sending haproxy logs (in HTTP mode) to Logstash.
A log line describing an individual request looks something like this (borrowed from the [haproxy config manual](https://www.haproxy.org/download/1.6/doc/configuration.txt)):

```log theme={}
Feb  6 12:14:14 localhost \
          haproxy[14389]: 10.0.1.2:33317 [06/Feb/2009:12:14:14.655] http-in \
          static/srv1 10/0/30/69/109 200 2750 - - ---- 1/1/1/1/0 0/0 {1wt.eu} \
          {} "GET /index.html HTTP/1.1"
```

Logstash puts this line in a `message` field, so in the filter parameter of the `logstash.yaml` config fragment below, we use the `grok` filter plugin and tell it to parse the message and make all the content available in top-level fields.
And, since we do not need it anymore, we tell `grok` to remove the `message` field.

The `mutate` filter plugin takes the numeric fields extracted by haproxy and turns them into integers so that Honeycomb can do math on them (later).

```ruby theme={}
filter {
  grok {
    match => ["message", "%{HAPROXYHTTP}"]
    remove_field => ["message"]
  }
  mutate {
    convert => {
      "actconn" => "integer"
      "backend_queue" => "integer"
      "beconn" => "integer"
      "bytes_read" => "integer"
      "feconn" => "integer"
      "http_status_code" => "integer"
      "retries" => "integer"
      "srv_queue" => "integer"
      "srvconn" => "integer"
      "time_backend_connect" => "integer"
      "time_backend_response" => "integer"
      "time_duration" => "integer"
      "time_queue" => "integer"
      "time_request" => "integer"
    }
  }
}
```

## Sending Data to Honeycomb

Now that all the fields in the `message` are nicely extracted into events, send them on to Honeycomb!
To send events, configure an output plugin.

You can use [Logstash's HTTP output plugin](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-http.html) to craft HTTP requests to the Honeycomb API.

This configuration example sends the data to a dataset called "logstash."

```ruby theme={}
filter {
  ruby {
    code => 'event.to_hash.each { |k, v| event.set("[data][" + k + "]" , v) }'
  }
  prune {
    whitelist_names => [ "^data$" ]
  }
}
output {
  http {
    url => "https://api.honeycomb.io/1/batch/logstash" # US instance
    #url => "https://api.eu1.honeycomb.io/1/batch/logstash" # EU instance
    http_method => "post"
    headers => {
      "X-Honeycomb-Team" => "YOUR_API_KEY"
    }
    format => "json_batch"
    http_compression => true
  }
}
```

To complete configuration in the example above:

* Use `filter` to nest the Logstash JSON fields under a `data` element in the JSON payload to Honeycomb. This filter is **required** for Honeycomb to ingest your Logstash logs. Learn more about `filter` in the [Elastic documentation](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html).
* Specify a URL (`url`) to send the data to:
  * for our US instance: `https://api.honeycomb.io/1/batch/<dataset_name>`
  * for our EU instance: `https://api.eu1.honeycomb.io/1/batch/<dataset_name>`
* Add your Honeycomb [API key](/configure/environments/manage-api-keys/) to `"X-Honeycomb-Team"` so that Logstash is authorized to send data to Honeycomb.
* Specify the output format as JSON batch (`json_batch`).
* Specify the use of HTTP compression (`http_compression => true`).

Then, restart Logstash.
When it is back up, you will find the new dataset on [your landing page](https://ui.honeycomb.io/).

### Set Event Timestamps

In Logstash, each event has a special `@timestamp` field.
In general, use the [date filter plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html) to extract the time attribute from log lines.

For example, if you have a JSON log line containing timestamps in the format:

```json theme={}
{"timestamp": "2018-02-04T14:55:10Z", "host": "app22", ...}
```

Then, extract the time value using the following Logstash configuration:

```ruby theme={}
filter {
  mutate{
    rename => {"timestamp" => "time"}
  }
  date {
    match => ["time", "ISO8601"]
  }
  ruby {
    code => 'event.to_hash.each { |k, v| event.set("[data][" + k + "]" , v) unless k == "time" }'
  }
  prune {
    allowlist_names => [ "^data$", "^time$" ]
  }
}
output {
  http {
    url => "https://api.honeycomb.io/1/batch/logstash" # US instance
    #url => "https://api.eu1.honeycomb.io/1/batch/logstash" # EU instance
    http_method => "post"
    headers => {
      "X-Honeycomb-Team" => "YOUR_API_KEY"
    }
    format => "json_batch"
    http_compression => true
  }
}
```