honeycloudtrail to support ingestions of AWS CloudTrail
These logs are useful to monitor your trails and visualizing questions such as
“Which API request is this account ID making most frequently?" or “Are there
specific IPs trying to access our S3 buckets?"
The source is available on Github and instructions for getting started are provided here.
honeycloudtrail assumes access to an AWS access key ID and AWS secret access
key with the proper permissions. It will attempt to obtain these via the default
~/.aws/config, by the proper environment variables, or by an IAM
EC2 instance profile. See the AWS guide on providing
for more details.
See the provided IAM policy
JSON in the
honeyaws repository for one example of a policy which has the proper
permissions. This can be scoped down to more specific resources if desired.
Use the following instructions to install
honeycloudtrail. It is available as
part of the Honeycomb AWS Bundle or
as a standalone binary.
wget -q https://honeycomb.io/download/honeyaws/v1.4.0/honeyaws_1.4.0_amd64.deb && \ echo 'b15409f56e67e0058f2766554c25b441abadc386cb2fc0ce41f385b4be909e09 honeyaws_1.4.0_amd64.deb' | sha256sum -c && \ sudo dpkg -i honeyaws_1.4.0_amd64.deb
wget -q https://honeycomb.io/download/honeyaws/v1.4.0/honeyaws-1.4.0-1.x86_64.rpm && \ echo '1abce68c81380e3facc53ed8cb018b0980d6e906ace5ca8b403a251130e582e0 honeyaws-1.4.0-1.x86_64.rpm' | sha256sum -c && \ sudo rpm -i honeyaws-1.4.0-1.x86_64.rpm
wget -q -O honeycloudtrail https://honeycomb.io/download/honeyaws/v1.4.0/honeycloudtrail-linux-amd64 && \ echo '45bc50eeb40bc8705057c6e04ae9bc44fc2efa9cdbe6a851f0b3e02a45484523 honeycloudtrail' | sha256sum -c && \ chmod 755 ./honeycloudtrail
wget -q -O honeycloudtrail https://honeycomb.io/download/honeyaws/v1.4.0/honeycloudtrail-linux-arm64 && \ echo '1ebe673b76e030d67a4429c3140e42148e35e1cbe362b78588e6e912cdc0385c honeycloudtrail' | sha256sum -c && \ chmod 755 ./honeycloudtrail
wget -q -O honeycloudtrail https://honeycomb.io/download/honeyaws/v1.4.0/honeycloudtrail-darwin-amd64 && \ echo 'd6463fecaf6b233f32b971c553960c598c733b3fdfe31b4957116d246d4fe7e0 honeycloudtrail' | shasum -a 256 -c && \ chmod 755 ./honeycloudtrail
honeycloudtrail interactively (for beginning exploration,
debugging credential management, etc) or as a daemon. Try running some commands
interactively at first to get a feel for using the tool and then configure it to
run as a proper system service when you’re ready to be ingesting continuously.
To show all trails, invoke
$ honeycloudtrail ls s3-trail elb-frontend-trail service-trail ...
To ingest access logs from a distribution, use
honeycloudtrail ingest with
one or more distribution names. Set your Honeycomb write
key with the
--writekey flag. By default the
events will be sent to a dataset called
Note: If an S3 bucket is not configured for the trail it will throw an error. See the documentation on CloudTrail to learn how to enable S3 bucket logging.
e.g Ingesting logs from one trail named
$ honeycloudtrail --writekey=YOUR_API_KEY \ ingest s3-trail ...
Ingesting logs from multiple specific load balancers (named
$ honeycloudtrail --writekey=YOUR_API_KEY \ ingest s3-trail elb-frontend-trail service-trail ...
honeycloudtrail ingest without any arguments will ingest all available CloudTrail trails
in your configured AWS region. With arguments, it will ingest
logs for the specified trail names.
Sampling is a great way to send fewer events (thereby keeping more history and
reducing costs) while still preserving most relevant information. To set a
sample rate while using one of the Honeycomb AWS tools, use the
flag. While the tools run, this base rate will be automatically adjusted by the
Honeycomb AWS tools using dynamic sampling to keep more interesting traffic at a
For instance, setting the sample flag to 20 will send 1 out of every 20 requests
processed to Honeycomb by default. Fields such as
elb_status_code are used to
lower this ratio for rarer, but relevant, events such as HTTP 500-level errors.
$ honeyelb --samplerate 20 ... ingest ...
honeycloudtrail, while supporting a interactive workflow for initial discovery and
experimentation, is meant to be invoked as a long-running process by a system
To do this, edit the system init files (Upstart and systemd are supported) installed by the package manager to add the API key.
Once you receive data from
honeycloudtrail you will want to explore it. The
descriptions of the sent fields is available in the AWS documentation for CloudTrail logs.
Here are some suggestions for things to try:
COUNT_DISTNCT(SourceIPAddress)to see which IPs are using which services.
ARNto see which IAM roles are being used the most frequently
EventNameto see the most commonly used API calls