Amazon’s Relational Database Service (RDS) lets you use a number of databases without having to administer them yourself. The Honeycomb RDS connector gives you access to the same data as if you were running MySQL on your own server.
The Honeycomb RDS connector surfaces attributes like:
Honeycomb allows you to calculate metrics and statistics on the fly while retaining the full-resolution log lines (and the original MySQL query that started it all).
Once you have got data flowing, be sure to take a look at our starter queries; our entry points provide our recommendations for comparing lock retention by normalized query, scan efficiency by collection, or read vs. write distribution by host.
This integration is a Lambda function subscribed to your instance’s RDS Log Group. It parses log events as they arrive and submits them to Honeycomb. Note that configuring your RDS instance to send its logs to Cloudwatch will incur additional AWS costs - see the Cloudwatch Pricing docs for details.
Before installing the integration, configure MySQL running on RDS to output the slow query log to a file. Refer to Amazon’s documentation on setting Parameter Groups to get started, and find more detail about the configuration options below in the MySQL docs for the slow query log.
Note: Audit logs are currently not supported. These instructions indicate setup for slow query logs.
Set the following options in the Parameter Group:
log_output to FILEslow_query_log to 1long_query_time to 0log_queries_not_using_indexes to 1 (optional)If you switch to a new Parameter Group when you make these changes, make sure you restart the database.
Note: Enabling slow query logs for all queries can generate a lot of log data, and can have an adverse impact on the performance of your RDS instance.
If your database already has a lot of traffic, consider starting out by only logging longer queries (for example, setting long_query_time to 0.1 or higher) before logging everything.
You may also wish to consider sampling your data.
Next, enable publishing of MySQL slow query logs to AWS Cloudwatch Logs. You can do this in the RDS console in the instance configuration. See the AWS docs for full details.
This change can be done without instance downtime. Once you have made the above changes, verify that logs are being received by Cloudwatch Logs via the Cloudwatch Logs Console.
The MySQL integration exists as an AWS Lambda function deployed in your AWS account. It subscribes to the Cloudwatch Log Group created by RDS, parses log lines, and submits them as events to Honeycomb. You can view the source here.
To install the integration, you will need:
Visit the AWS CloudFormation Stack creation wizard.
This opens the page with the Amazon S3 URL field populated with https://s3.amazonaws.com/honeycomb-builds/honeycombio/integrations-for-aws/LATEST/templates/cloudwatch-rds-mysql.yml
Confirm that the current AWS region contains the database.
Select “Next”. Specify the following stack details:
When installing the integration, you must supply your Honeycomb API Key via Cloudformation parameter. Cloudformation parameters are not encrypted, and are plainly viewable to anyone with access to your Cloudformation stacks or Lambda functions. For this reason, we strongly recommend that your Honeycomb API Key be encrypted. To encrypt your key, use AWS’s KMS service.
First, you will need to create a KMS key if you do not have one already. The default account keys are not suitable for this use case.
$ aws kms create-key --description "used to encrypt secrets"
{
"KeyMetadata": {
"AWSAccountId": "123455678910",
"KeyId": "a38f80cc-19b5-486a-a163-a4502b7a52cc",
"Arn": "arn:aws:kms:us-east-1:123455678910:key/a38f80cc-19b5-486a-a163-a4502b7a52cc",
"CreationDate": 1524160520.097,
"Enabled": true,
"Description": "used to encrypt honeycomb secrets",
"KeyUsage": "ENCRYPT_DECRYPT",
"KeyState": "Enabled",
"Origin": "AWS_KMS",
"KeyManager": "CUSTOMER"
}
}
# optionally, create an alias for the KMS key to describe the key's usage:
$ aws kms create-alias --alias-name alias/secrets_key --target-key-id=a38f80cc-19b5-486a-a163-a4502b7a52cc
Save a file containing only your Honeycomb API Key to be passed into the encryption step.
For example, if abc123 is your Honeycomb API Key and my-key is the name of the file, create the file like this:
echo -n abc123 > my-key
Next, encrypt your Honeycomb API Key:
$ aws kms encrypt --key-id=a38f80cc-19b5-486a-a163-a4502b7a52cc --plaintext fileb://my-key
{
"CiphertextBlob": "AQICAHge4+BhZ1sURk1UGUjTZxmcegPXyRqG8NCK8/schk381gGToGRb8n3PCjITQPDKjxuJAAAAcjBwBgkqhkiG9w0BBwagYzBhAgEAMFwGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQM0GLK36ChLOlHQiiiAgEQgC9lYlR3qvsQEhgILHhT0eD4atgdB7UAMW6TIAJw9vYsPpnbHhqhO7V8/mEa9Iej+g==",
"KeyId": "arn:aws:kms:us-east-1:702835727665:key/a38f80cc-19b5-486a-a163-a4502b7a52cc"
}
Record the CiphertextBlob and the last part of the KeyId from the encryption step.
In the example above, the last part of the KeyId is a38f80cc-19b5-486a-a163-a4502b7a52cc.
Enter the CiphertextBlob into the Cloudformation template as the HoneycombWriteKey.
Enter the KeyId into the Cloudformation template as the KMSKeyId.
For more information about the need to use fileb:// prefix, see the AWS Reference Guide.
To hash the concrete query, set the ScrubQuery parameter to true when installing the integration.
The normalized_query attribute will still be representative of the shape of the query and identifying patterns (including specific queries) will still be possible, but the sensitive information will be completely obscured before being submitted to our API.
Did you find what you were looking for?