- Start investigations automatically: Configure Honeycomb Triggers and SLO burn alerts to start AWS DevOps Agent investigations through a webhook.
- Give the agent trace-level context: AWS DevOps Agent queries Honeycomb telemetry through Honeycomb’s remote MCP server while it investigates, pulling in distributed trace data like latency percentiles, error rates, span attributes, and derived columns from your environments.
How it works
Honeycomb connects to AWS DevOps Agent through two independent mechanisms: an agent space can use either one without the other, though most setups use both.Telemetry introspection via MCP
AWS DevOps Agent queries Honeycomb telemetry through a registered MCP server while it investigates.How the connection works
AWS DevOps Agent connects to Honeycomb’s remote MCP server as a custom MCP server registration, authenticated with a Management API key scoped to read-only access. When an investigation needs telemetry context, the agent sends queries through this connection using Honeycomb’s standard query model: the same aggregations, filters, and time ranges available in the Honeycomb UI. Honeycomb returns results over the same connection; the agent never writes to Honeycomb, and its visibility is bounded by the API key’s scopes and the Honeycomb team that issued it. The credential is shared across everyone with access to the agent space; Honeycomb doesn’t distinguish between individual AWS users.Each registration connects to exactly one Honeycomb team; to give an agent space access to more than one team, register each team separately and enable all of them in that agent space.
What this gives an investigation
With this connection, an investigation can:- Break latency down per service and per endpoint, at the percentiles you query on in Honeycomb.
- Separate request latency from long-lived connections, which otherwise appear as extreme duration outliers.
- Attribute tail latency to a specific code path or downstream dependency.
- Cite the traces and span attributes behind a conclusion, so an operator can open the same view in Honeycomb.
Investigation triggering via webhook
A Honeycomb Trigger or SLO burn alert can start an AWS DevOps Agent investigation automatically, through a webhook.How Honeycomb sends the alert
Honeycomb Triggers and SLO burn alerts use a webhook Recipient to notify AWS DevOps Agent when a condition fires. Honeycomb authenticates each webhook delivery with a bearer token in the Authorization header, which AWS DevOps Agent generates during webhook creation. Honeycomb renders a payload template using Go template syntax, populating fields likepriority and description from the firing alert, and sends it to the webhook endpoint.
Honeycomb expects a response within 15 seconds and retries a failed delivery only for a narrow set of conditions: a 408 or 502 response, a 429 or 503 carrying a Retry-After header, or a connection-level failure.
How AWS DevOps Agent handles it
AWS DevOps Agent validates the payload, then starts a new investigation, deduplicating repeated deliveries for the same alert cycle using theincidentId field.
Before you begin
Make sure you have the following before you connect Honeycomb:- An agent space. If you haven’t created one, visit AWS’s documentation on Creating an Agent Space.
- A Honeycomb account with permission to create Management API keys and manage team integrations. In Honeycomb, both require the Team Owner role.
- The AWS DevOps Agent console open in a region where the service is available.
Set up telemetry introspection
Set up telemetry introspection so AWS DevOps Agent can query Honeycomb trace data while it investigates, instead of working from AWS telemetry alone.1
Create a Honeycomb Management API key
Honeycomb’s MCP server authenticates with a Management API key, a two-part credential: a Key ID and a Key Secret.
-
In Honeycomb, create a Management API key under Account > Team Settings > API Keys.
When creating your key:
-
Name the key something that identifies its purpose, so it’s easy to find later (for example,
aws-devops-agent). -
Grant these permissions:
Read access covers everything AWS DevOps Agent needs: it queries Honeycomb telemetry but doesn’t modify datasets, Triggers, SLOs, or Boards.
-
Name the key something that identifies its purpose, so it’s easy to find later (for example,
- Copy both the Key ID and the Key Secret; you will need them when you register the MCP server in the next step.
2
Register and authorize the Honeycomb MCP server
Register and authorize the Honeycomb MCP server to make it available to every agent space in the account.
You will enable the registration in a specific agent space in the next step.
- In the AWS DevOps Agent console, select Capability Providers from the side navigation.
- Locate the MCP Servers section, and select Register MCP Server.
-
On the Server details page, enter details:
- Select Next.
- On the Authorization flow page, select API key.
- Select Next.
-
On the Authorization configuration page, enter the following:
- Review and submit.
3
Enable Honeycomb in an agent space
Enable the Honeycomb MCP server in each agent space that should use it.
- In the AWS DevOps Agent console, from the agent spaces page, select an agent space, then select View details.
- Select the Capabilities tab.
- Locate the MCP Servers section.
- Select Add.
- Select the Honeycomb registration you want to enable.
- Select Next.
- Review and select Save.
4
Verify the configuration
Open the agent through Operator access on the agent space page and send the following prompt:You should receive a response naming your Honeycomb team and its environments, which confirms that the registration and the credential are both correct.
Set up investigation triggering
Set up investigation triggering so a Honeycomb Trigger or SLO burn alert can start an AWS DevOps Agent investigation automatically, without anyone needing to notice the alert and kick one off manually.1
Generate a webhook
Generate a webhook so Honeycomb has a destination to send alerts to, and a credential to authenticate the request.
- In the AWS DevOps Agent console, from the agent spaces page, select your agent space, then select View details.
- Select the Capabilities tab.
- In the Webhook section, select Configure.
- Select Generate webhook.
- For Webhook authentication type, choose API key.
- Copy the webhook endpoint URL and the API key.
2
Create the webhook integration in Honeycomb
Create a webhook integration in Honeycomb to connect the webhook you generated in the previous step to your Triggers and SLO burn alerts.
To learn more about how webhook recipients work, visit Send Alerts to Webhooks.
-
Create the integration.
- In Honeycomb, select Account > Team Settings from the navigation menu.
- Select the Integrations view.
- Locate Trigger and SLO Recipients and select Add Integration.
- Enter integration details:
-
Add the authorization header.
- Select the Headers tab.
- Select Add header.
- Add header details:
-
Add the payload template.
- Select the Payload tab.
- Toggle the Enable switch on for Triggers. A webhook can be used only with an alert type whose payload is enabled.
-
Replace the contents of the text area with this template:
AWS field to Honeycomb value mapping
To review the full list of available variables, visit Custom Webhook Variables. To reviewtoJsonand other transformations, visit Custom Webhook Functions. - Select Add.
3
Add the webhook as a Trigger Recipient
Add the webhook you created as a Recipient on each Trigger whose alerts should start an investigation.
- In Honeycomb, select Triggers from the navigation menu.
- Select an existing Trigger, or select New Trigger. For the full trigger creation flow, including how to define the query and alert condition, visit Create a Trigger.
- In the Recipients section, select Add Recipient.
- From the Recipient dropdown, select your webhook integration.
- Select Add, then select Save Trigger.
4
Verify the configuration
Select Test in the Trigger editor and confirm the following:
- Honeycomb reports that the delivery succeeded.
A
403 Forbiddenresponse points to a problem with theAuthorizationheader. - An investigation starts in your agent space.
A
200response without an investigation means the payload was accepted and then failed validation. The most common causes are apriorityvalue that isn’t one of the five literal strings, and a free-text field interpolated withouttoJson.
Troubleshooting
If AWS DevOps Agent isn’t behaving as expected, check the symptom against the likely cause before opening a support case.Telemetry introspection
Most connection problems trace back to how the API key was entered during registration, not the key itself.”Invalid input” error when saving the authorization configuration
This happens when the full header string, rather than just the header name, was entered in the API Key Header field. EnterAuthorization alone in that field, and move the rest of the value to API Key Value.
Registration saves, but the agent reports it can’t reach Honeycomb
The API Key Value is malformed. Confirm the value isBearer <KEY_ID>:<KEY_SECRET>: both halves present, joined by a colon, with a single space after Bearer and no wrapping quotation marks or whitespace.
Agent authenticates, but reports no environments
The key is missing the Environments (Read) scope, or it belongs to a different Honeycomb team than expected. Check the key’s scopes in Honeycomb, and confirm which team issued it.Agent lists environments, but returns no data for a service
The service isn’t sending traces to the environment the key can see, or the query window predates the data. Confirm in the Honeycomb UI that the dataset holds spans for the time range in question.Requests fail from an EU-hosted team
The registration used the US endpoint instead of the EU one. Re-register usinghttps://mcp.eu1.honeycomb.io/mcp.
To rotate the credential:
- Create a new Management API key in Honeycomb.
- Edit the registration’s authorization configuration with the new value.
- Verify with the same prompt used during setup.
- Delete the old key in Honeycomb.
Trigger investigations
Most delivery problems either get an explicit rejection, like a403, or fail silently with no error at all, so check the response code before assuming the payload itself is wrong.
Honeycomb reports “403 Forbidden” on every delivery
TheAuthorization header sent with the webhook doesn’t match what AWS DevOps Agent expects.
Delete the header on the Headers tab and re-enter it as Bearer <API_KEY>.
”403 Forbidden” persists after you re-enter the header
The webhook credentials were removed or regenerated on the AWS DevOps Agent side, or the endpoint URL doesn’t match the webhook that was generated. Regenerate the credentials in the Webhook section of the Capabilities tab, then update both the URL and the header in Honeycomb.Honeycomb doesn’t retry
403 responses, so correcting the header only takes effect on the next delivery.
Use Test in the Trigger editor to confirm the fix instead of waiting for a real alert to fire.A 200 response comes back, but no investigation starts
Honeycomb delivered the payload successfully, but AWS DevOps Agent rejected it during validation.
This usually means the priority value isn’t one of the five literal strings (CRITICAL, HIGH, MEDIUM, LOW, MINIMAL), or a free-text field like title or description was interpolated into a quoted string instead of passed through toJson, producing invalid JSON.
Confirm the priority value on the Payload tab matches one of the five literal strings, and check any free-text field for missing toJson wrapping.
Deliveries fail intermittently with no clear pattern
Honeycomb expects a response within 15 seconds and retries a delivery only for a408, 502, a 429 or 503 carrying a Retry-After header, or a connection-level failure such as a refused connection.
If AWS DevOps Agent is slow to acknowledge the request, or returns any other status while under load, the delivery fails without a retry.
Make sure the webhook endpoint acknowledges the request quickly and handles any slower work asynchronously.
A repeated alert doesn’t start a new investigation
Deliveries within the same alert cycle carry the sameincidentId, so AWS DevOps Agent treats them as updates to the existing investigation instead of starting a new one.
This is expected behavior; a new investigation starts when the Trigger recovers and fires again.
The webhook integration doesn’t appear as an option on an SLO burn alert
Only the Triggers payload type is enabled on the integration, and a webhook can only be used with alert types whose payload is configured. Enable the Budget Rate Burn or Exhaustion Time Burn payload on the Payload tab. To rotate the webhook API key, remove the existing credentials in the Webhook section of the Capabilities tab, generate new ones, and update theAuthorization header in Honeycomb.
Requests succeed again once the new header is saved.
An investigation starts, then gets cancelled immediately
This usually means your AWS account has hit its monthly investigation limit. Contact your AWS account team to request a rate limit change.Removing the integration
Honeycomb connects at two levels: the agent space level and the account level. To remove it completely, remove it from all agent spaces first, then unregister it. The webhook is a separate credential; remove each part you configured.1
Remove the Honeycomb webhook integration
Delete the integration in Honeycomb to remove it from every Trigger and SLO that used it.
- In Honeycomb, select Account > Team Settings from the navigation menu.
- Select the Integrations view.
- Locate the Trigger and SLO Recipients section.
- Find your webhook integration, then select Edit.
- In the form editor, select Remove.
2
Remove the webhook credentials
Remove the webhook credentials to stop Honeycomb alerts from triggering new investigations.
- In the AWS DevOps Agent console, from the agent spaces page, select the agent space, then select View details.
- Select the Capabilities tab.
- In the Webhook section, select Configure.
- Choose Remove.
3
Remove Honeycomb from each agent space
Remove the registration from an agent space to stop that space from querying Honeycomb, without affecting other agent spaces or the account-level registration.
- In the AWS DevOps Agent console, from the agent spaces page, select the agent space, then select View details.
- Select the Capabilities tab.
- Locate the MCP Servers section.
- Select your Honeycomb registration, then select Remove.
4
Deregister from the account
Once no agent space is using Honeycomb, deregister Honeycomb as an available MCP server for the entire account.
- In the AWS DevOps Agent console, select Capability Providers from the side navigation.
- Locate the Currently registered section.
- Check that the agent space count is zero. If it isn’t, repeat the previous step in your other agent spaces.
- Select your Honeycomb registration, then choose Deregister from the Actions menu.
5
Delete the Honeycomb credential
Deregistering the MCP server doesn’t revoke the credential it used, so delete the Management API key in Honeycomb as a separate, final step.
- In Honeycomb, select Account > Team Settings from the navigation menu.
- Select the API Keys view.
- Locate the Management API key you created for this integration and select it.
- Select Delete.
- In the Delete Management API Key? modal, enter the name of the key, then select Delete.
Next steps
Continue setting up AWS DevOps Agent with these resources:- AWS DevOps Agent IAM Permissions: Review who can register capability providers in your organization.
- Connecting MCP Servers: See the full range of ways to connect an MCP server to AWS DevOps Agent.
- Connecting to Privately Hosted Tools: Configure a private network proxy for AWS DevOps Agent.