1. Docs
2. Triggers
3. PagerDuty

Triggers with PagerDuty

Using the PagerDuty + Honeycomb Integration 

The PagerDuty + Honeycomb integration uses Honeycomb Triggers to notify on-call responders based on alerts sent from Honeycomb. This integration is maintained and supported by Honeycomb.

Requirements 

• a user account in your team’s PagerDuty organization
• a user account in your Honeycomb team

Integration Walkthrough 

In PagerDuty 

You will create an integration linked to a specific service. From the PagerDuty Configuration menu, select Services to get to the Service Directory. Your goal is to create an Integration Key which you can use for Honeycomb.

If you are creating a new service in PagerDuty, follow the instructions at PagerDuty’s Configuring Services and Integrations.

1. Once you have an existing service, click the name of the service you want to add the integration to. Then, select the Integrations tab and click the New Integration button.

2. Enter an Integration Name in the format monitoring-tool-service-name (Honeycomb-Shopping-Cart,for example) and select Honeycomb from the Integration Type menu.

3. Click the Add Integration button to save your new integration. You will be redirected to the Integrations tab for your service.

4. An Integration Key will be generated on this screen. Keep this key saved in a safe place, as it will be used when you configure the integration with Honeycomb in the next section.

   

In Honeycomb 

Adding the integration key for your PagerDuty service

1. Navigate to Team Settings and click on the Integrations tab.
2. Under Trigger Recipients, click Add Integration.
3. Select PagerDuty as the integration type. Fill out the form with a name that will be easy to find when you configure individual triggers later on, and paste the Integration Key you generated in PagerDuty.
4. Click Add.

Configuring Honeycomb Triggers to Alert PagerDuty 

1. Navigate to the Triggers page in the Honeycomb UI.
2. Click New Trigger to create a new trigger, or click on an existing trigger’s name to edit it.
3. Under Recipients, click Add Recipient.
4. You will see a pop-up form titled Add Trigger Recipient. In the Recipient dropdown, choose your service’s PagerDuty integration and desired severity, as Critical is the default, and select Add.
5. Click Save Trigger.

Configuring Honeycomb SLO Burn Alerts to Alert PagerDuty 

1. Navigate to the SLOs page in the Honeycomb UI.
2. Create an SLO if you do not already have one by clicking New SLO and filling out the form. (More detail on the SLOs page)
3. Find your SLO in the list, and click the Configure button under the Burn Alerts column.
4. Click New Burn Alert.
5. In the Create Burn Alert form, set your desired exhaustion time and choose your PagerDuty integration in the Notify dropdown. Set your desired Severity value, as Critical is the default value. Select Create Burn Alert.

PagerDuty Alert from Honeycomb 

When a configured trigger or burn alert fires, expect to see a display in PagerDuty similar to the example below.

Removing Integrations 

In PagerDuty 

You may need admin privileges to delete an integration in PagerDuty.

1. Navigate to the Service Directory in PagerDuty.
2. Locate the service that contains your Honeycomb integration.
3. Click on the Integrations tab.
4. Click the gear icon for the integration and select Delete.

In Honeycomb 

1. Navigate to Team Settings and click on the Integrations tab.
2. Find your PagerDuty integration under Trigger Recipients and click the Edit button.
3. In the form editor, click Remove.

Troubleshooting 

If you are not receiving alerts from PagerDuty when a trigger fires, examine your PagerDuty escalation policy. Ensure that a responder is always on-call in the escalation policy in PagerDuty. If there are gaps in your escalation policy schedule, PagerDuty will not create an incident if no responder is on-call during the moment the trigger fires.