Skip to content

Alerts: Feature Guide

Minimum configuration

When creating the first alert you will be guided through configuring everything necessary for alerting to be enabled. Overall, the minimum general configuration needed is:

  • Configure an integration for sending notifications in Integrations configuration.
  • For Slack, make sure you invite the @bitdrift Slack app into the channel(s) you want to send the notification to.
  • Configure a notification group that will select which channel will the notification fire into.
  • When creating an alert (by selecting Create alert in the top right of any chart in any workflow) select the desired setting and a notification group created earlier.

Create a Basic Alert

To create an alert, click on the options button of any workflow-powered Plot Chart. Configure the alert by providing a rule name, description, time window, and threshold. Then, select a notification group to determine where alerts will be sent.

Example Create Alert

Create a SLO Alert

For background on SLO alerts, first see the following chapters in the Google SRE book:

Example Create SLO Alert

Defining an SLO alert requires the following:

  • SLO alerts can only be defined on Rate Charts. The chart is assumed to be <good events> / <total events>
  • Select an SLO window (7 or 30 days is supported)
  • Select an SLO target percentage
  • Select the settings for Multiwindow, Multi-Burn-Rate (MWMBR) that apply to the alert. Configuring these settings is nuanced and discussed in detail in the Google SRE book chapter linked above. To aid users we supply default settings for both 7 and 30 day SLOs that are likely a good starting point for most users.

When customizing MWMBR windows, the following settings are required for each window:

  • The short time window
  • The long time window
  • The burn rate

Given the long window and the burn rate, the system will display the error budget % that would be consumed over the long window before the alert fires.

30 day SLO alerts default to the following settings:

Long Window Short Window Burn Rate Error Budget Consumed
1h 5m 14.4 2%
6h 30m 6 5%
3d 6h 1 10%

7 day SLO alerts default to the following settings:

Long Window Short Window Burn Rate Error Budget Consumed
1h 5m 16.8 10%
6h 30m 5.6 20%
1d 2h 2.8 40%

Alerts Home Page

The Alerts Home Page lets you monitor and filter alerts by status and ownership. It displays details such as the last fired timestamp, the user who created the alert, and its current status. You can also disable, edit, or delete alerts as needed.

Example Health Dashboard

Notification Groups

Manage notification groups from Company Settings. A notification group represents one or more delivery targets for alerts and can mix different target types. Bitdrift supports Slack channels and Pager Duty services. You may include any combination (e.g. a single group that posts to two Slack channels and triggers a Pager Duty incident).

Adding Slack Targets

To include Slack delivery you must first install the bitdrift Slack app and add it to the channels you want to target. When creating or editing a notification group, select one or more Slack channels to append them to the group. You can combine these with PagerDuty targets in the same group.

Example Slack Notification Groups

Adding PagerDuty Targets

Add PagerDuty delivery by supplying a Routing Key from a PagerDuty service. A group may contain multiple Pager Duty targets alongside Slack channels. Use separate targets when you need different services.

Steps (PagerDuty target addition):

  • In the Create A New Group form: select Pager Duty → paste Routing Key → click the add button.

Slack Notification Message & Mentions

Enter the exact Slack notification that will be posted when the alert fires. • Message text: The custom text of your alert (e.g. “Error rate above 90% for service X”). • @mentions: Include any user or team pings using Slack’s mention syntax (e.g. <@U12345678> for a specific user, <!subteam^S12345|@dev-team> for a user group). For more info, check out the Slack mention syntax docs.

Example Slack Notification Message

PagerDuty Incident Message & Formatting

When an alert fires via a notification group containing Pager Duty targets bitdrift sends the alert to each configured service and simultaneously delivers to any Slack channels in the same group.

Key fields:

  • Routing Key: Integration key added to the notification group.
  • Summary: "[Bitdrift] <alert name> breached: <short condition>".
  • Links: Deep link back to the alert in bitdrift.