Revert "Alerting docs: rework create alert rules definition and topic" (#83328)
This commit is contained in:
@@ -5,46 +5,54 @@ aliases:
|
||||
- unified-alerting/alerting-rules/
|
||||
- ./create-alerts/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/
|
||||
description: Create and manage alert rules
|
||||
description: Configure the features and integrations you need to create and manage your alerts
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Create and manage alert rules
|
||||
title: Create and manage alert rules
|
||||
menuTitle: Configure
|
||||
title: Configure Alerting
|
||||
weight: 120
|
||||
---
|
||||
|
||||
# Create and manage alert rules
|
||||
# Configure Alerting
|
||||
|
||||
An alert rule consists of one or more queries and expressions that select the data you want to measure. It also contains a condition, which is the threshold that an alert rule must meet or exceed in order to fire.
|
||||
Configure the features and integrations that you need to create and manage your alerts.
|
||||
|
||||
Create, manage, view, and adjust alert rules to alert on your metrics data or log entries from multiple data sources — no matter where your data is stored.
|
||||
**Configure alert rules**
|
||||
|
||||
The main parts of alert rule creation are:
|
||||
[Configure Grafana-managed alert rules][create-grafana-managed-rule].
|
||||
|
||||
1. Select your data source
|
||||
1. Query your data
|
||||
1. Normalize your data
|
||||
1. Set your threshold
|
||||
[Configure data source-managed alert rules][create-mimir-loki-managed-rule]
|
||||
|
||||
**Query, expressions, and alert condition**
|
||||
**Configure recording rules**
|
||||
|
||||
What are you monitoring? How are you measuring it?
|
||||
_Recording rules are only available for compatible Prometheus or Loki data sources._
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
Expressions can only be used for Grafana-managed alert rules.
|
||||
{{< /admonition >}}
|
||||
For more information, see [Configure recording rules][create-mimir-loki-managed-recording-rule].
|
||||
|
||||
**Evaluation**
|
||||
**Configure contact points**
|
||||
|
||||
How do you want your alert to be evaluated?
|
||||
For information on how to configure contact points, see [Configure contact points][manage-contact-points].
|
||||
|
||||
**Labels and notifications**
|
||||
**Configure notification policies**
|
||||
|
||||
How do you want to route your alert? What kind of additional labels could you add to annotate your alert rules and ease searching?
|
||||
For information on how to configure notification policies, see [Configure notification policies][create-notification-policy].
|
||||
|
||||
**Annotations**
|
||||
{{% docs/reference %}}
|
||||
[create-mimir-loki-managed-rule]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-mimir-loki-managed-rule"
|
||||
[create-mimir-loki-managed-rule]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-mimir-loki-managed-rule"
|
||||
|
||||
Do you want to add more context on the alert in your notification messages, for example, what caused the alert to fire? Which server did it happen on?
|
||||
[create-mimir-loki-managed-recording-rule]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-mimir-loki-managed-recording-rule"
|
||||
[create-mimir-loki-managed-recording-rule]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-mimir-loki-managed-recording-rule"
|
||||
|
||||
[create-grafana-managed-rule]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-grafana-managed-rule"
|
||||
[create-grafana-managed-rule]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-grafana-managed-rule"
|
||||
|
||||
[manage-contact-points]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points"
|
||||
[manage-contact-points]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/manage-contact-points"
|
||||
|
||||
[create-notification-policy]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-notification-policy"
|
||||
[create-notification-policy]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-notification-policy"
|
||||
{{% /docs/reference %}}
|
||||
|
||||
@@ -3,7 +3,7 @@ aliases:
|
||||
- ../unified-alerting/alerting-rules/create-cortex-loki-managed-recording-rule/
|
||||
- ../unified-alerting/alerting-rules/create-mimir-loki-managed-recording-rule/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-mimir-loki-managed-recording-rule/
|
||||
description: Create recording rules for an external Grafana Mimir or Loki instance
|
||||
description: Configure recording rules for an external Grafana Mimir or Loki instance
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
@@ -16,14 +16,13 @@ labels:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Create recording rules
|
||||
title: Configure recording rules
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# Create recording rules
|
||||
# Configure recording rules
|
||||
|
||||
You can create and manage recording rules for an external Grafana Mimir or Loki instance.
|
||||
Recording rules calculate frequently needed expressions or computationally expensive expressions in advance and save the result as a new set of time series. Querying this new time series is faster, especially for dashboards since they query the same expression every time the dashboards refresh.
|
||||
You can create and manage recording rules for an external Grafana Mimir or Loki instance. Recording rules calculate frequently needed expressions or computationally expensive expressions in advance and save the result as a new set of time series. Querying this new time series is faster, especially for dashboards since they query the same expression every time the dashboards refresh.
|
||||
|
||||
**Note:**
|
||||
|
||||
@@ -49,14 +48,13 @@ To create recording rules, follow these steps.
|
||||
|
||||
1. Click **Alerts & IRM** -> **Alerting** ->
|
||||
**Alert rules**.
|
||||
1. Select **Rule type** -> **Recording**.
|
||||
1. Click **+New recording rule**.
|
||||
1. Click **New recording rule**.
|
||||
|
||||
1. Enter recording rule name.
|
||||
1. Set rule name.
|
||||
|
||||
The recording rule name must be a Prometheus metric name and contain no whitespace.
|
||||
|
||||
1. Define recording rule.
|
||||
1. Define query.
|
||||
- Select your Loki or Prometheus data source.
|
||||
- Enter a query.
|
||||
1. Add namespace and group.
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
aliases:
|
||||
- ../notifications/
|
||||
- ../old-alerting/notifications/
|
||||
- ../unified-alerting/notifications/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-notification-policy/
|
||||
description: Configure notification policies to determine how alerts are routed to contact points
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- guide
|
||||
- notification policies
|
||||
- routes
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Configure notification policies
|
||||
weight: 420
|
||||
---
|
||||
|
||||
# Configure notification policies
|
||||
|
||||
Notification policies determine how alerts are routed to contact points.
|
||||
|
||||
Policies have a tree structure, where each policy can have one or more nested policies. Each policy, except for the default policy, can also match specific alert labels.
|
||||
|
||||
Each alert is evaluated by the default policy and subsequently by each nested policy.
|
||||
|
||||
If the **Continue matching subsequent sibling nodes** option is enabled for a nested policy, then evaluation continues even after one or more matches. A parent policy’s configuration settings and contact point information govern the behavior of an alert that does not match any of the nested policies. A default policy governs any alert that does not match a nested policy.
|
||||
|
||||
You can configure Grafana-managed notification policies as well as notification policies for an external Alertmanager data source.
|
||||
|
||||
For more information on notification policies, see [fundamentals of Notification Policies][notification-policies].
|
||||
|
||||
## Edit default notification policy
|
||||
|
||||
{{% admonition type="note" %}}
|
||||
Before Grafana v8.2, the configuration of the embedded Alertmanager was shared across organizations. Users of Grafana 8.0 and 8.1 are advised to use the new Grafana 8 Alerts only if they have one organization. Otherwise, silences for the Grafana managed alerts will be visible by all organizations.
|
||||
{{% /admonition %}}
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Notification policies**.
|
||||
1. From the **Choose Alertmanager** dropdown, select an external Alertmanager. By default, the **Grafana Alertmanager** is selected.
|
||||
1. In the Default policy section, click **...** -> **Edit**.
|
||||
1. In **Default contact point**, update the contact point to whom notifications should be sent for rules when alert rules do not match any specific policy.
|
||||
1. In **Group by**, choose labels to group alerts by. If multiple alerts are matched for this policy, then they are grouped by these labels. A notification is sent per group. If the field is empty (default), then all notifications are sent in a single group. Use a special label `...` to group alerts by all labels (which effectively disables grouping).
|
||||
1. In **Timing options**, select from the following options:
|
||||
- **Group wait** Time to wait to buffer alerts of the same group before sending an initial notification. Default is 30 seconds.
|
||||
- **Group interval** Minimum time interval between two notifications for a group. Default is 5 minutes.
|
||||
- **Repeat interval** Minimum time interval for re-sending a notification if no new alerts were added to the group. Default is 4 hours.
|
||||
1. Click **Save** to save your changes.
|
||||
|
||||
## Add new nested policy
|
||||
|
||||
To create a new notification policy, you need to follow its tree structure. New policies created on the trunk of the tree (default policy), are the tree branches. And, subsequently, each branch can bear their own child policies. This is why you will always be adding a new **nested** policy under either the default policy, or under a already nested policy.
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Notification policies**.
|
||||
1. From the **Choose Alertmanager** dropdown, select an Alertmanager. By default, the **Grafana Alertmanager** is selected.
|
||||
1. To add a top level specific policy, go to the Specific routing section (either to the default policy, or to another existing policy in which you would like to add a new nested policy) and click **+New nested policy**.
|
||||
1. In the Matching labels section, add one or more rules for matching alert labels.
|
||||
1. In the **Contact point** dropdown, select the contact point to send notification to if alert matches only this specific policy and not any of the nested policies.
|
||||
1. Optionally, enable **Continue matching subsequent sibling nodes** to continue matching sibling policies even after the alert matched the current policy. When this option is enabled, you can get more than one notification for one alert.
|
||||
1. Optionally, enable **Override grouping** to specify the same grouping as the default policy. If this option is not enabled, the default policy grouping is used.
|
||||
1. Optionally, enable **Override general timings** to override the timing options configured in the group notification policy.
|
||||
1. Click **Save policy** to save your changes.
|
||||
|
||||
## Add nested policy
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Notification policies**.
|
||||
1. Expand the specific policy you want to update.
|
||||
1. Click **+ Add nested policy**, then add the details using information in [Add new specific policy](#add-new-nested-policy).
|
||||
1. Click **Save policy** to save your changes.
|
||||
|
||||
## Edit notification policies
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM**, and then **Alerting**.
|
||||
1. Click **Notification policies**.
|
||||
1. Find the policy you want to edit, then click **...** -> **Edit**.
|
||||
1. Make any changes using instructions in [Add new specific policy](#add-new-nested-policy).
|
||||
1. Save your changes.
|
||||
|
||||
## Searching for policies
|
||||
|
||||
Grafana allows you to search within the tree of policies by the following:
|
||||
|
||||
- **Label matchers**
|
||||
- **Contact Points**
|
||||
|
||||
To search by contact point simply select a contact point from the **Search by contact point** dropdown. The policies that use that contact point will be highlighted in the user interface.
|
||||
|
||||
To search by label matchers simply enter a valid matcher in the **Search by matchers** input field. Multiple matchers can be combined with a comma (`,`).
|
||||
|
||||
An example of a valid matchers search input is:
|
||||
|
||||
`severity=high, region=~EMEA|NASA`
|
||||
|
||||
> All matched policies will be **exact** matches, we currently do not support regex-style or partial matching.
|
||||
|
||||
## Example
|
||||
|
||||
An example of an alert configuration.
|
||||
|
||||
- Create a "default" contact point for slack notifications, and set it on the default policy.
|
||||
- Edit the default policy grouping to group alerts by `cluster`, `namespace` and `severity` so that you get a notification per alert rule and specific kubernetes cluster and namespace.
|
||||
- Create specific route for alerts coming from the development cluster with an appropriate contact point.
|
||||
- Create a specific route for alerts with "critical" severity with a more invasive contact point integration, like pager duty notification.
|
||||
- Create specific routes for particular teams that handle their own on-call rotations.
|
||||
|
||||
{{% docs/reference %}}
|
||||
[notification-policies]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/notification-policies/notifications"
|
||||
[notification-policies]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notification-policies/notifications"
|
||||
{{% /docs/reference %}}
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
aliases:
|
||||
- ../contact-points/ # /docs/grafana/<GRAFANA_VERSION>/alerting/contact-points/
|
||||
- ../contact-points/create-contact-point/ # /docs/grafana/<GRAFANA_VERSION>/alerting/contact-points/create-contact-point/
|
||||
- ../contact-points/delete-contact-point/ # /docs/grafana/<GRAFANA_VERSION>/alerting/contact-points/delete-contact-point/
|
||||
- ../contact-points/edit-contact-point/ # /docs/grafana/<GRAFANA_VERSION>/alerting/contact-points/edit-contact-point/
|
||||
- ../contact-points/test-contact-point/ # /docs/grafana/<GRAFANA_VERSION>/alerting/contact-points/test-contact-point/
|
||||
- ../manage-notifications/manage-contact-points/ # /docs/grafana/<GRAFANA_VERSION>/alerting/manage-notifications/manage-contact-points/
|
||||
- create-contact-point/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-contact-point/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/
|
||||
description: Configure contact points to define how your contacts are notified when an alert rule fires
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- guide
|
||||
- contact point
|
||||
- templating
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Configure contact points
|
||||
weight: 410
|
||||
---
|
||||
|
||||
# Configure contact points
|
||||
|
||||
Use contact points to define how your contacts are notified when an alert rule fires. You can add, edit, delete, and test a contact point.
|
||||
|
||||
## Add a contact point
|
||||
|
||||
Complete the following steps to add a contact point.
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Contact points**.
|
||||
1. From the **Choose Alertmanager** dropdown, select an Alertmanager. By default, **Grafana Alertmanager** is selected.
|
||||
1. On the **Contact Points** tab, click **+ Add contact point**.
|
||||
1. Enter a descriptive name for the contact point.
|
||||
1. From **Integration**, select a type and fill out mandatory fields. For example, if you choose email, enter the email addresses. Or if you choose Slack, enter the Slack channel(s) and users who should be contacted.
|
||||
1. Some contact point integrations, like email or webhook, have optional settings. In **Optional settings**, specify additional settings for the selected contact point integration.
|
||||
1. In Notification settings, optionally select **Disable resolved message** if you do not want to be notified when an alert resolves.
|
||||
1. To add another contact point integration, click **Add contact point integration** and repeat steps 6 through 8.
|
||||
1. Save your changes.
|
||||
|
||||
## Edit a contact point
|
||||
|
||||
Complete the following steps to edit a contact point.
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Contact points** to view a list of existing contact points.
|
||||
1. On the **Contact Points** tab, find the contact point you want to edit, and then click **Edit**.
|
||||
1. Update the contact point and save your changes.
|
||||
|
||||
## Delete a contact point
|
||||
|
||||
Complete the following steps to delete a contact point.
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Contact points** to view a list of existing contact points.
|
||||
1. On the **Contact Points** tab, find the contact point you want to delete, and then click **More** -> **Delete**.
|
||||
1. In the confirmation dialog, click **Yes, delete**.
|
||||
|
||||
{{% admonition type="note" %}}
|
||||
You cannot delete contact points that are in use by a notification policy. Either delete the notification policy or update it to use another contact point.
|
||||
{{% /admonition %}}
|
||||
|
||||
## Test a contact point
|
||||
|
||||
Complete the following steps to test a contact point.
|
||||
|
||||
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
|
||||
1. Click **Contact points** to view a list of existing contact points.
|
||||
1. On the **Contact Points** tab, find the contact point you want to test, then click **Edit**. You can also create a new contact point if needed.
|
||||
1. Click **Test** to open the contact point testing modal.
|
||||
1. Choose whether to send a predefined test notification or choose custom to add your own custom annotations and labels to include in the notification.
|
||||
1. Click **Send test notification** to fire the alert.
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
aliases:
|
||||
- alerting/manage-notifications/manage-contact-points/configure-integrations/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/integrations/
|
||||
description: Configure contact point integrations to select your preferred communication channels for receiving notifications of firing alerts.
|
||||
keywords:
|
||||
- Grafana
|
||||
- alerting
|
||||
- guide
|
||||
- notifications
|
||||
- integrations
|
||||
- contact points
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Configure contact point integrations
|
||||
weight: 100
|
||||
---
|
||||
|
||||
# Configure contact point integrations
|
||||
|
||||
Configure contact point integrations in Grafana to select your preferred communication channel for receiving notifications when your alert rules are firing. Each integration has its own configuration options and setup process. In most cases, this involves providing an API key or a Webhook URL.
|
||||
|
||||
Once configured, you can use integrations as part of your contact points to receive notifications whenever your alert changes its state. In this section, we'll cover the basic steps to configure your integrations, so you can start receiving real-time alerts and stay on top of your monitoring data.
|
||||
|
||||
## List of supported integrations
|
||||
|
||||
| Name | Type |
|
||||
| ----------------------- | ------------------------- |
|
||||
| DingDing | `dingding` |
|
||||
| Discord | `discord` |
|
||||
| Email | `email` |
|
||||
| Google Chat | `googlechat` |
|
||||
| Hipchat | `hipchat` |
|
||||
| Kafka | `kafka` |
|
||||
| Line | `line` |
|
||||
| Microsoft Teams | `teams` |
|
||||
| Opsgenie | `opsgenie` |
|
||||
| Pagerduty | `pagerduty` |
|
||||
| Prometheus Alertmanager | `prometheus-alertmanager` |
|
||||
| Pushover | `pushover` |
|
||||
| Sensu | `sensu` |
|
||||
| Sensu Go | `sensugo` |
|
||||
| Slack | `slack` |
|
||||
| Telegram | `telegram` |
|
||||
| Threema | `threema` |
|
||||
| VictorOps | `victorops` |
|
||||
| Webhook | `webhook` |
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
---
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/integrations/configure-oncall/
|
||||
description: Configure the Alerting - Grafana OnCall integration to connect alerts generated by Grafana Alerting with Grafana OnCall
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- oncall
|
||||
- integration
|
||||
aliases:
|
||||
- ../configure-oncall/ # /docs/grafana/latest/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Grafana OnCall
|
||||
title: Configure Grafana OnCall for Alerting
|
||||
weight: 300
|
||||
---
|
||||
|
||||
## Configure Grafana OnCall for Alerting
|
||||
|
||||
Use the Grafana Alerting - Grafana OnCall integration to effortlessly connect alerts generated by Grafana Alerting with Grafana OnCall, where you can then route them according to defined escalation chains and schedules.
|
||||
|
||||
You can set up the integration using the Grafana Alerting application or the Grafana OnCall application. For more information on setting it up from the Grafana OnCall application, see [Grafana OnCall documentation][oncall-integration].
|
||||
|
||||
### Before you begin
|
||||
|
||||
- Ensure you have Installed and enabled the Grafana OnCall plugin
|
||||
- Ensure your version of Grafana is up-to-date and supports the new features
|
||||
|
||||
### Procedure
|
||||
|
||||
To set up the Grafana OnCall integration using the Grafana Alerting application, complete the following steps.
|
||||
|
||||
1. Navigate to **Alerts&IRM** -> **Alerting** -> **Contact points**.
|
||||
1. Click **+ Add contact point**.
|
||||
1. Enter a contact point name.
|
||||
1. From the Integration list, select Grafana OnCall.
|
||||
|
||||
**Note:**
|
||||
|
||||
The Grafana OnCall integration is only available for Grafana Alertmanager.
|
||||
|
||||
1. Choose whether to add a new OnCall integration or add an existing one.
|
||||
|
||||
- If you add a new one, enter an Integration name.
|
||||
- If you add an existing one, choose from the list of available integrations
|
||||
|
||||
1. Click **Save contact point**.
|
||||
|
||||
1. On the Contact points list view page, you can see the contact point with the Grafana OnCall icon.
|
||||
|
||||
If the integration is not yet being used anywhere in the notification policies tree, it will have **Unused** as its status in the **Health** column. It won’t receive any notifications, because there are no notifications using that integration.
|
||||
|
||||
1. Connect your contact point to a notification policy.
|
||||
|
||||
For more information on connecting your contact point to a notification policy, see
|
||||
[Create notification policy][create-notification-policy].
|
||||
|
||||
1. To view your integration in the Grafana OnCall application and set up routes and escalation chains, click the Link next to the integration on the Contact points list view page in the **Type** column.
|
||||
|
||||
This redirects you to the Grafana OnCall integration page in the Grafana OnCall application. From there, you can add [routes and escalation chains][escalation-chain].
|
||||
|
||||
{{% docs/reference %}}
|
||||
[create-notification-policy]: "/docs/grafana/ -> /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-notification-policy"
|
||||
[create-notification-policy]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-notification-policy"
|
||||
|
||||
[oncall-integration]: "/docs/grafana/ -> /docs/oncall/latest/integrations/grafana-alerting"
|
||||
[oncall-integration]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/oncall/integrations/grafana-alerting"
|
||||
|
||||
[escalation-chain]: "/docs/grafana/ -> /docs/oncall/latest/configure/escalation-chains-and-routes"
|
||||
[escalation-chain]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/oncall/configure/escalation-chains-and-routes"
|
||||
{{% /docs/reference %}}
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/integrations/pager-duty/
|
||||
description: Configure the PagerDuty integration for Alerting
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- pagerduty
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: PagerDuty
|
||||
title: Configure PagerDuty for Alerting
|
||||
weight: 400
|
||||
---
|
||||
|
||||
# Configure PagerDuty for Alerting
|
||||
|
||||
To set up PagerDuty, provide an integration key.
|
||||
|
||||
| Setting | Description |
|
||||
| --------------- | ------------------------------------------------------ |
|
||||
| Integration Key | Integration key for PagerDuty |
|
||||
| Severity | Level for dynamic notifications, default is `critical` |
|
||||
| Custom Details | Additional details about the event |
|
||||
|
||||
The `CustomDetails` field is an object containing arbitrary key-value pairs. The user-defined details are merged with the ones we use by default.
|
||||
|
||||
Our default values for `CustomDetails` are:
|
||||
|
||||
```go
|
||||
{
|
||||
"firing": `{{ template "__text_alert_list" .Alerts.Firing }}`,
|
||||
"resolved": `{{ template "__text_alert_list" .Alerts.Resolved }}`,
|
||||
"num_firing": `{{ .Alerts.Firing | len }}`,
|
||||
"num_resolved": `{{ .Alerts.Resolved | len }}`,
|
||||
}
|
||||
```
|
||||
|
||||
In case of duplicate keys, the user-defined details overwrite the default ones.
|
||||
+150
@@ -0,0 +1,150 @@
|
||||
---
|
||||
aliases:
|
||||
- ../contact-points/notifiers/webhook-notifier/
|
||||
- ../fundamentals/contact-points/webhook-notifier/
|
||||
- alerting/manage-notifications/manage-contact-points/webhook-notifier/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/alerting-rules/manage-contact-points/integrations/webhook-notifier/
|
||||
description: Configure the webhook notifier integration for Alerting
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- guide
|
||||
- contact point
|
||||
- templating
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Webhook notifier
|
||||
title: Configure the webhook notifier for Alerting
|
||||
weight: 200
|
||||
---
|
||||
|
||||
### Configure the webhook notifier for Alerting
|
||||
|
||||
Example JSON body:
|
||||
|
||||
```json
|
||||
{
|
||||
"receiver": "My Super Webhook",
|
||||
"status": "firing",
|
||||
"orgId": 1,
|
||||
"alerts": [
|
||||
{
|
||||
"status": "firing",
|
||||
"labels": {
|
||||
"alertname": "High memory usage",
|
||||
"team": "blue",
|
||||
"zone": "us-1"
|
||||
},
|
||||
"annotations": {
|
||||
"description": "The system has high memory usage",
|
||||
"runbook_url": "https://myrunbook.com/runbook/1234",
|
||||
"summary": "This alert was triggered for zone us-1"
|
||||
},
|
||||
"startsAt": "2021-10-12T09:51:03.157076+02:00",
|
||||
"endsAt": "0001-01-01T00:00:00Z",
|
||||
"generatorURL": "https://play.grafana.org/alerting/1afz29v7z/edit",
|
||||
"fingerprint": "c6eadffa33fcdf37",
|
||||
"silenceURL": "https://play.grafana.org/alerting/silence/new?alertmanager=grafana&matchers=alertname%3DT2%2Cteam%3Dblue%2Czone%3Dus-1",
|
||||
"dashboardURL": "",
|
||||
"panelURL": "",
|
||||
"values": {
|
||||
"B": 44.23943737541908,
|
||||
"C": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"status": "firing",
|
||||
"labels": {
|
||||
"alertname": "High CPU usage",
|
||||
"team": "blue",
|
||||
"zone": "eu-1"
|
||||
},
|
||||
"annotations": {
|
||||
"description": "The system has high CPU usage",
|
||||
"runbook_url": "https://myrunbook.com/runbook/1234",
|
||||
"summary": "This alert was triggered for zone eu-1"
|
||||
},
|
||||
"startsAt": "2021-10-12T09:56:03.157076+02:00",
|
||||
"endsAt": "0001-01-01T00:00:00Z",
|
||||
"generatorURL": "https://play.grafana.org/alerting/d1rdpdv7k/edit",
|
||||
"fingerprint": "bc97ff14869b13e3",
|
||||
"silenceURL": "https://play.grafana.org/alerting/silence/new?alertmanager=grafana&matchers=alertname%3DT1%2Cteam%3Dblue%2Czone%3Deu-1",
|
||||
"dashboardURL": "",
|
||||
"panelURL": "",
|
||||
"values": {
|
||||
"B": 44.23943737541908,
|
||||
"C": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"groupLabels": {},
|
||||
"commonLabels": {
|
||||
"team": "blue"
|
||||
},
|
||||
"commonAnnotations": {},
|
||||
"externalURL": "https://play.grafana.org/",
|
||||
"version": "1",
|
||||
"groupKey": "{}:{}",
|
||||
"truncatedAlerts": 0,
|
||||
"title": "[FIRING:2] (blue)",
|
||||
"state": "alerting",
|
||||
"message": "**Firing**\n\nLabels:\n - alertname = T2\n - team = blue\n - zone = us-1\nAnnotations:\n - description = This is the alert rule checking the second system\n - runbook_url = https://myrunbook.com\n - summary = This is my summary\nSource: https://play.grafana.org/alerting/1afz29v7z/edit\nSilence: https://play.grafana.org/alerting/silence/new?alertmanager=grafana&matchers=alertname%3DT2%2Cteam%3Dblue%2Czone%3Dus-1\n\nLabels:\n - alertname = T1\n - team = blue\n - zone = eu-1\nAnnotations:\nSource: https://play.grafana.org/alerting/d1rdpdv7k/edit\nSilence: https://play.grafana.org/alerting/silence/new?alertmanager=grafana&matchers=alertname%3DT1%2Cteam%3Dblue%2Czone%3Deu-1\n"
|
||||
}
|
||||
```
|
||||
|
||||
### Webhook fields
|
||||
|
||||
## Body
|
||||
|
||||
| Key | Type | Description |
|
||||
| ----------------- | ------------------------- | ------------------------------------------------------------------------------- |
|
||||
| receiver | string | Name of the webhook |
|
||||
| status | string | Current status of the alert, `firing` or `resolved` |
|
||||
| orgId | number | ID of the organization related to the payload |
|
||||
| alerts | array of [alerts](#alert) | Alerts that are triggering |
|
||||
| groupLabels | object | Labels that are used for grouping, map of string keys to string values |
|
||||
| commonLabels | object | Labels that all alarms have in common, map of string keys to string values |
|
||||
| commonAnnotations | object | Annotations that all alarms have in common, map of string keys to string values |
|
||||
| externalURL | string | External URL to the Grafana instance sending this webhook |
|
||||
| version | string | Version of the payload |
|
||||
| groupKey | string | Key that is used for grouping |
|
||||
| truncatedAlerts | number | Number of alerts that were truncated |
|
||||
| title | string | **Will be deprecated soon** |
|
||||
| state | string | **Will be deprecated soon** |
|
||||
| message | string | **Will be deprecated soon** |
|
||||
|
||||
### Alert
|
||||
|
||||
| Key | Type | Description |
|
||||
| ------------ | ------ | ---------------------------------------------------------------------------------- |
|
||||
| status | string | Current status of the alert, `firing` or `resolved` |
|
||||
| labels | object | Labels that are part of this alert, map of string keys to string values |
|
||||
| annotations | object | Annotations that are part of this alert, map of string keys to string values |
|
||||
| startsAt | string | Start time of the alert |
|
||||
| endsAt | string | End time of the alert, default value when not resolved is `0001-01-01T00:00:00Z` |
|
||||
| values | object | Values that triggered the current status |
|
||||
| generatorURL | string | URL of the alert rule in the Grafana UI |
|
||||
| fingerprint | string | The labels fingerprint, alarms with the same labels will have the same fingerprint |
|
||||
| silenceURL | string | URL to silence the alert rule in the Grafana UI |
|
||||
| dashboardURL | string | **Will be deprecated soon** |
|
||||
| panelURL | string | **Will be deprecated soon** |
|
||||
| imageURL | string | URL of a screenshot of a panel assigned to the rule that created this notification |
|
||||
|
||||
### Removed fields related to dashboards
|
||||
|
||||
Alerts are not coupled to dashboards anymore therefore the fields related to dashboards `dashboardId` and `panelId` have been removed.
|
||||
|
||||
## WeCom
|
||||
|
||||
WeCom contact points need a Webhook URL. These are obtained by setting up a WeCom robot on the corresponding group chat. To obtain a Webhook URL using the WeCom desktop Client please follow these steps:
|
||||
|
||||
1. Click the "..." in the top right corner of a group chat that you want your alerts to be delivered to
|
||||
2. Click "Add Group Robot", select "New Robot" and give your robot a name. Click "Add Robot"
|
||||
3. There should be a Webhook URL in the panel.
|
||||
|
||||
| Setting | Description |
|
||||
| ------- | ------------------ |
|
||||
| Url | WeCom webhook URL. |
|
||||
Reference in New Issue
Block a user