Backport docs changes for Grafana Alerting (#50610)

This commit is contained in:
Armand Grillet
2022-06-10 16:02:45 +02:00
committed by GitHub
parent a4958a437f
commit d8f5a3b799
40 changed files with 258 additions and 120 deletions
+6 -6
View File
@@ -6,27 +6,27 @@ title: Alerting
weight: 114
---
# Grafana alerting
# Grafana Alerting
Grafana alerts allow you to learn about problems in your systems moments after they occur. Robust and actionable alerts help you identify and resolve issues quickly, minimizing disruption to your services. It centralizes alerting information in a single, searchable view that allows you to:
Grafana Alerting allows you to learn about problems in your systems moments after they occur. Robust and actionable alerts help you identify and resolve issues quickly, minimizing disruption to your services. It centralizes alerting information in a single, searchable view that allows you to:
- Create and manage Grafana alerts
- Create and manage Grafana Mimir and Loki managed alerts
- View alerting information from Prometheus and Alertmanager compatible data sources
For new installations or existing installs without alerting configured, Grafana alerting is enabled by default.
For new installations or existing installs without alerting configured, Grafana Alerting is enabled by default.
| Release | Cloud | Enterprise | OSS |
| ----------- | ------------- | ------------- | ------------- |
| Grafana 9.0 | On by default | On by default | On by default |
Existing installations that upgrade to v9.0 will have Grafana alerting enabled by default. For more information on migrating from legacy or the cloud alerting plugin, see [Migrating to Grafana alerting]({{< relref "migrating-alerts/" >}}).
Existing installations that upgrade to v9.0 will have Grafana Alerting enabled by default. For more information on migrating from legacy or the cloud alerting plugin, see [Migrating to Grafana Alerting]({{< relref "migrating-alerts/" >}}).
Before you begin, we recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "fundamentals/" >}}) of Grafana alerting. Refer to [Role-based access control]({{< relref "../enterprise/access-control/" >}}) in Grafana Enterprise to learn more about controlling access to alerts using role-based permissions.
Before you begin, we recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "fundamentals/" >}}) of Grafana Alerting. Refer to [Role-based access control]({{< relref "../enterprise/access-control/" >}}) in Grafana Enterprise to learn more about controlling access to alerts using role-based permissions.
- [About alert rules]({{< relref "fundamentals/alert-rules/" >}})
- [Migrating legacy alerts]({{< relref "migrating-alerts/" >}})
- [Disable Grafana alerting in OSS]({{< relref "migrating-alerts/opt-out/" >}})
- [Disable Grafana Alerting in OSS]({{< relref "migrating-alerts/opt-out/" >}})
- [Create Grafana managed alerting rules]({{< relref "alerting-rules/create-grafana-managed-rule/" >}})
- [Create Grafana Mimir or Loki managed alerting rules]({{< relref "alerting-rules/create-mimir-loki-managed-rule/" >}})
- [View existing alerting rules and manage their current state]({{< relref "alerting-rules/rule-list/" >}})
+5 -5
View File
@@ -1,18 +1,18 @@
---
aliases:
- /docs/grafana/latest/alerting/about-alerting/
description: A quick overview of Grafana alerting
description: A quick overview of Grafana Alerting
keywords:
- grafana
- alerting
- overview
- concepts
- basics
title: About Grafana alerting
title: About Grafana Alerting
weight: 100
---
# About Grafana alerting
# About Grafana Alerting
Grafana Alerting consists of several individual concepts that are at the core of a flexible and powerful alerting engine.
@@ -24,9 +24,9 @@ We will also touch on various other concepts such as [silences]({{< relref "sile
## Overview
{{< figure src="/static/img/docs/alerting/unified/about-alerting-flow-diagram.jpg" caption="Grafana alerting overview" >}}
{{< figure src="/static/img/docs/alerting/unified/about-alerting-flow-diagram.jpg" caption="Grafana Alerting overview" >}}
As shown in the diagram above, Grafana alerting uses [labels]({{< relref "fundamentals/annotation-label/how-to-use-labels/" >}}) to match an alert rule and its instances to a specific notification policy. This concept of labels and label matching is important and is also used in [silences]({{< relref "silences/" >}}).
As shown in the diagram above, Grafana Alerting uses [labels]({{< relref "fundamentals/annotation-label/how-to-use-labels/" >}}) to match an alert rule and its instances to a specific notification policy. This concept of labels and label matching is important and is also used in [silences]({{< relref "silences/" >}}).
Each notification policy specifies a set of [label matchers]({{< relref "fundamentals/annotation-label/labels-and-label-matchers/" >}}) to indicate what alerts they are responsible for.
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/alert-groups/
- /docs/grafana/latest/alerting/unified-alerting/alert-groups/
description: Alert groups
keywords:
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/alert-groups/filter-alerts/
- /docs/grafana/latest/alerting/unified-alerting/alert-groups/
description: Alert groups
keywords:
@@ -1,6 +1,7 @@
---
aliases:
- /docs/grafana/latest/alerting/alert-groups/
- /docs/grafana/latest/alerting/alert-groups/view-alert-grouping/
- /docs/grafana/latest/alerting/unified-alerting/alert-groups/
description: Alert groups
keywords:
+19 -2
View File
@@ -7,5 +7,22 @@ weight: 552
# Limitations
- Grafana alerting system can retrieve rules from all available Prometheus, Loki, and Alertmanager data sources. It might not be able to fetch alerting rules from all other supported data sources at this time.
- We aim to support the latest two minor versions of both Prometheus and Alertmanager. We cannot guarantee that older versions will work. As an example, if the current Prometheus version is `2.31.1`, we support >= `2.29.0`.
## Limited rule sources support
Grafana Alerting can retrieve alerting and recording rules **stored** in most available Prometheus, Loki, Mimir, and Alertmanager compatible data sources.
It does not support reading or writing alerting rules from any other data sources but the ones previously mentioned at this time.
## Prometheus version support
We support the latest two minor versions of both Prometheus and Alertmanager. We cannot guarantee that older versions will work.
As an example, if the current Prometheus version is `2.31.1`, we support >= `2.29.0`.
## Grafana is not an alert receiver
Grafana is not an alert receiver; is it an alert generator. This means that Grafana cannot receive alerts from anything other than its internal alert generator.
Receiving alerts from Prometheus (or anything else) is not supported at the time.
For more information, refer to [this GitHub discussion](https://github.com/grafana/grafana/discussions/45773).
@@ -7,7 +7,7 @@ title: Create and manage rules
weight: 130
---
# Create and manage Grafana alerting rules
# Create and manage Grafana Alerting rules
An alerting rule is a set of evaluation criteria that determines whether an alert will fire. The rule consists of one or more queries and expressions, a condition, the frequency of evaluation, and optionally, the duration over which the condition is met.
@@ -15,7 +15,7 @@ weight: 400
# Create a Grafana managed alerting rule
Grafana allows you to create alerting rules that query one or more data sources, reduce or transform the results and compare them to each other or to fix thresholds. When these are executed, Grafana sends notifications to the contact point. For information on Grafana alerting, see [About Grafana alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana alerting.
Grafana allows you to create alerting rules that query one or more data sources, reduce or transform the results and compare them to each other or to fix thresholds. When these are executed, Grafana sends notifications to the contact point. For information on Grafana Alerting, see [About Grafana Alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana Alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana Alerting.
## Add Grafana managed rule
@@ -16,7 +16,7 @@ weight: 400
# Create a Grafana Mimir or Loki managed alerting rule
Grafana allows you to create alerting rules for an external Grafana Mimir or Loki instance that has ruler API enabled. For information on Grafana alerting, see [About Grafana alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana alerting.
Grafana allows you to create alerting rules for an external Grafana Mimir or Loki instance that has ruler API enabled. For information on Grafana Alerting, see [About Grafana Alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana Alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana Alerting.
## Before you begin
@@ -20,7 +20,7 @@ Use contact points to define how your contacts are notified when an alert fires.
You can configure Grafana managed contact points as well as contact points for an [external Alertmanager data source]({{< relref "../../datasources/alertmanager/" >}}). For more information, see [Alertmanager]({{< relref "../fundamentals/alertmanager/" >}}).
Before you begin, see [About Grafana alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana alerting.
Before you begin, see [About Grafana Alerting]({{< relref "../about-alerting/" >}}) which explains the various components of Grafana Alerting. We also recommend that you familiarize yourself with some of the [fundamental concepts]({{< relref "../fundamentals/" >}}) of Grafana Alerting.
- [Create contact point]({{< relref "create-contact-point/" >}})
- [Edit contact point]({{< relref "edit-contact-point/" >}})
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/contact-points/message-templating/
- /docs/grafana/latest/alerting/message-templating/
- /docs/grafana/latest/alerting/unified-alerting/message-templating/
description: Message templating
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/contact-points/message-templating/template-data/
- /docs/grafana/latest/alerting/message-templating/template-data/
- /docs/grafana/latest/alerting/unified-alerting/message-templating/template-data/
keywords:
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/contact-points/message-templating/template-functions/
- /docs/grafana/latest/alerting/message-templating/template-functions/
- /docs/grafana/latest/alerting/unified-alerting/message-templating/template-functions/
keywords:
+5 -5
View File
@@ -8,13 +8,13 @@ keywords:
- grafana
- alerting
- guide
title: What's new in Grafana alerting
title: What's new in Grafana Alerting
weight: 108
---
# What's new in Grafana alerting
# What's new in Grafana Alerting
Grafana alerting has several enhancements over legacy dashboard alerting.
Grafana Alerting has several enhancements over legacy dashboard alerting.
## Multi-dimensional alerting
@@ -28,7 +28,7 @@ Since unified alerts are no longer directly tied to panel queries, they do not i
## Create Loki and Grafana Mimir alerting rules
In Grafana alerting, you can manage Loki and Grafana Mimir alerting rules using the same UI and API as your Grafana managed alerts.
In Grafana Alerting, you can manage Loki and Grafana Mimir alerting rules using the same UI and API as your Grafana managed alerts.
## View and search for alerts from Prometheus compatible data sources
@@ -36,7 +36,7 @@ Alerts for Prometheus compatible data sources are now listed under the Grafana a
## Special alerts for alert state NoData and Error
Grafana alerting introduced a new concept of the alert states. When evaluation of an alerting rule produces state NoData or Error, Grafana alerting will generate special alerts that will have the following labels:
Grafana Alerting introduced a new concept of the alert states. When evaluation of an alerting rule produces state NoData or Error, Grafana Alerting will generate special alerts that will have the following labels:
- `alertname` with value DatasourceNoData or DatasourceError depending on the state.
- `rulename` name of the alert rule the special alert belongs to.
+1 -1
View File
@@ -9,7 +9,7 @@ weight: 110
# Alerting fundamentals
This section includes the following fundamental concepts of Grafana alerting:
This section includes the following fundamental concepts of Grafana Alerting:
- [Annotations and labels for alerting rules]({{< relref "annotation-label/" >}})
- [Alertmanager]({{< relref "alertmanager/" >}})
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/alert-rules/
description: About Grafana alert rules
keywords:
- grafana
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/alert-rules/alert-instances/
description: Learn about Grafana alert instances
keywords:
- grafana
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/alert-rules/alert-rule-types/
description: Learn about the different alert rule types
keywords:
- grafana
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/alert-rules/organising-alerts/
description: Learn how to organize alert rules
keywords:
- grafana
@@ -15,7 +15,7 @@ Grafana includes built-in support for Prometheus Alertmanager. By default, notif
> **Note:** Before v8.2, the configuration of the embedded Alertmanager was shared across organizations. If you are on an older Grafana version, we recommend that you use Grafana alerts only if you have one organization. Otherwise, your contact points are visible to all organizations.
Grafana alerting added support for external Alertmanager configuration. When you add an [Alertmanager data source]({{< relref "../../datasources/alertmanager/" >}}), the Alertmanager drop-down shows a list of available external Alertmanager data sources. Select a data source to create and manage alerting for standalone Grafana Mimir or Loki data sources.
Grafana Alerting added support for external Alertmanager configuration. When you add an [Alertmanager data source]({{< relref "../../datasources/alertmanager/" >}}), the Alertmanager drop-down shows a list of available external Alertmanager data sources. Select a data source to create and manage alerting for standalone Grafana Mimir or Loki data sources.
{{< figure max-width="40%" src="/static/img/docs/alerting/unified/contact-points-select-am-8-0.gif" max-width="250px" caption="Select Alertmanager" >}}
@@ -1,6 +1,7 @@
---
aliases:
- /docs/grafana/latest/alerting/alerting-rules/alert-annotation-label/
- /docs/grafana/latest/alerting/fundamentals/annotation-label/
- /docs/grafana/latest/alerting/unified-alerting/alerting-rules/alert-annotation-label/
description: Annotations and labels for alerting
keywords:
@@ -27,6 +28,6 @@ Labels are key-value pairs that contain information about, and are used to uniqu
Before you begin using annotations and labels, familiarize yourself with:
- [Labels in Grafana alerting]({{< relref "how-to-use-labels/" >}})
- [Labels in Grafana Alerting]({{< relref "how-to-use-labels/" >}})
- [How label matching works]({{< relref "how-to-use-labels/" >}})
- [Template variables for alerting rule labels and annotations]({{< relref "variables-label-annotation/" >}})
@@ -1,15 +1,17 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/annotation-label/how-to-use-labels/
description: Learn about labels and label matchers in alerting
keywords:
- grafana
- alerting
- guide
- fundamentals
title: Labels in Grafana alerting
title: Labels in Grafana Alerting
weight: 117
---
# Labels in Grafana alerting
# Labels in Grafana Alerting
This topic explains why labels are a fundamental component of alerting.
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/annotation-label/labels-and-label-matchers/
description: Learn about labels and label matchers in alerting
keywords:
- grafana
@@ -1,4 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/fundamentals/annotation-label/variables-label-annotation/
description: Learn about labels and label matchers in alerting
keywords:
- grafana
@@ -13,24 +13,51 @@ title: State and health of alerting rules
# State and health of alerting rules
The state and health of alerting rules help you understand several key status indicators about your alerts. There are three key components: alert state, alerting rule state, and alerting rule health. Although related, each component conveys subtly different information.
The state and health of alerting rules help you understand several key status indicators about your alerts.
## Alerting rule state
There are three key components: [alert rule state](#alert-rule-state), [alert instance state](#alert-instance-state), and [alert rule health](#alert-rule-health). Although related, each component conveys subtly different information.
- **Normal**: None of the time series returned by the evaluation engine is in a Pending or Firing state.
- **Pending**: At least one time series returned by the evaluation engine is Pending.
- **Firing**: At least one time series returned by the evaluation engine is Firing.
## Alert rule state
## Alert state
An alert rule can be in either of the following states:
- **Normal**: Condition for the alerting rule is **false** for every time series returned by the evaluation engine.
- **Alerting**: Condition of the alerting rule is **true** for at least one time series returned by the evaluation engine. The duration for which the condition must be true before an alert fires, if set, is met or has exceeded.
- **Pending**: Condition of the alerting rule is **true** for at least one time series returned by the evaluation engine. The duration for which the condition must be true before an alert fires, if set, **has not** been met.
- **NoData**: the alerting rule has not returned a time series, all values for the time series are null, or all values for the time series are zero.
- **Error**: Error when attempting to evaluate an alerting rule.
| State | Description |
| ----------- | ---------------------------------------------------------------------------------------------- |
| **Normal** | None of the time series returned by the evaluation engine is in a `Pending` or `Firing` state. |
| **Pending** | At least one time series returned by the evaluation engine is `Pending`. |
| **Firing** | At least one time series returned by the evaluation engine is `Firing`. |
## Alerting rule health
> **Note:** Alerts will transition first to `pending` and then `firing`, thus it will take at least two evaluation cycles before an alert is fired.
- **Ok**: No error when evaluating an alerting rule.
- **Error**: Error when evaluating an alerting rule.
- **NoData**: The absence of data in at least one time series returned during a rule evaluation.
## Alert instance state
An alert instance can be in either of the following states:
| State | Description |
| ------------ | --------------------------------------------------------------------------------------------- |
| **Normal** | The state of an alert that is neither firing nor pending, everything is working correctly. |
| **Pending** | The state of an alert that has been active for less than the configured threshold duration. |
| **Alerting** | The state of an alert that has been active for longer than the configured threshold duration. |
| **NoData** | No data has been received for the configured time window. |
| **Error** | The error that occurred when attempting to evaluate an alerting rule. |
## Alert rule health
An alert rule can have one the following health statuses:
| State | Description |
| ---------- | ---------------------------------------------------------------------------------- |
| **Ok** | No error when evaluating an alerting rule. |
| **Error** | An error occurred when evaluating an alerting rule. |
| **NoData** | The absence of data in at least one time series returned during a rule evaluation. |
## Special alerts for `NoData` and `Error`
When evaluation of an alerting rule produces state `NoData` or `Error`, Grafana Alerting will generate alert instances that have the following additional labels:
| Label | Description |
| ------------------ | ---------------------------------------------------------------------- |
| **alertname** | Either `DatasourceNoData` or `DatasourceError` depending on the state. |
| **datasource_uid** | The UID of the data source that caused the state. |
You can handle these alerts the same way as regular alerts by adding a silence, route to a contact point, and so on.
@@ -15,9 +15,9 @@ weight: 450
# About alerting high availability
The Grafana alerting system has two main components: a `Scheduler` and an internal `Alertmanager`. The `Scheduler` evaluates your [alert rules]({{< relref "../fundamentals/evaluate-grafana-alerts/" >}}), while the internal Alertmanager manages **routing** and **grouping**.
The Grafana Alerting system has two main components: a `Scheduler` and an internal `Alertmanager`. The `Scheduler` evaluates your [alert rules]({{< relref "../fundamentals/evaluate-grafana-alerts/" >}}), while the internal Alertmanager manages **routing** and **grouping**.
When running Grafana alerting in high availability, the operational mode of the scheduler remains unaffected, and each Grafana instance evaluates all alerts. The operational change happens in the Alertmanager when it deduplicates alert notifications across Grafana instances.
When running Grafana Alerting in high availability, the operational mode of the scheduler remains unaffected, and each Grafana instance evaluates all alerts. The operational change happens in the Alertmanager when it deduplicates alert notifications across Grafana instances.
{{< figure src="/static/img/docs/alerting/unified/high-availability-ua.png" class="docs-image--no-shadow" max-width= "750px" caption="High availability" >}}
@@ -1,27 +1,33 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/
description: Migrate Grafana alerts
title: Migrate to Grafana alerting
weight: 113
description: Upgrade Grafana alerts
title: Upgrade to Grafana Alerting
weight: 101
---
# Migrate to Grafana alerting
# Upgrade to Grafana Alerting
Grafana alerting is the default for new Cloud, Enterprise, and OSS installations. The new installations will only show the Grafana alerting icon in the left navigation panel.
Grafana Alerting is enabled by default for new installations or existing installations whether or not legacy alerting is configured.
Existing installations that upgrade to v9.0 will have Grafana alerting enabled by default.
> **Note**: We recommend that Grafana Enterprise customers with more than a dozen Grafana dashboard alert rules do not upgrade and remain on legacy alerting for now by [opting out]({{< relref "opt-out/" >}}). If you do want to upgrade to Grafana Alerting, contact customer support.
| Grafana instance upgraded to v 90 | |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Cloud | Existing Cloud installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting plugin icon and the new Grafana alerting icon. During upgrade, existing alerts from the Cloud alerting plugin are migrated to Grafana alerting. Once migration is complete, you can access aman manage the older alerts from the new alerting Grafana alerting icon in the navigation panel. The (older) Cloud alerting plugin is uninstalled from your cloud instance. Contact customer support if you **do not wish** to migrate to Grafana alerting for your Cloud stack. If you choose to use legacy alerting, use the You will see the new Grafana alerting icon as well as the old Cloud alerting plugin in the left navigation panel. |
| Enterprise | Existing Enterprise instances using legacy alerting will have both the old (marked as legacy) and the new alerting icons in the navigation panel. During upgrade, existing legacy alerts are migrated to Grafana alerting. If you wish, you can [opt-out]({{< relref "opt-out/" >}}) of Grafana alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
| OSS | Existing OSS installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting icon (marked as legacy) and the new Grafana alerting icon. During upgrade, existing legacy alerts are migrated to Grafana alerting. If you wish, you can [opt-out]({{< relref "opt-out/" >}}) of Grafana alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
Existing installations that do not use legacy alerting will have Grafana Alerting enabled by default unless alerting is disabled in the configuration.
During migration from legacy alerting to unified alerting, the legacy alerts are updated to the new alerts type, as a result, the user does not lose alerts or alerting data. However, if a user rolls back to legacy alerting after having migrated to unified alerting, they will only get the legacy alerts they had right before migration.
Likewise, existing installations that use legacy alerting will be automatically upgraded to Grafana Alerting unless you have [opted out]({{< relref "opt-out/" >}}) of Grafana Alerting before migration takes place. During the upgrade, legacy alerts are migrated to the new alerts type and no alerts or alerting data are lost.
## Roll back to legacy alerting
Once the upgrade has taken place, you still have the option to [roll back]({{< relref "roll-back/" >}}) to legacy alerting. However, we do not recommend choosing this option. If you do choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place. All new alerts and changes made exclusively in Grafana Alerting will be deleted.
Although we encourage you to use Grafana alerting, roll back to legacy alerting is supported in Grafana 9. Rolling back can result in data loss (you will loose all alerts that you created using Grafana alerting). This is applicable to the fresh installation as well as upgraded setups.
> **Note**: Cloud customers, who do not want to upgrade to Grafana Alerting, should contact customer support.
If you have opted out or rolled back, you can always choose to [opt in]({{< relref "opt-in/" >}}) to Grafana Alerting at a later point in time.
The following table provides details on the upgrade for Cloud, Enterprise, and OSS installations and the new Grafana Alerting UI.
| Grafana instance upgraded to 9.0 | |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Cloud | Existing Cloud installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting plugin icon and the new Grafana Alerting icon. During upgrade, existing alerts from the Cloud alerting plugin are migrated to Grafana Alerting. Once migration is complete, you can access and manage the older alerts from the new alerting Grafana Alerting icon in the navigation panel. The (older) Cloud alerting plugin is uninstalled from your cloud instance. Contact customer support if you **do not wish** to migrate to Grafana Alerting for your Cloud stack. If you choose to use legacy alerting, use the You will see the new Grafana Alerting icon as well as the old Cloud alerting plugin in the left navigation panel. |
| Enterprise | Existing Enterprise instances using legacy alerting will have both the old (marked as legacy) and the new alerting icons in the navigation panel. During upgrade, existing legacy alerts are migrated to Grafana Alerting. If you wish, you can [opt-out]({{< relref "opt-out/" >}}) of Grafana Alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
| OSS | Existing OSS installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting icon (marked as legacy) and the new Grafana Alerting icon. During upgrade, existing legacy alerts are migrated to Grafana Alerting. If you wish, you can [opt-out]({{< relref "opt-out/" >}}) of Grafana Alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
> **Note:** Legacy alerting will be deprecated in a future release (v10).
@@ -0,0 +1,23 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/disable-alerting/
- /docs/grafana/latest/alerting/disable-alerting/
- /docs/grafana/latest/alerting/unified-alerting/disable-alerting/
description: Disable alerting in Grafana
title: Disable alerting in Grafana
weight: 105
---
# Disable alerting in Grafana
To disable alerting in Grafana entirely (including both legacy and Grafana Alerting), enter the following in your configuration:
```
[alerting]
enabled = false
[unified_alerting]
enabled = false
```
If at any time you want to turn alerting back on, you can opt in.
@@ -1,17 +1,19 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/migrating-legacy-alerts/
- /docs/grafana/latest/alerting/migrating-legacy-alerts/
- /docs/grafana/latest/alerting/unified-alerting/opt-in/
description: Migrate legacy dashboard alerts
title: Migrating legacy dashboard alerts
weight: 114
title: Differences and limitations
weight: 106
---
# Migrating legacy dashboard alerts
# Differences and limitations
When Grafana alerting is enabled or Grafana is upgraded to the latest version, existing legacy dashboard alerts migrate in a format compatible with the Grafana alerting. In the Alerting page of your Grafana instance, you can view the migrated alerts alongside any new alerts. This topic explains how legacy dashboard alerts are migrated and some limitations.
When Grafana Alerting is enabled or upgraded to Grafana 9.0 or later, existing legacy dashboard alerts migrate in a format compatible with the Grafana Alerting. In the Alerting page of your Grafana instance, you can view the migrated alerts alongside any new alerts.
This topic explains how legacy dashboard alerts are migrated and some limitations of the migration.
> **Note:** This topic is only relevant for OSS and Enterprise customers. Contact customer support to enable or disable Grafana alerting for your Cloud stack.
> **Note:** This topic is only relevant for OSS and Enterprise customers. Contact customer support to enable or disable Grafana Alerting for your Cloud stack.
Read and write access to legacy dashboard alerts and Grafana alerts are governed by the permissions of the folders storing them. During migration, legacy dashboard alert permissions are matched to the new rules permissions as follows:
@@ -19,9 +21,11 @@ Read and write access to legacy dashboard alerts and Grafana alerts are governed
- If there are no dashboard permissions and the dashboard is under a folder, then the rule is linked to this folder and inherits its permissions.
- If there are no dashboard permissions and the dashboard is under the General folder, then the rule is linked to the `General Alerting` folder, and the rule inherits the default permissions.
> **Note:** Since there is no `Keep Last State` option for [`No Data`]({{< relref "../alerting-rules/create-grafana-managed-rule/#no-data--error-handling" >}}) in Grafana alerting, this option becomes `NoData` during the legacy rules migration. Option "Keep Last State" for [`Error handling`]({{< relref "../alerting-rules/create-grafana-managed-rule/#no-data--error-handling" >}}) is migrated to a new option `Error`. To match the behavior of the `Keep Last State`, in both cases, during the migration Grafana automatically creates a [silence]({{< relref "../silences/" >}}) for each alert rule with a duration of 1 year.
> **Note:** Since there is no `Keep Last State` option for [`No Data`]({{< relref "../alerting-rules/create-grafana-managed-rule/#no-data--error-handling" >}}) in Grafana Alerting, this option becomes `NoData` during the legacy rules migration. Option "Keep Last State" for [`Error handling`]({{< relref "../alerting-rules/create-grafana-managed-rule/#no-data--error-handling" >}}) is migrated to a new option `Error`. To match the behavior of the `Keep Last State`, in both cases, during the migration Grafana automatically creates a [silence]({{< relref "../silences/" >}}) for each alert rule with a duration of 1 year.
Notification channels are migrated to an Alertmanager configuration with the appropriate routes and receivers. Default notification channels are added as contact points to the default route. Notification channels not associated with any Dashboard alert go to the `autogen-unlinked-channel-recv` route.
Since `Hipchat` and `Sensu` notification channels are no longer supported, legacy alerts associated with these channels are not automatically migrated to Grafana alerting. Assign the legacy alerts to a supported notification channel so that you continue to receive notifications for those alerts.
## Limitations
Since `Hipchat` and `Sensu` notification channels are no longer supported, legacy alerts associated with these channels are not automatically migrated to Grafana Alerting. Assign the legacy alerts to a supported notification channel so that you continue to receive notifications for those alerts.
Silences (expiring after one year) are created for all paused dashboard alerts.
@@ -0,0 +1,25 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/opt-in/
- /docs/grafana/latest/alerting/opt-in/
- /docs/grafana/latest/alerting/unified-alerting/opt-in/
description: Opt in to Grafana Alerting
title: Opt in to Grafana Alerting
weight: 104
---
# Opt in to Grafana Alerting
If you have previously disabled alerting in Grafana, or opted out of Grafana Alerting and have decided that you would now like to use Grafana Alerting, you can choose to opt in at any time.
If you have been using legacy alerting up until now your existing alerts will be migrated to the new alerts type and no alerts or alerting data are lost. Even if you choose to opt in to Grafana Alerting, you can roll back to legacy alerting at any time.
To opt in to Grafana Alerting, enter the following in your configuration:
```
[alerting]
enabled = false
[unified_alerting]
enabled = true
```
@@ -1,42 +1,49 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/opt-out/
- /docs/grafana/latest/alerting/opt-in/
- /docs/grafana/latest/alerting/unified-alerting/opt-in/
description: Disable Grafana alerts
title: Opt-out of Grafana alerting
weight: 113
description: Opt out of Grafana Alerting
title: Opt out of Grafana Alerting
weight: 102
---
# Opt-out to Grafana alerting in OSS
# Opt out of Grafana Alerting
This topic discusses how to disable Grafana alerting and migrate to legacy dashboard alerting. It also provides guidance on how to enable Grafana alerting once you are ready to migrate to Grafana alerting.
If you have an existing installation, you can opt out of alerting in its entirety or opt out of Grafana Alerting in favor of using legacy alerting.
> **Note:** This topic is only relevant for OSS and Enterprise customers. Contact customer support to enable or disable Grafana alerting for your Grafana Cloud stack.
Existing installations that do not use legacy alerting will have Grafana Alerting enabled by default unless alerting is disabled in the configuration. To keep alerting disabled:
## Before you begin
1. Go to your custom configuration file ($WORKING_DIR/conf/custom.ini).
1. Enter the following in your configuration:
We recommend that you backup Grafana's database. If you are using PostgreSQL as the backend database, then the minimum required version is 9.5.
```
[alerting]
enabled = false
## Opt-out of Grafana alerts
[unified_alerting]
enabled = false
```
To opt-out of Grafana alerts and roll back to legacy dashboard alerting:
3. Restart Grafana for the configuration changes to take effect.
1. In your custom configuration file ($WORKING_DIR/conf/custom.ini), go to the [Grafana alerting]({{< relref "../../setup-grafana/configure-grafana/#unified_alerting" >}}) section.
1. Set the `enabled` property to `false`.
1. For [legacy dashboard alerting]({{< relref "../../setup-grafana/configure-grafana/#alerting" >}}), set the `enabled` flag to `true`.
1. Restart Grafana for the configuration changes to take effect.
If at any time you want to turn alerting back on, you can do so.
> **Note:** Rolling back from Grafana to legacy alerting can result in data loss. This is applicable to the fresh installation as well as upgraded setups.
Existing installations that use legacy alerting will automatically be upgraded to Grafana Alerting unless you have opted-out of Grafana Alerting before migration takes place. During the upgrade, legacy alerts are migrated to the new alerts type and no alerts or alerting data are lost. To keep using legacy alerting and disable Grafana Alerting:
## Opt-in to Grafana alerting
1. Go to your custom configuration file ($WORKING_DIR/conf/custom.ini).
2. Enter the following in your configuration:
When you are ready to make the switch, the following procedure will help you migrate to Grafana alerting.
```
[alerting]
enabled = false
To opt-in to Grafana alerts:
[unified_alerting]
enabled = true
```
1. In your custom configuration file ($WORKING_DIR/conf/custom.ini), go to the [unified alerts]({{< relref "../../setup-grafana/configure-grafana/#unified_alerting" >}}) section.
1. Set the `enabled` property to `true`.
1. Next, for [legacy dashboard alerting]({{< relref "../../setup-grafana/configure-grafana/#alerting" >}}), set the `enabled` flag to `false`.
1. Restart Grafana for the configuration changes to take effect.
Installations that have been migrated to Grafana Alerting can [roll back]({{< relref "roll-back/" >}}) to legacy alerting at any time.
> **Note:** The `ngalert` toggle previously used to enable or disable Grafana alerting is no longer available.
> **Note:** This topic is only relevant for OSS and Enterprise customers. Contact customer support to enable or disable Grafana Alerting for your Grafana Cloud stack.
The `ngalert` toggle previously used to enable or disable Grafana Alerting is no longer available.
@@ -0,0 +1,29 @@
---
aliases:
- /docs/grafana/latest/alerting/migrating-alerts/roll-back/
- /docs/grafana/latest/alerting/opt-in/
- /docs/grafana/latest/alerting/unified-alerting/roll-back/
description: Roll back to legacy alerting
title: Roll back to legacy alerting
weight: 103
---
# Roll back to legacy alerting
Once the upgrade has taken place, you still have the option to roll back to legacy alerting. If you choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place.
All new alerts and changes made exclusively in Grafana Alerting will be deleted.
To roll back to legacy alerting, enter the following in your configuration:
```
force_migration = true
[alerting]
enabled = true
[unified_alerting]
enabled = false
```
> **Note**: We do not recommend this option. If you choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place. All new alerts and changes made exclusively in Grafana Alerting will be deleted.
@@ -23,7 +23,7 @@ You can configure Grafana managed notification policies as well as notification
{{< figure max-width="40%" src="/static/img/docs/alerting/unified/notification-policies-grouping.png" max-width="650px" caption="Notification policies grouping" >}}
Grouping is a new and key concept of Grafana alerting that categorizes alert notifications of similar nature into a single funnel. This allows you to properly route alert notifications during larger outages when many parts of a system fail at once causing a high number of alerts to fire simultaneously.
Grouping is a new and key concept of Grafana Alerting that categorizes alert notifications of similar nature into a single funnel. This allows you to properly route alert notifications during larger outages when many parts of a system fail at once causing a high number of alerts to fire simultaneously.
For example, suppose you have 100 services connected to a database in different environments. These services are differentiated by the label `env=environmentname`. An alert rule is in place to monitor whether your services can reach the database named `alertname=DatabaseUnreachable`.
+3 -3
View File
@@ -2,12 +2,12 @@
title = "Performance considerations"
description = "Understanding alerting performance"
keywords = ["grafana", "alerting", "performance"]
weight = 100
weight = 555
+++
# Alerting performance considerations
Grafana alerting supports multi-dimensional alerting, where one alert rule can generate many alerts. For example, you can configure an alert rule to fire an alert every time the CPU of individual VMs max out. This topic discusses performance considerations resulting from multi-dimensional alerting.
Grafana Alerting supports multi-dimensional alerting, where one alert rule can generate many alerts. For example, you can configure an alert rule to fire an alert every time the CPU of individual VMs max out. This topic discusses performance considerations resulting from multi-dimensional alerting.
Evaluating alerting rules consumes RAM and CPU to compute the output of an alerting query, and network resources to send alert notifications and write the results to the Grafana SQL database. The configuration of individual alert rules affects the resource consumption and, therefore, the maximum number of rules a given configuration can support.
@@ -19,6 +19,6 @@ The following section provides a list of alerting performance considerations.
Each evaluation of an alert rule generates a set of alert instances; one for each member of the result set. The state of all the instances is written to the `alert_instance` table in Grafana's SQL database.
Grafana alerting exposes a metric, `grafana_alerting_rule_evaluations_total` that counts the number of alert rule evaluations. To get a feel for the influence of rule evaluations on your Grafana instance, you can observe the rate of evaluations and compare it with resource consumption. In a Prometheus-compatible database, you can use the query `rate(grafana_alerting_rule_evaluations_total[5m])` to compute the rate over 5 minute windows of time. It's important to remember that this isn't the full picture of rule evaluation. For example, the load will be unevenly distributed if you have some rules that evaluate every 10 seconds, and others every 30 minutes.
Grafana Alerting exposes a metric, `grafana_alerting_rule_evaluations_total` that counts the number of alert rule evaluations. To get a feel for the influence of rule evaluations on your Grafana instance, you can observe the rate of evaluations and compare it with resource consumption. In a Prometheus-compatible database, you can use the query `rate(grafana_alerting_rule_evaluations_total[5m])` to compute the rate over 5 minute windows of time. It's important to remember that this isn't the full picture of rule evaluation. For example, the load will be unevenly distributed if you have some rules that evaluate every 10 seconds, and others every 30 minutes.
These factors all affect the load on the Grafana instance, but you should also be aware of the performance impact that evaluating these rules has on your data sources. Alerting queries are often the vast majority of queries handled by monitoring databases, so the same load factors that affect the Grafana instance affect them as well.
+2 -1
View File
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/silences/
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Silences
keywords:
@@ -15,7 +16,7 @@ weight: 450
Use silences to stop notifications from one or more alerting rules. Silences do not prevent alert rules from being evaluated. Nor do they not stop alerting instances from being shown in the user interface. Silences only stop notifications from getting created. A silence lasts for only a specified window of time.
Silences do not prevent alert rules from being evaluated. They also do not stop alert instances being shown in the user interface. Silences only prevent notifications from being created.
Silences do not prevent alert rules from being evaluated. They also do not stop alert instances from being shown in the user interface. Silences only prevent notifications from being created.
You can configure Grafana managed silences as well as silences for an [external Alertmanager data source]({{< relref "../../datasources/alertmanager/" >}}). For more information, see [Alertmanager]({{< relref "../fundamentals/alertmanager/" >}}).
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/silences/create-silence/
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Add silence alert notification
keywords:
@@ -18,7 +19,7 @@ Silences stop notifications from getting created and last for only a specified w
To add a silence:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
2. In the Alerting page, click **Silences** to open the page listing existing contact points.
2. On the Alerting page, click **Silences** to open the page listing existing contact points.
3. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana.
4. Click **New Silence** to open the Create silence page.
5. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire.
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/silences/edit-silence/
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Edit alerting silences
keywords:
@@ -1,26 +0,0 @@
---
aliases:
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Silences alert notifications
keywords:
- grafana
- alerting
- silence
- mute
title: Label matching and alert suppression
weight: 452
---
# About label matching for alert suppression
Grafana suppresses notifications only for alerts with labels that match all the "Matching Labels" specified in the silence.
- The **Label** field is the name of the label to match. It must exactly match the label name.
- The **Operator** field is the operator to match against the label value. The available operators are:
- `=`: Select labels that are exactly equal to the provided string.
- `!=`: Select labels that are not equal to the provided string.
- `=~`: Select labels that regex-match the provided string.
- `!~`: Select labels that do not regex-match the provided string.
- The **Value** field matches against the corresponding value for the specified **Label** name. How it matches depends on the **Operator** value.
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/silences/linking-to-silence-form/
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Linking to a silence form
keywords:
@@ -1,5 +1,6 @@
---
aliases:
- /docs/grafana/latest/alerting/silences/remove-silence/
- /docs/grafana/latest/alerting/unified-alerting/silences/
description: Remove alerting silences
keywords: