From 68c44f1dd969188261b5d696c4c0736abccca424 Mon Sep 17 00:00:00 2001 From: Pepe Cano <825430+ppcano@users.noreply.github.com> Date: Mon, 3 Jun 2024 11:00:52 +0200 Subject: [PATCH] Alerting docs: Update Alertmanager docs (#88567) * Remove alertmanager page and set redirects * Update internal alertmanager links * Update `Alertmanager` docs * Change heading to `Configure Alertmanagers` --- .../configure-notifications/mute-timings.md | 13 ++--- docs/sources/alerting/fundamentals/_index.md | 5 -- .../fundamentals/alert-rules/_index.md | 12 ++--- .../notifications/alertmanager.md | 47 ------------------- .../set-up/configure-alertmanager/index.md | 41 ++++++++++++++-- 5 files changed, 46 insertions(+), 72 deletions(-) delete mode 100644 docs/sources/alerting/fundamentals/notifications/alertmanager.md diff --git a/docs/sources/alerting/configure-notifications/mute-timings.md b/docs/sources/alerting/configure-notifications/mute-timings.md index 214fa13732d..5311e0483c0 100644 --- a/docs/sources/alerting/configure-notifications/mute-timings.md +++ b/docs/sources/alerting/configure-notifications/mute-timings.md @@ -20,16 +20,11 @@ labels: title: Configure mute timings weight: 450 refs: - datasources/alertmanager: + external-alertmanager: - pattern: /docs/grafana/ - destination: /docs/grafana//datasources/alertmanager/ + destination: /docs/grafana//alerting/set-up/configure-alertmanager/ - pattern: /docs/grafana-cloud/ - destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/alertmanager/ - intro-alertmanager: - - pattern: /docs/grafana/ - destination: /docs/grafana//alerting/fundamentals/notifications/alertmanager/ - - pattern: /docs/grafana-cloud/ - destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/alertmanager/ + destination: /docs/grafana-cloud/alerting-and-irm/alerting/set-up/configure-alertmanager/ --- # Configure mute timings @@ -38,7 +33,7 @@ A mute timing is a recurring interval of time when no new notifications for a po Similar to silences, mute timings do not prevent alert rules from being evaluated, nor do they stop alert instances from being shown in the user interface. They only prevent notifications from being created. -You can configure Grafana managed mute timings as well as mute timings for an [external Alertmanager data source](ref:datasources/alertmanager). For more information, refer to [Alertmanager documentation](ref:intro-alertmanager). +You can configure Grafana managed mute timings as well as mute timings for an [external Alertmanager](ref:external-alertmanager). ## Mute timings vs silences diff --git a/docs/sources/alerting/fundamentals/_index.md b/docs/sources/alerting/fundamentals/_index.md index 63b616d97bd..b9dd88611e3 100644 --- a/docs/sources/alerting/fundamentals/_index.md +++ b/docs/sources/alerting/fundamentals/_index.md @@ -13,11 +13,6 @@ menuTitle: Introduction title: Introduction to Alerting weight: 100 refs: - alertmanager: - - pattern: /docs/grafana/ - destination: /docs/grafana//alerting/fundamentals/notifications/alertmanager/ - - pattern: /docs/grafana-cloud/ - destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/alertmanager/ notification-policies: - pattern: /docs/grafana/ destination: /docs/grafana//alerting/fundamentals/notifications/notification-policies/ diff --git a/docs/sources/alerting/fundamentals/alert-rules/_index.md b/docs/sources/alerting/fundamentals/alert-rules/_index.md index 1ebd1d00e26..ee8f31d3f03 100644 --- a/docs/sources/alerting/fundamentals/alert-rules/_index.md +++ b/docs/sources/alerting/fundamentals/alert-rules/_index.md @@ -34,11 +34,11 @@ refs: destination: /docs/grafana//alerting/configure-notifications/template-notifications/images-in-notifications/ - pattern: /docs/grafana-cloud/ destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/template-notifications/images-in-notifications/ - alert-manager: + notifications: - pattern: /docs/grafana/ - destination: /docs/grafana//alerting/fundamentals/notifications/alertmanager/ + destination: /docs/grafana//alerting/fundamentals/notifications/ - pattern: /docs/grafana-cloud/ - destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/alertmanager/ + destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/ create-recording-rules: - pattern: /docs/grafana/ destination: /docs/grafana//alerting/alerting-rules/create-mimir-loki-managed-recording-rule/ @@ -76,10 +76,8 @@ Additionally, you can also add [expressions to transform your data](ref:expressi {{< figure src="/media/docs/alerting/grafana-managed-alerting-architecture.png" max-width="750px" caption="How Grafana-managed alerting works by default" >}} 1. Alert rules are created within Grafana based on one or more data sources. - 1. Alert rules are evaluated by the Alert Rule Evaluation Engine from within Grafana. - -1. Firing and resolved alert instances are delivered to the internal Grafana [Alertmanager](ref:alert-manager) which handles notifications. +1. Firing and resolved alert instances are forwarded to [handle their notifications](ref:notifications). ### Supported data sources @@ -98,7 +96,7 @@ They are only supported for Prometheus-based or Loki data sources with the Ruler 1. Alert rules are created and stored within the data source itself. 1. Alert rules can only query Prometheus-based data. It can use either queries or [recording rules](#recording-rules). 1. Alert rules are evaluated by the Alert Rule Evaluation Engine. -1. Firing and resolved alert instances are delivered to the configured [Alertmanager](ref:alert-manager) which handles notifications. +1. Firing and resolved alert instances are forwarded to [handle their notifications](ref:notifications). ### Recording rules diff --git a/docs/sources/alerting/fundamentals/notifications/alertmanager.md b/docs/sources/alerting/fundamentals/notifications/alertmanager.md deleted file mode 100644 index d2a82c883bf..00000000000 --- a/docs/sources/alerting/fundamentals/notifications/alertmanager.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -aliases: - - ../../fundamentals/alertmanager/ # /docs/grafana//alerting/fundamentals/alertmanager/ - - ../../unified-alerting/fundamentals/alertmanager/ # /docs/grafana//alerting/unified-alerting/fundamentals/alertmanager/ - - ../../manage-notifications/alertmanager/ # /docs/grafana//alerting/manage-notifications/alertmanager/ -canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/alertmanager/ -description: Learn about Alertmanagers and the Alertmanager options for Grafana Alerting -labels: - products: - - cloud - - enterprise - - oss -title: Alertmanager -weight: 111 -refs: - configure-alertmanager: - - pattern: /docs/grafana/ - destination: /docs/grafana//alerting/set-up/configure-alertmanager/ - - pattern: /docs/grafana-cloud/ - destination: /docs/grafana-cloud/alerting-and-irm/alerting/set-up/configure-alertmanager/ ---- - -# Alertmanager - -Grafana sends firing and resolved alerts to Alertmanagers. The Alertmanager receives alerts, handles silencing, inhibition, grouping, and routing by sending notifications out via your channel of choice, for example, email or Slack. - -Grafana has its own Alertmanager, referred to as "Grafana" in the user interface, but also supports sending alerts to other Alertmanagers, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/). You can use both internal and external Alertmanagers. - -The Grafana Alertmanager uses notification policies and contact points to configure how and where a notification is sent; how often a notification should be sent; and whether alerts should all be sent in the same notification, sent in grouped notifications based on a set of labels, or as separate notifications. - -Alertmanagers are visible from the drop-down menu on the Alerting Contact Points, Notification Policies, and Silences pages. - -In Grafana, you can use the Cloud Alertmanager, Grafana Alertmanager, or an external Alertmanager. You can also run multiple Alertmanagers; your decision depends on your set up and where your alerts are being generated. - -- **Grafana Alertmanager** is an internal Alertmanager that is pre-configured and available for selection by default if you run Grafana on-premises or open source. - - The Grafana Alertmanager can receive alerts from Grafana, but it cannot receive alerts from outside Grafana, for example, from Mimir or Loki. Note that inhibition rules are not supported. - -- **Cloud Alertmanager** runs in Grafana Cloud and it can receive alerts from Grafana, Mimir, and Loki. - -- **External Alertmanager** can receive all your Grafana, Loki, Mimir, and Prometheus alerts. External Alertmanagers can be configured and administered from within Grafana itself. - -Here are two examples of when you may want to [add your own external Alertmanager](ref:configure-alertmanager) and send your alerts there instead of the Grafana Alertmanager: - -1. You may already have Alertmanagers on-premises in your own Cloud infrastructure that you have set up and still want to use, because you have other alert generators, such as Prometheus. - -2. You want to use both Prometheus on-premises and hosted Grafana to send alerts to the same Alertmanager that runs in your Cloud infrastructure. diff --git a/docs/sources/alerting/set-up/configure-alertmanager/index.md b/docs/sources/alerting/set-up/configure-alertmanager/index.md index cc8edc3731d..da8117d3662 100644 --- a/docs/sources/alerting/set-up/configure-alertmanager/index.md +++ b/docs/sources/alerting/set-up/configure-alertmanager/index.md @@ -1,8 +1,12 @@ --- aliases: - - ../configure-alertmanager/ # /docs/grafana//configure-alertmanager/ + - ../../configure-alertmanager/ # /docs/grafana//configure-alertmanager/ + - ../unified-alerting/fundamentals/alertmanager/ # /docs/grafana//alerting/unified-alerting/fundamentals/alertmanager/ + - ../manage-notifications/alertmanager/ # /docs/grafana//alerting/manage-notifications/alertmanager/ + - ../fundamentals/alertmanager/ # /docs/grafana//alerting/fundamentals/alertmanager/ + - ../fundamentals/notifications/alertmanager/ # /docs/grafana//alerting/fundamentals/notifications/alertmanager canonical: https://grafana.com/docs/grafana/latest/alerting/set-up/configure-alertmanager/ -description: Configure an Alertmanager to receive all of your alerts +description: Learn about Alertmanagers and set up Alerting to use an external Alertmanager keywords: - grafana - alerting @@ -11,15 +15,42 @@ keywords: - external Alertmanager labels: products: + - cloud - enterprise - oss title: Add an external Alertmanager weight: 200 +refs: + notifications: + - pattern: /docs/grafana/ + destination: /docs/grafana//alerting/fundamentals/notifications/ + - pattern: /docs/grafana-cloud/ + destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/ --- -# Add an external Alertmanager +## Configure Alertmanagers -Set up Grafana to use an external Alertmanager as a single Alertmanager to receive all of your alerts. This external Alertmanager can then be configured and administered from within Grafana itself. +Grafana Alerting is based on the architecture of the Prometheus alerting system. Grafana sends firing and resolved alerts to an Alertmanager, which is responsible for [handling notifications](ref:notifications). + +{{< figure src="/media/docs/alerting/alerting-alertmanager-architecture.png" max-width="750px" alt="A diagram with the alert generator and alert manager architecture" >}} + +Grafana has its own pre-configured Alertmanager, referred to as "Grafana" in the user interface: + +- **Grafana Alertmanager** is the default internal Alertmanager if you run Grafana on-premises or as open source. It can receive alerts from Grafana but cannot receive alerts from external alert generators such as Mimir or Loki. + +- **Cloud Alertmanager** runs in Grafana Cloud and can receive Grafana-managed alerts and Data sources-managed alerts like Mimir, Loki, and Prometheus. + +Grafana Alerting also supports sending alerts to **External Alertmanagers**, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), which can receive alerts from Grafana, Loki, Mimir, and Prometheus. + +You can use both internal and external Alertmanagers. The decision often depends on your alerting setup and where your alerts are being generated. Here are two examples of when you may want to [add an external Alertmanager](#add-an-external-alertmanager) and send your alerts there instead of the default Grafana Alertmanager: + +1. You may already have Alertmanagers on-premises in your own Cloud infrastructure that you still want to use because you have other alert generators, such as Prometheus. + +2. You want to use both Prometheus on-premises and hosted Grafana to send alerts to the same Alertmanager that runs in your Cloud infrastructure. + +## Add an external Alertmanager + +From Grafana, you can configure and administer your own external Alertmanager to receive Grafana alerts. {{% admonition type="note" %}} Grafana Alerting does not support sending alerts to the AWS Managed Service for Prometheus due to the lack of sigv4 support in Prometheus. @@ -27,6 +58,8 @@ Grafana Alerting does not support sending alerts to the AWS Managed Service for After you have added the Alertmanager, you can use the Grafana Alerting UI to manage silences, contact points, and notification policies. A drop-down option in these pages allows you to switch between alertmanagers. +{{< figure src="/media/docs/alerting/alerting-choose-alertmanager.png" max-width="750px" alt="A screenshot choosing an Alertmanager in the notification policies UI" >}} + External alertmanagers should now be configured as data sources using Grafana Configuration from the main Grafana navigation menu. This enables you to manage the contact points and notification policies of external alertmanagers from within Grafana and also encrypts HTTP basic authentication credentials. To add an external Alertmanager, complete the following steps.