public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[v10.0.x] Docs: bit of restructuring (#70380)

Browse Source 
* Docs: bit of restructuring (#70364)

* Docs: bit of restructuring

* corrects link

* fixes link

* updates

* fixes links

* fixes links

* fixes links

* fixes links

* fixes links

(cherry picked from commit d529636bc3)

* fixes link

---------

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
This commit is contained in:
Gilles De Mey
2023-06-20 14:55:47 +02:00
committed by GitHub
co-authored by brendamuir
parent 73feec5910
commit c6cfae6b97
20 changed files with 223 additions and 241 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/alerting/set-up/_index.md
+7 -6
View File
@@ -1,23 +1,24 @@
---
menuTitle: Set up
aliases:
- unified-alerting/set-up/
labels:
products:
- oss
description: How to set up alerting features and integrations
description: How to configure alerting features and integrations
title: Set up Alerting
weight: 107
---
# Set up Alerting
Configure the features and integrations that you need to create and manage your alerts.
Set up or upgrade your implementation of Grafana Alerting.
**Note:**
These are set up instructions for Grafana Alerting Open Source.
These are set-up instructions for Grafana Alerting Open Source.
To set up Grafana Alerting for Cloud, see ({{< relref "../set-up-cloud" >}})
To set up Grafana Alerting for Cloud, see ({{< relref "./_index.md" >}})
## Before you begin
@@ -47,9 +48,9 @@ To set up Alerting, you need to:
- [Optional] Add labels and label matchers to control alert routing
1. [Optional] Integrate with [Grafana OnCall]
(/docs/oncall/latest/integrations/grafana-alerting/)
(/docs/oncall/latest/integrations/grafana-alerting)
## Advanced set-up options
## Advanced set up options
Grafana Alerting supports many additional configuration options, from configuring external Alertmanagers to routing Grafana-managed alerts outside of Grafana, to defining your alerting setup as code.
docs/sources/alerting/set-up/migrating-alerts/_index.md
+170
View File
@@ -0,0 +1,170 @@
---
aliases:
- difference-old-new/
- unified-alerting/
- unified-alerting/difference-old-new/
description: Upgrade Grafana alerts
title: Upgrade Alerting
weight: 150
---
# Upgrade Alerting
> **Note:** Legacy alerting will be deprecated in a future release (v10).
Grafana Alerting is enabled by default for new installations or existing installations whether or not legacy alerting is configured.
{{% admonition type="note" %}}
When upgrading, your dashboard alerts are migrated to a new format. This migration can be rolled back easily by opting out. If you have any questions regarding this migration, please contact us.
{{% /admonition %}}
Existing installations that do not use legacy alerting will have Grafana Alerting enabled by default unless alerting is disabled in the configuration.
Likewise, existing installations that use legacy alerting will be automatically upgraded to Grafana Alerting unless you have opted out of Grafana Alerting before migration takes place. During the upgrade, legacy alerts are migrated to the new alerts type and no alerts or alerting data are lost.
Once the upgrade has taken place, you still have the option to roll back to legacy alerting. However, we do not recommend choosing this option. If you do choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place. All new alerts and changes made exclusively in Grafana Alerting will be deleted.
{{% admonition type="note" %}}
Cloud customers, who do not want to upgrade to Grafana Alerting, should contact customer support.
{{% /admonition %}}
If you have opted out or rolled back, you can always choose to opt in to Grafana Alerting at a later point in time.
The following table provides details on the upgrade for Cloud, Enterprise, and OSS installations and the new Grafana Alerting UI.
| Grafana instance upgraded to 9.0 | |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Cloud | Existing Cloud installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting plugin icon and the new Grafana Alerting icon. During upgrade, existing alerts from the Cloud alerting plugin are migrated to Grafana Alerting. Once migration is complete, you can access and manage the older alerts from the new alerting Grafana Alerting icon in the navigation panel. The (older) Cloud alerting plugin is uninstalled from your cloud instance. Contact customer support if you **do not wish** to migrate to Grafana Alerting for your Cloud stack. If you choose to use legacy alerting, use the You will see the new Grafana Alerting icon as well as the old Cloud alerting plugin in the left navigation panel. |
| Enterprise | Existing Enterprise instances using legacy alerting will have both the old (marked as legacy) and the new alerting icons in the navigation panel. During upgrade, existing legacy alerts are migrated to Grafana Alerting. If you wish, you can opt out of Grafana Alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
| OSS | Existing OSS installations with legacy dashboard alerting will have two alerting icons in the left navigation panel - the old alerting icon (marked as legacy) and the new Grafana Alerting icon. During upgrade, existing legacy alerts are migrated to Grafana Alerting. If you wish, you can opt out of Grafana Alerting and roll back to legacy alerting. In that case, you can manage your legacy alerts from the alerting icon marked as legacy. |
> **Note:** Starting with v9.0, legacy alerting is deprecated and will be removed in a future release.
## Opt out
You can opt out of Grafana Alerting at any time and switch to using legacy alerting. Alternatively, you can opt out of using alerting in its entirety.
### Stay on legacy alerting
When upgrading to Grafana > 9.0, existing installations that use legacy alerting are automatically upgraded to Grafana Alerting unless you have opted-out of Grafana Alerting before migration takes place. During the upgrade, legacy alerts are migrated to the new alerts type and no alerts or alerting data are lost. To keep using legacy alerting and disable Grafana Alerting:
1. Go to your custom configuration file ($WORKING_DIR/conf/custom.ini).
2. Enter the following in your configuration:
```
[alerting]
enabled = true
[unified_alerting]
enabled = false
```
Installations that have been migrated to Grafana Alerting can roll back to legacy alerting at any time.
{{% admonition type="note" %}}
This topic is only relevant for OSS and Enterprise customers. Contact customer support to enable or disable Grafana Alerting for your Grafana Cloud stack.
{{% /admonition %}}
The `ngalert` toggle previously used to enable or disable Grafana Alerting is no longer available.
### Disable alerting
You can disable both Grafana Alerting and legacy alerting in Grafana.
1. Go to your custom configuration file ($WORKING_DIR/conf/custom.ini).
2. Enter the following in your configuration:
```
[alerting]
enabled = false
[unified_alerting]
enabled = false
```
3. Restart Grafana for the configuration changes to take effect.
If you want to turn alerting back on, you can remove both flags to enable Grafana Alerting.
## Roll back
Once the upgrade has taken place, you still have the option to roll back to legacy alerting. If you choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place.
All new alerts and changes made exclusively in Grafana Alerting will be deleted.
To roll back to legacy alerting, enter the following in your configuration:
```
force_migration = true
[alerting]
enabled = true
[unified_alerting]
enabled = false
```
> **Note**: We do not recommend this option. If you choose to roll back, Grafana will restore your alerts to the alerts you had at the point in time when the upgrade took place. All new alerts and changes made exclusively in Grafana Alerting will be deleted.
## Opt in
If you have previously disabled alerting in Grafana, or opted out of Grafana Alerting and have decided that you would now like to use Grafana Alerting, you can choose to opt in at any time.
If you have been using legacy alerting up until now your existing alerts will be migrated to the new alerts type and no alerts or alerting data are lost. Even if you choose to opt in to Grafana Alerting, you can roll back to legacy alerting at any time.
To opt in to Grafana Alerting, enter the following in your configuration:
```
[alerting]
enabled = false
[unified_alerting]
enabled = true
```
## Disable
To disable alerting in Grafana entirely (including both legacy and Grafana Alerting), enter the following in your configuration:
```
[alerting]
enabled = false
[unified_alerting]
enabled = false
```
If at any time you want to turn alerting back on, you can opt in.
## Differences and limitations
There are some differences between Grafana Alerting and legacy dashboard alerts, and a number of features that are no
longer supported.
### Differences
1. When Grafana Alerting is enabled or upgraded to Grafana 9.0 or later, existing legacy dashboard alerts migrate in a format compatible with the Grafana Alerting. In the Alerting page of your Grafana instance, you can view the migrated alerts alongside any new alerts.
This topic explains how legacy dashboard alerts are migrated and some limitations of the migration.
2. Read and write access to legacy dashboard alerts and Grafana alerts are governed by the permissions of the folders storing them. During migration, legacy dashboard alert permissions are matched to the new rules permissions as follows:
- If there are dashboard permissions, a folder named `Migrated {"dashboardUid": "UID", "panelId": 1, "alertId": 1}` is created to match the permissions of the dashboard (including the inherited permissions from the folder).
- If there are no dashboard permissions and the dashboard is in a folder, then the rule is linked to this folder and inherits its permissions.
- If there are no dashboard permissions and the dashboard is in the General folder, then the rule is linked to the `General Alerting` folder and the rule inherits the default permissions.
3. `NoData` and `Error` settings are migrated as is to the corresponding settings in Grafana Alerting, except in two situations:
3.1. As there is no `Keep Last State` option for `No Data` in Grafana Alerting, this option becomes `NoData`. The `Keep Last State` option for `Error` is migrated to a new option `Error`. To match the behavior of the `Keep Last State`, in both cases, during the migration Grafana automatically creates a silence for each alert rule with a duration of 1 year.
3.2. Due to lack of validation, legacy alert rules imported via JSON or provisioned along with dashboards can contain arbitrary values for `NoData` and [`Error`](/docs/sources/alerting/alerting-rules/create-grafana-managed-rule.md#configure-no-data-and-error-handling). In this situation, Grafana will use the default setting: `NoData` for No data, and `Error` for Error.
4. Notification channels are migrated to an Alertmanager configuration with the appropriate routes and receivers. Default notification channels are added as contact points to the default route. Notification channels not associated with any Dashboard alert go to the `autogen-unlinked-channel-recv` route.
5. Unlike legacy dashboard alerts where images in notifications are enabled per contact point, images in notifications for Grafana Alerting must be enabled in the Grafana configuration, either in the configuration file or environment variables, and are enabled for either all or no contact points.
6. The JSON format for webhook notifications has changed in Grafana Alerting and uses the format from [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config).
### Limitations
1. Since `Hipchat` and `Sensu` notification channels are no longer supported, legacy alerts associated with these channels are not automatically migrated to Grafana Alerting. Assign the legacy alerts to a supported notification channel so that you continue to receive notifications for those alerts.
Silences (expiring after one year) are created for all paused dashboard alerts.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting-deprecation.md
+45
View File
@@ -0,0 +1,45 @@
---
title: Legacy alerting deprecation
description: Legacy alerting deprecation notice
weight: 109
keywords:
- grafana
- alerting
---
# Legacy alerting deprecation
Starting with Grafana v9.0.0, legacy alerting is deprecated, meaning that it is no longer actively maintained or supported by Grafana. As of Grafana v10.0.0, we do not contribute or accept external contributions to the codebase apart from CVE fixes.
Legacy alerting refers to the old alerting system that was used prior to the introduction of Grafana Alerting; the new alerting system in Grafana.
The decision to deprecate legacy alerting was made to encourage users to migrate to the new alerting system, which offers a more powerful and flexible alerting experience based on Prometheus Alertmanager.
Users who are still using legacy alerting are encouraged to migrate their alerts to the new system as soon as possible to ensure that they continue to receive new features, bug fixes, and support.
However, we will still patch CVEs until legacy alerting is completely removed in Grafana 11; honoring our commitment to building and distributing secure software.
We have provided [upgrade instructions]({{< relref "./_index.md" >}}) on how to migrate to the new alerting system, making the process as easy as possible for users.
## Why are we deprecating legacy alerting?
The new Grafana alerting system is more powerful and flexible than the legacy alerting feature.
The new system is based on Prometheus Alertmanager, which offers a more comprehensive set of features for defining and managing alerts. With the new alerting system, users can create alerts based on complex queries, configure alert notifications via various integrations, and set up sophisticated alerting rules with support for conditional expressions, aggregation, and grouping.
Overall, the new alerting system in Grafana is a major improvement over the legacy alerting feature, providing users with a more powerful and flexible alerting experience.
Additionally, legacy alerting still requires Angular to function and we are [planning to remove support for it]({{< relref "../../../developers/angular_deprecation" >}}) in Grafana 11.
## When will we remove legacy alerting completely?
Legacy alerting will be removed from the code-base in Grafana 11, following the same timeline as the [Angular deprecation]({{< relref "../../../developers/angular_deprecation" >}}).
## How do I migrate to the new Grafana alerting?
Refer to our [upgrade instructions]({{< relref "./_index.md" >}}).
### Links
- [Upgrade Alerting]({{< relref "./_index.md" >}})
- [Angular support deprecation]({{< relref "../../../developers/angular_deprecation" >}})
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/_index.md
+95
View File
@@ -0,0 +1,95 @@
---
aliases:
- /docs/grafana-cloud/alerts/
- /docs/grafana-cloud/how-do-i/alerts/
- /docs/grafana-cloud/legacy-alerting/
description: Legacy alerting
title: Legacy alerting
weight: 110
---
# Legacy alerting
**Note:**
Starting with Grafana v9.0.0, legacy alerting is deprecated. It is no longer actively maintained or supported by Grafana and will be removed in Grafana v11.0.0.
You have two options to configure alerts within the Grafana Cloud GUI and a third option that enables you to set Grafana Cloud Alerts using the command line.
- **Grafana alerts** are the same as in an on-prem instance of Grafana.
These alerts are created from a graph panel within a Grafana dashboard.
This is useful when you want to create a simple alert based on one metric from within a panel.
It also has a much simpler learning curve when you are getting started.
- **Grafana Cloud alerts - GUI** are an implementation of Prometheus-style rules that enable you to query your Grafana Cloud Metrics and then set up Prometheus Alertmanager-style alerts based on those rules.
This is useful when you want to create precise, PromQL-based rules or create alerts from across many metrics and logs being collected into your Grafana Cloud Metrics.
This form of alerting is much more powerful and configurable, but that comes with some complexity.
- **Grafana Cloud alerts - CLI** use mimirtool to create and upload the same types of Prometheus-style recording and alerting rules definitions to your Grafana Cloud Metrics instance.
Once created, you will also be able to view these rules from within the Grafana Cloud Alerting page in the GUI.
- **Synthetic Monitoring alerts** are built on Prometheus alerts, just like in Grafana Cloud alerting.
You can configure synthetic monitoring alerts separately using the UI in synthetic monitoring.
Another option to create alerts for synthetic monitoring checks is to simply use Grafana Cloud alerting.
## Using Grafana alerts in Grafana Cloud
Grafana alerts are dashboard panel-driven and can only be created using the Graph panel.
This style of alerting builds on top of the query defined for the graph visualization, so alerts and notifications are sent based on breaking some threshold in the associated panel.
This also means that there is a one-to-one relationship between a Grafana alert and a graph panel.
So although Grafana alerts can be viewed centrally, they can only be managed directly from the panel that they’re tied to.
As a result, Grafana alerting is best suited for smaller setups, where there are only a few individuals or teams responsible for a small set of dashboards and where there are few dependencies between the dashboards.
{{% admonition type="note" %}}
Most curated dashboards, such as those provided with an integration or with Synthetic Monitoring do not allow you to alert from panels.
This is to preserve the ability to upgrade these dashboards automatically when the integration or Synthetic Monitoring abilities are updated.
To create an editable copy that you can edit and alert from, click settings (the gear logo) within any dashboard and then click **Make Editable**.
The copy will not be upgraded when/if the curated dashboard receives an update.
This is one reason why Grafana Cloud Alerts may be considered a better option.
{{% /admonition %}}
### What makes Grafana alerts unique?
With Grafana alerts, alerts are limited to only graph panels within dashboards.
In addition:
- Alerts can be edited by both Editor and Admin roles
- Alerts are visual, with an associated alerting threshold line
- Alerts work with many non-Prometheus data sources, including Graphite
- Alert notifications can be routed to many external notifier systems, directly from Grafana
- Alerts are directly associated with a dashboard
- Alerts can be tested
## Using Grafana Cloud Alerts
Because the metrics you collect and send to Grafana Cloud are centrally stored in one large time-series database, Grafana Cloud Metrics, you can query across these metrics using [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) and build alerts directly around those metrics rather than around a panel.
You can also query across any logs you have sent using Loki.
Grafana Cloud Alerts are directly tied to metrics and log data.
They can be configured either through the UI or by uploading files containing Prometheus and Loki alert rules with mimirtool.
Grafana Cloud Alerting's Prometheus-style alerts are built by querying directly from the data source itself.
Because these alerts are based on the data, they are not tied to a single panel.
This makes it possible to evaluate and centrally manage alerts across several different Prometheus and Loki data source instances.
### What makes Grafana Cloud Alerts unique?
With Grafana Cloud Alerts, alerts are not limited to coming from a graph panel.
In addition, you can:
- Prevent alerts from being edited, except by users with accounts that are assigned Admin roles.
- Centrally manage and create alerts across many systems, teams, and dashboards.
Alerts are not bound to just one system, team, or dashboard.
- Create alerts for both metric _and_ log data, based on Prometheus and Loki, respectively.
- Silence and mute alerts in bulk, even using a schedule, using the Alertmanager.
- Route alert notifications to [many external notifier systems](https://prometheus.io/docs/operating/integrations/#alertmanager-webhook-receiver) using Alertmanager configurations
- Dedupe alert notifications automatically.
### Grafana Cloud Alert configuration methods
In a traditional on-prem environment, Prometheus-style alert configuration is done through the combination of defining a [Prometheus configuration file](https://prometheus.io/docs/prometheus/latest/configuration/configuration/) and an [Alertmanager configuration file](https://prometheus.io/docs/alerting/latest/configuration/), which live close to the Prometheus server.
With Grafana Cloud, you can still use this setup as well as more flexible architectures.
- You can use `mimirtool` to upload your configuration files to be hosted and evaluated entirely in Grafana Cloud.
- You can manage both alerting rules and Alertmanager configurations directly through the UI.
Configuration files are unnecessary with this setup.
- You can use both methods concurrently to manage the alerts.
For example, updates made using the `mimirtool` are automatically updated and visible within the Grafana Cloud Alerting interface in minutes.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/_index.md
+76
View File
@@ -0,0 +1,76 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/
description: Grafana Cloud Alerting
title: Grafana Cloud Alerting
weight: 100
---
# Grafana Cloud Alerting
Grafana Cloud Alerting allows you to create and manage all of your Prometheus-style alerting rules, for both Prometheus metrics and Loki log data. With this feature, you don't need to leave Grafana, upload or edit configuration files, or install additional tools.
![Grafana Cloud Alerting](/static/img/docs/grafana-cloud/grafana-cloud-alerting.png)
## Permissions
All members of an organization that have alerts set up can view alerts in Grafana Cloud Alerting. This includes everyone with a Viewer, Editor, or Admin role.
Users with the organization Admin role can also create, edit, or delete alerts.
## Data sources
Grafana Cloud Alerting supports rule management across multiple data sources, for both metrics and logs, across all of the stacks in your org. If you have more than one Prometheus or Loki data source, there will be a dropdown at the top for you to select the data source to configure rules.
{{% admonition type="note" %}}
Pay attention to which data source you select. Cloud alerts are tied to a specific data source. For example, if you have a Loki data source selected you will not be able to create an alert based on a Prometheus data source.
![Cloud Alerting Data Source](/static/img/docs/grafana-cloud/grafana-cloud-alerting-data-source.png)
{{% /admonition %}}
## Alerts and recording rules
Prometheus supports two types of rules:
- [Recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) - Recording rules allow you to execute expressions or queries, by saving them off as a stored rule instead.
- [Alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) - Alerting rules allow you to define alert conditions and to route those notifications to an external service. An alert fires if metrics meet criteria defined in the alerting rule.
Both of these rules are configurable from the Grafana Cloud Alerting interface and configured in the same way.
## Alert states
Alert states are identical to the standard format found in Prometheus rule configurations. In Grafana Cloud Alerting, each individual alert is highlighted by its state to more clearly distinguish between alerts.
- **Firing -** Alerts that have been active for longer than the configured threshold. Alerts are highlighted in red and tagged with a red `firing` label.
- **Pending -** Alerts that have been active for less than the configured threshold. Alerts are highlighted in orange.
- **Inactive -** Alerts that are neither firing nor pending. Alerts are highlighted in green.
## Notifications
The **Notifications** tab is where you can view all current notifications and sort them by various states, receivers, and labels.
![Grafana Cloud Alerting Notifications](/static/img/docs/grafana-cloud/grafana-cloud-alerting-notifications.png)
## Limits
There is a limit on how many rules can be created in a rule group. There is also a limit on how many rule groups can be created.
You can create:
- 20 rules per rule group
- 35 rule groups
> It is possible to increase these limits. Please contact customer support for further information.
If you exceed the limits, you will encounter an error similar to this:
```bash
ERROR[0000] requests failed fields.msg="request failed with response body
per-user rules per rule group limit (limit: 20 actual: 22) exceeded\n"
status="400 Bad Request"
ERROR[0000] unable to load rule group error="failed request to the cortex api"
group=limit_rules_per_group namespace=test
```
To increase the number of rules or rule groups you can configure, contact support to upgrade your account.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/alertmanager.md
+55
View File
@@ -0,0 +1,55 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/alertmanager/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/alertmanager/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/alertmanager/
description: Alertmanager
title: Alertmanager
weight: 500
---
# Alertmanager
Grafana Cloud Alerting allows you to edit and view configuration for your Alertmanager directly inside of Grafana. See the official [Alertmanager documentation](https://prometheus.io/docs/alerting/latest/configuration/) to learn how to configure.
{{% admonition type="note" %}}
Only organization Admins can view or update Alertmanger configurations.
{{% /admonition %}}
## Edit a config for Grafana Cloud Alerting
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alertmanager**.
1. If you have more than one Alertmanager source, there will be a dropdown at the top for you to select the data source to edit configurations.
1. Currently active configuration for the Alertmanager will be displayed. Click the **Edit** button to enter edit mode and start making changes. Click "Save and finish editing" once done to persist your changes.
1. Alternatively, updates to the Alertmanager configurations made using the mimirtool will also sync and appear here.
## Use the Grafana Labs-supplied SMTP option to configure email notifications
Grafana Cloud users who do not have an SMTP server available for sending alert emails may use Grafana-Labs supplied SMTP relay (available at `smtprelay:2525`).
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alertmanager**.
1. If you have more than one Alertmanager source, there will be a dropdown at the top for you to select the data source to edit configurations.
1. Find info box with heading **Send alert email notifications from Grafana Cloud** at the top
1. Enter desired email address into the **email address** field
1. Click **Update configuration** button. Alertmanager config will be updated with grafana SMTP relay settings and an "email" receiver that will send to the specified email address.
{{% admonition type="note" %}}
Following these steps will overwrite any custom global SMTP settings that you might have. Default route configuration will send all notifications to the "email" receiver. If you have already customized routes, they will not be updated and you will have to configure "email" receiver on the appropriate route.
{{% /admonition %}}
Use these settings in your Grafana Cloud Alerting YAML, if you do not find them already set. Most important is the `smtp_require_tls: false` line. If this is not set properly, alert emails will not be received. If you use mimirtool to configure alertmanager, by default this will be set to `true`, which will cause problems.
```yaml
global:
smtp_from: noreply@grafana.net
smtp_smarthost: smtprelay:2525
smtp_require_tls: false
```
## Troubleshooting Alertmanager failures
Configuration errors can cause Alertmanager notification failures, e.g. a typo in an email address recipient or an expired token for a webhook. Grafana Cloud provisions a Loki datasource `grafanacloud-<stack_slug>-usage-insights` which can be used to display select notification errors with a query similar to the example below. The `instance_type` label of `alerts` is what selects the Grafana Cloud Alertmanager logs.
```sql
{instance_type="alerts"} | logfmt | level="warn"
```
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/create-edit-rules.md
+59
View File
@@ -0,0 +1,59 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/create-edit-rules/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/create-edit-rules/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/create-edit-rules/
description: Create and edit alert rules
title: Create and edit alert rules
weight: 200
---
# Create and edit alert rules
Creating alerts in Grafana Cloud differs from creating alerts directly with Prometheus or Loki. While the rule format is the same, everything is done in the Grafana Cloud Alerting interface, rather than with configuration files.
{{% admonition type="note" %}}
Only organization Admins can create or edit alert rules.
{{% /admonition %}}
## Create an alert rule
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alerts and rules**.
1. If you have more than one Prometheus or Loki data source, there will be a dropdown at the top for you to select the data source to create or edit rules.
1. Click **Edit rules**.
1. Click **Add rule**.
Grafana creates a new rule with placeholders.
```
alert: ""
expr: ""
```
Enter text according to regular Prometheus rule configuration guidelines:
- [Recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/)
- [Alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
{{% admonition type="note" %}}
Grafana Cloud Alerting does not support comments.
{{% /admonition %}}
When you are finished, click **Save**. You can then repeat the process to create more rules or click **Finish editing** to return to the rules list.
## Edit an alert rule
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alerts and rules**.
1. If you have more than one Prometheus or Loki data source, there will be a dropdown at the top for you to select the data source to create or edit rules.
1. Click **Edit rules**.
1. Scroll down to the rule that you want to edit and then click **Edit**.
1. Make any necessary changes to the rule text and then click **Save**.
1. Click **Finish editing** to return to the rules list.
## Delete an alert rule
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alerts and rules**.
1. If you have more than one Prometheus or Loki data source, there will be a dropdown at the top for you to select the data source to create or edit rules.
1. Click **Edit rules**.
1. Scroll down to the rule that you want to edit and then click **Delete**.
1. Click **Finish editing** to return to the rules list.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/namespaces-and-groups.md
+50
View File
@@ -0,0 +1,50 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/namespaces-and-groups/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/namespaces-and-groups/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/namespaces-and-groups/
description: Namespaces and rule groups
title: Namespaces and rule groups
weight: 400
---
# Namespaces and rule groups
By default, all alerting and recording rules created in Grafana Cloud Alerting will default to a single namespace and a single rule group.
## Managing namespaces
While Grafana Cloud Alerting does support viewing multiple namespaces that have been added through the mimirtool, it is currently not possible to add new namespaces or to rename the existing ones.
## Managing rule groups
Rule groups can be managed directly within the Grafana Cloud Alerting interface or through the mimirtool, similar to managing namespaces.
{{% admonition type="note" %}}
By default, Grafana Cloud limits the number of rule groups to 20, with a limit of up to 15 rules per group. If you wish to increase the default limits, please [open a support ticket](/profile/org#support) or reach out to your account manager.
{{% /admonition %}}
### Create a new rule group:
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Alerts and rules**.
2. If you have more than one Prometheus or Loki data source, there will be a dropdown at the top for you to select the data source to create or edit rules.
3. Click **Create new rule group**.
4. Enter text to name your new rule group.
5. Enter text for the new rule in your new rule group, according to regular Prometheus rule configuration guidelines:
- [Recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/)
- [Alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
6. When you are finished naming your new rule group and adding new rule details, click **Save**.
{{% admonition type="note" %}}
In order to create a new rule group, you must also create a new rule for it.
{{% /admonition %}}
### Update a rule group
Existing rule groups can be renamed by selecting the **pencil** icon next to the rule group name.
### Delete a rule group
Rule groups will be automatically deleted once the all rules within a group are deleted.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/silences.md
+31
View File
@@ -0,0 +1,31 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/silences/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/silences/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/silences/
description: Silences
title: Silences
weight: 600
---
# Silences
Grafana Cloud Alerting allows you to manage silences for your alertmanager notifications directly inside of Grafana. This applies to alerting rules created for both Prometheus metrics and Loki logs.
## Create a silence
1. In Grafana, hover your cursor over the **Grafana Cloud Alerting** icon and then click **Silences**.
2. Click **New silence**.
3. Enter a date in **Start of silence** to indicate when the silence should go into effect.
4. Enter a date in **End of silence** to indicate when the silence should expire.
5. Enter one or more matchers by filling out the **Name** and **Value** fields. Matchers determine which rules the silence will apply to.
6. Enter the name of the owner in **Creator**.
7. Enter a **Comment**.
8. To view which rules will be affected by your silence, click **Preview alerts**.
9. Otherwise, when you are finished, click **Create**
## Update an existing silence
You can always update an existing silence by clicking the **Edit silence** button under the silence.
It is also possible to expire a silence, on-demand, by clicking the **Expire silence** button under the silence. This will override the original scheduled expiration date of the silence.
docs/sources/alerting/set-up/migrating-alerts/legacy-alerting/grafana-cloud-alerting/view-filter-rules.md
+42
View File
@@ -0,0 +1,42 @@
---
aliases:
- /docs/grafana-cloud/alerts/grafana-cloud-alerting/view-filter-rules/
- /docs/grafana-cloud/how-do-i/grafana-cloud-alerting/view-filter-alerts/
- /docs/grafana-cloud/legacy-alerting/grafana-cloud-alerting/view-filter-rules/
description: View and filter alert rules
title: View and filter alert rules
weight: 300
---
# View and filter alert rules
Grafana Cloud Alerting displays a list of all recording and alerting rules assigned to a selected data source in the Alerts and rules tab.
All members of an organization that have access to a particular data source can view the list of rules and filter or reorder their view.
## View alert rules
1. Hover your cursor over the **Grafana Cloud Alerting** icon (alarm bell with Prometheus logo) and then click **Alerts and rules**.
1. In the list at the top of the tab, select the data source for which you want to view rules.
Grafana displays rules according to rule groups. If your instance has added namespaces and alert groups, then they will be ordered alphabetically. Otherwise, you will have one namespace called `default` and an alert group called `rules`.
If an alert is firing, then click the down carrot arrow to see additional information. The Label and annotations section appears.
## Filter your alert rule view
You can control which alerts you see and in what order they appear several ways. Combine different filters to personalize your view so that you can quickly find the information that you need.
- **Filter by alert state -** Click the toggles to show or hide alerts in different states. Turn off the toggle to hide alerts matching the state.
- **Filter by rule type -** Click the toggles to show or hide alerting rules or recording rules.
- **View options -** Click the toggle to show or hide the Prometheus annotations shown in the Labels and annotations section.
- **Rule sorting -** Click an option to sort alert rules within each rule group.
- **None -** No special sort is applied and sorts as if in a file, ordered according to the editing list order.
- **A-Z -** Sorts rules alphabetically according to the rule name.
- **Alert state -** Sorts rules according to the alert state (Firing, Pending, or Inactive).
## View alert in Explore
Click **View in Explore** or click the `expr` link to open the `expr` in [Explore](/docs/grafana/latest/explore/).
> **Note:** Only users with Admin or Editor roles in an organization can use the Explore feature unless the viewers can edit.
docs/sources/alerting/set-up/set-up-cloud/_index.md
+19 -11
View File
@@ -11,14 +11,14 @@ aliases:
labels:
products:
- cloud
description: How to set up Alerting for Cloud
description: How to configure Alerting for Cloud
title: Set up Alerting for Cloud
weight: 100
---
# Set up Alerting for Cloud
Configure the features and integrations that you need to create and manage your alerts.
Set up your implementation of Grafana Alerting for Cloud.
Grafana Cloud alerts are directly tied to metrics and log data.
@@ -30,7 +30,7 @@ Grafana Cloud Alerting's Prometheus-style alerts are built by querying directly
These are set up instructions for Grafana Alerting Cloud.
To set up Grafana Alerting for Open Source, see ({{< relref "../set-up" >}})
To set up Grafana Alerting for Open Source, see ({{< relref "../set-up/" >}})
To set up Alerting, you need to:
@@ -47,11 +47,15 @@ To set up Alerting, you need to:
- [Optional] Add additional nested policies
- [Optional] Add labels and label matchers to control alert routing
4. [Optional] Integrate with [Grafana OnCall](/docs/oncall/latest/integrations/grafana-alerting/) and [Grafana Incident](/docs/grafana-cloud/incident/set-up/)
4. [Optional] Integrate with [Grafana OnCall](/docs/oncall/latest/integrations/grafana-alerting) and [Grafana Incident](/docs/grafana-cloud/incident/set-up)
## Advanced set up options
Grafana Alerting supports many additional configuration options, from configuring external Alertmanagers to routing Grafana-managed alerts outside of Grafana, to defining your alerting setup as code.
The following topics provide you with advanced configuration options for Grafana Alerting for Cloud.
## Provision alert rules using mimirtool
### Provision alert rules using mimirtool
Use `mimirtool` to create and upload alert and recording rules to your Grafana Cloud instance.
@@ -71,7 +75,7 @@ The following sections cover all of these concepts:
**Note:** You need an API key with proper permissions. You can use the same API key for your Metric, Log, and Alerting instances.
### Download and install mimirtool
#### Download and install mimirtool
`mimirtool` is a powerful command-line tool for interacting with Grafana Mimir, which powers Grafana Cloud Metrics and Alerts. Use `mimirtool` to upload your metric and log rules definition and the Alertmanager configuration using YAML files.
@@ -81,7 +85,7 @@ For more information, including installation instructions, see [Grafana Mimirtoo
For `mimirtool` to interact with Grafana Cloud, you must set the correct configuration variables. Set them using either environment variables or a command line flags.
{{% /admonition %}}
### Upload rules definition to your Grafana Cloud Metrics and Logs instance
#### Upload rules definition to your Grafana Cloud Metrics and Logs instance
First, you'll need to upload your alerting and recording rules to your Metrics and Logs instance. You'll need the instance ID and the URL. These should be part of /orgs/`<yourOrgName>`/.
@@ -93,7 +97,7 @@ Your Metrics instance is likely to be in the `us-central1` region. Its address w
Your Logs instance is likely to be in the `us-central1` region. Its address would be in the form of [https://logs-prod-us-central1.grafana.net](https://logs-prod-us-central1.grafana.net).
### Use mimirtool
#### Use mimirtool
With your instance ID, URL, and API key you're now ready to upload your rules to your metrics instance. Use the following commands and files as a reference.
@@ -170,13 +174,13 @@ first_rules:
expr: sum by(job) (up)
```
## Add an external Alertmanager using mimirtool
### Add an external Alertmanager using mimirtool
To receive alerts you need to upload your Alertmanager configuration to your Grafana Cloud Alerts instance. Similar to the previous step, you need the corresponding instance ID, URL and API key. These should be part of /orgs/​`<yourOrgName>`/.
Your Alerts instance is likely to be in the `us-central1` region. Its address would be in the form of [https://alertmanager-us-central1.grafana.net](https://alertmanager-us-central1.grafana.net).
### Use mimirtool
#### Use mimirtool
With your instance ID, URL, and API key you're now ready to upload your Alertmanager configuration to your Alerts instance. Use the following commands and files as a reference.
@@ -238,6 +242,10 @@ $ mimirtool alertmanager delete \
--key=<yourKey>
```
### UI access
#### UI access
After you upload a working Alertmanager configuration file, you can access the Alertmanager UI at: https://alertmanager-us-central1.grafana.net/alertmanager.
### Provision alert rules using Terraform
For information on how to provision alert rule using Terraform, see [Provision alert rules using Terraform]({{< relref "../provision-alerting-resources/terraform-provisioning" >}}).