public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: Refactor alerting documentation (part 1) (#48664)

Browse Source 
* Initial commit

* Moved files, ad fixed broken relrefs.

* Fixed other broken relrefs

* More changes.

* Fixing broken relrefs

* More changes.

* Fixed last of the broken links

* More re-org.

* Added aliases and some weight adjustments

* More aliases.

* Fix fundamentals topic.

* Fixed remaining metadata issues

* Ran prettier
This commit is contained in:
achatterjee-grafana
2022-05-04 08:22:11 -04:00
committed by GitHub
parent f00ffb190c
commit 082cfbdb06
49 changed files with 148 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/alerting/alerting-rules/_index.md
+20
View File
@@ -0,0 +1,20 @@
+++
title = "Create and manage rules"
aliases = ["/docs/grafana/latest/alerting/rules/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/"]
weight = 130
+++
# Create and manage Grafana alerting rules
An alerting rule is a set of evaluation criteria that determines whether an alert will fire. The rule consists of one or more queries and expressions, a condition, the frequency of evaluation, and optionally, the duration over which the condition is met.
While queries and expressions select the data set to evaluate, a condition sets the threshold that an alert must meet or exceed to create an alert. An interval specifies how frequently an alerting rule is evaluated. Duration, when configured, indicates how long a condition must be met. The rules can also define alerting behavior in the absence of data.
You can:
- [Create Grafana Mimir or Loki managed alert rule]({{< relref "./create-mimir-loki-managed-rule.md" >}})
- [Create Grafana Mimir or Loki managed recording rule]({{< relref "./create-mimir-loki-managed-recording-rule.md" >}})
- [Edit Grafana Mimir or Loki rule groups and namespaces]({{< relref "./edit-mimir-loki-namespace-group.md" >}})
- [Create Grafana managed alert rule]({{< relref "./create-grafana-managed-rule.md" >}})
- [State and health of alerting rules]({{< relref "../fundamentals/state-and-health.md" >}})
- [Manage alerting rules]({{< relref "./rule-list.md" >}})
docs/sources/alerting/alerting-rules/alert-annotation-label.md
+39
View File
@@ -0,0 +1,39 @@
+++
title = "Annotations and labels for alerting rules"
description = "Annotations and labels for alerting"
keywords = ["grafana", "alerting", "guide", "rules", "create"]
weight = 401
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/alert-annotation-label/"]
+++
# Annotations and labels for alerting rules
Annotations and labels are key value pairs associated with alerts originating from the alerting rule, datasource response, and as a result of alerting rule evaluation. They can be used in alert notifications directly or in [templates]({{< relref "../message-templating/" >}}) and [template functions]({{< relref "../message-templating/template-functions" >}}) to create notification contact dynamically.
## Annotations
Annotations are key-value pairs that provide additional meta-information about an alert. You can use the following annotations: `description`, `summary`, `runbook_url`, `alertId`, `dashboardUid`, and `panelId`. For example, a description, a summary, and a runbook URL. These are displayed in rule and alert details in the UI and can be used in contact point message templates.
## Labels
Labels are key-value pairs that contain information about, and are used to uniquely identify an alert. The label set for an alert is generated and added to throughout the alerting evaluation and notification process.
### How are labels used?
- The complete set of labels for an alert is what uniquely identifies an alert within Grafana Alerts.
- The Alertmanager uses labels to match alerts for [silences]({{< relref "../silences/" >}}) and [alert groups]({{< relref "../alert-groups/" >}}) in [notification policies]({{< relref "../notifications/_index.md" >}}).
- The alerting UI displays labels for every alert instance generated by the evaluation of that rule.
- Contact points can access labels to dynamically generate notifications that contain information specific to the alert that is resulting in a notification.
- Labels can be added to an [alerting rule]({{< relref "../alerting-rules/" >}}). These manually configured labels are able to use template functions and reference other labels. Labels added to an alerting rule here take precedence in the event of a collision between labels.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-details-8-0.png" max-width="550px" caption="Alert details" >}}
#### Variables available to alerting rule labels and annotations
The following template variables are available when expanding annotations and labels.
| Name | Description |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| $labels | The labels from the query or condition. For example, `{{ $labels.instance }}` and `{{ $labels.job }}`. This is unavailable when the rule uses a [classic condition]({{< relref "./create-grafana-managed-rule/#single-and-multi-dimensional-rule" >}}). |
| $values | The values of all reduce and math expressions that were evaluated for this alert rule. For example, `{{ $values.A }}`, `{{ $values.A.Labels }}` and `{{ $values.A.Value }}` where `A` is the `refID` of the expression. If the rule uses classic conditions, then a combination of the `refID` and position of the condition is used. For example, `{{ $values.A0.Value }}` or `{{ $values.A1.Value }}` |
| $value | The value string of the alert instance. For example, `[ var='A' labels={instance=foo} value=10 ]`. |
docs/sources/alerting/alerting-rules/create-grafana-managed-rule.md
+77
View File
@@ -0,0 +1,77 @@
+++
title = "Create Grafana managed alert rule"
description = "Create Grafana managed alert rule"
keywords = ["grafana", "alerting", "guide", "rules", "create"]
weight = 400
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-grafana-managed-rule/"]
+++
# Create a Grafana managed alerting rule
Grafana allows you to create alerting rules that query one or more data sources, reduce or transform the results and compare them to each other or to fix thresholds. When these are executed, Grafana sends notifications to the contact point.
## Add Grafana managed rule
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Click **New alert rule**.
1. In Step 1, add the rule name, type, and storage location.
- In **Rule name**, add a descriptive name. This name is displayed in the alert rule list. It is also the `alertname` label for every alert instance that is created from this rule.
- From the **Rule type** drop-down, select **Grafana managed alert**.
- From the **Folder** drop-down, select the folder where you want to store the rule. If you do not select a folder, the rule is stored in the General folder. To create a new folder, click the drop-down and enter the new folder name.
1. In Step 2, add queries and expressions to evaluate.
- Keep the default name or hover over and click the edit icon to change the name.
- For queries, select a data source from the drop-down.
- Add one or more [queries]({{< relref "../../panels/query-a-data-source/add-a-query.md" >}}) or [expressions]({{< relref "../../panels/query-a-data-source/use-expressions-to-manipulate-data/about-expressions.md" >}}).
- For each expression, select either **Classic condition** to create a single alert rule, or choose from **Math**, **Reduce**, **Resample** options to generate separate alert for each series. For details on these options, see [Single and multi dimensional rule](#single-and-multi-dimensional-rule).
- Click **Run queries** to verify that the query is successful.
1. In Step 3, add conditions.
- From the **Condition** drop-down, select the query or expression to trigger the alert rule.
- For **Evaluate every**, specify the frequency of evaluation. Must be a multiple of 10 seconds. For examples, `1m`, `30s`.
- For **Evaluate for**, specify the duration for which the condition must be true before an alert fires.
> **Note:** Once a condition is breached, the alert goes into the Pending state. If the condition remains breached for the duration specified, the alert transitions to the Firing state, else it reverts back to the Normal state.
- In **Configure no data and error handling**, configure alerting behavior in the absence of data. Use the guidelines in [No data and error handling](#no-data-and-error-handling).
- Click **Preview alerts** to check the result of running the query at this moment. Preview excludes no data and error handling.
1. In Step 4, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. Click **Save** to save the rule or **Save and exit** to save the rule and go back to the Alerting page.
### Single and multi dimensional rule
For Grafana managed alerts, you can create a rule with a classic condition or you can create a multi-dimensional rule.
**Rule with classic condition**
Use the classic condition expression to create a rule that triggers a single alert when its condition is met. For a query that returns multiple series, Grafana does not track the alert state of each series. As a result, Grafana sends only a single alert even when alert conditions are met for multiple series.
**Multi dimensional rule**
To generate a separate alert for each series, create a multi-dimensional rule. Use `Math`, `Reduce`, or `Resample` expressions to create a multi-dimensional rule. For example:
- Add a `Reduce` expression for each query to aggregate values in the selected time range into a single value. (Not needed for [rules using numeric data]({{< relref "../fundamentals/evaluate-grafana-alerts.md#alerting-on-numeric-data-1" >}})).
- Add a `Math` expression with the condition for the rule. Not needed in case a query or a reduce expression already returns 0 if rule should not fire, or a positive number if it should fire. Some examples: `$B > 70` if it should fire in case value of B query/expression is more than 70. `$B < $C * 100` in case it should fire if value of B is less than value of C multiplied by 100. If queries being compared have multiple series in their results, series from different queries are matched if they have the same labels or one is a subset of the other.
![Query section multi dimensional](/static/img/docs/alerting/unified/rule-edit-multi-8-0.png 'Query section multi dimensional screenshot')
> **Note:** Grafana does not support alert queries with template variables. More information is available at <https://community.grafana.com/t/template-variables-are-not-supported-in-alert-queries-while-setting-up-alert/2514>.
#### Rule with classic condition
For more information, see [expressions documentation]({{< relref "../../panels/query-a-data-source/use-expressions-to-manipulate-data/about-expressions.md" >}}).
### No data and error handling
Configure alerting behavior in the absence of data using information in the following tables.
| No Data Option | Description |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| No Data | Create a new alert `DatasourceNoData` with the name and UID of the alert rule, and UID of the datasource that returned no data as labels. |
| Alerting | Set alert rule state to `Alerting`. |
| Ok | Set alert rule state to `Normal`. |
| Error or timeout option | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| Alerting | Set alert rule state to `Alerting`. From Grafana 8.5, the alert rule waits for the entire duration for which the condition is true before firing. |
| OK | Set alert rule state to `Normal` |
| Error | Create a new alert `DatasourceError` with the name and UID of the alert rule, and UID of the datasource that returned no data as labels. |
docs/sources/alerting/alerting-rules/create-mimir-loki-managed-recording-rule.md
+42
View File
@@ -0,0 +1,42 @@
+++
title = "Create Grafana Mimir or Loki managed recording rule"
description = "Create Grafana Mimir or Loki managed recording rule"
keywords = ["grafana", "alerting", "guide", "rules", "recording rules", "create"]
weight = 400
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-mimir-loki-managed-recording-rule/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-cortex-loki-managed-recording-rule/"]
+++
# Create a Grafana Mimir or Loki managed recording rule
You can create and manage recording rules for an external Grafana Mimir or Loki instance. Recording rules calculate frequently needed expressions or computationally expensive expressions in advance and save the result as a new set of time series. Querying this new time series is faster, especially for dashboards since they query the same expression every time the dashboards refresh.
## Before you begin
For Grafana Mimir and Loki data sources to work with Grafana 8.0 alerting, enable the ruler API by configuring their respective services.
**Loki** - The `local` rule storage type, default for the Loki data source, supports only viewing of rules. To edit rules, configure one of the other rule storage types.
**Grafana Mimir** - When configuring a Grafana Prometheus data source to point to Grafana Mimir, use the legacy `/api/prom` prefix, not `/prometheus`. Currently, we support only single-binary mode and you cannot provide a separate URL for the ruler API.
> **Note:** If you do not want to manage alerting rules for a particular Loki or Prometheus data source, go to its settings page and clear the **Manage alerts via Alerting UI** checkbox.
## Add a Grafana Mimir or Loki managed recording rule
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Click **New alert rule**.
1. In Step 1, add the rule name, type, and storage location.
- In **Rule name**, add a descriptive name. This name is displayed in the alert rule list. It is also the `alertname` label for every alert instance that is created from this rule.
- From the **Rule type** drop-down, select **Mimir / Loki managed alert**.
- From the **Select data source** drop-down, select an external Prometheus, an external Loki, or a Grafana Cloud data source.
- From the **Namespace** drop-down, select an existing rule namespace. Otherwise, click **Add new** and enter a name to create a new one. Namespaces can contain one or more rule groups and only have an organizational purpose.
- From the **Group** drop-down, select an existing group within the selected namespace. Otherwise, click **Add new** and enter a name to create a new one. Newly created rules are appended to the end of the group. Rules within a group are run sequentially at a regular interval, with the same evaluation time.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-mimir-alert-type-8-0.png" max-width="550px" caption="Alert details" >}}
1. In Step 2, add the query to evaluate.
- Enter a PromQL or LogQL expression. The rule fires if the evaluation result has at least one series with a value that is greater than 0. An alert is created for each series.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-mimir-query-8-0.png" max-width="550px" caption="Alert details" >}}
1. In Step 3, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. Click **Save** to save the rule or **Save and exit** to save the rule and go back to the Alerting page.
docs/sources/alerting/alerting-rules/create-mimir-loki-managed-rule.md
+47
View File
@@ -0,0 +1,47 @@
+++
title = "Create Grafana Mimir or Loki managed alert rule"
description = "Create Grafana Mimir or Loki managed alerting rule"
keywords = ["grafana", "alerting", "guide", "rules", "create"]
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-cortex-loki-managed-recording-rule/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/create-mimir-loki-managed-recording-rule/"]
weight = 400
+++
# Create a Grafana Mimir or Loki managed alerting rule
Grafana allows you to create alerting rules for an external Grafana Mimir or Loki instance.
## Before you begin
- Verify that you have write permission to the Prometheus data source. Otherwise, you will not be able to create or update Grafana Mimir managed alerting rules.
- For Grafana Mimir and Loki data sources, enable the ruler API by configuring their respective services.
- **Loki** - The `local` rule storage type, default for the Loki data source, supports only viewing of rules. To edit rules, configure one of the other rule storage types.
- **Grafana Mimir** - use the [legacy `/api/prom` prefix](https://grafana.com/docs/mimir/latest/operators-guide/reference-http-api/#path-prefixes), not `/prometheus`. The Prometheus data source supports both Grafana Mimir and Prometheus, and Grafana expects that both the [Query API](https://grafana.com/docs/mimir/latest/operators-guide/reference-http-api/#querier--query-frontend) and [Ruler API](https://grafana.com/docs/mimir/latest/operators-guide/reference-http-api/#ruler) are under the same URL. You cannot provide a separate URL for the Ruler API.
> **Note:** If you do not want to manage alerting rules for a particular Loki or Prometheus data source, go to its settings and clear the **Manage alerts via Alerting UI** checkbox.
## Add a Grafana Mimir or Loki managed alerting rule
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Click **New alert rule**.
1. In Step 1, add the rule name, type, and storage location.
- In **Rule name**, add a descriptive name. This name is displayed in the alert rule list. It is also the `alertname` label for every alert instance that is created from this rule.
- From the **Rule type** drop-down, select **Mimir / Loki managed alert**.
- From the **Select data source** drop-down, select an external Prometheus, an external Loki, or a Grafana Cloud data source.
- From the **Namespace** drop-down, select an existing rule namespace. Otherwise, click **Add new** and enter a name to create a new one. Namespaces can contain one or more rule groups and only have an organizational purpose. For more information, see [Grafana Mimir or Loki rule groups and namespaces]({{< relref "./edit-mimir-loki-namespace-group.md" >}}).
- From the **Group** drop-down, select an existing group within the selected namespace. Otherwise, click **Add new** and enter a name to create a new one. Newly created rules are appended to the end of the group. Rules within a group are run sequentially at a regular interval, with the same evaluation time.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-mimir-alert-type-8-0.png" max-width="550px" caption="Alert details" >}}
1. In Step 2, add the query to evaluate.
- Enter a PromQL or LogQL expression. The rule fires if the evaluation result has at least one series with a value that is greater than 0. An alert is created for each series.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-mimir-query-8-0.png" max-width="550px" caption="Alert details" >}}
1. In Step 3, add conditions.
- In the **For** text box, specify the duration for which the condition must be true before an alert fires. If you specify `5m`, the condition must be true for 5 minutes before the alert fires.
> **Note:** Once a condition is met, the alert goes into the `Pending` state. If the condition remains active for the duration specified, the alert transitions to the `Firing` state, else it reverts to the `Normal` state.
1. In Step 4, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. To evaluate the rule and see what alerts it would produce, click **Preview alerts**. It will display a list of alerts with state and value of for each one.
1. Click **Save** to save the rule or **Save and exit** to save the rule and go back to the Alerting page.
docs/sources/alerting/alerting-rules/edit-mimir-loki-namespace-group.md
+40
View File
@@ -0,0 +1,40 @@
+++
title = "Grafana Mimir or Loki rule groups and namespaces"
description = "Edit Grafana Mimir or Loki rule groups and namespaces"
keywords = ["grafana", "alerting", "guide", "group", "namespace", "grafana mimir", "loki"]
weight = 405
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/edit-mimir-loki-namespace-group/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/edit-cortex-loki-namespace-group/"]
+++
# Grafana Mimir or Loki rule groups and namespaces
A namespace contains one or more groups. The rules within a group are run sequentially at a regular interval. The default interval is one (1) minute. You can rename Grafana Mimir or Loki rule namespaces and groups, and edit group evaluation intervals.
![Group list](/static/img/docs/alerting/unified/rule-list-edit-mimir-loki-icon-8-2.png 'Rule group list screenshot')
{{< figure src="/static/img/docs/alerting/unified/rule-list-edit-mimir-loki-icon-8-2.png" max-width="550px" caption="Alert details" >}}
## Rename a namespace
To rename a namespace:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Find a Grafana Mimir or Loki managed rule with the group that belongs to the namespace you want to edit.
1. Click the **Edit** (pen) icon.
1. Enter a new name in the **Namespace** field, then click **Save changes**.
A new namespace is created and all groups are copied into this namespace from the old one. The old namespace is deleted.
## Rename rule group or change the rule group evaluation interval
The rules within a group are run sequentially at a regular interval, the default interval is one (1) minute. You can modify this interval using the following instructions.
1. n the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Find a Grafana Mimir or Loki managed rule with the group you want to edit.
1. Click **Edit** (pen) icon.
1. Modify the **Rule group** and **Rule group evaluation interval** information as necessary.
1. Click **Save changes**.
When you rename the group, a new group with all the rules from the old group is created. The old group is deleted.
![Group edit modal](/static/img/docs/alerting/unified/rule-list-mimir-loki-edit-ns-group-8-2.png 'Rule group edit modal screenshot')
docs/sources/alerting/alerting-rules/rule-list.md
+59
View File
@@ -0,0 +1,59 @@
+++
title = "Manage alerting rules"
description = "Manage alerting rules"
keywords = ["grafana", "alerting", "guide", "rules", "view"]
weight = 402
aliases = ["/docs/grafana/latest/alerting/unified-alerting/alerting-rules/rule-list/"]
+++
# Manage alerting rules
The Alerting page lists existing Grafana 8 alerting rules. By default, rules are grouped by types of data sources. The Grafana section lists all Grafana managed rules. Alerting rules for Prometheus compatible data sources are also listed here. You can view alerting rules for Prometheus compatible data sources but you cannot edit them.
The Mimir/Loki rules section lists all rules for external Prometheus or Loki data sources. Cloud alerting rules are also listed in this section.
- [Manage alerting rules](#manage-alerting-rules)
- [View alerting rules](#view-alerting-rules)
- [Group view](#group-view)
- [State view](#state-view)
- [Filter alerting rules](#filter-alerting-rules)
- [Edit or delete an alerting rule](#edit-or-delete-an-alerting-rule)
## View alerting rules
To view alerting details:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page. By default, the group view displays.
1. In **View as**, toggle between group or state views by clicking the relevant option. See [Group view](#group-view) and [State view](#state-view) for more information.
1. Expand the rule row to view the rule labels, annotations, data sources the rule queries, and a list of alert instances resulting from this rule.
{{< figure src="/static/img/docs/alerting/unified/rule-details-8-0.png" max-width="650px" caption="Alerting rule details" >}}
### Group view
Group view shows Grafana alert rules grouped by folder and Loki or Prometheus alert rules grouped by `namespace` + `group`. This is the default rule list view, intended for managing rules. You can expand each group to view a list of rules in this group. Expand a rule further to view its details. You can also expand action buttons and alerts resulting from the rule to view their details.
{{< figure src="/static/img/docs/alerting/unified/rule-list-group-view-8-0.png" max-width="800px" caption="Alerting grouped view" >}}
### State view
State view shows alert rules grouped by state. Use this view to get an overview of which rules are in what state. Each rule can be expanded to view its details. Action buttons and any alerts generated by this rule, and each alert can be further expanded to view its details.
{{< figure src="/static/img/docs/alerting/unified/rule-list-state-view-8-0.png" max-width="800px" caption="Alerting state view" >}}
## Filter alerting rules
To filter alerting rules:
- From **Select data sources**, select a data source. You can see alerting rules that query the selected data source.
- In the **Search by label**, enter search criteria using label selectors. For example, `environment=production,region=~US|EU,severity!=warning`.
- From **Filter alerts by state**, select an alerting state you want to see. You can see alerting rules that match the state. Rules matching other states are hidden.
## Edit or delete an alerting rule
Grafana managed alerting rules can only be edited or deleted by users with Edit permissions for the folder storing the rules. Alerting rules for an external Grafana Mimir or Loki instance can be edited or deleted by users with Editor or Admin roles.
To edit or delete a rule:
1. Expand a rule row until you can see the rule controls of **View**, **Edit**, and **Delete**.
1. Click **Edit** to open the create rule page. Make updates following instructions in [Create a Grafana managed alerting rule]({{< relref "./create-grafana-managed-rule.md" >}}) or [Create a Grafana Mimir or Loki managed alerting rule]({{< relref "./create-mimir-loki-managed-rule.md" >}}).
1. Click **Delete** to delete a rule.