DOCS: Prometheus data sources documentation overhaul (#104427)

* initial updates

* additional updates - index and config dos

* made edits, renamed the index files.

* Close unclosed shortcode

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* updated main and config docs

* edits to query editor and templates

* final edits to landing page

* added updates

* made a few additional edits

* edits

* final edits and changes per dev feedback

* reviewed titles and headings; ran prettier

* added ref URIs

* Update docs/sources/datasources/prometheus/template-variables/_index.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update docs/sources/datasources/prometheus/configure/_index.md

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>

* Update docs/sources/datasources/prometheus/template-variables/_index.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* updates based on feedback

* ran prettier

* updates due to feedback

---------

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
This commit is contained in:
Larissa Wandzura
2025-05-06 14:02:52 +00:00
committed by GitHub
co-authored by Copilot ismail simsek Jack Baldry
parent b78044511f
commit fa93b3b74c
6 changed files with 645 additions and 600 deletions
@@ -13,7 +13,7 @@ labels:
- cloud
- enterprise
- oss
menuTitle: Template variables
menuTitle: Prometheus template variables
title: Prometheus template variables
weight: 400
refs:
@@ -42,13 +42,13 @@ refs:
# Prometheus template variables
Instead of hard-coding details such as server, application, and sensor names in metric queries, you can use variables. Grafana refers to such variables as **template** variables.
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the data displayed in your dashboard.
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the displayed data.
For an introduction to templating and template variables, see [Templating](ref:variables) and [Add and manage variables](ref:add-template-variables).
For an introduction to templating and template variables, refer to [Templating](ref:variables) and [Add and manage variables](ref:add-template-variables).
## Use query variables
You have the option to use several different variable types, but variables of the type `Query` will query Prometheus for a list of metrics, labels, label values, a query result or a series.
Grafana supports several types of variables, but Query variables are specifically used to query Prometheus. They can return a list of metrics, labels, label values, query results, or series.
Select a Prometheus data source query type and enter the required inputs:
@@ -61,18 +61,18 @@ Select a Prometheus data source query type and enter the required inputs:
| `Series query` | `metric`, `label` or both | Returns a list of time series associated with the entered data. | /api/v1/series |
| `Classic query` | classic query string | Deprecated, classic version of variable query editor. Enter a string with the query type using a syntax like the following: `label_values(<metric>, <label>)` | all |
For details on _metric names_, _label names_, and _label values_, refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
For details on `metric names`, `label names`, and `label values`, refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
### Query options
Under the query variable type, you can set the following query options:
With the query variable type, you can set the following query options:
| Option | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------- |
| **Data source** | Select your data source from the dropdown list. |
| **Data source** | Select your data source from the drop-down list. |
| **Select query type** | Options are `default`, `value` and `metric name`. Each query type hits a different Prometheus endpoint. |
| **Regex** | Optional, if you want to extract part of a series name or metric node segment. |
| **Sort** | Default is `disabled`. Options include `alphabetical`, `numerical` and `alphabetical case-sensitive`. |
| **Sort** | Default is `disabled`. Options include `alphabetical`, `numerical`, and `alphabetical case-sensitive`. |
| **Refresh** | When to update the values for the variable. Options are `On dashboard load` and `On time range change`. |
### Selection options
@@ -85,34 +85,41 @@ The following selection options are available:
### Use interval and range variables
You can use some global built-in variables in query variables, for example, `$__interval`, `$__interval_ms`, `$__range`, `$__range_s` and `$__range_ms`.
For details, see [Global built-in variables](ref:add-template-variables-global-variables).
You can use global built-in variables in query variables, including the following:
- `$__interval`
- `$__interval_ms`
- `$__range`
- `$__range_s`
- `$__range_ms`
For details, refer to [Global built-in variables](ref:add-template-variables-global-variables).
The `label_values` function doesn't support queries, so you can use these variables in conjunction with the `query_result` function to filter variable queries.
Make sure to set the variable's `refresh` trigger to be `On Time Range Change` to get the correct instances when changing the time range on the dashboard.
Configure the variable’s `refresh` setting to `On Time Range Change` to ensure it dynamically queries and displays the correct instances when the dashboard time range is modified.
**Example:**
Populate a variable with the busiest 5 request instances based on average QPS over the time range shown in the dashboard:
Populate a variable with the top 5 busiest request instances ranked by average QPS over the dashboard's selected time range:
```
Query: query_result(topk(5, sum(rate(http_requests_total[$__range])) by (instance)))
query_result(topk(5, sum(rate(http_requests_total[$__range])) by (instance)))
Regex: /"([^"]+)"/
```
Populate a variable with the instances having a certain state over the time range shown in the dashboard, using `$__range_s`:
```
Query: query_result(max_over_time(<metric>[${__range_s}s]) != <state>)
query_result(max_over_time(<metric>[${__range_s}s]) != <state>)
Regex:
```
## Use `$__rate_interval`
We recommend using `$__rate_interval` in the `rate` and `increase` functions instead of `$__interval` or a fixed interval value.
Because `$__rate_interval` is always at least four times the value of the Scrape interval, it avoid problems specific to Prometheus.
Grafana recommends using `$__rate_interval` with the `rate` and `increase` functions instead of `$__interval` or a fixed interval value.
Since `$__rate_interval` is always at least four times the scrape interval, it helps avoid issues specific to Prometheus, such as gaps or inaccuracies in query results.
For example, instead of using:
For example, instead of using the following:
```
rate(http_requests_total[5m])
@@ -124,20 +131,28 @@ or:
rate(http_requests_total[$__interval])
```
We recommend that you use:
Use the following:
```
rate(http_requests_total[$__rate_interval])
```
The value of `$__rate_interval` is defined as
<!-- The value of `$__rate_interval` is defined as
*max(`$__interval` + *Scrape interval*, 4 \* *Scrape interval*)*,
where _Scrape interval_ is the "Min step" setting (also known as `query*interval`, a setting per PromQL query) if any is set.
Otherwise, Grafana uses the Prometheus data source's "Scrape interval" setting.
Otherwise, Grafana uses the Prometheus data source's `scrape interval` setting. -->
The "Min interval" setting in the panel is modified by the resolution setting, and therefore doesn't have any effect on _Scrape interval_.
The value of `$__rate_interval` is calculated as:
For details, refer to the [Grafana blog](/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/).
```
max($__interval + scrape_interval, 4 * scrape_interval)
```
Here, `scrape_interval` refers to the `min step` setting (also known as `query_interval`) specified per PromQL query, if set. If not, Grafana falls back to the Prometheus data source’s scrape interval setting.
The `min interval` setting in the panel is modified by the resolution setting, and therefore doesn't have any effect on `scrape interval`.
For details, refer to the Grafana blog [$\_\_rate_interval for Prometheus rate queries that just work](https://grafana.com/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/).
## Choose a variable syntax
@@ -146,9 +161,8 @@ The Prometheus data source supports two variable syntaxes for use in the **Query
- `$<varname>`, for example `rate(http_requests_total{job=~"$job"}[$_rate_interval])`, which is easier to read and write but does not allow you to use a variable in the middle of a word.
- `[[varname]]`, for example `rate(http_requests_total{job=~"[[job]]"}[$_rate_interval])`
If you've enabled the _Multi-value_ or _Include all value_ options, Grafana converts the labels from plain text to a regex-compatible string, which requires you to use `=~` instead of `=`.
If you've enabled the `Multi-value` or `Include all value` options, Grafana converts the labels from plain text to a regex-compatible string, which requires you to use `=~` instead of `=`.
## Use the ad hoc filters variable type
Prometheus supports the special [ad hoc filters](ref:add-template-variables-add-ad-hoc-filters) variable type, which you can use to specify any number of label/value filters on the fly.
These filters are automatically applied to all your Prometheus queries.
Prometheus supports the special [ad hoc filters](ref:add-template-variables-add-ad-hoc-filters) variable type, which allows you to dynamically apply label/value filters across your dashboards. These filters are automatically added to all Prometheus queries, allowing dynamic filtering without modifying individual queries.