Docs: Refactor data sources content (#57573)
* Docs: Revise data source index * Docs: Consolidate data source administration docs * Docs: Revise panels docs related to data sources * Docs: Revise Alertmanager data source * Docs: Reorganize AWS CloudWatch data source docs * Docs: Reorganize Azure Monitor data source docs * Docs: Move azuremonitor to azure-monitor * Docs: Revise Elasticsearch docs * Docs: Move Elasticsearch index into bundle * Docs: Revise GCM docs * Docs: Revise Graphite docs * Docs: Move Graphite index into bundle * Docs: Revise InfluxDB docs * Docs: Revise Jaeger docs * Docs: Move Jaeger index into bundle * Docs: Revise Loki docs * Docs: Move Loki index into bundle * Docs: Revise MS SQL docs * Docs: Move MS SQL index into bundle * Docs: Revise Prometheus docs * Docs: Move Prometheus index into bundle * Docs: Revise Tempo docs * Docs: Move Tempo index into bundle * Docs: Revise TestData DB docs * Docs: Move TestData DB index into bundle * Docs: Revise Zipkin docs * Docs: Move Zipkin index into bundle * Docs: Move other data sources' index pages into bundles * Docs: Revise frontmatter * Fixing hugo markdown errors * Docs: Add query editor and template var sections to overview doc * Docs: Remove CTAs across data source docs * Docs: Remove CTA * Docs: Remove CTA * Docs: Fix links, images, typos, and usage consistency. * Docs: Fix typos * Docs: Fix CI issues * Update docs/sources/datasources/_index.md Co-authored-by: Torkel Ödegaard <torkel@grafana.com> * Update docs/sources/datasources/_index.md Co-authored-by: Torkel Ödegaard <torkel@grafana.com> * Docs: Fix query editor links * Update docs/sources/panels-visualizations/_index.md Co-authored-by: Torkel Ödegaard <torkel@grafana.com> * Update docs/sources/panels-visualizations/_index.md Co-authored-by: Torkel Ödegaard <torkel@grafana.com> * Docs: Rebundle child pages per writers' toolkit * Docs: Fix prettier for CI * Docs: Fix relrefs from outside data sources docs * Docs: Fix broken relrefs within datasources * Docs: Fix relrefs to data sources docs * Fixed some more refs Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
This commit is contained in:
Garrett Guillotte
co-authored by
Torkel Ödegaard
Torkel Ödegaard
parent
22648d8581
commit
852d069a3c
@@ -0,0 +1,164 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/features/datasources/prometheus/
|
||||
- /docs/grafana/latest/datasources/prometheus/
|
||||
- /docs/grafana/latest/data-sources/prometheus/
|
||||
description: Guide for using Prometheus in Grafana
|
||||
keywords:
|
||||
- grafana
|
||||
- prometheus
|
||||
- guide
|
||||
menuTitle: Prometheus
|
||||
title: Prometheus data source
|
||||
weight: 1300
|
||||
---
|
||||
|
||||
# Prometheus data source
|
||||
|
||||
Grafana ships with built-in support for Prometheus.
|
||||
This topic explains options, variables, querying, and other features specific to the Prometheus data source, which include its [feature-rich code editor for queries and visual query builder]({{< relref "./query-editor/" >}}).
|
||||
|
||||
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management/" >}}).
|
||||
Only users with the organization administrator role can add data sources.
|
||||
Administrators can also [configure the data source via YAML]({{< relref "#provision-the-data-source" >}}) with Grafana's provisioning system.
|
||||
|
||||
Once you've added the data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "./query-editor/" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}), use [Explore]({{< relref "../../explore/" >}}), and [annotate visualizations]({{< relref "./query-editor/#apply-annotations" >}}).
|
||||
|
||||
## Prometheus API
|
||||
|
||||
The Prometheus data source also works with other projects that implement the [Prometheus querying API](https://prometheus.io/docs/prometheus/latest/querying/api/).
|
||||
|
||||
For more information on how to query other Prometheus-compatible projects from Grafana, refer to the specific project's documentation:
|
||||
|
||||
- [Grafana Mimir](/docs/mimir/latest/)
|
||||
- [Thanos](https://thanos.io/tip/components/query.md/)
|
||||
|
||||
## Configure the data source
|
||||
|
||||
**To access the data source configuration page:**
|
||||
|
||||
1. Hover the cursor over the **Configuration** (gear) icon.
|
||||
1. Select **Data Sources**.
|
||||
1. Select the Prometheus data source.
|
||||
|
||||
Set the data source's basic configuration options carefully:
|
||||
|
||||
| Name | Description |
|
||||
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
|
||||
| **Default** | Sets whether the data source is pre-selected for new panels. |
|
||||
| **Url** | Sets the URL of your Prometheus server, such as `http://prometheus.example.org:9090`. |
|
||||
| **Access** | Only Server access mode is functional. If Server mode is already selected, this option is hidden. Otherwise, change this to Server mode to prevent errors. |
|
||||
| **Basic Auth** | Enables basic authentication to the Prometheus data source. |
|
||||
| **User** | Sets the user name for basic authentication. |
|
||||
| **Password** | Sets the password for basic authentication. |
|
||||
| **Scrape interval** | Sets the scrape and evaluation interval. We recommend the same value as the typical configured in Prometheus. Defaults to 15s. |
|
||||
| **Type** | Defines the type of your Prometheus server. Valid values are `Prometheus`, `Cortex`, `Thanos`, `Mimir`. When selected, the Prometheus version field attempts to detect the version automatically using the Prometheus [buildinfo](https://semver.org/) API. Some Prometheus types, such as Cortex, don't support this API, and you must provide their version. |
|
||||
| **Version** | Defines the version of your Prometheus server. This field is visible only after the **Type** field is defined. |
|
||||
| **HTTP method** | Sets the HTTP method used to query your data source. We recommend POST, which is pre-selected, because it allows for larger queries. Use GET if the Prometheus version is older than 2.1, or if POST requests are restricted in your network. |
|
||||
| **Disable metrics lookup** | Disables the metrics chooser and metric/label support in the query field's autocompletion. This can prevent performance issues with larger Prometheus instances. |
|
||||
| **Custom query parameters** | Adds custom parameters to the Prometheus query URL, such as `timeout`, `partial_response`, `dedup`, or `max_source_resolution`. Concatenate multiple parameters with '&'. |
|
||||
|
||||
**Exemplars configuration:**
|
||||
|
||||
| Name | Description |
|
||||
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Internal link** | Enable this option if you have an internal link. When enabled, this reveals the data source selector. Select the backend tracing data store for your exemplar data. |
|
||||
| **Data source** | _(Visible only if you enable `Internal link`)_ Selects the backend tracing data store for your exemplar data. |
|
||||
| **URL** | _(Visible only if you disable `Internal link`)_ Defines the external link's full URL. You can interpolate the value from the field by using the [`${__value.raw}` macro]({{< relref "../..//panels-visualizations/configure-data-links/#value-variables" >}}). |
|
||||
| **URL label** | _(Optional)_ Adds a custom display label to override the value of the `Label name` field. |
|
||||
| **Label name** | Adds a name for the exemplar traceID property. |
|
||||
|
||||
### Provision the data source
|
||||
|
||||
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
|
||||
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
|
||||
|
||||
#### Provisioning example
|
||||
|
||||
```yaml
|
||||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: Prometheus
|
||||
type: prometheus
|
||||
# Access mode - proxy (server in the UI) or direct (browser in the UI).
|
||||
access: proxy
|
||||
url: http://localhost:9090
|
||||
jsonData:
|
||||
httpMethod: POST
|
||||
exemplarTraceIdDestinations:
|
||||
# Field with internal link pointing to data source in Grafana.
|
||||
# datasourceUid value can be anything, but it should be unique across all defined data source uids.
|
||||
- datasourceUid: my_jaeger_uid
|
||||
name: traceID
|
||||
|
||||
# Field with external link.
|
||||
- name: traceID
|
||||
url: 'http://localhost:3000/explore?orgId=1&left=%5B%22now-1h%22,%22now%22,%22Jaeger%22,%7B%22query%22:%22$${__value.raw}%22%7D%5D'
|
||||
```
|
||||
|
||||
## Query the data source
|
||||
|
||||
The Prometheus query editor includes a code editor and visual query builder.
|
||||
|
||||
For details, see the [query editor documentation]({{< relref "./query-editor" >}}).
|
||||
|
||||
## View Grafana metrics with Prometheus
|
||||
|
||||
Grafana exposes metrics for Prometheus on the `/metrics` endpoint.
|
||||
We also bundle a dashboard within Grafana so you can start viewing your metrics faster.
|
||||
|
||||
**To import the bundled dashboard:**
|
||||
|
||||
1. Navigate to the data source's [configuration page]({{< relref "#configure-the-data-source" >}}).
|
||||
1. Select the **Dashboards** tab.
|
||||
|
||||
This displays dashboards for Grafana and Prometheus.
|
||||
|
||||
1. Select **Import** for the dashboard to import.
|
||||
|
||||
For details about these metrics, refer to [Internal Grafana metrics]({{< relref "../../setup-grafana/set-up-grafana-monitoring/" >}}).
|
||||
|
||||
### Amazon Managed Service for Prometheus
|
||||
|
||||
The Prometheus data source works with Amazon Managed Service for Prometheus.
|
||||
|
||||
If you use an AWS Identity and Access Management (IAM) policy to control access to your Amazon Elasticsearch Service domain, you must use AWS Signature Version 4 (AWS SigV4) to sign all requests to that domain.
|
||||
|
||||
For details on AWS SigV4, refer to the [AWS documentation](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html).
|
||||
|
||||
#### AWS Signature Version 4 authentication
|
||||
|
||||
> **Note:** Available in Grafana v7.3.5 and higher.
|
||||
|
||||
To connect the Prometheus data source to Amazon Managed Service for Prometheus using SigV4 authentication, refer to the AWS guide to [Set up Grafana open source or Grafana Enterprise for use with AMP](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-query-standalone-grafana.html).
|
||||
|
||||
If you run Grafana in an Amazon EKS cluster, follow the AWS guide to [Query using Grafana running in an Amazon EKS cluster](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-query-grafana-7.3.html).
|
||||
|
||||
### Configure exemplars
|
||||
|
||||
> **Note:** Available in Prometheus v2.26 and higher with Grafana v7.4 and higher.
|
||||
|
||||
Grafana 7.4 and higher can show exemplars data alongside a metric both in Explore and in Dashboards.
|
||||
Exemplars associate higher-cardinality metadata from a specific event with traditional time series data.
|
||||
|
||||
{{< figure src="/static/img/docs/v74/exemplars.png" class="docs-image--no-shadow" caption="Screenshot showing the detail window of an Exemplar" >}}
|
||||
|
||||
Configure Exemplars in the data source settings by adding external or internal links.
|
||||
|
||||
{{< figure src="/static/img/docs/v74/exemplars-setting.png" class="docs-image--no-shadow" caption="Screenshot of the Exemplars configuration" >}}
|
||||
|
||||
## Query the data source
|
||||
|
||||
You can create queries with the Prometheus data source's query editor.
|
||||
|
||||
For details, refer to the [query editor documentation]({{< relref "./query-editor/" >}}).
|
||||
|
||||
## Use template variables
|
||||
|
||||
Instead of hard-coding details such as server, application, and sensor names in metric queries, you can use variables.
|
||||
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the data displayed in your dashboard.
|
||||
Grafana refers to such variables as template variables.
|
||||
|
||||
For details, see the [template variables documentation]({{< relref "./template-variables/" >}}).
|
||||
@@ -0,0 +1,228 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/data-sources/prometheus/query-editor/
|
||||
description: Guide for using the Prometheus data source's query editor
|
||||
keywords:
|
||||
- grafana
|
||||
- prometheus
|
||||
- logs
|
||||
- queries
|
||||
menuTitle: Query editor
|
||||
title: Prometheus query editor
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# Prometheus query editor
|
||||
|
||||
This topic explains querying specific to the Prometheus data source.
|
||||
For general documentation on querying data sources in Grafana, see [Query and transform data]({{< relref "../../../panels-visualizations/query-transform-data" >}}).
|
||||
|
||||
## Choose a query editing mode
|
||||
|
||||
You can switch the Prometheus query editor between two modes:
|
||||
|
||||
- [Code mode](#code-mode), which provides a feature-rich editor for writing queries
|
||||
- [Builder mode](#builder-mode), which provides a visual query designer
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/header-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Editor toolbar" >}}
|
||||
|
||||
To switch between the editor modes, select the corresponding **Builder** and **Code** tabs.
|
||||
|
||||
To run a query, select **Run queries** located at the top of the editor.
|
||||
|
||||
> **Note:** To run Prometheus queries in [Explore]({{< relref "../../../explore" >}}), select **Run query**.
|
||||
|
||||
Each mode is synchronized with the other modes, so you can switch between them without losing your work, although there are some limitations.
|
||||
|
||||
Builder mode doesn't yet support some complex queries.
|
||||
When you switch from Code mode to Builder mode with such a query, the editor displays a popup that explains how you might lose parts of the query if you continue.
|
||||
You can then decide whether you still want to switch to Builder mode.
|
||||
|
||||
You can also use the [Explain feature]({{< relref "#use-explain-mode-to-understand-queries" >}}) to help understand how a query works, and augment queries by using [template variables]({{< relref "./template-variables/" >}}).
|
||||
|
||||
For options and functions common to all query editors, refer to [Query and transform data]({{< relref "../../../panels-visualizations/query-transform-data" >}}).
|
||||
|
||||
## Configure common options
|
||||
|
||||
You can configure Prometheus-specific options in the query editor by setting several options regardless of its mode.
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/options-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Options" >}}
|
||||
|
||||
### Legend
|
||||
|
||||
The **Legend** setting defines the time series's name. You can use a predefined or custom format.
|
||||
|
||||
| Option | Description |
|
||||
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Auto** | Shows the value of a single label for each series with only one label, or displays all labels if a series has multiple labels. |
|
||||
| **Verbose** | Displays all labels. |
|
||||
| **Custom** | Uses templating to select which labels will be included.<br/>For example, `{{hostname}}` is replaced by the label value for the label `hostname`.<br/>Clear the input and click outside of it to select another mode. |
|
||||
|
||||
### Min step
|
||||
|
||||
The **Min step** setting defines the lower bounds on the interval between data points.
|
||||
For example, set this to `1h` to hint that measurements are taken hourly.
|
||||
This setting supports the `$__interval` and `$__rate_interval` macros.
|
||||
|
||||
### Format
|
||||
|
||||
You can switch between **Table**, **Time series**, and **Heatmap** options by configuring the query's **Format**.
|
||||
|
||||
| Option | Description |
|
||||
| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Table** | This works only in a [Table panel]{{< relref "../../../panels-visualizations/visualizations/table" >}}. |
|
||||
| **Time series** | Uses the default time series format. |
|
||||
| **Heatmap** | Displays metrics of the Histogram type on a [Heatmap panel]{{< relref "../../../panels-visualizations/visualizations/heatmap" >}} by converting cumulative histograms to regular ones and sorting the series by the bucket bound. |
|
||||
|
||||
### Type
|
||||
|
||||
The **Type** setting selects the query type.
|
||||
|
||||
- A **Range** query returns a Range vector, comprised of a set of time series containing a range of data points over time for each time series.
|
||||
- An **Instant** query returns only the latest value that Prometheus has scraped for the requested time series. Instant queries can return results much faster than normal range queries and are well suited to look up label sets.
|
||||
|
||||
Instant query results are comprised of only one data point per series and can be shown in the time series panel by adding a field override, adding a property to the override named `Transform`, and selecting `Constant` from the **Transform** dropdown.
|
||||
|
||||
For more information, refer to the [Time Series Transform option documentation]({{< relref "../../../panels-visualizations/visualizations/time-series#transform" >}}).
|
||||
|
||||
- An **Exemplars** query runs with the regular query and shows exemplars in the graph.
|
||||
|
||||
> **Note:** Grafana modifies the request dates for queries to align them with the dynamically calculated step.
|
||||
> This ensures a consistent display of metrics data, but it can result in a small gap of data at the right edge of a graph.
|
||||
|
||||
## Code mode
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/code-mode-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Code mode" >}}
|
||||
|
||||
In **Code mode**, you can write complex queries using a text editor with autocompletion features and syntax highlighting.
|
||||
It also contains a [Metrics browser]({{< relref "#metrics-browser" >}}) to further help you write queries.
|
||||
|
||||
For more information about Prometheus's query language, refer to the [Prometheus documentation](http://prometheus.io/docs/querying/basics/).
|
||||
|
||||
### Use autocompletion
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/autocomplete-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Autocomplete" >}}
|
||||
|
||||
Code mode's autocompletion feature works automatically while typing.
|
||||
To manually trigger autocompletion, use the keyboard shortcut <key>Ctrl</key>/<key>Cmd</key> + <key>Space</key>.
|
||||
|
||||
The query editor can autocomplete static functions, aggregations, and keywords, and also dynamic items like metrics and labels.
|
||||
The autocompletion dropdown includes documentation for the suggested items where available.
|
||||
|
||||
To run a query in [Explore]({{< relref "../../../explore/" >}}), use the keyboard shortcut <key>Shift</key> + <key>Enter</key>.
|
||||
|
||||
### Metrics browser
|
||||
|
||||
The metrics browser locates metrics and selects relevant labels to help you build basic queries.
|
||||
When you open the browser, it displays all available metrics and labels.
|
||||
If supported by your Prometheus instance, each metric also displays its HELP and TYPE as a tooltip.
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/metric-browser-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Metrics browser" >}}
|
||||
|
||||
When you select a metric, the browser narrows down the available labels to show only the ones applicable to the metric.
|
||||
You can then select one or more labels for which the available label values are shown in lists in the bottom section.
|
||||
Select one or more values for each label to tighten your query scope.
|
||||
|
||||
> **Note:** If you do not remember a metric name to start with, you can also select a few labels to narrow down the list, then find relevant label values.
|
||||
|
||||
All lists in the metrics browser have a search field above them to quickly filter for metrics or labels that match a certain string.
|
||||
The values section has only one search field, and its filtering applies to all labels to help you find values across labels once selected.
|
||||
|
||||
For example, among your labels `app`, `job`, `job_name` only one might with the value you are looking for.
|
||||
|
||||
Once you are satisfied with your query, click "Use query" to run the query. The button "Use as rate query" adds a `rate(...)[$__interval]` around your query to help write queries for counter metrics.
|
||||
The "Validate selector" button will check with Prometheus how many time series are available for that selector.
|
||||
|
||||
## Builder mode
|
||||
|
||||
In **Builder mode**, you can build queries using a visual interface.
|
||||
|
||||
This video demonstrates how to use the visual Prometheus query builder available since Grafana v9.0:
|
||||
|
||||
{{< vimeo 720004179 >}}
|
||||
|
||||
</br>
|
||||
|
||||
### Toolbar
|
||||
|
||||
In addition to the **Run query** button and mode switcher, Builder mode includes additional elements:
|
||||
|
||||
| Name | Description |
|
||||
| ------------------ | ----------------------------------------------------------------------------------------- |
|
||||
| **Query patterns** | A list of operation patterns that help you quickly add multiple operations to your query. |
|
||||
| **Explain** | Displays a step-by-step explanation of all query parts and its operations. |
|
||||
| **Raw query** | Displays the raw query generated by the Builder that will be sent to Prometheus instance. |
|
||||
|
||||
### Metric and labels
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/metric-select-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Metric and labels" >}}
|
||||
|
||||
Select a specific metric name from the dropdown list.
|
||||
The data source requests the list of available metrics from the Prometheus server based on the selected time rage.
|
||||
You can also enter text into the selector when the dropdown is open to search and filter the list.
|
||||
|
||||
Select desired labels and their values from the dropdown list.
|
||||
When a metric is selected, the data source requests available labels and their values from the server.
|
||||
Use the `+` button to add a label, and the `x` button to remove a label.
|
||||
|
||||
### Operations
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/operations-9-1.png" max-width="500px" class="docs-image--no-shadow" caption="Operations" >}}
|
||||
|
||||
Select the `+ Operations` button to add operations to your query.
|
||||
The query editor groups operations into related sections, and you can type while the operations dropdown is open to search and filter the list.
|
||||
|
||||
The query editor displays a query's operations as boxes in the operations section.
|
||||
Each operation's header displays its name, and additional action buttons appear when you hover your cursor over the header:
|
||||
|
||||
| Button | Action |
|
||||
| ------ | ----------------------------------------------------------------- |
|
||||
| `v` | Replaces the operation with different operation of the same type. |
|
||||
| `info` | Opens the operation's description tooltip. |
|
||||
| `x` | Removes the operation. |
|
||||
|
||||
Some operations have additional parameters under the operation header.
|
||||
For details about each operation, use the `info` button to view the operation's description, or refer to the Prometheus documentation on [query functions](https://prometheus.io/docs/prometheus/latest/querying/functions/).
|
||||
|
||||
Some operations make sense only when used in a specific order.
|
||||
If adding an operation would result in nonsensical query, the query editor adds the operation to the correct place.
|
||||
To re-order operations manually, drag the operation box by its name and drop it into the desired place.
|
||||
|
||||
#### Hints
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/hint-8-5.png" max-width="500px" class="docs-image--no-shadow" caption="Hint" >}}
|
||||
|
||||
The query editor can detect which operations are most appropriate for some selected metrics.
|
||||
If it does, it displays a hint next to the `+ Operations` button.
|
||||
|
||||
To add the operations to your query, click the hint.
|
||||
|
||||
## Use Explain mode to understand queries
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/explain-8-5.png" max-width="500px" class="docs-image--no-shadow" caption="Explain mode" >}}
|
||||
|
||||
Explain mode helps you understand a query by displaying a step-by-step explanation of all query components and operations.
|
||||
|
||||
### Raw query
|
||||
|
||||
{{< figure src="/static/img/docs/prometheus/raw-query-8-5.png" max-width="500px" class="docs-image--no-shadow" caption="Raw query" >}}
|
||||
|
||||
The query editor displays the raw query only if the **Raw query** switch from the query editor toolbar is enabled.
|
||||
If visible, it displays the raw query that the query editor has created.
|
||||
|
||||
### Additional options
|
||||
|
||||
In addition to these Builder mode-specific options, the query editor also displays the options it shares in common with Code mode.
|
||||
For details, refer to the [common options]({{< relref "#configure-common-options" >}}).
|
||||
|
||||
## Apply annotations
|
||||
|
||||
[Annotations]({{< relref "../../../dashboards/build-dashboards/annotate-visualizations" >}}) overlay rich event information on top of graphs.
|
||||
You can add annotation queries in the Dashboard menu's Annotations view.
|
||||
|
||||
Prometheus supports two ways to query annotations.
|
||||
|
||||
- A regular metric query
|
||||
- A Prometheus query for pending and firing alerts (for details see [Inspecting alerts during runtime](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/#inspecting-alerts-during-runtime))
|
||||
|
||||
The step option is useful to limit the number of events returned from your query.
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/datasources/prometheus/template-variables/
|
||||
- /docs/grafana/latest/data-sources/prometheus/template-variables/
|
||||
description: Using template variables with Prometheus in Grafana
|
||||
keywords:
|
||||
- grafana
|
||||
- prometheus
|
||||
- templates
|
||||
- variables
|
||||
- queries
|
||||
menuTitle: Template variables
|
||||
title: Prometheus template variables
|
||||
weight: 400
|
||||
---
|
||||
|
||||
# Prometheus template variables
|
||||
|
||||
Instead of hard-coding details such as server, application, and sensor names in metric queries, you can use variables.
|
||||
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the data displayed in your dashboard.
|
||||
Grafana refers to such variables as template variables.
|
||||
|
||||
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../../dashboards/variables" >}}) and [Add and manage variables]({{< relref "../../../dashboards/variables/add-template-variables" >}}) documentation.
|
||||
|
||||
## Use query variables
|
||||
|
||||
You can use variables of the type _Query_ to query Prometheus for a list of metrics, labels, or label values.
|
||||
|
||||
You can use these Prometheus data source functions in the **Query** input field:
|
||||
|
||||
| Name | Description | Used API endpoints |
|
||||
| ----------------------------- | ----------------------------------------------------------------------- | --------------------------------- |
|
||||
| `label_names()` | Returns a list of label names. | /api/v1/labels |
|
||||
| `label_values(label)` | Returns a list of label values for the `label` in every metric. | /api/v1/label/`label`/values |
|
||||
| `label_values(metric, label)` | Returns a list of label values for the `label` in the specified metric. | /api/v1/series |
|
||||
| `metrics(metric)` | Returns a list of metrics matching the specified `metric` regex. | /api/v1/label/\_\_name\_\_/values |
|
||||
| `query_result(query)` | Returns a list of Prometheus query result for the `query`. | /api/v1/query |
|
||||
|
||||
For details on _metric names_, _label names_, and _label values_, refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
|
||||
|
||||
### Use interval and range variables
|
||||
|
||||
You can use some global built-in variables in query variables, for example, `$__interval`, `$__interval_ms`, `$__range`, `$__range_s` and `$__range_ms`.
|
||||
For details, see [Global built-in variables]({{< relref "../../../dashboards/variables/add-template-variables#global-variables" >}}).
|
||||
The `label_values` function doesn't support queries, so you can use these variables in conjunction with the `query_result` function to filter variable queries.
|
||||
|
||||
Make sure to set the variable's `refresh` trigger to be `On Time Range Change` to get the correct instances when changing the time range on the dashboard.
|
||||
|
||||
**Example:**
|
||||
|
||||
Populate a variable with the busiest 5 request instances based on average QPS over the time range shown in the dashboard:
|
||||
|
||||
```
|
||||
Query: query_result(topk(5, sum(rate(http_requests_total[$__range])) by (instance)))
|
||||
Regex: /"([^"]+)"/
|
||||
```
|
||||
|
||||
Populate a variable with the instances having a certain state over the time range shown in the dashboard, using `$__range_s`:
|
||||
|
||||
```
|
||||
Query: query_result(max_over_time(<metric>[${__range_s}s]) != <state>)
|
||||
Regex:
|
||||
```
|
||||
|
||||
## Use `$__rate_interval`
|
||||
|
||||
> **Note:** Available in Grafana v7.2 and higher.
|
||||
|
||||
We recommend using `$__rate_interval` in the `rate` and `increase` functions instead of `$__interval` or a fixed interval value.
|
||||
Because `$__rate_interval` is always at least four times the value of the Scrape interval, it avoid problems specific to Prometheus.
|
||||
|
||||
For example, instead of using:
|
||||
|
||||
```
|
||||
rate(http_requests_total[5m])
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```
|
||||
rate(http_requests_total[$__interval])
|
||||
```
|
||||
|
||||
We recommend that you use:
|
||||
|
||||
```
|
||||
rate(http_requests_total[$__rate_interval])
|
||||
```
|
||||
|
||||
The value of `$__rate_interval` is defined as
|
||||
*max(`$__interval` + *Scrape interval*, 4 \* *Scrape interval*)*,
|
||||
where _Scrape interval_ is the "Min step" setting (also known as `query*interval`, a setting per PromQL query) if any is set.
|
||||
Otherwise, Grafana uses the Prometheus data source's "Scrape interval" setting.
|
||||
|
||||
The "Min interval" setting in the panel is modified by the resolution setting, and therefore doesn't have any effect on _Scrape interval_.
|
||||
|
||||
For details, refer to the [Grafana blog](/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/).
|
||||
|
||||
## Choose a variable syntax
|
||||
|
||||
The Prometheus data source supports two variable syntaxes for use in the **Query** field:
|
||||
|
||||
- `$<varname>`, for example `rate(http_requests_total{job=~"\$job"}[$_rate_interval])`, which is easier to read and write but does not allow you to use a variable in the middle of a word.
|
||||
- `[[varname]]`, for example `rate(http_requests_total{job=~"[[job]]"}[$_rate_interval])`
|
||||
|
||||
If you've enabled the _Multi-value_ or _Include all value_ options, Grafana converts the labels from plain text to a regex-compatible string, which requires you to use `=~` instead of `=`.
|
||||
|
||||
## Use the ad hoc filters variable type
|
||||
|
||||
Prometheus supports the special [ad hoc filters]({{< relref "../../../dashboards/variables/add-template-variables#add-ad-hoc-filters" >}}) variable type, which you can use to specify any number of label/value filters on the fly.
|
||||
These filters are automatically applied to all your Prometheus queries.
|
||||
Reference in New Issue
Block a user