Consolidate Grafana visualization documentation (#112393)
Co-authored-by: Jacob Valdez <jacob.valdez@grafana.com>
This commit is contained in:
Jack Baldry
GitHub
@@ -0,0 +1,335 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../panels-visualizations/manage-queries/ # /docs/grafana/next/panels-visualizations/manage-queries/
|
||||
- ../../panels-visualizations/query-options/ # /docs/grafana/next/panels-visualizations/query-options/
|
||||
- ../../panels-visualizations/query-transform-data/ # /docs/grafana/next/panels-visualizations/query-transform-data/
|
||||
- ../../panels/expressions/ # /docs/grafana/next/panels/expressions/
|
||||
- ../../panels/inspect-panel/ # /docs/grafana/next/panels/inspect-panel/
|
||||
- ../../panels/queries/ # /docs/grafana/next/panels/queries/
|
||||
- ../../panels/query-a-data-source/ # /docs/grafana/next/panels/query-a-data-source/
|
||||
- ../../panels/query-a-data-source/about-queries/ # /docs/grafana/next/panels/query-a-data-source/about-queries/
|
||||
- ../../panels/query-a-data-source/add-a-query/ # /docs/grafana/next/panels/query-a-data-source/add-a-query/
|
||||
- ../../panels/query-a-data-source/manage-queries/ # /docs/grafana/next/panels/query-a-data-source/manage-queries/
|
||||
- ../../panels/query-a-data-source/navigate-query-tab/ # /docs/grafana/next/panels/query-a-data-source/navigate-query-tab/
|
||||
- ../../panels/query-options/ # /docs/grafana/next/panels/query-options/
|
||||
- ../../panels/reference-query-options/ # /docs/grafana/next/panels/reference-query-options/
|
||||
- ../../panels/share-query-results/ # /docs/grafana/next/panels/share-query-results/
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Query and transform data
|
||||
description: Query and transform your data
|
||||
weight: 40
|
||||
refs:
|
||||
data-sources:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/
|
||||
built-in-core-data-sources:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/#built-in-core-data-sources
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/#built-in-core-data-sources
|
||||
use-expressions-to-manipulate-data:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/query-transform-data/expression-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/visualizations/panels-visualizations/query-transform-data/expression-queries/
|
||||
global-variables:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/dashboards/variables/add-template-variables/#global-variables
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/visualizations/dashboards/variables/add-template-variables/#global-variables
|
||||
plugin-management:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/plugin-management/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/plugin-management/
|
||||
recorded-queries:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
special-data-sources:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/#special-data-sources
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/#special-data-sources
|
||||
---
|
||||
|
||||
# Query and transform data
|
||||
|
||||
Grafana supports many types of [data sources](ref:data-sources).
|
||||
Data source _queries_ return data that Grafana can _transform_ and visualize.
|
||||
Each data source uses its own query language, and data source plugins each implement a query-building user interface called a query editor.
|
||||
|
||||
## About queries
|
||||
|
||||
Grafana panels communicate with data sources using queries, which retrieve data for the visualization.
|
||||
A query is a question written in the query language used by the data source.
|
||||
|
||||
You can configure query frequency and data collection limits in the panel's data source options.
|
||||
Grafana supports up to 26 queries per panel.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
You **must** be familiar with a data source's query language.
|
||||
For more information, refer to [Data sources](ref:data-sources).
|
||||
{{< /admonition >}}
|
||||
|
||||
### Query editors
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-queries-tab-v11.6.png" max-width="750px" alt="The InfluxDB query editor" >}}
|
||||
|
||||
Each data source's query editor provides a customized user interface that helps you write queries that take advantage of its unique capabilities.
|
||||
|
||||
Because of the differences between query languages, each data source query editor looks and functions differently.
|
||||
Depending on your data source, the query editor might provide auto-completion features, metric names, variable suggestions, or a visual query-building interface.
|
||||
|
||||
For example, this video demonstrates the visual Prometheus query builder:
|
||||
|
||||
{{< vimeo 720004179 >}}
|
||||
|
||||
For details on a specific data source's unique query editor features, refer to its documentation:
|
||||
|
||||
- For data sources included with Grafana, refer to [Built-in core data sources](ref:built-in-core-data-sources), which links to each core data source's documentation.
|
||||
- For data sources installed as plugins, refer to the documentation for the plugin.
|
||||
- Data source plugins in Grafana's [plugin catalog](/grafana/plugins/) link to or include their documentation in their catalog listings.
|
||||
For details about the plugin catalog, refer to [Plugin management](ref:plugin-management).
|
||||
- For links to Grafana Enterprise data source plugin documentation, refer to the [Enterprise plugins index](/docs/plugins/).
|
||||
|
||||
### Query syntax
|
||||
|
||||
Each data source uses a different query languages to request data.
|
||||
For details on a specific data source's unique query language, refer to its documentation.
|
||||
|
||||
**PostgreSQL example:**
|
||||
|
||||
```
|
||||
SELECT hostname FROM host WHERE region IN($region)
|
||||
```
|
||||
|
||||
**PromQL example:**
|
||||
|
||||
```
|
||||
query_result(max_over_time(<metric>[${__range_s}s]) != <state>)
|
||||
```
|
||||
|
||||
### Saved queries
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
Saved queries is currently in [public preview](https://grafana.com/docs/release-life-cycle/). Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available.
|
||||
|
||||
This feature is only available on Grafana Enterprise and Grafana Cloud. It will gradually roll out to all Grafana Cloud users with no action required. To try out this feature on Grafana Enterprise, enable the `queryLibrary` feature toggle.
|
||||
{{< /admonition >}}
|
||||
|
||||
You can save queries that you've created so they can be reused by you and others in your organization.
|
||||
This helps users across your organization create dashboards or find insights in Explore without having to create their own queries or know a query language.
|
||||
It also helps you avoid having several users build the same queries for the same data sources multiple times.
|
||||
|
||||
You can see a list of these queries in the **Saved queries** drawer:
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-saved-queries-v12.2png.png" max-width="550px" alt="List of saved queries and the edit query form" caption="The Saved queries drawer accessed from Dashboards" >}}
|
||||
|
||||
When you first open the drawer, the list of queries in the **All** tab is filtered by the data source of the panel.
|
||||
However, you can clear that filter to display all saved queries.
|
||||
|
||||
The list in the **Favorites** tab is also filtered by data source, by default.
|
||||
The **Recent** tab displays the last 20 queries across all data sources from your **Query history** in Explore.
|
||||
From this tab, you can save queries for reuse as well.
|
||||
|
||||
In the **Saved queries** drawer, you can:
|
||||
|
||||
- Search for queries by data source name, query content, title, or description.
|
||||
- Sort queries alphabetically or by creation date.
|
||||
- Filter by data source name, author name, and tags (the tag filter uses the `OR` operator, while the others use the `AND` operator).
|
||||
- Set queries as favorites.
|
||||
- Duplicate, lock and unlock a query for editing, or delete a saved query.
|
||||
- Edit a query title, description, tags, or the availability of the query to other users in your organization. By default, saved queries are locked for editing.
|
||||
- When you access the **Saved queries** drawer from Explore, you can use the **Edit in Explore** option to edit the body of a query.
|
||||
|
||||
Access the duplicate, lock, unlock, and delete query options through the menu in the top-right corner of the query form next to the **Edit** button.
|
||||
|
||||
To access your saved queries, click **+ Add from saved queries** in the query editor:
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-add-from-saved-2-v12.2.png" max-width="750px" alt="Add a saved query" >}}
|
||||
|
||||
If you've already entered a query, you also have the option to replace it with a saved one:
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-replace-w-saved-v12.2.png" max-width="750px" alt="Replace a query with a saved one" >}}
|
||||
|
||||
#### Save a query
|
||||
|
||||
To save a query you've created:
|
||||
|
||||
1. From the query editor, click the **Save query** icon:
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-save-query-v12.2.png" max-width="750px" alt="Save a query" >}}
|
||||
|
||||
1. In the **Saved queries** drawer, enter a title for the query that will make it easy to find later.
|
||||
1. (Optional) Enter a description and relevant tags.
|
||||
1. Clear the **Share query with all users** checkbox if you only want the saved query to be available to you.
|
||||
1. Click **Save**.
|
||||
|
||||
#### Known limitations
|
||||
|
||||
- No validation is performed when you save a query, so it's possible to save an invalid query. You should confirm the query is working properly before you save it.
|
||||
- Saved queries are currently accessible from the query editors in Dashboards and Explore.
|
||||
- You can save a maximum of 1000 queries.
|
||||
- Users with the Viewer role who have access to Explore can use saved queries, but can't write them.
|
||||
- If you have multiple queries open in Explore and you edit one of them by way of the **Edit in Explore** function in the **Saved queries** drawer, the edited query replaces your open queries in Explore.
|
||||
|
||||
### Special data sources
|
||||
|
||||
Grafana also includes three special data sources: **Grafana**, **Mixed**, and **Dashboard**.
|
||||
For details, refer to [Data sources](ref:data-sources)
|
||||
|
||||
## Navigate the Queries tab {#navigate-the-query-tab}
|
||||
|
||||
A panel's **Queries** tab consists of the following elements:
|
||||
|
||||
- **Data source selector** - Selects the data source to query.
|
||||
For more information about data sources, refer to [Data sources](ref:data-sources).
|
||||
- **Query options** - Sets maximum data retrieval parameters and query execution time intervals.
|
||||
- **Query inspector button** - Opens the query inspector panel, where you can view and optimize your query.
|
||||
- **Query editor list** - The list of queries you've written. Each query can be expanded or collapsed.
|
||||
- **Expressions** - Uses the expression builder to create alert expressions.
|
||||
For more information about expressions, refer to [Use expressions to manipulate data](ref:use-expressions-to-manipulate-data).
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-queries-tab2-v11.6.png" max-width="750px" alt="The Query tab of the panel editor" >}}
|
||||
|
||||
## Add a query
|
||||
|
||||
A query returns data that Grafana visualizes in dashboard panels.
|
||||
When you create a panel, Grafana automatically selects the default data source.
|
||||
|
||||
To add a query, follow these steps:
|
||||
|
||||
1. Hover the cursor over any part of the panel to which you're adding a query to display the menu icon in the top-right corner.
|
||||
1. Click the menu and select **Edit**.
|
||||
1. In the panel editor, click the **Queries** tab.
|
||||
1. Click the **Data source** drop-down menu and select a data source.
|
||||
|
||||
If you're creating a new dashboard, you'll be prompted to select a data source when you add the first panel.
|
||||
|
||||
1. Click **Query options** to configure the maximum number of data points you need.
|
||||
|
||||
For more information about query options, refer to [Query options](#query-options).
|
||||
|
||||
1. To add a query, do one of the following:
|
||||
- Write or construct a query in the query language of your data source.
|
||||
- Click **+ Add from saved queries** to add a previously saved query.
|
||||
- If you've already written a query, you can click the **Replace with saved query** icon to use a previously saved query instead.
|
||||
|
||||
1. (Optional) To save the query for reuse, click the **Save query** icon.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
[Saved queries](#saved-queries) is currently in [public preview](https://grafana.com/docs/release-life-cycle/). Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available.
|
||||
|
||||
This feature is only available on Grafana Enterprise and Grafana Cloud.
|
||||
{{< /admonition >}}
|
||||
|
||||
1. Click **Run queries**.
|
||||
|
||||
Grafana queries the data source and visualizes the data.
|
||||
|
||||
## Manage queries
|
||||
|
||||
Grafana organizes queries in collapsible query rows.
|
||||
Each query row contains a query editor and is identified with a letter (A, B, C, and so on).
|
||||
|
||||
You can:
|
||||
|
||||
<!-- prettier-ignore-start -->
|
||||
| Icon | Description |
|
||||
| ------- | -------------------------------------------- |
|
||||
| {{< figure src="/static/img/docs/queries/query-editor-help-7-4.png" max-width="30px" max-height="30px" alt="Help icon" >}} | Toggles query editor help. If supported by the data source, click this icon to display information on how to use the query editor or provide quick access to common queries. |
|
||||
| {{< figure src="/media/docs/grafana/panels-visualizations/create-recorded-query-icon.png" max-width="30px" max-height="30px" alt="Create recorded query icon" >}} | Create [recorded queries](ref:recorded-queries) so you can see trends over time by taking a snapshot of a data point on a set interval (Enterprise and Cloud only). |
|
||||
| {{< figure src="/media/docs/grafana/panels-visualizations/save-to-query-icon.png" max-width="30px" max-height="30px" alt="Save query icon" >}} | Save query. Saves the query so it can be reused. Access saved queries by clicking **+ Add saved query**. For more information, refer to [Saved queries](#saved-queries) (Enterprise and Cloud only). |
|
||||
| {{< figure src="/static/img/docs/queries/duplicate-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Duplicate icon" >}} | Copies a query. Duplicating queries is useful when working with multiple complex queries that are similar and you want to either experiment with different variants or do minor alterations. |
|
||||
| {{< figure src="/static/img/docs/queries/hide-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Hide icon" >}} | Hides a query. Grafana does not send hidden queries to the data source. |
|
||||
| {{< figure src="/static/img/docs/queries/remove-query-icon-7-0.png" max-width="30px" max-height="30px" alt="Remove icon">}} | Removes a query. Removing a query permanently deletes it, but sometimes you can recover deleted queries by reverting to previously saved versions of the panel. |
|
||||
| {{< figure src="/static/img/docs/queries/query-drag-icon-7-2.png" max-width="30px" max-height="30px" alt="Drag icon" >}} | Reorders queries. Change the order of queries by clicking and holding the drag icon, then drag queries where desired. The order of results reflects the order of the queries, so you can often adjust your visual results based on query order. |
|
||||
<!-- prettier-ignore-end -->
|
||||
|
||||
## Query options
|
||||
|
||||
Click **Query options** next to the data source selector to see settings for the selected data source.
|
||||
Changes you make here affect only queries made in this panel.
|
||||
|
||||
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-query-options-v11.6.png" max-width="750px" alt="Data source query options" >}}
|
||||
|
||||
Grafana sets defaults that are shown in dark gray text.
|
||||
Changes are displayed in white text.
|
||||
To return a field to the default setting, delete the white text from the field.
|
||||
|
||||
Panel data source query options include:
|
||||
|
||||
- **Max data points** - If the data source supports it, this sets the maximum number of data points for each series returned.
|
||||
If the query returns more data points than the max data points setting, then the data source reduces the number of points returned by aggregating them together by average, max, or another function.
|
||||
|
||||
You can limit the number of points to improve query performance or smooth the visualized line.
|
||||
The default value is the width (or number of pixels) of the graph, because you can only visualize as many data points as the graph panel has room to display.
|
||||
|
||||
With streaming data, Grafana uses the max data points value for the rolling buffer.
|
||||
Streaming is a continuous flow of data, and buffering divides the stream into chunks.
|
||||
For example, Loki streams data in its live tailing mode.
|
||||
|
||||
- **Min interval** - Sets a minimum limit for the automatically calculated interval, which is typically the minimum scrape interval.
|
||||
If a data point is saved every 15 seconds, you don't benefit from having an interval lower than that.
|
||||
You can also set this to a higher minimum than the scrape interval to retrieve queries that are more coarse-grained and well-functioning.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
The **Min interval** corresponds to the min step in Prometheus. Changing the Prometheus interval can change the start and end of the query range because Prometheus aligns the range to the interval. Refer to [Min step](https://grafana.com/docs/grafana/latest/datasources/prometheus/query-editor/#min-step) for more details.
|
||||
{{< /admonition >}}
|
||||
|
||||
- **Interval** - Sets a time span that you can use when aggregating or grouping data points by time.
|
||||
|
||||
Grafana automatically calculates an appropriate interval that you can use as a variable in templated queries.
|
||||
The variable is measured in either seconds (`$__interval`) or milliseconds (`$__interval_ms`).
|
||||
|
||||
Intervals are typically used in aggregation functions like sum or average.
|
||||
For example, this is a Prometheus query that uses the interval variable: `rate(http_requests_total[$__interval])`.
|
||||
|
||||
This automatic interval is calculated based on the width of the graph.
|
||||
As the user zooms out on a visualization, the interval grows, resulting in a more coarse-grained aggregation.
|
||||
Likewise, if the user zooms in, the interval decreases, resulting in a more fine-grained aggregation.
|
||||
|
||||
For more information, refer to [Global variables](ref:global-variables).
|
||||
|
||||
- **Relative time** - Overrides the relative time range for individual panels, which causes them to be different than what is selected in the dashboard time picker in the top-right corner of the dashboard.
|
||||
You can use this to show metrics from different time periods or days on the same dashboard.
|
||||
|
||||
{{< admonition type="note">}}
|
||||
Panel time overrides have no effect when the dashboard's time range is absolute.
|
||||
{{< /admonition >}}
|
||||
|
||||
| Example | Relative time field |
|
||||
| ---------------- | ------------------- |
|
||||
| Last 5 minutes | `now-5m` |
|
||||
| The day so far | `now/d` |
|
||||
| Last 5 days | `now-5d/d` |
|
||||
| This week so far | `now/w` |
|
||||
| Last 2 years | `now-2y/y` |
|
||||
|
||||
{{< docs/play title="Time range override" url="https://play.grafana.org/d/000000041/" >}}
|
||||
|
||||
- **Time shift** - Overrides the time range for individual panels by shifting its start and end relative to the time picker.
|
||||
For example, you can shift the time range for the panel to be two hours earlier than the dashboard time picker.
|
||||
|
||||
{{< admonition type="note">}}
|
||||
Panel time overrides have no effect when the dashboard's time range is absolute.
|
||||
{{< /admonition >}}
|
||||
|
||||
| Example | Time shift field |
|
||||
| -------------------- | ---------------- |
|
||||
| Last entire week | `1w/w` |
|
||||
| Two entire weeks ago | `2w/w` |
|
||||
| Last entire month | `1M/M` |
|
||||
| This entire year | `1d/y` |
|
||||
| Last entire year | `1y/y` |
|
||||
|
||||
- **Cache timeout** - _(Visible only if available in the data source)_ Overrides the default cache timeout if your time series store has a query cache.
|
||||
Specify this value as a numeric value in seconds.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../../panels-visualizations/calculation-types/ # /docs/grafana/next/panels-visualizations/calculation-types/
|
||||
- ../../../panels-visualizations/query-transform-data/calculation-types/ # /docs/grafana/next/panels-visualizations/query-transform-data/calculation-types/
|
||||
- ../../../panels/calculation-types/ # /docs/grafana/<GRAFANA_VERSION>/panels/calculation-types/
|
||||
- ../../../panels/calculations-list/ # /docs/grafana/<GRAFANA_VERSION>/panels/calculations-list/
|
||||
- ../../../panels/reference-calculation-types/ # /docs/grafana/<GRAFANA_VERSION>/panels/reference-calculation-types/
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Calculation types
|
||||
description: Learn about the calculations you can apply to your data
|
||||
weight: 1100
|
||||
---
|
||||
|
||||
# Calculation types
|
||||
|
||||
The following table contains a list of calculations you can perform in Grafana. You can find these calculations in the **Transform** tab and in the bar gauge, gauge, stat, and table visualizations.
|
||||
|
||||
| Calculation | Description |
|
||||
| :----------------- | :-------------------------------------------------------- |
|
||||
| 1st % - 99th % | 1st - 99th percentile value. |
|
||||
| All nulls | True when all values are null |
|
||||
| All unique values | Array with all unique values |
|
||||
| All values | Array with all values |
|
||||
| All zeros | True when all values are 0 |
|
||||
| Change count | Number of times the field's value changes |
|
||||
| Count | Number of values in a field |
|
||||
| Delta | Cumulative change in value, only counts increments |
|
||||
| Difference | Difference between first and last value of a field |
|
||||
| Difference percent | Percentage change between first and last value of a field |
|
||||
| Distinct count | Number of unique values in a field |
|
||||
| First | First value in a field |
|
||||
| First\* (not null) | First, not null value in a field (also excludes NaNs) |
|
||||
| Last | Last value in a field |
|
||||
| Last\* (not null) | Last, not null value in a field (also excludes NaNs) |
|
||||
| Max | Maximum value of a field |
|
||||
| Median | Median value of all values in a field |
|
||||
| Mean | Mean value of all values in a field |
|
||||
| Min | Minimum value of a field |
|
||||
| Min (above zero) | Minimum, positive value of a field |
|
||||
| Range | Difference between maximum and minimum values of a field |
|
||||
| StdDev | Standard deviation of all values in a field |
|
||||
| Step | Minimal interval between values of a field |
|
||||
| Total | Sum of all values in a field |
|
||||
| Variance | Variance of all values in a field |
|
||||
+263
@@ -0,0 +1,263 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../../panels-visualizations/query-transform-data/ # /docs/grafana/next/panels-visualizations/query-transform-data/
|
||||
- ../../../panels-visualizations/query-transform-data/expression-queries/ # /docs/grafana/next/panels-visualizations/query-transform-data/expression-queries/
|
||||
- ../../../panels/query-a-data-source/use-expressions-to-manipulate-data/ # /docs/grafana/next/panels/query-a-data-source/use-expressions-to-manipulate-data/
|
||||
- ../../../panels/query-a-data-source/use-expressions-to-manipulate-data/about-expressions/ # /docs/grafana/next/panels/query-a-data-source/use-expressions-to-manipulate-data/about-expressions/
|
||||
- ../../../panels/query-a-data-source/use-expressions-to-manipulate-data/write-an-expression/ # /docs/grafana/next/panels/query-a-data-source/use-expressions-to-manipulate-data/write-an-expression/
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Write expression queries
|
||||
title: Write expression queries
|
||||
description: Write server-side expressions to manipulate data using math and other operations
|
||||
weight: 40
|
||||
refs:
|
||||
no-data-and-error-handling:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-grafana-managed-rule/#configure-no-data-and-error-handling
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-grafana-managed-rule/#configure-no-data-and-error-handling
|
||||
multiple-dimensional-data:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/fundamentals/timeseries-dimensions/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/fundamentals/timeseries-dimensions/
|
||||
grafana-alerting:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/
|
||||
labels:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/fundamentals/timeseries-dimensions/#labels
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/fundamentals/timeseries-dimensions/#labels
|
||||
---
|
||||
|
||||
# Write expression queries
|
||||
|
||||
Server-side expressions enable you to manipulate data returned from queries with math and other operations. Expressions create new data and do not manipulate the data returned by data sources.
|
||||
|
||||
## About expressions
|
||||
|
||||
Server-side expressions allow you to manipulate data returned from queries with math and other operations. Expressions create new data and do not manipulate the data returned by data sources, aside from some minor data restructuring to make the data acceptable input for expressions.
|
||||
|
||||
### Using expressions
|
||||
|
||||
Expressions are most commonly used for [Grafana Alerting](ref:grafana-alerting). The processing is done server-side, so expressions can operate without a browser session. However, expressions can also be used with backend data sources and visualization.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
Expressions do not work with legacy dashboard alerts.
|
||||
{{< /admonition >}}
|
||||
|
||||
Expressions are meant to augment data sources by enabling queries from different data sources to be combined or by providing operations unavailable in a data source.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
When possible, you should do data processing inside the data source. Copying data from storage to the Grafana server for processing is inefficient, so expressions are targeted at lightweight data processing.
|
||||
{{< /admonition >}}
|
||||
|
||||
Expressions work with data source queries that return time series or number data. They also operate on [multiple-dimensional data](ref:multiple-dimensional-data). For example, a query that returns multiple series, where each series is identified by labels or tags.
|
||||
|
||||
An individual expression takes one or more queries or other expressions as input and adds data to the result. Each individual expression or query is represented by a variable that is a named identifier known as its RefID (e.g., the default letter `A` or `B`).
|
||||
|
||||
To reference the output of an individual expression or a data source query in another expression, this identifier is used as a variable.
|
||||
|
||||
### Types of expressions
|
||||
|
||||
Expressions work with two types of data.
|
||||
|
||||
- A collection of time series.
|
||||
- A collection of numbers, where each number is an item.
|
||||
|
||||
Each collection is returned from a single data source query or expression and represented by the RefID. Each collection is a set, where each item in the set is uniquely identified by its dimensions which are stored as [labels](ref:labels) or key-value pairs.
|
||||
|
||||
### Data source queries
|
||||
|
||||
Server-side expressions only support data source queries for backend data sources. The data is generally assumed to be labeled time series data. In the future we intend to add an assertion of the query return type (number or time series) data so expressions can handle errors better.
|
||||
|
||||
Data source queries, when used with expressions, are executed by the expression engine. When it does this, it restructures data to be either one time series or one number per data frame. So for example if using a data source that returns multiple series on one frame in the table view, you might notice it looks different when executed with expressions.
|
||||
|
||||
Currently, the only non-time series format (number) is supported when you're using data frames and you have a table response that returns a data frame with no time, string columns, and one number column:
|
||||
|
||||
| Loc | Host | Avg_CPU |
|
||||
| --- | ---- | ------- |
|
||||
| MIA | A | 1 |
|
||||
| NYC | B | 2 |
|
||||
|
||||
The example above will produce a number that works with expressions. The string columns become labels and the number column the corresponding value. For example `{"Loc": "MIA", "Host": "A"}` with a value of 1.
|
||||
|
||||
### Operations
|
||||
|
||||
You can use the following operations in expressions: math, reduce, and resample.
|
||||
|
||||
#### Math
|
||||
|
||||
Math is for free-form math formulas on time series or number data. Math operations take numbers and time series as input and change them to different numbers and time series.
|
||||
|
||||
Data from other queries or expressions are referenced with the RefID prefixed with a dollar sign, for example `$A`. If the variable has spaces in the name, then you can use a brace syntax like `${my variable}`.
|
||||
|
||||
Numeric constants may be in decimal (`2.24`), octal (with a leading zero like `072`), or hex (with a leading 0x like `0x2A`). Exponentials and signs are also supported (e.g., `-0.8e-2`).
|
||||
|
||||
##### Operators
|
||||
|
||||
The arithmetic (`+`, binary and unary `-`, `*`, `/`, `%`, exponent `**`), relational (`<`, `>`, `==`, `!=`, `>=`, `<=`), and logical (`&&`, `||`, and unary `!`) operators are supported.
|
||||
|
||||
How the operation behaves with data depends on if it is a number or time series data.
|
||||
|
||||
With binary operations, such as `$A + $B` or `$A || $B`, the operator is applied in the following ways depending on the type of data:
|
||||
|
||||
- If both `$A` and `$B` are a number, then the operation is performed between the two numbers.
|
||||
- If one variable is a number, and the other variable is a time series, then the operation between the value of each point in the time series and the number is performed.
|
||||
- If both `$A` and `$B` are time series data, then the operation between each value in the two series is performed for each time stamp that exists in both `$A` and `$B`. The Resample operation can be used to line up time stamps. (**Note:** in the future, we plan to add options to the Math operation for different behaviors).
|
||||
|
||||
Summary:
|
||||
|
||||
- Number OP number = number
|
||||
- Number OP series = series
|
||||
- Series OP series = series
|
||||
|
||||
Because expressions work with multiple series or numbers represented by a single variable, binary operations also perform a union (join) between the two variables. This is done based on the identifying labels associated with each individual series or number.
|
||||
|
||||
So if you have numbers with labels like `{host=web01}` in `$A` and another number in `$B` with the same labels then the operation is performed between those two items within each variable, and the result will share the same labels. The rules for the behavior of this union are as follows:
|
||||
|
||||
- An item with no labels will join to anything.
|
||||
- If both `$A` and `$B` each contain only one item (one series, or one number), they will join.
|
||||
- If labels are exact match they will join.
|
||||
- If labels are a subset of the other, for example and item in `$A` is labeled `{host=A,dc=MIA}` and item in `$B` is labeled `{host=A}` they will join.
|
||||
- Currently, if within a variable such as `$A` there are different tag _keys_ for each item, the join behavior is undefined.
|
||||
|
||||
The relational and logical operators return 0 for false 1 for true.
|
||||
|
||||
##### Math Functions
|
||||
|
||||
While most functions exist in the own expression operations, the math operation does have some functions similar to math operators or symbols. When functions can take either numbers or series, than the same type as the argument will be returned. When it is a series, the operation of performed for the value of each point in the series.
|
||||
|
||||
###### abs
|
||||
|
||||
abs returns the absolute value of its argument which can be a number or a series. For example `abs(-1)` or `abs($A)`.
|
||||
|
||||
###### is_inf
|
||||
|
||||
is_inf takes a number or a series and returns `1` for `Inf` values (negative or positive) and `0` for other values. For example `is_inf($A)`.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
If you need to specifically check for negative infinity for example, you can do a comparison like `$A == infn()`.
|
||||
{{< /admonition >}}
|
||||
|
||||
###### is_nan
|
||||
|
||||
is_nan takes a number or a series and returns `1` for `NaN` values and `0` for other values. For example `is_nan($A)`. This function exists because `NaN` is not equal to `NaN`.
|
||||
|
||||
###### is_null
|
||||
|
||||
is_null takes a number or a series and returns `1` for `null` values and `0` for other values. For example `is_null($A)`.
|
||||
|
||||
###### is_number
|
||||
|
||||
is_number takes a number or a series and returns `1` for all real number values and `0` for other values (which are `null`, `Inf+`, `Inf-`, and `NaN`). For example `is_number($A)`.
|
||||
|
||||
###### log
|
||||
|
||||
Log returns the natural logarithm of of its argument which can be a number or a series. If the value is less than 0, NaN is returned. For example `log(-1)` or `log($A)`.
|
||||
|
||||
###### inf, infn, nan, and null
|
||||
|
||||
The inf, infn, nan, and null functions all return a single value of the name. They primarily exist for testing. Example: `null()`.
|
||||
|
||||
###### round
|
||||
|
||||
Round returns a rounded integer value. For example, `round(3.123)` or `round($A)`. (This function should probably take an argument so it can add precision to the rounded value).
|
||||
|
||||
###### ceil
|
||||
|
||||
Ceil rounds the number up to the nearest integer value. For example, `ceil(3.123)` returns 4.
|
||||
|
||||
###### floor
|
||||
|
||||
Floor rounds the number down to the nearest integer value. For example, `floor(3.123)` returns 3.
|
||||
|
||||
#### Reduce
|
||||
|
||||
Reduce takes one or more time series returned from a query or an expression and turns each series into a single number. The labels of the time series are kept as labels on each outputted reduced number.
|
||||
|
||||
**Fields:**
|
||||
|
||||
- **Function -** The reduction function to use
|
||||
- **Input -** The variable (refID (such as `A`)) to resample
|
||||
- **Mode -** Allows control behavior of reduction function when a series contains non-numerical values (null, NaN, +\-Inf)
|
||||
|
||||
##### Reduction Functions
|
||||
|
||||
###### Count
|
||||
|
||||
Count returns the number of points in each series.
|
||||
|
||||
###### Mean
|
||||
|
||||
Mean returns the total of all values in each series divided by the number of points in that series. In `strict` mode if any values in the series are null or nan, or if the series is empty, NaN is returned.
|
||||
|
||||
###### Min and Max
|
||||
|
||||
Min and Max return the smallest or largest value in the series respectively. In `strict` mode if any values in the series are null or nan, or if the series is empty, NaN is returned.
|
||||
|
||||
###### Sum
|
||||
|
||||
Sum returns the total of all values in the series. If series is of zero length, the sum will be 0. In `strict` mode if there are any NaN or Null values in the series, NaN is returned.
|
||||
|
||||
##### Last
|
||||
|
||||
Last returns the last number in the series. If the series has no values then returns NaN.
|
||||
|
||||
##### Reduction Modes
|
||||
|
||||
###### Strict
|
||||
|
||||
In Strict mode the input series is processed as is. If any values in the series are non-numeric (null, NaN or +\-Inf), NaN is returned.
|
||||
|
||||
###### Drop Non-Numeric
|
||||
|
||||
In this mode all non-numeric values (null, NaN or +\-Inf) in the input series are filtered out before executing the reduction function.
|
||||
|
||||
###### Replace Non-Numeric
|
||||
|
||||
In this mode all non-numeric values are replaced by a pre-defined value.
|
||||
|
||||
#### Resample
|
||||
|
||||
Resample changes the time stamps in each time series to have a consistent time interval. The main use case is so you can resample time series that do not share the same timestamps so math can be performed between them. This can be done by resample each of the two series, and then in a Math operation referencing the resampled variables.
|
||||
|
||||
**Fields:**
|
||||
|
||||
- **Input -** The variable of time series data (refID (such as `A`)) to resample
|
||||
- **Resample to -** The duration of time to resample to, for example `10s`. Units may be `s` seconds, `m` for minutes, `h` for hours, `d` for days, `w` for weeks, and `y` of years.
|
||||
- **Downsample -** The reduction function to use when there are more than one data point per window sample. See the reduction operation for behavior details.
|
||||
- **Upsample -** The method to use to fill a window sample that has no data points.
|
||||
- **pad** fills with the last know value
|
||||
- **backfill** with next known value
|
||||
- **fillna** to fill empty sample windows with NaNs
|
||||
|
||||
## Write an expression
|
||||
|
||||
If your data source supports them, then Grafana displays the **Expression** button and shows any existing expressions in the query editor list.
|
||||
|
||||
For more information about expressions, refer to [About expressions](#about-expressions).
|
||||
|
||||
1. Open the panel.
|
||||
1. Below the query, click **Expression**.
|
||||
1. In the **Operation** field, select the type of expression you want to write.
|
||||
|
||||
For more information about expression operations, refer to [About expressions](#about-expressions).
|
||||
|
||||
1. Write the expression.
|
||||
1. Click **Apply**.
|
||||
|
||||
## Special cases
|
||||
|
||||
When any queried data source returns no series or numbers, the expression engine returns `NoData`. For example, if a request contains two data source queries that are merged by an expression, if `NoData` is returned by at least one of the data source queries, then the returned result for the entire query is `NoData`.
|
||||
|
||||
For more information about how [Grafana Alerting](ref:grafana-alerting) processes `NoData` results, refer to [No data and error handling](ref:no-data-and-error-handling).
|
||||
|
||||
In the case of using an expression on multiple queries, the expression engine requires that all of the queries return an identical timestamp. For example, if using math to combine the results of multiple SQL queries which each use `SELECT NOW() AS "time"`, the expression will only work if all queries evaluate `NOW()` to an identical timestamp; which does not always happen. To resolve this, you can replace `NOW()` with an arbitrary time, such as `SELECT 1 AS "time"`, or any other valid UNIX timestamp.
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../../panels/query-a-data-source/share-query/ # /docs/grafana/next/panels/query-a-data-source/share-query/
|
||||
- ../../../panels-visualizations/query-transform-data/share-query/ # /docs/grafana/next/panels-visualizations/query-transform-data/share-query/
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Share query results
|
||||
title: Share query results with another panel
|
||||
description: Reduce the number of queries to your data source by sharing query results
|
||||
weight: 60
|
||||
refs:
|
||||
query:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/query-transform-data/#add-a-query
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/query-transform-data/#add-a-query
|
||||
create-a-dashboard:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/dashboards/build-dashboards/create-dashboard/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/dashboards/build-dashboards/create-dashboard/
|
||||
---
|
||||
|
||||
# Share query results with another panel
|
||||
|
||||
Grafana lets you use the query result from one panel for any other panel in the dashboard. Sharing query results across panels reduces the number of queries made to your data source, which can improve the performance of your dashboard.
|
||||
|
||||
The Dashboard data source lets you select a panel in your dashboard that contains the queries you want to share the results for. Instead of sending a separate query for each panel, Grafana sends one query and other panels use the query results to construct visualizations.
|
||||
|
||||
This strategy can drastically reduce the number of queries being made when you for example have several panels visualizing the same data.
|
||||
|
||||
1. [Create a dashboard](ref:create-a-dashboard).
|
||||
1. Create a panel.
|
||||
1. Change the panel title to "Source panel". You'll use this panel as a source for the other panels.
|
||||
1. Define the [query](ref:query) or queries that you want share.
|
||||
|
||||
If you don't have a data source available, use the **Grafana** data source, which returns a random time series that you can use for testing.
|
||||
|
||||
1. Add a new panel and select the **Dashboard** data source in the query editor.
|
||||
1. In the **Use results from panel list**, select the first panel you created.
|
||||
|
||||
All queries defined in the source panel are now available to the new panel. Queries defined in the source panel can be shared with multiple panels.
|
||||
|
||||
You can click on any of the queries to go to the panel where they are defined.
|
||||
+273
@@ -0,0 +1,273 @@
|
||||
---
|
||||
aliases:
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: SQL expressions
|
||||
title: SQL expressions
|
||||
description: Manipulate and transform data in Grafana using SQL expressions.
|
||||
weight: 45
|
||||
refs:
|
||||
expressions:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/query-transform-data/expression-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/visualizations/panels-visualizations/query-transform-data/expression-queries/
|
||||
---
|
||||
|
||||
# SQL expressions
|
||||
|
||||
{{< docs/public-preview product="SQL expressions" >}}
|
||||
|
||||
SQL Expressions are server-side expressions that manipulate and transform the results of data source queries using MySQL-like syntax. They allow you to easily query and transform your data after it has been queried, using SQL, which provides a familiar and powerful syntax that can handle everything from simple filters to highly complex, multi-step transformations.
|
||||
|
||||
In Grafana, a server-side expression is a way to transform or calculate data after it has been retrieved from the data source, but before it is sent to the frontend for visualization. Grafana evaluates these expressions on the server, not in the browser or at the data source.
|
||||
|
||||
For general information on Grafana expressions, refer to [Write expression queries](ref:expressions).
|
||||
|
||||
|
||||
|
||||
## Before you begin
|
||||
|
||||
- Enable SQL expressions under the feature toggle `sqlExpressions`.
|
||||
- If you self-host Grafana, you can find feature toggles in the configuration file `grafana.ini`.
|
||||
|
||||
```
|
||||
[feature_toggles]
|
||||
enable = sqlExpressions
|
||||
```
|
||||
|
||||
- If you are using Grafana Cloud, contact [Support](https://grafana.com/help/) to enable this feature.
|
||||
|
||||
## Transform data with SQL expressions
|
||||
|
||||
SQL expressions allow you to:
|
||||
|
||||
- Shape, transform, and modify query results without changing the original query.
|
||||
- JOIN data from multiple tables.
|
||||
- Create alerts or recording rules based on transformed data.
|
||||
- Perform final-stage modifications to datasets, including:
|
||||
- Show, hide, or rename columns.
|
||||
- Filter rows based on conditions.
|
||||
- Aggregate data (for example: sum, average, count).
|
||||
- Write subqueries and Common Table Expressions (CTEs) to support more complex logic:
|
||||
- **Subqueries** are nested queries used for filtering, calculations, or transformations.
|
||||
- **CTEs** are temporary named result sets that help make complex queries more readable and reusable.
|
||||
|
||||
A key capability of SQL expressions is the ability to JOIN data from multiple tables. This allows users to combine and transform data in a predictable, user-friendly way—even for complex use cases. You can JOIN data from an unlimited number of data source queries.
|
||||
|
||||
To work with SQL expressions, you must use data from a backend data source. In Grafana, a backend data source refers to a data source plugin or integration that communicates with a database, service, or API through the Grafana server, rather than directly from the browser (frontend).
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Currently, only one SQL expression is supported per panel or alert.
|
||||
- Grafana supports certain data sources. Refer to [compatible data sources](#compatible-data-sources) for a current list.
|
||||
- Autocomplete is available, but column/field autocomplete is only available after enabling the `sqlExpressionsColumnAutoComplete` feature toggle, which is provided on an experimental basis.
|
||||
|
||||
## Compatible data sources
|
||||
|
||||
The following are compatible data sources:
|
||||
|
||||
**Full support:** Grafana supports all query types for each of these data sources.
|
||||
|
||||
- Elasticsearch
|
||||
- MySQL
|
||||
- Loki
|
||||
- Graphite
|
||||
- Google Sheets
|
||||
- Amazon Athena
|
||||
|
||||
**Partial support:** The following data sources have limited or conditional support. Some support multiple query types depending on the service. For example, Azure Monitor can query multiple services, each with its own query format. In some cases, you can also switch the query type within a panel.
|
||||
|
||||
- InfluxDB
|
||||
- Infinity
|
||||
- Azure Monitor
|
||||
- TestData
|
||||
- Tempo
|
||||
- Prometheus
|
||||
- Cloudwatch
|
||||
- GitHub
|
||||
- BigQuery
|
||||
|
||||
## Create SQL expressions
|
||||
|
||||
To create a SQL expression, complete the following steps:
|
||||
|
||||
1. Navigate to **Dashboards** in the left-side menu.
|
||||
1. Select a dashboard and open a dashboard panel.
|
||||
1. Click the ellipsis in the upper right and select **Edit** .
|
||||
1. Click **+ Expression**.
|
||||
1. Select **SQL** from the drop-down.
|
||||
|
||||
After you have added a SQL expression, you can select from other data source queries by referencing the RefIDs of the queries in your SQL expression as if they were tables in a SQL database.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
The **RefID** is a unique identifier assigned to each query within a Grafana panel that serves as a reference name for that query's data.
|
||||
{{< /admonition >}}
|
||||
|
||||
|
||||
|
||||
## Workflow to build SQL expressions
|
||||
|
||||
Use the following workflow to create a SQL expression:
|
||||
|
||||
1. **Build your base queries.** Create the individual query and give it a meaningful name. Create the queries (A, B, etc.) that provide the data you want to combine or transform using SQL Expressions.
|
||||
1. **Hide your base queries.** Click the **👁️ Eye icon** next to each base query to hide them from visualization. This keeps your panel clean while still making the data available to the SQL Expression.
|
||||
1. **Switch to table view**. Set the panel visualization to **Table** to inspect and review the structure and output of your SQL expression as you build and refine it.
|
||||
1. **Add a SQL Expression**. Add a new query and add select SQL Expression as its type.
|
||||
**Inspect inputs**. Start with simple test queries to understand the shape of your input frames.
|
||||
|
||||
```sql
|
||||
SELECT * FROM A LIMIT 10.
|
||||
```
|
||||
|
||||
This lets you see the available columns and sample rows from `query A`. Repeat this for each input query you want to use (e.g., `SELECT * FROM B LIMIT 10`).
|
||||
|
||||
1. **Inspect your data**. Repeat this for each input query to understand the column structure and data types you're working with.
|
||||
|
||||
```sql
|
||||
SELECT * FROM <B, C, D, etc> LIMIT 10
|
||||
```
|
||||
|
||||
1. **Construct the SQL expression.** Once you understand your data, you can write your SQL expression to join, filter, or otherwise transform the data.
|
||||
1. **Validate and iterate**. Click **Refresh** every time you update your SQL query to re-evaluate and see the updated result.
|
||||
|
||||
When selecting a visualization type, **ensure your SQL expression returns data in the required shape**. For example, time series panels require a column with a time field (e.g., timestamp) and a numeric value column (e.g., \_\_value\_\_). If the output is not shaped correctly, your visualization may appear empty or fail to render.
|
||||
|
||||
The SQL expression workflow in Grafana is designed with the following behaviors:
|
||||
|
||||
- **Unhidden queries are visualized automatically.** If an input query is not hidden, Grafana will attempt to render it alongside your SQL expression. This can clutter the output, especially in table visualizations.
|
||||
|
||||
- **SQL expression results may not be immediately visible.** You might need to use the data frame selector (dropdown at the bottom of the table panel) to switch between the raw query and the SQL expression result.
|
||||
|
||||
- **Non-tabular or incorrectly shaped data will not render in certain panels.** Visualizations such as graphs or gauges require properly structured data. Mismatched formats will result in rendering issues or missing data.
|
||||
|
||||
## SQL conversion rules
|
||||
|
||||
When you reference a RefID within a SQL statement (e.g., `SELECT * FROM A`), the system invokes a distinct SQL conversion process.
|
||||
|
||||
The SQL conversion path:
|
||||
|
||||
- The query result appears as a single data frame, without labels, and is mapped directly to a tabular format.
|
||||
- If the frame type is present and is either numeric, wide time series, or multi-frame time series (for example: labeled formats), Grafana automatically converts the data into a table structure.
|
||||
|
||||
## Supported functions
|
||||
|
||||
Grafana maintains a complete list of supported SQL keywords, operators, and functions in the SQL expressions query validator implementation.
|
||||
|
||||
For the most up-to-date reference of all supported SQL functionality, refer to the `allowedNode` and `allowedFunction` definitions in the Grafana [codebase](https://github.com/grafana/grafana/blob/main/pkg/expr/sql/parser_allow.go).
|
||||
|
||||
## Alerting and recording rules
|
||||
|
||||
SQL expressions integrates alerting and recording rules, allowing you to define complex conditions and metrics using standard SQL queries. The system processes your query results and automatically creates alert instances or recorded metrics based on the returned data structure.
|
||||
|
||||
For SQL Expressions to work properly with alerting and recording rules, your query must return:
|
||||
|
||||
- One numeric column - **_required_**. This contains the value that triggers alerts or gets recorded.
|
||||
- Unique string column combinations - **_required_**. Each row must have a unique combination of string column values.
|
||||
- One or more string columns - _optional_. These become **labels** for the alert instances or metrics. Examples: `service`, `region`.
|
||||
|
||||
Consider the following query results:
|
||||
|
||||
```sql
|
||||
error_count,service,region
|
||||
25,auth-service,us-east
|
||||
0,payment-service,us-west
|
||||
15,user-service,eu-west
|
||||
```
|
||||
|
||||
This query returns:
|
||||
|
||||
- the numeric column `error_count` (values: 25, 0, 15)
|
||||
- the string columns `service` and `region`
|
||||
|
||||
For alert rules, this creates three alert instances:
|
||||
|
||||
- First instance with labels {service=auth-service, region=us-east} and value 25 (triggers alert - high error count)
|
||||
- Second instance with labels {service=payment-service, region=us-west} and value 0 (no alert - zero errors)
|
||||
- Third instance with labels {service=user-service, region=eu-west} and value 15 (triggers alert - elevated error count)
|
||||
|
||||
For recording rules, creates one metric with three series:
|
||||
|
||||
- First series: error_count_total{service=auth-service, region=us-east} 25
|
||||
- Second series: error_count_total{service=payment-service, region=us-west} 0
|
||||
- Third series: error_count_total{service=user-service, region=eu-west} 15
|
||||
|
||||
Following are some best practices for alerting and recording rules:
|
||||
|
||||
- Keep numeric values meaningful (for example: error counts, request duration).
|
||||
- Use clear, descriptive column names - these become your labels.
|
||||
- Keep string values short and consistent.
|
||||
- Avoid too many unique label combinations, as this can result in high cardinality.
|
||||
- Always use `GROUP BY` to avoid duplicate label errors.
|
||||
- Aggregate numeric values logically (for example: `SUM(error_count)`).
|
||||
|
||||
## Supported data source formats
|
||||
|
||||
Grafana supports three types of data source response formats:
|
||||
|
||||
1. **Single Table-like Frame**:
|
||||
This refers to data returned in a standard tabular structure, where all values are organized into rows and columns, similar to what you'd get from a SQL query.
|
||||
- **Example**: Any query against a SQL data source (e.g., PostgreSQL, MySQL) with the format set to Table.
|
||||
|
||||
2. **Dataplane: Time Series Format**:
|
||||
This format represents time series data with timestamps and associated values. It is typically returned from monitoring data sources.
|
||||
- **Example**: Prometheus or Loki Range Queries (queries that return a set of values over time).
|
||||
|
||||
3. **Dataplane: Numeric Long Format**:
|
||||
This format is used for point-in-time (instant) metric queries that return a single value (or a set of values) at a specific moment.
|
||||
- **Example**: Prometheus or Loki Instant Queries (queries that return the current value of a metric).
|
||||
|
||||
For more information on Dataplane formats, refer to [Grafana Dataplane Documentation](https://grafana.com/developers/dataplane).
|
||||
|
||||
The following non-tabular formats are automatically converted to a tabular format (`FullLong`) when used in SQL expressions:
|
||||
|
||||
- **Time Series Wide**: Label keys become column names.
|
||||
- **Time Series Multi**: Label values become the values in each row (or null if a label is missing).
|
||||
- **Numeric Wide**: The `value` column contains the numeric metric value.
|
||||
- **Numeric Multi**: If a display name exists, it will appear in the `display_name` column.
|
||||
|
||||
During conversion:
|
||||
|
||||
- Label keys become column names.
|
||||
- Label values populate the corresponding rows (null if a label is missing).
|
||||
- The `value` column contains the numeric metric.
|
||||
- If available, the `display_name` column contains a human-readable name.
|
||||
- The `metric_name` column stores the raw metric identifier.
|
||||
- For time series data, Grafana includes a `time` column with timestamps
|
||||
|
||||
## SQL expressions examples
|
||||
|
||||
1. Create the following Prometheus query:
|
||||
|
||||
```promql
|
||||
sum(
|
||||
rate(go_cpu_classes_gc_total_cpu_seconds_total{namespace=~".*(namespace).*5."}[$__rate_interval])
|
||||
) by (namespace)
|
||||
```
|
||||
|
||||
The panel displays the CPU usage by Go garbage collection (GC) over time, broken down by namespace.
|
||||
|
||||
|
||||
|
||||
2. Add the SQL expression `SELECT * from A`. After you add a SQL expression that selects from RefID A, Grafana converts it to a table response:
|
||||
|
||||
|
||||
|
||||
## LLM integration
|
||||
|
||||
The Grafana LLM plugin seamlessly integrates AI-powered assistance into your SQL expressions workflow.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
The Grafana LLM plugin is currently in public preview, meaning Grafana offers limited support, and breaking changes might occur prior to the feature being made generally available.
|
||||
{{< /admonition >}}
|
||||
|
||||
To use this integration, first [install and configure the LLM plugin](https://grafana.com/grafana/plugins/grafana-llm-app/). After installation, open your dashboard and select **Edit** to open the panel editor. Navigate to the **Queries** tab and scroll to the bottom where you'll find two new buttons positioned to the right of the **Run query** button in your SQL Expressions query.
|
||||
|
||||
{{< figure src="/media/docs/sql-expressions/sqlexpressions-LLM-integration-v12.2.png" caption="LLM integration" >}}
|
||||
|
||||
Click **Explain query** to open a drawer that displays a detailed explanation of your query, including its interpreted business meaning and performance statistics. Once the explanation is generated, the button changes to **View explanation**.
|
||||
|
||||
Click **Improve query** to open a suggestions drawer that contains performance and reliability enhancements, column naming best practices, and guidance on panel optimization. Click **Apply** to implement a suggestion. After you’ve interacted with the interface, you'll see a **Suggestions** button for quick access. Newer suggestions appear at the top, with older ones listed below, creating a history of improvements. If your SQL query has a parsing error, such as a syntax issue, the LLM will attempt to provide a corrected version. The LLM automatically identifies errors and helps you rewrite the query correctly.
|
||||
+1509
File diff suppressed because it is too large
Load Diff
+49
@@ -0,0 +1,49 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../../troubleshooting/troubleshoot-queries/ # /docs/grafana/next/troubleshooting/troubleshoot-queries/
|
||||
- ../../../panels-visualizations/query-transform-data/troubleshoot-queries/ # /docs/grafana/next/panels-visualizations/query-transform-data/troubleshoot-queries/
|
||||
description: Troubleshoot Grafana queries
|
||||
keywords:
|
||||
- grafana
|
||||
- troubleshooting
|
||||
- documentation
|
||||
- guide
|
||||
- queries
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Troubleshoot queries
|
||||
weight: 200
|
||||
refs:
|
||||
inspect:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/panel-inspector/#inspect-query-request-and-response-data
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/panels-visualizations/panel-inspector/#inspect-query-request-and-response-data
|
||||
---
|
||||
|
||||
# Troubleshoot queries
|
||||
|
||||
This page provides information to solve common dashboard problems.
|
||||
|
||||
## I get different results when I rearrange my functions
|
||||
|
||||
Function order is very important. Just like in math, the order that you place your functions can affect the result.
|
||||
|
||||
## Inspect your query request and response
|
||||
|
||||
The most common problems are related to the query and response from your data source. Even if it looks
|
||||
like a bug or visualization issue in Grafana, it is almost always a problem with the data source query or
|
||||
the data source response. Start by inspecting your panel query and response.
|
||||
|
||||
For more information, refer to [Inspect request and response data](ref:inspect).
|
||||
|
||||
## My query is slow
|
||||
|
||||
How many data points is your query returning? A query that returns lots of data points will be slow. Try this:
|
||||
|
||||
- In **Query options**, limit the **Max data points** returned.
|
||||
- In **Query options**, increase the **Min interval** time.
|
||||
- In your query, use a `group by` function.
|
||||
Reference in New Issue
Block a user