Docs: refactors config panels and visualizations, corrects relrefs (#55940)
* refactors config panels and visualizations, corrects relrefs * adds an alias * Remove some old content * moves visualizations topic to the root * moves out panels and visualization topics to the root * adds move and resize panel to add/organize panel; creates a create dashboard topic under build dashboards; adjusts context of add a panel to be from within an existing dashboard * updates aliases * creates search at root, moves dashboard preview to search, creates standalone search dashboard topic * moves Set dashboard time range to use-dashboards, creates modify dashboard settings and adds moves Modify dashboard time settings to that topic * moves existing query-options topic from working with panels to configure-panel-visualizations, moves panel time overrides and timeshift content to query options * Moving things to better category, fixing links, improving ordering * Move panel inspector to main panel topic * completes partial fix of relrefs * relref fixes con't * restructures remaining panels topics * more relref fixes * Minor fix * Minor tweak * finishes fixing relrefs Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
This commit is contained in:
co-authored by
Torkel Ödegaard
Jack Baldry
parent
f4a3400a9c
commit
414d536186
@@ -0,0 +1,177 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/panels/expressions/
|
||||
- /docs/grafana/latest/panels/inspect-panel/
|
||||
- /docs/grafana/latest/panels/queries/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/
|
||||
- /docs/grafana/latest/panels/share-query-results/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/about-queries/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/navigate-query-tab/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/add-a-query/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/manage-queries/
|
||||
- /docs/grafana/latest/panels-visualizations/manage-queries/
|
||||
- /docs/grafana/latest/panels/query-options/
|
||||
- /docs/grafana/latest/panels/reference-query-options/
|
||||
- /docs/grafana/latest/panels-visualizations/query-options/
|
||||
|
||||
title: Query and transform data
|
||||
weight: 200
|
||||
---
|
||||
|
||||
# Query and transform data
|
||||
|
||||
Data source queries return data that can then be transformed via transformations and then visualized by different types of visualizations. The query language and query builder UI depends on the data source type. Grafana supports many different types of data sources.
|
||||
|
||||
## About queries
|
||||
|
||||
_Queries_ are how Grafana panels communicate with data sources to get data for the visualization. A query is a question written in the query language used by the data source. How often the query is sent to the data source and how many data points are collected can be adjusted in the panel data source options.
|
||||
|
||||
Use you a query editor to write a query. Each data source has its own query editor that we have customized to include the features and capabilities of the data source. Grafana supports up to 26 queries per panel.
|
||||
|
||||
> Important! You must be familiar with the query language of the data source. For more information about data sources, refer to [Data sources](../../../datasources/).
|
||||
|
||||
### Query editors
|
||||
|
||||
Depending on your data source, the query editor might provide auto-completion, metric names, or variable suggestions.
|
||||
|
||||
Because of the difference between query languages, data sources have query editors that look different. Here are two examples of query editors.
|
||||
|
||||
**InfluxDB query editor**
|
||||
|
||||
{{< figure src="/static/img/docs/queries/influxdb-query-editor-7-2.png" class="docs-image--no-shadow" max-width="1000px" >}}
|
||||
|
||||
**Prometheus (PromQL) query editor**
|
||||
|
||||
{{< figure src="/static/img/docs/queries/prometheus-query-editor-7-4.png" class="docs-image--no-shadow" max-width="1000px" >}}
|
||||
|
||||
### Query syntax
|
||||
|
||||
Data sources use different query languages to return data. Here are two query examples:
|
||||
|
||||
**PostgreSQL**
|
||||
|
||||
```
|
||||
SELECT hostname FROM host WHERE region IN($region)
|
||||
```
|
||||
|
||||
**PromQL**
|
||||
|
||||
```
|
||||
query_result(max_over_time(<metric>[${__range_s}s]) != <state>)
|
||||
```
|
||||
|
||||
### Data sources used in queries
|
||||
|
||||
In addition to the data sources that you have configured in Grafana, there are three special data sources available:
|
||||
|
||||
- **Grafana -** A built-in data source that generates random walk data, which can be useful for testing visualizations and running experiments.
|
||||
- **Mixed -** Select this option to query multiple data sources in the same panel. When you select this data source, Grafana enables you to select a data source for every new query that you add.
|
||||
- The first query uses the data source that was selected before you selected **Mixed**.
|
||||
- You cannot change an existing query to use the Mixed Data Source.
|
||||
- **Dashboard -** Select this option to use a result set from another panel in the same dashboard.
|
||||
|
||||
You can combine data from multiple data sources onto a single dashboard, but each panel is tied to a specific data source that belongs to a particular Organization.
|
||||
|
||||
## Navigate the query tab
|
||||
|
||||
The Query tab consists of the following elements:
|
||||
|
||||
- Data source selector: Use the data source selector to select the source of the data you want to query. For more information about data sources, refer to [Data sources](../../../datasources/).
|
||||
- Query options: Enables you to set maximum data retrieved parameters and query execution time intervals.
|
||||
- Query inspector button: Open the query inspector panel where you can view and optimize your query.
|
||||
- Query editor list: Lists the queries you have written.
|
||||
- Expressions: Use the expression builder to create alert expressions. For more information about expressions, refer to [Use expressions to manipulate data](../use-expressions-to-manipulate-data/).
|
||||
|
||||
{{< figure src="/static/img/docs/queries/query-editor-7-2.png" class="docs-image--no-shadow" max-width="1000px" >}}
|
||||
|
||||
## Add a query
|
||||
|
||||
A query returns data that Grafana visualizes in dashboards. When you create a panel, Grafana automatically selects the default data source.
|
||||
|
||||
**To add a query**:
|
||||
|
||||
1. Edit the panel to which you are adding a query.
|
||||
1. Click the **Query** tab.
|
||||
1. Click the **Data source** drop-down menu and select a data source.
|
||||
1. Click **Query options** to configure the maximum number of data points you need.
|
||||
For more information about query options, refer to [Query options]({{< relref "#query-options" >}}).
|
||||
1. Write the query using the query editor.
|
||||
1. Click **Apply**.
|
||||
|
||||
The system queries the data source and presents the data in the visualization.
|
||||
|
||||
## Manage queries
|
||||
|
||||
Queries are organized in collapsible query rows. Each query row contains a query editor and is identified with a letter (A, B, C, and so on).
|
||||
|
||||
You can:
|
||||
|
||||
| Icon | Description |
|
||||
| :-----------------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| {{< figure src="/static/img/docs/queries/query-editor-help-7-4.png" class="docs-image--no-shadow" max-width="30px" max-height="30px" >}} | Toggle query editor help. If supported by the data source, click this icon to display information on how to use the query editor or provide quick access to common queries. |
|
||||
| {{< figure src="/static/img/docs/queries/duplicate-query-icon-7-0.png" class="docs-image--no-shadow" max-width="30px" max-height="30px" >}} | Copy a query. Duplicating queries is useful when working with multiple complex queries that are similar and you want to either experiment with different variants or do minor alterations. |
|
||||
| {{< figure src="/static/img/docs/queries/hide-query-icon-7-0.png" class="docs-image--no-shadow" max-width="30px" max-height="30px" >}} | Hide a query. Grafana does not send hidden queries to the data source. |
|
||||
| {{< figure src="/static/img/docs/queries/remove-query-icon-7-0.png" class="docs-image--no-shadow" max-width="30px" max-height="30px" >}} | Remove a query. Removing a query permanently deletes it, but sometimes you can recover deleted queries by reverting to previously saved versions of the panel. |
|
||||
| {{< figure src="/static/img/docs/queries/query-drag-icon-7-2.png" class="docs-image--no-shadow" max-width="30px" max-height="30px" >}} | Reorder queries. Change the order of queries by clicking and holding the drag icon, then drag queries where desired. The order of results reflects the order of the queries, so you can often adjust your visual results based on query order. |
|
||||
|
||||
## Query options
|
||||
|
||||
Click **Query options** next to the data source selector to see settings for your selected data source. Changes you make here affect only queries made in this panel.
|
||||
|
||||
{{< figure src="/static/img/docs/queries/data-source-options-7-0.png" class="docs-image--no-shadow" max-width="1000px" >}}
|
||||
|
||||
Grafana sets defaults that are shown in dark gray text. Changes are displayed in white text. To return a field to the default setting, delete the white text from the field.
|
||||
|
||||
Panel data source query options:
|
||||
|
||||
- **Max data points -** If the data source supports it, sets the maximum numbers of data points for each series returned. If the query returns more data points than the max data points setting, then the data source consolidates them (reduces the number of points returned by aggregating them together by average or max or other function).
|
||||
|
||||
There are two main reasons for limiting the number of points, performance and smoothing the line. The default value is the width (or number of pixels) of the graph as there is no point in having more data points than the graph panel can display.
|
||||
|
||||
With streaming data, the max data points value is used for the rolling buffer. (Streaming is a continuous flow of data and buffering is a way of dividing the stream into chunks). Loki streams data in the live tailing mode.
|
||||
|
||||
- **Min interval -** Sets a minimum limit for the automatically calculated interval, typically the minimum scrape interval. If a data point is saved every 15 seconds, then there's no point in having an interval lower than that. Another use case is to set it to a higher minimum than the scrape interval to get more coarse-grained, well-functioning queries.
|
||||
|
||||
- **Interval -** The interval is a time span that you can use when aggregating or grouping data points by time.
|
||||
|
||||
Grafana automatically calculates an appropriate interval and it can be used as a variable in templated queries. The variable is either in seconds: `$__interval` or in milliseconds: `$__interval_ms`. It is typically used in aggregation functions like sum or average. For example, a Prometheus query using the interval variable: `rate(http_requests_total[$__interval])`.
|
||||
|
||||
This automatic interval is calculated based on the width of the graph. If the user zooms out a lot then the interval becomes greater, resulting in a more coarse grained aggregation whereas if the user zooms in then the interval decreases resulting in a more fine grained aggregation.
|
||||
|
||||
For more information, refer to [Global variables]({{< relref "../../dashboards/variables/add-template-variables/#global-variables" >}}).
|
||||
|
||||
- **Relative time -** You can override the relative time range for individual panels, causing them to be different than what is selected in the dashboard time picker in the top right corner of the dashboard. This allows you to show metrics from different time periods or days on the same dashboard.
|
||||
|
||||
- **Time shift -** The time shift function is another way to override the time range for individual panels. It only works with relative time ranges and allows you to adjust the time range.
|
||||
|
||||
For example, you could shift the time range for the panel to be two hours earlier than the dashboard time picker. For more information, refer to [Time range controls]({{< relref "../../dashboards/manage-dashboards/#configure-dashboard-time-range-controls" >}}).
|
||||
|
||||
- **Cache timeout -** (This field is only visible if available in your data source.) If your time series store has a query cache, then this option can override the default cache timeout. Specified as a numeric value in seconds.
|
||||
|
||||
### Examples
|
||||
|
||||
- **Relative time:**
|
||||
|
||||
| Example | Relative time field |
|
||||
| ---------------- | ------------------- |
|
||||
| Last 5 minutes | `now-5m` |
|
||||
| The day so far | `now/d` |
|
||||
| Last 5 days | `now-5d/d` |
|
||||
| This week so far | `now/w` |
|
||||
| Last 2 years | `now-2y/y` |
|
||||
|
||||
- **Time shift:**
|
||||
|
||||
| Example | Time shift field |
|
||||
| -------------------- | ---------------- |
|
||||
| Last entire week | `1w/w` |
|
||||
| Two entire weeks ago | `2w/w` |
|
||||
| Last entire month | `1M/M` |
|
||||
| This entire year | `1d/y` |
|
||||
| Last entire year | `1y/y` |
|
||||
|
||||
### Panel time overrides and timeshift
|
||||
|
||||
In [Query options]({{< relref "#query-options" >}}), you can override the relative time range for individual panels, which causes them to be different than what is selected in the dashboard time picker located in the upper right. This enables you to show metrics from different time periods or days at the same time.
|
||||
|
||||
> **Note:** Panel time overrides have no effect when the time range for the dashboard is absolute.
|
||||
@@ -0,0 +1,223 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/panels/query-a-data-source/use-expressions-to-manipulate-data/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/use-expressions-to-manipulate-data/about-expressions/
|
||||
- /docs/grafana/latest/panels/query-a-data-source/use-expressions-to-manipulate-data/write-an-expression/
|
||||
- docs/grafana/latest/panels-visualizations/query-transform-data/expression-queries/
|
||||
title: Write expression queries
|
||||
menuTitle: Write expression queries
|
||||
weight: 40
|
||||
---
|
||||
|
||||
# Write expression queries
|
||||
|
||||
Server-side expressions enable you to manipulate data returned from queries with math and other operations. Expressions create new data and do not manipulate the data returned by data sources.
|
||||
|
||||
## About expressions
|
||||
|
||||
> **Note:** This documentation is for a beta feature.
|
||||
|
||||
Server-side expressions allow you to manipulate data returned from queries with math and other operations. Expressions create new data and do not manipulate the data returned by data sources, aside from some minor data restructuring to make the data acceptable input for expressions.
|
||||
|
||||
### Using expressions
|
||||
|
||||
Expressions are primarily used by [Grafana Alerting]({{< relref "../../../alerting/" >}}). The processing is done server-side, so expressions can operate without a browser session. However, expressions can also be used with backend data sources and visualization.
|
||||
|
||||
> **Note:** Expressions do not work with legacy dashboard alerts.
|
||||
|
||||
Expressions are meant to augment data sources by enabling queries from different data sources to be combined or by providing operations unavailable in a data source.
|
||||
|
||||
> **Note:** When possible, you should do data processing inside the data source. Copying data from storage to the Grafana server for processing is inefficient, so expressions are targeted at lightweight data processing.
|
||||
|
||||
Expressions work with data source queries that return time series or number data. They also operate on [multiple-dimensional data]({{< relref "../../../basics/timeseries-dimensions/" >}}). For example, a query that returns multiple series, where each series is identified by labels or tags.
|
||||
|
||||
An individual expression takes one or more queries or other expressions as input and adds data to the result. Each individual expression or query is represented by a variable that is a named identifier known as its RefID (e.g., the default letter `A` or `B`).
|
||||
|
||||
To reference the output of an individual expression or a data source query in another expression, this identifier is used as a variable.
|
||||
|
||||
### Types of expressions
|
||||
|
||||
Expressions work with two types of data.
|
||||
|
||||
- A collection of time series.
|
||||
- A collection of numbers, where each number is an item.
|
||||
|
||||
Each collection is returned from a single data source query or expression and represented by the RefID. Each collection is a set, where each item in the set is uniquely identified by its dimensions which are stored as [labels]({{< relref "../../../basics/timeseries-dimensions/#labels" >}}) or key-value pairs.
|
||||
|
||||
### Data source queries
|
||||
|
||||
Server-side expressions only support data source queries for backend data sources. The data is generally assumed to be labeled time series data. In the future we intended to add an assertion of the query return type (number or time series) data so expressions can handle errors better.
|
||||
|
||||
Data source queries, when used with expressions, are executed by the expression engine. When it does this, it restructures data to be either one time series or one number per data frame. So for example if using a data source that returns multiple series on one frame in the table view, you might notice it looks different when executed with expressions.
|
||||
|
||||
Currently, the only non-time series format (number) is supported when using data frames are you have a table response that returns a data frame with no time, string columns, and one number column:
|
||||
|
||||
| Loc | Host | Avg_CPU |
|
||||
| --- | ---- | ------- |
|
||||
| MIA | A | 1 |
|
||||
| NYC | B | 2 |
|
||||
|
||||
The example above will produce a number that works with expressions. The string columns become labels and the number column the corresponding value. For example `{"Loc": "MIA", "Host": "A"}` with a value of 1.
|
||||
|
||||
### Operations
|
||||
|
||||
You can use the following operations in expressions: math, reduce, and resample.
|
||||
|
||||
#### Math
|
||||
|
||||
Math is for free-form math formulas on time series or number data. Math operations take numbers and time series as input and changes them to different numbers and time series.
|
||||
|
||||
Data from other queries or expressions are referenced with the RefID prefixed with a dollar sign, for example `$A`. If the variable has spaces in the name, then you can use a brace syntax like `${my variable}`.
|
||||
|
||||
Numeric constants may be in decimal (`2.24`), octal (with a leading zero like `072`), or hex (with a leading 0x like `0x2A`). Exponentials and signs are also supported (e.g., `-0.8e-2`).
|
||||
|
||||
##### Operators
|
||||
|
||||
The arithmetic (`+`, binary and unary `-`, `*`, `/`, `%`, exponent `**`), relational (`<`, `>`, `==`, `!=`, `>=`, `<=`), and logical (`&&`, `||`, and unary `!`) operators are supported.
|
||||
|
||||
How the operation behaves with data depends on if it is a number or time series data.
|
||||
|
||||
With binary operations, such as `$A + $B` or `$A || $B`, the operator is applied in the following ways depending on the type of data:
|
||||
|
||||
- If both `$A` and `$B` are a number, then the operation is performed between the two numbers.
|
||||
- If one variable is a number, and the other variable is a time series, then the operation between the value of each point in the time series and the number is performed.
|
||||
- If both `$A` and `$B` are time series data, then the operation between each value in the two series is performed for each time stamp that exists in both `$A` and `$B`. The Resample operation can be used to line up time stamps. (**Note:** in the future, we plan to add options to the Math operation for different behaviors).
|
||||
|
||||
Summary:
|
||||
|
||||
- Number OP number = number
|
||||
- Number OP series = series
|
||||
- Series OP series = series
|
||||
|
||||
Because expressions work with multiple series or numbers represented by a single variable, binary operations also perform a union (join) between the two variables. This is done based on the identifying labels associated with each individual series or number.
|
||||
|
||||
So if you have numbers with labels like `{host=web01}` in `$A` and another number in `$B` with the same labels then the operation is performed between those two items within each variable, and the result will share the same labels. The rules for the behavior of this union are as follows:
|
||||
|
||||
- An item with no labels will join to anything.
|
||||
- If both `$A` and `$B` each contain only one item (one series, or one number), they will join.
|
||||
- If labels are exact math they will join.
|
||||
- If labels are a subset of the other, for example and item in `$A` is labeled `{host=A,dc=MIA}` and and item in `$B` is labeled `{host=A}` they will join.
|
||||
- Currently, if within a variable such as `$A` there are different tag _keys_ for each item, the join behavior is undefined.
|
||||
|
||||
The relational and logical operators return 0 for false 1 for true.
|
||||
|
||||
##### Math Functions
|
||||
|
||||
While most functions exist in the own expression operations, the math operation does have some functions that similar to math operators or symbols. When functions can take either numbers or series, than the same type as the argument will be returned. When it is a series, the operation of performed for the value of each point in the series.
|
||||
|
||||
###### abs
|
||||
|
||||
abs returns the absolute value of its argument which can be a number or a series. For example `abs(-1)` or `abs($A)`.
|
||||
|
||||
###### is_inf
|
||||
|
||||
is_inf takes a number or a series and returns `1` for `Inf` values (negative or positive) and `0` for other values. For example `is_inf($A)`.
|
||||
|
||||
> **Note:** If you need to specifically check for negative infinity for example, you can do a comparison like `$A == infn()`.
|
||||
|
||||
###### is_nan
|
||||
|
||||
is_nan takes a number or a series and returns `1` for `NaN` values and `0` for other values. For example `is_nan($A)`. This function exists because `NaN` is not equal to `NaN`.
|
||||
|
||||
###### is_null
|
||||
|
||||
is_null takes a number or a series and returns `1` for `null` values and `0` for other values. For example `is_null($A)`.
|
||||
|
||||
###### is_number
|
||||
|
||||
is_number takes a number or a series and returns `1` for all real number values and `0` for other values (which are `null`, `Inf+`, `Inf-`, and `NaN`). For example `is_number($A)`.
|
||||
|
||||
###### log
|
||||
|
||||
Log returns the natural logarithm of of its argument which can be a number or a series. If the value is less than 0, NaN is returned. For example `log(-1)` or `log($A)`.
|
||||
|
||||
###### inf, infn, nan, and null
|
||||
|
||||
The inf, infn, nan, and null functions all return a single value of the name. They primarily exist for testing. Example: `null()`.
|
||||
|
||||
###### round
|
||||
|
||||
Round returns a rounded integer value. For example, `round(3.123)` or `round($A)`. (This function should probably take an argument so it can add precision to the rounded value).
|
||||
|
||||
###### ceil
|
||||
|
||||
Ceil rounds the number up to the nearest integer value. For example, `ceil(3.123)` returns 4.
|
||||
|
||||
###### floor
|
||||
|
||||
Floor rounds the number down to the nearest integer value. For example, `floor(3.123)` returns 3.
|
||||
|
||||
#### Reduce
|
||||
|
||||
Reduce takes one or more time series returned from a query or an expression and turns each series into a single number. The labels of the time series are kept as labels on each outputted reduced number.
|
||||
|
||||
**Fields:**
|
||||
|
||||
- **Function -** The reduction function to use
|
||||
- **Input -** The variable (refID (such as `A`)) to resample
|
||||
- **Mode -** Allows control behavior of reduction function when a series contains non-numerical values (null, NaN, +\-Inf)
|
||||
|
||||
##### Reduction Functions
|
||||
|
||||
###### Count
|
||||
|
||||
Count returns the number of points in each series.
|
||||
|
||||
###### Mean
|
||||
|
||||
Mean returns the total of all values in each series divided by the number of points in that series. In `strict` mode if any values in the series are null or nan, or if the series is empty, NaN is returned.
|
||||
|
||||
###### Min and Max
|
||||
|
||||
Min and Max return the smallest or largest value in the series respectively. In `strict` mode if any values in the series are null or nan, or if the series is empty, NaN is returned.
|
||||
|
||||
###### Sum
|
||||
|
||||
Sum returns the total of all values in the series. If series is of zero length, the sum will be 0. In `strict` mode if there are any NaN or Null values in the series, NaN is returned.
|
||||
|
||||
##### Last
|
||||
|
||||
Last returns the last number in the series. If the series has no values then returns NaN.
|
||||
|
||||
##### Reduction Modes
|
||||
|
||||
###### Strict
|
||||
|
||||
In Strict mode the input series is processed as is. If any values in the series are non-numeric (null, NaN or +\-Inf), NaN is returned.
|
||||
|
||||
###### Drop Non-Numeric
|
||||
|
||||
In this mode all non-numeric values (null, NaN or +\-Inf) in the input series are filtered out before executing the reduction function.
|
||||
|
||||
###### Replace Non-Numeric
|
||||
|
||||
In this mode all non-numeric values are replaced by a pre-defined value.
|
||||
|
||||
#### Resample
|
||||
|
||||
Resample changes the time stamps in each time series to have a consistent time interval. The main use case is so you can resample time series that do not share the same timestamps so math can be performed between them. This can be done by resample each of the two series, and then in a Math operation referencing the resampled variables.
|
||||
|
||||
**Fields:**
|
||||
|
||||
- **Input -** The variable of time series data (refID (such as `A`)) to resample
|
||||
- **Resample to -** The duration of time to resample to, for example `10s`. Units may be `s` seconds, `m` for minutes, `h` for hours, `d` for days, `w` for weeks, and `y` of years.
|
||||
- **Downsample -** The reduction function to use when there are more than one data point per window sample. See the reduction operation for behavior details.
|
||||
- **Upsample -** The method to use to fill a window sample that has no data points.
|
||||
- **pad** fills with the last know value
|
||||
- **backfill** with next known value
|
||||
- **fillna** to fill empty sample windows with NaNs
|
||||
|
||||
## Write an expression
|
||||
|
||||
If your data source supports them, then Grafana displays the **Expression** button and shows any existing expressions in the query editor list.
|
||||
|
||||
For more information about expressions, refer to [About expressions]({{< relref "#about-expressions" >}}).
|
||||
|
||||
1. Open the panel.
|
||||
1. Below the query, click **Expression**.
|
||||
1. In the **Operation** field, select the type of expression you want to write.
|
||||
|
||||
For more information about expression operations, refer to [About expressions]({{< relref "#about-expressions" >}}).
|
||||
|
||||
1. Write the expression.
|
||||
1. Click **Apply**.
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/panels/query-a-data-source/share-query/
|
||||
- /docs/grafana/latest/panels-visualizations/query-transform-data/share-query/
|
||||
title: Share query results with another panel
|
||||
menuTitle: Share query results
|
||||
weight: 60
|
||||
---
|
||||
|
||||
# Share query results with another panel
|
||||
|
||||
Grafana let you use the query result from one panel for any other panel in the dashboard. Sharing query results across panels reduces the number of queries made to your data source, which can improve the performance of your dashboard.
|
||||
|
||||
The Dashboard data source lets you select a panel in your dashboard that contains the queries you want to share the results for. Instead of sending a separate query for each panel, Grafana sends one query and other panels use the query results to construct visualizations.
|
||||
|
||||
This strategy can drastically reduce the number of queries being made when you for example have several panels visualizing the same data.
|
||||
|
||||
1. [Create a dashboard]({{< relref "../../../dashboards/build-dashboards/create-dashboard/" >}}).
|
||||
1. Change the title to "Source panel". You'll use this panel as a source for the other panels.
|
||||
1. Define the [query]({{< relref "../#add-a-query" >}}) or queries that you want share.
|
||||
|
||||
If you don't have a data source available, use the **Grafana** data source, which returns a random time series that you can use for testing.
|
||||
|
||||
1. Add a second panel and select the **Dashboard** data source in the query editor.
|
||||
1. In the **Use results from panel list**, select the first panel you created.
|
||||
|
||||
All queries defined in the source panel are now available to the new panel. Queries made in the source panel can be shared with multiple panels.
|
||||
|
||||
You can click on any of the queries to go to the panel where they are defined.
|
||||
@@ -0,0 +1,703 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/panels/transform-data/
|
||||
- /docs/grafana/latest/panels/transformations/
|
||||
- /docs/grafana/latest/panels/transformations/apply-transformations/
|
||||
- /docs/grafana/latest/panels/transformations/config-from-query/
|
||||
- /docs/grafana/latest/panels/transformations/rows-to-fields/
|
||||
- /docs/grafana/latest/panels/transform-data/about-transformation/
|
||||
- /docs/grafana/latest/panels/transform-data/add-transformation-to-data/
|
||||
- /docs/grafana/latest/panels/transform-data/apply-transformation-to-data/
|
||||
- /docs/grafana/latest/panels/transform-data/debug-transformation/
|
||||
- /docs/grafana/latest/panels/transform-data/delete-transformation/
|
||||
- /docs/grafana/latest/panels/reference-transformation-functions/
|
||||
- /docs/grafana/latest/panels/transform-data/transformation-functions/
|
||||
- /docs/grafana/latest/panels/transformations/types-options/
|
||||
- /docs/grafana/latest/panels-visualizations/query-transform-data/transform-data/
|
||||
title: Transform data
|
||||
weight: 100
|
||||
---
|
||||
|
||||
# Transform data
|
||||
|
||||
Transformations are a powerful way to manipulate data returned by a query before the system applies a visualization. Using transformations, you can:
|
||||
|
||||
- Rename fields
|
||||
- Join time series data
|
||||
- Perform mathematical operations across queries
|
||||
- Use the output of one transformation as the input to another transformation
|
||||
|
||||
For users that rely on multiple views of the same dataset, transformations offer an efficient method of creating and maintaining numerous dashboards.
|
||||
|
||||
You can also use the output of one transformation as the input to another transformation, which results in a performance gain.
|
||||
|
||||
> Sometimes the system cannot graph transformed data. When that happens, click the `Table view` toggle above the visualization to switch to a table view of the data. This can help you understand the final result of your transformations.
|
||||
|
||||
## Transformation types
|
||||
|
||||
Grafana provides a number of ways that you can transform data. For a complete list of transformations, refer to [Transformation functions]({{< relref "#transformation-functions" >}}).
|
||||
|
||||
## Order of transformations
|
||||
|
||||
When there are multiple transformations, Grafana applies them in the order they are listed. Each transformation creates a result set that then passes on to the next transformation in the processing pipeline.
|
||||
|
||||
The order in which Grafana applies transformations directly impacts the results. For example, if you use a Reduce transformation to condense all the results of one column into a single value, then you can only apply transformations to that single value.
|
||||
|
||||
## Add a transformation function to data
|
||||
|
||||
The following steps guide you in adding a transformation to data. This documentation does not include steps for each type of transformation. For a complete list of transformations, refer to [Transformation functions]({{< relref "#transformation-functions" >}}).
|
||||
|
||||
1. Navigate to the panel where you want to add one or more transformations.
|
||||
1. Click the panel title and then click **Edit**.
|
||||
1. Click the **Transform** tab.
|
||||
1. Click a transformation.
|
||||
A transformation row appears where you configure the transformation options. For more information about how to configure a transformation, refer to [Transformation functions]({{< relref "#transformation-functions" >}}).
|
||||
For information about available calculations, refer to [Calculation types]({{< relref "../../calculation-types" >}}).
|
||||
1. To apply another transformation, click **Add transformation**.
|
||||
This transformation acts on the result set returned by the previous transformation.
|
||||
{{< figure src="/static/img/docs/transformations/transformations-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
## Debug a transformation
|
||||
|
||||
To see the input and the output result sets of the transformation, click the bug icon on the right side of the transformation row.
|
||||
|
||||
The input and output results sets can help you debug a transformation.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/debug-transformations-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
## Delete a transformation
|
||||
|
||||
We recommend that you remove transformations that you don't need. When you delete a transformation, you remove the data from the visualization.
|
||||
|
||||
**Before you begin:**
|
||||
|
||||
- Identify all dashboards that rely on the transformation and inform impacted dashboard users.
|
||||
|
||||
**To delete a transformation**:
|
||||
|
||||
1. Open a panel for editing.
|
||||
1. Click the **Transform** tab.
|
||||
1. Click the trash icon next to the transformation you want to delete.
|
||||
|
||||
## Transformation functions
|
||||
|
||||
You can perform the following transformations on your data.
|
||||
|
||||
### Add field from calculation
|
||||
|
||||
Use this transformation to add a new field calculated from two other fields. Each transformation allows you to add one new field.
|
||||
|
||||
- **Mode -** Select a mode:
|
||||
- **Reduce row -** Apply selected calculation on each row of selected fields independently.
|
||||
- **Binary option -** Apply basic math operation(sum, multiply, etc) on values in a single row from two selected fields.
|
||||
- **Field name -** Select the names of fields you want to use in the calculation for the new field.
|
||||
- **Calculation -** If you select **Reduce row** mode, then the **Calculation** field appears. Click in the field to see a list of calculation choices you can use to create the new field. For information about available calculations, refer to [Calculation types]({{< relref "../../calculation-types" >}}).
|
||||
- **Operation -** If you select **Binary option** mode, then the **Operation** fields appear. These fields allow you to do basic math operations on values in a single row from two selected fields. You can also use numerical values for binary operations.
|
||||
- **Alias -** (Optional) Enter the name of your new field. If you leave this blank, then the field will be named to match the calculation.
|
||||
- **Replace all fields -** (Optional) Select this option if you want to hide all other fields and display only your calculated field in the visualization.
|
||||
|
||||
In the example below, I added two fields together and named them Sum.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/add-field-from-calc-stat-example-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
### Concatenate fields
|
||||
|
||||
This transformation combines all fields from all frames into one result. Consider:
|
||||
|
||||
Query A:
|
||||
|
||||
| Temp | Uptime |
|
||||
| ---- | ------- |
|
||||
| 15.4 | 1230233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| AQI | Errors |
|
||||
| --- | ------ |
|
||||
| 3.2 | 5 |
|
||||
|
||||
After you concatenate the fields, the data frame would be:
|
||||
|
||||
| Temp | Uptime | AQI | Errors |
|
||||
| ---- | ------- | --- | ------ |
|
||||
| 15.4 | 1230233 | 3.2 | 5 |
|
||||
|
||||
### Config from query results
|
||||
|
||||
This transformation allow you to select one query and from it extract standard options like **Min**, **Max**, **Unit** and **Thresholds** and apply it to other query results. This enables dynamic query driven visualization configuration.
|
||||
|
||||
If you want to extract a unique config for every row in the config query result then try the rows to fields transformation.
|
||||
|
||||
#### Options
|
||||
|
||||
- **Config query**: Select the query that returns the data you want to use as configuration.
|
||||
- **Apply to**: Select what fields or series to apply the configuration to.
|
||||
- **Apply to options**: Usually a field type or field name regex depending on what option you selected in **Apply to**.
|
||||
|
||||
### Convert field type
|
||||
|
||||
This transformation changes the field type of the specified field.
|
||||
|
||||
- **Field -** Select from available fields
|
||||
- **as -** Select the FieldType to convert to
|
||||
- **Numeric -** attempts to make the values numbers
|
||||
- **String -** will make the values strings
|
||||
- **Time -** attempts to parse the values as time
|
||||
- Will show an option to specify a DateFormat as input by a string like yyyy-mm-dd or DD MM YYYY hh:mm:ss
|
||||
- **Boolean -** will make the values booleans
|
||||
|
||||
For example the following query could be modified by selecting the time field, as Time, and Date Format as YYYY.
|
||||
|
||||
| Time | Mark | Value |
|
||||
| ---------- | ----- | ----- |
|
||||
| 2017-07-01 | above | 25 |
|
||||
| 2018-08-02 | below | 22 |
|
||||
| 2019-09-02 | below | 29 |
|
||||
| 2020-10-04 | above | 22 |
|
||||
|
||||
The result:
|
||||
|
||||
| Time | Mark | Value |
|
||||
| ------------------- | ----- | ----- |
|
||||
| 2017-01-01 00:00:00 | above | 25 |
|
||||
| 2018-01-01 00:00:00 | below | 22 |
|
||||
| 2019-01-01 00:00:00 | below | 29 |
|
||||
| 2020-01-01 00:00:00 | above | 22 |
|
||||
|
||||
### Filter data by name
|
||||
|
||||
Use this transformation to remove portions of the query results.
|
||||
|
||||
Grafana displays the **Identifier** field, followed by the fields returned by your query.
|
||||
|
||||
You can apply filters in one of two ways:
|
||||
|
||||
- Enter a regex expression.
|
||||
- Click a field to toggle filtering on that field. Filtered fields are displayed with dark gray text, unfiltered fields have white text.
|
||||
|
||||
In the example below, I removed the Min field from the results.
|
||||
|
||||
Here is the original query table. (This is streaming data, so numbers change over time and between screenshots.)
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/filter-name-table-before-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
Here is the table after I applied the transformation to remove the Min field.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/filter-name-table-after-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
Here is the same query using a Stat visualization.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/filter-name-stat-after-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
### Filter data by query
|
||||
|
||||
Use this transformation in panels that have multiple queries, if you want to hide one or more of the queries.
|
||||
|
||||
Grafana displays the query identification letters in dark gray text. Click a query identifier to toggle filtering. If the query letter is white, then the results are displayed. If the query letter is dark, then the results are hidden.
|
||||
|
||||
In the example below, the panel has three queries (A, B, C). I removed the B query from the visualization.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/filter-by-query-stat-example-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
> **Note:** This transformation is not available for Graphite because this data source does not support correlating returned data with queries.
|
||||
|
||||
### Filter data by value
|
||||
|
||||
This transformation allows you to filter your data directly in Grafana and remove some data points from your query result. You have the option to include or exclude data that match one or more conditions you define. The conditions are applied on a selected field.
|
||||
|
||||
This transformation is very useful if your data source does not natively filter by values. You might also use this to narrow values to display if you are using a shared query.
|
||||
|
||||
The available conditions for all fields are:
|
||||
|
||||
- **Regex:** Match a regex expression
|
||||
- **Is Null:** Match if the value is null
|
||||
- **Is Not Null:** Match if the value is not null
|
||||
- **Equal:** Match if the value is equal to the specified value
|
||||
- **Different:** match if the value is different than the specified value
|
||||
|
||||
The available conditions for number fields are:
|
||||
|
||||
- **Greater:** Match if the value is greater than the specified value
|
||||
- **Lower:** Match if the value is lower than the specified value
|
||||
- **Greater or equal:** Match if the value is greater or equal
|
||||
- **Lower or equal:** Match if the value is lower or equal
|
||||
- **Range:** Match a range between a specified minimum and maximum, min and max included
|
||||
|
||||
Consider the following data set:
|
||||
|
||||
| Time | Temperature | Altitude |
|
||||
| ------------------- | ----------- | -------- |
|
||||
| 2020-07-07 11:34:23 | 32 | 101 |
|
||||
| 2020-07-07 11:34:22 | 28 | 125 |
|
||||
| 2020-07-07 11:34:21 | 26 | 110 |
|
||||
| 2020-07-07 11:34:20 | 23 | 98 |
|
||||
| 2020-07-07 10:32:24 | 31 | 95 |
|
||||
| 2020-07-07 10:31:22 | 20 | 85 |
|
||||
| 2020-07-07 09:30:57 | 19 | 101 |
|
||||
|
||||
If you **Include** the data points that have a temperature below 30°C, the configuration will look as follows:
|
||||
|
||||
- Filter Type: `Include`
|
||||
- Condition: Rows where `Temperature` matches `Lower Than` `30`
|
||||
|
||||
And you will get the following result, where only the temperatures below 30°C are included:
|
||||
|
||||
| Time | Temperature | Altitude |
|
||||
| ------------------- | ----------- | -------- |
|
||||
| 2020-07-07 11:34:22 | 28 | 125 |
|
||||
| 2020-07-07 11:34:21 | 26 | 110 |
|
||||
| 2020-07-07 11:34:20 | 23 | 98 |
|
||||
| 2020-07-07 10:31:22 | 20 | 85 |
|
||||
| 2020-07-07 09:30:57 | 19 | 101 |
|
||||
|
||||
You can add more than one condition to the filter. For example, you might want to include the data only if the altitude is greater than 100. To do so, add that condition to the following configuration:
|
||||
|
||||
- Filter type: `Include` rows that `Match All` conditions
|
||||
- Condition 1: Rows where `Temperature` matches `Lower` than `30`
|
||||
- Condition 2: Rows where `Altitude` matches `Greater` than `100`
|
||||
|
||||
When you have more than one condition, you can choose if you want the action (include / exclude) to be applied on rows that **Match all** conditions or **Match any** of the conditions you added.
|
||||
|
||||
In the example above we chose **Match all** because we wanted to include the rows that have a temperature lower than 30 _AND_ an altitude higher than 100. If we wanted to include the rows that have a temperature lower than 30 _OR_ an altitude higher than 100 instead, then we would select **Match any**. This would include the first row in the original data, which has a temperature of 32°C (does not match the first condition) but an altitude of 101 (which matches the second condition), so it is included.
|
||||
|
||||
Conditions that are invalid or incompletely configured are ignored.
|
||||
|
||||
### Group by
|
||||
|
||||
This transformation groups the data by a specified field (column) value and processes calculations on each group. Click to see a list of calculation choices. For information about available calculations, refer to [Calculation types]({{< relref "../../calculation-types" >}}).
|
||||
|
||||
Here's an example of original data.
|
||||
|
||||
| Time | Server ID | CPU Temperature | Server Status |
|
||||
| ------------------- | --------- | --------------- | ------------- |
|
||||
| 2020-07-07 11:34:20 | server 1 | 80 | Shutdown |
|
||||
| 2020-07-07 11:34:20 | server 3 | 62 | OK |
|
||||
| 2020-07-07 10:32:20 | server 2 | 90 | Overload |
|
||||
| 2020-07-07 10:31:22 | server 3 | 55 | OK |
|
||||
| 2020-07-07 09:30:57 | server 3 | 62 | Rebooting |
|
||||
| 2020-07-07 09:30:05 | server 2 | 88 | OK |
|
||||
| 2020-07-07 09:28:06 | server 1 | 80 | OK |
|
||||
| 2020-07-07 09:25:05 | server 2 | 88 | OK |
|
||||
| 2020-07-07 09:23:07 | server 1 | 86 | OK |
|
||||
|
||||
This transformation goes in two steps. First you specify one or multiple fields to group the data by. This will group all the same values of those fields together, as if you sorted them. For instance if we group by the Server ID field, then it would group the data this way:
|
||||
|
||||
| Time | Server ID | CPU Temperature | Server Status |
|
||||
| ------------------- | -------------- | --------------- | ------------- |
|
||||
| 2020-07-07 11:34:20 | **server 1** | 80 | Shutdown |
|
||||
| 2020-07-07 09:28:06 | **server 1** | 80 | OK |
|
||||
| 2020-07-07 09:23:07 | **server 1** | 86 | OK |
|
||||
| 2020-07-07 10:32:20 | server 2 | 90 | Overload |
|
||||
| 2020-07-07 09:30:05 | server 2 | 88 | OK |
|
||||
| 2020-07-07 09:25:05 | server 2 | 88 | OK |
|
||||
| 2020-07-07 11:34:20 | **_server 3_** | 62 | OK |
|
||||
| 2020-07-07 10:31:22 | **_server 3_** | 55 | OK |
|
||||
| 2020-07-07 09:30:57 | **_server 3_** | 62 | Rebooting |
|
||||
|
||||
All rows with the same value of Server ID are grouped together.
|
||||
|
||||
After choosing which field you want to group your data by, you can add various calculations on the other fields, and apply the calculation to each group of rows. For instance, we could want to calculate the average CPU temperature for each of those servers. So we can add the _mean_ calculation applied on the CPU Temperature field to get the following:
|
||||
|
||||
| Server ID | CPU Temperature (mean) |
|
||||
| --------- | ---------------------- |
|
||||
| server 1 | 82 |
|
||||
| server 2 | 88.6 |
|
||||
| server 3 | 59.6 |
|
||||
|
||||
And we can add more than one calculation. For instance:
|
||||
|
||||
- For field Time, we can calculate the _Last_ value, to know when the last data point was received for each server
|
||||
- For field Server Status, we can calculate the _Last_ value to know what is the last state value for each server
|
||||
- For field Temperature, we can also calculate the _Last_ value to know what is the latest monitored temperature for each server
|
||||
|
||||
We would then get :
|
||||
|
||||
| Server ID | CPU Temperature (mean) | CPU Temperature (last) | Time (last) | Server Status (last) |
|
||||
| --------- | ---------------------- | ---------------------- | ------------------- | -------------------- |
|
||||
| server 1 | 82 | 80 | 2020-07-07 11:34:20 | Shutdown |
|
||||
| server 2 | 88.6 | 90 | 2020-07-07 10:32:20 | Overload |
|
||||
| server 3 | 59.6 | 62 | 2020-07-07 11:34:20 | OK |
|
||||
|
||||
This transformation enables you to extract key information from your time series and display it in a convenient way.
|
||||
|
||||
### Join by field
|
||||
|
||||
Use this transformation to join multiple results into a single table. This is especially useful for converting multiple
|
||||
time series results into a single wide table with a shared time field.
|
||||
|
||||
#### Inner join
|
||||
|
||||
An inner join merges data from multiple tables where all tables share the same value from the selected field. This type of join excludes
|
||||
data where values do not match in every result.
|
||||
|
||||
Use this transformation to combine the results from multiple queries (combining on a passed join field or the first time column) into one result, and drop rows where a successful join cannot occur.
|
||||
|
||||
In the following example, two queries return table data. It is visualized as two separate tables before applying the inner join transformation.
|
||||
|
||||
Query A:
|
||||
|
||||
| Time | Job | Uptime |
|
||||
| ------------------- | ------- | --------- |
|
||||
| 2020-07-07 11:34:20 | node | 25260122 |
|
||||
| 2020-07-07 11:24:20 | postgre | 123001233 |
|
||||
| 2020-07-07 11:14:20 | postgre | 345001233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| Time | Server | Errors |
|
||||
| ------------------- | -------- | ------ |
|
||||
| 2020-07-07 11:34:20 | server 1 | 15 |
|
||||
| 2020-07-07 11:24:20 | server 2 | 5 |
|
||||
| 2020-07-07 11:04:20 | server 3 | 10 |
|
||||
|
||||
The result after applying the inner join transformation looks like the following:
|
||||
|
||||
| Time | Job | Uptime | Server | Errors |
|
||||
| ------------------- | ------- | --------- | -------- | ------ |
|
||||
| 2020-07-07 11:34:20 | node | 25260122 | server 1 | 15 |
|
||||
| 2020-07-07 11:24:20 | postgre | 123001233 | server 2 | 5 |
|
||||
|
||||
#### Outer join
|
||||
|
||||
An outer join includes all data from an inner join and rows where values do not match in every input.
|
||||
|
||||
Use this transformation to combine the results from multiple queries (combining on a passed join field or the first time column) into one result, and drop rows where a successful join cannot occur - performing an inner join.
|
||||
|
||||
In the following example, two queries return table data. It is visualized as two tables before applying the inner join transformation.
|
||||
|
||||
Query A:
|
||||
|
||||
| Time | Job | Uptime |
|
||||
| ------------------- | ------- | --------- |
|
||||
| 2020-07-07 11:34:20 | node | 25260122 |
|
||||
| 2020-07-07 11:24:20 | postgre | 123001233 |
|
||||
| 2020-07-07 11:14:20 | postgre | 345001233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| Time | Server | Errors |
|
||||
| ------------------- | -------- | ------ |
|
||||
| 2020-07-07 11:34:20 | server 1 | 15 |
|
||||
| 2020-07-07 11:24:20 | server 2 | 5 |
|
||||
| 2020-07-07 11:04:20 | server 3 | 10 |
|
||||
|
||||
The result after applying the inner join transformation looks like the following:
|
||||
|
||||
| Time | Job | Uptime | Server | Errors |
|
||||
| ------------------- | ------- | --------- | -------- | ------ |
|
||||
| 2020-07-07 11:34:20 | node | 25260122 | server 1 | 15 |
|
||||
| 2020-07-07 11:24:20 | postgre | 123001233 | server 2 | 5 |
|
||||
|
||||
In the following example, a template query displays time series data from multiple servers in a table visualization. The results of only one query can be viewed at a time.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/join-fields-before-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
I applied a transformation to join the query results using the time field. Now I can run calculations, combine, and organize the results in this new table.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/join-fields-after-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
### Labels to fields
|
||||
|
||||
This transformation changes time series results that include labels or tags into a table where each label keys and values are included in the table result. The labels can be displayed either as columns or as row values.
|
||||
|
||||
Given a query result of two time series:
|
||||
|
||||
- Series 1: labels Server=Server A, Datacenter=EU
|
||||
- Series 2: labels Server=Server B, Datacenter=EU
|
||||
|
||||
In "Columns" mode, the result looks like this:
|
||||
|
||||
| Time | Server | Datacenter | Value |
|
||||
| ------------------- | -------- | ---------- | ----- |
|
||||
| 2020-07-07 11:34:20 | Server A | EU | 1 |
|
||||
| 2020-07-07 11:34:20 | Server B | EU | 2 |
|
||||
|
||||
In "Rows" mode, the result has a table for each series and show each label value like this:
|
||||
|
||||
| label | value |
|
||||
| ---------- | -------- |
|
||||
| Server | Server A |
|
||||
| Datacenter | EU |
|
||||
|
||||
| label | value |
|
||||
| ---------- | -------- |
|
||||
| Server | Server B |
|
||||
| Datacenter | EU |
|
||||
|
||||
#### Value field name
|
||||
|
||||
If you selected Server as the **Value field name**, then you would get one field for every value of the Server label.
|
||||
|
||||
| Time | Datacenter | Server A | Server B |
|
||||
| ------------------- | ---------- | -------- | -------- |
|
||||
| 2020-07-07 11:34:20 | EU | 1 | 2 |
|
||||
|
||||
#### Merging behavior
|
||||
|
||||
The labels to fields transformer is internally two separate transformations. The first acts on single series and extracts labels to fields. The second is the [merge](#merge) transformation that joins all the results into a single table. The merge transformation tries to join on all matching fields. This merge step is required and cannot be turned off.
|
||||
|
||||
To illustrate this, here is an example where you have two queries that return time series with no overlapping labels.
|
||||
|
||||
- Series 1: labels Server=ServerA
|
||||
- Series 2: labels Datacenter=EU
|
||||
|
||||
This will first result in these two tables:
|
||||
|
||||
| Time | Server | Value |
|
||||
| ------------------- | ------- | ----- |
|
||||
| 2020-07-07 11:34:20 | ServerA | 10 |
|
||||
|
||||
| Time | Datacenter | Value |
|
||||
| ------------------- | ---------- | ----- |
|
||||
| 2020-07-07 11:34:20 | EU | 20 |
|
||||
|
||||
After merge:
|
||||
|
||||
| Time | Server | Value | Datacenter |
|
||||
| ------------------- | ------- | ----- | ---------- |
|
||||
| 2020-07-07 11:34:20 | ServerA | 10 | |
|
||||
| 2020-07-07 11:34:20 | | 20 | EU |
|
||||
|
||||
### Merge
|
||||
|
||||
Use this transformation to combine the result from multiple queries into one single result. This is helpful when using the table panel visualization. Values that can be merged are combined into the same row. Values are mergeable if the shared fields contain the same data. For information, refer to [Table panel]({{< relref "../../visualizations/table/" >}}).
|
||||
|
||||
In the example below, we have two queries returning table data. It is visualized as two separate tables before applying the transformation.
|
||||
|
||||
Query A:
|
||||
|
||||
| Time | Job | Uptime |
|
||||
| ------------------- | ------- | --------- |
|
||||
| 2020-07-07 11:34:20 | node | 25260122 |
|
||||
| 2020-07-07 11:24:20 | postgre | 123001233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| Time | Job | Errors |
|
||||
| ------------------- | ------- | ------ |
|
||||
| 2020-07-07 11:34:20 | node | 15 |
|
||||
| 2020-07-07 11:24:20 | postgre | 5 |
|
||||
|
||||
Here is the result after applying the Merge transformation.
|
||||
|
||||
| Time | Job | Errors | Uptime |
|
||||
| ------------------- | ------- | ------ | --------- |
|
||||
| 2020-07-07 11:34:20 | node | 15 | 25260122 |
|
||||
| 2020-07-07 11:24:20 | postgre | 5 | 123001233 |
|
||||
|
||||
### Organize fields
|
||||
|
||||
Use this transformation to rename, reorder, or hide fields returned by the query.
|
||||
|
||||
> **Note:** This transformation only works in panels with a single query. If your panel has multiple queries, then you must either apply an Outer join transformation or remove the extra queries.
|
||||
|
||||
Grafana displays a list of fields returned by the query. You can:
|
||||
|
||||
- Change field order by hovering your cursor over a field. The cursor turns into a hand and then you can drag the field to its new place.
|
||||
- Hide or show a field by clicking the eye icon next to the field name.
|
||||
- Rename fields by typing a new name in the **Rename <field>** box.
|
||||
|
||||
In the example below, I hid the value field and renamed Max and Min.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/organize-fields-stat-example-7-0.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
### Reduce
|
||||
|
||||
The _Reduce_ transformation applies a calculation to each field in the frame and return a single value. Time fields are removed when applying this transformation.
|
||||
|
||||
Consider the input:
|
||||
|
||||
Query A:
|
||||
|
||||
| Time | Temp | Uptime |
|
||||
| ------------------- | ---- | ------- |
|
||||
| 2020-07-07 11:34:20 | 12.3 | 256122 |
|
||||
| 2020-07-07 11:24:20 | 15.4 | 1230233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| Time | AQI | Errors |
|
||||
| ------------------- | --- | ------ |
|
||||
| 2020-07-07 11:34:20 | 6.5 | 15 |
|
||||
| 2020-07-07 11:24:20 | 3.2 | 5 |
|
||||
|
||||
The reduce transformer has two modes:
|
||||
|
||||
- **Series to rows -** Creates a row for each field and a column for each calculation.
|
||||
- **Reduce fields -** Keeps the existing frame structure, but collapses each field into a single value.
|
||||
|
||||
For example, if you used the **First** and **Last** calculation with a **Series to rows** transformation, then
|
||||
the result would be:
|
||||
|
||||
| Field | First | Last |
|
||||
| ------ | ------ | ------- |
|
||||
| Temp | 12.3 | 15.4 |
|
||||
| Uptime | 256122 | 1230233 |
|
||||
| AQI | 6.5 | 3.2 |
|
||||
| Errors | 15 | 5 |
|
||||
|
||||
The **Reduce fields** with the **Last** calculation,
|
||||
results in two frames, each with one row:
|
||||
|
||||
Query A:
|
||||
|
||||
| Temp | Uptime |
|
||||
| ---- | ------- |
|
||||
| 15.4 | 1230233 |
|
||||
|
||||
Query B:
|
||||
|
||||
| AQI | Errors |
|
||||
| --- | ------ |
|
||||
| 3.2 | 5 |
|
||||
|
||||
### Rename by regex
|
||||
|
||||
Use this transformation to rename parts of the query results using a regular expression and replacement pattern.
|
||||
|
||||
You can specify a regular expression, which is only applied to matches, along with a replacement pattern that support back references. For example, let's imagine you're visualizing CPU usage per host and you want to remove the domain name. You could set the regex to `([^\.]+)\..+` and the replacement pattern to `$1`, `web-01.example.com` would become `web-01`.
|
||||
|
||||
In the following example, we are stripping the prefix from event types. In the before image, you can see everything is prefixed with `system.`
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/rename-by-regex-before-7-3.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
With the transformation applied, you can see we are left with just the remainder of the string.
|
||||
|
||||
{{< figure src="/static/img/docs/transformations/rename-by-regex-after-7-3.png" class="docs-image--no-shadow" max-width= "1100px" >}}
|
||||
|
||||
### Rows to fields
|
||||
|
||||
The rows to fields transformation converts rows into separate fields. This can be useful as fields can be styled and configured individually. It can also use additional fields as sources for dynamic field configuration or map them to field labels. The additional labels can then be used to define better display names for the resulting fields.
|
||||
|
||||
This transformation includes a field table which lists all fields in the data returned by the config query. This table gives you control over what field should be mapped to each config property (the \*Use as\*\* option). You can also choose which value to select if there are multiple rows in the returned data.
|
||||
|
||||
This transformation requires:
|
||||
|
||||
- One field to use as the source of field names.
|
||||
|
||||
By default, the transform uses the first string field as the source. You can override this default setting by selecting **Field name** in the **Use as** column for the field you want to use instead.
|
||||
|
||||
- One field to use as the source of values.
|
||||
|
||||
By default, the transform uses the first number field as the source. But you can override this default setting by selecting **Field value** in the **Use as** column for the field you want to use instead.
|
||||
|
||||
Useful when visualizing data in:
|
||||
|
||||
- Gauge
|
||||
- Stat
|
||||
- Pie chart
|
||||
|
||||
#### Map extra fields to labels
|
||||
|
||||
If a field does not map to config property Grafana will automatically use it as source for a label on the output field-
|
||||
|
||||
Example:
|
||||
|
||||
| Name | DataCenter | Value |
|
||||
| ------- | ---------- | ----- |
|
||||
| ServerA | US | 100 |
|
||||
| ServerB | EU | 200 |
|
||||
|
||||
Output:
|
||||
|
||||
| ServerA (labels: DataCenter: US) | ServerB (labels: DataCenter: EU) |
|
||||
| -------------------------------- | -------------------------------- |
|
||||
| 10 | 20 |
|
||||
|
||||
The extra labels can now be used in the field display name provide more complete field names.
|
||||
|
||||
If you want to extract config from one query and appply it to another you should use the config from query results transformation.
|
||||
|
||||
#### Example
|
||||
|
||||
Input:
|
||||
|
||||
| Name | Value | Max |
|
||||
| ------- | ----- | --- |
|
||||
| ServerA | 10 | 100 |
|
||||
| ServerB | 20 | 200 |
|
||||
| ServerC | 30 | 300 |
|
||||
|
||||
Output:
|
||||
|
||||
| ServerA (config: max=100) | ServerB (config: max=200) | ServerC (config: max=300) |
|
||||
| ------------------------- | ------------------------- | ------------------------- |
|
||||
| 10 | 20 | 30 |
|
||||
|
||||
As you can see each row in the source data becomes a separate field. Each field now also has a max config option set. Options like **Min**, **Max**, **Unit** and **Thresholds** are all part of field configuration and if set like this will be used by the visualization instead of any options manually configured in the panel editor options pane.
|
||||
|
||||
### Prepare time series
|
||||
|
||||
> **Note:** This transformation is available in Grafana 7.5.10+ and Grafana 8.0.6+.
|
||||
|
||||
Prepare time series transformation is useful when a data source returns time series data in a format that isn't supported by the panel you want to use. For more information about data frame formats, refer to [Data frames]({{< relref "../../../developers/plugins/data-frames/" >}}).
|
||||
|
||||
This transformation helps you resolve this issue by converting the time series data from either the wide format to the long format or the other way around.
|
||||
|
||||
Select the `Multi-frame time series` option to transform the time series data frame from the wide to the long format.
|
||||
|
||||
Select the `Wide time series` option to transform the time series data frame from the long to the wide format.
|
||||
|
||||
### Series to rows
|
||||
|
||||
> **Note:** This transformation is available in Grafana 7.1+.
|
||||
|
||||
Use this transformation to combine the result from multiple time series data queries into one single result. This is helpful when using the table panel visualization.
|
||||
|
||||
The result from this transformation will contain three columns: Time, Metric, and Value. The Metric column is added so you easily can see from which query the metric originates from. Customize this value by defining Label on the source query.
|
||||
|
||||
In the example below, we have two queries returning time series data. It is visualized as two separate tables before applying the transformation.
|
||||
|
||||
Query A:
|
||||
|
||||
| Time | Temperature |
|
||||
| ------------------- | ----------- |
|
||||
| 2020-07-07 11:34:20 | 25 |
|
||||
| 2020-07-07 10:31:22 | 22 |
|
||||
| 2020-07-07 09:30:05 | 19 |
|
||||
|
||||
Query B:
|
||||
|
||||
| Time | Humidity |
|
||||
| ------------------- | -------- |
|
||||
| 2020-07-07 11:34:20 | 24 |
|
||||
| 2020-07-07 10:32:20 | 29 |
|
||||
| 2020-07-07 09:30:57 | 33 |
|
||||
|
||||
Here is the result after applying the Series to rows transformation.
|
||||
|
||||
| Time | Metric | Value |
|
||||
| ------------------- | ----------- | ----- |
|
||||
| 2020-07-07 11:34:20 | Temperature | 25 |
|
||||
| 2020-07-07 11:34:20 | Humidity | 22 |
|
||||
| 2020-07-07 10:32:20 | Humidity | 29 |
|
||||
| 2020-07-07 10:31:22 | Temperature | 22 |
|
||||
| 2020-07-07 09:30:57 | Humidity | 33 |
|
||||
| 2020-07-07 09:30:05 | Temperature | 19 |
|
||||
|
||||
### Sort by
|
||||
|
||||
This transformation will sort each frame by the configured field, When `reverse` is checked, the values will return in the opposite order.
|
||||
|
||||
### Limit
|
||||
|
||||
Use this transformation to limit the number of rows displayed.
|
||||
|
||||
In the example below, we have the following response from the data source:
|
||||
|
||||
| Time | Metric | Value |
|
||||
| ------------------- | ----------- | ----- |
|
||||
| 2020-07-07 11:34:20 | Temperature | 25 |
|
||||
| 2020-07-07 11:34:20 | Humidity | 22 |
|
||||
| 2020-07-07 10:32:20 | Humidity | 29 |
|
||||
| 2020-07-07 10:31:22 | Temperature | 22 |
|
||||
| 2020-07-07 09:30:57 | Humidity | 33 |
|
||||
| 2020-07-07 09:30:05 | Temperature | 19 |
|
||||
|
||||
Here is the result after adding a Limit transformation with a value of '3':
|
||||
|
||||
| Time | Metric | Value |
|
||||
| ------------------- | ----------- | ----- |
|
||||
| 2020-07-07 11:34:20 | Temperature | 25 |
|
||||
| 2020-07-07 11:34:20 | Humidity | 22 |
|
||||
| 2020-07-07 10:32:20 | Humidity | 29 |
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
aliases:
|
||||
- /docs/grafana/latest/troubleshooting/troubleshoot-queries/
|
||||
- /docs/grafana/latest/panels-visualizations/query-transform-data/troubleshoot-queries/
|
||||
description: Guide to troubleshooting Grafana queries
|
||||
keywords:
|
||||
- grafana
|
||||
- troubleshooting
|
||||
- documentation
|
||||
- guide
|
||||
- queries
|
||||
title: Troubleshoot queries
|
||||
weight: 200
|
||||
---
|
||||
|
||||
# Troubleshoot queries
|
||||
|
||||
This page provides information to solve common dashboard problems.
|
||||
|
||||
## I get different results when I rearrange my functions
|
||||
|
||||
Function order is very important. Just like in math, the order that you place your functions can affect the result.
|
||||
|
||||
## Inspect your query request and response
|
||||
|
||||
The most common problems are related to the query and response from your data source. Even if it looks
|
||||
like a bug or visualization issue in Grafana, it is almost always a problem with the data source query or
|
||||
the data source response. Start by inspecting your panel query and response.
|
||||
|
||||
For more information, refer to [Inspect request and response data]({{< relref "../../panel-inspector/#inspect-query-request-and-response data/" >}}).
|
||||
|
||||
## My query is slow
|
||||
|
||||
How many data points is your query returning? A query that returns lots of data points will be slow. Try this:
|
||||
|
||||
- In **Query options**, limit the **Max data points** returned.
|
||||
- In **Query options**, increase the **Min interval** time.
|
||||
- In your query, use a `group by` function.
|
||||
Reference in New Issue
Block a user