[v11.2.x] [DOC] Update TraceQL query editor doc (#92491)

Co-authored-by: Isabel Matwawana <76437239+imatwawana@users.noreply.github.com>
Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
This commit is contained in:
grafana-delivery-bot[bot]
2024-08-27 14:56:08 +01:00
committed by GitHub
co-authored by Isabel Matwawana Kim Nylander
parent 2a88694fd3
commit a8aefb1386
7 changed files with 317 additions and 110 deletions
@@ -23,19 +23,24 @@ Query results are returned faster because the queries limit what is searched.
To learn more about how to query by TraceQL, refer to the [TraceQL documentation](https://grafana.com/docs/tempo/latest/traceql/).
The TraceQL query editor, located on the **Explore** > **TraceQL** tab in Grafana, lets you search by trace ID and write TraceQL queries using autocomplete.
The TraceQL query editor in Grafana **Explore** lets you search by trace ID and write TraceQL queries using autocomplete.
![The TraceQL query editor](/media/docs/tempo/traceql/screenshot-traceql-query-editor-v10.png)
![The TraceQL query editor](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-traceql-v11.png)
## Enable the query editor
## Before you begin
This feature is automatically available in Grafana 10 (and newer) and Grafana Cloud.
To use the TraceQL query editor in self-hosted Grafana 9.3.2 and older, you need to [enable the `traceqlEditor` feature toggle](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/feature-toggles/).
If trying to query a self-managed Grafana Tempo or Grafana Enterprise Traces database with a gateway (e.g., nginx) in front of it from your hosted Grafana, that gateway (e.g., nginx) must allow gRPC connections. If it does not, streaming will not work and queries will fail to return results.
### Streaming and gRPC
If you cannot configure your gateway to allow gRPC, open a support escalation to request streaming query results be disabled in your hosted Grafana.
If you're trying to query a self-managed Grafana Tempo or Grafana Enterprise Traces database with a gateway, such as nginx, in front of it from your hosted Grafana, that gateway (for example, nginx) must allow gRPC connections.
If it doesn't, streaming won't work and queries will fail to return results.
If you can't configure your gateway to allow gRPC, deactivate streaming in your hosted Grafana.
In Grafana 11.2 and newer, you can deactivate the **Streaming** option in your Tempo data source settings from **Connections** > **Data sources** in the Grafana main menu.
You can also open a support escalation to request streaming query results be disabled in your hosted Grafana.
## Write TraceQL queries using the query editor
@@ -44,13 +49,19 @@ The Tempo data source’s TraceQL query editor helps you query and display trace
To access the query editor, follow these steps:
1. Sign into Grafana or Grafana Cloud.
1. Select your Tempo data source.
1. From the menu, choose **Explore** and select the **TraceQL** tab.
1. Select **Explore** from the main menu.
1. Select a Tempo data source.
1. Select the **TraceQL** tab.
1. Start your query on the text line by entering `{`. For help with TraceQL syntax, refer to the [Construct a TraceQL query documentation](https://grafana.com/docs/tempo/latest/traceql/#construct-a-traceql-query).
1. Optional: Use the Time picker drop-down to change the time and range for the query (refer to the [documentation for instructions](https://grafana.com/docs/grafana/latest/dashboards/use-dashboards/#set-dashboard-time-range)).
1. Once you have finished your query, select **Run query**.
This video provides and example of creating a TraceQL query using the custom tag grouping.
Optional: Select **Copy query from Search** to transfer a builder query to the editor.
1. Optional: Use the **Time picker** drop-down list to change the time and range for the query (refer to the [documentation for instructions](https://grafana.com/docs/grafana/latest/dashboards/use-dashboards/#set-dashboard-time-range)).
1. Once you've finished your query, select **Run query**.
![Query editor showing span results](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-ed-example-v11-a.png)
This video provides an example of creating a TraceQL query using the custom tag grouping.
{{< youtube id="fraepWra00Y" >}}
@@ -62,15 +73,15 @@ To query a particular trace by its trace ID:
1. Enter the trace ID into the query field. For example: `41928b92edf1cdbe0ba6594baee5ae9`
1. Click **Run query** or use the keyboard shortcut Shift + Enter.
![Search for a trace ID using the TraceQL query editor](/media/docs/tempo/traceql/screenshot-traceql-editor-traceID.png)
![Search for a trace ID using the TraceQL query editor](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-trace-id-v11.png)
## Use autocomplete to write queries
You can use the query editor’s autocomplete suggestions to write queries.
The editor detects span sets to provide relevant autocomplete options.
It uses regular expressions (regex) to detect where it's inside a spanset and provide attribute names, scopes, intrinsic names, logic operators, or attribute values from Tempo's API, depending on what's expected for the current situation.
The editor detects spansets to provide relevant autocomplete options.
It uses regular expressions (regex) to detect where it is inside a spanset and provide attribute names, scopes, intrinsic names, logic operators, or attribute values from the Tempo API, depending on what's expected for the current situation.
![Query editor showing the auto-complete feature](/media/docs/tempo/traceql/screenshot-traceql-query-editor-auto-complete-v10.png)
![Query editor showing the auto-complete feature](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-editor-autocomplete.png)
To create a query using autocomplete, follow these steps:
@@ -84,17 +95,37 @@ To create a query using autocomplete, follow these steps:
## View query results
Query results for both the editor and the builder are returned in a table. Selecting the Trace ID or Span ID provides more detailed information.
Query results appear in a table, such as **Table - Traces**, under the query editor.
Each span (and the trace it belongs to) matching the query conditions is returned by the query.
If there are no filter conditions, all spans are matching and thus returned with their associated traces.
Selecting the trace ID from the returned results opens a trace diagram. Selecting a span from the returned results opens a trace diagram and reveals the relevant span in the trace diagram (the highlighted blue line).
A query is performed against a defined time interval, relative (for example, the last 3 hours) or absolute (for example, from X date-time to Y date-time).
The query response is also limited by the number of traces (**Limit**) and spans per spanset (**Span Limit**).
In the trace diagram, the bold text on the left side of each span indicates the service name, for example `mythical-requester: requester`, and it is hidden when subsequent spans have the same service name (nested spans).
Each service has a color assigned to it, which is visible to the left of the name and timeline in the graph.
Spans with the same color belong to the same service. The grey text to the right of the service name indicates the span name.
![TraceQL in Grafana](/media/docs/tempo/traceql/TraceQL-in-Grafana-v11.png)
![Query editor showing span results](/media/docs/tempo/traceql/screenshot-traceql-query-editor-results-v10.png)
1. TraceQL query editor
1. Query options: **Limit**, **Span Limit** and **Table Format** (Traces or Spans).
1. Trace (by Trace ID). The **Name** and **Service** columns are displaying the trace root span name and associated service.
1. Spans associated with the Trace.
### Streaming results
Selecting the trace ID from the returned results opens a trace diagram.
Selecting a span from the returned results opens a trace diagram and reveals the relevant span in the trace diagram.
For more information on span details, refer to [Traces in Explore](https://grafana.com/docs/grafana/latest/explore/trace-integration/#span-details).
![Selecting a trace ID or a span to view span details](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-span-details-v11.png)
### Focus on traces or spans
Under **Options**, you can choose to display the table as **Traces** or **Spans** focused.
When the **Table Type** option is set to **Spans**, the traces and spansets are flattened into a list of spans.
The trace service and trace name are added to the row of each span to add context.
Using the **Spans** option makes it easier access the spans to apply transformations and plot them in dashboards.
### Stream results
The Tempo data source supports streaming responses to TraceQL queries so you can see partial query results as they come in without waiting for the whole query to finish.
@@ -17,45 +17,44 @@ labels:
# Write TraceQL queries using Search
The TraceQL query builder, located on the **Explore** > **Query type** > **Search** in Grafana, provides drop-downs and text fields to help you write a query.
The **Search** query builder, located on the **Explore** > **Query type** > **Search** in Grafana, provides drop-down lists and text fields to help you write a query.
The selections you make automatically generate a [TraceQL query](/docs/tempo/latest/traceql).
![The TraceQL query builder](/media/docs/tempo/traceql/screenshot-traceql-query-type-search-v10.png)
![The Search query builder](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-search-v11.png)
The builder lets you run the most common queries in as few clicks as possible. You don't need to know the underlying query language or database architecture to use it.
The builder supports a subset of TraceQL capabilities. For example, if you wish to use structural operators (`>>`, `>`, `~`), you need to use the query editor on the **TraceQL** tab.
You can use the query builder’s drop-downs to compose TraceQL queries. The selections you make automatically generate a [TraceQL query](/docs/tempo/latest/traceql).
The builder supports a subset of TraceQL capabilities, including some structural operators (`>>`, `>`, `=~`, `!=`).
To access **Search**, select your Tempo data source, and then choose **Explore** and select **Query type** > **Search**.
You can use the query builder to search trace data by resource service name, span name, duration, and one or more tags. The examples on this page use the default filters.
You can use the query builder to search trace data by resource service name, span name, duration, one or more tags. The examples on this page use the default filters.
In addition, you can add query builder blocks, view the query history, and use the **Inspector** to see details.
{{< figure src="/media/docs/tempo/traceql/screenshot-tempods-query-search.png" class="docs-image--no-shadow" max-width="750px" caption="Screenshot of the Tempo Search query type" >}}
{{< figure src="/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-builder-v11.png" class="docs-image--no-shadow" max-width="750px" caption="Screenshot of the Tempo Search query type" >}}
## Perform a search
To perform a search, you need to select filters and/or tags and then run the query. The results appear underneath the query builder.
To perform a search, you need to select filters and then run the query. The results appear underneath the query builder.
The screenshot identifies the areas used to perform a search.
{{< figure src="/media/docs/tempo/traceql/screenshot-tempods-query-search-parts.png" class="docs-image--no-shadow" max-width="750px" caption="Parts of Tempo Search query type" >}}
![Parts of the Search query builder](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-build-numbered-v11.png)
| Number | Name | Action | Comment |
| :----- | :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | Data source | Use the data source drop-down to select a Tempo data source. | Each data source has its own version of search. This Search is specific to the Tempo data source. |
| 2 | Query type tab | Select Search. | |
| 3 | Choose filter | Choose whether to filter using **Resource Service Name**, **Span Name**, and/or **Duration**. | Optional. You can execute an empty query in the Search tab. In TraceQL, `{}` is a valid query and is the default query until you choose options. |
| 4 | Filters conditions | Select options for one or more filters. For example, you can define a filter where **Resource Service Name** (`resource.service.name`) equals (`=`) `cloud-logs-archiver`. | Optional. At least one tag or filter must be defined. |
| 5 | Tags | Add tags for span, resource, or unscoped and define their conditions. | Optional. |
| 6 | TraceQL query | Displays the TraceQL query constructed by your selections. | This TraceQL query is executed when you select **Run query**. |
| Number | Name | Action | Comment |
| :----- | :-------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | Data source | Use the data source drop-down list to select a Tempo data source. | Each data source has its own version of search. This **Search** is specific to the Tempo data source. |
| 2 | Query type | Select **Search**. | |
| 3 | Choose filter | Choose one or more of the filters. | Optional. You can execute an empty query in the Search tab. In TraceQL, `{}` is a valid query and is the default query to provide a list of all traces or spans. |
| 4 | Filters conditions | Select options for one or more filters. For example, you can define a filter where **Service Name** (`resource.service.name`) equals (`=`) `user`. | Optional. At least one tag or filter must be defined. |
| 5 | Tags and Aggregate by | Add tags for span, resource, or unscoped and define their conditions. Use **Aggregate by** to group results. | Optional. |
| 6 | TraceQL query | Displays the TraceQL query constructed by your selections. | This TraceQL query is executed when you select **Run query**. |
Every query searches the data for the selected time frame.
By default, queries run against data from the last hour.
Select Time range to the left of Run query to choose the time range for the data your query runs against.
Select **Time range** to the left of **Run query** to choose the time range for the data your query runs against.
Read the [dashboard time range](/docs/grafana/latest/dashboards/use-dashboards/#set-dashboard-time-range) documentation to learn more.
To access Search, use the following steps:
To access the **Search** query builder, use the following steps:
1. Sign into Grafana.
1. Select your Tempo data source.
@@ -63,16 +62,20 @@ To access Search, use the following steps:
## Define filters
Using filters, you refine the data returned from the query by selecting **Resource Service Name**, **Span Name**, or **Duration**. The **Duration** represents span time, calculated by subtracting the end time from the start time of the span.
Using filters, you refine the data returned from the query by selecting **Service Name**, **Span Name**, **Status**, or **Duration**.
**Duration** represents span time, calculated by subtracting the end time from the start time of the span.
Grafana administrators can change the default filters using the Tempo data source configuration.
Filters can be limited by the operators. The available operators are determined by the field type.
For example, **Span Name** and **Resource Service Name** are string fields so the comparison operators are equals (`=`), not equal (`!=`), or regular expressions (`=~`).
Filters can be limited by the operators.
The field type determines the available operators.
For example, **Span Name** and **Service Name** are string fields so the comparison operators are equals (`=`), not equal (`!=`), matches regular expressions (`=~`), or doesn't match regular expression (`!~`).
**Duration** is a duration field type and uses range selections (`>`, `>=`, `<`, `<=`).
When you select multiple values for the same filter, Grafana automatically changes the operator to the regex operator `=~` and concatenates the values with a `|`. This capability only applies to fields with drop-down value selection.
When you select multiple values for the same filter, Grafana automatically changes the operator to the regular expression (regex) operator `=~` and concatenates the values with a `|`.
This capability only applies to fields with drop-down value selection.
For example, if you choose **Span Name** `= get` and then **Span Name** `= log_results_cache,` operator drop-down changes from `=` to `=~` and both `get` and `log_results_cache` are listed in the **Span Name** field. The resulting query is updated with this:
For example, if you choose **Span Name** `= get` and then **Span Name** `= log_results_cache,` operator drop-down list changes from `=` to `=~` and both `get` and `log_results_cache` are listed in the **Span Name** field.
The resulting query is updated with this:
`{duration>5ms && duration<10ms && name=~"get|log_results_cache"}`
@@ -80,7 +83,7 @@ To define filters, follow these steps:
1. Choose one of the filters.
1. Select a comparison operator from the drop-down.
1. **Resource Service Name** and **Span Name** only: Select one or more values from the drop-down.
1. **Service Name**, **Span Name**, and **Status** only: Select one or more values from the drop-down.
1. **Duration** only: Enter values and units for the range and choose comparison operators for the drop-downs. Units can be nanoseconds (`ns`), milliseconds (`ms`), seconds (`s`), minutes (`m`), and hours (`h`).
You can either select **Run query** to execute the query or define tags and then run the query.
@@ -97,9 +100,9 @@ To add a tag, follow these steps:
1. Select a tag from the **Select tag** drop-down.
1. Select a comparison operator.
1. Select a value from the **Select value** drop-down. This field is populated based upon the tag.
1. Optional: Select **+** to add an additional tag.
1. Optional: Select **+** to add another tag.
## Optional: Use Aggregate by
### Optional: Use Aggregate by
{{% admonition type="warning" %}}
**Aggregate by** is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. Documentation is either limited or not provided outside of code comments. No SLA is provided.
@@ -127,7 +130,7 @@ The results are shown in the same order used in **Aggregate by**.
For example, **Aggregate by** lists `intrinsic.name` followed by `span.http.user_agent`.
The first column in the results Table shows **name** and then **span.http.user_agent**.
![Use Aggregate by to calculate RED metrics for spans and group by attributes](/media/docs/tempo/traceql/screenshot-traces-aggregate-by.png)
![Use Aggregate by to calculate RED metrics for spans and group by attributes](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-build-aggregate-v11-a.png)
To use this capability:
@@ -141,22 +144,26 @@ To use this capability:
{{< youtube id="g97CjKOZqT4" >}}
### Optional: Add queries
### Optional: Add query and service graph blocks
Using **Add query**, you can have successive queries that run in sequential order.
Using **Add query**, you can have successive query or service node blocks that run in sequential order.
For example, query A runs and then query B.
You can reorder the queries by dragging and dropping them above or below other queries.
Select **+ Add query** to add another query block.
### Run queries and view results
For more information, refer to [Use query types together](/docs/grafana/<GRAFANA_VERSION>/datasources/tempo/query-editor/).
## Run queries and view results
Select **Run query** to run the TraceQL query (1 in the screenshot).
Queries can take a little while to return results. The results appear in a table underneath the query builder. Selecting a Trace ID (2 in the screenshot) displays more detailed information (3 in the screenshot).
Queries can take a little while to return results. The results appear in a table underneath the query builder.
Selecting a Trace ID (2 in the screenshot) displays more detailed information (3 in the screenshot).
**Span Filters** (4 in the screenshot) provide an additional to refine the query results.
{{< figure src="/media/docs/tempo/traceql/screenshot-tempods-query-results.png" class="docs-image--no-shadow" max-width="750px" caption="Tempo Search query type results" >}}
![Query results with numbered sections](/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-results-numbered-v11.png)
#### Streaming results
### Stream results
The Tempo data source supports streaming responses to TraceQL queries so you can see partial query results as they come in without waiting for the whole query to finish.
@@ -171,3 +178,16 @@ Streaming is available for both the **Search** and **TraceQL** query types.
You'll get immediate visibility of incoming traces on the results table.
{{< video-embed src="/media/docs/grafana/data-sources/tempo-streaming-v2.mp4" >}}
### Use filters and tags on spans
Using **Span Filters**, you can use filters to refine results when viewing span details.
These filters are available when viewing details for a trace.
You can continue to apply filters until you have narrowed down your resulting spans to the select few you are most interested in.
**Service Name**, **Span Name**, **Duration**, and **Tags** have the same function and operation as the filters of the same name in the **Search** query builder.
In addition, you can search for a keyword, opt to **Show matches only**, opt to **Show critical path only**, and browse matches using **Prev** and **Next**.
Use **Clear** to reset the filters.