[v11.2.x] [DOC] Update TraceQL query editor doc (#92491)
Co-authored-by: Isabel Matwawana <76437239+imatwawana@users.noreply.github.com> Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
This commit is contained in:
grafana-delivery-bot[bot]
co-authored by
Isabel Matwawana
Kim Nylander
Isabel Matwawana
Kim Nylander
parent
2a88694fd3
commit
a8aefb1386
@@ -26,86 +26,194 @@ refs:
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
service-graph:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/tempo/service-graph/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/service-graph/
|
||||
recorded-queries:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
query-history-management:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/query-management/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/query-management/
|
||||
query-inspector:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/explore-inspector/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/explore-inspector/
|
||||
---
|
||||
|
||||
# Query tracing data
|
||||
|
||||
The Tempo data source's query editor helps you query and display traces from Tempo in [Explore](ref:explore).
|
||||
The queries use [TraceQL](/docs/tempo/latest/traceql), the query language designed specifically for tracing.
|
||||
|
||||
This topic explains configuration and queries specific to the Tempo data source.
|
||||
For general documentation on querying data sources in Grafana, see [Query and transform data](ref:query-transform-data).
|
||||
For general documentation on querying data sources in Grafana, refer to [Query and transform data](ref:query-transform-data).
|
||||
|
||||
## Before you begin
|
||||
|
||||
You can compose TraceQL queries in Grafana and Grafana Cloud using **Explore** and a Tempo data source.
|
||||
|
||||
### TraceQL knowledge helpful, but not required
|
||||
|
||||
You don't have to know TraceQL to create a query.
|
||||
You can use the **Search** query builder's user interface to select options to search your data.
|
||||
These selections generate a TraceQL query.
|
||||
Any query generated using **Search** query builder can be transferred to the **TraceQL** query editor, where you can edit the query directly.
|
||||
|
||||
To learn more about how to query by TraceQL, refer to the [TraceQL documentation](/docs/tempo/latest/traceql).
|
||||
|
||||
## Choose a query editing mode
|
||||
|
||||
The query editor has three **Query types** that you can use to explore your tracing data.
|
||||
You can use these modes by themselves or in combination to create building blocks to generate custom queries.
|
||||
Adding another query adds a new query block.
|
||||
Refer to [Use query types together](#use-query-types-together) for more information.
|
||||
|
||||
|
||||
|
||||
The three query types are:
|
||||
|
||||
- **Search** query builder - Provides a user interface for building a TraceQL query.
|
||||
- **TraceQL** query editor - Lets you write your own TraceQL query with assistance from autocomplete.
|
||||
- **Service Graph** view - Displays a visual relationship between services. Refer to the [Service graph](ref:service-graph) documentation for more information.
|
||||
|
||||
### Search query builder
|
||||
|
||||
The **Search** query builder provides drop-down lists and text fields to help you write a query.
|
||||
The query builder is ideal for people who aren't familiar with or want to learn TraceQL.
|
||||
|
||||
Refer to the [Search using the TraceQL query builder documentation]({{< relref "./traceql-search" >}}) to learn more about creating queries using convenient drop-down menus.
|
||||
|
||||
|
||||
|
||||
### TraceQL query editor
|
||||
|
||||
The **TraceQL** query editor lets you search by trace ID and write TraceQL queries using autocomplete.
|
||||
|
||||
Refer to the [TraceQL query editor documentation]({{< relref "./traceql-editor" >}}) to learn more about constructing queries using a code-editor-like experience.
|
||||
|
||||
|
||||
|
||||
You can also search for a trace ID by entering it into the query field.
|
||||
|
||||
### Service graph view
|
||||
|
||||
Grafana’s **Service Graph** view uses metrics to display span request rates, error rates, and durations, as well as service graphs.
|
||||
Once the requirements are set up, this preconfigured view is immediately available.
|
||||
|
||||
Using the service graph view, you can:
|
||||
|
||||
- Discover spans which are consistently erroring and the rates at which they occur.
|
||||
- Get an overview of the overall rate of span calls throughout your services.
|
||||
- Determine how long the slowest queries in your service take to complete.
|
||||
- Examine all traces that contain spans of particular interest based on rate, error, and duration values (RED signals).
|
||||
|
||||
For more information about the service graph, refer to [Service graph](../service-graph/).
|
||||
|
||||
|
||||
|
||||
## Use TraceQL panels in dashboards
|
||||
|
||||
To add TraceQL panels to your dashboard, refer to the [Traces panel documentation](/docs/grafana/latest/panels-visualizations/visualizations/traces/).
|
||||
|
||||
To learn more about Grafana dashboards, refer to the [Use dashboards documentation](/docs/grafana/latest/dashboards/use-dashboards/).
|
||||
|
||||
## Write TraceQL queries in Grafana
|
||||
## Set options for query builder and editor
|
||||
|
||||
You can compose TraceQL queries in Grafana and Grafana Cloud using **Explore** and a Tempo data source. You can use either the **Query type** > **Search** (the TraceQL query builder) or the **TraceQL** tab (the TraceQL query editor).
|
||||
Both of these methods let you build queries and drill-down into result sets.
|
||||
The following options are available for the **Search** and **TraceQL** query types.
|
||||
You can modify these settings in the **Options** section.
|
||||
|
||||
To learn more about how to query by TraceQL, refer to the [TraceQL documentation](/docs/tempo/latest/traceql).
|
||||
|
||||
|
||||
### TraceQL query builder
|
||||
After changing any option, re-run the query to apply the updates.
|
||||
|
||||
The TraceQL query builder, located on the **Explore** > **Query type** > **Search** in Grafana, provides drop-downs and text fields to help you write a query.
|
||||
Limit
|
||||
: Determines the maximum number of traces to return. Default value is `20`.
|
||||
|
||||
Refer to the [Search using the TraceQL query builder documentation]({{< relref "./traceql-search" >}}) to learn more about creating queries using convenient drop-down menus.
|
||||
Span Limit
|
||||
: Sets the maximum number of spans to return for each spanset. Default value is `3`.
|
||||
|
||||
|
||||
Table Format
|
||||
: Determines whether the query results table is displayed focused on **Traces** or **Spans**. **Traces** is the default selection. When **Traces** is selected, the results table starts with the trace ID. When **Spans** is selected, the table starts with the trace service.
|
||||
|
||||
### TraceQL query editor
|
||||
Step
|
||||
: Defines the step for metrics queries. Use duration notation, for example, `30ms` or `1m`.
|
||||
|
||||
The TraceQL query editor, located on the **Explore** > **TraceQL** tab in Grafana, lets you search by trace ID and write TraceQL queries using autocomplete.
|
||||
Streaming
|
||||
: Indicates if streaming is active. Streaming lets you view partial query results before the entire query completes. Activating streaming adds the **Table - Streaming Progress** section to the query results.
|
||||
|
||||
Refer to the [TraceQL query editor documentation]({{< relref "./traceql-editor" >}}) to learn more about constructing queries using a code-editor-like experience.
|
||||
## Use query types together
|
||||
|
||||
|
||||
You can use **+ Add query** to create customized queries that use one or more of the query types together.
|
||||
Each time you add a new query, it adds a new section, or query block, that contains **Search**, **TraceQL**, or **Service Graph** user interface.
|
||||
|
||||
## Query by search (deprecated)
|
||||
The added query and results table appear in the navigation under **Queries** and **Tables** respectively.
|
||||
You can use the navigation to view query, results table, and service graph blocks.
|
||||
|
||||
{{% admonition type="caution" %}}
|
||||
Starting with Grafana v10.2, this query type has been deprecated. It will be removed in Grafana v10.3.
|
||||
{{% /admonition %}}
|
||||
{{< video-embed src="/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-editor.mp4" max-width="800px" class="my-cool-video" caption="Navigating through the query blocks" align="center" >}}
|
||||
|
||||
Use this to search for traces by service name, span name, duration range, or process-level attributes that are included in your application's instrumentation, such as HTTP status code and customer ID.
|
||||
To add a query block:
|
||||
|
||||
To configure Tempo and the Tempo data source for search, refer to [Configure the data source]({{< relref "../#configure-the-data-source" >}}).
|
||||
1. Select **+ Add query**.
|
||||
1. Choose a query type: **Search**, **TraceQL**, or **Service Graph**.
|
||||
|
||||
To search for traces:
|
||||
To remove a query block, select the **Remove query** trash can icon.
|
||||
|
||||
1. Select **Search** from the **Query** type selector.
|
||||
1. Fill out the search form:
|
||||
To rename a block, select the **Rename** edit icon next to the query block name.
|
||||
The name changes in the queries and table list.
|
||||
|
||||
| Name | Description |
|
||||
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Service Name** | Returns a list of services. |
|
||||
| **Span Name** | Returns a list of span names. |
|
||||
| **Tags** | Sets tags with values in the [logfmt](https://brandur.org/logfmt) format, such as `error=true db.statement="select * from User"`. |
|
||||
| **Min Duration** | Filters all traces with a duration higher than the set value. Possible values are `1.2s`, `100ms`, `500us`. |
|
||||
| **Max Duration** | Filters all traces with a duration lower than the set value. Possible values are `1.2s`, `100ms`, `500us`. |
|
||||
| **Limit** | Limits the number of traces returned. |
|
||||
### Additional query block options
|
||||
|
||||
{{< figure src="/static/img/docs/explore/tempo-search.png" class="docs-image--no-shadow" max-width="750px" caption="Screenshot of the Tempo search feature with a trace rendered in the right panel" >}}
|
||||
Each query block has a set of icons in the right top corner.
|
||||
|
||||
### Search recent traces
|
||||
|
||||
|
||||
You can search recent traces held in Tempo's ingesters.
|
||||
By default, ingesters store the last 15 minutes of tracing data.
|
||||
These icons include:
|
||||
|
||||
To configure your Tempo data source to use this feature, refer to the [Tempo documentation](/docs/tempo/latest/getting-started/tempo-in-grafana/#search-of-recent-traces).
|
||||
Show data source help
|
||||
: Displays the **Tempo Cheat Sheet** with links to documentation.
|
||||
|
||||
### Search the backend datastore
|
||||
Create recorded query
|
||||
: Lets you save the current query block as a recorded query. This option is available in Grafana Cloud and Grafana Enterprise. For more information, refer to [Recorded queries](ref:recorded-queries).
|
||||
|
||||
Tempo includes the ability to search the entire backend datastore.
|
||||
Duplicate query
|
||||
: Copies the current block and adds a new identical block.
|
||||
|
||||
To configure your Tempo data source to use this feature, refer to the [Tempo documentation](/docs/tempo/latest/getting-started/tempo-in-grafana/#search-of-the-backend-datastore).
|
||||
Remove query
|
||||
: Deletes the query block.
|
||||
|
||||
## Query by TraceID
|
||||
### Use query history and query inspector
|
||||
|
||||
To query a particular trace:
|
||||
**Explore** provides a history of all queries you've used within a data source and an inspector that lets you view stats, inspect queries, view JSON, and general information for your data source queries.
|
||||
|
||||
1. Select the **TraceQL** query type.
|
||||
1. Enter the trace's ID into the query field.
|
||||
For more information, refer to the [Query inspector in Explore](ref:query-inspector) and [Query management in Explore](ref:query-history-management) documentation.
|
||||
|
||||
{{< figure src="/static/img/docs/tempo/query-editor-traceid.png" class="docs-image--no-shadow" max-width="750px" caption="Screenshot of the Tempo TraceID query type" >}}
|
||||
## Cross-tenant TraceQL queries
|
||||
|
||||
If you've configured a multi-stack Tempo data source, you can perform TraceQL queries across those stacks and tenants.
|
||||
|
||||
Queries performed using the cross-tenant configured data source, in either **Explore** or inside of dashboards,
|
||||
are performed across all the tenants that you specified in the **X-Scope-OrgID** header.
|
||||
|
||||
<!-- vale Grafana.Spelling = NO -->
|
||||
|
||||
TraceQL queries that compare multiple spansets may not correctly return all traces in a cross-tenant query. For instance,
|
||||
|
||||
<!-- vale Grafana.Quotes = YES -->
|
||||
|
||||
```
|
||||
{ span.attr1 = "bar" } && { span.attr2 = "foo" }
|
||||
```
|
||||
|
||||
TraceQL evaluates a contiguously stored trace.
|
||||
If these two conditions are satisfied in separate tenants, then Tempo doesn't return the trace.
|
||||
|
||||
Refer to [Set up a multi-stack Tempo data source in Grafana](https://grafana.com/docs/grafana-cloud/connect-externally-hosted/multi-stack-data-sources/#set-up-a-multi-stack-tempo-data-source-in-grafana) for information about configuring the Tempo data source.
|
||||
|
||||
For information about Tempo configuration requirements, refer to the [Cross-tenant query](https://grafana.com/docs/tempo/<TEMPO_VERSION>/operations/cross_tenant_query/) and [Enable multitenancy](https://grafana.com/docs/tempo/<TEMPO_VERSION>/operations/multitenancy/) documentation.
|
||||
|
||||
@@ -13,6 +13,27 @@ labels:
|
||||
menuTitle: Write TraceQL queries
|
||||
title: Write TraceQL queries with the editor
|
||||
weight: 300
|
||||
refs:
|
||||
explore:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
service-graph:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/tempo/service-graph/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/service-graph/
|
||||
recorded-queries:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
tempo-query-editor:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/tempo/query-editor/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/query-editor/
|
||||
---
|
||||
|
||||
# Write TraceQL queries with the editor
|
||||
|
||||
@@ -11,11 +11,32 @@ labels:
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Search traces
|
||||
title: Search traces using TraceQL query builder
|
||||
title: Investigate traces using Search query builder
|
||||
weight: 300
|
||||
refs:
|
||||
explore:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/explore/
|
||||
service-graph:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/tempo/service-graph/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/service-graph/
|
||||
recorded-queries:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/administration/recorded-queries/
|
||||
tempo-query-editor:
|
||||
- pattern: /docs/grafana/
|
||||
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/tempo/query-editor/
|
||||
- pattern: /docs/grafana-cloud/
|
||||
destination: /docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/query-editor/
|
||||
---
|
||||
|
||||
# Search traces using TraceQL query builder
|
||||
# Investigate traces using Search query builder
|
||||
|
||||
Inspired by PromQL and LogQL, TraceQL is a query language designed for selecting traces.
|
||||
TraceQL provides a method for formulating precise queries so you can zoom in to the data you need.
|
||||
@@ -23,9 +44,9 @@ Query results are returned faster because the queries limit what is searched.
|
||||
|
||||
To learn more about how to query by TraceQL, refer to the [TraceQL documentation](/docs/tempo/latest/traceql).
|
||||
|
||||
The TraceQL query builder, located on the **Explore** > **Query type** > **Search** in Grafana, provides drop-downs and text fields to help you write a query.
|
||||
The **Search** query builder, located on the **Explore** > **Query type** > **Search** in Grafana, provides drop-down lists and text fields to help you write a query.
|
||||
|
||||
|
||||
|
||||
|
||||
## Enable Search with the query builder
|
||||
|
||||
|
||||
@@ -68,7 +68,7 @@ Each node on the graph represents a service such as an API or database.
|
||||
|
||||
You use the Service Graph to detect performance issues; track increases in error, fault, or throttle rates in services; and investigate root causes by viewing corresponding traces.
|
||||
|
||||
{{< figure src="/static/img/docs/node-graph/node-graph-8-0.png" class="docs-image--no-shadow" max-width="500px" caption="Screenshot of a Node Graph" >}}
|
||||
{{< figure src="/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-node-graph.png" class="docs-image--no-shadow" max-width="500px" alt="Screenshot of a Node Graph" >}}
|
||||
|
||||
## Display the Service Graph
|
||||
|
||||
@@ -100,9 +100,9 @@ Each circle's color represents the percentage of requests in each state:
|
||||
|
||||
Service graph view displays a table of request rate, error rate, and duration metrics (RED) calculated from your incoming spans. It also includes a node graph view built from your spans.
|
||||
|
||||
{{< figure src="/static/img/docs/tempo/apm-table.png" class="docs-image--no-shadow" max-width="500px" caption="Screenshot of the Service Graph view" >}}
|
||||
{{< figure src="/media/docs/grafana/data-sources/tempo/query-editor/tempo-ds-query-service-graph.png" class="docs-image--no-shadow" max-width="500px" alt="Screenshot of the Service Graph view" >}}
|
||||
|
||||
For details, refer to the [Service Graph view documentation](/docs/tempo/latest/metrics-generator/service-graph-view/).
|
||||
For details, refer to the [Service Graph view documentation](/docs/tempo/<TEMPO_VERSION>/metrics-generator/service-graph-view/).
|
||||
|
||||
To open the Service Graph view:
|
||||
|
||||
@@ -120,4 +120,6 @@ These metrics must exist in your Prometheus data source.
|
||||
|
||||
To open a query in Prometheus with the span name of that row automatically set in the query, click a row in the **rate**, **error rate**, or **duration** columns.
|
||||
|
||||
|
||||
|
||||
To open a query in Tempo with the span name of that row automatically set in the query, click a row in the **links** column.
|
||||
|
||||
Reference in New Issue
Block a user