public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[v9.4.x] [feat] docs; update admonition syntax (#68855)

Browse Source 
[feat] docs; update admonition syntax (#68842)

* [feat] docs; update admonition syntax

- Standardizes according to style conventions: https://grafana.com/docs/writers-toolkit/style-guide/style-conventions/#admonitions
- Prepares docs for better, uniform admonition style.

* Remove false positives and irregularities

* false positive removal

* Update docs/sources/datasources/mysql/_index.md

* Update docs/sources/developers/angular_deprecation/angular-plugins.md

* fix link errors

* Prettify some nested blockquotes

* remoe unnecessary admonition

(cherry picked from commit 1c4bb9ca00)

Co-authored-by: Matt Dodson <47385188+MattDodsonEnglish@users.noreply.github.com>
This commit is contained in:
Grot (@grafanabot)
2023-05-23 07:46:19 -05:00
committed by GitHub
co-authored by Matt Dodson
parent ad2a920191
commit 048de5d09a
144 changed files with 3887 additions and 1238 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/datasources/aws-cloudwatch/_index.md
+10 -4
View File
@@ -27,7 +27,9 @@ Administrators can also [provision the data source]({{< relref "#provision-the-d
Once you've added the data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "./query-editor/" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
> **Note:** To troubleshoot issues while setting up the CloudWatch data source, check the `/var/log/grafana/grafana.log` file.
{{% admonition type="note" %}}
To troubleshoot issues while setting up the CloudWatch data source, check the `/var/log/grafana/grafana.log` file.
{{% /admonition %}}
## Configure the data source
@@ -343,7 +345,9 @@ filter @message like /Exception/
| sort exceptionCount desc
```
> **Note:** If you receive an error like `input data must be a wide series but got ...` when trying to alert on a query, make sure that your query returns valid numeric data that can be output to a Time series panel.
{{% admonition type="note" %}}
If you receive an error like `input data must be a wide series but got ...` when trying to alert on a query, make sure that your query returns valid numeric data that can be output to a Time series panel.
{{% /admonition %}}
For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting" >}}).
@@ -354,8 +358,10 @@ Pricing for CloudWatch Logs is based on the amount of data ingested, archived, a
Each time you select a dimension in the query editor, Grafana issues a `ListMetrics` API request.
Each time you change queries in the query editor, Grafana issues a new request to the `GetMetricData` API.
> **Note:** Grafana v6.5 and higher replaced all `GetMetricStatistics` API requests with calls to GetMetricData to provide better support for CloudWatch metric math, and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option.
> The `GetMetricStatistics` API qualified for the CloudWatch API free tier, but `GetMetricData` calls don't.
{{% admonition type="note" %}}
Grafana v6.5 and higher replaced all `GetMetricStatistics` API requests with calls to GetMetricData to provide better support for CloudWatch metric math, and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option.
The `GetMetricStatistics` API qualified for the CloudWatch API free tier, but `GetMetricData` calls don't.
{{% /admonition %}}
For more information, refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
docs/sources/datasources/aws-cloudwatch/aws-authentication/index.md
+3 -1
View File
@@ -77,7 +77,9 @@ For more information on why and how to use service endpoints, refer to the [AWS
Create a file at `~/.aws/credentials`, the `HOME` path for the user running the `grafana-server` service.
> **Note:** If you think you have the credentials file in the right location, but it's not working, try moving your `.aws` file to `/usr/share/grafana/` and grant your credentials file at most 0644 permissions.
{{% admonition type="note" %}}
If you think you have the credentials file in the right location, but it's not working, try moving your `.aws` file to `/usr/share/grafana/` and grant your credentials file at most 0644 permissions.
{{% /admonition %}}
### Credentials file example
docs/sources/datasources/aws-cloudwatch/query-editor/index.md
+9 -3
View File
@@ -90,7 +90,9 @@ For details on the available functions, refer to [AWS Metric Math](https://docs.
For example, to apply arithmetic operations to a metric, apply a unique string id to the raw metric, then use this id and apply arithmetic operations to it in the Expression field of the new metric.
> **Note:** If you use the expression field to reference another query, like `queryA * 2`, you can't create an alert rule based on that query.
{{% admonition type="note" %}}
If you use the expression field to reference another query, like `queryA * 2`, you can't create an alert rule based on that query.
{{% /admonition %}}
##### Period macro
@@ -109,7 +111,9 @@ This feature is not available for metrics based on [metric math expressions](#me
### Create a Metric Insights query
> **Note:** This query option is available only in Grafana v8.3 and higher.
{{% admonition type="note" %}}
This query option is available only in Grafana v8.3 and higher.
{{% /admonition %}}
The Metrics Query option in the CloudWatch data source is referred to as **Metric Insights** in the AWS console.
It's a fast, flexible, SQL-based query engine that you can use to identify trends and patterns across millions of operational metrics in real time.
@@ -171,7 +175,9 @@ The suggestions appear after typing a space, comma, or dollar (`$`) character, o
{{< figure src="/static/img/docs/cloudwatch/cloudwatch-code-editor-autocomplete-8.3.0.png" max-width="500px" class="docs-image--right" caption="Code editor autocomplete" >}}
> **Note:** Template variables in the code editor can interfere with autocompletion.
{{% admonition type="note" %}}
Template variables in the code editor can interfere with autocompletion.
{{% /admonition %}}
To run the query, click **Run query** above the code editor.
docs/sources/datasources/elasticsearch/_index.md
+30 -30
View File
@@ -30,27 +30,30 @@ Once you've added the Elasticsearch data source, you can [configure it]({{< relr
This data source supports these versions of Elasticsearch:
- v7.10+
- v7.16+
- v8.x
## Configure the data source
**To access the data source configuration page:**
To configure basic settings for the data source, complete the following steps:
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the Elasticsearch data source.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `Elasticsearch` in the search bar.
1. Click **Elasticsearch**.
Set the data source's basic configuration options carefully:
The **Settings** tab of the data source is displayed.
| Name | Description |
| ----------- | -------------------------------------------------------------------------- |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Sets the data source that's pre-selected for new panels. |
| **Url** | Sets the HTTP protocol, IP, and port of your Elasticsearch server. |
| **Access** | Don't modify Access. Use "Server (default)" or the data source won't work. |
1. Set the data source's basic configuration options:
You must also configure settings specific to the Elasticsearch data source.
| Name | Description |
| ----------- | -------------------------------------------------------------------------- |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Sets the data source that's pre-selected for new panels. |
| **Url** | Sets the HTTP protocol, IP, and port of your Elasticsearch server. |
| **Access** | Don't modify Access. Use `Server (default)` or the data source won't work. |
You must also configure settings specific to the Elasticsearch data source. These options are described in the sections below.
### Index settings
@@ -59,18 +62,6 @@ You must also configure settings specific to the Elasticsearch data source.
Use the index settings to specify a default for the `time field` and your Elasticsearch index's name.
You can use a time pattern, such as `YYYY.MM.DD`, or a wildcard for the index name.
### Elasticsearch version
Select the version of your Elasticsearch data source from the version selection dropdown.
Different versions provide different query compositions and functionalities in the [query editor]({{< relref "./query-editor/" >}}).
Available Elasticsearch versions are `2.x`, `5.x`, `5.6+`, `6.0+`, `7.0+`, `7.7+`, and `7.10+`.
Grafana assumes you're running the lowest possible version for a specified range.
This ensures that new features or breaking changes in a future Elasticsearch release don't affect your configuration.
For example, if you run Elasticsearch `7.6.1` and select `7.0+`, and a new feature is made available for Elasticsearch `7.5.0` or newer releases, then a `7.5+` option will be available.
However, your configuration won't be affected until you explicitly select the new `7.5+` option in your settings.
### Configure Min time interval
The **Min time interval** setting defines a lower limit for the auto group-by time interval.
@@ -101,7 +92,9 @@ Toggle this to enable `X-Pack`-specific features and options, which provide the
When the "X-Pack enabled" setting is active and the configured Elasticsearch version is higher than `6.6.0`, you can configure Grafana to not ignore [frozen indices](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/frozen-indices.html) when performing search requests.
> **Note:** Frozen indices are [deprecated in Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/frozen-indices.html) since v7.14.
{{% admonition type="note" %}}
Frozen indices are [deprecated in Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/frozen-indices.html) since v7.14.
{{% /admonition %}}
### Logs
@@ -135,7 +128,9 @@ For details on AWS SigV4, refer to the [AWS documentation](https://docs.aws.amaz
#### AWS Signature Version 4 authentication
> **Note:** Available in Grafana v7.3 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.3 and higher.
{{% /admonition %}}
To sign requests to your Amazon Elasticsearch Service domain, you can enable SigV4 in Grafana's [configuration]({{< relref "../../setup-grafana/configure-grafana/#sigv4_auth_enabled" >}}).
@@ -149,6 +144,12 @@ For more information about AWS authentication options, refer to [AWS authenticat
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
{{% admonition type="note" %}}
The previously used `database` field has now been [deprecated](https://github.com/grafana/grafana/pull/58647).
You should now use the `index` field in `jsonData` to store the index name.
Please see the examples below.
{{% /admonition %}}
#### Provisioning examples
**Basic provisioning:**
@@ -160,9 +161,9 @@ datasources:
- name: Elastic
type: elasticsearch
access: proxy
database: '[metrics-]YYYY.MM.DD'
url: http://localhost:9200
jsonData:
index: '[metrics-]YYYY.MM.DD'
interval: Daily
timeField: '@timestamp'
```
@@ -176,12 +177,11 @@ datasources:
- name: elasticsearch-v7-filebeat
type: elasticsearch
access: proxy
database: '[filebeat-]YYYY.MM.DD'
url: http://localhost:9200
jsonData:
index: '[filebeat-]YYYY.MM.DD'
interval: Daily
timeField: '@timestamp'
esVersion: '7.0.0'
logMessageField: message
logLevelField: fields.level
dataLinks:
docs/sources/datasources/elasticsearch/template-variables/index.md
+3 -1
View File
@@ -47,7 +47,9 @@ These queries by default return results in term order (which can then be sorted
To produce a list of terms sorted by doc count (a top-N values list), add an `orderBy` property of "doc_count".
This automatically selects a descending sort.
> **Note:** To use an ascending sort (`asc`) with doc_count (a bottom-N list), set `order: "asc"`. However, Elasticsearch [discourages this](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-order) because sorting by ascending doc count can return inaccurate results.
{{% admonition type="note" %}}
To use an ascending sort (`asc`) with doc_count (a bottom-N list), set `order: "asc"`. However, Elasticsearch [discourages this](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-order) because sorting by ascending doc count can return inaccurate results.
{{% /admonition %}}
To keep terms in the doc count order, set the variable's Sort dropdown to **Disabled**.
You can alternatively use other sorting criteria, such as **Alphabetical**, to re-sort them.
docs/sources/datasources/google-cloud-monitoring/_index.md
+3 -1
View File
@@ -24,7 +24,9 @@ weight: 350
Grafana ships with built-in support for Google Cloud Monitoring.
This topic describes queries, templates, variables, and other configuration specific to the Google Cloud Monitoring data source.
> **Note:** Before Grafana v7.1, Google Cloud Monitoring was referred to as Google Stackdriver.
{{% admonition type="note" %}}
Before Grafana v7.1, Google Cloud Monitoring was referred to as Google Stackdriver.
{{% /admonition %}}
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management/" >}}).
Only users with the organization administrator role can add data sources.
docs/sources/datasources/google-cloud-monitoring/query-editor/index.md
+9 -3
View File
@@ -177,7 +177,9 @@ An expected result would look like: `gce_instance - compute.googleapis.com/insta
### Deep-link from Grafana panels to the Google Cloud Console Metrics Explorer
> **Note:** Available in Grafana v7.1 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.1 and higher.
{{% /admonition %}}
{{< figure src="/static/img/docs/v71/cloudmonitoring_deep_linking.png" max-width="500px" class="docs-image--no-shadow" caption="Google Cloud Monitoring deep linking" >}}
@@ -199,7 +201,9 @@ If the query editor rows return different units, Grafana uses the unit from the
### Use the Monitoring Query Language
> **Note:** Available in Grafana v7.4 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.4 and higher.
{{% /admonition %}}
The Monitoring Query Language (MQL) query builder helps you query and display MQL results in time series format.
To understand basic MQL concepts, refer to [Introduction to Monitoring Query Language](https://cloud.google.com/monitoring/mql).
@@ -222,7 +226,9 @@ However, `{{metric.service}}` is not supported, and `{{metric.type}}` and `{{met
## Query Service Level Objectives
> **Note:** Available in Grafana v7.0 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.0 and higher.
{{% /admonition %}}
{{< figure src="/static/img/docs/google-cloud-monitoring/slo-query-builder-8-0.png" max-width="400px" class="docs-image--no-shadow" caption="Service Level Objectives (SLO) query editor" >}}
docs/sources/datasources/graphite/query-editor/index.md
+8 -4
View File
@@ -66,8 +66,10 @@ Grafana consolidates all Graphite metrics so that Graphite doesn't return more d
By default, Grafana consolidates data points using the `avg` function.
To control how Graphite consolidates metrics, use the Graphite `consolidateBy()` function.
> **Note:** Legend summary values (max, min, total) can't all be correct at the same time because they are calculated client-side by Grafana.
> Depending on your consolidation function, only one or two can be correct at the same time.
{{% admonition type="note" %}}
Legend summary values (max, min, total) can't all be correct at the same time because they are calculated client-side by Grafana.
Depending on your consolidation function, only one or two can be correct at the same time.
{{% /admonition %}}
### Combine time series
@@ -82,8 +84,10 @@ To select data, use the `seriesByTag` function, which takes tag expressions (`=`
The Grafana query builder does this for you automatically when you select a tag.
> **Tip:** The regular expression search can be slow on high-cardinality tags, so try to use other tags to reduce the scope first.
> To help reduce the results, start by filtering on a particular name or namespace.
{{% admonition type="note" %}}
The regular expression search can be slow on high-cardinality tags, so try to use other tags to reduce the scope first.
To help reduce the results, start by filtering on a particular name or namespace.
{{% /admonition %}}
## Nest queries
docs/sources/datasources/graphite/template-variables/index.md
+3 -1
View File
@@ -104,7 +104,9 @@ For example, `apps.$app.servers.*` uses the variable `$app` in its query definit
### Use `__searchFilter` to filter query variable results
> **Note:** Available in Grafana v6.5 and higher.
{{% admonition type="note" %}}
Available in Grafana v6.5 and higher.
{{% /admonition %}}
You can use `__searchFilter` in the query field to filter the query result based on what the user types in the dropdown select box.
The default value for `__searchFilter` is `*` if you've not entered anything, and `` when used as part of a regular expression.
docs/sources/datasources/influxdb/_index.md
+32 -20
View File
@@ -30,23 +30,26 @@ Once you've added the InfluxDB data source, you can [configure it]({{< relref "#
## Configure the data source
**To access the data source configuration page:**
To configure basic settings for the data source, complete the following steps:
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the InfluxDB data source.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `InfluxDB` in the search bar.
1. Select **InfluxDB**.
Set the data source's basic configuration options carefully:
The **Settings** tab of the data source is displayed.
| Name | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. We recommend something like `InfluxDB-InfluxQL`. |
| **Default** | Sets whether the data source is pre-selected for new panels. |
| **URL** | The HTTP protocol, IP address, and port of your InfluxDB API. InfluxDB's default API port is 8086. |
| **Min time interval** | _(Optional)_ Refer to [Min time interval]({{< relref "#configure-min-time-interval" >}}). |
| **Max series** | _(Optional)_ Limits the number of series and tables that Grafana processes. Lower this number to prevent abuse, and increase it if you have many small time series and not all are shown. Defaults to 1,000. |
1. Set the data source's basic configuration options carefully:
You can also configure settings specific to the InfluxDB data source:
| Name | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. We recommend something like `InfluxDB-InfluxQL`. |
| **Default** | Sets whether the data source is pre-selected for new panels. |
| **URL** | The HTTP protocol, IP address, and port of your InfluxDB API. InfluxDB's default API port is 8086. |
| **Min time interval** | _(Optional)_ Refer to [Min time interval]({{< relref "#configure-min-time-interval" >}}). |
| **Max series** | _(Optional)_ Limits the number of series and tables that Grafana processes. Lower this number to prevent abuse, and increase it if you have many small time series and not all are shown. Defaults to 1,000. |
You can also configure settings specific to the InfluxDB data source. These options are described in the sections below.
### Min time interval
@@ -81,10 +84,13 @@ InfluxDB data source options differ depending on which query language you select
To help choose the best language for your needs, refer to a [comparison of Flux vs InfluxQL](https://docs.influxdata.com/influxdb/v1.8/flux/flux-vs-influxql/) and [why InfluxData created Flux](https://www.influxdata.com/blog/why-were-building-flux-a-new-data-scripting-and-query-language/).
> **Note:** Though not required, we recommend that you append your query language choice to the data source's **Name** setting:
>
> - InfluxDB-InfluxQL
> - InfluxDB-Flux
{{% admonition type="note" %}}
Though not required, we recommend that you append your query language choice to the data source's **Name** setting:
- InfluxDB-InfluxQL
- InfluxDB-Flux
{{% /admonition %}}
### Configure InfluxQL
@@ -114,6 +120,12 @@ Configure these options if you select the Flux query language:
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
{{% admonition type="note" %}}
`database` [field is deprecated](https://github.com/grafana/grafana/pull/58647).
We suggest to use `dbName` field in `jsonData`. Please see the examples below.
No need to change existing provisioning settings.
{{% /admonition %}}
#### Provisioning examples
**InfluxDB 1.x example:**
@@ -125,10 +137,10 @@ datasources:
- name: InfluxDB_v1
type: influxdb
access: proxy
database: site
user: grafana
url: http://localhost:8086
jsonData:
dbName: site
httpMode: GET
secureJsonData:
password: grafana
@@ -163,9 +175,9 @@ datasources:
type: influxdb
access: proxy
url: http://localhost:8086
# This database should be mapped to a bucket
database: site
jsonData:
# This database should be mapped to a bucket
dbName: site
httpMode: GET
httpHeaderName1: 'Authorization'
secureJsonData:
docs/sources/datasources/jaeger/_index.md
+150 -78
View File
@@ -15,75 +15,114 @@ weight: 800
# Jaeger data source
Grafana ships with built-in support for Jaeger, which provides open source, end-to-end distributed tracing.
This topic explains configuration and queries specific to the Jaeger data source.
Grafana ships with built-in support for Jaeger, which provides open source, end-to-end distributed tracing. This topic explains configuration and queries specific to the Jaeger data source.
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management" >}}).
Only users with the organization administrator role can add data sources.
Administrators can also [configure the data source via YAML]({{< relref "#provision-the-data-source" >}}) with Grafana's provisioning system.
You can also [upload a JSON trace file]({{< relref "#upload-a-json-trace-file" >}}), and [link a trace ID from logs]({{< relref "#link-a-trace-id-from-logs" >}}) in Loki.
Once you've added the data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so your Grafana instance's users can create queries in the [query editor]({{< relref "#query-the-data-source" >}}) when the users [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
You can also [upload a JSON trace file]({{< relref "#upload-a-json-trace-file" >}}), [link to a trace ID from logs]({{< relref "#link-to-a-trace-id-from-logs" >}}), and [link to a trace ID from metrics]({{< relref "#link-to-a-trace-id-from-metrics" >}}).
## Configure the data source
**To access the data source configuration page:**
To configure basic settings for the data source, complete the following steps:
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the Jaeger data source.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `Jaeger` in the search bar.
1. Select **Jaeger**.
Set the data source's basic configuration options carefully:
The **Settings** tab of the data source is displayed.
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Defines whether this data source is pre-selected for new panels. |
| **URL** | Sets the URL of the Jaeger instance, such as `http://localhost:16686`. |
| **Basic Auth** | Enables basic authentication for the Jaeger data source. |
| **User** | Defines the user name for basic authentication. |
| **Password** | Defines the password for basic authentication. |
1. Set the data source's basic configuration options:
You can also configure settings specific to the Jaeger data source:
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Defines whether this data source is pre-selected for new panels. |
| **URL** | Sets the URL of the Jaeger instance, such as `http://localhost:16686`. |
| **Basic Auth** | Enables basic authentication for the Jaeger data source. |
| **User** | Defines the user name for basic authentication. |
| **Password** | Defines the password for basic authentication. |
### Configure trace to logs
You can also configure settings specific to the Jaeger data source. These options are described in the sections below.
{{< figure src="/static/img/docs/explore/traces-to-logs-settings-8-2.png" class="docs-image--no-shadow" caption="Screenshot of the trace to logs settings" >}}
### Trace to logs
> **Note:** Available in Grafana v7.4 and higher.
![Trace to logs settings](/media/docs/tempo/tempo-trace-to-logs-9-4.png)
{{% admonition type="note" %}}
Available in Grafana v7.4 and higher.
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to logs** setting configures the [trace to logs feature]({{< relref "../../explore/trace-integration" >}}) that is available when you integrate Grafana with Jaeger.
**To configure trace to logs:**
There are two ways to configure the trace to logs feature:
- Use a simplified configuration with default query, or
- Configure a custom query where you can use a [template language]({{< relref "../../dashboards/variables/variable-syntax">}}) to interpolate variables from the trace or span.
#### Use a simple configuration
1. Select the target data source.
2. Select which tags to use in the logs query. The tags you configure must be present in the spans attributes or resources for a trace to logs span link to appear.
- **Single tag**
- Configuring `job` as a tag and clicking on a span link will take you to your configured logs datasource with the query `{job='value from clicked span'}`.
- **Multiple tags**
- If multiple tags are used they will be concatenated so the logs query would look like `{job='value from clicked span', service='value from clicked span'}`.
- **Mapped tags**
- For a mapped tag `service.name` with value `service`, clicking on a span link will take you to your configured logs datasource with the query `{service='value from clicked span'}` instead of `{service.name='value from clicked span'}`.
- This is useful for instances where your tracing datasource tags and your logs datasource tags don't match one-to-one.
1. Set start and end time shift. Since the logs timestamps may not exactly match the timestamps of the spans in trace, it may be necessary to search in larger or shifted time range to find the desired logs.
1. Select which tags to use in the logs query. The tags you configure must be present in the spans attributes or resources for a trace to logs span link to appear. You can optionally configure a new name for the tag. This is useful if the tag has dots in the name and the target data source does not allow dots in labels. In that case, you can, for example, remap `http.status` to `http_status`.
1. Optionally, switch on the **Filter by trace ID** and/or **Filter by span ID** setting to further filter the logs if your logs consistently contain trace or span IDs.
#### Configure a custom query
1. Select the target data source.
1. Set start and end time shift. Since the logs timestamps may not exactly match the timestamps of the spans in the trace, you may need to widen or shift the time range to find the desired logs.
1. Optionally, select tags to map. These tags can be used in the custom query with `${__tags}` variable. This variable will interpolate the mapped tags as a list in an appropriate syntax for the data source and will only include the tags that were present in the span omitting those that weren't present. You can optionally configure a new name for the tag. This is useful in cases where the tag has dots in the name and the target data source does not allow dots in labels. For example, you can remap `http.status` to `http_status`. If you don't map any tags here, you can still use any tag in the query like this `method="${__span.tags.method}"`.
1. Skip **Filter by trace ID** and **Filter by span ID** settings as these cannot be used with a custom query.
1. Switch on **Use custom query**.
1. Specify a custom query to be used to query the logs. You can use various variables to make that query relevant for current span. The link is present only if all the variables are interpolated with non-empty values to prevent creating an invalid query.
#### Variables that can be used in a custom query
To use a variable you need to wrap it in `${}`. For example: `${__span.name}`.
| Variable name | Description |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **\_\_tags** | This variable uses the tag mapping from the UI to create a label matcher string in the specific data source syntax. The variable only uses tags that are present in the span. The link is still created even if only one of those tags is present in the span. You can use this if all tags are not required for the query to be useful. |
| **\_\_span.spanId** | The ID of the span. |
| **\_\_span.traceId** | The ID of the trace. |
| **\_\_span.duration** | The duration of the span. |
| **\_\_span.name** | Name of the span. |
| **\_\_span.tags** | Namespace for the tags in the span. To access a specific tag named `version`, you would use `${__span.tags.version}`. In case the tag contains dot, you have to access it as `${__span.tags["http.status"]}`. |
| **\_\_trace.traceId** | The ID of the trace. |
| **\_\_trace.duration** | The duration of the trace. |
| **\_\_trace.name** | The name of the trace. |
The following table describes the ways in which you can configure your trace to logs settings:
| Name | Description |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Sets the target data source. |
| **Tags** | Defines the tags to use in the logs query. Default is `'cluster', 'hostname', 'namespace', 'pod'`. |
| **Map tag names** | Enables configuring how Jaeger tag names map to logs label names. For example, map `service.name` to `service`. |
| **Span start time shift** | Shifts the start time for the logs query based on the span start time. To extend to the past, use a negative value. Use time interval units like `5s`, `1m`, `3h`. Default is `0`. |
| **Span end time shift** | Shifts the end time for the logs query based on the span end time. Use time interval units. Default is `0`. |
| **Filter by Trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by Span ID** | Toggles whether to append the span ID to the logs query. |
| Setting name | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Defines the target data source. You can select only Loki or Splunk \[logs\] data sources. |
| **Span start time shift** | Shifts the start time for the logs query, based on the span's start time. You can use time units, such as `5s`, `1m`, `3h`. To extend the time to the past, use a negative value. Default: `0`. |
| **Span end time shift** | Shifts the end time for the logs query, based on the span's end time. You can use time units. Default: `0`. |
| **Tags** | Defines the tags to use in the logs query. Default: `cluster`, `hostname`, `namespace`, `pod`. You can change the tag name for example to remove dots from the name if they are not allowed in the target data source. For example, map `http.status` to `http_status`. |
| **Filter by trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by span ID** | Toggles whether to append the span ID to the logs query. |
| **Use custom query** | Toggles use of custom query with interpolation. |
| **Query** | Input to write custom query. Use variable interpolation to customize it with variables from span. |
### Configure trace to metrics
### Trace to metrics
> **Note:** This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
{{% admonition type="note" %}}
This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to metrics** section configures the [trace to metrics feature](/blog/2022/08/18/new-in-grafana-9.1-trace-to-metrics-allows-users-to-navigate-from-a-trace-span-to-a-selected-data-source/).
The **Trace to metrics** setting configures the [trace to metrics feature](/blog/2022/08/18/new-in-grafana-9.1-trace-to-metrics-allows-users-to-navigate-from-a-trace-span-to-a-selected-data-source/) available when integrating Grafana with Jaeger.
Use the settings to select the target Prometheus data source, and create any desired linked queries.
To configure trace to metrics:
1. Select the target data source.
1. Create any desired linked queries.
| Setting name | Description |
| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -97,15 +136,15 @@ Each linked query consists of:
Interpolate tags using the `$__tags` keyword.
For example, when you configure the query `requests_total{$__tags}`with the tags `k8s.pod=pod` and `cluster`, the result looks like `requests_total{pod="nginx-554b9", cluster="us-east-1"}`.
### Enable Node Graph
### Node Graph
The **Node Graph** setting enables the [Node Graph visualization]({{< relref "../../panels-visualizations/visualizations/node-graph/" >}}), which is disabled by default.
Once enabled, Grafana displays the Node Graph after loading the trace view.
Once enabled, Grafana displays the Node Graph above the trace view.
### Configure the span bar label
### Span bar
The **Span bar label** section helps you display additional information in the span bar row.
The **Span bar** setting helps you display additional information in the span bar row.
You can choose one of three options:
@@ -118,11 +157,9 @@ You can choose one of three options:
### Provision the data source
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning#data-sources" >}}).
For more information about provisioning and available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning#data-sources" >}}).
#### Provisioning examples
**Using basic auth and the trace-to-logs field:**
#### Provisioning example
```yaml
apiVersion: 1
@@ -130,51 +167,51 @@ apiVersion: 1
datasources:
- name: Jaeger
type: jaeger
uid: jaeger-spectra
uid: EbPG8fYoz
url: http://localhost:16686
access: proxy
url: http://localhost:16686/
basicAuth: true
basicAuthUser: my_user
editable: true
readOnly: false
isDefault: false
jsonData:
tracesToLogs:
# Field with internal link pointing to a logs data source in Grafana.
# datasourceUid value must match the datasourceUid value of the logs data source.
tracesToLogsV2:
# Field with an internal link pointing to a logs data source in Grafana.
# datasourceUid value must match the uid value of the logs data source.
datasourceUid: 'loki'
tags: ['job', 'instance', 'pod', 'namespace']
mappedTags: [{ key: 'service.name', value: 'service' }]
mapTagNamesEnabled: false
spanStartTimeShift: '1h'
spanEndTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: ['job', 'instance', 'pod', 'namespace']
filterByTraceID: false
filterBySpanID: false
customQuery: true
query: 'method="${__span.tags.method}"'
tracesToMetrics:
datasourceUid: 'prom'
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: [{ key: 'service.name', value: 'service' }, { key: 'job' }]
queries:
- name: 'Sample query'
query: 'sum(rate(traces_spanmetrics_latency_bucket{$__tags}[5m]))'
nodeGraph:
enabled: true
traceQuery:
timeShiftEnabled: true
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
spanBar:
type: 'None'
secureJsonData:
basicAuthPassword: my_password
```
## Query traces
## Query the data source
You can query and display traces from Jaeger via [Explore]({{< relref "../../explore" >}}).
{{< figure src="/static/img/docs/explore/jaeger-search-form.png" class="docs-image--no-shadow" caption="Screenshot of the Jaeger query editor" >}}
You can query by trace ID, or use the search form to find traces.
### Query by trace ID
{{< figure src="/static/img/docs/explore/jaeger-trace-id.png" class="docs-image--no-shadow" caption="Screenshot of the Jaeger query editor with trace ID selected" >}}
**To query by trace ID:**
1. Select **TraceID** from the **Query** type selector.
1. Insert the ID into the text input.
This topic explains configuration and queries specific to the Jaeger data source.
For general documentation on querying data sources in Grafana, see [Query and transform data]({{< relref "../../panels-visualizations/query-transform-data" >}}).
### Query by search
@@ -188,10 +225,21 @@ You can query by trace ID, or use the search form to find traces.
| **Service** | Returns a list of services. |
| **Operation** | Populated when you select a service with related operations. Select `All` to query all operations. |
| **Tags** | Sets tags with values in the [logfmt](https://brandur.org/logfmt) format, such as `error=true db.statement="select * from User"`. |
| **Min Duration** | Filters all traces with a duration higher than the set value. Valid values are `1.2s, 100ms, 500us`. |
| **Max Duration** | Filters all traces with a duration lower than the set value. Valid values are `1.2s, 100ms, 500us`. |
| **Min Duration** | Filters all traces with a duration higher than the set value. Possible values are `1.2s, 100ms, 500us`. |
| **Max Duration** | Filters all traces with a duration lower than the set value. Possible values are `1.2s, 100ms, 500us`. |
| **Limit** | Limits the number of traces returned. |
{{< figure src="/static/img/docs/explore/jaeger-search-form.png" class="docs-image--no-shadow" caption="Screenshot of the Jaeger query editor" >}}
### Query by trace ID
To query a particular trace:
1. Select the **TraceID** query type.
1. Enter the trace's ID into the **Trace ID** field.
{{< figure src="/static/img/docs/explore/jaeger-trace-id.png" class="docs-image--no-shadow" caption="Screenshot of the Jaeger query editor with TraceID selected" >}}
## Upload a JSON trace file
You can upload a JSON file that contains a single trace and visualize it.
@@ -249,8 +297,32 @@ If the file has multiple traces, Grafana visualizes its first trace.
}
```
## Link a trace ID from logs
## Span Filters
You can link to a Jaeger trace from logs in [Loki](/docs/loki/latest/) by configuring a derived field with an internal link.
{{% admonition type="note" %}}
This feature is behind the `newTraceViewHeader` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
For details, refer to [Derived fields]({{< relref "../loki/#configure-derived-fields" >}}) section of the [Loki data source]({{< relref "../loki/" >}}) documentation.
![Screenshot of span filtering](/media/docs/tempo/screenshot-grafana-tempo-span-filters.png)
Using span filters, you can filter your spans in the trace timeline viewer. The more filters you add, the more specific are the filtered spans.
You can add one or more of the following filters:
- Service name
- Span name
- Duration
- Tags (which include tags, process tags, and log fields)
## Link to a trace ID from logs
You can link to Jaeger traces from logs in Loki, Elasticsearch, Splunk, and other logs data sources by configuring an internal link.
To configure this feature, see the [Derived fields]({{< relref "../loki#configure-derived-fields" >}}) section of the Loki data source docs or the [Data links]({{< relref "../elasticsearch#data-links" >}}) section of the Elasticsearch or Splunk data source docs.
## Link to a trace ID from metrics
You can link to Jaeger traces from metrics in Prometheus data sources by configuring an exemplar.
To configure this feature, see the [introduction to exemplars]({{< relref "docs/grafana/latest/fundamentals/exemplars" >}}) documentation.
docs/sources/datasources/loki/_index.md
+6 -2
View File
@@ -42,7 +42,9 @@ Set the data source's basic configuration options carefully:
| **Allowed cookies** | Defines which cookies are forwarded to the data source. Grafana Proxy deletes all other cookies. |
| **Maximum lines** | Sets the upper limit for the number of log lines returned by Loki. Defaults to 1,000. Lower this limit if your browser is sluggish when displaying logs in Explore. |
> **Note:** To troubleshoot configuration and other issues, check the log file located at `/var/log/grafana/grafana.log` on Unix systems, or in `<grafana_install_dir>/data/log` on other platforms and manual installations.
{{% admonition type="note" %}}
To troubleshoot configuration and other issues, check the log file located at `/var/log/grafana/grafana.log` on Unix systems, or in `<grafana_install_dir>/data/log` on other platforms and manual installations.
{{% /admonition %}}
### Configure derived fields
@@ -54,7 +56,9 @@ The **Derived Fields** configuration helps you:
For example, you can link to your tracing backend directly from your logs, or link to a user profile page if the log line contains a corresponding userId.
These links appear in the [log details]({{< relref "../../explore/logs-integration/#labels-and-detected-fields" >}}).
> **Note:** If you use Grafana Cloud, you can request modifications to this feature by [opening a support ticket in the Cloud Portal](/profile/org#support).
{{% admonition type="note" %}}
If you use Grafana Cloud, you can request modifications to this feature by [opening a support ticket in the Cloud Portal](/profile/org#support).
{{% /admonition %}}
Each derived field consists of:
docs/sources/datasources/loki/query-editor/index.md
+6 -2
View File
@@ -30,7 +30,9 @@ To switch between the editor modes, select the corresponding **Builder** and **C
To run a query, select **Run queries** located at the top of the editor.
> **Note:** To run Loki queries in [Explore]({{< relref "../../../explore/" >}}), select **Run query**.
{{% admonition type="note" %}}
To run Loki queries in [Explore]({{< relref "../../../explore/" >}}), select **Run query**.
{{% /admonition %}}
Each mode is synchronized with the other modes, so you can switch between them without losing your work, although there are some limitations.
@@ -206,7 +208,9 @@ In your `server` section, add the following configuration:
}
```
> **Note:** Available in Grafana v6.3 and higher.
{{% admonition type="note" %}}
Available in Grafana v6.3 and higher.
{{% /admonition %}}
## Create a metric query
docs/sources/datasources/mysql/_index.md
+3 -1
View File
@@ -118,7 +118,9 @@ The response from MySQL can be formatted as either a table or as a time series.
### Dataset and Table selection
> **Note:** If your table or database name contains a reserved word or a [not permitted character](https://dev.mysql.com/doc/refman/8.0/en/identifiers.html) the editor will put quotes around them. For example a table name like `table-name` will be quoted with backticks `` `table-name` ``.
{{% admonition type="note" %}}
If your table or database name contains a reserved word or a [not permitted character](https://dev.mysql.com/doc/refman/8.0/en/identifiers.html) the editor will put quotes around them. For example a table name like `table-name` will be quoted with backticks `` `table-name` ``.
{{% /admonition %}}
In the dataset dropdown, choose the MySQL database to query. The dropdown is be populated with the databases that the user has access to.
When the dataset is selected, the table dropdown is populated with the tables that are available.
docs/sources/datasources/opentsdb/_index.md
+3 -1
View File
@@ -64,7 +64,9 @@ can be used to query opentsdb. Fill Policy is also introduced in OpenTSDB 2.2.
![](/static/img/docs/v43/opentsdb_query_editor.png)
> **Note:** While using OpenTSDB 2.2 data source, make sure you use either Filters or Tags as they are mutually exclusive. If used together, might give you weird results.
{{% admonition type="note" %}}
While using OpenTSDB 2.2 data source, make sure you use either Filters or Tags as they are mutually exclusive. If used together, might give you weird results.
{{% /admonition %}}
### Auto complete suggestions
docs/sources/datasources/postgres/_index.md
+3 -1
View File
@@ -138,7 +138,9 @@ datasources:
timescaledb: false
```
> **Note:** In the above code, the `postgresVersion` value of `10` refers to version PostgreSQL 10 and above.
{{% admonition type="note" %}}
In the above code, the `postgresVersion` value of `10` refers to version PostgreSQL 10 and above.
{{% /admonition %}}
#### Troubleshoot provisioning
docs/sources/datasources/prometheus/_index.md
+6 -2
View File
@@ -123,7 +123,9 @@ For details on AWS SigV4, refer to the [AWS documentation](https://docs.aws.amaz
#### AWS Signature Version 4 authentication
> **Note:** Available in Grafana v7.3.5 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.3.5 and higher.
{{% /admonition %}}
To connect the Prometheus data source to Amazon Managed Service for Prometheus using SigV4 authentication, refer to the AWS guide to [Set up Grafana open source or Grafana Enterprise for use with AMP](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-query-standalone-grafana.html).
@@ -131,7 +133,9 @@ If you run Grafana in an Amazon EKS cluster, follow the AWS guide to [Query usin
### Configure exemplars
> **Note:** Available in Prometheus v2.26 and higher with Grafana v7.4 and higher.
{{% admonition type="note" %}}
Available in Prometheus v2.26 and higher with Grafana v7.4 and higher.
{{% /admonition %}}
Grafana 7.4 and higher can show exemplars data alongside a metric both in Explore and in Dashboards.
Exemplars associate higher-cardinality metadata from a specific event with traditional time series data.
docs/sources/datasources/prometheus/query-editor/index.md
+7 -3
View File
@@ -85,8 +85,10 @@ The **Type** setting selects the query type.
- An **Exemplars** query runs with the regular query and shows exemplars in the graph.
> **Note:** Grafana modifies the request dates for queries to align them with the dynamically calculated step.
> This ensures a consistent display of metrics data, but it can result in a small gap of data at the right edge of a graph.
{{% admonition type="note" %}}
Grafana modifies the request dates for queries to align them with the dynamically calculated step.
This ensures a consistent display of metrics data, but it can result in a small gap of data at the right edge of a graph.
{{% /admonition %}}
## Code mode
@@ -118,7 +120,9 @@ When you select a metric, the browser narrows down the available labels to show
You can then select one or more labels for which the available label values are shown in lists in the bottom section.
Select one or more values for each label to tighten your query scope.
> **Note:** If you do not remember a metric name to start with, you can also select a few labels to narrow down the list, then find relevant label values.
{{% admonition type="note" %}}
If you do not remember a metric name to start with, you can also select a few labels to narrow down the list, then find relevant label values.
{{% /admonition %}}
All lists in the metrics browser have a search field above them to quickly filter for metrics or labels that match a certain string.
The values section has only one search field, and its filtering applies to all labels to help you find values across labels once selected.
docs/sources/datasources/prometheus/template-variables/index.md
+3 -1
View File
@@ -63,7 +63,9 @@ Regex:
## Use `$__rate_interval`
> **Note:** Available in Grafana v7.2 and higher.
{{% admonition type="note" %}}
Available in Grafana v7.2 and higher.
{{% /admonition %}}
We recommend using `$__rate_interval` in the `rate` and `increase` functions instead of `$__interval` or a fixed interval value.
Because `$__rate_interval` is always at least four times the value of the Scrape interval, it avoid problems specific to Prometheus.
docs/sources/datasources/tempo/_index.md
+152 -91
View File
@@ -15,7 +15,7 @@ weight: 1400
# Tempo data source
Grafana ships with built-in support for Tempo, a high-volume, minimal-dependency trace storage, open-source tracing solution from Grafana Labs.
Grafana ships with built-in support for [Tempo](https://grafana.com/docs/tempo/latest/), a high-volume, minimal-dependency trace storage, open-source tracing solution from Grafana Labs. This topic explains configuration and queries specific to the Tempo data source.
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management/" >}}).
Only users with the organization administrator role can add data sources.
@@ -23,92 +23,103 @@ Administrators can also [configure the data source via YAML]({{< relref "#provis
Once you've added the data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "./query-editor/" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
You can also [use the service graph]({{< relref "#use-the-service-graph" >}}) to view service relationships, [track RED metrics]({{< relref "#open-the-service-graph-view" >}}), [upload a JSON trace file]({{< relref "#upload-a-json-trace-file" >}}), and [link a trace ID from logs]({{< relref "#link-a-trace-id-from-logs" >}}) in Loki or Elasticsearch.
You can also [use the Service Graph]({{< relref "#use-the-service-graph" >}}) to view service relationships, [track RED metrics]({{< relref "#open-the-service-graph-view" >}}), [upload a JSON trace file]({{< relref "#upload-a-json-trace-file" >}}), [link to a trace ID from logs]({{< relref "#link-to-a-trace-id-from-logs" >}}), and [link to a trace ID from metrics]({{< relref "#link-to-a-trace-id-from-metrics" >}}).
## Configure the data source
**To access the data source configuration page:**
To configure basic settings for the data source, complete the following steps:
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the Tempo data source.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `Tempo` in the search bar.
1. Select **Tempo**.
Set the data source's basic configuration options carefully:
The **Settings** tab of the data source is displayed.
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Sets the data source that's pre-selected for new panels. |
| **URL** | Sets the URL of the Tempo instance, such as `http://tempo`. |
| **Basic Auth** | Enables basic authentication to the Tempo data source. |
| **User** | Sets the user name for basic authentication. |
| **Password** | Sets the password for basic authentication. |
1. Set the data source's basic configuration options:
You can also configure settings specific to the Tempo data source:
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Sets the data source that's pre-selected for new panels. |
| **URL** | Sets the URL of the Tempo instance, such as `http://tempo`. |
| **Basic Auth** | Enables basic authentication to the Tempo data source. |
| **User** | Sets the user name for basic authentication. |
| **Password** | Sets the password for basic authentication. |
### Configure trace to logs
You can also configure settings specific to the Tempo data source. These options are described in the sections below.
### Trace to logs
![Trace to logs settings](/media/docs/tempo/tempo-trace-to-logs-9-4.png)
> **Note:** Available in Grafana v7.4 and higher.
> If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% admonition type="note" %}}
Available in Grafana v7.4 and higher.
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to logs** setting configures the [trace to logs feature]({{< relref "../../explore/trace-integration" >}}) that is available when you integrate Grafana with Tempo.
There are two ways to configure the trace to logs feature. You can use simplified configuration with default query, or you can configure custom query where you can use a [template language]({{< relref "../../dashboards/variables/variable-syntax">}}) to interpolate variables from the trace or span.
There are two ways to configure the trace to logs feature:
**To use simple configuration:**
- Use a simplified configuration with default query, or
- Configure a custom query where you can use a [template language]({{< relref "../../dashboards/variables/variable-syntax">}}) to interpolate variables from the trace or span.
#### Use a simple configuration
1. Select the target data source.
1. Set start and end time shift. As the logs timestamps may not exactly match the timestamps of the spans in trace it may be necessary to search in larger or shifted time range to find the desired logs.
1. Select which tags to use in the logs query. The tags you configure must be present in the spans attributes or resources for a trace to logs span link to appear. You can optionally configure a new name for the tag. This is useful for example if the tag has dots in the name and the target data source does not allow using dots in labels. In that case you can for example remap `http.status` to `http_status`.
1. Optionally switch on Filter by Trace ID and/or Filter by Span ID to further filter the logs if your logs consistently contain trace or span IDs.
1. Optionally switch on the **Filter by trace ID** and/or **Filter by span ID** setting to further filter the logs if your logs consistently contain trace or span IDs.
**To use custom query configuration:**
#### Configure a custom query
1. Select the target data source.
1. Set start and end time shift. As the logs timestamps may not exactly match the timestamps of the spans in the trace it may be necessary to widen or shift the time range to find the desired logs.
1. Optionally select tags to map. These tags can be used in the custom query with `${__tags}` variable. This variable will interpolate the mapped tags as list in an appropriate syntax for the data source and will only include the tags that were present in the span omitting those that weren't present. You can optionally configure a new name for the tag. This is useful in cases where the tag has dots in the name and the target data source does not allow using dots in labels. For example, you can remap `http.status` to `http_status` in such a case. If you don't map any tags here, you can still use any tag in the query like this `method="${__span.tags.method}"`.
1. Skip Filter by Trace ID or Filter by Span ID as these cannot be used with custom query.
1. Switch on Use custom query.
1. Specify custom query to be used to query the logs. You can use various variables to make that query relevant for current span. The link will only be shown only if all the variables are interpolated with non-empty values to prevent creating invalid query.
1. Skip **Filter by trace ID** and **Filter by span ID** settings as these cannot be used with a custom query.
1. Switch on **Use custom query**.
1. Specify a custom query to be used to query the logs. You can use various variables to make that query relevant for current span. The link will only be shown only if all the variables are interpolated with non-empty values to prevent creating an invalid query.
#### Variables that can be used in a custom query
**Variables that can be used in custom query**
To use a variable you need to wrap it in `${}`. For example `${__span.name}`.
| Variable name | Description |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| \_\_tags | This variable is special because it uses the tag mapping in from the UI to create a label matcher string in the specific data source syntax. It uses only tags that are present in the span so the link will still be created even if only one of those tags is present in the span. You can use this if not all the tags are required for the query to be useful. |
| \_\_span.spanId | The ID of the span. |
| \_\_span.traceId | The ID of the trace. |
| \_\_span.duration | The duration of the span. |
| \_\_span.name | Name of the span. |
| \_\_span.tags | Namespace for the tags in the span. To access a specific tag named "version" you would use `${__span.tags.version}`. In case the tag contains dot you have to access it as `${__span.tags["http.status"]}`. |
| \_\_trace.traceId | The ID of the trace. |
| \_\_trace.duration | The duration of the trace. |
| \_\_trace.name | The name of the trace. |
| Variable name | Description |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **\_\_tags** | This variable uses the tag mapping from the UI to create a label matcher string in the specific data source syntax. The variable only uses tags that are present in the span. The link is still created even if only one of those tags is present in the span. You can use this if all tags are not required for the query to be useful. |
| **\_\_span.spanId** | The ID of the span. |
| **\_\_span.traceId** | The ID of the trace. |
| **\_\_span.duration** | The duration of the span. |
| **\_\_span.name** | Name of the span. |
| **\_\_span.tags** | Namespace for the tags in the span. To access a specific tag named `version`, you would use `${__span.tags.version}`. In case the tag contains dot, you have to access it as `${__span.tags["http.status"]}`. |
| **\_\_trace.traceId** | The ID of the trace. |
| **\_\_trace.duration** | The duration of the trace. |
| **\_\_trace.name** | The name of the trace. |
The following table describes the ways in which you can configure your trace to logs settings:
| Setting name | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Defines the target data source. You can select only Loki or Splunk \[logs\] data sources. |
| **Span start time shift** | Shifts the start time for the logs query, based on the span's start time. You can use time units, such as `5s`, `1m`, `3h`. To extend the time to the past, use a negative value. Default is 0. |
| **Span end time shift** | Shifts the end time for the logs query, based on the span's end time. You can use time units. Default is 0. |
| **Tags** | Defines the the tags to use in the logs query. Default is `'cluster', 'hostname', 'namespace', 'pod'`. You can change the tag name for example to remove dots from the name if they are not allowed in the target data source. For example map `http.status` to `http_status` |
| **Filter by Trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by Span ID** | Toggles whether to append the span ID to the logs query. |
| **Use custom query** | Toggles use of custom query with interpolation. |
| **Query** | Input to write custom query. Use variable interpolation to customize it with variables from span. |
| Setting name | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Defines the target data source. You can select only Loki or Splunk \[logs\] data sources. |
| **Span start time shift** | Shifts the start time for the logs query, based on the span's start time. You can use time units, such as `5s`, `1m`, `3h`. To extend the time to the past, use a negative value. Default: `0`. |
| **Span end time shift** | Shifts the end time for the logs query, based on the span's end time. You can use time units. Default: `0`. |
| **Tags** | Defines the tags to use in the logs query. Default: `cluster`, `hostname`, `namespace`, `pod`. You can change the tag name for example to remove dots from the name if they are not allowed in the target data source. For example, map `http.status` to `http_status`. |
| **Filter by trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by span ID** | Toggles whether to append the span ID to the logs query. |
| **Use custom query** | Toggles use of custom query with interpolation. |
| **Query** | Input to write custom query. Use variable interpolation to customize it with variables from span. |
### Configure trace to metrics
### Trace to metrics
> **Note:** This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
> If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% admonition type="note" %}}
This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to metrics** setting configures the [trace to metrics feature](/blog/2022/08/18/new-in-grafana-9.1-trace-to-metrics-allows-users-to-navigate-from-a-trace-span-to-a-selected-data-source/) available when integrating Grafana with Tempo.
**To configure trace to metrics:**
To configure trace to metrics:
1. Select the target data source.
1. Create any desired linked queries.
@@ -125,36 +136,48 @@ Each linked query consists of:
Interpolate tags using the `$__tags` keyword.
For example, when you configure the query `requests_total{$__tags}`with the tags `k8s.pod=pod` and `cluster`, the result looks like `requests_total{pod="nginx-554b9", cluster="us-east-1"}`.
### Configure service graph
### Service Graph
The **Service Graph** section configures the [Service Graph](/docs/tempo/latest/grafana-agent/service-graphs/) feature.
The **Service Graph** setting configures the [Service Graph](/docs/tempo/latest/grafana-agent/service-graphs/) feature.
Configure the **Data source** setting to define in which Prometheus instance the Service Graph data is stored.
To use the service graph, refer to the [Service graph section]({{< relref "#use-the-service-graph" >}}).
To use the Service Graph, refer to the [Service Graph documentation]({{< relref "#use-the-service-graph" >}}).
### Configure Tempo search integration
### Node Graph
The **Search** section configures [Tempo search](/docs/tempo/latest/configuration/#search).
The **Node Graph** setting enables the [node graph visualization]({{< relref "../../panels-visualizations/visualizations/node-graph/" >}}), which is disabled by default.
Optionally configure the **Hide search** setting to hide the search query option in Explore if search is not configured in the Tempo instance.
Once enabled, Grafana displays the node graph above the trace view.
### Enable Node Graph
### Tempo search
The **Node Graph** setting enables the [Node Graph visualization]({{< relref "../../panels-visualizations/visualizations/node-graph/" >}}), which is disabled by default.
The **Search** setting configures [Tempo search](/docs/tempo/latest/configuration/#search).
Once enabled, Grafana displays the Node Graph after loading the trace view.
You can configure the **Hide search** setting to hide the search query option in **Explore** if search is not configured in the Tempo instance.
### Configure Loki search
### Loki search
The **Loki search** section configures the Loki search query type.
The **Loki search** setting configures the Loki search query type.
Configure the **Data source** setting to define which Loki instance you want to use to search traces.
You must configure [derived fields]({{< relref "../loki#configure-derived-fields" >}}) in the Loki instance.
### Span bar label
### TraceID query
The **Span bar label** section helps you display additional information in the span bar row.
The **TraceID query** setting modifies how TraceID queries are run. The time range can be used when there are performance issues or timeouts since it will narrow down the search to the defined range. This setting is disabled by default.
You can configure this setting as follows:
| Name | Description |
| --------------------- | ----------------------------------------------------------- |
| **Enable time range** | Use a time range in the TraceID query. Default: `disabled`. |
| **Time shift start** | Time shift for start of search. Default: `30m`. |
| **Time shift end** | Time shift for end of search. Default: `30m`. |
### Span bar
The **Span bar** setting helps you display additional information in the span bar row.
You can choose one of three options:
@@ -167,9 +190,9 @@ You can choose one of three options:
### Provision the data source
You can define and configure the Tempo data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
For more information about provisioning and available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
#### Provisioning examples
#### Provisioning example
```yaml
apiVersion: 1
@@ -177,34 +200,45 @@ apiVersion: 1
datasources:
- name: Tempo
type: tempo
# Access mode - proxy (server in the UI) or direct (browser in the UI).
access: proxy
uid: EbPG8fYoz
url: http://localhost:3200
access: proxy
basicAuth: false
jsonData:
httpMethod: GET
tracesToLogs:
tracesToLogsV2:
# Field with an internal link pointing to a logs data source in Grafana.
# datasourceUid value must match the uid value of the logs data source.
datasourceUid: 'loki'
tags: ['job', 'instance', 'pod', 'namespace']
mappedTags: [{ key: 'service.name', value: 'service' }]
mapTagNamesEnabled: false
spanStartTimeShift: '1h'
spanEndTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: ['job', 'instance', 'pod', 'namespace']
filterByTraceID: false
filterBySpanID: false
customQuery: true
query: 'method="${__span.tags.method}"'
tracesToMetrics:
datasourceUid: 'prom'
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: [{ key: 'service.name', value: 'service' }, { key: 'job' }]
queries:
- name: 'Sample query'
query: 'sum(rate(traces_spanmetrics_latency_bucket{$__tags}[5m]))'
serviceMap:
datasourceUid: 'prometheus'
search:
hide: false
nodeGraph:
enabled: true
search:
hide: false
lokiSearch:
datasourceUid: 'loki'
traceQuery:
timeShiftEnabled: true
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
spanBar:
type: 'Tag'
tag: 'http.path'
```
## Query the data source
@@ -218,11 +252,11 @@ For details, refer to the [query editor documentation]({{< relref "./query-edito
You can upload a JSON file that contains a single trace and visualize it.
If the file has multiple traces, Grafana visualizes its first trace.
**To download a trace or service graph through the inspector:**
**To download a trace or Service Graph through the inspector:**
1. Open the inspector.
1. Navigate to the **Data** tab.
1. Click **Download traces** or **Download service graph**.
1. Click **Download traces** or **Download Service Graph**.
### Trace JSON example
@@ -268,18 +302,18 @@ If the file has multiple traces, Grafana visualizes its first trace.
}
```
## Use the service graph
## Use the Service Graph
A service graph is a visual representation of the relationships between services.
The Service Graph is a visual representation of the relationships between services.
Each node on the graph represents a service such as an API or database.
You use a service graph to detect performance issues; track increases in error, fault, or throttle rates in services; and investigate root causes by viewing corresponding traces.
You use the Service Graph to detect performance issues; track increases in error, fault, or throttle rates in services; and investigate root causes by viewing corresponding traces.
{{< figure src="/static/img/docs/node-graph/node-graph-8-0.png" class="docs-image--no-shadow" max-width="500px" caption="Screenshot of a Node Graph" >}}
**To display the service graph:**
**To display the Service Graph:**
1. [Configure Grafana Agent](/docs/tempo/latest/grafana-agent/service-graphs/#quickstart) or [Tempo or GET](/docs/tempo/latest/metrics-generator/service_graphs/#tempo) to generate service graph data.
1. [Configure Grafana Agent](/docs/tempo/latest/grafana-agent/service-graphs/#quickstart) or [Tempo or GET](/docs/tempo/latest/metrics-generator/service_graphs/#tempo) to generate Service Graph data.
1. Link a Prometheus data source in the Tempo data source's [Service Graph](#configure-service-graph) settings.
1. Navigate to [Explore]({{< relref "../../explore/" >}}).
1. Select the Tempo data source.
@@ -307,7 +341,7 @@ Each circle's color represents the percentage of requests in each state:
Service graph view displays a table of request rate, error rate, and duration metrics (RED) calculated from your incoming spans. It also includes a node graph view built from your spans.
{{< figure src="/static/img/docs/tempo/apm-table.png" class="docs-image--no-shadow" max-width="500px" caption="Screenshot of the Service Graph view table" >}}
{{< figure src="/static/img/docs/tempo/apm-table.png" class="docs-image--no-shadow" max-width="500px" caption="Screenshot of the Service Graph view" >}}
For details, refer to the [Service Graph view documentation](/docs/tempo/latest/metrics-generator/service-graph-view/).
@@ -316,18 +350,45 @@ To open the Service Graph view:
1. Link a Prometheus data source in the Tempo data source settings.
1. Navigate to [Explore]({{< relref "../../explore/" >}}).
1. Select the Tempo data source.
1. Select the **Service Graph** query type and run the query.
1. Select the **Service Graph** query type.
1. Run the query.
1. _(Optional)_ Filter your results.
> **Note:** Grafana uses the `traces_spanmetrics_calls_total` metric to display the name, rate, and error rate columns, and `traces_spanmetrics_latency_bucket` to display the duration column.
> These metrics must exist in your Prometheus data source.
{{% admonition type="note" %}}
Grafana uses the `traces_spanmetrics_calls_total` metric to display the name, rate, and error rate columns, and `traces_spanmetrics_latency_bucket` to display the duration column.
These metrics must exist in your Prometheus data source.
{{% /admonition %}}
To open a query in Prometheus with the span name of that row automatically set in the query, click a row in the **rate**, **error rate**, or **duration** columns.
To open a query in Tempo with the span name of that row automatically set in the query, click a row in the **links** column.
## Link a trace ID from logs
## Span Filters
You can link to Tempo trace from logs in [Loki](/docs/loki/latest) or Elasticsearch by configuring an internal link.
{{% admonition type="note" %}}
This feature is behind the `newTraceViewHeader` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
To configure this feature, see the [Derived fields]({{< relref "../loki#configure-derived-fields" >}}) section of the [Loki data source docs]({{< relref "../loki/" >}}), or the [Data links]({{< relref "../elasticsearch#data-links" >}}) section of the [Elasticsearch data source docs]({{< relref "../elasticsearch" >}}).
![Screenshot of span filtering](/media/docs/tempo/screenshot-grafana-tempo-span-filters.png)
Using span filters, you can filter your spans in the trace timeline viewer. The more filters you add, the more specific are the filtered spans.
You can add one or more of the following filters:
- Service name
- Span name
- Duration
- Tags (which include tags, process tags, and log fields)
## Link to a trace ID from logs
You can link to Tempo traces from logs in Loki, Elasticsearch, Splunk, and other logs data sources by configuring an internal link.
To configure this feature, see the [Derived fields]({{< relref "../loki#configure-derived-fields" >}}) section of the Loki data source docs or the [Data links]({{< relref "../elasticsearch#data-links" >}}) section of the Elasticsearch or Splunk data source docs.
## Link to a trace ID from metrics
You can link to Tempo traces from metrics in Prometheus data sources by configuring an exemplar.
To configure this feature, see the [introduction to exemplars]({{< relref "docs/grafana/latest/fundamentals/exemplars" >}}) documentation.
docs/sources/datasources/zipkin/_index.md
+172 -50
View File
@@ -15,73 +15,112 @@ weight: 1600
# Zipkin data source
Grafana ships with built-in support for Zipkin, an open source, distributed tracing system.
Grafana ships with built-in support for Zipkin, an open source, distributed tracing system. This topic explains configuration and queries specific to the Zipkin data source.
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management/" >}}).
Only users with the organization administrator role can add data sources.
Administrators can also [configure the data source via YAML]({{< relref "#provision-the-data-source" >}}) with Grafana's provisioning system.
Once you've added the Zipkin data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "#query-traces" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
Once you've added the Zipkin data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "#query-the-data-source" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
You can also [upload a JSON trace file]({{< relref "#upload-a-json-trace-file" >}}), [link to a trace ID from logs]({{< relref "#link-to-a-trace-id-from-logs" >}}), and [link to a trace ID from metrics]({{< relref "#link-to-a-trace-id-from-metrics" >}}).
## Configure the data source
**To access the data source configuration page:**
To configure basic settings for the data source, complete the following steps:
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the Zipkin data source.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `Zipkin` in the search bar.
1. Select **Zipkin**.
Set the data source's basic configuration options carefully:
The **Settings** tab of the data source is displayed.
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Defines whether this data source is pre-selected for new panels. |
| **URL** | Sets the URL of the Zipkin instance, such as `http://localhost:9411`. |
| **Basic Auth** | Enables basic authentication for the Zipkin data source. |
| **User** | Defines the user name for basic authentication. |
| **Password** | Defines the password for basic authentication. |
1. Set the data source's basic configuration options:
### Configure trace to logs
| Name | Description |
| -------------- | ------------------------------------------------------------------------ |
| **Name** | Sets the name you use to refer to the data source in panels and queries. |
| **Default** | Defines whether this data source is pre-selected for new panels. |
| **URL** | Sets the URL of the Zipkin instance, such as `http://localhost:9411`. |
| **Basic Auth** | Enables basic authentication for the Zipkin data source. |
| **User** | Defines the user name for basic authentication. |
| **Password** | Defines the password for basic authentication. |
{{< figure src="/static/img/docs/explore/traces-to-logs-settings-8-2.png" class="docs-image--no-shadow" caption="Screenshot of the trace to logs settings" >}}
### Trace to logs
> **Note:** Available in Grafana v7.4 and higher.
![Trace to logs settings](/media/docs/tempo/tempo-trace-to-logs-9-4.png)
{{% admonition type="note" %}}
Available in Grafana v7.4 and higher.
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to logs** setting configures the [trace to logs feature]({{< relref "../../explore/trace-integration" >}}) that is available when you integrate Grafana with Zipkin.
**To configure trace to logs:**
There are two ways to configure the trace to logs feature:
- Use a simplified configuration with default query, or
- Configure a custom query where you can use a [template language]({{< relref "../../dashboards/variables/variable-syntax">}}) to interpolate variables from the trace or span.
#### Use a simple configuration
1. Select the target data source.
1. Select which tags to use in the logs query. The tags you configure must be present in the spans attributes or resources for a trace to logs span link to appear.
1. Set start and end time shift. As the logs timestamps may not exactly match the timestamps of the spans in trace it may be necessary to search in larger or shifted time range to find the desired logs.
1. Select which tags to use in the logs query. The tags you configure must be present in the spans attributes or resources for a trace to logs span link to appear. You can optionally configure a new name for the tag. This is useful if the tag has dots in the name and the target data source does not allow using dots in labels. In that case, you can for example remap `http.status` to `http_status`.
1. Optionally, switch on the **Filter by trace ID** and/or **Filter by span ID** setting to further filter the logs if your logs consistently contain trace or span IDs.
- **Single tag**
- Configuring `job` as a tag and clicking on a span link will take you to your configured logs datasource with the query `{job='value from clicked span'}`.
- **Multiple tags**
- If multiple tags are used they will be concatenated so the logs query would look like `{job='value from clicked span', service='value from clicked span'}`.
- **Mapped tags**
- For a mapped tag `service.name` with value `service`, clicking on a span link will take you to your configured logs datasource with the query `{service='value from clicked span'}` instead of `{service.name='value from clicked span'}`.
- This is useful for instances where your tracing datasource tags and your logs datasource tags don't match one-to-one.
#### Configure a custom query
1. Select the target data source.
1. Set start and end time shift. Since the logs timestamps may not exactly match the timestamps of the spans in the trace, you may need to widen or shift the time range to find the desired logs.
1. Optionally, select tags to map. These tags can be used in the custom query with `${__tags}` variable. This variable will interpolate the mapped tags as list in an appropriate syntax for the data source and will only include the tags that were present in the span omitting those that weren't present. You can optionally configure a new name for the tag. This is useful when the tag has dots in the name and the target data source does not allow using dots in labels. For example, you can remap `http.status` to `http_status`. If you don't map any tags here, you can still use any tag in the query like this `method="${__span.tags.method}"`.
1. Skip **Filter by trace ID** and **Filter by span ID** settings as these cannot be used with a custom query.
1. Switch on **Use custom query**.
1. Specify a custom query to be used to query the logs. You can use various variables to make that query relevant for current span. The link will only be shown only if all the variables are interpolated with non-empty values to prevent creating an invalid query.
#### Variables that can be used in a custom query
To use a variable you need to wrap it in `${}`. For example `${__span.name}`.
| Variable name | Description |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **\_\_tags** | This variable uses the tag mapping from the UI to create a label matcher string in the specific data source syntax. The variable only uses tags that are present in the span. The link is still created even if only one of those tags is present in the span. You can use this if all tags are not required for the query to be useful. |
| **\_\_span.spanId** | The ID of the span. |
| **\_\_span.traceId** | The ID of the trace. |
| **\_\_span.duration** | The duration of the span. |
| **\_\_span.name** | Name of the span. |
| **\_\_span.tags** | Namespace for the tags in the span. To access a specific tag named `version`, you would use `${__span.tags.version}`. In case the tag contains dot, you have to access it as `${__span.tags["http.status"]}`. |
| **\_\_trace.traceId** | The ID of the trace. |
| **\_\_trace.duration** | The duration of the trace. |
| **\_\_trace.name** | The name of the trace. |
The following table describes the ways in which you can configure your trace to logs settings:
| Name | Description |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Sets the target data source. |
| **Tags** | Defines the tags to use in the logs query. Default is `'cluster', 'hostname', 'namespace', 'pod'`. |
| **Map tag names** | Enables configuring how Jaeger tag names map to logs label names. For example, map `service.name` to `service`. |
| **Span start time shift** | Shifts the start time for the logs query based on the span start time. To extend to the past, use a negative value. Use time interval units like `5s`, `1m`, `3h`. Default is `0`. |
| **Span end time shift** | Shifts the end time for the logs query based on the span end time. Use time interval units. Default is `0`. |
| **Filter by Trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by Span ID** | Toggles whether to append the span ID to the logs query. |
| Setting name | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Data source** | Defines the target data source. You can select only Loki or Splunk \[logs\] data sources. |
| **Span start time shift** | Shifts the start time for the logs query, based on the span's start time. You can use time units, such as `5s`, `1m`, `3h`. To extend the time to the past, use a negative value. Default: `0`. |
| **Span end time shift** | Shifts the end time for the logs query, based on the span's end time. You can use time units. Default: `0`. |
| **Tags** | Defines the tags to use in the logs query. Default: `cluster`, `hostname`, `namespace`, `pod`. You can change the tag name for example to remove dots from the name if they are not allowed in the target data source. For example, map `http.status` to `http_status`. |
| **Filter by trace ID** | Toggles whether to append the trace ID to the logs query. |
| **Filter by span ID** | Toggles whether to append the span ID to the logs query. |
| **Use custom query** | Toggles use of custom query with interpolation. |
| **Query** | Input to write custom query. Use variable interpolation to customize it with variables from span. |
### Configure trace to metrics
### Trace to metrics
> **Note:** This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
{{% admonition type="note" %}}
This feature is behind the `traceToMetrics` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
The **Trace to metrics** section configures the [trace to metrics feature](/blog/2022/08/18/new-in-grafana-9.1-trace-to-metrics-allows-users-to-navigate-from-a-trace-span-to-a-selected-data-source/).
The **Trace to metrics** setting configures the [trace to metrics feature](/blog/2022/08/18/new-in-grafana-9.1-trace-to-metrics-allows-users-to-navigate-from-a-trace-span-to-a-selected-data-source/) available when integrating Grafana with Zipkin.
Use the settings to select the target Prometheus data source, and create any desired linked queries.
To configure trace to metrics:
1. Select the target data source.
1. Create any desired linked queries.
| Setting name | Description |
| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -95,15 +134,15 @@ Each linked query consists of:
Interpolate tags using the `$__tags` keyword.
For example, when you configure the query `requests_total{$__tags}`with the tags `k8s.pod=pod` and `cluster`, the result looks like `requests_total{pod="nginx-554b9", cluster="us-east-1"}`.
### Enable Node Graph
### Node Graph
The **Node Graph** setting enables the [Node Graph visualization]({{< relref "../../panels-visualizations/visualizations/node-graph/" >}}), which is disabled by default.
Once enabled, Grafana displays the Node Graph after loading the trace view.
### Configure the span bar label
### Span bar
The **Span bar label** section helps you display additional information in the span bar row.
The **Span bar** setting helps you display additional information in the span bar row.
You can choose one of three options:
@@ -113,18 +152,75 @@ You can choose one of three options:
| **Duration** | _(Default)_ Displays the span duration on the span bar row. |
| **Tag** | Displays the span tag on the span bar row. You must also specify which tag key to use to get the tag value, such as `span.kind`. |
## Query traces
### Provision the data source
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning and available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning#data-sources" >}}).
#### Provisioning example
```yaml
apiVersion: 1
datasources:
- name: Zipkin
type: zipkin
uid: EbPG8fYoz
url: http://localhost:16686
access: proxy
basicAuth: true
basicAuthUser: my_user
readOnly: true
isDefault: false
jsonData:
tracesToLogsV2:
# Field with an internal link pointing to a logs data source in Grafana.
# datasourceUid value must match the uid value of the logs data source.
datasourceUid: 'loki'
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: ['job', 'instance', 'pod', 'namespace']
filterByTraceID: false
filterBySpanID: false
customQuery: true
query: 'method="${__span.tags.method}"'
tracesToMetrics:
datasourceUid: 'prom'
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
tags: [{ key: 'service.name', value: 'service' }, { key: 'job' }]
queries:
- name: 'Sample query'
query: 'sum(rate(traces_spanmetrics_latency_bucket{$__tags}[5m]))'
nodeGraph:
enabled: true
traceQuery:
timeShiftEnabled: true
spanStartTimeShift: '1h'
spanEndTimeShift: '-1h'
spanBar:
type: 'None'
secureJsonData:
basicAuthPassword: my_password
```
## Query the data source
You can query and display traces from Zipkin via [Explore]({{< relref "../../explore/" >}}).
This topic explains configuration and queries specific to the Zipkin data source.
For general documentation on querying data sources in Grafana, see [Query and transform data]({{< relref "../../panels-visualizations/query-transform-data" >}}).
### Query by trace ID
To query a particular trace:
1. Select the **TraceID** query type.
1. Enter the trace's ID into the **Trace ID** field.
{{< figure src="/static/img/docs/v70/zipkin-query-editor.png" class="docs-image--no-shadow" caption="Screenshot of the Zipkin query editor" >}}
To query by trace ID, enter it.
{{< figure src="/static/img/docs/v70/zipkin-query-editor-open.png" class="docs-image--no-shadow" caption="Screenshot of the Zipkin query editor with trace selector expanded" >}}
### Query by trace selector
To select a particular trace from all traces logged in the time range you have selected in Explore, you can also query by trace selector.
The trace selector has three levels of nesting:
@@ -133,6 +229,8 @@ The trace selector has three levels of nesting:
- Particular operation, part of the selected service
- Specific trace in which the selected operation occurred, represented by the root operation name and trace duration
{{< figure src="/static/img/docs/v70/zipkin-query-editor-open.png" class="docs-image--no-shadow" caption="Screenshot of the Zipkin query editor with trace selector expanded" >}}
## View data mapping in the trace UI
You can view Zipkin annotations in the trace view as logs with annotation value displayed under the annotation key.
@@ -168,8 +266,32 @@ If the file has multiple traces, Grafana visualizes its first trace.
]
```
## Link a trace ID from logs
## Span Filters
You can link to a Zipkin trace from logs in [Loki](/docs/loki/latest/) or Splunk by configuring a derived field with an internal link.
{{% admonition type="note" %}}
This feature is behind the `newTraceViewHeader` [feature toggle]({{< relref "../../setup-grafana/configure-grafana#feature_toggles" >}}).
If you use Grafana Cloud, open a [support ticket in the Cloud Portal](/profile/org#support) to access this feature.
{{% /admonition %}}
For details, refer to [Derived fields]({{< relref "../loki/#configure-derived-fields" >}}) section of the [Loki data source]({{< relref "../loki/" >}}) documentation.
![Screenshot of span filtering](/media/docs/tempo/screenshot-grafana-tempo-span-filters.png)
Using span filters, you can filter your spans in the trace timeline viewer. The more filters you add, the more specific are the filtered spans.
You can add one or more of the following filters:
- Service name
- Span name
- Duration
- Tags (which include tags, process tags, and log fields)
## Link to a trace ID from logs
You can link to Zipkin traces from logs in Loki, Elasticsearch, Splunk, and other logs data sources by configuring an internal link.
To configure this feature, see the [Derived fields]({{< relref "../loki#configure-derived-fields" >}}) section of the Loki data source docs or the [Data links]({{< relref "../elasticsearch#data-links" >}}) section of the Elasticsearch or Splunk data source docs.
## Link to a trace ID from metrics
You can link to Zipkin traces from metrics in Prometheus data sources by configuring an exemplar.
To configure this feature, see the [introduction to exemplars]({{< relref "docs/grafana/latest/fundamentals/exemplars" >}}) documentation.