public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: Refactor data sources content (#57573)

Browse Source 
* Docs: Revise data source index

* Docs: Consolidate data source administration docs

* Docs: Revise panels docs related to data sources

* Docs: Revise Alertmanager data source

* Docs: Reorganize AWS CloudWatch data source docs

* Docs: Reorganize Azure Monitor data source docs

* Docs: Move azuremonitor to azure-monitor

* Docs: Revise Elasticsearch docs

* Docs: Move Elasticsearch index into bundle

* Docs: Revise GCM docs

* Docs: Revise Graphite docs

* Docs: Move Graphite index into bundle

* Docs: Revise InfluxDB docs

* Docs: Revise Jaeger docs

* Docs: Move Jaeger index into bundle

* Docs: Revise Loki docs

* Docs: Move Loki index into bundle

* Docs: Revise MS SQL docs

* Docs: Move MS SQL index into bundle

* Docs: Revise Prometheus docs

* Docs: Move Prometheus index into bundle

* Docs: Revise Tempo docs

* Docs: Move Tempo index into bundle

* Docs: Revise TestData DB docs

* Docs: Move TestData DB index into bundle

* Docs: Revise Zipkin docs

* Docs: Move Zipkin index into bundle

* Docs: Move other data sources' index pages into bundles

* Docs: Revise frontmatter

* Fixing hugo markdown errors

* Docs: Add query editor and template var sections to overview doc

* Docs: Remove CTAs across data source docs

* Docs: Remove CTA

* Docs: Remove CTA

* Docs: Fix links, images, typos, and usage consistency.

* Docs: Fix typos

* Docs: Fix CI issues

* Update docs/sources/datasources/_index.md

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>

* Update docs/sources/datasources/_index.md

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>

* Docs: Fix query editor links

* Update docs/sources/panels-visualizations/_index.md

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>

* Update docs/sources/panels-visualizations/_index.md

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>

* Docs: Rebundle child pages per writers' toolkit

* Docs: Fix prettier for CI

* Docs: Fix relrefs from outside data sources docs

* Docs: Fix broken relrefs within datasources

* Docs: Fix relrefs to data sources docs

* Fixed some more refs

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
This commit is contained in:
Garrett Guillotte
2022-11-01 10:22:06 -05:00
committed by GitHub
co-authored by Torkel Ödegaard
parent 22648d8581
commit 852d069a3c
77 changed files with 5722 additions and 4410 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/datasources/aws-cloudwatch/_index.md
+154 -160
View File
@@ -1,37 +1,56 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/
- /docs/grafana/latest/datasources/cloudwatch/
description: Guide for using CloudWatch in Grafana
- /docs/grafana/latest/datasources/aws-cloudwatch/
- /docs/grafana/latest/datasources/aws-cloudwatch/provision-cloudwatch/
- /docs/grafana/latest/datasources/aws-cloudwatch/preconfig-cloudwatch-dashboards/
- /docs/grafana/latest/data-sources/aws-cloudwatch/
- /docs/grafana/latest/data-sources/aws-cloudwatch/provision-cloudwatch/
- /docs/grafana/latest/data-sources/aws-cloudwatch/preconfig-cloudwatch-dashboards/
description: Guide for using AWS CloudWatch in Grafana
keywords:
- grafana
- cloudwatch
- guide
title: AWS CloudWatch
menuTitle: AWS CloudWatch
title: AWS CloudWatch data source
weight: 200
---
# AWS CloudWatch data source
Grafana ships with built-in support for CloudWatch. This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source. For instructions on how to add a data source to Grafana, refer to [Add a data source]({{< relref "../add-a-data-source/" >}}). Only users with the organization admin role can add data sources.
Grafana ships with built-in support for AWS CloudWatch.
This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source.
Once you have added the Cloudwatch data source, you can build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs.
For instructions on how to add a data source to Grafana, refer to the [administration documentation]({{< relref "../../administration/data-source-management/" >}}).
Only users with the organization administrator role can add data sources.
Administrators can also [provision the data source]({{< relref "#provision-the-data-source" >}}) with Grafana's provisioning system, and should [control pricing]({{< relref "#control-pricing" >}}) and [manage service quotas]({{< relref "#manage-service-quotas" >}}) accordingly.
> **Note:** For troubleshooting issues when setting up the Cloudwatch data source, check the `/var/log/grafana/grafana.log` file.
Once you've added the data source, you can [configure it]({{< relref "#configure-the-data-source" >}}) so that your Grafana instance's users can create queries in its [query editor]({{< relref "./query-editor/" >}}) when they [build dashboards]({{< relref "../../dashboards/build-dashboards/" >}}) and use [Explore]({{< relref "../../explore/" >}}).
## Configure the CloudWatch data source
> **Note:** To troubleshoot issues while setting up the CloudWatch data source, check the `/var/log/grafana/grafana.log` file.
To access data source settings, hover your mouse over the **Configuration** (gear) icon, then click **Data Sources**, and then click the AWS Cloudwatch data source.
## Configure the data source
For authentication options and configuration details, see [AWS authentication]({{< relref "aws-authentication/" >}}) topic.
**To access the data source configuration page:**
### CloudWatch specific data source configuration
1. Hover the cursor over the **Configuration** (gear) icon.
1. Select **Data Sources**.
1. Select the CloudWatch data source.
#### IAM policies
### Configure AWS authentication
Grafana needs permissions granted via IAM to be able to read CloudWatch metrics and EC2 tags/instances/regions/alarms. You can attach these permissions to the IAM role or IAM user configured in the previous step.
A Grafana plugin's requests to AWS are made on behalf of an AWS Identity and Access Management (IAM) role or IAM user.
The IAM user or IAM role must have the associated policies to perform certain API actions.
##### Metrics only example:
For authentication options and configuration details, refer to [AWS authentication]({{< relref "./aws-authentication/" >}}).
#### IAM policy examples
To read CloudWatch metrics and EC2 tags, instances, regions, and alarms, you must grant Grafana permissions via IAM.
You can attach these permissions to the IAM role or IAM user you configured in [AWS authentication]({{< relref "./aws-authentication/" >}}).
**Metrics-only:**
```json
{
@@ -66,7 +85,7 @@ Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
}
```
##### Logs only example:
**Logs-only:**
```json
{
@@ -101,7 +120,7 @@ Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
}
```
##### Metrics and Logs example:
**Metrics and Logs:**
```json
{
@@ -149,178 +168,160 @@ Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
}
```
### Configure CloudWatch settings
#### Namespaces of Custom Metrics
Grafana is not able to load custom namespaces through the GetMetricData API. If you still want your custom metrics to show up in the fields in the query editor, you can specify the names of the namespaces containing the custom metrics in the _Namespaces of Custom Metrics_ field. The field accepts a multiple namespaces, separated by a comma.
Grafana can't load custom namespaces through the CloudWatch [GetMetricData API](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html).
To make custom metrics appear in the data source's query editor fields, specify the names of the namespaces containing the custom metrics in the data source configuration's _Namespaces of Custom Metrics_ field.
The field accepts multiple namespaces separated by commas.
#### Timeout
Timeout specifically, for CloudWatch Logs queries. Log queries don't recognize standard Grafana query timeout as they don't keep a single request open and instead periodically poll for results. Because of limits on concurrently running queries in CloudWatch they can also take a longer time to finish.
Configure the timeout specifically for CloudWatch Logs queries.
Log queries don't keep a single request open, and instead periodically poll for results.
Therefore, they don't recognize the standard Grafana query timeout.
Because of limits on concurrently running queries in CloudWatch, they can also take longer to finish.
#### X-Ray trace links
Link an X-Ray data source in the "X-Ray trace link" section of the configuration page to automatically add links in your logs when the log contains `@xrayTraceId` field.
To automatically add links in your logs when the log contains the `@xrayTraceId` field, link an X-Ray data source in the "X-Ray trace link" section of the data source configuration.
![Trace link configuration](/static/img/docs/cloudwatch/xray-trace-link-configuration-8-2.png 'Trace link configuration')
{{< figure src="/static/img/docs/cloudwatch/xray-trace-link-configuration-8-2.png" max-width="800px" class="docs-image--no-shadow" caption="Trace link configuration" >}}
The data source select will contain only existing data source instances of type X-Ray so in order to use this feature you need to have existing X-Ray data source already configured, see [X-Ray docs](https://grafana.com/grafana/plugins/grafana-x-ray-datasource/) for details.
The data source select contains only existing data source instances of type X-Ray.
To use this feature, you must already have an X-Ray data source configured.
For details, see the [X-Ray data source docs](/grafana/plugins/grafana-x-ray-datasource/).
The X-Ray link will then appear in the log details section which is accessible by clicking on the log row either in Explore or in dashboard [Logs panel]({{< relref "../../panels-visualizations/visualizations/logs/" >}}). To log the `@xrayTraceId` in your logs see the [AWS X-Ray documentation](https://docs.amazonaws.cn/en_us/xray/latest/devguide/xray-services.html). To provide the field to Grafana your log queries also have to contain the `@xrayTraceId` field, for example using query `fields @message, @xrayTraceId`.
To view the X-Ray link, select the log row in either the Explore view or dashboard [Logs panel]({{< relref "../../panels-visualizations/visualizations/logs" >}}) to view the log details section.
![Trace link in log details](/static/img/docs/cloudwatch/xray-link-log-details-8-2.png 'Trace link in log details')
To log the `@xrayTraceId`, see the [AWS X-Ray documentation](https://docs.amazonaws.cn/en_us/xray/latest/devguide/xray-services.html).
## CloudWatch query editor
To provide the field to Grafana, your log queries must also contain the `@xrayTraceId` field, for example by using the query `fields @message, @xrayTraceId`.
The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor. You select which API you want to query with using the query mode switch on top of the editor.
{{< figure src="/static/img/docs/cloudwatch/xray-link-log-details-8-2.png" max-width="800px" class="docs-image--no-shadow" caption="Trace link in log details" >}}
![CloudWatch API modes](/static/img/docs/cloudwatch/cloudwatch-query-editor-api-modes-8.3.0.png)
### Configure the data source with grafana.ini
### Metrics query editor
The Grafana [configuration file]({{< relref "../../setup-grafana/configure-grafana#aws" >}}) includes an `AWS` section where you can configure data source options:
The metrics query editor allows you to build two types of queries - **Metric Search** and **Metric Query**.
| Configuration option | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `allowed_auth_providers` | Specifies which authentication providers are allowed for the CloudWatch data source. The following providers are enabled by default in open-source Grafana: `default` (AWS SDK default), keys (Access and secret key), credentials (Credentials file), ec2_IAM_role (EC2 IAM role). |
| `assume_role_enabled` | Allows you to disable `assume role (ARN)` in the CloudWatch data source. The assume role (ARN) is enabled by default in open-source Grafana. |
| `list_metrics_page_limit` | Sets the limit of List Metrics API pages. When a custom namespace is specified in the query editor, the [List Metrics API](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) populates the _Metrics_ field and _Dimension_ fields. The API is paginated and returns up to 500 results per page, and the data source also limits the number of pages to 500 by default. This setting customizes that limit. |
#### Using the Metric Search option
### Provision the data source
To create a valid Metric Search query specify the namespace, metric name and at least one statistic.
You can define and configure the data source in YAML files as part of Grafana's provisioning system.
For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana]({{< relref "../../administration/provisioning/#data-sources" >}}).
If `Match Exact` is enabled, you also need to specify all the dimensions of the metric you’re querying, so that the [metric schema](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) matches exactly. If `Match Exact` is disabled, you can specify any number of dimensions by which you’d like to filter. Up to 100 metrics matching your filter criteria will be returned.
#### Provisioning examples
##### Dynamic queries using dimension wildcards
You can monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values.
![CloudWatch dimension wildcard](/static/img/docs/cloudwatch/cloudwatch-dimension-wildcard-8.3.0.png)
In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. When new instances are created as part of an auto scaling event, they will automatically appear in the graph without you having to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics.
You can expand the [Query inspector](https://grafana.com/docs/grafana/latest/panels/queries/#query-inspector-button) button and click `Meta Data` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with the name ‘InstanceId’ will be returned.
![CloudWatch Meta Inspector](/static/img/docs/cloudwatch/cloudwatch-meta-inspector-8.3.0.png)
You can disable `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.
##### Multi-value template variables
When defining dimension values based on multi-valued template variables, a search expression is used to query for the matching metrics. This enables the use of multiple template variables in one query and also allows you to use template variables for queries that have the `Match Exact` option disabled.
Search expressions are currently limited to 1024 characters, so your query may fail if you have a long list of values. We recommend using the asterisk (`*`) wildcard instead of the `All` option if you want to query all metrics that have any value for a certain dimension name.
The use of multi-valued template variables is only supported for dimension values. Using multi-valued template variables for `Region`, `Namespace`, or `Metric Name` is not supported.
##### Metric math expressions
You can create new time series metrics by operating on top of CloudWatch metrics using mathematical functions. Arithmetic operators, unary subtraction and other functions are supported and can be applied to CloudWatch metrics. More details on the available functions can be found on [AWS Metric Math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html)
As an example, if you want to apply arithmetic operations on a metric, you can do it by giving an id (a unique string) to the raw metric as shown below. You can then use this id and apply arithmetic operations to it in the Expression field of the new metric.
Please note that in the case you use the expression field to reference another query, like `queryA * 2`, it will not be possible to create an alert rule based on that query.
##### Deep linking from Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v65/cloudwatch-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch deep linking" >}}
Left clicking a time series in the panel shows a context menu with a link to `View in CloudWatch console`. Clicking that link will open a new tab that will take you to the CloudWatch console and display all the metrics for that query. If you're not currently logged in to the CloudWatch console, the link will forward you to the login page. The provided link is valid for any account but will only display the right metrics if you're logged in to the account that corresponds to the selected data source in Grafana.
This feature is not available for metrics that are based on metric math expressions.
### Using the Metric Query option
> **Note:** This query option is available in Grafana 8.3 and higher versions only.
Metrics Query in the CloudWatch plugin is what is referred to as **Metric Insights** in the AWS console. It's a fast, flexible, SQL-based query engine that enables you to identify trends and patterns across millions of operational metrics in real time. It uses a dialect of SQL. The query syntax is as follows.
**Using AWS SDK (default):**
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
defaultRegion: eu-west-2
```
SELECT FUNCTION(MetricName)
FROM Namespace | SCHEMA(...)
[ WHERE labelKey OPERATOR labelValue [AND|...]]
[ GROUP BY labelKey [, ...]]
[ ORDER BY FUNCTION() [DESC | ASC] ]
[ LIMIT number]
**Using credentials' profile name (non-default):**
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: credentials
defaultRegion: eu-west-2
customMetricsNamespaces: 'CWAgent,CustomNameSpace'
profile: secondary
```
The following table provides basic explanation of the query keywords. For details about the Metrics Insights syntax, refer to the [AWS documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights-syntax).
**Using accessKey and secretKey:**
| Keyword | Description |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FUNCTION` | Required. Specifies the aggregate function to use, and also specifies the name of the metric that is to be queried. Valid values are AVG, COUNT, MAX, MIN, and SUM |
| `MetricName` | Required. For example, `CPUUtilization`. |
| `FROM` | Required. Specifies the source of the metric. You can specify either the metric namespace that contains the metric that is to be queried, or a SCHEMA table function. Some namespace examples are 1`AWS/EC2`, `AWS/Lambda`. |
| `SCHEMA` | Optional. Allows you to narrow down the query results to only the metrics that is an exact match or to metrics that do noy match. |
| `WHERE` | Optional. Filters the results to only those metrics that match your specified expression. For example, `WHERE InstanceType != 'c3.4xlarge'`. |
| `GROUP BY` | Optional. Groups the query results into multiple time series. For example, `GROUP BY ServiceName`. |
| `ORDER BY` | Optional. Specifies the order of time series that are returned. Options are `ASC`, `DESC`. |
| `LIMIT` | Optional. Limits the number of time series returned. |
```yaml
apiVersion: 1
For information about limits for the Metrics Insights, please refer to the [AWS documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights).
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: keys
defaultRegion: eu-west-2
secureJsonData:
accessKey: '<your access key>'
secretKey: '<your secret key>'
```
**Builder mode**
**Using AWS SDK Default and ARN of IAM Role to Assume:**
To create a query in Builder mode:
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
assumeRoleArn: arn:aws:iam::123456789012:root
defaultRegion: eu-west-2
```
1. Browse and select a metric namespace, metric name, filter, group, and order options using information from the table above.
1. For each of these options, choose from the list of possible options.
## Query the data source
Grafana automatically constructs a SQL query based on your selections.
The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor.
**Code mode**
For details, see the [query editor documentation]({{< relref "./query-editor/" >}}).
To create a query in the Code mode:
## Use template variables
1. Write your SQL query.
1. To run the query, click the **Run query** above the code editor.
Instead of hard-coding details such as server, application, and sensor names in metric queries, you can use variables.
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the data displayed in your dashboard.
Grafana refers to such variables as template variables.
The code editor has a built in autocomplete feature that gives suggestions for keywords, aggregations, namespaces, metrics, labels and label values. The suggestions are shown when hitting space, comma or dollar character. You can also use the keyboard combination CTRL+Space.
For details, see the [template variables documentation]({{< relref "./template-variables/" >}}).
![Code editor autocomplete](/static/img/docs/cloudwatch/cloudwatch-code-editor-autocomplete-8.3.0.gif)
## Import pre-configured dashboards
> **Note:** Usage of template variables in the code editor might interfere the autocompletion.
The CloudWatch data source ships with curated and pre-configured dashboards for five of the most popular AWS services:
### Common metric query editor fields
- **Amazon Elastic Compute Cloud:** `Amazon EC2`
- **Amazon Elastic Block Store:** `Amazon EBS`
- **AWS Lambda:** `AWS Lambda`
- **Amazon CloudWatch Logs:** `Amazon CloudWatch Logs`
- **Amazon Relational Database Service:** `Amazon RDS`
At the bottom of the metric query editor, you'll find three fields that are common to both _Metric Search_ and _Metric Query_.
**To import curated dashboards:**
#### Id
1. Navigate to the data source's [configuration page]({{< relref "#configure-the-data-source" >}}).
1. Select the **Dashboards** tab.
The GetMetricData API requires that all queries have a unique ID. Use this field to specify an ID of choice. The ID can include numbers, letters, and underscore, and must start with a lowercase letter. If no ID is specified, grafana will generate an ID using the following pattern `query[refId of the current query row]`, e.g `queryA` for the first query row in the panel editor.
This displays the curated selection of importable dashboards.
The ID can be used to reference queries in Metric Math expressions.
1. Select **Import** for the dashboard to import.
#### Period
{{< figure src="/static/img/docs/v65/cloudwatch-dashboard-import.png" caption="CloudWatch dashboard import" >}}
A period is the length of time associated with a specific Amazon CloudWatch statistic. Periods are defined in numbers of seconds, and valid values for period are 1, 5, 10, 30, or any multiple of 60.
**To customize an imported dashboard:**
If the period field is left blank or set to `auto`, then it calculates automatically based on the time range and [cloudwatch's retention policy](https://aws.amazon.com/about-aws/whats-new/2016/11/cloudwatch-extends-metrics-retention-and-new-user-interface/). The formula used is `time range in seconds / 2000`, and then it snaps to the next higher value in an array of predefined periods `[60, 300, 900, 3600, 21600, 86400]` after removing periods based on retention. By clicking `Show Query Preview` in the query editor, you can see what period Grafana used.
To customize one of these dashboards, we recommend that you save it under a different name.
If you don't, upgrading Grafana can overwrite the customized dashboard with the new version.
#### Label
## Create queries for alerting
The label field allows you to override the default name of the metric legend using CloudWatch dynamic labels. If you're using a time-based dynamic label such as `${MIN_MAX_TIME_RANGE}`, then the legend value is derived from the current timezone specified in the time range picker. To see the full list of label patterns and the dynamic label limitations, refer to the [CloudWatch dynamic labels](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/graph-dynamic-labels.html) documentation.
Alerting requires queries that return numeric data, which CloudWatch Logs support.
For example, you can enable alerts through the use of the `stats` command.
**Alias pattern deprecation:** In Grafana 9, dynamic labels have replaced alias patterns in the CloudWatch data source. Any existing alias pattern will get migrated to a corresponding dynamic label pattern. If you wish to use alias patterns instead of dynamic labels, set the feature toggle `cloudWatchDynamicLabels` to `false` in the Grafana configuration file. It will revert to the alias pattern system and use the previous alias formatting logic.
The alias field will be deprecated and removed in a release. During this interim period, we won’t fix bugs related to the alias pattern system. For details on why we're doing this change, refer to [issue 48434](https://github.com/grafana/grafana/issues/48434).
## Using the Logs query editor
To query CloudWatch Logs:
1. Select the region and up to 20 log groups which you want to query.
1. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html).
You can also write queries returning time series data by using the [`stats` command](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_Insights-Visualizing-Log-Data.html). When making `stats` queries in Explore, you have to make sure you are in Metrics Explore mode.
{{< figure src="/static/img/docs/v70/explore-mode-switcher.png" max-width="500px" class="docs-image--right" caption="Explore mode switcher" >}}
### Deep linking from Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v70/cloudwatch-logs-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch Logs deep linking" >}}
If you'd like to view your query in the CloudWatch Logs Insights console, simply click the `CloudWatch Logs Insights` button next to the query editor.
If you're not currently logged in to the CloudWatch console, the link will forward you to the login page. The provided link is valid for any account but will only display the right metrics if you're logged in to the account that corresponds to the selected data source in Grafana.
## Alerting
Alerting require queries that return numeric data, which CloudWatch Logs support. For example through the use of the `stats` command, alerts are supported. For example, this is a valid query for alerting on messages that include the text "Exception":
This is also a valid query for alerting on messages that include the text "Exception":
```
filter @message like /Exception/
@@ -328,36 +329,29 @@ filter @message like /Exception/
| sort exceptionCount desc
```
**NOTE**: When trying to alert on a query, if an error like `input data must be a wide series but got ...` is received, make sure that your query returns valid numeric data that can be printed in a Time series panel.
> **Note:** If you receive an error like `input data must be a wide series but got ...` when trying to alert on a query, make sure that your query returns valid numeric data that can be output to a Time series panel.
For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting/" >}}) documentation.
For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting" >}}).
## Configure CloudWatch with grafana.ini
## Control pricing
The Grafana [configuration]({{< relref "../../setup-grafana/configure-grafana/#aws" >}}) file includes an `AWS` section where you can customize the data source.
| Configuration option | Description |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `allowed_auth_providers` | Specifies which authentication providers are allowed for the CloudWatch data source. The following providers are enabled by default in OSS Grafana: `default` (AWS SDK default), keys (Access and secret key), credentials (Credentials file), ec2_IAM_role (EC2 IAM role). |
| `assume_role_enabled` | Allows you to disable `assume role (ARN)` in the CloudWatch data source. By default, assume role (ARN) is enabled for OSS Grafana. |
| `list_metrics_page_limit` | When a custom namespace is specified in the query editor, the [List Metrics API](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) is used to populate the _Metrics_ field and the _Dimension_ fields. The API is paginated and returns up to 500 results per page. The CloudWatch data source also limits the number of pages to 500. However, you can change this limit using the `list_metrics_page_limit` variable in the [grafana configuration file](https://grafana.com/docs/grafana/latest/administration/configuration/#aws). |
## Pricing
The Amazon CloudWatch data source for Grafana uses the `ListMetrics` and `GetMetricData` CloudWatch API calls to list and retrieve metrics.
The Amazon CloudWatch data source for Grafana uses [`ListMetrics`](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) and [`GetMetricData`](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html) CloudWatch API calls to list and retrieve metrics.
Pricing for CloudWatch Logs is based on the amount of data ingested, archived, and analyzed via CloudWatch Logs Insights queries.
Every time you pick a dimension in the query editor Grafana will issue a ListMetrics request. Whenever you make a change to the queries in the query editor, one new request to GetMetricData will be issued.
Each time you select a dimension in the query editor, Grafana issues a `ListMetrics` API request.
Each time you change queries in the query editor, Grafana issues a new request to the `GetMetricData` API.
In Grafana version 6.5 or higher, all API requests to GetMetricStatistics have been replaced with calls to GetMetricData to provide better support for CloudWatch metric math and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option. While GetMetricStatistics qualified for the CloudWatch API free tier, this is not the case for GetMetricData calls.
> **Note:** Grafana v6.5 and higher replaced all `GetMetricStatistics` API requests with calls to GetMetricData to provide better support for CloudWatch metric math, and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option.
> The `GetMetricStatistics` API qualified for the CloudWatch API free tier, but `GetMetricData` calls don't.
For more information, please refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
For more information, refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
## Service quotas
## Manage service quotas
AWS defines quotas, or limits, for resources, actions, and items in your AWS account. Depending on the number of queries in your dashboard and the number of users accessing the dashboard, you may reach the usage limits for various CloudWatch and CloudWatch Logs resources. Note that quotas are defined per account and per region. If you're using multiple regions or have set up more than one CloudWatch data source to query against multiple accounts, you need to request a quota increase for each account and each region in which you hit the limit.
AWS defines quotas, or limits, for resources, actions, and items in your AWS account.
Depending on the number of queries in your dashboard and the number of users accessing the dashboard, you might reach the usage limits for various CloudWatch and CloudWatch Logs resources.
Quotas are defined per account and per region.
To request a quota increase, visit the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/home?r#!/services/monitoring/quotas/L-5E141212). For more information, refer to the AWS documentation for [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) and [CloudWatch limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html).
If you use multiple regions or configured more than one CloudWatch data source to query against multiple accounts, you must request a quota increase for each account and region in which you reach the limit.
```
```
To request a quota increase, visit the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/home?r#!/services/monitoring/quotas/L-5E141212).
For more information, refer to the AWS documentation for [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) and [CloudWatch limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html).
docs/sources/datasources/aws-cloudwatch/aws-authentication.md
-78
View File
@@ -1,78 +0,0 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/aws-authentication/
- /docs/grafana/latest/datasources/cloudwatch/
description: AWS authentication
keywords:
- grafana
- aws
- authentication
title: Authentication
weight: 5
---
# AWS authentication
Requests from a Grafana plugin to AWS are made on behalf of an IAM role or an IAM user. The IAM user or IAM role must have the associated policies to perform certain API actions. Since these policies are specific to each data source, refer to the data source documentation for details.
All requests to AWS APIs are performed on the server side by the Grafana backend using the official AWS SDK.
This topic has the following sections:
- [Authentication methods](#authentication-methods)
- [Assuming a role](#assuming-a-role)
- [Endpoint](#endpoint)
- [AWS credentials file](#aws-credentials-file)
- [EKS IAM roles for service accounts](#eks-iam-roles-for-service-accounts)
## Authentication methods
You can use one of the following authentication methods. Currently, `AWS SDK Default`, `Credentials file` and `Access and secret key` are enabled by default in open source Grafana. You can enable/disable them if necessary if you have server configuration access. For more information, refer to [allowed_auth_providers]({{< relref "../../setup-grafana/configure-grafana/#allowed_auth_providers" >}}) documentation.
- `AWS SDK Default` performs no custom configuration and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. It requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html), or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
- `Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. It reads the AWS shared credentials file to find a given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. This option doesn't have any implicit fallbacks to other credential providers, and it fails if the credentials provided from the file aren't correct.
- `Access and secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work.
- `Workspace IAM role` corresponds to the [EC2RoleProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/ec2rolecreds/#EC2RoleProvider). The EC2RoleProvider pulls credentials for a role attached to the EC2 instance that Grafana runs on. You can also achieve this by using the authentication method AWS SDK Default, but this option is different as it doesn’t have any fallbacks. This option is currently only enabled by default in Amazon Managed Grafana.
## Assuming a role
The `Assume Role ARN` field allows you to specify which IAM role to assume. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call.
You can disable this feature in the Grafana configuration. For more information, refer to [assume_role_enabled]({{< relref "../../setup-grafana/configure-grafana/#assume_role_enabled" >}}) documentation.
### External ID
If you are assuming a role in another account that was created with an external ID, then specify the external ID in this field. For more information, refer to the [AWS documentation on external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html).
## Endpoint
The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the AWS service API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).
## AWS credentials file
Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server.
> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
Example content:
```bash
[default]
aws_access_key_id = asdsadasdasdasd
aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```
## EKS IAM roles for service accounts
The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. To allow user 472 to access the credentials (and avoid falling back to the IAM role attached to the EC2 instance), you need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.
```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```
docs/sources/datasources/aws-cloudwatch/aws-authentication/index.md
+106
View File
@@ -0,0 +1,106 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/aws-authentication/
- /docs/grafana/latest/datasources/cloudwatch/
- /docs/grafana/latest/data-sources/aws-cloudwatch/aws-authentication/
- /docs/grafana/latest/data-sources/elasticsearch/aws-authentication/
description: Guide to configuring AWS authentication in Grafana
keywords:
- grafana
- aws
- authentication
menuTitle: AWS authentication
title: Configure AWS authentication
weight: 200
---
# Configure AWS authentication
A Grafana plugin's requests to AWS are made on behalf of an AWS Identity and Access Management (IAM) role or IAM user.
The IAM user or IAM role must have the associated policies to perform certain API actions.
Since these policies are specific to each data source, refer to the data source documentation for details.
All requests to AWS APIs are performed on the server side by the Grafana backend using the official AWS SDK.
This topic has the following sections:
- [Select an authentication method]({{< relref "#select-an-authentication-method" >}})
- [Assume a role]({{< relref "#assume-a-role" >}})
- [Use a custom endpoint]({{< relref "#use-a-custom-endpoint" >}})
- [Use an AWS credentials file]({{< relref "#use-an-aws-credentials-file" >}})
- [Use EKS IAM roles for service accounts]({{< relref "#use-eks-iam-roles-for-service-accounts" >}})
## Select an authentication method
You can use one of the following authentication methods.
Open source Grafana enables the `AWS SDK Default`, `Credentials file`, and `Access and secret key` methods by default.
- `AWS SDK Default` performs no custom configuration and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go.
It requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html), or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
- `Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK.
It reads the AWS shared credentials file to find a given profile.
While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables.
This option doesn't have any implicit fallbacks to other credential providers, and it fails if the credentials provided from the file aren't correct.
- `Access and secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate.
This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work.
- `Workspace IAM role` corresponds to the [EC2RoleProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/ec2rolecreds/#EC2RoleProvider).
The EC2RoleProvider pulls credentials for a role attached to the EC2 instance that Grafana runs on.
You can also achieve this by using the authentication method AWS SDK Default, but this option is different as it doesn't have any fallbacks.
This option is enabled by default only in Amazon Managed Grafana.
If necessary, you can enable or disable them if you have server configuration access.
For more information, refer to the [`allowed_auth_providers` documentation]({{< relref "../../../setup-grafana/configure-grafana#allowed_auth_providers" >}}).
## Assume a role
You can specify which IAM role to assume in the **Assume Role ARN** field.
If this field is left blank, Grafana uses the provided credentials directly, and the associated role or user should have the required permissions.
If this field isn't blank, Grafana uses the provided credentials to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call.
To disable this feature, refer to the [`assume_role_enabled` documentation]({{< relref "../../../setup-grafana/configure-grafana#assume_role_enabled" >}}).
### Use an external ID
To assume a role in another account that was created with an external ID, specify the external ID in the **External ID** field.
For more information, refer to the [AWS documentation on external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html).
## Use a custom endpoint
You can specify a custom endpoint URL in the **Endpoint** field, which overrides the default generated endpoint for the AWS service API.
Leave this field blank to use the default generated endpoint.
For more information on why and how to use service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).
## Use an AWS credentials file
Create a file at `~/.aws/credentials`, the `HOME` path for the user running the `grafana-server` service.
> **Note:** If you think you have the credentials file in the right location, but it's not working, try moving your `.aws` file to `/usr/share/grafana/` and grant your credentials file at most 0644 permissions.
### Credentials file example
```bash
[default]
aws_access_key_id = asdsadasdasdasd
aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```
## Use EKS IAM roles for service accounts
The Grafana process in the container runs as user 472 (called "grafana").
When Kubernetes mounts your projected credentials, they're available by default to only the root user.
To grant user 472 permission to access the credentials, and avoid falling back to the IAM role attached to the EC2 instance, you must provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.
### Security context example
```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```
docs/sources/datasources/aws-cloudwatch/preconfig-cloudwatch-dashboards.md
-35
View File
@@ -1,35 +0,0 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/preconfig-cloudwatch-dashboards/
- /docs/grafana/latest/datasources/cloudwatch/
description: Guide for using AWS CloudWatch in Grafana
keywords:
- grafana
- stackdriver
- google
- guide
- cloud
- monitoring
title: Curated CloudWatch dashboards
weight: 15
---
# Curated CloudWatch dashboards
The updated CloudWatch data source ships with pre-configured dashboards for five of the most popular AWS services:
- Amazon Elastic Compute Cloud `Amazon EC2`,
- Amazon Elastic Block Store `Amazon EBS`,
- AWS Lambda `AWS Lambda`,
- Amazon CloudWatch Logs `Amazon CloudWatch Logs`, and
- Amazon Relational Database Service `Amazon RDS`.
To import curated dashboards:
1. On the configuration page of your CloudWatch data source, click the **Dashboards** tab.
1. Click **Import** for the dashboard you would like to use.
In case you want to customize a dashboard, we recommend that you save it under a different name. Otherwise the dashboard will be overwritten when a new version of the dashboard is released.
{{< figure src="/static/img/docs/v65/cloudwatch-dashboard-import.png" caption="CloudWatch dashboard import" >}}
docs/sources/datasources/aws-cloudwatch/provision-cloudwatch.md
-70
View File
@@ -1,70 +0,0 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/provision-cloudwatch/
- /docs/grafana/latest/datasources/cloudwatch/
description: Guide for provisioning CloudWatch
title: Provision CloudWatch
weight: 400
---
# Provision CloudWatch data source
You can configure the CloudWatch data source by customizing configuration files in Grafana's provisioning system. To know more about provisioning and learn about available configuration options, refer to the [Provisioning Grafana]({{< relref "../../administration/provisioning/#datasources" >}}) topic.
Here are some provisioning examples for this data source.
## Using AWS SDK (default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
defaultRegion: eu-west-2
```
## Using credentials' profile name (non-default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: credentials
defaultRegion: eu-west-2
customMetricsNamespaces: 'CWAgent,CustomNameSpace'
profile: secondary
```
## Using accessKey and secretKey
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: keys
defaultRegion: eu-west-2
secureJsonData:
accessKey: '<your access key>'
secretKey: '<your secret key>'
```
## Using AWS SDK Default and ARN of IAM Role to Assume
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
assumeRoleArn: arn:aws:iam::123456789012:root
defaultRegion: eu-west-2
```
docs/sources/datasources/aws-cloudwatch/query-editor/index.md
+224
View File
@@ -0,0 +1,224 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/
- /docs/grafana/latest/datasources/cloudwatch/
- /docs/grafana/latest/data-sources/aws-cloudwatch/query-editor/
description: Guide for using the AWS CloudWatch data source's query editor
keywords:
- grafana
- aws
- cloudwatch
- guide
- queries
menuTitle: Query editor
title: AWS CloudWatch query editor
weight: 300
---
# AWS CloudWatch query editor
This topic explains querying specific to the CloudWatch data source.
For general documentation on querying data sources in Grafana, see [Query and transform data]({{< relref "../../../panels-visualizations/query-transform-data" >}}).
## Choose a query editing mode
The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor.
- [CloudWatch metrics]({{< relref "#query-cloudwatch-metrics" >}})
- [CloudWatch Logs]({{< relref "#query-cloudwatch-logs" >}})
{{< figure src="/static/img/docs/cloudwatch/cloudwatch-query-editor-api-modes-8.3.0.png" max-width="500px" class="docs-image--right" caption="CloudWatch API modes" >}}
Select which API to query by using the query mode switch on top of the editor.
## Query CloudWatch metrics
You can build two types of queries with the CloudWatch query editor:
- [Metric Search]({{< relref "#create-a-metric-search-query" >}})
- [Metric Query]({{< relref "#create-a-metric-insights-query" >}}), which uses the Metrics Insights feature
### Create a Metric Search query
To create a valid Metric Search query, specify the namespace, metric name, and at least one statistic.
If you enable `Match Exact`, you must also specify all dimensions of the metric you're querying so that the [metric schema](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) matches exactly.
If `Match Exact` is disabled, you can specify any number of dimensions on which you'd like to filter.
The data source returns up to 100 metrics matching your filter criteria.
You can also augment queries by using [template variables]({{< relref "./template-variables/" >}}).
#### Create dynamic queries with dimension wildcards
Use the asterisk (`*`) wildcard for one or more dimension values to monitor a dynamic list of metrics.
{{< figure src="/static/img/docs/cloudwatch/cloudwatch-dimension-wildcard-8.3.0.png" max-width="500px" class="docs-image--right" caption="CloudWatch dimension wildcard" >}}
In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization`, and also queries ANY value for the `InstanceId` dimension.
This can help you monitor metrics for AWS resources, like EC2 instances or containers.
When an auto-scaling event creates new instances, they automatically appear in the graph without you having to track the new instance IDs.
This capability is currently limited to retrieving up to 100 metrics.
You can expand the [Query inspector]({{< relref "../../../panels-visualizations/query-transform-data/#navigate-the-query-tab" >}}) button and click `Meta Data` to see the search expression that's automatically built to support wildcards.
To learn more about search expressions, refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html).
The search expression is defined by default in such a way that the queried metrics must match the defined dimension names exactly.
This means that in the example, the query returns only metrics with exactly one dimension containing the name 'InstanceId'.
{{< figure src="/static/img/docs/cloudwatch/cloudwatch-meta-inspector-8.3.0.png" max-width="500px" class="docs-image--right" caption="CloudWatch Meta Inspector" >}}
You can disable `Match Exact` to include metrics that have other dimensions defined.
Disabling `Match Exact` also creates a search expression even if you don't use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.
#### Use multi-value template variables
When defining dimension values based on multi-valued template variables, the data source uses a search expression to query for the matching metrics.
This enables the use of multiple template variables in one query, and also lets you use template variables for queries that have the `Match Exact` option disabled.
Search expressions are limited to 1,024 characters, so your query might fail if you have a long list of values.
We recommend using the asterisk (`*`) wildcard instead of the `All` option to query all metrics that have any value for a certain dimension name.
The use of multi-valued template variables is supported only for dimension values.
Using multi-valued template variables for `Region`, `Namespace`, or `Metric Name` is not supported.
#### Use metric math expressions
You can create new time series metrics by operating on top of CloudWatch metrics using mathematical functions.
This includes support for arithmetic operators, unary subtraction, and other functions, and can be applied to CloudWatch metrics.
For details on the available functions, refer to [AWS Metric Math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html).
For example, to apply arithmetic operations to a metric, apply a unique string id to the raw metric, then use this id and apply arithmetic operations to it in the Expression field of the new metric.
> **Note:** If you use the expression field to reference another query, like `queryA * 2`, you can't create an alert rule based on that query.
#### Deep-link Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v65/cloudwatch-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch deep linking" >}}
Left-clicking a time series in the panel shows a context menu with a link to `View in CloudWatch console`.
Clicking that link opens a new tab that takes you to the CloudWatch console and displays all metrics for that query.
If you're not logged in to the CloudWatch console, the link forwards you to the login page.
The provided link is valid for any account but displays the expected metrics only if you're logged in to the account that corresponds to the selected data source in Grafana.
This feature is not available for metrics based on [metric math expressions](#metric-math-expressions).
### Create a Metric Insights query
> **Note:** This query option is available only in Grafana v8.3 and higher.
The Metrics Query option in the CloudWatch data source is referred to as **Metric Insights** in the AWS console.
It's a fast, flexible, SQL-based query engine that you can use to identify trends and patterns across millions of operational metrics in real time.
The metrics query editor's Metrics Query option has two editing modes:
- [Builder mode](#create-a-query-in-builder-mode), which provides a visual query-building interface
- [Code mode](#create-a-query-in-code-mode), which provides a code editor for writing queries
#### Use Metric Insights syntax
Metric Insights uses a dialect of SQL and this query syntax:
```sql
SELECT FUNCTION(MetricName)
FROM Namespace | SCHEMA(...)
[ WHERE labelKey OPERATOR labelValue [AND|...]]
[ GROUP BY labelKey [, ...]]
[ ORDER BY FUNCTION() [DESC | ASC] ]
[ LIMIT number]
```
For details about the Metrics Insights syntax, refer to the [AWS reference documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights-syntax).
For information about Metrics Insights limits, refer to the [AWS feature documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights).
You can also augment queries by using [template variables]({{< relref "./template-variables/" >}}).
#### Use Metrics Insights keywords
This table summarizes common Metrics Insights query keywords:
| Keyword | Description |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `FUNCTION` | Required. Specifies the aggregate function to use, and also specifies the name of the metric to be queried. Valid values are AVG, COUNT, MAX, MIN, and SUM. |
| `MetricName` | Required. For example, `CPUUtilization`. |
| `FROM` | Required. Specifies the metric's source. You can specify either the metric namespace that contains the metric to be queried, or a SCHEMA table function. Namespace examples include `AWS/EC2`, `AWS/Lambda`. |
| `SCHEMA` | Optional. Narrows the query results to only the metrics that are an exact match, or to metrics that do not match. |
| `WHERE` | Optional. Filters the query results to only the metrics that match your specified expression. For example, `WHERE InstanceType != 'c3.4xlarge'`. |
| `GROUP BY` | Optional. Groups the query results into multiple time series. For example, `GROUP BY ServiceName`. |
| `ORDER BY` | Optional. Specifies the order in which time series are returned. Options are `ASC`, `DESC`. |
| `LIMIT` | Optional. Limits the number of time series returned. |
#### Create a query in Builder mode
**To create a query in Builder mode:**
1. Browse and select a metric namespace, metric name, filter, group, and order options using information from the [Metrics Insights keywords table](#metrics-insights-keywords).
1. For each of these keywords, choose from the list of possible options.
Grafana constructs a SQL query based on your selections.
#### Create a query in Code mode
You can also write your SQL query directly in a code editor by using Code mode.
The code editor has a built-in autocomplete feature that suggests keywords, aggregations, namespaces, metrics, labels, and label values.
The suggestions appear after typing a space, comma, or dollar (`$`) character, or the keyboard combination <key>CTRL</key>+<key>Space</key>.
{{< figure src="/static/img/docs/cloudwatch/cloudwatch-code-editor-autocomplete-8.3.0.png" max-width="500px" class="docs-image--right" caption="Code editor autocomplete" >}}
> **Note:** Template variables in the code editor can interfere with autocompletion.
To run the query, click **Run query** above the code editor.
### Common query editor fields
Three fields located at the bottom of the metrics query editor are common to both Metric Search and Metric Query editors.
#### Id
The GetMetricData API requires that all queries have a unique ID. Use this field to specify an ID of choice. The ID can include numbers, letters, and underscore, and must start with a lowercase letter. If no ID is specified, grafana will generate an ID using the following pattern `query[refId of the current query row]`, such as `queryA` for the first query row in the panel editor.
The ID can be used to reference queries in Metric Math expressions.
#### Period
A period is the length of time associated with a specific Amazon CloudWatch statistic. Periods are defined in numbers of seconds, and valid values for period are 1, 5, 10, 30, or any multiple of 60.
If the period field is left blank or set to `auto`, then it calculates automatically based on the time range and [cloudwatch's retention policy](https://aws.amazon.com/about-aws/whats-new/2016/11/cloudwatch-extends-metrics-retention-and-new-user-interface/). The formula used is `time range in seconds / 2000`, and then it snaps to the next higher value in an array of predefined periods `[60, 300, 900, 3600, 21600, 86400]` after removing periods based on retention. By clicking `Show Query Preview` in the query editor, you can see what period Grafana used.
#### Label
The label field allows you to override the default name of the metric legend using CloudWatch dynamic labels. If you're using a time-based dynamic label such as `${MIN_MAX_TIME_RANGE}`, then the legend value is derived from the current timezone specified in the time range picker. To see the full list of label patterns and the dynamic label limitations, refer to the [CloudWatch dynamic labels](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/graph-dynamic-labels.html) documentation.
> **Alias pattern deprecation:** Since Grafana v9.0, dynamic labels replaced alias patterns in the CloudWatch data source.
> Any existing alias pattern is migrated upon upgrade to a corresponding dynamic label pattern.
> To use alias patterns instead of dynamic labels, set the feature toggle `cloudWatchDynamicLabels` to `false` in the [Grafana configuration file]({{< relref "../../../setup-grafana/configure-grafana/" >}}).
> This reverts to the alias pattern system and uses the previous alias formatting logic.
The alias field will be deprecated and removed in a release.
During this interim period, we won't fix bugs related to the alias pattern system.
For details on why we're changing this feature, refer to [issue #48434](https://github.com/grafana/grafana/issues/48434).
## Query CloudWatch Logs
The logs query editor helps you write CloudWatch Logs Query Language queries across defined regions and log groups.
### Create a CloudWatch Logs query
1. Select the region and up to 20 log groups to query.
1. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html).
You can also write queries returning time series data by using the [`stats` command](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_Insights-Visualizing-Log-Data.html).
When making `stats` queries in [Explore]({{< relref "../../../explore/" >}}), make sure you are in Metrics Explore mode.
{{< figure src="/static/img/docs/v70/explore-mode-switcher.png" max-width="500px" class="docs-image--right" caption="Explore mode switcher" >}}
### Deep-link Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v70/cloudwatch-logs-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch Logs deep linking" >}}
To view your query in the CloudWatch Logs Insights console, click the `CloudWatch Logs Insights` button next to the query editor.
If you're not logged in to the CloudWatch console, the link forwards you to the login page.
The provided link is valid for any account, but displays the expected metrics only if you're logged in to the account that corresponds to the selected data source in Grafana.
docs/sources/datasources/aws-cloudwatch/template-queries-cloudwatch.md
-81
View File
@@ -1,81 +0,0 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/template-queries-cloudwatch/
- /docs/grafana/latest/datasources/cloudwatch/
description: Template variables in CloudWatch queryh
title: Template variables in CloudWatch query
weight: 10
---
# Using template variables in CloudWatch queries
Instead of hard-coding server, application, and sensor names in your metric queries, you can use variables. The variables are listed as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the display of data in your dashboard.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../dashboards/variables" >}}) documentation.
## Query variable
You can use the following CloudWatch data source queries to specify the `Query Type` field in the Variable edit view. Use them to fill a variables options list with values like `regions`, `namespaces`, `metric names`, and `dimension keys/values`.
Read more about the available dimensions in the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
| Name | Description |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Regions` | Returns a list of all AWS regions |
| `Namespaces` | Returns a list of all the namespaces CloudWatch supports. |
| `Metrics` | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics) |
| `Dimension Keys` | Returns a list of dimension keys in the namespace. |
| `Dimension Values` | Returns a list of dimension values matching the specified `region`, `namespace`, `metric`, and `dimension_key`. You can use dimension `filters` to get more specific results. |
| `EBS Volume IDs` | Returns a list of volume ids matching the specified `region` and `instance_id`. |
| `EC2 Instance Attributes` | Returns a list of attributes matching the specified `region`, `attribute_name`, and `filters`. |
| `Resource ARNs` | Returns a list of ARNs matching the specified `region`, `resource_type` and `tags`. |
| `Statistics` | Returns a list of all the standard statistics. |
| `LogGroups` | Returns a list of all log groups matching the specified `region`. |
For details about the metrics CloudWatch provides, please refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html).
### Using variables in queries
Variables can be used in the variable form. Refer to the [variable syntax documentation]({{< relref "../../dashboards/variables/variable-syntax" >}}).
## ec2_instance_attribute examples
### Filters
The `ec2_instance_attribute` query takes in `filters` as a filter name and a comma-separated list of values.
You can specify [pre-defined filters of ec2:DescribeInstances](http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html).
### Selecting attributes
Only 1 attribute per instance can be returned. Any flat attribute can be selected (i.e. if the attribute has a single value and isn't an object or array). Below is a list of available flat attributes:
- `AmiLaunchIndex`
- `Architecture`
- `ClientToken`
- `EbsOptimized`
- `EnaSupport`
- `Hypervisor`
- `IamInstanceProfile`
- `ImageId`
- `InstanceId`
- `InstanceLifecycle`
- `InstanceType`
- `KernelId`
- `KeyName`
- `LaunchTime`
- `Platform`
- `PrivateDnsName`
- `PrivateIpAddress`
- `PublicDnsName`
- `PublicIpAddress`
- `RamdiskId`
- `RootDeviceName`
- `RootDeviceType`
- `SourceDestCheck`
- `SpotInstanceRequestId`
- `SriovNetSupport`
- `SubnetId`
- `VirtualizationType`
- `VpcId`
You can select tags by prepending the tag name with `Tags.`. For example, the tag `Name` is selected with `Tags.Name`.
docs/sources/datasources/aws-cloudwatch/template-variables/index.md
+94
View File
@@ -0,0 +1,94 @@
---
aliases:
- /docs/grafana/latest/datasources/aws-cloudwatch/template-queries-cloudwatch/
- /docs/grafana/latest/data-sources/aws-cloudwatch/template-variables/
description: Guide on using template variables in CloudWatch queries
keywords:
- grafana
- aws
- cloudwatch
- templates
- variables
menuTitle: Template variables
title: CloudWatch template variables
weight: 400
---
# CloudWatch template variables
Instead of hard-coding details such as server, application, and sensor names in metric queries, you can use variables.
Grafana lists these variables in dropdown select boxes at the top of the dashboard to help you change the data displayed in your dashboard.
Grafana refers to such variables as template variables.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../../dashboards/variables" >}}) and [Add and manage variables]({{< relref "../../../dashboards/variables/add-template-variables" >}}) documentation.
## Use query variables
You can specify these CloudWatch data source queries in the Variable edit view's **Query Type** field.
Use them to fill a variable's options list with values like `regions`, `namespaces`, `metric names`, and `dimension keys/values`.
| Name | List returned |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Regions** | All AWS regions. |
| **Namespaces** | All namespaces CloudWatch supports. |
| **Metrics** | Metrics in the namespace. (Specify region or use "default" for custom metrics.) |
| **Dimension Keys** | Dimension keys in the namespace. |
| **Dimension Values** | Dimension values matching the specified `region`, `namespace`, `metric`, and `dimension_key`. Use dimension `filters` for more specific results. |
| **EBS Volume IDs** | Volume ids matching the specified `region` and `instance_id`. |
| **EC2 Instance Attributes** | Attributes matching the specified `region`, `attribute_name`, and `filters`. |
| **Resource ARNs** | ARNs matching the specified `region`, `resource_type`, and `tags`. |
| **Statistics** | All standard statistics. |
| **LogGroups** | All log groups matching the specified `region`. |
For details on the available dimensions, refer to the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
For details about the metrics CloudWatch provides, refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html).
### Use variables in queries
Use Grafana's variable syntax to include variables in queries.
For details, refer to the [variable syntax documentation]({{< relref "../../../dashboards/variables/variable-syntax" >}}).
## Use ec2_instance_attribute
### Filters
The `ec2_instance_attribute` query takes `filters` as a filter name and a comma-separated list of values.
You can specify [pre-defined filters of ec2:DescribeInstances](http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html).
### Select attributes
A query returns only one attribute per instance.
You can select any attribute that has a single value and isn't an object or array, also known as a flat attribute:
- `AmiLaunchIndex`
- `Architecture`
- `ClientToken`
- `EbsOptimized`
- `EnaSupport`
- `Hypervisor`
- `IamInstanceProfile`
- `ImageId`
- `InstanceId`
- `InstanceLifecycle`
- `InstanceType`
- `KernelId`
- `KeyName`
- `LaunchTime`
- `Platform`
- `PrivateDnsName`
- `PrivateIpAddress`
- `PublicDnsName`
- `PublicIpAddress`
- `RamdiskId`
- `RootDeviceName`
- `RootDeviceType`
- `SourceDestCheck`
- `SpotInstanceRequestId`
- `SriovNetSupport`
- `SubnetId`
- `VirtualizationType`
- `VpcId`
You can select tags by prepending the tag name with `Tags.`.
For example, select the tag `Name` by using `Tags.Name`.