diff --git a/.circleci/config.yml b/.circleci/config.yml index 87cd502ef18..c293ea26a9d 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -242,7 +242,7 @@ jobs: command: '/tmp/bootstrap.sh' - run: name: checkout enterprise - command: './scripts/build/prepare_enterprise.sh' + command: './scripts/build/prepare-enterprise.sh' - run: name: test enterprise command: 'go test ./pkg/extensions/...' @@ -274,7 +274,7 @@ jobs: command: '/tmp/bootstrap.sh' - run: name: checkout enterprise - command: './scripts/build/prepare_enterprise.sh' + command: './scripts/build/prepare-enterprise.sh' - restore_cache: key: phantomjs-binaries-{{ checksum "scripts/build/download-phantomjs.sh" }} - run: @@ -323,18 +323,18 @@ jobs: name: deploy to s3 command: 'aws s3 sync ./enterprise-dist s3://$ENTERPRISE_BUCKET_NAME/master' -deploy-enterprise-release: - docker: - - image: circleci/python:2.7-stretch - steps: - - attach_workspace: - at: . - - run: - name: install awscli - command: 'sudo pip install awscli' - - run: - name: deploy to s3 - command: 'aws s3 sync ./enterprise-dist s3://$ENTERPRISE_BUCKET_NAME/release' + deploy-enterprise-release: + docker: + - image: circleci/python:2.7-stretch + steps: + - attach_workspace: + at: . + - run: + name: install awscli + command: 'sudo pip install awscli' + - run: + name: deploy to s3 + command: 'aws s3 sync ./enterprise-dist s3://$ENTERPRISE_BUCKET_NAME/release' deploy-master: docker: diff --git a/CHANGELOG.md b/CHANGELOG.md index db5457147f5..778da9cd499 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ * **Postgres/MySQL/MSSQL**: Adds support for configuration of max open/idle connections and connection max lifetime. Also, panels with multiple SQL queries will now be executed concurrently [#11711](https://github.com/grafana/grafana/issues/11711), thx [@connection-reset](https://github.com/connection-reset) * **MSSQL**: Add encrypt setting to allow configuration of how data sent between client and server are encrypted [#13629](https://github.com/grafana/grafana/issues/13629), thx [@ramiro](https://github.com/ramiro) * **MySQL**: Support connecting thru Unix socket for MySQL datasource [#12342](https://github.com/grafana/grafana/issues/12342), thx [@Yukinoshita-Yukino](https://github.com/Yukinoshita-Yukino) +* **Stackdriver**: Not possible to authenticate using GCE metadata server [#13669](https://github.com/grafana/grafana/issues/13669) ### Minor @@ -17,13 +18,23 @@ * Postgres/MySQL/MSSQL datasources now per default uses `max open connections` = `unlimited` (earlier 10), `max idle connections` = `2` (earlier 10) and `connection max lifetime` = `4` hours (earlier unlimited) -# 5.3.2 (unreleased) +# 5.3.2 (2018-10-24) * **InfluxDB/Graphite/Postgres**: Prevent cross site scripting (XSS) in query editor [#13667](https://github.com/grafana/grafana/issues/13667), thx [@svenklemm](https://github.com/svenklemm) * **Postgres**: Fix template variables error [#13692](https://github.com/grafana/grafana/issues/13692), thx [@svenklemm](https://github.com/svenklemm) * **Cloudwatch**: Fix service panic because of race conditions [#13674](https://github.com/grafana/grafana/issues/13674), thx [@mtanda](https://github.com/mtanda) +* **Cloudwatch**: Fix check for invalid percentile statistics [#13633](https://github.com/grafana/grafana/issues/13633), thx [@apalaniuk](https://github.com/apalaniuk) * **Stackdriver/Cloudwatch**: Allow user to change unit in graph panel if cloudwatch/stackdriver datasource response doesn't include unit [#13718](https://github.com/grafana/grafana/issues/13718), thx [@mtanda](https://github.com/mtanda) +* **Stackdriver**: stackdriver user-metrics duplicated response when multiple resource types [#13691](https://github.com/grafana/grafana/issues/13691) +* **Variables**: Fix text box template variable doesn't work properly without a default value [#13666](https://github.com/grafana/grafana/issues/13666) +* **Variables**: Fix variable dependency check when using `${var}` format [#13600](https://github.com/grafana/grafana/issues/13600) +* **Dashboard**: Fix kiosk=1 url parameter should put dashboard in kiosk mode [#13764](https://github.com/grafana/grafana/pull/13764) * **LDAP**: Fix super admins can also be admins of orgs [#13710](https://github.com/grafana/grafana/issues/13710), thx [@adrien-f](https://github.com/adrien-f) +* **Provisioning**: Fix deleting provisioned dashboard folder should cleanup provisioning meta data [#13280](https://github.com/grafana/grafana/issues/13280) + +### Minor + +* **Docker**: adds curl back into the docker image for utility. [#13794](https://github.com/grafana/grafana/pull/13794) # 5.3.1 (2018-10-16) diff --git a/docs/sources/features/datasources/stackdriver.md b/docs/sources/features/datasources/stackdriver.md index 103a58587b1..d19dbe4ea50 100644 --- a/docs/sources/features/datasources/stackdriver.md +++ b/docs/sources/features/datasources/stackdriver.md @@ -35,7 +35,9 @@ Grafana ships with built-in support for Google Stackdriver. Just add it as a dat ## Authentication -### Service Account Credentials - Private Key File +There are two ways to authenticate the Stackdriver plugin - either by uploading a Google JWT file, or by automatically retrieving credentials from Google metadata server. The latter option is only available when running Grafana on GCE virtual machine. + +### Using a Google Service Account Key File To authenticate with the Stackdriver API, you need to create a Google Cloud Platform (GCP) Service Account for the Project you want to show data for. A Grafana datasource integrates with one GCP Project. If you want to visualize data from multiple GCP Projects then you need to create one datasource per GCP Project. @@ -74,6 +76,16 @@ Click on the links above and click the `Enable` button: {{< docs-imagebox img="/img/docs/v53/stackdriver_grafana_key_uploaded.png" class="docs-image--no-shadow" caption="Service key file is uploaded to Grafana" >}} +### Using GCE Default Service Account + +If Grafana is running on a Google Compute Engine (GCE) virtual machine, it is possible for Grafana to automatically retrieve default credentials from the metadata server. This has the advantage of not needing to generate a private key file for the service account and also not having to upload the file to Grafana. However for this to work, there are a few preconditions that need to be met. + +1. First of all, you need to create a Service Account that can be used by the GCE virtual machine. See detailed instructions on how to do that [here](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#createanewserviceaccount). +2. Make sure the GCE virtual machine instance is being run as the service account that you just created. See instructions [here](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#using). +3. Allow access to the `Stackdriver Monitoring API` scope. See instructions [here](changeserviceaccountandscopes). + +Read more about creating and enabling service accounts for GCE VM instances [here](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances). + ## Metric Query Editor {{< docs-imagebox img="/img/docs/v53/stackdriver_query_editor.png" max-width= "400px" class="docs-image--right" >}} @@ -144,6 +156,16 @@ Example Alias By: `{{metric.type}} - {{metric.labels.instance_name}}` Example Result: `compute.googleapis.com/instance/cpu/usage_time - server1-prod` +It is also possible to resolve the name of the Monitored Resource Type. + +| Alias Pattern Format | Description | Example Result | +| ------------------------ | ------------------------------------------------| ---------------- | +| `{{resource.type}}` | returns the name of the monitored resource type | `gce_instance` | + +Example Alias By: `{{resource.type}} - {{metric.type}}` + +Example Result: `gce_instance - compute.googleapis.com/instance/cpu/usage_time` + ## Templating Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place. @@ -194,7 +216,7 @@ Example Result: `monitoring.googleapis.com/uptime_check/http_status has this val It's now possible to configure datasources using config files with Grafana's provisioning system. You can read more about how it works and all the settings you can set for datasources on the [provisioning docs page](/administration/provisioning/#datasources) -Here is a provisioning example for this datasource. +Here is a provisioning example using the JWT (Service Account key file) authentication type. ```yaml apiVersion: 1 @@ -206,6 +228,7 @@ datasources: jsonData: tokenUri: https://oauth2.googleapis.com/token clientEmail: stackdriver@myproject.iam.gserviceaccount.com + authenticationType: jwt defaultProject: my-project-name secureJsonData: privateKey: | @@ -215,3 +238,16 @@ datasources: yA+23427282348234= -----END PRIVATE KEY----- ``` + +Here is a provisioning example using GCE Default Service Account authentication. + +```yaml +apiVersion: 1 + +datasources: + - name: Stackdriver + type: stackdriver + access: proxy + jsonData: + authenticationType: gce +``` diff --git a/docs/sources/installation/debian.md b/docs/sources/installation/debian.md index 2ae2e9dc40d..7ed44572533 100644 --- a/docs/sources/installation/debian.md +++ b/docs/sources/installation/debian.md @@ -28,7 +28,7 @@ installation. ```bash wget sudo apt-get install -y adduser libfontconfig -sudo dpkg -i grafana_5.1.4_amd64.deb +sudo dpkg -i grafana__amd64.deb ``` Example: diff --git a/docs/sources/tutorials/index.md b/docs/sources/tutorials/index.md index cb11940c6dd..90410e901d3 100644 --- a/docs/sources/tutorials/index.md +++ b/docs/sources/tutorials/index.md @@ -1,5 +1,6 @@ +++ title = "Tutorials" +type = "docs" [menu.docs] identifier = "tutorials" weight = 6 @@ -11,7 +12,11 @@ This section of the docs contains a series for tutorials and stack setup guides. ## Articles -- [How to integrate Hubot with Grafana](hubot_howto.md) +- [Running Grafana behind a reverse proxy]({{< relref "behind_proxy.md" >}}) +- [API Tutorial: How To Create API Tokens And Dashboards For A Specific Organization]({{< relref "api_org_token_howto.md" >}}) +- [How to Use IIS with URL Rewrite as a Reverse Proxy for Grafana on Windows]({{< relref "iis.md" >}}) +- [How to integrate Hubot with Grafana]({{< relref "hubot_howto.md" >}}) +- [How to setup Grafana for high availability]({{< relref "ha_setup.md" >}}) ## External links diff --git a/latest.json b/latest.json index 4355e9a64b7..992af0a8336 100644 --- a/latest.json +++ b/latest.json @@ -1,4 +1,4 @@ { - "stable": "5.3.1", - "testing": "5.3.1" + "stable": "5.3.2", + "testing": "5.3.2" } diff --git a/packaging/docker/Dockerfile b/packaging/docker/Dockerfile index 890d6a4fb11..dc8972b0ba0 100644 --- a/packaging/docker/Dockerfile +++ b/packaging/docker/Dockerfile @@ -25,7 +25,7 @@ ENV PATH=/usr/share/grafana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bi WORKDIR $GF_PATHS_HOME -RUN apt-get update && apt-get install -qq -y libfontconfig ca-certificates && \ +RUN apt-get update && apt-get install -qq -y libfontconfig ca-certificates curl && \ apt-get autoremove -y && \ rm -rf /var/lib/apt/lists/* diff --git a/pkg/api/pluginproxy/ds_auth_provider.go b/pkg/api/pluginproxy/ds_auth_provider.go index c68da839d13..edf43085c7c 100644 --- a/pkg/api/pluginproxy/ds_auth_provider.go +++ b/pkg/api/pluginproxy/ds_auth_provider.go @@ -12,6 +12,7 @@ import ( m "github.com/grafana/grafana/pkg/models" "github.com/grafana/grafana/pkg/plugins" "github.com/grafana/grafana/pkg/util" + "golang.org/x/oauth2/google" ) //ApplyRoute should use the plugin route data to set auth headers and custom headers @@ -54,15 +55,30 @@ func ApplyRoute(ctx context.Context, req *http.Request, proxyPath string, route } } - if route.JwtTokenAuth != nil { + authenticationType := ds.JsonData.Get("authenticationType").MustString("jwt") + if route.JwtTokenAuth != nil && authenticationType == "jwt" { if token, err := tokenProvider.getJwtAccessToken(ctx, data); err != nil { logger.Error("Failed to get access token", "error", err) } else { req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", token)) } } - logger.Info("Requesting", "url", req.URL.String()) + if authenticationType == "gce" { + tokenSrc, err := google.DefaultTokenSource(ctx, route.JwtTokenAuth.Scopes...) + if err != nil { + logger.Error("Failed to get default token from meta data server", "error", err) + } else { + token, err := tokenSrc.Token() + if err != nil { + logger.Error("Failed to get default access token from meta data server", "error", err) + } else { + req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", token.AccessToken)) + } + } + } + + logger.Info("Requesting", "url", req.URL.String()) } func interpolateString(text string, data templateData) (string, error) { diff --git a/pkg/services/guardian/guardian.go b/pkg/services/guardian/guardian.go index 7506338c5f0..366bc90fc37 100644 --- a/pkg/services/guardian/guardian.go +++ b/pkg/services/guardian/guardian.go @@ -40,7 +40,7 @@ var New = func(dashId int64, orgId int64, user *m.SignedInUser) DashboardGuardia user: user, dashId: dashId, orgId: orgId, - log: log.New("guardians.dashboard"), + log: log.New("dashboard.permissions"), } } @@ -66,15 +66,30 @@ func (g *dashboardGuardianImpl) CanAdmin() (bool, error) { func (g *dashboardGuardianImpl) HasPermission(permission m.PermissionType) (bool, error) { if g.user.OrgRole == m.ROLE_ADMIN { - return true, nil + return g.logHasPermissionResult(permission, true, nil) } acl, err := g.GetAcl() if err != nil { - return false, err + return g.logHasPermissionResult(permission, false, err) } - return g.checkAcl(permission, acl) + result, err := g.checkAcl(permission, acl) + return g.logHasPermissionResult(permission, result, err) +} + +func (g *dashboardGuardianImpl) logHasPermissionResult(permission m.PermissionType, hasPermission bool, err error) (bool, error) { + if err != nil { + return hasPermission, err + } + + if hasPermission { + g.log.Debug("User granted access to execute action", "userId", g.user.UserId, "orgId", g.orgId, "uname", g.user.Login, "dashId", g.dashId, "action", permission) + } else { + g.log.Debug("User denied access to execute action", "userId", g.user.UserId, "orgId", g.orgId, "uname", g.user.Login, "dashId", g.dashId, "action", permission) + } + + return hasPermission, err } func (g *dashboardGuardianImpl) checkAcl(permission m.PermissionType, acl []*m.DashboardAclInfoDTO) (bool, error) { diff --git a/pkg/services/sqlstore/dashboard.go b/pkg/services/sqlstore/dashboard.go index e43279208e7..1b853d17b5f 100644 --- a/pkg/services/sqlstore/dashboard.go +++ b/pkg/services/sqlstore/dashboard.go @@ -320,13 +320,18 @@ func DeleteDashboard(cmd *m.DeleteDashboardCommand) error { "DELETE FROM dashboard WHERE id = ?", "DELETE FROM playlist_item WHERE type = 'dashboard_by_id' AND value = ?", "DELETE FROM dashboard_version WHERE dashboard_id = ?", - "DELETE FROM dashboard WHERE folder_id = ?", "DELETE FROM annotation WHERE dashboard_id = ?", "DELETE FROM dashboard_provisioning WHERE dashboard_id = ?", } + if dashboard.IsFolder { + deletes = append(deletes, "DELETE FROM dashboard_provisioning WHERE dashboard_id in (select id from dashboard where folder_id = ?)") + deletes = append(deletes, "DELETE FROM dashboard WHERE folder_id = ?") + } + for _, sql := range deletes { _, err := sess.Exec(sql, dashboard.Id) + if err != nil { return err } diff --git a/pkg/services/sqlstore/dashboard_provisioning_test.go b/pkg/services/sqlstore/dashboard_provisioning_test.go index 7ef45df3152..1b7a3976727 100644 --- a/pkg/services/sqlstore/dashboard_provisioning_test.go +++ b/pkg/services/sqlstore/dashboard_provisioning_test.go @@ -13,17 +13,30 @@ func TestDashboardProvisioningTest(t *testing.T) { Convey("Testing Dashboard provisioning", t, func() { InitTestDB(t) - saveDashboardCmd := &models.SaveDashboardCommand{ + folderCmd := &models.SaveDashboardCommand{ OrgId: 1, FolderId: 0, - IsFolder: false, + IsFolder: true, Dashboard: simplejson.NewFromAny(map[string]interface{}{ "id": nil, "title": "test dashboard", }), } - Convey("Saving dashboards with extras", func() { + err := SaveDashboard(folderCmd) + So(err, ShouldBeNil) + + saveDashboardCmd := &models.SaveDashboardCommand{ + OrgId: 1, + IsFolder: false, + FolderId: folderCmd.Result.Id, + Dashboard: simplejson.NewFromAny(map[string]interface{}{ + "id": nil, + "title": "test dashboard", + }), + } + + Convey("Saving dashboards with provisioning meta data", func() { now := time.Now() cmd := &models.SaveProvisionedDashboardCommand{ @@ -67,6 +80,21 @@ func TestDashboardProvisioningTest(t *testing.T) { So(err, ShouldBeNil) So(query.Result, ShouldBeFalse) }) + + Convey("Deleteing folder should delete provision meta data", func() { + deleteCmd := &models.DeleteDashboardCommand{ + Id: folderCmd.Result.Id, + OrgId: 1, + } + + So(DeleteDashboard(deleteCmd), ShouldBeNil) + + query := &models.IsDashboardProvisionedQuery{DashboardId: cmd.Result.Id} + + err = GetProvisionedDataByDashboardId(query) + So(err, ShouldBeNil) + So(query.Result, ShouldBeFalse) + }) }) }) } diff --git a/pkg/tsdb/stackdriver/ensure_default_project.go b/pkg/tsdb/stackdriver/ensure_default_project.go new file mode 100644 index 00000000000..265fdda5151 --- /dev/null +++ b/pkg/tsdb/stackdriver/ensure_default_project.go @@ -0,0 +1,24 @@ +package stackdriver + +import ( + "context" + + "github.com/grafana/grafana/pkg/components/simplejson" + "github.com/grafana/grafana/pkg/tsdb" +) + +func (e *StackdriverExecutor) ensureDefaultProject(ctx context.Context, tsdbQuery *tsdb.TsdbQuery) (*tsdb.Response, error) { + queryResult := &tsdb.QueryResult{Meta: simplejson.New(), RefId: tsdbQuery.Queries[0].RefId} + result := &tsdb.Response{ + Results: make(map[string]*tsdb.QueryResult), + } + defaultProject, err := e.getDefaultProject(ctx) + if err != nil { + return nil, err + } + + e.dsInfo.JsonData.Set("defaultProject", defaultProject) + queryResult.Meta.Set("defaultProject", defaultProject) + result.Results[tsdbQuery.Queries[0].RefId] = queryResult + return result, nil +} diff --git a/pkg/tsdb/stackdriver/stackdriver.go b/pkg/tsdb/stackdriver/stackdriver.go index 96242dfdec4..b33d33fb41c 100644 --- a/pkg/tsdb/stackdriver/stackdriver.go +++ b/pkg/tsdb/stackdriver/stackdriver.go @@ -16,6 +16,7 @@ import ( "time" "golang.org/x/net/context/ctxhttp" + "golang.org/x/oauth2/google" "github.com/grafana/grafana/pkg/api/pluginproxy" "github.com/grafana/grafana/pkg/components/null" @@ -34,6 +35,11 @@ var ( metricNameFormat *regexp.Regexp ) +const ( + gceAuthentication string = "gce" + jwtAuthentication string = "jwt" +) + // StackdriverExecutor executes queries for the Stackdriver datasource type StackdriverExecutor struct { httpClient *http.Client @@ -71,6 +77,8 @@ func (e *StackdriverExecutor) Query(ctx context.Context, dsInfo *models.DataSour switch queryType { case "annotationQuery": result, err = e.executeAnnotationQuery(ctx, tsdbQuery) + case "ensureDefaultProjectQuery": + result, err = e.ensureDefaultProject(ctx, tsdbQuery) case "timeSeriesQuery": fallthrough default: @@ -85,6 +93,16 @@ func (e *StackdriverExecutor) executeTimeSeriesQuery(ctx context.Context, tsdbQu Results: make(map[string]*tsdb.QueryResult), } + authenticationType := e.dsInfo.JsonData.Get("authenticationType").MustString(jwtAuthentication) + if authenticationType == gceAuthentication { + defaultProject, err := e.getDefaultProject(ctx) + if err != nil { + return nil, fmt.Errorf("Failed to retrieve default project from GCE metadata server. error: %v", err) + } + + e.dsInfo.JsonData.Set("defaultProject", defaultProject) + } + queries, err := e.buildQueries(tsdbQuery) if err != nil { return nil, err @@ -337,11 +355,21 @@ func (e *StackdriverExecutor) unmarshalResponse(res *http.Response) (Stackdriver func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data StackdriverResponse, query *StackdriverQuery) error { metricLabels := make(map[string][]string) resourceLabels := make(map[string][]string) + var resourceTypes []string + + for _, series := range data.TimeSeries { + if !containsLabel(resourceTypes, series.Resource.Type) { + resourceTypes = append(resourceTypes, series.Resource.Type) + } + } for _, series := range data.TimeSeries { points := make([]tsdb.TimePoint, 0) defaultMetricName := series.Metric.Type + if len(resourceTypes) > 1 { + defaultMetricName += " " + series.Resource.Type + } for key, value := range series.Metric.Labels { if !containsLabel(metricLabels[key], value) { @@ -385,7 +413,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta points = append(points, tsdb.NewTimePoint(null.FloatFrom(value), float64((point.Interval.EndTime).Unix())*1000)) } - metricName := formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, make(map[string]string), query) + metricName := formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, make(map[string]string), query) queryRes.Series = append(queryRes.Series, &tsdb.TimeSeries{ Name: metricName, @@ -411,7 +439,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta bucketBound := calcBucketBound(point.Value.DistributionValue.BucketOptions, i) additionalLabels := map[string]string{"bucket": bucketBound} buckets[i] = &tsdb.TimeSeries{ - Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), + Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), Points: make([]tsdb.TimePoint, 0), } if maxKey < i { @@ -427,7 +455,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta bucketBound := calcBucketBound(point.Value.DistributionValue.BucketOptions, i) additionalLabels := map[string]string{"bucket": bucketBound} buckets[i] = &tsdb.TimeSeries{ - Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), + Name: formatLegendKeys(series.Metric.Type, defaultMetricName, series.Resource.Type, series.Metric.Labels, series.Resource.Labels, additionalLabels, query), Points: make([]tsdb.TimePoint, 0), } } @@ -442,6 +470,7 @@ func (e *StackdriverExecutor) parseResponse(queryRes *tsdb.QueryResult, data Sta queryRes.Meta.Set("resourceLabels", resourceLabels) queryRes.Meta.Set("metricLabels", metricLabels) queryRes.Meta.Set("groupBys", query.GroupBys) + queryRes.Meta.Set("resourceTypes", resourceTypes) return nil } @@ -455,7 +484,7 @@ func containsLabel(labels []string, newLabel string) bool { return false } -func formatLegendKeys(metricType string, defaultMetricName string, metricLabels map[string]string, resourceLabels map[string]string, additionalLabels map[string]string, query *StackdriverQuery) string { +func formatLegendKeys(metricType string, defaultMetricName string, resourceType string, metricLabels map[string]string, resourceLabels map[string]string, additionalLabels map[string]string, query *StackdriverQuery) string { if query.AliasBy == "" { return defaultMetricName } @@ -469,6 +498,10 @@ func formatLegendKeys(metricType string, defaultMetricName string, metricLabels return []byte(metricType) } + if metaPartName == "resource.type" && resourceType != "" { + return []byte(resourceType) + } + metricPart := replaceWithMetricPart(metaPartName, metricType) if metricPart != nil { @@ -550,8 +583,6 @@ func (e *StackdriverExecutor) createRequest(ctx context.Context, dsInfo *models. if !ok { return nil, errors.New("Unable to find datasource plugin Stackdriver") } - projectName := dsInfo.JsonData.Get("defaultProject").MustString() - proxyPass := fmt.Sprintf("stackdriver%s", "v3/projects/"+projectName+"/timeSeries") var stackdriverRoute *plugins.AppPluginRoute for _, route := range plugin.Routes { @@ -561,7 +592,22 @@ func (e *StackdriverExecutor) createRequest(ctx context.Context, dsInfo *models. } } + projectName := dsInfo.JsonData.Get("defaultProject").MustString() + proxyPass := fmt.Sprintf("stackdriver%s", "v3/projects/"+projectName+"/timeSeries") + pluginproxy.ApplyRoute(ctx, req, proxyPass, stackdriverRoute, dsInfo) return req, nil } + +func (e *StackdriverExecutor) getDefaultProject(ctx context.Context) (string, error) { + authenticationType := e.dsInfo.JsonData.Get("authenticationType").MustString(jwtAuthentication) + if authenticationType == gceAuthentication { + defaultCredentials, err := google.FindDefaultCredentials(ctx, "https://www.googleapis.com/auth/monitoring.read") + if err != nil { + return "", fmt.Errorf("Failed to retrieve default project from GCE metadata server. error: %v", err) + } + return defaultCredentials.ProjectID, nil + } + return e.dsInfo.JsonData.Get("defaultProject").MustString(), nil +} diff --git a/public/app/core/components/help/help.ts b/public/app/core/components/help/help.ts index eac47b6e0a2..8e8a5ed45d2 100644 --- a/public/app/core/components/help/help.ts +++ b/public/app/core/components/help/help.ts @@ -34,6 +34,7 @@ export class HelpCtrl { { keys: ['p', 's'], description: 'Open Panel Share Modal' }, { keys: ['p', 'd'], description: 'Duplicate Panel' }, { keys: ['p', 'r'], description: 'Remove Panel' }, + { keys: ['p', 'l'], description: 'Toggle panel legend' }, ], 'Time Range': [ { keys: ['t', 'z'], description: 'Zoom out time range' }, diff --git a/public/app/core/services/keybindingSrv.ts b/public/app/core/services/keybindingSrv.ts index f43dc96cd37..6fe57dfa77a 100644 --- a/public/app/core/services/keybindingSrv.ts +++ b/public/app/core/services/keybindingSrv.ts @@ -242,6 +242,18 @@ export class KeybindingSrv { } }); + // toggle panel legend + this.bind('p l', () => { + if (dashboard.meta.focusPanelId) { + const panelInfo = dashboard.getPanelInfoById(dashboard.meta.focusPanelId); + if (panelInfo.panel.legend) { + const panelRef = dashboard.getPanelById(dashboard.meta.focusPanelId); + panelRef.legend.show = !panelRef.legend.show; + panelRef.refresh(); + } + } + }); + // collapse all rows this.bind('d shift+c', () => { dashboard.collapseRows(); diff --git a/public/app/core/specs/table_model.test.ts b/public/app/core/specs/table_model.test.ts index 990daaaa2da..19b2a7543fb 100644 --- a/public/app/core/specs/table_model.test.ts +++ b/public/app/core/specs/table_model.test.ts @@ -1,4 +1,4 @@ -import TableModel from 'app/core/table_model'; +import TableModel, { mergeTablesIntoModel } from 'app/core/table_model'; describe('when sorting table desc', () => { let table; @@ -79,3 +79,118 @@ describe('when sorting with nulls', () => { expect(values).toEqual([null, null, 'd', 'c', 'b', 'a', '', '']); }); }); + +describe('mergeTables', () => { + const time = new Date().getTime(); + + const singleTable = new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value' }], + rows: [[time, 'Label Value 1', 42]], + }); + + const multipleTablesSameColumns = [ + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #A' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 42]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #B' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 13]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 4]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Label Key 2' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 1', 'Label Value 2', 7]], + }), + ]; + + const multipleTablesDifferentColumns = [ + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #A' }], + rows: [[time, 'Label Value 1', 42]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 2' }, { text: 'Value #B' }], + rows: [[time, 'Label Value 2', 13]], + }), + new TableModel({ + type: 'table', + columns: [{ text: 'Time' }, { text: 'Label Key 1' }, { text: 'Value #C' }], + rows: [[time, 'Label Value 3', 7]], + }), + ]; + + it('should return the single table as is', () => { + const table = mergeTablesIntoModel(new TableModel(), singleTable); + expect(table.columns.length).toBe(3); + expect(table.columns[0].text).toBe('Time'); + expect(table.columns[1].text).toBe('Label Key 1'); + expect(table.columns[2].text).toBe('Value'); + }); + + it('should return the union of columns for multiple tables', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesSameColumns); + expect(table.columns.length).toBe(6); + expect(table.columns[0].text).toBe('Time'); + expect(table.columns[1].text).toBe('Label Key 1'); + expect(table.columns[2].text).toBe('Label Key 2'); + expect(table.columns[3].text).toBe('Value #A'); + expect(table.columns[4].text).toBe('Value #B'); + expect(table.columns[5].text).toBe('Value #C'); + }); + + it('should return 1 row for a single table', () => { + const table = mergeTablesIntoModel(new TableModel(), singleTable); + expect(table.rows.length).toBe(1); + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe(42); + }); + + it('should return 2 rows for a multiple tables with same column values plus one extra row', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesSameColumns); + expect(table.rows.length).toBe(2); + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe('Label Value 2'); + expect(table.rows[0][3]).toBe(42); + expect(table.rows[0][4]).toBe(13); + expect(table.rows[0][5]).toBe(4); + expect(table.rows[1][0]).toBe(time); + expect(table.rows[1][1]).toBe('Label Value 1'); + expect(table.rows[1][2]).toBe('Label Value 2'); + expect(table.rows[1][3]).toBeUndefined(); + expect(table.rows[1][4]).toBeUndefined(); + expect(table.rows[1][5]).toBe(7); + }); + + it('should return 2 rows for multiple tables with different column values', () => { + const table = mergeTablesIntoModel(new TableModel(), ...multipleTablesDifferentColumns); + expect(table.rows.length).toBe(2); + expect(table.columns.length).toBe(6); + + expect(table.rows[0][0]).toBe(time); + expect(table.rows[0][1]).toBe('Label Value 1'); + expect(table.rows[0][2]).toBe(42); + expect(table.rows[0][3]).toBe('Label Value 2'); + expect(table.rows[0][4]).toBe(13); + expect(table.rows[0][5]).toBeUndefined(); + + expect(table.rows[1][0]).toBe(time); + expect(table.rows[1][1]).toBe('Label Value 3'); + expect(table.rows[1][2]).toBeUndefined(); + expect(table.rows[1][3]).toBeUndefined(); + expect(table.rows[1][4]).toBeUndefined(); + expect(table.rows[1][5]).toBe(7); + }); +}); diff --git a/public/app/core/table_model.ts b/public/app/core/table_model.ts index f8b96d0537b..99395258ba3 100644 --- a/public/app/core/table_model.ts +++ b/public/app/core/table_model.ts @@ -1,3 +1,5 @@ +import _ from 'lodash'; + interface Column { text: string; title?: string; @@ -14,11 +16,20 @@ export default class TableModel { type: string; columnMap: any; - constructor() { + constructor(table?: any) { this.columns = []; this.columnMap = {}; this.rows = []; this.type = 'table'; + + if (table) { + if (table.columns) { + table.columns.forEach(col => this.addColumn(col)); + } + if (table.rows) { + table.rows.forEach(row => this.addRow(row)); + } + } } sort(options) { @@ -52,3 +63,100 @@ export default class TableModel { this.rows.push(row); } } + +// Returns true if both rows have matching non-empty fields as well as matching +// indexes where one field is empty and the other is not +function areRowsMatching(columns, row, otherRow) { + let foundFieldToMatch = false; + for (let columnIndex = 0; columnIndex < columns.length; columnIndex++) { + if (row[columnIndex] !== undefined && otherRow[columnIndex] !== undefined) { + if (row[columnIndex] !== otherRow[columnIndex]) { + return false; + } + } else if (row[columnIndex] === undefined || otherRow[columnIndex] === undefined) { + foundFieldToMatch = true; + } + } + return foundFieldToMatch; +} + +export function mergeTablesIntoModel(dst?: TableModel, ...tables: TableModel[]): TableModel { + const model = dst || new TableModel(); + + // Single query returns data columns and rows as is + if (arguments.length === 2) { + model.columns = [...tables[0].columns]; + model.rows = [...tables[0].rows]; + return model; + } + + // Track column indexes of union: name -> index + const columnNames = {}; + + // Union of all non-value columns + const columnsUnion = tables.slice().reduce((acc, series) => { + series.columns.forEach(col => { + const { text } = col; + if (columnNames[text] === undefined) { + columnNames[text] = acc.length; + acc.push(col); + } + }); + return acc; + }, []); + + // Map old column index to union index per series, e.g., + // given columnNames {A: 0, B: 1} and + // data [{columns: [{ text: 'A' }]}, {columns: [{ text: 'B' }]}] => [[0], [1]] + const columnIndexMapper = tables.map(series => series.columns.map(col => columnNames[col.text])); + + // Flatten rows of all series and adjust new column indexes + const flattenedRows = tables.reduce((acc, series, seriesIndex) => { + const mapper = columnIndexMapper[seriesIndex]; + series.rows.forEach(row => { + const alteredRow = []; + // Shifting entries according to index mapper + mapper.forEach((to, from) => { + alteredRow[to] = row[from]; + }); + acc.push(alteredRow); + }); + return acc; + }, []); + + // Merge rows that have same values for columns + const mergedRows = {}; + const compactedRows = flattenedRows.reduce((acc, row, rowIndex) => { + if (!mergedRows[rowIndex]) { + // Look from current row onwards + let offset = rowIndex + 1; + // More than one row can be merged into current row + while (offset < flattenedRows.length) { + // Find next row that could be merged + const match = _.findIndex(flattenedRows, otherRow => areRowsMatching(columnsUnion, row, otherRow), offset); + if (match > -1) { + const matchedRow = flattenedRows[match]; + // Merge values from match into current row if there is a gap in the current row + for (let columnIndex = 0; columnIndex < columnsUnion.length; columnIndex++) { + if (row[columnIndex] === undefined && matchedRow[columnIndex] !== undefined) { + row[columnIndex] = matchedRow[columnIndex]; + } + } + // Don't visit this row again + mergedRows[match] = matchedRow; + // Keep looking for more rows to merge + offset = match + 1; + } else { + // No match found, stop looking + break; + } + } + acc.push(row); + } + return acc; + }, []); + + model.columns = columnsUnion; + model.rows = compactedRows; + return model; +} diff --git a/public/app/core/utils/explore.test.ts b/public/app/core/utils/explore.test.ts index 915b47e14e2..4252730338d 100644 --- a/public/app/core/utils/explore.test.ts +++ b/public/app/core/utils/explore.test.ts @@ -8,23 +8,17 @@ const DEFAULT_EXPLORE_STATE: ExploreState = { datasourceMissing: false, datasourceName: '', exploreDatasources: [], - graphResult: null, + graphRange: DEFAULT_RANGE, history: [], - latency: 0, - loading: false, - logsResult: null, queries: [], - queryErrors: [], - queryHints: [], + queryTransactions: [], range: DEFAULT_RANGE, - requestOptions: null, showingGraph: true, showingLogs: true, showingTable: true, supportsGraph: null, supportsLogs: null, supportsTable: null, - tableResult: null, }; describe('state functions', () => { diff --git a/public/app/features/explore/Explore.tsx b/public/app/features/explore/Explore.tsx index 4fe67d9d37b..bac063116f1 100644 --- a/public/app/features/explore/Explore.tsx +++ b/public/app/features/explore/Explore.tsx @@ -1,8 +1,17 @@ import React from 'react'; import { hot } from 'react-hot-loader'; import Select from 'react-select'; +import _ from 'lodash'; -import { ExploreState, ExploreUrlState, Query } from 'app/types/explore'; +import { + ExploreState, + ExploreUrlState, + HistoryItem, + Query, + QueryTransaction, + Range, + ResultType, +} from 'app/types/explore'; import kbn from 'app/core/utils/kbn'; import colors from 'app/core/utils/colors'; import store from 'app/core/store'; @@ -13,8 +22,8 @@ import ResetStyles from 'app/core/components/Picker/ResetStyles'; import PickerOption from 'app/core/components/Picker/PickerOption'; import IndicatorsContainer from 'app/core/components/Picker/IndicatorsContainer'; import NoOptionsMessage from 'app/core/components/Picker/NoOptionsMessage'; +import TableModel, { mergeTablesIntoModel } from 'app/core/table_model'; -import ElapsedTime from './ElapsedTime'; import QueryRows from './QueryRows'; import Graph from './Graph'; import Logs from './Logs'; @@ -24,16 +33,6 @@ import { ensureQueries, generateQueryKey, hasQuery } from './utils/query'; const MAX_HISTORY_ITEMS = 100; -function makeHints(hints) { - const hintsByIndex = []; - hints.forEach(hint => { - if (hint) { - hintsByIndex[hint.index] = hint; - } - }); - return hintsByIndex; -} - function makeTimeSeriesList(dataList, options) { return dataList.map((seriesData, index) => { const datapoints = seriesData.datapoints || []; @@ -52,6 +51,25 @@ function makeTimeSeriesList(dataList, options) { }); } +/** + * Update the query history. Side-effect: store history in local storage + */ +function updateHistory(history: HistoryItem[], datasourceId: string, queries: string[]): HistoryItem[] { + const ts = Date.now(); + queries.forEach(query => { + history = [{ query, ts }, ...history]; + }); + + if (history.length > MAX_HISTORY_ITEMS) { + history = history.slice(0, MAX_HISTORY_ITEMS); + } + + // Combine all queries of a datasource type into one history + const historyKey = `grafana.explore.history.${datasourceId}`; + store.setObject(historyKey, history); + return history; +} + interface ExploreProps { datasourceSrv: any; onChangeSplit: (split: boolean, state?: ExploreState) => void; @@ -82,6 +100,7 @@ export class Explore extends React.PureComponent { } else { const { datasource, queries, range } = props.urlState as ExploreUrlState; initialQueries = ensureQueries(queries); + const initialRange = range || { ...DEFAULT_RANGE }; this.state = { datasource: null, datasourceError: null, @@ -89,23 +108,17 @@ export class Explore extends React.PureComponent { datasourceMissing: false, datasourceName: datasource, exploreDatasources: [], - graphResult: null, + graphRange: initialRange, history: [], - latency: 0, - loading: false, - logsResult: null, queries: initialQueries, - queryErrors: [], - queryHints: [], - range: range || { ...DEFAULT_RANGE }, - requestOptions: null, + queryTransactions: [], + range: initialRange, showingGraph: true, showingLogs: true, showingTable: true, supportsGraph: null, supportsLogs: null, supportsTable: null, - tableResult: null, }; } this.queryExpressions = initialQueries.map(q => q.query); @@ -199,14 +212,32 @@ export class Explore extends React.PureComponent { }; onAddQueryRow = index => { - const { queries } = this.state; + // Local cache this.queryExpressions[index + 1] = ''; - const nextQueries = [ - ...queries.slice(0, index + 1), - { query: '', key: generateQueryKey() }, - ...queries.slice(index + 1), - ]; - this.setState({ queries: nextQueries }); + + this.setState(state => { + const { queries, queryTransactions } = state; + + // Add row by generating new react key + const nextQueries = [ + ...queries.slice(0, index + 1), + { query: '', key: generateQueryKey() }, + ...queries.slice(index + 1), + ]; + + // Ongoing transactions need to update their row indices + const nextQueryTransactions = queryTransactions.map(qt => { + if (qt.rowIndex > index) { + return { + ...qt, + rowIndex: qt.rowIndex + 1, + }; + } + return qt; + }); + + return { queries: nextQueries, queryTransactions: nextQueryTransactions }; + }); }; onChangeDatasource = async option => { @@ -214,12 +245,7 @@ export class Explore extends React.PureComponent { datasource: null, datasourceError: null, datasourceLoading: true, - graphResult: null, - latency: 0, - logsResult: null, - queryErrors: [], - queryHints: [], - tableResult: null, + queryTransactions: [], }); const datasourceName = option.value; const datasource = await this.props.datasourceSrv.get(datasourceName); @@ -230,24 +256,25 @@ export class Explore extends React.PureComponent { // Keep current value in local cache this.queryExpressions[index] = value; - // Replace query row on override if (override) { - const { queries } = this.state; - const nextQuery: Query = { - key: generateQueryKey(index), - query: value, - }; - const nextQueries = [...queries]; - nextQueries[index] = nextQuery; + this.setState(state => { + // Replace query row + const { queries, queryTransactions } = state; + const nextQuery: Query = { + key: generateQueryKey(index), + query: value, + }; + const nextQueries = [...queries]; + nextQueries[index] = nextQuery; - this.setState( - { - queryErrors: [], - queryHints: [], + // Discard ongoing transaction related to row query + const nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + + return { queries: nextQueries, - }, - this.onSubmit - ); + queryTransactions: nextQueryTransactions, + }; + }, this.onSubmit); } }; @@ -263,13 +290,8 @@ export class Explore extends React.PureComponent { this.queryExpressions = ['']; this.setState( { - graphResult: null, - logsResult: null, - latency: 0, queries: ensureQueries(), - queryErrors: [], - queryHints: [], - tableResult: null, + queryTransactions: [], }, this.saveState ); @@ -283,11 +305,41 @@ export class Explore extends React.PureComponent { }; onClickGraphButton = () => { - this.setState(state => ({ showingGraph: !state.showingGraph })); + this.setState( + state => { + const showingGraph = !state.showingGraph; + let nextQueryTransactions = state.queryTransactions; + if (!showingGraph) { + // Discard transactions related to Graph query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Graph'); + } + return { queryTransactions: nextQueryTransactions, showingGraph }; + }, + () => { + if (this.state.showingGraph) { + this.onSubmit(); + } + } + ); }; onClickLogsButton = () => { - this.setState(state => ({ showingLogs: !state.showingLogs })); + this.setState( + state => { + const showingLogs = !state.showingLogs; + let nextQueryTransactions = state.queryTransactions; + if (!showingLogs) { + // Discard transactions related to Logs query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Logs'); + } + return { queryTransactions: nextQueryTransactions, showingLogs }; + }, + () => { + if (this.state.showingLogs) { + this.onSubmit(); + } + } + ); }; onClickSplit = () => { @@ -299,7 +351,22 @@ export class Explore extends React.PureComponent { }; onClickTableButton = () => { - this.setState(state => ({ showingTable: !state.showingTable })); + this.setState( + state => { + const showingTable = !state.showingTable; + let nextQueryTransactions = state.queryTransactions; + if (!showingTable) { + // Discard transactions related to Table query + nextQueryTransactions = state.queryTransactions.filter(qt => qt.resultType !== 'Table'); + } + return { queryTransactions: nextQueryTransactions, showingTable }; + }, + () => { + if (this.state.showingTable) { + this.onSubmit(); + } + } + ); }; onClickTableCell = (columnKey: string, rowValue: string) => { @@ -307,39 +374,68 @@ export class Explore extends React.PureComponent { }; onModifyQueries = (action: object, index?: number) => { - const { datasource, queries } = this.state; + const { datasource } = this.state; if (datasource && datasource.modifyQuery) { - let nextQueries; - if (index === undefined) { - // Modify all queries - nextQueries = queries.map((q, i) => ({ - key: generateQueryKey(i), - query: datasource.modifyQuery(this.queryExpressions[i], action), - })); - } else { - // Modify query only at index - nextQueries = [ - ...queries.slice(0, index), - { - key: generateQueryKey(index), - query: datasource.modifyQuery(this.queryExpressions[index], action), - }, - ...queries.slice(index + 1), - ]; - } - this.queryExpressions = nextQueries.map(q => q.query); - this.setState({ queries: nextQueries }, () => this.onSubmit()); + this.setState( + state => { + const { queries, queryTransactions } = state; + let nextQueries; + let nextQueryTransactions; + if (index === undefined) { + // Modify all queries + nextQueries = queries.map((q, i) => ({ + key: generateQueryKey(i), + query: datasource.modifyQuery(this.queryExpressions[i], action), + })); + // Discard all ongoing transactions + nextQueryTransactions = []; + } else { + // Modify query only at index + nextQueries = [ + ...queries.slice(0, index), + { + key: generateQueryKey(index), + query: datasource.modifyQuery(this.queryExpressions[index], action), + }, + ...queries.slice(index + 1), + ]; + // Discard transactions related to row query + nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + } + this.queryExpressions = nextQueries.map(q => q.query); + return { + queries: nextQueries, + queryTransactions: nextQueryTransactions, + }; + }, + () => this.onSubmit() + ); } }; onRemoveQueryRow = index => { - const { queries } = this.state; - if (queries.length <= 1) { - return; - } - const nextQueries = [...queries.slice(0, index), ...queries.slice(index + 1)]; - this.queryExpressions = nextQueries.map(q => q.query); - this.setState({ queries: nextQueries }, () => this.onSubmit()); + // Remove from local cache + this.queryExpressions = [...this.queryExpressions.slice(0, index), ...this.queryExpressions.slice(index + 1)]; + + this.setState( + state => { + const { queries, queryTransactions } = state; + if (queries.length <= 1) { + return null; + } + // Remove row from react state + const nextQueries = [...queries.slice(0, index), ...queries.slice(index + 1)]; + + // Discard transactions related to row query + const nextQueryTransactions = queryTransactions.filter(qt => qt.rowIndex !== index); + + return { + queries: nextQueries, + queryTransactions: nextQueryTransactions, + }; + }, + () => this.onSubmit() + ); }; onSubmit = () => { @@ -348,7 +444,7 @@ export class Explore extends React.PureComponent { this.runTableQuery(); } if (showingGraph && supportsGraph) { - this.runGraphQuery(); + this.runGraphQueries(); } if (showingLogs && supportsLogs) { this.runLogsQuery(); @@ -356,32 +452,11 @@ export class Explore extends React.PureComponent { this.saveState(); }; - onQuerySuccess(datasourceId: string, queries: string[]): void { - // save queries to history - let { history } = this.state; - const { datasource } = this.state; - - if (datasource.meta.id !== datasourceId) { - // Navigated away, queries did not matter - return; - } - - const ts = Date.now(); - queries.forEach(query => { - history = [{ query, ts }, ...history]; - }); - - if (history.length > MAX_HISTORY_ITEMS) { - history = history.slice(0, MAX_HISTORY_ITEMS); - } - - // Combine all queries of a datasource type into one history - const historyKey = `grafana.explore.history.${datasourceId}`; - store.setObject(historyKey, history); - this.setState({ history }); - } - - buildQueryOptions(targetOptions: { format: string; hinting?: boolean; instant?: boolean }) { + buildQueryOptions( + query: string, + rowIndex: number, + targetOptions: { format: string; hinting?: boolean; instant?: boolean } + ) { const { datasource, range } = this.state; const resolution = this.el.offsetWidth; const absoluteRange = { @@ -389,88 +464,235 @@ export class Explore extends React.PureComponent { to: parseDate(range.to, true), }; const { interval } = kbn.calculateInterval(absoluteRange, resolution, datasource.interval); - const targets = this.queryExpressions.map(q => ({ - ...targetOptions, - expr: q, - })); + const targets = [ + { + ...targetOptions, + // Target identifier is needed for table transformations + refId: rowIndex + 1, + expr: query, + }, + ]; + + // Clone range for query request + const queryRange: Range = { ...range }; + return { interval, - range, targets, + range: queryRange, }; } - async runGraphQuery() { + startQueryTransaction(query: string, rowIndex: number, resultType: ResultType, options: any): QueryTransaction { + const queryOptions = this.buildQueryOptions(query, rowIndex, options); + const transaction: QueryTransaction = { + query, + resultType, + rowIndex, + id: generateQueryKey(), + done: false, + latency: 0, + options: queryOptions, + }; + + // Using updater style because we might be modifying queryTransactions in quick succession + this.setState(state => { + const { queryTransactions } = state; + // Discarding existing transactions of same type + const remainingTransactions = queryTransactions.filter( + qt => !(qt.resultType === resultType && qt.rowIndex === rowIndex) + ); + + // Append new transaction + const nextQueryTransactions = [...remainingTransactions, transaction]; + + return { + queryTransactions: nextQueryTransactions, + }; + }); + + return transaction; + } + + completeQueryTransaction( + transactionId: string, + result: any, + latency: number, + queries: string[], + datasourceId: string + ) { const { datasource } = this.state; + if (datasource.meta.id !== datasourceId) { + // Navigated away, queries did not matter + return; + } + + this.setState(state => { + const { history, queryTransactions } = state; + + // Transaction might have been discarded + const transaction = queryTransactions.find(qt => qt.id === transactionId); + if (!transaction) { + return null; + } + + // Get query hints + let hints; + if (datasource.getQueryHints) { + hints = datasource.getQueryHints(transaction.query, result); + } + + // Mark transactions as complete + const nextQueryTransactions = queryTransactions.map(qt => { + if (qt.id === transactionId) { + return { + ...qt, + hints, + latency, + result, + done: true, + }; + } + return qt; + }); + + const nextHistory = updateHistory(history, datasourceId, queries); + + return { + history: nextHistory, + queryTransactions: nextQueryTransactions, + }; + }); + } + + discardTransactions(rowIndex: number) { + this.setState(state => { + const remainingTransactions = state.queryTransactions.filter(qt => qt.rowIndex !== rowIndex); + return { queryTransactions: remainingTransactions }; + }); + } + + failQueryTransaction(transactionId: string, error: string, datasourceId: string) { + const { datasource } = this.state; + if (datasource.meta.id !== datasourceId) { + // Navigated away, queries did not matter + return; + } + + this.setState(state => { + // Transaction might have been discarded + if (!state.queryTransactions.find(qt => qt.id === transactionId)) { + return null; + } + + // Mark transactions as complete + const nextQueryTransactions = state.queryTransactions.map(qt => { + if (qt.id === transactionId) { + return { + ...qt, + error, + done: true, + }; + } + return qt; + }); + + return { + queryTransactions: nextQueryTransactions, + }; + }); + } + + async runGraphQueries() { const queries = [...this.queryExpressions]; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, graphResult: null, queryErrors: [], queryHints: [] }); - const now = Date.now(); - const options = this.buildQueryOptions({ format: 'time_series', instant: false, hinting: true }); - try { - const res = await datasource.query(options); - const result = makeTimeSeriesList(res.data, options); - const queryHints = res.hints ? makeHints(res.hints) : []; - const latency = Date.now() - now; - this.setState({ latency, loading: false, graphResult: result, queryHints, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Graph', { + format: 'time_series', + instant: false, + }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = makeTimeSeriesList(res.data, transaction.options); + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + this.setState({ graphRange: transaction.options.range }); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } + }); } async runTableQuery() { const queries = [...this.queryExpressions]; - const { datasource } = this.state; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, queryErrors: [], queryHints: [], tableResult: null }); - const now = Date.now(); - const options = this.buildQueryOptions({ - format: 'table', - instant: true, + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Table', { + format: 'table', + instant: true, + valueWithRefId: true, + }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = res.data[0]; + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } }); - try { - const res = await datasource.query(options); - const tableModel = res.data[0]; - const latency = Date.now() - now; - this.setState({ latency, loading: false, tableResult: tableModel, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } } async runLogsQuery() { const queries = [...this.queryExpressions]; - const { datasource } = this.state; if (!hasQuery(queries)) { return; } - this.setState({ latency: 0, loading: true, queryErrors: [], queryHints: [], logsResult: null }); - const now = Date.now(); - const options = this.buildQueryOptions({ - format: 'logs', + const { datasource } = this.state; + const datasourceId = datasource.meta.id; + // Run all queries concurrently + queries.forEach(async (query, rowIndex) => { + if (query) { + const transaction = this.startQueryTransaction(query, rowIndex, 'Logs', { format: 'logs' }); + try { + const now = Date.now(); + const res = await datasource.query(transaction.options); + const latency = Date.now() - now; + const results = res.data; + this.completeQueryTransaction(transaction.id, results, latency, queries, datasourceId); + } catch (response) { + console.error(response); + const queryError = response.data ? response.data.error : response; + this.failQueryTransaction(transaction.id, queryError, datasourceId); + } + } else { + this.discardTransactions(rowIndex); + } }); - - try { - const res = await datasource.query(options); - const logsData = res.data; - const latency = Date.now() - now; - this.setState({ latency, loading: false, logsResult: logsData, requestOptions: options }); - this.onQuerySuccess(datasource.meta.id, queries); - } catch (response) { - console.error(response); - const queryError = response.data ? response.data.error : response; - this.setState({ loading: false, queryErrors: [queryError] }); - } } request = url => { @@ -482,6 +704,7 @@ export class Explore extends React.PureComponent { // Copy state, but copy queries including modifications return { ...this.state, + queryTransactions: [], queries: ensureQueries(this.queryExpressions.map(query => ({ query }))), }; } @@ -499,23 +722,17 @@ export class Explore extends React.PureComponent { datasourceLoading, datasourceMissing, exploreDatasources, - graphResult, + graphRange, history, - latency, - loading, - logsResult, queries, - queryErrors, - queryHints, + queryTransactions, range, - requestOptions, showingGraph, showingLogs, showingTable, supportsGraph, supportsLogs, supportsTable, - tableResult, } = this.state; const showingBoth = showingGraph && showingTable; const graphHeight = showingBoth ? '200px' : '400px'; @@ -524,6 +741,20 @@ export class Explore extends React.PureComponent { const tableButtonActive = showingBoth || showingTable ? 'active' : ''; const exploreClass = split ? 'explore explore-split' : 'explore'; const selectedDatasource = datasource ? exploreDatasources.find(d => d.label === datasource.name) : undefined; + const graphLoading = queryTransactions.some(qt => qt.resultType === 'Graph' && !qt.done); + const tableLoading = queryTransactions.some(qt => qt.resultType === 'Table' && !qt.done); + const logsLoading = queryTransactions.some(qt => qt.resultType === 'Logs' && !qt.done); + const graphResult = _.flatten( + queryTransactions.filter(qt => qt.resultType === 'Graph' && qt.done && qt.result).map(qt => qt.result) + ); + const tableResult = mergeTablesIntoModel( + new TableModel(), + ...queryTransactions.filter(qt => qt.resultType === 'Table' && qt.done).map(qt => qt.result) + ); + const logsResult = _.flatten( + queryTransactions.filter(qt => qt.resultType === 'Logs' && qt.done).map(qt => qt.result) + ); + const loading = queryTransactions.some(qt => !qt.done); return (
@@ -581,9 +812,9 @@ export class Explore extends React.PureComponent {
- {loading || latency ? : null}
@@ -602,8 +833,6 @@ export class Explore extends React.PureComponent { { onExecuteQuery={this.onSubmit} onRemoveQueryRow={this.onRemoveQueryRow} supportsLogs={supportsLogs} + transactions={queryTransactions} />
{supportsGraph ? ( @@ -632,23 +862,22 @@ export class Explore extends React.PureComponent {
{supportsGraph && - showingGraph && - graphResult && ( + showingGraph && ( )} {supportsTable && showingTable ? ( -
- +
+
) : null} - {supportsLogs && showingLogs ? : null} + {supportsLogs && showingLogs ? : null} ) : null} diff --git a/public/app/features/explore/Graph.test.tsx b/public/app/features/explore/Graph.test.tsx index 043b843f0a5..fe4deaf17aa 100644 --- a/public/app/features/explore/Graph.test.tsx +++ b/public/app/features/explore/Graph.test.tsx @@ -4,24 +4,11 @@ import { Graph } from './Graph'; import { mockData } from './__mocks__/mockData'; const setup = (propOverrides?: object) => { - const props = Object.assign( - { - data: mockData().slice(0, 19), - options: { - interval: '20s', - range: { from: 'now-6h', to: 'now' }, - targets: [ - { - format: 'time_series', - instant: false, - hinting: true, - expr: 'prometheus_http_request_duration_seconds_bucket', - }, - ], - }, - }, - propOverrides - ); + const props = { + data: mockData().slice(0, 19), + range: { from: 'now-6h', to: 'now' }, + ...propOverrides, + }; // Enzyme.shallow did not work well with jquery.flop. Mocking the draw function. Graph.prototype.draw = jest.fn(); diff --git a/public/app/features/explore/Graph.tsx b/public/app/features/explore/Graph.tsx index cb5def07fab..d57f8d49a43 100644 --- a/public/app/features/explore/Graph.tsx +++ b/public/app/features/explore/Graph.tsx @@ -5,6 +5,8 @@ import { withSize } from 'react-sizeme'; import 'vendor/flot/jquery.flot'; import 'vendor/flot/jquery.flot.time'; + +import { Range } from 'app/types/explore'; import * as dateMath from 'app/core/utils/datemath'; import TimeSeries from 'app/core/time_series2'; @@ -74,7 +76,7 @@ interface GraphProps { height?: string; // e.g., '200px' id?: string; loading?: boolean; - options: any; + range: Range; split?: boolean; size?: { width: number; height: number }; } @@ -101,7 +103,7 @@ export class Graph extends PureComponent { componentDidUpdate(prevProps: GraphProps) { if ( prevProps.data !== this.props.data || - prevProps.options !== this.props.options || + prevProps.range !== this.props.range || prevProps.split !== this.props.split || prevProps.height !== this.props.height || (prevProps.size && prevProps.size.width !== this.props.size.width) @@ -120,22 +122,22 @@ export class Graph extends PureComponent { }; draw() { - const { options: userOptions, size } = this.props; + const { range, size } = this.props; const data = this.getGraphData(); const $el = $(`#${this.props.id}`); - if (!data) { - $el.empty(); - return; + let series = [{ data: [[0, 0]] }]; + + if (data && data.length > 0) { + series = data.map((ts: TimeSeries) => ({ + color: ts.color, + label: ts.label, + data: ts.getFlotPairs('null'), + })); } - const series = data.map((ts: TimeSeries) => ({ - color: ts.color, - label: ts.label, - data: ts.getFlotPairs('null'), - })); const ticks = (size.width || 0) / 100; - let { from, to } = userOptions.range; + let { from, to } = range; if (!moment.isMoment(from)) { from = dateMath.parse(from, false); } @@ -157,7 +159,6 @@ export class Graph extends PureComponent { const options = { ...FLOT_OPTIONS, ...dynamicOptions, - ...userOptions, }; $.plot($el, series, options); } @@ -166,16 +167,11 @@ export class Graph extends PureComponent { const { height = '100px', id = 'graph', loading = false } = this.props; const data = this.getGraphData(); - if (!loading && data.length === 0) { - return ( -
-
The queries returned no time series to graph.
-
- ); - } return ( -
- {this.props.data.length > MAX_NUMBER_OF_TIME_SERIES && +
+ {loading &&
} + {this.props.data && + this.props.data.length > MAX_NUMBER_OF_TIME_SERIES && !this.state.showAllTimeSeries && (
@@ -185,10 +181,8 @@ export class Graph extends PureComponent { }`}
)} -
-
- -
+
+
); } diff --git a/public/app/features/explore/QueryField.tsx b/public/app/features/explore/QueryField.tsx index c3c41b7ab17..ce0bcd71ed0 100644 --- a/public/app/features/explore/QueryField.tsx +++ b/public/app/features/explore/QueryField.tsx @@ -198,7 +198,7 @@ class QueryField extends React.PureComponent { diff --git a/public/app/features/explore/QueryRows.tsx b/public/app/features/explore/QueryRows.tsx index 3d71c2f3566..0b0d7085d2d 100644 --- a/public/app/features/explore/QueryRows.tsx +++ b/public/app/features/explore/QueryRows.tsx @@ -1,7 +1,18 @@ import React, { PureComponent } from 'react'; +import { QueryTransaction } from 'app/types/explore'; + // TODO make this datasource-plugin-dependent import QueryField from './PromQueryField'; +import QueryTransactions from './QueryTransactions'; + +function getFirstHintFromTransactions(transactions: QueryTransaction[]) { + const transaction = transactions.find(qt => qt.hints && qt.hints.length > 0); + if (transaction) { + return transaction.hints[0]; + } + return undefined; +} class QueryRow extends PureComponent { onChangeQuery = (value, override?: boolean) => { @@ -44,13 +55,19 @@ class QueryRow extends PureComponent { }; render() { - const { history, query, queryError, queryHint, request, supportsLogs } = this.props; + const { history, query, request, supportsLogs, transactions } = this.props; + const transactionWithError = transactions.find(t => t.error); + const hint = getFirstHintFromTransactions(transactions); + const queryError = transactionWithError ? transactionWithError.error : null; return (
+
+ +
{ export default class QueryRows extends PureComponent { render() { - const { className = '', queries, queryErrors, queryHints, ...handlers } = this.props; + const { className = '', queries, queryHints, transactions, ...handlers } = this.props; return (
{queries.map((q, index) => ( @@ -86,8 +103,7 @@ export default class QueryRows extends PureComponent { key={q.key} index={index} query={q.query} - queryError={queryErrors[index]} - queryHint={queryHints[index]} + transactions={transactions.filter(t => t.rowIndex === index)} {...handlers} /> ))} diff --git a/public/app/features/explore/QueryTransactions.tsx b/public/app/features/explore/QueryTransactions.tsx new file mode 100644 index 00000000000..0ce721f14e7 --- /dev/null +++ b/public/app/features/explore/QueryTransactions.tsx @@ -0,0 +1,42 @@ +import React, { PureComponent } from 'react'; + +import { QueryTransaction as QueryTransactionModel } from 'app/types/explore'; +import ElapsedTime from './ElapsedTime'; + +function formatLatency(value) { + return `${(value / 1000).toFixed(1)}s`; +} + +interface QueryTransactionProps { + transaction: QueryTransactionModel; +} + +class QueryTransaction extends PureComponent { + render() { + const { transaction } = this.props; + const className = transaction.done ? 'query-transaction' : 'query-transaction query-transaction--loading'; + return ( +
+
{transaction.resultType}:
+
+ {transaction.done ? formatLatency(transaction.latency) : } +
+
+ ); + } +} + +interface QueryTransactionsProps { + transactions: QueryTransactionModel[]; +} + +export default class QueryTransactions extends PureComponent { + render() { + const { transactions } = this.props; + return ( +
+ {transactions.map((t, i) => )} +
+ ); + } +} diff --git a/public/app/features/explore/Table.tsx b/public/app/features/explore/Table.tsx index e1c71fa55e4..0264bd3b4cc 100644 --- a/public/app/features/explore/Table.tsx +++ b/public/app/features/explore/Table.tsx @@ -5,6 +5,8 @@ import ReactTable from 'react-table'; import TableModel from 'app/core/table_model'; const EMPTY_TABLE = new TableModel(); +// Identify columns that contain values +const VALUE_REGEX = /^[Vv]alue #\d+/; interface TableProps { data: TableModel; @@ -34,6 +36,7 @@ export default class Table extends PureComponent { const columns = tableModel.columns.map(({ filterable, text }) => ({ Header: text, accessor: text, + className: VALUE_REGEX.test(text) ? 'text-right' : '', show: text !== 'Time', Cell: row => {row.value}, })); @@ -48,7 +51,7 @@ export default class Table extends PureComponent { minRows={0} noDataText={noDataText} resolveData={data => prepareRows(data, columnNames)} - showPagination={data} + showPagination={Boolean(data)} /> ); } diff --git a/public/app/features/explore/__snapshots__/Graph.test.tsx.snap b/public/app/features/explore/__snapshots__/Graph.test.tsx.snap index d6760dff59c..fd2010a76d3 100644 --- a/public/app/features/explore/__snapshots__/Graph.test.tsx.snap +++ b/public/app/features/explore/__snapshots__/Graph.test.tsx.snap @@ -1,468 +1,468 @@ // Jest Snapshot v1, https://goo.gl/fbAQLP exports[`Render should render component 1`] = ` -
+
-
+ - -
+ "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/label/:name/values", + "instance": "localhost:9090", + "job": "prometheus", + "le": "3", + }, + "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/label/:name/values", + "instance": "localhost:9090", + "job": "prometheus", + "le": "60", + }, + "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/label/:name/values", + "instance": "localhost:9090", + "job": "prometheus", + "le": "8", + }, + "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "+Inf", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "0.1", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "0.4", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "1", + }, + "values": Array [ + Array [ + 1537847900, + "953", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "120", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "20", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "3", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "60", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "8", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + ] + } + />
`; exports[`Render should render component with disclaimer 1`] = ` -
+
@@ -478,484 +478,480 @@ exports[`Render should render component with disclaimer 1`] = `
-
+ - -
+ "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/label/:name/values", + "instance": "localhost:9090", + "job": "prometheus", + "le": "60", + }, + "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/label/:name/values", + "instance": "localhost:9090", + "job": "prometheus", + "le": "8", + }, + "values": Array [ + Array [ + 1537858100, + "16", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "+Inf", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "0.1", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "0.4", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "1", + }, + "values": Array [ + Array [ + 1537847900, + "953", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "120", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "20", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "3", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "60", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/metrics", + "instance": "localhost:9090", + "job": "prometheus", + "le": "8", + }, + "values": Array [ + Array [ + 1537858060, + "1195", + ], + Array [ + 1537858080, + "1195", + ], + Array [ + 1537858100, + "1195", + ], + ], + }, + Object { + "metric": Object { + "__name__": "prometheus_http_request_duration_seconds_bucket", + "handler": "/query", + "instance": "localhost:9090", + "job": "prometheus", + "le": "+Inf", + }, + "values": Array [ + Array [ + 1537858100, + "55", + ], + Array [ + 1537861960, + "1", + ], + Array [ + 1537861980, + "1", + ], + ], + }, + ] + } + />
`; @@ -964,9 +960,16 @@ exports[`Render should show query return no time series 1`] = ` className="panel-container" >
- The queries returned no time series to graph. -
+ className="explore-graph" + id="graph" + style={ + Object { + "height": "100px", + } + } + /> +
`; diff --git a/public/app/features/templating/partials/editor.html b/public/app/features/templating/partials/editor.html index ac4450c20a2..c4463972177 100644 --- a/public/app/features/templating/partials/editor.html +++ b/public/app/features/templating/partials/editor.html @@ -115,7 +115,7 @@
Values - +
diff --git a/public/app/features/templating/specs/template_srv.test.ts b/public/app/features/templating/specs/template_srv.test.ts index 7f5ff959216..d279029d64d 100644 --- a/public/app/features/templating/specs/template_srv.test.ts +++ b/public/app/features/templating/specs/template_srv.test.ts @@ -429,6 +429,11 @@ describe('templateSrv', () => { name: 'period', current: { value: '$__auto_interval_interval', text: 'auto' }, }, + { + type: 'textbox', + name: 'empty_on_init', + current: { value: '', text: '' }, + }, ]); _templateSrv.setGrafanaVariable('$__auto_interval_interval', '13m'); _templateSrv.updateTemplateData(); @@ -438,6 +443,11 @@ describe('templateSrv', () => { const target = _templateSrv.replaceWithText('Server: $server, period: $period'); expect(target).toBe('Server: All, period: 13m'); }); + + it('should replace empty string-values with an empty string', () => { + const target = _templateSrv.replaceWithText('Hello $empty_on_init'); + expect(target).toBe('Hello '); + }); }); describe('built in interval variables', () => { diff --git a/public/app/features/templating/specs/variable.test.ts b/public/app/features/templating/specs/variable.test.ts index 6d5e88fa4bd..83f4af8bca9 100644 --- a/public/app/features/templating/specs/variable.test.ts +++ b/public/app/features/templating/specs/variable.test.ts @@ -22,6 +22,11 @@ describe('containsVariable', () => { expect(contains).toBe(true); }); + it('should find it with [[var:option]] syntax', () => { + const contains = containsVariable('this.[[test:csv]].filters', 'test'); + expect(contains).toBe(true); + }); + it('should find it when part of segment', () => { const contains = containsVariable('metrics.$env.$group-*', 'group'); expect(contains).toBe(true); @@ -36,6 +41,16 @@ describe('containsVariable', () => { const contains = containsVariable('asd', 'asd2.$env', 'env'); expect(contains).toBe(true); }); + + it('should find it with ${var} syntax', () => { + const contains = containsVariable('this.${test}.filters', 'test'); + expect(contains).toBe(true); + }); + + it('should find it with ${var:option} syntax', () => { + const contains = containsVariable('this.${test:csv}.filters', 'test'); + expect(contains).toBe(true); + }); }); }); diff --git a/public/app/features/templating/template_srv.ts b/public/app/features/templating/template_srv.ts index 70fd287402f..61326ad63ec 100644 --- a/public/app/features/templating/template_srv.ts +++ b/public/app/features/templating/template_srv.ts @@ -1,5 +1,6 @@ import kbn from 'app/core/utils/kbn'; import _ from 'lodash'; +import { variableRegex } from 'app/features/templating/variable'; function luceneEscape(value) { return value.replace(/([\!\*\+\-\=<>\s\&\|\(\)\[\]\{\}\^\~\?\:\\/"])/g, '\\$1'); @@ -8,13 +9,7 @@ function luceneEscape(value) { export class TemplateSrv { variables: any[]; - /* - * This regex matches 3 types of variable reference with an optional format specifier - * \$(\w+) $var1 - * \[\[([\s\S]+?)(?::(\w+))?\]\] [[var2]] or [[var2:fmt2]] - * \${(\w+)(?::(\w+))?} ${var3} or ${var3:fmt3} - */ - private regex = /\$(\w+)|\[\[([\s\S]+?)(?::(\w+))?\]\]|\${(\w+)(?::(\w+))?}/g; + private regex = variableRegex; private index = {}; private grafanaVariables = {}; private builtIns = {}; @@ -30,17 +25,14 @@ export class TemplateSrv { } updateTemplateData() { - this.index = {}; + const existsOrEmpty = value => value || value === ''; - for (let i = 0; i < this.variables.length; i++) { - const variable = this.variables[i]; - - if (!variable.current || (!variable.current.isNone && !variable.current.value)) { - continue; + this.index = this.variables.reduce((acc, currentValue) => { + if (currentValue.current && !currentValue.current.isNone && existsOrEmpty(currentValue.current.value)) { + acc[currentValue.name] = currentValue; } - - this.index[variable.name] = variable; - } + return acc; + }, {}); } variableInitialized(variable) { diff --git a/public/app/features/templating/variable.ts b/public/app/features/templating/variable.ts index 412426fb294..1994e86eff0 100644 --- a/public/app/features/templating/variable.ts +++ b/public/app/features/templating/variable.ts @@ -1,6 +1,19 @@ -import kbn from 'app/core/utils/kbn'; import { assignModelProperties } from 'app/core/utils/model_utils'; +/* + * This regex matches 3 types of variable reference with an optional format specifier + * \$(\w+) $var1 + * \[\[([\s\S]+?)(?::(\w+))?\]\] [[var2]] or [[var2:fmt2]] + * \${(\w+)(?::(\w+))?} ${var3} or ${var3:fmt3} + */ +export const variableRegex = /\$(\w+)|\[\[([\s\S]+?)(?::(\w+))?\]\]|\${(\w+)(?::(\w+))?}/g; + +// Helper function since lastIndex is not reset +export const variableRegexExec = (variableString: string) => { + variableRegex.lastIndex = 0; + return variableRegex.exec(variableString); +}; + export interface Variable { setValue(option); updateOptions(); @@ -14,15 +27,16 @@ export let variableTypes = {}; export { assignModelProperties }; export function containsVariable(...args: any[]) { - let variableName = args[args.length - 1]; - let str = args[0] || ''; + const variableName = args[args.length - 1]; + const variableString = args.slice(0, -1).join(' '); + const matches = variableString.match(variableRegex); + const isMatchingVariable = + matches !== null + ? matches.find(match => { + const varMatch = variableRegexExec(match); + return varMatch !== null && varMatch.indexOf(variableName) > -1; + }) + : false; - for (let i = 1; i < args.length - 1; i++) { - str += ' ' + args[i] || ''; - } - - variableName = kbn.regexEscape(variableName); - const findVarRegex = new RegExp('\\$(' + variableName + ')(?:\\W|$)|\\[\\[(' + variableName + ')\\]\\]', 'g'); - const match = findVarRegex.exec(str); - return match !== null; + return !!isMatchingVariable; } diff --git a/public/app/plugins/datasource/prometheus/datasource.ts b/public/app/plugins/datasource/prometheus/datasource.ts index 856ab035ea0..89f88a946c2 100644 --- a/public/app/plugins/datasource/prometheus/datasource.ts +++ b/public/app/plugins/datasource/prometheus/datasource.ts @@ -176,7 +176,6 @@ export class PrometheusDatasource { return this.$q.all(allQueryPromise).then(responseList => { let result = []; - let hints = []; _.each(responseList, (response, index) => { if (response.status === 'error') { @@ -196,19 +195,14 @@ export class PrometheusDatasource { end: queries[index].end, query: queries[index].expr, responseListLength: responseList.length, - responseIndex: index, refId: activeTargets[index].refId, + valueWithRefId: activeTargets[index].valueWithRefId, }; const series = this.resultTransformer.transform(response, transformerOptions); result = [...result, ...series]; - - if (queries[index].hinting) { - const queryHints = getQueryHints(series, this); - hints = [...hints, ...queryHints]; - } }); - return { data: result, hints }; + return { data: result }; }); } @@ -437,6 +431,10 @@ export class PrometheusDatasource { return state; } + getQueryHints(query: string, result: any[]) { + return getQueryHints(query, result, this); + } + loadRules() { this.metadataRequest('/api/v1/rules') .then(res => res.data || res.json()) diff --git a/public/app/plugins/datasource/prometheus/query_hints.ts b/public/app/plugins/datasource/prometheus/query_hints.ts index 5efaadb77be..cfd04c766ba 100644 --- a/public/app/plugins/datasource/prometheus/query_hints.ts +++ b/public/app/plugins/datasource/prometheus/query_hints.ts @@ -1,100 +1,92 @@ import _ from 'lodash'; -export function getQueryHints(series: any[], datasource?: any): any[] { - const hints = series.map((s, i) => { - const query: string = s.query; - const index: number = s.responseIndex; - if (query === undefined || index === undefined) { - return null; - } +export function getQueryHints(query: string, series?: any[], datasource?: any): any[] { + const hints = []; - // ..._bucket metric needs a histogram_quantile() - const histogramMetric = query.trim().match(/^\w+_bucket$/); - if (histogramMetric) { - const label = 'Time series has buckets, you probably wanted a histogram.'; - return { - index, + // ..._bucket metric needs a histogram_quantile() + const histogramMetric = query.trim().match(/^\w+_bucket$/); + if (histogramMetric) { + const label = 'Time series has buckets, you probably wanted a histogram.'; + hints.push({ + type: 'HISTOGRAM_QUANTILE', + label, + fix: { + label: 'Fix by adding histogram_quantile().', + action: { + type: 'ADD_HISTOGRAM_QUANTILE', + query, + }, + }, + }); + } + + // Check for monotony on series (table results are being ignored here) + if (series && series.length > 0) { + series.forEach(s => { + const datapoints: number[][] = s.datapoints; + if (query.indexOf('rate(') === -1 && datapoints.length > 1) { + let increasing = false; + const nonNullData = datapoints.filter(dp => dp[0] !== null); + const monotonic = nonNullData.every((dp, index) => { + if (index === 0) { + return true; + } + increasing = increasing || dp[0] > nonNullData[index - 1][0]; + // monotonic? + return dp[0] >= nonNullData[index - 1][0]; + }); + if (increasing && monotonic) { + const simpleMetric = query.trim().match(/^\w+$/); + let label = 'Time series is monotonously increasing.'; + let fix; + if (simpleMetric) { + fix = { + label: 'Fix by adding rate().', + action: { + type: 'ADD_RATE', + query, + }, + }; + } else { + label = `${label} Try applying a rate() function.`; + } + hints.push({ + type: 'APPLY_RATE', + label, + fix, + }); + } + } + }); + } + + // Check for recording rules expansion + if (datasource && datasource.ruleMappings) { + const mapping = datasource.ruleMappings; + const mappingForQuery = Object.keys(mapping).reduce((acc, ruleName) => { + if (query.search(ruleName) > -1) { + return { + ...acc, + [ruleName]: mapping[ruleName], + }; + } + return acc; + }, {}); + if (_.size(mappingForQuery) > 0) { + const label = 'Query contains recording rules.'; + hints.push({ + type: 'EXPAND_RULES', label, fix: { - label: 'Fix by adding histogram_quantile().', + label: 'Expand rules', action: { - type: 'ADD_HISTOGRAM_QUANTILE', + type: 'EXPAND_RULES', query, - index, + mapping: mappingForQuery, }, }, - }; - } - - // Check for monotony - const datapoints: number[][] = s.datapoints; - if (query.indexOf('rate(') === -1 && datapoints.length > 1) { - let increasing = false; - const nonNullData = datapoints.filter(dp => dp[0] !== null); - const monotonic = nonNullData.every((dp, index) => { - if (index === 0) { - return true; - } - increasing = increasing || dp[0] > nonNullData[index - 1][0]; - // monotonic? - return dp[0] >= nonNullData[index - 1][0]; }); - if (increasing && monotonic) { - const simpleMetric = query.trim().match(/^\w+$/); - let label = 'Time series is monotonously increasing.'; - let fix; - if (simpleMetric) { - fix = { - label: 'Fix by adding rate().', - action: { - type: 'ADD_RATE', - query, - index, - }, - }; - } else { - label = `${label} Try applying a rate() function.`; - } - return { - label, - index, - fix, - }; - } } - - // Check for recording rules expansion - if (datasource && datasource.ruleMappings) { - const mapping = datasource.ruleMappings; - const mappingForQuery = Object.keys(mapping).reduce((acc, ruleName) => { - if (query.search(ruleName) > -1) { - return { - ...acc, - [ruleName]: mapping[ruleName], - }; - } - return acc; - }, {}); - if (_.size(mappingForQuery) > 0) { - const label = 'Query contains recording rules.'; - return { - label, - index, - fix: { - label: 'Expand rules', - action: { - type: 'EXPAND_RULES', - query, - index, - mapping: mappingForQuery, - }, - }, - }; - } - } - - // No hint found - return null; - }); - return hints; + } + return hints.length > 0 ? hints : null; } diff --git a/public/app/plugins/datasource/prometheus/result_transformer.ts b/public/app/plugins/datasource/prometheus/result_transformer.ts index bf916bebf04..c9693eaf657 100644 --- a/public/app/plugins/datasource/prometheus/result_transformer.ts +++ b/public/app/plugins/datasource/prometheus/result_transformer.ts @@ -8,7 +8,14 @@ export class ResultTransformer { const prometheusResult = response.data.data.result; if (options.format === 'table') { - return [this.transformMetricDataToTable(prometheusResult, options.responseListLength, options.refId)]; + return [ + this.transformMetricDataToTable( + prometheusResult, + options.responseListLength, + options.refId, + options.valueWithRefId + ), + ]; } else if (options.format === 'heatmap') { let seriesList = []; prometheusResult.sort(sortSeriesByLabel); @@ -66,12 +73,11 @@ export class ResultTransformer { return { datapoints: dps, query: options.query, - responseIndex: options.responseIndex, target: metricLabel, }; } - transformMetricDataToTable(md, resultCount: number, refId: string) { + transformMetricDataToTable(md, resultCount: number, refId: string, valueWithRefId?: boolean) { const table = new TableModel(); let i, j; const metricLabels = {}; @@ -96,7 +102,7 @@ export class ResultTransformer { metricLabels[label] = labelIndex + 1; table.columns.push({ text: label, filterable: !label.startsWith('__') }); }); - const valueText = resultCount > 1 ? `Value #${refId}` : 'Value'; + const valueText = resultCount > 1 || valueWithRefId ? `Value #${refId}` : 'Value'; table.columns.push({ text: valueText }); // Populate rows, set value to empty string when label not present. diff --git a/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts b/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts index 2c72203137d..7eba54536fe 100644 --- a/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts +++ b/public/app/plugins/datasource/prometheus/specs/query_hints.test.ts @@ -2,34 +2,31 @@ import { getQueryHints } from '../query_hints'; describe('getQueryHints()', () => { it('returns no hints for no series', () => { - expect(getQueryHints([])).toEqual([]); + expect(getQueryHints('', [])).toEqual(null); }); it('returns no hints for empty series', () => { - expect(getQueryHints([{ datapoints: [], query: '' }])).toEqual([null]); + expect(getQueryHints('', [{ datapoints: [] }])).toEqual(null); }); it('returns no hint for a monotonously decreasing series', () => { - const series = [{ datapoints: [[23, 1000], [22, 1001]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[23, 1000], [22, 1001]] }]; + const hints = getQueryHints('metric', series); + expect(hints).toEqual(null); }); it('returns no hint for a flat series', () => { - const series = [ - { datapoints: [[null, 1000], [23, 1001], [null, 1002], [23, 1003]], query: 'metric', responseIndex: 0 }, - ]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[null, 1000], [23, 1001], [null, 1002], [23, 1003]] }]; + const hints = getQueryHints('metric', series); + expect(hints).toEqual(null); }); it('returns a rate hint for a monotonously increasing series', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('metric', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series is monotonously increasing.', - index: 0, fix: { action: { type: 'ADD_RATE', @@ -40,26 +37,25 @@ describe('getQueryHints()', () => { }); it('returns no rate hint for a monotonously increasing series that already has a rate', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'rate(metric[1m])', responseIndex: 0 }]; - const hints = getQueryHints(series); - expect(hints).toEqual([null]); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('rate(metric[1m])', series); + expect(hints).toEqual(null); }); it('returns a rate hint w/o action for a complex monotonously increasing series', () => { - const series = [{ datapoints: [[23, 1000], [24, 1001]], query: 'sum(metric)', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [24, 1001]] }]; + const hints = getQueryHints('sum(metric)', series); expect(hints.length).toBe(1); expect(hints[0].label).toContain('rate()'); expect(hints[0].fix).toBeUndefined(); }); it('returns a rate hint for a monotonously increasing series with missing data', () => { - const series = [{ datapoints: [[23, 1000], [null, 1001], [24, 1002]], query: 'metric', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000], [null, 1001], [24, 1002]] }]; + const hints = getQueryHints('metric', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series is monotonously increasing.', - index: 0, fix: { action: { type: 'ADD_RATE', @@ -70,12 +66,11 @@ describe('getQueryHints()', () => { }); it('returns a histogram hint for a bucket series', () => { - const series = [{ datapoints: [[23, 1000]], query: 'metric_bucket', responseIndex: 0 }]; - const hints = getQueryHints(series); + const series = [{ datapoints: [[23, 1000]] }]; + const hints = getQueryHints('metric_bucket', series); expect(hints.length).toBe(1); expect(hints[0]).toMatchObject({ label: 'Time series has buckets, you probably wanted a histogram.', - index: 0, fix: { action: { type: 'ADD_HISTOGRAM_QUANTILE', diff --git a/public/app/plugins/datasource/stackdriver/config_ctrl.ts b/public/app/plugins/datasource/stackdriver/config_ctrl.ts index d3847562156..e74659536c8 100644 --- a/public/app/plugins/datasource/stackdriver/config_ctrl.ts +++ b/public/app/plugins/datasource/stackdriver/config_ctrl.ts @@ -5,13 +5,23 @@ export class StackdriverConfigCtrl { jsonText: string; validationErrors: string[] = []; inputDataValid: boolean; + authenticationTypes: any[]; + defaultAuthenticationType: string; /** @ngInject */ constructor(datasourceSrv) { + this.defaultAuthenticationType = 'jwt'; this.datasourceSrv = datasourceSrv; this.current.jsonData = this.current.jsonData || {}; + this.current.jsonData.authenticationType = this.current.jsonData.authenticationType + ? this.current.jsonData.authenticationType + : this.defaultAuthenticationType; this.current.secureJsonData = this.current.secureJsonData || {}; this.current.secureJsonFields = this.current.secureJsonFields || {}; + this.authenticationTypes = [ + { key: this.defaultAuthenticationType, value: 'Google JWT File' }, + { key: 'gce', value: 'GCE Default Service Account' }, + ]; } save(jwt) { @@ -35,6 +45,10 @@ export class StackdriverConfigCtrl { this.validationErrors.push('Client Email field missing in JWT file.'); } + if (!jwt.project_id || jwt.project_id.length === 0) { + this.validationErrors.push('Project Id field missing in JWT file.'); + } + if (this.validationErrors.length === 0) { this.inputDataValid = true; return true; @@ -67,7 +81,7 @@ export class StackdriverConfigCtrl { this.inputDataValid = false; this.jsonText = ''; - this.current.jsonData = {}; + this.current.jsonData = Object.assign({}, { authenticationType: this.current.jsonData.authenticationType }); this.current.secureJsonData = {}; this.current.secureJsonFields = {}; } diff --git a/public/app/plugins/datasource/stackdriver/datasource.ts b/public/app/plugins/datasource/stackdriver/datasource.ts index b77abdbdab3..4a81eb8a619 100644 --- a/public/app/plugins/datasource/stackdriver/datasource.ts +++ b/public/app/plugins/datasource/stackdriver/datasource.ts @@ -1,11 +1,14 @@ import { stackdriverUnitMappings } from './constants'; import appEvents from 'app/core/app_events'; +import _ from 'lodash'; export default class StackdriverDatasource { id: number; url: string; baseUrl: string; projectName: string; + authenticationType: string; + queryPromise: Promise; /** @ngInject */ constructor(instanceSettings, private backendSrv, private templateSrv, private timeSrv) { @@ -14,6 +17,7 @@ export default class StackdriverDatasource { this.doRequest = this.doRequest; this.id = instanceSettings.id; this.projectName = instanceSettings.jsonData.defaultProject || ''; + this.authenticationType = instanceSettings.jsonData.authenticationType || 'jwt'; } async getTimeSeries(options) { @@ -46,16 +50,20 @@ export default class StackdriverDatasource { }; }); - const { data } = await this.backendSrv.datasourceRequest({ - url: '/api/tsdb/query', - method: 'POST', - data: { - from: options.range.from.valueOf().toString(), - to: options.range.to.valueOf().toString(), - queries, - }, - }); - return data; + if (queries.length > 0) { + const { data } = await this.backendSrv.datasourceRequest({ + url: '/api/tsdb/query', + method: 'POST', + data: { + from: options.range.from.valueOf().toString(), + to: options.range.to.valueOf().toString(), + queries, + }, + }); + return data; + } else { + return { results: [] }; + } } async getLabels(metricType, refId) { @@ -106,7 +114,7 @@ export default class StackdriverDatasource { if (!queryRes.series) { return; } - + this.projectName = queryRes.meta.defaultProject; const unit = this.resolvePanelUnitFromTargets(options.targets); queryRes.series.forEach(series => { let timeSerie: any = { @@ -121,9 +129,10 @@ export default class StackdriverDatasource { result.push(timeSerie); }); }); + return { data: result }; + } else { + return { data: [] }; } - - return { data: result }; } async annotationQuery(options) { @@ -173,76 +182,84 @@ export default class StackdriverDatasource { throw new Error('Template variables support is not yet imlemented'); } - testDatasource() { - const path = `v3/projects/${this.projectName}/metricDescriptors`; - return this.doRequest(`${this.baseUrl}${path}`) - .then(response => { - if (response.status === 200) { - return { - status: 'success', - message: 'Successfully queried the Stackdriver API.', - title: 'Success', - }; - } - - return { - status: 'error', - message: 'Returned http status code ' + response.status, - }; - }) - .catch(error => { - let message = 'Stackdriver: '; - message += error.statusText ? error.statusText + ': ' : ''; - + async testDatasource() { + let status, message; + const defaultErrorMessage = 'Cannot connect to Stackdriver API'; + try { + const projectName = await this.getDefaultProject(); + const path = `v3/projects/${projectName}/metricDescriptors`; + const response = await this.doRequest(`${this.baseUrl}${path}`); + if (response.status === 200) { + status = 'success'; + message = 'Successfully queried the Stackdriver API.'; + } else { + status = 'error'; + message = response.statusText ? response.statusText : defaultErrorMessage; + } + } catch (error) { + status = 'error'; + if (_.isString(error)) { + message = error; + } else { + message = 'Stackdriver: '; + message += error.statusText ? error.statusText : defaultErrorMessage; if (error.data && error.data.error && error.data.error.code) { - // 400, 401 - message += error.data.error.code + '. ' + error.data.error.message; - } else { - message += 'Cannot connect to Stackdriver API'; + message += ': ' + error.data.error.code + '. ' + error.data.error.message; } - return { - status: 'error', - message: message, - }; - }); + } + } finally { + return { + status, + message, + }; + } } - async getProjects() { - const response = await this.doRequest(`/cloudresourcemanager/v1/projects`); - return response.data.projects.map(p => ({ id: p.projectId, name: p.name })); + formatStackdriverError(error) { + let message = 'Stackdriver: '; + message += error.statusText ? error.statusText + ': ' : ''; + if (error.data && error.data.error) { + try { + const res = JSON.parse(error.data.error); + message += res.error.code + '. ' + res.error.message; + } catch (err) { + message += error.data.error; + } + } else { + message += 'Cannot connect to Stackdriver API'; + } + return message; } async getDefaultProject() { try { - const projects = await this.getProjects(); - if (projects && projects.length > 0) { - const test = projects.filter(p => p.id === this.projectName)[0]; - return test; + if (this.authenticationType === 'gce' || !this.projectName) { + const { data } = await this.backendSrv.datasourceRequest({ + url: '/api/tsdb/query', + method: 'POST', + data: { + queries: [ + { + refId: 'ensureDefaultProjectQuery', + type: 'ensureDefaultProjectQuery', + datasourceId: this.id, + }, + ], + }, + }); + this.projectName = data.results.ensureDefaultProjectQuery.meta.defaultProject; + return this.projectName; } else { - throw new Error('No projects found'); + return this.projectName; } } catch (error) { - let message = 'Projects cannot be fetched: '; - message += error.statusText ? error.statusText + ': ' : ''; - if (error && error.data && error.data.error && error.data.error.message) { - if (error.data.error.code === 403) { - message += ` - A list of projects could not be fetched from the Google Cloud Resource Manager API. - You might need to enable it first: - https://console.developers.google.com/apis/library/cloudresourcemanager.googleapis.com`; - } else { - message += error.data.error.code + '. ' + error.data.error.message; - } - } else { - message += 'Cannot connect to Stackdriver API'; - } - appEvents.emit('ds-request-error', message); + throw this.formatStackdriverError(error); } } - async getMetricTypes(projectId: string) { + async getMetricTypes(projectName: string) { try { - const metricsApiPath = `v3/projects/${projectId}/metricDescriptors`; + const metricsApiPath = `v3/projects/${projectName}/metricDescriptors`; const { data } = await this.doRequest(`${this.baseUrl}${metricsApiPath}`); const metrics = data.metricDescriptors.map(m => { @@ -256,7 +273,8 @@ export default class StackdriverDatasource { return metrics; } catch (error) { - console.log(error); + appEvents.emit('ds-request-error', this.formatStackdriverError(error)); + return []; } } diff --git a/public/app/plugins/datasource/stackdriver/filter_segments.ts b/public/app/plugins/datasource/stackdriver/filter_segments.ts index 9eb27f31975..5adb56e2fcf 100644 --- a/public/app/plugins/datasource/stackdriver/filter_segments.ts +++ b/public/app/plugins/datasource/stackdriver/filter_segments.ts @@ -44,7 +44,7 @@ export class FilterSegments { this.removeSegment.value = DefaultRemoveFilterValue; return Promise.resolve([this.removeSegment]); } else { - return this.getFilterKeysFunc(); + return this.getFilterKeysFunc(segment, DefaultRemoveFilterValue); } } diff --git a/public/app/plugins/datasource/stackdriver/partials/config.html b/public/app/plugins/datasource/stackdriver/partials/config.html index 46b79d8bb0d..9be7eef0e68 100644 --- a/public/app/plugins/datasource/stackdriver/partials/config.html +++ b/public/app/plugins/datasource/stackdriver/partials/config.html @@ -1,37 +1,54 @@
-
GCP Service Account
+

Stackdriver Authentication

+

There are two ways to authenticate the Stackdriver plugin - either by uploading a Service Account key file, or by + automatically retrieving credentials from the Google metadata server. The latter option is only available + when running Grafana on a GCE virtual machine.

+ +
Uploading a Service Account Key File

- To authenticate with the Stackdriver API, you need to create a Google Cloud Platform (GCP) Service Account for + First you need to create a Google Cloud Platform (GCP) Service Account for the Project you want to show data for. A Grafana datasource integrates with one GCP Project. If you want to visualize data from multiple GCP Projects then you need to create one datasource per GCP Project.

- The Monitoring Viewer role provides all the permissions that Grafana needs. + The Monitoring Viewer role provides all the permissions that Grafana needs. The following API + needs to be enabled on GCP for the datasource to work: Monitoring + API

+ +
GCE Default Service Account

- The following APIs need to be enabled on GCP for the datasource to work: -

+ If Grafana is running on a Google Compute Engine (GCE) virtual machine, it is possible for Grafana to + automatically retrieve the default project id and authentication token from the metadata server. In order for this to + work, you need to make sure that you have a service account that is setup as the default account for the virtual + machine and that the service account has been given read access to the Stackdriver Monitoring API.

+

Detailed instructions on how to create a Service Account can be found in - the documentation.

+ the documentation. +

-

Service Account Authentication

+

Authentication

Upload your Service Account key file or paste in the contents of the file. The file contents will be encrypted and saved in the Grafana database.
-
+
+
+ Authentication Type +
+ +
+
+
+ +
@@ -52,23 +69,23 @@
-
+
Uploaded Key Details
- Project + Project
- Client Email - -
+ Client Email + +
- Token URI + Token URI
- Private Key + Private Key
@@ -81,6 +98,8 @@
-
- Do not forget to save your changes after uploading a file. -
+

Do not forget to save your changes after uploading a file.

+ +

+ Verify GCE default service account by clicking Save & Test

diff --git a/public/app/plugins/datasource/stackdriver/partials/query.editor.html b/public/app/plugins/datasource/stackdriver/partials/query.editor.html index 7cc93e9d1a8..98c8fcc83e8 100755 --- a/public/app/plugins/datasource/stackdriver/partials/query.editor.html +++ b/public/app/plugins/datasource/stackdriver/partials/query.editor.html @@ -15,8 +15,7 @@
Project - +