The new table panel is very flexible, supporting both multiple modes for time series as well as for
-table, annotation and raw JSON data. It also provides date formating and value formating and coloring options.
+table, annotation and raw JSON data. It also provides date formatting and value formatting and coloring options.
### Time series to rows
diff --git a/docs/sources/guides/whats-new-in-v4-1.md b/docs/sources/guides/whats-new-in-v4-1.md
index bd2b0f1b75f..217b21b545e 100644
--- a/docs/sources/guides/whats-new-in-v4-1.md
+++ b/docs/sources/guides/whats-new-in-v4-1.md
@@ -33,7 +33,7 @@ You can enable/disable the shared tooltip from the dashboard settings menu or cy
{{< imgbox max-width="60%" img="/img/docs/v41/helptext_for_panel_settings.png" caption="Hovering help text" >}}
-You can set a help text in the general tab on any panel. The help text is using Markdown to enable better formating and linking to other sites that can provide more information.
+You can set a help text in the general tab on any panel. The help text is using Markdown to enable better formatting and linking to other sites that can provide more information.
diff --git a/docs/sources/guides/whats-new-in-v4-5.md b/docs/sources/guides/whats-new-in-v4-5.md
index b2de451308a..a5cd3ca982d 100644
--- a/docs/sources/guides/whats-new-in-v4-5.md
+++ b/docs/sources/guides/whats-new-in-v4-5.md
@@ -12,7 +12,7 @@ weight = -4
# What's New in Grafana v4.5
-## Hightlights
+## Highlights
### New prometheus query editor
@@ -62,7 +62,7 @@ Datas source selection & options & help are now above your metric queries.
### Minor Changes
* **InfluxDB**: Change time range filter for absolute time ranges to be inclusive instead of exclusive [#8319](https://github.com/grafana/grafana/issues/8319), thx [@Oxydros](https://github.com/Oxydros)
-* **InfluxDB**: Added paranthesis around tag filters in queries [#9131](https://github.com/grafana/grafana/pull/9131)
+* **InfluxDB**: Added parenthesis around tag filters in queries [#9131](https://github.com/grafana/grafana/pull/9131)
## Bug Fixes
diff --git a/docs/sources/guides/whats-new-in-v4-6.md b/docs/sources/guides/whats-new-in-v4-6.md
index fd75384761f..09955fa58cc 100644
--- a/docs/sources/guides/whats-new-in-v4-6.md
+++ b/docs/sources/guides/whats-new-in-v4-6.md
@@ -45,7 +45,7 @@ This makes exploring and filtering Prometheus data much easier.
* **GCS**: Adds support for Google Cloud Storage [#8370](https://github.com/grafana/grafana/issues/8370) thx [@chuhlomin](https://github.com/chuhlomin)
* **Prometheus**: Adds /metrics endpoint for exposing Grafana metrics. [#9187](https://github.com/grafana/grafana/pull/9187)
-* **Graph**: Add support for local formating in axis. [#1395](https://github.com/grafana/grafana/issues/1395), thx [@m0nhawk](https://github.com/m0nhawk)
+* **Graph**: Add support for local formatting in axis. [#1395](https://github.com/grafana/grafana/issues/1395), thx [@m0nhawk](https://github.com/m0nhawk)
* **Jaeger**: Add support for open tracing using jaeger in Grafana. [#9213](https://github.com/grafana/grafana/pull/9213)
* **Unit types**: New date & time unit types added, useful in singlestat to show dates & times. [#3678](https://github.com/grafana/grafana/issues/3678), [#6710](https://github.com/grafana/grafana/issues/6710), [#2764](https://github.com/grafana/grafana/issues/2764)
* **CLI**: Make it possible to install plugins from any url [#5873](https://github.com/grafana/grafana/issues/5873)
diff --git a/docs/sources/http_api/org.md b/docs/sources/http_api/org.md
index 4c1dff904c8..b9a15450786 100644
--- a/docs/sources/http_api/org.md
+++ b/docs/sources/http_api/org.md
@@ -307,7 +307,7 @@ Content-Type: application/json
`PUT /api/orgs/:orgId`
-Update Organisation, fields *Adress 1*, *Adress 2*, *City* are not implemented yet.
+Update Organisation, fields *Address 1*, *Address 2*, *City* are not implemented yet.
**Example Request**:
@@ -436,4 +436,4 @@ HTTP/1.1 200
Content-Type: application/json
{"message":"User removed from organization"}
-```
\ No newline at end of file
+```
diff --git a/docs/sources/installation/configuration.md b/docs/sources/installation/configuration.md
index 6169280b798..b7fe9040574 100644
--- a/docs/sources/installation/configuration.md
+++ b/docs/sources/installation/configuration.md
@@ -482,7 +482,7 @@ Set api_url to the resource that returns [OpenID UserInfo](https://connect2id.co
First set up Grafana as an OpenId client "webapplication" in Okta. Then set the Base URIs to `https://Teams settings
@@ -76,7 +76,7 @@ func (this *TeamsNotifier) Notify(evalContext *alerting.EvalContext) error { } message := this.Mention - if evalContext.Rule.State != m.AlertStateOK { //dont add message when going back to alert state ok. + if evalContext.Rule.State != m.AlertStateOK { //don't add message when going back to alert state ok. message += " " + evalContext.Rule.Message } else { message += " " // summary must not be empty diff --git a/pkg/services/alerting/notifiers/telegram_test.go b/pkg/services/alerting/notifiers/telegram_test.go index 05be787dced..98c8d884ad0 100644 --- a/pkg/services/alerting/notifiers/telegram_test.go +++ b/pkg/services/alerting/notifiers/telegram_test.go @@ -100,7 +100,7 @@ func TestTelegramNotifier(t *testing.T) { So(caption, ShouldContainSubstring, "Some kind of message that is too long for appending to our pretty little message, this line is actually exactly 197 chars long and I will get there in the end I promise ") }) - Convey("Metrics should be skipped if they dont fit", func() { + Convey("Metrics should be skipped if they don't fit", func() { evalContext := alerting.NewEvalContext(nil, &alerting.Rule{ Name: "This is an alarm", Message: "Some kind of message that is too long for appending to our pretty little message, this line is actually exactly 197 chars long and I will get there in the end I ", diff --git a/pkg/services/alerting/result_handler.go b/pkg/services/alerting/result_handler.go index 8f9deb758a6..5d95e090c9e 100644 --- a/pkg/services/alerting/result_handler.go +++ b/pkg/services/alerting/result_handler.go @@ -56,7 +56,7 @@ func (handler *DefaultResultHandler) Handle(evalContext *EvalContext) error { if err := bus.Dispatch(cmd); err != nil { if err == m.ErrCannotChangeStateOnPausedAlert { - handler.log.Error("Cannot change state on alert thats pause", "error", err) + handler.log.Error("Cannot change state on alert that's paused", "error", err) return err } diff --git a/pkg/services/alerting/scheduler.go b/pkg/services/alerting/scheduler.go index 151f802ec15..b0a3f8303c4 100644 --- a/pkg/services/alerting/scheduler.go +++ b/pkg/services/alerting/scheduler.go @@ -58,7 +58,7 @@ func (s *SchedulerImpl) Tick(tickTime time.Time, execQueue chan *Job) { if job.OffsetWait && now%job.Offset == 0 { job.OffsetWait = false - s.enque(job, execQueue) + s.enqueue(job, execQueue) continue } @@ -66,13 +66,13 @@ func (s *SchedulerImpl) Tick(tickTime time.Time, execQueue chan *Job) { if job.Offset > 0 { job.OffsetWait = true } else { - s.enque(job, execQueue) + s.enqueue(job, execQueue) } } } } -func (s *SchedulerImpl) enque(job *Job, execQueue chan *Job) { +func (s *SchedulerImpl) enqueue(job *Job, execQueue chan *Job) { s.log.Debug("Scheduler: Putting job on to exec queue", "name", job.Rule.Name, "id", job.Rule.Id) execQueue <- job } diff --git a/pkg/services/guardian/guardian.go b/pkg/services/guardian/guardian.go index 811b38cac86..6e13817b902 100644 --- a/pkg/services/guardian/guardian.go +++ b/pkg/services/guardian/guardian.go @@ -113,7 +113,7 @@ func (g *dashboardGuardianImpl) checkAcl(permission m.PermissionType, acl []*m.D return false, err } - // evalute team rules + // evaluate team rules for _, p := range acl { for _, ug := range teams { if ug.Id == p.TeamId && p.Permission >= permission { diff --git a/pkg/services/provisioning/dashboards/config_reader.go b/pkg/services/provisioning/dashboards/config_reader.go index 9030ba609b9..8ac79df0fac 100644 --- a/pkg/services/provisioning/dashboards/config_reader.go +++ b/pkg/services/provisioning/dashboards/config_reader.go @@ -58,7 +58,7 @@ func (cr *configReader) readConfig() ([]*DashboardsAsConfig, error) { files, err := ioutil.ReadDir(cr.path) if err != nil { - cr.log.Error("cant read dashboard provisioning files from directory", "path", cr.path) + cr.log.Error("can't read dashboard provisioning files from directory", "path", cr.path) return dashboards, nil } diff --git a/pkg/services/provisioning/datasources/config_reader.go b/pkg/services/provisioning/datasources/config_reader.go index 58ed5472a6b..4b8931f0ed3 100644 --- a/pkg/services/provisioning/datasources/config_reader.go +++ b/pkg/services/provisioning/datasources/config_reader.go @@ -19,7 +19,7 @@ func (cr *configReader) readConfig(path string) ([]*DatasourcesAsConfig, error) files, err := ioutil.ReadDir(path) if err != nil { - cr.log.Error("cant read datasource provisioning files from directory", "path", path) + cr.log.Error("can't read datasource provisioning files from directory", "path", path) return datasources, nil } diff --git a/pkg/services/sqlstore/alert_notification_test.go b/pkg/services/sqlstore/alert_notification_test.go index d37062fb58f..761114978a8 100644 --- a/pkg/services/sqlstore/alert_notification_test.go +++ b/pkg/services/sqlstore/alert_notification_test.go @@ -21,7 +21,7 @@ func TestAlertNotificationSQLAccess(t *testing.T) { } err := GetAlertNotifications(cmd) - fmt.Printf("errror %v", err) + fmt.Printf("error %v", err) So(err, ShouldBeNil) So(cmd.Result, ShouldBeNil) }) diff --git a/pkg/services/sqlstore/sqlstore.go b/pkg/services/sqlstore/sqlstore.go index 6aace350193..782318fa188 100644 --- a/pkg/services/sqlstore/sqlstore.go +++ b/pkg/services/sqlstore/sqlstore.go @@ -258,7 +258,7 @@ func InitTestDB(t *testing.T) *xorm.Engine { // x.ShowSQL() if err != nil { - t.Fatalf("Failed to init in memory sqllite3 db %v", err) + t.Fatalf("Failed to init test database: %v", err) } sqlutil.CleanDB(x) @@ -269,3 +269,19 @@ func InitTestDB(t *testing.T) *xorm.Engine { return x } + +func IsTestDbMySql() bool { + if db, present := os.LookupEnv("GRAFANA_TEST_DB"); present { + return db == dbMySql + } + + return false +} + +func IsTestDbPostgres() bool { + if db, present := os.LookupEnv("GRAFANA_TEST_DB"); present { + return db == dbPostgres + } + + return false +} diff --git a/pkg/tsdb/mssql/macros.go b/pkg/tsdb/mssql/macros.go index 9d41cd03255..bb9489cd654 100644 --- a/pkg/tsdb/mssql/macros.go +++ b/pkg/tsdb/mssql/macros.go @@ -82,11 +82,11 @@ func (m *MsSqlMacroEngine) evaluateMacro(name string, args []string) (string, er if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("%s >= DATEADD(s, %d, '1970-01-01') AND %s <= DATEADD(s, %d, '1970-01-01')", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), args[0], uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%s >= DATEADD(s, %d, '1970-01-01') AND %s <= DATEADD(s, %d, '1970-01-01')", args[0], m.TimeRange.GetFromAsSecondsEpoch(), args[0], m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeFrom": - return fmt.Sprintf("DATEADD(second, %d, '1970-01-01')", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("DATEADD(second, %d, '1970-01-01')", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__timeTo": - return fmt.Sprintf("DATEADD(second, %d, '1970-01-01')", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("DATEADD(second, %d, '1970-01-01')", m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeGroup": if len(args) < 2 { return "", fmt.Errorf("macro %v needs time column and interval", name) @@ -113,11 +113,11 @@ func (m *MsSqlMacroEngine) evaluateMacro(name string, args []string) (string, er if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), args[0], uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.TimeRange.GetFromAsSecondsEpoch(), args[0], m.TimeRange.GetToAsSecondsEpoch()), nil case "__unixEpochFrom": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__unixEpochTo": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetToAsSecondsEpoch()), nil default: return "", fmt.Errorf("Unknown macro %v", name) } diff --git a/pkg/tsdb/mssql/macros_test.go b/pkg/tsdb/mssql/macros_test.go index 12a9b0d82be..ae0d4f67d2b 100644 --- a/pkg/tsdb/mssql/macros_test.go +++ b/pkg/tsdb/mssql/macros_test.go @@ -1,6 +1,8 @@ package mssql import ( + "fmt" + "strconv" "testing" "time" @@ -13,112 +15,213 @@ import ( func TestMacroEngine(t *testing.T) { Convey("MacroEngine", t, func() { engine := &MsSqlMacroEngine{} - timeRange := &tsdb.TimeRange{From: "5m", To: "now"} query := &tsdb.Query{ Model: simplejson.New(), } - Convey("interpolate __time function", func() { - sql, err := engine.Interpolate(query, nil, "select $__time(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 2018-04-12 00:00 and 2018-04-12 00:05", func() { + from := time.Date(2018, 4, 12, 18, 0, 0, 0, time.UTC) + to := from.Add(5 * time.Minute) + timeRange := tsdb.NewFakeTimeRange("5m", "now", to) - So(sql, ShouldEqual, "select time_column AS time") + Convey("interpolate __time function", func() { + sql, err := engine.Interpolate(query, nil, "select $__time(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select time_column AS time") + }) + + Convey("interpolate __timeEpoch function", func() { + sql, err := engine.Interpolate(query, nil, "select $__timeEpoch(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select DATEDIFF(second, '1970-01-01', time_column) AS time") + }) + + Convey("interpolate __timeEpoch function wrapped in aggregation", func() { + sql, err := engine.Interpolate(query, nil, "select min($__timeEpoch(time_column))") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select min(DATEDIFF(second, '1970-01-01', time_column) AS time)") + }) + + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= DATEADD(s, %d, '1970-01-01') AND time_column <= DATEADD(s, %d, '1970-01-01')", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeGroup function", func() { + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) as bigint)*300") + }) + + Convey("interpolate __timeGroup function with spaces around arguments", func() { + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) as bigint)*300") + }) + + Convey("interpolate __timeGroup function with fill (value = NULL)", func() { + _, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m', NULL)") + + fill := query.Model.Get("fill").MustBool() + fillNull := query.Model.Get("fillNull").MustBool() + fillInterval := query.Model.Get("fillInterval").MustInt() + + So(err, ShouldBeNil) + So(fill, ShouldBeTrue) + So(fillNull, ShouldBeTrue) + So(fillInterval, ShouldEqual, 5*time.Minute.Seconds()) + }) + + Convey("interpolate __timeGroup function with fill (value = float)", func() { + _, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m', 1.5)") + + fill := query.Model.Get("fill").MustBool() + fillValue := query.Model.Get("fillValue").MustFloat64() + fillInterval := query.Model.Get("fillInterval").MustInt() + + So(err, ShouldBeNil) + So(fill, ShouldBeTrue) + So(fillValue, ShouldEqual, 1.5) + So(fillInterval, ShouldEqual, 5*time.Minute.Seconds()) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __timeEpoch function", func() { - sql, err := engine.Interpolate(query, nil, "select $__timeEpoch(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1965-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1965, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "select DATEDIFF(second, '1970-01-01', time_column) AS time") + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= DATEADD(s, %d, '1970-01-01') AND time_column <= DATEADD(s, %d, '1970-01-01')", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __timeEpoch function wrapped in aggregation", func() { - sql, err := engine.Interpolate(query, nil, "select min($__timeEpoch(time_column))") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1980-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1980, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "select min(DATEDIFF(second, '1970-01-01', time_column) AS time)") - }) + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) - Convey("interpolate __timeFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") - So(err, ShouldBeNil) + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= DATEADD(s, %d, '1970-01-01') AND time_column <= DATEADD(s, %d, '1970-01-01')", from.Unix(), to.Unix())) + }) - So(sql, ShouldEqual, "WHERE time_column >= DATEADD(s, 18446744066914186738, '1970-01-01') AND time_column <= DATEADD(s, 18446744066914187038, '1970-01-01')") - }) + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) - Convey("interpolate __timeGroup function", func() { - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") - So(err, ShouldBeNil) + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", from.Unix())) + }) - So(sql, ShouldEqual, "GROUP BY CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) as bigint)*300") - }) + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) - Convey("interpolate __timeGroup function with spaces around arguments", func() { - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") - So(err, ShouldBeNil) + So(sql, ShouldEqual, fmt.Sprintf("select DATEADD(second, %d, '1970-01-01')", to.Unix())) + }) - So(sql, ShouldEqual, "GROUP BY CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) as bigint)*300") - }) + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time_column)") + So(err, ShouldBeNil) - Convey("interpolate __timeGroup function with fill (value = NULL)", func() { - _, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m', NULL)") + So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix())) + }) - fill := query.Model.Get("fill").MustBool() - fillNull := query.Model.Get("fillNull").MustBool() - fillInterval := query.Model.Get("fillInterval").MustInt() + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) - So(err, ShouldBeNil) - So(fill, ShouldBeTrue) - So(fillNull, ShouldBeTrue) - So(fillInterval, ShouldEqual, 5*time.Minute.Seconds()) - }) + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) - Convey("interpolate __timeGroup function with fill (value = float)", func() { - _, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m', 1.5)") + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) - fill := query.Model.Get("fill").MustBool() - fillValue := query.Model.Get("fillValue").MustFloat64() - fillInterval := query.Model.Get("fillInterval").MustInt() - - So(err, ShouldBeNil) - So(fill, ShouldBeTrue) - So(fillValue, ShouldEqual, 1.5) - So(fillInterval, ShouldEqual, 5*time.Minute.Seconds()) - }) - - Convey("interpolate __timeFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select DATEADD(second, 18446744066914186738, '1970-01-01')") - }) - - Convey("interpolate __timeTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select DATEADD(second, 18446744066914187038, '1970-01-01')") - }) - - Convey("interpolate __unixEpochFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select time_column >= 18446744066914186738 AND time_column <= 18446744066914187038") - }) - - Convey("interpolate __unixEpochFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914186738") - }) - - Convey("interpolate __unixEpochTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914187038") + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) }) } diff --git a/pkg/tsdb/mssql/mssql.go b/pkg/tsdb/mssql/mssql.go index 2638fd8bb40..3a440859f9f 100644 --- a/pkg/tsdb/mssql/mssql.go +++ b/pkg/tsdb/mssql/mssql.go @@ -8,8 +8,6 @@ import ( "strconv" "strings" - "time" - "math" _ "github.com/denisenkom/go-mssqldb" @@ -231,15 +229,18 @@ func (e MssqlQueryEndpoint) transformToTimeSeries(query *tsdb.Query, rows *core. return err } + // converts column named time to unix timestamp in milliseconds to make + // native mysql datetime types and epoch dates work in + // annotation and table queries. + tsdb.ConvertSqlTimeColumnToEpochMs(values, timeIndex) + switch columnValue := values[timeIndex].(type) { case int64: - timestamp = float64(columnValue * 1000) + timestamp = float64(columnValue) case float64: - timestamp = columnValue * 1000 - case time.Time: - timestamp = (float64(columnValue.Unix()) * 1000) + float64(columnValue.Nanosecond()/1e6) // in case someone is trying to map times beyond 2262 :D + timestamp = columnValue default: - return fmt.Errorf("Invalid type for column time, must be of type timestamp or unix timestamp") + return fmt.Errorf("Invalid type for column time, must be of type timestamp or unix timestamp, got: %T %v", columnValue, columnValue) } if metricIndex >= 0 { diff --git a/pkg/tsdb/mssql/mssql_test.go b/pkg/tsdb/mssql/mssql_test.go index 4bd1e3a8ad7..599f4869f6a 100644 --- a/pkg/tsdb/mssql/mssql_test.go +++ b/pkg/tsdb/mssql/mssql_test.go @@ -16,10 +16,10 @@ import ( ) // To run this test, remove the Skip from SkipConvey -// and set up a MSSQL db named grafanatest and a user/password grafana/Password! +// The tests require a MSSQL db named grafanatest and a user/password grafana/Password! // Use the docker/blocks/mssql_tests/docker-compose.yaml to spin up a // preconfigured MSSQL server suitable for running these tests. -// Thers's also a dashboard.json in same directory that you can import to Grafana +// There is also a dashboard.json in same directory that you can import to Grafana // once you've created a datasource for the test server/database. // If needed, change the variable below to the IP address of the database. var serverIP string = "localhost" @@ -188,10 +188,8 @@ func TestMSSQL(t *testing.T) { }) } - for _, s := range series { - _, err = sess.Insert(s) - So(err, ShouldBeNil) - } + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) Convey("When doing a metric query using timeGroup", func() { query := &tsdb.TsdbQuery{ @@ -312,10 +310,18 @@ func TestMSSQL(t *testing.T) { Convey("Given a table with metrics having multiple values and measurements", func() { type metric_values struct { - Time time.Time - Measurement string - ValueOne int64 `xorm:"integer 'valueOne'"` - ValueTwo int64 `xorm:"integer 'valueTwo'"` + Time time.Time + TimeInt64 int64 `xorm:"bigint 'timeInt64' not null"` + TimeInt64Nullable *int64 `xorm:"bigint 'timeInt64Nullable' null"` + TimeFloat64 float64 `xorm:"float 'timeFloat64' not null"` + TimeFloat64Nullable *float64 `xorm:"float 'timeFloat64Nullable' null"` + TimeInt32 int32 `xorm:"int(11) 'timeInt32' not null"` + TimeInt32Nullable *int32 `xorm:"int(11) 'timeInt32Nullable' null"` + TimeFloat32 float32 `xorm:"float(11) 'timeFloat32' not null"` + TimeFloat32Nullable *float32 `xorm:"float(11) 'timeFloat32Nullable' null"` + Measurement string + ValueOne int64 `xorm:"integer 'valueOne'"` + ValueTwo int64 `xorm:"integer 'valueTwo'"` } if exist, err := sess.IsTableExist(metric_values{}); err != nil || exist { @@ -330,26 +336,219 @@ func TestMSSQL(t *testing.T) { return rand.Int63n(max-min) + min } + var tInitial time.Time + series := []*metric_values{} - for _, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric A", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric B", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) + for i, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { + if i == 0 { + tInitial = t + } + tSeconds := t.Unix() + tSecondsInt32 := int32(tSeconds) + tSecondsFloat32 := float32(tSeconds) + tMilliseconds := tSeconds * 1e3 + tMillisecondsFloat := float64(tMilliseconds) + first := metric_values{ + Time: t, + TimeInt64: tMilliseconds, + TimeInt64Nullable: &(tMilliseconds), + TimeFloat64: tMillisecondsFloat, + TimeFloat64Nullable: &tMillisecondsFloat, + TimeInt32: tSecondsInt32, + TimeInt32Nullable: &tSecondsInt32, + TimeFloat32: tSecondsFloat32, + TimeFloat32Nullable: &tSecondsFloat32, + Measurement: "Metric A", + ValueOne: rnd(0, 100), + ValueTwo: rnd(0, 100), + } + second := first + second.Measurement = "Metric B" + second.ValueOne = rnd(0, 100) + second.ValueTwo = rnd(0, 100) + + series = append(series, &first) + series = append(series, &second) } - for _, s := range series { - _, err = sess.Insert(s) + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) + + Convey("When doing a metric query using epoch (int64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeInt64 as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) So(err, ShouldBeNil) - } + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeInt64Nullable as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeFloat64 as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeFloat64Nullable as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeInt32 as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeInt32Nullable as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeFloat32 as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) + + Convey("When doing a metric query using epoch (float32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT TOP 1 timeFloat32Nullable as time, valueOne FROM metric_values ORDER BY time`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) Convey("When doing a metric query grouping by time and select metric column should return correct series", func() { query := &tsdb.TsdbQuery{ @@ -476,7 +675,6 @@ func TestMSSQL(t *testing.T) { resp, err := endpoint.Query(nil, nil, query) queryResult := resp.Results["A"] So(err, ShouldBeNil) - fmt.Println("query", "sql", queryResult.Meta) So(queryResult.Error, ShouldBeNil) So(len(queryResult.Series), ShouldEqual, 4) @@ -696,7 +894,7 @@ func TestMSSQL(t *testing.T) { columns := queryResult.Tables[0].Rows[0] //Should be in milliseconds - So(columns[0].(float64), ShouldEqual, float64(dt.Unix()*1000)) + So(columns[0].(float64), ShouldEqual, float64(dt.UnixNano()/1e6)) }) Convey("When doing an annotation query with a time column in epoch second format should return ms", func() { @@ -850,15 +1048,15 @@ func TestMSSQL(t *testing.T) { func InitMSSQLTestDB(t *testing.T) *xorm.Engine { x, err := xorm.NewEngine(sqlutil.TestDB_Mssql.DriverName, strings.Replace(sqlutil.TestDB_Mssql.ConnStr, "localhost", serverIP, 1)) + if err != nil { + t.Fatalf("Failed to init mssql db %v", err) + } + x.DatabaseTZ = time.UTC x.TZLocation = time.UTC // x.ShowSQL() - if err != nil { - t.Fatalf("Failed to init mssql db %v", err) - } - return x } diff --git a/pkg/tsdb/mysql/macros.go b/pkg/tsdb/mysql/macros.go index a292f209429..fadcbe4edbc 100644 --- a/pkg/tsdb/mysql/macros.go +++ b/pkg/tsdb/mysql/macros.go @@ -77,11 +77,11 @@ func (m *MySqlMacroEngine) evaluateMacro(name string, args []string) (string, er if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("%s >= FROM_UNIXTIME(%d) AND %s <= FROM_UNIXTIME(%d)", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), args[0], uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%s >= FROM_UNIXTIME(%d) AND %s <= FROM_UNIXTIME(%d)", args[0], m.TimeRange.GetFromAsSecondsEpoch(), args[0], m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeFrom": - return fmt.Sprintf("FROM_UNIXTIME(%d)", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("FROM_UNIXTIME(%d)", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__timeTo": - return fmt.Sprintf("FROM_UNIXTIME(%d)", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("FROM_UNIXTIME(%d)", m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeGroup": if len(args) < 2 { return "", fmt.Errorf("macro %v needs time column and interval", name) @@ -108,11 +108,11 @@ func (m *MySqlMacroEngine) evaluateMacro(name string, args []string) (string, er if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), args[0], uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.TimeRange.GetFromAsSecondsEpoch(), args[0], m.TimeRange.GetToAsSecondsEpoch()), nil case "__unixEpochFrom": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__unixEpochTo": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetToAsSecondsEpoch()), nil default: return "", fmt.Errorf("Unknown macro %v", name) } diff --git a/pkg/tsdb/mysql/macros_test.go b/pkg/tsdb/mysql/macros_test.go index a89ba16ab78..66ec143eac8 100644 --- a/pkg/tsdb/mysql/macros_test.go +++ b/pkg/tsdb/mysql/macros_test.go @@ -1,7 +1,10 @@ package mysql import ( + "fmt" + "strconv" "testing" + "time" "github.com/grafana/grafana/pkg/tsdb" . "github.com/smartystreets/goconvey/convey" @@ -11,79 +14,179 @@ func TestMacroEngine(t *testing.T) { Convey("MacroEngine", t, func() { engine := &MySqlMacroEngine{} query := &tsdb.Query{} - timeRange := &tsdb.TimeRange{From: "5m", To: "now"} - Convey("interpolate __time function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__time(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 2018-04-12 00:00 and 2018-04-12 00:05", func() { + from := time.Date(2018, 4, 12, 18, 0, 0, 0, time.UTC) + to := from.Add(5 * time.Minute) + timeRange := tsdb.NewFakeTimeRange("5m", "now", to) - So(sql, ShouldEqual, "select UNIX_TIMESTAMP(time_column) as time_sec") + Convey("interpolate __time function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__time(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select UNIX_TIMESTAMP(time_column) as time_sec") + }) + + Convey("interpolate __time function wrapped in aggregation", func() { + sql, err := engine.Interpolate(query, timeRange, "select min($__time(time_column))") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select min(UNIX_TIMESTAMP(time_column) as time_sec)") + }) + + Convey("interpolate __timeGroup function", func() { + + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY cast(cast(UNIX_TIMESTAMP(time_column)/(300) as signed)*300 as signed)") + }) + + Convey("interpolate __timeGroup function with spaces around arguments", func() { + + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY cast(cast(UNIX_TIMESTAMP(time_column)/(300) as signed)*300 as signed)") + }) + + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= FROM_UNIXTIME(%d) AND time_column <= FROM_UNIXTIME(%d)", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __time function wrapped in aggregation", func() { - sql, err := engine.Interpolate(query, timeRange, "select min($__time(time_column))") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1965-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1965, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "select min(UNIX_TIMESTAMP(time_column) as time_sec)") + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= FROM_UNIXTIME(%d) AND time_column <= FROM_UNIXTIME(%d)", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __timeFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1980-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1980, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "WHERE time_column >= FROM_UNIXTIME(18446744066914186738) AND time_column <= FROM_UNIXTIME(18446744066914187038)") + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE time_column >= FROM_UNIXTIME(%d) AND time_column <= FROM_UNIXTIME(%d)", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select FROM_UNIXTIME(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - - Convey("interpolate __timeFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select FROM_UNIXTIME(18446744066914186738)") - }) - - Convey("interpolate __timeGroup function", func() { - - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "GROUP BY cast(cast(UNIX_TIMESTAMP(time_column)/(300) as signed)*300 as signed)") - }) - - Convey("interpolate __timeGroup function with spaces around arguments", func() { - - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "GROUP BY cast(cast(UNIX_TIMESTAMP(time_column)/(300) as signed)*300 as signed)") - }) - - Convey("interpolate __timeTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select FROM_UNIXTIME(18446744066914187038)") - }) - - Convey("interpolate __unixEpochFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(18446744066914186738)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914186738 >= 18446744066914186738 AND 18446744066914186738 <= 18446744066914187038") - }) - - Convey("interpolate __unixEpochFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914186738") - }) - - Convey("interpolate __unixEpochTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914187038") - }) - }) } diff --git a/pkg/tsdb/mysql/mysql.go b/pkg/tsdb/mysql/mysql.go index 483974c55a4..83027d4b210 100644 --- a/pkg/tsdb/mysql/mysql.go +++ b/pkg/tsdb/mysql/mysql.go @@ -8,7 +8,6 @@ import ( "math" "reflect" "strconv" - "time" "github.com/go-sql-driver/mysql" "github.com/go-xorm/core" @@ -239,15 +238,18 @@ func (e MysqlQueryEndpoint) transformToTimeSeries(query *tsdb.Query, rows *core. return err } + // converts column named time to unix timestamp in milliseconds to make + // native mysql datetime types and epoch dates work in + // annotation and table queries. + tsdb.ConvertSqlTimeColumnToEpochMs(values, timeIndex) + switch columnValue := values[timeIndex].(type) { case int64: - timestamp = float64(columnValue * 1000) + timestamp = float64(columnValue) case float64: - timestamp = columnValue * 1000 - case time.Time: - timestamp = float64(columnValue.UnixNano() / 1e6) + timestamp = columnValue default: - return fmt.Errorf("Invalid type for column time, must be of type timestamp or unix timestamp, got: %T %v", columnValue, columnValue) + return fmt.Errorf("Invalid type for column time/time_sec, must be of type timestamp or unix timestamp, got: %T %v", columnValue, columnValue) } if metricIndex >= 0 { diff --git a/pkg/tsdb/mysql/mysql_test.go b/pkg/tsdb/mysql/mysql_test.go index 750704c9965..74cedea803a 100644 --- a/pkg/tsdb/mysql/mysql_test.go +++ b/pkg/tsdb/mysql/mysql_test.go @@ -3,25 +3,36 @@ package mysql import ( "fmt" "math/rand" + "strings" "testing" "time" "github.com/go-xorm/xorm" "github.com/grafana/grafana/pkg/components/simplejson" "github.com/grafana/grafana/pkg/log" + "github.com/grafana/grafana/pkg/services/sqlstore" "github.com/grafana/grafana/pkg/services/sqlstore/sqlutil" "github.com/grafana/grafana/pkg/tsdb" . "github.com/smartystreets/goconvey/convey" ) -// To run this test, remove the Skip from SkipConvey -// and set up a MySQL db named grafana_tests and a user/password grafana/password +// To run this test, set runMySqlTests=true +// Or from the commandline: GRAFANA_TEST_DB=mysql go test -v ./pkg/tsdb/mysql +// The tests require a MySQL db named grafana_ds_tests and a user/password grafana/password // Use the docker/blocks/mysql_tests/docker-compose.yaml to spin up a // preconfigured MySQL server suitable for running these tests. -// Thers's also a dashboard.json in same directory that you can import to Grafana +// There is also a dashboard.json in same directory that you can import to Grafana // once you've created a datasource for the test server/database. func TestMySQL(t *testing.T) { - SkipConvey("MySQL", t, func() { + // change to true to run the MySQL tests + runMySqlTests := false + // runMySqlTests := true + + if !(sqlstore.IsTestDbMySql() || runMySqlTests) { + t.Skip() + } + + Convey("MySQL", t, func() { x := InitMySQLTestDB(t) endpoint := &MysqlQueryEndpoint{ @@ -35,7 +46,7 @@ func TestMySQL(t *testing.T) { sess := x.NewSession() defer sess.Close() - fromStart := time.Date(2018, 3, 15, 13, 0, 0, 0, time.Local) + fromStart := time.Date(2018, 3, 15, 13, 0, 0, 0, time.UTC) Convey("Given a table with different native data types", func() { if exists, err := sess.IsTableExist("mysql_types"); err != nil || exists { @@ -121,9 +132,8 @@ func TestMySQL(t *testing.T) { So(column[7].(float64), ShouldEqual, 1.11) So(column[8].(float64), ShouldEqual, 2.22) So(*column[9].(*float32), ShouldEqual, 3.33) - _, offset := time.Now().Zone() - So(column[10].(time.Time), ShouldHappenWithin, time.Duration(10*time.Second), time.Now().Add(time.Duration(offset)*time.Second)) - So(column[11].(time.Time), ShouldHappenWithin, time.Duration(10*time.Second), time.Now().Add(time.Duration(offset)*time.Second)) + So(column[10].(time.Time), ShouldHappenWithin, time.Duration(10*time.Second), time.Now()) + So(column[11].(time.Time), ShouldHappenWithin, time.Duration(10*time.Second), time.Now()) So(column[12].(string), ShouldEqual, "11:11:11") So(column[13].(int64), ShouldEqual, 2018) So(*column[14].(*[]byte), ShouldHaveSameTypeAs, []byte{1}) @@ -137,8 +147,8 @@ func TestMySQL(t *testing.T) { So(column[22].(string), ShouldEqual, "longblob") So(column[23].(string), ShouldEqual, "val2") So(column[24].(string), ShouldEqual, "a,b") - So(column[25].(time.Time).Format("2006-01-02T00:00:00Z"), ShouldEqual, time.Now().Format("2006-01-02T00:00:00Z")) - So(column[26].(float64), ShouldEqual, float64(1514764861000)) + So(column[25].(time.Time).Format("2006-01-02T00:00:00Z"), ShouldEqual, time.Now().UTC().Format("2006-01-02T00:00:00Z")) + So(column[26].(float64), ShouldEqual, float64(1.514764861123456*1e12)) So(column[27], ShouldEqual, nil) So(column[28], ShouldEqual, nil) So(column[29], ShouldEqual, "") @@ -177,10 +187,8 @@ func TestMySQL(t *testing.T) { }) } - for _, s := range series { - _, err = sess.Insert(s) - So(err, ShouldBeNil) - } + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) Convey("When doing a metric query using timeGroup", func() { query := &tsdb.TsdbQuery{ @@ -301,10 +309,19 @@ func TestMySQL(t *testing.T) { Convey("Given a table with metrics having multiple values and measurements", func() { type metric_values struct { - Time time.Time - Measurement string - ValueOne int64 `xorm:"integer 'valueOne'"` - ValueTwo int64 `xorm:"integer 'valueTwo'"` + Time time.Time `xorm:"datetime 'time' not null"` + TimeNullable *time.Time `xorm:"datetime(6) 'timeNullable' null"` + TimeInt64 int64 `xorm:"bigint(20) 'timeInt64' not null"` + TimeInt64Nullable *int64 `xorm:"bigint(20) 'timeInt64Nullable' null"` + TimeFloat64 float64 `xorm:"double 'timeFloat64' not null"` + TimeFloat64Nullable *float64 `xorm:"double 'timeFloat64Nullable' null"` + TimeInt32 int32 `xorm:"int(11) 'timeInt32' not null"` + TimeInt32Nullable *int32 `xorm:"int(11) 'timeInt32Nullable' null"` + TimeFloat32 float32 `xorm:"double 'timeFloat32' not null"` + TimeFloat32Nullable *float32 `xorm:"double 'timeFloat32Nullable' null"` + Measurement string + ValueOne int64 `xorm:"integer 'valueOne'"` + ValueTwo int64 `xorm:"integer 'valueTwo'"` } if exist, err := sess.IsTableExist(metric_values{}); err != nil || exist { @@ -319,26 +336,265 @@ func TestMySQL(t *testing.T) { return rand.Int63n(max-min) + min } + var tInitial time.Time + series := []*metric_values{} - for _, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric A", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric B", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) + for i, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { + if i == 0 { + tInitial = t + } + tSeconds := t.Unix() + tSecondsInt32 := int32(tSeconds) + tSecondsFloat32 := float32(tSeconds) + tMilliseconds := tSeconds * 1e3 + tMillisecondsFloat := float64(tMilliseconds) + t2 := t + first := metric_values{ + Time: t, + TimeNullable: &t2, + TimeInt64: tMilliseconds, + TimeInt64Nullable: &(tMilliseconds), + TimeFloat64: tMillisecondsFloat, + TimeFloat64Nullable: &tMillisecondsFloat, + TimeInt32: tSecondsInt32, + TimeInt32Nullable: &tSecondsInt32, + TimeFloat32: tSecondsFloat32, + TimeFloat32Nullable: &tSecondsFloat32, + Measurement: "Metric A", + ValueOne: rnd(0, 100), + ValueTwo: rnd(0, 100), + } + second := first + second.Measurement = "Metric B" + second.ValueOne = rnd(0, 100) + second.ValueTwo = rnd(0, 100) + + series = append(series, &first) + series = append(series, &second) } - for _, s := range series { - _, err := sess.Insert(s) + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) + + Convey("When doing a metric query using time as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) So(err, ShouldBeNil) - } + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using time (nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeNullable as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeInt64 as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeInt64Nullable as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeFloat64 as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeFloat64Nullable as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeInt32 as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeInt32Nullable as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeFloat32 as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) + + Convey("When doing a metric query using epoch (float32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT timeFloat32Nullable as time, valueOne FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) Convey("When doing a metric query grouping by time and select metric column should return correct series", func() { query := &tsdb.TsdbQuery{ @@ -647,16 +903,16 @@ func TestMySQL(t *testing.T) { } func InitMySQLTestDB(t *testing.T) *xorm.Engine { - x, err := xorm.NewEngine(sqlutil.TestDB_Mysql.DriverName, sqlutil.TestDB_Mysql.ConnStr+"&parseTime=true") - x.DatabaseTZ = time.Local - x.TZLocation = time.Local - - // x.ShowSQL() - + x, err := xorm.NewEngine(sqlutil.TestDB_Mysql.DriverName, strings.Replace(sqlutil.TestDB_Mysql.ConnStr, "/grafana_tests", "/grafana_ds_tests", 1)) if err != nil { t.Fatalf("Failed to init mysql db %v", err) } + x.DatabaseTZ = time.UTC + x.TZLocation = time.UTC + + // x.ShowSQL() + return x } diff --git a/pkg/tsdb/opentsdb/opentsdb_test.go b/pkg/tsdb/opentsdb/opentsdb_test.go index 094deb9e8ec..fe03599f54d 100644 --- a/pkg/tsdb/opentsdb/opentsdb_test.go +++ b/pkg/tsdb/opentsdb/opentsdb_test.go @@ -35,7 +35,7 @@ func TestOpenTsdbExecutor(t *testing.T) { }) - Convey("Build metric with downsampling diabled", func() { + Convey("Build metric with downsampling disabled", func() { query := &tsdb.Query{ Model: simplejson.New(), diff --git a/pkg/tsdb/postgres/macros.go b/pkg/tsdb/postgres/macros.go index 23daeebec5a..f82997ace11 100644 --- a/pkg/tsdb/postgres/macros.go +++ b/pkg/tsdb/postgres/macros.go @@ -79,15 +79,15 @@ func (m *PostgresMacroEngine) evaluateMacro(name string, args []string) (string, } return fmt.Sprintf("extract(epoch from %s) as \"time\"", args[0]), nil case "__timeFilter": - // dont use to_timestamp in this macro for redshift compatibility #9566 + // don't use to_timestamp in this macro for redshift compatibility #9566 if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("extract(epoch from %s) BETWEEN %d AND %d", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("extract(epoch from %s) BETWEEN %d AND %d", args[0], m.TimeRange.GetFromAsSecondsEpoch(), m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeFrom": - return fmt.Sprintf("to_timestamp(%d)", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("to_timestamp(%d)", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__timeTo": - return fmt.Sprintf("to_timestamp(%d)", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("to_timestamp(%d)", m.TimeRange.GetToAsSecondsEpoch()), nil case "__timeGroup": if len(args) < 2 { return "", fmt.Errorf("macro %v needs time column and interval and optional fill value", name) @@ -114,11 +114,11 @@ func (m *PostgresMacroEngine) evaluateMacro(name string, args []string) (string, if len(args) == 0 { return "", fmt.Errorf("missing time column argument for macro %v", name) } - return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], uint64(m.TimeRange.GetFromAsMsEpoch()/1000), args[0], uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.TimeRange.GetFromAsSecondsEpoch(), args[0], m.TimeRange.GetToAsSecondsEpoch()), nil case "__unixEpochFrom": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetFromAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetFromAsSecondsEpoch()), nil case "__unixEpochTo": - return fmt.Sprintf("%d", uint64(m.TimeRange.GetToAsMsEpoch()/1000)), nil + return fmt.Sprintf("%d", m.TimeRange.GetToAsSecondsEpoch()), nil default: return "", fmt.Errorf("Unknown macro %v", name) } diff --git a/pkg/tsdb/postgres/macros_test.go b/pkg/tsdb/postgres/macros_test.go index b18acced963..f441690a429 100644 --- a/pkg/tsdb/postgres/macros_test.go +++ b/pkg/tsdb/postgres/macros_test.go @@ -1,7 +1,10 @@ package postgres import ( + "fmt" + "strconv" "testing" + "time" "github.com/grafana/grafana/pkg/tsdb" . "github.com/smartystreets/goconvey/convey" @@ -11,79 +14,179 @@ func TestMacroEngine(t *testing.T) { Convey("MacroEngine", t, func() { engine := &PostgresMacroEngine{} query := &tsdb.Query{} - timeRange := &tsdb.TimeRange{From: "5m", To: "now"} - Convey("interpolate __time function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__time(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 2018-04-12 00:00 and 2018-04-12 00:05", func() { + from := time.Date(2018, 4, 12, 18, 0, 0, 0, time.UTC) + to := from.Add(5 * time.Minute) + timeRange := tsdb.NewFakeTimeRange("5m", "now", to) - So(sql, ShouldEqual, "select time_column AS \"time\"") + Convey("interpolate __time function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__time(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select time_column AS \"time\"") + }) + + Convey("interpolate __time function wrapped in aggregation", func() { + sql, err := engine.Interpolate(query, timeRange, "select min($__time(time_column))") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "select min(time_column AS \"time\")") + }) + + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE extract(epoch from time_column) BETWEEN %d AND %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", from.Unix())) + }) + + Convey("interpolate __timeGroup function", func() { + + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY (extract(epoch from time_column)/300)::bigint*300 AS time") + }) + + Convey("interpolate __timeGroup function with spaces between args", func() { + + sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, "GROUP BY (extract(epoch from time_column)/300)::bigint*300 AS time") + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __time function wrapped in aggregation", func() { - sql, err := engine.Interpolate(query, timeRange, "select min($__time(time_column))") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1965-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1965, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "select min(time_column AS \"time\")") + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE extract(epoch from time_column) BETWEEN %d AND %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - Convey("interpolate __timeFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") - So(err, ShouldBeNil) + Convey("Given a time range between 1960-02-01 07:00 and 1980-02-03 08:00", func() { + from := time.Date(1960, 2, 1, 7, 0, 0, 0, time.UTC) + to := time.Date(1980, 2, 3, 8, 0, 0, 0, time.UTC) + timeRange := tsdb.NewTimeRange(strconv.FormatInt(from.UnixNano()/int64(time.Millisecond), 10), strconv.FormatInt(to.UnixNano()/int64(time.Millisecond), 10)) - So(sql, ShouldEqual, "WHERE extract(epoch from time_column) BETWEEN 18446744066914186738 AND 18446744066914187038") + Convey("interpolate __timeFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "WHERE $__timeFilter(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("WHERE extract(epoch from time_column) BETWEEN %d AND %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __timeFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", from.Unix())) + }) + + Convey("interpolate __timeTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select to_timestamp(%d)", to.Unix())) + }) + + Convey("interpolate __unixEpochFilter function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(time)") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix())) + }) + + Convey("interpolate __unixEpochFrom function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", from.Unix())) + }) + + Convey("interpolate __unixEpochTo function", func() { + sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") + So(err, ShouldBeNil) + + So(sql, ShouldEqual, fmt.Sprintf("select %d", to.Unix())) + }) }) - - Convey("interpolate __timeFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeFrom(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select to_timestamp(18446744066914186738)") - }) - - Convey("interpolate __timeGroup function", func() { - - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column,'5m')") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "GROUP BY (extract(epoch from time_column)/300)::bigint*300 AS time") - }) - - Convey("interpolate __timeGroup function with spaces between args", func() { - - sql, err := engine.Interpolate(query, timeRange, "GROUP BY $__timeGroup(time_column , '5m')") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "GROUP BY (extract(epoch from time_column)/300)::bigint*300 AS time") - }) - - Convey("interpolate __timeTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__timeTo(time_column)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select to_timestamp(18446744066914187038)") - }) - - Convey("interpolate __unixEpochFilter function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFilter(18446744066914186738)") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914186738 >= 18446744066914186738 AND 18446744066914186738 <= 18446744066914187038") - }) - - Convey("interpolate __unixEpochFrom function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochFrom()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914186738") - }) - - Convey("interpolate __unixEpochTo function", func() { - sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochTo()") - So(err, ShouldBeNil) - - So(sql, ShouldEqual, "select 18446744066914187038") - }) - }) } diff --git a/pkg/tsdb/postgres/postgres.go b/pkg/tsdb/postgres/postgres.go index 5f6b56ebcf1..e17cc783f38 100644 --- a/pkg/tsdb/postgres/postgres.go +++ b/pkg/tsdb/postgres/postgres.go @@ -7,7 +7,6 @@ import ( "math" "net/url" "strconv" - "time" "github.com/go-xorm/core" "github.com/grafana/grafana/pkg/components/null" @@ -219,13 +218,16 @@ func (e PostgresQueryEndpoint) transformToTimeSeries(query *tsdb.Query, rows *co return err } + // converts column named time to unix timestamp in milliseconds to make + // native mysql datetime types and epoch dates work in + // annotation and table queries. + tsdb.ConvertSqlTimeColumnToEpochMs(values, timeIndex) + switch columnValue := values[timeIndex].(type) { case int64: - timestamp = float64(columnValue * 1000) + timestamp = float64(columnValue) case float64: - timestamp = columnValue * 1000 - case time.Time: - timestamp = float64(columnValue.UnixNano() / 1e6) + timestamp = columnValue default: return fmt.Errorf("Invalid type for column time, must be of type timestamp or unix timestamp, got: %T %v", columnValue, columnValue) } diff --git a/pkg/tsdb/postgres/postgres_test.go b/pkg/tsdb/postgres/postgres_test.go index 3f2203ac7a4..d18251bac7d 100644 --- a/pkg/tsdb/postgres/postgres_test.go +++ b/pkg/tsdb/postgres/postgres_test.go @@ -3,26 +3,37 @@ package postgres import ( "fmt" "math/rand" + "strings" "testing" "time" "github.com/go-xorm/xorm" "github.com/grafana/grafana/pkg/components/simplejson" "github.com/grafana/grafana/pkg/log" + "github.com/grafana/grafana/pkg/services/sqlstore" "github.com/grafana/grafana/pkg/services/sqlstore/sqlutil" "github.com/grafana/grafana/pkg/tsdb" _ "github.com/lib/pq" . "github.com/smartystreets/goconvey/convey" ) -// To run this test, remove the Skip from SkipConvey -// and set up a PostgreSQL db named grafanatest and a user/password grafanatest/grafanatest! +// To run this test, set runPostgresTests=true +// Or from the commandline: GRAFANA_TEST_DB=postgres go test -v ./pkg/tsdb/postgres +// The tests require a PostgreSQL db named grafanadstest and a user/password grafanatest/grafanatest! // Use the docker/blocks/postgres_tests/docker-compose.yaml to spin up a // preconfigured Postgres server suitable for running these tests. -// Thers's also a dashboard.json in same directory that you can import to Grafana +// There is also a dashboard.json in same directory that you can import to Grafana // once you've created a datasource for the test server/database. func TestPostgres(t *testing.T) { - SkipConvey("PostgreSQL", t, func() { + // change to true to run the MySQL tests + runPostgresTests := false + // runPostgresTests := true + + if !(sqlstore.IsTestDbPostgres() || runPostgresTests) { + t.Skip() + } + + Convey("PostgreSQL", t, func() { x := InitPostgresTestDB(t) endpoint := &PostgresQueryEndpoint{ @@ -156,10 +167,8 @@ func TestPostgres(t *testing.T) { }) } - for _, s := range series { - _, err = sess.Insert(s) - So(err, ShouldBeNil) - } + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) Convey("When doing a metric query using timeGroup", func() { query := &tsdb.TsdbQuery{ @@ -280,10 +289,18 @@ func TestPostgres(t *testing.T) { Convey("Given a table with metrics having multiple values and measurements", func() { type metric_values struct { - Time time.Time - Measurement string - ValueOne int64 `xorm:"integer 'valueOne'"` - ValueTwo int64 `xorm:"integer 'valueTwo'"` + Time time.Time + TimeInt64 int64 `xorm:"bigint 'timeInt64' not null"` + TimeInt64Nullable *int64 `xorm:"bigint 'timeInt64Nullable' null"` + TimeFloat64 float64 `xorm:"double 'timeFloat64' not null"` + TimeFloat64Nullable *float64 `xorm:"double 'timeFloat64Nullable' null"` + TimeInt32 int32 `xorm:"int(11) 'timeInt32' not null"` + TimeInt32Nullable *int32 `xorm:"int(11) 'timeInt32Nullable' null"` + TimeFloat32 float32 `xorm:"double 'timeFloat32' not null"` + TimeFloat32Nullable *float32 `xorm:"double 'timeFloat32Nullable' null"` + Measurement string + ValueOne int64 `xorm:"integer 'valueOne'"` + ValueTwo int64 `xorm:"integer 'valueTwo'"` } if exist, err := sess.IsTableExist(metric_values{}); err != nil || exist { @@ -298,26 +315,219 @@ func TestPostgres(t *testing.T) { return rand.Int63n(max-min) + min } + var tInitial time.Time + series := []*metric_values{} - for _, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric A", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) - series = append(series, &metric_values{ - Time: t, - Measurement: "Metric B", - ValueOne: rnd(0, 100), - ValueTwo: rnd(0, 100), - }) + for i, t := range genTimeRangeByInterval(fromStart.Add(-30*time.Minute), 90*time.Minute, 5*time.Minute) { + if i == 0 { + tInitial = t + } + tSeconds := t.Unix() + tSecondsInt32 := int32(tSeconds) + tSecondsFloat32 := float32(tSeconds) + tMilliseconds := tSeconds * 1e3 + tMillisecondsFloat := float64(tMilliseconds) + first := metric_values{ + Time: t, + TimeInt64: tMilliseconds, + TimeInt64Nullable: &(tMilliseconds), + TimeFloat64: tMillisecondsFloat, + TimeFloat64Nullable: &tMillisecondsFloat, + TimeInt32: tSecondsInt32, + TimeInt32Nullable: &tSecondsInt32, + TimeFloat32: tSecondsFloat32, + TimeFloat32Nullable: &tSecondsFloat32, + Measurement: "Metric A", + ValueOne: rnd(0, 100), + ValueTwo: rnd(0, 100), + } + second := first + second.Measurement = "Metric B" + second.ValueOne = rnd(0, 100) + second.ValueTwo = rnd(0, 100) + + series = append(series, &first) + series = append(series, &second) } - for _, s := range series { - _, err := sess.Insert(s) + _, err = sess.InsertMulti(series) + So(err, ShouldBeNil) + + Convey("When doing a metric query using epoch (int64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeInt64" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) So(err, ShouldBeNil) - } + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeInt64Nullable" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeFloat64" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float64 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeFloat64Nullable" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeInt32" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (int32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeInt32Nullable" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(tInitial.UnixNano()/1e6)) + }) + + Convey("When doing a metric query using epoch (float32) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeFloat32" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) + + Convey("When doing a metric query using epoch (float32 nullable) as time column should return metric with time in milliseconds", func() { + query := &tsdb.TsdbQuery{ + Queries: []*tsdb.Query{ + { + Model: simplejson.NewFromAny(map[string]interface{}{ + "rawSql": `SELECT "timeFloat32Nullable" as time, "valueOne" FROM metric_values ORDER BY time LIMIT 1`, + "format": "time_series", + }), + RefId: "A", + }, + }, + } + + resp, err := endpoint.Query(nil, nil, query) + So(err, ShouldBeNil) + queryResult := resp.Results["A"] + So(queryResult.Error, ShouldBeNil) + + So(len(queryResult.Series), ShouldEqual, 1) + So(queryResult.Series[0].Points[0][1].Float64, ShouldEqual, float64(float64(float32(tInitial.Unix())))*1e3) + }) Convey("When doing a metric query grouping by time and select metric column should return correct series", func() { query := &tsdb.TsdbQuery{ @@ -473,7 +683,7 @@ func TestPostgres(t *testing.T) { columns := queryResult.Tables[0].Rows[0] //Should be in milliseconds - So(columns[0].(float64), ShouldEqual, float64(dt.Unix()*1000)) + So(columns[0].(float64), ShouldEqual, float64(dt.UnixNano()/1e6)) }) Convey("When doing an annotation query with a time column in epoch second format should return ms", func() { @@ -626,16 +836,16 @@ func TestPostgres(t *testing.T) { } func InitPostgresTestDB(t *testing.T) *xorm.Engine { - x, err := xorm.NewEngine(sqlutil.TestDB_Postgres.DriverName, sqlutil.TestDB_Postgres.ConnStr) + x, err := xorm.NewEngine(sqlutil.TestDB_Postgres.DriverName, strings.Replace(sqlutil.TestDB_Postgres.ConnStr, "dbname=grafanatest", "dbname=grafanadstest", 1)) + if err != nil { + t.Fatalf("Failed to init postgres db %v", err) + } + x.DatabaseTZ = time.UTC x.TZLocation = time.UTC // x.ShowSQL() - if err != nil { - t.Fatalf("Failed to init postgres db %v", err) - } - return x } diff --git a/pkg/tsdb/sql_engine.go b/pkg/tsdb/sql_engine.go index 16370a4ea7f..0f35cadf4d6 100644 --- a/pkg/tsdb/sql_engine.go +++ b/pkg/tsdb/sql_engine.go @@ -135,16 +135,16 @@ func (e *DefaultSqlEngine) Query( return result, nil } -// ConvertTimeColumnToEpochMs converts column named time to unix timestamp in milliseconds +// ConvertSqlTimeColumnToEpochMs converts column named time to unix timestamp in milliseconds // to make native datetime types and epoch dates work in annotation and table queries. func ConvertSqlTimeColumnToEpochMs(values RowValues, timeIndex int) { if timeIndex >= 0 { switch value := values[timeIndex].(type) { case time.Time: - values[timeIndex] = EpochPrecisionToMs(float64(value.Unix())) + values[timeIndex] = EpochPrecisionToMs(float64(value.UnixNano())) case *time.Time: if value != nil { - values[timeIndex] = EpochPrecisionToMs(float64((*value).Unix())) + values[timeIndex] = EpochPrecisionToMs(float64((*value).UnixNano())) } case int64: values[timeIndex] = int64(EpochPrecisionToMs(float64(value))) @@ -152,12 +152,36 @@ func ConvertSqlTimeColumnToEpochMs(values RowValues, timeIndex int) { if value != nil { values[timeIndex] = int64(EpochPrecisionToMs(float64(*value))) } + case uint64: + values[timeIndex] = int64(EpochPrecisionToMs(float64(value))) + case *uint64: + if value != nil { + values[timeIndex] = int64(EpochPrecisionToMs(float64(*value))) + } + case int32: + values[timeIndex] = int64(EpochPrecisionToMs(float64(value))) + case *int32: + if value != nil { + values[timeIndex] = int64(EpochPrecisionToMs(float64(*value))) + } + case uint32: + values[timeIndex] = int64(EpochPrecisionToMs(float64(value))) + case *uint32: + if value != nil { + values[timeIndex] = int64(EpochPrecisionToMs(float64(*value))) + } case float64: values[timeIndex] = EpochPrecisionToMs(value) case *float64: if value != nil { values[timeIndex] = EpochPrecisionToMs(*value) } + case float32: + values[timeIndex] = EpochPrecisionToMs(float64(value)) + case *float32: + if value != nil { + values[timeIndex] = EpochPrecisionToMs(float64(*value)) + } } } } diff --git a/pkg/tsdb/sql_engine_test.go b/pkg/tsdb/sql_engine_test.go index 48aac2c4d45..4c6951a0196 100644 --- a/pkg/tsdb/sql_engine_test.go +++ b/pkg/tsdb/sql_engine_test.go @@ -1,6 +1,7 @@ package tsdb import ( + "fmt" "testing" "time" @@ -9,37 +10,177 @@ import ( func TestSqlEngine(t *testing.T) { Convey("SqlEngine", t, func() { - Convey("Given row values with time columns when converting them", func() { - dt := time.Date(2018, 3, 14, 21, 20, 6, 527e6, time.UTC) - fixtures := make([]interface{}, 8) - fixtures[0] = dt - fixtures[1] = dt.Unix() * 1000 - fixtures[2] = dt.Unix() - fixtures[3] = float64(dt.Unix() * 1000) - fixtures[4] = float64(dt.Unix()) + dt := time.Date(2018, 3, 14, 21, 20, 6, int(527345*time.Microsecond), time.UTC) - var nilDt *time.Time - var nilInt64 *int64 - var nilFloat64 *float64 - fixtures[5] = nilDt - fixtures[6] = nilInt64 - fixtures[7] = nilFloat64 + Convey("Given row values with time.Time as time columns", func() { + var nilPointer *time.Time + + fixtures := make([]interface{}, 3) + fixtures[0] = dt + fixtures[1] = &dt + fixtures[2] = nilPointer for i := range fixtures { ConvertSqlTimeColumnToEpochMs(fixtures, i) } - Convey("Should convert sql time columns to epoch time in ms ", func() { - expected := float64(dt.Unix() * 1000) + Convey("When converting them should return epoch time with millisecond precision ", func() { + expected := float64(dt.UnixNano()) / float64(time.Millisecond) So(fixtures[0].(float64), ShouldEqual, expected) - So(fixtures[1].(int64), ShouldEqual, expected) - So(fixtures[2].(int64), ShouldEqual, expected) - So(fixtures[3].(float64), ShouldEqual, expected) - So(fixtures[4].(float64), ShouldEqual, expected) + So(fixtures[1].(float64), ShouldEqual, expected) + So(fixtures[2], ShouldBeNil) + }) + }) - So(fixtures[5], ShouldBeNil) + Convey("Given row values with int64 as time columns", func() { + tSeconds := dt.Unix() + tMilliseconds := dt.UnixNano() / 1e6 + tNanoSeconds := dt.UnixNano() + var nilPointer *int64 + + fixtures := make([]interface{}, 7) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = tMilliseconds + fixtures[3] = &tMilliseconds + fixtures[4] = tNanoSeconds + fixtures[5] = &tNanoSeconds + fixtures[6] = nilPointer + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(int64), ShouldEqual, tSeconds*1e3) + So(fixtures[1].(int64), ShouldEqual, tSeconds*1e3) + So(fixtures[2].(int64), ShouldEqual, tMilliseconds) + So(fixtures[3].(int64), ShouldEqual, tMilliseconds) + So(fixtures[4].(int64), ShouldEqual, tMilliseconds) + So(fixtures[5].(int64), ShouldEqual, tMilliseconds) So(fixtures[6], ShouldBeNil) - So(fixtures[7], ShouldBeNil) + }) + }) + + Convey("Given row values with uin64 as time columns", func() { + tSeconds := uint64(dt.Unix()) + tMilliseconds := uint64(dt.UnixNano() / 1e6) + tNanoSeconds := uint64(dt.UnixNano()) + var nilPointer *uint64 + + fixtures := make([]interface{}, 7) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = tMilliseconds + fixtures[3] = &tMilliseconds + fixtures[4] = tNanoSeconds + fixtures[5] = &tNanoSeconds + fixtures[6] = nilPointer + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(int64), ShouldEqual, tSeconds*1e3) + So(fixtures[1].(int64), ShouldEqual, tSeconds*1e3) + So(fixtures[2].(int64), ShouldEqual, tMilliseconds) + So(fixtures[3].(int64), ShouldEqual, tMilliseconds) + So(fixtures[4].(int64), ShouldEqual, tMilliseconds) + So(fixtures[5].(int64), ShouldEqual, tMilliseconds) + So(fixtures[6], ShouldBeNil) + }) + }) + + Convey("Given row values with int32 as time columns", func() { + tSeconds := int32(dt.Unix()) + var nilInt *int32 + + fixtures := make([]interface{}, 3) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = nilInt + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(int64), ShouldEqual, dt.Unix()*1e3) + So(fixtures[1].(int64), ShouldEqual, dt.Unix()*1e3) + So(fixtures[2], ShouldBeNil) + }) + }) + + Convey("Given row values with uint32 as time columns", func() { + tSeconds := uint32(dt.Unix()) + var nilInt *uint32 + + fixtures := make([]interface{}, 3) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = nilInt + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(int64), ShouldEqual, dt.Unix()*1e3) + So(fixtures[1].(int64), ShouldEqual, dt.Unix()*1e3) + So(fixtures[2], ShouldBeNil) + }) + }) + + Convey("Given row values with float64 as time columns", func() { + tSeconds := float64(dt.UnixNano()) / float64(time.Second) + tMilliseconds := float64(dt.UnixNano()) / float64(time.Millisecond) + tNanoSeconds := float64(dt.UnixNano()) + var nilPointer *float64 + + fixtures := make([]interface{}, 7) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = tMilliseconds + fixtures[3] = &tMilliseconds + fixtures[4] = tNanoSeconds + fixtures[5] = &tNanoSeconds + fixtures[6] = nilPointer + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(float64), ShouldEqual, tMilliseconds) + So(fixtures[1].(float64), ShouldEqual, tMilliseconds) + So(fixtures[2].(float64), ShouldEqual, tMilliseconds) + So(fixtures[3].(float64), ShouldEqual, tMilliseconds) + fmt.Println(fixtures[4].(float64)) + fmt.Println(tMilliseconds) + So(fixtures[4].(float64), ShouldEqual, tMilliseconds) + So(fixtures[5].(float64), ShouldEqual, tMilliseconds) + So(fixtures[6], ShouldBeNil) + }) + }) + + Convey("Given row values with float32 as time columns", func() { + tSeconds := float32(dt.Unix()) + var nilInt *float32 + + fixtures := make([]interface{}, 3) + fixtures[0] = tSeconds + fixtures[1] = &tSeconds + fixtures[2] = nilInt + + for i := range fixtures { + ConvertSqlTimeColumnToEpochMs(fixtures, i) + } + + Convey("When converting them should return epoch time with millisecond precision ", func() { + So(fixtures[0].(float64), ShouldEqual, float32(dt.Unix()*1e3)) + So(fixtures[1].(float64), ShouldEqual, float32(dt.Unix()*1e3)) + So(fixtures[2], ShouldBeNil) }) }) }) diff --git a/pkg/tsdb/time_range.go b/pkg/tsdb/time_range.go index fd0cb3f8e82..3bd4e228999 100644 --- a/pkg/tsdb/time_range.go +++ b/pkg/tsdb/time_range.go @@ -15,6 +15,14 @@ func NewTimeRange(from, to string) *TimeRange { } } +func NewFakeTimeRange(from, to string, now time.Time) *TimeRange { + return &TimeRange{ + From: from, + To: to, + now: now, + } +} + type TimeRange struct { From string To string @@ -25,10 +33,18 @@ func (tr *TimeRange) GetFromAsMsEpoch() int64 { return tr.MustGetFrom().UnixNano() / int64(time.Millisecond) } +func (tr *TimeRange) GetFromAsSecondsEpoch() int64 { + return tr.GetFromAsMsEpoch() / 1000 +} + func (tr *TimeRange) GetToAsMsEpoch() int64 { return tr.MustGetTo().UnixNano() / int64(time.Millisecond) } +func (tr *TimeRange) GetToAsSecondsEpoch() int64 { + return tr.GetToAsMsEpoch() / 1000 +} + func (tr *TimeRange) MustGetFrom() time.Time { if res, err := tr.ParseFrom(); err != nil { return time.Unix(0, 0) @@ -92,9 +108,14 @@ func (tr *TimeRange) ParseTo() (time.Time, error) { // EpochPrecisionToMs converts epoch precision to millisecond, if needed. // Only seconds to milliseconds supported right now func EpochPrecisionToMs(value float64) float64 { - if int64(value)/1e10 == 0 { - return float64(value * 1e3) + s := strconv.FormatFloat(value, 'e', -1, 64) + if strings.HasSuffix(s, "e+09") { + return value * float64(1e3) } - return float64(value) + if strings.HasSuffix(s, "e+18") { + return value / float64(time.Millisecond) + } + + return value } diff --git a/public/app/core/components/grafana_app.ts b/public/app/core/components/grafana_app.ts index 1e3f0cb9119..4f4b3a64fa5 100644 --- a/public/app/core/components/grafana_app.ts +++ b/public/app/core/components/grafana_app.ts @@ -117,6 +117,14 @@ export function grafanaAppDirective(playlistSrv, contextSrv, $timeout, $rootScop appEvents.emit('toggle-kiosk-mode'); } + // check for 'inactive' url param for clean looks like kiosk, but with title + if (data.params.inactive) { + body.addClass('user-activity-low'); + + // for some reason, with this class it looks cleanest + body.addClass('sidemenu-open'); + } + // close all drops for (let drop of Drop.drops) { drop.destroy(); diff --git a/public/app/core/components/json_explorer/helpers.ts b/public/app/core/components/json_explorer/helpers.ts index 5b053792d73..c445e1b0667 100644 --- a/public/app/core/components/json_explorer/helpers.ts +++ b/public/app/core/components/json_explorer/helpers.ts @@ -2,7 +2,7 @@ // Licence MIT, Copyright (c) 2015 Mohsen Azimi /* - * Escapes `"` charachters from string + * Escapes `"` characters from string */ function escapeString(str: string): string { return str.replace('"', '"'); @@ -100,7 +100,7 @@ export function cssClass(className: string): string { } /* - * Creates a new DOM element wiht given type and class + * Creates a new DOM element with given type and class * TODO: move me to helpers */ export function createElement(type: string, className?: string, content?: Element | string): Element { diff --git a/public/app/core/components/json_explorer/json_explorer.ts b/public/app/core/components/json_explorer/json_explorer.ts index 9cc1b53bc82..790ed442d5c 100644 --- a/public/app/core/components/json_explorer/json_explorer.ts +++ b/public/app/core/components/json_explorer/json_explorer.ts @@ -146,7 +146,7 @@ export class JsonExplorer { } /* - * did we recieve a key argument? + * did we receive a key argument? * This means that the formatter was called as a sub formatter of a parent formatter */ private get hasKey(): boolean { diff --git a/public/app/core/directives/dropdown_typeahead.js b/public/app/core/directives/dropdown_typeahead.js index 25772b4638a..9b677c95697 100644 --- a/public/app/core/directives/dropdown_typeahead.js +++ b/public/app/core/directives/dropdown_typeahead.js @@ -108,7 +108,7 @@ function (_, $, coreModule) { $input.val(''); $button.show(); $button.focus(); - // clicking the function dropdown menu wont + // clicking the function dropdown menu won't // work if you remove class at once setTimeout(function() { elem.removeClass('open'); @@ -222,7 +222,7 @@ function (_, $, coreModule) { $input.val(''); $button.show(); $button.focus(); - // clicking the function dropdown menu wont + // clicking the function dropdown menu won't // work if you remove class at once setTimeout(function() { elem.removeClass('open'); diff --git a/public/app/core/directives/value_select_dropdown.js b/public/app/core/directives/value_select_dropdown.js deleted file mode 100644 index a2bae1c34d3..00000000000 --- a/public/app/core/directives/value_select_dropdown.js +++ /dev/null @@ -1,283 +0,0 @@ -define([ - 'angular', - 'lodash', - '../core_module', -], -function (angular, _, coreModule) { - 'use strict'; - - coreModule.default.controller('ValueSelectDropdownCtrl', function($q) { - var vm = this; - - vm.show = function() { - vm.oldVariableText = vm.variable.current.text; - vm.highlightIndex = -1; - - vm.options = vm.variable.options; - vm.selectedValues = _.filter(vm.options, {selected: true}); - - vm.tags = _.map(vm.variable.tags, function(value) { - var tag = { text: value, selected: false }; - _.each(vm.variable.current.tags, function(tagObj) { - if (tagObj.text === value) { - tag = tagObj; - } - }); - return tag; - }); - - vm.search = { - query: '', - options: vm.options.slice(0, Math.min(vm.options.length, 1000)) - }; - - vm.dropdownVisible = true; - }; - - vm.updateLinkText = function() { - var current = vm.variable.current; - - if (current.tags && current.tags.length) { - // filer out values that are in selected tags - var selectedAndNotInTag = _.filter(vm.variable.options, function(option) { - if (!option.selected) { return false; } - for (var i = 0; i < current.tags.length; i++) { - var tag = current.tags[i]; - if (_.indexOf(tag.values, option.value) !== -1) { - return false; - } - } - return true; - }); - - // convert values to text - var currentTexts = _.map(selectedAndNotInTag, 'text'); - - // join texts - vm.linkText = currentTexts.join(' + '); - if (vm.linkText.length > 0) { - vm.linkText += ' + '; - } - } else { - vm.linkText = vm.variable.current.text; - } - }; - - vm.clearSelections = function() { - _.each(vm.options, function(option) { - option.selected = false; - }); - - vm.selectionsChanged(false); - }; - - vm.selectTag = function(tag) { - tag.selected = !tag.selected; - var tagValuesPromise; - if (!tag.values) { - tagValuesPromise = vm.variable.getValuesForTag(tag.text); - } else { - tagValuesPromise = $q.when(tag.values); - } - - tagValuesPromise.then(function(values) { - tag.values = values; - tag.valuesText = values.join(' + '); - _.each(vm.options, function(option) { - if (_.indexOf(tag.values, option.value) !== -1) { - option.selected = tag.selected; - } - }); - - vm.selectionsChanged(false); - }); - }; - - vm.keyDown = function (evt) { - if (evt.keyCode === 27) { - vm.hide(); - } - if (evt.keyCode === 40) { - vm.moveHighlight(1); - } - if (evt.keyCode === 38) { - vm.moveHighlight(-1); - } - if (evt.keyCode === 13) { - if (vm.search.options.length === 0) { - vm.commitChanges(); - } else { - vm.selectValue(vm.search.options[vm.highlightIndex], {}, true, false); - } - } - if (evt.keyCode === 32) { - vm.selectValue(vm.search.options[vm.highlightIndex], {}, false, false); - } - }; - - vm.moveHighlight = function(direction) { - vm.highlightIndex = (vm.highlightIndex + direction) % vm.search.options.length; - }; - - vm.selectValue = function(option, event, commitChange, excludeOthers) { - if (!option) { return; } - - option.selected = vm.variable.multi ? !option.selected: true; - - commitChange = commitChange || false; - excludeOthers = excludeOthers || false; - - var setAllExceptCurrentTo = function(newValue) { - _.each(vm.options, function(other) { - if (option !== other) { other.selected = newValue; } - }); - }; - - // commit action (enter key), should not deselect it - if (commitChange) { - option.selected = true; - } - - if (option.text === 'All' || excludeOthers) { - setAllExceptCurrentTo(false); - commitChange = true; - } - else if (!vm.variable.multi) { - setAllExceptCurrentTo(false); - commitChange = true; - } else if (event.ctrlKey || event.metaKey || event.shiftKey) { - commitChange = true; - setAllExceptCurrentTo(false); - } - - vm.selectionsChanged(commitChange); - }; - - vm.selectionsChanged = function(commitChange) { - vm.selectedValues = _.filter(vm.options, {selected: true}); - - if (vm.selectedValues.length > 1) { - if (vm.selectedValues[0].text === 'All') { - vm.selectedValues[0].selected = false; - vm.selectedValues = vm.selectedValues.slice(1, vm.selectedValues.length); - } - } - - // validate selected tags - _.each(vm.tags, function(tag) { - if (tag.selected) { - _.each(tag.values, function(value) { - if (!_.find(vm.selectedValues, {value: value})) { - tag.selected = false; - } - }); - } - }); - - vm.selectedTags = _.filter(vm.tags, {selected: true}); - vm.variable.current.value = _.map(vm.selectedValues, 'value'); - vm.variable.current.text = _.map(vm.selectedValues, 'text').join(' + '); - vm.variable.current.tags = vm.selectedTags; - - if (!vm.variable.multi) { - vm.variable.current.value = vm.selectedValues[0].value; - } - - if (commitChange) { - vm.commitChanges(); - } - }; - - vm.commitChanges = function() { - // if we have a search query and no options use that - if (vm.search.options.length === 0 && vm.search.query.length > 0) { - vm.variable.current = {text: vm.search.query, value: vm.search.query}; - } - else if (vm.selectedValues.length === 0) { - // make sure one option is selected - vm.options[0].selected = true; - vm.selectionsChanged(false); - } - - vm.dropdownVisible = false; - vm.updateLinkText(); - - if (vm.variable.current.text !== vm.oldVariableText) { - vm.onUpdated(); - } - }; - - vm.queryChanged = function() { - vm.highlightIndex = -1; - vm.search.options = _.filter(vm.options, function(option) { - return option.text.toLowerCase().indexOf(vm.search.query.toLowerCase()) !== -1; - }); - - vm.search.options = vm.search.options.slice(0, Math.min(vm.search.options.length, 1000)); - }; - - vm.init = function() { - vm.selectedTags = vm.variable.current.tags || []; - vm.updateLinkText(); - }; - - }); - - coreModule.default.directive('valueSelectDropdown', function($compile, $window, $timeout, $rootScope) { - return { - scope: { variable: "=", onUpdated: "&"}, - templateUrl: 'public/app/partials/valueSelectDropdown.html', - controller: 'ValueSelectDropdownCtrl', - controllerAs: 'vm', - bindToController: true, - link: function(scope, elem) { - var bodyEl = angular.element($window.document.body); - var linkEl = elem.find('.variable-value-link'); - var inputEl = elem.find('input'); - - function openDropdown() { - inputEl.css('width', Math.max(linkEl.width(), 80) + 'px'); - - inputEl.show(); - linkEl.hide(); - - inputEl.focus(); - $timeout(function() { bodyEl.on('click', bodyOnClick); }, 0, false); - } - - function switchToLink() { - inputEl.hide(); - linkEl.show(); - bodyEl.off('click', bodyOnClick); - } - - function bodyOnClick (e) { - if (elem.has(e.target).length === 0) { - scope.$apply(function() { - scope.vm.commitChanges(); - }); - } - } - - scope.$watch('vm.dropdownVisible', function(newValue) { - if (newValue) { - openDropdown(); - } else { - switchToLink(); - } - }); - - var cleanUp = $rootScope.$on('template-variable-value-updated', function() { - scope.vm.updateLinkText(); - }); - - scope.$on("$destroy", function() { - cleanUp(); - }); - - scope.vm.init(); - }, - }; - }); - -}); diff --git a/public/app/core/directives/value_select_dropdown.ts b/public/app/core/directives/value_select_dropdown.ts new file mode 100644 index 00000000000..2b8d5de5ad8 --- /dev/null +++ b/public/app/core/directives/value_select_dropdown.ts @@ -0,0 +1,305 @@ +import angular from 'angular'; +import _ from 'lodash'; +import coreModule from '../core_module'; + +export class ValueSelectDropdownCtrl { + dropdownVisible: any; + highlightIndex: any; + linkText: any; + oldVariableText: any; + options: any; + search: any; + selectedTags: any; + selectedValues: any; + tags: any; + variable: any; + + hide: any; + onUpdated: any; + + /** @ngInject */ + constructor(private $q) {} + + show() { + this.oldVariableText = this.variable.current.text; + this.highlightIndex = -1; + + this.options = this.variable.options; + this.selectedValues = _.filter(this.options, { selected: true }); + + this.tags = _.map(this.variable.tags, value => { + let tag = { text: value, selected: false }; + _.each(this.variable.current.tags, tagObj => { + if (tagObj.text === value) { + tag = tagObj; + } + }); + return tag; + }); + + this.search = { + query: '', + options: this.options.slice(0, Math.min(this.options.length, 1000)), + }; + + this.dropdownVisible = true; + } + + updateLinkText() { + let current = this.variable.current; + + if (current.tags && current.tags.length) { + // filer out values that are in selected tags + let selectedAndNotInTag = _.filter(this.variable.options, option => { + if (!option.selected) { + return false; + } + for (let i = 0; i < current.tags.length; i++) { + let tag = current.tags[i]; + if (_.indexOf(tag.values, option.value) !== -1) { + return false; + } + } + return true; + }); + + // convert values to text + let currentTexts = _.map(selectedAndNotInTag, 'text'); + + // join texts + this.linkText = currentTexts.join(' + '); + if (this.linkText.length > 0) { + this.linkText += ' + '; + } + } else { + this.linkText = this.variable.current.text; + } + } + + clearSelections() { + _.each(this.options, option => { + option.selected = false; + }); + + this.selectionsChanged(false); + } + + selectTag(tag) { + tag.selected = !tag.selected; + let tagValuesPromise; + if (!tag.values) { + tagValuesPromise = this.variable.getValuesForTag(tag.text); + } else { + tagValuesPromise = this.$q.when(tag.values); + } + + tagValuesPromise.then(values => { + tag.values = values; + tag.valuesText = values.join(' + '); + _.each(this.options, option => { + if (_.indexOf(tag.values, option.value) !== -1) { + option.selected = tag.selected; + } + }); + + this.selectionsChanged(false); + }); + } + + keyDown(evt) { + if (evt.keyCode === 27) { + this.hide(); + } + if (evt.keyCode === 40) { + this.moveHighlight(1); + } + if (evt.keyCode === 38) { + this.moveHighlight(-1); + } + if (evt.keyCode === 13) { + if (this.search.options.length === 0) { + this.commitChanges(); + } else { + this.selectValue(this.search.options[this.highlightIndex], {}, true, false); + } + } + if (evt.keyCode === 32) { + this.selectValue(this.search.options[this.highlightIndex], {}, false, false); + } + } + + moveHighlight(direction) { + this.highlightIndex = (this.highlightIndex + direction) % this.search.options.length; + } + + selectValue(option, event, commitChange, excludeOthers) { + if (!option) { + return; + } + + option.selected = this.variable.multi ? !option.selected : true; + + commitChange = commitChange || false; + excludeOthers = excludeOthers || false; + + let setAllExceptCurrentTo = function(newValue) { + _.each(this.options, other => { + if (option !== other) { + other.selected = newValue; + } + }); + }; + + // commit action (enter key), should not deselect it + if (commitChange) { + option.selected = true; + } + + if (option.text === 'All' || excludeOthers) { + setAllExceptCurrentTo(false); + commitChange = true; + } else if (!this.variable.multi) { + setAllExceptCurrentTo(false); + commitChange = true; + } else if (event.ctrlKey || event.metaKey || event.shiftKey) { + commitChange = true; + setAllExceptCurrentTo(false); + } + + this.selectionsChanged(commitChange); + } + + selectionsChanged(commitChange) { + this.selectedValues = _.filter(this.options, { selected: true }); + + if (this.selectedValues.length > 1) { + if (this.selectedValues[0].text === 'All') { + this.selectedValues[0].selected = false; + this.selectedValues = this.selectedValues.slice(1, this.selectedValues.length); + } + } + + // validate selected tags + _.each(this.tags, tag => { + if (tag.selected) { + _.each(tag.values, value => { + if (!_.find(this.selectedValues, { value: value })) { + tag.selected = false; + } + }); + } + }); + + this.selectedTags = _.filter(this.tags, { selected: true }); + this.variable.current.value = _.map(this.selectedValues, 'value'); + this.variable.current.text = _.map(this.selectedValues, 'text').join(' + '); + this.variable.current.tags = this.selectedTags; + + if (!this.variable.multi) { + this.variable.current.value = this.selectedValues[0].value; + } + + if (commitChange) { + this.commitChanges(); + } + } + + commitChanges() { + // if we have a search query and no options use that + if (this.search.options.length === 0 && this.search.query.length > 0) { + this.variable.current = { text: this.search.query, value: this.search.query }; + } else if (this.selectedValues.length === 0) { + // make sure one option is selected + this.options[0].selected = true; + this.selectionsChanged(false); + } + + this.dropdownVisible = false; + this.updateLinkText(); + + if (this.variable.current.text !== this.oldVariableText) { + this.onUpdated(); + } + } + + queryChanged() { + this.highlightIndex = -1; + this.search.options = _.filter(this.options, option => { + return option.text.toLowerCase().indexOf(this.search.query.toLowerCase()) !== -1; + }); + + this.search.options = this.search.options.slice(0, Math.min(this.search.options.length, 1000)); + } + + init() { + this.selectedTags = this.variable.current.tags || []; + this.updateLinkText(); + } +} + +/** @ngInject */ +export function valueSelectDropdown($compile, $window, $timeout, $rootScope) { + return { + scope: { variable: '=', onUpdated: '&' }, + templateUrl: 'public/app/partials/valueSelectDropdown.html', + controller: 'ValueSelectDropdownCtrl', + controllerAs: 'vm', + bindToController: true, + link: function(scope, elem) { + let bodyEl = angular.element($window.document.body); + let linkEl = elem.find('.variable-value-link'); + let inputEl = elem.find('input'); + + function openDropdown() { + inputEl.css('width', Math.max(linkEl.width(), 80) + 'px'); + + inputEl.show(); + linkEl.hide(); + + inputEl.focus(); + $timeout( + function() { + bodyEl.on('click', bodyOnClick); + }, + 0, + false + ); + } + + function switchToLink() { + inputEl.hide(); + linkEl.show(); + bodyEl.off('click', bodyOnClick); + } + + function bodyOnClick(e) { + if (elem.has(e.target).length === 0) { + scope.$apply(function() { + scope.vm.commitChanges(); + }); + } + } + + scope.$watch('vm.dropdownVisible', newValue => { + if (newValue) { + openDropdown(); + } else { + switchToLink(); + } + }); + + let cleanUp = $rootScope.$on('template-variable-value-updated', () => { + scope.vm.updateLinkText(); + }); + + scope.$on('$destroy', () => { + cleanUp(); + }); + + scope.vm.init(); + }, + }; +} + +coreModule.controller('ValueSelectDropdownCtrl', ValueSelectDropdownCtrl); +coreModule.directive('valueSelectDropdown', valueSelectDropdown); diff --git a/public/app/core/utils/css_loader.ts b/public/app/core/utils/css_loader.ts index 42f59a9c27c..ba8623df842 100644 --- a/public/app/core/utils/css_loader.ts +++ b/public/app/core/utils/css_loader.ts @@ -67,7 +67,7 @@ export function fetch(load): any { return ''; } - // dont reload styles loaded in the head + // don't reload styles loaded in the head for (var i = 0; i < linkHrefs.length; i++) { if (load.address === linkHrefs[i]) { return ''; diff --git a/public/app/core/utils/kbn.ts b/public/app/core/utils/kbn.ts index dcb04a3e38e..8c3e3e72dda 100644 --- a/public/app/core/utils/kbn.ts +++ b/public/app/core/utils/kbn.ts @@ -620,13 +620,13 @@ kbn.valueFormats.ms = function(size, decimals, scaledDecimals) { // Less than 1 min return kbn.toFixedScaled(size / 1000, decimals, scaledDecimals, 3, ' s'); } else if (Math.abs(size) < 3600000) { - // Less than 1 hour, devide in minutes + // Less than 1 hour, divide in minutes return kbn.toFixedScaled(size / 60000, decimals, scaledDecimals, 5, ' min'); } else if (Math.abs(size) < 86400000) { - // Less than one day, devide in hours + // Less than one day, divide in hours return kbn.toFixedScaled(size / 3600000, decimals, scaledDecimals, 7, ' hour'); } else if (Math.abs(size) < 31536000000) { - // Less than one year, devide in days + // Less than one year, divide in days return kbn.toFixedScaled(size / 86400000, decimals, scaledDecimals, 8, ' day'); } @@ -638,15 +638,15 @@ kbn.valueFormats.s = function(size, decimals, scaledDecimals) { return ''; } - // Less than 1 µs, devide in ns + // Less than 1 µs, divide in ns if (Math.abs(size) < 0.000001) { return kbn.toFixedScaled(size * 1e9, decimals, scaledDecimals - decimals, -9, ' ns'); } - // Less than 1 ms, devide in µs + // Less than 1 ms, divide in µs if (Math.abs(size) < 0.001) { return kbn.toFixedScaled(size * 1e6, decimals, scaledDecimals - decimals, -6, ' µs'); } - // Less than 1 second, devide in ms + // Less than 1 second, divide in ms if (Math.abs(size) < 1) { return kbn.toFixedScaled(size * 1e3, decimals, scaledDecimals - decimals, -3, ' ms'); } @@ -654,16 +654,16 @@ kbn.valueFormats.s = function(size, decimals, scaledDecimals) { if (Math.abs(size) < 60) { return kbn.toFixed(size, decimals) + ' s'; } else if (Math.abs(size) < 3600) { - // Less than 1 hour, devide in minutes + // Less than 1 hour, divide in minutes return kbn.toFixedScaled(size / 60, decimals, scaledDecimals, 1, ' min'); } else if (Math.abs(size) < 86400) { - // Less than one day, devide in hours + // Less than one day, divide in hours return kbn.toFixedScaled(size / 3600, decimals, scaledDecimals, 4, ' hour'); } else if (Math.abs(size) < 604800) { - // Less than one week, devide in days + // Less than one week, divide in days return kbn.toFixedScaled(size / 86400, decimals, scaledDecimals, 5, ' day'); } else if (Math.abs(size) < 31536000) { - // Less than one year, devide in week + // Less than one year, divide in week return kbn.toFixedScaled(size / 604800, decimals, scaledDecimals, 6, ' week'); } diff --git a/public/app/features/alerting/alert_def.ts b/public/app/features/alerting/alert_def.ts index d86461780ba..797a67abfd8 100644 --- a/public/app/features/alerting/alert_def.ts +++ b/public/app/features/alerting/alert_def.ts @@ -124,7 +124,7 @@ function joinEvalMatches(matches, separator: string) { } function getAlertAnnotationInfo(ah) { - // backward compatability, can be removed in grafana 5.x + // backward compatibility, can be removed in grafana 5.x // old way stored evalMatches in data property directly, // new way stores it in evalMatches property on new data object diff --git a/public/app/features/alerting/specs/threshold_mapper_specs.ts b/public/app/features/alerting/specs/threshold_mapper_specs.ts index 3b284776b8d..1d68fce7050 100644 --- a/public/app/features/alerting/specs/threshold_mapper_specs.ts +++ b/public/app/features/alerting/specs/threshold_mapper_specs.ts @@ -4,7 +4,7 @@ import { ThresholdMapper } from '../threshold_mapper'; describe('ThresholdMapper', () => { describe('with greater than evaluator', () => { - it('can mapp query conditions to thresholds', () => { + it('can map query conditions to thresholds', () => { var panel: any = { type: 'graph', alert: { @@ -25,7 +25,7 @@ describe('ThresholdMapper', () => { }); describe('with outside range evaluator', () => { - it('can mapp query conditions to thresholds', () => { + it('can map query conditions to thresholds', () => { var panel: any = { type: 'graph', alert: { @@ -49,7 +49,7 @@ describe('ThresholdMapper', () => { }); describe('with inside range evaluator', () => { - it('can mapp query conditions to thresholds', () => { + it('can map query conditions to thresholds', () => { var panel: any = { type: 'graph', alert: { diff --git a/public/app/features/annotations/events_processing.ts b/public/app/features/annotations/events_processing.ts index 040bf6425c1..667285d7d43 100644 --- a/public/app/features/annotations/events_processing.ts +++ b/public/app/features/annotations/events_processing.ts @@ -56,7 +56,7 @@ function isStartOfRegion(event): boolean { export function dedupAnnotations(annotations) { let dedup = []; - // Split events by annotationId property existance + // Split events by annotationId property existence let events = _.partition(annotations, 'id'); let eventsById = _.groupBy(events[0], 'id'); diff --git a/public/app/features/dashboard/dashboard_model.ts b/public/app/features/dashboard/dashboard_model.ts index 3fa8ed9973a..9130cb7e806 100644 --- a/public/app/features/dashboard/dashboard_model.ts +++ b/public/app/features/dashboard/dashboard_model.ts @@ -129,7 +129,7 @@ export class DashboardModel { this.meta = meta; } - // cleans meta data and other non peristent state + // cleans meta data and other non persistent state getSaveModelClone() { // make clone var copy: any = {}; @@ -606,7 +606,7 @@ export class DashboardModel { if (panel.gridPos.x + panel.gridPos.w * 2 <= GRID_COLUMN_COUNT) { newPanel.gridPos.x += panel.gridPos.w; } else { - // add bellow + // add below newPanel.gridPos.y += panel.gridPos.h; } diff --git a/public/app/features/dashboard/history/history.ts b/public/app/features/dashboard/history/history.ts index d9f0c087438..be6ad5af1ba 100644 --- a/public/app/features/dashboard/history/history.ts +++ b/public/app/features/dashboard/history/history.ts @@ -133,7 +133,7 @@ export class HistoryListCtrl { return this.historySrv .getHistoryList(this.dashboard, options) .then(revisions => { - // set formated dates & default values + // set formatted dates & default values for (let rev of revisions) { rev.createdDateString = this.formatDate(rev.created); rev.ageString = this.formatBasicDate(rev.created); diff --git a/public/app/features/dashboard/specs/dashboard_import_ctrl.jest.ts b/public/app/features/dashboard/specs/dashboard_import_ctrl.jest.ts index 1cb59ef5bac..737eb360461 100644 --- a/public/app/features/dashboard/specs/dashboard_import_ctrl.jest.ts +++ b/public/app/features/dashboard/specs/dashboard_import_ctrl.jest.ts @@ -56,7 +56,7 @@ describe('DashboardImportCtrl', function() { }); }); - describe('when specifing grafana.com url', function() { + describe('when specifying grafana.com url', function() { beforeEach(function() { ctx.ctrl.gnetUrl = 'http://grafana.com/dashboards/123'; // setup api mock @@ -73,7 +73,7 @@ describe('DashboardImportCtrl', function() { }); }); - describe('when specifing dashbord id', function() { + describe('when specifying dashboard id', function() { beforeEach(function() { ctx.ctrl.gnetUrl = '2342'; // setup api mock diff --git a/public/app/features/dashboard/specs/time_srv_specs.ts b/public/app/features/dashboard/specs/time_srv_specs.ts index ca75f0ffcf9..6e180679ff2 100644 --- a/public/app/features/dashboard/specs/time_srv_specs.ts +++ b/public/app/features/dashboard/specs/time_srv_specs.ts @@ -44,7 +44,7 @@ describe('timeSrv', function() { expect(time.raw.to).to.be('now'); }); - it('should handle formated dates', function() { + it('should handle formatted dates', function() { ctx.$location.search({ from: '20140410T052010', to: '20140520T031022' }); ctx.service.init(_dashboard); var time = ctx.service.timeRange(true); @@ -52,7 +52,7 @@ describe('timeSrv', function() { expect(time.to.valueOf()).to.equal(new Date('2014-05-20T03:10:22Z').getTime()); }); - it('should handle formated dates without time', function() { + it('should handle formatted dates without time', function() { ctx.$location.search({ from: '20140410', to: '20140520' }); ctx.service.init(_dashboard); var time = ctx.service.timeRange(true); diff --git a/public/app/features/dashboard/view_state_srv.ts b/public/app/features/dashboard/view_state_srv.ts index fa471b89989..1ed2d61df71 100644 --- a/public/app/features/dashboard/view_state_srv.ts +++ b/public/app/features/dashboard/view_state_srv.ts @@ -38,7 +38,7 @@ export class DashboardViewState { }); // this marks changes to location during this digest cycle as not to add history item - // dont want url changes like adding orgId to add browser history + // don't want url changes like adding orgId to add browser history $location.replace(); this.update(this.getQueryStringState()); } @@ -196,7 +196,7 @@ export class DashboardViewState { this.oldTimeRange = ctrl.range; this.fullscreenPanel = panelScope; - // Firefox doesn't return scrollTop postion properly if 'dash-scroll' is emitted after setViewMode() + // Firefox doesn't return scrollTop position properly if 'dash-scroll' is emitted after setViewMode() this.$scope.appEvent('dash-scroll', { animate: false, pos: 0 }); this.dashboard.setViewMode(ctrl.panel, true, ctrl.editMode); this.$scope.appEvent('panel-fullscreen-enter', { panelId: ctrl.panel.id }); diff --git a/public/app/features/org/partials/newOrg.html b/public/app/features/org/partials/newOrg.html index 424c55d6eb7..9777107c31a 100644 --- a/public/app/features/org/partials/newOrg.html +++ b/public/app/features/org/partials/newOrg.html @@ -5,7 +5,7 @@ New Organization -Each organization contains their own dashboards, data sources and configuration, and cannot be shared between orgs. While users may belong to more than one, mutiple organization are most frequently used in multi-tenant deployments.
+Each organization contains their own dashboards, data sources and configuration, and cannot be shared between orgs. While users may belong to more than one, multiple organization are most frequently used in multi-tenant deployments.