Commit Graph

216 Commits

Author SHA1 Message Date
Ryan McKinley 1caaa56de0 FeatureFlags: Use interface rather than manager (#80000) 2024-01-09 10:38:06 -08:00
ismail simsek 2edcf0edbd InfluxDB: Use database input for SQL configuration instead of metadata (#79579)
* introduce constants file

* update frontend to use database field

* use dbName field instead of metadata

* use secureGrpc

* betterer
2023-12-15 18:41:32 +01:00
ismail simsek 03da1a3f3f InfluxDB: Update SQL language url specification (#79408)
* accept urls with or without port

* remove redundant comment

* remove redundant function
2023-12-13 16:01:01 +01:00
ismail simsek c088d003f2 InfluxDB: Implement InfluxQL json streaming parser (#76934)
* Have the first iteration

* Prepare bench testing

* rename the test files

* Remove unnecessary test file

* Introduce influxqlStreamingParser feature flag

* Apply streaming parser feature flag

* Add new tests

* More tests

* return executedQueryString only in first frame

* add frame meta and config

* Update golden json files

* Support tags/labels

* more tests

* more tests

* Don't change original response_parser.go

* provide context

* create util package

* don't pass the row

* update converter with formatted frameName

* add executedQueryString info only to first frame

* update golden files

* rename

* update test file

* use pointer values

* update testdata

* update parsing

* update converter for null values

* prepare converter for table response

* clean up

* return timeField in fields

* handle no time column responses

* better nil field handling

* refactor the code

* add table tests

* fix config for table

* table response format

* fix value

* if there is no time column set name

* linting

* refactoring

* handle the status code

* add tracing

* Update pkg/tsdb/influxdb/influxql/converter/converter_test.go

Co-authored-by: İnanç Gümüş <m@inanc.io>

* fix import

* update test data

* sanity

* sanity

* linting

* simplicity

* return empty rsp

* rename to prevent confusion

* nullableJson field type for null values

* better handling null values

* remove duplicate test file

* fix healthcheck

* use util for pointer

* move bench test to root

* provide fake feature manager

* add more tests

* partial fix for null values in table response format

* handle partial null fields

* comments for easy testing

* move frameName allocation in readSeries

* one less append operation

* performance improvement by making string conversion once

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
             │ stream2.txt │            stream3.txt             │
             │   sec/op    │   sec/op     vs base               │
ParseJson-10   314.4m ± 1%   303.9m ± 1%  -3.34% (p=0.000 n=10)

             │ stream2.txt  │             stream3.txt              │
             │     B/op     │     B/op      vs base                │
ParseJson-10   425.2Mi ± 0%   382.7Mi ± 0%  -10.00% (p=0.000 n=10)

             │ stream2.txt │            stream3.txt             │
             │  allocs/op  │  allocs/op   vs base               │
ParseJson-10   7.224M ± 0%   6.689M ± 0%  -7.41% (p=0.000 n=10)

* add comment lines

---------

Co-authored-by: İnanç Gümüş <m@inanc.io>
2023-12-06 12:39:05 +01:00
ismail simsek 2b0e7afb4e InfluxDB: Fix an edge case while parsing table response (#78802)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again

* handle nil values

* append in the right array

* add comments

* remove redundant if condition
2023-12-01 18:31:49 +01:00
ismail simsek dff5022021 Chore: Return executedString information even with frames has no time column (#78906)
* return executedString for no time column responses

* remove comment
2023-11-30 14:25:15 +01:00
ismail simsek aea5a9f01a Chore: Introduce util package for InfluxDB backend testing (#78826)
have a util package
2023-11-29 17:48:53 +01:00
ismail simsek 23c7211f1d Chore: InlfuxDB testing update (#78663)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again
2023-11-29 16:19:46 +01:00
ismail simsek c5f3ce1371 InfluxDB: Parse data for table view to have parity with frontend parser (#78365)
* Use TimeSeriesWide format for table response

* fix group by query result parsing

* handle labels

* provide a test where result has no tags

* parsing results without time column

* clean the code

* remove the comment line

* more cleaning

* lint
2023-11-22 19:27:42 +01:00
ismail simsek 5096806eab InfluxDB: Don't sort retention policies on the backend (#78252)
don't sort retention policies
2023-11-16 10:39:15 +02:00
Ben Tasker f38f657f87 InfluxDB: Add new truthiness operators (Is and Is Not) to InfluxQL Query Builder (#77923)
* InfluxDB: Add new truthiness operators (`Is` and `Is Not`) to InfluxQL Editor for use with boolean fields

* InfluxDB: Make the front-end aware of the new operators so that translation between raw and builder works

* Chore: Add tests

* feat: identify type of field value to allow other types to work with Is/Is Not

Tags: always quote
Integer: Don't quote
Float: Don't quote
Boolean: Don't quote
String: Quote

* Chore: Add test-cases for type inference

* Update front-end to infer type for operators Is and Is Not

* chore: Add front-end tests

* chore: add additional front-end tests

* chore: fix failing lint test

* chore: fix tests (run prettier)
2023-11-15 21:10:37 +01:00
Kyle Brandt a94acf4b63 Revert "InfluxDB: Response parser improvements (#76852)" (#78224)
This reverts commit 046791e2be.

Is causing malformed frames, and resulting in errors about mismatched field lengths in the logs, No data in the UI, and will cause SSE to panic.

I think this is because of the global slice vars since it seems to take concurrency to replicate it.
2023-11-15 14:01:36 -05:00
Ben Tasker c06debe200 InfluxDB: Add support for >= and <= comparison operators to IQL Query Builder (#77917)
* InfluxDB: Add support for `>=` and `<=` comparison operators to InfluxQL Query Builder

* Add front-end support for the new operators

This ensures that the query translates correctly between raw and builder mode

* Chore: add test for new operators

* chore: add front-end tests

* fix: don't skip quoting on `<>`

This preserves the pre-existing behaviour, fixing a failing test

* chore: fix tests
2023-11-15 12:21:41 +01:00
ismail simsek 6b13064cf6 InfluxDB: Fix parsing multiple tags on backend mode (#77340)
* Multiple tags separated by comma in the result

* A non-flaky unit test
2023-10-30 15:45:20 +01:00
ismail simsek 4ed36cbc1d InfluxDB: Fix parsing empty response (#77353)
Fix parsing empty response
2023-10-30 11:39:15 +01:00
ismail simsek 046791e2be InfluxDB: Response parser improvements (#76852)
* remove retention policy lookup

* Back to one big function

* %10 less memory allocation

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    1.txt    │                2.txt                │
                │   sec/op    │   sec/op     vs base                │
ParseBigJson-10   540.9m ± 3%   474.0m ± 2%  -12.37% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   580.6Mi ± 0%   573.2Mi ± 0%  -1.28% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │  allocs/op   │  allocs/op   vs base                │
ParseBigJson-10   10.123M ± 0%   9.086M ± 0%  -10.25% (p=0.000 n=10)

* Slightly better results comparing with the previous commit
pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    2.txt    │               3.txt                │
                │   sec/op    │   sec/op     vs base               │
ParseBigJson-10   474.0m ± 1%   503.4m ± 3%  +6.21% (p=0.000 n=10)

                │    2.txt     │                3.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   573.2Mi ± 0%   564.0Mi ± 0%  -1.60% (p=0.000 n=10)

                │    2.txt    │               3.txt                │
                │  allocs/op  │  allocs/op   vs base               │
ParseBigJson-10   9.086M ± 0%   9.052M ± 0%  -0.37% (p=0.000 n=10)

* Split into smaller functions

* Unit test for parseTimestamp
2023-10-27 17:17:19 +02:00
ismail simsek 63abe25b74 InfluxDB: Fix table parsing with backend mode (#76899)
* Add resultFormat to query model

* don't add row name if the result format is table

* No need special formatting since we use unified dataframes

* betterer

* specify visualization type in response

* Unit tests

* fix unit tests
2023-10-23 19:21:30 +02:00
ismail simsek 5eb0b2bedb InfluxDB: Fix aliasing with $measurement or $m on backend mode (#76917)
* better interpolation $measurement aliasing

* unit tests
2023-10-23 17:36:51 +02:00
Will Browne 9d92818cae Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
Dan Cech 91cf4f0c1c Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216.
2023-10-13 12:35:16 -04:00
ismail simsek 2877723648 InfluxDB: Use apache/arrow v13 (#76548)
* Use apache/arrow v13

* remove apache/thrift

* go mod tidy with go1.21.1

* add metrics team as owner

---------

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2023-10-13 17:06:10 +02:00
Will Browne 151f6d6216 Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00
ismail simsek 2d603bed22 InfluxDB SQL: Add time filter automatically (#75953)
* Use double quotes instead of backticks

* render time field as datetime

* add @react-awesome-query-builder/core

* Add time range filtering

* Update test

* Update macro test
2023-10-04 17:00:24 +02:00
ismail simsek a64bfdfb94 InfluxDB SQL: Provide raw query preview for query history (#75030)
* log the sql query

* render the raw sql

* Update rawQuery

* unit tests
2023-09-22 17:40:18 +02:00
Will Browne 7fca1bde54 Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
Serge Zaitsev 93cdc94a94 Chore: capitalise logs in other backend packages (#74344)
* capitalise logs in observability logs

* capitalise oss-bit-tent packages

* capitalise logs in aws-datasources

* capitalise logs for traces and profiling

* capitalise logs for partner datasources

* capitalise logs in plugins platform

* capitalise logs for observability metrics
2023-09-04 22:25:43 +02:00
Ryan McKinley 025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
ismail simsek 29ea0886e3 InfluxDB: Fix sending retention policy with the backend request (#72763)
* Add retention policy to the request

* refactor

* refactor influxql query flow

* fix healthcheck

* organize imports

* handle queries separately

* fix tests

* update bench test
2023-08-30 12:42:02 +02:00
ismail simsek 1ac9e7eaab InfluxDB: Fix handling flux response with no time and value column (#72833)
* Upgrade the influxdb-client-go version

* Handle flux response with no time and value column
2023-08-08 15:00:52 +02:00
ismail simsek d333c09418 InfluxDB: SQL Query Editor (#72168)
* Add influxdbSqlSupport feature toggle

* Add SQL option to the config page

* Add SQL backend

* Add metadata support in config page

* Implement unified querying

* Fix healthcheck query

* fsql tests

* secure grpc by default

* code cleanup

* Query handing for sql mode

* Implement a placeholder sql editor

* Fix query language dropdown

* drop in SQL editor

* switch to use rawSql, get sql editor working

* fix healthcheck

* WIP

* memoize component to stop unwanted rerender onQuery

* dont reinit datasource on each render of the editor

* remove useless memo

* clean up

* Fix the link

* Alpha state warning

* Remove console.logs

* update model for fsql

* remove unused

---------

Co-authored-by: Galen <galen.kistler@grafana.com>
2023-08-02 19:04:16 +02:00
ismail simsek 77e7ae2a1b InfluxDB: SQL Support (#72167)
* Add influxdbSqlSupport feature toggle

* Add SQL option to the config page

* Add SQL backend

* Add metadata support in config page

* Implement unified querying

* Fix healthcheck query

* fsql tests

* secure grpc by default

* code cleanup

* Query handing for sql mode

* Implement a placeholder sql editor

* Fix query language dropdown

* go mod updates

* make lint-go

* more make lint-go

* remove unused runQuery

* switch statements with default case

* linting again
2023-08-02 18:27:43 +02:00
ismail simsek 61a77b4f6b InfluxDB: Return better error messages from backend (#70066)
Co-authored-by: ludovio <ludovic.viaud@gmail.com>
2023-07-04 13:55:08 +02:00
ismail simsek 80c432e524 InfluxDB: Fix backend mode table result with aliases (#69943)
Co-authored-by: ludovio <ludovic.viaud@gmail.com>
2023-07-03 14:50:08 +02:00
ismail simsek 903af7e29c InfluxDB: Backend parser compatibility with frontend parser (#69865)
* Reformatting and restructuring

* Update unit test

* Always send the default retention policy as first element

* Fix typo

* Update test

* Update test once more

* Field names start with capital letters

* Simplify the condition

* Case-insensitive checks

* Fix typo

* Update response_parser test

* Update imports
2023-06-26 14:03:02 +02:00
Andreas Deininger 95b1f3c875 Fixing typos (#70487) 2023-06-22 09:43:38 +01:00
ismail simsek e3ea5422ea InfluxDB: Return default retention policy from backend as first element (#69818)
* Reformatting and restructuring

* Update unit test

* Always send the default retention policy as first element

* Fix typo

* Update test

* Update test once more
2023-06-17 14:20:54 +02:00
Eng Zer Jun 7f7b03d794 Chore: Avoid unnecessary byte/string conversions (#69001)
Avoid unnecessary byte/string conversion

We can use alternative functions/methods to avoid unnecessary
byte/string conversion calls.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-05-26 10:08:50 +00:00
Will Browne 286b9e08e9 Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
Kyle Brandt 6c6427e63f Chore: Use latest go-sdk (v0.158.0) (#66054)
SDK release notes:
* Logs contract: ignore remaining fields by @gabor in #659
* Logs contract: more robust field finding, explicit approach by @gabor in #660
* Tracing: Support multiple OTel propagators by @xnyo in #663
* Tracing: Add more details to HTTP Outgoing Request by @xnyo in #664
* Data: Encode Nanosecond into JSON by @kylebrandt in #647
* Data: cmp tests using FrameTestCompareOptions() will no longer ignore time differences beyond millisecond resolution
2023-04-05 21:00:31 +03:00
ismail simsek ae07bf7ce5 Influxdb: Response parser performance improvement (#64776)
* Influxdb response parser allocation improvements

* More improvements

* more improvements

* embed file

* use json-iterator

* rename

* use encoding/json
2023-03-17 20:45:13 +01:00
Brendan O'Handley 50ef183208 InfluxDB: Fix type assertion panics for interface conversion: interface {} is nil (#64556)
* fix type assertions

* fix chk

* add nil and fix tests

* add test for null bool and strings in the JSON response to be parsed as nil in Go

* reduce complexity

* reduce complexity

* reduce complexity again

* switch case for valTypes
2023-03-10 11:11:00 -05:00
Serge Zaitsev 0bdb105df2 Chore: Remove xorcare/pointer dependency (#63900)
* Chore: remove pointer dependency

* fix type casts

* deprecate xorcare/pointer library in linter

* rooky mistake
2023-03-06 05:23:15 -05:00
Ludovic Viaud 5bd2fac9c8 InfluxDB: Update InfluxQL querybuilder to qualify identifiers (#62230) 2023-03-02 15:05:24 +01:00
Kyle Brandt 52ed297ed5 Backend: Use latest plugin-sdk-go v0.149.1 (#63498)
Use latest github.com/grafana/grafana-plugin-sdk-go which includes changes to the TypeVersion property (always present in JSON).

Also included is squtil changes: SQL util - allow using the database scan type for converters
2023-02-21 09:06:15 -05:00
ismail simsek 8684fa75c7 Remove explicit user agent definition (#63019) 2023-02-07 07:49:20 -08:00
ismail simsek 65fbbc06fd InfluxDB: Move database information into jsondata (#62308)
* Use dbName in jsonData instead of database

* Use dbName in instead of database

* Remove database fields and define dbName instead

* Fix tests

* set database field as empty string
2023-01-31 14:38:02 +01:00
Serge Zaitsev f8ec35e643 Chore: Fix goimports grouping (#62427)
* fix goimports order

* fix goimports order
2023-01-30 09:38:51 +01:00
Denis Limarev 812c90ec69 Perfomance: Add preallocation for some slices (#61632) 2023-01-23 17:44:27 +01:00
Giuseppe Guerra 7996a1c8ad Chore: Update grafana-plugin-sdk-go to v0.142.0 (#58115)
* Chore: Update grafana-plugin-sdk-go to v0.142.0

* Update tests and golden files for 207 status code

* Chore: Move update flag definition at the top in response_parser_test.go

* retrigger

Co-authored-by: Will Browne <will.browne@grafana.com>
2022-11-03 18:34:27 +01:00
Yuriy Tseretyan facf2b1ee8 Chore: Update prometheus, loki, graphite and influx plugins to support contextual logs. (#57708) 2022-10-27 12:05:06 -04:00