Commit Graph

124 Commits

Author SHA1 Message Date
grafana-delivery-bot[bot] 4c64aa6bcd [v10.2.x] Loki: Fix bug duplicating parsed labels across multiple log lines (#80368)
Loki: Fix bug duplicating parsed labels across multiple log lines (#80292)

(cherry picked from commit e1aa8a95d9)

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2024-01-11 16:22:39 +02:00
Sven Grossmann 777d119a80 Loki: Fix nil pointer in api.go (#78727) 2023-11-28 12:48:23 +02:00
Sven Grossmann 44586df7bf Loki: Fix scope of labels variable being wrong (#78417)
fix scope of labels var
2023-11-21 10:14:21 +01:00
Sven Grossmann a01f8c5b42 Loki: Add backend functionality to parse structured metadata from Loki (#77361)
* prepare backend for structured metadata

* add `lokiStructuredMetadata` feature toggle

* use `lokiStructuredMetadata` feature flag

* add field type check to `labelTypesField`

* remove fixme

* fix feature toggle

* add field in dataplane mode

* use `data.Labels` where possible

* adjust framing tests

* improve verbiage

* improve naming

* update tests to "attributes"
2023-11-16 17:06:14 +01:00
Ryan McKinley f69fd3726b FeatureToggles: Add context and and an explicit global check (#78081) 2023-11-14 12:50:27 -08:00
Ivana Huckova 8cd8eb7882 Revert "Loki: Add error source to DataQuery (#77876)" (#78006)
This reverts commit 934456dc1c.
2023-11-10 17:03:59 +01:00
Ivana Huckova 934456dc1c Loki: Add error source to DataQuery (#77876)
* Loki: Add error source to DataQuery

* Create middlewares to fix healtcheck test

* Add comment for errors from ReadPrometheusStyleResult
2023-11-10 10:26:56 +01:00
Ivana Huckova d4a257bc8e Logs: Update logic to process logs dataPlane frame with labels field (#77708)
* Logs: Update dataplane logic to use labels instead of attributes

* Update Loki logs data plane data frame according to specs

* Remove only in test
2023-11-07 10:53:10 +01:00
Will Browne 9d92818cae Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
Dan Cech 91cf4f0c1c Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216.
2023-10-13 12:35:16 -04:00
Will Browne 151f6d6216 Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00
Marcus Efraimsson e4c1a7a141 Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
Ivana Huckova 5510a0d8d1 Loki: Add instrumentation for grafana_loki_plugin_parse_response_duration_seconds_bucket (#75570)
* Add metric for loki parsing response duration

* Rename and add exemplars

* Fix lint
2023-09-28 17:25:09 +02:00
Marcus Efraimsson 0128d0403f Tracing: Use tracing.InitializeTracerForTest (#75479) 2023-09-27 09:51:57 +02:00
Will Browne 7fca1bde54 Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
Travis Patterson 98aa7db64a Loki: Run queries in parallel behind lokiRunQueriesInParallel feature toggle (#74064)
* issue loki queries in parallel so total query time is only the slowest query rather than the sum of all query times.

* Fix lint

* Add running of queries in parallel behind feature toggle to test the functonality before release

* Add span end

* Move shared logic to separate function

* Add logging and tracing around running of all queries

---------

Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
2023-09-19 12:34:01 +03:00
Sven Grossmann 727c83d562 Loki: Improve error logging in case statusCode is not 200 (#75075)
* improve loki logging

* fix error log

* fix logger arg
2023-09-19 11:11:27 +02:00
Sven Grossmann aaef8d3b40 Loki: Add query expression to backend logging (#74927)
add query expression to loki log
2023-09-14 17:23:02 +02:00
Sven Grossmann d14851c877 Loki: Adjust to logging convention (#74531)
* adjust Loki to logging convention

* Fix call resource logging

* Update dataquery

* Update

* Remove redundant logging

* Fix TODO

* Rename action to stage and use variables

* `resp` might be `nil`

* `resp` might be `nil` here as well

* change to `statusCode`

* use correct logger

* also here

* add query information to logging

---------

Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
2023-09-07 17:09:24 +02:00
Sven Grossmann a403027608 Loki: Add backend healthcheck (#74330)
* add loki healthcheck

* remove `testDatasource` call

* remove unused error check

* change success message

* improve error messages
2023-09-05 08:59:13 +02:00
Serge Zaitsev 93cdc94a94 Chore: capitalise logs in other backend packages (#74344)
* capitalise logs in observability logs

* capitalise oss-bit-tent packages

* capitalise logs in aws-datasources

* capitalise logs for traces and profiling

* capitalise logs for partner datasources

* capitalise logs in plugins platform

* capitalise logs for observability metrics
2023-09-04 22:25:43 +02:00
Sven Grossmann 056c83c788 Loki: Improve backend instrumentation of CallResource calls (#73899)
* instrument call resource

* capitalize log messages
2023-09-01 15:26:25 +02:00
Sven Grossmann 124f445db0 Loki: Improve backend instrumentation of QueryData calls (#73807)
* improve loki `queryData` instrumentation

* fix lint

* use correct logger in "queryData"

* capitalize log messages

* distinguish between requests from alerting
2023-09-01 11:44:18 +02:00
Ryan McKinley 025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Kyle Brandt 6d8fc42cdd Prometheus: (Chore) Remove experimental unshipped prometheusWideSerie… (#73751) 2023-08-24 17:47:19 +03:00
Sven Grossmann 703f4be15d Loki: Fix wrong context being passed to HTTP client (#72991)
use new context containing tracing information
2023-08-08 10:24:07 +02:00
Ivana Huckova 7bb0ff7055 Loki: Introduce $__auto range variable for metric queries (#72690)
* Loki: Add  interpolation to backend

* Loki: Replace default variable  with  in frontend

* Loki: Update docs in query builder fro __auto

* Loki: Update test for change default __auto

* Loki: Remove  and  from suggestions as  should be used

* Update docs

* Update pkg/tsdb/loki/parse_query.go

* Fix backend lint

* Fix lint and test

* Update

* Update docs/sources/datasources/loki/template-variables/index.md

Co-authored-by: lwandz13 <126723338+lwandz13@users.noreply.github.com>

* Update public/app/plugins/datasource/loki/querybuilder/operationUtils.ts

Co-authored-by: Matias Chomicki <matyax@gmail.com>

---------

Co-authored-by: lwandz13 <126723338+lwandz13@users.noreply.github.com>
Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-08-03 16:27:23 +02:00
Gábor Farkas e045860fbf loki: generate dataplane-compliant logs dataframes (behind feature flag) (#69909)
* add feature flag

* prepare tests

* enable dataplane-format in loki-backend
2023-07-13 09:58:00 +02:00
Ivana Huckova 55d18361dc Loki: Deprecate resolution (#70326)
* Loki: Deprecate resolution and only show it if it was selected before

* Deprecate

* Fix merge, add missing brackets
2023-06-21 12:21:29 +02:00
Ivana Huckova 82c125d450 Loki: Implement step editor (#69648)
* Loki: Implement step editor

* Update to keep value

* Remove console.log

* Remove white space

* Update public/app/plugins/datasource/loki/querybuilder/components/LokiQueryBuilderOptions.tsx

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* Import trim

* Update using of step in split queries

* Add tests

* Add tests

* Remove step interpolation

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-06-16 19:08:29 +03:00
Tania 0316350d16 Codegen: Generate any instead of interface{} (#70201)
* Chore: Update thema to latest

* Regenerate
2023-06-16 10:54:56 +02:00
Ivana Huckova 49c940de23 Loki: Add step parameter to Loki query (#69558) 2023-06-06 13:20:27 +02:00
Selene bd622a5381 Chore: Unify references for dataquery (#68963)
* Unify references for dataquery

* Fix ResultFormat in azure types

* Update azure-log with new schema

* Fix cloudwatch query

* Update thema

* Fix test

* More fixes

* More fixes

* Fix lint
2023-05-26 10:16:01 +02:00
Will Browne 286b9e08e9 Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
Ivana Huckova ee9620e4e0 Logs: Fix unresponsive log lines if duplicate ids in Elasticsearch (#68569)
* Logs: Fix duplicate uids by appending series refIds

* Fix id generation in loki live streams to be consistent

* Update public/app/core/logsModel.ts

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Fix test

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-05-17 16:28:25 +03:00
Gábor Farkas b928fce070 Loki: Remove alpha feature toggle lokiDataframeApi (#65918)
loki: remove experimental feature-flag
2023-04-13 15:22:09 +02:00
Gábor Farkas 531caec602 Loki: Add feature flag to enable dataplane-compliant metric frames (#66017)
* loki: add feature flag to enable dataplane-compatible metric frames

* linter fix
2023-04-13 15:07:08 +02:00
Kyle Brandt 6c6427e63f Chore: Use latest go-sdk (v0.158.0) (#66054)
SDK release notes:
* Logs contract: ignore remaining fields by @gabor in #659
* Logs contract: more robust field finding, explicit approach by @gabor in #660
* Tracing: Support multiple OTel propagators by @xnyo in #663
* Tracing: Add more details to HTTP Outgoing Request by @xnyo in #664
* Data: Encode Nanosecond into JSON by @kylebrandt in #647
* Data: cmp tests using FrameTestCompareOptions() will no longer ignore time differences beyond millisecond resolution
2023-04-05 21:00:31 +03:00
Sven Grossmann 7261c6f7cd DataSourceWithBackend: Add queryGroupId to find correlated/related queries (#64587)
* add `correlationId` to queries

* trace correlation id in backend

* add correlation id to loki's span

* add correlation id to query chunks

* fix test

* fix DataSourceWithBackend test

* change to `queryGroupId`

* remove empty line

* fix test in `DataSourceWithBackend`
2023-03-17 15:48:13 +01:00
Sven Grossmann 473013e3f5 Loki: Add error handling to CallResource (#64439)
* surface loki error message for `CallResource`

* use `data.Message` instead or `errorMessage`

* change struct coming from Loki

* remove whitespace
2023-03-09 11:12:33 +01:00
Ryan McKinley 4b94c7e5d2 Schema: Remove key from root DataQuery type (#64467) 2023-03-08 16:52:31 -05:00
Kyle Brandt 52ed297ed5 Backend: Use latest plugin-sdk-go v0.149.1 (#63498)
Use latest github.com/grafana/grafana-plugin-sdk-go which includes changes to the TypeVersion property (always present in JSON).

Also included is squtil changes: SQL util - allow using the database scan type for converters
2023-02-21 09:06:15 -05:00
Selene 7ace44c8b5 Chore: Update oapi-codegen library (#62962)
* Update oapi library and thema

* Use fork commit to fix elasticsearch and cloudwatch generators

* Update thema

* Fixes

* Update thema with last fixes

* Sync

* Fix test

* Update thema and schemas

* Update thema
2023-02-21 12:34:24 +01:00
Gábor Farkas 48a374f50b loki: implement schema for loki query (#62114)
* loki: implement schema for loki query

* removed forgotten code

* better types

* adjust cue-maturity-level

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* sync docs to schema change

* removed forgotten go code

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* update schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* update schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

* add documentation to the resolution field

* move direction-field to schema

* loki: cue: moved deprecated fields to the bottom

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
2023-02-02 16:18:30 +00:00
Serge Zaitsev 0bf4093005 Chore: Fix goimports grouping (#62428)
* fix goimports order

* fix goimports order
2023-01-30 08:50:27 +00:00
Gareth Dawson b2c8126e6e Loki: Show query size approximation (#62109)
* feat: make api request to /loki/api/v1/index/stats

* fix: add /index/stats to callResource valid urls

* feat: make call to getQueryStats when the query changes

* feat: render user tooltip displaying the estimated value for processed data

* fix: add new props to component tests

* test: add tests for query size estimation

* fix: disable error message on request failure

* refactor: add suggestions from code review

* refactor: only pass required query string
2023-01-27 17:32:53 +00:00
Ivana Huckova d0e95f8c95 Loki: Add X-Query-Tags header for logs sample and data sample (#62333) 2023-01-27 16:41:40 +01:00
Denis Limarev 812c90ec69 Perfomance: Add preallocation for some slices (#61632) 2023-01-23 17:44:27 +01:00
KonH 4421b31b66 Loki/Prometheus: Change signature - return non-reference from ReadPrometheusStyleResult (#57209)
* Change signature - return non-reference from ReadPrometheusStyleResult

Related to https://github.com/grafana/grafana/issues/52430

* Fix invalid reference usage

* Potential fix for test data

* Remove additional ref usage

* Revert "Potential fix for test data"

This reverts commit 94ed588ab3.

* Commit changed test data

* Revert unwanted conflict resolution

* Add additional condition to track empty responses

* Setup valid empty response

* Re-introduce changes reverted by merge
2023-01-13 09:00:39 +01:00
Marcus Efraimsson c35c689a96 Plugins: Automatically forward plugin request HTTP headers in outgoing HTTP requests (#60417)
Automatically forward core plugin request HTTP headers in outgoing HTTP requests. 
Core datasource plugin authors don't have to specifically handle forwarding of HTTP 
headers, e.g. do not have to "hardcode" the header-names in the datasource plugin, 
if not having custom needs.

Fixes #57065
2022-12-21 13:25:58 +01:00