Commit Graph

56 Commits

Author SHA1 Message Date
Gábor Farkas
2e5b55a855 datasources: querier: renamed the "mt" builder to "qs" builder (#109779) 2025-08-19 12:37:56 +02:00
Gábor Farkas
81531dcd7b datasources: strict local-time-range parsing (#109180)
* datasources: strict local-time-range parsing

* fixed unit test
2025-08-07 11:46:43 +02:00
Gábor Farkas
0ccfdbdfae datasources: querier: better handling of timestamps (#108973)
* query: support query-local timestamps

* make it configurable

* lint fix

* corrected unit test

* fixed unit test
2025-08-05 08:35:57 +02:00
Gábor Farkas
5a67e9d640 query: add unit test (#108958) 2025-07-31 15:26:43 +02:00
Serge Zaitsev
a95fb3a37c Chore: Omit integration tests if short test flag is passed (#108777)
* omit integration tests if short test flag is passed

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* fix the rest

* false positive

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-28 13:38:54 +02:00
Sarah Zinger
f34127c449 Query Service: Fix Time Bug (#108337) 2025-07-18 18:14:44 +00:00
Sarah Zinger
3fad863fd1 Query Service: Combine SSE handling in single tenant and multi tenant paths (#108041)
* parse via sse

I need to figure out how to handle the pipeline.execute with our own
client. I think this is important for MT reasons, just like using our
own cache (via legacy) is important.

parsing is done though!

* WIP nonsense

* horrible code but i think it works

* Add support for sql expressions config settings

* Cleanup:
- remove spew from nodes.go
- uncomment out plugin context and use in single tenant flow
- make code more readable and add comments

* Cleanup:
- create separate file for mt ds client builder
- ensure error handling is the same for both expressions and regular queries
- other cleanup

* not working but good thoughts

* WIP, vector not working for non sse

* super hacky but i think vectors work now

* delete delete delete

* Comments for future ref

* break out query handling and start test

* add prom debugger

* clean up: remove comments and commented out bits

* fix query_test

* add prom debugger

* create table-driven tests with testsdata files

* Fix test

* Add test

* go mod??

* idk

* Remove comment

* go enterprise issue maybe

* Fix codeowners

* Delete

* Remove test data

* Clean up

* logger

* Remove go changes hopefully

* idk go man

* sad

* idk i ran go mod tidy and this is what it wants

* Fix readme, with much help from adam

* some linting and testing errors

* lint

* fix lint

* fix lint register.go

* another lint

* address lint in test

* fix dead code and linters for query_test

* Go mod?

* Struggling with go mod

* Fix test

* Fix another test

* Revert headers change

* Its difficult to test this in OSS as it depends on functionality defined in enterprise, let's bring these tests back in some form in enterprise

* Fix codeowners

---------

Co-authored-by: Adam Simpson <adam@adamsimpson.net>
2025-07-17 17:22:55 -04:00
Serge Zaitsev
f66a693438 Chore: Rename integration tests to follow the common convention (#105987)
* automatically rename integration tests to follow the common convention

* name tests differently

* alter column type to bigint

* update another column to bigint

* add another alter

* fix subquery for mysql
2025-06-29 16:56:24 +02:00
Sofia Papagiannaki
d192a44469 Plugins: Modify interface for plugin validations to allow taking PDC into account (#96089)
* Request interceptor: Do not block PDC

* Apply change after feedback received

* Add test

* Check if secure socks proxy configured for the instance

* Apply suggestions from code review

* Add dedicated service for datasource request URL validation (#99179)

---------

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2025-01-24 17:01:46 +02:00
Ryan McKinley
99d8025829 Chore: Move identity and errutil to apimachinery module (#89116) 2024-06-13 07:11:35 +03:00
Serge Zaitsev
522a98c126 Chore: Make Cfg field private in SQLStore (#85593)
* make cfg private in sqlstore

* fix db init in tests

* fix case

* fix folder test init

* fix imports

* make another Cfg private

* remove another Cfg

* remove unused variable

* use store cfg, it has side-effects

* fix mutated cfg in tests
2024-04-24 10:38:40 +02:00
Dave Henderson
5687243d0b Feature Flags: use FeatureToggles interface where possible (#85131)
* Feature Flags: use FeatureToggles interface where possible

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Replace TestFeatureToggles with existing WithFeatures

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-04-04 12:22:31 -04:00
Will Browne
5edd96ae77 Plugins: Refactor plugin config into separate env var and request scoped services (#83261)
* seperate services for env + req

* merge with main

* fix tests

* undo changes to golden file

* fix linter

* remove unused fields

* split out new config struct

* provide config

* undo go mod changes

* more renaming

* fix tests

* undo bra.toml changes

* update go.work.sum

* undo changes

* trigger

* apply PR feedback
2024-02-27 12:38:02 +01:00
Dan Cech
790e1feb93 Chore: Update test database initialization (#81673)
* streamline initialization of test databases, support on-disk sqlite test db

* clean up test databases

* introduce testsuite helper

* use testsuite everywhere we use a test db

* update documentation

* improve error handling

* disable entity integration test until we can figure out locking error
2024-02-09 09:35:39 -05:00
Will Browne
3f30cbf91c DataSources: Add datasource fetching + querying interface (#80749)
* first pass

* separate oss + enterprise

* tidy things up

* add ctx

* fix tests

* use standalone svcs

* mv plugin context provide

* fix wire

* fix import
2024-01-19 15:56:52 +01:00
Will Browne
7fca1bde54 Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
Will Browne
e855efb13d Plugins: Move store and plugin dto to pluginsintegration (#74655)
move store and plugin dto
2023-09-11 13:59:24 +02:00
linoman
1b8e9b51b2 Replace signed in user for identity.requester (#74048)
* Make identity.Requester available at Context

* Clean pkg/services/guardian/guardian.go

* Clean guardian provider and guardian AC

* Clean pkg/api/team.go

* Clean ctxhandler, datasources, plugin and live

* Clean dashboards and guardian

* Implement NewUserDisplayDTOFromRequester

* Change status code numbers for http constants

* Upgrade signature of ngalert services

* log parsing errors instead of throwing error
2023-08-30 16:51:18 +02:00
Jo
a307582212 Revert "Replace signed in user for identity.requester (#73750)" (#73962)
This reverts commit 9b9c9e83dc.
2023-08-28 21:05:59 +02:00
linoman
9b9c9e83dc Replace signed in user for identity.requester (#73750)
* Make identity.Requester available at Context

* Clean pkg/services/guardian/guardian.go

* Clean guardian provider and guardian AC

* Clean pkg/api/team.go

* Clean ctxhandler, datasources, plugin and live

* Question: what to do with the UserDisplayDTO?

* Clean dashboards and guardian

* Remove identity.Requester from ReqContext

* Implement NewUserDisplayDTOFromRequester

* Fix tests

* Change status code numbers for http constants

* Upgrade signature of ngalert services

* log parsing errors instead of throwing error

* Fix tests and add logs

* linting
2023-08-28 12:04:36 -05:00
Torkel Ödegaard
3ee26df41e PublicDashboards: Variables refactor (#73476)
Co-authored-by: Juan Cabanas <juan.cabanas@grafana.com>
Co-authored-by: Ezequiel Victorero <ezequiel.victorero@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-08-25 13:56:02 -05:00
Will Browne
a8577c21ba Plugins: Migrate PluginStore mock to pre-existing fakes package (#71664)
* migrate to existing fakes package

* fix imports
2023-07-17 10:21:44 +00:00
Michael Mandrus
ff6d6659fb Query: Fix concurrency handling for mixed datasource queries (#70100)
* split queries and merge responses

* increase concurrency again

* update unit test to verify the headers are merged

* fix lint issue

* fix race condition in unit test

* Fix function name and add a bit more documentation about how the func should be used

* update function call after rename

* check for duplicate header vals

* make concurrent query limit configurable

* Update conf/sample.ini

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-07-06 17:15:43 +03:00
Will Browne
624777258b Plugins: Refactor creation of plugin context to dedicated service (#66451)
* first pass

* fix tests

* return errs

* change signature

* tidy

* delete unnecessary fields from test

* tidy

* fix tests

* simplify

* separate error check in API

* apply nits
2023-06-08 13:59:51 +02:00
Kyle Brandt
840fb32ad8 SSE: (Instrumentation) Add Tracing (#66700)
spans are prefixed `SSE.`
2023-04-18 08:04:51 -04:00
Kyle Brandt
2f13c851e4 SSE: (Chore/Instrumentation) Add ds_queries_total metric and move met… (#66695)
* SSE: (Chore/Instrumentation) Add ds_queries_total metric and move metrics to service
2023-04-17 16:12:44 -07:00
Kyle Brandt
e78be44e1a SSE: Dataplane Compliance (#65927)
Takes a specific code path for data that identifies itself as dataplane instead of "guessing" what the data is.

The data must identify itself by being in the dataplane by having both the following frame metadata properties:

- TypeVersion property that is greater than 0.0
- 'Type' property

The flag is disableSSEDataplane and disables this functionality and uses the old code for all queries regardless.

See https://github.com/grafana/grafana-plugin-sdk-go/blob/main/data/contract_docs/contract.md for dataplane details.
2023-04-12 12:24:34 -04:00
Ezequiel Victorero
8f0a9729f0 PublicDashboards: Allow hidden queries execution (#64858) 2023-03-16 15:39:17 -03:00
Andres Martinez Gotor
00b692c0f9 Chore: Validate batch query refIds (#63018) 2023-02-09 10:11:16 +01:00
idafurjes
23c27cffb3 Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
Andres Martinez Gotor
68862ce3e8 Plugins: Prefer to use the data source UID when querying (#62776) 2023-02-02 15:43:07 +01:00
ismail simsek
91221bc436 Expressions: Fixes the issue showing expressions editor (#62510)
* Use suggested value for uid

* update the snapshot

* use __expr__

* replace all -100 with __expr__

* update snapshot

* more changes

* revert redundant change

* Use expr.DatasourceUID where it's possible

* generate files
2023-01-31 18:50:10 +01:00
idafurjes
6c5a573772 Chore: Move ReqContext to contexthandler service (#62102)
* Chore: Move ReqContext to contexthandler service

* Rename package to contextmodel

* Generate ngalert files

* Remove unused imports
2023-01-27 08:50:36 +01:00
Marcus Efraimsson
6dbe3b555f Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132)
Adding support for backend plugin client middlewares. This allows headers in outgoing 
backend plugin and HTTP requests to be modified using client middlewares.

The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client 
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP 
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming 
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package, 
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with 
middlewares and/or longer term bootstrap/instantiate plugin management. 

Fixes #54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
2022-12-01 19:08:36 +01:00
Michael Mandrus
40d87d9d40 Datasources: Refactor mixed datasource support to improve concurrency and error handling (#58163)
* recover from panic inside mixed ds query loop

* remove accidental commit

* add messages from panics and errors to the query response

* refactor based on PR comments

* quick update to unit test to verify mixed errors and successes

* reduce concurrency limit
2022-11-28 10:21:54 -05:00
Kyle Brandt
5623b5afaf SSE: Make sure to forward headers, user and cookies/OAuth token (#58897)
Fixes #58793 and Fixes https://github.com/grafana/azure-data-explorer-datasource/issues/513

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-28 07:40:06 -05:00
Ryan McKinley
8e19a1618f QueryData: skip header validation (revert check) (#58871) 2022-11-18 05:46:50 -05:00
Ryan McKinley
934fb2f0ee QueryData: fix header parsing to support expressions (#58826)
fixes #58821
2022-11-16 10:17:24 -05:00
Ryan McKinley
d33939da55 DataSourceWithBackend: Add plugin id to the request headers (#58082) 2022-11-14 19:35:50 -05:00
Sofia Papagiannaki
9855e74b92 Chore: Refactor quota service (#58643)
Chore: Refactor quota service (#57586)

* Chore: refactore quota service

* Apply suggestions from code review
2022-11-14 21:08:10 +02:00
Sofia Papagiannaki
96cdf77995 Revert "Chore: Refactor quota service (#57586)" (#58394)
This reverts commit 326ea86a57.
2022-11-08 11:52:07 +02:00
Sofia Papagiannaki
326ea86a57 Chore: Refactor quota service (#57586)
* Chore: refactore quota service

* Apply suggestions from code review
2022-11-08 10:25:34 +02:00
Marcus Efraimsson
6f8fcae01b [main] Plugin fixes (#57399)
* Plugins: Remove support for V1 manifests

* Plugins: Make proxy endpoints not leak sensitive HTTP headers

* Security: Fix do not forward login cookie in outgoing requests

(cherry picked from commit 4539c33fce)

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-10-21 13:54:55 +02:00
Kristin Laemmert
05709ce411 chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
Misi
9c954d06ab Auth: Refresh OAuth access_token automatically using the refresh_token (#56076)
* Verify OAuth token expiration for oauth users in the ctx handler middleware

* Use refresh token to get a new access token

* Refactor oauth_token.go

* Add tests for the middleware changes

* Align other tests

* Add tests, wip

* Add more tests

* Add InvalidateOAuthTokens method

* Fix ExpiryDate update to default

* Invalidate OAuth tokens during logout

* Improve logout

* Add more comments

* Cleanup

* Fix import order

* Add error to HasOAuthEntry return values

* add dev debug logs

* Fix tests

Co-authored-by: jguer <joao.guerreiro@grafana.com>
2022-10-18 18:17:28 +02:00
Michael Mandrus
ea8549b8c2 Datasources: Support mixed datasources in a single query (#56832)
* initial cut at refactor - need to run more tests

* fix unit tests

* change newly unused function to test helper

* create unit tests for parsing query requests that cover a range of cases

* add some comments

* rename function to avoid dev confusion
2022-10-14 10:27:06 -04:00
juanicabanas
69c49f6ba2 PublicDashboards: Make mixed datasource calls concurrently (#56421)
PublicDashboards: Make mixed datasource calls concurrently
2022-10-05 18:45:17 -03:00
owensmallwood
f4bbce15a0 Public Dashboards: Add Expressions Support (#54336)
Adds support for expressions with public dashboards
2022-08-31 09:11:10 -06:00
Marcus Efraimsson
87afd9cadc Plugins: Remove various custom headers logic (#54146)
Removes various custom headers logic sprinkled around in the backend. 
It should automatically be applied to outgoing HTTP requests via the 
CustomHeadersMiddleware.
This also removes decryption of SecureJSONData to populate custom 
headers in ngalert which seemed to have caused a ton of CPU usage.
2022-08-26 11:56:10 +02:00
Guilherme Caulada
f25c7f6ddd Chore: Refactor secrets kvstore to organize testing and migrations (#54249)
* Refactor migrations and tests for secrets kvstore

* Use fake secrets store as a shortcut on tests

* Update wire

* Use global migration logger

* Fix ds proxy tests

* Fix linting issues

* Rename data source test setup function
2022-08-25 18:04:44 -03:00