Commit Graph
11973 Commits
Author SHA1 Message Date
Mihai Doarna d7641b0ecb Auth: encrypt secrets for oauth providers in SSO settings API service (#79081)
encrypt secrets for oauth providers
2023-12-06 14:37:10 +02:00
ismail simsekandİnanç Gümüş c088d003f2 InfluxDB: Implement InfluxQL json streaming parser (#76934)
* Have the first iteration

* Prepare bench testing

* rename the test files

* Remove unnecessary test file

* Introduce influxqlStreamingParser feature flag

* Apply streaming parser feature flag

* Add new tests

* More tests

* return executedQueryString only in first frame

* add frame meta and config

* Update golden json files

* Support tags/labels

* more tests

* more tests

* Don't change original response_parser.go

* provide context

* create util package

* don't pass the row

* update converter with formatted frameName

* add executedQueryString info only to first frame

* update golden files

* rename

* update test file

* use pointer values

* update testdata

* update parsing

* update converter for null values

* prepare converter for table response

* clean up

* return timeField in fields

* handle no time column responses

* better nil field handling

* refactor the code

* add table tests

* fix config for table

* table response format

* fix value

* if there is no time column set name

* linting

* refactoring

* handle the status code

* add tracing

* Update pkg/tsdb/influxdb/influxql/converter/converter_test.go

Co-authored-by: İnanç Gümüş <m@inanc.io>

* fix import

* update test data

* sanity

* sanity

* linting

* simplicity

* return empty rsp

* rename to prevent confusion

* nullableJson field type for null values

* better handling null values

* remove duplicate test file

* fix healthcheck

* use util for pointer

* move bench test to root

* provide fake feature manager

* add more tests

* partial fix for null values in table response format

* handle partial null fields

* comments for easy testing

* move frameName allocation in readSeries

* one less append operation

* performance improvement by making string conversion once

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
             │ stream2.txt │            stream3.txt             │
             │   sec/op    │   sec/op     vs base               │
ParseJson-10   314.4m ± 1%   303.9m ± 1%  -3.34% (p=0.000 n=10)

             │ stream2.txt  │             stream3.txt              │
             │     B/op     │     B/op      vs base                │
ParseJson-10   425.2Mi ± 0%   382.7Mi ± 0%  -10.00% (p=0.000 n=10)

             │ stream2.txt │            stream3.txt             │
             │  allocs/op  │  allocs/op   vs base               │
ParseJson-10   7.224M ± 0%   6.689M ± 0%  -7.41% (p=0.000 n=10)

* add comment lines

---------

Co-authored-by: İnanç Gümüş <m@inanc.io>
2023-12-06 12:39:05 +01:00
Ryan McKinley 3299809977 Chore: remove unused math function (#79120) 2023-12-06 01:38:48 -08:00
Gábor Farkas 9f0fc60477 sql: remove xorm dependency from postgres/mysql/mssql (#77870)
* sql: remove xorm

* sql: remove pkg/util dependency (#78821)

sql: removed md5-util dependency
2023-12-06 09:35:05 +01:00
Ryan McKinleyandCharandas Batra 439edebcd6 K8s: fix standalone command and add hack scripts (#79052)
Co-authored-by: Charandas Batra <charandas.batra@grafana.com>
2023-12-05 14:31:49 -08:00
Ezequiel Victorero 66df17869d Chore: Use dashboard service to retrieve dashboards (#79020) 2023-12-05 18:09:21 -03:00
gotjosh 0c9356a3c7 Unified Alerting: Set max_attempts to 1 by default (#79095)
* Unified Alerting: Set `max_attempts` to 1 by default

The retry logic for unified alerting has been broken as far as v9.4.x, rather than fixing it in one go and causing a headache to our users with rules putting extra load on their datasources - I think a better approach is to simply set 1 as a default and then let our users change it.

I see two cons with this approach:

- Configuration for legacy to unified alerting cannot be ported over automatically, users will have to manually set `max_attempts` to 3 when migrating.
- Users expecting to get any sort of retrying (as with legacy alerting) will not have it out of the box and will have to manually edit the configuration.

Signed-off-by: gotjosh <josue.abreu@gmail.com>
---------

Signed-off-by: gotjosh <josue.abreu@gmail.com>
2023-12-05 17:42:34 +00:00
Alexander Zobnin 959ebf82da Folders: Show dashboards and folders with directly assigned permissions in "Shared" folder (#78465)
* Folders: Show folders user has access to at the root level

* Refactor

* Refactor

* Hide parent folders user has no access to

* Skip expensive computation if possible

* Fix tests

* Fix potential nil access

* Fix duplicated folders

* Fix linter error

* Fix querying folders if no managed permissions set

* Update benchmark

* Add special shared with me folder and fetch available non-root folders on demand

* Fix parents query

* Improve db query for folders

* Reset benchmark changes

* Fix permissions for shared with me folder

* Simplify dedup

* Add option to include shared folder permission to user's permissions

* Fix nil UID

* Remove duplicated folders from shared list

* Folders: Fix fetching empty folder

* Nested folders: Show dashboards with directly assigned permissions

* Fix slow dashboards fetch

* Refactor

* Fix cycle dependencies

* Move shared folder to models

* Fix shared folder links

* Refactor

* Use feature flag for permissions

* Use feature flag

* Review comments

* Expose shared folder UID through frontend settings

* Add frontend type for sharedWithMeFolderUID option

* Refactor: apply review suggestions

* Fix parent uid for shared folder

* Fix listing shared dashboards for users with access to all folders

* Prevent creating folder with "shared" UID

* Add tests for shared folders

* Add test for shared dashboards

* Fix linter

* Add metrics for shared with me folder

* Add metrics for shared with me dashboards

* Fix tests

* Tests: add metrics as a dependency

* Fix access control metadata for shared with me folder

* Use constant for shared with me

* Optimize parent folders access check, fetch all folders in one query.

* Use labels for metrics
2023-12-05 16:13:31 +01:00
Dan Cech 647f576359 Server: Automatically generate a self-signed TLS cert if needed (#78243)
generate a self-signed cert when running in https or http2 mode without a provided cert
2023-12-05 10:04:39 -05:00
Todd Treece 7e2aad1d01 Revert "K8s: Refactor metrics to share k8s registry (#77957)" (#79079)
This reverts commit 38bc41651a.
2023-12-05 09:34:07 -05:00
Alexander Zobnin 043096d652 Team LBAC: Limit access to data source (#78376)
* Team LBAC: Limit access to data source

* Fix tests
2023-12-05 13:45:40 +02:00
Misi d099292d99 Auth: Fix for the github_oauth parse config error (#79063)
* Fix for reverting the github_oauth parse config behaviour

* minimal changes
2023-12-05 13:13:55 +02:00
Torkel ÖdegaardandRyan McKinley ed128ea964 Frontend: Foundations for multi tenant frontend (#78815)
* Frontend: Foundations for multi tenant frontend

* improve manifest parsing for multi-tenant frontend (#78876)

* add test

* add test

* ??

* Updates

* Added cache

* test cleanup

* lint

* fix test

* fix error templates

* cleanup

* remove copy

* revert changes to list testdata

* comment cleanup

* prepare integration tests

* Remove integrety

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-05 08:34:22 +01:00
Ryan McKinley 7b78061235 K8s: Add grafana metadata accessor (#79050) 2023-12-04 18:20:17 -08:00
Rodrigo VillablancaandYuri Tseretyan ab83bc7346 Alerting: Fix export of notification policy to JSON (#78021)
* Export Notification Policy correctly (#78020)

The JSON version of an exported Notification Policy now
inline correctly the policy in the same way the Yaml version
does.

Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2023-12-04 16:57:37 -05:00
Gabriel MABILLE 99580d60f5 Chore: Fix flaky test by removing the extsvcauth background service (#79044) 2023-12-04 22:26:55 +01:00
Todd TreeceandRyan McKinley 38bc41651a K8s: Refactor metrics to share k8s registry (#77957)
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-04 10:54:59 -08:00
Julien Duchesne cd584e9261 Swagger: Fix getLibraryElementByName response (#78221)
It returns an array of library elements, not a single one. This has been tested by generating the OpenAPI client and using it in Terraform
2023-12-04 11:20:13 -05:00
Julien Duchesne 3c51190392 ngalert make: Support GNU install on Darwin (#78482)
* ngalert `make`: Support GNU install on Darwin
Currently, the Makefile assumes that Darwin is using the Mac version of `sed`
I have the GNU version, so it failed. With this PR, it checks which version is installed
I also called `make` and there are some changes that came out of it

* swagger-gen
2023-12-04 10:11:39 -05:00
Dan Cech 318f51eaee Chore: Remove unused CacheService dependency from sqlstore (#78507)
remove unused CacheService dependency from sqlstore
2023-12-04 10:00:45 -05:00
Julien Duchesne 637cfa89be Swagger: Rename Spec to Preferences (#78226)
The definition for preferences is globally named `Spec` because that's the type that cue outputs
This adds a swagger annotation to rename the definition in the swagger schema to `Preferences`
This will be easier to use in generated clients
2023-12-04 09:51:09 -05:00
Gabriel MABILLE 5b70130e6c Plugins: Rename externalServiceAuthentication to iam (#78686)
Plugins: Rename externalServiceAuthentication to iam
2023-12-04 13:14:21 +01:00
Sofia Papagiannaki 6d4625ad52 Alerting: Fix deleting rules in a folder with matching UID in another organization (#78258)
* Remove usage of obsolete function for deleting alert rules under folder

* Apply suggestion from code review

* Update tests
2023-12-04 11:34:38 +02:00
Ryan McKinley f761ae1f02 FeatureFlags: Add created time property (#78996) 2023-12-03 20:45:51 -08:00
Ashley Harrison 4a6ff8cf69 Navigation: add application-observability icon (#78935)
add application-observability icon
2023-12-02 09:51:46 +00:00
Yuri Tseretyan 64feeddc23 Alerting: Update rule access control to return errutil errors (#78284)
* update rule access control to return errutil errors
* use alerting in msgID
2023-12-02 01:42:11 +02:00
Matthew Jacobson 6644e5e676 Alerting: Fix migration that is brittle to version downgrades (#78976) 2023-12-01 15:18:41 -05:00
ismail simsek 2b0e7afb4e InfluxDB: Fix an edge case while parsing table response (#78802)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again

* handle nil values

* append in the right array

* add comments

* remove redundant if condition
2023-12-01 18:31:49 +01:00
Alexander Weaver ab0ef5276f Alerting: Decouple quota configuration logic from API interfaces and add tests (#78930)
* Separate usage reporter from API

* Extract quota registration

* Decouple from API store interface

* Move to ngalert package and add tests

* linter
2023-12-01 10:47:19 -06:00
Ivan Ortega Alba e56a252158 GeneralSettings: Edit general dashboards settings to scenes (#78492) 2023-12-01 17:04:56 +02:00
Ieva c354c7bfff RBAC: Update fixed annotation roles (#78756)
* update fixed annotation roles if FlagAnnotationPermissionUpdate is enabled

* add dashboard type scope back in the fixed roles to make the migration easier
2023-12-01 14:50:55 +00:00
MisiandGabriel MABILLE d3a269ab3b Auth: Load ini/env vars settings in the fallback strategy (#78495)
* Return data in camelCase from the OAuth fb strategy

* changes

* wip

* Add defaults for oauth fb strategy

* revert other changes

* Add tests

* Add Defaults to cfg and use it in OAuthStrategy

* Return *OAuthInfo from OAuthStrategy

* lint

* Remove unnecessary Defaults

* Introduce const for fields, fix import order

* Align failing tests

* clean up

* Changes requested by @gamab

* Update pkg/services/ssosettings/strategies/oauth_strategy_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Load data on startup

* Rename + simplify

---------

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-12-01 15:35:44 +01:00
Steve Simpson 520c927931 Alerting: Only warm alert state cache if execute_alerts=true. (#78895)
* Alerting: Only warm alert state cache if execute_alerts=true.

If the Grafana instance is not executing alerts, then Warm()-ing the state
manager is wasteful and could lead to misleading rule status queries, as the
status returned will be always based on the state loaded from the database at
startup, and not the most recent evaluation state.

* Move Warm() down to shared conditional.
2023-12-01 10:17:32 +01:00
Matthew JacobsonandChristopher Moyer 5a80962de9 Alerting: Add clean_upgrade config and deprecate force_migration (#78324)
* Alerting: Add clean_upgrade config and deprecate force_migration

Upgrading to UA and rolling back will no longer delete any data by default. 
Instead, each set of tables will remain unchanged when switching between 
legacy and UA. As such, the force_migration config has been deprecated 
and no extra configuration is required to roll back to legacy anymore.

If clean_upgrade is set to true when upgrading from legacy alerting to Unified
Alerting, grafana will first delete all existing Unified Alerting resources,
thus re-upgrading all organizations from scratch. If false or unset,
organizations that have previously upgraded will not lose their existing Unified
 Alerting data when switching between legacy and Unified Alerting.

 Similar to force_migration, it should be kept false when not needed as it may
 cause unintended data-loss if left enabled.

---------

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-11-30 11:01:11 -05:00
Karl Persson 0f0249abea RBAC: Fix filter so that check for access on service account is correct (#78907)
Fix filter so that check for access on service account is in correct place
2023-11-30 17:32:04 +02:00
Matthew Jacobson cdad712547 Alerting: Keep track of individual org migration status (#78369)
* Alerting: Keep track of individual org migration status

Save migration status per migrated org.

Change the meaning (and key/value) of the org_id=0 entry 
to store the current (previous) config value used by alerting. 
This is so we can know when to upgrade/downgrade by 
comparing with the new config value in 
UnifiedAlerting.IsEnabled.
2023-11-30 10:25:59 -05:00
Ashley Harrisonandeledobleefe 329d0e79ec Navigation: Rearrange IA based on tree test (#78804)
* rearrange IA

* add landing page routes and subtitles

* Translate menu items

* add frontend-observability icon

* proper frontend-observability icon

---------

Co-authored-by: eledobleefe <laura.fernandez@grafana.com>
2023-11-30 15:18:05 +00:00
Gabriel MABILLEandMarcus Efraimsson ef2c79d22a Chore: use errutil for pluginRepo errors (#78647)
* Chore: use errutil for pluginRepo errors

* Update pkg/util/errutil/status.go

* Use errutil helper functions

Co-Authored-By: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Forgot the log level

* Use entity

---------

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2023-11-30 15:49:27 +01:00
Santiago d64c2b6f4e Alerting: Implement ApplyConfig in the forked Alertmanager (#78684)
* Alerting: Add a sync interval for ApplyConfig in remote secondary mode

* remove out of scope code

* remove parentheses after CleanUp for consistency in test comments

* Add comment to ApplyConfig
2023-11-30 15:36:41 +01:00
Santiago 316c8b50bc Alerting: Add SaveAndApply methods to the forked Alertmanager (remote secondary) (#78827)
* Alerting: Add configuration methods to the forked Alertmanager for remote secondary modes

* update comments
2023-11-30 15:18:56 +01:00
Eric LeijonmarckandGabriel MABILLE 3311467210 Auth: AzureAD surface error from group claims (#78808)
* Add error to surface for groups groups not valid

* Update pkg/login/social/azuread_oauth.go

---------

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-11-30 15:08:15 +01:00
ismail simsek dff5022021 Chore: Return executedString information even with frames has no time column (#78906)
* return executedString for no time column responses

* remove comment
2023-11-30 14:25:15 +01:00
Gilles De Mey 7cbf5ae78d Alerting: Remove old contact points view (#78704) 2023-11-30 13:37:14 +01:00
Andreas Christou f26ad881ed Plugins: Only set non-existing headers for core plugin requests (#78633)
* Only set non-existing headers

- Add test scenario

* Fix test
2023-11-30 11:51:40 +00:00
Misi 79577e4929 Auth: Remove oAuthProviders from Social service (#78732)
* Remove oauthProviders from social svc

* Add EnabledFn to supportbundles.Collector
2023-11-30 09:30:35 +01:00
ShirleyandFiona Liao 86311e3a33 CloudWatch: Refactor query batching (#78581)
Co-authored-by: Fiona Liao <fiona.y.liao@gmail.com>
2023-11-29 22:43:34 +01:00
Dan CechandMarcus Efraimsson 3c89e68fc1 Chore: Fix timeout issues when gathering prometheus datasource stats (#74618)
* fix timeout issues when gathering prometheus flavor stats

* workaround data race in sdk tracing middleware

* cap concurrency at 10

---------

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2023-11-29 16:01:54 -05:00
Alexander Weaver 8a56a94781 Usagestats: Add stat group for alert rule groups (#78825)
* Add rule group support to usagestats service

* Quote column name
2023-11-29 14:37:36 -06:00
406704a027 XYChart: Add demo gdev dashboard (#71168)
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
Co-authored-by: Adela Almasan <adela.almasan@grafana.com>
2023-11-29 19:54:16 +02:00
ismail simsek 5845f14075 InfluxDB: Introduce influxqlStreamingParser feature toggle (#78834)
introduce influxqlStreamingParser feature toggle
2023-11-29 18:29:35 +01:00