Commit Graph

606 Commits

Author SHA1 Message Date
Piotr Jamróz
6750e881e3 Query History: Use a search index on new queries to filter in mixed data sources (#88979)
* Add search index table

* Stab a test

* Add more tests

* Add basic index

* Switch to UID and add a test for the index

* Improve tests coverage

* Remove redundant whitespaces

* Load all data source APIs when query history is loaded

* Fix column type

* Fix migration

* Clean-up the index

* Fix linting

* Fix migrations

* Fix migrations

* Fix migrations

* Rename index to details
2024-07-16 11:47:21 +02:00
Matthew Jacobson
ba800692c6 Alerting: Persist AlertInstance ResolvedAt & LastSentAt (#89135)
* Alerting: Persist AlertInstance ResolvedAt & LastSentAt

* Fix test

* Modify existing tests

* Fix merge conflicts from nullable LastSentAt & ResolvedAt
2024-07-12 12:26:58 -04:00
Michael Mandrus
4d69213829 CloudMigrations: Break snapshot resources out into their own table (#89575)
* create a new table for migration resources

* remove raw result bytes from db

* more snapshot resource management stuff

* integrate new table with snapshots

* pass in result limit and offset as params

* combine create and update

* set up xorm store test

* add unit tests

* save some cpu

* remove unneeded arg

* regen swagger

* fix bug with result processing

* fix update create logic so that uid isn't required for lookup

* change offset to page

* regen swagger

* revert accidental changes to file

* curl command page should be 1 indexed
2024-06-24 23:50:07 -04:00
Michael Mandrus
8a8f97b0e4 CloudMigrations: Implement snapshot management apis (#89296)
* add new apis

* add payloads

* create snapshot status type

* add some impl

* finish implementing update

* start implementing build snapshot func

* add more fake build logic

* add cancel endpoint. do some cleanup

* implement GetSnapshot

* implement upload snapshot

* merge onprem status with gms result

* get it working

* update comment

* rename list endpoint

* add query limit and offset

* add helper method to snapshot

* little bit of cleanup

* work on swagger annotations

* manual merge

* generate swagger specs

* clean up curl commands

* fix bugs found during final testing

* fix linter issue

* fix unit test
2024-06-19 09:20:52 -04:00
Dave Henderson
6262c56132 chore(perf): Pre-allocate where possible (enable prealloc linter) (#88952)
* chore(perf): Pre-allocate where possible (enable prealloc linter)

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* fix TestAlertManagers_buildRedactedAMs

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* prealloc a slice that appeared after rebase

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-06-14 14:16:36 -04:00
Sofia Papagiannaki
63c4d7373a Update cloud_migrations.go (#89218)
Remove unnecessary newline
2024-06-14 15:14:37 +03:00
Michael Mandrus
9d3a4e236d CloudMigrations: Refactor API for async work (#89084)
* rename some stuff

* more renaming

* clean up api

* rename more functions

* rename cms -> gms

* update comment

* update swagger gen

* update endpoints

* overzealous

* final touches

* dont modify existing migrations

* break structs into domain and dtos

* add some conversion funcs

* fix build

* update frontend

* try to make swagger happy
2024-06-13 17:58:59 +00:00
Eric Leijonmarck
39bd13f877 User: add second migration for lowercasing login/email of users (#88915) 2024-06-07 15:07:14 +02:00
Ezequiel Victorero
42d75ac737 Dashboards: Add feature restore dashboards backend (#83131)
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2024-05-16 14:36:26 -03:00
Andres Martinez Gotor
d8904f3ca4 Add apiVersion to plugin models (#87510) 2024-05-14 13:58:27 +02:00
Alexander Weaver
36ef611cf4 Alerting: Add database migration for recording rule fields (#87012)
* Create recording rule fields in model

* Add migration

* Write to database, support in version table

* extend fingerprint

* Force fields to be empty on validate

* Another storage spot, tests for fingerprint

* Explicitly set defaults in provisioning API

* Tests for main API validation

* Add diff tests even though fields are unpopulated for now

* Use struct tag approach instead of FromDB/ToDB hooks as it better handles nulls when deserializing

* test for deser

* Backout RecordTo for now since it's not decided in the doc

* back out of migration too

* Drop datasourceref for now

* address linter complaints

* Try a single outer struct with all fields embedded
2024-05-09 12:12:44 -05:00
lean.dev
0719f73f35 Cloud Migration: Remove ID migration in favor of UID (#86550)
* - Added migration to add a new collumn UID to both migration and migration_run tables.
- Added migration to set UID for records already existent in the database before adding a new column.
- Added UID field to CloudMigration and CloudMigrationRun dtos (model.go)

* Fix db migration

* Updating store and model

* Updating API to use UID

* fix typo

* update openapi specs and generated endpoints

* fix spec

* update openapi specs and generated endpoints

* Fixing db mapping

* Fix frontend

* fix migration

* remove migration uid from run fetch endpoint

* Revert "merge"

This reverts commit d654e4c530, reversing
changes made to 5fe0b483eb.

* manual merge

* rename some funcs for consistency

* make interfaces consistent

* validate uids

* update generated frontend api

* fix enterprise spec

* manually resolve api

* try again

* try yet again

* once more

---------

Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: joshhunt <josh@trtr.co>
2024-05-01 12:29:25 -04:00
Eric Leijonmarck
e394e16073 Auth: Force lowercase login/email for users (#86359)
* [WIP]: Force lowercase login/email for user CRUD

* warn and remove use of userCaseInsensitiveLogin check

* remove log warning

* reimplementation of the caseinsensitive

* need to decide if we want the conflict check or not

* remvoved the tests for conflict user by getEmail, getLogin

* added tests for user lowercase migration

* wip: emails next

* tests for email lowercasing

* review comments

* optimized login and email lookup before migrating
2024-04-25 17:31:17 +01:00
Yuri Tseretyan
12605bfed2 Alerting: Update fixed roles to include silences permissions (#85826)
* update fixed roles to include silences
* add silence actions to managed permissions
* update documentation
2024-04-12 12:37:34 -04:00
idafurjes
5c4a2de59b Add FolderUID for library elements (#83819)
* Revert "Revert "Add FolderUID for library elements" (#83776)"

This reverts commit 0dfdb2ae47.

* Fix bug, dashboard id and library element fodler_id are the corresponding values
Dashboard table hold both dahboards and tables
2024-04-09 12:27:43 +02:00
Mikel Vuka
724517dc40 Provisioning: datasources auto deletion (#83034) 2024-04-08 11:45:39 +02:00
Leonard Gram
383ebb2bc4 Cloudmigration: create migration (#85386)
* Cloudmigration: create migration

* fix drone build

* lint fix

* fix unit test

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2024-03-29 01:55:27 +01:00
Sofia Papagiannaki
33b653534e SQLStore: Enable migration locking by default (#84983)
* Introduce new configuration for migration locking

* Remove feature toggle

* Fix test and turn it into an integration

* Fix docs
2024-03-22 21:22:29 +02:00
Eric Leijonmarck
6c8895e349 Service accounts: Same Org fix migration to account for duplicate entries (#84349)
* bug: fix migration to account for duplicate entries

* refactoring to create test folder for user migrations

* fix migration log

* added the migration

* additional tests

* added extSrv tests
2024-03-15 15:00:25 +01:00
Leonard Gram
f15aa2d30d CloudMigrations: draft of cloud migration tables (#83990)
* CloudMigrations: draft of cloud migration tables

* naming migration rules

* removes feature flag for migration

* tweaked db types

* removed comment
2024-03-12 15:44:10 +01:00
Eric Leijonmarck
e611a736ed Serviceaccounts: Add ability to add samename SA for different orgs (#83893)
* add ability to add samename SA for different orgs

* Update pkg/services/user/userimpl/user.go

* fix tests

* refactor name

* removed tests

* add migration

* fix linting
2024-03-06 09:53:58 +01:00
owensmallwood
0dfdb2ae47 Revert "Add FolderUID for library elements" (#83776)
Revert "Add FolderUID for library elements (#79572)"

This reverts commit 2532047e7a.
2024-03-01 18:16:43 +02:00
idafurjes
2532047e7a Add FolderUID for library elements (#79572)
* Add FolderUID in missing places for libraryelements

* Add migration for FolderUID in library elements table

* Add Folder UIDs tolibrary panels

* Adjust dashboard import with folder uid

* Fix lint

* Rename back FolderUID to UID

* Remove default

* Check if folderUID is nil

* Add unique indes on org_id,folder_uid,name and kind

* Update pkg/services/libraryelements/database.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Fix folder integration test, with unique index on library elements

* Make folder uids nullable and rewrite migration query

* Use dashboard uid instead of folder_uid

* Adjust test

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2024-03-01 10:16:33 +01:00
Gabriel MABILLE
80d6bf6da0 AuthN: Remove embedded oauth server (#83146)
* AuthN: Remove embedded oauth server

* Restore main

* go mod tidy

* Fix problem

* Remove permission intersection

* Fix test and lint

* Fix TestData test

* Revert to origin/main

* Update go.mod

* Update go.mod

* Update go.sum
2024-02-26 11:29:09 +01:00
Ieva
65534e62a6 RBAC: add kind, attribute and identifier to annotation permissions during the migration (#83299)
add kind, attribute and identifier to annotation permissions during the migration
2024-02-23 16:03:23 +00:00
Sofia Papagiannaki
38e8c62972 Folders: Switch order of the columns in folder table indexes so that org_id becomes first (#82454)
* Folders: Switch order of the columns in folder table so that org_id becomes first
2024-02-16 21:10:46 +02:00
Ieva
7343102d59 RBAC: Migration to remove the scope from permissions where action is alert.instances:read (#82202)
* add a migration to remove the scope from any permissions where action is alert.instances:read

* linting
2024-02-16 11:52:43 +00:00
Yuri Tseretyan
1eebd2a4de Alerting: Support for simplified notification settings in rule API (#81011)
* Add notification settings to storage\domain and API models. Settings are a slice to workaround XORM mapping
* Support validation of notification settings when rules are updated

* Implement route generator for Alertmanager configuration. That fetches all notification settings.
* Update multi-tenant Alertmanager to run the generator before applying the configuration.

* Add notification settings labels to state calculation
* update the Multi-tenant Alertmanager to provide validation for notification settings

* update GET API so only admins can see auto-gen
2024-02-15 09:45:10 -05:00
Dan Cech
790e1feb93 Chore: Update test database initialization (#81673)
* streamline initialization of test databases, support on-disk sqlite test db

* clean up test databases

* introduce testsuite helper

* use testsuite everywhere we use a test db

* update documentation

* improve error handling

* disable entity integration test until we can figure out locking error
2024-02-09 09:35:39 -05:00
Ryan McKinley
9c9e5e68c8 User: Add uid colum to user table (#81615) 2024-02-01 18:14:10 -08:00
Ryan McKinley
1fab107e79 FeatureFlags: Avoid using cfg.IsFeatureToggleEnabled (#81407) 2024-01-28 15:22:45 -08:00
Ieva
048d1e7c86 RBAC: Annotation permission migration (#78899)
* add annotation permissions to dashboard managed role and add migrations for annotation permissions

* fix a bug with conditional access level definitions

* add tests

* Update pkg/services/sqlstore/migrations/accesscontrol/dashboard_permissions.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* apply feedback

* add batching, fix tests and a typo

* add one more test

* undo unneeded change

* undo unwanted change

* only check the default basic permissions for non-OSS instances

* account for all wildcards and simplify the check a bit

* error handling and extra conditionals to avoid test failures

* fix a bug with admin permissions not appearing for folders

* fix the OSS check

---------

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2024-01-26 17:17:29 +00:00
Matthew Jacobson
e45f664ca4 Alerting: Replace index role_id, action, scope with action, scope, role_id on permission table (#80336)
* Alerting: Add action, scope, role_id to permission table

The existing role_id, action, scope index has the wrong ordering to be most
effectively used in dashboard/folder permission requests.

On a large tests set, the slow database calls were on the order of ~30-40ms, so
when performed individually they don't have that large of a latency impact.
However, when done in bulk in the migration this adds up to some very slow
requests.

After the index is added these same database calls are reduced to ~4-5ms

* Change index to action, scope, role_id

* Make new index unique and drop [role_id, action, scope] index
2024-01-24 16:18:14 -05:00
Gabriel MABILLE
63679813b0 RBAC: prevent seeding oncall access (#80862)
* RBAC: prevent seeding oncall access

* Add comments and an early exit

* Test SeedAssignmentOnCallAccessMigrator

* imports

* Comment rework

* Check error

* Nit.
2024-01-24 09:23:40 +01:00
Marcus Efraimsson
6768c6c059 Chore: Remove public vars in setting package (#81018)
Removes the public variable setting.SecretKey plus some other ones. 
Introduces some new functions for creating setting.Cfg.
2024-01-23 12:36:22 +01:00
Matthew Jacobson
5ecc7dd2fa Alerting: Increase size of kvstore value type for MySQL to LONGTEXT (#80331)
* Alerting: Increase size of kvstore value type for MySQL to LONGTEXT

alertmanager uses the kvstore to persist its notification log and the current
column limit for MySQL (16.7mb) puts the maximum entries at a level that is
potentially achievable for heavy alerting users (~40-80k entries).

In comparison, the current type for PSQL (TEXT) is effectively unlimited and
I believe SQLIte defaults to 2gb which is also plenty of leeway.
2024-01-11 22:51:36 +02:00
Gabriel MABILLE
c9ac069076 RBAC: Add origin column to seed_assignment (#80326)
* RBAC: Add origin column to seed_assignment

* Add OnCall permission migration
2024-01-11 18:43:43 +01:00
Yuri Tseretyan
f6a46744a6 Alerting: Support hysteresis command expression (#75189)
Backend: 

* Update the Grafana Alerting engine to provide feedback to HysteresisCommand. The feedback information is stored in state.Manager as a fingerprint of each state. The fingerprint is persisted to the database. Only fingerprints that belong to Pending and Alerting states are considered as "loaded" and provided back to the command.
   - add ResultFingerprint to state.State. It's different from other fingerprints we store in the state because it is calculated from the result labels.
  -  add rule_fingerprint column to alert_instance
   - update alerting evaluator to accept AlertingResultsReader via context, and update scheduler to provide it.
   - add AlertingResultsFromRuleState that implements the new interface in eval package
   - update getExprRequest to patch the hysteresis command.

* Only one "Recovery Threshold" query is allowed to be used in the alert rule and it must be the Condition.


Frontend:

* Add hysteresis option to Threshold in UI. It's called "Recovery Threshold"
* Add test for getUnloadEvaluatorTypeFromCondition
* Hide hysteresis in panel expressions

* Refactor isInvalid and add test for it
* Remove unnecesary React.memo
* Add tests for updateEvaluatorConditions

---------

Co-authored-by: Sonia Aguilar <soniaaguilarpeiron@gmail.com>
2024-01-04 11:47:13 -05:00
Serge Zaitsev
06d2ae3ada Chore: Move folder sync logic into one-shot migration (#78985)
Chore: move folder sync code into one-shot migration
2023-12-12 17:13:47 +01:00
Matthew Jacobson
6644e5e676 Alerting: Fix migration that is brittle to version downgrades (#78976) 2023-12-01 15:18:41 -05:00
Matthew Jacobson
cdad712547 Alerting: Keep track of individual org migration status (#78369)
* Alerting: Keep track of individual org migration status

Save migration status per migrated org.

Change the meaning (and key/value) of the org_id=0 entry 
to store the current (previous) config value used by alerting. 
This is so we can know when to upgrade/downgrade by 
comparing with the new config value in 
UnifiedAlerting.IsEnabled.
2023-11-30 10:25:59 -05:00
Jo
0de66a8099 Authz: Remove use of SignedInUser copy for permission evaluation (#78448)
* remove use of SignedInUserCopies

* add extra safety to not cross assign permissions

unwind circular dependency

dashboardacl->dashboardaccess

fix missing import

* correctly set teams for permissions

* fix missing inits

* nit: check err

* exit early for api keys
2023-11-22 14:20:22 +01:00
Ryan McKinley
f69fd3726b FeatureToggles: Add context and and an explicit global check (#78081) 2023-11-14 12:50:27 -08:00
Ryan McKinley
dec9a07738 Settings: Actually deprecate access to feature flags (#78073) 2023-11-13 11:39:01 -08:00
Ryan McKinley
3509a5abb9 FeatureFlags: Cleanup usage of cfg.IsFeatureToggleEnabled (#78014) 2023-11-13 07:55:15 -08:00
Misi
5285e9503b Auth: SSO settings foundations (#77724)
* inital changes, db migration

* changes

* Implement basic GetAll, Delete

* Add first batch of tests

* Add more tests

* Add service tests for GetForProvider, List

* Update http_server.go + wire.go

* Lint + update fixed role

* Update CODEOWNERS

* Change API init

* Change roles, rename

* Review with @kalleep

* Revert a mistakenly changed part

* Updates based on @dmihai 's feedback

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-11-08 10:50:01 +01:00
Sofia Papagiannaki
03a626f1d6 Search: Fix empty folder details for nested folder items (#76504)
* Introduce dashboard.folder_uid column

* Add data migration

* Search: Fix empty folder details for nested folders

* Set `dashboard.folder_uid` and update tests

* Add unique index

* lint

Ignore cyclomatic complexity of func
`(*DashboardServiceImpl).BuildSaveDashboardCommand

* Fix search by folder UID
2023-10-24 10:04:45 +03:00
Matthew Jacobson
82f3127e23 Alerting: Move legacy alert migration from sqlstore migration to service (#72702) 2023-10-12 13:43:10 +01:00
kay delaney
a12cb8cbf3 LibraryPanels: Add RBAC support (#73475) 2023-10-12 00:30:50 +01:00
Alexander Weaver
f6649d7a97 Revert "Alerting: Remove vendored models in migration service" (#76387)
Revert "Alerting: Remove vendored models in migration service (#74503)"

This reverts commit 6a8649d544.
2023-10-11 14:21:21 -05:00