Commit Graph

1984 Commits

Author SHA1 Message Date
Kristin Laemmert
daedb358dd chore (replstore): fix registration with multiple sql drivers, again (#90990)
* replstore: fix registration with multiple sql drivers, again
* only compile regex once
2024-07-25 15:13:36 -04:00
Ryan McKinley
9db3bc926e Identity: Rename "namespace" to "type" in the requester interface (#90567) 2024-07-25 12:52:14 +03:00
Kristin Laemmert
27b52b1507 ReplStore: Add support for round robin load balancing between multiple read replicas (#90530)
* ReplStore: Add support for multiple replicas and round-robin load balancing

* add check for zero-length repls list
2024-07-18 08:20:28 -04:00
Serge Zaitsev
f5da9edbba Chore: Remove unused method in db.DB (#90433)
* remove unused method

* clean up tests
2024-07-17 08:17:23 +02:00
Piotr Jamróz
6750e881e3 Query History: Use a search index on new queries to filter in mixed data sources (#88979)
* Add search index table

* Stab a test

* Add more tests

* Add basic index

* Switch to UID and add a test for the index

* Improve tests coverage

* Remove redundant whitespaces

* Load all data source APIs when query history is loaded

* Fix column type

* Fix migration

* Clean-up the index

* Fix linting

* Fix migrations

* Fix migrations

* Fix migrations

* Rename index to details
2024-07-16 11:47:21 +02:00
Matthew Jacobson
ba800692c6 Alerting: Persist AlertInstance ResolvedAt & LastSentAt (#89135)
* Alerting: Persist AlertInstance ResolvedAt & LastSentAt

* Fix test

* Modify existing tests

* Fix merge conflicts from nullable LastSentAt & ResolvedAt
2024-07-12 12:26:58 -04:00
Kristin Laemmert
8a6107cd35 DashboardStore: Use ReplDB and get dashboard quotas from the ReadReplica (#90235)
* Use ReplDB in dashboard store and update all fixtures - no other changes

* just moving dashboard counts for now

* find the missing test fixture
2024-07-12 10:47:49 -04:00
Kristin Laemmert
77a4869fca accesscontrol service read replica (#89963)
* accesscontrol service read replica
* now using the ReplDB interface
* ReadReplica for GetUser
2024-07-08 10:00:13 -04:00
Ieva
e9ebb6eaa4 Folders: Fix folder pagination for cloud instances with many folders (#90008)
* filter the k6 folder out in the SQL queries rather than during post processing to ensure that the correct number of results is always returned

* linting
2024-07-05 11:19:03 +01:00
Karl Persson
bfe77ab530 Users: Ensure default admin is created with a valid uid (#89981)
Users: Ensure default admin has a valid uid
2024-07-03 12:06:10 +02:00
Jeff Levin
cfe8317d45 Add auth spans and remove deduplication code for scopes (#89804)
Adds more spans for timing in accesscontrol and remove permission deduplicating code after benchmarking

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
2024-07-02 22:08:57 -08:00
Michael Mandrus
4d69213829 CloudMigrations: Break snapshot resources out into their own table (#89575)
* create a new table for migration resources

* remove raw result bytes from db

* more snapshot resource management stuff

* integrate new table with snapshots

* pass in result limit and offset as params

* combine create and update

* set up xorm store test

* add unit tests

* save some cpu

* remove unneeded arg

* regen swagger

* fix bug with result processing

* fix update create logic so that uid isn't required for lookup

* change offset to page

* regen swagger

* revert accidental changes to file

* curl command page should be 1 indexed
2024-06-24 23:50:07 -04:00
Kristin Laemmert
ab5a079bcc Fix: add mapping for the new mysqlRepl driver (#89551) 2024-06-21 08:35:46 -04:00
Kristin Laemmert
d988f5c3b0 fix: sql instrumentation dual registration error (#89508)
fix dual registration error
2024-06-20 14:45:39 -04:00
Michael Mandrus
8a8f97b0e4 CloudMigrations: Implement snapshot management apis (#89296)
* add new apis

* add payloads

* create snapshot status type

* add some impl

* finish implementing update

* start implementing build snapshot func

* add more fake build logic

* add cancel endpoint. do some cleanup

* implement GetSnapshot

* implement upload snapshot

* merge onprem status with gms result

* get it working

* update comment

* rename list endpoint

* add query limit and offset

* add helper method to snapshot

* little bit of cleanup

* work on swagger annotations

* manual merge

* generate swagger specs

* clean up curl commands

* fix bugs found during final testing

* fix linter issue

* fix unit test
2024-06-19 09:20:52 -04:00
Kristin Laemmert
50244ed4a1 Experimental Feature Toggle: databaseReadReplica (#89232)
This adds a version of the SQLStore that includes a ReadReplica. The primary DB can be accessed directly - from the caller's standpoint, there is no difference between the SQLStore and ReplStore unless they wish to explicitly call the ReadReplica() and use that for the DB sessions.

Currently only the stats service GetSystemStats and GetAdminStats are using the ReadReplica(); if it's misconfigured or if the databaseReadReplica feature flag is not turned on, it will fall back to the usual (SQLStore) behavior.

Testing requires a database and read replica - the replication should already be configured. I have been testing this locally with a docker mysql setup (https://medium.com/@vbabak/docker-mysql-master-slave-replication-setup-2ff553fceef2) and the following config:

[feature_toggles]
databaseReadReplica = true

[database]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3306

[database_replica]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3307
2024-06-18 11:07:15 -04:00
Dave Henderson
6262c56132 chore(perf): Pre-allocate where possible (enable prealloc linter) (#88952)
* chore(perf): Pre-allocate where possible (enable prealloc linter)

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* fix TestAlertManagers_buildRedactedAMs

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* prealloc a slice that appeared after rebase

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-06-14 14:16:36 -04:00
Sofia Papagiannaki
63c4d7373a Update cloud_migrations.go (#89218)
Remove unnecessary newline
2024-06-14 15:14:37 +03:00
Michael Mandrus
9d3a4e236d CloudMigrations: Refactor API for async work (#89084)
* rename some stuff

* more renaming

* clean up api

* rename more functions

* rename cms -> gms

* update comment

* update swagger gen

* update endpoints

* overzealous

* final touches

* dont modify existing migrations

* break structs into domain and dtos

* add some conversion funcs

* fix build

* update frontend

* try to make swagger happy
2024-06-13 17:58:59 +00:00
Ieva
3853f90528 RBAC: Include action sets in dashboard and folder permission filter (#89133)
take action sets into account in dashboard and folder permission filter
2024-06-13 19:40:47 +03:00
Ryan McKinley
99d8025829 Chore: Move identity and errutil to apimachinery module (#89116) 2024-06-13 07:11:35 +03:00
Ieva
35d0597367 RBAC: Only check for the write action when listing editable dashboards/folders (#88518)
* only check for the write action when listing editable resources

* test fix
2024-06-10 14:44:34 +03:00
Eric Leijonmarck
39bd13f877 User: add second migration for lowercasing login/email of users (#88915) 2024-06-07 15:07:14 +02:00
Dave Henderson
e21d357d67 chore: Updating linter, fixing some new warnings (#88863) 2024-06-06 13:01:27 -04:00
Ezequiel Victorero
42d75ac737 Dashboards: Add feature restore dashboards backend (#83131)
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2024-05-16 14:36:26 -03:00
Andres Martinez Gotor
d8904f3ca4 Add apiVersion to plugin models (#87510) 2024-05-14 13:58:27 +02:00
Alexander Weaver
36ef611cf4 Alerting: Add database migration for recording rule fields (#87012)
* Create recording rule fields in model

* Add migration

* Write to database, support in version table

* extend fingerprint

* Force fields to be empty on validate

* Another storage spot, tests for fingerprint

* Explicitly set defaults in provisioning API

* Tests for main API validation

* Add diff tests even though fields are unpopulated for now

* Use struct tag approach instead of FromDB/ToDB hooks as it better handles nulls when deserializing

* test for deser

* Backout RecordTo for now since it's not decided in the doc

* back out of migration too

* Drop datasourceref for now

* address linter complaints

* Try a single outer struct with all fields embedded
2024-05-09 12:12:44 -05:00
Dan Cech
41bee274fd Chore: Fix error handling in postDashboard, remove UserDisplayDTO, fix live redis client initialization (#87206)
* clean up error handling in postDashboard and remove UserDisplayDTO

* replace GetUserUID with GetUID and GetNamespacedUID, enforce namespace constant type

* lint fix

* lint fix

* more lint fixes
2024-05-06 14:17:34 -04:00
Serge Zaitsev
ad5613d7d4 Chore: Remove cfg from folder service (#87212)
remove cfg from folder service
2024-05-02 13:18:54 +02:00
lean.dev
0719f73f35 Cloud Migration: Remove ID migration in favor of UID (#86550)
* - Added migration to add a new collumn UID to both migration and migration_run tables.
- Added migration to set UID for records already existent in the database before adding a new column.
- Added UID field to CloudMigration and CloudMigrationRun dtos (model.go)

* Fix db migration

* Updating store and model

* Updating API to use UID

* fix typo

* update openapi specs and generated endpoints

* fix spec

* update openapi specs and generated endpoints

* Fixing db mapping

* Fix frontend

* fix migration

* remove migration uid from run fetch endpoint

* Revert "merge"

This reverts commit d654e4c530, reversing
changes made to 5fe0b483eb.

* manual merge

* rename some funcs for consistency

* make interfaces consistent

* validate uids

* update generated frontend api

* fix enterprise spec

* manually resolve api

* try again

* try yet again

* once more

---------

Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: joshhunt <josh@trtr.co>
2024-05-01 12:29:25 -04:00
Eric Leijonmarck
e394e16073 Auth: Force lowercase login/email for users (#86359)
* [WIP]: Force lowercase login/email for user CRUD

* warn and remove use of userCaseInsensitiveLogin check

* remove log warning

* reimplementation of the caseinsensitive

* need to decide if we want the conflict check or not

* remvoved the tests for conflict user by getEmail, getLogin

* added tests for user lowercase migration

* wip: emails next

* tests for email lowercasing

* review comments

* optimized login and email lookup before migrating
2024-04-25 17:31:17 +01:00
Alexander Zobnin
60ed6bfc33 Search: Fix slow query when user does not have roles assigned (#86791)
* Search: Fix slow query when user does not have roles assigned

* Check all required actions and skip if not found
2024-04-24 13:23:08 +03:00
Serge Zaitsev
522a98c126 Chore: Make Cfg field private in SQLStore (#85593)
* make cfg private in sqlstore

* fix db init in tests

* fix case

* fix folder test init

* fix imports

* make another Cfg private

* remove another Cfg

* remove unused variable

* use store cfg, it has side-effects

* fix mutated cfg in tests
2024-04-24 10:38:40 +02:00
Kristin Laemmert
03b795844c SQLStore: Improve recursive CTE support detection (#86397)
sqlstore: improve recursive CTE support detection

Vitess returns a not supported error, not a parse error

Co-authored-by: Derek Perkins <derek@nozzle.io>
2024-04-17 08:37:47 -04:00
Yuri Tseretyan
12605bfed2 Alerting: Update fixed roles to include silences permissions (#85826)
* update fixed roles to include silences
* add silence actions to managed permissions
* update documentation
2024-04-12 12:37:34 -04:00
idafurjes
5c4a2de59b Add FolderUID for library elements (#83819)
* Revert "Revert "Add FolderUID for library elements" (#83776)"

This reverts commit 0dfdb2ae47.

* Fix bug, dashboard id and library element fodler_id are the corresponding values
Dashboard table hold both dahboards and tables
2024-04-09 12:27:43 +02:00
Dan Cech
5dfe4cf407 Storage: Watch tests (#85496)
* basic watch tests working

* refactor to read previous event within poller

* add watch test files

* cleanup

* watch tests passing

* debug cleanup

* special handling for canceled context

* cleanup

* fix wire

* fix sqlite_sequence cleanup issue

* move watch tests to integration, wait for provisioned dashboards

* handle context deadline exceeded and eof errors

* add comment about sleep
2024-04-08 11:42:12 -04:00
Mikel Vuka
724517dc40 Provisioning: datasources auto deletion (#83034) 2024-04-08 11:45:39 +02:00
Serge Zaitsev
faa1244518 Chore: Replace sqlstore with db interface (#85366)
* replace sqlstore with db interface in a few packages

* remove from stats

* remove sqlstore in admin test

* remove sqlstore from api plugin tests

* fix another createUser

* remove sqlstore in publicdashboards

* remove sqlstore from orgs

* clean up orguser test

* more clean up in sso

* clean up service accounts

* further cleanup

* more cleanup in accesscontrol

* last cleanup in accesscontrol

* clean up teams

* more removals

* split cfg from db in testenv

* few remaining fixes

* fix test with bus

* pass cfg for testing inside db as an option

* set query retries when no opts provided

* revert golden test data

* rebase and rollback
2024-04-04 15:04:47 +02:00
Leonard Gram
383ebb2bc4 Cloudmigration: create migration (#85386)
* Cloudmigration: create migration

* fix drone build

* lint fix

* fix unit test

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2024-03-29 01:55:27 +01:00
Karl Persson
152cb47692 AuthN: Add IsAuthenticatedBy to identity interface and replace checks (#85262)
Add IsAuthenticatedBy to identity interface and replace checks
2024-03-27 15:22:13 +01:00
Sofia Papagiannaki
33b653534e SQLStore: Enable migration locking by default (#84983)
* Introduce new configuration for migration locking

* Remove feature toggle

* Fix test and turn it into an integration

* Fix docs
2024-03-22 21:22:29 +02:00
Eric Leijonmarck
2f7fd729ef Auth: Enable case insensitive logins/emails by default (#84840)
* wip

* wip

* wip

* wip postgres tests
2024-03-22 16:45:18 +01:00
Eric Leijonmarck
6c8895e349 Service accounts: Same Org fix migration to account for duplicate entries (#84349)
* bug: fix migration to account for duplicate entries

* refactoring to create test folder for user migrations

* fix migration log

* added the migration

* additional tests

* added extSrv tests
2024-03-15 15:00:25 +01:00
Leonard Gram
f15aa2d30d CloudMigrations: draft of cloud migration tables (#83990)
* CloudMigrations: draft of cloud migration tables

* naming migration rules

* removes feature flag for migration

* tweaked db types

* removed comment
2024-03-12 15:44:10 +01:00
Sofia Papagiannaki
22d8258e48 Postgres: Allow disabling SNI on SSL-enabled connections (#83892)
* Postgres: Allow disabling SNI on SSL-enabled connections

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

---------

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2024-03-12 12:03:24 +02:00
Eric Leijonmarck
e611a736ed Serviceaccounts: Add ability to add samename SA for different orgs (#83893)
* add ability to add samename SA for different orgs

* Update pkg/services/user/userimpl/user.go

* fix tests

* refactor name

* removed tests

* add migration

* fix linting
2024-03-06 09:53:58 +01:00
owensmallwood
0dfdb2ae47 Revert "Add FolderUID for library elements" (#83776)
Revert "Add FolderUID for library elements (#79572)"

This reverts commit 2532047e7a.
2024-03-01 18:16:43 +02:00
idafurjes
2532047e7a Add FolderUID for library elements (#79572)
* Add FolderUID in missing places for libraryelements

* Add migration for FolderUID in library elements table

* Add Folder UIDs tolibrary panels

* Adjust dashboard import with folder uid

* Fix lint

* Rename back FolderUID to UID

* Remove default

* Check if folderUID is nil

* Add unique indes on org_id,folder_uid,name and kind

* Update pkg/services/libraryelements/database.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Fix folder integration test, with unique index on library elements

* Make folder uids nullable and rewrite migration query

* Use dashboard uid instead of folder_uid

* Adjust test

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2024-03-01 10:16:33 +01:00
Gabriel MABILLE
80d6bf6da0 AuthN: Remove embedded oauth server (#83146)
* AuthN: Remove embedded oauth server

* Restore main

* go mod tidy

* Fix problem

* Remove permission intersection

* Fix test and lint

* Fix TestData test

* Revert to origin/main

* Update go.mod

* Update go.mod

* Update go.sum
2024-02-26 11:29:09 +01:00