Commit Graph

1645 Commits

Author SHA1 Message Date
Jo
062d255124 Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
Serge Zaitsev
6e4b537ba6 Chore: Remove datasources from the Store interface (#53515) 2022-08-10 15:04:51 +02:00
idafurjes
6afad51761 Move SignedInUser to user service and RoleType and Roles to org (#53445)
* Move SignedInUser to user service and RoleType and Roles to org

* Use go naming convention for roles

* Fix some imports and leftovers

* Fix ldap debug test

* Fix lint

* Fix lint 2

* Fix lint 3

* Fix type and not needed conversion

* Clean up messages in api tests

* Clean up api tests 2
2022-08-10 11:56:48 +02:00
Karl Persson
aa484a60c9 RBAC: Fix dashboard filter in SQLBuilder (#53379)
* Reuse DasbhoardPermissionFilter

*  Use rbac dashboard filter if enabled
2022-08-10 10:32:03 +02:00
ying-jeanne
ba89471598 Chore: Enable ANSI_QUOTES for Mysql Database (#53277)
* remove the quote dialect

* add environment variable
2022-08-08 12:14:17 -04:00
ying-jeanne
af09638a0d remove the sqlstore playlist methods (#53415) 2022-08-08 11:06:06 -05:00
Kat Yang
62b4dbf52f Chore: Remove pluginsettings methods from store and mockstore (#53347) 2022-08-05 12:26:39 -04:00
idafurjes
0c5b71694f Chore: Add user service method UpdateLastSeenAt (#53309)
* Chore: Add user service method UpdateLastSeenAt

* Fix lint
2022-08-04 15:44:14 +02:00
idafurjes
20f4191e56 Chore: Add user service method ChangePassword (#53303)
* Chore: Add user service method ChangePassword

* Fix lint
2022-08-04 15:05:05 +02:00
idafurjes
6fdb6ea6f6 Chore: Add user service method Update (#53300)
* Chore: Add user service method Update

* Remove UpdateUser from store interface
2022-08-04 14:22:44 +02:00
Serge Zaitsev
191ab3bb01 Chore: Move api key models into apikey service package (#53241)
* Chore: move api key models into apikey service package

* force table name for api key
2022-08-04 14:19:09 +02:00
idafurjes
af83a09a92 Chore: Add user service method GetByEmail (#53298) 2022-08-04 13:47:30 +02:00
idafurjes
1ecbe22751 Chore: Add user service method GetByLogin (#53204)
* Add wrapper around sqlstore method GetUserByLogin

* Use new method from user service

* Fix lint

* Fix lint 2

* fix middleware basic auth test

* Fix grafana login returning a user by login

* Remove GetUserByLogin from store interface

* Merge commit
2022-08-04 13:22:43 +02:00
idafurjes
66fd516c44 Remove GetUserById from store interface (#53226) 2022-08-04 10:34:05 +02:00
Sofia Papagiannaki
ae101bf935 Alerting: Fix migration (#53253) 2022-08-03 11:41:18 -04:00
Kyle Brandt
643d2bc890 Chore: Sql store split for legacy alerting (#52901)
Moves ~20 sqlstore methods for legacy alerting out of sqlstore (sqlstore.Store interface) and into alerting.
2022-08-03 11:17:26 -04:00
Serge Zaitsev
8d36f71673 Chore: Remove apikey store methods from sqlstore (#53197)
* Chore: remove apikey store methods from sqlstore

* remove GetAPIKeys

* remove GetAllAPIKeys

* remove the rest of apikey from sqlstore
2022-08-03 08:13:05 -04:00
Serge Zaitsev
64488f6b90 Chore: split APIKey store (#52781)
* move apikey store into a separate service

* add apikey service to wire graph

* fix linter

* switch api to use apikey service

* fix provideservice in tests

* add apikey service test double

* try different sql syntax

* rolling back the dialect

* trigger drone

* trigger drone
2022-08-02 16:55:19 +02:00
Alexander Emelin
e791a4e576 Search: Move entity events to transaction (#52394) 2022-08-01 18:56:36 +03:00
Artur Wierzbicki
449acb56a9 Previews: fix dashboard.thumbnails.ds_uids migration (#52960) 2022-07-28 21:50:02 +04:00
Artur Wierzbicki
18daa6754c Previews: datasource permissions (#52747)
* Previews: datasource permissions

* lint

* simplify - force non-null `ds_uids`

* add `canBeDisabled` to search service

* add `IncludeThumbnailsWithEmptyDsUids`

* remove force refresh migration

* refactor main preview service

* add safeguard

* revert ticker interval

* update testdata

* fix test

* add mock search service

* add datasources lookup test

* update migration

* extract ds lookup to its own package to avoid cyclic imports

* lint

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* remove stale log

* consistent casing

* pass context to `createServiceAccount`

* filter out the special grafana ds
2022-07-28 16:40:26 +04:00
Carl Bergquist
7825ad6aa3 Instrumentation: Collect database connection stats (#52797)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2022-07-27 13:42:28 +02:00
Ieva
b3a10202d4 Revert "Service accounts: Add service account to teams" (#52710)
* Revert "Service accounts: Add service account to teams (#51536)"

This reverts commit 0f919671e7.

* remove unneeded line

* fix test
2022-07-26 09:43:29 +01:00
Giordano Ricci
5ce4baf6f5 Correlations: Add CreateCorrelation HTTP API (#51630)
* Correlations: add migration

* Correlations: Add CreateCorrelation API

* Correlations: Make correlations work with provisioning

* Handle version changes

* Fix lining error

* lint fixes

* rebuild betterer results

* add a UID to each correlation

* Fix lint errors

* add docs

* better wording in API docs

* remove leftover comment

* handle ds updates

* Fix error message typo

* add bad data test

* make correlations a separate table

* skip readonly check when provisioning correlations

* delete stale correlations when datasources are deleted

* restore provisioned readonly ds

* publish deletion event with full data

* generate swagger and HTTP API docs

* apply source datasource permission to create correlation API

* Fix tests & lint errors

* ignore empty deletion events

* fix last lint errors

* fix more lint error

* Only publish deletion event if datasource was actually deleted

* delete DS provisioning deletes correlations, added & fixed tests

* Fix unmarshalling tests

* Fix linting errors

* Fix deltion event tests

* fix small linting error

* fix lint errors

* update betterer

* fix test

* make path singular

* Revert "make path singular"

This reverts commit 420c3d315e.

* add integration tests

* remove unneeded id from correlations table

* update spec

* update leftover references to CorrelationDTO

* fix tests

* cleanup tests

* fix lint error
2022-07-25 15:19:07 +01:00
Jean-Philippe Quéméner
50ae42130b Alerting: take datasources as external alertmanagers into consideration (#52534) 2022-07-20 16:50:49 +02:00
idafurjes
d3d8fdd878 Chore: Move user errors to user service (#52460)
* Move user not found err to user service

* User ErrCaseInsensitive from user pkg

* User ErrUserAlreadyExists from user pkg

* User ErrLastGrafanaAdmin from user pkg

* Remove errors from model
2022-07-20 14:50:06 +02:00
Ieva
39025bb4cd add logs to debug failing migration (#52447) 2022-07-19 09:34:49 -04:00
Joe Blubaugh
a0f96ed4e1 SQLStore: Support Upserting multiple rows. (#52228)
This will be used to reduce write load when the alerting system writes a
large number of events.
2022-07-19 16:42:48 +08:00
idafurjes
f5cace8bbd Rename Acl to ACL (#52342)
* Rename Acl to ACL

* Fix yaml files

* Add xorm tags and fix test
2022-07-18 15:14:58 +02:00
Artur Wierzbicki
b2736ac1fe Storage: limit the number of uploaded files (#50796)
* #50608: sql file upload quotas

* rename `files_in_sql` to `file`

* merge conflict
2022-07-18 15:24:39 +04:00
Sofia Papagiannaki
fb379ae436 Chore: Introduce playlist service (#52252)
* Store: Introduce playlist service

* Integrate playlist service

* Update swagger
2022-07-18 05:26:35 -04:00
Ieva
c851907fc3 Access Control: Fix missing folder permissions (#52153)
* add the migration

* Update pkg/services/sqlstore/migrations/accesscontrol/dashboard_permissions.go

Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>

Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
2022-07-14 12:01:21 -04:00
Sofia Papagiannaki
4ff0f006dd SQLstore: Fix fetching an inexistent playlist response (#51962)
* SQLstore: Fix fetching and deleting an inexistent playlist

xorm's session.Get() does not return an error if the raw does not exist.
It returns a boolean instead.
The playlist `sqlstore.GetPlaylist()` used to check only the error and in case
of inexistent UID didn't return an error.
2022-07-13 13:00:03 +03:00
Guilherme Caulada
2d8a91a846 Secrets: Improve unified secrets migration and implement compatibility flag (#50463)
* Implement disableSecretsCompatibility flag

* Allow secret deletion right after migration

* Use dialect.Quote for secure_json_data on secret deletion

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Set secure_json_data to NULL instead of empty json

* Run toggles_gen_test and use generated flag variable

* Add ID to delete data source secrets command on function call

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Remove extra query to get datasource on secret deletion

* Fix linting issues with CHANGELOG.md

* Use empty json string when deleting secure json data

* Implement secret migration as a background process

* Refactor secret migration as a background service

* Refactor migration to be inside secret store

* Re-add secret deletion function removed on merge

* Try using transaction to fix db lock during tests

* Disable migration for pipeline debugging

* Try adding sleep to fix database lock

* Remove unecessary time sleep from migration

* Fix merge issue, replace models with datasources

* Try event listener approach

* Fix merge issue, replace models with datasources

* Fix linting issues with unchecked error

* Remove unecessary trainling new line

* Increase wait interval on background secret migration

* Rename secret store migration folder for consistency

* Convert background migration to blocking

* Fix number of arguments on server tests

* Check error value of secret migration provider

* Fix linting issue with method varaible

* Revert unintended change on background services

* Move secret migration service provider to wire.go

* Remove unecessary else from datasource service

* Move transaction inside loop on secret migration

* Remove unecessary GetServices function

* Remove unecessary interface after method removal

* Rename Run to Migrate on secret migration interface

* Rename secret migrations service variable on server

* Use MustBool on datasource secret migration

* Revert changes to GetDataSources

* Implement GetAllDataSources function

* Remove DeleteDataSourceSecrets function

* Move datasource secret migration to datasource service

* Remove unecessary properties from datasource secret migration

* Make DecryptLegacySecrets a private method

* Remove context canceled check on secret migrator

* Log error when fail to unmarshal datasource secret

* Add necessary fields to update command on migration

* Handle high availability on secret migration

* Use kvstore for datasource secret migration status

* Add error check for migration status set on kvstore

* Remove NewSecretMigrationService from server tests

* Use const for strings on datasource secrets migration

* Test all cases for datasources secret migrations

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-07-12 17:27:37 -03:00
Matthew Jacobson
434e94ef2b Alerting: Update default route groupBy to [grafana_folder, alertname] (#50052)
* Alerting: Update default route groupBy to [grafana_folder, alertname]

Default group by for new routes and migrations is now [grafana_folder, alertname]
2022-07-11 12:24:43 -04:00
Jguer
9ab210a7d7 ServiceAccounts: Don't create new orgs for service accounts (#51819)
* Org: use constants for status codes

* ServiceAccounts: Avoid creating new orgs for service accounts

* Document createUserBehavior

* Update pkg/services/sqlstore/org_users_test.go

* add doc string to flag
2022-07-07 08:50:38 -04:00
Eric Leijonmarck
0f919671e7 Service accounts: Add service account to teams (#51536)
* Revert "Serviceaccounts: #48995

Do not display service accounts assigned to team (#48995)"

This reverts commit cbf71fbd7f.

* fix: test to not include more actions than necessary

* adding service accounts to teams - backend and frontend changes

* also support SA addition through the old team membership endpoints

* fix tests

* tests

* serviceaccounts permission tests

* serviceaccounts permission service tests run

* added back test that was removed by accident

* lint

* refactor: add testoptionsTeams

* fix a bug

* service account picker change

* explicitly set SA managed permissions to false for dash and folders

* lint

* allow team creator to list service accounts

Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
2022-07-06 05:34:36 -04:00
Ieva
75873d05d7 Access Control: Allow dashboard admins to query org users (#51652)
* allow dashboard admins to query org users

* rename one more variable
2022-07-04 10:43:06 +01:00
Sven Grossmann
b7e22c37a8 UsageStats: fixed elasticsearch version number to semver (#49054)
* UsageStats: fixed elasticsearch version number

- The version numbering was changed from plain numbers to a semver-ish approach

* added missing version assertion

* adapted tests
2022-07-04 11:08:01 +02:00
Kristin Laemmert
9de00c8eb2 chore/backend: move dashboard errors to dashboard service (#51593)
* chore/backend: move dashboard errors to dashboard service

Dashboard-related models are slowly moving out of the models package and into dashboard services. This commit moves dashboard-related errors; the rest will come in later commits.

There are no logical code changes, this is only a structural (package) move.

* lint lint lint
2022-06-30 09:31:54 -04:00
Jguer
6d0261263c ServiceAccounts: Add Service Account Token last used at date (#51446)
* ServiceAccounts Add api key last used at

* ServiceAccounts: LastUpdateAt tests
2022-06-28 16:42:40 +02:00
idafurjes
6c43eb0b4d Split Create User (#50502)
* Split Create User

* Use new create user and User from package user

* Add service to wire

* Making create user work

* Replace user from user pkg

* One more

* Move Insert to orguser Service/Store

* Remove unnecessary conversion

* Cleaunp

* Fix Get User and add fakes

* Fixing get org id for user logic, adding fakes and other adjustments

* Add some tests for ourguser service and store

* Fix insert org logic

* Add comment about deprecation

* Fix after merge with main

* Move orguser service/store to org service/store

* Remove orguser from wire

* Unimplement new Create user and use User from pkg user

* Fix wire generation

* Fix lint

* Fix lint - use only User and CrateUserCommand from user pkg

* Remove User and CreateUserCommand from models

* Fix lint 2
2022-06-28 14:32:25 +02:00
Emil Tullstedt
7d815a1db5 Alerting: Use google/uuid instead of gofrs/uuid (#51242) 2022-06-28 11:57:24 +02:00
Kristin Laemmert
945f015770 backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Jguer
0689c5839a Auth: Add option for case insensitive login (#49262)
* add case insensitive option

* treat id as case insensitive

* Users: Add integration tests for case insensitive querying

* Prefer config struct to global variable

* change key to case_insensitive_login

* impede conflicting users from logging in

* add tests for impeding user retrieval if conflicting

* nits and picks

Co-authored-by: gamab <gabi.mabs@gmail.com>

* Add check in transaction for conflicting user

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* add update tests

* skip on mysql

* add custom messages for user admin view

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* nit: extra else

* linting mistake

Co-authored-by: gamab <gabi.mabs@gmail.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2022-06-24 16:59:45 +02:00
Ezequiel Victorero
4c4d6fd425 PublicDashboards: collect stats for public dashboards (#50553)
* PublicDashboards: collect stats for public dashboards
2022-06-23 12:02:57 -03:00
owensmallwood
1802cc8aca Add correct table definition for migrations for old dashboard_public schema (#51292)
Co-authored-by: Jesse Weaver <jesse.weaver@grafana.com>
2022-06-22 15:35:33 -08:00
Jeff Levin
d076bedb5e public dashboards: finalize db schema & v1 feature complete (#50467)
This PR completes public dashboards v1 functionality and simplifies public dashboard conventions. It exists as a large PR so that we are not making constant changes to the database schema.

models.PublicDashboardConfig model replaced with models.PublicDashboard directly
dashboard_public_config table renamed to dashboard_public
models.Dashboard.IsPublic removed from the dashboard and replaced with models.PublicDashboard.isEnabled
Routing now uses a uuid v4 as an access token for viewing a public dashboard anonymously, PublicDashboard.Uid only used as database identifier
Frontend utilizes uuid for auth'd operations and access token for anonymous access
Default to time range defined on dashboard when viewing public dashboard
Add audit fields to public dashboard

Co-authored-by: Owen Smallwood <owen.smallwood@grafana.com>, Ezequiel Victorero <ezequiel.victorero@grafana.com>, Jesse Weaver <jesse.weaver@grafana.com>
2022-06-22 13:58:52 -08:00
gotjosh
90646e7f41 Alerting: Don't stop the migration when alert rule tags are invalid (#51253)
* Alerting: Don't stop the migration when alert rule tags are invalid

As we migrate we expect the `alertRuleTags` on a dashboard alert to be a JSON object. However, it seems this is not really validated by Grafana and an user can change the format to something else that the JSON parser is not able to marshal into a `map[string]string`.

Let's do a bit better by "attempting" to parse the tags and if we can't we'll simple return an empty map. The data is still there so if the user wishes they can go back, fix the data and attemp the migration again.
2022-06-22 17:39:17 +01:00
Yuriy Tseretyan
4d02f73e5f Alerting: Persist rule position in the group (#50051)
Migrations:
* add a new column alert_group_idx to alert_rule table
* add a new column alert_group_idx to alert_rule_version table
* re-index existing rules during migration

API:
* set group index on update. Use the natural order of items in  the array as group index
* sort rules in the group on GET
* update the version of all rules of all affected groups. This will make optimistic lock work in the case of multiple concurrent request touching the same groups.

UI:
* update UI to keep the order of alerts in a group
2022-06-22 10:52:46 -04:00