Commit Graph

553 Commits

Author SHA1 Message Date
George Robinson ed7d29f2b9 Alerting: Migrate old alerting templates to Go templates (#62911)
* Migrate old alerting templates to use $labels

* Fix imports

* Add test coverage and separate rewriting to Go templates

* Fix lint

* Check for additional closing braces

* Add logging of invalid message templates

* Fix tests

* Small fixes

* Update comments

* Panic on empty token

* Use logtest.Fake

* Fix lint

* Allow for spaces in variable names by not tokenizing spaces

* Add template function to deduplicate Labels in a Value map

* Fix behavior of mapLookupString

* Reference deduplicated labels in migrated message template

* Fix behavior of deduplicateLabelsFunc

* Don't create variable for parent logger

* Add more tests for deduplicateLabelsFunc

* Remove unused function

* Apply suggestions from code review

Co-authored by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>

* Give label val merge function better name

* Extract template migration and escape literal tokens

* Consolidate + simplify template migration

---------

Co-authored-by: William Wernert <william.wernert@grafana.com>
2023-10-02 11:25:33 -04:00
Ryan McKinley 0af7247612 Chore: cleanup old dashboardPreviews files (#75682) 2023-09-28 13:31:07 -07:00
Jo 40a1f8434d Anon: Scaffold anon service (#74744)
* remove API tagging method and authed tagging

* add anonstore

move debug to after cache

change test order

fix issue where mysql trims to second

* add old device cleanup

lint

utc-ize everything

trim whitespace

* remove dangling setting

* Add delete devices

* Move anonymous authnclient to anonimpl

* Add simple post login hook

* move registration of Background Service

cleanup

* add updated_at index

* do not untag device if login err

* add delete device integration test
2023-09-25 16:25:29 +02:00
Piotr Jamróz 946da57b6a Correlations: Allow creating correlations for provisioned data sources (#73737)
* Allow creating correlations for provisioned data sources

* Update docs

* Fix linting

* Add missing props

* Add missing props

* Fix linting

* Fix linting

* Clarify error name

* Removed error handling for a non-existing use case

* Create a list of deleted data datasources based on all configs

* Add org_id to correlations

* Add tests

* Allow org_id to be null in case org_id=0 is used

* Create organization to ensure stable id is generated

* Fix linting

* Ensure backwards compatibility

* Add deprecation information

* Update comments

* Override existing datasSource variable so the UID is retrieved correctly

* Migrate correlations indices

* Default org_id when migrating

* Remove redundant default

* Make PK non-nullable

* Post merge fixes

* Separate data sources / correlations provisioning

* Adjust comments

* Store new data sources in spy store so it can be used to test correlations as well

* Fix linting

* Update tests

* Ensure response is closed

* Avoid creating duplicates during provisioning

* Fix updating provisioned column and update tests

* Rename error message

* Fix linting errors

* Fix linting errors and rename variable

* Update test

* Update pkg/services/sqlstore/migrations/correlations_mig.go

Co-authored-by: Giordano Ricci <me@giordanoricci.com>

* Remove unused error

* Fix lining

---------

Co-authored-by: Giordano Ricci <me@giordanoricci.com>
2023-09-13 15:10:09 +02:00
Eric Leijonmarck b00f3216c1 Auth: Refactor for revoking user tokens within last hours (#74616)
* fix: revoked tokens within last hours

adds check for unlimited sessions out of index
adds a function for specifing the hours to look back when revoking users tokens, otherwise we "assume" the clean up takes care of them adds a index for the `user_auth_token` - `revoked_at` for faster queries when using `revoked_at`

* fix: sqllite datetime conversion with unixtimestamps

* fix: postgres dialect

* fix: mysql dialect

* fix: mysql dialect missing closing )

* refactor: delete revoked tokens directly

* fix: tests for sqlite

* AuthToken: Simplify DeleteUserRevokedTokens and add test

* fix: linting newline

* Reset get time after test

* fix: test order by revoked

* fix: order by different db

* ascending

* test with seen at

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-09-13 10:24:37 +01:00
Serge Zaitsev 58f6648505 Chore: capitalise messages for alerting (#74335) 2023-09-04 18:46:34 +02:00
Ryan McKinley 025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Piotr Jamróz b30e0aa5aa Correlations: Add organization id (#72258)
* Add org_id to correlations

* Add tests

* Allow org_id to be null in case org_id=0 is used

* Create organization to ensure stable id is generated

* Fix linting

* Ensure backwards compatibility

* Add deprecation information

* Migrate correlations indices

* Default org_id when migrating

* Remove redundant default

* Make PK non-nullable
2023-08-24 09:39:30 +02:00
Serge Zaitsev 7767ab6f43 Chore: Add folder data migration, fix unique index (#72602)
* add folder data migration, fix unique index

* fix unique index

* pass a fake store in tests

* pass store into other providers in tests

* and now with alerting!
2023-08-01 09:36:37 +02:00
Yuri Tseretyan 143683bd05 Alerting: Add more clear error to migration when rule cannot be parsed (#72374) 2023-07-27 09:47:04 -04:00
Matthew Jacobson c7eb7fb58a Alerting: Keep legacy alert rule maxDataPoints and intervalMs during migration (#71989)
Keep legacy alert rule max-data-points and interval during migration
2023-07-24 13:36:34 -04:00
Ieva cfa1a2c55f RBAC: Split non-empty scopes into kind, attribute and identifier fields for better search performance (#71933)
* add a feature toggle

* add the fields for attribute, kind and identifier to permission

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* set the new fields when new permissions are stored

* add migrations

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* remove comments

* Update pkg/services/accesscontrol/migrator/migrator.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* feedback: put column migrations behind the feature toggle, added an index, changed how wildcard scopes are split

* PR feedback: add a comment and revert an accidentally changed file

* PR feedback: handle the case with : in resource identifier

* switch from checking feature toggle through cfg to checking it through featuremgmt

* don't put the column migrations behind a feature toggle after all - this breaks permission queries from db

---------

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-07-21 15:23:01 +01:00
Matthew Jacobson 8c6cdf51fc Alerting: No longer silence paused alerts during legacy migration (#71596)
* Alerting: No longer silence paused alerts during legacy migration

Now that we migrate paused legacy alerts to paused UA alert rules, we no longer need to silence them.
2023-07-17 09:37:53 -04:00
Jo d6c468c1c2 Auth: Add empty role definition (#64694)
* Allow setting role as None

Co-authored-by: gamab <gabi.mabs@gmail.com>

Seeking for places where role.None would be used

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

Adding None role to the frontend

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

unify org role declaration and remove from add permission

fix backend test

fix backend lint

* remove role none from frontend

* Simplify checks

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* nits

---------

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
2023-07-06 15:40:06 +02:00
Matthew Jacobson 00d5f7fed7 Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration (#70781)
* Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration
2023-06-28 14:02:57 -04:00
Santiago ff9eff49bd Alerting: Bump grafana/alerting and refactor the ImageStore/Provider to provide image URL/bytes (#70182)
* implement alerting.images.Provider interface in our ImageStore

* add URLExists() method to fakeConfigStore

* make linter happy

* update integration tests
2023-06-21 20:53:30 -03:00
Dan Cech a6279b2d62 Database: Make dialects independent of xorm Engine (#69955)
* make dialects independent of xorm Engine

* goimports
2023-06-14 16:13:36 -04:00
Artur Wierzbicki 15f06a93b2 Storage: increase size of the contents column (#69152)
* fix size of contents column

* temporarily re-enable integration tests

* lint

* revert: temporarily re-enable integration tests
2023-05-30 21:11:04 +04:00
Alexander Weaver 0ed5d3bdf2 Revert "Alerting: Refactor the ImageStore/Provider to provide image URL/bytes" (#69265)
Revert "Alerting: Refactor the ImageStore/Provider to provide image URL/bytes (#67693)"

This reverts commit 72a187b0be.
2023-05-30 11:33:33 -05:00
Santiago 72a187b0be Alerting: Refactor the ImageStore/Provider to provide image URL/bytes (#67693)
* (WIP) Refactor the ImageStore interface to work with our latest alerting repository

* update alerting package

* refactor, new URLExists method in ImageProvider

* tests for the new methods

* fix linter warnings

* use alertingImages as an alias for grafana/alerting/images

* logs about image uris and not found images

* nerf image not found logs

* extract duplicated code to getImageFromURI() method

* refactor getImageFromURI()

* add index on url

* add comment about migration log

* sync generated files
2023-05-30 11:25:55 -03:00
Gabriel MABILLE edf1775d49 AuthN: Embed an OAuth2 server for external service authentication (#68086)
* Moving POC files from #64283 to a new branch

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Adding missing permission definition

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Force the service instantiation while client isn't merged

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Merge conf with main

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Leave go-sqlite3 version unchanged

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* tidy

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* User SearchUserPermissions instead of SearchUsersPermissions

* Replace DummyKeyService with signingkeys.Service

* Use user🆔<id> as subject

* Fix introspection endpoint issue

* Add X-Grafana-Org-Id to get_resources.bash script

* Regenerate toggles_gen.go
* Fix basic.go

* Add GetExternalService tests

* Add GetPublicKeyScopes tests

* Add GetScopesOnUser tests

* Add GetScopes tests

* Add ParsePublicKeyPem tests

* Add database test for GetByName

* re-add comments

* client tests added

* Add GetExternalServicePublicKey tests

* Add other test case to GetExternalServicePublicKey

* client_credentials grant test

* Add test to jwtbearer grant

* Test Comments

* Add handleKeyOptions tests

* Add RSA key generation test

* Add ECDSA by default to EmbeddedSigningKeysService

* Clean up org id scope and audiences

* Add audiences to the DB

* Fix check on Audience

* Fix double import

* Add AC Store mock and align oauthserver tests

* Fix test after rebase

* Adding missing store function to mock

* Fix double import

* Add CODEOWNER

* Fix some linting errors

* errors don't need type assertion

* Typo codeowners

* use mockery for oauthserver store

* Add feature toggle check

* Fix db tests to handle the feature flag

* Adding call to DeleteExternalServiceRole

* Fix flaky test

* Re-organize routes comments and plan futur work

* Add client_id check to Extended JWT client

* Clean up

* Fix

* Remove background service registry instantiation of the OAuth server

* Comment cleanup

* Remove unused client function

* Update go.mod to use the latest ory/fosite commit

* Remove oauth2_server related configs from defaults.ini

* Add audiences to DTO

* Fix flaky test

* Remove registration endpoint and demo scripts. Document code

* Rename packages

* Remove the OAuthService vs OAuthServer confusion

* fix incorrect import ext_jwt_test

* Comments and order

* Comment basic auth

* Remove unecessary todo

* Clean api

* Moving ParsePublicKeyPem to utils

* re ordering functions in service.go

* Fix comment

* comment on the redirect uri

* Add RBAC actions, not only scopes

* Fix tests

* re-import featuremgmt in migrations

* Fix wire

* Fix scopes in test

* Fix flaky test

* Remove todo, the intersection should always return the minimal set

* Remove unecessary check from intersection code

* Allow env overrides on settings

* remove the term app name

* Remove app keyword for client instead and use Name instead of ExternalServiceName

* LogID remove ExternalService ref

* Use Name instead of ExternalServiceName

* Imports order

* Inline

* Using ExternalService and ExternalServiceDTO

* Remove xorm tags

* comment

* Rename client files

* client -> external service

* comments

* Move test to correct package

* slimmer test

* cachedUser -> cachedExternalService

* Fix aggregate store test

* PluginAuthSession -> AuthSession

* Revert the nil cehcks

* Remove unecessary extra

* Removing custom session

* fix typo in test

* Use constants for tests

* Simplify HandleToken tests

* Refactor the HandleTokenRequest test

* test message

* Review test

* Prevent flacky test on client as well

* go imports

* Revert changes from 526e48ad45

* AuthN: Change the External Service registration form (#68649)

* AuthN: change the External Service registration form

* Gen default permissions

* Change demo script registration form

* Remove unecessary comment

* Nit.

* Reduce cyclomatic complexity

* Remove demo_scripts

* Handle case with no service account

* Comments

* Group key gen

* Nit.

* Check the SaveExternalService test

* Rename cachedUser to cachedClient in test

* One more test case to database test

* Comments

* Remove last org scope

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Update pkg/services/oauthserver/utils/utils_test.go

* Update pkg/services/sqlstore/migrations/oauthserver/migrations.go

Remove comment

* Update pkg/setting/setting.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

---------

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
2023-05-25 15:38:30 +02:00
Yuri Tseretyan 3af95bebe1 Alerting: Migrate unknown NoData\Error settings to the default (#68403)
* use default execution if legacy is not known

* update docs

* Update docs/sources/alerting/migrating-alerts/migrating-legacy-alerts.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/migrating-alerts/migrating-legacy-alerts.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

---------

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
2023-05-24 13:09:17 -04:00
Ryan McKinley 26658d172f Migrations: Support independent migration log for entity api (#68511) 2023-05-22 11:31:07 -07:00
Dan Cech 7572988305 add uid to team table (#66920)
* add uid to team table

* concat

* add prefix to transition uids

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-05-09 13:03:28 -04:00
Yuri Tseretyan 0ce7f7eaf4 Alerting: Migration to not fail if alert_configuration table is not empty (#67924) 2023-05-05 12:03:53 -04:00
Yuri Tseretyan 61484fa826 Alerting: Mention title of alert rule that caused migration to fail (#67451)
* add debug log for migration of alert rules
* add alert rule name and some information to conversion error
2023-05-01 10:32:16 -04:00
Yuri Tseretyan a8b4a4bb45 Alerting: Update alerting module to 20230418161049-5f374e58cb32 + refactoring (#66622)
* update to alerting 20230418161049-5f374e58cb32
* rename renamed structs in https://github.com/grafana/alerting/pull/73
* update ValidateContactPoint to use BuildReceiverConfiguration
* update logger factory according to changes
* rewrite integration builder
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2023-04-25 13:39:46 -04:00
Ieva 035bf29146 RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652)
* RBAC: Stop reading enabeld from ini file and always set to true

* Migrations: Add a migration for rbac to reset data migrations if rbac
was disabled

* If rbac was disabled we reset the data and data migrations that rbac
  has to perform to get it to a correct state

* Migrator: Store migration logs on migrator and add function to clear it from the
in-memory stored logs

* update tests

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-04-19 16:34:19 +01:00
Artur Wierzbicki 2136e680c4 Previews: remove dashboard previews backend (#66176)
* remove dashboard previews backend

* remove dashboard previews backend

* bring back the migration

* bring back the migration

* bring back the migration
2023-04-13 21:42:24 +04:00
Yuri Tseretyan afd52d0866 Alerting: use alerting GrafanaReceiver and BuildReceiverConfiguration in Grafana (#65224)
* replace receiver errors with one from alerting
* add the converter to alerting models
* update buildReceiverIntegration to accept GrafanaReceiver
---------

Co-authored-by: George Robinson <george.robinson@grafana.com>
2023-04-13 12:25:32 -04:00
Yuri Tseretyan 7b2f44762e Alerting: Update migration to put alerts to the default folder if dashboard folder is missing (#65577)
* extract function

* use context logger

* put alert to general folder if folder is missing

* move folderHelper init

* add test

* Update pkg/services/sqlstore/migrations/ualert/ualert.go

Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>

---------

Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
2023-03-31 18:40:21 +02:00
ismail simsek 3cfb7ac0dd Chore: Remove expr imports (#64543)
* Remove expr imports

* Add comment
2023-03-23 21:55:54 +01:00
Ryan McKinley d5a9a0cea0 Chore: remove comments feature (#64644) 2023-03-11 04:28:12 -08:00
Sofia Papagiannaki e41311e8be Library elements: Add MySQL migration for increasing the column size (#63856)
* Library elements: Add MySQL migration for increasing column size
2023-03-06 12:13:30 +01:00
George Robinson 030f6c948f Alerting: Fix migration pauses all alert rules on PostgreSQL (#63951)
This commit fixes a serious bug in Grafana 9.4.1 where on upgrade
a migration would pause all existing alert rules and change the
default value of the column to true.
2023-03-01 17:32:29 +00:00
Alex Moreno a05bf41ff9 Alerting: Fix boolean default in migration from false to 0 (#63952)
Fix boolean default in migration from false to 0
2023-03-01 16:58:30 +00:00
owensmallwood dc01e1ee6a PublicDashboards: Backfills share column with default value (#63407)
adds migration to backfill empty share column fields with default value of public
2023-02-27 15:54:16 -06:00
Emil Tullstedt 3abaf32cf2 Chore: Upgrade golangci-lint to v1.51.2 (#63630)
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-02-23 15:10:03 +01:00
Ryan McKinley b1e58eb47e Chore: Replace short UID generation with more standard UUIDs (#62731) 2023-02-06 20:44:37 -05:00
Ryan McKinley 72790b0614 Live: Remove dev code that would save config and messages in SQL (#62912) 2023-02-03 16:21:48 -05:00
Yuri Tseretyan f066e8cdcd Alerting: Update to alerting 20230203015918-0e4e2675d7aa (after refactoring) (#62823)
* add alerting prefix to some packages from alerting that have similar names in prometheus alertmanager
2023-02-03 11:36:49 -05:00
idafurjes 00d954320f Chore: rename Id to ID in alert notification models (#62868) 2023-02-03 15:46:55 +01:00
George Robinson f49efa6e27 Alerting: Pause dash alerts on migration (#62798)
* Alerting: Pause dash alerts on migration
2023-02-02 16:49:05 -05:00
Gilles De Mey c3cd4a720f Alerting: Adds a default value to the last_applied column (#62818) 2023-02-02 19:35:28 +00:00
Santiago ba731f7865 Alerting: Mark AM configuration as applied (#61330)
* Mark AM configuration as applied

* add missing checks, make linter happy

* fix deadlock, mark as valid on save and on load

* mark configurations only if needed

* check error after applyConfig()

* code review comments

* code review changes

* more code review changes

* clean HistoricConfigFromAlertConfig function
2023-02-02 14:45:17 -03:00
idafurjes 23c27cffb3 Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
Sofia Papagiannaki 4eaff63eda SQLStore: Fix folder migration for MySQL < 5.7 (#62521)
* Nested folders: Do not skip integration tests

* SQLStore: Fix folder migration

It reduces the length of the title column to be equal with the respective
dashboard column.
2023-02-02 16:21:25 +02:00
idafurjes 2c50c44d89 Chore: Move library elements models to library elements service (#62507)
* Chore: Move library elements models to library elements service

* Fix stat import

* Fix faulty error assignment
2023-02-01 17:32:05 +01:00
ismail simsek 91221bc436 Expressions: Fixes the issue showing expressions editor (#62510)
* Use suggested value for uid

* update the snapshot

* use __expr__

* replace all -100 with __expr__

* update snapshot

* more changes

* revert redundant change

* Use expr.DatasourceUID where it's possible

* generate files
2023-01-31 18:50:10 +01:00
suntala 51bef166c2 Chore: Remove Result field from serviceaccounts, ualert (#62476)
* Chore: Remove Result field from serviceaccounts
* Chore: Remove Result field from ualert
2023-01-31 09:51:55 +01:00