* Publish event when one or more rules are changed
* Publish affected rules
* Use a fake bus to test publish event without listening
* Wire alerting store into provisioning service
* Revert "chore: add replDB to team service (#91799)"
This reverts commit c6ae2d7999.
* Revert "experiment: use read replica for Get and Find Dashboards (#91706)"
This reverts commit 54177ca619.
* Revert "QuotaService: refactor to use ReplDB for Get queries (#91333)"
This reverts commit 299c142f6a.
* Revert "refactor replCfg to look more like plugins/plugin config (#91142)"
This reverts commit ac0b4bb34d.
* Revert "chore (replstore): fix registration with multiple sql drivers, again (#90990)"
This reverts commit daedb358dd.
* Revert "Chore (sqlstore): add validation and testing for repl config (#90683)"
This reverts commit af19f039b6.
* Revert "ReplStore: Add support for round robin load balancing between multiple read replicas (#90530)"
This reverts commit 27b52b1507.
* Revert "DashboardStore: Use ReplDB and get dashboard quotas from the ReadReplica (#90235)"
This reverts commit 8a6107cd35.
* Revert "accesscontrol service read replica (#89963)"
This reverts commit 77a4869fca.
* Revert "Fix: add mapping for the new mysqlRepl driver (#89551)"
This reverts commit ab5a079bcc.
* Revert "fix: sql instrumentation dual registration error (#89508)"
This reverts commit d988f5c3b0.
* Revert "Experimental Feature Toggle: databaseReadReplica (#89232)"
This reverts commit 50244ed4a1.
* introduce storage model for alert rule tables
* remove AlertRuleVersion from models because it's not used anywhere other than in storage
* update historian xorm store to use alerting store to fetch rules
* fix folder tests
---------
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* Let alert rule service implement registry service
* Add count method to RuleStore interface
* Add implementation for deletion of alert rules
* Rename uid to folderUID in registry methods
* Check forceDeleteRule value for registry deletion
* Register alerting store with folder service
* Move folder test functions to separate package
* Add testing for alert rule counting, deletion
* Remove redundant count method
* Fix deleteChildrenInFolder signature
* Update pkg/services/ngalert/store/alert_rule.go
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Add tests for nested folder deletion
* Refactor TestIntegrationNestedFolderService
* Add rules store as parameter for alertng provider
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* (WIP) Refactor the ImageStore interface to work with our latest alerting repository
* update alerting package
* refactor, new URLExists method in ImageProvider
* tests for the new methods
* fix linter warnings
* use alertingImages as an alias for grafana/alerting/images
* logs about image uris and not found images
* nerf image not found logs
* extract duplicated code to getImageFromURI() method
* refactor getImageFromURI()
* add index on url
* add comment about migration log
* sync generated files
* use tokens or urls in image annotations
* improve tests, fix some comments
* fix empty tokens
* code review changes, check for url before checking for token (support old token formats)
* Delete folders, dashboards with registry service
Co-authored-by: Serge Zaitsev <hello@zserge.com>
* Update signature of ProvideDashboardServiceImpl
* Regenerate mockery file
* Add test for DeleteInFolder
* Add test for DeleteDashboardsInFolder
* Delete child dashboard associations via registry
* Add validation of folder uid and org id
---------
Co-authored-by: Serge Zaitsev <hello@zserge.com>
* Move fakeRuleStore to tests/fakes package
* Break stub dependencies on store
* Update existing tests to point to new location
* Remove unused stub of TimeNow
* Rename fake to take advantage of package name
* Refactor state manager to not depend on rule store interface
* Refactor grafana and proxied ruler APIs to not depend on store.RuleStore
* Refactor folder subscription logic to not use store.RuleStore
* Delete dead code
* Delete store.RuleStore
* Add consumer-side store interface to state manager
* Remove dead dependency
* Delete dead dependency in API struct
* Delete store-layer InstanceStore interface
* Move fake for state's InstanceStore interface to state package
* Alerting: Change screenshots to use components
This commit changes screenshots to use a number of components instead of a set of functional wrappers.
It moves the uploading of screenshots from the screenshot package to the image package so we can re-use the same code for both uploading screenshots and server-side images; SingleFlight from the screenshot package to the image package so we can use it for both taking and uploading the screenshot, where as before it was used just for taking the screenshot; and it also removes the use of a cache because we know that screenshots can be taken at most once per tick of the scheduler.
Prior to this change, all alert instance writes and deletes happened
individually, in their own database transaction. This change batches up
writes or deletes for a given rule's evaluation loop into a single
transaction before applying it.
Before:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s
```
After:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s
```
So we cut time by about 75% and memory allocations by about 60% when
storing and deleting 100 instances.
This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1
* Update GetAlertRulesForScheduling to query for folders (if needed)
* Update scheduler's alertRulesRegistry to cache folder titles along with rules
* Update rule eval loop to take folder title from the
* Extract interface RuleStore
* Pre-fetch the rule keys with the version to detect changes, and query the full table only if there are changes.
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* remove support for bus from scheduler
* rename event to FolderTitleUpdated and fire only if title has changed
* add method to increase version of all rules that belong to a folder
* update ngalert service to subscribe to folder title change event call data store and update scheduler
* add tests
* update GetAlertRulesForSchedulingQuery to have result AlertRule
* update fetcher utils and registry to support AlertRule
* alertRuleInfo to use alert rule instead of version
* update updateCh hanlder of ruleRoutine to just clean up the state. The updated rule will be provided at the next evaluation
* update evalCh handler of ruleRoutine to use rule from the message and clear state as well as update extra labels
* remove unused function in ruleRoutine
* remove unused model SchedulableAlertRule
* store rule version in ruleRoutine instead of rule
* do not call the sender if nothing to send
* move fake FakeExternalAlertmanager to sender package
* move tests from scheduler to router
* update alerts router to have all fields private
* update scheduler tests to use sender mock
* Alerting: Add first Grafana reserved label g_label
g_label holds the title of the folder container the alert. The intention of this label
is to use it as part of the new default notification policy groupBy.
* Add nil check on updateRule labels map
* Disable gocyclo lint on schedule.ruleRoutine
will remove later in a separate refactoring PR to reduce complexity.
* Address doc suggestions
* Update g_folder for rules in folder when folder title changes
* Remove global bus in FolderService
* Modify tests to fit new common g_folder label
* Add changelog entry
* Fix merge conflicts
* Switch GrafanaReservedLabelPrefix from `g_` to `grafana_`
* update authz to exclude entire group if user does not have access to rule
* change rule update authz to not return changes because if user does not have access to any rule in group, they do not have access to the rule
* a new query that returns alerts in group by UID of alert that belongs to that group
* collect all affected groups during calculate changes
* update authorize to check access to groups
* update tests for calculateChanges to assert new fields
* add authorization tests
* Alerting: unwrap upsert into insert and update function
* add changelog entry
* remove changelog entry
* rename upsertrule to updaterule
* use directly alertrule model for inserts
* add test for updating a rule with a conflicting name
* add check for access to rule's data source in GET APIs
* use more general method GetAlertRules instead of GetNamespaceAlertRules.
* remove unused GetNamespaceAlertRules.
Tests:
* create a method to generate permissions for rules
* extract method to create RuleSrv
* add tests for RouteGetNamespaceRulesConfig
* move validation at the beginning of method
* remove usage of GetOrgRuleGroups because it is not necessary. All information is already available in memory.
* remove unused method
* Use alert:create action for folder search with edit permissions. This matches the action that is used to query dashboards (the update will be addressed later)
* Update rule store to use FindDashboards instead of folder service to list folders the user has access to view alerts. Folder service does not support query type and additional filters.
* Do not check whether the user can save to folder if FGAC is enabled because it is checked on API level.
* rename GetRuleGroupAlertRules to GetAlertRules
* make rule group optional in GetAlertRulesQuery
* simplify FakeStore. the current structure did not support optional rule group
* Update API controller
- add validation of rules API model
- add function to calculate changes between the submitted alerts and existing alerts
- update RoutePostNameRulesConfig to validate input models, calculate changes and apply in a transaction
* Update DBStore
- delete unused storage method. All the logic is moved upstream.
- upsert to not modify fields of new by values from the existing alert
- if rule has UID do not try to pull it from db. (it is done upstream)
* Add rule generator
William Wernert
Jeff Levin
Yuri Tseretyan
Santiago
Arati R
Alexander Weaver
gotjosh
George Robinson
Sofia Papagiannaki
Joe Blubaugh
idafurjes
Matthew Jacobson
Jean-Philippe Quéméner
Kat Yang
ying-jeanne