* add metrics and tracing to state manager
* propagate tracer to state manager
* add scheduler metrics
* fix backtesting
* add test for state metrics
* remove StateUpdateCount
* update docs
* metrics can be null
* add tracer to new tests
* Alerting: Repurpose rule testing endpoint to return potential alerts
This feature replaces the existing no-longer in-use grafana ruler testing API endpoint /api/v1/rule/test/grafana. The new endpoint returns a list of potential alerts created by the given alert rule, including built-in + interpolated labels and annotations.
The key priority of this endpoint is that it is intended to be as true as possible to what would be generated by the ruler except that the resulting alerts are not filtered to only Resolved / Firing and ready to be sent.
This means that the endpoint will, among other things:
- Attach static annotations and labels from the rule configuration to the alert instances.
- Attach dynamic annotations from the datasource to the alert instances.
- Attach built-in labels and annotations created by the Grafana Ruler (such as alertname and grafana_folder) to the alert instances.
- Interpolate templated annotations / labels and accept allowed template functions.
* Alerting: Allow hooking into request handler functions.
Adds a facility to AlertNG for hooking into API handlers, allowing the
replacement of request handlers for specific paths. One of goals of this
approach was to allow hooking as late as possible in the request, e.g.
after all middleware has been applied, to simplfiy usage.
* Update pkg/services/ngalert/api/hooks.go
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Update pkg/services/ngalert/api/hooks.go
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Update pkg/services/ngalert/ngalert.go
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Fixes to review comments
* Fix passing logger in
---------
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* stop using the scheduler's Update and Delete methods all communication must be via the database
* update scheduler's registry to calculate diff before re-setting the cache
* update fetcher to return the diff generated by registry
* update processTick to update rule eval routine if the rule was updated and it is not going to be evaluated at this tick.
* remove references to the scheduler from api package
* remove unused methods in the scheduler
* Define endpoint and generate
* Wire up and register endpoint
* Cleanup, define authorization
* Forgot the leading slash
* Wire up query and SignedInUser
* Wire up timerange query params
* Add todo for label queries
* Drop comment
* Update path to rules subtree
* Implement backtesting engine that can process regular rule specification (with queries to datasource) as well as special kind of rules that have data frame instead of query.
* declare a new API endpoint and model
* add feature toggle `alertingBacktesting`
* create contextual log context provider
* use contextual provider in scheduler
* init logger in the package
* use context for log context
* use context in state manager
* (WIP) switch to fork AM, first implementation of the API, generate spec
* get receivers avoiding race conditions
* use latest version of our forked AM, tests
* make linter happy, delete TODO comment
* update number of expected paths to += 2
* delete unused endpoint code, code review comments, tests
* Update pkg/services/ngalert/notifier/alertmanager.go
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* remove call to fmt.Println
* clear naming for fields
* shorter variable names in GetReceivers
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* Refactor state manager to not depend on rule store interface
* Refactor grafana and proxied ruler APIs to not depend on store.RuleStore
* Refactor folder subscription logic to not use store.RuleStore
* Delete dead code
* Delete store.RuleStore
* Add consumer-side store interface to state manager
* Remove dead dependency
* Delete dead dependency in API struct
* Delete store-layer InstanceStore interface
* Move fake for state's InstanceStore interface to state package
Removes various custom headers logic sprinkled around in the backend.
It should automatically be applied to outgoing HTTP requests via the
CustomHeadersMiddleware.
This also removes decryption of SecureJSONData to populate custom
headers in ngalert which seemed to have caused a ton of CPU usage.
This changes the API codegen template (controller-api.mustache) to simplify some names. When this package was created, most APIs "forked" to either a Grafana backend implementation or a "Lotex" remote implementation. As we have added APIs it's no longer the case. Provisioning, configuration, and testing APIs do not fork, and we are likely to add additional APIs that don't fork.
This change replaces {{classname}}ForkingService with {{classname}} for interface names, and names the concrete implementation {{classname}}Handler. It changes the implied implementation of a route handler from fork{{nickname}} to handle{{nickname}}. So PrometheusApiForkingService becomes PrometheusApi, ForkedPrometheusApi becomes PrometheusApiHandler and forkRouteGetGrafanaAlertStatuses becomes handleRouteGetGrafanaAlertStatuses
It also renames some files - APIs that do no forking go from forked_{{name}}.go to {{name}}.go and APIs that still fork go from forked_{{name}}.go to forking_{{name}}.go to capture the idea that those files a "doing forking" rather than "are a fork of something."
Signed-off-by: Joe Blubaugh <joe.blubaugh@grafana.com>
* Remove user from preferences, stars, orguser, team member
* Fix lint
* Add Delete user from org and dashboard acl
* Delete user from user auth
* Add DeleteUser to quota
* Add test files and adjust user auth store
* Rename package in wire for user auth
* Import Quota Service interface in other services
* do the same in tests
* fix lint tests
* Fix tests
* Add some tests
* Rename InsertUser and DeleteUser to InsertOrgUser and DeleteOrgUser
* Rename DeleteUser to DeleteByUser in quota
* changing a method name in few additional places
* Fix in other places
* Fix lint
* Fix tests
* Rename DeleteOrgUser to DeleteUserFromAll
* Update pkg/services/org/orgimpl/org_test.go
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Update pkg/services/preference/prefimpl/inmemory_test.go
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Rename Acl to ACL
* Fix wire after merge with main
* Move test to uni test
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Introduce AlertsRouter in the sender package, and move all fields and methods related to notifications out of the scheduler to this router.
* Introduce a new interface AlertsSender in the schedule package and replace calls of anonymous function `notify` inside the ruleRoutine to calling methods of that interface.
* Rename interface Scheduler in api package to ExternalAlertmanagerProvider, and replace scheduler with AlertRouter as struct that implements the interface.
* Template service
* Add GET routes and implement them
* Generate mock for persist layer
* Unit tests for reading templates
* Set up composition root and get integration tests working
* Fix prealloc issue
* Extract setup boilerplate
* Update AuthorizationTest
* Rebase and resolve
* Fix linter error
* Refactor GET am config to be extensible
* Extract post config route
* Fix tests
* Remove temporary duplication
* Fix broken test due to layer shift
* Fix duplicated error message
* Properly return 400 on config rejection
* Revert weird half method extraction
* Move things to notifier package and avoid redundant interface
* Simplify documentation
* Split encryption service and depend on minimal abstractions
* Properly initialize things all the way up to the composition root
* Encryption -> Crypto
* Address misc feedback
* Missing docstring
* Few more simple polish improvements
* Unify on MultiOrgAlertmanager. Discover bug in existing test
* Fix rebase conflicts
* Misc feedback, renames, docs
* Access crypto hanging off MultiOrgAlertmanager rather than having a separate API to initialize
* add check for access to rule's data source in GET APIs
* use more general method GetAlertRules instead of GetNamespaceAlertRules.
* remove unused GetNamespaceAlertRules.
Tests:
* create a method to generate permissions for rules
* extract method to create RuleSrv
* add tests for RouteGetNamespaceRulesConfig
* Base-line API for provisioning notification policies
* Wire API up, some simple tests
* Return provenance status through API
* Fix missing call
* Transactions
* Clarity in package dependencies
* Unify receivers in definitions
* Fix issue introduced by receiver change
* Drop unused internal test implementation
* FGAC hooks for provisioning routes
* Polish, swap names
* Asserting on number of exposed routes
* Don't bubble up updated object
* Integrate with new concurrency token feature in store
* Back out duplicated changes
* Remove redundant tests
* Regenerate and create unit tests for API layer
* Integration tests for auth
* Address linter errors
* Put route behind toggle
* Use alternative store API and fix feature toggle in tests
* Fixes, polish
* Fix whitespace
* Re-kick drone
* Rename services to provisioning
* make eval.Evaluator an interface
* inject Evaluator to TestingApiSrv
* move conditionEval to RouteTestGrafanaRuleConfig because it is the only place where it is used
* update rule test api to check data source permissions
* Remove InTransaction from RuleStore and make it its own interface
* Ensure that ctx-based is clear from name
* Resolve merge conflicts
* Refactor tests to work in terms of the introduced abstraction rather than concrete dbstore
* add actions, roles and route mapping for rule permission
* add instance\notification actions
* do not declare alerting roles if no feature flag is set (temporary)
* Update API controller
- add validation of rules API model
- add function to calculate changes between the submitted alerts and existing alerts
- update RoutePostNameRulesConfig to validate input models, calculate changes and apply in a transaction
* Update DBStore
- delete unused storage method. All the logic is moved upstream.
- upsert to not modify fields of new by values from the existing alert
- if rule has UID do not try to pull it from db. (it is done upstream)
* Add rule generator
* Fix evaluation of alert rules for datasources with custom headers
* Fix unit tests
* Fix integration tests
* Evaluator fields should be package private
* Update API to call the scheduler to remove\update an alert rule. When a rule is updated by a user, the scheduler will remove the currently firing alert instances and clean up the state cache.
* Update evaluation loop in the scheduler to support one more channel that is used to communicate updates to it.
* Improved rule deletion from the internal registry.
* Move alert rule version from the internal registry (structure alertRuleInfo) closer rule evaluation loop (to evaluation task structure), which will make the registry values immutable.
* Extract notification code to a separate function to reuse in update flow.
* Create API test for overwriting invalid alertmanager config
* Avoid requiring alertmanager readiness for config changes
* AlertmanagerSrv depends on functionality rather than concrete types
* Add test for non-ready alertmanagers
* Additional cleanup and polish
* Back out previous integration test changes
* Refactor of tests incorrectly caused a test to become redundant
* Use pre-existing fake secret service
* Drop unused interface
* Test against concrete MultiOrgAlertmanager re-using fake infra from other tests
* Fix linter error
* Empty commit to rerun checks
Yuri Tseretyan
Santiago
Matthew Jacobson
Steve Simpson
Serge Zaitsev
Alexander Weaver
Jo
Sofia Papagiannaki
Marcus Efraimsson
Jean-Philippe Quéméner
Joe Blubaugh
idafurjes
George Robinson
Yuriy Tseretyan