Commit Graph
13 Commits
Author SHA1 Message Date
Yuri Tseretyan 420db99d16 Alerting: Update state manager to have immutable state in cache (#95985)
* create a new state and set at the end
* propagate labels datasource_uid and ref_id from current state if it's error
* copy the state when apply to all
2024-11-15 15:01:02 -05:00
Yuri Tseretyan 06d5850396 Alerting: Update alerting state history API to authorize access using RBAC (#89579)
* add method CanReadAllRules to rule authorization service

* add alias type Namespace for Folder in ngalert's models package. It implements the Namespacer interface that is used by authz logic

* update state history's backends to authorize access to rules.
* update Loki to add folders UIDs to query. 
    * Update BuildLogQuery to drop filter by folders if it's too long and fall back to in-memory filtering.
2024-06-26 10:25:37 -04:00
Matthew Jacobson 47c9259d75 Alerting: Ensure we update State.LastSentAt before persisting (#89427) 2024-06-25 13:01:26 -04:00
Steve Simpson 67fa96f88d Alerting: Pass logger into NewAnnotationBackend. (#87812)
* Alerting: Pass logger into NewAnnotationBackend.

Make it possible to pass loggers into more places for code reuse.

* Mistake in passing logger
2024-05-14 15:51:27 +02:00
Jean-Philippe Quéméner 82638d059f feat(alerting): add state persister interface (#80384) 2024-01-17 13:33:13 +01:00
William Wernert f7bf818527 Alerting: Make alert state history Loki http client public (#78291)
* Make state history Loki client public

* Make historian metrics subsystem configurable
2023-11-27 09:20:50 -05:00
Steve Simpson 894f420014 Alerting: Pass loggers into SchedulerCfg and ManagerCfg. (#75158) 2023-09-20 15:07:02 +02:00
Yuri Tseretyan 938e26b59f Alerting: Add new metrics and tracings to state manager and scheduler (#71398)
* add metrics and tracing to state manager

* propagate tracer to state manager

* add scheduler metrics

* fix backtesting

* add test for state metrics

* remove StateUpdateCount

* update docs

* metrics can be null

* add tracer to new tests
2023-08-16 09:04:18 +02:00
Alexander Weaver 18b910e654 Alerting: Refactor annotation historian to isolate dashboard service dependency (#71689)
* Refactor annotation historian to isolate dashboard service dependency

* Export PanelKey

* Don't export parsePanelKey

* Remove commented out code
2023-07-18 08:18:55 -05:00
George Robinson 7edbe72483 Alerting: Support concurrent queries for saving alert instances (#70525)
This commit adds support for concurrent queries when saving alert
instances to the database. This is an experimental feature in
response to some customers experiencing delays between rule evaluation
and sending alerts to Alertmanager, resulting in flapping. It is
disabled by default.
2023-06-23 11:36:07 +01:00
Alexander Weaver 19d01dff91 Alerting: Expose Prometheus metrics for persisting state history (#63157)
* Create historian metrics and dependency inject

* Record counter for total number of state transitions logged

* Track write failures

* Track current number of active write goroutines

* Record histogram of how long it takes to write history data

* Don't copy the registerer

* Adjust naming of write failures metric

* Introduce WritesTotal to complement WritesFailedTotal

* Measure TransitionsFailedTotal to complement TransitionsTotal

* Rename all to state_history

* Remove redundant Total suffix

* Increment totals all the time, not just on success

* Drop ActiveWriteGoroutines

* Drop PersistDuration in favor of WriteDuration

* Drop unused gauge

* Make writes and writesFailed per org

* Add metric indicating backend and a spot for future metadata

* Drop _batch_ from names and update help

* Add metric for bytes written

* Better pairing of total + failure metric updates

* Few tweaks to wording and naming

* Record info metric during composition

* Create fakeRequester and simple happy path test using it

* Blocking test for the full historian and test for happy path metrics

* Add tests for failure case metrics

* Smoke test for full annotation persistence

* Create test for metrics on annotation persistence, both happy and failing paths

* Address linter complaints

* More linter complaints

* Remove unnecessary whitespace

* Consistency improvements to help texts

* Update tests to match new descs
2023-03-06 10:40:37 -06:00
Joey Tawadrous 121260e0dd Parca: Use data query schema (#62840)
* Parca data query schema

* Remove groupBy
2023-02-07 09:56:21 +00:00
Alexander Weaver 0efb84617e Alerting: Create benchmarking test for state.ProcessEvalResults (#62041)
* Create benchmark for ProcessEvalResults

* Simplify the test
2023-02-03 15:38:08 -06:00