Commit Graph
21 Commits
Author SHA1 Message Date
Santiago dab39c873f Remote Alertmanager: Use the correct OrgID when creating the store (#111634)
* Remote Alertmanager: Use the correct OrgID when creating the store

* fix test
2025-09-25 16:53:07 +00:00
Santiago a31323578f Remote Alertmanager: Move factory functions to the remote package (#108582)
* Remote Alertmanager: Move factory functions to the remote package

* remove createRemoteAlertmanager

* modify comment

* unexport functions to create remote secondary and primary forked AMs

* RemoteFactory -> NewRemoteFactory

* avoid passing a logger

* avoid panics if creating the internal AM fails

* remove lines

* rephrase comment

* fix source of sync interval
2025-08-19 15:34:58 +00:00
Vadim Stepanov bccc980b90 Alerting: Notifiication history (#107644)
* Add unified_alerting.notification_history to ini files

* Parse notification history settings

* Move Loki client to a separate package

* Loki client: add params for metrics and traces

* add NotificationHistorian

* rm writeDuration

* remove RangeQuery stuff

* wip

* wip

* wip

* wip

* pass notification historian in tests

* unify loki settings

* unify loki settings

* add test

* update grafana/alerting

* make update-workspace

* add feature toggle

* fix configureNotificationHistorian

* Revert "add feature toggle"

This reverts commit de7af8f7

* add feature toggle

* more tests

* RuleUID

* fix metrics test

* met.Info.Set(0)
2025-07-17 14:26:26 +01:00
Santiago a314b99589 Remote Alertmanager: Use the same struct for Grafana state and Mimir full state (#107791)
Remote Alertmanager: Use the same struct for Grafana stat and Mimir full state
2025-07-11 10:10:30 +02:00
Alexander Akhmetov 894944dcb0 Alerting: Refactor remote alertmanager to use Crypto interface (#107228) 2025-06-26 12:39:06 +02:00
Santiago 7edace5e88 Remote Alertmanager: Remove comparison before sending the state (#104930)
* Remote Alertmanager: Remove comparison before sending the state

* fix test

* fix test
2025-05-12 10:34:06 +02:00
Santiago 57640e40a2 Remote Alertmanager: Consider auto-gen routes when flagging a config as "default" (#105120)
* Remote Alertmanager: Consider auto-gen routes when flagging a config as 'default'

* remove always-nil error from isDefaultConfiguration

* remove unnecessary context.Background() in test

* pass orgID to autogenFn call during AM creation

* fix test

* make update-workspace
2025-05-08 21:04:30 +02:00
Matthew Jacobson e86929eb0a Alerting: Managed receiver resource permission in config api (#93632)
* Alerting: Managed receiver resource permission in config api
2024-09-25 09:39:36 -04:00
Alexander Weaver 8491e02caf Alerting: Instrument outbound requests for Loki Historian and Remote Alertmanager with tracing (#89185)
* Add TracedClient

* Handle errors and status codes

* Wire up tracing to normal ASH and loki annotation mapping

* Add tracing to remote alertmanager

* one more spot

* and not or

* More consistency with other grafana traces, lower cardinality name
2024-06-14 13:24:12 -05:00
Santiago 12d5251c12 Alerting: Alertmanager configuration sync loop (#88822)
* make the config sync happen on each call to ApplyConfig(), fix tests

* send autogen config

* add fake autogen function for tests

* update stale comments, tidy things up, make linter happy

* add auto-gen routes only if the feature toggle is enabled

* remove unnecessary fake autogen function

* throttle configuration syncs

* restore pkg/services/store/entity/sqlstash/sql_storage_server.go

* test sync loop in ApplyConfig, skip invalid autogen routes

* restore conf/defaults.ini

* restore conf/defaults.ini

* avoid skipping invalid auto-gen routes in SaveAndApplyConfig

* test that autogenFn is called and its errors are returned

* add debug message about the sync interval not having elapsed

* collapse two log lines into one
2024-06-12 10:13:34 +02:00
Santiago e41434c332 Alerting: Promote configuration in the remote Alertmanager (#87388) 2024-05-16 12:06:03 +02:00
Santiago 529f55cfe8 Alerting: Remove isDefault field from receivers (Alertmanager configuration) (#86605)
Alerting: Remove isDefault field from receivers in the Alertmanager configuration
2024-04-19 15:44:20 +02:00
Santiago 309a7e7684 Alerting: Implement SaveAndApplyDefaultConfig in the remote Alertmanager struct (#85005)
* Alerting: Implement SaveAndApplyDefaultConfig in the remote Alertmanager struct

* send the hash of the encrypted configuration

* tests, default config hash in AM struct

* add missing default config to test

* restore build directory

* go work file...

* fix broken test

* remove unnecessary conversion to []byte

* go work again...

* make things work again with latest main branch changes

* update error messages in tests for decrypting config
2024-04-19 15:11:07 +02:00
Matthew Jacobson f79dd7c7f9 Alerting: Persist silence state immediately on Create/Delete (#84705)
* Alerting: Persist silence state immediately on Create/Delete

Persists the silence state to the kvstore immediately instead of waiting for the
 next maintenance run. This is used after Create/Delete to prevent silences from
 being lost when a new Alertmanager is started before the state has persisted.
 This can happen, for example, in a rolling deployment scenario.

* Fix test that requires real data

* Don't error if silence state persist fails, maintenance will correct
2024-04-09 13:39:34 -04:00
Dave Henderson 5687243d0b Feature Flags: use FeatureToggles interface where possible (#85131)
* Feature Flags: use FeatureToggles interface where possible

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Replace TestFeatureToggles with existing WithFeatures

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-04-04 12:22:31 -04:00
Matthew Jacobson 0c3c5c5607 Alerting: Stop persisting silences and nflog to disk (#84706)
With this change, we no longer need to persist silence/nflog states to disk in addition to the kvstore
2024-03-23 00:37:33 +02:00
Santiago c9bb18101c Alerting: Decrypt secrets before sending configuration to the remote Alertmanager (#83640)
* (WIP) Alerting: Decrypt secrets before sending configuration to the remote Alertmanager

* refactor, fix tests

* test decrypting secrets

* tidy up

* test SendConfiguration, quote keys, refactor tests

* make linter happy

* decrypt configuration before comparing

* copy configuration struct before decrypting

* reduce diff in TestCompareAndSendConfiguration

* clean up remote/alertmanager.go

* make linter happy

* avoid serializing into JSON to copy struct

* codeowners
2024-03-19 12:12:03 +01:00
Yuri Tseretyan 1eebd2a4de Alerting: Support for simplified notification settings in rule API (#81011)
* Add notification settings to storage\domain and API models. Settings are a slice to workaround XORM mapping
* Support validation of notification settings when rules are updated

* Implement route generator for Alertmanager configuration. That fetches all notification settings.
* Update multi-tenant Alertmanager to run the generator before applying the configuration.

* Add notification settings labels to state calculation
* update the Multi-tenant Alertmanager to provide validation for notification settings

* update GET API so only admins can see auto-gen
2024-02-15 09:45:10 -05:00
William Wernert 2203bc2a3d Alerting: Refactor provisioning tests/fakes (#81205)
* Fix up test Alertmanager config JSON

* Move fake AM config and provisioning stores to fakes package
2024-01-24 17:15:55 -05:00
Santiago 9e78faa7ba Alerting: Add metrics to the remote Alertmanager struct (#79835)
* Alerting: Add metrics to the remote Alertmanager struct

* rephrase http_requests_failed description

* make linter happy

* remove unnecessary metrics

* extract timed client to separate package

* use histogram collector from dskit

* remove weaveworks dependency

* capture metrics for all requests to the remote Alertmanager (both clients)

* use the timed client in the MimirAuthRoundTripper

* HTTPRequestsDuration -> HTTPRequestDuration, clean up mimir client factory function

* refactor

* less git diff

* gauge for last readiness check in seconds

* initialize LastReadinesCheck to 0, tweak metric names and descriptions

* add counters for sync attempts/errors

* last config sync and last state sync timestamps (gauges)

* change latency metric name

* metric for remote Alertmanager mode

* code review comments

* move label constants to metrics package
2024-01-10 11:18:24 +01:00
Santiago 1f6575e65e Alerting: Test MOA in remote secondary mode (#79828) 2024-01-05 11:05:27 +01:00