Commit Graph

106 Commits

Author SHA1 Message Date
Yuri Tseretyan
0351a37e99 Alerting: Remote Alertmanager to calculate hash of the request payload instead of just the configuration v2 (#109139)
* Revert "Revert "Alerting: Remote Alertmanager to calculate hash of the reques…"

This reverts commit cbf256120e.

* log the decision

Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>

---------

Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2025-09-03 14:01:25 +00:00
Santiago
29aff8c387 Remote Alertmanager: Ignore tenant ID when merging remote state (#109928)
* Remote Alertmanager: Ignore tenant ID when merging remote state

* update tests
2025-08-21 09:00:37 +00:00
Santiago
a31323578f Remote Alertmanager: Move factory functions to the remote package (#108582)
* Remote Alertmanager: Move factory functions to the remote package

* remove createRemoteAlertmanager

* modify comment

* unexport functions to create remote secondary and primary forked AMs

* RemoteFactory -> NewRemoteFactory

* avoid passing a logger

* avoid panics if creating the internal AM fails

* remove lines

* rephrase comment

* fix source of sync interval
2025-08-19 15:34:58 +00:00
Santiago
1914fb95ee Remote Alertmanager: Wire up remoteSecondaryAlertmanagerWithRemoteState and fix path (#109253) 2025-08-06 17:33:31 +02:00
Yuri Tseretyan
cbf256120e Revert "Alerting: Remote Alertmanager to calculate hash of the request payload instead of just the configuration" (#109086)
Revert "Alerting: Remote Alertmanager to calculate hash of the request payloa…"

This reverts commit 32434810e1.
2025-08-01 21:34:31 +00:00
Yuri Tseretyan
32434810e1 Alerting: Remote Alertmanager to calculate hash of the request payload instead of just the configuration (#108632)
* update CreateGrafanaAlertmanagerConfig to accept UserGrafanaConfig move construction logic to alertmanager
* consolidate building UserGrafanaConfig into buildConfig
* use config to determine whether it needs to be send calculate hash of the entire request struct rather than configuration
2025-07-31 10:25:43 -04:00
Santiago
3c4c3b3f5c Remote Alertmanager: Remove unneeded SmtpFrom and StaticHeaders fields (#108781)
* Remote Alertmanager: Remove unneeded SmtpFrom and StaticHeaders fields

* remove smtpFrom (am struct)

* move smtp field (am struct)

* fix test
2025-07-29 12:14:42 +02:00
Serge Zaitsev
a95fb3a37c Chore: Omit integration tests if short test flag is passed (#108777)
* omit integration tests if short test flag is passed

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* fix the rest

* false positive

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-28 13:38:54 +02:00
Santiago
dcb965b7dc Remote Alertmanager: Optionally merge remote state before starting the internal Alertmanager (#107710)
* Remote Alertmanager: Use the same struct for Grafana stat and Mimir full state

* Alertmanager: Add methods to merge nflog and silences

* update grafana/alerting version

* make update-workspace

* update mocks

* remove unnecesary methods from the remote Alertmanager implementation, create separate StateMerger interface

* (WIP) Remote Alertmanager: Optionally merge remote state before starting the internal Alertmanager

* cleanup ngalert.go

* restore defaults.ini

* move state parsing logic to 'remote' package, clean up ngalert.go

* remove GetBase, implement MegeNflog and MergeSilences

* delete fmt.Println

* FetchRemoteState -> GetRemoteState

* UserGrafanaState -> UserState

* remove duplicate clusterpb import

* reorder MimirClient interface

* use general getState() method for Grafana state and Mimir full state

* remove unnecessary state merging methods from the Alertmanager interface

* remove pullState field

* reduce diff

* add info log after merging

* merge silences and nflog entries in the same method

* merge the remote state in the forked AM

* reduce diff

* update remote AM mock

* tests

* make error more specific

* typo
2025-07-25 15:06:59 +00:00
Yuri Tseretyan
e280b949e3 Alerting: Refactor remote Alertmanager and Mimir client (#107808)
* deduplicate config preparation logic squashes config preparation in 3 places into a single method buildConfiguration
* remove copying config from decrypt because we already use copy
* move logic from decryptConfiguration to buildConfiguration
* move logic from mergeExtraConfigs to buildConfiguration
* load default config with buildConfiguration method and skip if fails
---------

Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2025-07-24 12:00:38 -04:00
Yuri Tseretyan
0895d63516 Alerting: Remote Alertmanager to compare current hash with hash from response (#108554) 2025-07-24 09:22:44 -04:00
Yuri Tseretyan
5097dd5c7d Alerting: Send templates from extra configuration to remote Alertmanager (#107981)
* extract logging of MergedResult into method

* convert GetMergedTemplateDefinitions to return PostableApiTemplate

* update mergeExtracConfigs to return GrafanaAlertmanagerConfig

* pass by value, not pointer

* add template definition to payload

* update tests

* rename to Templates

* log merge results

* fix reference in workspace
2025-07-18 15:16:17 -04:00
Santiago
8548530dc4 Alertmanager: Add MergeState method (#108242)
* Alertmanager: Add MergeState method

* remove RemoteState in favor of ExternalState

* fix tests
2025-07-17 16:47:14 +02:00
Santiago
71a425f912 Remote Alertmanager: Fetch full state from Mimir (#107905)
* Remote Alertmanager: Add method to fetch the full state

* decode and parse the remote Alertmanager state string

* test
2025-07-15 11:38:38 +02:00
Santiago
a314b99589 Remote Alertmanager: Use the same struct for Grafana state and Mimir full state (#107791)
Remote Alertmanager: Use the same struct for Grafana stat and Mimir full state
2025-07-11 10:10:30 +02:00
Yuri Tseretyan
4bb6926eee Alerting: Separate configuration model for remote Alertmanager Mimir client (#107741)
* replace PostableUserConfig with GrafanaAlertmanagerConfig to decouple from internal Grafana models
* update alertmanager + tests
* calculate hash of the GrafanaAlertmanagerConfig
2025-07-09 12:42:10 -04:00
Ryan McKinley
3f502f305d Chore: Update mocks with recent mockery (#107816) 2025-07-09 09:15:34 +02:00
Alexander Akhmetov
b483a04aec Alerting: Send merged configuration to the remote alertmanager (#107004) 2025-07-02 21:35:24 +02:00
Serge Zaitsev
f66a693438 Chore: Rename integration tests to follow the common convention (#105987)
* automatically rename integration tests to follow the common convention

* name tests differently

* alter column type to bigint

* update another column to bigint

* add another alter

* fix subquery for mysql
2025-06-29 16:56:24 +02:00
Alexander Akhmetov
894944dcb0 Alerting: Refactor remote alertmanager to use Crypto interface (#107228) 2025-06-26 12:39:06 +02:00
Matthew Jacobson
5520f48153 Alerting: Use new receiver models for encrypt/decrypt in remote AM (#107042)
Several niche bugs have surfaced as a result of the decrypt code Grafana uses in receivers API being different than what is used to decrypt secrets before sending to remote AM. Example:
- Dingding notifier not abiding by new Patching added to local AM, thus causing missing url errors.

* noop refactor to simplify decryptConfiguration

* Move compat function package

* Use new receiver models to encrypt/decrypt in remote AM
2025-06-20 11:45:01 -04:00
Santiago
3fe73b8de9 Remote Alertmanager: Send SMTP config (#106337)
* (WIP) Remote Alertmanager: Send SMTP config

* send SMTP configs separately

* bring back deleted fields

* actually send stuff over

* remove redundant type, fix comments

* smtp -> smtpConfig

* also send SmtpFrom an StaticHeaders separately

* tests

* restore defaults.ini
2025-06-13 12:44:39 -03:00
Santiago
180b67ca6c Remote Alertmanager: Make timeout configurable in alert senders (#105599) 2025-05-19 12:52:12 +02:00
Santiago
6c3d89f390 Remote Alertmanager: Add timeouts to the HTTP client (#105279)
* Remote Alertmanage: Add timeouts to the HTTP client

* code review suggestions
2025-05-13 13:25:56 +02:00
Alexander Akhmetov
29128f7ae4 Alerting: Copy alertmanager configuration before decrypting (#105271) 2025-05-13 11:46:17 +02:00
Santiago
b434925adc Remote Alertmanager: Add tracing to the HTTP client used for POSTing alerts and the readiness check (#105235) 2025-05-12 14:57:42 +03:00
Tito Lins
638972c787 Alerting: fix tests (#105240) 2025-05-12 14:00:12 +03:00
Santiago
51d7aa2bef Remote Alertmanager: Configure SMTP From address (#104925)
* Remote Alertmanager: Configure SMTP From address

* include smtp from address in config comparison

* updte tests

* trigger build

* make linter happy

* trigger build

* fix test
2025-05-12 10:37:27 +02:00
Santiago
7edace5e88 Remote Alertmanager: Remove comparison before sending the state (#104930)
* Remote Alertmanager: Remove comparison before sending the state

* fix test

* fix test
2025-05-12 10:34:06 +02:00
beejeebus
8f79e4882f Replace usage of http.DefaultClient and http.DefaultTransport (#104135)
Remove usage of http.DefaultClient and http.DefaultTransport

Part of grafana/data-sources#484
2025-05-09 13:26:39 -04:00
Santiago
57640e40a2 Remote Alertmanager: Consider auto-gen routes when flagging a config as "default" (#105120)
* Remote Alertmanager: Consider auto-gen routes when flagging a config as 'default'

* remove always-nil error from isDefaultConfiguration

* remove unnecessary context.Background() in test

* pass orgID to autogenFn call during AM creation

* fix test

* make update-workspace
2025-05-08 21:04:30 +02:00
Mariell Hoversholm
757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
Matthew Jacobson
371ea5cda7 Alerting: Fix loss of TimeInterval location on remote AM apply (#102510)
* Alerting: Fix loss of TimeInterval location on remote AM apply

deepcopy.Copy does not correctly copy PostableUserConfig because it ignores
unexported fields. As a result, TimeInterval locations default to UTC instead
of retaining their original values.

* make update-workspace
2025-03-20 09:54:33 +01:00
Yuri Tseretyan
c3f00eb403 Alerting: log body of unexpected response from Mimir (#102382)
log body of unexpected response

Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2025-03-19 10:14:05 -04:00
Santiago
7d4895c3c9 Alerting: Use exponential backoff in the remote Alertmanager readiness check (#99756)
* Alerting: Use exponential backoff in the remote Alertmanager readiness check

* fix capitalized error

* remove unnecessary 'for'

* refactor, use time.After() instead of channel
2025-01-29 18:53:30 +01:00
Santiago
361312bbd7 Alerting: Expect 406s from the remote Alertmanager during the readiness check (#99507)
* Alerting: Expect 406s from the remote Alertmanager during the readiness check

* make it clear in the warning logs that we'll attempt to send the confgiuration/state without comparing in case of error pulling the current state/config
2025-01-24 16:26:57 +01:00
Santiago
9e408f842c Alerting: Skip sanitizing labels when sending alerts to the remote Alertmanager (#96251)
* Alerting: Skip sanitizing labels when sending alerts to the remote Alertmanager

* fix drone image name
2024-11-13 11:21:44 -03:00
Santiago
4f8f82f5f1 Alerting: Fix remote Alertmanager readiness check path (#95063) 2024-10-21 17:24:51 +02:00
Santiago
4c15266a77 Alerting: Add X-Remote-Alertmanager header to the remote AM client (#94913) 2024-10-17 22:38:13 +02:00
Santiago
80611b381c Alerting: Decrypt secure settings when testing receivers in the remote Alertmanager (#93864)
* Alerting: Decrypt secure settings when testing receivers in the remote Alertmanager

* go work sync

* make update-workspace

* point to latest main in grafana/alerting

* unit test

* import definitions only once
2024-09-30 13:28:30 -03:00
Tito Lins
4a124469fa Check is config is default by comparing hashes (#92296) 2024-08-23 11:22:06 +02:00
Fayzal Ghantiwala
8d725a641c Alerting: Integration test for testing template via remote alertmanager (#92147)
* Add integration test for testing template

* Update drone signature
2024-08-21 13:06:01 +01:00
Fayzal Ghantiwala
e321dbb690 Alerting: Use remote Alertmanager to test templates and receivers when enabled (#91570)
* Initial impl

* Add code to test templates and receivers

* Fix linter

* Fix forked am tests

* Update mimir client

* Remove trailing whitespace

* re-trigger CI
2024-08-15 16:56:14 +01:00
Santiago
f852bf684a Alerting: Fix duplicated silences in remote primary mode bug (#91902)
* Alerting: Fix duplicated silences in remote primary mode bug

* test that a new silence id returned by calling CreateSilence() on the internal Alertmanager is ignored
2024-08-15 17:14:55 +02:00
Fayzal Ghantiwala
25dbb32cea Alerting: Vendor in latest grafana/alerting package (#91786)
* temp

* vendor

* Remove dead code

* Vendoring
2024-08-12 15:37:15 +01:00
Santiago
5487ea444a Alerting: Update remote Alertmanager config marshalling in test (#91791) 2024-08-12 13:45:22 +02:00
Santiago
e097ffc771 Alerting: Update grafana/alerting dependency (#90365)
* update grafana/alerting to latest main
* update alertmanager to  66ec17e3aa45
2024-07-12 14:05:17 -04:00
Santiago
3bb861b9f0 Alerting: Remove empty/namespace labels when sending alerts to the remote Alertmanager (#90284)
* Alerting: Remove empty/namespace labels when sending alerts to the remote Alertmanager

* update tests

* fix typo in comment
2024-07-11 15:20:12 +02:00
Santiago
fce03cd724 Alerting: Send static headers to the remote Alertmanager (#89846) 2024-07-01 17:48:40 +02:00
Santiago
fe1309dd96 Alerting: Send external URL to the remote Alertmanager (#89701)
* Alerting: Send external URL to the remote Alertmanager

* test that the URL is sent to the remote Alertmanager

* AppURL -> ExternalURL
2024-06-26 14:02:02 +02:00