Commit Graph
1920 Commits
Author SHA1 Message Date
Matthew Jacobson 2444fb1caf Alerting: Update grafana/alerting to 8eef376f49f8 (#108190)
Update grafana/alerting to 8eef376f49f8

Includes support for OAuth2 in most notifiers but does not enable that support
yet in FE. Fixes tests related to the change.
2025-07-16 17:33:52 +00:00
Santiago 71a425f912 Remote Alertmanager: Fetch full state from Mimir (#107905)
* Remote Alertmanager: Add method to fetch the full state

* decode and parse the remote Alertmanager state string

* test
2025-07-15 11:38:38 +02:00
Santiago d4670b1cee Alertmanager: Add methods to merge nflog and silences (#108006) 2025-07-14 16:23:23 +02:00
Santiago a314b99589 Remote Alertmanager: Use the same struct for Grafana state and Mimir full state (#107791)
Remote Alertmanager: Use the same struct for Grafana stat and Mimir full state
2025-07-11 10:10:30 +02:00
Yuri Tseretyan 4bb6926eee Alerting: Separate configuration model for remote Alertmanager Mimir client (#107741)
* replace PostableUserConfig with GrafanaAlertmanagerConfig to decouple from internal Grafana models
* update alertmanager + tests
* calculate hash of the GrafanaAlertmanagerConfig
2025-07-09 12:42:10 -04:00
Ryan McKinley 3f502f305d Chore: Update mocks with recent mockery (#107816) 2025-07-09 09:15:34 +02:00
Fayzal Ghantiwala de50c5a497 Alerting: Fix crypto_test.go errors in CI (#107759)
Fix test
2025-07-08 11:03:26 +02:00
Yuri TseretyanandAlexander Akhmetov c93fd3ee9e Alerting: idempotent decrypt\encrypt operations on extra configs (#107592)
* add prefix to encrypted string to distinguish between unencrypted and encrypted ones

---------

Co-authored-by: Alexander Akhmetov <me@alx.cx>
2025-07-07 12:17:41 -04:00
Yuri Tseretyan d15e1ad8d0 Alerting: Fix route validation in provisioning service (#107550)
* introduce ExtraConfigsCrypto
* delete getLastConfiguration and move all code into method Get
* update legacy store to encrypt ExtraConfigs same way we do in MultiOrgAlertmanager
* update legacy store init
* delete PersistConfig
* remove unnecessary unmarshalling
* add better error

* add tests for legacy store

* add tests that cover extra config validation

* add integration test for conflicting routes

---------

Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2025-07-07 12:16:31 -04:00
Alexander Akhmetov b483a04aec Alerting: Send merged configuration to the remote alertmanager (#107004) 2025-07-02 21:35:24 +02:00
Moustafa BaiouandMatt Jacobson 11ea1dfba8 Alerting: Fix notification policy conflicts originating from provenance mismatch (#107343)
* Alerting: Fix notification policy conflicts originating from provenance mismatch

Sometimes the provenance field on the notification policy route can mismatch with the actual provenance in the provenance store.

This leads to 409s when trying to update the notification policy via the k8s apis.

We avoid this by ignoring the provenance field when calculating the fingerprint.

Co-authored-by: Matt Jacobson <matthew.jacobson@grafana.com>

* fix tests which validate stability

---------

Co-authored-by: Matt Jacobson <matthew.jacobson@grafana.com>
2025-07-02 17:26:27 +00:00
Serge Zaitsev f66a693438 Chore: Rename integration tests to follow the common convention (#105987)
* automatically rename integration tests to follow the common convention

* name tests differently

* alter column type to bigint

* update another column to bigint

* add another alter

* fix subquery for mysql
2025-06-29 16:56:24 +02:00
Alexander Akhmetov b93cde0adb Alerting: Use default identifier for extra configs in Alertmanager API (#107049) 2025-06-26 12:44:49 +02:00
Alexander Akhmetov abbae41f60 Alerting: Sanitize Prometheus state history labels before writing (#107181) 2025-06-26 12:40:37 +02:00
Alexander Akhmetov 894944dcb0 Alerting: Refactor remote alertmanager to use Crypto interface (#107228) 2025-06-26 12:39:06 +02:00
Alexander Akhmetov 478f9bf597 Alerting: Emit metrics from prometheus state history backend (#107121)
Alerting: Emit metrics from prometheus historian backend
2025-06-24 18:27:52 +02:00
Alexander Akhmetov 3b45f7daa5 Alerting: Fix flaky test (#107090) 2025-06-24 11:25:51 +02:00
c4c2510a04 Alerting: Update alerting module to 863b09716e600ae15235b0fe2f996a389becd850 (#107027)
* [create-pull-request] automated change

* Fix TestRemoveSecretsForContactPoint test

---------

Co-authored-by: alexander-akhmetov <1875873+alexander-akhmetov@users.noreply.github.com>
Co-authored-by: Alexander Akhmetov <me@alx.cx>
2025-06-21 00:01:12 +02:00
Matthew Jacobson 5520f48153 Alerting: Use new receiver models for encrypt/decrypt in remote AM (#107042)
Several niche bugs have surfaced as a result of the decrypt code Grafana uses in receivers API being different than what is used to decrypt secrets before sending to remote AM. Example:
- Dingding notifier not abiding by new Patching added to local AM, thus causing missing url errors.

* noop refactor to simplify decryptConfiguration

* Move compat function package

* Use new receiver models to encrypt/decrypt in remote AM
2025-06-20 11:45:01 -04:00
Alexander Akhmetov ac832c157e Alerting: Maintain endsAt for missing alert instances that are not stale yet (#107011) 2025-06-20 14:31:21 +02:00
Alexander Akhmetov 700c9a857c Alerting: Return extra alertmanager config as a string for mimirtool (#106959) 2025-06-19 01:12:26 +02:00
Alexander Akhmetov 4b46da9214 Alerting: Hide extra Prometheus configuration secrets (#106916) 2025-06-18 17:01:19 +02:00
Alexander Akhmetov 426c334973 Alerting: API to create extra Alertmanager configurations (#106904)
What is this feature?

Implements the POST endpoint for deleting imported Mimir Alertmanager configurations:

    POST /api/convert/api/v1/alerts

The API endpoint creates the extra Alertmanager configuration with the provided identifier and matchers.
2025-06-18 15:05:06 +02:00
Alexander Akhmetov def5d889d0 Alerting: API to delete extra Alertmanager configurations (#106892) 2025-06-18 12:37:56 +02:00
Alexander Akhmetov 8c6df8b449 Alerting: Add support for retrieving imported Prometheus Alertmanager configurations (#106864) 2025-06-18 10:49:40 +02:00
Alexander Akhmetov 78bec77ca1 Alerting: Fix NewDatasourceWriter initialization (#106869)
Fix NewDatasourceWriter initialization
2025-06-18 07:53:48 +02:00
Alexander Akhmetov ad683f83ff Alerting: Add state history backend to write ALERTS metric (#104361)
**What is this feature?**

This PR implements a new Prometheus historian backend that allows Grafana alerting to write alert state history as Prometheus-compatible `ALERTS` metrics to remote Prometheus-compatible data sources.

The metric includes a few additional labels:

* `grafana_alertstate`: Grafana's full alert state, more granular than Prometheus.
* `grafana_rule_uid`: Grafana's alert rule UID.

Grafana states are included in the `grafana_alertstate` label also mapped to Prometheus-compatible `alertstate` values:

| Grafana alert state | `alertstate`          | `grafana_alertstate`  |
|---------------------|-----------------------|-----------------------|
| `Alerting`          | `firing`              | `alerting`            |
| `Recovering`        | `firing`              | `recovering`          |
| `Pending`           | `pending`             | `pending`             |
| `Error`             | `firing`              | `error`               |
| `NoData`            | `firing`              | `nodata`              |
| `Normal`            | _(no metric emitted)_ | _(no metric emitted)_ |
2025-06-18 07:17:57 +02:00
Alexander Akhmetov 5a78ce8a4b Alerting: Support saving extra Mimir configurations (#106721) 2025-06-17 23:33:16 +02:00
Moustafa Baiou 74e800e427 Alerting: Add provenance to Prometheus API (#106596)
This commit adds provenance information to the Prometheus API in the ngalert service to enable compatibility with the new alert list page.
2025-06-17 23:31:42 +02:00
Moustafa Baiou 0e6c7f84c3 Alerting: Add filters for health and contact point in Prometheus Rules api (#106580)
This adds the ability to filter rules with the prometheus compatible api using:
1. `receiver_name` to filter by contact point name
2. `health` to filter by the health status of the rule (one of `ok`, `error`, `nodata`, or `unknown`)

This also ensures that groups with no rules (due to filters) are not returned.
2025-06-17 16:57:03 +02:00
Alexander Akhmetov e92baba748 Alerting: Support PDC in Grafana-managed recording rules (#106677) 2025-06-17 11:46:34 +02:00
Stephanie Hingtgen a8886ad5ec Annotations: Use dashboard uids instead of dashboard ids (#106676) 2025-06-13 13:59:24 -05:00
Santiago 3fe73b8de9 Remote Alertmanager: Send SMTP config (#106337)
* (WIP) Remote Alertmanager: Send SMTP config

* send SMTP configs separately

* bring back deleted fields

* actually send stuff over

* remove redundant type, fix comments

* smtp -> smtpConfig

* also send SmtpFrom an StaticHeaders separately

* tests

* restore defaults.ini
2025-06-13 12:44:39 -03:00
Yuri Tseretyan b0ff51a903 Alerting: Support for Mimir configuration in Grafana Alertmanager (#106402) 2025-06-13 16:32:23 +02:00
Kevin MinehartandYuri Tseretyan 910eb1dd9e Security: apply patch 428 (#106710)
* declare dingding url as secret

patch raw settings before parsing because DingDing's config parser does not know about secrets

* fix integration test

---------

Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
2025-06-13 15:56:26 +02:00
Pepe Cano 98ea0193e4 alerting(ui): update the descriptions of the MQTT Message settings (#106601)
alerting(ui): update the description of the MQTT Message settings
2025-06-13 10:12:11 +02:00
Matthew Jacobson 0016b57486 Alerting: Add OAuth2 Support for Webhook Receiver (#106302)
* Add to available channels

* Export

* Fix bug in deeply nested secrets

BE: Slice re-use bug when traversing deeply.

FE: Only at most one level of nesting was being taken into account
when determining secureFields keys. This change adds a new field on
NotificationChannelOption: secureFieldKey. This is populated on API GET via
transform. This change gives us the option to hardcode secureFieldKey in the
backend and no longer calculate the key via settings topology.

* Update grafana/alerting to 3e20fda3b872

* Prettier

* Linting

* Fix IntegrationConfig test to catch secure field mismatch
2025-06-12 23:00:09 +02:00
Fayzal Ghantiwala 85df859589 Alerting: Correctly persist FiredAt in SyncRuleStatePersister (#106658)
Correctly persist FiredAt
2025-06-12 18:07:16 +01:00
Alexander Akhmetov e1ce9ceac1 Alerting: Simplify alert rule unique constraint violation errors (#106608)
Alerting: Simplify alert rule storage unique constraint violation errors
2025-06-12 13:27:08 +02:00
Alexander Akhmetov eaac4a69fb Alerting: Empty endpoints to manage alertmanager configurations (#106546) 2025-06-11 13:45:02 +02:00
Tito Lins 7688089a57 alerting: stop using rule group idx to calculate alert fingerprint (#106407) 2025-06-11 11:49:45 +02:00
Alexander Akhmetov 1a75787e74 Alerting: Send notifications immediately on Error|NoData -> Normal transitions (#106421) 2025-06-10 16:36:30 +02:00
Alexander Akhmetov a4fa8ab891 Alerting: Empty endpoint to load alertmanager config with mimirtool (#106266) 2025-06-10 11:35:57 +02:00
Vadim Stepanov 1df888c517 Alerting: Provisioning API returns 403 on quota exceeded for rule group PUT (#106409) 2025-06-09 11:12:22 +01:00
Alexander Akhmetov 3bb4c92028 Alerting: Fix resolved notifications for same-label Error to Normal transitions (#106210)
What is this feature?

Ensures that resolved notifications are sent when alert states transition from Error to Normal after the configured number of evaluation intervals: Missing series evaluations to resolve.

Why do we need this feature?

Before this change, when an alert was transitioning from Error to Normal, in case when the labels on the new Normal alert instance are the same, Grafana would not send resolved notifications for the Error alert state. The alert would be resolved after a few evaluation intervals automatically in the alertmanager, following the endsAt.

With this change the resolved notification is sent after the configured number of evaluation intervals: Missing series evaluations to resolve.
2025-06-07 14:03:11 +02:00
Alexander Akhmetov f7a52bc04e Alerting: Fix group-level labels and query_offset in the import API (#106379)
What is this feature?

Fixes a bug when group-level query_offset and labels parameters are ignored and not saved

Why do we need this feature?

In the import API Prometheus YAML rule definitions are supported:

groups:
  - name: group-1
    interval: 1m
    query_offset: 10m
    labels:
      severity: "warning"
    rules:
      - alert: Alert 0 > 0
        expr: vector(0) > 0

But applying group-level labels and query_offset is broken and they are not saved right now because during the conversion of the API model to PrometheusRuleGroup they aren't saved to the new structure.
2025-06-06 11:21:39 +02:00
5137995830 Alerting: Add support for Redis Sentinel for Alerting HA (#106322)
* Alerting: Add support for Redis Sentinel

* docs

* docs

* Use minisentinel in test

* Apply suggestions from code review

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
Co-authored-by: Fayzal Ghantiwala <114010985+fayzal-g@users.noreply.github.com>

* "address(es)" -> "address or addresses"

* make update-workspace

* make lint-go-diff

---------

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
Co-authored-by: Fayzal Ghantiwala <114010985+fayzal-g@users.noreply.github.com>
2025-06-05 15:02:40 +01:00
Moustafa BaiouandKonrad Lalik 941162ca79 Alerting: Optimize prometheus api permission checks (#106299)
* Alerting: Optimize prometheus api permission checks

This improves the performance of the Prometheus API by performing the permission checks for rule read permission in a folder upfront, rather than checking permissions for each rule group individually. This reduces the number of permission checks and should speed up the API response time.

* refactor vars

---------

Co-authored-by: Konrad Lalik <konradlalik@gmail.com>
2025-06-05 09:43:06 -04:00
Alexander Akhmetov 82549ea8b3 Alerting: Add state label to prometheus_imported_rules metric (#106365) 2025-06-05 14:24:48 +02:00
Alexander Akhmetov da88e5912f Alerting: Evaluate all imported from Prometheus rules sequentially (#106295)
What is this feature?

Makes all alert rules imported from a Prometheus YAML or Prometheus-compatible data source evaluate sequentially.

Why do we need this feature?

Currently only alert rules [imported via the API](https://grafana.com/docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/alerting-migration/migration-api/) are evaluated sequentially, because only they have the original alert rule definition in YAML. But alert rules can be imported [in the UI, and from a YAML file](https://grafana.com/docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/alerting-migration/), and they won't be evaluated sequentially which can lead to issues with recording rules.
2025-06-05 12:08:44 +02:00