Alerting: Add backend support for keep_firing_for (#100750)
What is this feature? This PR introduces a new alert rule configuration option, keep_firing_for (Prometheus documentation). keep_firing_for prevents alerts from resolving immediately after the alert condition returns to normal. Instead, they transition into a "Recovering" state and are not considered resolved by the Alertmanager. Once the recovery period ends (or after the next evaluation if it is bigger than keep_firing_for), the alert transitions to "Normal" if it doesn't start alerting again: Before +----------+ +----------+ | Alerting |---->| Normal | +----------+ +----------+ ----- After +----------+ +------------+ +----------+ | Alerting |----->| Recovering |---->| Normal | +----------+ +------------+ +----------+ Why do we need this feature? This feature prevents flapping alerts by adding a recovery period. This helps avoid false resolutions caused by brief alert
This commit is contained in:
@@ -340,6 +340,7 @@ func TestRouteGetRuleByUID(t *testing.T) {
|
||||
gen.WithUniqueGroupIndex(), gen.WithUniqueID(),
|
||||
gen.WithEditorSettingsSimplifiedQueryAndExpressionsSection(true),
|
||||
gen.WithEditorSettingsSimplifiedNotificationsSection(true),
|
||||
gen.WithKeepFiringFor(30*time.Second),
|
||||
).GenerateManyRef(3)
|
||||
require.Len(t, createdRules, 3)
|
||||
ruleStore.PutRule(context.Background(), createdRules...)
|
||||
@@ -358,6 +359,7 @@ func TestRouteGetRuleByUID(t *testing.T) {
|
||||
require.Equal(t, expectedRule.UID, result.GrafanaManagedAlert.UID)
|
||||
require.Equal(t, expectedRule.RuleGroup, result.GrafanaManagedAlert.RuleGroup)
|
||||
require.Equal(t, expectedRule.Title, result.GrafanaManagedAlert.Title)
|
||||
require.Equal(t, int64(expectedRule.KeepFiringFor), int64(*(result.KeepFiringFor)))
|
||||
require.True(t, result.GrafanaManagedAlert.Metadata.EditorSettings.SimplifiedQueryAndExpressionsSection)
|
||||
require.True(t, result.GrafanaManagedAlert.Metadata.EditorSettings.SimplifiedNotificationsSection)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user