From 30d38c1775381d84328ea0006b62ec97577ff13e Mon Sep 17 00:00:00 2001 From: George Robinson Date: Mon, 22 Apr 2024 10:13:02 +0100 Subject: [PATCH] Update the docs for state_periodic_save_interval (#86611) This commit updates the docs for state_periodic_save_interval following a question in the forums about what the consequences of this change could be. --- .../sources/alerting/set-up/performance-limitations/index.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/sources/alerting/set-up/performance-limitations/index.md b/docs/sources/alerting/set-up/performance-limitations/index.md index 5cd9a894462..ce38f8545b2 100644 --- a/docs/sources/alerting/set-up/performance-limitations/index.md +++ b/docs/sources/alerting/set-up/performance-limitations/index.md @@ -67,3 +67,8 @@ can also be configured using the `state_periodic_save_interval` configuration fl The time it takes to write to the database periodically can be monitored using the `state_full_sync_duration_seconds` metric that is exposed by Grafana. + +If Grafana crashes or is force killed, then the database can be up to `state_periodic_save_interval` seconds out of date. +When Grafana restarts, the UI might show incorrect state for some alerts until the alerts are re-evaluated. +In some cases, alerts that were firing before the crash might fire again. +If this happens, Grafana might send duplicate notifications for firing alerts.