public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
62,676 Commits 655 Branches 650 Tags
b50737a76ceaa301d07deda42b8336e20b6d7953
Commit Graph

916 Commits

Author SHA1 Message Date
Seunghun Shin
512c292e04 Alerting: Add jitter support for periodic alert state storage to reduce database load spikes (#111357) 
What is this feature?

This PR implements a jitter mechanism for periodic alert state storage to distribute database load over time instead of processing all alert instances simultaneously. When enabled via the state_periodic_save_jitter_enabled configuration option, the system spreads batch write operations across 85% of the save interval window, preventing database load spikes in high-cardinality alerting environments.

Why do we need this feature?

In production environments with high alert cardinality, the current periodic batch storage can cause database performance issues by processing all alert instances simultaneously at fixed intervals. Even when using periodic batch storage to improve performance, concentrating all database operations at a single point in time can overwhelm database resources, especially in resource-constrained environments.

Rather than performing all INSERT operations at once during the periodic save, distributing these operations across the time window until the next save cycle can maintain more stable service operation within limited database resources. This approach prevents resource saturation by spreading the database load over the available time interval, allowing the system to operate more gracefully within existing resource constraints.

For example, with 200,000 alert instances using a 5-minute interval and 4,000 batch size, instead of executing 50 batch operations simultaneously, the jitter mechanism distributes these operations across approximately 4.25 minutes (85% of 5 minutes), with each batch executed roughly every 5.2 seconds.

This PR provides system-level protection against such load spikes by distributing operations across time, reducing peak resource usage while maintaining the benefits of periodic batch storage. The jitter mechanism is particularly valuable in resource-constrained environments where maintaining consistent database performance is more critical than precise timing of state updates.
 2025-09-29 11:22:36 +02:00
Johnny Kartheiser
82e5019333 alerting docs: update list view filtering description (#110885) 
* alerting docs: update list view filtering description

update the description of the list view filter on the view alert rules page.

* img

* Update view-alert-rules.md

* Update view-alert-rules.md

* edit via lauren
 2025-09-17 14:51:35 +02:00
Johnny Kartheiser
73a241da86 alerting docs: update contact point emails (SE #17651) (#110780) 2025-09-10 19:47:33 +00:00
Alex Bikfalvi
4d55358fd2 docs: Add Grafana-managed recording rules documentation for Tempo (#110097) 
Creates comprehensive guide for configuring recording rules with TraceQL metrics queries,
including Tempo-specific considerations for time ranges, evaluation delays, and examples.

Signed-off-by: Alex Bikfalvi <alex.bikfalvi@grafana.com>
Co-authored-by: Kim Nylander <kim.nylander@grafana.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
 2025-08-28 10:22:56 +02:00
Johnny Kartheiser
fe6985f2ac docs: alerting list view UI changes (#108876) 2025-08-06 15:42:33 +02:00
Alexander Akhmetov
8b5b9b68c2 Alerting: Document Accept header in Prometheus conversion API (#109080) 2025-08-01 22:39:54 +02:00
Alexander Akhmetov
b36a8e84cc Alerting: Document "Get rule group" Prometheus conversion API endpoint (#109075) 2025-08-01 21:15:33 +02:00
Eve Meelan
147df3de08 Pricing update: no more Cloud Advanced (#109056) 
* scrub Cloud Advanced

* prettier edit
 2025-08-01 16:23:58 +00:00
Jack Baldry
0faa03edbe Add snippets for 'Create log alert rules with Grafana Alerting' learning journey (#109059) 2025-08-01 15:57:18 +00:00
Alexander Akhmetov
c827ddf790 Alerting: Add meta-monitoring documentation for GRAFANA_ALERTS (#108785) 2025-07-29 21:54:44 +02:00
Alexander Akhmetov
412704c9de Alerting: Document Mimir compatibility for Prometheus conversion API endpoints (#108741) 
Document Mimir compatibility for Prometheus conversion API endpoints
 2025-07-29 21:53:24 +02:00
renovate[bot]
c94f930950 Update dependency prettier to v3.6.2 (#108689) 
* Update dependency prettier to v3.6.2

* run prettier

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
 2025-07-25 17:47:44 +01:00
Johnny Kartheiser
a49b35a7ec Update _index.md (#108192) 2025-07-18 15:10:16 +01:00
Johnny Kartheiser
08e8a71ad6 alerting docs: activate active timing time (#107928) 
* alerting docs: activate active timing time

rough draft: documentation for active timing feature.

* purdier

* more tweaks

* edits via yuri advice

* prettier
 2025-07-17 10:56:25 -05:00
Blueswen
1cd6ef6b84 Docs: Align Jira documentation with contact point options (#105069) 
Align with option names in contact point.

To avoid confusion, ensure that the documentation aligns with the option names used in the Jira contact point.
 2025-07-10 21:15:04 +00:00
Pepe Cano
61efc8b609 docs(alerting): clarify usage of different Alertmanagers and fix misleading details (#107498) 
* docs(alerting): clarify usage of different Alertmanagers and fix misleading details

* address review changes
 2025-07-02 21:46:29 +02:00
Pepe Cano
f5b79fca55 docs(alerting): performance considerations minor clarifications (#107333) 2025-06-30 07:42:26 +00:00
Alexander Akhmetov
f4b0e793aa Alerting: Document label sanitization in GRAFANA_ALERTS (#107285) 
* Alerting: Document label sanitization in GRAFANA_ALERTS
 2025-06-27 23:33:42 +02:00
Pepe Cano
2f1a6ae171 docs(alerting): Add Detect missing series in Prometheus section to the MissingData guide (#107329) 2025-06-27 21:13:13 +02:00
Pepe Cano
ff4d7d35c6 docs(alerting): add new guidelines to the missing data alerting guide (#107232) 2025-06-26 21:06:34 +02:00
Pepe Cano
5c1b263664 docs(alerting): simplify Intro to Grafana Alerting docs (#106944) 
* docs(alerting): improve `Intro > Alert rule evaluation` docs

* Update Introduction to Grafana Alerting

* Simplify `Intro > Alert rules` and related docs

* minor copy change phrasing GMA and DS differences

* fix vale error
 2025-06-26 12:25:32 +00:00
Pepe Cano
f14492baf8 docs(alerting): prom backend to write ALERTS metric (#107006) 
* docs(alerting): prom backend to write ALERTS metric

* add enterprise label

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

* Update docs/sources/alerting/set-up/meta-monitoring.md

Co-authored-by: Alexander Akhmetov <me@alx.cx>

---------

Co-authored-by: Alexander Akhmetov <me@alx.cx>
 2025-06-24 10:38:52 +02:00
Jack Baldry
244ffad99d Fix all the old usage of admonition syntax (#106984) 2025-06-19 17:31:13 +01:00
Pepe Cano
286a6638b8 docs(alerting): fix Grafana Play links due to provisioning (#106816) 2025-06-17 23:08:46 +02:00
Bryan Boreham
fca89d0d4c Docs: Typo: mediam->median (#106305) 2025-06-16 12:35:29 +03:00
Pepe Cano
493e7ba75f docs(alerting): enhancements for MQTT docs (#106566) 
* docs(alerting): enhancements for MQTT docs

* Update docs/sources/alerting/configure-notifications/manage-contact-points/integrations/configure-mqtt.md

Co-authored-by: Simon Prickett <simon@crudworks.org>

---------

Co-authored-by: Simon Prickett <simon@crudworks.org>
 2025-06-13 10:27:08 +02:00
Pepe Cano
79ff67268f docs(alerting): Add Tutorials directory page under Best Practices (#106159) 
* docs(alerting): Add Tutorials directoy page under Best Practices

* run prettier

* Include latest tutorials

* fix tutorial list
 2025-06-10 16:10:20 +00:00
Pepe Cano
f76e4f8fda docs(alerting): Import to Grafana-managed rules (#106384) 
* docs(alerting): Import to Grafana-managed rules

* apply latest evaluation changes

* Add additional conversion details to How it works section

* fix ref link

* fix Data source input name

* more details about the `Target data source` input
 2025-06-10 17:22:48 +02:00
Pepe Cano
0d0aa35ba7 docs(alerting): add a short new guideline for handling NoData scenarios. (#106412) 
docs(alerting): add consideration for handling NoData scenarios
 2025-06-06 21:19:21 +02:00
Vadim Stepanov
5137995830 Alerting: Add support for Redis Sentinel for Alerting HA (#106322) 
* Alerting: Add support for Redis Sentinel

* docs

* docs

* Use minisentinel in test

* Apply suggestions from code review

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
Co-authored-by: Fayzal Ghantiwala <114010985+fayzal-g@users.noreply.github.com>

* "address(es)" -> "address or addresses"

* make update-workspace

* make lint-go-diff

---------

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
Co-authored-by: Fayzal Ghantiwala <114010985+fayzal-g@users.noreply.github.com>
 2025-06-05 15:02:40 +01:00
Alexander Akhmetov
da88e5912f Alerting: Evaluate all imported from Prometheus rules sequentially (#106295) 
What is this feature?

Makes all alert rules imported from a Prometheus YAML or Prometheus-compatible data source evaluate sequentially.

Why do we need this feature?

Currently only alert rules [imported via the API](https://grafana.com/docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/alerting-migration/migration-api/) are evaluated sequentially, because only they have the original alert rule definition in YAML. But alert rules can be imported [in the UI, and from a YAML file](https://grafana.com/docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/alerting-migration/), and they won't be evaluated sequentially which can lead to issues with recording rules.
 2025-06-05 12:08:44 +02:00
Pepe Cano
afdf1441f7 docs(alerting): Migration API format enhancements (#106317) 
* docs(alerting): Migration API format enhancements

* fix canonical URL

* small command change

* More changes
 2025-06-05 00:55:31 +02:00
Pepe Cano
be8d153dc3 docs(alerting): updates Missing series evaluations to resolve option (#106319) 
docs(alerting): correct detailed process of the missingSeriesToEval option
 2025-06-05 00:05:22 +02:00
Pepe Cano
f36d5c5206 docs(alerting): new notification template example: link to a dashboard with time range (#106254) 
* docs(alerting): new notification template example: link to a dashboard with time range

* remind configuring the `MyDashboardURL` annotation

* Update docs/sources/alerting/configure-notifications/template-notifications/examples.md

Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>

---------

Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
 2025-06-03 17:16:43 +02:00
Pepe Cano
cce28ec351 docs(alerting): updates link to outdated tutorial (#106178) 
* docs(alerting): remove admonition to an outdated tutorial

* amended note and link to new tutorial

* link

* minor copy changes

* run prettier

* update copy

---------

Co-authored-by: tonypowa <tonypowa@gmail.com>
 2025-06-03 14:20:24 +00:00
Johnny Kartheiser
de0c639417 Alerting docs: document Prometheus YAML import tool update (#106219) 
* Alerting docs: document ability to import rules to GMA from Prometheus YAML

documentation for prom YAML import feature.
- took out reference that said users can't import prometheus rules in the tool.
- added the More > Import alert rules UI path to the tool
- added a step for the YAML file upload button.

* 💅prettier✨

* Update _index.md
 2025-06-03 09:20:02 -05:00
Alexander Akhmetov
e256f2d5e2 Alerting: Enable recording rules by default (#105603) 2025-06-02 10:56:05 +02:00
Pepe Cano
e1400e67d3 docs(alerting): document Keep firing for in Alert rule evaluation intro (#106139) 2025-05-29 18:24:26 +02:00
Pepe Cano
b2d317cc23 docs(alerting): add notes for Redis HA setup (#106144) 
* docs(alerting): add note for Redis HA setup

* run prettier
 2025-05-29 11:21:32 -05:00
Pepe Cano
8de013c7e6 docs(alerting): OpsGenie instructions for other alertmanagers (#106049) 
* docs(alerting): OpsGenie instructions for other alertmanagers

* run prettier
 2025-05-28 10:59:05 +02:00
Pepe Cano
c84388f550 docs(alerting): Examples of dynamic labels and dynamic thresholds (#105776) 
* docs(alerting): Add dynamic thresholds example

* update intro

* docs(alerting): Example of dynamic labels

* fix template example

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-thresholds.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-thresholds.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-thresholds.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/best-practices/dynamic-labels.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* fix typo

* fix typo

* Add section `CSV data with Infinity` in Table data example

* Link dynamic threshold example to tabular data requirements

* minor copy changes

* minor heading fix

* Add links (admonition) to Grafana Play examples

* Use `Caveat` instead of `Gotcha`

* Dynamic thresholds: caution message when Math operates on missing series

* Exampleof latency threshold based on traffic

---------

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
 2025-05-27 17:36:32 +02:00
Alexander Akhmetov
d57ce72969 Alerting: Document X-Grafana-Alerting-Notification-Settings header (#105300) 2025-05-19 22:01:13 +02:00
Alexander Akhmetov
d5a8401499 Alerting: Mention recording rules setup in the import API documentation (#105565) 
* Alerting: Update recording rules mentions in the import API documentation

* Apply suggestions from code review

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

---------

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
 2025-05-19 19:01:46 +02:00
Alexander Akhmetov
c58ac15031 Alerting: Remove grafanaManagedRecordingRules feature flag (#105569) 2025-05-19 12:15:49 +02:00
Johnny Kartheiser
bf87c6f774 docs: edits for alerting learning content (#105500) 
* docs: edits for alerting learning content

edits for alerting learning content

* vale'd

* left nav change

* final adjustments

link fixes and the like

* Update _index.md
 2025-05-18 22:08:52 -05:00
Sonia Aguilar
f7cad3a2fe Alerting: Update docs rules bulk actions (#105442) 
* enable pause/unpause bulk actions

* Update docs for pause/resume rules evaluations within a folder
 2025-05-15 16:06:15 +02:00
Pepe Cano
bb04b9c55b docs(alerting): MissingSeriesEvalsToResolve and New Stale alert instances docs page (#105415) 
* docs(alerting): New `Stale alert instances` docs page and document the `Missing series evaluations to resolve` option.

* update meta description
 2025-05-15 13:36:59 +02:00
Pepe Cano
4ae91715df docs(alerting): Add two common examples in Learn section (#105325) 
* docs(alerting): Add two common examples in `Learn` section

* Update docs/sources/alerting/learn/examples/multi-dimensional-alerts.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* Update docs/sources/alerting/learn/examples/multi-dimensional-alerts.md

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>

* mention `summary` annotation in multi-dimensional alerts example

* Remove note about alert grouping

* minor edits to section: `Differences with time series`

* minor grammar change

---------

Co-authored-by: Johnny Kartheiser <140559259+JohnnyK-Grafana@users.noreply.github.com>
 2025-05-14 17:10:32 +02:00
Pepe Cano
b53f68ea5e docs(alerting): Correct gotcha when using dynamic labels (#105347) 2025-05-14 10:00:30 +02:00
Johnny Kartheiser
a26d907dae docs: bulk delete alert rule group (#105087) 
* docs: bulk delete alert rule group

first draft

* icon name fix

* Update docs/sources/alerting/alerting-rules/create-data-source-managed-rule.md

Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>

* Update docs/sources/alerting/alerting-rules/create-data-source-managed-rule.md

Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>

* Update docs/sources/alerting/alerting-rules/create-grafana-managed-rule.md

Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>

* Update docs/sources/alerting/alerting-rules/create-grafana-managed-rule.md

Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>

* prettier

* prettiest

---------

Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>
 2025-05-13 16:32:15 -05:00