* Move filtering code to generators for performance reasons
Discarding rules and groups early in the iterable chain limits the number of promises we need to wait for which improves performance significantly
* Add error handling for generators
* Add support for data source filter for GMA rules
* search WIP fix
* Fix datasource filter
* Move filtering back to filtered rules hook, use paged groups for improved performance
* Add queriedDatasources field to grafana managed rules and update filtering logic to rely on it
- Introduced a new field `queriedDatasources` in the AlertingRule struct to track data sources used in rules.
- Updated the Prometheus API to populate `queriedDatasources` when creating alerting rules.
- Modified filtering logic in the ruleFilter function to utilize the new `queriedDatasources` field for improved data source matching.
- Adjusted related tests to reflect changes in rule structure and filtering behavior.
* Add FilterView performance logging
* Improve GMA Prometheus types, rename queried datasources property
* Use custom generator helpers for flattening and filtering rule groups
* Fix lint errors, add missing translations
* Revert test condition
* Refactor api prom changes
* Fix lint errors
* Update backend tests
* Refactor rule list components to improve error handling and data source management
- Enhanced error handling in FilterViewResults by logging errors before returning an empty iterable.
- Simplified conditional rendering in GrafanaRuleLoader for better readability.
- Updated data source handling in PaginatedDataSourceLoader and PaginatedGrafanaLoader to use new individual rule group generator.
- Renamed toPageless function to toIndividualRuleGroups for clarity in prometheusGroupsGenerator.
- Improved filtering logic in useFilteredRulesIterator to utilize a dedicated function for data source type validation.
- Added isRulesDataSourceType utility function for better data source type checks.
- Removed commented-out code in PromRuleDTOBase for cleaner interface definition.
* Fix abort controller on FilterView
* Improve generators filtering
* fix abort controller
* refactor cancelSearch
* make states exclusive
* Load full page in one loadResultPage call
* Update tests, update translations
* Refactor filter status into separate component
* hoist hook
* Use the new function for supported rules source type
---------
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
This makes it so that it is:
- No longer required to have datasource permissions to delete a rule.
- No longer required to have datasource permissions to update non-query related fields of a rule.
* wip
* Add prom flavor support for data source variables and export/import dashboards (#103321)
* add dashboard and data source var selection
* use match plugin id instead
* use updated matchpluginid
* formatting
* cleanup
* regex anchor
* update error msg
* Alerting: Clean up prometheus-flavored types and functions (#103703)
* clean up types and utility functions for dealing with
prometheus-flavored data sources
* Refactor alerting datasource types to use constants as source of truth
* Alerting: Clean up prometheus-flavored types and functions on the bac… (#103716)
Alerting: Clean up prometheus-flavored types and functions on the backend
* add matchPluginId tests
* Update matchPluginId func to bidirectional (#103746)
* update matchpluginid func to bidirectional
* lint
* formatting
* use actual isSupportedExternalRulesSourceType in test
* add tests in datasource_srv
* betterer
* remove type assertion
* remove unnecessary case
* use satisifies to not have to convert tuple to an array of string
* add prometheus_flavor test
---------
Co-authored-by: Andrew Hackmann <5140848+bossinc@users.noreply.github.com>
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
Co-authored-by: Alexander Akhmetov <me@alx.cx>
* placeholder commit
* Complete function in api_convert_prometheus.go
* MVP before extensive testing
* Cleanup
* Updated tests
* cleanup
* Fix random logs and lint
* Remove comment
* Fix errors after rebase
* Update test
* Update swagger
* swagger
* Refactor to accept groups in body
* Fix auth tests and some cleanup
* Some cleanup before refactoring
* Remove unnecessary fields
* Also refactor RouteConvertPrometheusPostRuleGroup
* Remove unused code
* Rebase + cleanup
* Update authorization_test
* address comments
* Regen swagger files
* Remove namespace and group filters
* Final comments
What is this feature?
A follow-up for #101184, adds AlertRule.MissingSeriesEvalsToResolve to the APIs.
missing_series_evals_to_resolve must be specified too and it must be > 0.
POST /api/ruler/grafana/api/v1/rules/{folderUID} works in the following way:
If missing_series_evals_to_resolve is not sent or null, the rule keeps its existing value
If missing_series_evals_to_resolve > 0: updates to that value
If missing_series_evals_to_resolve = 0: resets to default (nil).
AlertRule.MissingSeriesEvalsToResolve can't be 0, so I used it to reset
In the Provisioning API, the value is just set if present and > 0. Otherwise it's reset:
PUT to /api/v1/provisioning/alert-rules/{UID}:
If missing_series_evals_to_resolve is nil, it's reset to the default value
If missing_series_evals_to_resolve > 0, it's updated
Adds support for rule group-level query_offset in Prometheus to Grafana rule conversion. It allows specifying a time offset for rule evaluation, which gets applied and saved during the conversion.
Adds a new configuration option to specify a time offset for rule evaluation, which gets applied and saved during the Prometheus -> Grafana conversion. For example:
[unified_alerting.prometheus_conversion]
rule_query_offset = 1m
Changing this option affects only the rules imported after the change. If query_offset is set at the group level, it takes precedence over this setting.
Default is set to 1m.
* Alerting: Fix loss of TimeInterval location on remote AM apply
deepcopy.Copy does not correctly copy PostableUserConfig because it ignores
unexported fields. As a result, TimeInterval locations default to UTC instead
of retaining their original values.
* make update-workspace
What is this feature?
This PR introduces a new alert rule configuration option, keep_firing_for (Prometheus documentation).
keep_firing_for prevents alerts from resolving immediately after the alert condition returns to normal. Instead, they transition into a "Recovering" state and are not considered resolved by the Alertmanager. Once the recovery period ends (or after the next evaluation if it is bigger than keep_firing_for), the alert transitions to "Normal" if it doesn't start alerting again:
Before
+----------+ +----------+
| Alerting |---->| Normal |
+----------+ +----------+
-----
After
+----------+ +------------+ +----------+
| Alerting |----->| Recovering |---->| Normal |
+----------+ +------------+ +----------+
Why do we need this feature?
This feature prevents flapping alerts by adding a recovery period. This helps avoid false resolutions caused by brief alert
* add query parameter to existing APIs to control the permanent deletion of rules
* add GUID to gettable rule
* add new endpoint /ruler/grafana/api/v1/trash/rule/guid/{RuleGUID} to delete rules from trash permanently
---------
Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
When creating Grafana-managed alerts from Prometheus rule definitions with mimirtool or cortextool, the rules are marked as "provisioned" and are not editable in the Grafana UI. This PR allows changing this by providing an extra header: --extra-header="X-Disable-Provenance=true".
When provenance is disabled, we do not keep the original rule definition in YAML, so it is impossible to read it back using the Prometheus conversion API (mimirtool/cortextool). This is intentional because if we did keep it and the rule was later changed in the UI, its Prometheus YAML definition would no longer reflect the latest version of the alert rule, as it would be unchanged.
What is this feature?
Adds target datasource UID to the recording rules so that they write to the same datasource used for alerting rule queries after the import.
Why do we need this feature?
Target datasourse support was added in #101678, and under a feature flag grafanaManagedRecordingRulesDatasources (#101778).
This PR makes the importing process:
Check if the import contains recording rules
Verify both recording rules and the grafanaManagedRecordingRulesDatasources feature flag are enabled
If either check fails, return an error
If both checks pass, create recording rules with the provided datasource UID set as both the query and target datasource
Extend the recording rule definition to include the target data source, allowing
configuration of where the output of the recording rule is written to. Also
extends the relevant interfaces in preparation for the next set of changes.
* add feature toggle alertRuleRestore
* Update delete rule to require UserUID, remove all versions and create "delete" version that holds information about who and when deleted the rule
What is this feature?
Allows the creation of alert rules with mimirtool in a specified folder.
Why do we need this feature?
Currently, the APIs for mimirtool create namespaces and rule groups in the root folder without the ability to set a custom folder. For example, it could be a special "Imported" folder, etc.
This PR makes it possible with a special header: mimirtool ... --extra-headers="X-Grafana-Alerting-Folder-UID=123". If it's not present, the root folder is used, otherwise, the specified one is used.
mimirtool does not support nested folder structures, while Grafana allows folder nesting. To keep compatibility, we return only direct child folders of the working folder (as namespaces) with rule groups and rules that are directly in these child folders as if there are no nested folders.
For example, given this folder structure in Grafana:
```
grafana/
├── production/
│ ├── service1/
│ │ └── alerts/
│ └── service2/
└── testing/
└── service3/
```
If the working folder is "grafana":
Only namespaces "production" and "testing" are returned
Only rule groups directly within these folders are included
If the working folder is "production":
- Only namespaces "service1" and "service2" are returned
Only rule groups directly within these folders are included
Expand template testing to try additional scopes if the root scope fails.
This mitigates errors for definitions like pagerduty.default.instances,
which require the .Alerts scope. Added support for .Alerts and .Alert
scopes.
Initially, Metadata had only the EditorSettings, and HasMetadata was used to understand if the incoming update request had metadata in the body because it could be omitted if it was empty. For example, when the rule is updated via the provisioning API or has only false values. If it was in the request, we used that; if not, we used the metadata from the existing rule from the database. If the rule was updated via the AlertRuleService, we didn't change Metadata at all if the rule already existed.
But now, Metadata also has the Prometheus rule definition, and we always need to update it with the new version of the AlertRuleService when the rule exists in the DB and has the same UID. HasMetadata is renamed to HasEditorSettings to keep the old behaviour only for EditorSettings.
Now, the provisioning API and the conversion API will overwrite everything except EditorSettings with the new data.
What is this feature?
Adds an API endpoint to create alert rules with mimirtool:
- POST /convert/prometheus/config/v1/rules/{NamespaceTitle} - Accepts a single rule group in a Prometheus YAML format and creates or updates a Grafana rule group from it.
The endpoint uses the conversion package from #100224.
Key parts
The API works similarly to the provisioning API. If the rule does not exist, it will be created, otherwise updated. Any rules not present in the new group will be deleted, ensuring the group is fully synchronized with the provided configuration.
Since the API works with namespace titles (folders), the handler automatically creates a folder in the root based on the provided title if it does not exist. It also requires a special header, X-Grafana-Alerting-Datasource-UID. This header specifies which datasource to use for the new rules.
If the rule group's evaluation interval is not specified, it uses the DefaultRuleEvaluationInterval from settings.
Konrad Lalik
Moustafa Baiou
Kevin Yu
Mariell Hoversholm
Alexander Akhmetov
Will Assis
maicon
Fayzal Ghantiwala
William Wernert
Matthew Jacobson
Yuri Tseretyan
Steve Simpson
Ryan McKinley
Stephanie Hingtgen
Peter Štibraný