Adds HMAC-SHA256 signature support to webhook notifications, providing a way to verify the authenticity and integrity of webhook requests. The implementation allows to specify the header in which the signature will be sent. The signature is calculated from the request body.
An optional timestamp header name can be provided. If set, the HMAC signature will be generated by concatenating the timestamp, a ":" and the request body: {timestamp}:{body}. The timestamp will also be sent in the provided header name.
When creating Grafana-managed alerts from Prometheus rule definitions with mimirtool or cortextool, the rules are marked as "provisioned" and are not editable in the Grafana UI. This PR allows changing this by providing an extra header: --extra-header="X-Disable-Provenance=true".
When provenance is disabled, we do not keep the original rule definition in YAML, so it is impossible to read it back using the Prometheus conversion API (mimirtool/cortextool). This is intentional because if we did keep it and the rule was later changed in the UI, its Prometheus YAML definition would no longer reflect the latest version of the alert rule, as it would be unchanged.
The remote write path differs based on whether the data source is actually
Prometheus, Mimir, Cortex, or an older version of Cortex. We do not want
users to have to specify the path, so this change determines the path as
best it can.
It may be in the future we have to make this configurable per-datasource
to cater for setups where it's impossible to determine the correct path.
Alerting: Include time range in templated dashboard and panel urls
Time range:
from=Alert.StartsAt-1hr
Firing Alerts: to=Current Timestamp
Resolved Alerts: to=Alert.EndsAt
What is this feature?
Adds target datasource UID to the recording rules so that they write to the same datasource used for alerting rule queries after the import.
Why do we need this feature?
Target datasourse support was added in #101678, and under a feature flag grafanaManagedRecordingRulesDatasources (#101778).
This PR makes the importing process:
Check if the import contains recording rules
Verify both recording rules and the grafanaManagedRecordingRulesDatasources feature flag are enabled
If either check fails, return an error
If both checks pass, create recording rules with the provided datasource UID set as both the query and target datasource
* Alerting: Allow selection of recording rule write target on per-rule basis.
Introduces a new feature flag (`grafanaManagedRecordingRulesDatasources`),
disabled by default, to enable the ability to write recording rules data using
data source settings, and selecting the data source to use on a per-rule basis.
To cope with the scenario of users upgrading, a configuration file option
allows setting the default data source to use, if none is specified in the rule,
emulating the behaviour of recording rules without the flag enabled.
* Lint
* Update conf/sample.ini
Co-authored-by: Alexander Akhmetov <me@alx.cx>
---------
Co-authored-by: Alexander Akhmetov <me@alx.cx>
The change to use WriteDatasource was done in a previous commit, this adds a
test case using DatasourceWriter, in addition to the one using PrometheusWriter.
* Alerting: Implement arbitrary data source remote writer.
Recording rules currently remote write to the endpoint speficied in the
configuration file, but we want to change this so that it can be specified
on a per-rule basis. This change implements the updated RemoteWriter
interface, and dispatches writes to the correct data source.
Notes about the implementation:
- There is a defaulting behaviour, because we will need to handle situations
when upgrading from older Grafana versions, where recording rules do not a
data source target specified.
- In order to avoid re-creating the remote write clients for every request,
a cache is used. The entries have a TTL of 30 seconds at which point they
are recreated; this is the delay for picking up new data source settings.
Known limitations:
- There is no per data source configuration for the remote write path suffix.
- Not all data source settings will be used, but basic auth and TLS will be.
- There is no RBAC. This will be a follow up feature.
* Lint
* Alerting: Refactor NewPrometheusWriter function.
In order to re-use PrometheusWriter, changing the function take a
PrometheusWriterConfig instead of RecordingRulesSettings, and adapt the old
interface onto the new interface.
* Make linter happy
Extend the recording rule definition to include the target data source, allowing
configuration of where the output of the recording rule is written to. Also
extends the relevant interfaces in preparation for the next set of changes.
* add feature toggle alertRuleRestore
* Update delete rule to require UserUID, remove all versions and create "delete" version that holds information about who and when deleted the rule
What is this feature?
Allows the creation of alert rules with mimirtool in a specified folder.
Why do we need this feature?
Currently, the APIs for mimirtool create namespaces and rule groups in the root folder without the ability to set a custom folder. For example, it could be a special "Imported" folder, etc.
This PR makes it possible with a special header: mimirtool ... --extra-headers="X-Grafana-Alerting-Folder-UID=123". If it's not present, the root folder is used, otherwise, the specified one is used.
mimirtool does not support nested folder structures, while Grafana allows folder nesting. To keep compatibility, we return only direct child folders of the working folder (as namespaces) with rule groups and rules that are directly in these child folders as if there are no nested folders.
For example, given this folder structure in Grafana:
```
grafana/
├── production/
│ ├── service1/
│ │ └── alerts/
│ └── service2/
└── testing/
└── service3/
```
If the working folder is "grafana":
Only namespaces "production" and "testing" are returned
Only rule groups directly within these folders are included
If the working folder is "production":
- Only namespaces "service1" and "service2" are returned
Only rule groups directly within these folders are included
Expand template testing to try additional scopes if the root scope fails.
This mitigates errors for definitions like pagerduty.default.instances,
which require the .Alerts scope. Added support for .Alerts and .Alert
scopes.
* add column guid to alert rule table and rule_guid to rule version table
+ populate the new field with UUID
* update storage and domain models
* patch GUID
* ignore GUID in fingerprint tests
Initially, Metadata had only the EditorSettings, and HasMetadata was used to understand if the incoming update request had metadata in the body because it could be omitted if it was empty. For example, when the rule is updated via the provisioning API or has only false values. If it was in the request, we used that; if not, we used the metadata from the existing rule from the database. If the rule was updated via the AlertRuleService, we didn't change Metadata at all if the rule already existed.
But now, Metadata also has the Prometheus rule definition, and we always need to update it with the new version of the AlertRuleService when the rule exists in the DB and has the same UID. HasMetadata is renamed to HasEditorSettings to keep the old behaviour only for EditorSettings.
Now, the provisioning API and the conversion API will overwrite everything except EditorSettings with the new data.
When you use a math expression with out any operators, the dataFrame pointer is identical between the expression result and the input query/expression.
This was resulting in the values returned from an evaluation overshadowing each other, depending on the order of the processing of the result map.
For example:
```
A: some_metric
B: reduce of A
C: math expression -> "${B}"
D: Threshold evaluation of C -> "C > 0"
```
With a value of 1 for `some_metric`, might result in a evaluation result of one of the following (somewhat at random):
1. { B: 1, D: 1 }
2. { C: 1, D: 1}
While you would expect to see:
{ B: 1, C: 1, D: 1 }
update scheduler's aler rule to accept regular Evaluation in update channel
This makes it accept the full rule definition, which is required in reset state.
What is this feature?
Adds an API endpoint to create alert rules with mimirtool:
- POST /convert/prometheus/config/v1/rules/{NamespaceTitle} - Accepts a single rule group in a Prometheus YAML format and creates or updates a Grafana rule group from it.
The endpoint uses the conversion package from #100224.
Key parts
The API works similarly to the provisioning API. If the rule does not exist, it will be created, otherwise updated. Any rules not present in the new group will be deleted, ensuring the group is fully synchronized with the provided configuration.
Since the API works with namespace titles (folders), the handler automatically creates a folder in the root based on the provided title if it does not exist. It also requires a special header, X-Grafana-Alerting-Datasource-UID. This header specifies which datasource to use for the new rules.
If the rule group's evaluation interval is not specified, it uses the DefaultRuleEvaluationInterval from settings.
* Send new annotation containing image url
* Use new image TokenProvider with TokenStore
New abstraction GetImage no longer needs to support parsing both token and
url from annotations, as remote AM will use the new URLProvider. Instead, we
use the new generic TokenProvider and give it a TokenStore backed by the
grafana database.
That means we revert back to always using token simplifying code and security
considerations.
* Upgrade grafana/alerting to merged commit SHA
* split create to create and patch and move to state
patch will be refactored further
* move setNextState to state transition
* move tests
* split tests for patch function
Alexander Akhmetov
Steve Simpson
Yuri Tseretyan
Matthew Jacobson
Ryan McKinley
Moustafa Baiou
Pepe Cano
Santiago
Stephanie Hingtgen
Peter Štibraný