Provisioning: Look up provisioned folders by UID when possible (#87465)
look up provisioned folders by uid when possible
(cherry picked from commit 1884b03511)
* Snapshots: Require delete within same org (backport) (#84707)
* check orgId on delete
* test from main
(cherry picked from commit d80f83be01)
* manual fix
Chore: Bump update checker interval to 1 day (#84404)
* Bump interval to 1hr
* 2 hours is better than 1
* Bump further to 1 day
(cherry picked from commit 391d14d091)
Co-authored-by: Andreas Christou <andreas.christou@grafana.com>
* Annotations: Split cleanup into separate queries and deletes to avoid deadlocks on MySQL (#80329)
* Split subquery when cleaning annotations
* update comment
* Raise batch size, now that we pay attention to it
* Iterate in batches
* Separate cancellable batch implementation to allow for multi-statement callbacks, add overload for single-statement use
* Use split-out utility in outer batching loop so it respects context cancellation
* guard against empty queries
* Use SQL parameters
* Use same approach for tags
* drop unused function
* Work around parameter limit on sqlite for large batches
* Bulk insert test data in DB
* Refactor test to customise test data creation
* Add test for catching SQLITE_MAX_VARIABLE_NUMBER limit
* Turn annotation cleanup test to integration tests
* lint
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
(cherry picked from commit 81c45bfe44)
* Fix logs and interval per backport
* empty commit to kick actions
Release: Deprecate latest.json and replace with api call to grafana.com (#80537)
* remove latest.json and replace with api call to grafana.com
* remove latest.json
* Revert "remove latest.json"
This reverts commit bcff43d898.
* Revert "remove latest.json and replace with api call to grafana.com"
This reverts commit 02b867d84e.
* add deprecation message to latest.json
(cherry picked from commit 127decee1e)
Auth: id response header (#77871)
* Add config options for identity id response header
* Add feature to add identity id response header to all responses
* Use util.SplitString
(cherry picked from commit 21f94c5b78)
Alerting: Attempt to retry retryable errors (#79161)
* Alerting: Attempt to retry retryable errors
Retrying has been broken for a good while now (at least since version 9.4) - this change attempts to re-introduce them in their simplest and safest form possible.
I first introduced #79095 to make sure we don't disrupt or put additional load on our customer's data sources with this change in a patch release. Paired with this change, retries can now work as expected.
There's two small differences between how retries work now and how they used to work in legacy alerting.
Retries only occur for valid alert definitions - if we suspect that that error comes from a malformed alert definition we skip retrying.
We have added a constant backoff of 1s in between retries.
---------
(cherry picked from commit c631261681)
Signed-off-by: gotjosh <josue.abreu@gmail.com>
Unified Alerting: Set `max_attempts` to 1 by default (#79095)
* Unified Alerting: Set `max_attempts` to 1 by default
The retry logic for unified alerting has been broken as far as v9.4.x, rather than fixing it in one go and causing a headache to our users with rules putting extra load on their datasources - I think a better approach is to simply set 1 as a default and then let our users change it.
I see two cons with this approach:
- Configuration for legacy to unified alerting cannot be ported over automatically, users will have to manually set `max_attempts` to 3 when migrating.
- Users expecting to get any sort of retrying (as with legacy alerting) will not have it out of the box and will have to manually edit the configuration.
Signed-off-by: gotjosh <josue.abreu@gmail.com>
---------
Signed-off-by: gotjosh <josue.abreu@gmail.com>
(cherry picked from commit 0c9356a3c7)
* Alerting: Fix state manager to not keep datasource_uid and ref_id labels in state after Error (#72216)
(cherry picked from commit 78fc3bcdf4)
* user QueryError
---------
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
* CI: Update CI/CD tooling and pipelines from main (#76814)
* CI: Update CI/CD tooling and pipelines from main
* Update Makefile
* Comment out validate_openapi_spec_step
* Update broken frontend tests
* Fix validate-npm-packages regex to work without suffix
* Fix cypress image version
(cherry picked from commit 03ecb1db39)
* Comment out kindsysreport/codegen/report.go on Makefile
* Disable steps that don't work for older versions
* Fix permissions of validate-npm-packages.sh
Prometheus: Handle the response with different field key order (#74567)
* Handle the response with different field key order
* More unit tests to cover edge cases
* Cover more edge cases
* make it simpler
* Better test inputs
(cherry picked from commit 3107459e57)
* [LDAP] Disable removed users on login (#74016)
* [LDAP] Disable removed users on login
* Fix tests
* Add test for user disabling
* Add tests for disabling user behind auth proxy
* Linting.
* Rename setup func
* Account for reviews comments
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
---------
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
(cherry picked from commit f900098cc9)
* manual backport of #74016
* LDAP: Fix active sync with large quantities of users (#73834)
* Fix middleware test
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
Co-authored-by: Gabriel MABILLE <gabriel.mabille@grafana.com>
* SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData) (#71246)
* introduce a function checkIfSeriesNeedToBeFixed to scan all value fields in the response and provide a function that updates Series so they can be uniquely identifiable. Only Graphite and TestData are checked.
* update `DSNode.Execute` to run this function and provide it to WideToMany
* update WideToMany to run the fix function if it is not nil
Grafana versioning: Remove `pre` suffix from Grafana version (#73357)
* Remove pre suffix from version
* Fix tests
(cherry picked from commit 0aba319ea0)
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
* Alerting: Support concurrent queries for saving alert instances (#70525)
This commit adds support for concurrent queries when saving alert
instances to the database. This is an experimental feature in
response to some customers experiencing delays between rule evaluation
and sending alerts to Alertmanager, resulting in flapping. It is
disabled by default.
(cherry picked from commit 7edbe72483)
* Trigger PR automation
---------
Co-authored-by: George Robinson <george.robinson@grafana.com>
Alerting: Add debug logs for EndsAt timestamp (#70336)
This commit adds debug logs for previous_ends_at and next_ends_at
to state.go to help us debug issues where alerts are resolved in
Alertmanager due to expiration. This change is in response to a
support escalation where this information was needed but unavailable.
(cherry picked from commit 815e98ed95)
Co-authored-by: George Robinson <george.robinson@grafana.com>
GenericOAuth: Set sub as auth id (#65902)
* GenericOAuth: Set sud as auth id
* GenericOAuth: Extract function to reduce complexity
(cherry picked from commit 7cd6018253)
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Alerting: Improve performance of matching captures (#71828)
This commit updates eval.go to improve the performance of matching
captures in the general case. In some cases we have reduced the
runtime of the function from 10s of minutes to a couple 100ms.
In the case where no capture matches the exact labels, we revert to
the current subset/superset match, but with a reduced search space
due to grouping captures.
(cherry picked from commit 8dd3eb856d)
* Add label fingerprints from grafana-plugin-sdk-go
* Remove unsafe.StringData as we use Go 1.19
* Fix lint
---------
Co-authored-by: George Robinson <george.robinson@grafana.com>
Alerting: Add tests for matching captures (#71928)
This commit adds tests for matching captures, which we do not have
at present.
(cherry picked from commit f1af0502db)
Co-authored-by: George Robinson <george.robinson@grafana.com>
Alerting: Sort NumberCaptureValues in EvaluationString (#71927)
This commit changes extractEvalString to sort NumberCaptureValues
in ascending order of Var before building the output string. This
means that users will see EvaluationString in a consistent order,
but also make it possible to assert its output in tests.
(cherry picked from commit 89dcaaf049)
Co-authored-by: George Robinson <george.robinson@grafana.com>
Alerting: No longer silence paused alerts during legacy migration (#71596)
* Alerting: No longer silence paused alerts during legacy migration
Now that we migrate paused legacy alerts to paused UA alert rules, we no longer need to silence them.
(cherry picked from commit 8c6cdf51fc)
Plugins: Only configure plugin proxy transport once (#71735)
only configure plugin proxy transport once
(cherry picked from commit b59ca7fb22)
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Alerting: Fix unique violation when updating rule group with title chains/cycles (#67868)
* Alerting: Fix unique violation when updating rule group with title chains/cycles
The uniqueness constraint for titles within an org+folder is enforced on every update within a transaction instead of on commit (deferred constraint). This means that there could be a set of updates that will throw a unique constraint violation in an intermediate step even though the final state is valid. For example, a chain of updates RuleA -> RuleB -> RuleC could fail if not executed in the correct order, or a swap of titles RuleA <-> RuleB cannot be executed in any order without violating the constraint.
The exact solution to this is complex and requires determining directed paths and cycles in the update graph, adding in temporary updates to break cycles, and then executing the updates in reverse topological order (see first commit in PR if curious).
This is not implemented here.
Instead, we choose a simpler solution that works in all cases but might perform more updates than necessary. This simpler solution makes a determination of whether an intermediate collision could occur and if so, adds a temporary title on all updated rules to break any cycles and remove the need for specific ordering.
In addition, we make sure diffs are executed in the following order: DELETES, UPDATES, INSERTS.
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
Fix: Change getExistingDashboardByTitleAndFolder to get dashboard by title, not slug (#70723)
* Fix: Change getExistingDashboardByTitleAndFolder to get dashboard by title, not slug
* test: add tests for get dashboard with existing name, get dashboard with non existing name, get dashboard with existing name in a folder
* Update pkg/services/dashboards/database/database_test.go
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* require specific error for Should be able to get dashboard with existing name
* Update pkg/services/dashboards/database/database_test.go
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* implement sofia suggestions to check for specific err, remove logs
* give test more specific name
* implement daniel suggestion of formatting documentation comment in safe way
* fix test title to refer to root directory not specific folder
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
(cherry picked from commit 67cdae4b7d)
Co-authored-by: Kat Yang <69819079+yangkb09@users.noreply.github.com>
Alerting: Remove and revert flag alertingBigTransactions (#65976)
* Alerting: Remove and revert flag alertingBigTransactions
This is a partial revert of #56575 and a removal of the `alertingBigTransactions` flag.
Real-word use has seen no clear performance incentive to maintain this flag. Lowered db connection count
came at the cost of significant increase in CPU usage and query latency.
* Fix lint backend
* Removed last bits of alertingBigTransactions
---------
Co-authored-by: Armand Grillet <2117580+armandgrillet@users.noreply.github.com>
(cherry picked from commit 63187fae0c)
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration (#70781)
* Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration
(cherry picked from commit 00d5f7fed7)
Kevin Minehart
Piotr Jamróz
grafana-delivery-bot[bot]
Stephanie Hingtgen
Dan Cech
Ryan McKinley
Will Browne
Andreas Christou
Alexander Weaver
Ashley Harrison
Victor Marin
Karl Persson
gotjosh
Horst Gutmann
Andres Martinez Gotor
Guilherme Caulada
Gabriel MABILLE
Yuri Tseretyan
lean.dev
ismail simsek
Santiago
Dimitris Sotirakis
Matthew Jacobson
George Robinson