Alerting: Prevent uid collision in migration when db is case-insensitive (#60494)
* Alerting: Prevent short uid collision in legacy migration when db is case-insensitive
Two factors come into play that cause sporadic uid conflicts during legacy alert migration:
- MySQL and MySQL-compatible backends use case-insensitive collation.
- Our short uid generator is not a uniform RNG and generates uids in such a way that generations in quick succession have a higher probability of creating similar uids.
Normally we would be guaranteed unique short uid generation, however if the source alphabet contains
duplicate characters (for example, if we use case-insensitive comparison) this guarantee is void.
Generating even ~1000 uids in quick succession is nearly guaranteed to create a case-insensitive
duplicate.
(cherry picked from commit 570b62091c)
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* FeatureToggle: for storing sessions in a Remote Cache
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* Fix conflicting modifications :D
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* rename the flag to userRemoteCache
* undo unintended change
* Users: Add option to use remote cache for SignedInUsers (#59892)
* Add remote cache to GetSignedInUserWithCacheCtx
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* Populate SignedInUser remote cache
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* Line
* minor fixes to make this work
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* Fix tests
* change flag to updated name
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
* Alerting: Improve notification policies created during migration
Previously, migrated legacy alerts were connected to notification policies through
a `rule_uid` label in a 1:1 fashion. While this correctly mimicked pre-migration routing,
it didn't create a notification policy structure that is easy to view/modify. In addition,
having one policy per migrated alert is, in some ways, counter to the recommended approach of
Unified Alerting.
This change replaces `rule_uid`-based migrated notification policies with a private
label called `__contacts__`. This label stores a list of double quoted strings containing the names of
all contact points an AlertRule should route to (based on legacy notification channels). Finally,
one notification policy is created per contact point with each matching AlertRules via regex on this
`__contacts__` label.
The result is a simpler, clearer, and easier to modify notification policy structure, with the
added benefit that you can see which contact points an AlertRule is being routed to from the
AlertRule creation page.
(cherry picked from commit 0db339d82f)
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* RBAC: Change the generate uid function to be deterministic so we can avoid collision
* RBAC: Use fmt.Errorf
* RBAC: Add comment
* RBAC: Export GenerateManagedRoleUID
(cherry picked from commit 21792fdf37)
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Chore: move team store implementation to a separate package
* trying to fix more tests
* fix tests in service accounts and access control
* fix common tests
* restore commented out test
* add todos
The length of the identifier from the underlying library is 9 or more characters depending on the rate at which the identifiers are generated. See https://pkg.go.dev/github.com/teris-io/shortid
The test previously made the assumption that the length will always be 10, which would intermittently fail.
Introduces a FromContext method on the log.Logger interface that
allows contextual key/value pairs to be attached, e.g. per request,
so that any logger using this API will automatically get the per request
context attached. The proposal makes the traceID available for
contextual logger , if available, and would allow logs originating from
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
* Extract standardized UID field length to constant
* Extract default length to constant
* Truncate rule names that are too long
* Add tests for name normalization
* Fix whitespace lint error
* Another linter fix
* Empty commit to kick build
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* Remove single member team restriction
* Add label when permissions list is empty
* Fix unit tests
* Add co-author.
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
Prior to this change, all alert instance writes and deletes happened
individually, in their own database transaction. This change batches up
writes or deletes for a given rule's evaluation loop into a single
transaction before applying it.
Before:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s
```
After:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s
```
So we cut time by about 75% and memory allocations by about 60% when
storing and deleting 100 instances.
This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1
* Change of sqlstore to use sqlx
* Use sqlx in the playlist store
* Refectory of the interface
* update playlist service
* go mod tidy
* some refectory on interface
* fix kyle
* Remove delete suer from store interface
* Remove get signed in user with cache ctx from store interface
* Support options when setting up access control tests
* Fix broken tests
* Fix lint
* Add user fake to middleware
* Fix middleware tests, remove usertest being initialised twice
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Chore: Remove disable user and searchusers methods from store interface
* Remove disable batch user from sqlstore interface
* Remove sqlstore from search store
* Fix lint
