* Alerting: Update migration to put alerts to the default folder if dashboard folder is missing (#65577)
* extract function
* use context logger
* put alert to general folder if folder is missing
* move folderHelper init
* add test
* Update pkg/services/sqlstore/migrations/ualert/ualert.go
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
---------
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
(cherry picked from commit 7b2f44762e)
* rename ID to Id and dashboards.Dashboard to models.Dashboard
---------
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
Expressions: Fixes the issue showing expressions editor (#62510)
* Use suggested value for uid
* update the snapshot
* use __expr__
* replace all -100 with __expr__
* update snapshot
* more changes
* revert redundant change
* Use expr.DatasourceUID where it's possible
* generate files
(cherry picked from commit 91221bc436)
Alerting: Prevent uid collision in migration when db is case-insensitive (#60494)
* Alerting: Prevent short uid collision in legacy migration when db is case-insensitive
Two factors come into play that cause sporadic uid conflicts during legacy alert migration:
- MySQL and MySQL-compatible backends use case-insensitive collation.
- Our short uid generator is not a uniform RNG and generates uids in such a way that generations in quick succession have a higher probability of creating similar uids.
Normally we would be guaranteed unique short uid generation, however if the source alphabet contains
duplicate characters (for example, if we use case-insensitive comparison) this guarantee is void.
Generating even ~1000 uids in quick succession is nearly guaranteed to create a case-insensitive
duplicate.
(cherry picked from commit 570b62091c)
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* Alerting: Improve notification policies created during migration
Previously, migrated legacy alerts were connected to notification policies through
a `rule_uid` label in a 1:1 fashion. While this correctly mimicked pre-migration routing,
it didn't create a notification policy structure that is easy to view/modify. In addition,
having one policy per migrated alert is, in some ways, counter to the recommended approach of
Unified Alerting.
This change replaces `rule_uid`-based migrated notification policies with a private
label called `__contacts__`. This label stores a list of double quoted strings containing the names of
all contact points an AlertRule should route to (based on legacy notification channels). Finally,
one notification policy is created per contact point with each matching AlertRules via regex on this
`__contacts__` label.
The result is a simpler, clearer, and easier to modify notification policy structure, with the
added benefit that you can see which contact points an AlertRule is being routed to from the
AlertRule creation page.
The length of the identifier from the underlying library is 9 or more characters depending on the rate at which the identifiers are generated. See https://pkg.go.dev/github.com/teris-io/shortid
The test previously made the assumption that the length will always be 10, which would intermittently fail.
* Extract standardized UID field length to constant
* Extract default length to constant
* Truncate rule names that are too long
* Add tests for name normalization
* Fix whitespace lint error
* Another linter fix
* Empty commit to kick build
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* Alerting: Update default route groupBy to [grafana_folder, alertname]
Default group by for new routes and migrations is now [grafana_folder, alertname]
* chore/backend: move dashboard errors to dashboard service
Dashboard-related models are slowly moving out of the models package and into dashboard services. This commit moves dashboard-related errors; the rest will come in later commits.
There are no logical code changes, this is only a structural (package) move.
* lint lint lint
* Alerting: Don't stop the migration when alert rule tags are invalid
As we migrate we expect the `alertRuleTags` on a dashboard alert to be a JSON object. However, it seems this is not really validated by Grafana and an user can change the format to something else that the JSON parser is not able to marshal into a `map[string]string`.
Let's do a bit better by "attempting" to parse the tags and if we can't we'll simple return an empty map. The data is still there so if the user wishes they can go back, fix the data and attemp the migration again.
Migrations:
* add a new column alert_group_idx to alert_rule table
* add a new column alert_group_idx to alert_rule_version table
* re-index existing rules during migration
API:
* set group index on update. Use the natural order of items in the array as group index
* sort rules in the group on GET
* update the version of all rules of all affected groups. This will make optimistic lock work in the case of multiple concurrent request touching the same groups.
UI:
* update UI to keep the order of alerts in a group
* Alerting: Fix force_migration when alerting is disabled
This commit fixes a bug where force_migration must be set to true
when both unified and legacy alerting is disabled.
* Update comment
* Fix typo in comment
Co-authored-by: Armand Grillet <armand.grillet@outlook.com>
After migrating to unified alerting, users must explicitly allow rolling
back to legacy alerting by setting force_migration = true in config.
This updates the panic message to clarify why that's required and what
the consequences of rolling back will be.
Fixes#50469
* Remove dashbpard version from models
* Fix lint
* Fix api & sqlstore tests
* Remove integration tags
* Fix lint again
* Add integration test to correct namespace
* Lont fix 2
* Change Id to ID in dashVersionMeta
This change adds a field to state.State and models.AlertInstance
that indicate the "Reason" that an instance has its current state. This
helps us account for cases where the state is "Normal" but the
underlying evaluation returned "NoData" or "Error", for example.
Fixes#42606
Signed-off-by: Joe Blubaugh <joe.blubaugh@grafana.com>
This change extracts screenshot data from alert messages via a private annotation `__alertScreenshotToken__` and attaches a URL to a Slack message or uploads the data to an image upload endpoint if needed.
This change also implements a few foundational functions for use in other notifiers.
This commit adds a pkg/services/screenshot package for taking and uploading screenshots of Grafana dashboards. It supports taking screenshots of both dashboards and individual panels within a dashboard, using the rendering service.
The screenshot package has the following services, most of which can be composed:
BrowserScreenshotService (Takes screenshots with headless Chrome)
CachableScreenshotService (Caches screenshots taken with another service such as BrowserScreenshotService)
NoopScreenshotService (A no-op screenshot service for tests)
SingleFlightScreenshotService (Prevents duplicate screenshots when taking screenshots of the same dashboard or panel in parallel)
ScreenshotUnavailableService (A screenshot service that returns ErrScreenshotsUnavailable)
UploadingScreenshotService (A screenshot service that uploads taken screenshots)
The screenshot package does not support wire dependency injection yet. ngalert constructs its own version of the service. See https://github.com/grafana/grafana/issues/49296
This PR also adds an ImageScreenshotService to ngAlert. This is used to take screenshots with a screenshotservice and then store their location reference for use by alert instances and notifiers.
* Alerting: Create fewer contact points on migration
Previously a new contact point was created for every unique combination
of channels attached to any legacy alert. This was very hard to maintain,
requiring modifications in every generated contact point.
This change deduplicates the generated contact points to a more
reasonable state. There should now only be one contact point per legacy
channel, and we attached multiple contact points to a route by nesting
them. The sole exception to this is if there were multiple default
legacy channels, in which case we create a redundant contact point
containing all of them used only in the root policy. This allows for a
much simpler notification policy structure.
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Alerting: Add integration test for AddDashAlertMigration
* Add more targeted test cases
* Apply suggestions from code review
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Reorganize file and improve comments
* Replace custom sort+trim with go-cmp
* Add test for AddDashAlertMigration
* Rename test cases to standard format
* Apply suggestions from code review
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Remove test-only snapshots of PostableUserConfig et al.
* Organize imports
* Fix linting
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Alerting: add collision safe update function for alertmanager configurations
* fix typo
* use bootstrap func for tests
* move hash calculation to store
* remove icons lol
* remove removed field
* Add missing OK option to models
* add ok to legacy legacy UI does not support it but it is possible to do so via provisioning.
* use enums in migration so linter would catch missing cases
* Simplistic store API for provenance lookups on arbitrary types
* Add a few notes in comments
* Improved type safety for provisioned objects
* Clean-up TODOs for future PRs
* Clean up provisioning model
* Clean up tests
* Restrict allowable types in interface
* Fix linter error
* Move AlertRule domain methods to same file as AlertRule definition
* Update pkg/services/ngalert/models/provisioning.go
Co-authored-by: George Robinson <george.robinson@grafana.com>
* Complete interface rename
* Pass context through store API
* More idiomatic method names
* Better error description
* Improve code-docs
* Use ORM language instead of raw sql
* Add support for records in different orgs
* ResourceTypeID -> ResourceType since it's not an ID
Co-authored-by: George Robinson <george.robinson@grafana.com>
* (WIP) send alerts to external, internal, or both alertmanagers
* Modify admin configuration endpoint, update swagger docs
* Integration test for admin config updated
* Code review changes
* Fix alertmanagers choice not changing bug, add unit test
* Add AlertmanagersChoice as enum in swagger, code review changes
* Fix API and tests errors
* Change enum from int to string, use 'SendAlertsTo' instead of 'AlertmanagerChoice' where necessary
* Fix tests to reflect last changes
* Keep senders running when alerts are handled just internally
* Check if any external AM has been discovered before sending alerts, update tests
* remove duplicate data from logs
* update comment
* represent alertmanagers choice as an int instead of a string
* default alertmanagers choice to all alertmanagers, test cases
* update definitions and generate spec
* pass notification service down to the notifiers
* add ns to all notifiers
* remove bus from ngalert notifiers
* use smaller interfaces for notificationservice
* attempt to fix the tests
* remove unused struct field
* simplify notification service mock
* trying to resolve issues in the tests
* make linter happy
* make linter even happier
* linter, you are annoying
* update AlertingEnabled and UnifiedAlertingSettings.Enabled to be pointers
* add a pseudo migration to fix the AlertingEnabled and UnifiedAlertingSettings.Enabled if the latter is not defined
* update the default configuration file to make default value for both 'enabled' flags be undefined
Misc
* update Migrator to expose DB engine. This is needed for a ualert migration to access the database while the list of migrations is created.
* add more verbose failure when migrations do not match
Co-authored-by: gotjosh <josue@grafana.com>
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
* do not suppress NoData state
* extract conversion of state to postable alert + tests
* create a special alert instance if nodata
* use NoData when converting from Keep Last State instead of Alerting
* add silence during migration if NoData is mapped to KeepLastState.
