* Create API test for overwriting invalid alertmanager config
* Avoid requiring alertmanager readiness for config changes
* AlertmanagerSrv depends on functionality rather than concrete types
* Add test for non-ready alertmanagers
* Additional cleanup and polish
* Back out previous integration test changes
* Refactor of tests incorrectly caused a test to become redundant
* Use pre-existing fake secret service
* Drop unused interface
* Test against concrete MultiOrgAlertmanager re-using fake infra from other tests
* Fix linter error
* Empty commit to rerun checks
* Fix flaky tests in several notifiers
- Non-mocked time in sensu go tests
- Close server in Slack tests
- Use a mutex for writing responses in the fake slack server
* Remove mutex at the fake slack server
* add value to email template
* add value to default template
* update test string
* test: fix ngalert test suite
* test: run CI
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
* Alerting: accept mute_timing_intervals through the api for the embedded alertmanager
* add workaround for mutetimeinterval
* add mute timings to routes
* revert changes
* Update pkg/services/ngalert/api/api_alertmanager.go
* Update pkg/services/ngalert/api/api_alertmanager.go
* Update pkg/services/ngalert/api/api_alertmanager.go
* update prometheus/alertmanager dependency
* add some var docs
* Use secrets service in pluginproxy
* Use secrets service in pluginxontext
* Use secrets service in pluginsettings
* Use secrets service in provisioning
* Use secrets service in authinfoservice
* Use secrets service in api
* Use secrets service in sqlstore
* Use secrets service in dashboardshapshots
* Use secrets service in tsdb
* Use secrets service in datasources
* Use secrets service in alerting
* Use secrets service in ngalert
* Break cyclic dependancy
* Refactor service
* Break cyclic dependancy
* Add FakeSecretsStore
* Setup Secrets Service in sqlstore
* Fix
* Continue secrets service refactoring
* Fix cyclic dependancy in sqlstore tests
* Fix secrets service references
* Fix linter errors
* Add fake secrets service for tests
* Refactor SetupTestSecretsService
* Update setting up secret service in tests
* Fix missing secrets service in multiorg_alertmanager_test
* Use fake db in tests and sort imports
* Use fake db in datasources tests
* Fix more tests
* Fix linter issues
* Attempt to fix plugin proxy tests
* Pass secrets service to getPluginProxiedRequest in pluginproxy tests
* Fix pluginproxy tests
* Revert using secrets service in alerting and provisioning
* Update decryptFn in alerting migration
* Rename defaultProvider to currentProvider
* Use fake secrets service in alert channels tests
* Refactor secrets service test helper
* Update setting up secrets service in tests
* Revert alerting changes in api
* Add comments
* Remove secrets service from background services
* Convert global encryption functions into vars
* Revert "Convert global encryption functions into vars"
This reverts commit 498eb19859.
* Add feature toggle for envelope encryption
* Rename toggle
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
* Added an option to discord notifier to use discord's webhook name (useful for customizing notifications).
* Support ngalert system with discord username toggle
* Added ngalert discord test
* Apply suggestions from code review
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Docs updated with discord username setting
* Fix api integration test
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* Alerting: Validate contact point configuration during the migration
This minimises the chances of generating broken configuration as part of the migration. Originally, we wanted to generate it and not produce a hard stop in Grafana but this strategy has the chance to avoid delivering notifications for our users.
We now think it's better to hard stop the migration and let the user take care of resolving the configuration manually.
* Alerting: Fixes a bug when trying to sync broken alertmanager config
Broken alertmanager configuration has the potential to be introduced as part of a migration e.g. due to incompatible data between what grafana accepts and what the Alertmanager expects. When this happens, we expect an eventually consistent behaviour where we'll keep trying to apply the configuration until it works.
As part of change in https://github.com/grafana/grafana/pull/39237 we introduced a regression that modified this behaviour and instead tried to create a new Alertmanager for that organization everytime, which eventually ended up in a panic due to a duplicate metrics being registered.
This PR fixes that and introduces a test to catch further regressions.
* Remove disable orgs
* Encryption: Add support to encrypt/decrypt sjd
* Add datasources.Service as a proxy to datasources db operations
* Encrypt ds.SecureJsonData before calling SQLStore
* Move ds cache code into ds service
* Fix tlsmanager tests
* Fix pluginproxy tests
* Remove some securejsondata.GetEncryptedJsonData usages
* Add pluginsettings.Service as a proxy for plugin settings db operations
* Add AlertNotificationService as a proxy for alert notification db operations
* Remove some securejsondata.GetEncryptedJsonData usages
* Remove more securejsondata.GetEncryptedJsonData usages
* Fix lint errors
* Minor fixes
* Remove encryption global functions usages from ngalert
* Fix lint errors
* Minor fixes
* Minor fixes
* Remove securejsondata.DecryptedValue usage
* Refactor the refactor
* Remove securejsondata.DecryptedValue usage
* Move securejsondata to migrations package
* Move securejsondata to migrations package
* Minor fix
* Fix integration test
* Fix integration tests
* Undo undesired changes
* Fix tests
* Add context.Context into encryption methods
* Fix tests
* Fix tests
* Fix tests
* Trigger CI
* Fix test
* Add names to params of encryption service interface
* Remove bus from CacheServiceImpl
* Add logging
* Add keys to logger
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Add missing key to logger
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Undo changes in markdown files
* Fix formatting
* Add context to secrets service
* Rename decryptSecureJsonData to decryptSecureJsonDataFn
* Name args in GetDecryptedValueFn
* Add template back to NewAlertmanagerNotifier
* Copy GetDecryptedValueFn to ngalert
* Add logging to pluginsettings
* Fix pluginsettings test
Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Remove validation for labels to be accepted in the Alertmanager, This helps with datasources that produce non-compatible labels.
Adds an "object_matchers" to alert manager routers so we can support labels names with extended characters beyond prometheus/openmetrics. It only does this for the internal Grafana managed Alert Manager.
This requires a change to alert manager, so for now we use grafana/alertmanager which is a slight fork, with the intention of going back to upstream.
The frontend handles the migration of "matchers" -> "object_matchers" when the route is edited and saved. Once this is done, downgrades will not work old versions will not recognize the "object_matchers".
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Nathan Rodman <nathanrodman@gmail.com>
* Add method GetAllLatestAlertmanagerConfiguration to DBStore
* add method ApplyConfig to AlertManager
* update multiorg alert manager to load all alertmanager configs at once
* Alerting: Refactor & fix unified alerting metrics structure
Fixes and refactors the metrics structure we have for the ngalert service. Now, each component has its own metric struct that includes the JUST the metrics it uses. Additionally, I have fixed the configuration metrics and added new metrics to determine if we have discovered and started all the necessary configurations of an instance.
This allows us to alert on `grafana_alerting_discovered_configurations - grafana_alerting_active_configurations != 0` to know whether an alertmanager instance did not start successfully.
* Alerting: Persist notification log and silences to the database
This removes the dependency of having persistent disk to run grafana alerting. Instead of regularly flushing the notification log and silences to disk we now flush the binary content of those files to the database encoded as a base64 string.
Introduces org-level isolation for the Alertmanager and its components.
Silences, Alerts and Contact points are not separated by org and are not shared between them.
Co-authored with @davidmparrott and @papagian
This commit adds contact point testing to ngalerts via a new API
endpoint. This endpoint accepts JSON containing a list of
receiver configurations which are validated and then tested
with a notification for a test alert. The endpoint returns JSON
for each receiver with a status and error message. It accepts
a configurable timeout via the Request-Timeout header (in seconds)
up to a maximum of 30 seconds.
idafurjes
Alexander Weaver
Jean-Philippe Quéméner
Gilles De Mey
smallpathandgotjosh
Yuriy Tseretyan
gotjosh
Sofia Papagiannaki
George Robinson
Peter Holmbergandgillesdemey
ying-jeanne
Yuriy Tseretyan
Santiago
David Parrott
Domas
George Robinson