Commit Graph

63 Commits

Author SHA1 Message Date
Grot (@grafanabot) 4fe55be39e Alerting: Do not hard fail on templating errors in channels (#35165) (#35193)
* Alerting: Do not hard fail on templating errors in channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
(cherry picked from commit a30e60a0b8)

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-06-03 16:48:37 +02:00
Grot (@grafanabot) eae88f8ce8 Alerting: Migrate tags as labels and not annotations (#34990) (#34992)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
(cherry picked from commit a23674ef99)

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-05-31 16:43:10 +02:00
Grot (@grafanabot) 2335c3d0d5 Alerting: Add support for configuring avatar URL for the Discord notifier (#33355) (#34940)
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
(cherry picked from commit badec6c6ad)

Co-authored-by: Chip Wolf ‮ <hello@chipwolf.uk>
2021-05-31 09:02:32 +02:00
Grot (@grafanabot) 8fef4d7a8a alerting: fixes per-receiver metric cardinality (#34915) (#34924)
(cherry picked from commit cc38613ba4)

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
2021-05-28 18:54:30 +02:00
Grot (@grafanabot) 638735a37c Clarify that Threema Gateway Alerts support only Basic IDs (#34828) (#34871)
Threema Gateway supports two types of IDs: Basic IDs (where the
encryption is managed by the API server) and End-to-End IDs (where the
keys are managed by the user).

This plugin currently does not support End-to-End IDs (since it's much
more complex to implement, because the encryption needs to happen
locally). Add a few clarifications to the UI.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
(cherry picked from commit 83a83de10a)

Co-authored-by: Danilo Bargen <mail@dbrgn.ch>
2021-05-28 09:29:12 +02:00
Grot (@grafanabot) 72b3478935 NGAlert: Update the default template to include more URLs (#34715) (#34750)
* NGAlert: Update the default template to include more URLs

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
(cherry picked from commit d69c21acb6)

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-05-26 17:14:15 +02:00
Grot (@grafanabot) 81f2f68535 NGAlert: Add integration tests for remaining notification channels (#34662) (#34728)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
(cherry picked from commit b168223029)

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-05-26 19:59:34 +05:30
Grot (@grafanabot) ec09a5995d [Alerting]: alertmanager notifier fixes (#34575) (#34589)
(cherry picked from commit b48832c0f7)

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2021-05-24 17:32:52 +03:00
Grot (@grafanabot) 52d6afbae7 [Alerting]: namespace fixes (#34470) (#34489)
* [Alerting]: forbid viewers for updating rules if viewers can edit

check for CanSave instead of CanEdit

* Clear ngalert tables when deleting the folder

* Apply suggestions from code review

* Log failure to check save permission

Co-authored-by: gotjosh <josue@grafana.com>
(cherry picked from commit 23939eab10)

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2021-05-24 10:42:56 +03:00
gotjosh 7b04278834 Alerting: Opsgenie notification channel (#34418)
* Alerting: Opsgenie notification channel

This translate the opsgenie notification channel from the old alerting
system to the new alerting system with a few changes:

- The tag system has been replaced in favour of annotation.
- TBD
- TBD

Signed-off-by: Josue Abreu <josue@grafana.com>

* Fix template URL

* Bugfig: dont send resolved when autoClose is false

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix integration tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix URLs in all other channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-20 10:12:08 +02:00
Ganesh Vernekar 533be16787 NGAlert: Add Threema notification channel (#34159)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 20:20:52 +02:00
Ganesh Vernekar b2e84277a3 NGAlert: Add Kafka notification channel (#34156)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 20:02:09 +02:00
Ganesh Vernekar ad1d0ae0bf NGAlert: Add VictorOps notification channel (#34161)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 19:52:14 +02:00
Ganesh Vernekar fb9223ab42 NGAlert: Add Line notification channel (#34157)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 19:04:48 +02:00
Domas 54c33c6cdd Alerting: update email template (#34205) 2021-05-19 18:58:31 +02:00
Ganesh Vernekar 01e0faf800 NGAlert: Add GoogleChat notification channel (#34153)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 18:24:04 +02:00
David Parrott b9f4ec2030 Add discord notifier channel and test (#34150)
* Add discord notifier channel and test

* Correct payload

* remove print statement

* PR feedback and update due to changes in main

* Add discord notifier channel and test

* Correct payload

* remove print statement

* PR feedback and update due to changes in main

* update constructor and tests

* group imports sensibly

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 17:31:55 +02:00
Sofia Papagiannaki a79a4838b8 [Alerting]: Add Pushover integration with the alert manager (#34371)
* [Alerting]: Add Pushover integration with the alert manager

* lint

* Set boundary only for tests

* Remove title field

* fix imports
2021-05-19 16:48:46 +02:00
Arve Knudsen 9dfaa037d1 Alerting: Migrate Alertmanager notifier (#34304)
* Alerting: Port Alertmanager notifier to v8

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 15:27:41 +02:00
Sofia Papagiannaki ff112f07e3 [Alerting]: Add Sensu Go integration with the alert manager (#34045)
* [Alerting]: Add sensugo notification channel

* Apply suggestions from code review

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Do not include labels with concatenated rule UID and names

* Modifications after syncing with main

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-05-18 17:31:51 +03:00
Ganesh Vernekar 89c2b5e863 NGAlert: Remove unwanted fields from notification channel config (#34036)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-18 10:04:47 +02:00
gotjosh 6384f86fb9 Alerting: Allow the notifier to log (#34232)
* Alerting: Allow the notifier to log

The notifier upstream code uses go-kit as its logging library. The
grafana specific logger is not compatible with this API. In this PR, I
have created a wrapper that implements io.Writer to make them
compatible.
2021-05-17 18:06:47 +01:00
Ganesh Vernekar d5ae55c5dd NGAlert: Add message field to email notification channel (#34044)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-17 16:05:09 +05:30
Owen Diehl 1367f7171e Alerting/ruler metrics (#34144)
* adds active configurations metric

* rule evaluation metrics

* ruler metrics

* pr feedback
2021-05-14 16:13:44 -04:00
gotjosh eb74994b8b Alerting: Modify configuration apply and save semantics - v2 (#34143)
* Save default configuration to the database and copy over secure settings
2021-05-14 19:49:54 +01:00
Owen Diehl baca873a84 extracts alertmanager from DI, including migrations (#34071)
* extracts alertmanager from DI, including migrations

* includes alertmanager Run method in ngalert

* removes 3s test shutdown timeout

* lint
2021-05-13 14:01:38 -04:00
Ganesh Vernekar ec3214bac2 NGAlert: Add integration tests for notification channels (#33431)
* NGAlert: Add integration tests for notification channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix the failing tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Override creation of rule UID, remove only namespace UID

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-13 22:58:19 +05:30
Ganesh Vernekar 5f44ccff0c NGAlert: Fix unit test to write files in temporary directory (#34032)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-13 16:08:12 +05:30
Ganesh Vernekar 8d442c9b44 NGAlert: Fix templating and remove unwanted default templates (#33918)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-12 15:13:43 +05:30
Sofia Papagiannaki 1c58fd380f [Alerting]: store encrypted receiver secure settings (#33832)
* [Alerting]: Store secure settings encrypted

* Move encryption to the API handler
2021-05-10 15:30:42 +03:00
Ganesh Vernekar 1b8c0ce88b NGAlert: Fix some TODOs in notification channels (#33739)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-05 17:48:40 +05:30
Ganesh Vernekar 918552d34b NGAlert: Send list of available ngalert notification channels via API (#33489)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-04 13:58:39 +02:00
Owen Diehl 5e48b54549 Alerting/metrics (#33547)
* moves alerting metrics to their own pkg

* adds grafana_alerting_alerts (by state) metric

* alerts_received_{total,invalid}

* embed alertmanager alerting struct in ng metrics & remove duplicated notification metrics (already embed alertmanager notifier metrics)

* use silence metrics from alertmanager lib

* fix - manager has metrics

* updates ngalert tests

* comment lint
Signed-off-by: Owen Diehl <ow.diehl@gmail.com>

* cleaner prom registry code

* removes ngalert global metrics

* new registry use in all tests

* ngalert metrics impl service, hack testinfra code to prevent duplicate metric registrations

* nilmetrics unexported
2021-04-30 12:28:06 -04:00
Ganesh Vernekar be1affe0a4 NGAlert: Fix flaky test (#33415)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-27 17:03:22 +05:30
Ganesh Vernekar 659ea20c3c NGAlert: Run the maintenance cycle for the silences (#33301)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-23 16:19:03 +02:00
Ganesh Vernekar d66a5e65a4 AlertingNG: Add webhook notification channel (#33229)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-23 18:59:28 +05:30
Ganesh Vernekar a0e567f80f AlertingNG: Add Dingding notification channel (#32995)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 19:30:49 +02:00
Ganesh Vernekar 4ec1edfca3 AlertingNG: Add Teams notification channel (#32979)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 18:16:26 +02:00
Ganesh Vernekar c9cd7ea701 AlertingNG: Add Telegram notification channel (#32795)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 17:24:59 +02:00
Ganesh Vernekar 0a03d5c29e AlertingNG: Correctly set StartsAt, EndsAt, UpdatedAt after alert reception (#33109)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 20:42:18 +05:30
Ganesh Vernekar 3056f86f76 AlertingNG: Fix TODOs in email notification channel (#33169)
* AlertingNG: Fix TODOs in email notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Test fixup

* Remove the receiver field it is not needed for the email notification

Co-authored-by: Josue Abreu <josue@grafana.com>
2021-04-22 10:01:55 -04:00
Arve Knudsen 6408b55a7c Slack: Use chat.postMessage API by default (#32511)
* Slack: Use only chat.postMessage API

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Slack: Check for response error

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Slack: Support custom webhook URL

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Simplify

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Rewrite tests to use stdlib

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/alerting/notifiers/slack.go

Co-authored-by: Dimitris Sotirakis <sotirakis.dim@gmail.com>

* Clarify URL field name

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix linting issue

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix up new Slack notifier

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix lint

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Slack: Make token not required

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Alerting: Send validation errors back to client

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Document how token is required

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Make recipient required when using Slack API

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix field description

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Dimitris Sotirakis <sotirakis.dim@gmail.com>
2021-04-22 16:00:21 +02:00
Arve Knudsen 66020b419c NGAlert: Consolidate on standard errors package (#33249)
* NGAlert: Don't use pkg/errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/ngalert/notifier/alertmanager.go

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>

* Fix logging

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2021-04-22 11:18:25 +02:00
gotjosh 23c7e7ab60 Alerting: Various fixes for the alerts endpoint (#33182)
A set of fixes for the GET alert and groups endpoints.

- First, is the fact that the default values where not being for the query params. I've introduced a new method in the Grafana context that allow us to do this.
- Second, is the fact that alerts were never being transitioned to active. To my surprise this is actually done by the inhibitor in the pipeline - if an alert is not muted, or inhibited then it's active.
- Third, I have added an integration test to cover for regressions.

Signed-off-by: Josue Abreu <josue@grafana.com>
2021-04-21 06:34:42 -04:00
Owen Diehl e37a780e14 Inhouse alerting api (#33129)
* init

* autogens AM route

* POST dashboards/db spec

* POST alert-notifications spec

* fix description

* re inits vendor, updates grafana to master

* go mod updates

* alerting routes

* renames to receivers

* prometheus endpoints

* align config endpoint with cortex, include templates

* Change grafana receiver type

* Update receivers.go

* rename struct to stop swagger thrashing

* add rules API

* index html

* standalone swagger ui html page

* Update README.md

* Expose GrafanaManagedAlert properties

* Some fixes

- /api/v1/rules/{Namespace} should return a map
- update ExtendedUpsertAlertDefinitionCommand properties

* am alerts routes

* rename prom swagger section for clarity, remove example endpoints

* Add missing json and yaml tags

* folder perms

* make folders POST again

* fix grafana receiver type

* rename fodler->namespace for perms

* make ruler json again

* PR fixes

* silences

* fix Ok -> Ack

* Add id to POST /api/v1/silences (#9)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add POST /api/v1/alerts (#10)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* fix silences

* Add testing endpoints

* removes grpc replace directives

* [wip] starts validation

* pkg cleanup

* go mod tidy

* ignores vendor dir

* Change response type for Cortex/Loki alerts

* receiver unmarshaling tests

* ability to split routes between AM & Grafana

* api marshaling & validation

* begins work on routing lib

* [hack] ignores embedded field in generation

* path specific datasource for alerting

* align endpoint names with cloud

* single route per Alerting config

* removes unused routing pkg

* regens spec

* adds datasource param to ruler/prom route paths

* Modifications for supporting migration

* Apply suggestions from code review

* hack for cleaning circular refs in swagger definition

* generates files

* minor fixes for prom endpoints

* decorate prom apis with required: true where applicable

* Revert "generates files"

This reverts commit ef7e975584.

* removes server autogen

* Update imported structs from ngalert

* Fix listing rules response

* Update github.com/prometheus/common dependency

* Update get silence response

* Update get silences response

* adds ruler validation & backend switching

* Fix GET /alertmanager/{DatasourceId}/config/api/v1/alerts response

* Distinct gettable and postable grafana receivers

* Remove permissions routes

* Latest JSON specs

* Fix testing routes

* inline yaml annotation on apirulenode

* yaml test & yamlv3 + comments

* Fix yaml annotations for embedded type

* Rename DatasourceId path parameter

* Implement Backend.String()

* backend zero value is a real backend

* exports DiscoveryBase

* Fix GO initialisms

* Silences: Use PostableSilence as the base struct for creating silences

* Use type alias instead of struct embedding

* More fixes to alertmanager silencing routes

* post and spec JSONs

* Split rule config to postable/gettable

* Fix empty POST /silences payload

Recreating the generated JSON specs fixes the issue
without further modifications

* better yaml unmarshaling for nested yaml docs in cortex-am configs

* regens spec

* re-adds config.receivers

* omitempty to align with prometheus API behavior

* Prefix routes with /api

* Update Alertmanager models

* Make adjustments to follow the Alertmanager API

* ruler: add for and annotations to grafana alert (#45)

* Modify testing API routes

* Fix grafana rule for field type

* Move PostableUserConfig validation to this library

* Fix PostableUserConfig YAML encoding/decoding

* Use common fields for grafana and lotex rules

* Add namespace id in GettableGrafanaRule

* Apply suggestions from code review

* fixup

* more changes

* Apply suggestions from code review

* aligns structure pre merge

* fix new imports & tests

* updates tooling readme

* goimports

* lint

* more linting!!

* revive lint

Co-authored-by: Sofia Papagiannaki <papagian@gmail.com>
Co-authored-by: Domas <domasx2@gmail.com>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Co-authored-by: gotjosh <josue@grafana.com>
Co-authored-by: David Parrott <stomp.box.yo@gmail.com>
Co-authored-by: Kyle Brandt <kyle@grafana.com>
2021-04-19 14:26:04 -04:00
Ganesh Vernekar 6271777ec6 AlertingNG: Remove the receivers field from postable alerts (#33068)
* AlertingNG: Remove the receivers field from postable alerts and update tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-19 12:28:44 +05:30
Ganesh Vernekar 04a8d5407e AlertingNG: Slack notification channel (#32675)
* AlertingNG: Slack notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments and small refactoring

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-15 16:01:41 +05:30
gotjosh 528ca9134b Alerting: Use a default configuration and periodically poll for new ones (#32851)
* Alerting: Use a default configuration and periodically poll for new ones

Use a default configuration to make sure we always start the grafana
instance. Then, regularly poll for new ones.

I've also made sure that failures to apply configuration do not stop the
Grafana server but instead keep polling until it is a success.
2021-04-13 13:02:44 +01:00
Ganesh Vernekar e3a1d3d158 AlertingNG: PagerDuty notification channel (#32604)
* AlertingNG: PagerDuty notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix reviews

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-08 19:21:09 +02:00
Ganesh Vernekar b1c84c795f AlertingNG: Add a global registry for notification channels (#32781)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-08 22:01:23 +05:30