Commit Graph

700 Commits

Author SHA1 Message Date
Carl Bergquist b1e112923b aligns the severity dropdown with other fields (#22779) 2020-03-13 14:19:35 +01:00
Vikky Omkar 066d5cf4da Alerting: Fixed the issue/bug of diff and percent_diff functions *Breaking change* (#21338)
fixes #16270 #10129
2020-02-27 20:25:03 +01:00
Arve Knudsen bf908424a7 Alerting: Don't include image_url field with Slack message if empty (#22372) 2020-02-21 15:43:54 +01:00
Marcus Efraimsson d0a80c59f3 Rendering: Store render key in remote cache (#22031)
By storing render key in remote cache it will enable
image renderer to use public facing url or load
balancer url to render images and thereby remove
the requirement of image renderer having to use the
url of the originating Grafana instance when running
HA setup (multiple Grafana instances).

Fixes #17704
Ref grafana/grafana-image-renderer#91
2020-02-19 19:47:39 +01:00
Arve Knudsen 019834ae12 Alerting: Update the types of recipient supported by the Slack notifier (#22205)
* Alerting: Update types of supported Slack recipients
* Document Slack recipient requirements
2020-02-14 17:11:23 +01:00
Arve Knudsen 6f09bc9fb4 Fix mentioning Slack users/groups (#21734)
* alerting/slack: Allow mentioning users, groups, and channels separately
2020-02-11 21:43:28 +01:00
Andrii Melnyk 47314d0f13 Alerting: Handle NaN in reducers (#22053)
- Fix bug with NaN in alerting - Closes #21953
- Alert reducers (avg/max/etc) drop null values from their calculation. This change makes it so NaN values are handled in the same way as null values.
2020-02-10 19:28:07 -05:00
John Dyer 0fcb2d0b2f Alerting: Support passing tags to Pagerduty and allow notification on specific event categories (#21335)
Add support for passing tags to Pagerduty and allow notification 
on specific event categories such as Class, Group and Component.

Ref #19912, #19913

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-01-23 13:20:07 +01:00
Marcus Efraimsson 71ffd1d108 Alerting: Fix image rendering and uploading timeout preventing to send alert notifications (#21536)
* svc alerting - use a shorter ctx to upload the img
This will prevent timeout on img upload to cancel the notifications from being sent

* components img uploader - pass the ctx to aws lib

* make webdavuploader use the ctx

* make azureblobuploader use the ctx

* rename uploadImage() to renderAndUploadImage()
for better clarity about what this method work

* Use timeout + 2s for plugin renderer (same as service and phantomjs)

* Make sure that original EvalContext is updated after render and upload

* Verify notification sent even if render or image upload times out

* fix lint

* fixes after review

Co-authored-by: Edouard Hur <3418467+hekmon@users.noreply.github.com>

Fixes #21018
2020-01-17 12:07:16 +01:00
Marcus Efraimsson 492912845f Chore: Adds basic alerting notification service tests (#21467)
Adds basic alerting notification tests. First steps towards 
getting #18011 merged.

Ref #18011
2020-01-14 17:27:16 +01:00
Sofia Papagiannaki d135f1229d Alerting: new min_interval_seconds options to enforce a minimum eval frequency (#21188)
* add min_interval_seconds setting to alerting config

It will let operator enforce a minimum time for the scheduler to enqueue evaluations

* Introduce UI modifications

* Update docs

Co-authored-by: Martin <uepoch@users.noreply.github.com>
2020-01-14 11:13:34 +02:00
zabullet bcac76f5d4 Alerting: Enable setting of OpsGenie priority via a tag (#21298)
OpsGenie's model works heavily off of the priority of an alert, 
e.g. routing and escalation. Currently this plugin only supports 
the default "P3".
Setting a tag og_priority to the correct P-value, e.g. P1, P2, P3, 
P4 or P5, will call the OpsGenie API with the correct priority value 
set.
2020-01-13 21:04:24 +01:00
Alvaro Olmedo Rodriguez abc806e17b Alerting: fallbackText added to Google Chat notifier (#21464)
Add fallbackText to Google Chat notifications. 
Definition of fallbackText: A plain-text description of the message's 
cards, used when the actual cards cannot be displayed, 
e.g. mobile notifications.

Fixes #21463
2020-01-13 20:42:52 +01:00
Marcus Efraimsson 0f0772b629 Alerting: Adds support for sending a single email to all recipients in notification channel (#21091)
When an alert is sent by e-mail, the process sends an e-mail to 
each recipient separately. This PR is a single delivery to all recipients.
For companies that use e-mail extensively, this is necessary in order 
not to overload the sending queue.

Replaces #18013
Fixes #12650

Co-authored-by: Henrique Oliveira <holiiveira@users.noreply.github.com>
2020-01-10 16:06:33 +01:00
Danilo Bargen 8a02fa7691 ThreemaNotifier: Use fully qualified status emoji (#21305) 2020-01-10 16:00:20 +01:00
Zach Bagnall d0360de8f1 Alerting: Add configurable severity support for PagerDuty notifier (#19425)
* Pagerduty notifier: configurable severity

Instead of hardcoding `critical` make it configurable per notification channel instance.

* fix html

* Add a test to ensure default severity is correct

* Notifications doc

* Add a non-default test

* Add err check on NewJson (all tests)

* Add default severity (critical) to AlertNotificationEditCtrl class
2019-12-24 09:32:05 +02:00
michael-az 5b3ff90377 Alerting: Add more information to webhook notifications (#20420)
Adds the orgId, dashboardId and panelId of the Webhook alert notifier.
2019-12-18 14:03:59 +01:00
Christoph 8c089b98b3 PagerDuty: Fix custom_details to be a JSON object instead of a string (#21150) 2019-12-18 11:42:25 +02:00
csyangchen 34299a1bd2 Alerting: Fix panic in dingding notifier (#20378)
dingding notifier fails to send alert messages containing JSON, 
This change use map and json.Marshal to always produce valid 
JSON body.

Fixes #9771
2019-12-11 17:01:01 +01:00
Dima Ryskin 2027e1aaee AlertNotifier: Support alert tags in OpsGenie notifier (#20810)
* support alert tags in OpsGenie notifier

* update readme: OpsGenie alert tags support

* lintfix: remove redundant string formatting
2019-12-03 11:54:37 +01:00
Marcus Efraimsson 5b42bb58f6 Telegram: Check error before adding defer close of image (#20331)
Properly handles file opening error and returns before deferring 
close of file.

Fixes #20156
2019-11-13 13:33:03 +01:00
stratomonitor 4d7d13175c Alerting: Add alert_state to the kafka message Fixes #11401 (#20099)
* Notifications: Add alert_state to the kafka message Fixes #11401
2019-11-07 14:08:58 +02:00
Arve Knudsen 2a78d2a61c pkg/services: Check errors (#19712)
* pkg/services: Check errors
* pkg/services: Don't treat context.Canceled|context.DeadlineExceeded as error
2019-10-22 14:08:18 +02:00
Abhilash Gnan 7f702f881c Alerting: All notification channels should always send (#19807)
Fixes so that all notification channels configured for an alert should 
try to send notification even if one notification channel fails to send
a notification.

Signed-off-by: Abhilash Gnan <abhilashgnan@gmail.com>

Fixes #19768
2019-10-16 09:55:51 +02:00
Andrew Rabert 8a991244d5 Alerting: Truncate PagerDuty summary when greater than 1024 characters (#18730)
Requests to PagerDuty fail with an HTTP 400 if the `summary` 
attribute contains more than 1024 characters, this fixes this.
API spec:
https://v2.developer.pagerduty.com/docs/send-an-event-events-api-v2

Fixes #18727
2019-09-20 10:39:27 +02:00
lzd c9c0dd1ea8 Alerting: fix response popover prompt when add notification channels (#18967) 2019-09-09 14:09:21 +02:00
Sofia Papagiannaki 388d3d3714 Notification is sent when state changes from no_data to ok (#18920) 2019-09-05 18:54:27 +03:00
Kyle Brandt 364d2358d8 alerting: add lock on job to prevent a race condition (#18218)
without this lock there is a race condition between the scheduler and job processing.
2019-09-03 09:14:28 -04:00
Gali Alykoff f942fecc52 Chore: Improve err message for notifications (#18757)
Improve message of notification json error
Fix test after change error message
2019-08-28 16:49:36 +02:00
Marcus Efraimsson e83f55f394 Chore: Use ruleId instead of alertId as log keyword (#18738) 2019-08-27 08:40:03 +02:00
Oleg Gaidarenko 35b74a99a8 Emails: resurrect template notification (#18686)
* Emails: resurrect template notification

* Phantomjs (oh yeah, there is another dev dep phantom :-) was failing for
  the generation of the html templates so I had to update the dependencies
  in order to fix it. While doing that I update the scripts field and docs
  for it as well. yarn.lock is included

* Move splitting of the emails to separate helper function, since more services
  coming up that would need to use this functionality

* Add support for enterprise specific email letters. Probably could
  be done in the better way, but it's not a priority right now
2019-08-26 17:19:03 +02:00
Dominic Miglar 32d6740b8f Alerting: Also include configured AlertRuleTags in Webhooks (#18233)
* added alert rule tags in webhook notifications

* fix: don't include whole list of Tag objects but only key/value pairs in Webhook JSON

* marked webhook alerts to support alert rule tags
2019-08-03 17:00:40 +02:00
Dima Kurguzov a4b0ccc138 Instrumentation: Add failed notifications metric (#18089) 2019-07-24 11:43:24 +02:00
Kyle Brandt ea0cee4c35 alerting: more specific error when missing threshold (#18221)
fixes #18184
2019-07-22 13:23:33 -04:00
Oleg Gaidarenko 75fa1f0207 Metrics: use consistent naming for exported variables (#18134)
* Metrics: remove unused metrics

Metric `M_Grafana_Version` is not used anywhere, nor the mentioned
`M_Grafana_Build_Version`. Seems to be an artefact?

* Metrics: make the naming consistent

* Metrics: add comments to exported vars

* Metrics: use proper naming

Fixes #18110
2019-07-16 16:58:46 +02:00
Oleg Gaidarenko caa1314f44 Build: use golangci-lint as a make command (#17739)
* Build: use golangci-lint as a make command

* Since gometalinter was deprecated in favor of golangci-lint so it was
  replaced by it. Responsibilities held by the gometalinter was moved to
  golangci-lint

* There was some changes in implementation (that was also mentioned in
  the code comment) between the tools, which uncovered couple errors
  in the code. Those issues were either solved or disabled by
  the inline comments

* Introduce the golangci-lint config, to make their
  configuration more manageable

* Build: replace backend-lint.sh script with make
2019-07-02 16:06:59 +03:00
Marcus Efraimsson 5713048f48 Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
Carl Bergquist 1aadb4426a metrics: expose stats about roles as metrics (#17469) 2019-06-06 16:39:35 +02:00
Thibault Chataigner e06abb30aa Alerting: Add tags to alert rules (#10989)
Ref #6552
2019-06-06 13:29:30 +02:00
Mario Trangoni 87760d4fde Codestyle: Fix govet issues (#17178)
ref #10381

Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>
2019-06-04 22:00:05 +02:00
Carl Bergquist d8736a2547 Alerting: golint fixes for alerting (#17246) 2019-06-03 10:25:58 +02:00
Abhilash Gnan ca6151e23f Alerting: Support for configuring content field for Discord alert notifier (#17017) 2019-05-23 08:31:02 +02:00
Mario Trangoni 66ba2aa524 Fix gosimple issues (#17179)
Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>
2019-05-21 07:50:44 +02:00
Carl Bergquist bfa7c3d963 alerting: golint fixes for alert notifiers. (#17167) 2019-05-20 15:23:06 +02:00
Carl Bergquist bd5bcea5d0 alerting: fix a bunch of lint issues. (#17128) 2019-05-20 12:13:32 +02:00
Carl Bergquist 2904e2d9fe Alertmanager: Replace illegal chars with underscore in label names (#17002)
closes #16624
2019-05-16 11:41:57 +02:00
Carl Bergquist 13f137a17d tech: avoid alias for importing models in alerting (#17041)
ref #14679
2019-05-14 08:15:05 +02:00
zhulongcheng 2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
Carl Bergquist f001815d9d alerting: no notification when going from nodata -> pending (#16905)
ref #16496
2019-05-07 10:34:00 +02:00
Andrej Ocenas 0433af6385 Config: Fixes bug where timeouts for alerting was not parsed correctly (#16784)
* Fix parsing of the config

* Remove unnecessary conversion

* Remove timeout modification

Co-Authored-By: aocenas <mr.ocenas@gmail.com>

* Remove unused import
2019-04-30 12:05:38 +02:00