diff --git a/docs/sources/alerting/alerting.md b/docs/sources/alerting/alerting.md deleted file mode 100644 index a5e374cfe8e..00000000000 --- a/docs/sources/alerting/alerting.md +++ /dev/null @@ -1,19 +0,0 @@ -+++ -title = "Configuring Alert Rules" -description = "Configuring Alert Rules" -keywords = ["grafana", "alerting", "guide", "rules"] -type = "docs" -[menu.docs] -name = "Alert Rules" -parent = "alerting" -weight = 1 -+++ - -# Alert Rules - -> Alerting is only available in Grafana v4.0 and above. - - -hk - - diff --git a/docs/sources/alerting/rules.md b/docs/sources/alerting/rules.md new file mode 100644 index 00000000000..4fa89b391f0 --- /dev/null +++ b/docs/sources/alerting/rules.md @@ -0,0 +1,131 @@ ++++ +title = "Alerting Engine Rules Guide" +description = "Configuring Alert Rules" +keywords = ["grafana", "alerting", "guide", "rules"] +type = "docs" +[menu.docs] +name = "Engine & Rules" +parent = "alerting" +weight = 1 ++++ + +# Alerting Engine & Rules Guide + +> Alerting is only available in Grafana v4.0 and above. + +## Introduction + +{{< imgbox max-width="40%" img="/img/docs/v4/drag_handles_gif.gif" caption="Alerting overview" >}} + +Alerting in Grafana allows you to attach rules to your graph panels. When you save the dashboard +Grafana will extract the alert rules into a seperate alert rule storage and schedule them for evaluation. + +In the alert tab of the graph panel you can configure how often the alert rule should be evaluated +and the conditions that need to be met for the alert to change state and trigger its +[notifications]({{< relref "/notifications.md" >}}). + +## Execution + +The alert rules are evaluated in the Grafana backend in a scheduler and query execution engine that is part +of core Grafana. We have not implemented clustering for this scheduler yet, so if you run multiple +instances of grafana-server you have to make sure [execute_alerts]({{< relref "/installation/configuration.md#alerting" >}}) +is true on only one instance or otherwise you will get duplicated notifications. + +
+ +## Rule Config + +{{< imgbox max-width="40%" img="/img/docs/v4/alerting_conditions.png" caption="Alerting Conditions" >}} + +Currently only the graph panel supports alert rules but this will be added to the **Singlestat** and **Table** +panels as well in a future release. The rule config allows you to specify a name, how often the rule +should be evaluated and a series of conditions that all need to be true for the alert to fire. + +### Name & Evaluation interval + +Here you can specify the name of the alert rule and how often the scheduler should evaluate the alert rule. + +### Conditions + +Currently the only condition type that exists is a `Query` condition that allows you to +specify a query letter, time range and an aggregation function. The letter refers to +a query you already have added in the **Metrics** tab. The result from the query and the aggregation function is +a single value that is then used in the threshold check. The query used in an alert rule cannot +contain any template variables. Currently we only support `AND` operator between conditions. + +We plan to add other condition types in the future, like `Other Alert`, where you can include the state +of another alert in your conditions, and `Time Of Day`. + +#### Multiple Series + +If a query returns multiple series then the aggregation function and threshold check will be evaluated for each series. +What Grafana does not do currently is track alert rule state **per series**. This has implications that is exemplified +in the scenario below. + +- Alert condition with query that returns 2 series: **server1** and **server2** +- **server1** series cause the alert rule to fire and switch to state `Alerting` +- Notifications are sent out with message: _load peaking (server1)_ +- In a subsequence evaluation of the same alert rule the **server2** series also cause the alert rule to fire +- No new notifications are sent as the alert rule is already in state `Alerting`. + +So as you can see from the above scenario Grafana will not send out notifications when other series cause the alert +to fire if the rule already is in state ´Alerting`. To improve support for queries that return multiple series +we plan to track state **per series** in a future release. + +### No Data / Null values + +Below you condition you can configure how the rule evaluation engine should handle queries that return no data or only null valued +data. + +No Data Option | Description +------------ | ------------- +NoData | Set alert rule state to `NoData` +Alerting | Set alert rule state to `Alerting` +Keep Last State | Keep the current alert rule state, what ever it is. + +### Execution errors or timeouts + +The last option is how to handle execution or timeout errors. + +Error or timeout option | Description +------------ | ------------- +Alerting | Set alert rule state to `Alerting` +Keep Last State | Keep the current alert rule state, what ever it is. + +If you an unreliable time series store that where queries sometime timesout or fail randomly you can set this option +t `Keep Last State` to basically ignore them. + +## Troubleshooting + +{{< imgbox max-width="40%" img="/img/docs/v4/alert_test_rule.png" caption="Test Rule" >}} + +First level of troubleshooting you can do is hit the **Test Rule** button. You will get result back that you can expand +to the point where you can see the raw data that was returned form your query. + +Further troubleshooting can also be done by inspecting the grafana-server log. If it's not an error or for some reason +the log does not say anything you can enable debug logging for some relevant components. This is done +in grafana's ini config file. + +Example showing loggers that could be relevant when troubleshooting alerting. + +```ini +[log] +filters = alerting.scheduler:debug \ + alerting.engine:debug \ + alerting.resultHandler:debug \ + alerting.evalHandler:debug \ + alerting.evalContext:debug \ + alerting.extractor:debug \ + alerting.notifier:debug \ + alerting.notifier.slack:debug \ + alerting.notifier.pagerduty:debug \ + alerting.notifier.email:debug \ + alerting.notifier.webhook:debug \ + tsdb.graphite:debug \ + tsdb.prometheus:debug \ + tsdb.opentsdb:debug \ + tsdb.influxdb:debug \ +``` + +If you want to see raw query sent to Graphite and raw response in log you also have to set grafana.ini option `app_mode` to +`development`. diff --git a/docs/sources/guides/whats-new-in-v4.md b/docs/sources/guides/whats-new-in-v4.md new file mode 100644 index 00000000000..39b2437d054 --- /dev/null +++ b/docs/sources/guides/whats-new-in-v4.md @@ -0,0 +1,180 @@ ++++ +title = "What's New in Grafana v4.0" +description = "Feature & improvement highlights for Grafana v4.0" +keywords = ["grafana", "new", "documentation", "4.0"] +type = "docs" +[menu.docs] +name = "Version 4.0" +identifier = "v4.0" +parent = "whatsnew" +weight = -1 ++++ + +# What's New in Grafana v4.0 + +As usual this release contains a ton of minor new features, fixes and improved UX. But on top of the usual new goodies +is a core new feature: Alerting! Read on below for a detailed description of what's new in v4.0. + +## Alerting + +{{< imgbox max-width="40%" img="/img/docs/v4/drag_handles_gif.gif" caption="Alerting overview" >}} + +Alerting is a really revolutionary feature for Grafana. It transforms Grafana from a +visualization tool into a truly mission critical monitoring tool. The alert rules are very easy to +configure using your existing graph panels and threshold levels can be set simply by dragging handles to +the right side of the graph. The rules will continually be evaluated by grafana-server and +notifications will be sent out when the rule conditions are met. + +This feature has been worked on for over a year with many iterations and rewrites +just to make sure the foundations are really solid. We are really proud to finally release it! +Since the alerting execution is processed in the backend all data source plugins are not supported. +Right now Graphite, Prometheus, InfluxDB and OpenTSDB are supported. Elasticsearch is being worked +on but will be not ready for v4 release. + +
+ +### Rules + +{{< imgbox max-width="40%" img="/img/docs/v4/alerting_conditions.png" caption="Alerting Conditions" >}} + +The rule config allows you to specify a name, how often the rule should be evaluated and a series +of conditions that all need to be true for the alert to fire. + +Currently the only condition type that exists is a `Query` condition that allows you to +specify a query letter, time range and an aggregation function. The letter refers to +a query you already have added in the **Metrics** tab. The result from the +query and the aggregation function is a single value that is then used in the threshold check. + +We plan to add other condition types in the future, like `Other Alert`, where you can include the state +of another alert in your conditions, and `Time Of Day`. + +### Notifications + +{{< imgbox max-width="40%" img="/img/docs/v4/slack_notification.png" caption="Alerting Slack Notification" >}} + +Alerting would not be very useful if there was no way to send notifications when rules trigger and change state. You +can setup notifications of different types. We currently have `Slack`, `PagerDuty`, `Email` and `Webhook` with more in the +pipe that will be added during beta period. The notifications can then be added to your alert rules. +If you have configured an external image store in the grafana.ini config file (s3 and webdav options available) +you can get very rich notifications with an image of the graph and the metric +values all included in the notification. + +### Annotations + +Alert state changes are recorded in a new annotation store that is built into Grafana. This store +currently only supports storing annotations in Grafana's own internal database (mysql, postgres or sqlite). +The Grafana annotation storage is currently only used for alert state changes but we hope to add the ability for users +to add graph comments in the form of annotations directly from within Grafana in a future release. + +### Alert List Panel + +{{< imgbox max-width="30%" img="/img/docs/v4/alert_list_panel.png" caption="Alert List Panel" >}} + +This new panel allows you to show alert rules or a history of alert rule state changes. You can filter based on states your +interested in. Very useful panel for overview style dashboards. + +
+ +## Ad-hoc filter variable + +{{< imgbox max-width="30%" img="/img/docs/v4/adhoc_filters.gif" caption="Ad-hoc filters variable" >}} + +This is a new and very different type of template variable. It will allow you to create new key/value filters on the fly. +With autocomplete for both key and values. The filter condition will be automatically applied to all +queries that use that data source. This feature opens up more exploratory dashboards. In the gif animation to the right +you have a dashboard for Elasticsearch log data. It uses one query variable that allow you to quickly change how the data +is grouped, and an interval variable for controlling the granularity of the time buckets. What was missing +was a way to dynamically apply filters to the log query. With the `Ad-Hoc Filters` variable you can +dynamically add filters to any log property! + +## UX Improvements + +We always try to bring some UX/UI refinements & polish in every release. + +### TV-mode & Kiosk mode + + +
+
+

+ Grafana is so often used on wall mounted TVs that we figured a clean TV mode would be + really nice. In TV mode the top navbar, row & panel controls will all fade to transparent. +

+ +

+ This happens automatically after one minute of user inactivity but can also be toggled manually + with the d v sequence shortcut. Any mouse movement or keyboard action will + restore navbar & controls. +

+ +

+ Another feature is the kiosk mode. This can be enabled with d k + shortcut or by adding &kiosk to the URL when you load a dashboard. + In kiosk mode the navbar is completely hidden/removed from view. +

+
+
+ {{< lightboxhelper max-width="100%" img="/img/docs/v4/tvmode.png" caption="TV mode" >}} + +
+
+ +### New row menu & add panel experience + +{{< imgbox max-width="50%" img="/img/docs/v4/add_panel.gif" caption="Add Panel flow" >}} + +We spent a lot of time improving the dashboard building experience. Trying to make it both +more efficient and easier for beginners. After many good but not great experiments +with a `build mode` we eventually decided to just improve the green row menu and +continue work on a `build mode` for a future release. + +The new row menu automatically slides out when you mouse over the edge of the row. You no longer need +to hover over the small green icon and the click it to expand the row menu. + +There is some minor improvements to drag and drop behaviour. Now when dragging a panel from one row +to another you will insert the panel and Grafana will automatically make room for it. +When you drag a panel within a row you will simply reorder the panels. + +If you look at the animation to the right you can see that you can drag and drop a new panel. This is not +required, you can also just click the panel type and it will be inserted at the end of the row +automatically. Dragging a new panel has an advantage in that you can insert a new panel where ever you want +not just at the end of the row. + +We plan to further improve dashboard building in the future with a more rich grid & layout system. + +### Keyboard shortcuts + +{{< imgbox max-width="40%" img="/img/docs/v4/shortcuts.png" caption="Shortcuts" >}} + +Grafana v4 introduces a number of really powerful keyboard shortcuts. You can now focus a panel +by hovering over it with your mouse. With a panel focused you can simple hit `e` to toggle panel +edit mode, or `v` to toggle fullscreen mode. `p r` removes the panel. `p s` opens share +modal. + +Some nice navigation shortcuts are: + +- `g h` for go to home dashboard +- `s s` open search with starred pre-selected +- `s t` open search in tags list view + +
+ +## Upgrade & Breaking changes + +There are no breaking changes. Old dashboards and features should work the same. Grafana-server will automatically upgrade it's db +schema on restart. It's advisable to do a backup of Grafana's database before updating. + +If your are using plugins make sure to update your plugins as some might not work perfectly v4. + +You can update plugins using grafana-cli + + grafana-cli plugins update-all + +## Changelog + +Checkout the [CHANGELOG.md](https://github.com/grafana/grafana/blob/master/CHANGELOG.md) file for a complete list +of new features, changes, and bug fixes. + diff --git a/docs/sources/installation/configuration.md b/docs/sources/installation/configuration.md index 73724bbe12d..2fc727d659f 100644 --- a/docs/sources/installation/configuration.md +++ b/docs/sources/installation/configuration.md @@ -601,7 +601,15 @@ secret key. ex AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ### url Url to where Grafana will send PUT request with images + ### username basic auth username + ### password basic auth password + +## [alerting] + +### execute_alerts = true + +Makes it possible to turn off alert rule execution.