public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: Added Intro to histograms and heatmaps (#24174)

Browse Source 
* Moved files

Moved files from guides folder into getting-started folder, added aliases, added Intro to histograms topic

* Updated links

* Update intro-histograms.md

Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
This commit is contained in:
Diana Payton
2020-05-05 15:23:30 -07:00
committed by GitHub
co-authored by Daniel Lee
parent f597a686fa
commit 376749ee32
13 changed files with 93 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/getting-started/_index.md
+9
View File
@@ -0,0 +1,9 @@
+++
title = "Getting started"
type = "docs"
[menu.docs]
name = "Getting started"
identifier = "getting-started"
weight = 100
+++
docs/sources/getting-started/getting-started.md
+61
View File
@@ -0,0 +1,61 @@
+++
title = "Getting started"
description = "Guide for getting started with Grafana"
keywords = ["grafana", "intro", "guide", "started"]
type = "docs"
aliases = ["/docs/grafana/latest/guides/gettingstarted","/docs/grafana/latest/guides/getting_started"]
[menu.docs]
name = "Getting started"
identifier = "getting_started_guide"
parent = "guides"
weight = 200
+++
# Getting started
This guide will help you get started and acquainted with Grafana. To learn more about Grafana in general, refer to [What is Grafana?]({{< relref "what-is-grafana.md" >}}).
## Install Grafana
This step varies according to your computer operating system. Refer to the instructions for your OS in the [Installation]({{< relref "../installation/_index.md" >}}) section for instructions.
## Log in for the first time
1. Open your web browser and go to http://localhost:3000/. `3000` is the default HTTP port that Grafana listens to if you haven’t configured a different port.
1. On the login page, type `admin` for the username and password.
1. Change your password.
> **Note:** We strongly encourage you to follow Grafana best practices and change the default administrator password. Don't forget to record your credentials!
## Create a dashboard
1. Click **New dashboard**.
1. Click **Add Query**. Grafana creates a basic graph panel with the Random Walk scenario.
1. Save your dashboard. Click the **Save dashboard** icon in the top corner of the screen.
Congratulations, you have gotten started with Grafana! You have a dashboard and are displaying results. Feel free to experiment with what you have built, continue on to add another data source, or explore [Next steps](#next-steps).
## Next steps
Different user types will have different interests. Some suggestions are listed below, or refer to [What is Grafana?]({{< relref "what-is-grafana.md" >}}) for a general overview of Grafana features.
### All users
All users might want to learn about:
* [Panels]({{< relref "../features/panels/panels.md" >}})
* [Dashboards]({{< relref "../features/dashboard/dashboards.md" >}})
* [Data sources]({{< relref "../features/datasources/data-sources.md" >}}) and [Add a data source]({{< relref "../features/datasources/add-a-data-source.md" >}})
* [Keyboard shortcuts]({{< relref "../features/shortcuts.md" >}})
* [Explore workflow]({{< relref "../features/explore/index.md" >}})
* [Plugins](https://grafana.com/grafana/plugins?orderBy=weight&direction=asc)
### Admins
Administrators might want to learn about:
* [Grafana configuration]({{< relref "../installation/configuration.md" >}})
* [Authentication]({{< relref "../auth/overview.md" >}})
* [User permissions and roles]({{< relref "../permissions/overview.md" >}})
* [Provisioning]({{< relref "../administration/provisioning.md" >}})
* [Grafana CLI]({{< relref "../administration/cli.md" >}})
docs/sources/getting-started/glossary.md
+57
View File
@@ -0,0 +1,57 @@
+++
title = "Glossary"
description = "Grafana glossary"
keywords = ["grafana", "intro", "glossary", "dictionary"]
type = "docs"
aliases = ["/docs/grafana/latest/guides/glossary"]
[menu.docs]
name = "Glossary"
identifier = "glossary"
parent = "guides"
weight = 500
+++
# Glossary
This topic lists words and abbreviations that are commonly used in the Grafana documentation and community.
<table>
<tr>
<td style = "vertical-align: top;">Dashboard</td>
<td>A set of one or more panels, organized and arranged into one or more rows, that provide an at-a-glance view of related information.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Data source</td>
<td>A file, database, or service providing the data. Grafana supports a several data sources by default, and can be extended to support additional ones through plugins.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Graph</td>
<td>A commonly-used visualization that displays data as points, lines, or bars.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Panel</td>
<td>Basic building block in Grafana, composed by a query and a visualization. Can be moved and resized within a dashboard.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Plugin</td>
<td>An extension of Grafana that allows users to provide additional functionality to enhance their experience. The types of plugins currently supported are:
<ul>
<li><b>App plugin:</b> Extends Grafana with a customized experience. It includes a set of panel and data source plugins, as well as custom pages.</li>
<li><b>Data source plugin:</b> Extends Grafana with support for additional data sources.</li>
<li><b>Panel plugin:</b> Extends Grafana with additional visualization options.</li>
</ul>
</td>
</tr>
<tr>
<td style = "vertical-align: top;">Query</td>
<td>Used to request data from a data source. The structure and format of the query depend on the specific data source.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Time series</td>
<td>A series of measurements, ordered by time. Time series are stored in data sources and returned as the result of a query.</td>
</tr>
<tr>
<td style = "vertical-align: top;">Visualization</td>
<td>A graphical representation of query results.</td>
</tr>
</table>
docs/sources/getting-started/intro-histograms.md
+58
View File
@@ -0,0 +1,58 @@
+++
title = "Histograms and heatmaps"
description = "An introduction to histograms and heatmaps"
keywords = ["grafana", "heatmap", "panel", "documentation", "histogram"]
type = "docs"
[menu.docs]
name = "intro-to-histograms"
parent = "panels"
weight = 400
+++
# Introduction to histograms and heatmaps
A histogram is a graphical representation of the distribution of numerical data. It groups values into buckets
(sometimes also called bins) and then counts how many values fall into each bucket.
Instead of graphing the actual values, histograms graph the buckets. Each bar represents a bucket,
and the bar height represents the frequency (such as count) of values that fell into that bucket's interval.
## Histogram example
This histogram shows the value distribution of a couple of time series. You can easily see that
most values land between 240-300 with a peak between 260-280.
![](/img/docs/v43/heatmap_histogram.png)
Histograms only look at _value distributions_ over a specific time range. The problem with histograms is you cannot see any trend or changes in the distribution over time.
This is where heatmaps become useful.
## Heatmaps
A _heatmap_ is like a histogram, but over time where each time slice represents its own histogram. Instead of using bar height as a representation of frequency, it uses cells and colors the cell proportional to the number of values in the bucket.
In this example, you can clearly see what values are more common and how they trend over time.
![](/img/docs/v43/heatmap_histogram_over_time.png)
## Pre-bucketed data
There are a number of data sources supporting histogram over time like Elasticsearch (by using a Histogram bucket
aggregation) or Prometheus (with [histogram](https://prometheus.io/docs/concepts/metric_types/#histogram) metric type
and *Format as* option set to Heatmap). But generally, any data source could be used if it meets the requirements:
returns series with names representing bucket bound or returns series sorted by the bound in ascending order.
## Raw data vs aggregated
If you use the heatmap with regular time series data (not pre-bucketed), then it's important to keep in mind that your data
is often already aggregated by your time series backend. Most time series queries do not return raw sample data
but include a group by time interval or maxDataPoints limit coupled with an aggregation function (usually average).
This all depends on the time range of your query of course. But the important point is to know that the histogram bucketing
that Grafana performs might be done on already aggregated and averaged data. To get more accurate heatmaps it is better
to do the bucketing during metric collection or store the data in Elasticsearch, or in the other data source which
supports doing histogram bucketing on the raw data.
If you remove or lower the group by time (or raise maxDataPoints) in your query to return more data points your heatmap will be
more accurate but this can also be very CPU and memory taxing for your browser and could cause hangs and crashes if the number of
data points becomes unreasonably large.
docs/sources/getting-started/timeseries.md
+128
View File
@@ -0,0 +1,128 @@
+++
title = "Time series"
description = "Introduction to time series"
keywords = ["grafana", "intro", "guide", "concepts", "timeseries"]
type = "docs"
[menu.docs]
aliases = ["/docs/grafana/latest/guides/timeseries"]
name = "Time series"
identifier = "time_series"
parent = "guides"
weight = 300
+++
# Introduction to time series
Imagine you wanted to know how the temperature outside changes throughout the day. Once every hour, you'd check the thermometer and write down the time along with the current temperature. After a while, you'd have something like this:
| Time | Value |
| ----- | ----- |
| 09:00 | 24°C |
| 10:00 | 26°C |
| 11:00 | 27°C |
Temperature data like this is one example of what we call a *time series*—a sequence of measurements, ordered in time. Every row in the table represents one individual measurement at a specific time.
Tables are useful when you want to identify individual measurements but make it difficult to see the big picture. A more common visualization for time series is the _graph_, which instead places each measurement along a time axis. Visual representations like the graph make it easier to discover patterns and features of the data that otherwise would be difficult to see.
{{< docs-imagebox img="/img/docs/example_graph.png" class="docs-image--no-shadow" max-width="850px" >}}
Temperature data like the one in the example, is far from the only example of a time series. Other examples of time series are:
- CPU and memory usage
- Sensor data
- Stock market index
While each of these examples are sequences of chronologically ordered measurements, they also share other attributes:
- New data is appended at the end, at regular intervals—for example, hourly at 09:00, 10:00, 11:00, and so on.
- Measurements are seldom updated after they were added—for example, yesterday's temperature doesn't change.
Time series are powerful. They help you understand the past by letting you analyze the state of the system at any point in time. Time series could tell you that the server crashed moments after the free disk space went down to zero.
Time series can also help you predict the future, by uncovering trends in your data. If the number of registered users has been increasing monthly by 4% for the past few months, you can predict how big your user base is going to be at the end of the year.
Some time series have patterns that repeat themselves over a known period. For example, the temperature is typically higher during the day, before it dips down at night. By identifying these periodic, or _seasonal_, time series, you can make confident predictions about the next period. If we know that the system load peaks every day around 18:00, we can add more machines right before.
## Aggregating time series
Depending on what you're measuring, the data can vary greatly. What if you wanted to compare periods longer than the interval between measurements? If you'd measure the temperature once every hour, you'd end up with 24 data points per day. To compare the temperature in August over the years, you'd have to combine the 31 times 24 data points into one.
Combining a collection of measurements is called _aggregation_. There are several ways to aggregate time series data. Here are some common ones:
- **Average** returns the sum of all values divided by the total number of values.
- **Min** and **Max** return the smallest, and largest value in the collection.
- **Sum** returns the sum of all values in the collection.
- **Count** returns the number of values in the collection.
For example, by aggregating the data in a month, you can determine that August 2017 was, on average, warmer than the year before. Instead, to see which month had the highest temperature, you'd compare the maximum temperature for each month.
How you choose to aggregate your time series data is an important decision and depends on the story you want to tell with your data. It's common to use different aggregations to visualize the same time series data in different ways.
## Time series and monitoring
In the IT industry, time series data is often collected to monitor things like infrastructure, hardware, or application events. Machine-generated time series data is typically collected with short intervals, which allows you to react to any unexpected changes, moments after they occur. As a consequence, data accumulates at a rapid pace, making it vital to have a way to store and query data efficiently. As a result, databases optimized for time series data have seen a rise in popularity in recent years.
### Time series databases
A time series database (TSDB) is a database explicitly designed for time series data. While it's possible to use any regular database to store measurements, a TSDB comes with some useful optimizations.
Modern time series databases take advantage of the fact that measurements are only ever appended, and rarely updated or removed. For example, the timestamps for each measurement change very little over time, which results in redundant data being stored.
Look at this sequence of Unix timestamps:
```
1572524345, 1572524375, 1572524404, 1572524434, 1572524464
```
Looking at these timestamps, they all start with `1572524`, leading to poor use of disk space. Instead, we could store each subsequent timestamp as the difference, or _delta_, from the first one:
```
1572524345, +30, +29, +30, +30
```
We could even take it a step further, by calculating the deltas of these deltas:
```
1572524345, +30, -1, +1, +0
```
If measurements are taken at regular intervals, most of these delta-of-deltas will be 0. Because of optimizations like these, TSDBs uses drastically less space than other databases.
Another feature of a TSDB is the ability to filter measurements using _tags_. Each data point is labeled with a tag that adds context information, such as where the measurement was taken. Here's an example of the [InfluxDB data format](https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/#syntax) that demonstrates how each measurement is stored.
Here are some of the TSDBs supported by Grafana:
- [Graphite](https://graphiteapp.org/)
- [InfluxDB](https://www.influxdata.com/products/influxdb-overview/)
- [Prometheus](https://prometheus.io/)
```
weather,location=us-midwest temperature=82 1465839830100400200
| -------------------- -------------- |
| | | |
| | | |
+-----------+--------+-+---------+-+---------+
|measurement|,tag_set| |field_set| |timestamp|
+-----------+--------+-+---------+-+---------+
```
### Collecting time series data
Now that we have a place to store our time series, how do we actually gather the measurements? To collect time series data, you'd typically install a _collector_ on the device, machine, or instance you want to monitor. Some collectors are made with a specific database in mind, and some support different output destinations.
Here are some examples of collectors:
- [collectd](https://collectd.org/)
- [statsd](https://github.com/statsd/statsd)
- [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/)
- [Telegraf](https://github.com/influxdata/telegraf)
A collector either _pushes_ data to a database or lets the database _pull_ the data from it. Both methods come with their own set of pros and cons:
| | Pros | Cons |
| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| Push | Easier to replicate data to multiple destinations. | The TSDB has no control over how much data gets sent. |
| Pull | Better control of how much data that gets ingested, and its authenticity. | Firewalls, VPNs or load balancers can make it hard to access the agents. |
Since it would be inefficient to write every measurement to the database, collectors pre-aggregate the data and write to the time series database at regular intervals.
docs/sources/getting-started/what-is-grafana.md
+94
View File
@@ -0,0 +1,94 @@
+++
title = "What is Grafana?"
description = "Overview of Grafana and features"
keywords = ["grafana", "intro", "guide", "started"]
type = "docs"
[menu.docs]
aliases = ["/docs/grafana/latest/guides/what-is-grafana"]
name = "What is Grafana?"
identifier = "what_is_grafana"
parent = "guides"
weight = 200
+++
# What is Grafana?
This topic provides a high-level look at Grafana, the Grafana process, and Grafana features. It's a good place to start if you want to learn more about Grafana software. To jump right in, refer to [Getting started]({{< relref "getting-started.md" >}}).
Grafana is open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.
After creating a dashboard like you do in [Getting started]({{< relref "getting-started.md" >}}), there are many possible things you might do next. It all depends on your needs and your use case.
For example, if you want to view weather data and statistics about your smart home, then you might create a playlist. If you are the administrator for a corporation and are managing Grafana for multiple teams, then you might need to set up provisioning and authentication.
The following sections provide an overview of things you might want to do with your Grafana database and links so you can learn more. For more guidance and ideas, check out the [Grafana Community forums](https://community.grafana.com/).
## Explore metrics and logs
Explore your data through ad-hoc queries and dynamic drilldown. Split view and compare different time ranges, queries and data sources side by side.
Refer to [Explore]({{< relref "../features/explore/index.md" >}}) for more information.
## Alerts
If you're using Grafana alerting, then you can have alerts sent through a number of different [alert notifiers]({{< relref "../alerting/notifications.md" >}}), including PagerDuty, SMS, email, VictorOps, OpsGenie, or Slack.
Alert hooks allow you to create different notifiers with a bit of code if you prefer some other channels of communication. Visually define [alert rules]({{< relref "../alerting/rules.md" >}}) for your most important metrics.
## Annotations
Annotate graphs with rich events from different data sources. Hover over events to see the full event metadata and tags.
This feature, which shows up as a graph marker in Grafana, is useful for correlating data in case something goes wrong. You can create the annotations manually—just control-click on a graph and input some text—or you can fetch data from any data source.
Refer to [Annotations]({{< relref "../reference/annotations.md" >}}) for more information.
## Dashboard variables
[Template variables]({{< relref "../variables/templates-and-variables.md" >}}) allow you to create dashboards that can be reused for lots of different use cases. Values aren't hard-coded with these templates, so for instance, if you have a production server and a test server, you can use the same dashboard for both.
Templating allows you to drill down into your data, say, from all data to North America data, down to Texas data, and beyond. You can also share these dashboards across teams within your organization—or if you create a great dashboard template for a popular data source, you can contribute it to the whole community to customize and use.
## Configure Grafana
If you're a Grafana administrator, then you'll want to thoroughly familiarize yourself with [Grafana configuration options]({{< relref "../installation/configuration.md" >}}) and the [Grafana CLI]({{< relref "../administration/cli.md" >}}).
Configuration covers both config files and environment variables. You can set up default ports, logging levels, email IP addresses, security, and more.
## Import dashboards and plugins
Discover hundreds of [dashboards](https://grafana.com/grafana/dashboards) and [plugins](https://grafana.com/grafana/plugins) in the official library. Thanks to the passion and momentum of community members, new ones are added every week.
## Authentication
Grafana supports different authentication methods, such as LDAP and OAuth, and allows you to map users to organizations. Refer to the [User authentication overview]({{< relref "../auth/overview.md" >}}) for more information.
In Grafana Enterprise, you can also map users to teams: If your company has its own authentication system, Grafana allows you to map the teams in your internal systems to teams in Grafana. That way, you can automatically give people access to the dashboards designated for their teams.
Refer to [Grafana Enterprise]({{< relref "../enterprise/_index.md" >}}) for more information.
## Provisioning
While it's easy to click, drag, and drop to create a single dashboard, power users in need of many dashboards will want to automate the setup with a script. You can script anything in Grafana.
For example, if you're spinning up a new Kubernetes cluster, you can also spin up a Grafana automatically with a script that would have the right server, IP address, and data sources preset and locked in so users cannot change them. It's also a way of getting control over a lot of dashboards.
Refer to [Provisioning]({{< relref "../administration/provisioning.md" >}}) for more information.
## Permissions
When organizations have one Grafana and multiple teams, they often want the ability to both keep things separate and share dashboards. You can create a team of users and then set [permissions]({{< relref "../permissions/overview.md" >}}) on folders, dashboards, and down to the [data source level]({{< relref "../enterprise/datasource_permissions.md" >}}) if you're using [Grafana Enterprise]({{< relref "../enterprise/_index.md" >}}).
## Grafana Cloud
Grafana Cloud is a highly available, fast, fully managed OpenSaaS logging and metrics platform. Everything you love about Grafana, but Grafana Labs hosts it for you and handles all the headaches.
[Learn more about Grafana Cloud.](https://grafana.com/cloud/)
## Grafana Enterprise
[Grafana Enterprise]({{< relref "../enterprise/_index.md" >}}) is a commercial edition of Grafana that includes additional features not found in the open source version.
Building on everything you already know and love about Grafana, Grafana Enterprise adds enterprise data sources, advanced authentication options, more permission controls, 24x7x365 support, and training from the core Grafana team.
[Learn more about Grafana Enterprise](https://grafana.com/enterprise). To purchase Enterprise or obtain a trial license, contact the Grafana Labs [Sales Team](https://grafana.com/contact?about=support&topic=Grafana%20Enterprise).