diff --git a/content/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/cluster-monitoring/prometheus/_index.md b/content/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/cluster-monitoring/prometheus/_index.md index f817e8603ca..b265606bc01 100644 --- a/content/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/cluster-monitoring/prometheus/_index.md +++ b/content/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/cluster-monitoring/prometheus/_index.md @@ -9,9 +9,16 @@ aliases: _Available as of v2.2.0_ - While configuring monitoring at either the [cluster level]({{}}/rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/#enabling-cluster-monitoring) or [project level]({{}}/rancher/v2.x/en/project-admin/tools/monitoring/#enabling-project-monitoring), there are multiple options that can be configured. +- [Basic Configuration](#basic-configuration) +- [Advanced Options](#advanced-options) +- [Node Exporter](#node-exporter) +- [Persistent Storage](#persistent-storage) +- [Remote Storage](#remote-storage) + +# Basic Configuration + Option | Description -------|------------- Data Retention | How long your Prometheus instance retains monitoring data scraped from Rancher objects before it's purged. @@ -24,9 +31,45 @@ Prometheus [CPU Reservation](https://kubernetes.io/docs/concepts/configuration/m Prometheus [Memory Limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource limit for the Prometheus pod. Prometheus [Memory Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource requests for the Prometheus pod. Selector | Ability to select the nodes in which Prometheus and Grafana pods are deployed to. To use this option, the nodes must have labels. -Advanced Options | Since monitoring is an [application](https://github.com/rancher/system-charts/tree/dev/charts/rancher-monitoring) from the [Rancher catalog]({{}}/rancher/v2.x/en/catalog/), it can be [configured like any other catalog application]({{}}/rancher/v2.x/en/catalog/catalog-config/). _Warning: Any modification to the application without understanding the entire application can lead to catastrophic errors._ -## Node Exporter +# Advanced Options + +Since monitoring is an [application](https://github.com/rancher/system-charts/tree/dev/charts/rancher-monitoring) from the [Rancher catalog]({{}}/rancher/v2.x/en/catalog/), it can be configured like any other catalog application, by passing in values to Helm. + +> **Warning:** Any modification to the application without understanding the entire application can lead to catastrophic errors. + +### Prometheus RemoteRead and RemoteWrite + +_Available as of v2.4.0_ + +Prometheus RemoteRead and RemoteWrite can be configured as custom answers in the **Advanced Options** section. + +For more information on remote endpoints and storage, refer to the [Prometheus documentation.](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) + +The Prometheus operator documentation contains the full [RemoteReadSpec](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#remotereadspec) and [RemoteWriteSpec.](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#remotewritespec) + +An example configuration would be: + +| Variable | Value | +|--------------|------------| +| `prometheus.remoteWrite[0].url` | `http://mytarget.com` | + +### LivenessProbe and ReadinessProbe + +_Available as of v2.4.0_ + +Prometheus LivenessProbe and ReadinessProbe can be configured as custom answers in the **Advanced Options** section. + +The Kubernetes probe spec is [here.](https://v1-17.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/#probe-v1-core) + +Some example key-value pairs are: + +| Variable | Value | +|--------------|------------| +| `prometheus.livenessProbe.timeoutSeconds` | 60 | +| `prometheus.readinessProbe.timeoutSeconds` | 60 | + +# Node Exporter The [node exporter](https://github.com/prometheus/node_exporter/blob/master/README.md) is a popular open source exporter, which exposes the metrics for hardware and \*NIX kernels OS. It is designed to monitor the host system. However, there are still issues with namespaces when running it in a container, mostly around filesystem mount spaces. In order to monitor actual network metrics for the container network, the node exporter must be deployed with the `hostNetwork` mode. @@ -34,7 +77,7 @@ When configuring Prometheus and enabling the node exporter, enter a host port in >**Warning:** In order for Prometheus to collect the metrics of the node exporter, after enabling cluster monitoring, you must open the Node Exporter Host Port in the host firewall rules to allow intranet access. By default, `9796` is used as that host port. -## Persistent Storage +# Persistent Storage >**Prerequisite:** Configure one or more [storage class]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/#adding-storage-classes) to use as [persistent storage]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/) for your Prometheus or Grafana pod. @@ -42,7 +85,7 @@ By default, when you enable Prometheus for either a cluster or project, all moni When enabling persistent storage for Prometheus or Grafana, specify the size of the persistent volume and select the [storage class]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/#storage-classes). -## Remote Storage +# Remote Storage >**Prerequisite:** Need a remote storage endpoint to be available. The possible list of integrations is available [here](https://prometheus.io/docs/operating/integrations/)