Update persisting Grafana dashboards docs for v2.5.8

This commit is contained in:
Catherine Luse
2021-04-11 13:33:47 -07:00
parent 5a34ddcc1e
commit 90b6a238df
3 changed files with 79 additions and 9 deletions
@@ -169,6 +169,8 @@ For more information on configuring Alertmanager in Rancher, see [this page.](./
**Result:** `rancher-monitoring` is uninstalled.
> **Note on Persistent Grafana Dashboards:** For users who are using Monitoring V2 v9.4.203 or below, uninstalling the Monitoring chart will delete the cattle-dashboards namespace, which will delete all persisted dashboards, unless the namespace is marked with the annotation `helm.sh/resource-policy: "keep"`. This annotation is added by default in Rancher v2.5.8+.
# Setting Resource Limits and Requests
The resource requests and limits can be configured when installing `rancher-monitoring`.
@@ -7,13 +7,66 @@ aliases:
To allow the Grafana dashboard to persist after the Grafana instance restarts, add the dashboard configuration JSON into a ConfigMap. ConfigMaps also allow the dashboards to be deployed with a GitOps or CD based approach. This allows the dashboard to be put under version control.
- [Creating a Persistent Grafana Dashboard](#creating-a-persistent-grafana-dashboard)
- [Known Issues](#known-issues)
# Creating a Persistent Grafana Dashboard
{{% tabs %}}
{{% tab "Rancher v2.5.8+" %}}
> **Prerequisites:**
>
> - The monitoring application needs to be installed.
> - You must have at least the **Manage Config Maps** Rancher RBAC permissions assigned to you in the project or namespace that contains the Grafana Dashboards. This correlates to the `monitoring-dashboard-edit` or `monitoring-dashboard-admin` Kubernetes native RBAC Roles exposed by the Monitoring chart.
1. Open the Grafana dashboard. From the **Cluster Explorer,** click **Cluster Explorer > Monitoring.**
1. Log in to Grafana. Note: The default Admin username and password for the Grafana instance is `admin/prom-operator`. (Regardless of who has the password, the **Manage Config Maps** permission in Rancher is still required to access the Grafana instance.) Alternative credentials can also be supplied on deploying or upgrading the chart.
1. Go to the dashboard that you want to persist. In the top navigation menu, go to the dashboard settings by clicking the gear icon.
1. In the left navigation menu, click **JSON Model.**
1. Copy the JSON data structure that appears.
1. Create a ConfigMap in the `cattle-dashboards` namespace.
Paste the JSON into the ConfigMap in the format shown in the example below:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
labels:
grafana_dashboard: "1"
name: <dashboard-name>
namespace: cattle-dashboards
data:
<dashboard-name>.json: |-
<copied-json>
```
> By default, Grafana is configured to watch all ConfigMaps with the `grafana_dashboard` label within the `cattle-dashboards` namespace.
>
> To specify that you would like Grafana to watch for ConfigMaps across all namespaces, refer to [this section.](#configuring-namespaces-for-the-grafana-dashboard-configmap)
**Result:** After the ConfigMap is created, it should show up on the Grafana UI and be persisted even if the Grafana pod is restarted.
Dashboards that are persisted using ConfigMaps cannot be deleted from the Grafana UI. If you attempt to delete the dashboard in the Grafana UI, you will see the error message "Dashboard cannot be deleted because it was provisioned." To delete the dashboard, you will need to delete the ConfigMap.
### Configuring Namespaces for the Grafana Dashboard ConfigMap
To specify that you would like Grafana to watch for ConfigMaps across all namespaces, set:
```
grafana.sidecar.dashboards.searchNamespace=ALL
```
Note that the RBAC roles exposed by the Monitoring chart to add Grafana Dashboards are still restricted to giving permissions for users to add dashboards in the namespace defined in `grafana.dashboards.namespace`, which defaults to `cattle-dashboards`.
{{% /tab %}}
{{% tab "Rancher v2.5.0-v2.5.8" %}}
> **Prerequisites:**
>
> - The monitoring application needs to be installed.
> - You must have the cluster-admin ClusterRole permission.
1. Open the Grafana dashboard. From the **Cluster Explorer,** click **Cluster Explorer > Monitoring.**
1. Log in to Grafana. Note: The default Admin username and password for the Grafana instance is `admin/prom-operator`. (Regardless of who has the password, cluster administrator permission in Rancher is still required access the Grafana instance.) Alternative credentials can also be supplied on deploying or upgrading the chart.
1. Log in to Grafana. Note: The default Admin username and password for the Grafana instance is `admin/prom-operator`. (Regardless of who has the password, cluster administrator permission in Rancher is still required to access the Grafana instance.) Alternative credentials can also be supplied on deploying or upgrading the chart.
1. Go to the dashboard that you want to persist. In the top navigation menu, go to the dashboard settings by clicking the gear icon.
1. In the left navigation menu, click **JSON Model.**
1. Copy the JSON data structure that appears.
@@ -35,3 +88,18 @@ To allow the Grafana dashboard to persist after the Grafana instance restarts, a
**Result:** After the ConfigMap is created, it should show up on the Grafana UI and be persisted even if the Grafana pod is restarted.
Dashboards that are persisted using ConfigMaps cannot be deleted from the Grafana UI. If you attempt to delete the dashboard in the Grafana UI, you will see the error message "Dashboard cannot be deleted because it was provisioned." To delete the dashboard, you will need to delete the ConfigMap.
To prevent the persistent dashboard from being deleted when Monitoring v2 is uninstalled, add the following annotation to the `cattle-dashboards` namespace:
```
helm.sh/resource-policy: "keep"
```
{{% /tab %}}
{{% /tabs %}}
# Known Issues
For users who are using Monitoring V2 v9.4.203 or below, uninstalling the Monitoring chart will delete the `cattle-dashboards` namespace, which will delete all persisted dashboards, unless the namespace is marked with the annotation `helm.sh/resource-policy: "keep"`.
This annotation will be added by default in the new monitoring chart released by Rancher v2.5.8, but it still needs to be manually applied for users of earlier Rancher versions.
@@ -7,7 +7,7 @@ aliases:
---
This section describes the expectations for RBAC for Rancher Monitoring.
## Cluster Admins
# Cluster Admins
By default, only those with the cluster-admin `ClusterRole` should be able to:
@@ -18,7 +18,7 @@ By default, only those with the cluster-admin `ClusterRole` should be able to:
- Persist new Grafana dashboards or datasources via creating ConfigMaps in the appropriate namespace
- Expose certain Prometheus metrics to the k8s Custom Metrics API for HPA via a Secret in the `cattle-monitoring-system` namespace
## Users with k8s ClusterRole-based Permissions
# Users with k8s ClusterRole-based Permissions
The `rancher-monitoring` chart installs the following three `ClusterRoles`. By default, they aggregate into the corresponding k8s `ClusterRoles`:
@@ -37,23 +37,23 @@ These `ClusterRoles` provide different levels of access to the Monitoring CRDs b
On a high level, the following permissions are assigned by default as a result.
### Users with k8s Admin / Edit Permissions
### Users with Kubernetes Admin/Edit Permissions
Only those with the the cluster-admin / admin / edit `ClusterRole` should be able to:
Only those with the the cluster-admin, admin or edit `ClusterRole` should be able to:
- Modify the scrape configuration of Prometheus deployments via ServiceMonitor and PodMonitor CRs
- Modify the alerting / recording rules of a Prometheus deployment via PrometheusRules CRs
### Users with k8s View Permissions
### Users with Kubernetes View Permissions
Only those with who have some k8s `ClusterRole` should be able to:
Only those with who have some Kubernetes `ClusterRole` should be able to:
- View the configuration of Prometheuses that are deployed within the cluster
- View the configuration of Alertmanagers that are deployed within the cluster
- View the scrape configuration of Prometheus deployments via ServiceMonitor and PodMonitor CRs
- View the alerting / recording rules of a Prometheus deployment via PrometheusRules CRs
- View the alerting/recording rules of a Prometheus deployment via PrometheusRules CRs
## Additional Monitoring Roles
# Additional Monitoring Roles
Monitoring also creates six additional `Roles` that are not assigned to users by default but are created within the cluster. Admins should use these roles to provide more fine-grained access to users: