mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-05-13 16:43:22 +00:00
Merge pull request #2778 from catherineluse/master-merge
Merge staging into master
This commit is contained in:
Catherine Luse
GitHub
+1
-1
@@ -5,7 +5,7 @@ title = "Rancher Labs"
|
||||
theme = "rancher-website-theme"
|
||||
themesDir = "node_modules"
|
||||
pluralizeListTitles = false
|
||||
timeout = 30000
|
||||
timeout = 60000
|
||||
|
||||
enableRobotsTXT = true
|
||||
pygmentsCodeFences = true
|
||||
|
||||
@@ -3,120 +3,6 @@ title: Backups and Disaster Recovery
|
||||
weight: 5
|
||||
---
|
||||
|
||||
In this section, you'll learn how to create backups of Rancher, how to restore Rancher from backup, and how to migrate Rancher to a new Kubernetes cluster.
|
||||
If you are using Rancher v2.5, refer to [this section.](./v2.5)
|
||||
|
||||
As of Rancher v2.5, the `rancher-backup` operator is used to backup and restore Rancher. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/main/charts/rancher-backup)
|
||||
|
||||
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
|
||||
|
||||
The Rancher version must be v2.5.0 and up to use this approach of backing up and restoring Rancher.
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [Backup and Restore for Rancher v2.5 installed with Docker](#backup-and-restore-for-rancher-v2-5-installed-with-docker)
|
||||
- [Backup and Restore for Rancher installed on a Kubernetes Cluster Prior to v2.5](#backup-and-restore-for-rancher-installed-on-a-kubernetes-cluster-prior-to-v2-5)
|
||||
- [How Backups and Restores Work](#how-backups-and-restores-work)
|
||||
- [Installing the rancher-backup Operator](#installing-the-rancher-backup-operator)
|
||||
- [Installing rancher-backup with the Rancher UI](#installing-rancher-backup-with-the-rancher-ui)
|
||||
- [Installing rancher-backup with the Helm CLI](#installing-rancher-backup-with-the-helm-cli)
|
||||
- [Backing up Rancher](#backing-up-rancher)
|
||||
- [Restoring Rancher](#restoring-rancher)
|
||||
- [Migrating Rancher to a New Cluster](#migrating-rancher-to-a-new-cluster)
|
||||
- [Default Storage Location Configuration](#default-storage-location-configuration)
|
||||
- [Example values.yaml for the rancher-backup Helm Chart](#example-values-yaml-for-the-rancher-backup-helm-chart)
|
||||
|
||||
# Changes in Rancher v2.5
|
||||
|
||||
The new `rancher-backup` operator allows Rancher to be backed up and restored on any Kubernetes cluster. This application is a Helm chart, and it can be deployed through the Rancher **Apps & Marketplace** page, or by using the Helm CLI.
|
||||
|
||||
Previously, the way that cluster data was backed up depended on the type of Kubernetes cluster that was used.
|
||||
|
||||
In Rancher v2.4, it was only supported to install Rancher on two types of Kubernetes clusters: an RKE cluster, or a K3s cluster with an external database. If Rancher was installed on an RKE cluster, RKE would be used to take a snapshot of the etcd database and restore the cluster. If Rancher was installed on a K3s cluster with an external database, the database would need to be backed up and restored using the upstream documentation for the database.
|
||||
|
||||
In Rancher v2.5, it is now supported to install Rancher hosted Kubernetes clusters, such as Amazon EKS clusters, which do not expose etcd to a degree that would allow snapshots to be created by an external tool. etcd doesn't need to be exposed for `rancher-backup` to work, because the operator gathers resources by making calls to `kube-apiserver`.
|
||||
|
||||
### Backup and Restore for Rancher v2.5 installed with Docker
|
||||
|
||||
For Rancher installed with Docker, refer to the same steps used up till 2.5 for [backups](./docker-installs/docker-backups) and [restores.](./docker-installs/docker-backups)
|
||||
|
||||
### Backup and Restore for Rancher installed on a Kubernetes Cluster Prior to v2.5
|
||||
|
||||
For Rancher prior to v2.5, the way that Rancher is backed up and restored differs based on the way that Rancher was installed. Our legacy backup and restore documentation is here:
|
||||
|
||||
- For Rancher installed on an RKE Kubernetes cluster, refer to the legacy [backup]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/backup/ha-backups) and [restore]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/restore/rke-restore) documentation.
|
||||
- For Rancher installed on a K3s Kubernetes cluster, refer to the legacy [backup]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/backup/k3s-backups) and [restore]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/restore/k3s-restore) documentation.
|
||||
|
||||
# How Backups and Restores Work
|
||||
|
||||
The `rancher-backup` operator introduces three custom resources: Backups, Restores, and ResourceSets. The following cluster-scoped custom resource definitions are added to the cluster:
|
||||
|
||||
- `backups.resources.cattle.io`
|
||||
- `resourcesets.resources.cattle.io`
|
||||
- `restores.resources.cattle.io`
|
||||
|
||||
The ResourceSet defines which Kubernetes resources need to be backed up. The ResourceSet is not available to be configured in the Rancher UI because the values required to back up Rancher are predefined. This ResourceSet should not be modified.
|
||||
|
||||
When a Backup custom resource is created, the `rancher-backup` operator calls the `kube-apiserver` to get the resources in the ResourceSet (specifically, the predefined `rancher-resource-set`) that the Backup custom resource refers to.
|
||||
|
||||
The operator then creates the backup file in the .tar.gz format and stores it in the location configured in the Backup resource.
|
||||
|
||||
When a Restore custom resource is created, the operator accesses the backup .tar.gz file specified by the Restore, and restores the application from that file.
|
||||
|
||||
The Backup and Restore custom resources can be created in the Rancher UI, or by using `kubectl apply`.
|
||||
|
||||
# Installing the rancher-backup Operator
|
||||
|
||||
The `rancher-backup` operator can be installed from the Rancher UI, or with the Helm CLI. In both cases, the `rancher-backup` Helm chart is installed on the Kubernetes cluster running the Rancher server. It is a cluster-admin only feature and available only for the local cluster.
|
||||
|
||||
### Installing rancher-backup with the Rancher UI
|
||||
|
||||
1. In the Rancher UI, go to the **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click the `rancher-backup` operator.
|
||||
1. Optional: Configure the default storage location. For help, refer to the [configuration section.](./configuration/storage-config)
|
||||
|
||||
**Result:** The `rancher-backup` operator is installed.
|
||||
|
||||
From the **Cluster Explorer,** you can see the `rancher-backup` operator listed under **Deployments.**
|
||||
|
||||
To configure the backup app in Rancher, click **Cluster Explorer** in the upper left corner and click **Rancher Backups.**
|
||||
|
||||
### Installing rancher-backup with the Helm CLI
|
||||
|
||||
Install the backup app as a Helm chart:
|
||||
|
||||
```
|
||||
helm repo add rancher-charts https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install rancher-backup-crd rancher-charts/rancher-backup-crd -n cattle-resources-system --create-namespace
|
||||
helm install rancher-backup rancher-charts/rancher-backup -n cattle-resources-system
|
||||
```
|
||||
|
||||
### RBAC
|
||||
|
||||
Only the rancher admins, and local cluster’s cluster-owner can:
|
||||
|
||||
* Install the Chart
|
||||
* See the navigation links for Backup and Restore CRDs
|
||||
* Perform a backup or restore by creating a Backup CR and Restore CR respectively, list backups/restores performed so far
|
||||
|
||||
# Backing up Rancher
|
||||
|
||||
A backup is performed by creating a Backup custom resource. For a tutorial, refer to [this page.](./back-up-rancher)
|
||||
|
||||
# Restoring Rancher
|
||||
|
||||
A restore is performed by creating a Restore custom resource. For a tutorial, refer to [this page.](./restoring-rancher)
|
||||
|
||||
# Migrating Rancher to a New Cluster
|
||||
|
||||
A migration is performed by following [these steps.](./migrating-rancher)
|
||||
|
||||
# Default Storage Location Configuration
|
||||
|
||||
Configure a storage location where all backups are saved by default. You will have the option to override this with each backup, but will be limited to using an S3-compatible or Minio object store.
|
||||
|
||||
For information on configuring these options, refer to [this page.](./configuration/storage-config)
|
||||
|
||||
### Example values.yaml for the rancher-backup Helm Chart
|
||||
|
||||
The example [values.yaml file](./configuration/storage-config/#example-values-yaml-for-the-rancher-backup-helm-chart) can be used to configure the `rancher-backup` operator when the Helm CLI is used to install it.
|
||||
If you are using Rancher v2.0-v2.4, refer to [this section.](./v2.0.x-v2.4.x)
|
||||
@@ -1,22 +0,0 @@
|
||||
---
|
||||
title: Backup and Restore for Rancher 2.4.x and Below
|
||||
weight: 6
|
||||
---
|
||||
|
||||
> In Rancher v2.5, the `rancher-backup` operator is used to back up and restore Rancher. This section preserves the older documentation for backups and restores in Rancher v2.0.x-v2.4.x.
|
||||
|
||||
This section is devoted to protecting your data in a disaster scenario.
|
||||
|
||||
To protect yourself from a disaster scenario, you should create backups on a regular basis.
|
||||
|
||||
- Rancher server backups:
|
||||
- [Rancher installed on a K3s Kubernetes cluster](./backups/k3s-backups)
|
||||
- [Rancher installed on an RKE Kubernetes cluster](./backups/ha-backups)
|
||||
- [Backing up Rancher Launched Kubernetes Clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/backing-up-etcd/)
|
||||
|
||||
In a disaster scenario, you can restore your `etcd` database by restoring a backup.
|
||||
|
||||
- [Rancher Server Restorations]({{<baseurl>}}/rancher/v2.x/en/backups/restorations)
|
||||
- [Restoring Rancher Launched Kubernetes Clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/restoring-etcd/)
|
||||
|
||||
For Rancher installed with Docker, the backup and restore procedure is the same in Rancher v2.5. The backup and restore instructions for Docker installs are [here.]({{<baseurl>}}/rancher/v2.x/en/backups/docker-installs)
|
||||
@@ -1,12 +0,0 @@
|
||||
---
|
||||
title: Restore
|
||||
weight: 1010
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/restorations
|
||||
---
|
||||
If you lose the data on your Rancher Server, you can restore it if you have backups stored in a safe location.
|
||||
|
||||
- [Restoring Backups—Docker Installs]({{<baseurl>}}/rancher/v2.x/en/backups/docker-installs/docker-restores)
|
||||
- [Restoring Backups—Kubernetes installs]({{<baseurl>}}/rancher/v2.x/en/backups/restorations/ha-restoration/)
|
||||
|
||||
If you are looking to restore your [Rancher launched Kubernetes cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), please refer [here]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/restoring-etcd/).
|
||||
@@ -1,135 +0,0 @@
|
||||
---
|
||||
title: Restoring Backups—Kubernetes installs
|
||||
shortTitle: Kubernetes Installs
|
||||
weight: 370
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/after-installation/ha-backup-and-restoration/
|
||||
- /rancher/v2.x/en/installation/backups/restores
|
||||
---
|
||||
|
||||
This procedure describes how to use RKE to restore a snapshot of the Rancher Kubernetes cluster.
|
||||
This will restore the Kubernetes configuration and the Rancher database and state.
|
||||
|
||||
> **Note:** This document covers clusters set up with RKE >= v0.2.x, for older RKE versions refer to the [RKE Documentation]({{<baseurl>}}/rke/latest/en/etcd-snapshots/restoring-from-backup).
|
||||
|
||||
## Restore Outline
|
||||
|
||||
<!-- TOC -->
|
||||
|
||||
- [1. Preparation](#1-preparation)
|
||||
- [2. Place Snapshot](#2-place-snapshot)
|
||||
- [3. Configure RKE](#3-configure-rke)
|
||||
- [4. Restore the Database and bring up the Cluster](#4-restore-the-database-and-bring-up-the-cluster)
|
||||
|
||||
<!-- /TOC -->
|
||||
|
||||
### 1. Preparation
|
||||
|
||||
It is advised that you run the restore from your local host or a jump box/bastion where your cluster yaml, rke statefile, and kubeconfig are stored. You will need [RKE]({{<baseurl>}}/rke/latest/en/installation/) and [kubectl]({{<baseurl>}}/rancher/v2.x/en/faq/kubectl/) CLI utilities installed locally.
|
||||
|
||||
Prepare by creating 3 new nodes to be the target for the restored Rancher instance. We recommend that you start with fresh nodes and a clean state. For clarification on the requirements, review the [Installation Requirements](https://rancher.com/docs/rancher/v2.x/en/installation/requirements/).
|
||||
|
||||
Alternatively you can re-use the existing nodes after clearing Kubernetes and Rancher configurations. This will destroy the data on these nodes. See [Node Cleanup]({{<baseurl>}}/rancher/v2.x/en/faq/cleaning-cluster-nodes/) for the procedure.
|
||||
|
||||
> **IMPORTANT:** Before starting the restore make sure all the Kubernetes services on the old cluster nodes are stopped. We recommend powering off the nodes to be sure.
|
||||
|
||||
### 2. Place Snapshot
|
||||
|
||||
As of RKE v0.2.0, snapshots could be saved in an S3 compatible backend. To restore your cluster from the snapshot stored in S3 compatible backend, you can skip this step and retrieve the snapshot in [4. Restore the Database and bring up the Cluster](#4-restore-the-database-and-bring-up-the-cluster). Otherwise, you will need to place the snapshot directly on one of the etcd nodes.
|
||||
|
||||
Pick one of the clean nodes that will have the etcd role assigned and place the zip-compressed snapshot file in `/opt/rke/etcd-snapshots` on that node.
|
||||
|
||||
> **Note:** Because of a current limitation in RKE, the restore process does not work correctly if `/opt/rke/etcd-snapshots` is a NFS share that is mounted on all nodes with the etcd role. The easiest options are to either keep `/opt/rke/etcd-snapshots` as a local folder during the restore process and only mount the NFS share there after it has been completed, or to only mount the NFS share to one node with an etcd role in the beginning.
|
||||
|
||||
### 3. Configure RKE
|
||||
|
||||
Use your original `rancher-cluster.yml` and `rancher-cluster.rkestate` files. If they are not stored in a version control system, it is a good idea to back them up before making any changes.
|
||||
|
||||
```
|
||||
cp rancher-cluster.yml rancher-cluster.yml.bak
|
||||
cp rancher-cluster.rkestate rancher-cluster.rkestate.bak
|
||||
```
|
||||
|
||||
If the replaced or cleaned nodes have been configured with new IP addresses, modify the `rancher-cluster.yml` file to ensure the address and optional internal_address fields reflect the new addresses.
|
||||
|
||||
> **IMPORTANT:** You should not rename the `rancher-cluster.yml` or `rancher-cluster.rkestate` files. It is important that the filenames match each other.
|
||||
|
||||
### 4. Restore the Database and bring up the Cluster
|
||||
|
||||
You will now use the RKE command-line tool with the `rancher-cluster.yml` and the `rancher-cluster.rkestate` configuration files to restore the etcd database and bring up the cluster on the new nodes.
|
||||
|
||||
> **Note:** Ensure your `rancher-cluster.rkestate` is present in the same directory as the `rancher-cluster.yml` file before starting the restore, as this file contains the certificate data for the cluster.
|
||||
|
||||
#### Restoring from a Local Snapshot
|
||||
|
||||
When restoring etcd from a local snapshot, the snapshot is assumed to be located on the target node in the directory `/opt/rke/etcd-snapshots`.
|
||||
|
||||
```
|
||||
rke etcd snapshot-restore --name snapshot-name --config ./rancher-cluster.yml
|
||||
```
|
||||
|
||||
> **Note:** The --name parameter expects the filename of the snapshot without the extension.
|
||||
|
||||
#### Restoring from a Snapshot in S3
|
||||
|
||||
_Available as of RKE v0.2.0_
|
||||
|
||||
When restoring etcd from a snapshot located in an S3 compatible backend, the command needs the S3 information in order to connect to the S3 backend and retrieve the snapshot.
|
||||
|
||||
```
|
||||
$ rke etcd snapshot-restore --config ./rancher-cluster.yml --name snapshot-name \
|
||||
--s3 --access-key S3_ACCESS_KEY --secret-key S3_SECRET_KEY \
|
||||
--bucket-name s3-bucket-name --s3-endpoint s3.amazonaws.com \
|
||||
--folder folder-name # Available as of v2.3.0
|
||||
```
|
||||
|
||||
#### Options for `rke etcd snapshot-restore`
|
||||
|
||||
S3 specific options are only available for RKE v0.2.0+.
|
||||
|
||||
| Option | Description | S3 Specific |
|
||||
| --- | --- | ---|
|
||||
| `--name` value | Specify snapshot name | |
|
||||
| `--config` value | Specify an alternate cluster YAML file (default: "cluster.yml") [$RKE_CONFIG] | |
|
||||
| `--s3` | Enabled backup to s3 |* |
|
||||
| `--s3-endpoint` value | Specify s3 endpoint url (default: "s3.amazonaws.com") | * |
|
||||
| `--access-key` value | Specify s3 accessKey | *|
|
||||
| `--secret-key` value | Specify s3 secretKey | *|
|
||||
| `--bucket-name` value | Specify s3 bucket name | *|
|
||||
| `--folder` value | Specify s3 folder in the bucket name _Available as of v2.3.0_ | *|
|
||||
| `--region` value | Specify the s3 bucket location (optional) | *|
|
||||
| `--ssh-agent-auth` | [Use SSH Agent Auth defined by SSH_AUTH_SOCK]({{<baseurl>}}/rke/latest/en/config-options/#ssh-agent) | |
|
||||
| `--ignore-docker-version` | [Disable Docker version check]({{<baseurl>}}/rke/latest/en/config-options/#supported-docker-versions) |
|
||||
|
||||
#### Testing the Cluster
|
||||
|
||||
Once RKE completes it will have created a credentials file in the local directory. Configure `kubectl` to use the `kube_config_rancher-cluster.yml` credentials file and check on the state of the cluster. See [Installing and Configuring kubectl]({{<baseurl>}}/rancher/v2.x/en/faq/kubectl/#configuration) for details.
|
||||
|
||||
#### Check Kubernetes Pods
|
||||
|
||||
Wait for the pods running in `kube-system`, `ingress-nginx` and the `rancher` pod in `cattle-system` to return to the `Running` state.
|
||||
|
||||
> **Note:** `cattle-cluster-agent` and `cattle-node-agent` pods will be in an `Error` or `CrashLoopBackOff` state until Rancher server is up and the DNS/Load Balancer have been pointed at the new cluster.
|
||||
|
||||
```
|
||||
kubectl get pods --all-namespaces
|
||||
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
cattle-system cattle-cluster-agent-766585f6b-kj88m 0/1 Error 6 4m
|
||||
cattle-system cattle-node-agent-wvhqm 0/1 Error 8 8m
|
||||
cattle-system rancher-78947c8548-jzlsr 0/1 Running 1 4m
|
||||
ingress-nginx default-http-backend-797c5bc547-f5ztd 1/1 Running 1 4m
|
||||
ingress-nginx nginx-ingress-controller-ljvkf 1/1 Running 1 8m
|
||||
kube-system canal-4pf9v 3/3 Running 3 8m
|
||||
kube-system cert-manager-6b47fc5fc-jnrl5 1/1 Running 1 4m
|
||||
kube-system kube-dns-7588d5b5f5-kgskt 3/3 Running 3 4m
|
||||
kube-system kube-dns-autoscaler-5db9bbb766-s698d 1/1 Running 1 4m
|
||||
kube-system metrics-server-97bc649d5-6w7zc 1/1 Running 1 4m
|
||||
kube-system tiller-deploy-56c4cf647b-j4whh 1/1 Running 1 4m
|
||||
```
|
||||
|
||||
#### Finishing Up
|
||||
|
||||
Rancher should now be running and available to manage your Kubernetes clusters. Review the [recommended architecture]({{<baseurl>}}/rancher/v2.x/en/installation/k8s-install/#recommended-architecture) for Kubernetes installations and update the endpoints for Rancher DNS or the Load Balancer that you built during Step 1 of the Kubernetes install ([1. Create Nodes and Load Balancer]({{<baseurl>}}/rancher/v2.x/en/installation/k8s-install/create-nodes-lb/#load-balancer)) to target the new cluster. Once the endpoints are updated, the agents on your managed clusters should automatically reconnect. This may take 10-15 minutes due to reconnect back off timeouts.
|
||||
|
||||
> **IMPORTANT:** Remember to save your updated RKE config (`rancher-cluster.yml`) state file (`rancher-cluster.rkestate`) and `kubectl` credentials (`kube_config_rancher-cluster.yml`) files in a safe place for future maintenance for example in a version control system.
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: Rancher v2.0-v2.4
|
||||
weight: 2
|
||||
---
|
||||
|
||||
> In Rancher v2.5, the `rancher-backup` operator is used to back up and restore Rancher. This section preserves the older documentation for backups and restores in Rancher v2.0.x-v2.4.x.
|
||||
|
||||
This section is devoted to protecting your data in a disaster scenario.
|
||||
|
||||
To protect yourself from a disaster scenario, you should create backups on a regular basis.
|
||||
|
||||
- [Backup](./backup)
|
||||
- [Restore](./restore)
|
||||
|
||||
+7
-3
@@ -5,11 +5,15 @@ aliases:
|
||||
- /rancher/v2.x/en/installation/after-installation/
|
||||
- /rancher/v2.x/en/backups/
|
||||
- /rancher/v2.x/en/backups/backups
|
||||
- /rancher/v2.x/en/backups/legacy/backup
|
||||
---
|
||||
This section contains information about how to create backups of your Rancher data and how to restore them in a disaster scenario.
|
||||
|
||||
- [Backing up Rancher installed on a K3s Kubernetes cluster](./k3s-backups)
|
||||
- [Backing up Rancher installed on an RKE Kubernetes cluster](./ha-backups/)
|
||||
- [Backing up Rancher installed with Docker]({{<baseurl>}}/rancher/v2.x/en/backups/docker-installs/docker-backups)
|
||||
- Rancher server backups:
|
||||
- [Rancher installed on a K3s Kubernetes cluster](./k3s-backups)
|
||||
- [Rancher installed on an RKE Kubernetes cluster](./rke-backups)
|
||||
- [Rancher installed with Docker](./docker-backups)
|
||||
|
||||
For information on backing up Rancher launched Kubernetes clusters, refer to [this section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/backing-up-etcd/)
|
||||
|
||||
If you are looking to back up your [Rancher launched Kubernetes cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), please refer [here]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/backing-up-etcd/).
|
||||
+1
@@ -7,6 +7,7 @@ aliases:
|
||||
- /rancher/v2.x/en/backups/backups/k8s-backups/k3s-backups
|
||||
- /rancher/v2.x/en/backups/legacy/backup/k8s-backups/k3s-backups/
|
||||
- /rancher/v2.x/en/backups/legacy/backups/k3s-backups
|
||||
- /rancher/v2.x/en/backups/legacy/backup/k3s-backups
|
||||
---
|
||||
|
||||
When Rancher is installed on a high-availability Kubernetes cluster, we recommend using an external database to store the cluster data.
|
||||
+1
@@ -9,6 +9,7 @@ aliases:
|
||||
- /rancher/v2.x/en/backups/backups/k8s-backups/ha-backups
|
||||
- /rancher/v2.x/en/backups/legacy/backup/k8s-backups/ha-backups/
|
||||
- /rancher/v2.x/en/backups/legacy/backups/ha-backups
|
||||
- /rancher/v2.x/en/backups/legacy/backup/ha-backups
|
||||
---
|
||||
This section describes how to create backups of your high-availability Rancher install.
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: Restore
|
||||
weight: 1010
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/restorations
|
||||
- /rancher/v2.x/en/backups/legacy/restore
|
||||
---
|
||||
If you lose the data on your Rancher Server, you can restore it if you have backups stored in a safe location.
|
||||
|
||||
- [Restoring backups for Rancher installed with Docker](./docker-restores)
|
||||
- [Restoring backups for Rancher installed on an RKE Kubernetes cluster](./rke-restore)
|
||||
- [Restoring backups for Rancher installed on a K3s Kubernetes cluster](./k3s-restore)
|
||||
|
||||
If you are looking to restore your [Rancher launched Kubernetes cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), please refer to [this section]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/restoring-etcd/).
|
||||
+1
@@ -6,6 +6,7 @@ aliases:
|
||||
- /rancher/v2.x/en/backups/restorations/k3s-restoration
|
||||
- /rancher/v2.x/en/backups/restorations/k8s-restore/k3s-restore
|
||||
- /rancher/v2.x/en/backups/legacy/restore/k8s-restore/k3s-restore/
|
||||
- /rancher/v2.x/en/backups/legacy/restore/k3s-restore
|
||||
---
|
||||
|
||||
When Rancher is installed on a high-availability Kubernetes cluster, we recommend using an external database to store the cluster data.
|
||||
+1
@@ -7,6 +7,7 @@ aliases:
|
||||
- /rancher/v2.x/en/backups/restorations/ha-restoration
|
||||
- /rancher/v2.x/en/backups/restorations/k8s-restore/rke-restore
|
||||
- /rancher/v2.x/en/backups/legacy/restore/k8s-restore/rke-restore/
|
||||
- /rancher/v2.x/en/backups/legacy/restore/rke-restore
|
||||
---
|
||||
|
||||
This procedure describes how to use RKE to restore a snapshot of the Rancher Kubernetes cluster.
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
title: Backup and Restore in Rancher v2.5
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
In this section, you'll learn how to create backups of Rancher, how to restore Rancher from backup, and how to migrate Rancher to a new Kubernetes cluster.
|
||||
|
||||
As of Rancher v2.5, the `rancher-backup` operator is used to backup and restore Rancher. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/main/charts/rancher-backup)
|
||||
|
||||
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
|
||||
|
||||
The Rancher version must be v2.5.0 and up to use this approach of backing up and restoring Rancher.
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [Backup and Restore for Rancher v2.5 installed with Docker](#backup-and-restore-for-rancher-v2-5-installed-with-docker)
|
||||
- [Backup and Restore for Rancher installed on a Kubernetes Cluster Prior to v2.5](#backup-and-restore-for-rancher-installed-on-a-kubernetes-cluster-prior-to-v2-5)
|
||||
- [How Backups and Restores Work](#how-backups-and-restores-work)
|
||||
- [Installing the rancher-backup Operator](#installing-the-rancher-backup-operator)
|
||||
- [Installing rancher-backup with the Rancher UI](#installing-rancher-backup-with-the-rancher-ui)
|
||||
- [Installing rancher-backup with the Helm CLI](#installing-rancher-backup-with-the-helm-cli)
|
||||
- [Backing up Rancher](#backing-up-rancher)
|
||||
- [Restoring Rancher](#restoring-rancher)
|
||||
- [Migrating Rancher to a New Cluster](#migrating-rancher-to-a-new-cluster)
|
||||
- [Default Storage Location Configuration](#default-storage-location-configuration)
|
||||
- [Example values.yaml for the rancher-backup Helm Chart](#example-values-yaml-for-the-rancher-backup-helm-chart)
|
||||
|
||||
# Changes in Rancher v2.5
|
||||
|
||||
The new `rancher-backup` operator allows Rancher to be backed up and restored on any Kubernetes cluster. This application is a Helm chart, and it can be deployed through the Rancher **Apps & Marketplace** page, or by using the Helm CLI.
|
||||
|
||||
Previously, the way that cluster data was backed up depended on the type of Kubernetes cluster that was used.
|
||||
|
||||
In Rancher v2.4, it was only supported to install Rancher on two types of Kubernetes clusters: an RKE cluster, or a K3s cluster with an external database. If Rancher was installed on an RKE cluster, RKE would be used to take a snapshot of the etcd database and restore the cluster. If Rancher was installed on a K3s cluster with an external database, the database would need to be backed up and restored using the upstream documentation for the database.
|
||||
|
||||
In Rancher v2.5, it is now supported to install Rancher hosted Kubernetes clusters, such as Amazon EKS clusters, which do not expose etcd to a degree that would allow snapshots to be created by an external tool. etcd doesn't need to be exposed for `rancher-backup` to work, because the operator gathers resources by making calls to `kube-apiserver`.
|
||||
|
||||
### Backup and Restore for Rancher v2.5 installed with Docker
|
||||
|
||||
For Rancher installed with Docker, refer to the same steps used up till 2.5 for [backups](./docker-installs/docker-backups) and [restores.](./docker-installs/docker-backups)
|
||||
|
||||
### Backup and Restore for Rancher installed on a Kubernetes Cluster Prior to v2.5
|
||||
|
||||
For Rancher prior to v2.5, the way that Rancher is backed up and restored differs based on the way that Rancher was installed. Our legacy backup and restore documentation is here:
|
||||
|
||||
- For Rancher installed on an RKE Kubernetes cluster, refer to the legacy [backup]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/backup/ha-backups) and [restore]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/restore/rke-restore) documentation.
|
||||
- For Rancher installed on a K3s Kubernetes cluster, refer to the legacy [backup]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/backup/k3s-backups) and [restore]({{<baseurl>}}/rancher/v2.x/en/backups/legacy/restore/k3s-restore) documentation.
|
||||
|
||||
# How Backups and Restores Work
|
||||
|
||||
The `rancher-backup` operator introduces three custom resources: Backups, Restores, and ResourceSets. The following cluster-scoped custom resource definitions are added to the cluster:
|
||||
|
||||
- `backups.resources.cattle.io`
|
||||
- `resourcesets.resources.cattle.io`
|
||||
- `restores.resources.cattle.io`
|
||||
|
||||
The ResourceSet defines which Kubernetes resources need to be backed up. The ResourceSet is not available to be configured in the Rancher UI because the values required to back up Rancher are predefined. This ResourceSet should not be modified.
|
||||
|
||||
When a Backup custom resource is created, the `rancher-backup` operator calls the `kube-apiserver` to get the resources in the ResourceSet (specifically, the predefined `rancher-resource-set`) that the Backup custom resource refers to.
|
||||
|
||||
The operator then creates the backup file in the .tar.gz format and stores it in the location configured in the Backup resource.
|
||||
|
||||
When a Restore custom resource is created, the operator accesses the backup .tar.gz file specified by the Restore, and restores the application from that file.
|
||||
|
||||
The Backup and Restore custom resources can be created in the Rancher UI, or by using `kubectl apply`.
|
||||
|
||||
# Installing the rancher-backup Operator
|
||||
|
||||
The `rancher-backup` operator can be installed from the Rancher UI, or with the Helm CLI. In both cases, the `rancher-backup` Helm chart is installed on the Kubernetes cluster running the Rancher server. It is a cluster-admin only feature and available only for the local cluster.
|
||||
|
||||
### Installing rancher-backup with the Rancher UI
|
||||
|
||||
1. In the Rancher UI, go to the **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click the `rancher-backup` operator.
|
||||
1. Optional: Configure the default storage location. For help, refer to the [configuration section.](./configuration/storage-config)
|
||||
|
||||
**Result:** The `rancher-backup` operator is installed.
|
||||
|
||||
From the **Cluster Explorer,** you can see the `rancher-backup` operator listed under **Deployments.**
|
||||
|
||||
To configure the backup app in Rancher, click **Cluster Explorer** in the upper left corner and click **Rancher Backups.**
|
||||
|
||||
### Installing rancher-backup with the Helm CLI
|
||||
|
||||
Install the backup app as a Helm chart:
|
||||
|
||||
```
|
||||
helm repo add rancher-charts https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install rancher-backup-crd rancher-charts/rancher-backup-crd -n cattle-resources-system --create-namespace
|
||||
helm install rancher-backup rancher-charts/rancher-backup -n cattle-resources-system
|
||||
```
|
||||
|
||||
### RBAC
|
||||
|
||||
Only the rancher admins, and local cluster’s cluster-owner can:
|
||||
|
||||
* Install the Chart
|
||||
* See the navigation links for Backup and Restore CRDs
|
||||
* Perform a backup or restore by creating a Backup CR and Restore CR respectively, list backups/restores performed so far
|
||||
|
||||
# Backing up Rancher
|
||||
|
||||
A backup is performed by creating a Backup custom resource. For a tutorial, refer to [this page.](./back-up-rancher)
|
||||
|
||||
# Restoring Rancher
|
||||
|
||||
A restore is performed by creating a Restore custom resource. For a tutorial, refer to [this page.](./restoring-rancher)
|
||||
|
||||
# Migrating Rancher to a New Cluster
|
||||
|
||||
A migration is performed by following [these steps.](./migrating-rancher)
|
||||
|
||||
# Default Storage Location Configuration
|
||||
|
||||
Configure a storage location where all backups are saved by default. You will have the option to override this with each backup, but will be limited to using an S3-compatible or Minio object store.
|
||||
|
||||
For information on configuring these options, refer to [this page.](./configuration/storage-config)
|
||||
|
||||
### Example values.yaml for the rancher-backup Helm Chart
|
||||
|
||||
The example [values.yaml file](./configuration/storage-config/#example-values-yaml-for-the-rancher-backup-helm-chart) can be used to configure the `rancher-backup` operator when the Helm CLI is used to install it.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Backing up Rancher
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/back-up-rancher
|
||||
---
|
||||
|
||||
In this section, you'll learn how to back up Rancher running on any Kubernetes cluster. To backup Rancher installed with Docker, refer the instructions for [single node backups](../legacy/backup/single-node-backups/)
|
||||
+2
@@ -2,6 +2,8 @@
|
||||
title: Rancher Backup Configuration Reference
|
||||
shortTitle: Configuration
|
||||
weight: 4
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/configuration
|
||||
---
|
||||
|
||||
- [Backup configuration](./backup-config)
|
||||
+2
@@ -2,6 +2,8 @@
|
||||
title: Backup Configuration
|
||||
shortTitle: Backup
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/configuration/backup-config
|
||||
---
|
||||
|
||||
The Backup Create page lets you configure a schedule, enable encryption and specify the storage location for your backups.
|
||||
+2
@@ -2,6 +2,8 @@
|
||||
title: Restore Configuration
|
||||
shortTitle: Restore
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/configuration/restore-config
|
||||
---
|
||||
|
||||
The Restore Create page lets you provide details of the backup to restore from
|
||||
+2
@@ -2,6 +2,8 @@
|
||||
title: Backup Storage Location Configuration
|
||||
shortTitle: Storage
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/configuration/storage-config
|
||||
---
|
||||
|
||||
Configure a storage location where all backups are saved by default. You will have the option to override this with each backup, but will be limited to using an S3-compatible object store.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
title: Backing up Rancher Installed with Docker
|
||||
shortTitle: Backups
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/
|
||||
- /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/
|
||||
- /rancher/v2.x/en/backups/backups/single-node-backups/
|
||||
- /rancher/v2.x/en/backups/legacy/backup/single-node-backups/
|
||||
---
|
||||
|
||||
|
||||
After completing your Docker installation of Rancher, we recommend creating backups of it on a regular basis. Having a recent backup will let you recover quickly from an unexpected disaster.
|
||||
|
||||
## Before You Start
|
||||
|
||||
During the creation of your backup, you'll enter a series of commands, replacing placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (`<EXAMPLE>`). Here's an example of a command with a placeholder:
|
||||
|
||||
```
|
||||
docker run --volumes-from rancher-data-<DATE> -v $PWD:/backup busybox tar pzcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
|
||||
```
|
||||
|
||||
In this command, `<DATE>` is a placeholder for the date that the data container and backup were created. `9-27-18` for example.
|
||||
|
||||
Cross reference the image and reference table below to learn how to obtain this placeholder data. Write down or copy this information before starting the [procedure below](#creating-a-backup).
|
||||
|
||||
<sup>Terminal `docker ps` Command, Displaying Where to Find `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>`</sup>
|
||||
|
||||
|
||||
| Placeholder | Example | Description |
|
||||
| -------------------------- | -------------------------- | --------------------------------------------------------- |
|
||||
| `<RANCHER_CONTAINER_TAG>` | `v2.0.5` | The rancher/rancher image you pulled for initial install. |
|
||||
| `<RANCHER_CONTAINER_NAME>` | `festive_mestorf` | The name of your Rancher container. |
|
||||
| `<RANCHER_VERSION>` | `v2.0.5` | The version of Rancher that you're creating a backup for. |
|
||||
| `<DATE>` | `9-27-18` | The date that the data container or backup was created. |
|
||||
<br/>
|
||||
|
||||
You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by logging into your Rancher Server by remote connection and entering the command to view the containers that are running: `docker ps`. You can also view containers that are stopped with `docker ps -a`. Use these commands for help anytime while creating backups.
|
||||
|
||||
## Creating a Backup
|
||||
|
||||
This procedure creates a backup that you can restore if Rancher encounters a disaster scenario.
|
||||
|
||||
|
||||
1. Using a remote Terminal connection, log into the node running your Rancher Server.
|
||||
|
||||
1. Stop the container currently running Rancher Server. Replace `<RANCHER_CONTAINER_NAME>` with the [name of your Rancher container](#before-you-start).
|
||||
|
||||
```
|
||||
docker stop <RANCHER_CONTAINER_NAME>
|
||||
```
|
||||
1. <a id="backup"></a>Use the command below, replacing each [placeholder](#before-you-start), to create a data container from the Rancher container that you just stopped.
|
||||
|
||||
```
|
||||
docker create --volumes-from <RANCHER_CONTAINER_NAME> --name rancher-data-<DATE> rancher/rancher:<RANCHER_CONTAINER_TAG>
|
||||
```
|
||||
|
||||
1. <a id="tarball"></a>From the data container that you just created (`rancher-data-<DATE>`), create a backup tarball (`rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz`). Use the following command, replacing each [placeholder](#before-you-start).
|
||||
|
||||
```
|
||||
docker run --volumes-from rancher-data-<DATE> -v $PWD:/backup:z busybox tar pzcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
|
||||
```
|
||||
|
||||
**Step Result:** A stream of commands runs on the screen.
|
||||
|
||||
1. Enter the `ls` command to confirm that the backup tarball was created. It will have a name similar to `rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz`.
|
||||
|
||||
1. Move your backup tarball to a safe location external to your Rancher Server. Then delete the `rancher-data-<DATE>` container from your Rancher Server.
|
||||
|
||||
1. Restart Rancher Server. Replace `<RANCHER_CONTAINER_NAME>` with the name of your [Rancher container](#before-you-start).
|
||||
|
||||
```
|
||||
docker start <RANCHER_CONTAINER_NAME>
|
||||
```
|
||||
|
||||
**Result:** A backup tarball of your Rancher Server data is created. See [Restoring Backups: Docker Installs]({{<baseurl>}}/rancher/v2.x/en/backups/restorations/single-node-restoration) if you need to restore backup data.
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
title: Restoring Backups—Docker Installs
|
||||
shortTitle: Restores
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/
|
||||
- /rancher/v2.x/en/backups/restorations/single-node-restoration
|
||||
---
|
||||
|
||||
If you encounter a disaster scenario, you can restore your Rancher Server to your most recent backup.
|
||||
|
||||
## Before You Start
|
||||
|
||||
During restore of your backup, you'll enter a series of commands, filling placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (`<EXAMPLE>`). Here's an example of a command with a placeholder:
|
||||
|
||||
```
|
||||
docker run --volumes-from <RANCHER_CONTAINER_NAME> -v $PWD:/backup \
|
||||
busybox sh -c "rm /var/lib/rancher/* -rf && \
|
||||
tar pzxvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>"
|
||||
```
|
||||
|
||||
In this command, `<RANCHER_CONTAINER_NAME>` and `<RANCHER_VERSION>-<DATE>` are environment variables for your Rancher deployment.
|
||||
|
||||
Cross reference the image and reference table below to learn how to obtain this placeholder data. Write down or copy this information before starting the procedure below.
|
||||
|
||||
<sup>Terminal `docker ps` Command, Displaying Where to Find `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>`</sup>
|
||||
|
||||
|
||||
| Placeholder | Example | Description |
|
||||
| -------------------------- | -------------------------- | --------------------------------------------------------- |
|
||||
| `<RANCHER_CONTAINER_TAG>` | `v2.0.5` | The rancher/rancher image you pulled for initial install. |
|
||||
| `<RANCHER_CONTAINER_NAME>` | `festive_mestorf` | The name of your Rancher container. |
|
||||
| `<RANCHER_VERSION>` | `v2.0.5` | The version number for your Rancher backup. |
|
||||
| `<DATE>` | `9-27-18` | The date that the data container or backup was created. |
|
||||
<br/>
|
||||
|
||||
You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by logging into your Rancher Server by remote connection and entering the command to view the containers that are running: `docker ps`. You can also view containers that are stopped using a different command: `docker ps -a`. Use these commands for help anytime during while creating backups.
|
||||
|
||||
## Restoring Backups
|
||||
|
||||
Using a [backup]({{<baseurl>}}/rancher/v2.x/en/backups/backups/single-node-backups/) that you created earlier, restore Rancher to its last known healthy state.
|
||||
|
||||
1. Using a remote Terminal connection, log into the node running your Rancher Server.
|
||||
|
||||
1. Stop the container currently running Rancher Server. Replace `<RANCHER_CONTAINER_NAME>` with the [name of your Rancher container](#before-you-start).
|
||||
|
||||
```
|
||||
docker stop <RANCHER_CONTAINER_NAME>
|
||||
```
|
||||
1. Move the backup tarball that you created during completion of [Creating Backups—Docker Installs]({{<baseurl>}}/rancher/v2.x/en/backups/backups/single-node-backups/) onto your Rancher Server. Change to the directory that you moved it to. Enter `dir` to confirm that it's there.
|
||||
|
||||
If you followed the naming convention we suggested in [Creating Backups—Docker Installs]({{<baseurl>}}/rancher/v2.x/en/backups/backups/single-node-backups/), it will have a name similar to `rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz`.
|
||||
|
||||
1. Enter the following command to delete your current state data and replace it with your backup data, replacing the [placeholders](#before-you-start). Don't forget to close the quotes.
|
||||
|
||||
>**Warning!** This command deletes all current state data from your Rancher Server container. Any changes saved after your backup tarball was created will be lost.
|
||||
|
||||
```
|
||||
docker run --volumes-from <RANCHER_CONTAINER_NAME> -v $PWD:/backup \
|
||||
busybox sh -c "rm /var/lib/rancher/* -rf && \
|
||||
tar pzxvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz"
|
||||
```
|
||||
|
||||
**Step Result:** A series of commands should run.
|
||||
|
||||
1. Restart your Rancher Server container, replacing the [placeholder](#before-you-start). It will restart using your backup data.
|
||||
|
||||
```
|
||||
docker start <RANCHER_CONTAINER_NAME>
|
||||
```
|
||||
|
||||
1. Wait a few moments and then open Rancher in a web browser. Confirm that the restore succeeded and that your data is restored.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Examples
|
||||
weight: 5
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/examples
|
||||
---
|
||||
|
||||
This section contains examples of Backup and Restore custom resources.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Migrating Rancher to a New Cluster
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/backups/migrating-rancher
|
||||
---
|
||||
|
||||
If you are migrating Rancher to a new Kubernetes cluster, you don't need to install Rancher on the new cluster first. If Rancher is restored to a new cluster with Rancher already installed, it can cause problems.
|
||||
+4
-2
@@ -3,13 +3,15 @@ title: Restoring Rancher
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/backups/restores
|
||||
- /rancher/v2.x/en/backups/restoring-rancher
|
||||
---
|
||||
|
||||
A restore is performed by creating a Restore custom resource.
|
||||
|
||||
> **Important**
|
||||
* Follow the instructions from this page for restoring rancher on the same cluster where it was backed up from. In order to migrate rancher to a new cluster, follow the steps to [migrate rancher.](../migrating-rancher)
|
||||
* While restoring rancher on the same setup, the operator will scale down the rancher deployment when restore starts, and it will scale back up the deployment once restore completes. So Rancher will be unavailable during the restore.
|
||||
>
|
||||
> * Follow the instructions from this page for restoring rancher on the same cluster where it was backed up from. In order to migrate rancher to a new cluster, follow the steps to [migrate rancher.](../migrating-rancher)
|
||||
> * While restoring rancher on the same setup, the operator will scale down the rancher deployment when restore starts, and it will scale back up the deployment once restore completes. So Rancher will be unavailable during the restore.
|
||||
|
||||
### Create the Restore Custom Resource
|
||||
|
||||
@@ -5,232 +5,7 @@ weight: 18
|
||||
|
||||
_Available as of v2.4.0_
|
||||
|
||||
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark.
|
||||
|
||||
The `rancher-cis-benchmark` app leverages <a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench,</a> an open-source tool from Aqua Security, to check clusters for CIS Kubernetes Benchmark compliance. Also, to generate a cluster-wide report, the application utilizes <a href="https://github.com/vmware-tanzu/sonobuoy" target="_blank">Sonobuoy</a> for report aggregation.
|
||||
If you are using Rancher v2.5, refer to the CIS scan documentation [here.](./v2.5)
|
||||
|
||||
> The CIS scan feature was improved in Rancher v2.5. If you are using Rancher v2.4, refer to the older version of the CIS scan documentation [here.](./legacy)
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [About the CIS Benchmark](#about-the-cis-benchmark)
|
||||
- [Installing rancher-cis-benchmark](#installing-rancher-cis-benchmark)
|
||||
- [Uninstalling rancher-cis-benchmark](#uninstalling-rancher-cis-benchmark)
|
||||
- [Running a Scan](#running-a-scan)
|
||||
- [Skipping Tests](#skipping-tests)
|
||||
- [Viewing Reports](#viewing-reports)
|
||||
- [About the generated report](#about-the-generated-report)
|
||||
- [Test Profiles](#test-profiles)
|
||||
- [About Skipped and Not Applicable Tests](#about-skipped-and-not-applicable-tests)
|
||||
- [Roles-based access control](./rbac)
|
||||
- [Configuration](./configuration)
|
||||
|
||||
### Changes in Rancher v2.5
|
||||
|
||||
We now support running CIS scans on any Kubernetes cluster, including hosted Kubernetes providers such as EKS, AKS, and GKE. Previously it was only supported to run CIS scans on RKE Kubernetes clusters.
|
||||
|
||||
In Rancher v2.4, the CIS scan tool was available from the **cluster manager** in the Rancher UI. Now it is available in the **Cluster Explorer** and it can be enabled and deployed using a Helm chart. It can be installed from the Rancher UI, but it can also be installed independently of Rancher. It deploys a CIS scan operator for the cluster, and deploys Kubernetes custom resources for cluster scans. The custom resources can be managed directly from the **Cluster Explorer.**
|
||||
|
||||
In v1 of the CIS scan tool, which was available in Rancher v2.4 through the cluster manager, recurring scans could be scheduled. The ability to schedule recurring scans is not yet available in Rancher v2.5.
|
||||
|
||||
Support for alerting for the cluster scan results is not available for Rancher v2.5 yet.
|
||||
|
||||
More test profiles were added. In Rancher v2.4, permissive and hardened profiles were included. In Rancher v2.5, the following profiles are available:
|
||||
|
||||
- Generic CIS 1.5
|
||||
- RKE permissive
|
||||
- RKE hardened
|
||||
- EKS
|
||||
- GKE
|
||||
|
||||
The default profile depends on the type of cluster that will be scanned:
|
||||
|
||||
- For RKE Kubernetes clusters, the RKE permissive profile is the default.
|
||||
- EKS and GKE have their own CIS Benchmarks published by `kube-bench`. The corresponding test profiles are used by default for those clusters.
|
||||
- For cluster types other than RKE, EKS and GKE, the Generic CIS 1.5 profile will be used by default.
|
||||
|
||||
The `rancher-cis-benchmark` currently supports the CIS 1.5 Benchmark version.
|
||||
|
||||
> **Note:** CIS v1 cannot run on a cluster when CIS v2 is deployed. In other words, after `rancher-cis-benchmark` is installed, you can't run scans by going to the Cluster Manager view in the Rancher UI and clicking **Tools > CIS Scans.**
|
||||
|
||||
# About the CIS Benchmark
|
||||
|
||||
The Center for Internet Security is a 501(c)(3) nonprofit organization, formed in October 2000, with a mission is to "identify, develop, validate, promote, and sustain best practice solutions for cyber defense and build and lead communities to enable an environment of trust in cyberspace". The organization is headquartered in East Greenbush, New York, with members including large corporations, government agencies, and academic institutions.
|
||||
|
||||
CIS Benchmarks are best practices for the secure configuration of a target system. CIS Benchmarks are developed through the generous volunteer efforts of subject matter experts, technology vendors, public and private community members, and the CIS Benchmark Development team.
|
||||
|
||||
The official Benchmark documents are available through the CIS website. The sign-up form to access the documents is
|
||||
<a href="https://learn.cisecurity.org/benchmarks" target="_blank">here.</a>
|
||||
|
||||
# Installing rancher-cis-benchmark
|
||||
|
||||
The application can be installed with the Rancher UI or with Helm.
|
||||
|
||||
### Installing with the Rancher UI
|
||||
|
||||
1. In the Rancher UI, go to the **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click `rancher-cis-benchmark`.
|
||||
1. Click **Install.**
|
||||
|
||||
**Result:** The CIS scan application is deployed on the Kubernetes cluster.
|
||||
|
||||
### Installing with Helm
|
||||
|
||||
There are two Helm charts for the application:
|
||||
|
||||
- `rancher-cis-benchmark-crds`, the custom resource definition chart
|
||||
- `rancher-cis-benchmark`, the chart deploying <a href="https://github.com/rancher/cis-operator" target="_blank">rancher/cis-operator</a>
|
||||
|
||||
To install the charts, run the following commands:
|
||||
```
|
||||
helm repo add rancherchart https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install rancher-cis-benchmark-crd --kubeconfig <> rancherchart/rancher-cis-benchmark-crd --create-namespace -n cis-operator-system
|
||||
helm install rancher-cis-benchmark --kubeconfig <> rancherchart/rancher-cis-benchmark -n cis-operator-system
|
||||
```
|
||||
|
||||
# Uninstalling rancher-cis-benchmark
|
||||
|
||||
The application can be uninstalled with the Rancher UI or with Helm.
|
||||
|
||||
### Uninstalling with the Rancher UI
|
||||
|
||||
1. From the **Cluster Explorer,** go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
1. Click **Installed Apps.**
|
||||
1. Go to the `cis-operator-system` namespace and check the boxes next to `rancher-cis-benchmark-crd` and `rancher-cis-benchmark`.
|
||||
1. Click **Delete** and confirm **Delete.**
|
||||
|
||||
**Result:** The `rancher-cis-benchmark` application is uninstalled.
|
||||
|
||||
### Uninstalling with Helm
|
||||
|
||||
Run the following commands:
|
||||
|
||||
```
|
||||
helm uninstall rancher-cis-benchmark -n cis-operator-system
|
||||
helm uninstall rancher-cis-benchmark-crd -n cis-operator-system
|
||||
```
|
||||
|
||||
# Running a Scan
|
||||
|
||||
When a ClusterScan custom resource is created, it launches a new CIS scan on the cluster for the chosen ClusterScanProfile.
|
||||
|
||||
Note: There is currently a limitation of running only one CIS scan at a time for a cluster. If you create multiple ClusterScan custom resources, they will be run one after the other by the operator, and until one scan finishes, the rest of the ClusterScan custom resources will be in the "Pending" state.
|
||||
|
||||
To run a scan,
|
||||
|
||||
1. Go to the **Cluster Explorer** in the Rancher UI. In the top left dropdown menu, click **Cluster Explorer > CIS Benchmark.**
|
||||
1. In the **Scans** section, click **Create.**
|
||||
1. Choose a cluster scan profile. The profile determines which CIS Benchmark version will be used and which tests will be performed. If you choose the Default profile, then the CIS Operator will choose a profile applicable to the type of Kubernetes cluster it is installed on.
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** A report is generated with the scan results. To see the results, click the name of the scan that appears.
|
||||
|
||||
# Skipping Tests
|
||||
|
||||
CIS scans can be run using test profiles with user-defined skips.
|
||||
|
||||
To skip tests, you will create a custom CIS scan profile. A profile contains the configuration for the CIS scan, which includes the benchmark versions to use and any specific tests to skip in that benchmark.
|
||||
|
||||
1. In the **Cluster Explorer,** go to the top-left dropdown menu and click **CIS Benchmark.**
|
||||
1. Click **Profiles.**
|
||||
1. From here, you can create a profile in multiple ways. To make a new profile, click **Create** and fill out the form in the UI. To make a new profile based on an existing profile, go to the existing profile, click the three vertical dots, and click **Clone as YAML.** If you are filling out the form, add the tests to skip using the test IDs, using the relevant CIS Benchmark as a reference. If you are creating the new test profile as YAML, you will add the IDs of the tests to skip in the `skipTests` directive. You will also give the profile a name:
|
||||
|
||||
```yaml
|
||||
apiVersion: cis.cattle.io/v1
|
||||
kind: ClusterScanProfile
|
||||
metadata:
|
||||
annotations:
|
||||
meta.helm.sh/release-name: clusterscan-operator
|
||||
meta.helm.sh/release-namespace: cis-operator-system
|
||||
labels:
|
||||
app.kubernetes.io/managed-by: Helm
|
||||
name: "<example-profile>"
|
||||
spec:
|
||||
benchmarkVersion: cis-1.5
|
||||
skipTests:
|
||||
- "1.1.20"
|
||||
- "1.1.21"
|
||||
```
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** A new CIS scan profile is created.
|
||||
|
||||
When you [run a scan](#running-a-scan) that uses this profile, the defined tests will be skipped during the scan. The skipped tests will be marked in the generated report as `Skip`.
|
||||
|
||||
# Viewing Reports
|
||||
|
||||
To view the generated CIS scan reports,
|
||||
|
||||
1. In the **Cluster Explorer,** go to the top left dropdown menu and click **Cluster Explorer > CIS Benchmark.**
|
||||
1. The **Scans** page will show the generated reports. To see a detailed report, go to a scan report and click the name.
|
||||
|
||||
One can download the report from the Scans list or from the scan detail page.
|
||||
|
||||
# About the Generated Report
|
||||
|
||||
Each scan generates a report can be viewed in the Rancher UI and can be downloaded in CSV format.
|
||||
|
||||
In Rancher v2.5, the scan will use the CIS Benchmark v1.5. The Benchmark version is included in the generated report.
|
||||
|
||||
The Benchmark provides recommendations of two types: Scored and Not Scored. Recommendations marked as Not Scored in the Benchmark are not included in the generated report.
|
||||
|
||||
Some tests are designated as "Not Applicable." These tests will not be run on any CIS scan because of the way that Rancher provisions RKE clusters. For information on how test results can be audited, and why some tests are designated to be not applicable, refer to Rancher's <a href="{{<baseurl>}}/rancher/v2.x/en/security/#the-cis-benchmark-and-self-assessment" target="_blank">self-assessment guide for the corresponding Kubernetes version.</a>
|
||||
|
||||
The report contains the following information:
|
||||
|
||||
| Column in Report | Description |
|
||||
|------------------|-------------|
|
||||
| `id` | The ID number of the CIS Benchmark. |
|
||||
| `description` | The description of the CIS Benchmark test. |
|
||||
| `remediation` | What needs to be fixed in order to pass the test. |
|
||||
| `state` | Indicates if the test passed, failed, was skipped, or was not applicable. |
|
||||
| `node_type` | The node role, which affects which tests are run on the node. Master tests are run on controlplane nodes, etcd tests are run on etcd nodes, and node tests are run on the worker nodes. |
|
||||
| `audit` | This is the audit check that `kube-bench` runs for this test. |
|
||||
| `audit_config` | Any configuration applicable to the audit script. |
|
||||
| `test_info` | Test-related info as reported by `kube-bench`, if any. |
|
||||
| `commands` | Test-related commands as reported by `kube-bench`, if any. |
|
||||
| `config_commands` | Test-related configuration data as reported by `kube-bench`, if any. |
|
||||
| `actual_value` | The test's actual value, present if reported by `kube-bench`. |
|
||||
| `expected_result` | The test's expected result, present if reported by `kube-bench`. |
|
||||
|
||||
Refer to <a href="{{<baseurl>}}/rancher/v2.x/en/security/" target="_blank">the table in the cluster hardening guide</a> for information on which versions of Kubernetes, the Benchmark, Rancher, and our cluster hardening guide correspond to each other. Also refer to the hardening guide for configuration files of CIS-compliant clusters and information on remediating failed tests.
|
||||
|
||||
# Test Profiles
|
||||
|
||||
The following profiles are available:
|
||||
|
||||
- Generic CIS 1.5 (default)
|
||||
- RKE permissive
|
||||
- RKE hardened
|
||||
- EKS
|
||||
- GKE
|
||||
|
||||
You also have the ability to customize a profile by saving a set of tests to skip.
|
||||
|
||||
All profiles will have a set of not applicable tests that will be skipped during the CIS scan. These tests are not applicable based on how a RKE cluster manages Kubernetes.
|
||||
|
||||
There are 2 types of RKE cluster scan profiles:
|
||||
|
||||
- **Permissive:** This profile has a set of tests that have been will be skipped as these tests will fail on a default RKE Kubernetes cluster. Besides the list of skipped tests, the profile will also not run the not applicable tests.
|
||||
- **Hardened:** This profile will not skip any tests, except for the non-applicable tests.
|
||||
|
||||
The EKS and GKE cluster scan profiles are based on CIS Benchmark versions that are specific to those types of clusters.
|
||||
|
||||
In order to pass the "Hardened" profile, you will need to follow the steps on the <a href="{{<baseurl>}}/rancher/v2.x/en/security/#rancher-hardening-guide" target="_blank">hardening guide</a> and use the `cluster.yml` defined in the hardening guide to provision a hardened cluster.
|
||||
|
||||
# About Skipped and Not Applicable Tests
|
||||
|
||||
For a list of skipped and not applicable tests, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/skipped-tests" target="_blank">this page.</a>
|
||||
|
||||
For now, only user-defined skipped tests are marked as skipped in the generated report.
|
||||
|
||||
Any skipped tests that are defined as being skipped by one of the default profiles are marked as not applicable.
|
||||
|
||||
# Roles-based Access Control
|
||||
|
||||
For information about permissions, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/rbac" target="_blank">this page.</a>
|
||||
|
||||
# Configuration
|
||||
|
||||
For more information about configuring the custom resources for the scans, profiles, and benchmark versions, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/configuration" target="_blank">this page.</a>
|
||||
If you are using Rancher v2.4, refer to the CIS scan documentation [here.](./v2.4)
|
||||
+6
-3
@@ -1,8 +1,11 @@
|
||||
---
|
||||
title: Cluster Manager CIS Scan (Deprecated)
|
||||
shortTitle: Cluster Manager
|
||||
weight: 1
|
||||
title: CIS Scans in Rancher v2.4 (Deprecated)
|
||||
shortTitle: Rancher v2.4
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cis-scans/legacy
|
||||
---
|
||||
|
||||
_Available as of v2.4.0_
|
||||
|
||||
This section contains the legacy documentation for the CIS Scan tool that was released in Rancher v2.4, and was available under the **Tools** menu in the top navigation bar of the cluster manager.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Skipped and Not Applicable Tests
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cis-scans/legacy/skipped-tests
|
||||
---
|
||||
|
||||
This section lists the tests that are skipped in the permissive test profile for RKE.
|
||||
@@ -0,0 +1,234 @@
|
||||
---
|
||||
title: CIS Scans in Rancher v2.5
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
|
||||
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark.
|
||||
|
||||
The `rancher-cis-benchmark` app leverages <a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench,</a> an open-source tool from Aqua Security, to check clusters for CIS Kubernetes Benchmark compliance. Also, to generate a cluster-wide report, the application utilizes <a href="https://github.com/vmware-tanzu/sonobuoy" target="_blank">Sonobuoy</a> for report aggregation.
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [About the CIS Benchmark](#about-the-cis-benchmark)
|
||||
- [Installing rancher-cis-benchmark](#installing-rancher-cis-benchmark)
|
||||
- [Uninstalling rancher-cis-benchmark](#uninstalling-rancher-cis-benchmark)
|
||||
- [Running a Scan](#running-a-scan)
|
||||
- [Skipping Tests](#skipping-tests)
|
||||
- [Viewing Reports](#viewing-reports)
|
||||
- [About the generated report](#about-the-generated-report)
|
||||
- [Test Profiles](#test-profiles)
|
||||
- [About Skipped and Not Applicable Tests](#about-skipped-and-not-applicable-tests)
|
||||
- [Roles-based access control](./rbac)
|
||||
- [Configuration](./configuration)
|
||||
|
||||
### Changes in Rancher v2.5
|
||||
|
||||
We now support running CIS scans on any Kubernetes cluster, including hosted Kubernetes providers such as EKS, AKS, and GKE. Previously it was only supported to run CIS scans on RKE Kubernetes clusters.
|
||||
|
||||
In Rancher v2.4, the CIS scan tool was available from the **cluster manager** in the Rancher UI. Now it is available in the **Cluster Explorer** and it can be enabled and deployed using a Helm chart. It can be installed from the Rancher UI, but it can also be installed independently of Rancher. It deploys a CIS scan operator for the cluster, and deploys Kubernetes custom resources for cluster scans. The custom resources can be managed directly from the **Cluster Explorer.**
|
||||
|
||||
In v1 of the CIS scan tool, which was available in Rancher v2.4 through the cluster manager, recurring scans could be scheduled. The ability to schedule recurring scans is not yet available in Rancher v2.5.
|
||||
|
||||
Support for alerting for the cluster scan results is not available for Rancher v2.5 yet.
|
||||
|
||||
More test profiles were added. In Rancher v2.4, permissive and hardened profiles were included. In Rancher v2.5, the following profiles are available:
|
||||
|
||||
- Generic CIS 1.5
|
||||
- RKE permissive
|
||||
- RKE hardened
|
||||
- EKS
|
||||
- GKE
|
||||
|
||||
The default profile depends on the type of cluster that will be scanned:
|
||||
|
||||
- For RKE Kubernetes clusters, the RKE permissive profile is the default.
|
||||
- EKS and GKE have their own CIS Benchmarks published by `kube-bench`. The corresponding test profiles are used by default for those clusters.
|
||||
- For cluster types other than RKE, EKS and GKE, the Generic CIS 1.5 profile will be used by default.
|
||||
|
||||
The `rancher-cis-benchmark` currently supports the CIS 1.5 Benchmark version.
|
||||
|
||||
> **Note:** CIS v1 cannot run on a cluster when CIS v2 is deployed. In other words, after `rancher-cis-benchmark` is installed, you can't run scans by going to the Cluster Manager view in the Rancher UI and clicking **Tools > CIS Scans.**
|
||||
|
||||
# About the CIS Benchmark
|
||||
|
||||
The Center for Internet Security is a 501(c)(3) nonprofit organization, formed in October 2000, with a mission is to "identify, develop, validate, promote, and sustain best practice solutions for cyber defense and build and lead communities to enable an environment of trust in cyberspace". The organization is headquartered in East Greenbush, New York, with members including large corporations, government agencies, and academic institutions.
|
||||
|
||||
CIS Benchmarks are best practices for the secure configuration of a target system. CIS Benchmarks are developed through the generous volunteer efforts of subject matter experts, technology vendors, public and private community members, and the CIS Benchmark Development team.
|
||||
|
||||
The official Benchmark documents are available through the CIS website. The sign-up form to access the documents is
|
||||
<a href="https://learn.cisecurity.org/benchmarks" target="_blank">here.</a>
|
||||
|
||||
# Installing rancher-cis-benchmark
|
||||
|
||||
The application can be installed with the Rancher UI or with Helm.
|
||||
|
||||
### Installing with the Rancher UI
|
||||
|
||||
1. In the Rancher UI, go to the **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click `rancher-cis-benchmark`.
|
||||
1. Click **Install.**
|
||||
|
||||
**Result:** The CIS scan application is deployed on the Kubernetes cluster.
|
||||
|
||||
### Installing with Helm
|
||||
|
||||
There are two Helm charts for the application:
|
||||
|
||||
- `rancher-cis-benchmark-crds`, the custom resource definition chart
|
||||
- `rancher-cis-benchmark`, the chart deploying <a href="https://github.com/rancher/cis-operator" target="_blank">rancher/cis-operator</a>
|
||||
|
||||
To install the charts, run the following commands:
|
||||
```
|
||||
helm repo add rancherchart https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install rancher-cis-benchmark-crd --kubeconfig <> rancherchart/rancher-cis-benchmark-crd --create-namespace -n cis-operator-system
|
||||
helm install rancher-cis-benchmark --kubeconfig <> rancherchart/rancher-cis-benchmark -n cis-operator-system
|
||||
```
|
||||
|
||||
# Uninstalling rancher-cis-benchmark
|
||||
|
||||
The application can be uninstalled with the Rancher UI or with Helm.
|
||||
|
||||
### Uninstalling with the Rancher UI
|
||||
|
||||
1. From the **Cluster Explorer,** go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
1. Click **Installed Apps.**
|
||||
1. Go to the `cis-operator-system` namespace and check the boxes next to `rancher-cis-benchmark-crd` and `rancher-cis-benchmark`.
|
||||
1. Click **Delete** and confirm **Delete.**
|
||||
|
||||
**Result:** The `rancher-cis-benchmark` application is uninstalled.
|
||||
|
||||
### Uninstalling with Helm
|
||||
|
||||
Run the following commands:
|
||||
|
||||
```
|
||||
helm uninstall rancher-cis-benchmark -n cis-operator-system
|
||||
helm uninstall rancher-cis-benchmark-crd -n cis-operator-system
|
||||
```
|
||||
|
||||
# Running a Scan
|
||||
|
||||
When a ClusterScan custom resource is created, it launches a new CIS scan on the cluster for the chosen ClusterScanProfile.
|
||||
|
||||
Note: There is currently a limitation of running only one CIS scan at a time for a cluster. If you create multiple ClusterScan custom resources, they will be run one after the other by the operator, and until one scan finishes, the rest of the ClusterScan custom resources will be in the "Pending" state.
|
||||
|
||||
To run a scan,
|
||||
|
||||
1. Go to the **Cluster Explorer** in the Rancher UI. In the top left dropdown menu, click **Cluster Explorer > CIS Benchmark.**
|
||||
1. In the **Scans** section, click **Create.**
|
||||
1. Choose a cluster scan profile. The profile determines which CIS Benchmark version will be used and which tests will be performed. If you choose the Default profile, then the CIS Operator will choose a profile applicable to the type of Kubernetes cluster it is installed on.
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** A report is generated with the scan results. To see the results, click the name of the scan that appears.
|
||||
|
||||
# Skipping Tests
|
||||
|
||||
CIS scans can be run using test profiles with user-defined skips.
|
||||
|
||||
To skip tests, you will create a custom CIS scan profile. A profile contains the configuration for the CIS scan, which includes the benchmark versions to use and any specific tests to skip in that benchmark.
|
||||
|
||||
1. In the **Cluster Explorer,** go to the top-left dropdown menu and click **CIS Benchmark.**
|
||||
1. Click **Profiles.**
|
||||
1. From here, you can create a profile in multiple ways. To make a new profile, click **Create** and fill out the form in the UI. To make a new profile based on an existing profile, go to the existing profile, click the three vertical dots, and click **Clone as YAML.** If you are filling out the form, add the tests to skip using the test IDs, using the relevant CIS Benchmark as a reference. If you are creating the new test profile as YAML, you will add the IDs of the tests to skip in the `skipTests` directive. You will also give the profile a name:
|
||||
|
||||
```yaml
|
||||
apiVersion: cis.cattle.io/v1
|
||||
kind: ClusterScanProfile
|
||||
metadata:
|
||||
annotations:
|
||||
meta.helm.sh/release-name: clusterscan-operator
|
||||
meta.helm.sh/release-namespace: cis-operator-system
|
||||
labels:
|
||||
app.kubernetes.io/managed-by: Helm
|
||||
name: "<example-profile>"
|
||||
spec:
|
||||
benchmarkVersion: cis-1.5
|
||||
skipTests:
|
||||
- "1.1.20"
|
||||
- "1.1.21"
|
||||
```
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** A new CIS scan profile is created.
|
||||
|
||||
When you [run a scan](#running-a-scan) that uses this profile, the defined tests will be skipped during the scan. The skipped tests will be marked in the generated report as `Skip`.
|
||||
|
||||
# Viewing Reports
|
||||
|
||||
To view the generated CIS scan reports,
|
||||
|
||||
1. In the **Cluster Explorer,** go to the top left dropdown menu and click **Cluster Explorer > CIS Benchmark.**
|
||||
1. The **Scans** page will show the generated reports. To see a detailed report, go to a scan report and click the name.
|
||||
|
||||
One can download the report from the Scans list or from the scan detail page.
|
||||
|
||||
# About the Generated Report
|
||||
|
||||
Each scan generates a report can be viewed in the Rancher UI and can be downloaded in CSV format.
|
||||
|
||||
In Rancher v2.5, the scan will use the CIS Benchmark v1.5. The Benchmark version is included in the generated report.
|
||||
|
||||
The Benchmark provides recommendations of two types: Scored and Not Scored. Recommendations marked as Not Scored in the Benchmark are not included in the generated report.
|
||||
|
||||
Some tests are designated as "Not Applicable." These tests will not be run on any CIS scan because of the way that Rancher provisions RKE clusters. For information on how test results can be audited, and why some tests are designated to be not applicable, refer to Rancher's <a href="{{<baseurl>}}/rancher/v2.x/en/security/#the-cis-benchmark-and-self-assessment" target="_blank">self-assessment guide for the corresponding Kubernetes version.</a>
|
||||
|
||||
The report contains the following information:
|
||||
|
||||
| Column in Report | Description |
|
||||
|------------------|-------------|
|
||||
| `id` | The ID number of the CIS Benchmark. |
|
||||
| `description` | The description of the CIS Benchmark test. |
|
||||
| `remediation` | What needs to be fixed in order to pass the test. |
|
||||
| `state` | Indicates if the test passed, failed, was skipped, or was not applicable. |
|
||||
| `node_type` | The node role, which affects which tests are run on the node. Master tests are run on controlplane nodes, etcd tests are run on etcd nodes, and node tests are run on the worker nodes. |
|
||||
| `audit` | This is the audit check that `kube-bench` runs for this test. |
|
||||
| `audit_config` | Any configuration applicable to the audit script. |
|
||||
| `test_info` | Test-related info as reported by `kube-bench`, if any. |
|
||||
| `commands` | Test-related commands as reported by `kube-bench`, if any. |
|
||||
| `config_commands` | Test-related configuration data as reported by `kube-bench`, if any. |
|
||||
| `actual_value` | The test's actual value, present if reported by `kube-bench`. |
|
||||
| `expected_result` | The test's expected result, present if reported by `kube-bench`. |
|
||||
|
||||
Refer to <a href="{{<baseurl>}}/rancher/v2.x/en/security/" target="_blank">the table in the cluster hardening guide</a> for information on which versions of Kubernetes, the Benchmark, Rancher, and our cluster hardening guide correspond to each other. Also refer to the hardening guide for configuration files of CIS-compliant clusters and information on remediating failed tests.
|
||||
|
||||
# Test Profiles
|
||||
|
||||
The following profiles are available:
|
||||
|
||||
- Generic CIS 1.5 (default)
|
||||
- RKE permissive
|
||||
- RKE hardened
|
||||
- EKS
|
||||
- GKE
|
||||
|
||||
You also have the ability to customize a profile by saving a set of tests to skip.
|
||||
|
||||
All profiles will have a set of not applicable tests that will be skipped during the CIS scan. These tests are not applicable based on how a RKE cluster manages Kubernetes.
|
||||
|
||||
There are 2 types of RKE cluster scan profiles:
|
||||
|
||||
- **Permissive:** This profile has a set of tests that have been will be skipped as these tests will fail on a default RKE Kubernetes cluster. Besides the list of skipped tests, the profile will also not run the not applicable tests.
|
||||
- **Hardened:** This profile will not skip any tests, except for the non-applicable tests.
|
||||
|
||||
The EKS and GKE cluster scan profiles are based on CIS Benchmark versions that are specific to those types of clusters.
|
||||
|
||||
In order to pass the "Hardened" profile, you will need to follow the steps on the <a href="{{<baseurl>}}/rancher/v2.x/en/security/#rancher-hardening-guide" target="_blank">hardening guide</a> and use the `cluster.yml` defined in the hardening guide to provision a hardened cluster.
|
||||
|
||||
# About Skipped and Not Applicable Tests
|
||||
|
||||
For a list of skipped and not applicable tests, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/skipped-tests" target="_blank">this page.</a>
|
||||
|
||||
For now, only user-defined skipped tests are marked as skipped in the generated report.
|
||||
|
||||
Any skipped tests that are defined as being skipped by one of the default profiles are marked as not applicable.
|
||||
|
||||
# Roles-based Access Control
|
||||
|
||||
For information about permissions, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/rbac" target="_blank">this page.</a>
|
||||
|
||||
# Configuration
|
||||
|
||||
For more information about configuring the custom resources for the scans, profiles, and benchmark versions, refer to <a href="{{<baseurl>}}/rancher/v2.x/en/cis-scans/configuration" target="_blank">this page.</a>
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Configuration
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cis-scans/configuration
|
||||
---
|
||||
|
||||
This configuration reference is intended to help you manage the custom resources created by the `rancher-cis-benchmark` application. These resources are used for performing CIS scans on a cluster, skipping tests, setting the test profile that will be used during a scan, and other customization.
|
||||
+2
@@ -2,6 +2,8 @@
|
||||
title: Roles-based Access Control
|
||||
shortTitle: RBAC
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cis-scans/rbac
|
||||
---
|
||||
|
||||
This section describes the permissions required to use the rancher-cis-benchmark App.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Skipped and Not Applicable Tests
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cis-scans/skipped-tests
|
||||
---
|
||||
|
||||
This section lists the tests that are skipped in the permissive test profile for RKE.
|
||||
@@ -9,11 +9,11 @@ Rancher contains a variety of tools that aren't included in Kubernetes to assist
|
||||
|
||||
<!-- TOC -->
|
||||
|
||||
- [Notifiers and Alerts](#notifiers-and-alerts)
|
||||
- [Logging](#logging)
|
||||
- [Monitoring](#monitoring)
|
||||
- [Monitoring and Alerts](#monitoring-and-alerts)
|
||||
- [Istio](#istio)
|
||||
- [OPA Gatekeeper](#opa-gatekeeper)
|
||||
- [CIS Scans](#cis-scans)
|
||||
|
||||
<!-- /TOC -->
|
||||
|
||||
@@ -30,13 +30,14 @@ Logging is helpful because it allows you to:
|
||||
|
||||
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
|
||||
|
||||
For details, refer to the [logging section.]({{<baseurl>}}/rancher/v2.x/en/logging)
|
||||
Logging was improved in Rancher v2.5.
|
||||
|
||||
# Monitoring
|
||||
- If you are using Rancher v2.5, refer to the logging documentation [here.]({{<baseurl>}}/rancher/v2.x/en/logging/v2.5)
|
||||
- If you are using Rancher v2.0.x-v2.4.x, refer to the logging documentation [here.]({{<baseurl>}}/rancher/v2.x/en/logging/v2.0.x-v2.4.x)
|
||||
|
||||
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.]({{<baseurl>}}/rancher/v2.x/en/monitoring)
|
||||
# Monitoring and Alerts
|
||||
|
||||
### Notifiers and Alerts
|
||||
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
|
||||
|
||||
After monitoring is enabled, you can set up alerts and notifiers that provide the mechanism to receive them.
|
||||
|
||||
@@ -44,10 +45,29 @@ Notifiers are services that inform you of alert events. You can configure notifi
|
||||
|
||||
Alerts are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
|
||||
|
||||
- If you are using Rancher v2.5, refer to the monitoring documentation [here.]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting/v2.5)
|
||||
- If you are using Rancher v2.0.x-v2.4.x, refer to the monitoring documentation [here.]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x)
|
||||
|
||||
# Istio
|
||||
|
||||
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices. For details on how to enable Istio in Rancher, refer to the [Istio section.]({{<baseurl>}}/rancher/v2.x/en/istio)
|
||||
_Available as of v2.3_
|
||||
|
||||
## OPA Gatekeeper
|
||||
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices.
|
||||
|
||||
[OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is an open-source project that provides integration between OPA and Kubernetes to provide policy control via admission controller webhooks. For details on how to enable Gatekeeper in Rancher, refer to the [OPA Gatekeeper section.]({{<baseurl>}}/rancher/v2.x/en/opa-gatekeper)
|
||||
Rancher's integration with Istio was improved in Rancher v2.5.
|
||||
|
||||
- If you are using Rancher v2.5, refer to the Istio documentation [here.]({{<baseurl>}}/rancher/v2.x/en/istio/v2.5)
|
||||
- If you are using Rancher v2.3.x-v2.4.x, refer to the Istio documentation [here.]({{<baseurl>}}/rancher/v2.x/en/istio/v2.3.x-v2.4.x)
|
||||
|
||||
# OPA Gatekeeper
|
||||
|
||||
[OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is an open-source project that provides integration between OPA and Kubernetes to provide policy control via admission controller webhooks. For details on how to enable Gatekeeper in Rancher, refer to the [OPA Gatekeeper section.]({{<baseurl>}}/rancher/v2.x/en/opa-gatekeper)
|
||||
|
||||
|
||||
# CIS Scans
|
||||
|
||||
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark.
|
||||
|
||||
If you are using Rancher v2.5, refer to the CIS scan documentation [here.](./v2.5)
|
||||
|
||||
If you are using Rancher v2.4, refer to the CIS scan documentation [here.](./v2.4)
|
||||
@@ -3,14 +3,16 @@ title: Deploying Applications across Clusters
|
||||
weight: 13
|
||||
---
|
||||
|
||||
Rancher v2.5 introduced Fleet, a new way to deploy applications across clusters.
|
||||
|
||||
|
||||
### Fleet
|
||||
|
||||
_Available in v2.5_
|
||||
|
||||
Rancher v2.5 introduced Fleet, a new way to deploy applications across clusters.
|
||||
|
||||
Fleet is GitOps at scale. For more information, refer to the [Fleet section.](./fleet)
|
||||
|
||||
### Cluster Manager UI Documentation for Multi-cluster Apps
|
||||
### Multi-cluster Apps
|
||||
|
||||
In Rancher prior to v2.5, the multi-cluster apps feature was used to deploy applications across clusters. Refer to the documentation [here.](./multi-cluster-apps)
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Fleet - GitOps at Scale
|
||||
shortTitle: Fleet
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
---
|
||||
title: Cluster Manager Multi-Cluster App Documentation
|
||||
shortTitle: Cluster Manager
|
||||
title: Multi-cluster Apps
|
||||
shortTitle: Rancher v2.2-v2.4
|
||||
weight: 2
|
||||
---
|
||||
|
||||
_Available as of v2.2.0_
|
||||
|
||||
> As of Rancher v2.5, we recommend using [Fleet]({{<baseurl>}}/rancher/v2.x/en/deploy-across-clusters/fleet) for deploying apps across clusters.
|
||||
|
||||
Typically, most applications are deployed on a single Kubernetes cluster, but there will be times you might want to deploy multiple copies of the same application across different clusters and/or projects. In Rancher, a _multi-cluster application_, is an application deployed using a Helm chart across multiple clusters. With the ability to deploy the same application across multiple clusters, it avoids the repetition of the same action on each cluster, which could introduce user error during application configuration. With multi-cluster applications, you can customize to have the same configuration across all projects/clusters as well as have the ability to change the configuration based on your target project. Since multi-cluster application is considered a single application, it's easy to manage and maintain this application.
|
||||
|
||||
Any Helm charts from a global catalog can be used to deploy and manage multi-cluster applications.
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
---
|
||||
title: Apps and Marketplace
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Cluster Manager Catalog Documentation
|
||||
shortTitle: Cluster Manager
|
||||
title: Rancher v2.0-v2.4 Catalogs (Deprecated)
|
||||
shortTitle: Rancher v2.0-v2.4
|
||||
description: Rancher enables the use of catalogs to repeatedly deploy applications easily. Catalogs are GitHub or Helm Chart repositories filled with deployment-ready apps.
|
||||
weight: 1
|
||||
aliases:
|
||||
@@ -11,6 +11,8 @@ aliases:
|
||||
- /rancher/v2.x/en/catalog/apps
|
||||
---
|
||||
|
||||
> As of Rancher v2.5, the catalog system is deprecated and has been replaced with [Apps and Marketplace]({{<baseurl>}}/rancher/v2.x/en/helm-charts/apps-marketplace) in the Cluster Explorer.
|
||||
|
||||
Rancher provides the ability to use a catalog of Helm charts that make it easy to repeatedly deploy applications.
|
||||
|
||||
- **Catalogs** are GitHub repositories or Helm Chart repositories filled with applications that are ready-made for deployment. Applications are bundled in objects called _Helm charts_.
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ Provision three Linux hosts according to our [Requirements]({{<baseurl>}}/ranche
|
||||
|
||||
## 2. Configure Load Balancer
|
||||
|
||||
We will be using NGINX as our Layer 4 Load Balancer (TCP). NGINX will forward all connections to one of your Rancher nodes. If you want to use Amazon NLB, you can skip this step and use [Amazon NLB configuration]({{<baseurl>}}/rancher/v2.x/en/installation/k8s-install-server-install/nlb/)
|
||||
We will be using NGINX as our Layer 4 Load Balancer (TCP). NGINX will forward all connections to one of your Rancher nodes. If you want to use Amazon NLB, you can skip this step and use [Amazon NLB configuration](./nlb)
|
||||
|
||||
>**Note:**
|
||||
> In this configuration, the load balancer is positioned in front of your Linux hosts. The load balancer can be any host that you have available that's capable of running NGINX.
|
||||
|
||||
+2
-2
@@ -27,8 +27,8 @@ NO_PROXY | Network address(es), network address range(s) and do
|
||||
|
||||
When using Kubernetes installation, the environment variables need to be added to the RKE Config File template.
|
||||
|
||||
* [Kubernetes Installation with External Load Balancer (TCP/Layer 4) RKE Config File Template]({{<baseurl>}}/rancher/v2.x/en/installation/k8s-install-server-install/#5-download-rke-config-file-template)
|
||||
* [Kubernetes Installation with External Load Balancer (HTTPS/Layer 7) RKE Config File Template]({{<baseurl>}}/rancher/v2.x/en/installation/k8s-install-server-install-external-lb/#5-download-rke-config-file-template)
|
||||
* [Kubernetes Installation with External Load Balancer (TCP/Layer 4) RKE Config File Template]({{<baseurl>}}/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/#5-download-rke-config-file-template)
|
||||
* [Kubernetes Installation with External Load Balancer (HTTPS/Layer 7) RKE Config File Template]({{<baseurl>}}/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/#5-download-rke-config-file-template)
|
||||
|
||||
The environment variables should be defined in the `Deployment` inside the RKE Config File Template. You only have to add the part starting with `env:` to (but not including) `ports:`. Make sure the indentation is identical to the preceding `name:`. Required values for `NO_PROXY` are:
|
||||
|
||||
|
||||
+1
-1
@@ -52,7 +52,7 @@ Provision three Linux hosts according to our [Requirements]({{<baseurl>}}/ranche
|
||||
|
||||
## 2. Configure Load Balancer
|
||||
|
||||
We will be using NGINX as our Layer 4 Load Balancer (TCP). NGINX will forward all connections to one of your Rancher nodes. If you want to use Amazon NLB, you can skip this step and use [Amazon NLB configuration]({{<baseurl>}}/rancher/v2.x/en/installation/options/rke-add-on/layer-4-lb/nlb/)
|
||||
We will be using NGINX as our Layer 4 Load Balancer (TCP). NGINX will forward all connections to one of your Rancher nodes. If you want to use Amazon NLB, you can skip this step and use [Amazon NLB configuration]({{<baseurl>}}/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/nlb)
|
||||
|
||||
>**Note:**
|
||||
> In this configuration, the load balancer is positioned in front of your Linux hosts. The load balancer can be any host that you have available that's capable of running NGINX.
|
||||
|
||||
@@ -5,91 +5,9 @@ aliases:
|
||||
- /rancher/v2.x/en/dashboard/istio
|
||||
---
|
||||
|
||||
# Istio in Cluster Manager
|
||||
If you are using a Rancher version from **v2.3.x** to **v2.4.x**, the older way of setting up Istio in th **Cluster Manager** is documented in [this section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/)
|
||||
In Rancher 2.5, the Istio application was improved.
|
||||
|
||||
|
||||
# Istio in Cluster Explorer
|
||||
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, secure, control, and troubleshoot the traffic within a complex network of microservices.
|
||||
If you are using Rancher v2.5, refer to the Istio documentation [here.]({{<baseurl>}}/rancher/v2.x/en/istio/v2.5)
|
||||
|
||||
As a network of microservices changes and grows, the interactions between them can become increasingly difficult to manage and understand. In such a situation, it is useful to have a service mesh as a separate infrastructure layer. Istio's service mesh lets you manipulate traffic between microservices without changing the microservices directly.
|
||||
|
||||
Our integration of Istio is designed so that a Rancher operator, such as an administrator or cluster owner, can deliver Istio to a team of developers. Then developers can use Istio to enforce security policies, troubleshoot problems, or manage traffic for green/blue deployments, canary deployments, or A/B testing.
|
||||
|
||||
This core service mesh provides features that include but are not limited to the following:
|
||||
|
||||
- **Traffic Management** such as ingress and egress routing, circuit breaking, mirroring.
|
||||
- **Security** with resources to authenticate and authorize traffic and users, mTLS included.
|
||||
- **Observability** of logs, metrics, and distributed traffic flows.
|
||||
|
||||
After [setting up istio]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup) you can leverage Istio's control plane functionality through the Cluster Explorer, `kubectl`, or `istioctl`.
|
||||
|
||||
Rancher's Istio integration comes with a comprehensive visualization aid:
|
||||
|
||||
- **Get the full picture of your microservice architecture with Kiali.** [Kiali](https://www.kiali.io/) provides a diagram that shows the services within a service mesh and how they are connected, including the traffic rates and latencies between them. You can check the health of the service mesh, or drill down to see the incoming and outgoing requests to a single component.
|
||||
|
||||
Istio needs to be set up by a `cluster-admin` before it can be used in a project.
|
||||
|
||||
# What's New in Rancher v2.5
|
||||
|
||||
The overall architecture of Istio has been simplified. A single component, Istiod, has been created by combining Pilot, Citadel, Galley and the sidecar injector. Node Agent functionality has also been merged into istio-agent.
|
||||
|
||||
Addons that were previously installed by Istio (cert-manager, Grafana, Jaeger, Kiali, Prometheus, Zipkin) will now need to be installed separately. Istio will support installation of integrations that are from the Istio Project and will maintain compatibility with those that are not.
|
||||
|
||||
A Prometheus integration will still be available through an installation of [Rancher Monitoring]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting/), or by installing your own Prometheus operator. Rancher's Istio chart will also install Kiali by default to ensure you can get a full picture of your microservices out of the box.
|
||||
|
||||
Istio has migrated away from Helm as a way to install Istio and now provides installation through the istioctl binary or Istio Operator. To ensure the easiest interaction with Istio, Rancher's Istio will maintain a Helm chart that utilizes the istioctl binary to manage your Istio installation.
|
||||
|
||||
This Helm chart will be available via the Apps and Marketplace in the UI. A user that has access to the Rancher Chart's catalog will need to set up Istio before it can be used in the project.
|
||||
|
||||
# Prerequisites
|
||||
|
||||
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough [CPU and memory]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources) to run all of the components of Istio.
|
||||
|
||||
# Setup Guide
|
||||
|
||||
Refer to the [setup guide]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup) for instructions on how to set up Istio and use it in a project.
|
||||
|
||||
# Remove Istio
|
||||
|
||||
To remove Istio components from a cluster, namespace, or workload, refer to the section on [uninstalling Istio.]({{<baseurl>}}/rancher/v2.x/en/istio/disabling-istio/)
|
||||
|
||||
# Migrate From Previous Istio Version
|
||||
|
||||
There is no upgrade path for Istio versions less than 1.7
|
||||
|
||||
# Accessing Visualizations
|
||||
|
||||
> By default, only cluster-admins have access to Kiali. For instructions on how to allow admin, edit or views roles to access them, refer to [Access to Visualizations.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/rbac/#access-to-visualizations)
|
||||
|
||||
After Istio is set up in a cluster, Grafana, Prometheus,and Kiali are available in the Rancher UI.
|
||||
|
||||
To access the Grafana and Prometheus visualizations, from the **Cluster Explorer** navigate to the **Monitoring** app overview page, and click on **Grafana** or **Prometheus**
|
||||
|
||||
To access the Kiali visualization, from the **Cluster Explorer** navigate to the **Istio** app overview page, and click on **Kiali**. From here you can access the **Traffic Graph** tab or the **Traffic Metrics** tab to see network visualizations and metrics.
|
||||
|
||||
By default, all namespace will picked up by prometheus and make data available for Kiali graphs. Refer to [selector/scrape config setup]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#selectors-scrape-configs) if you would like to use a different configuration for prometheus data scraping.
|
||||
|
||||
Your access to the visualizations depend on your role. Grafana and Prometheus are only available for `cluster-admin` roles. The Kiali UI is available only to `cluster-admin` by default, but `cluster-admin` can allow other roles to access them by editing the Istio values.yaml.
|
||||
|
||||
# Architecture
|
||||
|
||||
Istio installs a service mesh that uses [Envoy](https://www.envoyproxy.io/learn/service-mesh) sidecar proxies to intercept traffic to each workload. These sidecars intercept and manage service-to-service communication, allowing fine-grained observation and control over traffic within the cluster.
|
||||
|
||||
Only workloads that have the Istio sidecar injected can be tracked and controlled by Istio.
|
||||
|
||||
When a namespace has Istio enabled, new workloads deployed in the namespace will automatically have the Istio sidecar. You need to manually enable Istio in preexisting workloads.
|
||||
|
||||
For more information on the Istio sidecar, refer to the [Istio sidecare-injection docs](https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/) and for more information on Istio's architecture, refer to the [Istio Architecture docs](https://istio.io/latest/docs/ops/deployment/architecture/)
|
||||
|
||||
### Multiple Ingresses
|
||||
|
||||
By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster. Istio also installs an ingress gateway by default into the `istio-system` namespace. The result is that your cluster will have two ingresses in your cluster.
|
||||
|
||||
|
||||
|
||||
Additional Istio Ingress gateways can be enabled via the [overlay file]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#overlay-file).
|
||||
|
||||
### Egress Support
|
||||
|
||||
By default the Egress gateway is disabled, but can be enabled on install or upgrade through the values.yaml or via the [overlay file]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#overlay-file).
|
||||
If you are using Rancher v2.3-v2.4, refer to the Istio documentation [here.]({{<baseurl>}}/rancher/v2.x/en/istio/v2.3.x-v2.4.x)
|
||||
+4
-4
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: Cluster Manager Istio Documentation
|
||||
shortTitle: Cluster Manager
|
||||
weight: 1
|
||||
title: Istio in Rancher v2.3-v2.4
|
||||
shortTitle: Rancher v2.3-v2.4
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/project-admin/istio/configuring-resource-allocations/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/
|
||||
@@ -10,7 +10,7 @@ aliases:
|
||||
---
|
||||
_Available as of v2.3.0_
|
||||
|
||||
> In Rancher 2.5, the Istio application was improved. There are now two ways to enable Istio. The older way is documented in this section, and the new application for Istio is documented [here.]({{<baseurl>}}/rancher/v2.x/en/istio)
|
||||
> In Rancher 2.5, the Istio application was improved. There are now two ways to enable Istio. The older way is documented in this section, and the new application for Istio is documented [here.]({{<baseurl>}}/rancher/v2.x/en/istio/v2.5)
|
||||
|
||||
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices.
|
||||
|
||||
+1
@@ -3,6 +3,7 @@ title: Disabling Istio
|
||||
weight: 4
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/disabling-istio
|
||||
- /rancher/v2.x/en/istio/legacy/disabling-istio
|
||||
---
|
||||
|
||||
This section describes how to disable Istio in a cluster, namespace, or workload.
|
||||
+1
@@ -3,6 +3,7 @@ title: Role-based Access Control
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/rbac
|
||||
- /rancher/v2.x/en/istio/legacy/rbac
|
||||
---
|
||||
|
||||
This section describes the permissions required to access Istio features and how to configure access to the Kiali and Jaeger visualizations.
|
||||
+1
@@ -2,6 +2,7 @@
|
||||
title: Release Notes
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/release-notes
|
||||
- /rancher/v2.x/en/istio/legacy/release-notes
|
||||
---
|
||||
|
||||
|
||||
+1
@@ -5,6 +5,7 @@ aliases:
|
||||
- /rancher/v2.x/en/project-admin/istio/configuring-resource-allocations/
|
||||
- /rancher/v2.x/en/project-admin/istio/config/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/resources
|
||||
- /rancher/v2.x/en/istio/legacy/resources
|
||||
---
|
||||
_Available as of v2.3.0_
|
||||
|
||||
+1
@@ -3,6 +3,7 @@ title: Setup Guide
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup
|
||||
- /rancher/v2.x/en/istio/legacy/setup
|
||||
---
|
||||
|
||||
This section describes how to enable Istio and start using it in your projects.
|
||||
+1
@@ -3,6 +3,7 @@ title: 4. Add Deployments and Services with the Istio Sidecar
|
||||
weight: 4
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads
|
||||
- /rancher/v2.x/en/istio/legacy/setup/deploy-workloads
|
||||
---
|
||||
|
||||
> **Prerequisite:** To enable Istio for a workload, the cluster and namespace must have Istio enabled.
|
||||
+1
@@ -3,6 +3,7 @@ title: 1. Enable Istio in the Cluster
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster
|
||||
- /rancher/v2.x/en/istio/legacy/setup/enable-istio-in-cluster
|
||||
---
|
||||
|
||||
This cluster uses the default Nginx controller to allow traffic into the cluster.
|
||||
+1
@@ -2,6 +2,7 @@
|
||||
title: Enable Istio with Pod Security Policies
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/enable-istio-with-psp
|
||||
- /rancher/v2.x/en/istio/legacy/setup/enable-istio-in-cluster/enable-istio-with-psp
|
||||
---
|
||||
|
||||
>**Note:** The following guide is only for RKE provisioned clusters.
|
||||
+1
@@ -3,6 +3,7 @@ title: 2. Enable Istio in a Namespace
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace
|
||||
- /rancher/v2.x/en/istio/legacy/setup/enable-istio-in-namespace
|
||||
---
|
||||
|
||||
You will need to manually enable Istio in each namespace that you want to be tracked or controlled by Istio. When Istio is enabled in a namespace, the Envoy sidecar proxy will be automatically injected into all new workloads that are deployed in the namespace.
|
||||
+1
@@ -3,6 +3,7 @@ title: 5. Set up the Istio Gateway
|
||||
weight: 5
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway
|
||||
- /rancher/v2.x/en/istio/legacy/setup/gateway
|
||||
---
|
||||
|
||||
The gateway to each cluster can have its own port or load balancer, which is unrelated to a service mesh. By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster.
|
||||
+1
@@ -3,6 +3,7 @@ title: 3. Select the Nodes Where Istio Components Will be Deployed
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/node-selectors
|
||||
- /rancher/v2.x/en/istio/legacy/setup/node-selectors
|
||||
---
|
||||
|
||||
> **Prerequisite:** Your cluster needs a worker node that can designated for Istio. The worker node should meet the [resource requirements.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources)
|
||||
+1
@@ -3,6 +3,7 @@ title: 6. Set up Istio's Components for Traffic Management
|
||||
weight: 6
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/set-up-traffic-management
|
||||
- /rancher/v2.x/en/istio/legacy/setup/set-up-traffic-management
|
||||
---
|
||||
|
||||
A central advantage of traffic management in Istio is that it allows dynamic request routing. Some common applications for dynamic request routing include canary deployments and blue/green deployments. The two key resources in Istio traffic management are *virtual services* and *destination rules*.
|
||||
+1
@@ -3,6 +3,7 @@ title: 7. Generate and View Traffic
|
||||
weight: 7
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/view-traffic
|
||||
- /rancher/v2.x/en/istio/legacy/setup/view-traffic
|
||||
---
|
||||
|
||||
This section describes how to view the traffic that is being managed by Istio.
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
title: Istio in Rancher v2.5
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, secure, control, and troubleshoot the traffic within a complex network of microservices.
|
||||
|
||||
As a network of microservices changes and grows, the interactions between them can become increasingly difficult to manage and understand. In such a situation, it is useful to have a service mesh as a separate infrastructure layer. Istio's service mesh lets you manipulate traffic between microservices without changing the microservices directly.
|
||||
|
||||
Our integration of Istio is designed so that a Rancher operator, such as an administrator or cluster owner, can deliver Istio to a team of developers. Then developers can use Istio to enforce security policies, troubleshoot problems, or manage traffic for green/blue deployments, canary deployments, or A/B testing.
|
||||
|
||||
This core service mesh provides features that include but are not limited to the following:
|
||||
|
||||
- **Traffic Management** such as ingress and egress routing, circuit breaking, mirroring.
|
||||
- **Security** with resources to authenticate and authorize traffic and users, mTLS included.
|
||||
- **Observability** of logs, metrics, and distributed traffic flows.
|
||||
|
||||
After [setting up istio]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup) you can leverage Istio's control plane functionality through the Cluster Explorer, `kubectl`, or `istioctl`.
|
||||
|
||||
Rancher's Istio integration comes with a comprehensive visualization aid:
|
||||
|
||||
- **Get the full picture of your microservice architecture with Kiali.** [Kiali](https://www.kiali.io/) provides a diagram that shows the services within a service mesh and how they are connected, including the traffic rates and latencies between them. You can check the health of the service mesh, or drill down to see the incoming and outgoing requests to a single component.
|
||||
|
||||
Istio needs to be set up by a `cluster-admin` before it can be used in a project.
|
||||
|
||||
- [What's New in Rancher v2.5](#what-s-new-in-rancher-v2-5)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Setup Guide](#setup-guide)
|
||||
- [Remove Istio](#remove-istio)
|
||||
- [Migrate from Previous Istio Version](#migrate-from-previous-istio-version)
|
||||
- [Accessing Visualizations](#accessing-visualizations)
|
||||
- [Architecture](#architecture)
|
||||
|
||||
# What's New in Rancher v2.5
|
||||
|
||||
The overall architecture of Istio has been simplified. A single component, Istiod, has been created by combining Pilot, Citadel, Galley and the sidecar injector. Node Agent functionality has also been merged into istio-agent.
|
||||
|
||||
Addons that were previously installed by Istio (cert-manager, Grafana, Jaeger, Kiali, Prometheus, Zipkin) will now need to be installed separately. Istio will support installation of integrations that are from the Istio Project and will maintain compatibility with those that are not.
|
||||
|
||||
A Prometheus integration will still be available through an installation of [Rancher Monitoring]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting/), or by installing your own Prometheus operator. Rancher's Istio chart will also install Kiali by default to ensure you can get a full picture of your microservices out of the box.
|
||||
|
||||
Istio has migrated away from Helm as a way to install Istio and now provides installation through the istioctl binary or Istio Operator. To ensure the easiest interaction with Istio, Rancher's Istio will maintain a Helm chart that utilizes the istioctl binary to manage your Istio installation.
|
||||
|
||||
This Helm chart will be available via the Apps and Marketplace in the UI. A user that has access to the Rancher Chart's catalog will need to set up Istio before it can be used in the project.
|
||||
|
||||
# Prerequisites
|
||||
|
||||
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough [CPU and memory]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources) to run all of the components of Istio.
|
||||
|
||||
# Setup Guide
|
||||
|
||||
Refer to the [setup guide]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup) for instructions on how to set up Istio and use it in a project.
|
||||
|
||||
# Remove Istio
|
||||
|
||||
To remove Istio components from a cluster, namespace, or workload, refer to the section on [uninstalling Istio.]({{<baseurl>}}/rancher/v2.x/en/istio/disabling-istio/)
|
||||
|
||||
# Migrate From Previous Istio Version
|
||||
|
||||
There is no upgrade path for Istio versions less than 1.7.x. To successfully install Istio in the **Cluster Explorer**, you will need to disable your existing Istio in the **Cluster Manager**.
|
||||
|
||||
If you have a significant amount of additional Istio CRDs you might consider manually migrating CRDs that are supported in both versions of Istio. You can do this by running `kubectl get <resource> -n istio-system -o yaml`, save the output yaml and re-apply in the new version.
|
||||
|
||||
Another option is to manually uninstall istio resources one at a time, but leave the resources that are supported in both versions of Istio and that will not be installed by the newest version. This method is more likely to result in issues installing the new version, but could be a good option depending on your situation.
|
||||
|
||||
# Accessing Visualizations
|
||||
|
||||
> By default, only cluster-admins have access to Kiali. For instructions on how to allow admin, edit or views roles to access them, refer to [Access to Visualizations.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/rbac/#access-to-visualizations)
|
||||
|
||||
After Istio is set up in a cluster, Grafana, Prometheus,and Kiali are available in the Rancher UI.
|
||||
|
||||
To access the Grafana and Prometheus visualizations, from the **Cluster Explorer** navigate to the **Monitoring** app overview page, and click on **Grafana** or **Prometheus**
|
||||
|
||||
To access the Kiali visualization, from the **Cluster Explorer** navigate to the **Istio** app overview page, and click on **Kiali**. From here you can access the **Traffic Graph** tab or the **Traffic Metrics** tab to see network visualizations and metrics.
|
||||
|
||||
By default, all namespace will picked up by prometheus and make data available for Kiali graphs. Refer to [selector/scrape config setup]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#selectors-scrape-configs) if you would like to use a different configuration for prometheus data scraping.
|
||||
|
||||
Your access to the visualizations depend on your role. Grafana and Prometheus are only available for `cluster-admin` roles. The Kiali UI is available only to `cluster-admin` by default, but `cluster-admin` can allow other roles to access them by editing the Istio values.yaml.
|
||||
|
||||
# Architecture
|
||||
|
||||
Istio installs a service mesh that uses [Envoy](https://www.envoyproxy.io/learn/service-mesh) sidecar proxies to intercept traffic to each workload. These sidecars intercept and manage service-to-service communication, allowing fine-grained observation and control over traffic within the cluster.
|
||||
|
||||
Only workloads that have the Istio sidecar injected can be tracked and controlled by Istio.
|
||||
|
||||
When a namespace has Istio enabled, new workloads deployed in the namespace will automatically have the Istio sidecar. You need to manually enable Istio in preexisting workloads.
|
||||
|
||||
For more information on the Istio sidecar, refer to the [Istio sidecare-injection docs](https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/) and for more information on Istio's architecture, refer to the [Istio Architecture docs](https://istio.io/latest/docs/ops/deployment/architecture/)
|
||||
|
||||
### Multiple Ingresses
|
||||
|
||||
By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster. Istio also installs an ingress gateway by default into the `istio-system` namespace. The result is that your cluster will have two ingresses in your cluster.
|
||||
|
||||
|
||||
|
||||
Additional Istio Ingress gateways can be enabled via the [overlay file]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#overlay-file).
|
||||
|
||||
### Egress Support
|
||||
|
||||
By default the Egress gateway is disabled, but can be enabled on install or upgrade through the values.yaml or via the [overlay file]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#overlay-file).
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
---
|
||||
title: Disabling Istio
|
||||
weight: 4
|
||||
|
||||
aliases:
|
||||
- /rancher/v2.x/en/istio/disabling-istio
|
||||
---
|
||||
|
||||
This section describes how to uninstall Istio in a cluster or disable a namespace, or workload.
|
||||
+2
-1
@@ -3,6 +3,7 @@ title: Role-based Access Control
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/rbac
|
||||
- /rancher/v2.x/en/istio/rbac
|
||||
---
|
||||
|
||||
This section describes the permissions required to access Istio features.
|
||||
@@ -36,7 +37,7 @@ ClusterRole create by chart | Default K8s ClusterRole | Rancher Role |
|
||||
`istio-edit`| edit | Project Member |
|
||||
`istio-view` | view | Read-only |
|
||||
|
||||
Rancher will continue to use cluster-owner, cluster-member, project-owner, project-member, etc as role names, but will utilize default roles to determine access. For each default K8s `ClusterRole` there are different Istio CRD permissions and K8s actions (Create ( C ), Get ( G ), List ( L ), Update ( U ), Patch ( P ), Delete( D ), All ( * )) that can be performed.
|
||||
Rancher will continue to use cluster-owner, cluster-member, project-owner, project-member, etc as role names, but will utilize default roles to determine access. For each default K8s `ClusterRole` there are different Istio CRD permissions and K8s actions (Create ( C ), Get ( G ), List ( L ), Watch ( W ), Update ( U ), Patch ( P ), Delete( D ), All ( * )) that can be performed.
|
||||
|
||||
|
||||
|CRDs | Admin | Edit | View
|
||||
+2
-1
@@ -2,9 +2,10 @@
|
||||
title: Release Notes
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/release-notes
|
||||
- /rancher/v2.x/en/istio/release-notes
|
||||
---
|
||||
|
||||
## Istio 1.5.9 release notes
|
||||
# Istio 1.5.9 release notes
|
||||
|
||||
**Bug fixes**
|
||||
|
||||
+1
@@ -5,6 +5,7 @@ aliases:
|
||||
- /rancher/v2.x/en/project-admin/istio/configuring-resource-allocations/
|
||||
- /rancher/v2.x/en/project-admin/istio/config/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/resources
|
||||
- /rancher/v2.x/en/istio/resources
|
||||
---
|
||||
_This section applies to Istio in Rancher v2.5.0. If you are using Rancher v2.4.x, refer to [this section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/)_
|
||||
|
||||
+3
-2
@@ -3,6 +3,7 @@ title: 4. Add Deployments and Services with the Istio Sidecar
|
||||
weight: 4
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads
|
||||
- /rancher/v2.x/en/istio/setup/deploy-workloads
|
||||
---
|
||||
|
||||
> **Prerequisite:** To enable Istio for a workload, the cluster and namespace must have the Istio app installed.
|
||||
@@ -13,7 +14,7 @@ To inject the Istio sidecar on an existing workload in the namespace, from the *
|
||||
|
||||
Wait a few minutes for the workload to upgrade to have the istio sidecar. Click it and go to the Containers section. You should be able to see `istio-proxy` alongside your original workload. This means the Istio sidecar is enabled for the workload. Istio is doing all the wiring for the sidecar envoy. Now Istio can do all the features automatically if you enable them in the yaml.
|
||||
|
||||
### 3. Add Deployments and Services
|
||||
### Add Deployments and Services
|
||||
|
||||
There are a few ways to add new **Deployments** in your namespace
|
||||
|
||||
@@ -38,7 +39,7 @@ You can also create deployments and services using the kubectl **shell**
|
||||
1. Run `kubectl create -f <name of service/deployment file>.yaml` if your file is stored locally in the cluster
|
||||
1. Or run `cat<< EOF | kubectl apply -f -`, paste the file contents into the terminal, then run `EOF` to complete the command.
|
||||
|
||||
### 4. Example Deployments and Services
|
||||
### Example Deployments and Services
|
||||
|
||||
Next we add the Kubernetes resources for the sample deployments and services for the BookInfo app in Istio's documentation.
|
||||
|
||||
+1
@@ -3,6 +3,7 @@ title: 2. Enable Istio in a Namespace
|
||||
weight: 2
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace
|
||||
- /rancher/v2.x/en/istio/setup/enable-istio-in-namespace
|
||||
---
|
||||
|
||||
You will need to manually enable Istio in each namespace that you want to be tracked or controlled by Istio. When Istio is enabled in a namespace, the Envoy sidecar proxy will be automatically injected into all new workloads that are deployed in the namespace.
|
||||
+1
@@ -3,6 +3,7 @@ title: 5. Set up the Istio Gateway
|
||||
weight: 5
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway
|
||||
- /rancher/v2.x/en/istio/setup/gateway
|
||||
---
|
||||
|
||||
The gateway to each cluster can have its own port or load balancer, which is unrelated to a service mesh. By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster.
|
||||
+1
@@ -3,6 +3,7 @@ title: 6. Set up Istio's Components for Traffic Management
|
||||
weight: 6
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/set-up-traffic-management
|
||||
- /rancher/v2.x/en/istio/setup/set-up-traffic-management
|
||||
---
|
||||
|
||||
A central advantage of traffic management in Istio is that it allows dynamic request routing. Some common applications for dynamic request routing include canary deployments and blue/green deployments. The two key resources in Istio traffic management are *virtual services* and *destination rules*.
|
||||
+2
-1
@@ -3,6 +3,7 @@ title: 7. Generate and View Traffic
|
||||
weight: 7
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/istio/setup/view-traffic
|
||||
- /rancher/v2.x/en/istio/setup/view-traffic
|
||||
---
|
||||
|
||||
This section describes how to view the traffic that is being managed by Istio.
|
||||
@@ -11,7 +12,7 @@ This section describes how to view the traffic that is being managed by Istio.
|
||||
|
||||
The Istio overpage provides a link to the Kiali dashboard. From the Kiali dashboard, you are able to view graphs for each namespace. The Kiali graph provides a powerful way to visualize the topology of your Istio service mesh. It shows you which services communicate with each other.
|
||||
|
||||
>**Prerequisite:** To enable traffic to show up in the graph, ensure you have enabled one of the [Selectors & Scrape Configs]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#selectors-scrape-configs)options. If you do not have this configured, you will not see information on the graph.
|
||||
>**Prerequisite:** To enable traffic to show up in the graph, ensure you have prometheus installed in the cluster. Rancher-istio installs Kiali configured by default to work with the rancher-monitoring chart. You can use rancher-monitoring or install your own monitoring solution. Optional: you can change configuration on how data scraping occurs by setting the [Selectors & Scrape Configs]({{<baseurl>}}/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/#selectors-scrape-configs) options.
|
||||
|
||||
To see the traffic graph,
|
||||
|
||||
@@ -40,7 +40,7 @@ For more information, see [load balancers]({{<baseurl>}}/rancher/v2.x/en/k8s-in-
|
||||
|
||||
Load Balancers can only handle one IP address per service, which means if you run multiple services in your cluster, you must have a load balancer for each service. Running multiples load balancers can be expensive. You can get around this issue by using an ingress.
|
||||
|
||||
Ingress is a set or rules that act as a load balancer. Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster program the load balancer to direct the request to the correct service based on service subdomains or path rules that you've configured.
|
||||
Ingress is a set of rules that act as a load balancer. Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster program the load balancer to direct the request to the correct service based on service subdomains or path rules that you've configured.
|
||||
|
||||
For more information, see [Ingress]({{<baseurl>}}/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress).
|
||||
|
||||
|
||||
@@ -8,238 +8,7 @@ aliases:
|
||||
- /rancher/v2.x/en/dashboard/logging
|
||||
---
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [Configuring the Logging Output for the Rancher Kubernetes Cluster](#configuring-the-logging-output-for-the-rancher-kubernetes-cluster)
|
||||
- [Enabling Logging for Rancher Managed Clusters](#enabling-logging-for-rancher-managed-clusters)
|
||||
- [Configuring the Logging Application](#configuring-the-logging-application)
|
||||
|
||||
|
||||
### Changes in Rancher v2.5
|
||||
|
||||
The following changes were introduced to logging in Rancher v2.5:
|
||||
|
||||
- Rancher's logging feature is now powered the [Banzai Cloud Logging operator](https://banzaicloud.com/docs/one-eye/logging-operator/) instead of Rancher's in-house logging solution.
|
||||
- [Fluent Bit](https://fluentbit.io/) is now used to aggregate the logs. [Fluentd](https://www.fluentd.org/) is used for filtering the messages and routing them to the outputs. Previously, only Fluentd was used.
|
||||
- Logging can be configured with a Kubernetes manifest, because now the logging uses a Kubernetes operator with Custom Resource Definitions.
|
||||
- We now support filtering logs.
|
||||
- We now support writing logs to multiple outputs.
|
||||
- We now always collect Control Plane and etcd logs.
|
||||
|
||||
|
||||
The following figure from the [Banzai documentation](https://banzaicloud.com/docs/one-eye/logging-operator/#architecture) shows the new logging architecture:
|
||||
|
||||
<figcaption>How the Banzai Cloud Logging Operator Works with Fluentd and Fluent Bit</figcaption>
|
||||
|
||||
|
||||
|
||||
### Configuring the Logging Output for the Rancher Kubernetes Cluster
|
||||
|
||||
If you install Rancher as a Helm chart, you'll configure the Helm chart options to select a logging output for all the logs in the local Kubernetes cluster.
|
||||
|
||||
If you install Rancher using the Rancher CLI on an Linux OS, the Rancher Helm chart will be installed on a Kubernetes cluster with default options. Then when the Rancher UI is available, you'll enable the logging app from the Apps section of the UI. Then during the process of installing the logging application, you will configure the logging output.
|
||||
|
||||
### Enabling Logging for Rancher Managed Clusters
|
||||
|
||||
If you have Enterprise Cluster Manager enabled, you can enable the logging for a Rancher managed cluster by going to the Apps page and installing the logging app.
|
||||
|
||||
### Configuring the Logging Application
|
||||
|
||||
The following Custom Resource Definitions are used to configure logging:
|
||||
|
||||
- [Flow and ClusterFlow](https://banzaicloud.com/docs/one-eye/logging-operator/crds/#flows-clusterflows)
|
||||
- [Output and ClusterOutput](https://banzaicloud.com/docs/one-eye/logging-operator/crds/#outputs-clusteroutputs)
|
||||
|
||||
According to the [Banzai Cloud documentation,](https://banzaicloud.com/docs/one-eye/logging-operator/#architecture)
|
||||
|
||||
> You can define `outputs` (destinations where you want to send your log messages, for example, Elasticsearch, or and Amazon S3 bucket), and `flows` that use filters and selectors to route log messages to the appropriate outputs. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users cannot modify.
|
||||
|
||||
### RBAC
|
||||
Rancher logging has two roles, `logging-admin` and `logging-view`. `logging-admin` allows users full access to namespaced flows and outputs. The `logging-view` role allows users to view namespaced flows and outputs, and cluster flows and outputs. Edit access to the cluster flow and cluster output resources is powerful as it allows any user with edit access control of all logs in the cluster. Cluster admin is the only role with full access to all rancher-logging resources. Cluster members are not able to edit or read any logging resources. Project owners are able to create namespaced flows and outputs in the namespaces under their projects. This means that project owners can collect logs from anything in their project namespaces. Project members are able to view the flows and outputs in the namespaces under their projects. Project owners and project members require at least 1 namespace in their project to use logging. If they do not have at least one namespace in their project they may not see the logging button in the top nav dropdown.
|
||||
|
||||
|
||||
### Examples
|
||||
|
||||
Let's say you wanted to send all logs in your cluster to an elasticsearch cluster.
|
||||
|
||||
First lets create our cluster output:
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterOutput
|
||||
metadata:
|
||||
name: "example-es"
|
||||
namespace: "cattle-logging-system"
|
||||
spec:
|
||||
elasticsearch:
|
||||
host: elasticsearch.example.com
|
||||
port: 9200
|
||||
scheme: http
|
||||
```
|
||||
|
||||
We have created a cluster output, without elasticsearch configuration, in the same namespace as our operator `cattle-logging-system.`. Any time we create a cluster flow or cluster output we have to put it in the `cattle-logging-system` namespace.
|
||||
|
||||
Now we have configured where we want the logs to go, lets configure all logs to go to that output.
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterFlow
|
||||
metadata:
|
||||
name: "all-logs"
|
||||
namespace: "cattle-logging-system"
|
||||
spec:
|
||||
globalOutputRefs:
|
||||
- "example-es
|
||||
```
|
||||
|
||||
We should now see our configured index with logs in it.
|
||||
|
||||
What if we have an application team who only wants logs from a specific namespaces sent to a splunk server? For this case can use namespaced outputs and flows.
|
||||
|
||||
Before we start lets set up a scenario.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: devteam
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: coolapp
|
||||
namespace: devteam
|
||||
labels:
|
||||
app: coolapp
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: coolapp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: coolapp
|
||||
spec:
|
||||
containers:
|
||||
- name: generator
|
||||
image: paynejacob/loggenerator:latest
|
||||
```
|
||||
|
||||
like before we start with an output, unlike cluster outputs we create our output in our application's namespace:
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: Output
|
||||
metadata:
|
||||
name: "devteam-splunk"
|
||||
namespace: "devteam"
|
||||
spec:
|
||||
SplunkHec:
|
||||
host: splunk.example.com
|
||||
port: 8088
|
||||
protocol: http
|
||||
```
|
||||
|
||||
Once again, lets give our output some logs:
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: Flow
|
||||
metadata:
|
||||
name: "devteam-logs"
|
||||
namespace: "devteam"
|
||||
spec:
|
||||
localOutputRefs:
|
||||
- "devteam-splunk"
|
||||
```
|
||||
|
||||
For the final example we create an output to write logs to a destination that is not supported out of the box (e.g. syslog):
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: syslog-config
|
||||
namespace: cattle-logging-system
|
||||
type: Opaque
|
||||
stringData:
|
||||
fluent-bit.conf: |
|
||||
[INPUT]
|
||||
Name forward
|
||||
Port 24224
|
||||
|
||||
[OUTPUT]
|
||||
Name syslog
|
||||
InstanceName syslog-output
|
||||
Match *
|
||||
Addr syslog.example.com
|
||||
Port 514
|
||||
Cluster ranchers
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: fluentbit-syslog-forwarder
|
||||
namespace: cattle-logging-system
|
||||
labels:
|
||||
output: syslog
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
output: syslog
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
output: syslog
|
||||
spec:
|
||||
containers:
|
||||
- name: fluentbit
|
||||
image: paynejacob/fluent-bit-out-syslog:latest
|
||||
ports:
|
||||
- containerPort: 24224
|
||||
volumeMounts:
|
||||
- mountPath: "/fluent-bit/etc/"
|
||||
name: configuration
|
||||
volumes:
|
||||
- name: configuration
|
||||
secret:
|
||||
secretName: syslog-config
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: syslog-forwarder
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
selector:
|
||||
output: syslog
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 24224
|
||||
targetPort: 24224
|
||||
---
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterFlow
|
||||
metadata:
|
||||
name: all-logs
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
globalOutputRefs:
|
||||
- syslog
|
||||
---
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterOutput
|
||||
metadata:
|
||||
name: syslog
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
forward:
|
||||
servers:
|
||||
- host: "syslog-forwarder.cattle-logging-system"
|
||||
require_ack_response: false
|
||||
ignore_network_errors_at_startup: false
|
||||
```
|
||||
|
||||
if we break down what is happening, first we create a deployment of a container that has the additional syslog plugin and accepts logs forwarded from another fluentd. Next we create an output configured as a forwarder to our deployment. The deployment fluentd will then forward all logs to the configured syslog destination.
|
||||
|
||||
If you are using Rancher v2.5, refer to [this section.](./v2.5)
|
||||
|
||||
If you are using Rancher v2.0-v2.4, refer to [this section.](./v2.0.x-v2.4.x)
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
title: Cluster Manager Logging Documentation
|
||||
shortTitle: Cluster Manager
|
||||
weight: 1
|
||||
---
|
||||
|
||||
This section contains documentation for the logging features that were available in Rancher prior to v2.5.
|
||||
@@ -0,0 +1,11 @@
|
||||
---
|
||||
title: Logging in Rancher v2.0-v2.4
|
||||
shortTitle: Rancher v2.0-v2.4
|
||||
weight: 2
|
||||
---
|
||||
|
||||
|
||||
This section contains documentation for the logging features that were available in Rancher prior to v2.5.
|
||||
|
||||
- [Cluster logging](./cluster-logging)
|
||||
- [Project logging](./project-logging)
|
||||
+4
-2
@@ -1,14 +1,16 @@
|
||||
---
|
||||
title: Cluster Logging
|
||||
title: Cluster Logging in Rancher v2.0.x-v2.4.x
|
||||
shortTitle: Cluster Logging
|
||||
description: Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster.
|
||||
metaDescription: "Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster."
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/tasks/logging/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging
|
||||
---
|
||||
|
||||
> In Rancher 2.5, the logging application was improved. There are now two ways to enable logging. The older way is documented in this section, and the new application for logging is documented [dashboard section.]({{<baseurl>}}/rancher/v2.x/en/logging)
|
||||
> In Rancher 2.5, the logging application was improved. There are now two ways to enable logging. The older way is documented in this section, and the new application for logging is documented [here.]({{<baseurl>}}/rancher/v2.x/en/logging/v2.5)
|
||||
|
||||
Logging is helpful because it allows you to:
|
||||
|
||||
+1
@@ -4,6 +4,7 @@ weight: 200
|
||||
aliases:
|
||||
- /rancher/v2.x/en/tools/logging/elasticsearch/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging/elasticsearch
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging/elasticsearch
|
||||
---
|
||||
|
||||
If your organization uses [Elasticsearch](https://www.elastic.co/), either on premise or in the cloud, you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Elasticsearch deployment to view logs.
|
||||
+1
@@ -3,6 +3,7 @@ title: Fluentd
|
||||
weight: 600
|
||||
aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging/fluentd
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging/fluentd
|
||||
---
|
||||
|
||||
If your organization uses [Fluentd](https://www.fluentd.org/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Fluentd server to view logs.
|
||||
+1
@@ -4,6 +4,7 @@ weight: 400
|
||||
aliases:
|
||||
- /rancher/v2.x/en/tools/logging/kafka/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging/kafka
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging/kafka
|
||||
---
|
||||
|
||||
If your organization uses [Kafka](https://kafka.apache.org/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Kafka server to view logs.
|
||||
+1
@@ -5,6 +5,7 @@ aliases:
|
||||
- /rancher/v2.x/en/tasks/logging/splunk/
|
||||
- /rancher/v2.x/en/tools/logging/splunk/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging/splunk
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging/splunk
|
||||
---
|
||||
|
||||
If your organization uses [Splunk](https://www.splunk.com/), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Splunk server to view logs.
|
||||
+1
@@ -4,6 +4,7 @@ weight: 500
|
||||
aliases:
|
||||
- /rancher/v2.x/en/tools/logging/syslog/
|
||||
- /rancher/v2.x/en/cluster-admin/tools/logging/syslog
|
||||
- /rancher/v2.x/en/logging/legacy/cluster-logging/syslog
|
||||
---
|
||||
|
||||
If your organization uses [Syslog](https://tools.ietf.org/html/rfc5424), you can configure Rancher to send it Kubernetes logs. Afterwards, you can log into your Syslog server to view logs.
|
||||
+3
-1
@@ -1,8 +1,10 @@
|
||||
---
|
||||
title: Project Logging
|
||||
title: Project Logging in Rancher v2.0.x-v2.4.x
|
||||
shortTitle: Project Logging
|
||||
weight: 2527
|
||||
aliases:
|
||||
- /rancher/v2.x/en/project-admin/tools/logging
|
||||
- /rancher/v2.x/en/logging/legacy/project-logging
|
||||
---
|
||||
|
||||
Rancher can integrate with a variety of popular logging services and tools that exist outside of your Kubernetes clusters.
|
||||
@@ -0,0 +1,300 @@
|
||||
---
|
||||
title: Logging in Rancher v2.5
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
- [Changes in Rancher v2.5](#changes-in-rancher-v2-5)
|
||||
- [Configuring the Logging Output for the Rancher Kubernetes Cluster](#configuring-the-logging-output-for-the-rancher-kubernetes-cluster)
|
||||
- [Enabling Logging for Rancher Managed Clusters](#enabling-logging-for-rancher-managed-clusters)
|
||||
- [Configuring the Logging Application](#configuring-the-logging-application)
|
||||
- [Working with Taints and Tolerations](#working-with-taints-and-tolerations)
|
||||
|
||||
|
||||
### Changes in Rancher v2.5
|
||||
|
||||
The following changes were introduced to logging in Rancher v2.5:
|
||||
|
||||
- The [Banzai Cloud Logging operator](https://banzaicloud.com/docs/one-eye/logging-operator/) now powers Rancher's logging in place of the former, in-house logging solution.
|
||||
- [Fluent Bit](https://fluentbit.io/) is now used to aggregate the logs. [Fluentd](https://www.fluentd.org/) is used for filtering the messages and routing them to the outputs. Previously, only Fluentd was used.
|
||||
- Logging can be configured with a Kubernetes manifest, because now the logging uses a Kubernetes operator with Custom Resource Definitions.
|
||||
- We now support filtering logs.
|
||||
- We now support writing logs to multiple outputs.
|
||||
- We now always collect Control Plane and etcd logs.
|
||||
|
||||
|
||||
The following figure from the [Banzai documentation](https://banzaicloud.com/docs/one-eye/logging-operator/#architecture) shows the new logging architecture:
|
||||
|
||||
<figcaption>How the Banzai Cloud Logging Operator Works with Fluentd and Fluent Bit</figcaption>
|
||||
|
||||
|
||||
|
||||
### Configuring the Logging Output for the Rancher Kubernetes Cluster
|
||||
|
||||
If you install Rancher as a Helm chart, you'll configure the Helm chart options to select a logging output for all the logs in the local Kubernetes cluster.
|
||||
|
||||
If you install Rancher using the Rancher CLI on an Linux OS, the Rancher Helm chart will be installed on a Kubernetes cluster with default options. Then when the Rancher UI is available, you'll enable the logging app from the Apps section of the UI. Then during the process of installing the logging application, you will configure the logging output.
|
||||
|
||||
### Enabling Logging for Rancher Managed Clusters
|
||||
|
||||
If you have Enterprise Cluster Manager enabled, you can enable the logging for a Rancher managed cluster by going to the Apps page and installing the logging app.
|
||||
|
||||
### Configuring the Logging Application
|
||||
|
||||
The following Custom Resource Definitions are used to configure logging:
|
||||
|
||||
- [Flow and ClusterFlow](https://banzaicloud.com/docs/one-eye/logging-operator/crds/#flows-clusterflows)
|
||||
- [Output and ClusterOutput](https://banzaicloud.com/docs/one-eye/logging-operator/crds/#outputs-clusteroutputs)
|
||||
|
||||
According to the [Banzai Cloud documentation,](https://banzaicloud.com/docs/one-eye/logging-operator/#architecture)
|
||||
|
||||
> You can define `outputs` (destinations where you want to send your log messages, for example, Elasticsearch, or and Amazon S3 bucket), and `flows` that use filters and selectors to route log messages to the appropriate outputs. You can also define cluster-wide outputs and flows, for example, to use a centralized output that namespaced users cannot modify.
|
||||
|
||||
**RBAC**
|
||||
|
||||
Rancher logging has two roles, `logging-admin` and `logging-view`. `logging-admin` allows users full access to namespaced flows and outputs. The `logging-view` role allows users to view namespaced flows and outputs, and cluster flows and outputs. Edit access to the cluster flow and cluster output resources is powerful as it allows any user with edit access control of all logs in the cluster. Cluster admin is the only role with full access to all rancher-logging resources. Cluster members are not able to edit or read any logging resources. Project owners are able to create namespaced flows and outputs in the namespaces under their projects. This means that project owners can collect logs from anything in their project namespaces. Project members are able to view the flows and outputs in the namespaces under their projects. Project owners and project members require at least 1 namespace in their project to use logging. If they do not have at least one namespace in their project they may not see the logging button in the top nav dropdown.
|
||||
|
||||
**Examples**
|
||||
|
||||
Let's say you wanted to send all logs in your cluster to an elasticsearch cluster.
|
||||
|
||||
First lets create our cluster output:
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterOutput
|
||||
metadata:
|
||||
name: "example-es"
|
||||
namespace: "cattle-logging-system"
|
||||
spec:
|
||||
elasticsearch:
|
||||
host: elasticsearch.example.com
|
||||
port: 9200
|
||||
scheme: http
|
||||
```
|
||||
|
||||
We have created a cluster output, without elasticsearch configuration, in the same namespace as our operator `cattle-logging-system.`. Any time we create a cluster flow or cluster output we have to put it in the `cattle-logging-system` namespace.
|
||||
|
||||
Now we have configured where we want the logs to go, lets configure all logs to go to that output.
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterFlow
|
||||
metadata:
|
||||
name: "all-logs"
|
||||
namespace: "cattle-logging-system"
|
||||
spec:
|
||||
globalOutputRefs:
|
||||
- "example-es
|
||||
```
|
||||
|
||||
We should now see our configured index with logs in it.
|
||||
|
||||
What if we have an application team who only wants logs from a specific namespaces sent to a splunk server? For this case can use namespaced outputs and flows.
|
||||
|
||||
Before we start lets set up a scenario.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: devteam
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: coolapp
|
||||
namespace: devteam
|
||||
labels:
|
||||
app: coolapp
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: coolapp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: coolapp
|
||||
spec:
|
||||
containers:
|
||||
- name: generator
|
||||
image: paynejacob/loggenerator:latest
|
||||
```
|
||||
|
||||
like before we start with an output, unlike cluster outputs we create our output in our application's namespace:
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: Output
|
||||
metadata:
|
||||
name: "devteam-splunk"
|
||||
namespace: "devteam"
|
||||
spec:
|
||||
SplunkHec:
|
||||
host: splunk.example.com
|
||||
port: 8088
|
||||
protocol: http
|
||||
```
|
||||
|
||||
Once again, lets give our output some logs:
|
||||
|
||||
```yaml
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: Flow
|
||||
metadata:
|
||||
name: "devteam-logs"
|
||||
namespace: "devteam"
|
||||
spec:
|
||||
localOutputRefs:
|
||||
- "devteam-splunk"
|
||||
```
|
||||
|
||||
For the final example we create an output to write logs to a destination that is not supported out of the box (e.g. syslog):
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: syslog-config
|
||||
namespace: cattle-logging-system
|
||||
type: Opaque
|
||||
stringData:
|
||||
fluent-bit.conf: |
|
||||
[INPUT]
|
||||
Name forward
|
||||
Port 24224
|
||||
|
||||
[OUTPUT]
|
||||
Name syslog
|
||||
InstanceName syslog-output
|
||||
Match *
|
||||
Addr syslog.example.com
|
||||
Port 514
|
||||
Cluster ranchers
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: fluentbit-syslog-forwarder
|
||||
namespace: cattle-logging-system
|
||||
labels:
|
||||
output: syslog
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
output: syslog
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
output: syslog
|
||||
spec:
|
||||
containers:
|
||||
- name: fluentbit
|
||||
image: paynejacob/fluent-bit-out-syslog:latest
|
||||
ports:
|
||||
- containerPort: 24224
|
||||
volumeMounts:
|
||||
- mountPath: "/fluent-bit/etc/"
|
||||
name: configuration
|
||||
volumes:
|
||||
- name: configuration
|
||||
secret:
|
||||
secretName: syslog-config
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: syslog-forwarder
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
selector:
|
||||
output: syslog
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 24224
|
||||
targetPort: 24224
|
||||
---
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterFlow
|
||||
metadata:
|
||||
name: all-logs
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
globalOutputRefs:
|
||||
- syslog
|
||||
---
|
||||
apiVersion: logging.banzaicloud.io/v1beta1
|
||||
kind: ClusterOutput
|
||||
metadata:
|
||||
name: syslog
|
||||
namespace: cattle-logging-system
|
||||
spec:
|
||||
forward:
|
||||
servers:
|
||||
- host: "syslog-forwarder.cattle-logging-system"
|
||||
require_ack_response: false
|
||||
ignore_network_errors_at_startup: false
|
||||
```
|
||||
|
||||
if we break down what is happening, first we create a deployment of a container that has the additional syslog plugin and accepts logs forwarded from another fluentd. Next we create an output configured as a forwarder to our deployment. The deployment fluentd will then forward all logs to the configured syslog destination.
|
||||
|
||||
### Working with Taints and Tolerations
|
||||
|
||||
"Tainting" a Kubernetes node causes pods to repel running on that node.
|
||||
Unless the pods have a ```toleration``` for that node's taint, they will run on other nodes in the cluster.
|
||||
[Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) can work in conjunction with the ```nodeSelector``` [field](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) within the ```PodSpec```, which enables the *opposite* effect of a taint.
|
||||
Using ```nodeSelector``` gives pods an affinity towards certain nodes.
|
||||
Both provide choice for the what node(s) the pod will run on.
|
||||
|
||||
**Default Implementation in Rancher's Logging Stack**
|
||||
|
||||
By default, Rancher taints all Linux nodes with ```cattle.io/os=linux```, and does not taint Windows nodes.
|
||||
The logging stack pods have ```tolerations``` for this taint, which enables them to run on Linux nodes.
|
||||
Moreover, we can populate the ```nodeSelector``` to ensure that our pods *only* run on Linux nodes.
|
||||
Let's look at an example pod YAML file with these settings...
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
# metadata:
|
||||
spec:
|
||||
# containers:
|
||||
tolerations:
|
||||
- key: cattle.io/os
|
||||
operator: "Equal"
|
||||
value: "linux"
|
||||
effect: NoSchedule
|
||||
nodeSelector:
|
||||
kubernetes.io/os: linux
|
||||
```
|
||||
|
||||
In the above example, we ensure that our pod only runs on Linux nodes, and we add a ```toleration``` for the taint we have on all of our Linux nodes.
|
||||
You can do the same with Rancher's existing taints, or with your own custom ones.
|
||||
|
||||
**Why do we not schedule logging-related pods on Windows nodes?**
|
||||
|
||||
No parts of the logging stack are compatible with Windows Kubernetes nodes.
|
||||
For instance, if a logging pod is attempting to pull its image from a container registry, there may only be Linux-compatible images available.
|
||||
In this scenario, the pod would be stuck in an ```ImagePullBackOff``` status; and would eventually change to a ```ErrImagePull``` status.
|
||||
|
||||
**Adding NodeSelector Settings and Tolerations for Custom Taints**
|
||||
|
||||
If you would like to add your own ```nodeSelector``` settings, or if you would like to add ```tolerations``` for additional taints, you can pass the following to the chart's values.
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
# insert tolerations list
|
||||
nodeSelector:
|
||||
# insert nodeSelector settings
|
||||
```
|
||||
|
||||
These values will add both settings to the ```fluentd```, ```fluentbit```, and ```logging-operator``` containers.
|
||||
Essentially, these are global settings for all pods in the logging stack.
|
||||
|
||||
However, if you would like to add tolerations for *only* the ```fluentbit``` container, you can add the following to the chart's values.
|
||||
|
||||
```yaml
|
||||
fluentbit_tolerations:
|
||||
# insert tolerations list for fluentbit containers only
|
||||
```
|
||||
@@ -21,7 +21,9 @@ With Longhorn, you can:
|
||||
|
||||
### New in Rancher v2.5
|
||||
|
||||
Prior to Rancher v2.5, Longhorn could be installed as a Rancher catalog app. In Rancher v2.5, the catalog system was replaced by the **Apps & Marketplace,** and it became possible to install Longhorn as an app from that page. The **Cluster Explorer** now allows you to manipulate Longhorn's Kubernetes resources from the Rancher UI. So now you can control the Longhorn functionality with the Longhorn UI, or with kubectl, or by manipulating Longhorn's Kubernetes custom resources in the Rancher UI.
|
||||
Prior to Rancher v2.5, Longhorn could be installed as a Rancher catalog app. In Rancher v2.5, the catalog system was replaced by the **Apps & Marketplace,** and it became possible to install Longhorn as an app from that page.
|
||||
|
||||
The **Cluster Explorer** now allows you to manipulate Longhorn's Kubernetes resources from the Rancher UI. So now you can control the Longhorn functionality with the Longhorn UI, or with kubectl, or by manipulating Longhorn's Kubernetes custom resources in the Rancher UI.
|
||||
|
||||
These instructions assume you are using Rancher v2.5, but Longhorn can be installed with earlier Rancher versions. For documentation about installing Longhorn as a catalog app using the legacy Rancher UI, refer to the [Longhorn documentation.](https://longhorn.io/docs/1.0.2/deploy/install/install-with-rancher/)
|
||||
|
||||
|
||||
@@ -9,185 +9,6 @@ aliases:
|
||||
- /rancher/v2.x/en/cluster-admin/tools/monitoring/
|
||||
---
|
||||
|
||||
Using Rancher, you can quickly deploy leading open-source monitoring & alerting solutions such as [Prometheus](https://prometheus.io/), [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), and [Grafana](https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/) onto your cluster.
|
||||
If you are using Rancher v2.5, refer to [this section.](./v2.5)
|
||||
|
||||
Rancher's solution (powered by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator)) allows users to:
|
||||
|
||||
- Monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments via [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
|
||||
|
||||
- Defines alerts based on metrics collected via [Prometheus](https://prometheus.io/)
|
||||
- Creates custom dashboards to make it easy to visualize collected metrics via [Grafana](https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/)
|
||||
- Configures alert-based notifications via Email, Slack, PagerDuty, etc. using [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/)
|
||||
- Defines precomputed frequently needed / computationally expensive expressions as new time series based on metrics collected via [Prometheus](https://prometheus.io/) (only available in 2.5.x)
|
||||
- Exposes collected metrics from Prometheus to the Kubernetes Custom Metrics API via [Prometheus Adapter](https://github.com/DirectXMan12/k8s-prometheus-adapter) for use in HPA (only available in 2.5)
|
||||
|
||||
More information about the resources that get deployed onto your cluster to support this solution can be found in the [`rancher-monitoring`](https://github.com/rancher/charts/tree/main/charts/rancher-monitoring) Helm chart, which closely tracks the upstream [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) Helm chart maintained by the Prometheus community with certain changes tracked in the [CHANGELOG.md](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/CHANGELOG.md).
|
||||
|
||||
This page describes how to enable monitoring & alerting within a cluster using Rancher's new monitoring application, which was introduced in Rancher v2.5.
|
||||
|
||||
If you previously enabled Monitoring, Alerting, or Notifiers in Rancher prior to v2.5, there is no upgrade path for switching to the new monitoring/ alerting solution. You will need to disable monitoring/ alerting/notifiers in Cluster Manager before deploying the new monitoring solution via Cluster Explorer.
|
||||
|
||||
For more information about upgrading the Monitoring app in Rancher 2.5, please refer to the [migration docs](../migrating).
|
||||
|
||||
For the docs about monitoring for earlier Rancher versions, refer to [this section.](../legacy)
|
||||
|
||||
> Before enabling monitoring, be sure to review the resource requirements. The default values in [this section](#setting-resource-limits-and-requests) are the minimum required resource limits and requests.
|
||||
|
||||
- [Monitoring Components](#monitoring-components)
|
||||
- [Prometheus](#about-prometheus)
|
||||
- [Grafana](#about-grafana)
|
||||
- [Alertmanager](#about-alertmanager)
|
||||
- [Prometheus Operator](#about-prometheus-operator)
|
||||
- [Prometheus Adapter](#about-prometheus-adapter)
|
||||
- [Enable Monitoring](#enable-monitoring)
|
||||
- [Default Alerts, Targets and Grafana Dashboards](#default-alerts-targets-and-grafana-dashboards)
|
||||
- [Using Monitoring](#using-monitoring)
|
||||
- [Grafana UI](#grafana-ui)
|
||||
- [Prometheus UI](#prometheus-ui)
|
||||
- [Viewing the Prometheus Targets](#viewing-the-prometheus-targets)
|
||||
- [Viewing the Prometheus Rules](#viewing-the-prometheus-rules)
|
||||
- [Viewing Active Alerts in Alertmanager](#viewing-active-alerts-in-alertmanager)
|
||||
- [Uninstall Monitoring](#uninstall-monitoring)
|
||||
- [Setting Resource Limits and Requests](#setting-resource-limits-and-requests)
|
||||
- [Known Issues](#known-issues)
|
||||
|
||||
# Monitoring Components
|
||||
|
||||
The `rancher-monitoring` operator is powered by Prometheus, Grafana, Alertmanager, the Prometheus Operator, and the Prometheus adapter.
|
||||
|
||||
### About Prometheus
|
||||
|
||||
Prometheus provides a time series of your data, which is, according to the [Prometheus documentation:](https://prometheus.io/docs/concepts/data_model/)
|
||||
|
||||
> A stream of timestamped values belonging to the same metric and the same set of labeled dimensions, along with comprehensive statistics and metrics of the monitored cluster.
|
||||
|
||||
In other words, Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or Grafana, which is an analytics viewing platform deployed along with Prometheus.
|
||||
|
||||
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc.
|
||||
|
||||
### About Grafana
|
||||
|
||||
[Grafana](https://grafana.com/grafana/) allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data driven culture.
|
||||
|
||||
# Enabling Cluster Monitoring
|
||||
|
||||
As an [administrator]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), you can configure Rancher to deploy Prometheus to monitor your Kubernetes cluster.
|
||||
|
||||
> If you want to set up Alertmanager, Grafana or Ingress, it has to be done with the settings on the Helm chart deployment. It's problematic to create Ingress outside the deployment.
|
||||
|
||||
> **Prerequisites:**
|
||||
>
|
||||
> - Make sure that you are allowing traffic on port 9796 for each of your nodes because Prometheus will scrape metrics from here.
|
||||
> - Make sure your cluster fulfills the resource requirements. The cluster should have at least 1950Mi memory available, 2700m CPU, and 50Gi storage. A breakdown of the resource limits and requests is [here.](#resource-requirements)
|
||||
|
||||
1. In the Rancher UI, go to the cluster where you want to install monitoring and click **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click the `rancher-monitoring` app.
|
||||
1. Optional: Click **Chart Options** and configure alerting, Prometheus and Grafana. For help, refer to the [configuration reference.](../configuration)
|
||||
1. Scroll to the bottom of the Helm chart README and click **Install.**
|
||||
|
||||
**Result:** The monitoring app is deployed in the `cattle-monitoring-system` namespace.
|
||||
|
||||
### Default Alerts, Targets and Grafana Dashboards
|
||||
|
||||
By default, Rancher Monitoring deploys exporters (such as [node-exporter](https://github.com/prometheus/node_exporter) and [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)) as well as default Prometheus alerts and Grafana dashboards (curated by the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project) onto a cluster.
|
||||
|
||||
To see the default alerts, go to the [Alertmanager UI](#alertmanager-ui) and click **Expand all groups.**
|
||||
|
||||
To see what services you are monitoring, you will need to see your targets. To view the default targets, refer to [Viewing the Prometheus Targets.](#viewing-the-prometheus-targets)
|
||||
|
||||
To see the default dashboards, go to the [Grafana UI.](#grafana-ui) In the left navigation bar, click the icon with four boxes and click **Manage.**
|
||||
|
||||
### Next Steps
|
||||
|
||||
To configure Prometheus resources from the Rancher UI, click **Apps & Marketplace > Monitoring** in the upper left corner.
|
||||
|
||||
# Using Monitoring
|
||||
|
||||
Installing `rancher-monitoring` makes the following dashboards available from the Rancher UI.
|
||||
|
||||
### Grafana UI
|
||||
|
||||
Rancher allows any users who are authenticated by Kubernetes and have access the Grafana service deployed by the Rancher Monitoring chart to access Grafana via the Rancher Dashboard UI. By default, all users who are able to access Grafana are given the [Viewer](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#viewer-role) role, which allows them to view any of the default dashboards deployed by Rancher.
|
||||
|
||||
However, users can choose to log in to Grafana as an [Admin](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#admin-role) if necessary. The default Admin username and password for the Grafana instance will be `admin`/`prom-operator`, but alternative credentials can also be supplied on deploying or upgrading the chart.
|
||||
|
||||
To see the Grafana UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Grafana.
|
||||
|
||||
<figcaption>Cluster Compute Resources Dashboard in Grafana</figcaption>
|
||||
|
||||
|
||||
<figcaption>Default Dashboards in Grafana</figcaption>
|
||||
|
||||
|
||||
To allow the Grafana dashboard to persist after it restarts, you will need to add the configuration JSON into a ConfigMap. You can add this configuration to the ConfigMap using the Rancher UI.
|
||||
|
||||
### Prometheus UI
|
||||
|
||||
To see the Prometheus UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Graph.**
|
||||
|
||||
<figcaption>Prometheus Graph UI</figcaption>
|
||||
|
||||
|
||||
### Viewing the Prometheus Targets
|
||||
|
||||
To see the Prometheus Targets, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Targets.**
|
||||
|
||||
<figcaption>Targets in the Prometheus UI</figcaption>
|
||||
|
||||
|
||||
### Viewing the Prometheus Rules
|
||||
|
||||
To see the Prometheus Rules, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Rules.**
|
||||
|
||||
<figcaption>Rules in the Prometheus UI</figcaption>
|
||||
|
||||
|
||||
### Viewing Active Alerts in Alertmanager
|
||||
|
||||
When `rancher-monitoring` is installed, the Prometheus Alertmanager UI is deployed.
|
||||
|
||||
The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.
|
||||
|
||||
In the Alertmanager UI, you can view your alerts and the current Alertmanager configuration.
|
||||
|
||||
To see the Prometheus Rules, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Alertmanager.**
|
||||
|
||||
**Result:** The Alertmanager UI opens in a new tab. For help with configuration, refer to the [official Alertmanager documentation.](https://prometheus.io/docs/alerting/latest/alertmanager/)
|
||||
|
||||
<figcaption>The Alertmanager UI</figcaption>
|
||||
|
||||
|
||||
# Uninstall Monitoring
|
||||
|
||||
1. From the **Cluster Explorer,** click Apps & Marketplace.
|
||||
1. Click **Installed Apps.**
|
||||
1. Go to the `cattle-monitoring-system` namespace and check the boxes for `rancher-monitoring-crd` and `rancher-monitoring`.
|
||||
1. Click **Delete.**
|
||||
1. Confirm **Delete.**
|
||||
|
||||
**Result:** `rancher-monitoring` is uninstalled.
|
||||
|
||||
# Setting Resource Limits and Requests
|
||||
|
||||
The resource requests and limits can be configured when installing `rancher-monitoring`.
|
||||
|
||||
The default values are in the [values.yaml](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/values.yaml) in the `rancher-monitoring` Helm chart.
|
||||
|
||||
The default values in the table below are the minimum required resource limits and requests.
|
||||
|
||||
| Resource Name | Memory Limit | CPU Limit | Memory Request | CPU Request |
|
||||
| ------------- | ------------ | ----------- | ---------------- | ------------------ |
|
||||
| alertmanager | 500Mi | 1000m | 100Mi | 100m |
|
||||
| grafana | 200Mi | 200m | 100Mi | 100m |
|
||||
| kube-state-metrics subchart | 200Mi | 100m | 130Mi | 100m |
|
||||
| prometheus-node-exporter subchart | 50Mi | 200m | 30Mi | 100m |
|
||||
| prometheusOperator | 500Mi | 200m | 100Mi | 100m |
|
||||
| prometheus | 2500Mi | 1000m | 1750Mi | 750m |
|
||||
| **Total** | **3950Mi** | **2700m** | **2210Mi** | **1250m** |
|
||||
|
||||
At least 50Gi storage is recommended.
|
||||
|
||||
# Known Issues
|
||||
|
||||
There is a [known issue](https://github.com/rancher/rancher/issues/28787#issuecomment-693611821) that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend to setting `prometheus.prometheusSpec.resources.memory.limit` to 2500Mi` and `prometheus.prometheusSpec.resources.memory.request` to 1750Mi.
|
||||
If you are using Rancher v2.0-v2.4, refer to [this section.](./v2.0.x-v2.4.x)
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
title: Cluster Manager Monitoring/Alerting Documentation
|
||||
shortTitle: Cluster Manager
|
||||
weight: 1
|
||||
---
|
||||
|
||||
This section contains documentation related to the monitoring features available in Rancher prior to v2.5.
|
||||
@@ -1,4 +0,0 @@
|
||||
---
|
||||
title: Monitoring
|
||||
weight: 1
|
||||
---
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: Monitoring in Rancher v2.0-v2.4
|
||||
shortTitle: Rancher v2.0-v2.4
|
||||
weight: 2
|
||||
---
|
||||
|
||||
This section contains documentation related to the monitoring features available in Rancher prior to v2.5.
|
||||
|
||||
|
||||
|
||||
- [Cluster Monitoring](./cluster-monitoring)
|
||||
- [Project Monitoring](./cluster-monitoring/project-monitoring)
|
||||
- [Cluster Alerts](./cluster-alerts)
|
||||
- [Project Alerts](./cluster-alerts/project-alerts)
|
||||
- [Notifiers](./notifiers)
|
||||
+2
-1
@@ -2,7 +2,8 @@
|
||||
title: Cluster Alerts
|
||||
weight: 2
|
||||
aliases:
|
||||
- rancher/v2.x/en/cluster-admin/tools/alerts
|
||||
- /rancher/v2.x/en/cluster-admin/tools/alerts
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/alerts/cluster-alerts
|
||||
---
|
||||
|
||||
> In Rancher 2.5, the monitoring application was improved. There are now two ways to enable monitoring and alerting. The older way is documented in this section, and the new application for monitoring and alerting is documented [here.]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting)
|
||||
+2
-1
@@ -2,7 +2,8 @@
|
||||
title: Default Alerts for Cluster Monitoring
|
||||
weight: 1
|
||||
aliases:
|
||||
- rancher/v2.x/en/cluster-admin/tools/alerts/default-alerts
|
||||
- /rancher/v2.x/en/cluster-admin/tools/alerts/default-alerts
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/alerts/cluster-alerts/default-alerts
|
||||
---
|
||||
|
||||
When you create a cluster, some alert rules are predefined. These alerts notify you about signs that the cluster could be unhealthy. You can receive these alerts if you configure a [notifier]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/notifiers) for them.
|
||||
+2
-1
@@ -2,7 +2,8 @@
|
||||
title: Project Alerts
|
||||
weight: 2526
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/alerts
|
||||
- /rancher/v2.x/en/project-admin/tools/alerts
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/alerts/project-alerts
|
||||
---
|
||||
|
||||
To keep your clusters and applications healthy and driving your organizational productivity forward, you need to stay informed of events occurring in your clusters and projects, both planned and unplanned. When an event occurs, your alert is triggered, and you are sent a notification. You can then, if necessary, follow up with corrective actions.
|
||||
+4
-2
@@ -2,9 +2,11 @@
|
||||
title: Integrating Rancher and Prometheus for Cluster Monitoring
|
||||
shortTitle: Cluster Monitoring
|
||||
description: Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Learn about the scope of monitoring and how to enable cluster monitoring
|
||||
weight: 4
|
||||
weight: 1
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring
|
||||
- /rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/monitoring/cluster-monitoring
|
||||
---
|
||||
|
||||
_Available as of v2.2.0_
|
||||
+1
@@ -4,6 +4,7 @@ weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring/cluster-metrics
|
||||
- /rancher/v2.x/en/cluster-admin/tools/monitoring/cluster-metrics
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/cluster-metrics
|
||||
---
|
||||
|
||||
_Available as of v2.2.0_
|
||||
+2
-1
@@ -2,7 +2,8 @@
|
||||
title: Prometheus Custom Metrics Adapter
|
||||
weight: 5
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring/custom-metrics
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring/custom-metrics
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/custom-metrics
|
||||
---
|
||||
|
||||
After you've enabled [cluster level monitoring]({{< baseurl >}}/rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/#enabling-cluster-monitoring), You can view the metrics data from Rancher. You can also deploy the Prometheus custom metrics adapter then you can use the HPA with metrics stored in cluster monitoring.
|
||||
+2
-1
@@ -2,8 +2,9 @@
|
||||
title: Prometheus Expressions
|
||||
weight: 4
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring/expression
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring/expression
|
||||
- /rancher/v2.x/en/cluster-admin/tools/monitoring/expression
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/expression
|
||||
---
|
||||
|
||||
The PromQL expressions in this doc can be used to configure [alerts.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/alerts/)
|
||||
+3
-2
@@ -1,8 +1,9 @@
|
||||
---
|
||||
title: Project Monitoring
|
||||
weight: 2528
|
||||
weight: 2
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring
|
||||
- /rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x/monitoring/project-monitoring
|
||||
---
|
||||
|
||||
_Available as of v2.2.4_
|
||||
+2
-1
@@ -2,8 +2,9 @@
|
||||
title: Prometheus Configuration
|
||||
weight: 1
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring/prometheus
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring/prometheus
|
||||
- /rancher/v2.x/en/cluster-admin/tools/monitoring/prometheus/
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/prometheus
|
||||
---
|
||||
|
||||
_Available as of v2.2.0_
|
||||
+3
-2
@@ -2,8 +2,9 @@
|
||||
title: Viewing Metrics
|
||||
weight: 2
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/monitoring/viewing-metrics
|
||||
- rancher/v2.x/en/cluster-admin/tools/monitoring/viewing-metrics
|
||||
- /rancher/v2.x/en/project-admin/tools/monitoring/viewing-metrics
|
||||
- /rancher/v2.x/en/cluster-admin/tools/monitoring/viewing-metrics
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/viewing-metrics
|
||||
---
|
||||
|
||||
_Available as of v2.2.0_
|
||||
+3
-3
@@ -1,10 +1,10 @@
|
||||
---
|
||||
title: Notifiers
|
||||
weight: 1
|
||||
weight: 4
|
||||
aliases:
|
||||
- rancher/v2.x/en/project-admin/tools/notifiers
|
||||
- rancher/v2.x/en/cluster-admin/tools/notifiers
|
||||
- /rancher/v2.x/en/project-admin/tools/notifiers
|
||||
- /rancher/v2.x/en/cluster-admin/tools/notifiers
|
||||
- /rancher/v2.x/en/monitoring-alerting/legacy/notifiers
|
||||
---
|
||||
|
||||
> In Rancher 2.5, the notifier application was improved. There are now two ways to enable notifiers. The older way is documented in this section, and the new application for notifiers is documented [here.]({{<baseurl>}}/rancher/v2.x/en/monitoring-alerting)
|
||||
@@ -0,0 +1,187 @@
|
||||
---
|
||||
title: Monitoring in Rancher v2.5
|
||||
shortTitle: Rancher v2.5
|
||||
weight: 1
|
||||
---
|
||||
|
||||
|
||||
Using Rancher, you can quickly deploy leading open-source monitoring & alerting solutions such as [Prometheus](https://prometheus.io/), [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), and [Grafana](https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/) onto your cluster.
|
||||
|
||||
Rancher's solution (powered by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator)) allows users to:
|
||||
|
||||
- Monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments via [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
|
||||
|
||||
- Defines alerts based on metrics collected via [Prometheus](https://prometheus.io/)
|
||||
- Creates custom dashboards to make it easy to visualize collected metrics via [Grafana](https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/)
|
||||
- Configures alert-based notifications via Email, Slack, PagerDuty, etc. using [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/)
|
||||
- Defines precomputed frequently needed / computationally expensive expressions as new time series based on metrics collected via [Prometheus](https://prometheus.io/) (only available in 2.5.x)
|
||||
- Exposes collected metrics from Prometheus to the Kubernetes Custom Metrics API via [Prometheus Adapter](https://github.com/DirectXMan12/k8s-prometheus-adapter) for use in HPA (only available in 2.5)
|
||||
|
||||
More information about the resources that get deployed onto your cluster to support this solution can be found in the [`rancher-monitoring`](https://github.com/rancher/charts/tree/main/charts/rancher-monitoring) Helm chart, which closely tracks the upstream [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) Helm chart maintained by the Prometheus community with certain changes tracked in the [CHANGELOG.md](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/CHANGELOG.md).
|
||||
|
||||
This page describes how to enable monitoring & alerting within a cluster using Rancher's new monitoring application, which was introduced in Rancher v2.5.
|
||||
|
||||
If you previously enabled Monitoring, Alerting, or Notifiers in Rancher prior to v2.5, there is no upgrade path for switching to the new monitoring/ alerting solution. You will need to disable monitoring/ alerting/notifiers in Cluster Manager before deploying the new monitoring solution via Cluster Explorer.
|
||||
|
||||
For more information about upgrading the Monitoring app in Rancher 2.5, please refer to the [migration docs](./migrating).
|
||||
|
||||
> Before enabling monitoring, be sure to review the resource requirements. The default values in [this section](#setting-resource-limits-and-requests) are the minimum required resource limits and requests.
|
||||
|
||||
- [Monitoring Components](#monitoring-components)
|
||||
- [Prometheus](#about-prometheus)
|
||||
- [Grafana](#about-grafana)
|
||||
- [Alertmanager](#about-alertmanager)
|
||||
- [Prometheus Operator](#about-prometheus-operator)
|
||||
- [Prometheus Adapter](#about-prometheus-adapter)
|
||||
- [Enable Monitoring](#enable-monitoring)
|
||||
- [Default Alerts, Targets and Grafana Dashboards](#default-alerts-targets-and-grafana-dashboards)
|
||||
- [Using Monitoring](#using-monitoring)
|
||||
- [Grafana UI](#grafana-ui)
|
||||
- [Prometheus UI](#prometheus-ui)
|
||||
- [Viewing the Prometheus Targets](#viewing-the-prometheus-targets)
|
||||
- [Viewing the Prometheus Rules](#viewing-the-prometheus-rules)
|
||||
- [Viewing Active Alerts in Alertmanager](#viewing-active-alerts-in-alertmanager)
|
||||
- [Uninstall Monitoring](#uninstall-monitoring)
|
||||
- [Setting Resource Limits and Requests](#setting-resource-limits-and-requests)
|
||||
- [Known Issues](#known-issues)
|
||||
|
||||
# Monitoring Components
|
||||
|
||||
The `rancher-monitoring` operator is powered by Prometheus, Grafana, Alertmanager, the Prometheus Operator, and the Prometheus adapter.
|
||||
|
||||
### About Prometheus
|
||||
|
||||
Prometheus provides a time series of your data, which is, according to the [Prometheus documentation:](https://prometheus.io/docs/concepts/data_model/)
|
||||
|
||||
> A stream of timestamped values belonging to the same metric and the same set of labeled dimensions, along with comprehensive statistics and metrics of the monitored cluster.
|
||||
|
||||
In other words, Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or Grafana, which is an analytics viewing platform deployed along with Prometheus.
|
||||
|
||||
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc.
|
||||
|
||||
### About Grafana
|
||||
|
||||
[Grafana](https://grafana.com/grafana/) allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data driven culture.
|
||||
|
||||
# Enabling Cluster Monitoring
|
||||
|
||||
As an [administrator]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), you can configure Rancher to deploy Prometheus to monitor your Kubernetes cluster.
|
||||
|
||||
> If you want to set up Alertmanager, Grafana or Ingress, it has to be done with the settings on the Helm chart deployment. It's problematic to create Ingress outside the deployment.
|
||||
|
||||
> **Prerequisites:**
|
||||
>
|
||||
> - Make sure that you are allowing traffic on port 9796 for each of your nodes because Prometheus will scrape metrics from here.
|
||||
> - Make sure your cluster fulfills the resource requirements. The cluster should have at least 1950Mi memory available, 2700m CPU, and 50Gi storage. A breakdown of the resource limits and requests is [here.](#resource-requirements)
|
||||
|
||||
1. In the Rancher UI, go to the cluster where you want to install monitoring and click **Cluster Explorer.**
|
||||
1. Click **Apps.**
|
||||
1. Click the `rancher-monitoring` app.
|
||||
1. Optional: Click **Chart Options** and configure alerting, Prometheus and Grafana. For help, refer to the [configuration reference.](./configuration)
|
||||
1. Scroll to the bottom of the Helm chart README and click **Install.**
|
||||
|
||||
**Result:** The monitoring app is deployed in the `cattle-monitoring-system` namespace.
|
||||
|
||||
### Default Alerts, Targets and Grafana Dashboards
|
||||
|
||||
By default, Rancher Monitoring deploys exporters (such as [node-exporter](https://github.com/prometheus/node_exporter) and [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)) as well as default Prometheus alerts and Grafana dashboards (curated by the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project) onto a cluster.
|
||||
|
||||
To see the default alerts, go to the [Alertmanager UI](#alertmanager-ui) and click **Expand all groups.**
|
||||
|
||||
To see what services you are monitoring, you will need to see your targets. To view the default targets, refer to [Viewing the Prometheus Targets.](#viewing-the-prometheus-targets)
|
||||
|
||||
To see the default dashboards, go to the [Grafana UI.](#grafana-ui) In the left navigation bar, click the icon with four boxes and click **Manage.**
|
||||
|
||||
### Next Steps
|
||||
|
||||
To configure Prometheus resources from the Rancher UI, click **Apps & Marketplace > Monitoring** in the upper left corner.
|
||||
|
||||
# Using Monitoring
|
||||
|
||||
Installing `rancher-monitoring` makes the following dashboards available from the Rancher UI.
|
||||
|
||||
### Grafana UI
|
||||
|
||||
Rancher allows any users who are authenticated by Kubernetes and have access the Grafana service deployed by the Rancher Monitoring chart to access Grafana via the Rancher Dashboard UI. By default, all users who are able to access Grafana are given the [Viewer](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#viewer-role) role, which allows them to view any of the default dashboards deployed by Rancher.
|
||||
|
||||
However, users can choose to log in to Grafana as an [Admin](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#admin-role) if necessary. The default Admin username and password for the Grafana instance will be `admin`/`prom-operator`, but alternative credentials can also be supplied on deploying or upgrading the chart.
|
||||
|
||||
To see the Grafana UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Grafana.
|
||||
|
||||
<figcaption>Cluster Compute Resources Dashboard in Grafana</figcaption>
|
||||
|
||||
|
||||
<figcaption>Default Dashboards in Grafana</figcaption>
|
||||
|
||||
|
||||
To allow the Grafana dashboard to persist after it restarts, you will need to add the configuration JSON into a ConfigMap. You can add this configuration to the ConfigMap using the Rancher UI.
|
||||
|
||||
### Prometheus UI
|
||||
|
||||
To see the Prometheus UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Graph.**
|
||||
|
||||
<figcaption>Prometheus Graph UI</figcaption>
|
||||
|
||||
|
||||
### Viewing the Prometheus Targets
|
||||
|
||||
To see the Prometheus Targets, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Targets.**
|
||||
|
||||
<figcaption>Targets in the Prometheus UI</figcaption>
|
||||
|
||||
|
||||
### Viewing the Prometheus Rules
|
||||
|
||||
To see the Prometheus Rules, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Rules.**
|
||||
|
||||
<figcaption>Rules in the Prometheus UI</figcaption>
|
||||
|
||||
|
||||
### Viewing Active Alerts in Alertmanager
|
||||
|
||||
When `rancher-monitoring` is installed, the Prometheus Alertmanager UI is deployed.
|
||||
|
||||
The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.
|
||||
|
||||
In the Alertmanager UI, you can view your alerts and the current Alertmanager configuration.
|
||||
|
||||
To see the Prometheus Rules, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Alertmanager.**
|
||||
|
||||
**Result:** The Alertmanager UI opens in a new tab. For help with configuration, refer to the [official Alertmanager documentation.](https://prometheus.io/docs/alerting/latest/alertmanager/)
|
||||
|
||||
<figcaption>The Alertmanager UI</figcaption>
|
||||
|
||||
|
||||
# Uninstall Monitoring
|
||||
|
||||
1. From the **Cluster Explorer,** click Apps & Marketplace.
|
||||
1. Click **Installed Apps.**
|
||||
1. Go to the `cattle-monitoring-system` namespace and check the boxes for `rancher-monitoring-crd` and `rancher-monitoring`.
|
||||
1. Click **Delete.**
|
||||
1. Confirm **Delete.**
|
||||
|
||||
**Result:** `rancher-monitoring` is uninstalled.
|
||||
|
||||
# Setting Resource Limits and Requests
|
||||
|
||||
The resource requests and limits can be configured when installing `rancher-monitoring`.
|
||||
|
||||
The default values are in the [values.yaml](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/values.yaml) in the `rancher-monitoring` Helm chart.
|
||||
|
||||
The default values in the table below are the minimum required resource limits and requests.
|
||||
|
||||
| Resource Name | Memory Limit | CPU Limit | Memory Request | CPU Request |
|
||||
| ------------- | ------------ | ----------- | ---------------- | ------------------ |
|
||||
| alertmanager | 500Mi | 1000m | 100Mi | 100m |
|
||||
| grafana | 200Mi | 200m | 100Mi | 100m |
|
||||
| kube-state-metrics subchart | 200Mi | 100m | 130Mi | 100m |
|
||||
| prometheus-node-exporter subchart | 50Mi | 200m | 30Mi | 100m |
|
||||
| prometheusOperator | 500Mi | 200m | 100Mi | 100m |
|
||||
| prometheus | 2500Mi | 1000m | 1750Mi | 750m |
|
||||
| **Total** | **3950Mi** | **2700m** | **2210Mi** | **1250m** |
|
||||
|
||||
At least 50Gi storage is recommended.
|
||||
|
||||
# Known Issues
|
||||
|
||||
There is a [known issue](https://github.com/rancher/rancher/issues/28787#issuecomment-693611821) that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend to setting `prometheus.prometheusSpec.resources.memory.limit` to 2500Mi` and `prometheus.prometheusSpec.resources.memory.request` to 1750Mi.
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: Configuration
|
||||
weight: 3
|
||||
aliases:
|
||||
- /rancher/v2.x/en/monitoring-alerting/configuration
|
||||
---
|
||||
|
||||
This page captures some of the most important options for configuring the custom resources for monitoring.
|
||||
+3
-1
@@ -1,13 +1,15 @@
|
||||
---
|
||||
title: Migrating to Rancher v2.5 Monitoring
|
||||
weight: 5
|
||||
aliases:
|
||||
- /rancher/v2.x/en/monitoring-alerting/migrating
|
||||
---
|
||||
|
||||
If you previously enabled Monitoring, Alerting, or Notifiers in Rancher prior to v2.5, there is no upgrade path for switching to the new monitoring/alerting solution. You will need to disable monitoring/alerting/notifiers in the same way it was disabled in Rancher v2.4 before deploying the new monitoring solution via Cluster Explorer.
|
||||
|
||||
### Monitoring Prior to Rancher v2.5
|
||||
|
||||
As of v2.2.0, Rancher's Cluster Manager allowed users to enable Monitoring & Alerting V1 (both powered by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator)) independently within a cluster. For more information on how to configure Monitoring & Alerting V1, see the [docs about monitoring prior to Rancher v2.5](/rancher/v2.x/en/monitoring-alerting/legacy).
|
||||
As of v2.2.0, Rancher's Cluster Manager allowed users to enable Monitoring & Alerting V1 (both powered by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator)) independently within a cluster. For more information on how to configure Monitoring & Alerting V1, see the [docs about monitoring prior to Rancher v2.5](/rancher/v2.x/en/monitoring-alerting/v2.0.x-v2.4.x).
|
||||
|
||||
When Monitoring is enabled, Monitoring V1 deploys [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/) onto a cluster to monitor the state of processes of your cluster nodes, Kubernetes components, and software deployments and create custom dashboards to make it easy to visualize collected metrics.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user