Merge branch 'staging' into grafana

2026-05-25 06:08:29 +00:00 · 2021-04-30 10:28:30 -07:00
parent 8e12f3c31b be3f3d224a
commit 5deb78dc56
48 changed files with 937 additions and 210 deletions
@@ -1,6 +1,14 @@
 Rancher Docs
 ------------

+## Contributing
+
+We have transitioned to versioned documentation for Rancher (files within `content/rancher`).
+
+New contributions should be made to the applicable versioned directories (e.g. `content/rancher/v2.5` and `content/rancher/v2.0-v2.4`).
+
+Contents under the `content/rancher/v2.x` directory are no longer maintained after v2.5.6.
+
 ## Running for development/editing

 The `rancher/docs:dev` docker image runs a live-updating server.  To run on your workstation, run:
@@ -21,6 +21,7 @@ This section contains advanced information describing the different ways you can
 - [Enabling legacy iptables on Raspbian Buster](#enabling-legacy-iptables-on-raspbian-buster)
 - [Enabling cgroups for Raspbian Buster](#enabling-cgroups-for-raspbian-buster)
 - [SELinux Support](#selinux-support)
+- [Additional preparation for (Red Hat/CentOS) Enterprise Linux](#additional-preparation-for-red-hat-centos-enterprise-linux)

 # Certificate Rotation

@@ -228,7 +229,7 @@ $ k3s server
 INFO[2019-01-22T15:16:19.908493986-07:00] Starting k3s dev                             
 INFO[2019-01-22T15:16:19.908934479-07:00] Running kube-apiserver --allow-privileged=true --authorization-mode Node,RBAC --service-account-signing-key-file /var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range 10.43.0.0/16 --advertise-port 6445 --advertise-address 127.0.0.1 --insecure-port 0 --secure-port 6444 --bind-address 127.0.0.1 --tls-cert-file /var/lib/rancher/k3s/server/tls/localhost.crt --tls-private-key-file /var/lib/rancher/k3s/server/tls/localhost.key --service-account-key-file /var/lib/rancher/k3s/server/tls/service.key --service-account-issuer k3s --api-audiences unknown --basic-auth-file /var/lib/rancher/k3s/server/cred/passwd --kubelet-client-certificate /var/lib/rancher/k3s/server/tls/token-node.crt --kubelet-client-key /var/lib/rancher/k3s/server/tls/token-node.key 
 Flag --insecure-port has been deprecated, This flag will be removed in a future version.
-INFO[2019-01-22T15:16:20.196766005-07:00] Running kube-scheduler --kubeconfig /var/lib/rancher/k3s/server/cred/kubeconfig-system.yaml --port 0 --secure-port 0 --leader-elect=false 
+INFO[2019-01-22T15:16:20.196766005-07:00] Running kube-scheduler --kubeconfig /var/lib/rancher/k3s/server/cred/kubeconfig-system.yaml --port 0 --secure-port 0 --leader-elect=false
 INFO[2019-01-22T15:16:20.196880841-07:00] Running kube-controller-manager --kubeconfig /var/lib/rancher/k3s/server/cred/kubeconfig-system.yaml --service-account-private-key-file /var/lib/rancher/k3s/server/tls/service.key --allocate-node-cidrs --cluster-cidr 10.42.0.0/16 --root-ca-file /var/lib/rancher/k3s/server/tls/token-ca.crt --port 0 --secure-port 0 --leader-elect=false 
 Flag --port has been deprecated, see --secure-port instead.
 INFO[2019-01-22T15:16:20.273441984-07:00] Listening on :6443                           
@@ -366,3 +367,10 @@ Using a custom `--data-dir` under SELinux is not supported. To customize it, you

 {{%/tab%}}
 {{% /tabs %}}
+
+# Additional preparation for (Red Hat/CentOS) Enterprise Linux
+
+It is recommended to turn off firewalld:
+```
+systemctl disable firewalld --now
+```
@@ -13,6 +13,8 @@ This section contains instructions for installing K3s in various environments. P

 [Air-Gap Installation]({{<baseurl>}}/k3s/latest/en/installation/airgap/) details how to set up K3s in environments that do not have direct access to the Internet.

+[Disable Components Flags]({{<baseurl>}}/k3s/latest/en/installation/disable-flags/) details how to set up K3s with etcd only nodes and controlplane only nodes
+
 ### Uninstalling

 If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation. The script is created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`).
@@ -0,0 +1,73 @@
+---
+title: "Disable Components Flags"
+weight: 60
+---
+
+When starting K3s server with --cluster-init it will run all control plane components that includes (api server, controller manager, scheduler, and etcd). However you can run server nodes with certain components and execlude others, the following sectiohs will explain how to do that.
+
+# ETCD Only Nodes
+
+This document assumes you run K3s server with embedded etcd by passing `--cluster-init` flag to the server process.
+
+To run a K3s server with only etcd components you can pass `--disable-api-server --disable-controller-manager --disable-scheduler` flags to k3s, this will result in running a server node with only etcd, for example to run K3s server with those flags:
+
+```
+curl -fL https://get.k3s.io | sh -s - server --cluster-init --disable-api-server --disable-controller-manager --disable-scheduler
+```
+
+You can join other nodes to the cluster normally after that.
+
+# Disable ETCD
+
+You can also disable etcd from a server node and this will result in a k3s server running control components other than etcd, that can be accomplished by running k3s server with flag `--disable-etcd` for example to join another node with only control components to the etcd node created in the previous section:
+
+```
+curl -fL https://get.k3s.io | sh -s - server --token <token> --disable-etcd --server https://<etcd-only-node>:6443 
+```
+
+The end result will be a two nodes one of them is etcd only node and the other one is controlplane only node, if you check the node list you should see something like the following:
+
+```
+kubectl get nodes
+NAME              STATUS   ROLES                       AGE     VERSION
+ip-172-31-13-32   Ready    etcd                        5h39m   v1.20.4+k3s1
+ip-172-31-14-69   Ready    control-plane,master        5h39m   v1.20.4+k3s1
+```
+
+Note that you can run `kubectl` commands only on the k3s server that has the api running, and you cant run `kubectl` commands on etcd only nodes.
+
+
+### Re-enabling control components
+
+In both cases you can re-enable any component that you already disabled simply by removing the corresponding flag that disables them, so for example if you want to revert the etcd only node back to a full k3s server with all components you can just remove the following 3 flags `--disable-api-server --disable-controller-manager --disable-scheduler`, so in our example to revert back node `ip-172-31-13-32` to a full k3s server you can just re-run the curl command without the disable flags:
+```
+curl -fL https://get.k3s.io | sh -s - server --cluster-init
+``` 
+
+you will notice that all components started again and you can run kubectl commands again:
+
+```
+kubectl get nodes
+NAME              STATUS   ROLES                       AGE     VERSION
+ip-172-31-13-32   Ready    control-plane,etcd,master   5h45m   v1.20.4+k3s1
+ip-172-31-14-69   Ready    control-plane,master        5h45m   v1.20.4+k3s1
+```
+
+Notice that role labels has been re-added to the node `ip-172-31-13-32` with the correct labels (control-plane,etcd,master).
+
+# Add disable flags using the config file
+
+In any of the previous situation you can use the config file instead of running the curl commands with the associated flags, for example to run an etcd only node you can add the following options to the `/etc/rancher/k3s/config.yaml` file:
+
+```
+---
+disable-api-server: true
+disable-controller-manager: true
+disable-scheduler: true
+cluster-init: true
+```
+and then start K3s using the curl command without any arguents:
+
+```
+curl -fL https://get.k3s.io | sh -
+```
@@ -23,6 +23,7 @@ Some OSs have specific requirements:

 - If you are using **Raspbian Buster**, follow [these steps]({{<baseurl>}}/k3s/latest/en/advanced/#enabling-legacy-iptables-on-raspbian-buster) to switch to legacy iptables.
 - If you are using **Alpine Linux**, follow [these steps]({{<baseurl>}}/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup) for additional setup.
+- If you are using **(Red Hat/CentOS) Enterprise Linux**, follow [these steps]({{<baseurl>}}/k3s/latest/en/advanced/#additional-preparation-for-red-hat-centos-enterprise-linux) for additional setup.

 For more information on which OSs were tested with Rancher managed K3s clusters, refer to the [Rancher support and maintenance terms.](https://rancher.com/support-maintenance-terms/)

@@ -53,7 +53,7 @@ sudo k3s kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role
 ### Obtain the Bearer Token

 ```bash
-sudo k3s kubectl -n kubernetes-dashboard describe secret admin-user-token | grep ^token
+sudo k3s kubectl -n kubernetes-dashboard describe secret admin-user-token | grep '^token'
 ```

 ### Local Access to the Dashboard
@@ -18,7 +18,7 @@ aliases:
 ---
 The following instructions will guide you through upgrading a Rancher server that was installed on a Kubernetes cluster with Helm. These steps also apply to air gap installs with Helm.

-For the instructions to upgrade Rancher installed with Docker, refer to [ths page.]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)
+For the instructions to upgrade Rancher installed with Docker, refer to [this page.]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)

 To upgrade the components in your Kubernetes cluster, or the definition of the [Kubernetes services]({{<baseurl>}}/rke/latest/en/config-options/services/) or [add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/), refer to the [upgrade documentation for RKE]({{<baseurl>}}/rke/latest/en/upgrades/), the Rancher Kubernetes Engine.

@@ -62,7 +62,7 @@ The following tables break down the port requirements for inbound and outbound t
 | Protocol | Port | Destination                                              | Description                                   |
 | -------- | ---- | -------------------------------------------------------- | --------------------------------------------- |
 | TCP      | 22   | Any node IP from a node created using Node Driver        | SSH provisioning of nodes using Node Driver   |
-| TCP      | 443  | `35.160.43.145/32`, `35.167.242.46/32`, `52.33.59.17/32` | git.rancher.io (catalogs)                     |
+| TCP      | 443  | git.rancher.io |  Rancher catalog                    |
 | TCP      | 2376 | Any node IP from a node created using Node driver        | Docker daemon TLS port used by Docker Machine |
 | TCP      | 6443 | Hosted/Imported Kubernetes API                           | Kubernetes API server                         |

@@ -130,7 +130,7 @@ The following tables break down the port requirements for Rancher nodes, for inb
 | Protocol | Port | Source | Description |
 |-----|-----|----------------|---|
 | TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
-| TCP | 443 | `35.160.43.145/32`,`35.167.242.46/32`,`52.33.59.17/32` | git.rancher.io (catalogs) |
+| TCP | 443 | git.rancher.io |  Rancher catalog |
 | TCP | 2376 | Any node IP from a node created using a node driver | Docker daemon TLS port used by Docker Machine |
 | TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |

@@ -45,14 +45,14 @@ If the Rancher server is installed in a single Docker container, you only need o
 ```
 sudo ssh -i [path-to-private-key] ubuntu@[public-DNS-of-instance]
 ```
-1. When you are connected to the instance, run the following command on the instance to create a user:
-```
-sudo usermod -aG docker ubuntu
-```
 1. Run the following command on the instance to install Docker with one of Rancher's installation scripts:
 ```
 curl https://releases.rancher.com/install-docker/18.09.sh | sh
 ```
+1. When you are connected to the instance, run the following command on the instance to create a user:
+```
+sudo usermod -aG docker ubuntu
+```
 1. Repeat these steps so that Docker is installed on each node that will eventually run the Rancher management server.

 > To find out whether a script is available for installing a certain Docker version, refer to this [GitHub repository,](https://github.com/rancher/install-docker) which contains all of Rancher’s Docker installation scripts.
@@ -14,7 +14,7 @@ Double check if all the [required ports]({{<baseurl>}}/rancher/v2.0-v2.4/en/clus

 The pod can be scheduled to any of the hosts you used for your cluster, but that means that the NGINX ingress controller needs to be able to route the request from `NODE_1` to `NODE_2`. This happens over the overlay network. If the overlay network is not functioning, you will experience intermittent TCP/HTTP connection failures due to the NGINX ingress controller not being able to route to the pod.

-To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/leodotcloud/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
+To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.

 1. Save the following file as `overlaytest.yml`

@@ -35,7 +35,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
          tolerations:
          - operator: Exists
          containers:
-          - image: leodotcloud/swiss-army-knife
+          - image: rancherlabs/swiss-army-knife
            imagePullPolicy: Always
            name: overlaytest
            command: ["sh", "-c", "tail -f /dev/null"]
@@ -51,5 +51,5 @@ After you complete [Configuring Microsoft AD FS for Rancher]({{<baseurl>}}/ranch
 **Tip:** You can generate a certificate using an openssl command. For example:

 ```
-openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
+openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=https://myservice.example.com"
 ```
@@ -29,6 +29,7 @@ This section covers the following topics:
 # Node Options Available for Each Cluster Creation Option

 The following table lists which node options are available for each type of cluster in Rancher. Click the links in the **Option** column for more detailed information about each feature.
+
 | Option                                           | [Nodes Hosted by an Infrastructure Provider][1]                                   | [Custom Node][2] | [Hosted Cluster][3] | [Registered EKS Nodes][4] | [All Other Registered Nodes][5] | Description                                                        |
 | ------------------------------------------------ | ------------------------------------------------ | ---------------- | ------------------- | ------------------- | -------------------| ------------------------------------------------------------------ |
 | [Cordon](#cordoning-a-node)                      | ✓                                                | ✓                | ✓                   | ✓                   | ✓                  | Marks the node as unschedulable.                                   |
@@ -18,6 +18,7 @@ This section covers the following topics:

 <!-- TOC -->

+- [Cluster Management Capabilities by Cluster Type](#cluster-management-capabilities-by-cluster-type)
 - [Setting up clusters in a hosted Kubernetes provider](#setting-up-clusters-in-a-hosted-kubernetes-provider)
 - [Launching Kubernetes with Rancher](#launching-kubernetes-with-rancher)
  - [Launching Kubernetes and Provisioning Nodes in an Infrastructure Provider](#launching-kubernetes-and-provisioning-nodes-in-an-infrastructure-provider)
@@ -27,6 +28,8 @@ This section covers the following topics:

  <!-- /TOC -->

+### Cluster Management Capabilities by Cluster Type
+
 The following table summarizes the options and settings available for each cluster type:

 {{% include file="/rancher/v2.5/en/cluster-provisioning/cluster-capabilities-table" %}}
@@ -1,22 +1,59 @@
 ---
 headless: true
 ---
-| Action | [Rancher launched Kubernetes Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/) | [Hosted Kubernetes Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/) | [Imported Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/imported-clusters) |
-| --- | --- | ---| ---|
-| [Using kubectl and a kubeconfig file to Access a Cluster]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/) | ✓ | ✓ | ✓ |
-| [Managing Cluster Members]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/cluster-members/) | ✓ | ✓ | ✓ |
-| [Editing and Upgrading Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/editing-clusters/) | ✓ | ✓ | * |
-| [Managing Nodes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/nodes) | ✓ | ✓ | ✓ |
-| [Managing Persistent Volumes and Storage Classes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/) | ✓ | ✓ | ✓ |
-| [Managing Projects, Namespaces and Workloads]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/) | ✓ | ✓ | ✓ |
-| [Using App Catalogs]({{<baseurl>}}/rancher/v2.5/en/catalog/) | ✓ | ✓ | ✓ |
-| [Configuring Tools (Alerts, Notifiers, Logging, Monitoring, Istio)]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/tools/) | ✓ | ✓ | ✓ |
-| [Cloning Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cloning-clusters/)| ✓ | ✓ | |
-| [Ability to rotate certificates]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/certificate-rotation/) | ✓ |  | |
-| [Ability to back up your Kubernetes Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/backing-up-etcd/) | ✓ | | |
-| [Ability to recover and restore etcd]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/restoring-etcd/) | ✓ | | |
-| [Cleaning Kubernetes components when clusters are no longer reachable from Rancher]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cleaning-cluster-nodes/) | ✓ | | |
-| [Configuring Pod Security Policies]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/pod-security-policy/) | ✓ |  | |
-| [Running Security Scans]({{<baseurl>}}/rancher/v2.5/en/security/security-scan/) | ✓ |  | |

-\* Cluster configuration options can't be edited for imported clusters, except for [K3s clusters.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/imported-clusters/)
+{{% tabs %}}
+{{% tab "Rancher v2.5.8" %}}
+
+| Action | Rancher Launched Kubernetes Clusters |  EKS and GKE Clusters* | Other Hosted Kubernetes Clusters | Non-EKS or GKE Registered Clusters |
+| --- | --- | ---| ---|----|
+| [Using kubectl and a kubeconfig file to Access a Cluster]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/) | ✓ | ✓ | ✓ | ✓ |
+| [Managing Cluster Members]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/cluster-members/) | ✓ | ✓ | ✓ | ✓ |
+| [Editing and Upgrading Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/editing-clusters/) | ✓ | ✓ | ✓ | ** |
+| [Managing Nodes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/nodes) | ✓ | ✓ | ✓ | ✓ *** |
+| [Managing Persistent Volumes and Storage Classes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/) | ✓ | ✓ | ✓ | ✓ |
+| [Managing Projects, Namespaces and Workloads]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/) | ✓ | ✓ | ✓ | ✓ |
+| [Using App Catalogs]({{<baseurl>}}/rancher/v2.5/en/catalog/) | ✓ | ✓ | ✓ | ✓ |
+| [Configuring Tools (Alerts, Notifiers, Logging, Monitoring, Istio)]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/tools/) | ✓ | ✓ | ✓ | ✓ |
+| [Running Security Scans]({{<baseurl>}}/rancher/v2.5/en/security/security-scan/) | ✓ | ✓ | ✓ | ✓ |
+| [Cloning Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cloning-clusters/)| ✓ | ✓ |✓ | |
+| [Ability to rotate certificates]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/certificate-rotation/) | ✓ | ✓  |  | |
+| [Ability to back up your Kubernetes Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/backing-up-etcd/) | ✓ | ✓ |  | |
+| [Ability to recover and restore etcd]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/restoring-etcd/) | ✓ |  ✓ |  | |
+| [Cleaning Kubernetes components when clusters are no longer reachable from Rancher]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cleaning-cluster-nodes/) | ✓ | | | |
+| [Configuring Pod Security Policies]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/pod-security-policy/) | ✓ | ✓ |  ||
+
+\* Registered GKE and EKS clusters have the same options available as GKE and EKS clusters created from the Rancher UI. The  difference is that when a registered cluster is deleted from the Rancher UI, [it is not destroyed.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/registered-clusters/#additional-features-for-registered-eks-and-gke-clusters)
+
+\* \* Cluster configuration options can't be edited for imported clusters, except for [K3s and RKE2 clusters.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/imported-clusters/)
+
+\* \* \* For registered cluster nodes, the Rancher UI exposes the ability to cordon drain, and edit the node.
+
+{{% /tab %}}
+{{% tab "Rancher v2.5.0-v2.5.7" %}}
+
+| Action | Rancher Launched Kubernetes Clusters | Hosted Kubernetes Clusters | Registered EKS Clusters | All Other Registered Clusters |
+| --- | --- | ---| ---|----|
+| [Using kubectl and a kubeconfig file to Access a Cluster]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/) | ✓ | ✓ | ✓ | ✓ |
+| [Managing Cluster Members]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/cluster-members/) | ✓ | ✓ | ✓ | ✓ |
+| [Editing and Upgrading Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/editing-clusters/) | ✓ | ✓ | ✓ | * |
+| [Managing Nodes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/nodes) | ✓ | ✓ | ✓ | ✓ ** |
+| [Managing Persistent Volumes and Storage Classes]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/) | ✓ | ✓ | ✓ | ✓ |
+| [Managing Projects, Namespaces and Workloads]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/) | ✓ | ✓ | ✓ | ✓ |
+| [Using App Catalogs]({{<baseurl>}}/rancher/v2.5/en/catalog/) | ✓ | ✓ | ✓ | ✓ |
+| [Configuring Tools (Alerts, Notifiers, Logging, Monitoring, Istio)]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/tools/) | ✓ | ✓ | ✓ | ✓ |
+| [Running Security Scans]({{<baseurl>}}/rancher/v2.5/en/security/security-scan/) | ✓ | ✓ | ✓ | ✓ |
+| [Cloning Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cloning-clusters/)| ✓ | ✓ |✓ | |
+| [Ability to rotate certificates]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/certificate-rotation/) | ✓ |  | ✓ | |
+| [Ability to back up your Kubernetes Clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/backing-up-etcd/) | ✓ | | ✓ | |
+| [Ability to recover and restore etcd]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/restoring-etcd/) | ✓ | | ✓ | |
+| [Cleaning Kubernetes components when clusters are no longer reachable from Rancher]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cleaning-cluster-nodes/) | ✓ | | | |
+| [Configuring Pod Security Policies]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/pod-security-policy/) | ✓ |  | ✓ ||
+
+\* Cluster configuration options can't be edited for imported clusters, except for [K3s and RKE2 clusters.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/imported-clusters/)
+
+\* \* For registered cluster nodes, the Rancher UI exposes the ability to cordon drain, and edit the node.
+
+
+{{% /tab %}}
+{{% /tabs %}}
@@ -712,30 +712,31 @@ Permissions required for Rancher to create VPC and associated resources.
 }
 ```

-
 # Syncing

-Syncing is the feature that causes Rancher to update its EKS clusters' values so they are up to date with their corresponding cluster object in the EKS console. This enables Rancher to not be the sole owner of an EKS cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other.
-
-### How it works
-
-There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
-
-1. EKSConfig which is located on the Spec of the Cluster.
-2. UpstreamSpec which is located on the EKSStatus field on the Status of the Cluster.
-
-Both of which are defined by the struct EKSClusterConfigSpec found in the eks-operator project: https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go
-
-All fields with the exception of DisplayName, AmazonCredentialSecret, Region, and Imported are nillable on the EKSClusterConfigSpec.
-
-The EKSConfig represents desired state for its non-nil values. Fields that are non-nil in the EKSConfig can be thought of as “managed".When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher.
-
-UpstreamSpec represents the cluster as it is in EKS and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed rancher checks if the EKS cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
-
-The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig. This is what is displayed in the UI.
-
-If Rancher and another source attempt to update an EKS cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. For example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioned case.
+The EKS provisioner can synchronize the state of an EKS cluster between Rancher and the provider. For an in-depth technical explanation of how this works, see [Syncing.](../syncing)

 ### Configuring the Refresh Interval

-It is possible to change the refresh interval through the setting “eks-refresh-cron". This setting accepts values in the Cron format. The default is `*/5 * * * *`. The shorter the refresh window is the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
+{{% tabs %}}
+{{% tab "Rancher v2.5.8+" %}}
+
+The `eks-refresh-cron` setting is deprecated. It has been migrated to the `eks-refresh` setting, which is an integer representing seconds.
+
+The default value is 300 seconds.
+
+The syncing interval can be changed by running `kubectl edit setting eks-refresh`.
+
+If the `eks-refresh-cron` setting was previously set, the migration will happen automatically.
+
+The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
+
+{{% /tab %}}
+{{% tab "Before v2.5.8" %}}
+
+It is possible to change the refresh interval through the setting `eks-refresh-cron`. This setting accepts values in the Cron format. The default is `*/5 * * * *`. 
+
+The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
+
+{{% /tab %}}
+{{% /tabs %}}
@@ -1,5 +1,5 @@
 ---
-title: Creating a GKE Cluster
+title: Managing GKE Clusters
 shortTitle: Google Kubernetes Engine
 weight: 2105
 aliases:
@@ -9,6 +9,13 @@ aliases:
 {{% tabs %}}
 {{% tab "Rancher v2.5.8+" %}}

+- [Prerequisites](#prerequisites)
+- [Provisioning a GKE Cluster](#provisioning-a-gke-cluster)
+- [Private Clusters](#private-clusters)
+- [Configuration Reference](./config-reference)
+- [Updating Kubernetes Version](#updating-kubernetes-version)
+- [Syncing](#syncing)
+
 # Prerequisites

 Some setup in Google Kubernetes Engine is required.
@@ -36,10 +43,12 @@ To create a new project, refer to the Google cloud documentation [here.](https:/

 To get the project ID of an existing project, refer to the Google cloud documentation [here.](https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects)

+# Provisioning a GKE Cluster
+
 >**Note**
 >Deploying to GKE will incur charges.

-# 1. Create a Cloud Credential
+### 1. Create a Cloud Credential

 1. In the upper right corner, click the user profile dropdown menu and click **Cloud Credentials.**
 1. Click **Add Cloud Credential.**
@@ -50,7 +59,7 @@ To get the project ID of an existing project, refer to the Google cloud document

 **Result:** You have created credentials that Rancher will use to provision the new GKE cluster.

-# 2. Create the GKE Cluster
+### 2. Create the GKE Cluster
 Use Rancher to set up and configure your Kubernetes cluster.

 1. From the **Clusters** page, click **Add Cluster**.
@@ -73,6 +82,34 @@ You can access your cluster after its state is updated to **Active.**
 - `Default`, containing the `default` namespace
 - `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces

+# Private Clusters
+
+We now support private GKE clusters. Note: This advanced setup can require more steps during the cluster provisioning process. For details, see [this section.](./private-clusters)
+
+# Configuration Reference
+
+More configuration options are available for v2.5.8. For details on configuring GKE clusters in Rancher, see [this page.](./config-reference)
+# Updating Kubernetes Version
+
+The Kubernetes version of a cluster can be upgraded to any version available in the region or zone fo the GKE cluster. Upgrading the master Kubernetes version does not automatically upgrade worker nodes. Nodes can be upgraded independently.
+
+>**Note**
+>GKE has removed basic authentication in 1.19+. In order to upgrade a cluster to 1.19+, basic authentication must be disabled in the Google Cloud. Otherwise, an error will appear in Rancher when an upgrade to 1.19+ is attempted. You can follow the [Google documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/api-server-authentication#disabling_authentication_with_a_static_password). After this, the Kubernetes version can be updated to 1.19+ via Rancher.
+
+# Syncing
+
+The GKE provisioner can synchronize the state of an GKE cluster between Rancher and the provider. For an in-depth technical explanation of how this works, see [Syncing.](../syncing)
+
+### Configuring the Refresh Interval
+
+The refresh interval can be configured through the setting "gke-refresh", which is an integer representing seconds.
+
+The default value is 300 seconds.
+
+The syncing interval can be changed by running `kubectl edit setting gke-refresh`.
+
+The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for GCP APIs.
+
 {{% /tab %}}
 {{% tab "Rancher before v2.5.8" %}}

@@ -8,20 +8,17 @@ weight: 1

 ### Changes in v2.5.8

- We now support private GKE clusters.
- Shared VPCs are now supported.  You can use an organization level VPC as well as a project level VPC.
- We now support more configuration options for Rancher managed GKE clusters:
-	- Project ID
-	- Service address range
-	- Node Ipv4 CIDR block
-	- Cluster pod address range
-	- Subnetwork name
-	- Network policy
-	- Network policy config
-	- Node groups and node configuration options:
-		- More image types are available for the nodes
-		- The max pods per node can be configured
- When provisioning a GKE cluster, you can now use reusable cloud credentials instead of using a service account token directly to create the cluster.
+-   We now support private GKE clusters. Note: This advanced setup can require more steps during the cluster provisioning process. For details, see [this section.](https://staging.rancher.com/docs/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/#private-clusters)
+-   [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc) are now supported.
+-   We now support more configuration options for Rancher managed GKE clusters:
+    -   Project
+    -   Network policy
+    -   Network policy config
+    -   Node pools and node configuration options:
+        -   More image types are available for the nodes
+        -   The maximum number of pods per node can be configured
+	-   Node pools can be added while configuring the GKE cluster
+-   When provisioning a GKE cluster, you can now use reusable cloud credentials instead of using a service account token directly to create the cluster.

 # Cluster Location

@@ -35,61 +32,127 @@ weight: 1
 # Cluster Options

 ### Kubernetes Version
+
 For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)

 ### Container Address Range

+The IP address range for pods in the cluster. Must be a valid CIDR range, e.g. 10.42.0.0/16. If not specified, a random range is automatically chosen from 10.0.0.0/8 and will exclude ranges already allocated to VMs, other clusters, or routes. Automatically chosen ranges may conflict with reserved IP addresses, dynamic routes, or routes within VPCs peering with the cluster.
+
 ### Network

-### Node Subnet
+The Compute Engine Network that the cluster connects to. Routes and firewalls will be created using this network. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC networks that are shared to your project will appear here. will be available to select in this field. For more information, refer to [this page](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets).
+
+### Node Subnet / Subnet
+
+The Compute Engine subnetwork that the cluster connects to. This subnetwork must belong to the network specified in the **Network** field. Select an existing subnetwork, or select "Auto Create Subnetwork" to have one automatically created. If not using an existing network, **Subnetwork Name** is required to generate one. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC subnets that are shared to your project will appear here. If using a Shared VPC network, you cannot select "Auto Create Subnetwork". For more information, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
+
+### Subnetwork Name
+
+Automatically create a subnetwork with the provided name. Required if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. For more information on subnetworks, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)

 ### Ip Aliases

+Enable [alias IPs](https://cloud.google.com/vpc/docs/alias-ip). This enables VPC-native traffic routing. Required if using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc).
+
 ### Network Policy

-### Subnetwork Name (required)
-
-### Cluster Pod Address Range
+Enable network policy enforcement on the cluster. A network policy defines the level of communication that can occur between pods and services in the cluster. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/network-policy)

 ### Node Ipv4 CIDR Block

+The IP address range of the instance IPs in this cluster. Can be set if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. Must be a valid CIDR range, e.g. 10.96.0.0/14. For more information on how to determine the IP address range, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing)
+
+### Cluster Secondary Range Name
+
+The name of an existing secondary range for Pod IP addresses. If selected, **Cluster Pod Address Range** will automatically be populated. Required if using a Shared VPC network.
+
+### Cluster Pod Address Range
+
+The IP address range assigned to pods in the cluster. Must be a valid CIDR range, e.g. 10.96.0.0/11. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your pods, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_pods)
+
+### Services Secondary Range Name
+
+The name of an existing secondary range for service IP addresses. If selected, **Service Address Range** will be automatically populated. Required if using a Shared VPC network.
+
 ### Service Address Range

+The address range assigned to the services in the cluster. Must be a valid CIDR range, e.g. 10.94.0.0/18. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your services, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_svcs)
+
+### Private Cluster
+
+> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/private-clusters/).
+
+Assign nodes only internal IP addresses. Private cluster nodes cannot access the public internet unless additional networking steps are taken in GCP.
+
+### Enable Private Endpoint
+
+> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/#private-clusters).
+
+Locks down external access to the control plane endpoint. Only available if **Private Cluster** is also selected. If selected, and if Rancher does not have direct access to the Virtual Private Cloud network the cluster is running in, Rancher will provide a registration command to run on the cluster to enable Rancher to connect to it.
+
+### Master IPV4 CIDR Block
+
+The IP range for the control plane VPC.
+
+### Master Authorized Network
+
+Enable control plane authorized networks to block untrusted non-GCP source IPs from accessing the Kubernetes master through HTTPS. If selected, additional authorized networks may be added. If the cluster is created with a public endpoint, this option is useful for locking down access to the public endpoint to only certain networks, such as the network where your Rancher service is running. If the cluster only has a private endpoint, this setting is required.

 # Additional Options

 ### Cluster Addons

- Horizontal Pod Autoscaling
- HTTP (L7) Load Balancing
- Network Policy Config (master only
+Additional Kubernetes cluster components. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters#Cluster.AddonsConfig)
+
+#### Horizontal Pod Autoscaling
+
+The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler)
+
+#### HTTP (L7) Load Balancing
+
+HTTP (L7) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on GKE. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer)
+
+#### Network Policy Config (master only)
+
+Configuration for NetworkPolicy. This only tracks whether the addon is enabled or not on the master, it does not track whether network policy is enabled for the nodes.

 ### Cluster Features (Alpha Features)

+Turns on all Kubernetes alpha API groups and features for the cluster. When enabled, the cluster cannot be upgraded and will be deleted automatically after 30 days. Alpha clusters are not recommended for production use as they are not covered by the GKE SLA. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters)
+
 ### Logging Service

+The logging service the cluster uses to write logs. Use either [Cloud Logging](https://cloud.google.com/logging) or no logging service in which case no logs are exported from the cluster.

 ### Monitoring Service

+The monitoring service the cluster uses to write metrics. Use either [Cloud Monitoring](https://cloud.google.com/monitoring) or monitoring service in which case no metrics are exported from the cluster.
+

 ### Maintenance Window

-
+Set the start time for a 4 hour maintenance window. The time is specified in the UTC time zone using the HH:MM format. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions)

 # Node Pools

 In this section, enter details describing the configuration of each node in the node pool.
+
 ### Kubernetes Version

 The Kubernetes version for each node in the node pool. For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)

 ### Image Type

-For more information for the node image options that GKE offers for each OS, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#available_node_images)
+The node operating system image. For more information for the node image options that GKE offers for each OS, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#available_node_images)
+
+> Note: the default option is "Container-Optimized OS with Docker". The read-only filesystem on GCP's Container-Optimized OS is not compatible with the [legacy logging]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/tools/logging) implementation in Rancher. If you need to use the legacy logging feature, select "Ubuntu with Docker" or "Ubuntu with Containerd". The [logging feature as of v2.5]({{<baseurl>}}/rancher/v2.5/en/logging) is compatible with the Container-Optimized OS image.
+
+> Note: if selecting "Windows Long Term Service Channel" or "Windows Semi-Annual Channel" for the node pool image type, you must also add at least one Container-Optimized OS or Ubuntu node pool.

 ### Machine Type

-For more information on Google Cloud machine types, refer to [this page.](https://cloud.google.com/compute/docs/machine-types#machine_types)
+The virtualized hardware resources available to node instances. For more information on Google Cloud machine types, refer to [this page.](https://cloud.google.com/compute/docs/machine-types#machine_types)

 ### Root Disk Type

@@ -117,18 +180,20 @@ You can apply labels to the node pool, which applies the labels to all nodes in
 In this section, enter details describing the node pool.

 ### Name
-Enter a name for the node group.
+
+Enter a name for the node pool.

 ### Initial Node Count
+
 Integer for the starting number of nodes in the node pool.

 ### Max Pod Per Node

 GKE has a hard limit of 110 Pods per node. For more information on the Kubernetes limits, see [this section.](https://cloud.google.com/kubernetes-engine/docs/best-practices/scalability#dimension_limits)

-### Horizontal Pod Autoscaling
+### Autoscaling

-The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler)
+Node pool autoscaling dynamically creates or deletes nodes based on the demands of your workload. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler)

 ### Auto Repair

@@ -178,61 +243,55 @@ Link to list of GKE kubernetes versions

 ### Container Address Range

+The IP address range for pods in the cluster. Must be a valid CIDR range, e.g. 10.42.0.0/16. If not specified, a random range is automatically chosen from 10.0.0.0/8 and will exclude ranges already allocated to VMs, other clusters, or routes. Automatically chosen ranges may conflict with reserved IP addresses, dynamic routes, or routes within VPCs peering with the cluster.
+
 ### Alpha Features

+Turns on all Kubernetes alpha API groups and features for the cluster. When enabled, the cluster cannot be upgraded and will be deleted automatically after 30 days. Alpha clusters are not recommended for production use as they are not covered by the GKE SLA. For more information, refer to [this page](https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters).

 ### Legacy Authorization

-
+This option is deprecated and it is recommended to leave it disabled. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster#leave_abac_disabled)
 ### Stackdriver Logging

-
+Enable logging with Google Cloud's Operations Suite, formerly called Stackdriver. For details, see the [documentation.](https://cloud.google.com/logging/docs/basic-concepts)
 ### Stackdriver Monitoring

-
+Enable monitoring with Google Cloud's Operations Suite, formerly called Stackdriver. For details, see the [documentation.](https://cloud.google.com/monitoring/docs/monitoring-overview)
 ### Kubernetes Dashboard

-
+Enable the [Kubernetes dashboard add-on.](https://cloud.google.com/kubernetes-engine/docs/concepts/dashboards#kubernetes_dashboard) Starting with GKE v1.15, you will no longer be able to enable the Kubernetes Dashboard by using the add-on API.
 ### Http Load Balancing

-
+Set up [HTTP(S) load balancing.](https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer) To use Ingress, you must have the HTTP(S) Load Balancing add-on enabled.
 ### Horizontal Pod Autoscaling

-
+The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster. For more information, see the [documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler)
 ### Maintenance Window

+Set the start time for a 4 hour maintenance window. The time is specified in the UTC time zone using the HH:MM format. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions)

 ### Network

+The Compute Engine Network that the cluster connects to. Routes and firewalls will be created using this network. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC networks that are shared to your project will appear here. will be available to select in this field. For more information, refer to [this page](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets).

-### Node Subnet
-
+### Node Subnet / Subnet

+The Compute Engine subnetwork that the cluster connects to. This subnetwork must belong to the network specified in the **Network** field. Select an existing subnetwork, or select "Auto Create Subnetwork" to have one automatically created. If not using an existing network, **Subnetwork Name** is required to generate one. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC subnets that are shared to your project will appear here. If using a Shared VPC network, you cannot select "Auto Create Subnetwork". For more information, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
 ### Ip Aliases

+Enable [alias IPs](https://cloud.google.com/vpc/docs/alias-ip). This enables VPC-native traffic routing. Required if using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc).

-### Cluster Secondary Range Name
-This option is available after the cluster is created.
-
-### Services Secondary Range Name
-This option is available after the cluster is created.
 ### Pod address range

+When you create a VPC-native cluster, you specify a subnet in a VPC network. The cluster uses three unique subnet IP address ranges for nodes, pods, and services. For more information on IP address ranges, see [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing)

 ### Service address range

-
+When you create a VPC-native cluster, you specify a subnet in a VPC network. The cluster uses three unique subnet IP address ranges for nodes, pods, and services. For more information on IP address ranges, see [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing)
 ### Cluster Labels

 A [cluster label](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-managing-labels) is a key-value pair that helps you organize your Google Cloud clusters. You can attach a label to each resource, then filter the resources based on their labels. Information about labels is forwarded to the billing system, so you can break down your billing charges by label.
-## Advanced Cluster Options
-
-### Private Cluster
-
-
-### Master Authorized Network
-
-

 ## Node Options

@@ -242,12 +301,12 @@ Integer for the starting number of nodes in the node pool.
 ### Machine Type
 For more information on Google Cloud machine types, refer to [this page.](https://cloud.google.com/compute/docs/machine-types#machine_types)

-### Image Types
+### Image Type
 Ubuntu or Container-Optimized OS images are available.

 For more information about GKE node image options, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#available_node_images)

-### Root disk type
+### Root Disk Type

 Standard persistent disks are backed by standard hard disk drives (HDD), while SSD persistent disks are backed by solid state drives (SSD). For more information, refer to [this section.](https://cloud.google.com/compute/docs/disks)

@@ -260,6 +319,7 @@ Configure each node's local SSD disk storage in GB.
 Local SSDs are physically attached to the server that hosts your VM instance. Local SSDs have higher throughput and lower latency than standard persistent disks or SSD persistent disks. The data that you store on a local SSD persists only until the instance is stopped or deleted. For more information, see [this section.](https://cloud.google.com/compute/docs/disks#localssds)

 ### Preemptible nodes (beta)
+
 Preemptible nodes, also called preemptible VMs, are Compute Engine VM instances that last a maximum of 24 hours in general, and provide no availability guarantees. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms)

 ### Auto Upgrade
@@ -269,6 +329,7 @@ Preemptible nodes, also called preemptible VMs, are Compute Engine VM instances
 When enabled, the auto-upgrade feature keeps the nodes in your cluster up-to-date with the cluster control plane (master) version when your control plane is [updated on your behalf.](https://cloud.google.com/kubernetes-engine/upgrades#automatic_cp_upgrades) For more information about auto-upgrading nodes, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-upgrades)

 ### Auto Repair
+
 GKE's node auto-repair feature helps you keep the nodes in your cluster in a healthy, running state. When enabled, GKE makes periodic checks on the health state of each node in your cluster. If a node fails consecutive health checks over an extended time period, GKE initiates a repair process for that node. For more information, see the section on [auto-repairing nodes.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-repair)

 ### Node Pool Autoscaling
@@ -284,7 +345,6 @@ You can apply labels to the node pool, which applies the labels to all nodes in

 ### Service Account

-
 Create a [Service Account](https://console.cloud.google.com/projectselector/iam-admin/serviceaccounts) with a JSON private key and provide the JSON here. See [Google Cloud docs](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances) for more info about creating a service account. These IAM roles are required: Compute Viewer (`roles/compute.viewer`), (Project) Viewer (`roles/viewer`), Kubernetes Engine Admin (`roles/container.admin`), Service Account User (`roles/iam.serviceAccountUser`). More info on roles can be found [here.](https://cloud.google.com/kubernetes-engine/docs/how-to/iam-integration)

 ### Access Scopes
@@ -0,0 +1,42 @@
+---
+title: Private Clusters
+weight: 2
+---
+
+In GKE, [private clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept) are clusters whose nodes are isolated from inbound and outbound traffic by assigning them internal IP addresses only. Private clusters in GKE have the option of exposing the control plane endpoint as a publicly accessible address or as a private address. This is different from other Kubernetes providers, which may refer to clusters with private control plane endpoints as "private clusters" but still allow traffic to and from nodes. You may want to create a cluster with private nodes, with or without a public control plane endpoint, depending on your organization's networking and security requirements. A GKE cluster hosted in Rancher can use isolated nodes by selecting "Private Cluster" in the Cluster Options (under "Show advanced options"). The control plane endpoint can optionally be made private by selecting "Enable Private Endpoint".
+
+### Private Nodes
+
+Because the nodes in a private cluster only have internal IP addresses, they will not be able to install the cluster agent and Rancher will not be able to fully manage the cluster. This can be overcome in a few ways.
+
+#### Cloud NAT
+
+>**Note**
+>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
+
+If restricting outgoing internet access is not a concern for your organization, use Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service to allow nodes in the private network to access the internet, allowing them to download the required images from Dockerhub. This is the simplest solution.
+
+#### Private registry
+
+>**Note**
+>This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
+
+If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent.
+
+### Private Control Plane Endpoint
+
+If the cluster has a public endpoint exposed, Rancher will be able to reach the cluster, and no additional steps need to be taken. However, if the cluster has no public endpoint, then considerations must be made to ensure Rancher can access the cluster.
+
+#### Cloud NAT
+
+>**Note**
+>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
+
+As above, if restricting outgoing internet access to the nodes is not a concern, then Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service can be used to allow the nodes to access the internet. While the cluster is provisioning, Rancher will provide a registration command to run on the cluster. Download the [kubeconfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) for the new cluster and run the provided kubectl command on the cluster. Gaining access
+to the cluster in order to run this command can be done by creating a temporary node or using an existing node in the VPC, or by logging on to or creating an SSH tunnel through one of the cluster nodes.
+
+#### Direct access
+
+If the Rancher server is run on the same VPC as the cluster's control plane, it will have direct access to the control plane's private endpoint. The cluster nodes will need to have access to a private registry to download images as described above.
+
+You can also use services from Google such as [Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or [Cloud Interconnect VLAN](https://cloud.google.com/network-connectivity/docs/interconnect) to facilitate connectivity between your organization's network and your Google VPC.
@@ -0,0 +1,32 @@
+# Syncing
+
+Syncing is the feature for EKS and GKE clusters that causes Rancher to update the clusters' values so they are up to date with their corresponding cluster object in the hosted Kubernetes provider. This enables Rancher to not be the sole owner of a hosted cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other.
+
+### How it works
+
+There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
+
+1. The config object for the cluster, located on the Spec of the Cluster:
+
+   * For EKS, the field is called EKSConfig
+   * For GKE, the field is called GKEConfig
+
+2. The UpstreamSpec object
+
+   * For EKS, this is located on the EKSStatus field on the Status of the Cluster.
+   * For GKE, this is located on the GKEStatus field on the Status of the Cluster.
+
+The struct types that define these objects can be found in their corresponding operator projects:
+
+  * [eks-operator](https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go)
+  * [gke-operator](https://github.com/rancher/gke-operator/blob/master/pkg/apis/gke.cattle.io/v1/types.go)
+
+All fields with the exception of the cluster name, the location (region or zone), Imported, and the cloud credential reference, are nillable on this Spec object.
+
+The EKSConfig or GKEConfig represents desired state for its non-nil values. Fields that are non-nil in the config object can be thought of as “managed". When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher.
+
+UpstreamSpec represents the cluster as it is in the hosted Kubernetes provider and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed, Rancher checks if the cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig or GKEConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
+
+The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig or GKEConfig. This is what is displayed in the UI.
+
+If Rancher and another source attempt to update a cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. To use EKS as an example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioned case.
@@ -5,20 +5,13 @@ aliases:
  - /rancher/v2.5/en/cluster-provisioning/imported-clusters
 ---

-_Available as of v2.5_
-
 The cluster registration feature replaced the feature to import clusters.

 The control that Rancher has to manage a registered cluster depends on the type of cluster. For details, see [Management Capabilities for Registered Clusters.](#management-capabilities-for-registered-clusters)

-Registering EKS clusters now provides additional benefits.
-
 - [Prerequisites](#prerequisites)
 - [Registering a Cluster](#registering-a-cluster)
 - [Management Capabilities for Registered Clusters](#management-capabilities-for-registered-clusters)
-  - [Features for All Registered Clusters](#features-for-all-registered-clusters)
-  - [Additional Features for Registered K3s Clusters](#additional-features-for-registered-k3s-clusters)
-  - [Additional Features for Registered EKS Clusters](#additional-features-for-registered-eks-clusters)
 - [Configuring K3s Cluster Upgrades](#configuring-k3s-cluster-upgrades)
 - [Debug Logging and Troubleshooting for Registered K3s Clusters](#debug-logging-and-troubleshooting-for-registered-k3s-clusters)
 - [Annotating Registered Clusters](#annotating-registered-clusters)
@@ -85,6 +78,19 @@ $ curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s -

 The control that Rancher has to manage a registered cluster depends on the type of cluster.

+{{% tabs %}}
+{{% tab "Rancher v2.5.8+" %}}
+
+- [Changes in v2.5.8](#changes-in-v2-5-8)
+- [Features for All Registered Clusters](#2-5-8-features-for-all-registered-clusters)
+- [Additional Features for Registered K3s Clusters](#2-5-8-additional-features-for-registered-k3s-clusters)
+- [Additional Features for Registered EKS and GKE Clusters](#additional-features-for-registered-eks-and-gke-clusters)
+
+### Changes in v2.5.8
+
+Greater management capabilities are now available for [registered GKE clusters.](#additional-features-for-registered-eks-and-gke-clusters) The same configuration options are available for registered GKE clusters as for the GKE clusters created through the Rancher UI.
+
+<a id="2-5-8-features-for-all-registered-clusters"></a>
 ### Features for All Registered Clusters

 After registering a cluster, the cluster owner can:
@@ -96,6 +102,48 @@ After registering a cluster, the cluster owner can:
 - Use [pipelines]({{<baseurl>}}/rancher/v2.5/en/project-admin/pipelines/)
 - Manage projects and workloads

+<a id="2-5-8-additional-features-for-registered-k3s-clusters"></a>
+### Additional Features for Registered K3s Clusters
+
+[K3s]({{<baseurl>}}/k3s/latest/en/) is a lightweight, fully compliant Kubernetes distribution.
+
+When a K3s cluster is registered in Rancher, Rancher will recognize it as K3s. The Rancher UI will expose the features for [all registered clusters,](#features-for-all-registered-clusters) in addition to the following features for editing and upgrading the cluster:
+
+- The ability to [upgrade the K3s version]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/upgrading-kubernetes/)
+- The ability to configure the maximum number of nodes that will be upgraded concurrently
+- The ability to see a read-only version of the K3s cluster's configuration arguments and environment variables used to launch each node in the cluster
+
+### Additional Features for Registered EKS and GKE Clusters
+
+Registering an Amazon EKS cluster or GKE cluster allows Rancher to treat it as though it were created in Rancher.
+
+Amazon EKS clusters and GKE clusters can now be registered in Rancher. For the most part, these registered clusters are treated the same way as clusters created in the Rancher UI, except for deletion.
+
+When you delete an EKS cluster or GKE cluster that was created in Rancher, the cluster is destroyed. When you delete a cluster that was registered in Rancher, it is disconnected from the Rancher server, but it still exists and you can still access it in the same way you did before it was registered in Rancher.
+
+The capabilities for registered clusters are listed in the table on [this page.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/)
+
+
+{{% /tab %}}
+{{% tab "Rancher v2.5.0-v2.5.8" %}}
+
+- [Features for All Registered Clusters](#before-2-5-8-features-for-all-registered-clusters)
+- [Additional Features for Registered K3s Clusters](#before-2-5-8-additional-features-for-registered-k3s-clusters)
+- [Additional Features for Registered EKS Clusters](#additional-features-for-registered-eks-clusters)
+
+<a id="before-2-5-8-features-for-all-registered-clusters"></a>
+### Features for All Registered Clusters
+
+After registering a cluster, the cluster owner can:
+
+- [Manage cluster access]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/cluster-project-roles/) through role-based access control
+- Enable [monitoring, alerts and notifiers]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/v2.5/)
+- Enable [logging]({{<baseurl>}}/rancher/v2.5/en/logging/v2.5/)
+- Enable [Istio]({{<baseurl>}}/rancher/v2.5/en/istio/v2.5/)
+- Use [pipelines]({{<baseurl>}}/rancher/v2.5/en/project-admin/pipelines/)
+- Manage projects and workloads
+
+<a id="before-2-5-8-additional-features-for-registered-k3s-clusters"></a>
 ### Additional Features for Registered K3s Clusters

 [K3s]({{<baseurl>}}/k3s/latest/en/) is a lightweight, fully compliant Kubernetes distribution.
@@ -115,6 +163,10 @@ Amazon EKS clusters can now be registered in Rancher. For the most part, registe
 When you delete an EKS cluster that was created in Rancher, the cluster is destroyed. When you delete an EKS cluster that was registered in Rancher, it is disconnected from the Rancher server, but it still exists and you can still access it in the same way you did before it was registered in Rancher.

 The capabilities for registered EKS clusters are listed in the table on [this page.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/)
+{{% /tab %}}
+{{% /tabs %}}
+
+

 # Configuring K3s Cluster Upgrades

@@ -96,7 +96,7 @@ If you have configured your cluster to use Amazon as **Cloud Provider**, tag you
 >**Note:** You can use Amazon EC2 instances without configuring a cloud provider in Kubernetes. You only have to configure the cloud provider if you want to use specific Kubernetes cloud provider functionality. For more information, see [Kubernetes Cloud Providers](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/)


-The following resources need to tagged with a `ClusterID`:
+The following resources need to be tagged with a `ClusterID`:

 - **Nodes**: All hosts added in Rancher.
 - **Subnet**: The subnet used for your cluster
@@ -123,4 +123,4 @@ Key=kubernetes.io/cluster/CLUSTERID, Value=shared
 After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:

 - **Access your cluster with the kubectl CLI:** Follow [these steps]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#accessing-clusters-with-kubectl-on-your-workstation) to access clusters with kubectl on your workstation. In this case, you will be authenticated through the Rancher server’s authentication proxy, then Rancher will connect you to the downstream cluster. This method lets you manage the cluster without the Rancher UI.
- **Access your cluster with the kubectl CLI, using the authorized cluster endpoint:** Follow [these steps]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#authenticating-directly-with-a-downstream-cluster) to access your cluster with kubectl directly, without authenticating through Rancher. We recommend setting up this alternative method to access your cluster so that in case you can’t connect to Rancher, you can still access the cluster.
+- **Access your cluster with the kubectl CLI, using the authorized cluster endpoint:** Follow [these steps]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#authenticating-directly-with-a-downstream-cluster) to access your cluster with kubectl directly, without authenticating through Rancher. We recommend setting up this alternative method to access your cluster so that in case you can’t connect to Rancher, you can still access the cluster.
@@ -32,6 +32,15 @@ The capability to access a downstream cluster without Rancher depends on the typ

 If you [installed Rancher on a Kubernetes cluster,]({{<baseurl>}}/rancher/v2.5/en/installation/install-rancher-on-k8s/) remove Rancher by using the [System Tools]({{<baseurl>}}/rancher/v2.5/en/system-tools/) with the `remove` subcommand.

+As of Rancher v2.5.8, uninstalling Rancher in high-availability (HA) mode will also remove all `helm-operation-*` pods and the following apps:
+
+- fleet
+- fleet-agent
+- rancher-operator
+- rancher-webhook
+
+Custom resources (CRDs) and custom namespaces will still need to be manually removed.
+
 If you installed Rancher with Docker, you can uninstall Rancher by removing the single Docker container that it runs in.

 Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
@@ -20,7 +20,7 @@ The following instructions will guide you through upgrading a Rancher server tha

 For the instructions to upgrade Rancher installed on Kubernetes with RancherD, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/installation/install-rancher-on-linux/upgrades)

-For the instructions to upgrade Rancher installed with Docker, refer to [ths page.]({{<baseurl>}}/rancher/v2.5/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)
+For the instructions to upgrade Rancher installed with Docker, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)

 To upgrade the components in your Kubernetes cluster, or the definition of the [Kubernetes services]({{<baseurl>}}/rke/latest/en/config-options/services/) or [add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/), refer to the [upgrade documentation for RKE]({{<baseurl>}}/rke/latest/en/upgrades/), the Rancher Kubernetes Engine.

@@ -65,7 +65,7 @@ The following tables break down the port requirements for inbound and outbound t
 | Protocol | Port | Destination                                              | Description                                   |
 | -------- | ---- | -------------------------------------------------------- | --------------------------------------------- |
 | TCP      | 22   | Any node IP from a node created using Node Driver        | SSH provisioning of nodes using Node Driver   |
-| TCP      | 443  | `35.160.43.145/32`, `35.167.242.46/32`, `52.33.59.17/32` | git.rancher.io (catalogs)                     |
+| TCP      | 443  | git.rancher.io |  Rancher catalog                     |
 | TCP      | 2376 | Any node IP from a node created using Node driver        | Docker daemon TLS port used by Docker Machine |
 | TCP      | 6443 | Hosted/Imported Kubernetes API                           | Kubernetes API server                         |

@@ -162,7 +162,7 @@ The following tables break down the port requirements for Rancher nodes, for inb
 | Protocol | Port | Source | Description |
 |-----|-----|----------------|---|
 | TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
-| TCP | 443 | `35.160.43.145/32`,`35.167.242.46/32`,`52.33.59.17/32` | git.rancher.io (catalogs) |
+| TCP | 443 | git.rancher.io |  Rancher catalog |
 | TCP | 2376 | Any node IP from a node created using a node driver | Docker daemon TLS port used by Docker Machine |
 | TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |

@@ -48,14 +48,14 @@ If the Rancher server is installed in a single Docker container, you only need o
 ```
 sudo ssh -i [path-to-private-key] ubuntu@[public-DNS-of-instance]
 ```
-1. When you are connected to the instance, run the following command on the instance to create a user:
-```
-sudo usermod -aG docker ubuntu
-```
 1. Run the following command on the instance to install Docker with one of Rancher's installation scripts:
 ```
 curl https://releases.rancher.com/install-docker/18.09.sh | sh
 ```
+1. When you are connected to the instance, run the following command on the instance to create a user:
+```
+sudo usermod -aG docker ubuntu
+```
 1. Repeat these steps so that Docker is installed on each node that will eventually run the Rancher management server.

 > To find out whether a script is available for installing a certain Docker version, refer to this [GitHub repository,](https://github.com/rancher/install-docker) which contains all of Rancher’s Docker installation scripts.
@@ -32,6 +32,7 @@ These instructions assume you are using Rancher v2.5, but Longhorn can be instal

 ### Installing Longhorn with Rancher

+1. Fulfill all [Installation Requirements.](https://longhorn.io/docs/1.1.0/deploy/install/#installation-requirements)
 1. Go to the **Cluster Explorer** in the Rancher UI.
 1. Click **Apps.**
 1. Click `longhorn`.
@@ -43,7 +44,7 @@ These instructions assume you are using Rancher v2.5, but Longhorn can be instal
 ### Accessing Longhorn from the Rancher UI

 1. From the **Cluster Explorer," go to the top left dropdown menu and click **Cluster Explorer > Longhorn.**
-1. On this page, you can edit Kubernetes resources managed by Longhorn. To view the Longhorn UI, click the **Longhorn** button in the **Overview**section.
+1. On this page, you can edit Kubernetes resources managed by Longhorn. To view the Longhorn UI, click the **Longhorn** button in the **Overview** section.

 **Result:** You will be taken to the Longhorn UI, where you can manage your Longhorn volumes and their replicas in the Kubernetes cluster, as well as secondary backups of your Longhorn storage that may exist in another Kubernetes cluster or in S3.

@@ -73,4 +74,4 @@ The storage controller and replicas are themselves orchestrated using Kubernetes
 You can learn more about its architecture [here.](https://longhorn.io/docs/1.0.2/concepts/)

 <figcaption>Longhorn Architecture</figcaption>
-![Longhorn Architecture]({{<baseurl>}}/img/rancher/longhorn-architecture.svg)
+![Longhorn Architecture]({{<baseurl>}}/img/rancher/longhorn-architecture.svg)
@@ -62,6 +62,10 @@ As an [administrator]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/global-p
 > - Make sure your cluster fulfills the resource requirements. The cluster should have at least 1950Mi memory available, 2700m CPU, and 50Gi storage. A breakdown of the resource limits and requests is [here.](#setting-resource-limits-and-requests)
 > - When installing monitoring on an RKE cluster using RancherOS or Flatcar Linux nodes, change the etcd node certificate directory to `/opt/rke/etc/kubernetes/ssl`.

+{{% tabs %}}
+{{% tab "Rancher v2.5.8" %}}
+
+### Enable Monitoring for use without SSL

 1. In the Rancher UI, go to the cluster where you want to install monitoring and click **Cluster Explorer.**
 1. Click **Apps.**
@@ -71,6 +75,46 @@ As an [administrator]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/global-p

 **Result:** The monitoring app is deployed in the `cattle-monitoring-system` namespace.

+### Enable Monitoring for use with SSL
+
+1. Follow the steps on [this page]({{<baseurl>}}/rancher/v2.5/en/k8s-in-rancher/secrets/) to create a secret in order for SSL to be used for alerts.
+ - The secret should be created in the `cattle-monitoring-system` namespace. If it doesn't exist, create it first.
+ - Add the `ca`, `cert`, and `key` files to the secret.
+1. In the Rancher UI, go to the cluster where you want to install monitoring and click **Cluster Explorer.**
+1. Click **Apps.**
+1. Click the `rancher-monitoring` app.
+1. Click **Alerting**.
+1. Click **Additional Secrets** and add the secrets created earlier.
+ 
+**Result:** The monitoring app is deployed in the `cattle-monitoring-system` namespace.
+
+When [creating a receiver,]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/configuration/alertmanager/#creating-receivers-in-the-rancher-ui) SSL-enabled receivers such as email or webhook will have a **SSL** section with fields for **CA File Path**, **Cert File Path**, and **Key File Path**. Fill in these fields with the paths to each of `ca`, `cert`, and `key`. The path will be of the form `/etc/alertmanager/secrets/name-of-file-in-secret`.
+
+For example, if you created a secret with these key-value pairs:
+
+```yaml
+ca.crt=`base64-content`
+cert.pem=`base64-content`
+key.pfx=`base64-content`
+```
+
+Then **Cert File Path** would be set to `/etc/alertmanager/secrets/cert.pem`.
+
+{{% /tab %}}
+{{% tab "Rancher v2.5.0-2.5.7" %}}
+
+1. In the Rancher UI, go to the cluster where you want to install monitoring and click **Cluster Explorer.**
+1. Click **Apps.**
+1. Click the `rancher-monitoring` app.
+1. Optional: Click **Chart Options** and configure alerting, Prometheus and Grafana. For help, refer to the [configuration reference.](./configuration)
+1. Scroll to the bottom of the Helm chart README and click **Install.**
+
+**Result:** The monitoring app is deployed in the `cattle-monitoring-system` namespace.
+
+{{% /tab %}}
+
+{{% /tabs %}}
+
 ### Default Alerts, Targets, and Grafana Dashboards

 By default, Rancher Monitoring deploys exporters (such as [node-exporter](https://github.com/prometheus/node_exporter) and [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)) as well as default Prometheus alerts and Grafana dashboards (curated by the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project) onto a cluster.
@@ -20,6 +20,8 @@ The [Alertmanager Config](https://prometheus.io/docs/alerting/latest/configurati
  - [Opsgenie](#opsgenie)
  - [Webhook](#webhook)
  - [Custom](#custom)
+  - [Teams](#teams)
+  - [SMS](#sms)
 - [Route Configuration](#route-configuration)
  - [Receiver](#receiver)
  - [Grouping](#grouping)
@@ -70,19 +72,30 @@ To create notification receivers in the Rancher UI,

 The notification integrations are configured with the `receiver`, which is explained in the [Prometheus documentation.](https://prometheus.io/docs/alerting/latest/configuration/#receiver)

+### Native vs. Non-native Receivers
+
+
+### Changes in Rancher v2.5.8
+
+Rancher v2.5.8 added Microsoft Teams and SMS as configurable receivers in the Rancher UI.
+
+### Changes in Rancher v2.5.4
+
 Rancher v2.5.4 introduced the capability to configure receivers by filling out forms in the Rancher UI.

 {{% tabs %}}
-{{% tab "Rancher v2.5.4+" %}}
+{{% tab "Rancher v2.5.8" %}}

 The following types of receivers can be configured in the Rancher UI:

- <a href="#slack">Slack</Slack>
- <a href="#email">Email</Slack>
- <a href="#pagerduty">PagerDuty</Slack>
- <a href="#opsgenie">Opsgenie</Slack>
- <a href="#webhook">Webhook</Slack>
- <a href="#custom">Custom</Slack>
+- <a href="#slack">Slack</a>
+- <a href="#email">Email</a>
+- <a href="#pagerduty">PagerDuty</a>
+- <a href="#opsgenie">Opsgenie</a>
+- <a href="#webhook">Webhook</a>
+- <a href="#custom">Custom</a>
+- <a href="#teams">Teams</a>
+- <a href="#sms">SMS</a>

 The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.

@@ -144,10 +157,162 @@ Opsgenie Responders:
 | Proxy URL | Proxy for the webhook notification. |
 | Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage).    |

+<!-- TODO add info on webhook for teams and sms and link to them -->
+
 ### Custom

 The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.

+### Teams
+
+#### Enabling the Teams Receiver for Rancher Managed Clusters
+
+The Teams receiver is not a native receiver and must be enabled before it can be used. You can enable the Teams receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the Teams option selected.
+
+1. In the Rancher UI, go to the cluster where you want to install rancher-alerting-drivers and click **Cluster Explorer**.
+1. Click **Apps**.
+1. Click the **Alerting Drivers** app.
+1. Click the **Helm Deploy Options** tab
+1. Select the **Teams** option and click **Install**.
+1. Take note of the namespace used as it will be required in a later step.
+
+#### Configure the Teams Receiver
+
+The Teams receiver can be configured by updating its ConfigMap. For example, the following is a minimal Teams receiver configuration.
+
+```yaml
+[Microsoft Teams]
+teams-instance-1: https://your-teams-webhook-url
+```
+
+When configuration is complete, add the receiver using the steps in [this section](#creating-receivers-in-the-rancher-ui).
+
+Use the example below as the URL where:
+
+- `ns-1` is replaced with the namespace where the `rancher-alerting-drivers` app is installed
+
+```yaml
+url: http://rancher-alerting-drivers-prom2teams.ns-1.svc:8089/v2/teams-instance-1
+```
+
+<!-- https://github.com/idealista/prom2teams -->
+
+### SMS
+
+#### Enabling the SMS Receiver for Rancher Managed Clusters
+
+The SMS receiver is not a native receiver and must be enabled before it can be used. You can enable the SMS receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the SMS option selected.
+
+1. In the Rancher UI, go to the cluster where you want to install rancher-alerting-drivers and click **Cluster Explorer**.
+1. Click **Apps**.
+1. Click the **Alerting Drivers** app.
+1. Click the **Helm Deploy Options** tab
+1. Select the **SMS** option and click **Install**.
+1. Take note of the namespace used as it will be required in a later step.
+
+#### Configure the SMS Receiver
+
+The SMS receiver can be configured by updating its ConfigMap. For example, the following is a minimal SMS receiver configuration.
+
+```yaml
+receivers:
+- name: 'telegram-receiver-1'
+  provider: 'telegram'
+  to:
+    - '123456789'
+```
+
+When configuration is complete, add the receiver using the steps in [this section](#creating-receivers-in-the-rancher-ui).
+
+Use the example below as the name and URL, where:
+
+- the name assigned to the receiver, e.g. `telegram-receiver-1`, must match the name in the `receivers.name` field in the ConfigMap, e.g. `telegram-receiver-1`
+- `ns-1` in the URL is replaced with the namespace where the `rancher-alerting-drivers` app is installed
+
+```yaml
+name: telegram-receiver-1
+url http://rancher-alerting-drivers-sachet.ns-1.svc:9876/alert
+```
+
+<!-- https://github.com/messagebird/sachet -->
+
+{{% /tab %}}
+
+{{% tab "Rancher v2.5.4-2.5.7" %}}
+
+The following types of receivers can be configured in the Rancher UI:
+
+- <a href="#slack-254-257">Slack</a>
+- <a href="#email-254-257">Email</a>
+- <a href="#pagerduty-254-257">PagerDuty</a>
+- <a href="#opsgenie-254-257">Opsgenie</a>
+- <a href="#webhook-254-257">Webhook</a>
+- <a href="#custom-254-257">Custom</a>
+
+The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.
+
+### Slack {#slack-254-257}
+
+| Field | Type | Description |
+|------|--------------|------|
+| URL | String   |  Enter your Slack webhook URL. For instructions to create a Slack webhook, see the [Slack documentation.](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack)  |
+| Default Channel |  String   |  Enter the name of the channel that you want to send alert notifications in the following format: `#<channelname>`. | 
+| Proxy URL   |    String    |  Proxy for the webhook notifications.  |
+| Enable Send Resolved Alerts |   Bool    |  Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
+
+### Email {#email-254-257}
+
+| Field | Type | Description |
+|------|--------------|------|
+| Default Recipient Address |   String    |   The email address that will receive notifications.    |
+| Enable Send Resolved Alerts |  Bool    |   Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). | 
+
+SMTP options:
+
+| Field | Type | Description |
+|------|--------------|------|
+| Sender |   String       |  Enter an email address available on your SMTP mail server that you want to send the notification from.   |
+| Host |   String         | Enter the IP address or hostname for your SMTP server. Example: `smtp.email.com`. |
+| Use TLS |   Bool     | Use TLS for encryption. |
+| Username |   String   | Enter a username to authenticate with the SMTP server. |
+| Password |   String    | Enter a password to authenticate with the SMTP server. |
+
+### PagerDuty {#pagerduty-254-257}
+
+| Field | Type | Description |
+|------|------|-------|
+| Integration Type | String | `Events API v2` or `Prometheus`. |
+| Default Integration Key | String |  For instructions to get an integration key, see the [PagerDuty documentation.](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/)  |
+| Proxy URL | String |  Proxy for the PagerDuty notifications.  |
+| Enable Send Resolved Alerts |  Bool    |   Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). | 
+
+### Opsgenie {#opsgenie-254-257}
+
+| Field | Description |
+|------|-------------|
+| API Key |   For instructions to get an API key, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/api-key-management)             |
+| Proxy URL |   Proxy for the Opsgenie notifications.        |
+| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage).  |
+
+Opsgenie Responders:
+
+| Field |    Type | Description |
+|-------|------|--------|
+| Type | String | Schedule, Team, User, or Escalation. For more information on alert responders, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/alert-recipients-and-teams) |
+| Send To | String | Id, Name, or Username of the Opsgenie recipient. |
+
+### Webhook {#webhook-1}
+
+| Field |    Description |
+|-------|--------------|
+| URL | Webhook URL for the app of your choice. |
+| Proxy URL | Proxy for the webhook notification. |
+| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage).    |
+
+### Custom {#custom-254-257}
+
+The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.
+
 {{% /tab %}}
 {{% tab "Rancher v2.5.0-2.5.3" %}}
 The Alertmanager must be configured in YAML, as shown in this [example.](#example-alertmanager-config)
@@ -237,4 +402,4 @@ spec:
 #    key: string
 ```

-For more information on enabling alerting for `rancher-cis-benchmark`, see [this section.]({{<baseurl>}}/rancher/v2.5/en/cis-scans/v2.5/#enabling-alerting-for-rancher-cis-benchmark)
+For more information on enabling alerting for `rancher-cis-benchmark`, see [this section.]({{<baseurl>}}/rancher/v2.5/en/cis-scans/v2.5/#enabling-alerting-for-rancher-cis-benchmark)
@@ -14,10 +14,12 @@ To allow the Grafana dashboard to persist after the Grafana instance restarts, a

 {{% tabs %}}
 {{% tab "Rancher v2.5.8+" %}}
-### Prerequisites

- The monitoring application needs to be installed.
- You must have at least the **Manage Config Maps** Rancher RBAC permissions assigned to you in the project that contains the Grafana Dashboards namespace (e.g. cattle-dashboards by default). This correlates to the `monitoring-dashboard-edit` or `monitoring-dashboard-admin` Kubernetes native RBAC Roles exposed by the Monitoring chart.
+> **Prerequisites:**
+> 
+> - The monitoring application needs to be installed.
+> - To create the persistent dashboard, you must have at least the **Manage Config Maps** Rancher RBAC permissions assigned to you in the project or namespace that contains the Grafana Dashboards. This correlates to the `monitoring-dashboard-edit` or `monitoring-dashboard-admin` Kubernetes native RBAC Roles exposed by the Monitoring chart.
+> - To see the links to the external monitoring UIs, including Grafana dashboards, you will need at least a [project-member role.]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/rbac/#users-with-rancher-cluster-manager-based-permissions)

 ### 1. Get the JSON model of the dashboard that you want to persist

@@ -85,7 +85,7 @@ Monitoring also creates additional `ClusterRoles` that are not assigned to users

 | Role | Purpose  |
 | ------------------------------| ---------------------------|
-| monitoring-ui-view | <a id="monitoring-ui-view"></a>_Available as of Monitoring v2 9.4.204+_ Provides read-only access to external Monitoring UIs by giving a user permission to list the Prometheus, Alertmanager, and Grafana endpoints and make GET requests to Prometheus, Grafana, and Alertmanager UIs through the Rancher proxy. |
+| monitoring-ui-view | <a id="monitoring-ui-view"></a>_Available as of Monitoring v2 14.5.100+_ Provides read-only access to external Monitoring UIs by giving a user permission to list the Prometheus, Alertmanager, and Grafana endpoints and make GET requests to Prometheus, Grafana, and Alertmanager UIs through the Rancher proxy. |

 # Users with Rancher Cluster Manager Based Permissions

@@ -14,7 +14,7 @@ Double check if all the [required ports]({{<baseurl>}}/rancher/v2.5/en/cluster-p

 The pod can be scheduled to any of the hosts you used for your cluster, but that means that the NGINX ingress controller needs to be able to route the request from `NODE_1` to `NODE_2`. This happens over the overlay network. If the overlay network is not functioning, you will experience intermittent TCP/HTTP connection failures due to the NGINX ingress controller not being able to route to the pod.

-To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/leodotcloud/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
+To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.

 1. Save the following file as `overlaytest.yml`

@@ -35,7 +35,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
          tolerations:
          - operator: Exists
          containers:
-          - image: leodotcloud/swiss-army-knife
+          - image: rancher/swiss-army-knife
            imagePullPolicy: Always
            name: overlaytest
            command: ["sh", "-c", "tail -f /dev/null"]
@@ -113,4 +113,4 @@ To check if your cluster is affected, the following command will list nodes that
 kubectl get nodes -o json | jq '.items[].metadata | select(.annotations["flannel.alpha.coreos.com/public-ip"] == null or .annotations["flannel.alpha.coreos.com/kube-subnet-manager"] == null or .annotations["flannel.alpha.coreos.com/backend-type"] == null or .annotations["flannel.alpha.coreos.com/backend-data"] == null) | .name'
 ```

-If there is no output, the cluster is not affected.
+If there is no output, the cluster is not affected.
@@ -52,5 +52,5 @@ After you complete [Configuring Microsoft AD FS for Rancher]({{<baseurl>}}/ranch
 **Tip:** You can generate a certificate using an openssl command. For example:

 ```
-openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
-```
+openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=https://myservice.example.com"
+```
@@ -36,12 +36,13 @@ The Cloud Provider Interface (CPI) should be installed first before installing t
 	```
 	kubectl describe nodes | grep "ProviderID"
 	```
+
 ### 3. Installing the CSI plugin

- 1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
-1. Select the **vSphere CSI** chart. Fill out the required vCenter details.
-2. Set **Enable CSI Migration** to **false**.
-3. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. Fill out the details for the StorageClass and launch the chart.
+1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
+2. Select the **vSphere CSI** chart. Fill out the required vCenter details.
+3. Set **Enable CSI Migration** to **false**.
+4. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. Fill out the details for the StorageClass and launch the chart.

 # Using the CSI driver for provisioning volumes
 The CSI chart by default creates a storageClass.
@@ -27,7 +27,7 @@ Set up the Rancher server's local Kubernetes cluster.

 The cluster requirements depend on the Rancher version:

- **As of Rancher v2.5,** Rancher can be installed on any Kubernetes cluster. This cluster can use upstream Kubernetes, or it can use one of Rancher's Kubernetes distributions, or it can be a managed Kubernetes cluster from a provider such as Amazon EKS. Note: To deploy Rancher v2.5 on a hosted Kubernetes cluster such as EKS, GKE, or AKS, you should deploy a compatible Ingress controller first to configure [SSL termination on Rancher.]({{<baseurl>}}/rancher/v2.x/en/installation/install-rancher-on-k8s/#4-choose-your-ssl-configuration)
+- **As of Rancher v2.5,** Rancher can be installed on any Kubernetes cluster. This cluster can use upstream Kubernetes, or it can use one of Rancher's Kubernetes distributions, or it can be a managed Kubernetes cluster from a provider such as Amazon EKS. Note: To deploy Rancher v2.5 on a hosted Kubernetes cluster such as EKS, GKE, or AKS, you should deploy a compatible Ingress controller first to configure [SSL termination on Rancher.]({{<baseurl>}}/rancher/v2.x/en/installation/install-rancher-on-k8s/#3-choose-your-ssl-configuration)
 - **In Rancher v2.4.x,** Rancher needs to be installed on a K3s Kubernetes cluster or an RKE Kubernetes cluster.
 - **In Rancher before v2.4,** Rancher needs to be installed on an RKE Kubernetes cluster.

@@ -20,7 +20,7 @@ The following instructions will guide you through upgrading a Rancher server tha

 For the instructions to upgrade Rancher installed on Kubernetes with RancherD, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/installation/install-rancher-on-linux/upgrades)

-For the instructions to upgrade Rancher installed with Docker, refer to [ths page.]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)
+For the instructions to upgrade Rancher installed with Docker, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/single-node-upgrades)

 To upgrade the components in your Kubernetes cluster, or the definition of the [Kubernetes services]({{<baseurl>}}/rke/latest/en/config-options/services/) or [add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/), refer to the [upgrade documentation for RKE]({{<baseurl>}}/rke/latest/en/upgrades/), the Rancher Kubernetes Engine.

@@ -67,7 +67,7 @@ The following tables break down the port requirements for inbound and outbound t
 | Protocol | Port | Destination                                              | Description                                   |
 | -------- | ---- | -------------------------------------------------------- | --------------------------------------------- |
 | TCP      | 22   | Any node IP from a node created using Node Driver        | SSH provisioning of nodes using Node Driver   |
-| TCP      | 443  | `35.160.43.145/32`, `35.167.242.46/32`, `52.33.59.17/32` | git.rancher.io (catalogs)                     |
+| TCP      | 443  | git.rancher.io |  Rancher catalog                    |
 | TCP      | 2376 | Any node IP from a node created using Node driver        | Docker daemon TLS port used by Docker Machine |
 | TCP      | 6443 | Hosted/Imported Kubernetes API                           | Kubernetes API server                         |

@@ -164,7 +164,7 @@ The following tables break down the port requirements for Rancher nodes, for inb
 | Protocol | Port | Source | Description |
 |-----|-----|----------------|---|
 | TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
-| TCP | 443 | `35.160.43.145/32`,`35.167.242.46/32`,`52.33.59.17/32` | git.rancher.io (catalogs) |
+| TCP | 443 | git.rancher.io |  Rancher catalog |
 | TCP | 2376 | Any node IP from a node created using a node driver | Docker daemon TLS port used by Docker Machine |
 | TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |

@@ -47,14 +47,14 @@ If the Rancher server is installed in a single Docker container, you only need o
 ```
 sudo ssh -i [path-to-private-key] ubuntu@[public-DNS-of-instance]
 ```
-1. When you are connected to the instance, run the following command on the instance to create a user:
-```
-sudo usermod -aG docker ubuntu
-```
 1. Run the following command on the instance to install Docker with one of Rancher's installation scripts:
 ```
 curl https://releases.rancher.com/install-docker/18.09.sh | sh
 ```
+1. When you are connected to the instance, run the following command on the instance to add user `ubuntu` to group `docker`:
+```
+sudo usermod -aG docker ubuntu
+```
 1. Repeat these steps so that Docker is installed on each node that will eventually run the Rancher management server.

 > To find out whether a script is available for installing a certain Docker version, refer to this [GitHub repository,](https://github.com/rancher/install-docker) which contains all of Rancher’s Docker installation scripts.
@@ -14,7 +14,7 @@ Double check if all the [required ports]({{<baseurl>}}/rancher/v2.x/en/cluster-p

 The pod can be scheduled to any of the hosts you used for your cluster, but that means that the NGINX ingress controller needs to be able to route the request from `NODE_1` to `NODE_2`. This happens over the overlay network. If the overlay network is not functioning, you will experience intermittent TCP/HTTP connection failures due to the NGINX ingress controller not being able to route to the pod.

-To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/leodotcloud/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
+To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.

 1. Save the following file as `overlaytest.yml`

@@ -35,7 +35,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
          tolerations:
          - operator: Exists
          containers:
-          - image: leodotcloud/swiss-army-knife
+          - image: rancherlabs/swiss-army-knife
            imagePullPolicy: Always
            name: overlaytest
            command: ["sh", "-c", "tail -f /dev/null"]
@@ -18,7 +18,12 @@ There are a few things worth noting:
 * As of v0.1.8, RKE will update an add-on if it is the same name.
 * Before v0.1.8, update any add-ons by using `kubectl edit`.

-## Critical and Non-Critical Add-ons
+- [Critical and Non-Critical Add-ons](#critical-and-non-critical-add-ons)
+- [Add-on Deployment Jobs](#add-on-deployment-jobs)
+- [Add-on Placement](#add-on-placement)
+- [Tolerations](#tolerations)
+
+# Critical and Non-Critical Add-ons

 As of version v0.1.7, add-ons are split into two categories:

@@ -27,7 +32,7 @@ As of version v0.1.7, add-ons are split into two categories:

 Currently, only the [network plug-in]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/) is considered critical. KubeDNS, [ingress controllers]({{<baseurl>}}/rke/latest/en/config-options/add-ons/ingress-controllers/) and [user-defined add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/) are considered non-critical.

-## Add-on deployment jobs
+# Add-on Deployment Jobs

 RKE uses Kubernetes jobs to deploy add-ons. In some cases, add-ons deployment takes longer than expected. As of with version v0.1.7, RKE provides an option to control the job check timeout in seconds. This timeout is set at the cluster level.

@@ -35,7 +40,7 @@ RKE uses Kubernetes jobs to deploy add-ons. In some cases, add-ons deployment ta
 addon_job_timeout: 30
 ```

-## Add-on placement
+# Add-on Placement

 _Applies to v0.2.3 and higher_

@@ -50,7 +55,7 @@ _Applies to v0.2.3 and higher_
 | nginx-ingress      | - `beta.kubernetes.io/os:NotIn:windows`<br/>- `node-role.kubernetes.io/worker` `Exists` | none | - `NoSchedule:Exists`<br/>- `NoExecute:Exists` |
 | metrics-server     | - `beta.kubernetes.io/os:NotIn:windows`<br/>- `node-role.kubernetes.io/worker` `Exists` | none | - `NoSchedule:Exists`<br/>- `NoExecute:Exists` |

-## Tolerations
+# Tolerations

 _Available as of v1.2.4_

@@ -3,6 +3,25 @@ title: DNS providers
 weight: 262
 ---

+- [Available DNS Providers](#available-dns-providers)
+- [Disabling deployment of a DNS Provider](#disabling-deployment-of-a-dns-provider)
+- [CoreDNS](#coredns)
+  - [Scheduling CoreDNS](#scheduling-coredns)
+  - [Upstream nameservers](#coredns-upstream-nameservers)
+  - [Priority Class Name](#coredns-priority-class-name)
+  - [Tolerations](#coredns-tolerations)
+- [kube-dns](#kube-dns)
+  - [Scheduling kube-dns](#scheduling-kube-dns)
+  - [Upstream nameservers](#kube-dns-upstream-nameservers)
+  - [Priority Class Name](#kube-dns-priority-class-name)
+  - [Tolerations](#kube-dns-tolerations)
+- [NodeLocal DNS](#nodelocal-dns)
+  - [Configuring NodeLocal DNS](#configuring-nodelocal-dns)
+  - [Priority Class Name](#nodelocal-priority-class-name)
+  - [Removing NodeLocal DNS](#removing-nodelocal-dns)
+
+# Available DNS Providers
+
 RKE provides the following DNS providers that can be deployed as add-ons:

  * [CoreDNS](https://coredns.io)
@@ -18,6 +37,17 @@ CoreDNS was made the default in RKE v0.2.5 when using Kubernetes 1.14 and higher

 > **Note:** If you switch from one DNS provider to another, the existing DNS provider will be removed before the new one is deployed.

+# Disabling Deployment of a DNS Provider
+
+_Available as of v0.2.0_
+
+You can disable the default DNS provider by specifying `none` to  the dns `provider` directive in the cluster configuration. Be aware that this will prevent your pods from doing name resolution in your cluster.
+
+```yaml
+dns:
+  provider: none
+```
+
 # CoreDNS

 _Available as of v0.2.5_
@@ -28,7 +58,7 @@ RKE will deploy CoreDNS as a Deployment with the default replica count of 1. The

 The images used for CoreDNS are under the [`system_images` directive]({{<baseurl>}}/rke/latest/en/config-options/system-images/). For each Kubernetes version, there are default images associated with CoreDNS, but these can be overridden by changing the image tag in `system_images`.

-## Scheduling CoreDNS
+### Scheduling CoreDNS

 If you only want the CoreDNS pod to be deployed on specific nodes, you can set a `node_selector` in the `dns` section. The label in the `node_selector` would need to match the label on the nodes for the CoreDNS pod to be deployed.

@@ -46,9 +76,8 @@ dns:
    app: dns
 ```

-## Configuring CoreDNS

-### Upstream nameservers
+### CoreDNS Upstream nameservers

 By default, CoreDNS will use the host configured nameservers (usually residing at `/etc/resolv.conf`) to resolve external queries. If you want to configure specific upstream nameservers to be used by CoreDNS, you can use the `upstreamnameservers` directive.

@@ -62,13 +91,28 @@ dns:
  - 8.8.4.4
 ```

-### Tolerations
+
+### CoreDNS Priority Class Name
+
+_Available as of RKE v1.2.6+_
+
+The [pod priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#pod-priority) is set by configuring a priority class name under `options`:
+
+```yaml
+dns:
+    options:
+      coredns_autoscaler_priority_class_name: system-cluster-critical
+      coredns_priority_class_name: system-cluster-critical
+    provider: coredns
+```
+
+### CoreDNS Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `coredns` and the `coredns-autoscaler` Deployment.

-```
+```yaml
 dns:
  provider: coredns
  tolerations:
@@ -95,7 +139,7 @@ RKE will deploy kube-dns as a Deployment with the default replica count of 1. Th

 The images used for kube-dns are under the [`system_images` directive]({{<baseurl>}}/rke/latest/en/config-options/system-images/). For each Kubernetes version, there are default images associated with kube-dns, but these can be overridden by changing the image tag in `system_images`.

-## Scheduling kube-dns
+### Scheduling kube-dns

 _Available as of v0.2.0_

@@ -115,9 +159,7 @@ dns:
    app: dns
 ```

-## Configuring kube-dns
-
-### Upstream nameservers
+### kube-dns Upstream nameservers

 _Available as of v0.2.0_

@@ -133,13 +175,28 @@ dns:
  - 8.8.4.4
 ```

-### Tolerations
+### kube-dns Priority Class Name
+
+_Available as of RKE v1.2.6+_
+
+The [pod priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#pod-priority) is set by configuring a priority class name under `options`:
+
+```yaml
+dns:
+    options:
+      kube_dns_autoscaler_priority_class_name: system-cluster-critical
+      kube_dns_priority_class_name: system-cluster-critical
+    provider: kube-dns
+```
+
+
+### kube-dns Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `kube-dns` and the `kube-dns-autoscaler` Deployment.

-```
+```yaml
 dns:
  provider: kube-dns
  tolerations:
@@ -161,16 +218,7 @@ kubectl get deploy kube-dns -n kube-system -o jsonpath='{.spec.template.spec.tol
 kubectl get deploy kube-dns-autoscaler -n kube-system -o jsonpath='{.spec.template.spec.tolerations}'
 ```

-# Disabling deployment of a DNS provider

-_Available as of v0.2.0_
-
-You can disable the default DNS provider by specifying `none` to  the dns `provider` directive in the cluster configuration. Be aware that this will prevent your pods from doing name resolution in your cluster.
-
-```yaml
-dns:
-  provider: none
-```

 # NodeLocal DNS

@@ -186,7 +234,7 @@ NodeLocal DNS is an additional component that can be deployed on each node to im

 Enable NodeLocal DNS by configuring an IP address.

-## Configuring NodeLocal DNS
+### Configuring NodeLocal DNS

 The `ip_address` parameter is used to configure what link-local IP address will be configured one each host to listen on, make sure this IP address is not already configured on the host.

@@ -199,7 +247,21 @@ dns:

 > **Note:** When enabling NodeLocal DNS on an existing cluster, pods that are currently running will not be modified, the updated `/etc/resolv.conf` configuration will take effect only for pods started after enabling NodeLocal DNS.

-## Removing NodeLocal DNS
+### NodeLocal Priority Class Name
+
+_Available as of RKE v1.2.6+_
+
+The [pod priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#pod-priority) is set by configuring a priority class name under `options`:
+
+```yaml
+dns:
+    options:
+      nodelocal_autoscaler_priority_class_name: system-cluster-critical
+      nodelocal_priority_class_name: system-cluster-critical
+    provider: coredns # a DNS provider must be configured
+```
+
+### Removing NodeLocal DNS

 By removing the `ip_address` value, NodeLocal DNS will be removed from the cluster.

@@ -4,6 +4,17 @@ description: By default, RKE deploys the NGINX ingress controller. Learn how to
 weight: 262
 ---

+- [Default Ingress](#default-ingress)
+- [Scheduling Ingress Controllers](#scheduling-ingress-controllers)
+- [Ingress Priority Class Name](#ingress-priority-class-name)
+- [Tolerations](#tolerations)
+- [Disabling the Default Ingress Controller](#disabling-the-default-ingress-controller)
+- [Configuring NGINX Ingress Controller](#configuring-nginx-ingress-controller)
+- [Disabling NGINX Ingress Default Backend](#disabling-nginx-ingress-default-backend)
+- [Configuring an NGINX Default Certificate](#configuring-an-nginx-default-certificate)
+
+### Default Ingress
+
 By default, RKE deploys the NGINX ingress controller on all schedulable nodes.

 > **Note:** As of v0.1.8, only workers are considered schedulable nodes, but before v0.1.8, worker and controlplane nodes were considered schedulable nodes.  
@@ -12,7 +23,7 @@ RKE will deploy the ingress controller as a DaemonSet with `hostnetwork: true`,

 The images used for ingress controller is under the [`system_images` directive]({{<baseurl>}}/rke/latest/en/config-options/system-images/). For each Kubernetes version, there are default images associated with the ingress controller, but these can be overridden by changing the image tag in `system_images`.

-## Scheduling Ingress Controllers
+### Scheduling Ingress Controllers

 If you only wanted ingress controllers to be deployed on specific nodes, you can set a `node_selector` for the ingress. The label in the `node_selector` would need to match the label on the nodes for the ingress controller to be deployed.

@@ -30,13 +41,25 @@ ingress:
    app: ingress
 ```

-## Tolerations
+### Ingress Priority Class Name
+
+_Available as of RKE v1.2.6+_
+
+The [pod priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#pod-priority) is set by configuring a priority class name:
+
+```yaml
+ingress:
+  provider: nginx
+  ingress_priority_class_name: system-cluster-critical
+```
+
+### Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `default-http-backend` Deployment.

-```
+```yaml
 ingress:
  tolerations:
  - key: "node.kubernetes.io/unreachable"
@@ -55,7 +78,7 @@ To check for applied tolerations `default-http-backend` Deployment, use the foll
 kubectl -n ingress-nginx get deploy default-http-backend -o jsonpath='{.spec.template.spec.tolerations}'
 ```

-## Disabling the Default Ingress Controller
+### Disabling the Default Ingress Controller

 You can disable the default controller by specifying `none` to  the ingress `provider` directive in the cluster configuration.

@@ -63,7 +86,7 @@ You can disable the default controller by specifying `none` to  the ingress `pro
 ingress:
    provider: none
 ```
-## Configuring NGINX Ingress Controller
+### Configuring NGINX Ingress Controller

 For the configuration of NGINX, there are configuration options available in Kubernetes. There are a [list of options for the NGINX config map](https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/configmap.md) , [command line extra_args](https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/cli-arguments.md) and [annotations](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/).

@@ -88,7 +111,7 @@ ingress:

 > **What happens if the field is omitted?** The value of `default_backend` will default to `true`. This maintains behavior with older versions of `rke`. However, a future version of `rke` will change the default value to `false`.

-## Configuring an NGINX Default Certificate
+### Configuring an NGINX Default Certificate

 When configuring an ingress object with TLS termination, you must provide it with a certificate used for encryption/decryption. Instead of explicitly defining a certificate each time you configure an ingress, you can set up a custom certificate that's used by default.

@@ -9,13 +9,17 @@ RKE will deploy Metrics Server as a Deployment.

 The image used for Metrics Server is under the [`system_images` directive]({{<baseurl>}}/rke/latest/en/config-options/system-images/). For each Kubernetes version, there is a default image associated with the Metrics Server, but these can be overridden by changing the image tag in `system_images`.

-## Tolerations
+- [Tolerations](#tolerations)
+- [Priority Class Name](#metrics-server-priority-class-name)
+- [Disabling the Metrics Server](#disabling-the-metrics-server)
+
+### Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `metrics-server` Deployment.

-```
+```yaml
 monitoring:
  tolerations:
  - key: "node.kubernetes.io/unreachable"
@@ -34,7 +38,19 @@ To check for applied tolerations on the `metrics-server` Deployment, use the fol
 kubectl -n kube-system get deploy metrics-server -o jsonpath='{.spec.template.spec.tolerations}'
 ```

-## Disabling the Metrics Server
+### Metrics Server Priority Class Name
+
+_Available as of RKE v1.2.6+_
+
+The [pod priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#pod-priority) is set by configuring a priority class name:
+
+```yaml
+monitoring:
+  provider: metrics-server
+  metrics_server_priority_class_name: system-cluster-critical
+```
+
+### Disabling the Metrics Server

 _Available as of v0.2.0_

@@ -10,7 +10,27 @@ RKE provides the following network plug-ins that are deployed as add-ons:
 - Canal
 - Weave

-> **Note:** After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn’t allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
+> After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn’t allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
+
+- [Changing the Default Network Plug-in](#changing-the-default-network-plug-in)
+- [Disabling Deployment of a Network Plug-in](#disabling-deployment-of-a-network-plug-in)
+- [Network Plug-in Options](#network-plug-in-options)
+- [Canal](#canal)
+  - [Canal Network Plug-in Options](#canal-network-plug-in-options)
+  - [Canal Interface](#canal-interface)
+  - [Canal Network Plug-in Tolerations](#canal-network-plug-in-tolerations)
+- [Flannel](#flannel)
+  - [Flannel Network Plug-in Options](#flannel-network-plug-in-options)
+  - [Flannel Interface](#flannel-interface)
+- [Calico](#calico)
+  - [Calico Network Plug-in Options](#calico-network-plug-in-options)
+  - [Calico Cloud Provider](#calico-cloud-provider)
+  - [Calico Network Plug-in Tolerations](#calico-network-plug-in-tolerations)
+- [Weave](#weave)
+  - [Weave Network Plug-in Options](#weave-network-plug-in-options)
+- [Custom Network Plug-ins](#custom-network-plug-ins)
+
+# Changing the Default Network Plug-in

 By default, the network plug-in is `canal`. If you want to use another network plug-in, you need to specify which network plug-in to enable at the cluster level in the `cluster.yml`.

@@ -35,7 +55,14 @@ network:

 Besides the different images that could be used to deploy network plug-ins, certain network plug-ins support additional options that can be used to customize the network plug-in.

-## Canal Network Plug-in Options
+- [Canal](#canal)
+- [Flannel](#flannel)
+- [Calico](#calico)
+- [Weave](#weave)
+
+# Canal
+
+### Canal Network Plug-in Options

 ```yaml
 network:
@@ -43,20 +70,23 @@ network:
  options:
    canal_iface: eth1
    canal_flannel_backend_type: vxlan
+    canal_autoscaler_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
+    canal_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
 ```

-#### Canal Interface
+### Canal Interface

 By setting the `canal_iface`, you can configure the interface to use for inter-host communication.
+
 The `canal_flannel_backend_type` option allows you to specify the type of [flannel backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md) to use. By default the `vxlan` backend is used.

-## Canal Network Plug-in Tolerations
+### Canal Network Plug-in Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `calico-kube-controllers` Deployment.

-```
+```yaml
 network:
  plugin: canal
  tolerations:
@@ -76,7 +106,8 @@ To check for applied tolerations on the `calico-kube-controllers` Deployment, us
 kubectl -n kube-system get deploy calico-kube-controllers -o jsonpath='{.spec.template.spec.tolerations}'
 ```

-## Flannel Network Plug-in Options
+# Flannel
+### Flannel Network Plug-in Options

 ```yaml
 network:
@@ -84,22 +115,29 @@ network:
  options:
    flannel_iface: eth1
    flannel_backend_type: vxlan
+    flannel_autoscaler_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
+    flannel_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
 ```

-#### Flannel Interface
+### Flannel Interface

 By setting the `flannel_iface`, you can configure the interface to use for inter-host communication.
 The `flannel_backend_type` option allows you to specify the type of [flannel backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md) to use. By default the `vxlan` backend is used.

-## Calico Network Plug-in Options
+
+# Calico
+
+### Calico Network Plug-in Options

 ```yaml
 network:
  plugin: calico
  options:
    calico_cloud_provider: aws
+    calico_autoscaler_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
+    calico_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
 ```
-#### Calico Cloud Provider
+### Calico Cloud Provider

 Calico currently only supports 2 cloud providers, AWS or GCE, which can be set using `calico_cloud_provider`.

@@ -108,13 +146,13 @@ Calico currently only supports 2 cloud providers, AWS or GCE, which can be set u
 - `aws`
 - `gce`

-## Calico Network Plug-in Tolerations
+### Calico Network Plug-in Tolerations

 _Available as of v1.2.4_

 The configured tolerations apply to the `calico-kube-controllers` Deployment.

-```
+```yaml
 network:
  plugin: calico
  tolerations:
@@ -134,19 +172,23 @@ To check for applied tolerations on the `calico-kube-controllers` Deployment, us
 kubectl -n kube-system get deploy calico-kube-controllers -o jsonpath='{.spec.template.spec.tolerations}'
 ```

-## Weave Network Plug-in Options
+# Weave
+### Weave Network Plug-in Options

 ```yaml
 network:
  plugin: weave
+  options:
+    weave_autoscaler_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
+    weave_priority_class_name: system-cluster-critical # Available as of RKE v1.2.6+
  weave_network_provider:
    password: "Q]SZOQ5wp@n$oijz"
 ```

-#### Weave encryption
+### Weave Encryption

 Weave encryption can be enabled by passing a string password to the network provider config.

-## Custom Network Plug-ins
+# Custom Network Plug-ins

 It is possible to add a custom network plug-in by using the [user-defined add-on functionality]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. In the `addons` field, you can add the add-on manifest of a cluster that has the network plugin-that you want, as shown in [this example.]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
@@ -37,7 +37,7 @@ rancher_kubernetes_engine_config:
      workspace:
        server: vc.example.com
        folder: myvmfolder
-        default-datastore: /eu-west-1/datastore/ds-1
+        default-datastore: ds-1
        datacenter: /eu-west-1
        resourcepool-path: /eu-west-1/host/hn1/resources/myresourcepool

@@ -176,7 +176,7 @@ Consult the project pages for openSUSE MicroOS and Kubic for installation
 Designed to host container workloads with automated administration & patching. Installing openSUSE MicroOS you get a quick, small environment for deploying Containers, or any other workload that benefits from Transactional Updates. As rolling release distribution the software is always up-to-date.  
 https://microos.opensuse.org  
 #### openSUSE Kubic
-Based on MicroOS, but not a rolling release distribution. Designed with the same things in mind but also a Certified Kubernetes Distribution.   
+Based on openSUSE MicroOS, designed with the same things in mind but is focused on being a Certified Kubernetes Distribution.   
 https://kubic.opensuse.org  
 Installation instructions:  
 https://kubic.opensuse.org/blog/2021-02-08-MicroOS-Kubic-Rancher-RKE/  
@@ -18,7 +18,7 @@
          <td></td>
          <td></td>
          <td></td>
-          <td style="background-color: #3497DA; color:#ffffff;">git.rancher.io <sup>(2)</sup>:<br>35.160.43.145:32<br>35.167.242.46:32<br>52.33.59.17:32</td>
+          <td style="background-color: #3497DA; color:#ffffff;">git.rancher.io</td>
        </tr>
        <tr>
          <td rowspan="6">etcd Plane Nodes</td>
@@ -16,7 +16,7 @@
          <td></td>
          <td colspan="3" style="background-color: #3497DA; color:#ffffff;">22 TCP</td>
          <td></td>
-          <td rowspan="2" style="background-color: #3497DA; color:#ffffff;">git.rancher.io <sup>(2)</sup>:<br>35.160.43.145:32<br>35.167.242.46:32<br>52.33.59.17:32</td>
+          <td rowspan="2" style="background-color: #3497DA; color:#ffffff;">git.rancher.io</td>
        </tr>
        <tr>
          <td></td>
@@ -14,7 +14,7 @@
        <td></td>
        <td style="background-color: #3497DA; color:#ffffff;">Kubernetes API <br>Endpoint Port <sup>(2)</sup></td>
        <td></td>
-        <td rowspan="3" style="background-color: #3497DA; color:#ffffff;">git.rancher.io <sup>(3)</sup>:<br>35.160.43.145:32<br>35.167.242.46:32<br>52.33.59.17:32</td>
+        <td rowspan="3" style="background-color: #3497DA; color:#ffffff;">git.rancher.io</td>
        </tr>
        <tr>
        <td></td>