Merge pull request #297 from btat/main-to-release/v2.7

Main to release/v2.7
2026-09-25 20:48:11 +00:00 · 2022-11-16 10:33:27 -08:00
parent 3597c79a43 421d791dd2
commit 77fb55f742
257 changed files with 1947 additions and 1888 deletions
@@ -0,0 +1,16 @@
+---
+name: Request a New Feature
+about: For requesting new feature(s) to be added to the docs.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Summary:**
+Describe the new feature. If QA has not yet tested the new feature/process, please also file a ticket with QA for their review.
+
+
+**Details:**
+- Include all pertinent information, e.g., screenshots, resource requirements, workarounds, links, etc.
+- List page link(s) in the current docs where the new feature applies, if applicable.
@@ -0,0 +1,11 @@
+---
+name: Request an Update
+about: For fixing docs errors/typos, adding needed/missing information, updating verbiage, deleting outdated info, etc.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Summary:**
+Describe the requested update giving as much detail as possible. Please also list page link(s) in the current docs where the update applies.
@@ -72,7 +72,7 @@ Use `kubectl` to apply the rendered manifests.
 kubectl -n cattle-system apply -R -f ./rancher
 ```

-# Verify the Upgrade
+## Verify the Upgrade

 Log into Rancher to confirm that the upgrade succeeded.

@@ -84,6 +84,6 @@ See [Restoring Cluster Networking](../../../../versioned_docs/version-2.0-2.4/ge

 :::

-# Known Upgrade Issues
+## Known Upgrade Issues

 A list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
@@ -168,7 +168,7 @@ kubectl -n cattle-system create secret generic tls-ca-additional --from-file=ca-

 For details on installing Rancher with a private registry, see the [air gap installation docs.](../../../pages-for-subheaders/air-gapped-helm-cli-install.md)

-# External TLS Termination
+## External TLS Termination

 We recommend configuring your load balancer as a Layer 4 balancer, forwarding plain 80/tcp and 443/tcp to the Rancher Management cluster nodes. The Ingress Controller on the cluster will redirect http traffic on port 80 to https on port 443.

@@ -4,14 +4,14 @@ title: TLS Settings

 Changing the default TLS settings depends on the chosen installation method.

-# Running Rancher in a highly available Kubernetes cluster
+## Running Rancher in a highly available Kubernetes cluster

 When you install Rancher inside of a Kubernetes cluster, TLS is offloaded at the cluster's ingress controller. The possible TLS settings depend on the used ingress controller:

 * nginx-ingress-controller (default for RKE1 and RKE2): [Default TLS Version and Ciphers](https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-tls-version-and-ciphers).
 * traefik (default for K3s): [TLS Options](https://doc.traefik.io/traefik/https/tls/#tls-options).

-# Running Rancher in a single Docker container
+## Running Rancher in a single Docker container

 The default TLS configuration only accepts TLS 1.2 and secure TLS cipher suites. You can change this by setting the following environment variables:

@@ -11,7 +11,7 @@ import PortsImportedHosted from '@site/src/components/PortsImportedHosted'

 To operate properly, Rancher requires a number of ports to be open on Rancher nodes and on downstream Kubernetes cluster nodes.

-# Rancher Nodes
+## Rancher Nodes

 The following table lists the ports that need to be open to and from nodes that are running the Rancher server.

@@ -174,7 +174,7 @@ The following tables break down the port requirements for Rancher nodes, for inb

 </details>

-# Downstream Kubernetes Cluster Nodes
+## Downstream Kubernetes Cluster Nodes

 Downstream Kubernetes clusters run your apps and services. This section describes what ports need to be opened on the nodes in downstream clusters so that Rancher can communicate with them.

@@ -249,7 +249,7 @@ The following table depicts the port requirements for [registered clusters](../.
 </details>


-# Other Port Considerations
+## Other Port Considerations

 ### Commonly Used Ports

@@ -74,7 +74,7 @@ When setting up the Rancher Helm template, there are several options in the Helm
 | `systemDefaultRegistry` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters.  |
 | `useBundledSystemChart` | `true`                           | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. |

-### 3. Render the Rancher Helm Template
+### 3. Fetch the Cert-Manager chart 

 Based on the choice your made in [2. Choose your SSL Configuration](#2-choose-your-ssl-configuration), complete one of the procedures below.

@@ -112,30 +112,62 @@ New in v2.6.4, cert-manager versions 1.6.2 and 1.7.1 are compatible. We recommen
 helm fetch jetstack/cert-manager --version v1.7.1
 ```

-##### 3. Render the cert-manager template

-Render the cert-manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files.
-
-```plain
-helm template cert-manager ./cert-manager-v1.7.1.tgz --output-dir . \
-    --namespace cert-manager \
-    --set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller \
-    --set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook \
-    --set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector \
-    --set startupapicheck.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-ctl
-```
-
-##### 4. Download the cert-manager CRD
+##### 3. Retrieve the Cert-Manager CRDs

 Download the required CRD file for cert-manager:
   ```plain
   curl -L -o cert-manager/cert-manager-crd.yaml https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.crds.yaml
   ```

-##### 5. Render the Rancher template
+### 4. Install Rancher

-Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
+Copy the fetched charts to a system that has access to the Rancher server cluster to complete installation.

+##### 1. Install Cert-Manager
+
+Install cert-manager with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files.
+
+<details id="install-cert-manager">
+  <summary>Click to expand</summary>
+
+If you are using self-signed certificates, install cert-manager:
+
+1. Create the namespace for cert-manager.
+
+    ```plain
+    kubectl create namespace cert-manager
+    ```
+
+2. Create the cert-manager CustomResourceDefinitions (CRDs). 
+
+    ```plain
+    kubectl apply -f cert-manager/cert-manager-crd.yaml
+    ```
+
+3. Install cert-manager.
+
+    ```plain
+    helm install cert-manager ./cert-manager-v1.7.1.tgz \
+        --namespace cert-manager \
+        --set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller \
+        --set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook \
+        --set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector \
+        --set startupapicheck.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-ctl
+    ```
+
+</details>
+
+##### 2. Install Rancher
+First, refer to [Adding TLS Secrets](../../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
+
+Then, create the namespace for Rancher using kubectl:
+
+```plain
+kubectl create namespace cattle-system
+```
+
+Next, install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.

 Placeholder | Description
 ------------|-------------
@@ -145,8 +177,7 @@ Placeholder | Description
 `<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.

 ```plain
-helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
-    --no-hooks \ # prevent files for Helm hooks from being generated
+   helm install rancher ./rancher-<VERSION>.tgz \
    --namespace cattle-system \
    --set hostname=<RANCHER.YOURDOMAIN.COM> \
    --set certmanager.version=<CERTMANAGER_VERSION> \
@@ -159,14 +190,13 @@ helm template rancher ./rancher-<VERSION>.tgz --output-dir . \

 #### Option B: Certificates From Files using Kubernetes Secrets

-
 ##### 1. Create secrets

 Create Kubernetes secrets from your own certificates for Rancher to use. The common name for the cert will need to match the `hostname` option in the command below, or the ingress controller will fail to provision the site for Rancher.

-##### 2. Render the Rancher template
+##### 2. Install Rancher

-Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
+Install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.

 | Placeholder                      | Description                                     |
 | -------------------------------- | ----------------------------------------------- |
@@ -175,8 +205,7 @@ Render the Rancher template, declaring your chosen options. Use the reference ta
 | `<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.         |

 ```plain
-   helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
-    --no-hooks \ # prevent files for Helm hooks from being generated
+   helm install rancher ./rancher-<VERSION>.tgz --output-dir . \
    --namespace cattle-system \
    --set hostname=<RANCHER.YOURDOMAIN.COM> \
    --set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
@@ -188,8 +217,7 @@ Render the Rancher template, declaring your chosen options. Use the reference ta
 If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret`:

 ```plain
-   helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
-    --no-hooks \ # prevent files for Helm hooks from being generated
+   helm install rancher ./rancher-<VERSION>.tgz --output-dir . \
    --namespace cattle-system \
    --set hostname=<RANCHER.YOURDOMAIN.COM> \
    --set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
@@ -199,55 +227,8 @@ If you are using a Private CA signed cert, add `--set privateCA=true` following
    --set useBundledSystemChart=true # Use the packaged Rancher system charts
 ```

-**Optional**: To install a specific Rancher version, set the `rancherImageTag` value, example: `--set rancherImageTag=v2.3.6`

-Then refer to [Adding TLS Secrets](../../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
-
-### 4. Install Rancher
-
-Copy the rendered manifest directories to a system that has access to the Rancher server cluster to complete installation.
-
-Use `kubectl` to create namespaces and apply the rendered manifests.
-
-If you choose to use self-signed certificates in [B. Choose your SSL Configuration](#b-choose-your-ssl-configuration), install cert-manager.
-
-#### For Self-Signed Certificate Installs, Install Cert-manager
-
-<details id="install-cert-manager">
-  <summary>Click to expand</summary>
-
-If you are using self-signed certificates, install cert-manager:
-
-1. Create the namespace for cert-manager.
-    ```plain
-    kubectl create namespace cert-manager
-    ```
-2. Create the cert-manager CustomResourceDefinitions (CRDs).
-    ```plain
-    kubectl apply -f cert-manager/cert-manager-crd.yaml
-    ```
-
-    :::note
-
-    If you are running Kubernetes v1.15 or below, you will need to add the `--validate=false` flag to your `kubectl apply` command above, or else you will receive a validation error relating to the `x-kubernetes-preserve-unknown-fields` field in cert-manager’s CustomResourceDefinition resources. This is a benign error and occurs due to the way kubectl performs resource validation.
-
-    :::
-
-3. Launch cert-manager.
-    ```plain
-    kubectl apply -R -f ./cert-manager
-    ```
-
-</details>
-
-#### Install Rancher with kubectl
-
-```plain
-kubectl create namespace cattle-system
-kubectl -n cattle-system apply -R -f ./rancher
-```
 The installation is complete.
-
 :::caution

 If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login. Leaving this active in an air-gapped environment can cause issues if the sockets cannot be opened successfully.
@@ -127,7 +127,7 @@ The `rancher-images.txt` is expected to be on the workstation in the same direct

 For Rancher servers that will provision Linux and Windows clusters, there are distinctive steps to populate your private registry for the Windows images and the Linux images. Since a Windows cluster is a mix of Linux and Windows nodes, the Linux images pushed into the private registry are manifests.

-# Windows Steps
+## Windows Steps

 The Windows images need to be collected and pushed from a Windows server workstation.

@@ -206,7 +206,7 @@ The `rancher-windows-images.txt` is expected to be on the workstation in the sam
   ./rancher-load-images.ps1 --registry <REGISTRY.YOURDOMAIN.COM:PORT>
   ```

-# Linux Steps
+## Linux Steps

 The Linux images need to be collected and pushed from a Linux host, but _must be done after_ populating the Windows images into the private registry. These step are different from the Linux only steps as the Linux images that are pushed will actually manifests that support Windows and Linux images.

@@ -2,7 +2,7 @@
 title: '1. Set up Infrastructure'
 ---

-In this section, you will provision the underlying infrastructure for your Rancher management server with internete access through a HTTP proxy.
+In this section, you will provision the underlying infrastructure for your Rancher management server with internet access through a HTTP proxy.

 To install the Rancher management server on a high-availability RKE cluster, we recommend setting up the following infrastructure:

@@ -23,7 +23,7 @@ If you want to replace the certificate, you can delete the `tls-rancher-ingress`

 :::

-# Using a Private CA Signed Certificate
+## Using a Private CA Signed Certificate

 If you are using a private CA, Rancher requires a copy of the CA certificate which is used by the Rancher Agent to validate the connection to the server.

@@ -40,6 +40,6 @@ The configured `tls-ca` secret is retrieved when Rancher starts. On a running Ra

 :::

-# Updating a Private CA Certificate
+## Updating a Private CA Certificate

 Follow the steps on [this page](update-rancher-certificate.md) to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation](../../../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md) or to switch from the default self-signed certificate to a custom certificate.
@@ -6,7 +6,7 @@ The [System Charts](https://github.com/rancher/system-charts) repository contain

 In an air gapped installation of Rancher, you will need to configure Rancher to use a local copy of the system charts. This section describes how to use local system charts using a CLI flag.

-# Using Local System Charts
+## Using Local System Charts

 A local copy of `system-charts` has been packaged into the `rancher/rancher` container. To be able to use these features in an air gap install, you will need to run the Rancher install command with an extra environment variable, `CATTLE_SYSTEM_CATALOG=bundled`, which tells Rancher to use the local copy of the charts instead of attempting to fetch them from GitHub.

@@ -2,7 +2,7 @@
 title: Updating the Rancher Certificate
 ---

-# Updating a Private CA Certificate
+## Updating a Private CA Certificate

 Follow these steps to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation](../../../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md) or to switch from the default self-signed certificate to a custom certificate.

@@ -16,7 +16,7 @@ A summary of the steps is as follows:

 The details of these instructions are below.

-## 1. Create/update the certificate secret resource
+### 1. Create/update the certificate secret resource

 First, concatenate the server certificate followed by any intermediate certificate(s) to a file named `tls.crt` and provide the corresponding certificate key in a file named `tls.key`.

@@ -37,7 +37,7 @@ $ kubectl -n cattle-system create secret tls tls-rancher-ingress \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-## 2. Create/update the CA certificate secret resource
+### 2. Create/update the CA certificate secret resource

 If the new certificate was signed by a private CA, you will need to copy the corresponding root CA certificate into a file named `cacerts.pem` and create or update the `tls-ca secret` in the `cattle-system` namespace. If the certificate was signed by an intermediate CA, then the `cacerts.pem` must contain both the intermediate and root CA certificates (in this order).

@@ -56,7 +56,7 @@ $ kubectl -n cattle-system create secret generic tls-ca \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-## 3. Reconfigure the Rancher deployment
+### 3. Reconfigure the Rancher deployment

 :::note

@@ -95,18 +95,18 @@ helm upgrade rancher rancher-stable/rancher \

 When the upgrade is completed, navigate to `https://<Rancher_SERVER>/v3/settings/cacerts` to verify that the value matches the CA certificate written in the `tls-ca` secret earlier.

-## 4. Reconfigure Rancher agents to trust the private CA
+### 4. Reconfigure Rancher agents to trust the private CA

 This section covers three methods to reconfigure Rancher agents to trust the private CA. This step is required if either of the following is true:

 - Rancher was initially configured to use the Rancher self-signed certificate (`ingress.tls.source=rancher`) or with a Let's Encrypt issued certificate (`ingress.tls.source=letsEncrypt`)
 - The root CA certificate for the new custom certificate has changed

-### Why is this step required?
+#### Why is this step required?

 When Rancher is configured with a certificate signed by a private CA, the CA certificate chain is downloaded into Rancher agent containers. Agents compare the checksum of the downloaded certificate against the `CATTLE_CA_CHECKSUM` environment variable. This means that, when the private CA certificate is changed on Rancher server side, the environvment variable `CATTLE_CA_CHECKSUM` must be updated accordingly.

-### Which method should I choose?
+#### Which method should I choose?

 Method 1 is the easiest one but requires all clusters to be connected to Rancher after the certificates have been rotated. This is usually the case if the process is performed right after updating the Rancher deployment (Step 3).

@@ -114,7 +114,7 @@ If the clusters have lost connection to Rancher but you have [Authorized Cluster

 Method 3 can be used as a fallback if method 1 and 2 are unfeasible.

-### Method 1: Kubectl command
+#### Method 1: Kubectl command

 For each cluster under Rancher management (except the `local` Rancher management cluster) run the following command using the Kubeconfig file of the Rancher management cluster (RKE or K3S).

@@ -125,7 +125,7 @@ kubectl patch clusters.management.cattle.io <REPLACE_WITH_CLUSTERID> -p '{"statu
 This command will cause all Agent Kubernetes resources to be reconfigured with the checksum of the new certificate.


-### Method 2: Manually update checksum
+#### Method 2: Manually update checksum

 Manually patch the agent Kubernetes resources by updating the `CATTLE_CA_CHECKSUM` environment variable to the value matching the checksum of the new CA certificate. Generate the new checksum value like so:

@@ -141,7 +141,7 @@ $ kubectl edit -n cattle-system ds/cattle-node-agent
 $ kubectl edit -n cattle-system deployment/cattle-cluster-agent
 ```

-### Method 3: Recreate Rancher agents
+#### Method 3: Recreate Rancher agents

 With this method you are recreating the Rancher agents by running a set of commands on a controlplane node of each downstream cluster.

@@ -150,15 +150,15 @@ First, generate the agent definitions as described here: https://gist.github.com
 Then, connect to a controlplane node of the downstream cluster via SSH, create a Kubeconfig and apply the definitions:
 https://gist.github.com/superseb/b14ed3b5535f621ad3d2aa6a4cd6443b

-## 5. Select Force Update of Fleet clusters to connect fleet-agent to Rancher
+### 5. Select Force Update of Fleet clusters to connect fleet-agent to Rancher

 Select 'Force Update' for the clusters within the [Continuous Delivery](../../../how-to-guides/new-user-guides/deploy-apps-across-clusters/fleet.md#accessing-fleet-in-the-rancher-ui) view of the Rancher UI to allow the fleet-agent in downstream clusters to successfully connect to Rancher.

-### Why is this step required?
+#### Why is this step required?

 Fleet agents in Rancher managed clusters store kubeconfig that is used to connect to the Rancher proxied kube-api in the fleet-agent secret of the fleet-system namespace. The kubeconfig contains a certificate-authority-data block containing the Rancher CA. When changing the Rancher CA, this block needs to be updated for a successful connection of the fleet-agent to Rancher.

-# Updating from a Private CA Certificate to a Common Certificate
+## Updating from a Private CA Certificate to a Common Certificate

 :::note

@@ -166,7 +166,7 @@ It is possible to perform the opposite procedure as shown above: you may change

 :::

-## 1. Create/update the certificate secret resource
+### 1. Create/update the certificate secret resource

 First, concatenate the server certificate followed by any intermediate certificate(s) to a file named `tls.crt` and provide the corresponding certificate key in a file named `tls.key`.

@@ -187,7 +187,7 @@ $ kubectl -n cattle-system create secret tls tls-rancher-ingress \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-## 2. Delete the CA certificate secret resource
+### 2. Delete the CA certificate secret resource

 You will delete the `tls-ca secret` in the `cattle-system` namespace as it is no longer needed. You may also optionally save a copy of the `tls-ca secret` if desired.

@@ -203,7 +203,7 @@ To delete the existing `tls-ca` secret:
 kubectl -n cattle-system delete secret tls-ca
 ```

-## 3. Reconfigure the Rancher deployment
+### 3. Reconfigure the Rancher deployment

 :::note Important:

@@ -249,14 +249,14 @@ On upgrade, you can either
 set privateCA=false
 ```

-## 4. Reconfigure Rancher agents for the non-private/common certificate
+### 4. Reconfigure Rancher agents for the non-private/common certificate

 `CATTLE_CA_CHECKSUM` environment variable on the downstream cluster agents should be removed or set to "" (an empty string).

-## 5. Select Force Update of Fleet clusters to connect fleet-agent to Rancher
+### 5. Select Force Update of Fleet clusters to connect fleet-agent to Rancher

 Select 'Force Update' for the clusters within the [Continuous Delivery](../../../how-to-guides/new-user-guides/deploy-apps-across-clusters/fleet.md#accessing-fleet-in-the-rancher-ui) view of the Rancher UI to allow the fleet-agent in downstream clusters to successfully connect to Rancher.

-### Why is this step required?
+#### Why is this step required?

 Fleet agents in Rancher managed clusters store kubeconfig that is used to connect to the Rancher proxied kube-api in the fleet-agent secret of the fleet-system namespace. The kubeconfig contains a certificate-authority-data block containing the Rancher CA. When changing the Rancher CA, this block needs to be updated for a successful connection of the fleet-agent to Rancher.
@@ -19,7 +19,7 @@ If you are upgrading cert-manager to the latest version from a version older tha

 :::

-# Upgrade Cert-Manager
+## Upgrade Cert-Manager

 The namespace used in these instructions depends on the namespace cert-manager is currently installed in. If it is in kube-system use that in the instructions below. You can verify by running `kubectl get pods --all-namespaces` and checking which namespace the cert-manager-\* pods are listed in. Do not change the namespace cert-manager is running in or this can cause issues.

@@ -4,11 +4,11 @@ title: Overview

 Rancher is a container management platform built for organizations that deploy containers in production. Rancher makes it easy to run Kubernetes everywhere, meet IT requirements, and empower DevOps teams.

-# Run Kubernetes Everywhere
+## Run Kubernetes Everywhere

 Kubernetes has become the container orchestration standard. Most cloud and virtualization vendors now offer it as standard infrastructure. Rancher users have the choice of creating Kubernetes clusters with Rancher Kubernetes Engine (RKE) or cloud Kubernetes services, such as GKE, AKS, and EKS. Rancher users can also import and manage their existing Kubernetes clusters created using any Kubernetes distribution or installer.

-# Meet IT Requirements
+## Meet IT Requirements

 Rancher supports centralized authentication, access control, and monitoring for all Kubernetes clusters under its control. For example, you can:

@@ -16,7 +16,7 @@ Rancher supports centralized authentication, access control, and monitoring for
 - Setup and enforce access control and security policies across all users, groups, projects, clusters, and clouds.
 - View the health and capacity of your Kubernetes clusters from a single-pane-of-glass.

-# Empower DevOps Teams
+## Empower DevOps Teams

 Rancher provides an intuitive user interface for DevOps engineers to manage their application workload. The user does not need to have in-depth knowledge of Kubernetes concepts to start using Rancher. Rancher catalog contains a set of useful DevOps tools. Rancher is certified with a wide selection of cloud native ecosystem products, including, for example, security tools, monitoring systems, container registries, and storage and networking drivers.

@@ -24,7 +24,7 @@ The following figure illustrates the role Rancher plays in IT and DevOps organiz

 ![Platform](/img/platform.png)

-# Features of the Rancher API Server
+## Features of the Rancher API Server

 The Rancher API server is built on top of an embedded Kubernetes API server and an etcd database. It implements the following functionalities:

@@ -52,7 +52,7 @@ The Rancher API server is built on top of an embedded Kubernetes API server and
 - **Monitoring:** Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution.
 - **Alerting:** To keep your clusters and applications healthy and driving your organizational productivity forward, you need to stay informed of events occurring in your clusters and projects, both planned and unplanned.

-# Editing Downstream Clusters with Rancher
+## Editing Downstream Clusters with Rancher

 The options and settings available for an existing cluster change based on the method that you used to provision it. For example, only clusters [provisioned by RKE](../pages-for-subheaders/launch-kubernetes-with-rancher.md) have **Cluster Options** available for editing.

@@ -3,4 +3,4 @@ title: Rancher AWS Marketplace Quick Start
 description: Use Amazon EKS to deploy Rancher server.
 ---

-There is now an additional way for you to deploy the Rancher server in AWS by using Amazon EKS. To learn more, see our [Amazon Marketplace listing](https://aws.amazon.com/marketplace/pp/prodview-2yzbnvagmi4as).
+There is now an additional way for you to deploy the Rancher server in AWS by using Amazon EKS. To learn more, see our [Amazon Marketplace listing](https://aws.amazon.com/marketplace/pp/prodview-go7ent7goo5ae).
@@ -16,7 +16,7 @@ title: 1. Enable Istio in the Cluster
 1. Click **Apps & Marketplace**.
 1. Click **Charts**.
 1. Click **Istio**.
-1. If you have not already installed your own monitoring app, you will be prompted to install the rancher-monitoring app. Optional: Set your Selector or Scrape config options on rancher-monitoring app install. 
+1. If you have not already installed your own monitoring app, you will be prompted to install the rancher-monitoring app. Optional: Set your Selector or Scrape config options on rancher-monitoring app install.
 1. Optional: Configure member access and [resource limits](../../../integrations-in-rancher/istio/cpu-and-memory-allocations.md) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio.
 1. Optional: Make additional configuration changes to values.yaml if needed.
 1. Optional: Add additional resources or configuration via the [overlay file.](../../../pages-for-subheaders/configuration-options.md#overlay-file)
@@ -24,6 +24,6 @@ title: 1. Enable Istio in the Cluster

 **Result:** Istio is installed at the cluster level.

-# Additional Config Options
+## Additional Config Options

 For more information on configuring Istio, refer to the [configuration reference.](../../../pages-for-subheaders/configuration-options.md)
@@ -4,7 +4,7 @@ title: 6. Generate and View Traffic

 This section describes how to view the traffic that is being managed by Istio.

-# The Kiali Traffic Graph
+## The Kiali Traffic Graph

 The Istio overview page provides a link to the Kiali dashboard. From the Kiali dashboard, you are able to view graphs for each namespace. The Kiali graph provides a powerful way to visualize the topology of your Istio service mesh. It shows you which services communicate with each other.

@@ -16,7 +16,7 @@ For more information on the Istio gateway, refer to the [Istio documentation.](h

 ![In an Istio-enabled cluster, you can have two Ingresses: the default Nginx Ingress, and the default Istio controller.](/img/istio-ingress.svg)

-# Enable an Istio Gateway
+## Enable an Istio Gateway

 The ingress gateway is a Kubernetes service that will be deployed in your cluster. The Istio Gateway allows for more extensive customization and flexibility.

@@ -29,7 +29,7 @@ The ingress gateway is a Kubernetes service that will be deployed in your cluste

 **Result:** The gateway is deployed, and will now route traffic with applied rules.

-# Example Istio Gateway
+## Example Istio Gateway

 We add the BookInfo app deployments in services when going through the Workloads example. Next we add an Istio Gateway so that the app is accessible from outside your cluster.

@@ -127,7 +127,7 @@ To get the ingress gateway URL and port,

 For help inspecting the Istio controller URL and ports, try the commands the [Istio documentation.](https://istio.io/docs/tasks/traffic-management/ingress/ingress-control/#determining-the-ingress-ip-and-ports)

-# Troubleshooting
+## Troubleshooting

 The [official Istio documentation](https://istio.io/docs/tasks/traffic-management/ingress/ingress-control/#troubleshooting) suggests `kubectl` commands to inspect the correct ingress host and ingress port for external requests.

@@ -10,7 +10,7 @@ To allow the Grafana dashboard to persist after the Grafana instance restarts, a
 - [Creating a Persistent Grafana Dashboard](#creating-a-persistent-grafana-dashboard)
 - [Known Issues](#known-issues)

-# Creating a Persistent Grafana Dashboard
+## Creating a Persistent Grafana Dashboard

 <Tabs>
 <TabItem value="Rancher v2.5.8+">
@@ -144,7 +144,7 @@ helm.sh/resource-policy: "keep"
 </TabItem>
 </Tabs>

-# Known Issues
+## Known Issues

 For users who are using Monitoring V2 v9.4.203 or below, uninstalling the Monitoring chart will delete the `cattle-dashboards` namespace, which will delete all persisted dashboards, unless the namespace is marked with the annotation `helm.sh/resource-policy: "keep"`.

@@ -8,7 +8,7 @@ This page describes how to enable monitoring and alerting within a cluster using

 You can enable monitoring with or without SSL.

-# Requirements
+## Requirements

 - Make sure that you are allowing traffic on port 9796 for each of your nodes because Prometheus will scrape metrics from here.
 - Make sure your cluster fulfills the resource requirements. The cluster should have at least 1950Mi memory available, 2700m CPU, and 50Gi storage. A breakdown of the resource limits and requests is [here.](../../../reference-guides/monitoring-v2-configuration/helm-chart-options.md#configuring-resource-limits-and-requests)
@@ -27,13 +27,13 @@ If you want to set up Alertmanager, Grafana or Ingress, it has to be done with t

 :::

-# Setting Resource Limits and Requests
+#Setting Resource Limits and Requests

 The resource requests and limits can be configured when installing `rancher-monitoring`.  To configure Prometheus resources from the Rancher UI, click **Apps & Marketplace > Monitoring** in the upper left corner.

 For more information about the default limits, see [this page.](../../../reference-guides/monitoring-v2-configuration/helm-chart-options.md#configuring-resource-limits-and-requests)

-# Install the Monitoring Application
+## Install the Monitoring Application

 ### Enable Monitoring for use without SSL

@@ -10,7 +10,7 @@ This section assumes familiarity with how monitoring components work together. F

 :::

-# About the Prometheus Custom Resource
+## About the Prometheus Custom Resource

 The Prometheus CR defines a desired Prometheus deployment. The Prometheus Operator observes the Prometheus CR. When the CR changes, the Prometheus Operator creates `prometheus-rancher-monitoring-prometheus`, a Prometheus deployment based on the CR configuration.

@@ -45,7 +45,7 @@ Use the label selector field `ruleSelector` in the Prometheus object to define t

 For examples, refer to the Prometheus documentation on [recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) and [alerting rules.](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)

-# Configuration
+## Configuration

 ### Rule Group

@@ -6,7 +6,7 @@ Administrators have the permission to create RKE templates, and only administrat

 For more information on administrator permissions, refer to the [documentation on global permissions](../manage-role-based-access-control-rbac/global-permissions.md).

-# Giving Users Permission to Create Templates
+## Giving Users Permission to Create Templates

 Templates can only be created by users who have the global permission **Create RKE Templates**.

@@ -218,7 +218,7 @@ The following table maps the custom config values you copied in the Azure portal

 ### Migrating from Azure AD Graph API to Microsoft Graph API

-Since [Azure AD Graph API](https://docs.microsoft.com/en-us/graph/migrate-azure-ad-graph-overview) was deprecated in June 2022 and will be retired at the end of 2022, users should update their Azure AD App to use the new [Microsoft Graph API](https://docs.microsoft.com/en-us/graph/use-the-api) in Rancher.
+Since [Azure AD Graph API](https://docs.microsoft.com/en-us/graph/migrate-azure-ad-graph-overview) was deprecated in June 2022 and will be retired on June 30, 2023, users should update their Azure AD App to use the new [Microsoft Graph API](https://docs.microsoft.com/en-us/graph/use-the-api) in Rancher.

 #### Updating Endpoints in the Rancher UI

@@ -312,4 +312,4 @@ Token Endpoint   | https://login.partner.microsoftonline.cn/{tenantID}/oauth2/v2
    - Use another third-party auth system and set that up in Rancher. Please see the [authentication docs](../../../../pages-for-subheaders/authentication-config.md) to learn how to configure other open authentication providers.

 </TabItem>
-</Tabs>
+</Tabs>
@@ -8,7 +8,7 @@ Only admins of the G Suite domain have access to the Admin SDK. Therefore, only

 Within Rancher, only administrators or users with the **Manage Authentication** [global role](../../authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md) can configure authentication.

-# Prerequisites
+## Prerequisites

 - You must have a [G Suite admin account](https://admin.google.com) configured.
 - G Suite requires a [top private domain FQDN](https://github.com/google/guava/wiki/InternetDomainNameExplained#public-suffixes-and-private-domains) as an authorized domain. One way to get an FQDN is by creating an A-record in Route53 for your Rancher server. You do not need to update your Rancher Server URL setting with that record, because there could be clusters using that URL.
@@ -17,7 +17,7 @@ Within Rancher, only administrators or users with the **Manage Authentication**
 After the Admin SDK API is enabled, your G Suite domain's API screen should look like this:
 ![Enable Admin APIs](/img/Google-Enable-APIs-Screen.png)

-# Setting up G Suite for OAuth with Rancher
+## Setting up G Suite for OAuth with Rancher

 Before you can set up Google OAuth in Rancher, you need to log in to your G Suite account and do the following:

@@ -93,7 +93,7 @@ Using the Unique ID of the service account key, register it as an Oauth Client u

 **Result:** The service account is registered as an OAuth client in your G Suite account.

-# Configuring Google OAuth in Rancher
+## Configuring Google OAuth in Rancher

 1. Sign into Rancher using a local user assigned the [administrator](../../authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md) role. This user is also called the local principal.
 1.	In the top left corner, click **☰ > Users & Authentication**.
@@ -28,7 +28,7 @@ After you complete [Configuring Microsoft AD FS for Rancher](configure-ms-adfs-f

 **Result:** Rancher is configured to work with MS FS. Your users can now sign into Rancher using their MS FS logins.

-# Configuration
+## Configuration

 | Field | Description                |
 |---------------------------|-----------------|
@@ -16,7 +16,7 @@ For further details on configuring OpenLDAP, refer to the [official documentatio
 3. Once the user has been found, they are authenticated with another LDAP bind request using the user's DN and provided password.
 4. Once authentication succeeded, Rancher then resolves the group memberships both from the membership attribute in the user's object and by performing a group search based on the configured user mapping attribute.

-# OpenLDAP Server Configuration
+## OpenLDAP Server Configuration

 You will need to enter the address, port, and protocol to connect to your OpenLDAP server. `389` is the standard port for insecure traffic, `636` for TLS traffic.

@@ -39,7 +39,7 @@ If you are in doubt about the correct values to enter in the user/group Search B
 | User Search Base | Enter the Distinguished Name of the node in your directory tree from which to start searching for user objects. All users must be descendents of this base DN. For example: "ou=people,dc=acme,dc=com".|
 | Group Search Base | If your groups live under a different node than the one configured under `User Search Base` you will need to provide the Distinguished Name here. Otherwise leave this field empty. For example: "ou=groups,dc=acme,dc=com".|

-# User/Group Schema Configuration
+## User/Group Schema Configuration

 If your OpenLDAP directory deviates from the standard OpenLDAP schema, you must complete the **Customize Schema** section to match it.

@@ -160,7 +160,7 @@ Can be any link, even to links outside of the cluster.

 It is required to provide either the `toService` directive or the `toURL` directive.

-# Link Examples
+## Link Examples

 ### Example of Link with `toUrl`

@@ -8,7 +8,7 @@ Rancher recommends configuring recurrent `etcd` snapshots for all production clu

 Snapshots of the etcd database are taken and saved either [locally onto the etcd nodes](#local-backup-target) or to a [S3 compatible target](#s3-backup-target). The advantages of configuring S3 is that if all etcd nodes are lost, your snapshot is saved remotely and can be used to restore the cluster.

-# How Snapshots Work
+## How Snapshots Work

 ### Snapshot Components

@@ -64,7 +64,7 @@ On restore, the following process is used:
 4. The other etcd nodes download the snapshot and validate the checksum so that they all use the same snapshot for the restore.
 5.  The cluster is restored and post-restore actions will be done in the cluster.

-# Configuring Recurring Snapshots
+## Configuring Recurring Snapshots

 Select how often you want recurring snapshots to be taken as well as how many snapshots to keep. The amount of time is measured in hours. With timestamped snapshots, the user has the ability to do a point-in-time recovery.

@@ -81,7 +81,7 @@ In the **Advanced Cluster Options** section, there are several options available
 | Recurring etcd Snapshot Creation Period | Time in hours between recurring snapshots| 12 hours |
 | Recurring etcd Snapshot Retention Count | Number of snapshots to retain| 6 |

-# One-Time Snapshots
+## One-Time Snapshots

 In addition to recurring snapshots, you may want to take a "one-time" snapshot. For example, before upgrading the Kubernetes version of a cluster it's best to backup the state of the cluster to protect against upgrade failure.

@@ -91,7 +91,7 @@ In addition to recurring snapshots, you may want to take a "one-time" snapshot.

 **Result:** Based on your [snapshot backup target](#snapshot-backup-targets), a one-time snapshot will be taken and saved in the selected backup target.

-# Snapshot Backup Targets
+## Snapshot Backup Targets

 Rancher supports two different backup targets:

@@ -130,7 +130,7 @@ The `S3` backup target supports using IAM authentication to AWS API in addition

 To give an application access to S3, refer to the AWS documentation on [Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances.](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)

-# Viewing Available Snapshots
+## Viewing Available Snapshots

 The list of all available snapshots for the cluster is available in the Rancher UI.

@@ -138,7 +138,7 @@ The list of all available snapshots for the cluster is available in the Rancher
 1. In the **Clusters** page, go to the cluster where you want to view the snapshots and click its name.
 1. Click the **Snapshots** tab to view the list of saved snapshots. These snapshots include a timestamp of when they were created.

-# Safe Timestamps
+## Safe Timestamps

 Snapshot files are timestamped to simplify processing the files using external tools and scripts, but in some S3 compatible backends, these timestamps were unusable.

@@ -146,6 +146,6 @@ The option `safe_timestamp` is added to support compatible file names. When this

 This option is not available directly in the UI, and is only available through the `Edit as Yaml` interface.

-# Enabling Snapshot Features for Clusters Created Before Rancher v2.2.0
+## Enabling Snapshot Features for Clusters Created Before Rancher v2.2.0

 If you have any Rancher launched Kubernetes clusters that were created before v2.2.0, after upgrading Rancher, you must [edit the cluster](../../../pages-for-subheaders/cluster-configuration.md) and _save_ it, in order to enable the updated snapshot features. Even if you were already creating snapshots before v2.2.0, you must do this step as the older snapshots will not be available to use to [back up and restore etcd through the UI](restore-rancher-launched-kubernetes-clusters-from-backup.md).
@@ -70,8 +70,8 @@ If the group of etcd nodes loses quorum, the Kubernetes cluster will report a fa

 5. Run the revised command.

-6. After the single nodes is up and running, Rancher recommends adding additional etcd nodes to your cluster. If you have a [custom cluster](../../../pages-for-subheaders/use-existing-nodes.md) and you want to reuse an old node, you are required to [clean up the nodes](../manage-clusters/clean-cluster-nodes.md) before attempting to add them back into a cluster.  
+6. After the single nodes is up and running, Rancher recommends adding additional etcd nodes to your cluster. If you have a [custom cluster](../../../pages-for-subheaders/use-existing-nodes.md) and you want to reuse an old node, you are required to [clean up the nodes](../manage-clusters/clean-cluster-nodes.md) before attempting to add them back into a cluster.

-# Enabling Snapshot Features for Clusters Created Before Rancher v2.2.0
+## Enabling Snapshot Features for Clusters Created Before Rancher v2.2.0

 If you have any Rancher launched Kubernetes clusters that were created before v2.2.0, after upgrading Rancher, you must [edit the cluster](../../../pages-for-subheaders/cluster-configuration.md) and _save_ it, in order to enable the updated snapshot features. Even if you were already creating snapshots before v2.2.0, you must do this step as the older snapshots will not be available to use to [back up and restore etcd through the UI](restore-rancher-launched-kubernetes-clusters-from-backup.md).
@@ -12,7 +12,7 @@ For a complete walkthrough of developing charts, see the [Chart Template Develop



-# Chart Types
+## Chart Types

 Rancher supports two different types of charts: Helm charts and Rancher charts.

@@ -26,7 +26,7 @@ Rancher charts are native helm charts with two files that enhance user experienc

 Rancher charts add simplified chart descriptions and configuration forms to make the application deployment easy. Rancher users do not need to read through the entire list of Helm variables to understand how to launch an application.

-# Chart Directory Structure
+## Chart Directory Structure

 You can provide Helm Charts in a standard, HTTP based Helm Repository. For more information see the [Chart Repository Guide](https://helm.sh/docs/topics/chart_repository) in the official Helm documentation.

@@ -48,7 +48,7 @@ The following table demonstrates the directory structure for a Git repository. T
 │   │   │   ├── templates/        # Directory containing templates that, when combined with values.yml, generates Kubernetes YAML.
 ```

-# Additional Files for Rancher Charts
+## Additional Files for Rancher Charts

 Before you create your own custom catalog, you should have a basic understanding about how a Rancher chart differs from a native Helm chart. Rancher charts differ slightly from Helm charts in their directory structures. Rancher charts include two files that Helm charts do not.

@@ -149,7 +149,7 @@ Look over the load balancer details and click **Create** when you're satisfied.

 After AWS creates the NLB, click **Close**.

-# 4. Add listener to NLB for TCP port 80
+## 4. Add listener to NLB for TCP port 80

 1. Select your newly created NLB and select the **Listeners** tab.

@@ -163,7 +163,7 @@ After AWS creates the NLB, click **Close**.

 6. Click **Save** in the top right of the screen.

-# Health Check Paths for NGINX Ingress and Traefik Ingresses
+## Health Check Paths for NGINX Ingress and Traefik Ingresses

 K3s and RKE Kubernetes clusters handle health checks differently because they use different Ingresses by default.

@@ -16,12 +16,12 @@ In both single-node setups, Rancher can be installed with Helm on the Kubernetes

 :::

-# Prerequisites
+## Prerequisites

 These instructions assume you have set up two nodes, a load balancer, a DNS record, and an external MySQL database as described in [this section.](../infrastructure-setup/ha-k3s-kubernetes-cluster.md)

 Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/) To specify the K3s version, use the INSTALL_K3S_VERSION environment variable when running the K3s installation script.
-# Installing Kubernetes
+## Installing Kubernetes

 ### 1. Install Kubernetes and Set up the K3s Server

@@ -23,7 +23,7 @@ In both single-node setups, Rancher can be installed with Helm on the Kubernetes

 :::

-# Installing Kubernetes
+## Installing Kubernetes

 ### Required CLI Tools

@@ -85,7 +85,7 @@ ingress:
 RKE has many configuration options for customizing the install to suit your specific environment.

 Please see the [RKE Documentation](https://rancher.com/docs/rke/latest/en/config-options/) for the full list of options and capabilities.
- 
+
 For tuning your etcd cluster for larger Rancher installations, see the [etcd settings guide](../../advanced-user-guides/tune-etcd-for-large-installs.md).

 For more information regarding Dockershim support, refer to [this page](../../../getting-started/installation-and-upgrade/installation-requirements/dockershim.md)
@@ -6,14 +6,15 @@ _Tested on v2.5.6_

 This section describes how to install a Kubernetes cluster according to the [best practices for the Rancher server environment.](../../../reference-guides/rancher-manager-architecture/architecture-recommendations.md#environment-for-kubernetes-installations)

-# Prerequisites
+## Prerequisites

 These instructions assume you have set up three nodes, a load balancer, and a DNS record, as described in [this section.](../infrastructure-setup/ha-rke2-kubernetes-cluster.md)

 Note that in order for RKE2 to work correctly with the load balancer, you need to set up two listeners: one for the supervisor on port 9345, and one for the Kubernetes API on port 6443.

 Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/) To specify the RKE2 version, use the INSTALL_RKE2_VERSION environment variable when running the RKE2 installation script.
-# Installing Kubernetes
+
+## Installing Kubernetes

 ### 1. Install Kubernetes and Set up the RKE2 Server

@@ -15,7 +15,7 @@ The following account types are not supported for Azure Subscriptions:
 - Single tenant accounts (i.e. accounts with no subscriptions).
 - Multi-subscription accounts.

-# Prerequisites for RKE and RKE2
+## Prerequisites for RKE and RKE2

 To set up the Azure cloud provider for both RKE and RKE2, the following credentials need to be configured:

@@ -70,7 +70,7 @@ You should already assign custom hosts to this Network Security Group during pro

 Only hosts expected to be load balancer back ends need to be in this group.

-# RKE2 Cluster Set-up in Rancher
+## RKE2 Cluster Set-up in Rancher

 1. Choose "Azure" from the Cloud Provider drop-down in the Cluster Configuration section.

@@ -6,7 +6,7 @@ Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has

 This page covers how to install the Cloud Provider Interface (CPI) and Cloud Storage Interface (CSI) plugins after bringing up a cluster.

-# Prerequisites
+## Prerequisites

 The vSphere versions supported:

@@ -17,7 +17,7 @@ The Kubernetes version must be 1.19 or higher.

 Using the vSphere out-of-tree cloud provider requires Linux nodes and is not supported on Windows.

-# Installation
+## Installation

 The Cloud Provider Interface (CPI) should be installed first before installing the Cloud Storage Interface (CSI).

@@ -55,7 +55,7 @@ The Cloud Provider Interface (CPI) should be installed first before installing t
 1. Click **Install**.


-# Using the CSI driver for provisioning volumes
+## Using the CSI driver for provisioning volumes

 The CSI chart by default creates a storageClass.

@@ -29,12 +29,12 @@ Upstream bug: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/628

 Rancher issue tracking this bug: https://github.com/rancher/rancher/issues/31105

-# Prerequisites
+## Prerequisites

 - vSphere CSI Migration requires vSphere 7.0u1. In order to be able to manage existing in-tree vSphere volumes, upgrade vSphere to 7.0u1.
 - The Kubernetes version must be 1.19 or higher.

-# Migration
+## Migration

 ### 1. Install the CPI plugin

@@ -13,7 +13,7 @@ Deployments use the Kubernetes registry secret to authenticate with a private Do

 Currently, deployments pull the private registry credentials automatically only if the workload is created in the Rancher UI and not when it is created via kubectl.

-# Creating a Registry in Namespaces
+## Creating a Registry in Namespaces

 :::note Prerequisite:

@@ -44,7 +44,7 @@ You must have a [private registry](https://docs.docker.com/registry/deploying/)
 - You can view the secret in the Rancher UI by clicking either **Storage > Secrets** or **More Resources > Core > Secrets**.
 - Any workload that you create in the Rancher UI will have the credentials to access the registry if the workload is within the registry's scope.

-# Creating a Registry in Projects
+## Creating a Registry in Projects

 :::note Prerequisites:

@@ -87,7 +87,7 @@ Project-scoped registries on the local cluster are only visible when a single pr

 :::

-# Using a Private Registry
+## Using a Private Registry

 You can deploy a workload with an image from a private registry through the Rancher UI, or with `kubectl`.

@@ -7,7 +7,7 @@ description: Ingress configuration

 For Kubernetes v1.21 and up, the NGINX Ingress controller no longer runs in hostNetwork but uses hostPorts for port 80 and port 443. This was done so the admission webhook can be configured to be accessed using ClusterIP so it can only be reached inside the cluster.

-# Ingress Rule Configuration
+## Ingress Rule Configuration

 - [Specify a hostname to use](#specify-a-hostname-to-use)
 - [Use as the default backend](#use-as-the-default-backend)
@@ -14,7 +14,7 @@ When configuring a workload, you'll be able to choose which secrets to include.

 Mounted secrets will be updated automatically unless they are mounted as subpath volumes. For details on how updated secrets are propagated, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/secret/#mounted-secrets-are-updated-automatically)

-# Creating Secrets in Namespaces
+## Creating Secrets in Namespaces

 1. In the upper left corner, click **☰ > Cluster Management**.
 1. Go to the cluster where you want to add a secret and click **Explore**.
@@ -47,7 +47,7 @@ Mounted secrets will be updated automatically unless they are mounted as subpath
 Mounted secrets will be updated automatically unless they are mounted as subpath volumes. For details on how updated secrets are propagated, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/secret/#mounted-secrets-are-updated-automatically)


-# Creating Secrets in Projects
+## Creating Secrets in Projects

 Before v2.6, secrets were required to be in a project scope. Projects are no longer required, and you may use the namespace scope instead. As a result, the Rancher UI was updated to reflect this new functionality. However, you may still create project-scoped secrets if desired. Note that you have to first enable the `legacy` feature flag and look at a single project to do so. Use the following steps to set up your project-level secret:

@@ -69,7 +69,7 @@ Project-scoped secrets on the local cluster are only visible when a single proje

 :::

-# What's Next?
+## What's Next?

 Now that you have a secret added to a namespace, you can add it to a workload that you deploy.

@@ -93,7 +93,8 @@ You can access your cluster after its state is updated to **Active**.

 - `Default`, containing the `default` namespace
 - `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces
-# Optional Next Steps
+
+## Optional Next Steps

 After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:

@@ -21,7 +21,7 @@ Then you will create an EC2 cluster in Rancher, and when configuring the new clu
  - [Example IAM Policy to allow encrypted EBS volumes](#example-iam-policy-to-allow-encrypted-ebs-volumes)
 - **IAM Policy added as Permission** to the user. See [Amazon Documentation: Adding Permissions to a User (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) how to attach it to an user.

-# Creating an EC2 Cluster
+## Creating an EC2 Cluster

 The steps to create a cluster differ based on your Rancher version.

@@ -133,7 +133,7 @@ After creating your cluster, you can access it through the Rancher UI. As a best
 - **Access your cluster with the kubectl CLI:** Follow [these steps](../../../new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#accessing-clusters-with-kubectl-on-your-workstation) to access clusters with kubectl on your workstation. In this case, you will be authenticated through the Rancher server’s authentication proxy, then Rancher will connect you to the downstream cluster. This method lets you manage the cluster without the Rancher UI.
 - **Access your cluster with the kubectl CLI, using the authorized cluster endpoint:** Follow [these steps](../../../new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) to access your cluster with kubectl directly, without authenticating through Rancher. We recommend setting up this alternative method to access your cluster so that in case you can’t connect to Rancher, you can still access the cluster.

-# IAM Policies
+## IAM Policies

 ### Example IAM Policy

@@ -29,7 +29,7 @@ For more information on configuring Azure node templates, refer to the [Azure no
 - [Preparation in Azure](#preparation-in-azure)
 - [Creating an Azure Cluster](#creating-an-azure-cluster)

-# Preparation in Azure
+## Preparation in Azure

 Before creating a node template in Rancher using a cloud infrastructure such as Azure, we must configure Rancher to allow the manipulation of resources in an Azure subscription.

@@ -46,7 +46,7 @@ az ad sp create-for-rbac \

 The creation of this service principal returns three pieces of identification information, *The application ID, also called the client ID*, and *The client secret*. This information will be used when you create a node template for Azure.

-# Creating an Azure Cluster
+## Creating an Azure Cluster

 <Tabs>
 <TabItem value="RKE">
@@ -17,7 +17,7 @@ For details on configuring RKE Kubernetes clusters in Rancher, refer to the [clu
 - [Preparation in Nutanix AOS](#preparation-in-nutanix-aos)
 - [Creating a Nutanix AOS Cluster](#creating-a-nutanix-aos-cluster)

-# Preparation in Nutanix AOS
+## Preparation in Nutanix AOS

 The following sections describe the requirements for setting up Nutanix AOS so that Rancher can provision VMs and clusters.

@@ -26,6 +26,7 @@ The following sections describe the requirements for setting up Nutanix AOS so t
 The node templates are documented and tested with Nutanix AOS version 5.20.2 and 6.0.1.

 :::
+
 ### Create Credentials in Nutanix AOS

 Before proceeding to create a cluster, you must ensure that you have a [Nutanix Prism Central user account](https://portal.nutanix.com/page/documents/details?targetId=Nutanix-Security-Guide-v6_0:wc-user-create-wc-t.html) with admin permissions. When you set up a node template, the template will need to use these credentials.
@@ -43,7 +44,7 @@ See [Node Networking Requirements](../../../kubernetes-clusters-in-rancher-setup

 Setting up [VM-VM Anti-Affinity Policies](https://portal.nutanix.com/page/documents/details?targetId=AHV-Admin-Guide-v6_1:ahv-vm-anti-affinity-t.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate AHV hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.

-# Creating a Nutanix AOS Cluster
+## Creating a Nutanix AOS Cluster

 1. [Create a node template ](#1-create-a-node-template)
 2. [Create a cluster with node pools using the node template](#2-create-a-cluster-with-node-pools-using-the-node-template)
@@ -82,8 +83,7 @@ You can access your cluster after its state is updated to **Active**.
 - `Default`, containing the `default` namespace
 - `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces

-
-# Optional Next Steps
+## Optional Next Steps

 After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:

@@ -16,7 +16,7 @@ For details on configuring RKE Kubernetes clusters in Rancher, refer to the [clu
 - [Preparation in vSphere](#preparation-in-vsphere)
 - [Creating a vSphere Cluster](#creating-a-vsphere-cluster)

-# Preparation in vSphere
+## Preparation in vSphere

 This section describes the requirements for setting up vSphere so that Rancher can provision VMs and clusters.

@@ -513,7 +513,7 @@ Cluster-autoscaler deployment can also be set up using [manual configuration](ht

 :::

-# Testing
+## Testing

 At this point, we should have a cluster-scaler up and running in our Rancher custom cluster. Cluster-scale should manage `K8sWorkerAsg` ASG to scale up and down between 2 and 10 nodes, when one of the following conditions is true:

@@ -14,7 +14,7 @@ For public downstream clusters, it is sufficient to [set the required environmen

 For private nodes or private clusters, the environment variables need to be set on the nodes themselves. Then the environment variables are configured from the Rancher UI, typically when provisioning a custom cluster or when registering the private cluster. For an example of how to set the environment variables on Ubuntu node in a K3s Kubernetes cluster, see [this section.](#setting-environment-variables-on-private-nodes)

-# Required Environment Variables
+## Required Environment Variables

 When adding Fleet agent environment variables for the proxy, replace <PROXY_IP> with your private proxy IP.

@@ -65,6 +65,43 @@ Admins should use these roles to provide more fine-grained access to users:
 | monitoring-dashboard-edit | Allow admins to assign roles to users to be able to edit / view ConfigMaps within the cattle-dashboards namespace. ConfigMaps in this namespace will correspond to Grafana Dashboards that are persisted onto the cluster. |
 | monitoring-dashboard-view | Allow admins to assign roles to users to be able to view ConfigMaps within the cattle-dashboards namespace. ConfigMaps in this namespace will correspond to Grafana Dashboards that are persisted onto the cluster. |

+
+### Assigning Monitoring Roles through Custom Roles
+
+Admins may assign custom roles in the Rancher UI for admin, editing, and viewing monitoring. These "roles" are created by default when the monitoring app is installed. Additionally, these roles are also deployed to the corresponding Kubernetes roles: admin, edit, and view `ClusterRoles`.
+
+:::note Important
+
+The UI will not offer `monitoring-admin`, `monitoring-edit`, and `monitoring-view` options when users are being added to a cluster. These monitoring roles can only be assigned by manually creating a custom role that inherits from Project Owner and Project Monitoring View roles. 
+
+:::
+
+  1. Create the custom role:
+
+    1.1 Click **☰ > Users & Authentication > Roles**.
+
+    1.2 Select the appropriate tab, e.g., **Cluster** role. Then click **Create Cluster Role**.
+
+    1.3 In the **Name** field, create a custom role such as `View Monitoring`, `Edit Monitoring`, or `Admin Monitoring`.
+
+    1.4 Click **Inherit From > Add Resource**, then select the Kubernetes role, as applicable, from the dropdown.
+
+    1.5 Click **Create**.
+
+
+  2. Assign the custom role to a new user:
+
+    2.1 Click **☰ > Cluster Management > Cluster Explore > Cluster > Cluster Members > Add**.
+
+    2.2 Search for your new user name from **Select Member** options displayed.
+
+    2.3 Assign the new custom role from **Cluster Permissions** to the new user.
+
+    2.4 Click **Create**.
+
+
+**Result:** The new user should now be able to see the monitoring tools.
+
 ### Additional Monitoring ClusterRoles

 Monitoring also creates additional `ClusterRoles` that are not assigned to users by default but are created within the cluster.  They are not aggregated by default but can be bound to a namespace by deploying a `RoleBinding` or `ClusterRoleBinding` that references it. To define a `RoleBinding` with `kubectl` instead of through Rancher, click [here](#assigning-roles-and-clusterroles-with-kubectl).
@@ -16,14 +16,14 @@ If you install Rancher following the Docker installation guide, there is no upgr

 :::

-# Installation Outline
+## Installation Outline

 1. [Set up infrastructure and private registry](../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry.md)
 2. [Collect and publish images to your private registry](../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/publish-images.md)
 3. [Set up a Kubernetes cluster (Skip this step for Docker installations)](../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-kubernetes.md)
 4. [Install Rancher](../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md)

-# Upgrades
+## Upgrades

 To upgrade Rancher with Helm CLI in an air gap environment, follow [this procedure.](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades.md)

@@ -2,11 +2,11 @@
 title: AWS Marketplace Integration
 ---

-# Overview
+## Overview

 Rancher offers an integration with the AWS Marketplace which allows users to purchase a support contract with SUSE. This integration allows you easily adjust your support needs as you start to support more clusters.

-# Limitations
+## Limitations

 - You must be running Rancher v2.6.7 or higher
 - Rancher must be deployed with additional metrics enabled.
@@ -14,11 +14,12 @@ Rancher offers an integration with the AWS Marketplace which allows users to pur
 - You must purchase at least one entitlement to Rancher support through AWS Marketplace.
 - You may need additional setup to support proxy/airgap use cases. See the [prerequisites](../integrations-in-rancher/cloud-marketplace/aws-cloud-marketplace/adapter-requirements.md) for more information.

-# How to Use
+## How to Use
+
 1. Complete the [prerequisite steps](../integrations-in-rancher/cloud-marketplace/aws-cloud-marketplace/adapter-requirements.md).
 2. [Install the CSP Adapter](../integrations-in-rancher/cloud-marketplace/aws-cloud-marketplace/install-adapter.md).

-# FAQ
+## FAQ

 **Can I purchase support for more nodes later on?**

@@ -11,7 +11,7 @@ If you also configure OpenLDAP as the back end to Shibboleth, it will return a S
 > The instructions in this section assume that you understand how Rancher, Shibboleth, and OpenLDAP work together. For a more detailed explanation of how it works, refer to [this page.](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/configure-shibboleth-saml/about-group-permissions.md)


-# Setting up Shibboleth in Rancher
+## Setting up Shibboleth in Rancher

 ### Shibboleth Prerequisites
 >
@@ -73,7 +73,7 @@ If you configure Shibboleth without OpenLDAP, the following caveats apply due to

 To enable searching for groups when assigning permissions in Rancher, you will need to configure a back end for the SAML provider that supports groups, such as OpenLDAP.

-# Setting up OpenLDAP in Rancher
+## Setting up OpenLDAP in Rancher

 If you also configure OpenLDAP as the back end to Shibboleth, it will return a SAML assertion to Rancher with user attributes that include groups. Then authenticated users will be able to access resources in Rancher that their groups have permissions for.

@@ -92,10 +92,10 @@ Configure the settings for the OpenLDAP server, groups and users. For help filli
 > Before you proceed with the configuration, please familiarise yourself with the concepts of [External Authentication Configuration and Principal Users](authentication-config.md#external-authentication-configuration-and-principal-users).

 1. Log into the Rancher UI using the initial local `admin` account.
-1.	In the top left corner, click **☰ > Users & Authentication**.
+1. In the top left corner, click **☰ > Users & Authentication**.
 1. In the left navigation menu, click **Auth Provider**.
 1. Click **OpenLDAP**. The **Configure an OpenLDAP server** form will be displayed.

-# Troubleshooting
+## Troubleshooting

 If you are experiencing issues while testing the connection to the OpenLDAP server, first double-check the credentials entered for the service account as well as the search base configuration. You may also inspect the Rancher logs to help pinpointing the problem cause. Debug logs may contain more detailed information about the error. Please refer to [How can I enable debug logging](../faq/technical-items.md#how-can-i-enable-debug-logging) in this documentation.
@@ -20,7 +20,7 @@ Because the API sets the actual value and the command line sets the default valu

 For example, if you install Rancher, then set a feature flag to true with the Rancher API, then upgrade Rancher with a command that sets the feature flag to false, the default value will still be false, but the feature will still be enabled because it was set with the Rancher API. If you then deleted the set value (true) with the Rancher API, setting it to NULL, the default value (false) would take effect. See the [feature flags page](../getting-started/installation-and-upgrade/installation-references/feature-flags.md) for more information.

-# Enabling Features when Starting Rancher
+## Enabling Features when Starting Rancher

 When you install Rancher, enable the feature you want with a feature flag. The command is different depending on whether you are installing Rancher on a single node or if you are doing a Kubernetes Installation of Rancher.

@@ -81,7 +81,7 @@ docker run -d -p 80:80 -p 443:443 \
 ```


-# Enabling Features with the Rancher UI
+## Enabling Features with the Rancher UI

 1. In the upper left corner, click **☰ > Global Settings**.
 1. Click **Feature Flags**.
@@ -97,7 +97,7 @@ docker run -d -p 80:80 -p 443:443 \

 **Result:** The feature is disabled.

-# Enabling Features with the Rancher API
+## Enabling Features with the Rancher API

 1. Go to `<RANCHER-SERVER-URL>/v3/features`.
 1. In the `data` section, you will see an array containing all of the features that can be turned on with feature flags. The name of the feature is in the `id` field. Click the name of the feature you want to enable.
@@ -133,7 +133,7 @@ _Mutable: yes_

 Enable control plane authorized networks to block untrusted non-GCP source IPs from accessing the Kubernetes master through HTTPS. If selected, additional authorized networks may be added. If the cluster is created with a public endpoint, this option is useful for locking down access to the public endpoint to only certain networks, such as the network where your Rancher service is running. If the cluster only has a private endpoint, this setting is required.

-# Additional Options
+## Additional Options

 ### Cluster Addons

@@ -182,7 +182,7 @@ _Mutable: yes_

 Set the start time for a 4 hour maintenance window. The time is specified in the UTC time zone using the HH:MM format. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions)

-# Node Pools
+## Node Pools

 In this section, enter details describing the configuration of each node in the node pool.

@@ -256,7 +256,7 @@ You can add network tags to the node pool to make firewall rules and routes betw

 For details on tag syntax and requirements, see the [Kubernetes documentation](https://cloud.google.com/vpc/docs/add-remove-network-tags).

-# Group Details
+## Group Details

 In this section, enter details describing the node pool.

@@ -15,7 +15,7 @@ Cluster Autoscaler is designed to run on Kubernetes master nodes. It can run in

 It's possible to run a customized deployment of Cluster Autoscaler on worker nodes, but extra care needs to be taken to ensure that Cluster Autoscaler remains up and running.

-# Cloud Providers
+## Cloud Providers

 Cluster Autoscaler provides support to distinct cloud providers. For more information, go to [cluster-autoscaler supported cloud providers.](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler#deployment)

@@ -9,7 +9,7 @@ import TabItem from '@theme/TabItem';
 In this section, you'll learn how to deploy Rancher on a Kubernetes cluster using the Helm CLI.


-# Prerequisites
+## Prerequisites

 - [Kubernetes Cluster](#kubernetes-cluster)
 - [Ingress Controller](#ingress-controller)
@@ -47,7 +47,7 @@ The following CLI tools are required for setting up the Kubernetes cluster. Plea
 - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool.
 - [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements](../getting-started/installation-and-upgrade/resources/helm-version-requirements.md) to choose a version of Helm to install Rancher. Refer to the [instructions provided by the Helm project](https://helm.sh/docs/intro/install/) for your specific platform.

-# Install the Rancher Helm Chart
+## Install the Rancher Helm Chart

 Rancher is installed using the [Helm](https://helm.sh/) package manager for Kubernetes. Helm charts provide templating syntax for Kubernetes YAML manifest documents. With Helm, we can create configurable deployments instead of just using static files.

@@ -5,7 +5,7 @@ description: Learn how to install Rancher in development and production environm

 This section provides an overview of the architecture options of installing Rancher, describing advantages of each option.

-# Terminology
+## Terminology

 In this section,

@@ -16,7 +16,7 @@ In this section,

 Note the `restrictedAdmin` Helm chart option available for **the Rancher Server**. When this option is set to true, the initial Rancher user has restricted access to the local Kubernetes cluster to prevent privilege escalation. For more information, see the section about the [restricted-admin role.](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin)

-# Overview of Installation Options
+## Overview of Installation Options

 Rancher can be installed on these main architectures:

@@ -62,10 +62,11 @@ When the nodes in your Kubernetes cluster are running and fulfill the [node requ

 For a longer discussion of Rancher architecture, refer to the [architecture overview,](rancher-manager-architecture.md) [recommendations for production-grade architecture,](../reference-guides/rancher-manager-architecture/architecture-recommendations.md) or our [best practices guide.](../reference-guides/best-practices/rancher-server/tips-for-running-rancher.md)

-# Prerequisites
+## Prerequisites
+
 Before installing Rancher, make sure that your nodes fulfill all of the [installation requirements.](installation-requirements.md)

-# Architecture Tip
+## Architecture Tip

 For the best performance and greater security, we recommend a separate, dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](kubernetes-clusters-in-rancher-setup.md) for running your workloads.

@@ -35,7 +35,7 @@ For a list of best practices that we recommend for running the Rancher server in

 The Rancher UI works best in Firefox or Chromium based browsers (Chrome, Edge, Opera, Brave, ...).

-# Operating Systems and Container Runtime Requirements
+## Operating Systems and Container Runtime Requirements

 Rancher should work with any modern Linux distribution.

@@ -77,11 +77,11 @@ For the container runtime, RKE2 bundles its own containerd. Docker is not requir

 For details on which OS versions were tested with RKE2, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)

-# Hardware Requirements
+## Hardware Requirements

 The following sections describe the CPU, memory, and disk requirements for the nodes where the Rancher server is installed.

-# CPU and Memory
+## CPU and Memory

 Hardware requirements scale based on the size of your Rancher deployment. Provision each individual node according to the requirements. The requirements are different depending on if you are installing Rancher in a single container with Docker, or if you are installing Rancher on a Kubernetes cluster.

@@ -134,7 +134,7 @@ These CPU and memory requirements apply to a host with a [single-node](rancher-o
 | Small           | Up to 5  | Up to 50  | 1     | 4 GB |
 | Medium          | Up to 15 | Up to 200 | 2     | 8 GB |

-# Ingress
+## Ingress

 Each node in the Kubernetes cluster that Rancher is installed on should run an Ingress.

@@ -148,11 +148,11 @@ For hosted Kubernetes clusters (EKS, GKE, AKS), you will need to set up the ingr
 - **AKS:** For details on how to install Rancher with Azure Kubernetes Service, including how to install an ingress so that the Rancher server can be accessed, refer to [this page.](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md)
 - **GKE:** For details on how to install Rancher with Google Kubernetes Engine, including how to install an ingress so that the Rancher server can be accessed, refer to [this page.](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-gke.md)

-# Disks
+## Disks

 Rancher performance depends on etcd in the cluster performance. To ensure optimal speed, we recommend always using SSD disks to back your Rancher management Kubernetes cluster. On cloud providers, you will also want to use the minimum size that allows the maximum IOPS. In larger clusters, consider using dedicated storage devices for etcd data and wal directories.

-# Networking Requirements
+## Networking Requirements

 This section describes the networking requirements for the node(s) where the Rancher server is installed.

@@ -170,6 +170,6 @@ Each node used should have a static IP configured, regardless of whether you are

 To operate properly, Rancher requires a number of ports to be open on Rancher nodes and on downstream Kubernetes cluster nodes. [Port Requirements](../getting-started/installation-and-upgrade/installation-requirements/port-requirements.md) lists all the necessary ports for Rancher and Downstream Clusters for the different cluster types.

-# Dockershim Support
+## Dockershim Support

 For more information on Dockershim support, refer to [this page](../getting-started/installation-and-upgrade/installation-requirements/dockershim.md).
@@ -6,7 +6,7 @@ This section describes how to enable Istio and start using it in your projects.

 If you use Istio for traffic management, you will need to allow external traffic to the cluster. In that case, you will need to follow all of the steps below.

-# Prerequisites
+## Prerequisites

 This guide assumes you have already [installed Rancher,](installation-and-upgrade.md) and you have already [provisioned a separate Kubernetes cluster](kubernetes-clusters-in-rancher-setup.md) on which you will install Istio.

@@ -14,8 +14,7 @@ The nodes in your cluster must meet the [CPU and memory requirements.](../integr

 The workloads and services that you want to be controlled by Istio must meet [Istio's requirements.](https://istio.io/docs/setup/additional-setup/requirements/)

-
-# Install
+## Install

 :::tip Quick Setup Tip:

@@ -11,7 +11,7 @@ This section includes troubleshooting tips in the following categories:
 - [Troubleshooting nginx-proxy Nodes](../troubleshooting/kubernetes-components/troubleshooting-nginx-proxy.md)
 - [Troubleshooting Worker Nodes and Generic Components](../troubleshooting/kubernetes-components/troubleshooting-worker-nodes-and-generic-components.md)

-# Kubernetes Component Diagram
+## Kubernetes Component Diagram

 ![Cluster diagram](/img/clusterdiagram.svg)<br/>
 <sup>Lines show the traffic flow between components. Colors are used purely for visual aid</sup>
@@ -34,7 +34,7 @@ If you want to reuse a node from a previous custom cluster, [clean the node](../

 For more information, refer to the section on [custom nodes.](use-existing-nodes.md)

-# Programmatically Creating RKE Clusters
+### Programmatically Creating RKE Clusters

 The most common way to programmatically deploy RKE clusters through Rancher is by using the Rancher2 Terraform provider. The documentation for creating clusters with Terraform is [here.](https://registry.terraform.io/providers/rancher/rancher2/latest/docs/resources/cluster)

@@ -75,6 +75,6 @@ RKE2 provisioning also allows you to install custom clusters on pre-provisioned

 If you want to reuse a node from a previous custom cluster, clean the node before using it in a cluster again. If you reuse a node that hasn't been cleaned, cluster provisioning may fail.

-# Programmatically Creating RKE2 Clusters
+### Programmatically Creating RKE2 Clusters

 The most common way to programmatically deploy RKE2 clusters through Rancher is by using the Rancher2 Terraform provider. The documentation for creating clusters with Terraform is [here.](https://registry.terraform.io/providers/rancher/rancher2/latest/docs/resources/cluster_v2)
@@ -23,11 +23,11 @@ The monitoring application allows you to:
 - Defines precomputed, frequently needed or computationally expensive expressions as new time series based on metrics collected via Prometheus
 - Expose collected metrics from Prometheus to the Kubernetes Custom Metrics API via Prometheus Adapter for use in HPA

-# How Monitoring Works
+## How Monitoring Works

 For an explanation of how the monitoring components work together, see [this page.](../integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)

-# Default Components and Deployments
+## Default Components and Deployments

 ### Built-in Dashboards

@@ -48,11 +48,11 @@ The monitoring application deploys some alerts by default. To see the default al

 For a list of monitoring components exposed in the Rancher UI, along with common use cases for editing them, see [this section.](../integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md#components-exposed-in-the-rancher-ui)

-# Role-based Access Control
+## Role-based Access Control

 For information on configuring access to monitoring, see [this page.](../integrations-in-rancher/monitoring-and-alerting/rbac-for-monitoring.md)

-# Guides
+## Guides

 - [Enable monitoring](../how-to-guides/advanced-user-guides/monitoring-alerting-guides/enable-monitoring.md)
 - [Uninstall monitoring](../how-to-guides/advanced-user-guides/monitoring-alerting-guides/uninstall-monitoring.md)
@@ -61,7 +61,7 @@ For information on configuring access to monitoring, see [this page.](../integra
 - [Persistent Grafana dashboards](../how-to-guides/advanced-user-guides/monitoring-alerting-guides/create-persistent-grafana-dashboard.md)
 - [Debugging high memory usage](../how-to-guides/advanced-user-guides/monitoring-alerting-guides/debug-high-memory-usage.md)

-# Configuration
+## Configuration

 ### Configuring Monitoring Resources in Rancher

@@ -78,7 +78,7 @@ For information on configuring access to monitoring, see [this page.](../integra

 For more information on `rancher-monitoring` chart options, including options to set resource limits and requests, see [this page.](../reference-guides/monitoring-v2-configuration/helm-chart-options.md)

-# Windows Cluster Support
+## Windows Cluster Support

 When deployed onto an RKE1 Windows cluster, Monitoring V2 will now automatically deploy a [windows-exporter](https://github.com/prometheus-community/windows_exporter) DaemonSet and set up a ServiceMonitor to collect metrics from each of the deployed Pods. This will populate Prometheus with `windows_` metrics that are akin to the `node_` metrics exported by [node_exporter](https://github.com/prometheus/node_exporter) for Linux hosts.

@@ -87,8 +87,7 @@ To be able to fully deploy Monitoring V2 for Windows, all of your Windows hosts
 For more details on how to upgrade wins on existing Windows hosts, refer to the section on [Windows cluster support for Monitoring V2.](../integrations-in-rancher/monitoring-and-alerting/windows-support.md)


-
-# Known Issues
+## Known Issues

 There is a [known issue](https://github.com/rancher/rancher/issues/28787#issuecomment-693611821) that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend to setting `prometheus.prometheusSpec.resources.memory.limit` to 2500 Mi and `prometheus.prometheusSpec.resources.memory.request` to 1750 Mi.

@@ -6,7 +6,7 @@ This page captures some of the most important options for configuring Monitoring

 For information on configuring custom scrape targets and rules for Prometheus, please refer to the upstream documentation for the [Prometheus Operator.](https://github.com/prometheus-operator/prometheus-operator) Some of the most important custom resources are explained in the Prometheus Operator [design documentation.](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/design.md) The Prometheus Operator documentation can help also you set up RBAC, Thanos, or custom configuration.

-# Setting Resource Limits and Requests
+## Setting Resource Limits and Requests

 The resource requests and limits for the monitoring application can be configured when installing `rancher-monitoring`. For more information about the default limits, see [this page.](../reference-guides/monitoring-v2-configuration/helm-chart-options.md#configuring-resource-limits-and-requests)

@@ -16,7 +16,7 @@ On an idle cluster, Monitoring may have high CPU usage. To improve performance,

 :::

-# Prometheus Configuration
+## Prometheus Configuration

 It is usually not necessary to directly edit the Prometheus custom resource.

@@ -31,7 +31,7 @@ For details, see [this page.](../reference-guides/monitoring-v2-configuration/se

 For more information about directly editing the Prometheus custom resource, which may be helpful in advanced use cases, see [this page.](../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/prometheus.md)

-# Alertmanager Configuration
+## Alertmanager Configuration

 The Alertmanager custom resource usually doesn't need to be edited directly. For most common use cases, you can manage alerts by updating Routes and Receivers.

@@ -12,6 +12,6 @@ A Nutanix cluster may consist of multiple groups of VMs with distinct properties
 - [Creating a Nutanix Cluster](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/nutanix/provision-kubernetes-clusters-in-aos.md#creating-a-nutanix-aos-cluster)
 - [Provisioning Storage](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/nutanix/provision-kubernetes-clusters-in-aos)

-# Creating a Nutanix Cluster
+## Creating a Nutanix Cluster

 In [this section,](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/nutanix/provision-kubernetes-clusters-in-aos.md) you'll learn how to use Rancher to install an [RKE](https://rancher.com/docs/rke/latest/en/) Kubernetes cluster in Nutanix AOS.
@@ -6,7 +6,7 @@ In a lot of enterprise environments, servers or VMs running on premise do not ha

 Alternatively, it is also possible to set up Rancher completely air-gapped without any Internet access. This process is described in detail in the [Rancher docs](air-gapped-helm-cli-install.md).

-# Installation Outline
+## Installation Outline

 1. [Set up infrastructure](../getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/set-up-infrastructure.md)
 2. [Set up a Kubernetes cluster](../getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/install-kubernetes.md)
@@ -17,19 +17,19 @@ A Docker installation of Rancher is recommended only for development and testing

 The Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster.](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)

-### Privileged Access for Rancher
+## Privileged Access for Rancher

 When the Rancher server is deployed in the Docker container, a local Kubernetes cluster is installed within the container for Rancher to use. Because many features of Rancher run as deployments, and privileged mode is required to run containers within containers, you will need to install Rancher with the `--privileged` option.

-# Requirements for OS, Docker, Hardware, and Networking
+## Requirements for OS, Docker, Hardware, and Networking

 Make sure that your node fulfills the general [installation requirements.](installation-requirements.md)

-# 1. Provision Linux Host
+## 1. Provision Linux Host

 Provision a single Linux host according to our [Requirements](installation-requirements.md) to launch your Rancher server.

-# 2. Choose an SSL Option and Install Rancher
+## 2. Choose an SSL Option and Install Rancher

 For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.

@@ -9,7 +9,7 @@ To use this option you'll need access to servers you intend to use in your Kuber

 This section describes how to set up a custom cluster.

-# Creating a Cluster with Custom Nodes
+## Creating a Cluster with Custom Nodes

 :::note Want to use Windows hosts as Kubernetes workers?

@@ -17,7 +17,6 @@ See [Configuring Custom Clusters for Windows](use-windows-clusters.md) before yo

 :::

-
 ### 1. Provision a Linux Host

 Begin creation of a custom cluster by provisioning a Linux host. Your host can be:
@@ -130,7 +129,7 @@ If you share resources between clusters, you can change the tag to:
 Key=kubernetes.io/cluster/CLUSTERID, Value=shared
 ```

-# Optional Next Steps
+## Optional Next Steps

 After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:

@@ -242,7 +242,7 @@ spec:
  encryptionConfigSecretName: test-encryptionconfig
 ```

-# Example Credential Secret for Storing Backups in S3
+## Example Credential Secret for Storing Backups in S3

 ```yaml
 apiVersion: v1
@@ -255,7 +255,7 @@ data:
  secretKey: <Enter your base64-encoded secret key>
 ```

-# Example EncryptionConfiguration
+## Example EncryptionConfiguration

 The snippet below demonstrates two different types of secrets and their relevance with respect to Backup and Restore of custom resources.

@@ -13,7 +13,7 @@ This guide outlines a reference architecture for provisioning downstream Rancher

 ![Solution Overview](/img/solution_overview.drawio.svg)

-# 1. VM Considerations
+## 1. VM Considerations

 ### Leverage VM Templates to Construct the Environment

@@ -31,7 +31,7 @@ Doing so will ensure node VM's are spread across multiple datastores - preventin

 It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.

-# 2. Network Considerations
+## 2. Network Considerations

 ### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes

@@ -41,13 +41,13 @@ Deploy etcd members within a single data center where possible to avoid latency

 Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.

-# 3. Storage Considerations
+## 3. Storage Considerations

 ### Leverage SSD Drives for ETCD Nodes

 ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible.

-# 4. Backups and Disaster Recovery
+## 4. Backups and Disaster Recovery

 ### Perform Regular Downstream Cluster Backups

@@ -2,7 +2,7 @@
 title: kubectl Utility
 ---

-# kubectl
+## kubectl

 Interact with Rancher using kubectl.

@@ -4,7 +4,7 @@ title: Architecture Recommendations

 If you are installing Rancher on a single node, the main architecture recommendation that applies to your installation is that the node running Rancher should be [separate from downstream clusters.](#separation-of-rancher-and-user-clusters)

-# Separation of Rancher and User Clusters
+## Separation of Rancher and User Clusters

 A user cluster is a downstream Kubernetes cluster that runs your apps and services.

@@ -14,7 +14,7 @@ If Rancher is intended to manage downstream Kubernetes clusters, the Kubernetes

 ![Separation of Rancher Server from User Clusters](/img/rancher-architecture-separation-of-rancher-server.svg)

-# Why HA is Better for Rancher in Production
+## Why HA is Better for Rancher in Production

 We recommend installing the Rancher server on a high-availability Kubernetes cluster, primarily because it protects the Rancher server data. In a high-availability installation, a load balancer serves as the single point of contact for clients, distributing network traffic across multiple servers in the cluster and helping to prevent any one server from becoming a point of failure.

@@ -36,7 +36,7 @@ In an RKE installation, the cluster data is replicated on each of three etcd nod

 ![Architecture of an RKE Kubernetes cluster running the Rancher management server](/img/rke-server-storage.svg)

-# Recommended Load Balancer Configuration for Kubernetes Installations
+## Recommended Load Balancer Configuration for Kubernetes Installations

 We recommend the following configurations for the load balancer and Ingress controllers:

@@ -49,13 +49,13 @@ We recommend the following configurations for the load balancer and Ingress cont

 ![Rancher HA](/img/ha/rancher2ha.svg)

-# Environment for Kubernetes Installations
+## Environment for Kubernetes Installations

 It is strongly recommended to install Rancher on a Kubernetes cluster on hosted infrastructure such as Amazon's EC2 or Google Compute Engine.

 For the best performance and greater security, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../pages-for-subheaders/kubernetes-clusters-in-rancher-setup.md) for running your workloads.

-# Recommended Node Roles for Kubernetes Installations
+## Recommended Node Roles for Kubernetes Installations

 The below recommendations apply when Rancher is installed on a K3s Kubernetes cluster or an RKE Kubernetes cluster.

@@ -97,7 +97,7 @@ Because no additional workloads will be deployed on the Rancher server cluster,

 For more best practices for downstream clusters, refer to the [production checklist](../../pages-for-subheaders/checklist-for-production-ready-clusters.md) or our [best practices guide.](../../pages-for-subheaders/best-practices.md)

-# Architecture for an Authorized Cluster Endpoint (ACE)
+## Architecture for an Authorized Cluster Endpoint (ACE)

 If you are using an [authorized cluster endpoint (ACE),](../../pages-for-subheaders/rancher-manager-architecture.md#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.

@@ -71,11 +71,11 @@ Like the authorized cluster endpoint, the `kube-api-auth` authentication service

 > **Example scenario:** Let's say that the Rancher server is located in the United States, and User Cluster 1 is located in Australia. A user, Alice, also lives in Australia. Alice can manipulate resources in User Cluster 1 by using the Rancher UI, but her requests will have to be sent from Australia to the Rancher server in the United States, then be proxied back to Australia, where the downstream user cluster is. The geographical distance may cause significant latency, which Alice can reduce by using the authorized cluster endpoint.

-With this endpoint enabled for the downstream cluster, Rancher generates an extra Kubernetes context in the kubeconfig file in order to connect directly to the cluster. This file has the credentials for `kubectl` and `helm`. 
+With this endpoint enabled for the downstream cluster, Rancher generates an extra Kubernetes context in the kubeconfig file in order to connect directly to the cluster. This file has the credentials for `kubectl` and `helm`.

 You will need to use a context defined in this kubeconfig file to access the cluster if Rancher goes down. Therefore, we recommend exporting the kubeconfig file so that if Rancher goes down, you can still use the credentials in the file to access your cluster. For more information, refer to the section on accessing your cluster with [kubectl and the kubeconfig file.](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md)

-# Important Files
+## Important Files

 The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster:

@@ -87,7 +87,7 @@ The files mentioned below are needed to maintain, troubleshoot and upgrade your

 For more information on connecting to a cluster without the Rancher authentication proxy and other configuration options, refer to the [kubeconfig file](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md) documentation.

-# Tools for Provisioning Kubernetes Clusters
+## Tools for Provisioning Kubernetes Clusters

 The tools that Rancher uses to provision downstream user clusters depends on the type of cluster that is being provisioned.

@@ -113,7 +113,7 @@ Rancher provisions this type of cluster using [kontainer-engine.](https://github

 In this type of cluster, Rancher connects to a Kubernetes cluster that has already been set up. Therefore, Rancher does not provision Kubernetes, but only sets up the Rancher agents to communicate with the cluster.

-# Rancher Server Components and Source Code
+## Rancher Server Components and Source Code

 This diagram shows each component that the Rancher server is composed of:

@@ -8,11 +8,11 @@ System Tools has been deprecated since June 2022.

 :::

-# Logs
+## Logs

 Please use [logs-collector](https://github.com/rancherlabs/support-tools/tree/master/collection/rancher/v2.x/logs-collector) to collect logs from your cluster.

-# Stats
+## Stats

 If you want to replicate the stats command, you can run the following command on your cluster nodes:

@@ -26,6 +26,6 @@ This command below requires the package `sysstat` on the cluster node.
 /usr/bin/sar -u -r -F 1 1
 ```

-# Remove
+## Remove

 Please use the [Rancher Cleanup](https://github.com/rancher/rancher-cleanup) tool.
@@ -15,7 +15,7 @@
 | [Cleaning Kubernetes components when clusters are no longer reachable from Rancher](../how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md) | ✓ | | | |
 | [Configuring Pod Security Policies](../how-to-guides/new-user-guides/manage-clusters/add-a-pod-security-policy.md) | ✓ | ✓ |  ||

-1. Registered GKE and EKS clusters have the same options available as GKE and EKS clusters created from the Rancher UI. The  difference is that when a registered cluster is deleted from the Rancher UI, it is not destroyed.
+1. Registered EKS, GKE and AKS clusters have the same options available as EKS, GKE and AKS clusters created from the Rancher UI. The  difference is that when a registered cluster is deleted from the Rancher UI, it is not destroyed.

 2. Cluster configuration options can't be edited for registered clusters, except for [K3s and RKE2 clusters.](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md)

@@ -4,7 +4,7 @@ title: Troubleshooting Controlplane Nodes

 This section applies to nodes with the `controlplane` role.

-# Check if the Controlplane Containers are Running
+## Check if the Controlplane Containers are Running

 There are three specific containers launched on nodes with the `controlplane` role:

@@ -26,7 +26,7 @@ f3d287ca4549        rancher/hyperkube:v1.11.5-rancher1   "/opt/rke-tools/en..."
 bdf3898b8063        rancher/hyperkube:v1.11.5-rancher1   "/opt/rke-tools/en..."   3 hours ago         Up 3 hours                              kube-controller-manager
 ```

-# Controlplane Container Logging
+## Controlplane Container Logging

 :::note

@@ -42,7 +42,7 @@ docker logs kube-controller-manager
 docker logs kube-scheduler
 ```

-# RKE2 Server Logging
+## RKE2 Server Logging

 If Rancher provisions an RKE2 cluster that can't communicate with Rancher, you can run this command on a server node in the downstream cluster to get the RKE2 server logs:

@@ -4,7 +4,7 @@ title: Troubleshooting nginx-proxy

 The `nginx-proxy` container is deployed on every node that does not have the `controlplane` role. It provides access to all the nodes with the `controlplane` role by dynamically generating the NGINX configuration based on available nodes with the `controlplane` role.

-# Check if the Container is Running
+## Check if the Container is Running

 The container is called `nginx-proxy` and should have status `Up`. The duration shown after `Up` is the time the container has been running.

@@ -20,7 +20,7 @@ CONTAINER ID        IMAGE                       COMMAND                  CREATED
 c3e933687c0e        rancher/rke-tools:v0.1.15   "nginx-proxy CP_HO..."   3 hours ago         Up 3 hours                              nginx-proxy
 ```

-# Check Generated NGINX Configuration
+## Check Generated NGINX Configuration

 The generated configuration should include the IP addresses of the nodes with the `controlplane` role. The configuration can be checked using the following command:

@@ -59,7 +59,7 @@ stream {
 }
 ```

-# nginx-proxy Container Logging
+## nginx-proxy Container Logging

 The logging of the containers can contain information on what the problem could be.

@@ -4,7 +4,7 @@ title: Troubleshooting Worker Nodes and Generic Components

 This section applies to every node as it includes components that run on nodes with any role.

-# Check if the Containers are Running
+## Check if the Containers are Running

 There are two specific containers launched on nodes with the `worker` role:

@@ -24,7 +24,7 @@ CONTAINER ID        IMAGE                                COMMAND
 a30717ecfb55        rancher/hyperkube:v1.11.5-rancher1   "/opt/rke-tools/en..."   3 hours ago         Up 3 hours                              kubelet
 ```

-# Container Logging
+## Container Logging

 The logging of the containers can contain information on what the problem could be.

@@ -26,7 +26,7 @@ Harvester 已 GA。有关所有更新，请参阅 [Harvester 发行说明](https

 ### Harvester 主机驱动

-在 Rancher 的 RKE 和 RKE2 选项中，[Harvester 主机驱动](https://docs.harvesterhci.io/v0.3/rancher/node-driver/)被标记为`技术预览`。在**创建**页面和启用驱动后的页面都是一样的情况。无论 Harvester 功能开关是否启用，主机驱动都是可用的。请注意，默认情况下主机驱动是关闭的。用户只能通过**集群管理**页面在 Harvester 上创建 RKE 或 RKE2 集群。
+从 Rancher v2.6.3 开始，[Harvester 主机驱动](https://docs.harvesterhci.io/v0.3/rancher/node-driver/) 已在 Rancher 的 RKE 和 RKE2 选项中 GA。无论 Harvester 功能开关是否启用，主机驱动都是可用的。请注意，默认情况下主机驱动是关闭的。用户只能通过**集群管理**页面在 Harvester 上创建 RKE 或 RKE2 集群。

 Harvester 允许通过 Harvester UI 上传和显示 `.ISO` 镜像，但 Rancher UI 不支持。这是因为 `.ISO` 镜像通常需要额外的设置，这会干扰干净的部署（即无需用户干预），并且它们通常不用于云环境。

@@ -103,3 +103,9 @@ Rancher 依赖用户和组来决定允许登录到 Rancher 的用户，以及他
   ![Sign In External Principal](/img/users-page.png)

 6. 外部主体和本地主体共享相同的访问权限。
+
+:::note 重新配置以前设置的身份验证提供程序
+
+如果你需要重新配置或禁用以前设置的提供程序然后再重新启用它，请确保进行此操作的用户使用外部用户身份登录 Rancher，而不是本地管理员。
+
+:::
@@ -6,7 +6,8 @@ When you create a cluster, some alert rules are predefined. These alerts notify

 Several of the alerts use Prometheus expressions as the metric that triggers the alert. For more information on how expressions work, you can refer to the Rancher [documentation about Prometheus expressions](../cluster-monitoring/expression.md) or the Prometheus [documentation about querying metrics](https://prometheus.io/docs/prometheus/latest/querying/basics/).

-# Alerts for etcd
+## Alerts for etcd
+
 Etcd is the key-value store that contains the state of the Kubernetes cluster. Rancher provides default alerts if the built-in monitoring detects a potential problem with etcd. You don't have to enable monitoring to receive these alerts.

 A leader is the node that handles all client requests that need cluster consensus. For more information, you can refer to this [explanation of how etcd works.](https://rancher.com/blog/2019/2019-01-29-what-is-etcd/#how-does-etcd-work)
@@ -20,8 +21,7 @@ The leader of the cluster can change in response to certain events. It is normal
 | Etcd is unavailable | A critical alert is triggered when etcd becomes unavailable. |
 | Etcd member has no leader | A critical alert is triggered when the etcd cluster does not have a leader for at least three minutes. |

-
-# Alerts for Kubernetes Components
+## Alerts for Kubernetes Components
 Rancher provides alerts when core Kubernetes system components become unhealthy.

 Controllers update Kubernetes resources based on changes in etcd. The [controller manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) monitors the cluster desired state through the Kubernetes API server and makes the necessary changes to the current state to reach the desired state.
@@ -33,16 +33,16 @@ The [scheduler](https://kubernetes.io/docs/reference/command-line-tools-referenc
 | Controller Manager is unavailable |  A critical warning is triggered when the cluster’s controller-manager becomes unavailable. |
 | Scheduler is unavailable | A critical warning is triggered when the cluster’s scheduler becomes unavailable. |

+## Alerts for Events

-# Alerts for Events
 Kubernetes events are objects that provide insight into what is happening inside a cluster, such as what decisions were made by the scheduler or why some pods were evicted from the node. In the Rancher UI, from the project view, you can see events for each workload.

 | Alert | Explanation |
 |-------|-------------|
 | Get warning deployment event | A warning alert is triggered when a warning event happens on a deployment. |

+## Alerts for Nodes

-# Alerts for Nodes
 Alerts can be triggered based on node metrics. Each computing resource in a Kubernetes cluster is called a node. Nodes can be either bare-metal servers or virtual machines.

 | Alert | Explanation |
@@ -51,5 +51,6 @@ Alerts can be triggered based on node metrics. Each computing resource in a Kube
 | High node memory utilization | A warning alert is triggered if the node uses more than 80 percent of its available memory for at least three minutes. |
 | Node disk is running full within 24 hours | A critical alert is triggered if the disk space on the node is expected to run out in the next 24 hours based on the disk growth over the last 6 hours. |

-# Project-level Alerts
+## Project-level Alerts
+
 When you enable monitoring for the project, some project-level alerts are provided. For details, refer to the [section on project-level alerts.](../../../reference-guides/rancher-project-tools/project-alerts.md)
@@ -8,8 +8,7 @@ The PromQL expressions in this doc can be used to configure [alerts.](../../../p

 For more information about querying Prometheus, refer to the official [Prometheus documentation.](https://prometheus.io/docs/prometheus/latest/querying/basics/)

-
-# Cluster Metrics
+## Cluster Metrics

 ### Cluster CPU Utilization

@@ -60,7 +59,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr></table> |
 | Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr></table> |

-# Node Metrics
+## Node Metrics

 ### Node CPU Utilization

@@ -111,7 +110,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr></table> |
 | Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr></table> |

-# Etcd Metrics
+## Etcd Metrics

 ### Etcd Has a Leader

@@ -181,7 +180,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | <table><tr><td>wal</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le))`</td></tr><tr><td>db</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le))`</td></tr></table> |
 | Summary | <table><tr><td>wal</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr><tr><td>db</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr></table> |

-# Kubernetes Components Metrics
+## Kubernetes Components Metrics

 ### API Server Request Latency

@@ -239,8 +238,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | `topk(10, histogram_quantile(0.95,sum by (le, host, path)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |
 | Summary | `topk(10, histogram_quantile(0.95,sum by (le, host)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |

-# Rancher Logging Metrics
-
+## Rancher Logging Metrics

 ### Fluentd Buffer Queue Rate

@@ -270,7 +268,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | `sum(rate(fluentd_output_status_num_records_total[5m])) by (instance)` |
 | Summary | `sum(rate(fluentd_output_status_num_records_total[5m]))` |

-# Workload Metrics
+## Workload Metrics

 ### Workload CPU Utilization

@@ -307,7 +305,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
 | Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |

-# Pod Metrics
+## Pod Metrics

 ### Pod CPU Utilization

@@ -344,7 +342,7 @@ For more information about querying Prometheus, refer to the official [Prometheu
 | Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr></table> |
 | Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |

-# Container Metrics
+## Container Metrics

 ### Container CPU Utilization

@@ -12,7 +12,7 @@ While configuring monitoring at either the [cluster level](../../../pages-for-su
 - [Persistent Storage](#persistent-storage)
 - [Remote Storage](#remote-storage)

-# Basic Configuration
+## Basic Configuration

 Option | Description
 -------|-------------
@@ -27,7 +27,7 @@ Prometheus [Memory Limit](https://kubernetes.io/docs/concepts/configuration/mana
 Prometheus [Memory Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource requests for the Prometheus pod.
 Selector | Ability to select the nodes in which Prometheus and Grafana pods are deployed to. To use this option, the nodes must have labels.

-# Advanced Options
+## Advanced Options

 Since monitoring is an [application](https://github.com/rancher/system-charts/tree/dev/charts/rancher-monitoring) from the [Rancher catalog](../../../pages-for-subheaders/helm-charts-in-rancher.md), it can be configured like any other catalog application, by passing in values to Helm.

@@ -64,7 +64,7 @@ Some example key-value pairs are:
 | `prometheus.livenessProbe.timeoutSeconds` | 60 |
 | `prometheus.readinessProbe.timeoutSeconds` | 60 |

-# Node Exporter
+## Node Exporter

 The [node exporter](https://github.com/prometheus/node_exporter/blob/master/README.md) is a popular open source exporter, which exposes the metrics for hardware and \*NIX kernels OS. It is designed to monitor the host system. However, there are still issues with namespaces when running it in a container, mostly around filesystem mount spaces. In order to monitor actual network metrics for the container network, the node exporter must be deployed with the `hostNetwork` mode.

@@ -72,7 +72,7 @@ When configuring Prometheus and enabling the node exporter, enter a host port in

 >**Warning:** In order for Prometheus to collect the metrics of the node exporter, after enabling cluster monitoring, you must open the <b>Node Exporter Host Port</b> in the host firewall rules to allow intranet access. By default, `9796` is used as that host port.

-# Persistent Storage
+## Persistent Storage

 >**Prerequisite:** Configure one or more StorageClasses to use as [persistent storage](../../../pages-for-subheaders/create-kubernetes-persistent-storage.md) for your Prometheus or Grafana pod.

@@ -80,7 +80,7 @@ By default, when you enable Prometheus for either a cluster or project, all moni

 When enabling persistent storage for Prometheus or Grafana, specify the size of the persistent volume and select the StorageClass.

-# Remote Storage
+## Remote Storage

 >**Prerequisite:** Need a remote storage endpoint to be available. The possible list of integrations is available [here](https://prometheus.io/docs/operating/integrations/)

@@ -4,7 +4,7 @@ title: Disabling Istio

 This section describes how to disable Istio in a cluster, namespace, or workload.

-# Disable Istio in a Cluster
+## Disable Istio in a Cluster

 To disable Istio,

@@ -14,13 +14,13 @@ To disable Istio,

 **Result:** The `cluster-istio` application in the cluster's `system` project gets removed. The Istio sidecar cannot be deployed on any workloads in the cluster.

-# Disable Istio in a Namespace
+## Disable Istio in a Namespace

 1. In the Rancher UI, go to the project that has the namespace where you want to disable Istio.
 1. On the **Workloads** tab, you will see a list of namespaces and the workloads deployed in them. Go to the namespace where you want to disable and click the **&#8942; > Disable Istio Auto Injection.**

 **Result:** When workloads are deployed in this namespace, they will not have the Istio sidecar.

-# Remove the Istio Sidecar from a Workload
+## Remove the Istio Sidecar from a Workload

 Disable Istio in the namespace, then redeploy the workloads with in it. They will be deployed without the Istio sidecar.
@@ -4,7 +4,7 @@ title: Role-based Access Control

 This section describes the permissions required to access Istio features and how to configure access to the Kiali and Jaeger visualizations.

-# Cluster-level Access
+## Cluster-level Access

 By default, only cluster administrators can:

@@ -12,7 +12,7 @@ By default, only cluster administrators can:
 - Configure resource allocations for Istio
 - View each UI for Prometheus, Grafana, Kiali, and Jaeger

-# Project-level Access
+## Project-level Access

 After Istio is enabled in a cluster, project owners and members have permission to:

@@ -22,7 +22,7 @@ After Istio is enabled in a cluster, project owners and members have permission
 - View the Kiali and Jaeger visualizations if cluster administrators give access to project members
 - Configure Istio's resources (such as the gateway, destination rules, or virtual services) with `kubectl` (This does not apply to read-only project members)

-# Access to Visualizations
+## Access to Visualizations

 By default,  the Kiali and Jaeger visualizations are restricted to the cluster owner because the information in them could be sensitive.

@@ -40,7 +40,7 @@ To configure who has permission to access the Kiali and Jaeger UI,

 **Result:** The access levels for Kiali and Jaeger have been updated.

-# Summary of Default Permissions for Istio Users
+## Summary of Default Permissions for Istio Users

 | Permission                         | Cluster Administrators | Project Owners | Project Members | Read-only Project Members |
 |------------------------------------------|----------------|----------------|-----------------|---------------------------|
@@ -3,7 +3,7 @@ title: Release Notes
 ---


-# Istio 1.5.8
+## Istio 1.5.8

 ### Important note on 1.5.x versions

@@ -15,11 +15,11 @@ Rancher integrates with a variety of popular IT services, including:
 - **Microsoft Teams**: (Available as of v2.4.6) Send alert notifications to Teams using a webhook.


-# Roles-based Access Control for Notifiers
+## Roles-based Access Control for Notifiers

 Notifiers are configured at the cluster level. This model ensures that only cluster owners need to configure notifiers, leaving project owners to simply configure alerts in the scope of their projects. You don't need to dispense privileges like SMTP server access or cloud account access.

-# Adding Notifiers
+## Adding Notifiers

 Set up a notifier so that you can begin configuring and sending alerts.

@@ -31,7 +31,7 @@ Set up a notifier so that you can begin configuring and sending alerts.

 **Result:** Your notifier is added to Rancher.

-# Configuration
+## Configuration

 - [Slack](#slack)
 - [Email](#email)
@@ -140,7 +140,7 @@ _Available as of v2.4.6_

 **Validation:** Click **Test.** If the test is successful, the Teams notifier output is **MicrosoftTeams setting validated.**

-# Managing Notifiers
+## Managing Notifiers

 After you set up notifiers, you can manage them. From the **Global** view, open the cluster that you want to manage your notifiers. Select **Tools > Notifiers**. You can:

@@ -148,7 +148,7 @@ After you set up notifiers, you can manage them. From the **Global** view, open
 - **Clone** them, to quickly setup slightly different notifiers.
 - **Delete** them when they're no longer necessary.

-# Example Payload for a Webhook Alert Notifier
+## Example Payload for a Webhook Alert Notifier

 ```json
 {
@@ -184,7 +184,8 @@ After you set up notifiers, you can manage them. From the **Global** view, open
  }
 }
 ```
-# What's Next?
+
+## What's Next?

 After creating a notifier, set up alerts to receive notifications of Rancher system events.

@@ -4,7 +4,7 @@ title: Questions about Upgrading to Rancher v2.x

 This page contains frequently asked questions about the changes between Rancher v1.x and v2.x, and how to upgrade from Rancher v1.x to v2.x.

-# Kubernetes
+## Kubernetes

 **What does it mean when you say Rancher v2.x is built on Kubernetes?**

@@ -34,7 +34,7 @@ No.  Unfortunately, the differences were enough such that we cannot support Ranc

 Absolutely.

-# Cattle
+## Cattle

 **How does Rancher v2.x affect Cattle?**

@@ -46,7 +46,7 @@ Cattle will not supported in v2.x as Rancher has been re-architected to be based

 Yes. In the upcoming Rancher v2.1 release we will provide a tool to help translate existing Cattle workloads in Compose format to Kubernetes YAML format.  You will then be able to deploy those workloads on the v2.x platform.

-# Feature Changes
+## Feature Changes

 **Can we still add our own infrastructure services, which had a separate view/filter in 1.6.x?**

@@ -70,7 +70,7 @@ Yes. You can do so by leveraging Kubernetes' network policies.

 Yes. Definitely.

-# Environments & Clusters
+## Environments & Clusters

 **Can I still create templates for environments and clusters?**

@@ -84,7 +84,7 @@ Kubernetes RKE Templates is on our roadmap for 2.x. Please refer to our Release

 Yes. We still provide you with the same way of executing our Rancher agents directly on hosts.

-# Upgrading/Migrating
+## Upgrading/Migrating

 **How would the migration from v1.x to v2.x work?**

@@ -96,7 +96,7 @@ Due to the technical difficulty in transforming a Docker container into a pod ru

 At this time, we are still exploring this scenario and taking feedback. We anticipate that you will need to launch a new Rancher instance and then relaunch on v2.x. Once you've moved to v2.x, upgrades will be in place, as they are in v1.6.

-# Support
+## Support

 **Are you planning some long-term support releases for Rancher v1.6?**

@@ -7,7 +7,7 @@ import TabItem from '@theme/TabItem';

 This section is about how to prepare your node(s) to install Rancher for your air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. There are _tabs_ for either a high availability (recommended) or a Docker installation.

-# Prerequisites
+## Prerequisites

 <Tabs>
 <TabItem value="Kubernetes Install (Recommended)">
@@ -47,7 +47,7 @@ If you need help with creating a private registry, please refer to the [Docker d
 </TabItem>
 </Tabs>

-# Set up Infrastructure
+## Set up Infrastructure

 <Tabs>
 <TabItem value="Kubernetes Install (Recommended)">
@@ -33,7 +33,7 @@ sudo iptables --list

 This section describes how to use `firewalld` to apply the [firewall port rules](../../installation-requirements/port-requirements.md) for nodes in a high-availability Rancher server cluster.

-# Prerequisite
+## Prerequisite

 Install v7.x or later ofv`firewalld`:

@@ -43,7 +43,7 @@ systemctl start firewalld
 systemctl enable firewalld
 ```

-# Applying Firewall Port Rules
+## Applying Firewall Port Rules

 In the Rancher high-availability installation instructions, the Rancher server is set up on three nodes that have all three Kubernetes roles: etcd, controlplane, and worker. If your Rancher server nodes have all three roles, run the following commands on each node:

@@ -13,7 +13,7 @@ Environment Variable Key | Default Value | Status | Available as of
 `istio-virtual-service-ui` |`false` |  Experimental | v2.3.0
 `istio-virtual-service-ui` | `true` | GA                    | v2.3.2

-# About this Feature
+## About this Feature

 A central advantage of Istio's traffic management features is that they allow dynamic request routing, which is useful for canary deployments, blue/green deployments, or A/B testing.

@@ -23,12 +23,12 @@ If you installed Rancher using the RKE Add-on yaml, follow the directions to [mi
 > - If you are upgrading Rancher from v2.x to v2.3+, and you are using external TLS termination, you will need to edit the cluster.yml to [enable using forwarded host headers.](../../../../reference-guides/installation-references/helm-chart-options.md#configuring-ingress-for-external-tls-when-using-nginx-v0-25)
 > - The upgrade instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 migration docs.](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) This [section](../../../../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/helm2.md) provides a copy of the older upgrade instructions that used Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible.

-# Prerequisites
+## Prerequisites

 - **Review the [known upgrade issues](../../../../pages-for-subheaders/upgrades.md)** in the Rancher documentation for the most noteworthy issues to consider when upgrading Rancher. A more complete list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
 - **For [air gap installs only,](../../../../pages-for-subheaders/air-gapped-helm-cli-install.md) collect and populate images for the new Rancher server version.** Follow the guide to [populate your private registry](../../other-installation-methods/air-gapped-helm-cli-install/publish-images.md) with the images for the Rancher version that you want to upgrade to.

-# Upgrade Outline
+## Upgrade Outline

 Follow the steps to upgrade Rancher server:

@@ -9,7 +9,7 @@ import PortsImportedHosted from '@site/src/components/PortsImportedHosted'

 To operate properly, Rancher requires a number of ports to be open on Rancher nodes and on downstream Kubernetes cluster nodes.

-# Rancher Nodes
+## Rancher Nodes

 The following table lists the ports that need to be open to and from nodes that are running the Rancher server.

@@ -127,7 +127,7 @@ The following tables break down the port requirements for Rancher nodes, for inb

 </details>

-# Downstream Kubernetes Cluster Nodes
+## Downstream Kubernetes Cluster Nodes

 Downstream Kubernetes clusters run your apps and services. This section describes what ports need to be opened on the nodes in downstream clusters so that Rancher can communicate with them.

@@ -191,8 +191,7 @@ The following table depicts the port requirements for [imported clusters](../../

 </details>

-
-# Other Port Considerations
+## Other Port Considerations

 ### Commonly Used Ports

@@ -20,7 +20,7 @@ This section describes installing Rancher in five parts:
 - [4. Install Rancher](#4-install-rancher)
 - [5. For Rancher versions before v2.3.0, Configure System Charts](#5-for-rancher-versions-before-v2-3-0-configure-system-charts)

-# 1. Add the Helm Chart Repository
+## 1. Add the Helm Chart Repository

 From a system that has access to the internet, fetch the latest Helm chart and copy the resulting manifests to a system that has access to the Rancher server cluster.

@@ -51,7 +51,7 @@ From a system that has access to the internet, fetch the latest Helm chart and c
    helm fetch rancher-stable/rancher --version=v2.4.8
    ```

-# 2. Choose your SSL Configuration
+## 2. Choose your SSL Configuration

 Rancher Server is designed to be secure by default and requires SSL/TLS configuration.

@@ -64,7 +64,7 @@ When Rancher is installed on an air gapped Kubernetes cluster, there are two rec
 | Rancher Generated Self-Signed Certificates | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)<br/> This is the **default** and does not need to be added when rendering the Helm template. | yes                   |
 | Certificates from Files                    | `ingress.tls.source=secret`  | Use your own certificate files by creating Kubernetes Secret(s). <br/> This option must be passed when rendering the Rancher Helm template.                  | no                    |

-# 3. Render the Rancher Helm Template
+## 3. Render the Rancher Helm Template

 When setting up the Rancher Helm template, there are several options in the Helm chart that are designed specifically for air gap installations.

@@ -180,7 +180,7 @@ Then refer to [Adding TLS Secrets](../../resources/add-tls-secrets.md) to publis

 </details>

-# 4. Install Rancher
+## 4. Install Rancher

 Copy the rendered manifest directories to a system that has access to the Rancher server cluster to complete installation.

@@ -225,11 +225,11 @@ kubectl -n cattle-system apply -R -f ./rancher

 > **Note:** If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login. Leaving this active in an air-gapped environment can cause issues if the sockets cannot be opened successfully.

-# 5. For Rancher versions before v2.3.0, Configure System Charts
+## 5. For Rancher versions before v2.3.0, Configure System Charts

 If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in Github, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../resources/local-system-charts.md).

-# Additional Resources
+## Additional Resources

 These resources could be helpful when installing Rancher:

@@ -112,7 +112,7 @@ _Available as of v2.3.0_

 For Rancher servers that will provision Linux and Windows clusters, there are distinctive steps to populate your private registry for the Windows images and the Linux images. Since a Windows cluster is a mix of Linux and Windows nodes, the Linux images pushed into the private registry are manifests.

-# Windows Steps
+## Windows Steps

 The Windows images need to be collected and pushed from a Windows server workstation.

@@ -191,7 +191,7 @@ The `rancher-windows-images.txt` is expected to be on the workstation in the sam
   ./rancher-load-images.ps1 --registry <REGISTRY.YOURDOMAIN.COM:PORT>
   ```

-# Linux Steps
+## Linux Steps

 The Linux images needs to be collected and pushed from a Linux host, but _must be done after_ populating the Windows images into the private registry. These step are different from the Linux only steps as the Linux images that are pushed will actually manifests that support Windows and Linux images.

@@ -2,7 +2,7 @@
 title: '1. Set up Infrastructure'
 ---

-In this section, you will provision the underlying infrastructure for your Rancher management server with internete access through a HTTP proxy.
+In this section, you will provision the underlying infrastructure for your Rancher management server with internet access through a HTTP proxy.

 To install the Rancher management server on a high-availability RKE cluster, we recommend setting up the following infrastructure:

@@ -7,12 +7,12 @@ import TabItem from '@theme/TabItem';

 The following instructions will guide you through upgrading a Rancher server that was installed with Docker.

-# Prerequisites
+## Prerequisites

 - **Review the [known upgrade issues](../../../../pages-for-subheaders/upgrades.md#known-upgrade-issues) in the Rancher documentation for the most noteworthy issues to consider when upgrading Rancher. A more complete list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12) Note that upgrades to or from any chart in the [rancher-alpha repository](../../../../reference-guides/installation-references/helm-chart-options.md#helm-chart-repositories/) aren’t supported.
 - **For [air gap installs only,](../../../../pages-for-subheaders/air-gapped-helm-cli-install.md) collect and populate images for the new Rancher server version.** Follow the guide to [populate your private registry](../air-gapped-helm-cli-install/publish-images.md) with the images for the Rancher version that you want to upgrade to.

-# Placeholder Review
+## Placeholder Review

 During upgrade, you'll enter a series of commands, filling placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (`<EXAMPLE>`).

@@ -24,7 +24,7 @@ docker stop <RANCHER_CONTAINER_NAME>

 In this command, `<RANCHER_CONTAINER_NAME>` is the name of your Rancher container.

-# Get Data for Upgrade Commands
+## Get Data for Upgrade Commands

 To obtain the data to replace the placeholders, run:

@@ -48,7 +48,7 @@ Write down or copy this information before starting the upgrade.

 You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by logging into your Rancher server by remote connection and entering the command to view the containers that are running: `docker ps`. You can also view containers that are stopped using a different command: `docker ps -a`. Use these commands for help anytime during while creating backups.

-# Upgrade Outline
+## Upgrade Outline

 During upgrade, you create a copy of the data from your current Rancher container and a backup in case something goes wrong. Then you deploy the new version of Rancher in a new container using your existing data. Follow the steps to upgrade Rancher server:

@@ -59,7 +59,7 @@ During upgrade, you create a copy of the data from your current Rancher containe
 - [5. Verify the Upgrade](#5-verify-the-upgrade)
 - [6. Clean up your old Rancher server container](#6-clean-up-your-old-rancher-server-container)

-# 1. Create a copy of the data from your Rancher server container
+## 1. Create a copy of the data from your Rancher server container

 1. Using a remote Terminal connection, log into the node running your Rancher server.

@@ -75,7 +75,7 @@ During upgrade, you create a copy of the data from your current Rancher containe
    docker create --volumes-from <RANCHER_CONTAINER_NAME> --name rancher-data rancher/rancher:<RANCHER_CONTAINER_TAG>
    ```

-# 2. Create a backup tarball
+## 2. Create a backup tarball

 1. <a id="tarball"></a>From the data container that you just created (<code>rancher-data</code>), create a backup tarball (<code>rancher-data-backup-&lt;RANCHER_VERSION&gt;-&lt;DATE&gt;.tar.gz</code>).

@@ -97,7 +97,7 @@ During upgrade, you create a copy of the data from your current Rancher containe

 1. Move your backup tarball to a safe location external from your Rancher server.

-# 3. Pull the New Docker Image
+## 3. Pull the New Docker Image

 Pull the image of the Rancher version that you want to upgrade to.

@@ -109,7 +109,7 @@ Placeholder | Description
 docker pull rancher/rancher:<RANCHER_VERSION_TAG>
 ```

-# 4. Start the New Rancher Server Container
+## 4. Start the New Rancher Server Container

 Start a new Rancher server container using the data from the `rancher-data` container. Remember to pass in all the environment variables that you had used when you started the original container.

@@ -338,7 +338,7 @@ docker run -d --volumes-from rancher-data \

 **Result:** You have upgraded Rancher. Data from your upgraded server is now saved to the `rancher-data` container for use in future upgrades.

-# 5. Verify the Upgrade
+## 5. Verify the Upgrade

 Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window.

@@ -346,11 +346,10 @@ Log into Rancher. Confirm that the upgrade succeeded by checking the version dis
 >
 > See [Restoring Cluster Networking](../../install-upgrade-on-a-kubernetes-cluster/upgrades/namespace-migration.md#restoring-cluster-networking).

-
-# 6. Clean up Your Old Rancher Server Container
+## 6. Clean up Your Old Rancher Server Container

 Remove the previous Rancher server container. If you only stop the previous Rancher server container (and don't remove it), the container may restart after the next server reboot.

-# Rolling Back
+## Rolling Back

 If your upgrade does not complete successfully, you can roll back Rancher server and its data back to its last healthy state. For more information, see [Docker Rollback](./roll-back-docker-installed-rancher.md).
@@ -19,7 +19,7 @@ kubectl -n cattle-system create secret tls tls-rancher-ingress \

 > **Note:** If you want to replace the certificate, you can delete the `tls-rancher-ingress` secret using `kubectl -n cattle-system delete secret tls-rancher-ingress` and add a new one using the command shown above. If you are using a private CA signed certificate, replacing the certificate is only possible if the new certificate is signed by the same CA as the certificate currently in use.

-# Using a Private CA Signed Certificate
+## Using a Private CA Signed Certificate

 If you are using a private CA, Rancher requires a copy of the CA certificate which is used by the Rancher Agent to validate the connection to the server.

@@ -32,6 +32,6 @@ kubectl -n cattle-system create secret generic tls-ca \

 > **Note:** The configured `tls-ca` secret is retrieved when Rancher starts. On a running Rancher installation the updated CA will take effect after new Rancher pods are started.

-# Updating a Private CA Certificate
+## Updating a Private CA Certificate

 Follow the steps on [this page](update-rancher-certificate.md) to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation](../../../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md) or to switch from the default self-signed certificate to a custom certificate.
@@ -9,13 +9,13 @@ The [System Charts](https://github.com/rancher/system-charts) repository contain

 In an air gapped installation of Rancher, you will need to configure Rancher to use a local copy of the system charts. This section describes how to use local system charts using a CLI flag in Rancher v2.3.0, and using a Git mirror for Rancher versions before v2.3.0.

-# Using Local System Charts in Rancher v2.3.0
+## Using Local System Charts in Rancher v2.3.0

 In Rancher v2.3.0, a local copy of `system-charts` has been packaged into the `rancher/rancher` container. To be able to use these features in an air gap install, you will need to run the Rancher install command with an extra environment variable, `CATTLE_SYSTEM_CATALOG=bundled`, which tells Rancher to use the local copy of the charts instead of attempting to fetch them from GitHub.

 Example commands for a Rancher installation with a bundled `system-charts` are included in the [air gap Docker installation](../advanced-options/advanced-use-cases/air-gap-helm2/install-rancher.md) instructions and the [air gap Kubernetes installation](../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md) instructions.

-# Setting Up System Charts for Rancher Before v2.3.0
+## Setting Up System Charts for Rancher Before v2.3.0

 ### A. Prepare System Charts

@@ -2,7 +2,7 @@
 title: Updating the Rancher Certificate
 ---

-# Updating a Private CA Certificate
+## Updating a Private CA Certificate

 Follow these steps to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation](../../../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md) or to switch from the default self-signed certificate to a custom certificate.

@@ -15,7 +15,7 @@ A summary of the steps is as follows:

 The details of these instructions are below.

-## 1. Create/update the certificate secret resource
+### 1. Create/update the certificate secret resource

 First, concatenate the server certificate followed by any intermediate certificate(s) to a file named `tls.crt` and provide the corresponding certificate key in a file named `tls.key`.

@@ -36,7 +36,7 @@ $ kubectl -n cattle-system create secret tls tls-rancher-ingress \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-##  2. Create/update the CA certificate secret resource
+### 2. Create/update the CA certificate secret resource

 If the new certificate was signed by a private CA, you will need to copy the corresponding root CA certificate into a file named `cacerts.pem` and create or update the `tls-ca secret` in the `cattle-system` namespace. If the certificate was signed by an intermediate CA, then the `cacerts.pem` must contain both the intermediate and root CA certificates (in this order).

@@ -55,7 +55,7 @@ $ kubectl -n cattle-system create secret generic tls-ca \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-## 3. Reconfigure the Rancher deployment
+### 3. Reconfigure the Rancher deployment

 > Before proceeding, [generate an API token in the Rancher UI](../../../reference-guides/user-settings/api-keys.md#creating-an-api-key) (<b>User > API & Keys</b>).

@@ -90,18 +90,18 @@ helm upgrade rancher rancher-stable/rancher \

 When the upgrade is completed, navigate to `https://<Rancher_SERVER>/v3/settings/cacerts` to verify that the value matches the CA certificate written in the `tls-ca` secret earlier.

-## 4. Reconfigure Rancher agents to trust the private CA
+### 4. Reconfigure Rancher agents to trust the private CA

 This section covers three methods to reconfigure Rancher agents to trust the private CA. This step is required if either of the following is true:

 - Rancher was initially configured to use the Rancher self-signed certificate (`ingress.tls.source=rancher`) or with a Let's Encrypt issued certificate (`ingress.tls.source=letsEncrypt`)
 - The root CA certificate for the new custom certificate has changed

-### Why is this step required?
+#### Why is this step required?

 When Rancher is configured with a certificate signed by a private CA, the CA certificate chain is downloaded into Rancher agent containers. Agents compare the checksum of the downloaded certificate against the `CATTLE_CA_CHECKSUM` environment variable. This means that, when the private CA certificate is changed on Rancher server side, the environvment variable `CATTLE_CA_CHECKSUM` must be updated accordingly.

-### Which method should I choose?
+#### Which method should I choose?

 Method 1 is the easiest one but requires all clusters to be connected to Rancher after the certificates have been rotated. This is usually the case if the process is performed right after updating the Rancher deployment (Step 3).

@@ -109,7 +109,7 @@ If the clusters have lost connection to Rancher but you have [Authorized Cluster

 Method 3 can be used as a fallback if method 1 and 2 are unfeasible.

-### Method 1: Kubectl command
+#### Method 1: Kubectl command

 For each cluster under Rancher management (including `local`) run the following command using the Kubeconfig file of the Rancher management cluster (RKE or K3S).

@@ -120,7 +120,7 @@ kubectl patch clusters <REPLACE_WITH_CLUSTERID> -p '{"status":{"agentImage":"dum
 This command will cause all Agent Kubernetes resources to be reconfigured with the checksum of the new certificate.


-### Method 2: Manually update checksum
+#### Method 2: Manually update checksum

 Manually patch the agent Kubernetes resources by updating the `CATTLE_CA_CHECKSUM` environment variable to the value matching the checksum of the new CA certificate. Generate the new checksum value like so:

@@ -136,7 +136,7 @@ $ kubectl edit -n cattle-system ds/cattle-node-agent
 $ kubectl edit -n cattle-system deployment/cluster-agent
 ```

-### Method 3: Recreate Rancher agents
+#### Method 3: Recreate Rancher agents

 With this method you are recreating the Rancher agents by running a set of commands on a controlplane node of each downstream cluster.

@@ -145,11 +145,11 @@ First, generate the agent definitions as described here: https://gist.github.com
 Then, connect to a controlplane node of the downstream cluster via SSH, create a Kubeconfig and apply the definitions:
 https://gist.github.com/superseb/b14ed3b5535f621ad3d2aa6a4cd6443b

-# Updating from a Private CA Certificate to a Common Certificate
+## Updating from a Private CA Certificate to a Common Certificate

 >It is possible to perform the opposite procedure as shown above: you may change from a private certificate to a common, or non-private, certificate. The steps involved are outlined below.

-## 1. Create/update the certificate secret resource
+### 1. Create/update the certificate secret resource

 First, concatenate the server certificate followed by any intermediate certificate(s) to a file named `tls.crt` and provide the corresponding certificate key in a file named `tls.key`.

@@ -170,7 +170,7 @@ $ kubectl -n cattle-system create secret tls tls-rancher-ingress \
  --dry-run --save-config -o yaml | kubectl apply -f -
 ```

-## 2. Delete the CA certificate secret resource
+### 2. Delete the CA certificate secret resource

 You will delete the `tls-ca secret` in the `cattle-system` namespace as it is no longer needed. You may also optionally save a copy of the `tls-ca secret` if desired.

@@ -186,7 +186,7 @@ To delete the existing `tls-ca` secret:
 kubectl -n cattle-system delete secret tls-ca
 ```

-## 3. Reconfigure the Rancher deployment
+### 3. Reconfigure the Rancher deployment

 > Before proceeding, [generate an API token in the Rancher UI](https://rancher.com/docs/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) (<b>User > API & Keys</b>) and save the Bearer Token which you might need in step 4.

@@ -228,6 +228,6 @@ On upgrade, you can either
 set privateCA=false
 ```

-## 4. Reconfigure Rancher agents for the non-private/common certificate
+### 4. Reconfigure Rancher agents for the non-private/common certificate

 `CATTLE_CA_CHECKSUM` environment variable on the downstream cluster agents should be removed or set to "" (an empty string).
--- a/Show More
+++ b/Show More