Fix order of headings (#1465)

* Fix 'title out of sequence' errors fixed Dockershim.md * fixed deprecated-features.md * fixed install-and-configure-kubectl.md * fixed rancher-is-no-longer-needed.md * fixed security.md * fixed technical-items.md + spacing, duplicate section, admonitions * fixed telemetry.md * fixed upgrades.md * fixed upgrade-kubernetes-without-upgrading-rancher.md * fixed air-gapped-upgrades.md * fixed dockershim.md * fixed docker-install-commands.md * fixed install-kubernetes.md * fixed infrastructure-private-registry.md * fixed install-rancher-ha * fixed manage-namespaces and tune-etcd-for-large-installs.md * fixed cis-scans/configuration-reference.md * fixed custom-benchmark.md * fixed supportconfig.md * fixed harvester/overview.md * fixed logging-architecture.md * fixed logging-helm-chart-options.md + rm'd unnecessary annotation title * fixed taints-and-tolerances.md * fixed longhorn/overview.md * fixed neuvector/overview.md * fixed monitoring-and-alerting * fixed rancher-cli.md * fixed cluster-configuration.md * fixed monitoring-v2-configuration/examples.md * fixed servicemonitors-and-podmonitors.md * fixed other-troubleshooting-tips/dns.md
2026-09-25 12:38:05 +00:00 · 2024-09-18 14:02:55 -04:00
parent 9f8e1a7a53
commit c869ea69ac
191 changed files with 1059 additions and 1135 deletions
@@ -8,11 +8,11 @@ title: Installing and Configuring kubectl

 `kubectl` is a CLI utility for running commands against Kubernetes clusters. It's required for many maintenance and administrative tasks in Rancher 2.x.

-### Installation
+## Installation

 See [kubectl Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation on your operating system.

-### Configuration
+## Configuration

 When you create a Kubernetes cluster with RKE, RKE creates a `kube_config_rancher-cluster.yml` in the local directory that contains credentials to connect to your new cluster with tools like `kubectl` or `helm`.

@@ -9,11 +9,11 @@ title: Rancher is No Longer Needed
 This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted.


-### If the Rancher server is deleted, what happens to the workloads in my downstream clusters?
+## If the Rancher server is deleted, what happens to the workloads in my downstream clusters?

 If Rancher is ever deleted or unrecoverable, all workloads in the downstream Kubernetes clusters managed by Rancher will continue to function as normal.

-### If the Rancher server is deleted, how do I access my downstream clusters?
+## If the Rancher server is deleted, how do I access my downstream clusters?

 The capability to access a downstream cluster without Rancher depends on the type of cluster and the way that the cluster was created. To summarize:

@@ -21,7 +21,7 @@ The capability to access a downstream cluster without Rancher depends on the typ
 - **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
 - **RKE clusters:** To access an [RKE cluster,](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) the cluster must have the [authorized cluster endpoint](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.](../how-to-guides/advanced-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.

-### What if I don't want Rancher anymore?
+## What if I don't want Rancher anymore?

 If you [installed Rancher on a Kubernetes cluster,](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) remove Rancher by using the [System Tools](../reference-guides/system-tools.md) with the `remove` subcommand.

@@ -29,7 +29,7 @@ If you installed Rancher with Docker, you can uninstall Rancher by removing the

 Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)

-### What if I don't want my imported cluster managed by Rancher?
+## What if I don't want my imported cluster managed by Rancher?

 If an imported cluster is deleted from the Rancher UI, the cluster is detached from Rancher, leaving it intact and accessible by the same methods that were used to access it before it was imported into Rancher.

@@ -41,7 +41,7 @@ To detach the cluster,

 **Result:** The imported cluster is detached from Rancher and functions normally outside of Rancher.

-### What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?
+## What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?

 At this time, there is no functionality to detach these clusters from Rancher. In this context, "detach" is defined as the ability to remove Rancher components from the cluster and manage access to the cluster independently of Rancher.

@@ -6,12 +6,10 @@ title: Security FAQ
  <link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/security"/>
 </head>

-**Is there a Hardening Guide?**
+## Is there a Hardening Guide?

 The Hardening Guide is now located in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.

-<br/>
-
-**What are the results of Rancher's Kubernetes cluster when it is CIS benchmarked?**
+## What are the results of Rancher's Kubernetes cluster when it is CIS benchmarked?

 We have run the CIS Kubernetes benchmark against a hardened Rancher Kubernetes cluster.  The results of that assessment can be found in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
@@ -6,9 +6,10 @@ title: Technical FAQ
  <link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/technical-items"/>
 </head>

-### How can I reset the administrator password?
+## How can I reset the administrator password?
+
+Docker install:

-Docker Install:
 ```
 $ docker exec -ti <container_id> reset-password
 New password for default administrator (user-xxxxx):
@@ -16,6 +17,7 @@ New password for default administrator (user-xxxxx):
 ```

 Kubernetes install (Helm):
+
 ```
 $ KUBECONFIG=./kube_config_rancher-cluster.yml
 $ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password
@@ -23,11 +25,14 @@ New password for default administrator (user-xxxxx):
 <new_password>
 ```

-> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8**
->
->If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to start using the Helm chart.
+### RKE add-on install (only supported up to Rancher v2.0.8)
+
+:::tip
+> If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to start using the Helm chart.
+:::

 Kubernetes install (RKE add-on):
+
 ```
 $ KUBECONFIG=./kube_config_rancher-cluster.yml
 $ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- reset-password
@@ -35,8 +40,10 @@ New password for default administrator (user-xxxxx):
 <new_password>
 ```

-### I deleted/deactivated the last admin, how can I fix it?
-Docker Install:
+## I deleted/deactivated the last admin, how can I fix it?
+
+Docker install:
+
 ```
 $ docker exec -ti <container_id> ensure-default-admin
 New default administrator (user-xxxxx)
@@ -52,44 +59,32 @@ New password for default administrator (user-xxxxx):
 <new_password>
 ```

-> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8**
->
->If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to start using the Helm chart.
-
-Kubernetes install (RKE add-on):
-```
-$ KUBECONFIG=./kube_config_rancher-cluster.yml
-$ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- ensure-default-admin
-New password for default admin user (user-xxxxx):
-<new_password>
-```
-
-### How can I enable debug logging?
+## How can I enable debug logging?

 See [Troubleshooting: Logging](../troubleshooting/other-troubleshooting-tips/logging.md)

-### My ClusterIP does not respond to ping
+## My ClusterIP does not respond to ping

 ClusterIP is a virtual IP, which will not respond to ping. Best way to test if the ClusterIP is configured correctly, is by using `curl` to access the IP and port to see if it responds.

-### Where can I manage Node Templates?
+## Where can I manage Node Templates?

 Node Templates can be accessed by opening your account menu (top right) and selecting `Node Templates`.

-### Why is my Layer-4 Load Balancer in `Pending` state?
+## Why is my Layer-4 Load Balancer in `Pending` state?

 The Layer-4 Load Balancer is created as `type: LoadBalancer`. In Kubernetes, this needs a cloud provider or controller that can satisfy these requests, otherwise these will be in `Pending` state forever. More information can be found on [Cloud Providers](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/set-up-cloud-providers/set-up-cloud-providers.md) or [Create External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)

-### Where is the state of Rancher stored?
+## Where is the state of Rancher stored?

 - Docker Install: in the embedded etcd of the `rancher/rancher` container, located at `/var/lib/rancher`.
 - Kubernetes install: in the etcd of the RKE cluster created to run Rancher.

-### How are the supported Docker versions determined?
+## How are the supported Docker versions determined?

 We follow the validated Docker versions for upstream Kubernetes releases. The validated versions can be found under [External Dependencies](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#external-dependencies) in the Kubernetes release CHANGELOG.md.

-### How can I access nodes created by Rancher?
+## How can I access nodes created by Rancher?

 SSH keys to access the nodes created by Rancher can be downloaded via the **Nodes** view. Choose the node which you want to access and click on the vertical &#8942; button at the end of the row, and choose **Download Keys** as shown in the picture below.

@@ -101,14 +96,14 @@ Unzip the downloaded zip file, and use the file `id_rsa` to connect to you host.
 $ ssh -i id_rsa user@ip_of_node
 ```

-### How can I automate task X in Rancher?
+## How can I automate task X in Rancher?

 The UI consists of static files, and works based on responses of the API. That means every action/task that you can execute in the UI, can be automated via the API. There are 2 ways to do this:

 * Visit `https://your_rancher_ip/v3` and browse the API options.
 * Capture the API calls when using the UI (Most commonly used for this is [Chrome Developer Tools](https://developers.google.com/web/tools/chrome-devtools/#network) but you can use anything you like)

-### The IP address of a node changed, how can I recover?
+## The IP address of a node changed, how can I recover?

 A node is required to have a static IP configured (or a reserved IP via DHCP). If the IP of a node has changed, you will have to remove it from the cluster and readd it. After it is removed, Rancher will update the cluster to the correct state. If the cluster is no longer in `Provisioning` state, the node is removed from the cluster.

@@ -116,15 +111,17 @@ When the IP address of the node changed, Rancher lost connection to the node, so

 When the node is removed from the cluster, and the node is cleaned, you can readd the node to the cluster.

-### How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?
+## How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?

 You can add more arguments/binds/environment variables via the [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#cluster-config-file) option in Cluster Options. For more information, see the [Extra Args, Extra Binds, and Extra Environment Variables](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/) in the RKE documentation or browse the [Example Cluster.ymls](https://rancher.com/docs/rke/latest/en/example-yamls/).

-### How do I check if my certificate chain is valid?
+## How do I check if my certificate chain is valid?

 Use the `openssl verify` command to validate your certificate chain:

->**Note:** Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS installed certificates are not used when verifying manually.
+:::tip
+Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS installed certificates are not used when verifying manually.
+:::

 ```
 SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem rancher.yourdomain.com.pem
@@ -157,7 +154,7 @@ subject= /C=GB/ST=England/O=Alice Ltd/CN=rancher.yourdomain.com
 issuer= /C=GB/ST=England/O=Alice Ltd/CN=Alice Intermediate CA
 ```

-### How do I check `Common Name` and `Subject Alternative Names` in my server certificate?
+## How do I check `Common Name` and `Subject Alternative Names` in my server certificate?

 Although technically an entry in `Subject Alternative Names` is required, having the hostname in both `Common Name` and as entry in `Subject Alternative Names` gives you maximum compatibility with older browser/applications.

@@ -175,7 +172,7 @@ openssl x509 -noout -in cert.pem -text | grep DNS
                DNS:rancher.my.org
 ```

-### Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?
+## Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?

 This is due to a combination of the following default Kubernetes settings:

@@ -194,6 +191,6 @@ In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. Se
  * `default-not-ready-toleration-seconds`: Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.
  * `default-unreachable-toleration-seconds`: Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.

-### Can I use keyboard shortcuts in the UI?
+## Can I use keyboard shortcuts in the UI?

 Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
@@ -6,11 +6,11 @@ title: Telemetry FAQ
  <link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/telemetry"/>
 </head>

-### What is Telemetry?
+## What is Telemetry?

 Telemetry collects aggregate information about the size of Rancher installations, versions of components used, and which features are used.  This information is used by Rancher Labs to help make the product better and is not shared with third-parties.

-### What information is collected?
+## What information is collected?

 No specific identifying information like usernames, passwords, or the names or addresses of user resources will ever be collected.

@@ -24,12 +24,12 @@ The primary things collected include:
  - The image name & version of Rancher that is running.
  - A unique randomly-generated identifier for this installation.

-### Can I see the information that is being sent?
+## Can I see the information that is being sent?

 If Telemetry is enabled, you can go to `https://<your rancher server>/v1-telemetry` in your installation to see the current data.

 If Telemetry is not enabled, the process that collects the data is not running, so there is nothing being collected to look at.

-### How do I turn it on or off?
+## How do I turn it on or off?

 After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
@@ -6,7 +6,7 @@ When running larger Rancher installations with 15 or more clusters it is recomme

 The etcd data set is automatically cleaned up on a five minute interval by Kubernetes. There are situations, e.g. deployment thrashing, where enough events could be written to etcd and deleted before garbage collection occurs and cleans things up causing the keyspace to fill up. If you see `mvcc: database space exceeded` errors, in the etcd logs or Kubernetes API server logs, you should consider increasing the keyspace size. This can be accomplished by setting the [quota-backend-bytes](https://etcd.io/docs/v3.3/op-guide/maintenance/#space-quota) setting on the etcd servers.

-### Example: This snippet of the RKE cluster.yml file increases the keyspace size to 5GB
+## Example: This Snippet of the RKE Cluster.yml file Increases the Keyspace Size to 5GB

 ```yaml
 # RKE cluster.yml
@@ -17,7 +17,7 @@ services:
      quota-backend-bytes: 5368709120
 ```

-## Scaling etcd disk performance
+## Scaling etcd Disk Performance

 You can follow the recommendations from [the etcd docs](https://etcd.io/docs/v3.3/tuning/#disk) on how to tune the disk priority on the host.

@@ -108,7 +108,6 @@ You'll use the backup as a restoration point if something goes wrong during upgr

    > **Note:** If you want to switch to a different Helm chart repository, please follow the [steps on how to switch repositories](../../resources/choose-a-rancher-version.md#switching-to-a-different-helm-chart-repository). If you switch repositories, make sure to list the repositories again before continuing onto Step 3 to ensure you have the correct one added.

-
 1. Fetch the latest chart to install Rancher from the Helm chart repository.

    This command will pull down the latest charts and save it in the current directory as a `.tgz` file.
@@ -188,7 +187,6 @@ If you are currently running the cert-manager whose version is older than v0.11,
    --set hostname=rancher.my.org
    ```

-
 </TabItem>
 <TabItem value="Kubernetes Air Gap Upgrade">

@@ -203,7 +201,6 @@ Placeholder | Description
 `<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.
 `<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.

-
 ### Option A: Default Self-signed Certificate

 ```plain
@@ -23,7 +23,7 @@ We recommend setting up the following infrastructure for a high-availability ins
 - **A DNS record** to map a URL to the load balancer. This will become the Rancher server URL, and downstream Kubernetes clusters will need to reach it.
 - **A private Docker registry** to distribute Docker images to your machines.

-### 1. Set up Linux Nodes
+## 1. Set up Linux Nodes

 These hosts will be disconnected from the internet, but require being able to connect with your private registry.

@@ -31,7 +31,7 @@ Make sure that your nodes fulfill the general installation requirements for [OS,

 For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.

-### 2. Set up External Datastore
+## 2. Set up External Datastore

 The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to Kubernetes operators. The available options allow you to select a datastore that best fits your use case.

@@ -47,7 +47,7 @@ For an example of one way to set up the database, refer to this [tutorial](../..

 For the complete list of options that are available for configuring a K3s cluster datastore, refer to the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/datastore/)

-### 3. Set up the Load Balancer
+## 3. Set up the Load Balancer

 You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server.

@@ -67,7 +67,7 @@ For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to
 > **Important:**
 > Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.

-### 4. Set up the DNS Record
+## 4. Set up the DNS Record

 Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer.

@@ -77,7 +77,7 @@ You will need to specify this hostname in a later step when you install Rancher,

 For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)

-### 5. Set up a Private Docker Registry
+## 5. Set up a Private Docker Registry

 Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines.

@@ -97,13 +97,13 @@ To install the Rancher management server on a high-availability RKE cluster, we

 These nodes must be in the same region/data center. You may place these servers in separate availability zones.

-### Why three nodes?
+## Why Three Nodes?

 In an RKE cluster, Rancher server data is stored on etcd. This etcd database runs on all three nodes.

 The etcd database requires an odd number of nodes so that it can always elect a leader with a majority of the etcd cluster. If the etcd database cannot elect a leader, etcd can suffer from [split brain](https://www.quora.com/What-is-split-brain-in-distributed-systems), requiring the cluster to be restored from backup. If one of the three etcd nodes fails, the two remaining nodes can elect a leader because they have the majority of the total number of etcd nodes.

-### 1. Set up Linux Nodes
+## 1. Set up Linux Nodes

 These hosts will be disconnected from the internet, but require being able to connect with your private registry.

@@ -111,7 +111,7 @@ Make sure that your nodes fulfill the general installation requirements for [OS,

 For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.

-### 2. Set up the Load Balancer
+## 2. Set up the Load Balancer

 You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server.

@@ -131,7 +131,7 @@ For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to
 > **Important:**
 > Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.

-### 3. Set up the DNS Record
+## 3. Set up the DNS Record

 Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer.

@@ -141,7 +141,7 @@ You will need to specify this hostname in a later step when you install Rancher,

 For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)

-### 4. Set up a Private Docker Registry
+## 4. Set up a Private Docker Registry

 Rancher supports air gap installs using a secure Docker private registry. You must have your own private registry or other means of distributing Docker images to your machines.

@@ -157,7 +157,7 @@ If you need help with creating a private registry, please refer to the [official
 >
 > For Rancher v2.0-v2.4, there is no migration path from a Docker installation to a high-availability installation. Therefore, you may want to use a Kubernetes installation from the start.

-### 1. Set up a Linux Node
+## 1. Set up a Linux Node

 This host will be disconnected from the Internet, but needs to be able to connect to your private registry.

@@ -165,7 +165,7 @@ Make sure that your node fulfills the general installation requirements for [OS,

 For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.

-### 2. Set up a Private Docker Registry
+## 2. Set up a Private Docker Registry

 Rancher supports air gap installs using a Docker private registry on your bastion server. You must have your own private registry or other means of distributing Docker images to your machines.

@@ -174,4 +174,4 @@ If you need help with creating a private registry, please refer to the [official
 </TabItem>
 </Tabs>

-### [Next: Collect and Publish Images to your Private Registry](publish-images.md)
+## [Next: Collect and Publish Images to your Private Registry](publish-images.md)
@@ -21,14 +21,15 @@ The steps to set up an air-gapped Kubernetes cluster on RKE or K3s are shown bel

 In this guide, we are assuming you have created your nodes in your air gapped environment and have a secure Docker private registry on your bastion server.

-### Installation Outline
+## Installation Outline

 1. [Prepare Images Directory](#1-prepare-images-directory)
 2. [Create Registry YAML](#2-create-registry-yaml)
 3. [Install K3s](#3-install-k3s)
 4. [Save and Start Using the kubeconfig File](#4-save-and-start-using-the-kubeconfig-file)

-### 1. Prepare Images Directory
+## 1. Prepare Images Directory
+
 Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running.

 Place the tar file in the `images` directory before starting K3s on each node, for example:
@@ -38,7 +39,8 @@ sudo mkdir -p /var/lib/rancher/k3s/agent/images/
 sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/
 ```

-### 2. Create Registry YAML
+## 2. Create Registry YAML
+
 Create the registries.yaml file at `/etc/rancher/k3s/registries.yaml`. This will tell K3s the necessary details to connect to your private registry.

 The registries.yaml file should look like this before plugging in the necessary information:
@@ -64,7 +66,7 @@ Note, at this time only secure registries are supported with K3s (SSL with custo

 For more information on private registries configuration file for K3s, refer to the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/private-registry/)

-### 3. Install K3s
+## 3. Install K3s

 Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)

@@ -93,7 +95,7 @@ The node-token is on the server at `/var/lib/rancher/k3s/server/node-token`

 >**Note:** K3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks.

-### 4. Save and Start Using the kubeconfig File
+## 4. Save and Start Using the kubeconfig File

 When you installed K3s on each Rancher server node, a `kubeconfig` file was created on the node at `/etc/rancher/k3s/k3s.yaml`. This file contains credentials for full access to the cluster, and you should save this file in a secure location.

@@ -133,7 +135,7 @@ kubectl --kubeconfig ~/.kube/config/k3s.yaml get pods --all-namespaces

 For more information about the `kubeconfig` file, refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/cluster-access/) or the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) about organizing cluster access using `kubeconfig` files.

-### Note on Upgrading
+## Note on Upgrading

 Upgrading an air-gap environment can be accomplished in the following manner:

@@ -146,11 +148,11 @@ Upgrading an air-gap environment can be accomplished in the following manner:

 We will create a Kubernetes cluster using Rancher Kubernetes Engine (RKE). Before being able to start your Kubernetes cluster, you’ll need to install RKE and create a RKE config file.

-### 1. Install RKE
+## 1. Install RKE

 Install RKE by following the instructions in the [RKE documentation.](https://rancher.com/docs/rke/latest/en/installation/)

-### 2. Create an RKE Config File
+## 2. Create an RKE Config File

 From a system that can access ports 22/TCP and 6443/TCP on the Linux host node(s) that you set up in a previous step, use the sample below to create a new file named `rancher-cluster.yml`.

@@ -197,7 +199,7 @@ private_registries:
    is_default: true
 ```

-### 3. Run RKE
+## 3. Run RKE

 After configuring `rancher-cluster.yml`, bring up your Kubernetes cluster:

@@ -205,7 +207,7 @@ After configuring `rancher-cluster.yml`, bring up your Kubernetes cluster:
 rke up --config ./rancher-cluster.yml
 ```

-### 4. Save Your Files
+## 4. Save Your Files

 > **Important**
 > The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster.
@@ -221,8 +223,8 @@ Save a copy of the following files in a secure location:

 > **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file.

-### Issues or errors?
+## Issues or Errors?

 See the [Troubleshooting](../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md) page.

-### [Next: Install Rancher](install-rancher-ha.md)
+## [Next: Install Rancher](install-rancher-ha.md)
@@ -182,7 +182,7 @@ Use `kubectl` to create namespaces and apply the rendered manifests.

 If you choose to use self-signed certificates in [2. Choose your SSL Configuration](#2-choose-your-ssl-configuration), install cert-manager.

-### For Self-Signed Certificate Installs, Install Cert-manager
+### For Self-Signed Certificate Installs, Install cert-manager

 <details id="install-cert-manager">
  <summary>Click to expand</summary>
@@ -219,7 +219,7 @@ kubectl -n cattle-system apply -R -f ./rancher

 > **Note:** If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login. Leaving this active in an air-gapped environment can cause issues if the sockets cannot be opened successfully.

-## 5. For Rancher versions before v2.3.0, Configure System Charts
+## 5. For Rancher Versions Before v2.3.0, Configure System Charts

 If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in GitHub, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../resources/local-system-charts.md).

@@ -26,7 +26,7 @@ To manage permissions in a vanilla Kubernetes cluster, cluster admins configure
 > **Note:** If you create a namespace with `kubectl`, it may be unusable because `kubectl` doesn't require your new namespace to be scoped within a project that you have access to. If your permissions are restricted to the project level, it is better to [create a namespace through Rancher](manage-namespaces.md) to ensure that you will have permission to access the namespace.


-### Creating Namespaces
+## Creating Namespaces

 Create a new namespace to isolate apps and resources in a project.

@@ -44,7 +44,7 @@ Create a new namespace to isolate apps and resources in a project.

 **Result:** Your namespace is added to the project. You can begin assigning cluster resources to the namespace.

-### Moving Namespaces to Another Project
+## Moving Namespaces to Another Project

 Cluster admins and members may occasionally need to move a namespace to another project, such as when you want a different team to start using the application.

@@ -64,7 +64,7 @@ Cluster admins and members may occasionally need to move a namespace to another

 **Result:** Your namespace is moved to a different project (or is unattached from all projects). If any project resources are attached to the namespace, the namespace releases them and then attached resources from the new project.

-### Editing Namespace Resource Quotas
+## Editing Namespace Resource Quotas

 You can always override the namespace default limit to provide a specific namespace with access to more (or less) project resources.

@@ -9,7 +9,7 @@ description: Interact with Rancher using command line interface (CLI) tools from

 The Rancher CLI (Command Line Interface) is a unified tool that you can use to interact with Rancher. With this tool, you can operate Rancher using a command line rather than the GUI.

-### Download Rancher CLI
+## Download Rancher CLI

 The binary can be downloaded directly from the UI. The link can be found in the right hand side of the footer in the UI. We have binaries for Windows, Mac, and Linux. You can also check the [releases page for our CLI](https://github.com/rancher/cli/releases) for direct downloads of the binary.

@@ -17,14 +17,14 @@ The binary can be downloaded directly from the UI. The link can be found in the
 1. At the bottom, click **v2.6.x**, where **v2.6.x** is a hyperlinked text indicating the installed Rancher version.
 1. Under the **CLI Downloads section**, there are links to download the binaries for Windows, Mac, and Linux. You can also check the [releases page for our CLI](https://github.com/rancher/cli/releases) for direct downloads of the binary.

-### Requirements
+## Requirements

 After you download the Rancher CLI, you need to make a few configurations. Rancher CLI requires:

 - Your Rancher Server URL, which is used to connect to Rancher Server.
 - An API Bearer Token, which is used to authenticate with Rancher. For more information about obtaining a Bearer Token, see [Creating an API Key](../user-settings/api-keys.md).

-### CLI Authentication
+## CLI Authentication

 Before you can use Rancher CLI to control your Rancher Server, you must authenticate using an API Bearer Token. Log in using the following command (replace `<BEARER_TOKEN>` and `<SERVER_URL>` with your information):

@@ -34,7 +34,7 @@ $ ./rancher login https://<SERVER_URL> --token <BEARER_TOKEN>

 If Rancher Server uses a self-signed certificate, Rancher CLI prompts you to continue with the connection.

-### Project Selection
+## Project Selection

 Before you can perform any commands, you must select a Rancher project to perform those commands against. To select a [project](../../how-to-guides/advanced-user-guides/manage-clusters/projects-and-namespaces.md) to work on, use the command `./rancher context switch`. When you enter this command, a list of available projects displays. Enter a number to choose your project.

@@ -58,7 +58,7 @@ INFO[0005] Saving config to /Users/markbishop/.ranchcli2.json

 Ensure you can run `rancher kubectl get pods` successfully.

-### Commands
+## Commands

 The following commands are available for use in Rancher CLI.

@@ -79,13 +79,12 @@ The following commands are available for use in Rancher CLI.
 | `ssh`  | Connects to one of your cluster nodes using the SSH protocol.  |
 | `help, [h]`  | Shows a list of commands or help for one command.  |

-
-### Rancher CLI Help
+## Rancher CLI Help

 Once logged into Rancher Server using the CLI, enter `./rancher --help` for a list of commands.

 All commands accept the `--help` flag, which documents each command's usage.

-### Limitations
+## Limitations

 The Rancher CLI **cannot** be used to install [dashboard apps or Rancher feature charts](../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md).
@@ -15,7 +15,7 @@ For information on editing cluster membership, go to [this page.](../../how-to-g
 - [Editing Clusters with YAML](#editing-clusters-with-yaml)
 - [Updating ingress-nginx](#updating-ingress-nginx)

-### Cluster Management Capabilities by Cluster Type
+## Cluster Management Capabilities by Cluster Type

 The options and settings available for an existing cluster change based on the method that you used to provision it. For example, only clusters [provisioned by RKE](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) have **Cluster Options** available for editing.

@@ -25,7 +25,7 @@ import ClusterCapabilitiesTable from '../../shared-files/_cluster-capabilities-t

 <ClusterCapabilitiesTable />

-### Editing Clusters in the Rancher UI
+## Editing Clusters in the Rancher UI

 To edit your cluster, open the **Global** view, make sure the **Clusters** tab is selected, and then select **&#8942; > Edit** for the cluster that you want to edit.

@@ -46,7 +46,7 @@ Option | Description |
 Default Pod Security Policy | If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster. |
 Cloud Provider | If you're using a cloud provider to host cluster nodes launched by RKE, enable [this option](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/set-up-cloud-providers/set-up-cloud-providers.md) so that you can use the cloud provider's native features. If you want to store persistent data for your cloud-hosted cluster, this option is required.  |

-### Editing Clusters with YAML
+## Editing Clusters with YAML

 Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.

@@ -63,10 +63,7 @@ In Rancher v2.0.0-v2.2.x, the config file is identical to the  [cluster config f

 >**Note:** In Rancher v2.0.5 and v2.0.6, the names of services in the Config File (YAML) should contain underscores only: `kube_api` and `kube_controller`.

-
-
-
-### Updating ingress-nginx
+## Updating ingress-nginx

 Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.

@@ -12,7 +12,7 @@ Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG

 Before running the DNS checks, check the [default DNS provider](../../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#default-dns-provider) for your cluster and make sure that [the overlay network is functioning correctly](networking.md#check-if-overlay-network-is-functioning-correctly) as this can also be the reason why DNS resolution (partly) fails.

-### Check if DNS pods are running
+## Check if DNS pods are running

 ```
 kubectl -n kube-system get pods -l k8s-app=kube-dns
@@ -30,7 +30,7 @@ NAME                        READY   STATUS    RESTARTS   AGE
 kube-dns-5fd74c7488-h6f7n   3/3     Running   0          4m13s
 ```

-### Check if the DNS service is present with the correct cluster-ip
+## Check if the DNS service is present with the correct cluster-ip

 ```
 kubectl -n kube-system get svc -l k8s-app=kube-dns
@@ -41,7 +41,7 @@ NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
 service/kube-dns   ClusterIP   10.43.0.10   <none>        53/UDP,53/TCP   4m13s
 ```

-### Check if domain names are resolving
+## Check if domain names are resolving

 Check if internal cluster names are resolving (in this example, `kubernetes.default`), the IP shown after `Server:` should be the same as the `CLUSTER-IP` from the `kube-dns` service.

@@ -132,15 +132,15 @@ command terminated with exit code 1

 Cleanup the alpine DaemonSet by running `kubectl delete ds/dnstest`.

-### CoreDNS specific
+## CoreDNS specific

-#### Check CoreDNS logging
+### Check CoreDNS logging

 ```
 kubectl -n kube-system logs -l k8s-app=kube-dns
 ```

-#### Check configuration
+### Check configuration

 CoreDNS configuration is stored in the configmap `coredns` in the `kube-system` namespace.

@@ -148,7 +148,7 @@ CoreDNS configuration is stored in the configmap `coredns` in the `kube-system`
 kubectl -n kube-system get configmap coredns -o go-template={{.data.Corefile}}
 ```

-#### Check upstream nameservers in resolv.conf
+### Check upstream nameservers in resolv.conf

 By default, the configured nameservers on the host (in `/etc/resolv.conf`) will be used as upstream nameservers for CoreDNS. You can check this file on the host or run the following Pod with `dnsPolicy` set to `Default`, which will inherit the `/etc/resolv.conf` from the host it is running on.

@@ -156,7 +156,7 @@ By default, the configured nameservers on the host (in `/etc/resolv.conf`) will
 kubectl run -i --restart=Never --rm test-${RANDOM} --image=ubuntu --overrides='{"kind":"Pod", "apiVersion":"v1", "spec": {"dnsPolicy":"Default"}}' -- sh -c 'cat /etc/resolv.conf'
 ```

-#### Enable query logging
+### Enable query logging

 Enabling query logging can be done by enabling the [log plugin](https://coredns.io/plugins/log/) in the Corefile configuration in the configmap `coredns`. You can do so by using `kubectl -n kube-system edit configmap coredns` or use the command below to replace the configuration in place:

@@ -166,9 +166,9 @@ kubectl get configmap -n kube-system coredns -o json | sed -e 's_loadbalance_log

 All queries will now be logged and can be checked using the command in [Check CoreDNS logging](#check-coredns-logging).

-### kube-dns specific
+## kube-dns specific

-#### Check upstream nameservers in kubedns container
+### Check upstream nameservers in kubedns container

 By default, the configured nameservers on the host (in `/etc/resolv.conf`) will be used as upstream nameservers for kube-dns. Sometimes the host will run a local caching DNS nameserver, which means the address in `/etc/resolv.conf` will point to an address in the loopback range (`127.0.0.0/8`) which will be unreachable by the container. In case of Ubuntu 18.04, this is done by `systemd-resolved`. Since Rancher v2.0.7, we detect if `systemd-resolved` is running, and will automatically use the `/etc/resolv.conf` file with the correct upstream nameservers (which is located at `/run/systemd/resolve/resolv.conf`).