mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-05-15 09:33:30 +00:00
swapping use cases per Denise and Mohamed
This commit is contained in:
Mark Bishop
@@ -7,10 +7,10 @@ When you deploy Rancher and use it to provision clusters, Rancher installs its c
|
||||
|
||||
There are two contexts in which you'd remove Rancher from a Kubernetes cluster node.
|
||||
|
||||
- [Removing Rancher from Your Rancher Cluster Nodes]({{< baseurl >}}/rancher/v2.x/en/admin-settings/removing-rancher/rancher-cluster-nodes/)
|
||||
- [Removing Rancher from Rancher-Provisioned Nodes]({{< baseurl >}}/rancher/v2.x/en/admin-settings/removing-rancher/user-cluster-nodes/)
|
||||
|
||||
In this context, you are removing Rancher from Kubernetes clusters that you provisioned using Rancher.
|
||||
|
||||
- [Removing Rancher from Your Rancher Server Nodes]({{< baseurl >}}/rancher/v2.x/en/admin-settings/removing-rancher/rancher-cluster-nodes/)
|
||||
|
||||
In this context, you are removing Rancher from the Kubernetes cluster that your configured for your Rancher installation.
|
||||
|
||||
- [Removing Rancher from User Cluster Nodes]({{< baseurl >}}/rancher/v2.x/en/admin-settings/removing-rancher/user-cluster-nodes/)
|
||||
|
||||
In this context, you are removing Rancher from Kubernetes clusters that you provisioned using Rancher.
|
||||
+100
-155
@@ -1,192 +1,137 @@
|
||||
---
|
||||
title: Removing Rancher from Your Rancher Cluster Nodes
|
||||
weight: 375
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/removing-rancher/cleaning-cluster-nodes/
|
||||
- /rancher/v2.x/en/installation/removing-rancher/
|
||||
- /rancher/v2.x/en/faq/cleaning-cluster-nodes/
|
||||
title: Removing Rancher from Rancher Server Nodes
|
||||
weight: 2000
|
||||
---
|
||||
When you deploy Rancher to the Kubernetes nodes that host your [Rancher installation]({{< baseurl >}}/rancher/v2.x/en/installation/), resources (containers/virtual network interfaces) and configuration items (certificates/configuration files) are created.
|
||||
|
||||
When removing nodes from your installation cluster (provided that they are in `Active` state), those resources automatically cleaned, and the only action needed is to restart the node. When a node has become unreachable and the automatic cleanup process cannot be used, we describe the steps that need to be executed before the node can be added to a cluster again.
|
||||
When you no longer have use for Rancher in your [installation cluster]({{< baseurl >}}/rancher/v2.x/en/installation/ha/), and you want to remove Rancher from its nodes, follow one of the sets of instructions below based on your [cluster type]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-options). The method you'll use to remove Rancher changes based on the type of cluster.
|
||||
|
||||
## Removing A Node from a Cluster by Rancher UI
|
||||
|
||||
When the node is in `Active` state, removing the node from a cluster will trigger a process to clean up the node. Please restart the node after the automatic cleanup process is done to make sure any non-persistent data is properly removed.
|
||||
## Nodes Launched by RKE / Nodes Hosted by a Provider
|
||||
|
||||
**To restart a node:**
|
||||
For clusters nodes provisioned using [RKE](({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/)) or a [hosted Kubernetes provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#hosted-kubernetes-cluster), you can remove Rancher by downloading and running the Rancher system-tools.
|
||||
|
||||
### Download and Configuration
|
||||
|
||||
You can download the latest version of Rancher system-tools from its GitHub [releases page](https://github.com/rancher/system-tools/releases). Download the version of system-tools for the OS that you're removing Rancher from.
|
||||
|
||||
Operating System | File
|
||||
-----------------|-----
|
||||
MacOS | `system-tools_darwin-amd64`
|
||||
Linux | `system-tools_linux-amd64`
|
||||
Windows | `system-tools_windows-amd64.exe`
|
||||
|
||||
<br>
|
||||
|
||||
After you download the tools, move it onto the node you're removing Rancher from, and then complete the following actions:
|
||||
|
||||
1. Rename the file to `system-tools`.
|
||||
|
||||
1. Give the file executable permissions by running the following command:
|
||||
|
||||
```
|
||||
chmod +x system-tools
|
||||
```
|
||||
|
||||
### Using the System-Tool
|
||||
|
||||
System-tools is a utility that cleans up Rancher. In this use case, it will help you remove the Rancher from your cluster nodes.
|
||||
|
||||
#### Usage
|
||||
|
||||
After you move the `system-tools` file to your Rancher node, you can run it by changing to the file's directory and running the following command.
|
||||
|
||||
>**Warning:** This command will remove data from your nodes. Make sure you have created a backup of files you want to keep before executing the command, as data will be lost.
|
||||
|
||||
```
|
||||
# using reboot
|
||||
reboot
|
||||
|
||||
# using shutdown
|
||||
shutdown -r now
|
||||
./system-tools remove --kubeconfig <$KUBECONFIG> --namespace <NAMESPACE>
|
||||
```
|
||||
|
||||
## Cleaning a Node Manually
|
||||
<br/>
|
||||
When you run this command, the components listed in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
|
||||
When a node is unreachable and removed from the cluster, the automatic cleaning process can't be triggered because the node is unreachable. Please follow the steps below to manually clean the node.
|
||||
|
||||
>**Warning:** The commands listed below will remove data from the node. Make sure you have created a backup of files you want to keep before executing any of the commands as data will be lost.
|
||||
##### Options
|
||||
|
||||
### Docker Containers, Images, and Volumes
|
||||
| Option | Description |
|
||||
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--kubeconfig <$KUBECONFIG>, -c <$KUBECONFIG>` | The cluster's kubeconfig file absolute path (`<$KUBECONFIG>`). |
|
||||
| `--namespace <NAMESPACE>, -n cattle-system` | Rancher 2.x deployment namespace (`<NAMESPACE>`). If no namespace is defined, the options defaults to `cattle-system`. |
|
||||
| `--force` | Skips the the interactive removal confirmation and removes the Rancher deployment without prompt. |
|
||||
|
||||
Based on what role you assigned to the node, Kubernetes components in containers, containers belonging to overlay networking, DNS, ingress controller and Rancher agent. (and pods you created that have been scheduled to this node)
|
||||
## Imported Cluster Nodes
|
||||
|
||||
**To clean all Docker containers, images and volumes:**
|
||||
For imported clusters, the process for removing Rancher from its nodes is a little different. You can the option of simply deleting the cluster in the Rancher UI, or your can run a script that removes Rancher components from the nodes. Both options make the same deletions.
|
||||
|
||||
```
|
||||
docker rm -f $(docker ps -qa)
|
||||
docker rmi -f $(docker images -q)
|
||||
docker volume rm $(docker volume ls -q)
|
||||
```
|
||||
{{% tabs %}}
|
||||
{{% tab "By UI / API" %}}
|
||||
>**Warning:** This process will remove data from your nodes. Make sure you have created a backup of files you want to keep before executing the command, as data will be lost.
|
||||
|
||||
### Mounts
|
||||
After you initiate the removal of an [imported cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#import-existing-cluster) using the Rancher UI (or API), the following events occur.
|
||||
|
||||
Kubernetes components and secrets leave behind mounts on the system that need to be unmounted.
|
||||
1. Rancher creates a `serviceAccount` that it uses to remove the cluster. This account is assigned the [clusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) and [clusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) permissions, which are required to remove the cluster.
|
||||
|
||||
Mounts |
|
||||
--------|
|
||||
`/var/lib/kubelet/pods/XXX` (miscellaneous mounts) |
|
||||
`/var/lib/kubelet` |
|
||||
`/var/lib/rancher` |
|
||||
1. Using the `serviceAccount`, Rancher schedules and runs a [job](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) that cleans the Rancher and Kubernetes components off of the node. This job also references the `serviceAccount` and its roles as dependencies, so the job deletes them before its completion.
|
||||
|
||||
1. Rancher is removed from the cluster nodes. However, the cluster persists, running the native version of Kubernetes.
|
||||
|
||||
**To unmount all mounts:**
|
||||
**Result:** All components listed for imported clusters in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
|
||||
```
|
||||
for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab "By Script" %}}
|
||||
Rather than cleaning imported cluster nodes using the Rancher UI, you can run a script instead.
|
||||
|
||||
### Directories and Files
|
||||
>**Prerequisite:**
|
||||
>
|
||||
>Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
|
||||
|
||||
The following directories are used when adding a node to a cluster, and should be removed. You can remove a directory using `rm -rf /directory_name`.
|
||||
1. Open a web browser, navigate to [GitHub](https://github.com/rancher/rancher/blob/master/cleanup/user-cluster.sh), and download `user-cluster.sh`.
|
||||
|
||||
>**Note:** Depending on the role you assigned to the node, some of the directories will or won't be present on the node.
|
||||
1. Open kubectl.
|
||||
|
||||
Directories |
|
||||
--------|
|
||||
`/etc/ceph` |
|
||||
`/etc/cni` |
|
||||
`/etc/kubernetes` |
|
||||
`/opt/cni` |
|
||||
`/opt/rke` |
|
||||
`/run/secrets/kubernetes.io` |
|
||||
`/run/calico` |
|
||||
`/run/flannel` |
|
||||
`/var/lib/calico` |
|
||||
`/var/lib/etcd` |
|
||||
`/var/lib/cni` |
|
||||
`/var/lib/kubelet` |
|
||||
`/var/lib/rancher/rke/log` |
|
||||
`/var/log/containers` |
|
||||
`/var/log/pods` |
|
||||
`/var/run/calico` |
|
||||
1. Using kubectl, make the script executable by running the following command from the same directory as `user-cluster.sh`:
|
||||
|
||||
**To clean the directories:**
|
||||
```
|
||||
chmod +x user-cluster.sh
|
||||
```
|
||||
|
||||
```
|
||||
rm -rf /etc/ceph \
|
||||
/etc/cni \
|
||||
/etc/kubernetes \
|
||||
/opt/cni \
|
||||
/opt/rke \
|
||||
/run/secrets/kubernetes.io \
|
||||
/run/calico \
|
||||
/run/flannel \
|
||||
/var/lib/calico \
|
||||
/var/lib/etcd \
|
||||
/var/lib/cni \
|
||||
/var/lib/kubelet \
|
||||
/var/lib/rancher/rke/log \
|
||||
/var/log/containers \
|
||||
/var/log/pods \
|
||||
/var/run/calico
|
||||
```
|
||||
1. **Air Gap Users Only:** Open `user-cluster.sh` and replace `yaml_url` with the URL in `user-cluster.yml`.
|
||||
|
||||
### Network Interfaces and Iptables
|
||||
If you aren't an air gap user, skip this step.
|
||||
|
||||
The remaining two components that are changed/configured are (virtual) network interfaces and iptables rules. Both are non-persistent to the node, meaning that they will be cleared after a restart of the node.
|
||||
1. From the same directory, run the script:
|
||||
|
||||
This is the recommended method.
|
||||
>**Tip:**
|
||||
>
|
||||
>Add the `-dry-run` flag to preview the script's outcome without making changes.
|
||||
|
||||
**To restart a node:**
|
||||
```
|
||||
./user-cluster.sh rancher/agent:latest
|
||||
```
|
||||
|
||||
```
|
||||
# using reboot
|
||||
reboot
|
||||
**Result:** The script runs. All components listed for imported clusters in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# using shutdown
|
||||
shutdown -r now
|
||||
```
|
||||
## What Gets Removed?
|
||||
|
||||
If you want to know more on (virtual) network interfaces or iptables rules, please see the specific subjects below.
|
||||
When cleaning nodes provisioned using Rancher, the following components are deleted based on the type of cluster node you're removing.
|
||||
|
||||
### Network Interfaces
|
||||
| Removed Component | [IaaS Nodes][1] | [Custom Nodes][2] | [Hosted Cluster][3] | [Imported Nodes][4] |
|
||||
| ------------------------------------------------------------------------------ | --------------- | ----------------- | ------------------- | ------------------- |
|
||||
| The Rancher deployment namespace (`cattle-system` by default) | ✓ | ✓ | ✓ | ✓ |
|
||||
| `serviceAccount`, `clusterRoles`, and `clusterRoleBindings` labeled by Rancher | ✓ | ✓ | ✓ | ✓ |
|
||||
| Labels, Annotations, and Finalizers | ✓ | ✓ | ✓ | ✓ |
|
||||
| Rancher Deployment | ✓ | ✓ | ✓ | |
|
||||
| Machines, clusters, projects, and user custom resource deployments (CRDs) | ✓ | ✓ | ✓ | |
|
||||
| All resources create under the `management.cattle.io` API Group | ✓ | ✓ | ✓ | |
|
||||
| All CRDs created by Rancher v2.0.x | ✓ | ✓ | ✓ | |
|
||||
|
||||
>**Note:** Depending on the network provider configured for the cluster the node was part of, some of the interfaces will or won't be present on the node.
|
||||
[1]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/
|
||||
[2]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/
|
||||
[3]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/
|
||||
[4]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/
|
||||
|
||||
Interfaces |
|
||||
--------|
|
||||
`flannel.1` |
|
||||
`cni0` |
|
||||
`tunl0` |
|
||||
`caliXXXXXXXXXXX` (random interface names) |
|
||||
`vethXXXXXXXX` (random interface names) |
|
||||
|
||||
**To list all interfaces:**
|
||||
|
||||
```
|
||||
# Using ip
|
||||
ip address show
|
||||
|
||||
# Using ifconfig
|
||||
ifconfig -a
|
||||
```
|
||||
|
||||
**To remove an interface:**
|
||||
|
||||
```
|
||||
ip link delete interface_name
|
||||
```
|
||||
|
||||
### Iptables
|
||||
|
||||
>**Note:** Depending on the network provider configured for the cluster the node was part of, some of the chains will or won't be present on the node.
|
||||
|
||||
Iptables rules are used to route traffic from and to containers. The created rules are not persistent, so restarting the node will restore iptables to it's original state.
|
||||
|
||||
Chains |
|
||||
--------|
|
||||
`cali-failsafe-in` |
|
||||
`cali-failsafe-out` |
|
||||
`cali-fip-dnat` |
|
||||
`cali-fip-snat` |
|
||||
`cali-from-hep-forward` |
|
||||
`cali-from-host-endpoint` |
|
||||
`cali-from-wl-dispatch` |
|
||||
`cali-fw-caliXXXXXXXXXXX` (random chain names) |
|
||||
`cali-nat-outgoing` |
|
||||
`cali-pri-kns.NAMESPACE` (chain per namespace) |
|
||||
`cali-pro-kns.NAMESPACE` (chain per namespace) |
|
||||
`cali-to-hep-forward` |
|
||||
`cali-to-host-endpoint` |
|
||||
`cali-to-wl-dispatch` |
|
||||
`cali-tw-caliXXXXXXXXXXX` (random chain names) |
|
||||
`cali-wl-to-host` |
|
||||
`KUBE-EXTERNAL-SERVICES` |
|
||||
`KUBE-FIREWALL` |
|
||||
`KUBE-MARK-DROP` |
|
||||
`KUBE-MARK-MASQ` |
|
||||
`KUBE-NODEPORTS` |
|
||||
`KUBE-SEP-XXXXXXXXXXXXXXXX` (random chain names) |
|
||||
`KUBE-SERVICES` |
|
||||
`KUBE-SVC-XXXXXXXXXXXXXXXX` (random chain names) |
|
||||
|
||||
**To list all iptables rules:**
|
||||
|
||||
```
|
||||
iptables -L -t nat
|
||||
iptables -L -t mangle
|
||||
iptables -L
|
||||
```
|
||||
>**Using 2.0.7 or Earlier?**
|
||||
>
|
||||
>These versions of Rancher do not automatically delete the `serviceAccount`, `clusterRole`, and `clusterRole` resources after the job runs. You'll have to delete them yourself.
|
||||
+155
-101
@@ -1,138 +1,192 @@
|
||||
---
|
||||
title: Removing Rancher from User Cluster Nodes
|
||||
weight: 2000
|
||||
title: Removing Rancher from Rancher-Provisioned Nodes
|
||||
weight: 375
|
||||
aliases:
|
||||
- /rancher/v2.x/en/installation/removing-rancher/cleaning-cluster-nodes/
|
||||
- /rancher/v2.x/en/installation/removing-rancher/
|
||||
- /rancher/v2.x/en/faq/cleaning-cluster-nodes/
|
||||
---
|
||||
When you use Rancher to [provision nodes for a cluster]({{< baseurl >}}rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher), resources (containers/virtual network interfaces) and configuration items (certificates/configuration files) are created.
|
||||
|
||||
When removing nodes from your Rancher-provisioned cluster (provided that they are in `Active` state), those resources automatically cleaned, and the only action needed is to restart the node. When a node has become unreachable and the automatic cleanup process cannot be used, we describe the steps that need to be executed before the node can be added to a cluster again.
|
||||
|
||||
When you no longer have use for Rancher in a cluster that you've [provisioned using Rancher]({{< baseurl >}}rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher), and you want to remove Rancher from its nodes, follow one of the sets of instructions below based on your [cluster type]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-options). The method you'll use to remove Rancher changes based on the type of cluster.
|
||||
## Removing A Node from a Cluster by Rancher UI
|
||||
|
||||
When the node is in `Active` state, removing the node from a cluster will trigger a process to clean up the node. Please restart the node after the automatic cleanup process is done to make sure any non-persistent data is properly removed.
|
||||
|
||||
## Nodes Launched by RKE / Nodes Hosted by a Provider
|
||||
|
||||
For clusters nodes provisioned using [RKE](({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/)) or a [hosted Kubernetes provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#hosted-kubernetes-cluster), you can remove Rancher by downloading and running the Rancher system-tools.
|
||||
|
||||
### Download and Configuration
|
||||
|
||||
You can download the latest version of Rancher system-tools from its GitHub [releases page](https://github.com/rancher/system-tools/releases). Download the version of system-tools for the OS that you're removing Rancher from.
|
||||
|
||||
Operating System | File
|
||||
-----------------|-----
|
||||
MacOS | `system-tools_darwin-amd64`
|
||||
Linux | `system-tools_linux-amd64`
|
||||
Windows | `system-tools_windows-amd64.exe`
|
||||
|
||||
<br>
|
||||
|
||||
After you download the tools, move it onto the node you're removing Rancher from, and then complete the following actions:
|
||||
|
||||
1. Rename the file to `system-tools`.
|
||||
|
||||
1. Give the file executable permissions by running the following command:
|
||||
|
||||
```
|
||||
chmod +x system-tools
|
||||
```
|
||||
|
||||
### Using the System-Tool
|
||||
|
||||
System-tools is a utility that cleans up Rancher. In this use case, it will help you remove the Rancher from your cluster nodes.
|
||||
|
||||
#### Usage
|
||||
|
||||
After you move the `system-tools` file to your Rancher node, you can run it by changing to the file's directory and running the following command.
|
||||
|
||||
>**Warning:** This command will remove data from your nodes. Make sure you have created a backup of files you want to keep before executing the command, as data will be lost.
|
||||
**To restart a node:**
|
||||
|
||||
```
|
||||
./system-tools remove --kubeconfig <$KUBECONFIG> --namespace <NAMESPACE>
|
||||
# using reboot
|
||||
reboot
|
||||
|
||||
# using shutdown
|
||||
shutdown -r now
|
||||
```
|
||||
|
||||
<br/>
|
||||
When you run this command, the components listed in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
## Cleaning a Node Manually
|
||||
|
||||
When a node is unreachable and removed from the cluster, the automatic cleaning process can't be triggered because the node is unreachable. Please follow the steps below to manually clean the node.
|
||||
|
||||
##### Options
|
||||
>**Warning:** The commands listed below will remove data from the node. Make sure you have created a backup of files you want to keep before executing any of the commands as data will be lost.
|
||||
|
||||
| Option | Description |
|
||||
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `--kubeconfig <$KUBECONFIG>, -c <$KUBECONFIG>` | The cluster's kubeconfig file absolute path (`<$KUBECONFIG>`). |
|
||||
| `--namespace <NAMESPACE>, -n cattle-system` | Rancher 2.x deployment namespace (`<NAMESPACE>`). If no namespace is defined, the options defaults to `cattle-system`. |
|
||||
| `--force` | Skips the the interactive removal confirmation and removes the Rancher deployment without prompt. |
|
||||
### Docker Containers, Images, and Volumes
|
||||
|
||||
## Imported Cluster Nodes
|
||||
Based on what role you assigned to the node, Kubernetes components in containers, containers belonging to overlay networking, DNS, ingress controller and Rancher agent. (and pods you created that have been scheduled to this node)
|
||||
|
||||
For imported clusters, the process for removing Rancher from its nodes is a little different. You can the option of simply deleting the cluster in the Rancher UI, or your can run a script that removes Rancher components from the nodes. Both options make the same deletions.
|
||||
**To clean all Docker containers, images and volumes:**
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "By UI / API" %}}
|
||||
>**Warning:** This process will remove data from your nodes. Make sure you have created a backup of files you want to keep before executing the command, as data will be lost.
|
||||
```
|
||||
docker rm -f $(docker ps -qa)
|
||||
docker rmi -f $(docker images -q)
|
||||
docker volume rm $(docker volume ls -q)
|
||||
```
|
||||
|
||||
After you initiate the removal of an [imported cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#import-existing-cluster) using the Rancher UI (or API), the following events occur.
|
||||
### Mounts
|
||||
|
||||
1. Rancher creates a `serviceAccount` that it uses to remove the cluster. This account is assigned the [clusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole) and [clusterRoleBinding](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) permissions, which are required to remove the cluster.
|
||||
Kubernetes components and secrets leave behind mounts on the system that need to be unmounted.
|
||||
|
||||
1. Using the `serviceAccount`, Rancher schedules and runs a [job](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) that cleans the Rancher and Kubernetes components off of the node. This job also references the `serviceAccount` and its roles as dependencies, so the job deletes them before its completion.
|
||||
|
||||
1. Rancher is removed from the cluster nodes. However, the cluster persists, running the native version of Kubernetes.
|
||||
Mounts |
|
||||
--------|
|
||||
`/var/lib/kubelet/pods/XXX` (miscellaneous mounts) |
|
||||
`/var/lib/kubelet` |
|
||||
`/var/lib/rancher` |
|
||||
|
||||
**Result:** All components listed for imported clusters in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
**To unmount all mounts:**
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "By Script" %}}
|
||||
Rather than cleaning imported cluster nodes using the Rancher UI, you can run a script instead.
|
||||
```
|
||||
for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
|
||||
```
|
||||
|
||||
>**Prerequisite:**
|
||||
>
|
||||
>Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
|
||||
### Directories and Files
|
||||
|
||||
1. Open a web browser, navigate to [GitHub](https://github.com/rancher/rancher/blob/master/cleanup/user-cluster.sh), and download `user-cluster.sh`.
|
||||
The following directories are used when adding a node to a cluster, and should be removed. You can remove a directory using `rm -rf /directory_name`.
|
||||
|
||||
1. Open kubectl.
|
||||
>**Note:** Depending on the role you assigned to the node, some of the directories will or won't be present on the node.
|
||||
|
||||
1. Using kubectl, make the script executable by running the following command from the same directory as `user-cluster.sh`:
|
||||
Directories |
|
||||
--------|
|
||||
`/etc/ceph` |
|
||||
`/etc/cni` |
|
||||
`/etc/kubernetes` |
|
||||
`/opt/cni` |
|
||||
`/opt/rke` |
|
||||
`/run/secrets/kubernetes.io` |
|
||||
`/run/calico` |
|
||||
`/run/flannel` |
|
||||
`/var/lib/calico` |
|
||||
`/var/lib/etcd` |
|
||||
`/var/lib/cni` |
|
||||
`/var/lib/kubelet` |
|
||||
`/var/lib/rancher/rke/log` |
|
||||
`/var/log/containers` |
|
||||
`/var/log/pods` |
|
||||
`/var/run/calico` |
|
||||
|
||||
```
|
||||
chmod +x user-cluster.sh
|
||||
```
|
||||
**To clean the directories:**
|
||||
|
||||
1. **Air Gap Users Only:** Open `user-cluster.sh` and replace `yaml_url` with the URL in `user-cluster.yml`.
|
||||
```
|
||||
rm -rf /etc/ceph \
|
||||
/etc/cni \
|
||||
/etc/kubernetes \
|
||||
/opt/cni \
|
||||
/opt/rke \
|
||||
/run/secrets/kubernetes.io \
|
||||
/run/calico \
|
||||
/run/flannel \
|
||||
/var/lib/calico \
|
||||
/var/lib/etcd \
|
||||
/var/lib/cni \
|
||||
/var/lib/kubelet \
|
||||
/var/lib/rancher/rke/log \
|
||||
/var/log/containers \
|
||||
/var/log/pods \
|
||||
/var/run/calico
|
||||
```
|
||||
|
||||
If you aren't an air gap user, skip this step.
|
||||
### Network Interfaces and Iptables
|
||||
|
||||
1. From the same directory, run the script:
|
||||
The remaining two components that are changed/configured are (virtual) network interfaces and iptables rules. Both are non-persistent to the node, meaning that they will be cleared after a restart of the node.
|
||||
|
||||
>**Tip:**
|
||||
>
|
||||
>Add the `-dry-run` flag to preview the script's outcome without making changes.
|
||||
This is the recommended method.
|
||||
|
||||
```
|
||||
./user-cluster.sh rancher/agent:latest
|
||||
```
|
||||
**To restart a node:**
|
||||
|
||||
**Result:** The script runs. All components listed for imported clusters in [What Gets Removed?](#what-gets-removed) are deleted.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
```
|
||||
# using reboot
|
||||
reboot
|
||||
|
||||
## What Gets Removed?
|
||||
# using shutdown
|
||||
shutdown -r now
|
||||
```
|
||||
|
||||
When cleaning nodes provisioned using Rancher, the following components are deleted based on the type of cluster node you're removing.
|
||||
If you want to know more on (virtual) network interfaces or iptables rules, please see the specific subjects below.
|
||||
|
||||
| Removed Component | [IaaS Nodes][1] | [Custom Nodes][2] | [Hosted Cluster][3] | [Imported Nodes][4] |
|
||||
| ------------------------------------------------------------------------------ | --------------- | ----------------- | ------------------- | ------------------- |
|
||||
| The Rancher deployment namespace (`cattle-system` by default) | ✓ | ✓ | ✓ | ✓ |
|
||||
| `serviceAccount`, `clusterRoles`, and `clusterRoleBindings` labeled by Rancher | ✓ | ✓ | ✓ | ✓ |
|
||||
| Labels, Annotations, and Finalizers | ✓ | ✓ | ✓ | ✓ |
|
||||
| Rancher Deployment | ✓ | ✓ | ✓ | |
|
||||
| Machines, clusters, projects, and user custom resource deployments (CRDs) | ✓ | ✓ | ✓ | |
|
||||
| All resources create under the `management.cattle.io` API Group | ✓ | ✓ | ✓ | |
|
||||
| All CRDs created by Rancher v2.0.x | ✓ | ✓ | ✓ | |
|
||||
### Network Interfaces
|
||||
|
||||
[1]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/
|
||||
[2]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/
|
||||
[3]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/
|
||||
[4]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/
|
||||
>**Note:** Depending on the network provider configured for the cluster the node was part of, some of the interfaces will or won't be present on the node.
|
||||
|
||||
>**Using 2.0.7 or Earlier?**
|
||||
>
|
||||
>These versions of Rancher do not automatically delete the `serviceAccount`, `clusterRole`, and `clusterRole` resources after the job runs. You'll have to delete them yourself.
|
||||
Interfaces |
|
||||
--------|
|
||||
`flannel.1` |
|
||||
`cni0` |
|
||||
`tunl0` |
|
||||
`caliXXXXXXXXXXX` (random interface names) |
|
||||
`vethXXXXXXXX` (random interface names) |
|
||||
|
||||
**To list all interfaces:**
|
||||
|
||||
```
|
||||
# Using ip
|
||||
ip address show
|
||||
|
||||
# Using ifconfig
|
||||
ifconfig -a
|
||||
```
|
||||
|
||||
**To remove an interface:**
|
||||
|
||||
```
|
||||
ip link delete interface_name
|
||||
```
|
||||
|
||||
### Iptables
|
||||
|
||||
>**Note:** Depending on the network provider configured for the cluster the node was part of, some of the chains will or won't be present on the node.
|
||||
|
||||
Iptables rules are used to route traffic from and to containers. The created rules are not persistent, so restarting the node will restore iptables to it's original state.
|
||||
|
||||
Chains |
|
||||
--------|
|
||||
`cali-failsafe-in` |
|
||||
`cali-failsafe-out` |
|
||||
`cali-fip-dnat` |
|
||||
`cali-fip-snat` |
|
||||
`cali-from-hep-forward` |
|
||||
`cali-from-host-endpoint` |
|
||||
`cali-from-wl-dispatch` |
|
||||
`cali-fw-caliXXXXXXXXXXX` (random chain names) |
|
||||
`cali-nat-outgoing` |
|
||||
`cali-pri-kns.NAMESPACE` (chain per namespace) |
|
||||
`cali-pro-kns.NAMESPACE` (chain per namespace) |
|
||||
`cali-to-hep-forward` |
|
||||
`cali-to-host-endpoint` |
|
||||
`cali-to-wl-dispatch` |
|
||||
`cali-tw-caliXXXXXXXXXXX` (random chain names) |
|
||||
`cali-wl-to-host` |
|
||||
`KUBE-EXTERNAL-SERVICES` |
|
||||
`KUBE-FIREWALL` |
|
||||
`KUBE-MARK-DROP` |
|
||||
`KUBE-MARK-MASQ` |
|
||||
`KUBE-NODEPORTS` |
|
||||
`KUBE-SEP-XXXXXXXXXXXXXXXX` (random chain names) |
|
||||
`KUBE-SERVICES` |
|
||||
`KUBE-SVC-XXXXXXXXXXXXXXXX` (random chain names) |
|
||||
|
||||
**To list all iptables rules:**
|
||||
|
||||
```
|
||||
iptables -L -t nat
|
||||
iptables -L -t mangle
|
||||
iptables -L
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user