Docs: Migrate worker nodes troubleshooting guide from RKE/Docker to RKE2/K3s/containerd

This commit is contained in:
Manuel Simón Nóvoa
2026-04-20 17:12:58 +02:00
parent 7cca9e7574
commit a321c9fa41
4 changed files with 272 additions and 56 deletions
@@ -8,31 +8,85 @@ title: Troubleshooting Worker Nodes and Generic Components
This section applies to every node as it includes components that run on nodes with any role. This section applies to every node as it includes components that run on nodes with any role.
## Check if the Containers are Running ## Prerequisites
There are two specific containers launched on nodes with the `worker` role: Since RKE2 and K3s utilize `containerd` as the container runtime, `crictl` serves as the primary tool for container management, replacing the Docker CLI. To allow `crictl` to communicate with `containerd`, you must configure your environment by exporting the following variables:
* kubelet ### RKE2
* kube-proxy
The containers should have status `Up`. The duration shown after `Up` is the time the container has been running.
```bash
export PATH=$PATH:/var/lib/rancher/rke2/bin/
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
``` ```
docker ps -a -f=name='kubelet|kube-proxy'
### K3s
```bash
export PATH=$PATH:/usr/local/bin
export CRI_CONFIG_FILE=/var/lib/rancher/k3s/agent/etc/crictl.yaml
```
## Check if the Components are Running
There are two specific components launched on nodes with the `worker` role:
* `kubelet`
* `kube-proxy`
### RKE2
The `kubelet` runs natively as part of the `rke2-agent` (or `rke2-server`) systemd process, while `kube-proxy` runs as a Static Pod managed by `containerd`.
Check the status of the `kubelet` via the agent service:
```bash
systemctl status rke2-agent
```
:::note
If you are checking a controlplane node, use `systemctl status rke2-server` instead.
:::
Check the status of `kube-proxy` using `crictl`:
```bash
crictl ps --name kube-proxy
``` ```
Example output: Example output:
``` ```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
158d0dcc33a5 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kube-proxy 26c7159abbcc rancher/hardened-kubernetes:v1.28.8-rke2r1-build20240404 3 hours ago Running kube-proxy 0 1a2b3c4d5e6f7
a30717ecfb55 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kubelet
``` ```
## Container Logging ### K3s
The logging of the containers can contain information on what the problem could be. Both `kubelet` and `kube-proxy` run as embedded processes inside the `k3s-agent` (or `k3s` server) systemd service. There are no separate containers for them.
Check their status by checking the K3s service:
```bash
systemctl status k3s-agent
``` ```
docker logs kubelet :::note
docker logs kube-proxy If you are checking a controlplane node, use `systemctl status k3s` instead.
:::
## Component Logging
The logging of the components can contain information on what the problem could be.
### RKE2
```bash
# kubelet logs are part of the systemd service
journalctl -u rke2-agent -f | grep -i "kubelet"
# kube-proxy logs are retrieved from containerd
crictl logs $(crictl ps --name kube-proxy -q)
```
### K3s
```bash
# Both components log to the systemd service
journalctl -u k3s-agent -f | grep -iE "kubelet|kube-proxy"
``` ```
@@ -8,31 +8,85 @@ title: Troubleshooting Worker Nodes and Generic Components
This section applies to every node as it includes components that run on nodes with any role. This section applies to every node as it includes components that run on nodes with any role.
## Check if the Containers are Running ## Prerequisites
There are two specific containers launched on nodes with the `worker` role: Since RKE2 and K3s utilize `containerd` as the container runtime, `crictl` serves as the primary tool for container management, replacing the Docker CLI. To allow `crictl` to communicate with `containerd`, you must configure your environment by exporting the following variables:
* kubelet ### RKE2
* kube-proxy
The containers should have status `Up`. The duration shown after `Up` is the time the container has been running.
```bash
export PATH=$PATH:/var/lib/rancher/rke2/bin/
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
``` ```
docker ps -a -f=name='kubelet|kube-proxy'
### K3s
```bash
export PATH=$PATH:/usr/local/bin
export CRI_CONFIG_FILE=/var/lib/rancher/k3s/agent/etc/crictl.yaml
```
## Check if the Components are Running
There are two specific components launched on nodes with the `worker` role:
* `kubelet`
* `kube-proxy`
### RKE2
The `kubelet` runs natively as part of the `rke2-agent` (or `rke2-server`) systemd process, while `kube-proxy` runs as a Static Pod managed by `containerd`.
Check the status of the `kubelet` via the agent service:
```bash
systemctl status rke2-agent
```
:::note
If you are checking a controlplane node, use `systemctl status rke2-server` instead.
:::
Check the status of `kube-proxy` using `crictl`:
```bash
crictl ps --name kube-proxy
``` ```
Example output: Example output:
``` ```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
158d0dcc33a5 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kube-proxy 26c7159abbcc rancher/hardened-kubernetes:v1.28.8-rke2r1-build20240404 3 hours ago Running kube-proxy 0 1a2b3c4d5e6f7
a30717ecfb55 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kubelet
``` ```
## Container Logging ### K3s
The logging of the containers can contain information on what the problem could be. Both `kubelet` and `kube-proxy` run as embedded processes inside the `k3s-agent` (or `k3s` server) systemd service. There are no separate containers for them.
Check their status by checking the K3s service:
```bash
systemctl status k3s-agent
``` ```
docker logs kubelet :::note
docker logs kube-proxy If you are checking a controlplane node, use `systemctl status k3s` instead.
:::
## Component Logging
The logging of the components can contain information on what the problem could be.
### RKE2
```bash
# kubelet logs are part of the systemd service
journalctl -u rke2-agent -f | grep -i "kubelet"
# kube-proxy logs are retrieved from containerd
crictl logs $(crictl ps --name kube-proxy -q)
```
### K3s
```bash
# Both components log to the systemd service
journalctl -u k3s-agent -f | grep -iE "kubelet|kube-proxy"
``` ```
@@ -8,31 +8,85 @@ title: Troubleshooting Worker Nodes and Generic Components
This section applies to every node as it includes components that run on nodes with any role. This section applies to every node as it includes components that run on nodes with any role.
## Check if the Containers are Running ## Prerequisites
There are two specific containers launched on nodes with the `worker` role: Since RKE2 and K3s utilize `containerd` as the container runtime, `crictl` serves as the primary tool for container management, replacing the Docker CLI. To allow `crictl` to communicate with `containerd`, you must configure your environment by exporting the following variables:
* kubelet ### RKE2
* kube-proxy
The containers should have status `Up`. The duration shown after `Up` is the time the container has been running.
```bash
export PATH=$PATH:/var/lib/rancher/rke2/bin/
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
``` ```
docker ps -a -f=name='kubelet|kube-proxy'
### K3s
```bash
export PATH=$PATH:/usr/local/bin
export CRI_CONFIG_FILE=/var/lib/rancher/k3s/agent/etc/crictl.yaml
```
## Check if the Components are Running
There are two specific components launched on nodes with the `worker` role:
* `kubelet`
* `kube-proxy`
### RKE2
The `kubelet` runs natively as part of the `rke2-agent` (or `rke2-server`) systemd process, while `kube-proxy` runs as a Static Pod managed by `containerd`.
Check the status of the `kubelet` via the agent service:
```bash
systemctl status rke2-agent
```
:::note
If you are checking a controlplane node, use `systemctl status rke2-server` instead.
:::
Check the status of `kube-proxy` using `crictl`:
```bash
crictl ps --name kube-proxy
``` ```
Example output: Example output:
``` ```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
158d0dcc33a5 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kube-proxy 26c7159abbcc rancher/hardened-kubernetes:v1.28.8-rke2r1-build20240404 3 hours ago Running kube-proxy 0 1a2b3c4d5e6f7
a30717ecfb55 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kubelet
``` ```
## Container Logging ### K3s
The logging of the containers can contain information on what the problem could be. Both `kubelet` and `kube-proxy` run as embedded processes inside the `k3s-agent` (or `k3s` server) systemd service. There are no separate containers for them.
Check their status by checking the K3s service:
```bash
systemctl status k3s-agent
``` ```
docker logs kubelet :::note
docker logs kube-proxy If you are checking a controlplane node, use `systemctl status k3s` instead.
:::
## Component Logging
The logging of the components can contain information on what the problem could be.
### RKE2
```bash
# kubelet logs are part of the systemd service
journalctl -u rke2-agent -f | grep -i "kubelet"
# kube-proxy logs are retrieved from containerd
crictl logs $(crictl ps --name kube-proxy -q)
```
### K3s
```bash
# Both components log to the systemd service
journalctl -u k3s-agent -f | grep -iE "kubelet|kube-proxy"
``` ```
@@ -8,31 +8,85 @@ title: Troubleshooting Worker Nodes and Generic Components
This section applies to every node as it includes components that run on nodes with any role. This section applies to every node as it includes components that run on nodes with any role.
## Check if the Containers are Running ## Prerequisites
There are two specific containers launched on nodes with the `worker` role: Since RKE2 and K3s utilize `containerd` as the container runtime, `crictl` serves as the primary tool for container management, replacing the Docker CLI. To allow `crictl` to communicate with `containerd`, you must configure your environment by exporting the following variables:
* kubelet ### RKE2
* kube-proxy
The containers should have status `Up`. The duration shown after `Up` is the time the container has been running.
```bash
export PATH=$PATH:/var/lib/rancher/rke2/bin/
export CRI_CONFIG_FILE=/var/lib/rancher/rke2/agent/etc/crictl.yaml
``` ```
docker ps -a -f=name='kubelet|kube-proxy'
### K3s
```bash
export PATH=$PATH:/usr/local/bin
export CRI_CONFIG_FILE=/var/lib/rancher/k3s/agent/etc/crictl.yaml
```
## Check if the Components are Running
There are two specific components launched on nodes with the `worker` role:
* `kubelet`
* `kube-proxy`
### RKE2
The `kubelet` runs natively as part of the `rke2-agent` (or `rke2-server`) systemd process, while `kube-proxy` runs as a Static Pod managed by `containerd`.
Check the status of the `kubelet` via the agent service:
```bash
systemctl status rke2-agent
```
:::note
If you are checking a controlplane node, use `systemctl status rke2-server` instead.
:::
Check the status of `kube-proxy` using `crictl`:
```bash
crictl ps --name kube-proxy
``` ```
Example output: Example output:
``` ```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
158d0dcc33a5 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kube-proxy 26c7159abbcc rancher/hardened-kubernetes:v1.28.8-rke2r1-build20240404 3 hours ago Running kube-proxy 0 1a2b3c4d5e6f7
a30717ecfb55 rancher/hyperkube:v1.11.5-rancher1 "/opt/rke-tools/en..." 3 hours ago Up 3 hours kubelet
``` ```
## Container Logging ### K3s
The logging of the containers can contain information on what the problem could be. Both `kubelet` and `kube-proxy` run as embedded processes inside the `k3s-agent` (or `k3s` server) systemd service. There are no separate containers for them.
Check their status by checking the K3s service:
```bash
systemctl status k3s-agent
``` ```
docker logs kubelet :::note
docker logs kube-proxy If you are checking a controlplane node, use `systemctl status k3s` instead.
:::
## Component Logging
The logging of the components can contain information on what the problem could be.
### RKE2
```bash
# kubelet logs are part of the systemd service
journalctl -u rke2-agent -f | grep -i "kubelet"
# kube-proxy logs are retrieved from containerd
crictl logs $(crictl ps --name kube-proxy -q)
```
### K3s
```bash
# Both components log to the systemd service
journalctl -u k3s-agent -f | grep -iE "kubelet|kube-proxy"
``` ```