k3s: update instruction for rootless mode

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This commit is contained in:
Akihiro Suda
2021-05-06 17:41:12 +09:00
parent 8144e09ef0
commit fb301936a9
2 changed files with 38 additions and 21 deletions
+36 -19
View File
@@ -13,7 +13,7 @@ This section contains advanced information describing the different ways you can
- [Using Docker as the container runtime](#using-docker-as-the-container-runtime)
- [Configuring containerd](#configuring-containerd)
- [Secrets Encryption Config (Experimental)](#secrets-encryption-config-experimental)
- [Running K3s with RootlessKit (Experimental)](#running-k3s-with-rootlesskit-experimental)
- [Running K3s with Rootless mode (Experimental)](#running-k3s-with-rootless-mode-experimental)
- [Node labels and taints](#node-labels-and-taints)
- [Starting the server with the installation script](#starting-the-server-with-the-installation-script)
- [Additional preparation for Alpine Linux setup](#additional-preparation-for-alpine-linux-setup)
@@ -163,18 +163,15 @@ As of v1.17.4+k3s1, K3s added the experimental feature of enabling secrets encry
Once enabled any created secret will be encrypted with this key. Note that if you disable encryption then any encrypted secrets will not be readable until you enable encryption again.
# Running K3s with RootlessKit (Experimental)
# Running K3s with Rootless mode (Experimental)
> **Warning:** This feature is experimental.
RootlessKit is a kind of Linux-native "fake root" utility, made for mainly [running Docker and Kubernetes as an unprivileged user,](https://github.com/rootless-containers/usernetes) so as to protect the real root on the host from potential container-breakout attacks.
Rootless mode allows running the entire k3s an unprivileged user, so as to protect the real root on the host from potential container-breakout attacks.
Initial rootless support has been added but there are a series of significant usability issues surrounding it.
See also https://rootlesscontaine.rs/ to learn about Rootless mode.
We are releasing the initial support for those interested in rootless and hopefully some people can help to improve the usability. First, ensure you have a proper setup and support for user namespaces. Refer to the [requirements section](https://github.com/rootless-containers/rootlesskit#setup) in RootlessKit for instructions.
In short, latest Ubuntu is your best bet for this to work.
### Known Issues with RootlessKit
### Known Issues with Rootless mode
* **Ports**
@@ -184,24 +181,44 @@ In short, latest Ubuntu is your best bet for this to work.
Currently, only `LoadBalancer` services are automatically bound.
* **Daemon lifecycle**
Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left
with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace.
The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65
* **Cgroups**
Cgroups are not supported.
Cgroup v1 is not supported. v2 is supported.
* **Multi-node cluster**
Multi-cluster installation is untested and undocumented.
### Running Servers and Agents with Rootless
* Enable cgroup v2 delegation, see https://rootlesscontaine.rs/getting-started/common/cgroup2/ .
This step is optional, but highly recommended for enabling CPU and memory resource limtitation.
Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message `Wrote kubeconfig [SOME PATH]` for where your kubeconfig file is.
* Download `k3s-rootless.service` from [`https://github.com/k3s-io/k3s/blob/<VERSION>/k3s-rootless.service`](https://github.com/k3s-io/k3s/blob/master/k3s-rootless.service).
Make sure to use the same version of `k3s-rootless.service` and `k3s`.
For more information about setting up the kubeconfig file, refer to the [section about cluster access.](../cluster-access)
* Install `k3s-rootless.service` to `~/.config/systemd/user/k3s-rootless.service`.
Installing this file as a system-wide service (`/etc/systemd/...`) is not supported.
Depending on the path of `k3s` binary, you might need to modify the `ExecStart=/usr/local/bin/k3s ...` line of the file.
> Be careful, if you use `-o` to write the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different mount namespace.
* Run `systemctl --user daemon-reload`
* Run `systemctl --user enable --now k3s-rootless`
* Run `KUBECONFIG=~/.kube/k3s.yaml kubectl get pods -A`, and make sure the pods are running.
> **Note:** Don't try to run `k3s server --rootless` on a terminal, as it doesn't enable cgroup v2 delegation.
> If you really need to try it on a terminal, prepend `systemd-run --user -p Delegate=yes --tty` to create a systemd scope.
>
> i.e.,
> ```console
> $ systemd-run --user -p Delegate=yes --tty k3s server --rootless
> ```
### Troubleshooting
* Run `systemctl --user status k3s-rootless` to check the daemon status
* Run `journalctl --user -f -u k3s-rootless` to see the daemon log
* See also https://rootlesscontaine.rs/
# Node Labels and Taints
+2 -2
View File
@@ -12,6 +12,6 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r
If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues.
**RootlessKit**
**Rootless Mode**
Running K3s with RootlessKit is experimental and has several [known issues.]({{<baseurl>}}/k3s/latest/en/advanced/#known-issues-with-rootlesskit)
Running K3s with Rootless mode is experimental and has several [known issues.]({{<baseurl>}}/k3s/latest/en/advanced/#known-issues-with-rootless-mode)