diff --git a/.gitignore b/.gitignore index 130a0023abe..3db5f73ddd2 100644 --- a/.gitignore +++ b/.gitignore @@ -5,4 +5,4 @@ package-lock.json *.tern-port */**/.tern-port .DS_Store -.vscode/settings.json \ No newline at end of file +.vscode/settings.json diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index ede2b86a14d..58cb7baa2e8 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -14,13 +14,12 @@ Great for: * ARM * Situations where a PhD in k8s clusterology is infeasible -What is this? ---- +# What is K3s? K3s is a fully compliant Kubernetes distribution with the following enhancements: * An embedded SQLite database has replaced etcd as the default datastore. External datastores such as PostgreSQL, MySQL, and etcd are also supported. -* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a helm controller, and the Traefik ingress controller. +* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller. * Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates. * In-tree cloud providers and storage plugins have been removed. * External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including: diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 426a40d4afd..ca549701510 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -1,23 +1,174 @@ --- -title: "Advanced Options" +title: "Advanced Options and Configuration" weight: 40 aliases: - /k3s/latest/en/running/ + - /k3s/latest/en/configuration/ --- -This section contains advanced information describing the different ways you can run and manage K3s. +This section contains advanced information describing the different ways you can run and manage K3s: -Starting the Server ------------------- +- [Auto-deploying manifests](#auto-deploying-manifests) +- [Using the Helm CRD](#using-the-helm-crd) +- [Accessing the cluster from outside with kubectl](#accessing-the-cluster-from-outside-with-kubectl) +- [Using Docker as the container runtime](#using-docker-as-the-container-runtime) +- [Running K3s with RootlessKit (Experimental)](#running-k3s-with-rootlesskit-experimental) +- [Node labels and taints](#node-labels-and-taints) +- [Starting the server with the installation script](#starting-the-server-with-the-installation-script) +- [Additional preparation for Alpine Linux setup](#additional-preparation-for-alpine-linux-setup) +- [Running K3d (K3s in Docker) and docker-compose](#running-k3d-k3s-in-docker-and-docker-compose) + +# Auto-Deploying Manifests + +Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to +Kubernetes in a manner similar to `kubectl apply`. + +It is also possible to deploy Helm charts. K3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`): + +```yaml +apiVersion: helm.cattle.io/v1 +kind: HelmChart +metadata: + name: traefik + namespace: kube-system +spec: + chart: stable/traefik + set: + rbac.enabled: "true" + ssl.enabled: "true" +``` + +Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual Helm release, you can do that using `targetNamespace` key under the `spec` directive, as shown in the configuration example below. + +Also note that besides `set`, you can use `valuesContent` under the `spec` directive. And it's okay to use both of them: + +```yaml +apiVersion: helm.cattle.io/v1 +kind: HelmChart +metadata: + name: grafana + namespace: kube-system +spec: + chart: stable/grafana + targetNamespace: monitoring + set: + adminPassword: "NotVerySafePassword" + valuesContent: |- + image: + tag: master + env: + GF_EXPLORE_ENABLED: true + adminUser: admin + sidecar: + datasources: + enabled: true +``` + +K3s versions `<= v0.5.0` used `k3s.cattle.io` for the API group of HelmCharts. This has been changed to `helm.cattle.io` for later versions. + +# Using the Helm CRD + +You can deploy a 3rd party Helm chart using an example like this: + +```yaml +apiVersion: helm.cattle.io/v1 +kind: HelmChart +metadata: + name: nginx + namespace: kube-system +spec: + chart: nginx + repo: https://charts.bitnami.com/bitnami + targetNamespace: default +``` + +You can install a specific version of a Helm chart using an example like this: + +```yaml +apiVersion: helm.cattle.io/v1 +kind: HelmChart +metadata: + name: stable/nginx-ingress + namespace: kube-system +spec: + chart: nginx-ingress + version: 1.24.4 + targetNamespace: default +``` + +# Accessing the Cluster from Outside with kubectl + +Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace "localhost" with the IP or name of your K3s server. `kubectl` can now manage your K3s cluster. + +# Using Docker as the Container Runtime + +K3s includes and defaults to [containerd,](https://containerd.io/) an industry-standard container runtime. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag. + +K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`. For advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead. + +The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template, the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32 + +# Running K3s with RootlessKit (Experimental) + +> **Warning:** This feature is experimental. + +RootlessKit is a kind of Linux-native "fake root" utility, made for mainly [running Docker and Kubernetes as an unprivileged user,](https://github.com/rootless-containers/usernetes) so as to protect the real root on the host from potential container-breakout attacks. + +Initial rootless support has been added but there are a series of significant usability issues surrounding it. + +We are releasing the initial support for those interested in rootless and hopefully some people can help to improve the usability. First, ensure you have a proper setup and support for user namespaces. Refer to the [requirements section](https://github.com/rootless-containers/rootlesskit#setup) in RootlessKit for instructions. +In short, latest Ubuntu is your best bet for this to work. + +### Known Issues with RootlessKit + +* **Ports** + + When running rootless a new network namespace is created. This means that K3s instance is running with networking fairly detached from the host. The only way to access services run in K3s from the host is to set up port forwards to the K3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000. + + That means service port 80 will become 10080 on the host, but 8080 will become 8080 without any offset. + + Currently, only `LoadBalancer` services are automatically bound. + +* **Daemon lifecycle** + + Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left + with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace. + + The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65 + +* **Cgroups** + + Cgroups are not supported. + +### Running Servers and Agents with Rootless + +Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message +`Wrote kubeconfig [SOME PATH]` for where your kubeconfig to access you cluster is. + +> Be careful, if you use `-o` to write +the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different +mount namespace. + +# Node Labels and Taints + +K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints [at registration time,]({{}}/k3s/latest/en/installation/install-options/#node-labels-and-taints-for-agents) so they can only be added once and not changed after that again by running K3s commands. + +If you want to change node labels and taints after node registration you should use `kubectl`. Refer to the official Kubernetes documentation for details on how to add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) and [node labels.](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node) + +# Starting the Server with the Installation Script The installation script will auto-detect if your OS is using systemd or openrc and start the service. -When running with openrc logs will be created at `/var/log/k3s.log`, or with systemd in `/var/log/syslog` and viewed using `journalctl -u k3s`. An example of installing and auto-starting with the install script: +When running with openrc, logs will be created at `/var/log/k3s.log`. + +When running with systemd, logs will be created in `/var/log/syslog` and viewed using `journalctl -u k3s`. + +An example of installing and auto-starting with the install script: ```bash curl -sfL https://get.k3s.io | sh - ``` -When running the server manually you should get an output similar to: +When running the server manually you should get an output similar to the following: ``` $ k3s server @@ -38,10 +189,9 @@ INFO[2019-01-22T15:16:20.541049100-07:00] Run: k3s kubectl The output will likely be much longer as the agent will create a lot of logs. By default the server will register itself as a node (run the agent). -Alpine Linux ------------- +# Additional Preparation for Alpine Linux Setup -In order to pre-setup Alpine Linux you have to go through the following steps: +In order to set up Alpine Linux, you have to go through the following preparation: ```bash echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab @@ -75,19 +225,26 @@ reboot After rebooting: -- download **k3s** to **/usr/local/bin/k3s** -- create an openrc file in **/etc/init.d** +- Download **k3s** to **/usr/local/bin/k3s** +- Create an openrc file in **/etc/init.d** -Running in Docker (and docker-compose) ------------------ +# Running K3d (K3s in Docker) and docker-compose -[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS. +[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker. -`rancher/k3s` images are also available to run K3s server and agent from Docker. A `docker-compose.yml` is in the root of the K3s repo that -serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo run: +It can be installed via the the [brew](https://brew.sh/) utility on MacOS: + +``` +brew install k3d +``` + +`rancher/k3s` images are also available to run the K3s server and agent from Docker. + +A `docker-compose.yml` is in the root of the K3s repo that serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo, run: docker-compose up --scale node=3 # kubeconfig is written to current dir + kubectl --kubeconfig kubeconfig.yaml get node NAME STATUS ROLES AGE VERSION @@ -95,12 +252,14 @@ serves as an example of how to run K3s from Docker. To run from `docker-compose d54c8b17c055 Ready 11s v1.13.2-k3s2 db7a5a5a5bdd Ready 12s v1.13.2-k3s2 -To run the agent only in Docker, use `docker-compose up node`. Alternatively the Docker run command can also be used; +To run the agent only in Docker, use `docker-compose up node`. + +Alternatively the `docker run` command can also be used: sudo docker run \ - -d --tmpfs /run \ - --tmpfs /var/run \ - -e K3S_URL=${SERVER_URL} \ - -e K3S_TOKEN=${NODE_TOKEN} \ - --privileged rancher/k3s:vX.Y.Z + -d --tmpfs /run \ + --tmpfs /var/run \ + -e K3S_URL=${SERVER_URL} \ + -e K3S_TOKEN=${NODE_TOKEN} \ + --privileged rancher/k3s:vX.Y.Z diff --git a/content/k3s/latest/en/architecture/_index.md b/content/k3s/latest/en/architecture/_index.md new file mode 100644 index 00000000000..4f551fb99bf --- /dev/null +++ b/content/k3s/latest/en/architecture/_index.md @@ -0,0 +1,52 @@ +--- +title: Architecture +weight: 1 +--- + +This page describes the architecture of a high-availability K3s server cluster and how it differs from a single-node server cluster. + +It also describes how agent nodes are registered with K3s servers. + +A server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command. + +This page covers the following topics: + +- [Single-server setup with an embedded database](#single-server-setup-with-an-embedded-db) +- [High-availability K3s server with an external database](#high-availability-k3s-server-with-an-external-db) + - [Fixed registration address for agent nodes](#fixed-registration-address-for-agent-nodes) +- [How agent node registration works](#how-agent-node-registration-works) + +# Single-server Setup with an Embedded DB + +The following diagram shows an example of a cluster that has a single-node K3s server with an embedded SQLite database. + +In this configuration, each agent node is registered to the same server node. A K3s user can manipulate Kubernetes resources by calling the K3s API on the server node. + +![Architecture]({{}}/img/rancher/k3s-single-node-server-architecture.svg) + +# High-Availability K3s Server with an External DB + +Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of: + +* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services +* An **external datastore** (as opposed to the embedded SQLite datastore used in single-server setups) + +![Architecture]({{< baseurl >}}/img/rancher/k3s-ha-architecture.svg) + +### Fixed Registration Address for Agent Nodes + +In the high-availability server configuration, each node must also register with the Kubernetes API by using a fixed registration address, as shown in the diagram below. + +After registration, the agent nodes establish a connection directly to one of the server nodes. + +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg) + +# How Agent Node Registration Works + +Agent nodes are registered with a websocket connection initiated by the `k3s agent` process, and the connection is maintained by a client-side load balancer running as part of the agent process. + +Agents will register with the server using the node cluster secret along with a randomly generated password for the node, stored at `/etc/rancher/node/password`. The server will store the passwords for individual nodes at `/var/lib/rancher/k3s/server/cred/node-passwd`, and any subsequent attempts must use the same password. + +If the `/etc/rancher/node` directory of an agent is removed, the password file should be recreated for the agent, or the entry removed from the server. + +A unique node ID can be appended to the hostname by launching K3s servers or agents using the `--with-node-id` flag. \ No newline at end of file diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index 65387bb5bc1..4173c2092c8 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -27,7 +27,11 @@ spec: ssl.enabled: "true" ``` -Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section: +Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. + +If you want to specify the namespace for the actual Helm release, you can do that using `targetNamespace` key under the `spec` directive, as shown in the configuration example below. + +Also note that besides `set`, you can also use `valuesContent` under the `spec` directive. And it's okay to use both of them: ```yaml apiVersion: helm.cattle.io/v1 @@ -51,8 +55,6 @@ spec: enabled: true ``` -Also note that besides `set` you can use `valuesContent` in the spec section. And it's okay to use both of them. - K3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions. Using the helm CRD diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index f631fe9c543..3a6fb03fa7f 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -8,12 +8,12 @@ This section contains instructions for installing K3s in various environments. P [Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) provides guidance on the options available to you when installing K3s. -[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to setup an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd. +[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to set up an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd. -[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to setup an HA K3s cluster that leverages a built-in distributed database. +[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to set up an HA K3s cluster that leverages a built-in distributed database. -[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to setup K3s in environments that do not have direct access to the Internet. +[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to set up K3s in environments that do not have direct access to the Internet. ### Uninstalling -If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). +If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation. The script is created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index e96967163c1..66564948a5d 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -5,11 +5,11 @@ weight: 60 In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server. -Installation Outline --------------------- -1. Prepare Images Directory -2. Create Registry YAML -3. Install K3s +# Installation Outline + +1. [Prepare Images Directory](#prepare-images-directory) +2. [Create Registry YAML](#create-registry-YAML) +3. [Install K3s](#install-k3s) ### Prepare Images Directory Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. diff --git a/content/k3s/latest/en/installation/datastore/_index.md b/content/k3s/latest/en/installation/datastore/_index.md index 7d5c2775871..63ef6baa32b 100644 --- a/content/k3s/latest/en/installation/datastore/_index.md +++ b/content/k3s/latest/en/installation/datastore/_index.md @@ -94,4 +94,4 @@ k3s server ``` ### Embedded DQLite for HA (Experimental) -K3s's use of DQLite is similar to its use of SQLite. It is simple to setup and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option. +K3s's use of DQLite is similar to its use of SQLite. It is simple to set up and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option. diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index ca6897ae48a..024e7a1714f 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -1,36 +1,35 @@ --- -title: "High Availability with an External DB" +title: High Availability with an External DB weight: 30 --- >**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. +This section describes how to install a high-availability K3s cluster with an external database. + Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of: * Two or more **server nodes** that will serve the Kubernetes API and run other control plane services -* An **external datastore** (as opposed to the embedded SQLite datastore used in single server setups) -* A **fixed registration address** placed in front of the server nodes to allow worker nodes to register with the cluster +* An **external datastore** (as opposed to the embedded SQLite datastore used in single-server setups) +* A **fixed registration address** that is placed in front of the server nodes to allow worker nodes to register with the cluster -The following diagram illustrates the above configuration: -![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg) - -In this architecture a server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command. +For more details on how these components work together, refer to the [architecture section.]({{}}/k3s/latest/en/architecture/#high-availability-with-an-external-db) Workers register through the fixed registration address, but after registration they establish a connection directly to one of the server nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process. -Installation Outline --------------------- +# Installation Outline + Setting up an HA cluster requires the following steps: -1. Create an external datastore -2. Launch server nodes -3. Configure fixed registration address -4. Join worker nodes +1. [Create an external datastore](#1-create-an-external-datastore) +2. [Launch server nodes](#2-launch-server-nodes) +3. [Configure the fixed registration address](#3-configure-the-fixed-registration-address) +4. [Join worker nodes](#4-join-worker-nodes) -### Create an External Datastore +### 1. Create an External Datastore You will first need to create an external datastore for the cluster. See the [Cluster Datastore Options]({{< baseurl >}}/k3s/latest/en/installation/datastore/) documentation for more details. -### Launch Server Nodes +### 2. Launch Server Nodes K3s requires two or more server nodes for this HA configuration. See the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) guide for minimum machine requirements. When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. Please see the [datastore configuration guide]({{< baseurl >}}/k3s/latest/en/installation/datastore/#external-datastore-configuration-parameters) for information on configuring this parameter. @@ -41,16 +40,16 @@ By default, server nodes will be schedulable and thus your workloads can get lau Once you've launched the `k3s server` process on all server nodes, you can ensure that the cluster has come up properly by checking that the nodes are in the Ready state with `k3s kubectl get nodes`. -### Configure the Fixed Registration Address -Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be setup using any number approaches, such as: +### 3. Configure the Fixed Registration Address +Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be set up using any number approaches, such as: * A layer-4 (TCP) load balancer * Round-robin DNS -* A virtual or elastic IP addresses +* Virtual or elastic IP addresses -This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your kubeconfig file to point to it instead of a specific node. +This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file to point to it instead of a specific node. -### Join Worker Nodes +### 4. Join Worker Nodes Joining worker nodes in an HA cluster is the same as joining worker nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use. ``` K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443 diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md index b9c3aa859e2..424cbd9ae73 100644 --- a/content/k3s/latest/en/installation/install-options/_index.md +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -1,9 +1,18 @@ --- -title: "Installation and Configuration Options" +title: "Installation Options" weight: 20 --- -### Installation script options +This page focuses on the options that can be used when you set up K3s for the first time: + +- [Installation script options](#installation-script-options) +- [Installing K3s from the binary](#installing-k3s-from-the-binary) +- [Registration options for the K3s server](#registration-options-for-the-k3s-server) +- [Registration options for the K3s agent](#registration-options-for-the-k3s-agent) + +For more advanced options, refer to [this page.]({{}}/k3s/latest/en/advanced) + +# Installation Script Options As mentioned in the [Quick-Start Guide]({{< baseurl >}}/k3s/latest/en/quick-start/), you can use the installation script available at https://get.k3s.io to install K3s as a service on systemd and openrc based systems. @@ -36,7 +45,7 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_BIN_DIR_READ_ONLY` - If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting INSTALL_K3S_SKIP_DOWNLOAD=true. + If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting `INSTALL_K3S_SKIP_DOWNLOAD=true`. - `INSTALL_K3S_SYSTEMD_DIR` @@ -44,7 +53,9 @@ When using this method to install K3s, the following environment variables can b - `INSTALL_K3S_EXEC` - Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior: + Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. + + The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior of registering a server without flannel: ```sh curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s - curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s - @@ -65,7 +76,8 @@ When using this method to install K3s, the following environment variables can b Environment variables which begin with `K3S_` will be preserved for the systemd and openrc services to use. Setting `K3S_URL` without explicitly setting an exec command will default the command to "agent". When running the agent `K3S_TOKEN` must also be set. -### Beyond the Installation Script +# Installing K3s from the Binary + As stated, the installation script is primarily concerned with configuring K3s to run as a service. If you choose to not use the script, you can run K3s simply by downloading the binary from our [release page](https://github.com/rancher/k3s/releases/latest), placing it on your path, and executing it. The K3s binary supports the following commands: Command | Description @@ -79,7 +91,7 @@ Command | Description The `k3s server` and `k3s agent` commands have additional configuration options that can be viewed with `k3s server --help` or `k3s agent --help`. For convenience, that help text is presented here: -### `k3s server` +# Registration Options for the K3s Server ``` NAME: k3s server - Run management server @@ -145,7 +157,7 @@ OPTIONS: --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] ``` -### `k3s agent` +# Registration Options for the K3s Agent ``` NAME: k3s agent - Run node agent @@ -181,3 +193,16 @@ OPTIONS: --no-flannel (deprecated) use --flannel-backend=none --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] ``` + +### Node Labels and Taints for Agents + +K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s commands. + +Below is an example showing how to add labels and a taint: +``` + --node-label foo=bar \ + --node-label hello=world \ + --node-taint key1=value1:NoExecute +``` + +If you want to change node labels and taints after node registration you should use `kubectl`. Refer to the official Kubernetes documentation for details on how to add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) and [node labels.](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node) \ No newline at end of file diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index 25e2a8b3119..0d3a0aaeee5 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -18,6 +18,8 @@ K3s should run on just about any flavor of Linux. However, K3s is tested on the * Ubuntu 18.04 (amd64) * Raspbian Buster (armhf) +> If you are using Alpine Linux, follow [these steps]({{}}/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup) for additional setup. + ## Hardware Hardware requirements scale based on the size of your deployments. Minimum recommendations are outlined here. diff --git a/content/k3s/latest/en/known-issues/_index.md b/content/k3s/latest/en/known-issues/_index.md index c79f84bc653..8107e8a7451 100644 --- a/content/k3s/latest/en/known-issues/_index.md +++ b/content/k3s/latest/en/known-issues/_index.md @@ -11,3 +11,7 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r **Iptables** If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues. + +**RootlessKit** + +Running K3s with RootlessKit is experimental and has several [known issues.]({{}}/k3s/latest/en/advanced/#known-issues-with-rootlesskit) diff --git a/content/k3s/latest/en/networking/_index.md b/content/k3s/latest/en/networking/_index.md index f7a37ed7354..d4f780d8dc5 100644 --- a/content/k3s/latest/en/networking/_index.md +++ b/content/k3s/latest/en/networking/_index.md @@ -12,27 +12,26 @@ Please reference the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installat CoreDNS ------- -CoreDNS is deployed on start of the agent, to disable run each server with the `--no-deploy coredns` option. +CoreDNS is deployed on start of the agent. To disable, run each server with the `--no-deploy coredns` option. -If you don't install CoreDNS you will need to install a cluster DNS provider yourself. +If you don't install CoreDNS, you will need to install a cluster DNS provider yourself. Traefik Ingress Controller -------------------------- -Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/configuration/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. +[Traefik](https://traefik.io/) is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. It simplifies networking complexity while designing, deploying, and running applications. + +Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/advanced/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. The Traefik ingress controller will use ports 80, 443, and 8080 on the host (i.e. these will not be usable for HostPort or NodePort). -You can tweak traefik to meet your needs by setting options in the traefik.yaml file. -Reference the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information. +You can tweak traefik to meet your needs by setting options in the traefik.yaml file. Refer to the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information. To disable it, start each server with the `--no-deploy traefik` option. Service Load Balancer --------------------- -K3s includes a basic service load balancer that uses available host ports. If you try to create -a load balancer that listens on port 80, for example, it will try to find a free host in the cluster -for port 80. If no port is available the load balancer will stay in Pending. +K3s includes a basic service load balancer that uses available host ports. If you try to create a load balancer that listens on port 80, for example, it will try to find a free host in the cluster for port 80. If no port is available, the load balancer will stay in Pending. -To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. +To disable the embedded load balancer, run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. \ No newline at end of file diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index e4e8156bf34..8ec057e36a0 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -3,7 +3,9 @@ title: "Quick-Start Guide" weight: 10 --- ->**Note:** This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up. +This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up. + +For information on how K3s components work together, refer to the [architecture section.]({{}}/k3s/latest/en/architecture/#high-availability-with-an-external-db) > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). @@ -18,7 +20,7 @@ After running this installation: * The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed * Additional utilities will be installed, including `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh` -* A kubeconfig file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it +* A [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it To install on worker nodes and add them to the cluster, run the installation script with the `K3S_URL` and `K3S_TOKEN` environment variables. Here is an example showing how to join a worker node: diff --git a/content/k3s/latest/en/storage/_index.md b/content/k3s/latest/en/storage/_index.md index a567420905b..790cbd04e38 100644 --- a/content/k3s/latest/en/storage/_index.md +++ b/content/k3s/latest/en/storage/_index.md @@ -5,7 +5,11 @@ weight: 30 When deploying an application that needs to retain data, you’ll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the application’s pod fails. -# Local Storage Provider +A persistent volume (PV) is a piece of storage in the Kubernetes cluster, while a persistent volume claim (PVC) is a request for storage. For details on how PVs and PVCs work, refer to the official Kubernetes documentation on [storage.](https://kubernetes.io/docs/concepts/storage/volumes/) + +This page describes how to set up persistent storage with a local storage provider, or with [Longhorn.](#setting-up-longhorn) + +# Setting up the Local Storage Provider K3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage). Create a hostPath backed persistent volume claim and a pod to utilize it: @@ -51,19 +55,33 @@ spec: claimName: local-path-pvc ``` -Apply the yaml `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` +Apply the yaml: -Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. +``` +kubectl create -f pvc.yaml +kubectl create -f pod.yaml +``` -# Longhorn +Confirm the PV and PVC are created: + +``` +kubectl get pv +kubectl get pvc +``` + +The status should be Bound for each. + +# Setting up Longhorn [comment]: <> (pending change - longhorn may support arm64 and armhf in the future.) > **Note:** At this time Longhorn only supports amd64. -K3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md). +K3s supports [Longhorn](https://github.com/longhorn/longhorn). Longhorn is an open-source distributed block storage system for Kubernetes. -Apply the longhorn.yaml to install Longhorn. +Below we cover a simple example. For more information, refer to the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md). + +Apply the longhorn.yaml to install Longhorn: ``` kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml @@ -71,13 +89,18 @@ kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/depl Longhorn will be installed in the namespace `longhorn-system`. -Before we create a PVC, we will create a storage class for longhorn with this yaml. +Before we create a PVC, we will create a storage class for Longhorn with this yaml: ``` kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/storageclass.yaml ``` -Now, apply the following yaml to create the PVC and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` +Apply the yaml to create the PVC and pod: + +``` +kubectl create -f pvc.yaml +kubectl create -f pod.yaml +``` ### pvc.yaml @@ -119,4 +142,11 @@ spec: claimName: longhorn-volv-pvc ``` -Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. +Confirm the PV and PVC are created: + +``` +kubectl get pv +kubectl get pvc +``` + +The status should be Bound for each. diff --git a/content/k3s/latest/en/upgrades/_index.md b/content/k3s/latest/en/upgrades/_index.md index 62d3dd47cd8..3ce3a0591a3 100644 --- a/content/k3s/latest/en/upgrades/_index.md +++ b/content/k3s/latest/en/upgrades/_index.md @@ -3,7 +3,11 @@ title: "Upgrades" weight: 25 --- ->**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes. +You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version. + +>**Note:** When upgrading, upgrade server nodes first one at a time, then any worker nodes. + +### Upgrade K3s Using the Installation Script To upgrade K3s from an older version you can re-run the installation script using the same flags, for example: @@ -17,6 +21,8 @@ If you want to upgrade to specific version you can run the following command: curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - ``` +### Manually Upgrade K3s Using the Binary + Or to manually upgrade K3s: 1. Download the desired version of K3s from [releases](https://github.com/rancher/k3s/releases/latest) @@ -24,6 +30,8 @@ Or to manually upgrade K3s: 3. Stop the old version 4. Start the new version +### Restarting K3s + Restarting K3s is supported by the installation script for systemd and openrc. To restart manually for systemd use: ```sh diff --git a/static/img/rancher/k3s-ha-architecture.svg b/static/img/rancher/k3s-ha-architecture.svg new file mode 100644 index 00000000000..70493a11a0c --- /dev/null +++ b/static/img/rancher/k3s-ha-architecture.svg @@ -0,0 +1,3 @@ + + +
External Traffic
External Traffic
kubectl get pods
[Not supported by viewer]
K3s User
K3s User
Load
Balancer
[Not supported by viewer]
K3s
Server
[Not supported by viewer]

Server
Node
[Not supported by viewer]

Server
Node
[Not supported by viewer]

Server
Node
[Not supported by viewer]
External
Database
[Not supported by viewer]
K3s
Agents
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Load
Balancer
[Not supported by viewer]
also called worker nodes
also called worker nodes
Example configuration
for nodes running your apps and services
[Not supported by viewer] \ No newline at end of file diff --git a/static/img/rancher/k3s-single-node-server-architecture.svg b/static/img/rancher/k3s-single-node-server-architecture.svg new file mode 100644 index 00000000000..00bf13979ec --- /dev/null +++ b/static/img/rancher/k3s-single-node-server-architecture.svg @@ -0,0 +1,3 @@ + + +
External Traffic
External Traffic
K3s
Server
[Not supported by viewer]

Server
Node
[Not supported by viewer]
Embedded
SQLite
Database
[Not supported by viewer]
K3s
Agents
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Agent
Node
[Not supported by viewer]
Load
Balancer
[Not supported by viewer]
also called worker nodes
also called worker nodes
Example configuration
for nodes running your apps and services
[Not supported by viewer]
kubectl get pods
[Not supported by viewer]
K3s User
K3s User \ No newline at end of file