diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index 02dae30dcfd..85d522e5e37 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -1,6 +1,6 @@ --- -title: "k3s - 5 less than k8s" -shortTitle: k3s +title: "K3s - 5 less than K8s" +shortTitle: K3s date: 2019-02-05T09:52:46-07:00 name: "menu" --- @@ -18,16 +18,14 @@ Great for: What is this? --- -K3s is intended to be a fully compliant Kubernetes distribution with the following changes: +K3s is a fully compliant Kubernetes distribution with the following enhancements: -1. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced - with out of tree addons. -2. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. -3. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. -4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. -5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). K3s packages required dependencies +* An embedded SQLite database has replaced etcd as the default datastore. External datastores such as PostgreSQL, MySQL, and etcd are also supported. +* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a helm controller, and the Traefik ingress controller. +* Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates. +* In-tree cloud providers and storage plugins have been removed. +* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including: * containerd * Flannel * CoreDNS - * CNI * Host utilities (iptables, socat, etc) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index b1c54edc481..f631fe9c543 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -5,20 +5,15 @@ weight: 20 This section contains instructions for installing K3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing K3s. -### Installation Options +[Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) provides guidance on the options available to you when installing K3s. -* [Single Server Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install K3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. +[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to setup an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd. -* [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) +[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to setup an HA K3s cluster that leverages a built-in distributed database. - Install K3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any downtime. - -* [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) - - Install K3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any downtime. +[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to setup K3s in environments that do not have direct access to the Internet. ### Uninstalling -If you installed K3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). +If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index d8b26320885..f919b3768ce 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -1,6 +1,6 @@ --- title: "Air-Gap Install" -weight: 40 +weight: 60 --- In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server. @@ -12,7 +12,7 @@ Installation Outline 3. Install K3s ### Prepare Images Directory -Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. +Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. Place the tar file in the `images` directory before starting K3s on each node, for example: @@ -73,6 +73,5 @@ The node-token is on the server at `/var/lib/rancher/k3s/server/node-token` Upgrading an air-gap environment can be accomplished in the following manner: 1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file. -2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same enviornment variables. +2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same environment variables. 3. Restart the K3s service (if not restarted automatically by installer). - diff --git a/content/k3s/latest/en/installation/datastore/_index.md b/content/k3s/latest/en/installation/datastore/_index.md new file mode 100644 index 00000000000..6e5ac992ab3 --- /dev/null +++ b/content/k3s/latest/en/installation/datastore/_index.md @@ -0,0 +1,97 @@ +--- +title: "Cluster Datastore Options" +weight: 50 +--- + +The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to Kubernetes operators. The available datastore options allow you to select a datastore that best fits your use case. For example: + +* If your team doesn't have expertise in operating etcd, you can choose an enterprise-grade SQL database like MySQL or PostgreSQL +* If you need to run a simple, short-lived cluster in your CI/CD environment, you can use the embedded SQLite database +* If you wish to deploy Kubernetes on the edge and require a highly available solution but can't afford the operational overhead of managing a database at the edge, you can use K3s's embedded HA datastore built on top of DQLite (currently experimental) + +K3s supports the following datastore options: + +* Embedded [SQLite](https://www.sqlite.org/index.html) +* [PostgreSQL](https://www.postgresql.org/) (certified against versions 10.7 and 11.5) +* [MySQL](https://www.mysql.com/) (certified against version 5.7) +* [etcd](https://etcd.io/) (certified against version 3.3.15) +* Embedded [DQLite](https://dqlite.io/) for High Availability (experimental) + +### External Datastore Configuration Parameters +If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you must set the `datastore-endpoint` parameter so that K3s knows how to connect to it. You may also specify parameters to configure the authentication and encryption of the connection. The below table summarizes these parameters, which can be passed as either CLI flags or environment variables. + + CLI Flag | Environment Variable | Description + ------------|-------------|------------------ + `--datastore-endpoint` | `K3S_DATASTORE_ENDPOINT` | Specify a PostgresSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below. + `--datastore-cafile` | `K3S_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate. | +| `--datastore-certfile` | `K3S_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. | +| `--datastore-keyfile` | `K3S_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. | + +As a best practice we recommend setting these parameters as environment variables rather than command line arguments so that your database credentials or other sensitive information aren't exposed as part of the process info. + +### Datastore Endpoint Format and Functionality +As mentioned, the format of the value passed to the `datastore-endpoint` parameter is dependent upon the datastore backend. The following details this format and functionality for each supported external datastore. + +{{% tabs %}} +{{% tab "PostgreSQL" %}} + +In its most common form, the datastore-endpoint parameter for PostgreSQL has the following format: + +`postgres://username:password@hostname:port/database-name` + +More advanced configuration parameters are available. For more information on these, please see https://godoc.org/github.com/lib/pq. + +If you specify a database name and it does not exist, the server will attempt to create it. + +If you only supply `postgres://` as the endpoint, K3s will attempt to do the following: + +* Connect to localhost using `postgres` as the username and password +* Create a database named `kubernetes` + + +{{% /tab %}} +{{% tab "MySQL" %}} + +In its most common form, the `datastore-endpoint` parameter for MySQL has the following format: + +`mysql://username:password@tcp(hostname:3306)/database-name` + +More advanced configuration parameters are available. For more information on these, please see https://github.com/go-sql-driver/mysql#dsn-data-source-name + +Note that due to a [known issue](https://github.com/rancher/k3s/issues/1093) in K3s, you cannot set the `tls` parameter. TLS communication is supported, but you cannot, for example, set this parameter to "skip-verify" to cause K3s to skip certificate verification. + +If you specify a database name and it does not exist, the server will attempt to create it. + +If you only supply `mysql://` as the endpoint, K3s will attempt to do the following: + +* Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user and no password +* Create a database with the name `kubernetes` + + +{{% /tab %}} +{{% tab "etcd" %}} + +In its most common form, the `datastore-endpoint` parameter for etcd has the following format: + +`https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379` + +The above assumes a typical three node etcd cluster. The parameter can accept one more comma separated etcd URLs. + +{{% /tab %}} +{{% /tabs %}} + +
Based on the above, the following example command could be used to launch a server instance that connects to a PostgresSQL database named k3s-db: +``` +K3S_DATASTORE_ENDPOINT='postgres://username:password@hostname:5432/k3s-db' k3s server +``` + +And the following example could be used to connect to a MySQL database using client certificate authentication: +``` +K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/k3s-db' \ +K3S_DATASTORE_CERTFILE='/path/to/client.crt' \ +K3S_DATASTORE_KEYFILE='/path/to/client.key' \ +k3s server +``` + +### Embedded DQLite for HA (Experimental) +K3s's use of DQLite is similar to its use of SQLite. It is simple to setup and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option. diff --git a/content/k3s/latest/en/installation/ha-embedded/_index.md b/content/k3s/latest/en/installation/ha-embedded/_index.md new file mode 100644 index 00000000000..58526b65280 --- /dev/null +++ b/content/k3s/latest/en/installation/ha-embedded/_index.md @@ -0,0 +1,22 @@ +--- +title: "High Availability with Embedded DB (Experimental)" +weight: 40 +--- + +As of v1.0.0, K3s is previewing support for running a highly available control plane without the need for an external database. This means there is no need to manage an external etcd or SQL datastore in order to run a reliable production-grade setup. While this feature is currently experimental, we expect it to be the primary architecture for running HA K3s clusters in the future. + +This architecture is achieved by embedding a dqlite database within the K3s server process. DQLite is short for "distributed SQLite." According to https://dqlite.io, it is "*a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices.*" This makes it a natural fit for K3s. + +To run K3s in this mode, you must have an odd number of server nodes. We recommend starting with three nodes. + +To get started, first launch a server node with the `cluster-init` flag to enable clustering and a token that will be used as a shared secret to join additional servers to the cluster. +``` +K3S_TOKEN=SECRET k3s server --cluster-init +``` + +After launching the first server, join the second and third servers to the cluster using the shared secret: +``` +K3S_TOKEN=SECRET k3s server --server https://:6443 +``` + +Now you have a highly available control plane. Joining additional worker nodes to the cluster follows the same procedure as a single server cluster. diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 5b5364420df..81adf97d1a7 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -1,150 +1,57 @@ --- -title: "High Availability (HA) Install" +title: "High Availability with an External DB" weight: 30 --- >**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. ->k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. +Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of: -For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-server cluster with k3s with an external database for the cluster datastore. +* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services +* An **external datastore** (as opposed to the embedded SQLite datastore used in single server setups) +* A **fixed registration address** placed in front of the server nodes to allow worker nodes to register with the cluster -# Recommended Architecture -![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v5.svg) -This image depicts a k3s HA install with two load balancers: +The following diagram illustrates the above configuration: +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg) -* A load balancer to expose workloads to external traffic -* A load balancer to expose the Kubernetes API for clients such as kubectl and to expose a stable k3s worker registration endpoint +In this architecture a server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command. -The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the server nodepool will be auto scaling and thus server nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. +Workers register through the fixed registration address, but after registration they establish a connection directly to one of the sever nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process. Installation Outline -------------------- -1. Create Database for Cluster Datastore (PostgreSQL, MySQL, or etcd) -2. Create Server Nodes -3. Join Worker Nodes +Setting up an HA cluster requires the following steps: -### Create Database for Cluster Datastore -You will first need to create the database for the backend (cluster datastore). k3s must have a single endpoint it can reach to talk to the database. +1. Create an external datastore +2. Launch server nodes +3. Configure fixed registration address +4. Join worker nodes -Here is a list of supported backend databases as of our v1.0.0 release: +### Create an External Datastore +You will first need to create an external datastore for the cluster. See the [Cluster Datastore Options]({{< baseurl >}}/k3s/latest/en/installation/datastore/) documentation for more details. -* PostgreSQL 10.7 and 11.5 -* MySQL 5.7 -* etcd 3.3.15 +### Launch Server Nodes +K3s requires two or more server nodes for this HA configuration. See the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) guide for minimum machine requirements. -### Create Server Nodes -Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. -The sections below will indicate what you need to run (depending on the type of database) to bring the server nodes up. +When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. Please see the [datastore configuration guide]({{< baseurl >}}/k3s/latest/en/installation/datastore/#external-datastore-configuration-parameters) for information on configuring this parameter. ->**Note:** You may wish to taint the server nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. +> **Note:** The same installation options available to single-server installs are also available for HA installs. For more details, see the [Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) documentation. ->If your server node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. +By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use taints. The `node-taint` parameter will allow you to configure nodes with taints, for example `--node-taint k3s-controlplane=true:NoExecute`. +Once you've launched the `k3s server` process on all server nodes, you can ensure that the cluster has come up properly by checking that the nodes are in the Ready state with `k3s kubectl get nodes`. -{{% tabs %}} -{{% tab "PostgreSQL" %}} +### Configure the Fixed Registration Address +Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be setup using any number approaches, such as: ->**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='postgres://'` then the server will attempt to connect with the following: +* A layer-4 (TCP) load balancer +* Round-robin DNS +* A virtual or elastic IP addresses -> * Connect to `localhost` using the `postgres` user. -> * Use `postgres` as the password. -> * Create a database with the name `kubernetes`. - -> If you specify a database name and it does not exist, then the server will attempt to create it. -> For more information about the Postgres driver DSN, please refer to https://godoc.org/github.com/lib/pq - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile postgres.crt \ - --storage-keyfile postgres.key - -``` - -The above command will use these certificates to generate the TLS config to communicate with PostgreSQL securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% tab "MySQL" %}} - ->**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='mysql://'` then the server will attempt to connect with the following: - -> * Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user. -> * Use no password. -> * Create a database with the name `kubernetes`. - -> If you specify a database name and it does not exist, then the server will attempt to create it. -> For more information about the MySQL driver DSN, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile mysql.crt \ - --storage-keyfile mysql.key - - -``` -The above command will use these certificates to generate the TLS config to communicate with MySQL securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% tab "etcd" %}} - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostname:2379" \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile etcd.crt \ - --storage-keyfile etcd.key -``` -The above command will use these certificates to generate the TLS config to communicate with etcd securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='http://hostname:2379' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% /tabs %}} - -Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes` +This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your kubeconfig file to point to it instead of a specific node. ### Join Worker Nodes -Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s). - -Run the following command to join a worker node to the server nodes. We are leveraging the cluster-secret here. Since our server nodes were set up to use this, so too must any agent nodes. - +Joining worker nodes in an HA cluster is the same as joining worker nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use. ``` -curl -sfL https://get.k3s.io | K3S_URL=https://:6443 K3S_CLUSTER_SECRET='mysecret' sh - +K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443 ``` - -Provide the IP or DNS in place of ``. This can be any one server node. k3s automatically handles load balancing the server nodes. - -Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. - diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md new file mode 100644 index 00000000000..b9c3aa859e2 --- /dev/null +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -0,0 +1,183 @@ +--- +title: "Installation and Configuration Options" +weight: 20 +--- + +### Installation script options + +As mentioned in the [Quick-Start Guide]({{< baseurl >}}/k3s/latest/en/quick-start/), you can use the installation script available at https://get.k3s.io to install K3s as a service on systemd and openrc based systems. + +The simplest form of this command is as follows: +```sh +curl -sfL https://get.k3s.io | sh - +``` + +When using this method to install K3s, the following environment variables can be used to configure the installation: + +- `INSTALL_K3S_SKIP_DOWNLOAD` + + If set to true will not download K3s hash or binary. + +- `INSTALL_K3S_SYMLINK` + + If set to 'skip' will not create symlinks, 'force' will overwrite, default will symlink if command does not exist in path. + +- `INSTALL_K3S_SKIP_START` + + If set to true will not start K3s service. + +- `INSTALL_K3S_VERSION` + + Version of K3s to download from github. Will attempt to download the latest version if not specified. + +- `INSTALL_K3S_BIN_DIR` + + Directory to install K3s binary, links, and uninstall script to, or use `/usr/local/bin` as the default. + +- `INSTALL_K3S_BIN_DIR_READ_ONLY` + + If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting INSTALL_K3S_SKIP_DOWNLOAD=true. + +- `INSTALL_K3S_SYSTEMD_DIR` + + Directory to install systemd service and environment files to, or use `/etc/systemd/system` as the default. + +- `INSTALL_K3S_EXEC` + + Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior: + ```sh + curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s - + curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s - + curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel + curl ... | sh -s - server --no-flannel + curl ... | sh -s - --no-flannel + ``` + + - `INSTALL_K3S_NAME` + + Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'. + + - `INSTALL_K3S_TYPE` + + Type of systemd service to create, will default from the K3s exec command if not specified. + + +Environment variables which begin with `K3S_` will be preserved for the systemd and openrc services to use. Setting `K3S_URL` without explicitly setting an exec command will default the command to "agent". When running the agent `K3S_TOKEN` must also be set. + + +### Beyond the Installation Script +As stated, the installation script is primarily concerned with configuring K3s to run as a service. If you choose to not use the script, you can run K3s simply by downloading the binary from our [release page](https://github.com/rancher/k3s/releases/latest), placing it on your path, and executing it. The K3s binary supports the following commands: + +Command | Description +--------|------------------ +`k3s server` | Run the K3s management server, which will also launch Kubernetes control plane components such as the API server, controller-manager, and scheduler. +`k3s agent` | Run the K3s node agent. This will cause K3s to run as a worker node, launching the Kubernetes node services `kubelet` and `kube-proxy`. +`k3s kubectl` | Run an embedded [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) CLI. If the `KUBECONFIG` environment variable is not set, this will automatically attempt to use the config file that is created at `/etc/rancher/k3s/k3s.yaml` when launching a K3s server node. +`k3s crictl` | Run an embedded [crictl](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md). This is a CLI for interacting with Kubernetes's container runtime interface (CRI). Useful for debugging. +`k3s ctr` | Run an embedded [ctr](https://github.com/projectatomic/containerd/blob/master/docs/cli.md). This is a CLI for containerd, the container daemon used by K3s. Useful for debugging. +`k3s help` | Shows a list of commands or help for one command + +The `k3s server` and `k3s agent` commands have additional configuration options that can be viewed with `k3s server --help` or `k3s agent --help`. For convenience, that help text is presented here: + +### `k3s server` +``` +NAME: + k3s server - Run management server + +USAGE: + k3s server [OPTIONS] + +OPTIONS: + -v value (logging) Number for the log level verbosity (default: 0) + --vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging + --log value, -l value (logging) Log to file + --alsologtostderr (logging) Log to standard error as well as file (if set) + --bind-address value (listener) k3s bind address (default: 0.0.0.0) + --https-listen-port value (listener) HTTPS listen port (default: 6443) + --advertise-address value (listener) IP address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip) + --advertise-port value (listener) Port that apiserver uses to advertise to members of the cluster (default: listen-port) (default: 0) + --tls-san value (listener) Add additional hostname or IP as a Subject Alternative Name in the TLS cert + --data-dir value, -d value (data) Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root + --cluster-cidr value (networking) Network CIDR to use for pod IPs (default: "10.42.0.0/16") + --service-cidr value (networking) Network CIDR to use for services IPs (default: "10.43.0.0/16") + --cluster-dns value (networking) Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10) + --cluster-domain value (networking) Cluster Domain (default: "cluster.local") + --flannel-backend value (networking) One of 'none', 'vxlan', 'ipsec', or 'flannel' (default: "vxlan") + --token value, -t value (cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN] + --token-file value (cluster) File containing the cluster-secret/token [$K3S_TOKEN_FILE] + --write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT] + --write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE] + --kube-apiserver-arg value (flags) Customized flag for kube-apiserver process + --kube-scheduler-arg value (flags) Customized flag for kube-scheduler process + --kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process + --kube-cloud-controller-manager-arg value (flags) Customized flag for kube-cloud-controller-manager process + --datastore-endpoint value (db) Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$K3S_DATASTORE_ENDPOINT] + --datastore-cafile value (db) TLS Certificate Authority file used to secure datastore backend communication [$K3S_DATASTORE_CAFILE] + --datastore-certfile value (db) TLS certification file used to secure datastore backend communication [$K3S_DATASTORE_CERTFILE] + --datastore-keyfile value (db) TLS key file used to secure datastore backend communication [$K3S_DATASTORE_KEYFILE] + --default-local-storage-path value (storage) Default local storage path for local provisioner storage class + --no-deploy value (components) Do not deploy packaged components (valid items: coredns, servicelb, traefik, local-storage, metrics-server) + --disable-scheduler (components) Disable Kubernetes default scheduler + --disable-cloud-controller (components) Disable k3s default cloud controller manager + --disable-network-policy (components) Disable k3s default network policy controller + --node-name value (agent/node) Node name [$K3S_NODE_NAME] + --with-node-id (agent/node) Append id to node name + --node-label value (agent/node) Registering kubelet with set of labels + --node-taint value (agent/node) Registering kubelet with set of taints + --docker (agent/runtime) Use docker instead of containerd + --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation + --pause-image value (agent/runtime) Customized pause image for containerd sandbox + --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") + --node-ip value, -i value (agent/networking) IP address to advertise for node + --node-external-ip value (agent/networking) External IP address to advertise for node + --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] + --flannel-iface value (agent/networking) Override default flannel interface + --flannel-conf value (agent/networking) Override default flannel config file + --kubelet-arg value (agent/flags) Customized flag for kubelet process + --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process + --rootless (experimental) Run rootless + --agent-token value (experimental/cluster) Shared secret used to join agents to the cluster, but not servers [$K3S_AGENT_TOKEN] + --agent-token-file value (experimental/cluster) File containing the agent secret [$K3S_AGENT_TOKEN_FILE] + --server value, -s value (experimental/cluster) Server to connect to, used to join a cluster [$K3S_URL] + --cluster-init (experimental/cluster) Initialize new cluster master [$K3S_CLUSTER_INIT] + --cluster-reset (experimental/cluster) Forget all peers and become a single cluster new cluster master [$K3S_CLUSTER_RESET] + --no-flannel (deprecated) use --flannel-backend=none + --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] +``` + +### `k3s agent` +``` +NAME: + k3s agent - Run node agent + +USAGE: + k3s agent [OPTIONS] + +OPTIONS: + -v value (logging) Number for the log level verbosity (default: 0) + --vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging + --log value, -l value (logging) Log to file + --alsologtostderr (logging) Log to standard error as well as file (if set) + --token value, -t value (cluster) Token to use for authentication [$K3S_TOKEN] + --token-file value (cluster) Token file to use for authentication [$K3S_TOKEN_FILE] + --server value, -s value (cluster) Server to connect to [$K3S_URL] + --data-dir value, -d value (agent/data) Folder to hold state (default: "/var/lib/rancher/k3s") + --node-name value (agent/node) Node name [$K3S_NODE_NAME] + --with-node-id (agent/node) Append id to node name + --node-label value (agent/node) Registering kubelet with set of labels + --node-taint value (agent/node) Registering kubelet with set of taints + --docker (agent/runtime) Use docker instead of containerd + --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation + --pause-image value (agent/runtime) Customized pause image for containerd sandbox + --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") + --node-ip value, -i value (agent/networking) IP address to advertise for node + --node-external-ip value (agent/networking) External IP address to advertise for node + --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] + --flannel-iface value (agent/networking) Override default flannel interface + --flannel-conf value (agent/networking) Override default flannel config file + --kubelet-arg value (agent/flags) Customized flag for kubelet process + --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process + --rootless (experimental) Run rootless + --no-flannel (deprecated) use --flannel-backend=none + --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] +``` diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md deleted file mode 100644 index 17ea504c0d1..00000000000 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: "Single Server Install" -weight: 20 ---- - ->**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) K3s cluster. A High-Availability (HA) K3s cluster is recommended for production environments that cannot tolerate down time. - -Installation ------------- - -K3s is easy to install. To install the latest version, simply run: - -```sh -curl -sfL https://get.k3s.io | sh - -``` - -The install script will attempt to download the latest release. To specify a version for download we can use the `INSTALL_K3S_VERSION` environment variable. For example: -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - -``` - -To install with a specific flag we can use the `INSTALL_K3S_EXEC` -environment variable. For example: -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-flannel" sh - -``` - -The installer can also be run without performing downloads by setting `INSTALL_K3S_SKIP_DOWNLOAD=true`. For example: -```sh -curl -sfL https://github.com/rancher/k3s/releases/download/vX.Y.Z/k3s -o /usr/local/bin/k3s -chmod 0755 /usr/local/bin/k3s - -curl -sfL https://get.k3s.io -o install-k3s.sh -chmod 0755 install-k3s.sh - -export INSTALL_K3S_SKIP_DOWNLOAD=true -./install-k3s.sh -``` - -The full help text for the install script environment variables are as follows: - - `K3S_*` - - Environment variables which begin with `K3S_` will be preserved for the - systemd service to use. Setting `K3S_URL` without explicitly setting - a systemd exec command will default the command to "agent", and we - enforce that `K3S_TOKEN` or `K3S_CLUSTER_SECRET` is also set. - - - `INSTALL_K3S_SKIP_DOWNLOAD` - - If set to true will not download K3s hash or binary. - - - `INSTALL_K3S_SYMLINK` - - If set to 'skip' will not create symlinks, 'force' will overwrite, - default will symlink if command does not exist in path. - - - `INSTALL_K3S_VERSION` - - Version of K3s to download from github. Will attempt to download the - latest version if not specified. - - - `INSTALL_K3S_BIN_DIR` - - Directory to install K3s binary, links, and uninstall script to, or use - /usr/local/bin as the default - - - `INSTALL_K3S_SYSTEMD_DIR` - - Directory to install systemd service and environment files to, or use - /etc/systemd/system as the default - - - `INSTALL_K3S_EXEC` or script arguments - - Command with flags to use for launching K3s in the systemd service, if - the command is not specified will default to "agent" if `K3S_URL` is set - or "server" if not. The final systemd command resolves to a combination - of EXEC and script args ($@). - - The following commands result in the same behavior: - ```sh - curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s - - curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s - - curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel - curl ... | sh -s - server --no-flannel - curl ... | sh -s - --no-flannel - ``` - - - `INSTALL_K3S_NAME` - - Name of systemd service to create, will default from the K3s exec command - if not specified. If specified the name will be prefixed with 'k3s-'. - - - `INSTALL_K3S_TYPE` - - Type of systemd service to create, will default from the K3s exec command - if not specified. - diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index fa3b36c2b72..e4e8156bf34 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -3,26 +3,28 @@ title: "Quick-Start Guide" weight: 10 --- ->**Note:** The intent of this guide is to quickly launch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. -The [installation options](../installation) section covers in greater detail how K3s can be set up. +>**Note:** This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). Install Script -------------- -The K3s `install.sh` script provides a convenient way for installing to systemd or openrc, -to install K3s as a service just run: +K3s provides an installation script that is a convenient way to install it as a service on systemd or openrc based systems. This script is available at https://get.k3s.io. To install K3s using this method, just run: ```bash curl -sfL https://get.k3s.io | sh - ``` -A kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` and the service is automatically started or restarted. -The install script will install K3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`. +After running this installation: -To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with the `K3S_TOKEN` environment variable. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node: +* The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed +* Additional utilities will be installed, including `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh` +* A kubeconfig file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it + +To install on worker nodes and add them to the cluster, run the installation script with the `K3S_URL` and `K3S_TOKEN` environment variables. Here is an example showing how to join a worker node: ```bash curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh - ``` +Setting the `K3S_URL` parameter causes K3s to run in worker mode. The K3s agent will register with the K3s server listening at the supplied URL. The value to use for `K3S_TOKEN` is stored at `/var/lib/rancher/k3s/server/node-token` on your server node. Note: Each machine must have a unique hostname. If your machines do not have unique hostnames, pass the `K3S_NODE_NAME` environment variable and provide a value with a valid and unique hostname for each node. diff --git a/static/img/k3s/k3s-production-setup-v5.svg b/static/img/k3s/k3s-production-setup-v5.svg deleted file mode 100644 index cb50963c422..00000000000 --- a/static/img/k3s/k3s-production-setup-v5.svg +++ /dev/null @@ -1,2018 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/static/img/k3s/k3s-production-setup.svg b/static/img/k3s/k3s-production-setup.svg new file mode 100644 index 00000000000..2d132eb9566 --- /dev/null +++ b/static/img/k3s/k3s-production-setup.svg @@ -0,0 +1,1176 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +