diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 4d60414d649..051965c81b3 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -7,9 +7,9 @@ This section contains instructions for installing k3s in various environments. P ### Installation Options -* [Single Master Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) +* [Single Server Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install k3s on a single Linux host. Single master installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. + Install k3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 7d6f5920a9b..e2ec3a3701d 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -1,12 +1,13 @@ --- -title: "High Availability (HA) Install (Experimental)" +title: "High Availability (HA) Install" weight: 30 --- ->**Important:** High-Availability (HA) was introduced in the v0.10.0 release of k3s and is _experimental_. Our v1.0 release plans to support HA in production environments. HA should currently only be used for testing purposes in non-production environments. ->**Note:** k3s does not utilize etcd by default so only a 2-node cluster is needed for HA at a minimum. The following will guide you through setting up a 2-node cluster with PostgreSQL. You could optionally add one or more nodes for additional redundancy. In the future we plan to add support for additional database providers. +>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. HA may work on older releases but has limited support. Older versions of k3s bring up server nodes slightly differently. Reference [this]({{< baseurl >}}/k3s/latest/en/installation/older-installations/) page for more information. -For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7 and 11.5 +>k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. + +For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-server cluster with k3s with an external database for the cluster datastore. # Recommended Architecture ![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v5.svg) @@ -15,148 +16,135 @@ This image depicts a k3s HA install with two load balancers: * A load balancer to expose workloads to external traffic * A load balancer to expose the Kubernetes API for clients such as kubectl and to expose a stable k3s worker registration endpoint -The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. +The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the server nodepool will be auto scaling and thus server nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. Installation Outline -------------------- -1. Create backend database (PostgreSQL) -2. Create master nodes -3. Join worker nodes +1. Create Database for Cluster Datastore (PostgreSQL, MySQL, or etcd) +2. Create Server Nodes +3. Join Worker Nodes ### Create Database for Cluster Datastore -The first step for setting up High Availability (HA) is to create the database for the backend (cluster datastore). As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. +You will first need to create the database for the backend (cluster datastore). k3s must have a single endpoint it can reach to talk to the database. -### Create Master Nodes +Here is a list of supported backend databases as of our v1.0.0 release: + +* PostgreSQL 10.7 and 11.5 +* MySQL 5.7 +* etcd 3.3.15 + +### Create Server Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. +The sections below will indicate what you need to run (depending on the type of database) to bring the server nodes up. -On the first machine, run the following command to install k3s and connect it to the database. +>**Note:** You may wish to taint the server nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. ->**Note:** You may wish to taint the master nodes. They will run the kubelet by default and be scheduleable. You can only add node labels and taints during the install process. If you wish to do this, use the `--node-taint` flag. For example `--node-taint key1=value1:NoExecute` the following examples do not include this flag. +>If your server node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. ->If your master node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The following examples include this optional flag. + +{{% tabs %}} +{{% tab "PostgreSQL" %}} + +>**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='postgres://'` then the server will attempt to connect with the following: + +> * Connect to `localhost` using the `postgres` user. +> * Use `postgres` as the password. +> * Create a database with the name `kubernetes`. + +> If you specify a database name and it does not exist, then the server will attempt to create it. +> For more information about the Postgres driver DSN, please refer to https://godoc.org/github.com/lib/pq + +### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. ``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' --bootstrap-save -``` - -The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. - -Note: You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. - -On the second machine, run the following command. Since we ran the first node with the `--bootstrap-save` flag the second and any additional machines will now automatically bootstrap HA. +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile postgres.crt \ + --storage-keyfile postgres.key ``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' + +The above command will use these certificates to generate the TLS config to communicate with PostgreSQL securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + ``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% tab "MySQL" %}} + +>**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='mysql://'` then the server will attempt to connect with the following: + +> * Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user. +> * Use no password. +> * Create a database with the name `kubernetes`. + +> If you specify a database name and it does not exist, then the server will attempt to create it. +> For more information about the MySQL driver DSN, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name + +### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile mysql.crt \ + --storage-keyfile mysql.key + + +``` +The above command will use these certificates to generate the TLS config to communicate with MySQL securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% tab "etcd" %}} + +### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostname:2379" \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile etcd.crt \ + --storage-keyfile etcd.key +``` +The above command will use these certificates to generate the TLS config to communicate with etcd securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='http://hostname:2379' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% /tabs %}} Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes` ### Join Worker Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s). -Run the following command to join a worker node to the master nodes. We are leveraging the cluster-secret here. Since our master nodes were set up to use this, so too must any agent nodes. +Run the following command to join a worker node to the server nodes. We are leveraging the cluster-secret here. Since our server nodes were set up to use this, so too must any agent nodes. ``` -curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - +curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - ``` -Provide the IP or DNS in place of `` this can be any one master node. k3s automatically handles load balancing the master nodes. +Provide the IP or DNS in place of ``. This can be any one server node. k3s automatically handles load balancing the server nodes. Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. -# Cluster Datastore Options - ->**Note:** As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. - -k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enhancement depends on the following arguments that can be passed to k3s server: - -* `--storage-endpoint` _value_ - - Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] - -* `--storage-cafile` _value_ - - SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] - -* `--storage-certfile` _value_ - - SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] - -* `--storage-keyfile` _value_ - - SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] - -### MySQL - -To use k3s with MySQL storage backend, you can specify the following for insecure connection: - -``` - --storage-endpoint="mysql://" -``` -By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" -``` - -This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name - -To connect to MySQL securely, you can use the following example: -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" \ - --storage-cafile ca.crt \ - --storage-certfile mysql.crt \ - --storage-keyfile mysql.key -``` -The above command will use these certificates to generate the tls config to communicate with mysql securely. - - -### Postgres - -Connection to postgres can be established using the following command: - -``` - --storage-endpoint="postgres://" -``` - -By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" -``` - -This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq - -To connect to Postgres securely, you can use the following example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" \ - --storage-certfile postgres.crt \ - --storage-keyfile postgres.key \ - --storage-cafile ca.crt -``` - -The above command will use these certificates to generate the tls config to communicate with postgres securely. - -### etcd - -Connection to etcd3 can be established using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" -``` -The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" \ - --storage-cafile ca.crt \ - --storage-certfile etcd.crt \ - --storage-keyfile etcd.key -``` - -The above command will use these certificates to generate the tls config to communicate with etcd securely. - diff --git a/content/k3s/latest/en/installation/older-installations/_index.md b/content/k3s/latest/en/installation/older-installations/_index.md new file mode 100644 index 00000000000..21525e7875c --- /dev/null +++ b/content/k3s/latest/en/installation/older-installations/_index.md @@ -0,0 +1,40 @@ +--- +title: "Installations Prior to v1.0.0" +weight: 50 +--- + +Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. + +In v1.0.0 the method for adding server nodes is easier. If you are running a version of k3s older than v1.0.0, use these instructions for adding additional servers. Otherwise, the process is unchanged for joining workers. + +First, create your initial server, for example: + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' --cluster-secret='mysecret' --bootstrap-save +``` + +Then, add additional servers: + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' +``` + +Substitute the value for `--storage-endpoint` with different strings for PostgreSQL, MySQL, or etcd accordingly. Below are examples of the `--storage-endpoint` flag for each type of database: + +### PostgreSQL + +``` +--storage-endpoint='postgres://username:password@hostname:5432/dbname' +``` + +### MySQL + +``` +--storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname'' +``` + +### etcd + +``` +--storage-endpoint='https://etcd_hostname:2379' +``` diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 942c566f2d6..38d0416a320 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -1,9 +1,9 @@ --- -title: "Single Master Install" +title: "Single Server Install" weight: 20 --- ->**Note:** This section contains information on flags and environment variables used for starting a single-master (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. +>**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. Installation ------------ @@ -95,308 +95,3 @@ The full help text for the install script environment variables are as follows: Type of systemd service to create, will default from the k3s exec command if not specified. -Server Options --------------- - -The following information on server options is also available through `k3s server --help` : - -* `--bind-address` _value_ - - k3s bind address (default: localhost) - -* `--https-listen-port` _value_ - - HTTPS listen port (default: 6443) - -* `--http-listen-port` _value_ - - HTTP listen port (for /healthz, HTTPS redirect, and port for TLS terminating LB) (default: 0) - -* `--data-dir` _value_, `-d` _value_ - - Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root - -* `--log` _value_, `-l` _value_ - - Log to file - -* `--cluster-cidr` _value_ - - Network CIDR to use for pod IPs (default: "10.42.0.0/16") - -* `--cluster-secret` _value_ - - Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`] - -* `--service-cidr` _value_ - - Network CIDR to use for services IPs (default: "10.43.0.0/16") - -* `--cluster-dns` _value_ - - Cluster IP for coredns service. Should be in your service-cidr range - -* `--cluster-domain` _value_ - - Cluster Domain (default: "cluster.local") - -* `--no-deploy` _value_ - - Do not deploy packaged components (valid items: coredns, servicelb, traefik) - -* `--write-kubeconfig` _value_, `-o` _value_ - - Write kubeconfig for admin client to this file [$`K3S_KUBECONFIG_OUTPUT`] - -* `--write-kubeconfig-mode` _value_ - - Write kubeconfig with this mode [$`K3S_KUBECONFIG_MODE`] - -* `--tls-san` _value_ - - Add additional hostname or IP as a Subject Alternative Name in the TLS cert - -* `--kube-apiserver-arg` _value_ - - Customized flag for kube-apiserver process - -* `--kube-scheduler-arg` _value_ - - Customized flag for kube-scheduler process - -* `--kube-controller-arg` _value_ - - Customized flag for kube-controller-manager process - -* `--kube-cloud-controller-arg` _value_ - - Customized flag for kube-cloud-controller-manager process - -* `--rootless` - - (experimental) Run rootless - -* `--storage-endpoint` _value_ - - Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] - -* `--storage-cafile` _value_ - - SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] - -* `--storage-certfile` _value_ - - SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] - -* `--storage-keyfile` _value_ - - SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] - -* `--advertise-address` _value_ - - IP address that apiserver uses to advertise to members of the cluster - -* `--advertise-port` _value_ - - Port that apiserver uses to advertise to members of the cluster (default: 0) - -* `--disable-scheduler` - - Disable Kubernetes default scheduler - -* `--disable-cloud-controller` - - Disable k3s default cloud controller manager - -* `--disable-network-policy` - - Disable k3s default network policy controller - -* `--default-local-storage-path` _value_ - - Default local storage path for local provisioner storage class - -* `--node-ip` _value_, `-i` _value_ - - (agent) IP address to advertise for node - -* `--node-name` _value_ - - (agent) Node name [$`K3S_NODE_NAME`] - -* `--docker` - - (agent) Use docker instead of containerd - -* `--no-flannel` - - (agent) Disable embedded flannel - -* `--flannel-iface` _value_ - - (agent) Override default flannel interface - -* `--flannel-conf` _value_ - - (agent) (experimental) Override default flannel config file - -* `--flannel-backend` _value_ - - (agent) Specify the flannel backend you would like to use: vxlan (default), ipsec, or wireguard - -* `--container-runtime-endpoint` _value_ - - (agent) Disable embedded containerd and use alternative CRI implementation - -* `--pause-image` _value_ - - (agent) Customized pause image for containerd sandbox - -* `--resolv-conf` _value_ - - (agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`] - -* `--kubelet-arg` _value_ - - (agent) Customized flag for kubelet process - -* `--kube-proxy-arg` _value_ - - (agent) Customized flag for kube-proxy process - -* `--node-label` _value_ - - (agent) Registering kubelet with set of labels - -* `--node-taint` _value_ - - (agent) Registering kubelet with set of taints - -* `--private-registry` _value_ - - (agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") - -* `--node-external-ip` _value_ - - (agent) External IP address to advertise for node - -Agent Options ------------------- - -The following information on agent options is also available through `k3s agent --help` : - -* `--token` _value_, `-t` _value_ - - Token to use for authentication [$`K3S_TOKEN`] - -* `--token-file` _value_ - - Token file to use for authentication [$`K3S_TOKEN_FILE`] - -* `--server` _value_, `-s` _value_ - - Server to connect to [$`K3S_URL`] - -* `--data-dir` _value_, `-d` _value_ - - Folder to hold state (default: "/var/lib/rancher/k3s") - -* `--cluster-secret` _value_ - - Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`] - -* `--rootless` - - (experimental) Run rootless - -* `--docker` - - (agent) Use docker instead of containerd - -* `--no-flannel` - - (agent) Disable embedded flannel - -* `--flannel-iface` _value_ - - (agent) Override default flannel interface - -* `--flannel-conf` _value_ - - (agent) (experimental) Override default flannel config file - -* `--node-name` _value_ - - (agent) Node name [$`K3S_NODE_NAME`] - -* `--node-ip` _value_, `-i` _value_ - - (agent) IP address to advertise for node - -* `--container-runtime-endpoint` _value_ - - (agent) Disable embedded containerd and use alternative CRI implementation - -* `--pause-image` _value_ - - (agent) Customized pause image for containerd sandbox - -* `--resolv-conf` _value_ - - (agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`] - -* `--kubelet-arg` _value_ - - (agent) Customized flag for kubelet process - -* `--kube-proxy-arg` _value_ - - (agent) Customized flag for kube-proxy process - -* `--node-label` _value_ - - (agent) Registering kubelet with set of labels - -* `--node-taint` _value_ - - (agent) Registering kubelet with set of taints - -* `--private-registry` _value_ - - (agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") - -* `--node-external-ip` _value_ - - (agent) External IP address to advertise for node - -Customizing components ----------------------- - -As of v0.3.0 any of the following processes can be customized with extra flags: - -* `--kube-apiserver-arg` _value_ - - (server) [kube-apiserver options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) - -* `--kube-controller-arg` _value_ - - (server) [kube-controller-manager options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) - -* `--kube-scheduler-arg` _value_ - - (server) [kube-scheduler options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) - -* `--kubelet-arg` _value_ - - (agent) [kubelet options](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) - -* `--kube-proxy-arg` _value_ - - (agent) [kube-proxy options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/) - -Adding extra arguments can be done by passing the following flags to server or agent. -For example to add the following arguments `-v=9` and `log-file=/tmp/kubeapi.log` to the kube-apiserver, you should add the following options to k3s server: - -``` ---kube-apiserver-arg v=9 --kube-apiserver-arg log-file=/tmp/kubeapi.log -```