mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-24 20:18:18 +00:00
Merge branch 'master' into audit-30.1
This commit is contained in:
Robert Parker
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "k3s - 5 less than k8s"
|
||||
shortTitle: k3s
|
||||
title: "K3s - 5 less than K8s"
|
||||
shortTitle: K3s
|
||||
date: 2019-02-05T09:52:46-07:00
|
||||
name: "menu"
|
||||
---
|
||||
@@ -18,18 +18,14 @@ Great for:
|
||||
What is this?
|
||||
---
|
||||
|
||||
k3s is intended to be a fully compliant Kubernetes distribution with the following changes:
|
||||
K3s is a fully compliant Kubernetes distribution with the following enhancements:
|
||||
|
||||
1. Legacy, alpha, non-default features are removed. Hopefully, you shouldn't notice the
|
||||
stuff that has been removed.
|
||||
2. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced
|
||||
with out of tree addons.
|
||||
3. Add sqlite3 as the default storage mechanism. etcd3 is still available, but not the default.
|
||||
4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options.
|
||||
5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required
|
||||
dependencies
|
||||
* An embedded SQLite database has replaced etcd as the default datastore. External datastores such as PostgreSQL, MySQL, and etcd are also supported.
|
||||
* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a helm controller, and the Traefik ingress controller.
|
||||
* Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates.
|
||||
* In-tree cloud providers and storage plugins have been removed.
|
||||
* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including:
|
||||
* containerd
|
||||
* Flannel
|
||||
* CoreDNS
|
||||
* CNI
|
||||
* Host utilities (iptables, socat, etc)
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
---
|
||||
title: "Advanced Options"
|
||||
weight: 3
|
||||
weight: 40
|
||||
aliases:
|
||||
- /k3s/latest/en/running/
|
||||
---
|
||||
|
||||
This section contains advanced information describing the different ways you can run and manage k3s.
|
||||
This section contains advanced information describing the different ways you can run and manage K3s.
|
||||
|
||||
Starting the Server
|
||||
------------------
|
||||
@@ -81,10 +81,10 @@ After rebooting:
|
||||
Running in Docker (and docker-compose)
|
||||
-----------------
|
||||
|
||||
[k3d](https://github.com/rancher/k3d) is a utility designed to easily run k3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS.
|
||||
[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS.
|
||||
|
||||
`rancher/k3s` images are also available to run k3s server and agent from Docker. A `docker-compose.yml` is in the root of the k3s repo that
|
||||
serves as an example of how to run k3s from Docker. To run from `docker-compose` from this repo run:
|
||||
`rancher/k3s` images are also available to run K3s server and agent from Docker. A `docker-compose.yml` is in the root of the K3s repo that
|
||||
serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo run:
|
||||
|
||||
docker-compose up --scale node=3
|
||||
# kubeconfig is written to current dir
|
||||
@@ -104,74 +104,3 @@ To run the agent only in Docker, use `docker-compose up node`. Alternatively the
|
||||
-e K3S_TOKEN=${NODE_TOKEN} \
|
||||
--privileged rancher/k3s:vX.Y.Z
|
||||
|
||||
Air-Gap Support
|
||||
---------------
|
||||
|
||||
k3s supports pre-loading of containerd images by placing them in the `images` directory for the agent before starting, for example:
|
||||
```sh
|
||||
sudo mkdir -p /var/lib/rancher/k3s/agent/images/
|
||||
sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/
|
||||
```
|
||||
Images needed for a base install are provided through the releases page, additional images can be created with the `docker save` command.
|
||||
|
||||
Offline Helm charts are served from the `/var/lib/rancher/k3s/server/static` directory, and Helm chart manifests may reference the static files with a `%{KUBERNETES_API}%` templated variable. For example, the default traefik manifest chart installs from `https://%{KUBERNETES_API}%/static/charts/traefik-X.Y.Z.tgz`.
|
||||
|
||||
If networking is completely disabled k3s may not be able to start (ie ethernet unplugged or wifi disconnected), in which case it may be necessary to add a default route. For example:
|
||||
```sh
|
||||
sudo ip -c address add 192.168.123.123/24 dev eno1
|
||||
sudo ip route add default via 192.168.123.1
|
||||
```
|
||||
|
||||
k3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks.
|
||||
|
||||
Upgrades
|
||||
--------
|
||||
|
||||
To upgrade k3s from an older version you can re-run the installation script using the same flags, for example:
|
||||
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
```
|
||||
|
||||
If you want to upgrade to specific version you can run the following command:
|
||||
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh -
|
||||
```
|
||||
|
||||
Or to manually upgrade k3s:
|
||||
|
||||
1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest)
|
||||
2. Install to an appropriate location (normally `/usr/local/bin/k3s`)
|
||||
3. Stop the old version
|
||||
4. Start the new version
|
||||
|
||||
Restarting k3s is supported by the installation script for systemd and openrc.
|
||||
To restart manually for systemd use:
|
||||
```sh
|
||||
sudo systemctl restart k3s
|
||||
```
|
||||
|
||||
To restart manually for openrc use:
|
||||
```sh
|
||||
sudo service k3s restart
|
||||
```
|
||||
|
||||
Upgrading an air-gap environment can be accomplished in the following manner:
|
||||
|
||||
1. Download air-gap images and install if changed
|
||||
2. Install new k3s binary (from installer or manual download)
|
||||
3. Restart k3s (if not restarted automatically by installer)
|
||||
|
||||
Uninstalling
|
||||
------------
|
||||
|
||||
If you installed k3s with the help of `install.sh` script an uninstall script is generated during installation, which will be created on your server node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`).
|
||||
|
||||
Hyperkube
|
||||
---------
|
||||
|
||||
k3s is bundled in a nice wrapper to remove the majority of the headache of running k8s. If
|
||||
you don't want that wrapper and just want a smaller k8s distro, the releases includes
|
||||
the `hyperkube` binary you can use. It's then up to you to know how to use `hyperkube`. If
|
||||
you want individual binaries you will need to compile them yourself from source.
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
---
|
||||
title: "Building from Source"
|
||||
weight: 99999
|
||||
---
|
||||
|
||||
This section provides information on building k3s from source.
|
||||
|
||||
See the [release](https://github.com/rancher/k3s/releases/latest) page for pre-built releases.
|
||||
|
||||
The clone will be much faster on this repo if you do
|
||||
|
||||
git clone --depth 1 https://github.com/rancher/k3s.git
|
||||
|
||||
This repo includes all of Kubernetes history so `--depth 1` will avoid most of that.
|
||||
|
||||
To build the full release binary run `make` and that will create `./dist/artifacts/k3s`.
|
||||
|
||||
Optionally to build the binaries without running linting or building docker images:
|
||||
```sh
|
||||
./scripts/download && ./scripts/build && ./scripts/package-cli
|
||||
```
|
||||
|
||||
For development, you just need go 1.12 and a sane GOPATH. To compile the binaries run:
|
||||
```bash
|
||||
go build -o k3s
|
||||
go build -o kubectl ./cmd/kubectl
|
||||
go build -o hyperkube ./vendor/k8s.io/kubernetes/cmd/hyperkube
|
||||
```
|
||||
|
||||
This will create the main executable, but it does not include the dependencies like containerd, CNI,
|
||||
etc. To run a server and agent with all the dependencies for development run the following
|
||||
helper scripts:
|
||||
```bash
|
||||
# Server
|
||||
./scripts/dev-server.sh
|
||||
|
||||
# Agent
|
||||
./scripts/dev-agent.sh
|
||||
```
|
||||
|
||||
|
||||
Kubernetes Source
|
||||
-----------------
|
||||
|
||||
The source code for Kubernetes is in `vendor/` and the location from which that is copied
|
||||
is in `./vendor.conf`. Go to the referenced repo/tag and you'll find all the patches applied
|
||||
to upstream Kubernetes.
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
title: "Configuration Info"
|
||||
weight: 4
|
||||
weight: 50
|
||||
---
|
||||
|
||||
This section contains information on using k3s with various configurations.
|
||||
This section contains information on using K3s with various configurations.
|
||||
|
||||
|
||||
Auto-Deploying Manifests
|
||||
@@ -12,7 +12,7 @@ Auto-Deploying Manifests
|
||||
Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to
|
||||
Kubernetes in a manner similar to `kubectl apply`.
|
||||
|
||||
It is also possible to deploy Helm charts. k3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`):
|
||||
It is also possible to deploy Helm charts. K3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`):
|
||||
|
||||
```yaml
|
||||
apiVersion: helm.cattle.io/v1
|
||||
@@ -27,7 +27,7 @@ spec:
|
||||
ssl.enabled: "true"
|
||||
```
|
||||
|
||||
Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because k3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section:
|
||||
Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section:
|
||||
|
||||
```
|
||||
apiVersion: helm.cattle.io/v1
|
||||
@@ -53,7 +53,7 @@ spec:
|
||||
|
||||
Also note that besides `set` you can use `valuesContent` in the spec section. And it's okay to use both of them.
|
||||
|
||||
k3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions.
|
||||
K3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions.
|
||||
|
||||
Using the helm CRD
|
||||
---------------------
|
||||
@@ -90,36 +90,24 @@ Accessing Cluster from Outside
|
||||
-----------------------------
|
||||
|
||||
Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace
|
||||
"localhost" with the IP or name of your k3s server. `kubectl` can now manage your k3s cluster.
|
||||
|
||||
Open Ports / Network Security
|
||||
---------------------------
|
||||
|
||||
The server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach
|
||||
other nodes over UDP port 8472. The nodes also need to be able to reach the server on UDP port 8472. This is used for flannel VXLAN. If you don't use flannel
|
||||
and provide your own custom CNI, then 8472 is not needed by k3s. The node should not listen
|
||||
on any other port. k3s uses reverse tunneling such that the nodes make outbound connections
|
||||
to the server and all kubelet traffic runs through that tunnel.
|
||||
|
||||
IMPORTANT. The VXLAN port on nodes should not be exposed to the world, it opens up your
|
||||
cluster network to accessed by anyone. Run your nodes behind a firewall/security group that
|
||||
disables access to port 8472.
|
||||
"localhost" with the IP or name of your K3s server. `kubectl` can now manage your K3s cluster.
|
||||
|
||||
Node Registration
|
||||
-----------------
|
||||
|
||||
Agents will register with the server using the node cluster secret along with a randomly generated
|
||||
password for the node, stored at `/var/lib/rancher/k3s/agent/node-password.txt`. The server will
|
||||
password for the node, stored at `/etc/rancher/node/password`. The server will
|
||||
store the passwords for individual nodes at `/var/lib/rancher/k3s/server/cred/node-passwd`, and any
|
||||
subsequent attempts must use the same password. If the data directory of an agent is removed the
|
||||
password file should be recreated for the agent, or the entry removed from the server.
|
||||
subsequent attempts must use the same password. If the `/etc/rancher/node` directory of an agent is removed the
|
||||
password file should be recreated for the agent, or the entry removed from the server. A unique node
|
||||
id can be appended to the hostname by launching k3s servers or agents using the `--with-node-id` flag.
|
||||
|
||||
Containerd and Docker
|
||||
----------
|
||||
|
||||
k3s includes and defaults to containerd. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag.
|
||||
K3s includes and defaults to containerd. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag.
|
||||
|
||||
k3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
|
||||
K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
|
||||
|
||||
The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template, the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32
|
||||
|
||||
@@ -139,9 +127,9 @@ In short, latest Ubuntu is your best bet for this to work.
|
||||
|
||||
* **Ports**
|
||||
|
||||
When running rootless a new network namespace is created. This means that k3s instance is running with networking
|
||||
fairly detached from the host. The only way to access services run in k3s from the host is to setup port forwards
|
||||
to the k3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000.
|
||||
When running rootless a new network namespace is created. This means that K3s instance is running with networking
|
||||
fairly detached from the host. The only way to access services run in K3s from the host is to setup port forwards
|
||||
to the K3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000.
|
||||
|
||||
That means service port 80 will become 10080 on the host, but 8080 will become 8080 without any offset.
|
||||
|
||||
@@ -149,7 +137,7 @@ In short, latest Ubuntu is your best bet for this to work.
|
||||
|
||||
* **Daemon lifecycle**
|
||||
|
||||
Once you kill k3s and then start a new instance of k3s it will create a new network namespace, but it doesn't kill the old pods. So you are left
|
||||
Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left
|
||||
with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace.
|
||||
|
||||
The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65
|
||||
@@ -162,149 +150,16 @@ In short, latest Ubuntu is your best bet for this to work.
|
||||
|
||||
Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message
|
||||
`Wrote kubeconfig [SOME PATH]` for where your kubeconfig to access you cluster is. Be careful, if you use `-o` to write
|
||||
the kubeconfig to a different directory it will probably not work. This is because the k3s instance in running in a different
|
||||
the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different
|
||||
mount namespace.
|
||||
|
||||
Node Labels and Taints
|
||||
----------------------
|
||||
|
||||
k3s agents can be configured with options `--node-label` and `--node-taint` which adds set of Labels and Taints to kubelet, the two options only adds labels/taints at registration time, so they can only be added once and not changed after that, an example of options to add new label is:
|
||||
K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s. If you want to change node labels and taints after node registration you should use `kubectl`. Below is an example showing how to add labels and a taint:
|
||||
```
|
||||
--node-label foo=bar \
|
||||
--node-label hello=world \
|
||||
--node-taint key1=value1:NoExecute
|
||||
```
|
||||
|
||||
Flannel
|
||||
-------
|
||||
|
||||
Flannel is included by default, if you don't want flannel then run the agent with `--no-flannel` option.
|
||||
|
||||
In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network)
|
||||
|
||||
CoreDNS
|
||||
-------
|
||||
|
||||
CoreDNS is deployed on start of the agent, to disable run the server with the `--no-deploy coredns` option.
|
||||
|
||||
If you don't install CoreDNS you will need to install a cluster DNS provider yourself.
|
||||
|
||||
Traefik
|
||||
-------
|
||||
|
||||
Traefik is deployed by default when starting the server; to disable it, start the server with the `--no-deploy traefik` option. The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`.
|
||||
|
||||
Service Load Balancer
|
||||
---------------------
|
||||
|
||||
k3s includes a basic service load balancer that uses available host ports. If you try to create
|
||||
a load balancer that listens on port 80, for example, it will try to find a free host in the cluster
|
||||
for port 80. If no port is available the load balancer will stay in Pending.
|
||||
|
||||
To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB.
|
||||
|
||||
Metrics Server
|
||||
--------------
|
||||
|
||||
To add functionality for commands such as `k3s kubectl top nodes` metrics-server must be installed,
|
||||
to install see the instructions located at https://github.com/kubernetes-incubator/metrics-server/.
|
||||
|
||||
**NOTE** : By default the image used in `metrics-server-deployment.yaml` is valid only for **amd64** devices,
|
||||
this should be edited as appropriate for your architecture. As of this writing metrics-server provides
|
||||
the following images relevant to k3s: `amd64:v0.3.3`, `arm64:v0.3.2`, and `arm:v0.3.2`. Further information
|
||||
on the images provided through gcr.io can be found at https://console.cloud.google.com/gcr/images/google-containers/GLOBAL.
|
||||
|
||||
Storage Backends (Experimental)
|
||||
----------------
|
||||
|
||||
As of version 0.6.0, k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enhancement depends on the following arguments that can be passed to k3s server:
|
||||
|
||||
* `--storage-endpoint` _value_
|
||||
|
||||
Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`]
|
||||
|
||||
* `--storage-cafile` _value_
|
||||
|
||||
SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`]
|
||||
|
||||
* `--storage-certfile` _value_
|
||||
|
||||
SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`]
|
||||
|
||||
* `--storage-keyfile` _value_
|
||||
|
||||
SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`]
|
||||
|
||||
### MySQL
|
||||
|
||||
To use k3s with MySQL storage backend, you can specify the following for insecure connection:
|
||||
|
||||
```
|
||||
--storage-endpoint="mysql://"
|
||||
```
|
||||
By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN.
|
||||
|
||||
To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example:
|
||||
|
||||
```
|
||||
--storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest"
|
||||
```
|
||||
|
||||
This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name
|
||||
|
||||
To connect to MySQL securely, you can use the following example:
|
||||
```
|
||||
--storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" \
|
||||
--storage-cafile ca.crt \
|
||||
--storage-certfile mysql.crt \
|
||||
--storage-keyfile mysql.key
|
||||
```
|
||||
The above command will use these certificates to generate the tls config to communicate with mysql securely.
|
||||
|
||||
|
||||
### Postgres
|
||||
|
||||
Connection to postgres can be established using the following command:
|
||||
|
||||
```
|
||||
--storage-endpoint="postgres://"
|
||||
```
|
||||
|
||||
By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN.
|
||||
|
||||
To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example:
|
||||
|
||||
```
|
||||
--storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest"
|
||||
```
|
||||
|
||||
This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq
|
||||
|
||||
To connect to Postgres securely, you can use the following example:
|
||||
|
||||
```
|
||||
--storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" \
|
||||
--storage-certfile postgres.crt \
|
||||
--storage-keyfile postgres.key \
|
||||
--storage-cafile ca.crt
|
||||
```
|
||||
|
||||
The above command will use these certificates to generate the tls config to communicate with postgres securely.
|
||||
|
||||
### etcd
|
||||
|
||||
Connection to etcd3 can be established using the following command:
|
||||
|
||||
```
|
||||
--storage-endpoint="https://127.0.0.1:2379"
|
||||
```
|
||||
The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command:
|
||||
|
||||
```
|
||||
--storage-endpoint="https://127.0.0.1:2379" \
|
||||
--storage-cafile ca.crt \
|
||||
--storage-certfile etcd.crt \
|
||||
--storage-keyfile etcd.key
|
||||
```
|
||||
|
||||
The above command will use these certificates to generate the tls config to communicate with etcd securely.
|
||||
|
||||
@@ -1,18 +1,22 @@
|
||||
---
|
||||
title: FAQ
|
||||
weight: 8000
|
||||
weight: 60
|
||||
---
|
||||
|
||||
The FAQ is updated periodically and designed to answer the questions our users most frequently ask about k3s.
|
||||
The FAQ is updated periodically and designed to answer the questions our users most frequently ask about K3s.
|
||||
|
||||
**Is k3s a suitable replacement for k8s?**
|
||||
**Is K3s a suitable replacement for k8s?**
|
||||
|
||||
k3s is capable of nearly everything k8s can do. It is just a more lightweight version. See the [main]({{<baseurl>}}/k3s/latest/en/) docs page for more details.
|
||||
K3s is capable of nearly everything k8s can do. It is just a more lightweight version. See the [main]({{<baseurl>}}/k3s/latest/en/) docs page for more details.
|
||||
|
||||
**How can I use my own Ingress instead of Traefik?**
|
||||
|
||||
Simply start k3s server with `--no-deploy=traefik` and deploy your ingress.
|
||||
Simply start K3s server with `--no-deploy=traefik` and deploy your ingress.
|
||||
|
||||
**Does k3s support Windows?**
|
||||
**Does K3s support Windows?**
|
||||
|
||||
At this time k3s does not natively support Windows, however we are open to the idea in the future.
|
||||
At this time K3s does not natively support Windows, however we are open to the idea in the future.
|
||||
|
||||
**How can I build from source?**
|
||||
|
||||
Please reference the K3s [BUILDING.md](https://github.com/rancher/k3s/blob/master/BUILDING.md) with instructions.
|
||||
|
||||
@@ -1,16 +1,19 @@
|
||||
---
|
||||
title: "Installation Options"
|
||||
weight: 2
|
||||
title: "Installation"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
This section contains instructions for installing k3s in testing and production environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing k3s.
|
||||
This section contains instructions for installing K3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing K3s.
|
||||
|
||||
### Installation Options
|
||||
[Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) provides guidance on the options available to you when installing K3s.
|
||||
|
||||
* [Single Master Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/)
|
||||
|
||||
Install k3s on a single Linux host. Single master installs are recommended for development and test environments, as setup is simple and the cluster doesn't have to be readily available for a user-base.
|
||||
[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to setup an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd.
|
||||
|
||||
* [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/)
|
||||
[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to setup an HA K3s cluster that leverages a built-in distributed database.
|
||||
|
||||
Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments.
|
||||
[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to setup K3s in environments that do not have direct access to the Internet.
|
||||
|
||||
### Uninstalling
|
||||
|
||||
If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`).
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "Air-Gap Install"
|
||||
weight: 60
|
||||
---
|
||||
|
||||
In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server.
|
||||
|
||||
Installation Outline
|
||||
--------------------
|
||||
1. Prepare Images Directory
|
||||
2. Create Registry YAML
|
||||
3. Install K3s
|
||||
|
||||
### Prepare Images Directory
|
||||
Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running.
|
||||
|
||||
Place the tar file in the `images` directory before starting K3s on each node, for example:
|
||||
|
||||
```sh
|
||||
sudo mkdir -p /var/lib/rancher/k3s/agent/images/
|
||||
sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/
|
||||
```
|
||||
|
||||
### Create Registry YAML
|
||||
Create the registries.yaml file at `/etc/rancher/k3s/registries.yaml`. This will tell K3s the necessary details to connect to your private registry.
|
||||
The registries.yaml file should look like this before plugging in the necessary information:
|
||||
|
||||
```
|
||||
---
|
||||
mirrors:
|
||||
customreg:
|
||||
endpoint:
|
||||
- "https://ip-to-server:5000"
|
||||
configs:
|
||||
customreg:
|
||||
auth:
|
||||
username: xxxxxx # this is the registry username
|
||||
password: xxxxxx # this is the registry password
|
||||
tls:
|
||||
cert_file: <path to the cert file used in the registry>
|
||||
key_file: <path to the key file used in the registry>
|
||||
ca_file: <path to the ca file used in the registry>
|
||||
```
|
||||
|
||||
Note, at this time only secure registries are supported with K3s (SSL with custom CA)
|
||||
|
||||
### Install K3s
|
||||
|
||||
Obtain the K3s binary from the [releases](https://github.com/rancher/k3s/releases) page, matching the same version used to get the airgap images tar.
|
||||
Also obtain the K3s install script at https://get.k3s.io
|
||||
|
||||
Place the binary in `/usr/local/bin` on each node.
|
||||
Place the install script anywhere on each node, name it `install.sh`.
|
||||
|
||||
Install K3s on each server:
|
||||
|
||||
```
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
|
||||
```
|
||||
|
||||
Install K3s on each agent:
|
||||
|
||||
```
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken ./install.sh
|
||||
```
|
||||
|
||||
Note, take care to ensure you replace `myserver` with the IP or valid DNS of the server and replace `mynodetoken` with the node-token from the server.
|
||||
The node-token is on the server at `/var/lib/rancher/k3s/server/node-token`
|
||||
|
||||
|
||||
>**Note:** K3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks.
|
||||
|
||||
# Upgrading
|
||||
|
||||
Upgrading an air-gap environment can be accomplished in the following manner:
|
||||
|
||||
1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file.
|
||||
2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same environment variables.
|
||||
3. Restart the K3s service (if not restarted automatically by installer).
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: "Cluster Datastore Options"
|
||||
weight: 50
|
||||
---
|
||||
|
||||
The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to Kubernetes operators. The available datastore options allow you to select a datastore that best fits your use case. For example:
|
||||
|
||||
* If your team doesn't have expertise in operating etcd, you can choose an enterprise-grade SQL database like MySQL or PostgreSQL
|
||||
* If you need to run a simple, short-lived cluster in your CI/CD environment, you can use the embedded SQLite database
|
||||
* If you wish to deploy Kubernetes on the edge and require a highly available solution but can't afford the operational overhead of managing a database at the edge, you can use K3s's embedded HA datastore built on top of DQLite (currently experimental)
|
||||
|
||||
K3s supports the following datastore options:
|
||||
|
||||
* Embedded [SQLite](https://www.sqlite.org/index.html)
|
||||
* [PostgreSQL](https://www.postgresql.org/) (certified against versions 10.7 and 11.5)
|
||||
* [MySQL](https://www.mysql.com/) (certified against version 5.7)
|
||||
* [etcd](https://etcd.io/) (certified against version 3.3.15)
|
||||
* Embedded [DQLite](https://dqlite.io/) for High Availability (experimental)
|
||||
|
||||
### External Datastore Configuration Parameters
|
||||
If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you must set the `datastore-endpoint` parameter so that K3s knows how to connect to it. You may also specify parameters to configure the authentication and encryption of the connection. The below table summarizes these parameters, which can be passed as either CLI flags or environment variables.
|
||||
|
||||
CLI Flag | Environment Variable | Description
|
||||
------------|-------------|------------------
|
||||
<span style="white-space: nowrap">`--datastore-endpoint`</span> | `K3S_DATASTORE_ENDPOINT` | Specify a PostgresSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below.
|
||||
<span style="white-space: nowrap">`--datastore-cafile`</span> | `K3S_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate. |
|
||||
| <span style="white-space: nowrap">`--datastore-certfile`</span> | `K3S_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. |
|
||||
| <span style="white-space: nowrap">`--datastore-keyfile`</span> | `K3S_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. |
|
||||
|
||||
As a best practice we recommend setting these parameters as environment variables rather than command line arguments so that your database credentials or other sensitive information aren't exposed as part of the process info.
|
||||
|
||||
### Datastore Endpoint Format and Functionality
|
||||
As mentioned, the format of the value passed to the `datastore-endpoint` parameter is dependent upon the datastore backend. The following details this format and functionality for each supported external datastore.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "PostgreSQL" %}}
|
||||
|
||||
In its most common form, the datastore-endpoint parameter for PostgreSQL has the following format:
|
||||
|
||||
`postgres://username:password@hostname:port/database-name`
|
||||
|
||||
More advanced configuration parameters are available. For more information on these, please see https://godoc.org/github.com/lib/pq.
|
||||
|
||||
If you specify a database name and it does not exist, the server will attempt to create it.
|
||||
|
||||
If you only supply `postgres://` as the endpoint, K3s will attempt to do the following:
|
||||
|
||||
* Connect to localhost using `postgres` as the username and password
|
||||
* Create a database named `kubernetes`
|
||||
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "MySQL" %}}
|
||||
|
||||
In its most common form, the `datastore-endpoint` parameter for MySQL has the following format:
|
||||
|
||||
`mysql://username:password@tcp(hostname:3306)/database-name`
|
||||
|
||||
More advanced configuration parameters are available. For more information on these, please see https://github.com/go-sql-driver/mysql#dsn-data-source-name
|
||||
|
||||
Note that due to a [known issue](https://github.com/rancher/k3s/issues/1093) in K3s, you cannot set the `tls` parameter. TLS communication is supported, but you cannot, for example, set this parameter to "skip-verify" to cause K3s to skip certificate verification.
|
||||
|
||||
If you specify a database name and it does not exist, the server will attempt to create it.
|
||||
|
||||
If you only supply `mysql://` as the endpoint, K3s will attempt to do the following:
|
||||
|
||||
* Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user and no password
|
||||
* Create a database with the name `kubernetes`
|
||||
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "etcd" %}}
|
||||
|
||||
In its most common form, the `datastore-endpoint` parameter for etcd has the following format:
|
||||
|
||||
`https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379`
|
||||
|
||||
The above assumes a typical three node etcd cluster. The parameter can accept one more comma separated etcd URLs.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
<br/>Based on the above, the following example command could be used to launch a server instance that connects to a PostgresSQL database named k3s-db:
|
||||
```
|
||||
K3S_DATASTORE_ENDPOINT='postgres://username:password@hostname:5432/k3s-db' k3s server
|
||||
```
|
||||
|
||||
And the following example could be used to connect to a MySQL database using client certificate authentication:
|
||||
```
|
||||
K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/k3s-db' \
|
||||
K3S_DATASTORE_CERTFILE='/path/to/client.crt' \
|
||||
K3S_DATASTORE_KEYFILE='/path/to/client.key' \
|
||||
k3s server
|
||||
```
|
||||
|
||||
### Embedded DQLite for HA (Experimental)
|
||||
K3s's use of DQLite is similar to its use of SQLite. It is simple to setup and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option.
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: "High Availability with Embedded DB (Experimental)"
|
||||
weight: 40
|
||||
---
|
||||
|
||||
As of v1.0.0, K3s is previewing support for running a highly available control plane without the need for an external database. This means there is no need to manage an external etcd or SQL datastore in order to run a reliable production-grade setup. While this feature is currently experimental, we expect it to be the primary architecture for running HA K3s clusters in the future.
|
||||
|
||||
This architecture is achieved by embedding a dqlite database within the K3s server process. DQLite is short for "distributed SQLite." According to https://dqlite.io, it is "*a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices.*" This makes it a natural fit for K3s.
|
||||
|
||||
To run K3s in this mode, you must have an odd number of server nodes. We recommend starting with three nodes.
|
||||
|
||||
To get started, first launch a server node with the `cluster-init` flag to enable clustering and a token that will be used as a shared secret to join additional servers to the cluster.
|
||||
```
|
||||
K3S_TOKEN=SECRET k3s server --cluster-init
|
||||
```
|
||||
|
||||
After launching the first server, join the second and third servers to the cluster using the shared secret:
|
||||
```
|
||||
K3S_TOKEN=SECRET k3s server --server https://<ip or hostname of server1>:6443
|
||||
```
|
||||
|
||||
Now you have a highly available control plane. Joining additional worker nodes to the cluster follows the same procedure as a single server cluster.
|
||||
@@ -1,50 +1,57 @@
|
||||
---
|
||||
title: "High Availability (HA) Install (Experimental)"
|
||||
title: "High Availability with an External DB"
|
||||
weight: 30
|
||||
---
|
||||
|
||||
>**Important:** High-Availability (HA) was introduced in the v0.10.0 release of k3s and is _experimental_. Our v1.0 release plans to support HA in production environments. HA should currently only be used for testing purposes in non-production environments.
|
||||
>**Note:** k3s does not utilize etcd by default so only a 2-node cluster is needed for HA at a minimum. The following will guide you through setting up a 2-node cluster with PostgreSQL. You could optionally add one or more nodes for additional redundancy. In the future we plan to add support for additional database providers.
|
||||
>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release.
|
||||
|
||||
For production environments, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1
|
||||
Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of:
|
||||
|
||||
* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services
|
||||
* An **external datastore** (as opposed to the embedded SQLite datastore used in single server setups)
|
||||
* A **fixed registration address** placed in front of the server nodes to allow worker nodes to register with the cluster
|
||||
|
||||
The following diagram illustrates the above configuration:
|
||||
|
||||
|
||||
In this architecture a server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command.
|
||||
|
||||
Workers register through the fixed registration address, but after registration they establish a connection directly to one of the sever nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process.
|
||||
|
||||
Installation Outline
|
||||
--------------------
|
||||
1. Create backend database (PostgreSQL)
|
||||
2. Create master nodes
|
||||
3. Join worker nodes
|
||||
Setting up an HA cluster requires the following steps:
|
||||
|
||||
### Create Database
|
||||
The first step for setting up High Availability (HA) is to create the database for the backend. As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1
|
||||
1. Create an external datastore
|
||||
2. Launch server nodes
|
||||
3. Configure fixed registration address
|
||||
4. Join worker nodes
|
||||
|
||||
### Create Master Nodes
|
||||
Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines.
|
||||
### Create an External Datastore
|
||||
You will first need to create an external datastore for the cluster. See the [Cluster Datastore Options]({{< baseurl >}}/k3s/latest/en/installation/datastore/) documentation for more details.
|
||||
|
||||
On the first machine, run the following command to install k3s and connect it to the database.
|
||||
### Launch Server Nodes
|
||||
K3s requires two or more server nodes for this HA configuration. See the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) guide for minimum machine requirements.
|
||||
|
||||
>**Note:** You may wish to taint the master nodes. They will run the kubelet by default and be scheduleable. You can only add node labels and taints during the install process. If you wish to do this, use the `--node-taint` flag. For example `--node-taint key1=value1:NoExecute` the following examples do not include this flag.
|
||||
When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. Please see the [datastore configuration guide]({{< baseurl >}}/k3s/latest/en/installation/datastore/#external-datastore-configuration-parameters) for information on configuring this parameter.
|
||||
|
||||
```
|
||||
curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --bootstrap-save
|
||||
```
|
||||
Note: You may want to provide the password temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine.
|
||||
> **Note:** The same installation options available to single-server installs are also available for HA installs. For more details, see the [Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) documentation.
|
||||
|
||||
On the second machine, run the following command. Since we ran the first node with the `--bootstrap-save` flag the second and any additional machines will now automatically bootstrap HA.
|
||||
By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use taints. The <span style='white-space: nowrap'>`node-taint`</span> parameter will allow you to configure nodes with taints, for example <span style='white-space: nowrap'>`--node-taint k3s-controlplane=true:NoExecute`</span>.
|
||||
|
||||
```
|
||||
curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname'
|
||||
```
|
||||
Once you've launched the `k3s server` process on all server nodes, you can ensure that the cluster has come up properly by checking that the nodes are in the Ready state with `k3s kubectl get nodes`.
|
||||
|
||||
Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes`
|
||||
### Configure the Fixed Registration Address
|
||||
Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be setup using any number approaches, such as:
|
||||
|
||||
* A layer-4 (TCP) load balancer
|
||||
* Round-robin DNS
|
||||
* A virtual or elastic IP addresses
|
||||
|
||||
This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your kubeconfig file to point to it instead of a specific node.
|
||||
|
||||
### Join Worker Nodes
|
||||
Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s).
|
||||
|
||||
Run the following command to join a worker node to the master nodes. You can get the node-token from any of the servers at `/var/lib/rancher/k3s/server/node-token`
|
||||
|
||||
Joining worker nodes in an HA cluster is the same as joining worker nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use.
|
||||
```
|
||||
curl -sfL https://get.k3s.io | K3S_URL=https:/<master_node>:6443 K3S_TOKEN=XXX sh -
|
||||
K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443
|
||||
```
|
||||
|
||||
Provide the IP or DNS in place of `<master_node>` this can be any one master node. k3s automatically handles load balancing the master nodes.
|
||||
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
---
|
||||
title: "Installation and Configuration Options"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
### Installation script options
|
||||
|
||||
As mentioned in the [Quick-Start Guide]({{< baseurl >}}/k3s/latest/en/quick-start/), you can use the installation script available at https://get.k3s.io to install K3s as a service on systemd and openrc based systems.
|
||||
|
||||
The simplest form of this command is as follows:
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
```
|
||||
|
||||
When using this method to install K3s, the following environment variables can be used to configure the installation:
|
||||
|
||||
- `INSTALL_K3S_SKIP_DOWNLOAD`
|
||||
|
||||
If set to true will not download K3s hash or binary.
|
||||
|
||||
- `INSTALL_K3S_SYMLINK`
|
||||
|
||||
If set to 'skip' will not create symlinks, 'force' will overwrite, default will symlink if command does not exist in path.
|
||||
|
||||
- `INSTALL_K3S_SKIP_START`
|
||||
|
||||
If set to true will not start K3s service.
|
||||
|
||||
- `INSTALL_K3S_VERSION`
|
||||
|
||||
Version of K3s to download from github. Will attempt to download the latest version if not specified.
|
||||
|
||||
- `INSTALL_K3S_BIN_DIR`
|
||||
|
||||
Directory to install K3s binary, links, and uninstall script to, or use `/usr/local/bin` as the default.
|
||||
|
||||
- `INSTALL_K3S_BIN_DIR_READ_ONLY`
|
||||
|
||||
If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting INSTALL_K3S_SKIP_DOWNLOAD=true.
|
||||
|
||||
- `INSTALL_K3S_SYSTEMD_DIR`
|
||||
|
||||
Directory to install systemd service and environment files to, or use `/etc/systemd/system` as the default.
|
||||
|
||||
- `INSTALL_K3S_EXEC`
|
||||
|
||||
Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior:
|
||||
```sh
|
||||
curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s -
|
||||
curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s -
|
||||
curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel
|
||||
curl ... | sh -s - server --no-flannel
|
||||
curl ... | sh -s - --no-flannel
|
||||
```
|
||||
|
||||
- `INSTALL_K3S_NAME`
|
||||
|
||||
Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'.
|
||||
|
||||
- `INSTALL_K3S_TYPE`
|
||||
|
||||
Type of systemd service to create, will default from the K3s exec command if not specified.
|
||||
|
||||
|
||||
Environment variables which begin with `K3S_` will be preserved for the systemd and openrc services to use. Setting `K3S_URL` without explicitly setting an exec command will default the command to "agent". When running the agent `K3S_TOKEN` must also be set.
|
||||
|
||||
|
||||
### Beyond the Installation Script
|
||||
As stated, the installation script is primarily concerned with configuring K3s to run as a service. If you choose to not use the script, you can run K3s simply by downloading the binary from our [release page](https://github.com/rancher/k3s/releases/latest), placing it on your path, and executing it. The K3s binary supports the following commands:
|
||||
|
||||
Command | Description
|
||||
--------|------------------
|
||||
<span class='nowrap'>`k3s server`</span> | Run the K3s management server, which will also launch Kubernetes control plane components such as the API server, controller-manager, and scheduler.
|
||||
<span class='nowrap'>`k3s agent`</span> | Run the K3s node agent. This will cause K3s to run as a worker node, launching the Kubernetes node services `kubelet` and `kube-proxy`.
|
||||
<span class='nowrap'>`k3s kubectl`</span> | Run an embedded [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) CLI. If the `KUBECONFIG` environment variable is not set, this will automatically attempt to use the config file that is created at `/etc/rancher/k3s/k3s.yaml` when launching a K3s server node.
|
||||
<span class='nowrap'>`k3s crictl`</span> | Run an embedded [crictl](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md). This is a CLI for interacting with Kubernetes's container runtime interface (CRI). Useful for debugging.
|
||||
<span class='nowrap'>`k3s ctr`</span> | Run an embedded [ctr](https://github.com/projectatomic/containerd/blob/master/docs/cli.md). This is a CLI for containerd, the container daemon used by K3s. Useful for debugging.
|
||||
<span class='nowrap'>`k3s help`</span> | Shows a list of commands or help for one command
|
||||
|
||||
The `k3s server` and `k3s agent` commands have additional configuration options that can be viewed with <span class='nowrap'>`k3s server --help`</span> or <span class='nowrap'>`k3s agent --help`</span>. For convenience, that help text is presented here:
|
||||
|
||||
### `k3s server`
|
||||
```
|
||||
NAME:
|
||||
k3s server - Run management server
|
||||
|
||||
USAGE:
|
||||
k3s server [OPTIONS]
|
||||
|
||||
OPTIONS:
|
||||
-v value (logging) Number for the log level verbosity (default: 0)
|
||||
--vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging
|
||||
--log value, -l value (logging) Log to file
|
||||
--alsologtostderr (logging) Log to standard error as well as file (if set)
|
||||
--bind-address value (listener) k3s bind address (default: 0.0.0.0)
|
||||
--https-listen-port value (listener) HTTPS listen port (default: 6443)
|
||||
--advertise-address value (listener) IP address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip)
|
||||
--advertise-port value (listener) Port that apiserver uses to advertise to members of the cluster (default: listen-port) (default: 0)
|
||||
--tls-san value (listener) Add additional hostname or IP as a Subject Alternative Name in the TLS cert
|
||||
--data-dir value, -d value (data) Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root
|
||||
--cluster-cidr value (networking) Network CIDR to use for pod IPs (default: "10.42.0.0/16")
|
||||
--service-cidr value (networking) Network CIDR to use for services IPs (default: "10.43.0.0/16")
|
||||
--cluster-dns value (networking) Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10)
|
||||
--cluster-domain value (networking) Cluster Domain (default: "cluster.local")
|
||||
--flannel-backend value (networking) One of 'none', 'vxlan', 'ipsec', or 'flannel' (default: "vxlan")
|
||||
--token value, -t value (cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN]
|
||||
--token-file value (cluster) File containing the cluster-secret/token [$K3S_TOKEN_FILE]
|
||||
--write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT]
|
||||
--write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE]
|
||||
--kube-apiserver-arg value (flags) Customized flag for kube-apiserver process
|
||||
--kube-scheduler-arg value (flags) Customized flag for kube-scheduler process
|
||||
--kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process
|
||||
--kube-cloud-controller-manager-arg value (flags) Customized flag for kube-cloud-controller-manager process
|
||||
--datastore-endpoint value (db) Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$K3S_DATASTORE_ENDPOINT]
|
||||
--datastore-cafile value (db) TLS Certificate Authority file used to secure datastore backend communication [$K3S_DATASTORE_CAFILE]
|
||||
--datastore-certfile value (db) TLS certification file used to secure datastore backend communication [$K3S_DATASTORE_CERTFILE]
|
||||
--datastore-keyfile value (db) TLS key file used to secure datastore backend communication [$K3S_DATASTORE_KEYFILE]
|
||||
--default-local-storage-path value (storage) Default local storage path for local provisioner storage class
|
||||
--no-deploy value (components) Do not deploy packaged components (valid items: coredns, servicelb, traefik, local-storage, metrics-server)
|
||||
--disable-scheduler (components) Disable Kubernetes default scheduler
|
||||
--disable-cloud-controller (components) Disable k3s default cloud controller manager
|
||||
--disable-network-policy (components) Disable k3s default network policy controller
|
||||
--node-name value (agent/node) Node name [$K3S_NODE_NAME]
|
||||
--with-node-id (agent/node) Append id to node name
|
||||
--node-label value (agent/node) Registering kubelet with set of labels
|
||||
--node-taint value (agent/node) Registering kubelet with set of taints
|
||||
--docker (agent/runtime) Use docker instead of containerd
|
||||
--container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation
|
||||
--pause-image value (agent/runtime) Customized pause image for containerd sandbox
|
||||
--private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml")
|
||||
--node-ip value, -i value (agent/networking) IP address to advertise for node
|
||||
--node-external-ip value (agent/networking) External IP address to advertise for node
|
||||
--resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF]
|
||||
--flannel-iface value (agent/networking) Override default flannel interface
|
||||
--flannel-conf value (agent/networking) Override default flannel config file
|
||||
--kubelet-arg value (agent/flags) Customized flag for kubelet process
|
||||
--kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process
|
||||
--rootless (experimental) Run rootless
|
||||
--agent-token value (experimental/cluster) Shared secret used to join agents to the cluster, but not servers [$K3S_AGENT_TOKEN]
|
||||
--agent-token-file value (experimental/cluster) File containing the agent secret [$K3S_AGENT_TOKEN_FILE]
|
||||
--server value, -s value (experimental/cluster) Server to connect to, used to join a cluster [$K3S_URL]
|
||||
--cluster-init (experimental/cluster) Initialize new cluster master [$K3S_CLUSTER_INIT]
|
||||
--cluster-reset (experimental/cluster) Forget all peers and become a single cluster new cluster master [$K3S_CLUSTER_RESET]
|
||||
--no-flannel (deprecated) use --flannel-backend=none
|
||||
--cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET]
|
||||
```
|
||||
|
||||
### `k3s agent`
|
||||
```
|
||||
NAME:
|
||||
k3s agent - Run node agent
|
||||
|
||||
USAGE:
|
||||
k3s agent [OPTIONS]
|
||||
|
||||
OPTIONS:
|
||||
-v value (logging) Number for the log level verbosity (default: 0)
|
||||
--vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging
|
||||
--log value, -l value (logging) Log to file
|
||||
--alsologtostderr (logging) Log to standard error as well as file (if set)
|
||||
--token value, -t value (cluster) Token to use for authentication [$K3S_TOKEN]
|
||||
--token-file value (cluster) Token file to use for authentication [$K3S_TOKEN_FILE]
|
||||
--server value, -s value (cluster) Server to connect to [$K3S_URL]
|
||||
--data-dir value, -d value (agent/data) Folder to hold state (default: "/var/lib/rancher/k3s")
|
||||
--node-name value (agent/node) Node name [$K3S_NODE_NAME]
|
||||
--with-node-id (agent/node) Append id to node name
|
||||
--node-label value (agent/node) Registering kubelet with set of labels
|
||||
--node-taint value (agent/node) Registering kubelet with set of taints
|
||||
--docker (agent/runtime) Use docker instead of containerd
|
||||
--container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation
|
||||
--pause-image value (agent/runtime) Customized pause image for containerd sandbox
|
||||
--private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml")
|
||||
--node-ip value, -i value (agent/networking) IP address to advertise for node
|
||||
--node-external-ip value (agent/networking) External IP address to advertise for node
|
||||
--resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF]
|
||||
--flannel-iface value (agent/networking) Override default flannel interface
|
||||
--flannel-conf value (agent/networking) Override default flannel config file
|
||||
--kubelet-arg value (agent/flags) Customized flag for kubelet process
|
||||
--kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process
|
||||
--rootless (experimental) Run rootless
|
||||
--no-flannel (deprecated) use --flannel-backend=none
|
||||
--cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET]
|
||||
```
|
||||
@@ -3,20 +3,20 @@ title: Node Requirements
|
||||
weight: 1
|
||||
---
|
||||
|
||||
k3s is very lightweight, but has some minimum requirements as outlined below.
|
||||
K3s is very lightweight, but has some minimum requirements as outlined below.
|
||||
|
||||
Whether you're configuring a k3s cluster to run in a single-node or high-availability (HA) setup, each node running k3s should meet the following minimum requirements. You may need more resources to fit your needs.
|
||||
Whether you're configuring a K3s cluster to run in a single-node or high-availability (HA) setup, each node running K3s should meet the following minimum requirements. You may need more resources to fit your needs.
|
||||
|
||||
## Pre-requisites
|
||||
* Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or `$K3S_NODE_NAME` with a unique name for each node you add to the cluster.
|
||||
## Prerequisites
|
||||
* Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or set `$K3S_NODE_NAME` with a unique name for each node you add to the cluster.
|
||||
|
||||
## Operating Systems
|
||||
|
||||
k3s should run on just about any flavor of Linux. However, k3s is tested on the following operating systems and their subsequent non-major releases.
|
||||
K3s should run on just about any flavor of Linux. However, K3s is tested on the following operating systems and their subsequent non-major releases.
|
||||
|
||||
* Ubuntu 16.04 (amd64)
|
||||
* Ubuntu 18.04 (amd64)
|
||||
* Raspian Buster (armhf)
|
||||
* Raspbian Buster (armhf)
|
||||
|
||||
## Hardware
|
||||
|
||||
@@ -27,10 +27,12 @@ Hardware requirements scale based on the size of your deployments. Minimum recom
|
||||
|
||||
#### Disks
|
||||
|
||||
k3s performance depends on the performance of the database. To ensure optimal speed, we recommend using an SSD when possible. Disk performance will vary on ARM devices utilizing an SD card or eMMC.
|
||||
K3s performance depends on the performance of the database. To ensure optimal speed, we recommend using an SSD when possible. Disk performance will vary on ARM devices utilizing an SD card or eMMC.
|
||||
|
||||
## Networking
|
||||
|
||||
The k3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use flannel and provide your own custom CNI, then port 8472 is not needed by k3s. The node should not listen on any other port. k3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel.
|
||||
The K3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use flannel and provide your own custom CNI, then port 8472 is not needed by K3s. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel.
|
||||
|
||||
IMPORTANT: The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disabled access to port 8472.
|
||||
|
||||
If you wish to utilize the metrics server, you will need to open port 10250 on each node.
|
||||
|
||||
@@ -1,405 +0,0 @@
|
||||
---
|
||||
title: "Single Master Install"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
>**Note:** This section contains information on flags and environment variables used for starting a single-master
|
||||
(non-HA) k3s cluster. A High-Availability (HA) k3s cluster is required for production. A single server install is
|
||||
intended only for development and testing environments.
|
||||
|
||||
Installation
|
||||
------------
|
||||
|
||||
k3s is easy to install. To install the latest version, simply run:
|
||||
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
```
|
||||
|
||||
The install script will attempt to download the latest release. To specify a specific
|
||||
version for download we can use the `INSTALL_K3S_VERSION` environment variable. For example:
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh -
|
||||
```
|
||||
|
||||
To install with a specific flag we can use the `INSTALL_K3S_EXEC`
|
||||
environment variable. For example:
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-flannel" sh -
|
||||
```
|
||||
|
||||
The installer can also be run without performing downloads by setting `INSTALL_K3S_SKIP_DOWNLOAD=true`. For example:
|
||||
```sh
|
||||
curl -sfL https://github.com/rancher/k3s/releases/download/vX.Y.Z/k3s -o /usr/local/bin/k3s
|
||||
chmod 0755 /usr/local/bin/k3s
|
||||
|
||||
curl -sfL https://get.k3s.io -o install-k3s.sh
|
||||
chmod 0755 install-k3s.sh
|
||||
|
||||
export INSTALL_K3S_SKIP_DOWNLOAD=true
|
||||
./install-k3s.sh
|
||||
```
|
||||
|
||||
The full help text for the install script environment variables are as follows:
|
||||
- `K3S_*`
|
||||
|
||||
Environment variables which begin with `K3S_` will be preserved for the
|
||||
systemd service to use. Setting `K3S_URL` without explicitly setting
|
||||
a systemd exec command will default the command to "agent", and we
|
||||
enforce that `K3S_TOKEN` or `K3S_CLUSTER_SECRET` is also set.
|
||||
|
||||
- `INSTALL_K3S_SKIP_DOWNLOAD`
|
||||
|
||||
If set to true will not download k3s hash or binary.
|
||||
|
||||
- `INSTALL_K3S_SYMLINK`
|
||||
|
||||
If set to 'skip' will not create symlinks, 'force' will overwrite,
|
||||
default will symlink if command does not exist in path.
|
||||
|
||||
- `INSTALL_K3S_VERSION`
|
||||
|
||||
Version of k3s to download from github. Will attempt to download the
|
||||
latest version if not specified.
|
||||
|
||||
- `INSTALL_K3S_BIN_DIR`
|
||||
|
||||
Directory to install k3s binary, links, and uninstall script to, or use
|
||||
/usr/local/bin as the default
|
||||
|
||||
- `INSTALL_K3S_SYSTEMD_DIR`
|
||||
|
||||
Directory to install systemd service and environment files to, or use
|
||||
/etc/systemd/system as the default
|
||||
|
||||
- `INSTALL_K3S_EXEC` or script arguments
|
||||
|
||||
Command with flags to use for launching k3s in the systemd service, if
|
||||
the command is not specified will default to "agent" if `K3S_URL` is set
|
||||
or "server" if not. The final systemd command resolves to a combination
|
||||
of EXEC and script args ($@).
|
||||
|
||||
The following commands result in the same behavior:
|
||||
```sh
|
||||
curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s -
|
||||
curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s -
|
||||
curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel
|
||||
curl ... | sh -s - server --no-flannel
|
||||
curl ... | sh -s - --no-flannel
|
||||
```
|
||||
|
||||
- `INSTALL_K3S_NAME`
|
||||
|
||||
Name of systemd service to create, will default from the k3s exec command
|
||||
if not specified. If specified the name will be prefixed with 'k3s-'.
|
||||
|
||||
- `INSTALL_K3S_TYPE`
|
||||
|
||||
Type of systemd service to create, will default from the k3s exec command
|
||||
if not specified.
|
||||
|
||||
Server Options
|
||||
--------------
|
||||
|
||||
The following information on server options is also available through `k3s server --help` :
|
||||
|
||||
* `--bind-address` _value_
|
||||
|
||||
k3s bind address (default: localhost)
|
||||
|
||||
* `--https-listen-port` _value_
|
||||
|
||||
HTTPS listen port (default: 6443)
|
||||
|
||||
* `--http-listen-port` _value_
|
||||
|
||||
HTTP listen port (for /healthz, HTTPS redirect, and port for TLS terminating LB) (default: 0)
|
||||
|
||||
* `--data-dir` _value_, `-d` _value_
|
||||
|
||||
Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root
|
||||
|
||||
* `--log` _value_, `-l` _value_
|
||||
|
||||
Log to file
|
||||
|
||||
* `--cluster-cidr` _value_
|
||||
|
||||
Network CIDR to use for pod IPs (default: "10.42.0.0/16")
|
||||
|
||||
* `--cluster-secret` _value_
|
||||
|
||||
Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`]
|
||||
|
||||
* `--service-cidr` _value_
|
||||
|
||||
Network CIDR to use for services IPs (default: "10.43.0.0/16")
|
||||
|
||||
* `--cluster-dns` _value_
|
||||
|
||||
Cluster IP for coredns service. Should be in your service-cidr range
|
||||
|
||||
* `--cluster-domain` _value_
|
||||
|
||||
Cluster Domain (default: "cluster.local")
|
||||
|
||||
* `--no-deploy` _value_
|
||||
|
||||
Do not deploy packaged components (valid items: coredns, servicelb, traefik)
|
||||
|
||||
* `--write-kubeconfig` _value_, `-o` _value_
|
||||
|
||||
Write kubeconfig for admin client to this file [$`K3S_KUBECONFIG_OUTPUT`]
|
||||
|
||||
* `--write-kubeconfig-mode` _value_
|
||||
|
||||
Write kubeconfig with this mode [$`K3S_KUBECONFIG_MODE`]
|
||||
|
||||
* `--tls-san` _value_
|
||||
|
||||
Add additional hostname or IP as a Subject Alternative Name in the TLS cert
|
||||
|
||||
* `--kube-apiserver-arg` _value_
|
||||
|
||||
Customized flag for kube-apiserver process
|
||||
|
||||
* `--kube-scheduler-arg` _value_
|
||||
|
||||
Customized flag for kube-scheduler process
|
||||
|
||||
* `--kube-controller-arg` _value_
|
||||
|
||||
Customized flag for kube-controller-manager process
|
||||
|
||||
* `--kube-cloud-controller-arg` _value_
|
||||
|
||||
Customized flag for kube-cloud-controller-manager process
|
||||
|
||||
* `--rootless`
|
||||
|
||||
(experimental) Run rootless
|
||||
|
||||
* `--storage-endpoint` _value_
|
||||
|
||||
Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`]
|
||||
|
||||
* `--storage-cafile` _value_
|
||||
|
||||
SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`]
|
||||
|
||||
* `--storage-certfile` _value_
|
||||
|
||||
SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`]
|
||||
|
||||
* `--storage-keyfile` _value_
|
||||
|
||||
SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`]
|
||||
|
||||
* `--advertise-address` _value_
|
||||
|
||||
IP address that apiserver uses to advertise to members of the cluster
|
||||
|
||||
* `--advertise-port` _value_
|
||||
|
||||
Port that apiserver uses to advertise to members of the cluster (default: 0)
|
||||
|
||||
* `--disable-scheduler`
|
||||
|
||||
Disable Kubernetes default scheduler
|
||||
|
||||
* `--disable-cloud-controller`
|
||||
|
||||
Disable k3s default cloud controller manager
|
||||
|
||||
* `--disable-network-policy`
|
||||
|
||||
Disable k3s default network policy controller
|
||||
|
||||
* `--default-local-storage-path` _value_
|
||||
|
||||
Default local storage path for local provisioner storage class
|
||||
|
||||
* `--node-ip` _value_, `-i` _value_
|
||||
|
||||
(agent) IP address to advertise for node
|
||||
|
||||
* `--node-name` _value_
|
||||
|
||||
(agent) Node name [$`K3S_NODE_NAME`]
|
||||
|
||||
* `--docker`
|
||||
|
||||
(agent) Use docker instead of containerd
|
||||
|
||||
* `--no-flannel`
|
||||
|
||||
(agent) Disable embedded flannel
|
||||
|
||||
* `--flannel-iface` _value_
|
||||
|
||||
(agent) Override default flannel interface
|
||||
|
||||
* `--flannel-conf` _value_
|
||||
|
||||
(agent) (experimental) Override default flannel config file
|
||||
|
||||
* `--flannel-backend` _value_
|
||||
|
||||
(agent) Specify the flannel backend you would like to use: vxlan (default), ipsec, or wireguard
|
||||
|
||||
* `--container-runtime-endpoint` _value_
|
||||
|
||||
(agent) Disable embedded containerd and use alternative CRI implementation
|
||||
|
||||
* `--pause-image` _value_
|
||||
|
||||
(agent) Customized pause image for containerd sandbox
|
||||
|
||||
* `--resolv-conf` _value_
|
||||
|
||||
(agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`]
|
||||
|
||||
* `--kubelet-arg` _value_
|
||||
|
||||
(agent) Customized flag for kubelet process
|
||||
|
||||
* `--kube-proxy-arg` _value_
|
||||
|
||||
(agent) Customized flag for kube-proxy process
|
||||
|
||||
* `--node-label` _value_
|
||||
|
||||
(agent) Registering kubelet with set of labels
|
||||
|
||||
* `--node-taint` _value_
|
||||
|
||||
(agent) Registering kubelet with set of taints
|
||||
|
||||
* `--private-registry` _value_
|
||||
|
||||
(agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml")
|
||||
|
||||
* `--node-external-ip` _value_
|
||||
|
||||
(agent) External IP address to advertise for node
|
||||
|
||||
Agent Options
|
||||
------------------
|
||||
|
||||
The following information on agent options is also available through `k3s agent --help` :
|
||||
|
||||
* `--token` _value_, `-t` _value_
|
||||
|
||||
Token to use for authentication [$`K3S_TOKEN`]
|
||||
|
||||
* `--token-file` _value_
|
||||
|
||||
Token file to use for authentication [$`K3S_TOKEN_FILE`]
|
||||
|
||||
* `--server` _value_, `-s` _value_
|
||||
|
||||
Server to connect to [$`K3S_URL`]
|
||||
|
||||
* `--data-dir` _value_, `-d` _value_
|
||||
|
||||
Folder to hold state (default: "/var/lib/rancher/k3s")
|
||||
|
||||
* `--cluster-secret` _value_
|
||||
|
||||
Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`]
|
||||
|
||||
* `--rootless`
|
||||
|
||||
(experimental) Run rootless
|
||||
|
||||
* `--docker`
|
||||
|
||||
(agent) Use docker instead of containerd
|
||||
|
||||
* `--no-flannel`
|
||||
|
||||
(agent) Disable embedded flannel
|
||||
|
||||
* `--flannel-iface` _value_
|
||||
|
||||
(agent) Override default flannel interface
|
||||
|
||||
* `--flannel-conf` _value_
|
||||
|
||||
(agent) (experimental) Override default flannel config file
|
||||
|
||||
* `--node-name` _value_
|
||||
|
||||
(agent) Node name [$`K3S_NODE_NAME`]
|
||||
|
||||
* `--node-ip` _value_, `-i` _value_
|
||||
|
||||
(agent) IP address to advertise for node
|
||||
|
||||
* `--container-runtime-endpoint` _value_
|
||||
|
||||
(agent) Disable embedded containerd and use alternative CRI implementation
|
||||
|
||||
* `--pause-image` _value_
|
||||
|
||||
(agent) Customized pause image for containerd sandbox
|
||||
|
||||
* `--resolv-conf` _value_
|
||||
|
||||
(agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`]
|
||||
|
||||
* `--kubelet-arg` _value_
|
||||
|
||||
(agent) Customized flag for kubelet process
|
||||
|
||||
* `--kube-proxy-arg` _value_
|
||||
|
||||
(agent) Customized flag for kube-proxy process
|
||||
|
||||
* `--node-label` _value_
|
||||
|
||||
(agent) Registering kubelet with set of labels
|
||||
|
||||
* `--node-taint` _value_
|
||||
|
||||
(agent) Registering kubelet with set of taints
|
||||
|
||||
* `--private-registry` _value_
|
||||
|
||||
(agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml")
|
||||
|
||||
* `--node-external-ip` _value_
|
||||
|
||||
(agent) External IP address to advertise for node
|
||||
|
||||
Customizing components
|
||||
----------------------
|
||||
|
||||
As of v0.3.0 any of the following processes can be customized with extra flags:
|
||||
|
||||
* `--kube-apiserver-arg` _value_
|
||||
|
||||
(server) [kube-apiserver options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/)
|
||||
|
||||
* `--kube-controller-arg` _value_
|
||||
|
||||
(server) [kube-controller-manager options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/)
|
||||
|
||||
* `--kube-scheduler-arg` _value_
|
||||
|
||||
(server) [kube-scheduler options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
|
||||
|
||||
* `--kubelet-arg` _value_
|
||||
|
||||
(agent) [kubelet options](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/)
|
||||
|
||||
* `--kube-proxy-arg` _value_
|
||||
|
||||
(agent) [kube-proxy options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/)
|
||||
|
||||
Adding extra arguments can be done by passing the following flags to server or agent.
|
||||
For example to add the following arguments `-v=9` and `log-file=/tmp/kubeapi.log` to the kube-apiserver, you should add the following options to k3s server:
|
||||
|
||||
```
|
||||
--kube-apiserver-arg v=9 --kube-apiserver-arg log-file=/tmp/kubeapi.log
|
||||
```
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
title: Known Issues
|
||||
weight: 9000
|
||||
weight: 70
|
||||
---
|
||||
The Known Issues are updated periodically and designed to inform you about any issues that may not be immediately addressed in the next upcoming release.
|
||||
|
||||
**Snap Docker**
|
||||
|
||||
If you plan to use k3s with docker, Docker installed via a snap package is not recommended as it has been known to cause issues running k3s.
|
||||
If you plan to use K3s with docker, Docker installed via a snap package is not recommended as it has been known to cause issues running K3s.
|
||||
|
||||
**Iptables**
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "Networking"
|
||||
weight: 35
|
||||
---
|
||||
|
||||
Open Ports
|
||||
----------
|
||||
Please reference the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/#networking) page for port information.
|
||||
|
||||
Flannel
|
||||
-------
|
||||
|
||||
Flannel is included by default, if you don't want flannel then run each agent with `--no-flannel` option.
|
||||
|
||||
In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network)
|
||||
|
||||
CoreDNS
|
||||
-------
|
||||
|
||||
CoreDNS is deployed on start of the agent, to disable run each server with the `--no-deploy coredns` option.
|
||||
|
||||
If you don't install CoreDNS you will need to install a cluster DNS provider yourself.
|
||||
|
||||
Traefik Ingress Controller
|
||||
--------------------------
|
||||
|
||||
Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/configuration/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`.
|
||||
|
||||
The Traefik ingress controller will use ports 80, 443, and 8080 on the host (i.e. these will not be usable for HostPort or NodePort).
|
||||
|
||||
You can tweak traefik to meet your needs by setting options in the traefik.yaml file.
|
||||
Reference the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information.
|
||||
|
||||
To disable it, start each server with the `--no-deploy traefik` option.
|
||||
|
||||
Service Load Balancer
|
||||
---------------------
|
||||
|
||||
K3s includes a basic service load balancer that uses available host ports. If you try to create
|
||||
a load balancer that listens on port 80, for example, it will try to find a free host in the cluster
|
||||
for port 80. If no port is available the load balancer will stay in Pending.
|
||||
|
||||
To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB.
|
||||
@@ -1,27 +1,30 @@
|
||||
---
|
||||
title: "Quick-Start Guide"
|
||||
weight: 1
|
||||
weight: 10
|
||||
---
|
||||
|
||||
>**Note:** The intent of this guide is to quickly launch a cluster that you can use to evaluate k3s. This guide is not intended for production environments. Production environments should utilize a High-Availiability solution. The [installation options](../installation) section covers in greater detail how k3s can be setup.
|
||||
>**Note:** This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up.
|
||||
|
||||
> New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/).
|
||||
|
||||
Install Script
|
||||
--------------
|
||||
The k3s `install.sh` script provides a convenient way for installing to systemd or openrc,
|
||||
to install k3s as a service just run:
|
||||
K3s provides an installation script that is a convenient way to install it as a service on systemd or openrc based systems. This script is available at https://get.k3s.io. To install K3s using this method, just run:
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
```
|
||||
|
||||
A kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` and the service is automatically started or restarted.
|
||||
The install script will install k3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`.
|
||||
After running this installation:
|
||||
|
||||
To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with `K3S_TOKEN` or `K3S_CLUSTER_SECRET` environment variables. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node:
|
||||
* The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed
|
||||
* Additional utilities will be installed, including `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`
|
||||
* A kubeconfig file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it
|
||||
|
||||
To install on worker nodes and add them to the cluster, run the installation script with the `K3S_URL` and `K3S_TOKEN` environment variables. Here is an example showing how to join a worker node:
|
||||
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=XXX sh -
|
||||
curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -
|
||||
```
|
||||
Setting the `K3S_URL` parameter causes K3s to run in worker mode. The K3s agent will register with the K3s server listening at the supplied URL. The value to use for `K3S_TOKEN` is stored at `/var/lib/rancher/k3s/server/node-token` on your server node.
|
||||
|
||||
Note: Each machine must have a unique hostname. If your machines do not have unique hostnames, pass the `K3S_NODE_NAME` environment variable and provide a value with a valid and unique hostname for each node.
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
---
|
||||
title: "Volumes and Storage"
|
||||
weight: 30
|
||||
---
|
||||
|
||||
When deploying an application that needs to retain data, you’ll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the application’s pod fails.
|
||||
|
||||
# Local Storage Provider
|
||||
K3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage).
|
||||
|
||||
Create a hostPath backed persistent volume claim and a pod to utilize it:
|
||||
|
||||
### pvc.yaml
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: local-path-pvc
|
||||
namespace: default
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: local-path
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
```
|
||||
|
||||
### pod.yaml
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: volume-test
|
||||
namespace: default
|
||||
spec:
|
||||
containers:
|
||||
- name: volume-test
|
||||
image: nginx:stable-alpine
|
||||
imagePullPolicy: IfNotPresent
|
||||
volumeMounts:
|
||||
- name: volv
|
||||
mountPath: /data
|
||||
ports:
|
||||
- containerPort: 80
|
||||
volumes:
|
||||
- name: volv
|
||||
persistentVolumeClaim:
|
||||
claimName: local-path-pvc
|
||||
```
|
||||
|
||||
Apply the yaml `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml`
|
||||
|
||||
Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each.
|
||||
|
||||
# Longhorn
|
||||
|
||||
[comment]: <> (pending change - longhorn may support arm64 and armhf in the future.)
|
||||
|
||||
> **Note:** At this time Longhorn only supports amd64.
|
||||
|
||||
K3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md).
|
||||
|
||||
Apply the longhorn.yaml to install Longhorn.
|
||||
|
||||
```
|
||||
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
|
||||
```
|
||||
|
||||
Longhorn will be installed in the namespace `longhorn-system`.
|
||||
|
||||
Before we create a PVC, we will create a storage class for longhorn with this yaml.
|
||||
|
||||
```
|
||||
kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/storageclass.yaml
|
||||
```
|
||||
|
||||
Now, apply the following yaml to create the PVC and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml`
|
||||
|
||||
### pvc.yaml
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: longhorn-volv-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: longhorn
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
```
|
||||
|
||||
### pod.yaml
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: volume-test
|
||||
namespace: default
|
||||
spec:
|
||||
containers:
|
||||
- name: volume-test
|
||||
image: nginx:stable-alpine
|
||||
imagePullPolicy: IfNotPresent
|
||||
volumeMounts:
|
||||
- name: volv
|
||||
mountPath: /data
|
||||
ports:
|
||||
- containerPort: 80
|
||||
volumes:
|
||||
- name: volv
|
||||
persistentVolumeClaim:
|
||||
claimName: longhorn-volv-pvc
|
||||
```
|
||||
|
||||
Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "Upgrades"
|
||||
weight: 25
|
||||
---
|
||||
|
||||
>**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes.
|
||||
|
||||
To upgrade K3s from an older version you can re-run the installation script using the same flags, for example:
|
||||
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | sh -
|
||||
```
|
||||
|
||||
If you want to upgrade to specific version you can run the following command:
|
||||
|
||||
```sh
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh -
|
||||
```
|
||||
|
||||
Or to manually upgrade K3s:
|
||||
|
||||
1. Download the desired version of K3s from [releases](https://github.com/rancher/k3s/releases/latest)
|
||||
2. Install to an appropriate location (normally `/usr/local/bin/k3s`)
|
||||
3. Stop the old version
|
||||
4. Start the new version
|
||||
|
||||
Restarting K3s is supported by the installation script for systemd and openrc.
|
||||
To restart manually for systemd use:
|
||||
```sh
|
||||
sudo systemctl restart k3s
|
||||
```
|
||||
|
||||
To restart manually for openrc use:
|
||||
```sh
|
||||
sudo service k3s restart
|
||||
```
|
||||
@@ -52,7 +52,7 @@ The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/ra
|
||||
|
||||
Rancher Kubernetes Metadata contains Kubernetes version information which Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
|
||||
|
||||
For more information on how metadata works and how to configure metadata config, see Rancher Kubernetes Metadata.
|
||||
For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata).
|
||||
|
||||
## Enabling Experimental Features
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ The table below details the parameters for the user schema section configuration
|
||||
|
||||
| Parameter | Description |
|
||||
|:--|:--|
|
||||
| Object Class | The name of the object class used for user objects in your domain. |
|
||||
| Object Class | The name of the object class used for user objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
|
||||
| Username Attribute | The user attribute whose value is suitable as a display name. |
|
||||
| Login Attribute | The attribute whose value matches the username part of credentials entered by your users when logging in to Rancher. If your users authenticate with their UPN (e.g. "jdoe@acme.com") as username then this field must normally be set to `userPrincipalName`. Otherwise for the old, NetBIOS-style logon names (e.g. "jdoe") it's usually `sAMAccountName`. |
|
||||
| User Member Attribute | The attribute containing the groups that a user is a member of. |
|
||||
@@ -93,7 +93,7 @@ The table below details the parameters for the group schema configuration.
|
||||
|
||||
| Parameter | Description |
|
||||
|:--|:--|
|
||||
| Object Class | The name of the object class used for group objects in your domain. |
|
||||
| Object Class | The name of the object class used for group objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
|
||||
| Name Attribute | The group attribute whose value is suitable for a display name. |
|
||||
| Group Member User Attribute | The name of the **user attribute** whose format matches the group members in the `Group Member Mapping Attribute`. |
|
||||
| Group Member Mapping Attribute | The name of the group attribute containing the members of a group. |
|
||||
@@ -110,7 +110,7 @@ Once you have completed the configuration, proceed by testing the connection to
|
||||
|
||||
> **Note:**
|
||||
>
|
||||
> The AD user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned admin privileges in Rancher. You should therefore make a conscious decision on which AD account you use to perform this step.
|
||||
> The AD user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned administrator privileges in Rancher. You should therefore make a conscious decision on which AD account you use to perform this step.
|
||||
|
||||
1. Enter the **username** and **password** for the AD account that should be mapped to the local principal account.
|
||||
2. Click **Authenticate with Active Directory** to finalise the setup.
|
||||
|
||||
@@ -9,6 +9,8 @@ _Available as of v2.0.3_
|
||||
|
||||
If you have an instance of Active Directory (AD) hosted in Azure, you can configure Rancher to allow your users to log in using their AD accounts. Configuration of Azure AD external authentication requires you to make configurations in both Azure and Rancher.
|
||||
|
||||
>**Note:** Azure AD integration only supports Service Provider initiated logins.
|
||||
|
||||
>**Prerequisite:** Have an instance of Azure AD configured.
|
||||
|
||||
>**Note:** Most of this procedure takes place from the [Microsoft Azure Portal](https://portal.azure.com/).
|
||||
|
||||
@@ -7,6 +7,8 @@ _Available as of v2.2.0_
|
||||
|
||||
If your organization uses Okta Identity Provider (IdP) for user authentication, you can configure Rancher to allow your users to log in using their IdP credentials.
|
||||
|
||||
>**Note:** Okta integration only supports Service Provider initiated logins.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
In Okta, create a SAML Application with the settings below. See the [Okta documentation](https://developer.okta.com/standards/SAML/setting_up_a_saml_application_in_okta) for help.
|
||||
|
||||
@@ -22,7 +22,7 @@ If your organization uses LDAP for user authentication, you can configure Ranche
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Rancher must be configured with a LDAP bind account (aka service account) to search and retrieve LDAP entries pertaining to users and groups that should have access. It is recommended to not use an admin account or personal account for this purpose and instead create a dedicated account in OpenLDAP with read-only access to users and groups under the configured search base (see below).
|
||||
Rancher must be configured with a LDAP bind account (aka service account) to search and retrieve LDAP entries pertaining to users and groups that should have access. It is recommended to not use an administrator account or personal account for this purpose and instead create a dedicated account in OpenLDAP with read-only access to users and groups under the configured search base (see below).
|
||||
|
||||
> **Using TLS?**
|
||||
>
|
||||
@@ -75,7 +75,7 @@ The table below details the parameters for the user schema configuration.
|
||||
|
||||
| Parameter | Description |
|
||||
|:--|:--|
|
||||
| Object Class | The name of the object class used for user objects in your domain. |
|
||||
| Object Class | The name of the object class used for user objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
|
||||
| Username Attribute | The user attribute whose value is suitable as a display name. |
|
||||
| Login Attribute | The attribute whose value matches the username part of credentials entered by your users when logging in to Rancher. This is typically `uid`. |
|
||||
| User Member Attribute | The user attribute containing the Distinguished Name of groups a user is member of. Usually this is one of `memberOf` or `isMemberOf`. |
|
||||
@@ -93,7 +93,7 @@ The table below details the parameters for the group schema configuration.
|
||||
|
||||
| Parameter | Description |
|
||||
|:--|:--|
|
||||
| Object Class | The name of the object class used for group entries in your domain. |
|
||||
| Object Class | The name of the object class used for group entries in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
|
||||
| Name Attribute | The group attribute whose value is suitable for a display name. |
|
||||
| Group Member User Attribute | The name of the **user attribute** whose format matches the group members in the `Group Member Mapping Attribute`. |
|
||||
| Group Member Mapping Attribute | The name of the group attribute containing the members of a group. |
|
||||
@@ -109,7 +109,7 @@ Once you have completed the configuration, proceed by testing the connection to
|
||||
|
||||
> **Note:**
|
||||
>
|
||||
> The OpenLDAP user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned admin privileges in Rancher. You should therefore make a conscious decision on which LDAP account you use to perform this step.
|
||||
> The OpenLDAP user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned administrator privileges in Rancher. You should therefore make a conscious decision on which LDAP account you use to perform this step.
|
||||
|
||||
1. Enter the **username** and **password** for the OpenLDAP account that should be mapped to the local principal account.
|
||||
2. Click **Authenticate With OpenLDAP** to test the OpenLDAP connection and finalise the setup.
|
||||
|
||||
@@ -19,7 +19,7 @@ If your private registry requires credentials, it cannot be used as the default
|
||||
|
||||
# Setting a Private Registry with No Credentials as the Default Registry
|
||||
|
||||
1. Log into Rancher and configure the default admin password.
|
||||
1. Log into Rancher and configure the default administrator password.
|
||||
|
||||
1. Go into the **Settings** view.
|
||||
|
||||
|
||||
@@ -6,19 +6,20 @@ _Available as of v2.3.0_
|
||||
|
||||
Rancher includes some features that are experimental and disabled by default. You might want to enable these features, for example, if you decide that the benefits of using an [unsupported storage type]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers) outweighs the risk of using an untested feature. Feature flags were introduced to allow you to try these features that are not enabled by default.
|
||||
|
||||
The features can be enabled in two ways:
|
||||
The features can be enabled in three ways:
|
||||
|
||||
- When installing Rancher with a CLI, you can use a feature flag to enable a feature by default
|
||||
- After installing Rancher, you can turn on the features with the Rancher API
|
||||
- [Enable features when starting Rancher.](#enabling-features-when-starting-rancher) When installing Rancher with a CLI, you can use a feature flag to enable a feature by default.
|
||||
- [Enable features from the Rancher UI](#enabling-features-with-the-rancher-ui) in Rancher v2.3.3+ by going to the **Settings** page.
|
||||
- [Enable features with the Rancher API](#enabling-features-with-the-rancher-api) after installing Rancher.
|
||||
|
||||
Each feature has two values:
|
||||
|
||||
- A default value, which can be configured with a flag or environment variable from the command line
|
||||
- A set value, which can be configured with the Rancher API
|
||||
- A set value, which can be configured with the Rancher API or UI
|
||||
|
||||
If no value has been set, Rancher uses the default value.
|
||||
|
||||
Because the API sets the actual value and the command line sets the default value, that means that if you enable or disable a feature with the API, it will override any value set with the command line.
|
||||
Because the API sets the actual value and the command line sets the default value, that means that if you enable or disable a feature with the API or UI, it will override any value set with the command line.
|
||||
|
||||
For example, if you install Rancher, then set a feature flag to true with the Rancher API, then upgrade Rancher with a command that sets the feature flag to false, the default value will still be false, but the feature will still be enabled because it was set with the Rancher API. If you then deleted the set value (true) with the Rancher API, setting it to NULL, the default value (false) would take effect.
|
||||
|
||||
@@ -82,6 +83,25 @@ docker run -d -p 80:80 -p 443:443 \
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# Enabling Features with the Rancher UI
|
||||
|
||||
_Available as of Rancher v2.3.3_
|
||||
|
||||
1. Go to the **Global** view and click **Settings.**
|
||||
1. Click the **Feature Flags** tab. You will see a list of experimental features.
|
||||
1. To enable a feature, go to the disabled feature you want to enable and click **Ellipsis (...) > Activate.**
|
||||
|
||||
**Result:** The feature is enabled.
|
||||
|
||||
### Disabling Features with the Rancher UI
|
||||
|
||||
1. Go to the **Global** view and click **Settings.**
|
||||
1. Click the **Feature Flags** tab. You will see a list of experimental features.
|
||||
1. To disable a feature, go to the enabled feature you want to disable and click **Ellipsis (...) > Deactivate.**
|
||||
|
||||
**Result:** The feature is disabled.
|
||||
|
||||
|
||||
# Enabling Features with the Rancher API
|
||||
|
||||
1. Go to `<RANCHER-SERVER-URL>/v3/features`.
|
||||
@@ -94,7 +114,7 @@ docker run -d -p 80:80 -p 443:443 \
|
||||
|
||||
**Result:** The feature is enabled.
|
||||
|
||||
# Disabling Features with the Rancher API
|
||||
### Disabling Features with the Rancher API
|
||||
|
||||
1. Go to `<RANCHER-SERVER-URL>/v3/features`.
|
||||
1. In the `data` section, you will see an array containing all of the features that can be turned on with feature flags. The name of the feature is in the `id` field. Click the name of the feature you want to enable.
|
||||
@@ -104,4 +124,4 @@ docker run -d -p 80:80 -p 443:443 \
|
||||
1. Click **Send Request.**
|
||||
1. Click **Close.**
|
||||
|
||||
**Result:** The feature is disabled.
|
||||
**Result:** The feature is disabled.
|
||||
@@ -17,6 +17,8 @@ _Pod Security Policies_ (or PSPs) are objects that control security-sensitive as
|
||||
- You can override the default PSP by assigning a different PSP directly to the project.
|
||||
- Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked if it complies with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP.
|
||||
|
||||
>**Note:** You must enable PSPs at the cluster level before you can assign them to a project. This can be configured by [editing the cluster.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/editing-clusters/)
|
||||
|
||||
Read more about Pod Security Policies in the [Kubernetes Documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).
|
||||
|
||||
>**Best Practice:** Set pod security at the cluster level.
|
||||
|
||||
@@ -11,7 +11,7 @@ The projects and clusters accessible to non-administrative users is determined b
|
||||
|
||||
When you create a cluster or project, Rancher automatically assigns you as the `Owner` for it. Users assigned the `Owner` role can assign other users roles in the cluster or project.
|
||||
|
||||
> **Note:** Non-administrative users cannot access any existing projects/clusters by default. A user with appropriate permissions (typically the owner) must explicitly assign the user membership.
|
||||
> **Note:** Non-administrative users cannot access any existing projects/clusters by default. A user with appropriate permissions (typically the owner) must explicitly assign the project and cluster membership.
|
||||
|
||||
### Cluster Roles
|
||||
|
||||
@@ -27,31 +27,50 @@ _Cluster roles_ are roles that you can assign to users, granting them access to
|
||||
|
||||
#### Custom Cluster Roles
|
||||
|
||||
Rancher lets you assign _custom cluster roles_ to a user instead of the typical `Owner` or `Member` roles. These roles can be either a built-in custom cluster role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a user within a cluster. See the table below for a list of built-in custom cluster roles.
|
||||
Rancher lets you assign _custom cluster roles_ to a standard user instead of the typical `Owner` or `Member` roles. These roles can be either a built-in custom cluster role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a standard user within a cluster. See the table below for a list of built-in custom cluster roles.
|
||||
|
||||
#### Cluster Role Reference
|
||||
|
||||
The following table lists each built-in custom cluster role available in Rancher and whether it is also granted by the `Owner` or `Member` role.
|
||||
The following table lists each built-in custom cluster role available and whether that level of access is included in the default cluster-level permissions, `Cluster Owner` and `Cluster Member`.
|
||||
|
||||
| Built-in Cluster Role | Owner | Member <a id="clus-roles"></a> |
|
||||
| ---------------------------------- | ------------- | --------------------------------- |
|
||||
| Create Projects | ✓ | |
|
||||
| Manage Cluster Backups | ✓ | |
|
||||
| Manage Cluster Catalogs | ✓ | |
|
||||
| Manage Cluster Members | ✓ | |
|
||||
| Manage Cluster Catalogs | ✓ |
|
||||
| Manage Nodes | ✓ | |
|
||||
| Manage Snapshots | ✓ ||
|
||||
| Manage Storage | ✓ | |
|
||||
| View All Projects | ✓ | |
|
||||
| Create Project | ✓ | ✓ |
|
||||
| View Cluster Members | ✓ | ✓ |
|
||||
| View All Projects | ✓ | ✓ |
|
||||
| View Cluster Catalogs | ✓ | ✓ |
|
||||
| View Cluster Members | ✓ | ✓ |
|
||||
| View Nodes | ✓ | ✓ |
|
||||
| View Snapshots | ✓ | ✓ |
|
||||
|
||||
> **Notes:**
|
||||
>
|
||||
>- Each cluster role listed above, including `Owner` and `Member`, is comprised of multiple rules granting access to various resources. You can view the roles and their rules on the Global > Security > Roles page.
|
||||
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
|
||||
>- The `Manage Cluster Members` role allows the user to manage any members of the cluster **and** grant them any cluster scoped role regardless of their access to the cluster resources. Be cautious when assigning this role out individually.
|
||||
For details on how each cluster role can access Kubernetes resources, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Clusters** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role.
|
||||
|
||||
> **Note:**
|
||||
>When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
|
||||
|
||||
### Giving a Custom Cluster Role to a Cluster Member
|
||||
|
||||
After an administrator [sets up a custom cluster role,]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/default-custom-roles/#adding-a-custom-role) cluster owners and admins can then assign those roles to cluster members.
|
||||
|
||||
To assign a custom role to a new cluster member, you can use the Rancher UI. To modify the permissions of an existing member, you will need to use the Rancher API view.
|
||||
|
||||
To assign the role to a new cluster member,
|
||||
|
||||
1. Go to the **Cluster** view, then go to the **Members** tab.
|
||||
1. Click **Add Member.** Then in the **Cluster Permissions** section, choose the custom cluster role that should be assigned to the member.
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** The member has the assigned role.
|
||||
|
||||
To assign any custom role to an existing cluster member,
|
||||
|
||||
1. Go to the member you want to give the role to. Click the **Ellipsis (...) > View in API.**
|
||||
1. In the **roleTemplateId** field, go to the drop-down menu and choose the role you want to assign to the member. Click **Show Request** and **Send Request.**
|
||||
|
||||
**Result:** The member has the assigned role.
|
||||
|
||||
### Project Roles
|
||||
|
||||
@@ -76,7 +95,7 @@ _Project roles_ are roles that can be used to grant users access to a project. T
|
||||
|
||||
#### Custom Project Roles
|
||||
|
||||
Rancher lets you assign _custom project roles_ to a user instead of the typical `Owner`, `Member`, or `Read Only` roles. These roles can be either a built-in custom project role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a user within a project. See the table below for a list of built-in custom project roles.
|
||||
Rancher lets you assign _custom project roles_ to a standard user instead of the typical `Owner`, `Member`, or `Read Only` roles. These roles can be either a built-in custom project role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a standard user within a project. See the table below for a list of built-in custom project roles.
|
||||
|
||||
#### Project Role Reference
|
||||
|
||||
@@ -108,7 +127,7 @@ The following table lists each built-in custom project role available in Rancher
|
||||
>
|
||||
>- Each project role listed above, including `Owner`, `Member`, and `Read Only`, is comprised of multiple rules granting access to various resources. You can view the roles and their rules on the Global > Security > Roles page.
|
||||
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
|
||||
>- The `Manage Project Members` role allows the user to manage any members of the project **and** grant them any project scoped role regardless of their access to the project resources. Be cautious when assigning this role out individually.
|
||||
>- The `Manage Project Members` role allows the project owner to manage any members of the project **and** grant them any project scoped role regardless of their access to the project resources. Be cautious when assigning this role out individually.
|
||||
|
||||
### Defining Custom Roles
|
||||
As previously mentioned, custom roles can be defined for use at the cluster or project level. The context field defines whether the role will appear on the cluster member page, project member page, or both.
|
||||
@@ -117,7 +136,7 @@ When defining a custom role, you can grant access to specific resources or speci
|
||||
|
||||
### Default Cluster and Project Roles
|
||||
|
||||
By default, when a user creates a new cluster or project, they are automatically assigned an ownership role: either [cluster owner](#cluster-roles) or [project owner](#project-roles). However, in some organizations, these roles may overextend administrative access. In this use case, you can change the default role to something more restrictive, such as a set of individual roles or a custom role.
|
||||
By default, when a standard user creates a new cluster or project, they are automatically assigned an ownership role: either [cluster owner](#cluster-roles) or [project owner](#project-roles). However, in some organizations, these roles may overextend administrative access. In this use case, you can change the default role to something more restrictive, such as a set of individual roles or a custom role.
|
||||
|
||||
There are two methods for changing default cluster/project roles:
|
||||
|
||||
@@ -132,7 +151,7 @@ There are two methods for changing default cluster/project roles:
|
||||
>- Although you can [lock]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles/) a default role, the system still assigns the role to users who create a cluster/project.
|
||||
>- Only users that create clusters/projects inherit their roles. Users added to the cluster/project membership afterward must be explicitly assigned their roles.
|
||||
|
||||
### Configuring Default Roles
|
||||
### Configuring Default Roles for Cluster and Project Creators
|
||||
|
||||
You can change the cluster or project role(s) that are automatically assigned to the creating user.
|
||||
|
||||
@@ -156,9 +175,10 @@ You can change the cluster or project role(s) that are automatically assigned to
|
||||
|
||||
### Cluster Membership Revocation Behavior
|
||||
|
||||
When you revoke the cluster membership for a user that's explicitly assigned membership to both the cluster _and_ a project within the cluster, that user [loses their cluster roles](#clus-roles) but [retains their project roles](#proj-roles). In other words, although you have revoked the user's permissions to access the cluster and its nodes, the user can still:
|
||||
When you revoke the cluster membership for a standard user that's explicitly assigned membership to both the cluster _and_ a project within the cluster, that standard user [loses their cluster roles](#clus-roles) but [retains their project roles](#proj-roles). In other words, although you have revoked the user's permissions to access the cluster and its nodes, the standard user can still:
|
||||
|
||||
- Access the projects they hold membership in.
|
||||
- Exercise any [individual project roles](#project-role-reference) they are assigned.
|
||||
|
||||
If you want to completely revoke a user's access within a cluster, revoke both their cluster and project memberships.
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ title: Global Permissions
|
||||
weight: 1126
|
||||
---
|
||||
|
||||
_Permissions_ are individual access rights that you can assign when selecting a custom permission for a user.
|
||||
|
||||
Global Permissions define user authorization outside the scope of any particular cluster. Out-of-the-box, there are two default global permissions: `Administrator` and `Standard User`.
|
||||
|
||||
- **Administrator:**
|
||||
@@ -15,7 +17,7 @@ Global Permissions define user authorization outside the scope of any particular
|
||||
|
||||
>**Note:** You cannot create, update, or delete Global Permissions.
|
||||
|
||||
### Global Permission Assignment
|
||||
# Global Permission Assignment
|
||||
|
||||
Assignment of global permissions to a user depends on their authentication source: external or local.
|
||||
|
||||
@@ -27,53 +29,83 @@ Assignment of global permissions to a user depends on their authentication sourc
|
||||
|
||||
When you create a new local user, you assign them a global permission as you complete the **Add User** form.
|
||||
|
||||
### Custom Global Permissions
|
||||
# Custom Global Permissions
|
||||
|
||||
Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a custom set of permissions.
|
||||
Using custom permissions is convenient for providing users with narrow or specialized access to Rancher.
|
||||
|
||||
_Permissions_ are individual access rights that you can assign when selecting a custom permission for a user.
|
||||
When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, after a user logs in from the first time, they are created as a user and assigned the default `user` permission. The standard `user` permission allows users to login and create clusters.
|
||||
|
||||
Using custom permissions is convenient for providing users with narrow or specialized access to Rancher. See the [table below](#global-permissions-reference) for a list of individual permissions available.
|
||||
However, in some organizations, these permissions may extend too much access. Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a more restrictive set of custom global permissions.
|
||||
|
||||
### Global Permissions Reference
|
||||
The default roles, Administrator and Standard User, each come with multiple global permissions built into them. The Administrator role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role.
|
||||
|
||||
The following table lists each custom global permission available and whether it is assigned to the default global permissions, `Administrator` and `Standard User`.
|
||||
Administrators can enforce custom global permissions in two ways:
|
||||
|
||||
- Changing the [default permissions for new users](#configuring-default-global-permissions)
|
||||
|
||||
- Editing the [permissions of an existing user](#configuring-global-permissions-for-individual-users)
|
||||
|
||||
### Custom Global Permissions Reference
|
||||
|
||||
The following table lists each custom global permission available and whether it is included in the default global permissions, `Administrator` and `Standard User`.
|
||||
|
||||
| Custom Global Permission | Administrator | Standard User |
|
||||
| ---------------------------------- | ------------- | ------------- |
|
||||
| Create Clusters | ✓ | ✓ |
|
||||
| Create RKE Templates | ✓ | ✓ |
|
||||
| Manage Authentication | ✓ | |
|
||||
| Manage Catalogs | ✓ | |
|
||||
| Manage Cluster Drivers | ✓ | |
|
||||
| Manage Node Drivers | ✓ | |
|
||||
| Manage PodSecurityPolicy Templates | ✓ | |
|
||||
| Manage Roles | ✓ | |
|
||||
| Manage Settings | ✓ | |
|
||||
| Manage Users | ✓ | |
|
||||
| Create Clusters | ✓ | ✓ |
|
||||
| Create RKE Templates | ✓ | ✓ |
|
||||
| Use Catalog Templates | ✓ | ✓ |
|
||||
| Login Access | ✓ | ✓ |
|
||||
| User Base* (Basic log-in access) | ✓ | ✓ |
|
||||
|
||||
> *This role has two names:
|
||||
>
|
||||
> - When you go to the <b>Users</b> tab and edit a user's global role, this role is called <b>Login Access</b> in the custom global permissions list.
|
||||
> - When you go to the <b>Security</b> tab and edit the roles from the roles page, this role is called <b>User Base.</b>
|
||||
|
||||
For details on which Kubernetes resources correspond to each global permission, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Global** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role.
|
||||
|
||||
> **Notes:**
|
||||
>
|
||||
>- Each permission listed above is comprised of multiple individual permissions not listed in the Rancher UI. For a full list of these permissions and the rules they are comprised of, access through the API at `/v3/globalRoles`.
|
||||
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
|
||||
|
||||
When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, new users are assigned the [user](#user) permissions. However, in some organizations, these permissions may extend too much access. In this use case, you can change the default permissions to something more restrictive, such as a set of individual permissions.
|
||||
### Configuring Default Global Permissions
|
||||
|
||||
You can assign one or more default permissions. For example, the `user` permission assigns new users a [set of individual global permissions](#global-permissions-reference). If you want to restrict the default permissions for new users, you can remove the `user` permission as default role and then assign multiple individual permissions as default instead. Conversely, you can also add administrative permissions on top of a set of other standard permissions.
|
||||
If you want to restrict the default permissions for new users, you can remove the `user` permission as default role and then assign multiple individual permissions as default instead. Conversely, you can also add administrative permissions on top of a set of other standard permissions.
|
||||
|
||||
>**Note:** Default roles are only assigned to users added from an external authentication provider. For local users, you must explicitly assign global permissions when adding a user to Rancher. You can customize these global permissions when adding the user.
|
||||
|
||||
### Configuring Default Global Permissions
|
||||
|
||||
You can change the default global permissions that are assigned to external users upon their first log in.
|
||||
To change the default global permissions that are assigned to external users upon their first log in, follow these steps:
|
||||
|
||||
1. From the **Global** view, select **Security > Roles** from the main menu. Make sure the **Global** tab is selected.
|
||||
|
||||
1. Find the permissions set that you want to use as default. Then edit the permission by selecting **Ellipsis > Edit**.
|
||||
1. Find the permissions set that you want to add or remove as a default. Then edit the permission by selecting **Ellipsis > Edit**.
|
||||
|
||||
1. Select **Yes: Default role for new users** and then click **Save**.
|
||||
1. If you want to add the permission as a default, Select **Yes: Default role for new users** and then click **Save**.
|
||||
|
||||
1. If you want to remove a default permission, edit the permission and select **No** from **New User Default**.
|
||||
|
||||
**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column.
|
||||
**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column.
|
||||
|
||||
### Configuring Global Permissions for Individual Users
|
||||
|
||||
To configure permission for a user,
|
||||
|
||||
1. Go to the **Users** tab.
|
||||
|
||||
1. On this page, go to the user whose access level you want to change and click **Ellipsis (...) > Edit.**
|
||||
|
||||
1. In the **Global Permissions** section, click **Custom.**
|
||||
|
||||
1. Check the boxes for each subset of permissions you want the user to have access to.
|
||||
|
||||
1. Click **Save.**
|
||||
|
||||
> **Result:** The user's global permissions have been updated.
|
||||
|
||||
@@ -17,7 +17,9 @@ Admins control which cluster options can be changed by end users. RKE templates
|
||||
|
||||
If a cluster was created with an RKE template, you can't change it to a different RKE template. You can only update the cluster to a new revision of the same template.
|
||||
|
||||
To summarize, RKE templates allow DevOps and security teams to:
|
||||
As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. The new template can also be used to launch new clusters.
|
||||
|
||||
The core features of RKE templates allow DevOps and security teams to:
|
||||
|
||||
- Standardize cluster configuration and ensure that Rancher-provisioned clusters are created following best practices
|
||||
- Prevent less technical users from making uninformed choices when provisioning clusters
|
||||
@@ -48,7 +50,10 @@ RKE templates are supported for Rancher-provisioned clusters. The templates can
|
||||
|
||||
RKE templates are for defining Kubernetes and Rancher settings. Node templates are responsible for configuring nodes. For tips on how to use RKE templates in conjunction with hardware, refer to [RKE Templates and Hardware]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware).
|
||||
|
||||
RKE templates can be applied to new clusters, but not existing clusters.
|
||||
RKE templates can be created from scratch to pre-define cluster configuration. They can be applied to launch new clusters, or templates can also be exported from existing running clusters.
|
||||
|
||||
As of v2.3.3, the settings of an existing cluster can be [saved as an RKE template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) This creates a new template and binds the cluster settings to the template, so that the cluster can only be upgraded if the [template is updated]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template), and the cluster is upgraded to [use a newer version of the template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) The new template can also be used to create new clusters.
|
||||
|
||||
|
||||
# Example Scenarios
|
||||
When an organization has both basic and advanced Rancher users, administrators might want to give the advanced users more options for cluster creation, while restricting the options for basic users.
|
||||
@@ -58,13 +63,13 @@ These [example scenarios]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templ
|
||||
Some of the example scenarios include the following:
|
||||
|
||||
- **Enforcing templates:** Administrators might want to [enforce one or more template settings for everyone]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#enforcing-a-template-setting-for-everyone) if they want all new Rancher-provisioned clusters to have those settings.
|
||||
- **Sharing different templates with different users:** Administrators might give [different templates to basic and advanced users,]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#templates-for-basic-and-advanced-users) so that basic users can have more restricted options and advanced users can have more discretion when creating clusters.
|
||||
- **Updating template settings:** If an organization's security and DevOps teams decide to embed best practices into the required settings for new clusters, those best practices could change over time. If the best practices change, [a template can be updated to a new revision]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#updating-templates-and-clusters-created-with-them) and clusters created from the template can upgrade to the new version of the template.
|
||||
- **Sharing different templates with different users:** Administrators might give [different templates to basic and advanced users,]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#templates-for-basic-and-advanced-users) so that basic users can have more restricted options and advanced users can use more discretion when creating clusters.
|
||||
- **Updating template settings:** If an organization's security and DevOps teams decide to embed best practices into the required settings for new clusters, those best practices could change over time. If the best practices change, [a template can be updated to a new revision]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#updating-templates-and-clusters-created-with-them) and clusters created from the template can [upgrade to the new version]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) of the template.
|
||||
- **Sharing ownership of a template:** When a template owner no longer wants to maintain a template, or wants to share ownership of the template, this scenario describes how [template ownership can be shared.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#allowing-other-users-to-control-and-share-a-template)
|
||||
|
||||
# Template Management
|
||||
|
||||
When you create a RKE template, it is available in the Rancher UI from the **Global** view under **Tools > RKE Templates.** When you create a template, you become the template owner, which gives you permission to revise and share the template. You can share the RKE templates with specific users or groups, and you can also make it public.
|
||||
When you create an RKE template, it is available in the Rancher UI from the **Global** view under **Tools > RKE Templates.** When you create a template, you become the template owner, which gives you permission to revise and share the template. You can share the RKE templates with specific users or groups, and you can also make it public.
|
||||
|
||||
Administrators can turn on template enforcement to require users to always use RKE templates when creating a cluster. This allows administrators to guarantee that Rancher always provisions clusters with specific settings.
|
||||
|
||||
@@ -91,9 +96,12 @@ You can [create a cluster from a template]({{<baseurl>}}/rancher/v2.x/en/admin-s
|
||||
|
||||
If the RKE template owner creates a new revision of the template, you can [upgrade your cluster to that revision.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#updating-a-cluster-created-with-an-rke-template)
|
||||
|
||||
RKE templates can only be applied to new clusters, not existing clusters.
|
||||
RKE templates can be created from scratch to pre-define cluster configuration. They can be applied to launch new clusters, or templates can also be exported from existing running clusters.
|
||||
|
||||
As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated.
|
||||
|
||||
# Standardizing Hardware
|
||||
|
||||
RKE templates are designed to standardize Kubernetes and Rancher settings. If you want to standardize your infrastructure as well, you use RKE templates [in conjuction with other tools]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware).
|
||||
|
||||
# YAML Customization
|
||||
|
||||
@@ -5,9 +5,17 @@ weight: 50
|
||||
|
||||
You can create a cluster from an RKE template that you created, or from a template that has been [shared with you.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/template-access-and-sharing)
|
||||
|
||||
RKE templates can only be applied to new clusters, not existing clusters.
|
||||
RKE templates can be applied to new clusters.
|
||||
|
||||
You can't change the cluster to use a different RKE template. You can only update the cluster to a new revision of the same template.
|
||||
As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.](#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated.
|
||||
|
||||
You can't change a cluster to use a different RKE template. You can only update the cluster to a new revision of the same template.
|
||||
|
||||
This section covers the following topics:
|
||||
|
||||
- [Creating a cluster from an RKE template](#creating-a-cluster-from-an-rke-template)
|
||||
- [Updating a cluster created with an RKE template](#updating-a-cluster-created-with-an-rke-template)
|
||||
- [Converting an existing cluster to use an RKE template](#converting-an-existing-cluster-to-use-an-rke-template)
|
||||
|
||||
### Creating a Cluster from an RKE Template
|
||||
|
||||
@@ -30,4 +38,26 @@ When the template owner creates a template, each setting has a switch in the Ran
|
||||
|
||||
If a cluster was created from an RKE template, you can edit the cluster to update the cluster to a new revision of the template.
|
||||
|
||||
As of Rancher v2.3.3, an existing cluster's settings can be [saved as an RKE template.](#converting-an-existing-cluster-to-use-an-rke-template) In that situation, you can also edit the cluster to update the cluster to a new revision of the template.
|
||||
|
||||
> **Note:** You can't change the cluster to use a different RKE template. You can only update the cluster to a new revision of the same template.
|
||||
|
||||
### Converting an Existing Cluster to Use an RKE Template
|
||||
|
||||
_Available as of v2.3.3_
|
||||
|
||||
This section describes how to create an RKE template from an existing cluster.
|
||||
|
||||
RKE templates cannot be applied to existing clusters, except if you save an existing cluster's settings as an RKE template. This exports the cluster's settings as a new RKE template, and also binds the cluster to that template. The result is that the cluster can only be changed if the [template is updated,]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template) and the cluster is upgraded to [use a newer version of the template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision)
|
||||
|
||||
To convert an existing cluster to use an RKE template,
|
||||
|
||||
1. From the **Global** view in Rancher, click the **Clusters** tab.
|
||||
1. Go to the cluster that will be converted to use an RKE template. Click **Ellipsis (...)** > **Save as RKE Template.**
|
||||
1. Enter a name for the template in the form that appears, and click **Create.**
|
||||
|
||||
**Results:**
|
||||
|
||||
- A new RKE template is created.
|
||||
- The cluster is converted to use the new template.
|
||||
- New clusters can be [created from the new template.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#creating-a-cluster-from-an-rke-template)
|
||||
+52
-5
@@ -7,10 +7,25 @@ This section describes how to manage RKE templates and revisions. You an create,
|
||||
|
||||
Template updates are handled through a revision system. When template owners want to change or update a template, they create a new revision of the template. Individual revisions cannot be edited. However, if you want to prevent a revision from being used to create a new cluster, you can disable it.
|
||||
|
||||
Template revisions can be used in two ways: to create a new cluster, or to upgrade a cluster that was created with an earlier version of the template. The template creator can choose a default revision, but when end users create a cluster, they can choose any template and any template revision that is available to them. After the cluster is created from a specific revision, it cannot change to another template, but the cluster can be upgraded to another available revision of the same template.
|
||||
Template revisions can be used in two ways: to create a new cluster, or to upgrade a cluster that was created with an earlier version of the template. The template creator can choose a default revision, but when end users create a cluster, they can choose any template and any template revision that is available to them. After the cluster is created from a specific revision, it cannot change to another template, but the cluster can be upgraded to a newer available revision of the same template.
|
||||
|
||||
The template owner has full control over template revisions, and can create new revisions to update the template, delete or disable revisions that should not be used to create clusters, and choose which template revision is the default.
|
||||
|
||||
This section covers the following topics:
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Creating a template](#creating-a-template)
|
||||
- [Updating a template](#updating-a-template)
|
||||
- [Deleting a template](#deleting-a-template)
|
||||
- [Creating a revision based on the default revision](#creating-a-revision-based-on-the-default-revision)
|
||||
- [Creating a revision based on a cloned revision](#creating-a-revision-based-on-a-cloned-revision)
|
||||
- [Disabling a template revision](#disabling-a-template-revision)
|
||||
- [Re-enabling a disabled template revision](#re-enabling-a-disabled-template-revision)
|
||||
- [Setting a template revision as default](#setting-a-template-revision-as-default)
|
||||
- [Deleting a template revision](#deleting-a-template-revision)
|
||||
- [Upgrading a cluster to use a new template revision](#upgrading-a-cluster-to-use-a-new-template-revision)
|
||||
- [Exporting a running cluster to a new RKE template and revision](#exporting-a-running-cluster-to-a-new-rke-template-and-revision)
|
||||
|
||||
### Prerequisites
|
||||
|
||||
You can create RKE templates if you have the **Create RKE Templates** permission, which can be [given by an administrator.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creator-permissions)
|
||||
@@ -25,7 +40,7 @@ You can revise, share, and delete a template if you are an owner of the template
|
||||
1. Optional: Share the template with other users or groups by [adding them as members.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/template-access-and-sharing/#sharing-templates-with-specific-users) You can also make the template public to share with everyone in the Rancher setup.
|
||||
1. Then follow the form on screen to save the cluster configuration parameters as part of the template's revision. The revision can be marked as default for this template.
|
||||
|
||||
**Result:** An RKE template with one revision is configured. You can use this RKE template revision later when you [provision a Rancher-launched cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters).
|
||||
**Result:** An RKE template with one revision is configured. You can use this RKE template revision later when you [provision a Rancher-launched cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters).
|
||||
|
||||
### Updating a Template
|
||||
|
||||
@@ -33,14 +48,14 @@ When you update an RKE template, you are creating a revision of the existing tem
|
||||
|
||||
You can't edit individual revisions. Since you can't edit individual revisions of a template, in order to prevent a revision from being used, you can [disable it.](#disabling-a-template-revision)
|
||||
|
||||
New template revisions can be created without affecting clusters already using a revision of the template.
|
||||
When new template revisions are created, clusters using an older revision of the template are unaffected.
|
||||
|
||||
1. From the **Global** view, click **Tools > RKE Templates.**
|
||||
1. Go to the template that you want to edit and click the **Vertical Ellipsis (...) > Edit.**
|
||||
1. Edit the required information and click **Save.**
|
||||
1. Optional: You can change the default revision of this template and also change who it is shared with.
|
||||
|
||||
**Result:** The template is updated.
|
||||
**Result:** The template is updated. To apply it to a cluster using an older version of the template, refer to the section on [upgrading a cluster to use a new revision of a template.](#upgrading-a-cluster-to-use-a-new-template-revision)
|
||||
|
||||
### Deleting a Template
|
||||
|
||||
@@ -112,4 +127,36 @@ To permanently delete a revision,
|
||||
1. From the **Global** view, click **Tools > RKE Templates.**
|
||||
1. Go to the RKE template revision that should be deleted and click the **Ellipsis (...) > Delete.**
|
||||
|
||||
**Result:** The RKE template revision is deleted.
|
||||
**Result:** The RKE template revision is deleted.
|
||||
|
||||
### Upgrading a Cluster to Use a New Template Revision
|
||||
|
||||
> This section assumes that you already have a cluster that [has an RKE template applied.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates)
|
||||
> This section also assumes that you have [updated the template that the cluster is using](#updating-a-template) so that a new template revision is available.
|
||||
|
||||
To upgrade a cluster to use a new template revision,
|
||||
|
||||
1. From the **Global** view in Rancher, click the **Clusters** tab.
|
||||
1. Go to the cluster that you want to upgrade and click **Ellipsis (...) > Edit.**
|
||||
1. In the **Cluster Options** section, click the dropdown menu for the template revision, then select the new template revision.
|
||||
1. Click **Save.**
|
||||
|
||||
**Result:** The cluster is upgraded to use the settings defined in the new template revision.
|
||||
|
||||
### Exporting a Running Cluster to a New RKE Template and Revision
|
||||
|
||||
You can save an existing cluster's settings as an RKE template.
|
||||
|
||||
This exports the cluster's settings as a new RKE template, and also binds the cluster to that template. The result is that the cluster can only be changed if the [template is updated,]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template) and the cluster is upgraded to [use a newer version of the template.]
|
||||
|
||||
To convert an existing cluster to use an RKE template,
|
||||
|
||||
1. From the **Global** view in Rancher, click the **Clusters** tab.
|
||||
1. Go to the cluster that will be converted to use an RKE template. Click **Ellipsis (...)** > **Save as RKE Template.**
|
||||
1. Enter a name for the template in the form that appears, and click **Create.**
|
||||
|
||||
**Results:**
|
||||
|
||||
- A new RKE template is created.
|
||||
- The cluster is converted to use the new template.
|
||||
- New clusters can be [created from the new template and revision.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#creating-a-cluster-from-an-rke-template)
|
||||
@@ -12,8 +12,21 @@ The YAML in the RKE template uses the same customization that is used when you c
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
# This setting is not enforced. Clusters
|
||||
# created with this sample template
|
||||
# would have alerting turned off by default,
|
||||
# but end users could still turn alerting
|
||||
# on or off.
|
||||
|
||||
enable_cluster_monitoring: true
|
||||
# This setting is not enforced. Clusters
|
||||
# created with this sample template
|
||||
# would have monitoring turned on
|
||||
# by default, but end users could still
|
||||
# turn monitoring on or off.
|
||||
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
|
||||
@@ -99,7 +99,7 @@ This reference contains variables that you can use in `questions.yml` nested und
|
||||
| variable | string | true | Define the variable name specified in the `values.yml` file, using `foo.bar` for nested objects. |
|
||||
| label | string | true | Define the UI label. |
|
||||
| description | string | false | Specify the description of the variable.|
|
||||
| type | string | false | Default to `string` if not specified (current supported types are string, boolean, int, enum, password, storageclass and hostname).|
|
||||
| type | string | false | Default to `string` if not specified (current supported types are string, multiline, boolean, int, enum, password, storageclass, hostname, pvc, and secret).|
|
||||
| required | bool | false | Define if the variable is required or not (true \| false)|
|
||||
| default | string | false | Specify the default value. |
|
||||
| group | string | false | Group questions by input value. |
|
||||
@@ -121,7 +121,7 @@ This reference contains variables that you can use in `questions.yml` nested und
|
||||
|
||||
You can fill your custom catalogs with either Helm Charts or Rancher Charts, although we recommend Rancher Charts due to their enhanced user experience.
|
||||
|
||||
>**Note:** For a complete walkthrough of developing charts, see the upstream Helm chart [developer reference](https://docs.helm.sh/developing_charts/).
|
||||
>**Note:** For a complete walkthrough of developing charts, see the upstream Helm chart [developer reference](https://helm.sh/docs/topics/chart_template_guide/).
|
||||
|
||||
1. Within the GitHub repo that you're using as your custom catalog, create a directory structure that mirrors the structure listed in [Chart Directory Structure](#chart-directory-structure).
|
||||
|
||||
|
||||
@@ -23,6 +23,6 @@ If you use Istio for traffic management, you will need to allow external traffic
|
||||
|
||||
This guide assumes you have already [installed Rancher,]({{<baseurl>}}/rancher/v2.x/en/installation) and you have already [provisioned a separate Kubernetes cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning) on which you will install Istio.
|
||||
|
||||
The nodes in your cluster must meet the [CPU and memory requirements.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/istio/#cpu-and-memory-requirements)
|
||||
The nodes in your cluster must meet the [CPU and memory requirements.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources/)
|
||||
|
||||
The workloads and services that you want to be controlled by Istio must meet [Istio's requirements.](https://istio.io/docs/setup/additional-setup/requirements/)
|
||||
|
||||
@@ -15,9 +15,10 @@ Wait a few minutes for the workload to upgrade to have the istio sidecar. Click
|
||||
|
||||
Next we add the Kubernetes resources for the sample deployments and services for the BookInfo app in Istio's documentation.
|
||||
|
||||
1. Go to the cluster view and click **Import YAML.**
|
||||
1. Go to the project inside the cluster you want to deploy the workload on.
|
||||
1. In Workloads, click **Import YAML.**
|
||||
1. Copy the below resources into the form.
|
||||
1. Click **Import.**
|
||||
1. Click **Import.**
|
||||
|
||||
This will set up the following sample resources from Istio's example BookInfo app:
|
||||
|
||||
@@ -318,4 +319,4 @@ spec:
|
||||
---
|
||||
```
|
||||
|
||||
### [Next: Set up the Istio Gateway]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway)
|
||||
### [Next: Set up the Istio Gateway]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway)
|
||||
|
||||
+2
-2
@@ -7,9 +7,9 @@ This cluster uses the default Nginx controller to allow traffic into the cluster
|
||||
|
||||
A Rancher [administrator]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles) can configure Rancher to deploy Istio in a Kubernetes cluster.
|
||||
|
||||
1. From the **Global** view, navigate to the cluster where you want to enable Istio.
|
||||
1. From the **Global** view, navigate to the **cluster** where you want to enable Istio.
|
||||
1. Click **Tools > Istio.**
|
||||
1. Optional: Configure member access and resource limits for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio.
|
||||
1. Optional: Configure member access and [resource limits]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio.
|
||||
1. Click **Enable**.
|
||||
1. Click **Save**.
|
||||
|
||||
|
||||
+21
@@ -21,4 +21,25 @@ This namespace setting will only affect new workloads in the namespace. Any pree
|
||||
|
||||
To verify that Istio is enabled, deploy a hello-world workload in the namespace. Go to the workload and click the pod name. In the **Containers** section, you should see the `istio-proxy` container.
|
||||
|
||||
### Excluding Workloads from Being Injected with the Istio Sidecar
|
||||
|
||||
If you need to exclude a workload from getting injected with the Istio sidecar, use the following annotation on the workload:
|
||||
|
||||
```
|
||||
sidecar.istio.io/inject: “false”
|
||||
```
|
||||
|
||||
To add the annotation to a workload,
|
||||
|
||||
1. From the **Global** view, open the project that has the workload that should not have the sidecar.
|
||||
1. Click **Resources > Workloads.**
|
||||
1. Go to the workload that should not have the sidecar and click **Ellipsis (...) > Edit.**
|
||||
1. Click **Show Advanced Options.** Then expand the **Labels & Annotations** section.
|
||||
1. Click **Add Annotation.**
|
||||
1. In the **Key** field, enter `sidecar.istio.io/inject`.
|
||||
1. In the **Value** field, enter `false`.
|
||||
1. Click **Save.**
|
||||
|
||||
**Result:** The Istio sidecar will not be injected into the workload.
|
||||
|
||||
### [Next: Set up Taints and Tolerations]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/node-selectors)
|
||||
@@ -86,7 +86,7 @@ spec:
|
||||
|
||||
Confirm that the resource exists by running:
|
||||
```
|
||||
kubectl get gateway
|
||||
kubectl get gateway -A
|
||||
```
|
||||
|
||||
The result should be something like this:
|
||||
|
||||
@@ -189,6 +189,33 @@ services:
|
||||
- "/sbin/iscsiadm:/sbin/iscsiadm"
|
||||
```
|
||||
|
||||
## GlusterFS Volumes With Rancher Launched Kubernetes Clusters
|
||||
|
||||
In [Rancher Launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) that store data on GlusterFS volumes, you may experience an issue where pods fail to mount volumes after restarting the `kubelet`. The logging of the `kubelet` will show: `transport endpoint is not connected`. To prevent this from happening, you can configure your cluster to mount the `systemd-run` binary in the `kubelet` container. There are two requirements before you can change the cluster configuration:
|
||||
|
||||
- The node needs to have the `systemd-run` binary installed (this can be checked by using the command `which systemd-run` on each cluster node)
|
||||
- The `systemd-run` binary needs to be compatible with Debian OS on which the hyperkube image is based (this can be checked using the following command on each cluster node, replacing the image tag with the Kubernetes version you want to use)
|
||||
|
||||
```
|
||||
docker run -v /usr/bin/systemd-run:/usr/bin/systemd-run --entrypoint /usr/bin/systemd-run rancher/hyperkube:v1.16.2-rancher1 --version
|
||||
```
|
||||
|
||||
>**Note:**
|
||||
>
|
||||
>Before updating your Kubernetes YAML to mount the `systemd-run` binary, make sure the `systemd` package is installed on your cluster nodes. If this package isn't installed _before_ the bind mounts are created in your Kubernetes YAML, Docker will automatically create the directories and files on each node and will not allow the package install to succeed.
|
||||
|
||||
```
|
||||
services:
|
||||
kubelet:
|
||||
extra_binds:
|
||||
- "/usr/bin/systemd-run:/usr/bin/systemd-run"
|
||||
```
|
||||
|
||||
After the cluster has finished provisioning, you can check the `kubelet` container logging to see if the functionality is activated by looking for the following logline:
|
||||
|
||||
```
|
||||
Detected OS with systemd
|
||||
```
|
||||
|
||||
## What's Next?
|
||||
|
||||
|
||||
@@ -45,6 +45,8 @@ _Available as of Rancher v2.3.0_
|
||||
|
||||
If a node is in a node pool, Rancher can automatically replace unreachable nodes. Rancher will use the existing node template for the given node pool to recreate the node if it becomes inactive for a specified number of minutes.
|
||||
|
||||
> **Important** Self-healing node pools are designed to help you replace worker nodes for stateless applications. It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications.
|
||||
|
||||
{{% accordion id="how-does-node-auto-replace-work" label="How does Node Auto-replace Work?" %}}
|
||||
Node auto-replace works on top of the Kubernetes node controller. The node controller periodically checks the status of all the nodes (configurable via the `--node-monitor-period` flag of the `kube-controller`). When a node is unreachable, the node controller will taint that node. When this occurs, Rancher will begin its deletion countdown. You can configure the amount of time Rancher waits to delete the node. If the taint is not removed before the deletion countdown ends, Rancher will proceed to delete the node object. Rancher will then provision a node in accordance with the set quantity of the node pool.
|
||||
{{% /accordion %}}
|
||||
|
||||
+19
-148
@@ -5,170 +5,41 @@ weight: 2225
|
||||
aliases:
|
||||
- /rancher/v2.x/en/tasks/clusters/creating-a-cluster/create-cluster-vsphere/
|
||||
---
|
||||
Use {{< product >}} to create a Kubernetes cluster in vSphere.
|
||||
|
||||
## Introduction
|
||||
By using Rancher with vSphere, you can bring cloud operations on-premises.
|
||||
|
||||
When creating a vSphere cluster, Rancher first provisions the specified amount of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them. A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for the data, control, and worker plane respectively.
|
||||
Rancher can provision nodes in vSphere and install Kubernetes on them. When creating a Kubernetes cluster in vSphere, Rancher first provisions the specified number of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them.
|
||||
|
||||
>**Note:**
|
||||
>The vSphere node driver included in Rancher currently only supports the provisioning of VMs with [RancherOS]({{< baseurl >}}/os/v1.x/en/) as the guest operating system.
|
||||
A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for each Kubernetes role.
|
||||
|
||||
## Prerequisites
|
||||
# vSphere Enhancements
|
||||
|
||||
### vSphere API permissions
|
||||
The vSphere node templates have been updated, allowing you to bring cloud operations on-premises with the following enhancements:
|
||||
|
||||
Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. If you are planning to make use of vSphere volumes for persistent storage in the cluster, there are [additional requirements]({{< baseurl >}}/rke/latest/en/config-options/cloud-providers/vsphere/) that must be met.
|
||||
### Self-healing Node Pools
|
||||
|
||||
### Network permissions
|
||||
_Available as of v2.3.0_
|
||||
|
||||
You must ensure that the hosts running Rancher servers are able to establish network connections to the following network endpoints:
|
||||
One of the biggest advantages of provisioning vSphere nodes with Rancher is that it allows you to take advantage of Rancher's self-healing node pools, also called the [node auto-replace feature,]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-auto-replace) in your on-premises clusters. Self-healing node pools are designed to help you replace worker nodes for stateless applications. When Rancher provisions nodes from a node template, Rancher can automatically replace unreachable nodes.
|
||||
|
||||
- vCenter server (usually port 443/TCP)
|
||||
- Every ESXi host that is part of the datacenter to be used to provision virtual machines for your clusters (port 443/TCP).
|
||||
> **Important:** It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications.
|
||||
|
||||
### Dynamically Populated Options for Instances and Scheduling
|
||||
|
||||
## Provisioning a vSphere Cluster
|
||||
_Available as of v2.3.3_
|
||||
|
||||
The following steps create a role with the required privileges and then assign it to a new user in the vSphere console:
|
||||
Node templates for vSphere have been updated so that when you create a node template with your vSphere credentials, the template is automatically populated with the same options for provisioning VMs that you have access to in the vSphere console.
|
||||
|
||||
1. From the **vSphere** console, go to the **Administration** page.
|
||||
For the fields to be populated, your setup needs to fulfill the [prerequisites.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#prerequisites)
|
||||
|
||||
2. Go to the **Roles** tab.
|
||||
### More Supported Operating Systems
|
||||
|
||||
3. Create a new role. Give it a name and select the privileges listed in the [permissions table](#annex-vsphere-permissions).
|
||||
In Rancher v2.3.3+, you can provision VMs with any operating system that supports `cloud-init`. Only YAML format is supported for the [cloud config.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html)
|
||||
|
||||
{{< img "/img/rancher/rancherroles1.png" "image">}}
|
||||
In Rancher prior to v2.3.3, the vSphere node driver included in Rancher only supported the provisioning of VMs with [RancherOS]({{<baseurl>}}/os/v1.x/en/) as the guest operating system.
|
||||
|
||||
4. Go to the **Users and Groups** tab.
|
||||
# Video Walkthrough of v2.3.3 Node Template Features
|
||||
|
||||
5. Create a new user. Fill out the form and then click **OK**. Make sure to note the username and password, as you will need it when configuring node templates in Rancher.
|
||||
In this YouTube video, we demonstrate how to set up a node template with the new features designed to help you bring cloud operations to on-premises clusters.
|
||||
|
||||
{{< img "/img/rancher/rancheruser.png" "image">}}
|
||||
|
||||
6. Go to the **Global Permissions** tab.
|
||||
|
||||
7. Create a new Global Permission. Add the user you created earlier and assign it the role you created earlier. Click **OK**.
|
||||
|
||||
{{< img "/img/rancher/globalpermissionuser.png" "image">}}
|
||||
|
||||
{{< img "/img/rancher/globalpermissionrole.png" "image">}}
|
||||
|
||||
## Creating vSphere Clusters
|
||||
|
||||
### Create a vSphere Node Template
|
||||
|
||||
To create a cluster, you need to create at least one vSphere [node template]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) that specifies how VMs are created in vSphere.
|
||||
|
||||
>**Note:**
|
||||
>Once you create a node template, it is saved, and you can re-use it whenever you create additional vSphere clusters.
|
||||
|
||||
1. Log in with an admin account to the Rancher UI.
|
||||
|
||||
2. From the user settings menu, select **Node Templates**.
|
||||
|
||||
3. Click **Add Template** and then click on the **vSphere** icon.
|
||||
|
||||
4. Under [Account Access](#account-access) enter the vCenter FQDN or IP address and the credentials for the vSphere user account (see [Prerequisites](#prerequisites)).
|
||||
|
||||
{{< step_create-cloud-credential >}}
|
||||
|
||||
5. Under [Instance Options](#instance-options), configure the number of vCPUs, memory, and disk size for the VMs created by this template.
|
||||
|
||||
6. **Optional:** Enter the URL pointing to a [RancherOS]({{< baseurl >}}/os/v1.x/en/) cloud-config file in the [Cloud Init](#instance-options) field.
|
||||
|
||||
7. Ensure that the [OS ISO URL](#instance-options) contains the URL of a VMware ISO release for RancherOS (`rancheros-vmware.iso`).
|
||||
|
||||
{{< img "/img/rancher/vsphere-node-template-1.png" "image">}}
|
||||
|
||||
8. **Optional:** Provide a set of [Configuration Parameters](#instance-options) for the VMs.
|
||||
|
||||
9. Under **Scheduling**, enter the name/path of the **Data Center** to create the VMs in, the name of the **VM Network** to attach to, and the name/path of the **Datastore** to store the disks in.
|
||||
|
||||
{{< img "/img/rancher/vsphere-node-template-2.png" "image">}}
|
||||
|
||||
10. **Optional:** Assign labels to the VMs that can be used as a base for scheduling rules in the cluster.
|
||||
|
||||
11. **Optional:** Customize the configuration of the Docker daemon on the VMs that will be created.
|
||||
|
||||
10. Assign a descriptive **Name** for this template and click **Create**.
|
||||
|
||||
___
|
||||
|
||||
### Create a vSphere Cluster
|
||||
|
||||
After you've created a template, you can use it stand up the vSphere cluster itself.
|
||||
|
||||
1. From the **Global** view, click **Add Cluster**.
|
||||
|
||||
2. Choose **vSphere**.
|
||||
|
||||
3. Enter a **Cluster Name**.
|
||||
|
||||
4. {{< step_create-cluster_member-roles >}}
|
||||
|
||||
5. {{< step_create-cluster_cluster-options >}}
|
||||
|
||||
6. {{< step_create-cluster_node-pools >}}
|
||||
|
||||
{{< img "/img/rancher/vsphere-cluster-create-1.png" "Image">}}
|
||||
|
||||
7. Review your configuration, then click **Create**.
|
||||
|
||||
> **Note:**
|
||||
>
|
||||
> If you have a cluster with DRS enabled, setting up [VM-VM Affinity Rules](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.resmgmt.doc/GUID-7297C302-378F-4AF2-9BD6-6EDB1E0A850A.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.
|
||||
|
||||
{{< result_create-cluster >}}
|
||||
|
||||
## Annex - Node Template Configuration Reference
|
||||
|
||||
The tables below describe the configuration options available in the vSphere node template.
|
||||
|
||||
### Account Access
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------:|:--------:|:------------------------------------------------------------:|
|
||||
| vCenter or ESXi Server | * | IP or FQDN of the vCenter or ESXi server used for managing VMs. |
|
||||
| Port | * | Port to use when connecting to the server. Defaults to `443`. |
|
||||
| Username | * | vCenter/ESXi user to authenticate with the server. |
|
||||
| Password | * | User's password. |
|
||||
|
||||
___
|
||||
|
||||
### Instance Options
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------:|:--------:|:------------------------------------------------------------:|
|
||||
| CPUs | * | Number of vCPUS to assign to VMs. |
|
||||
| Memory | * | Amount of memory to assign to VMs. |
|
||||
| Disk | * | Size of the disk (in MB) to attach to the VMs. |
|
||||
| Cloud Init | | URL of a [RancherOS cloud-config]({{< baseurl >}}/os/v1.x/en/installation/configuration/) file to provision VMs with. This file allows further customization of the RancherOS operating system, such as network configuration, DNS servers, or system daemons.|
|
||||
| OS ISO URL | * | URL of a RancherOS vSphere ISO file to boot the VMs from. You can find URLs for specific versions in the [Rancher OS GitHub Repo](https://github.com/rancher/os). |
|
||||
| Configuration Parameters | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). |
|
||||
|
||||
___
|
||||
|
||||
### Scheduling Options
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------:|:--------:|:------------------------------------------------------------:|
|
||||
| Data Center | * | Name/path of the datacenter to create VMs in. |
|
||||
| Pool | | Name/path of the resource pool to schedule the VMs in. If not specified, the default resource pool is used. |
|
||||
| Host | | Name/path of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. |
|
||||
| Network | * | Name of the VM network to attach VMs to. |
|
||||
| Data Store | * | Datastore to store the VM disks. |
|
||||
| Folder | | Name/path of folder in the datastore to create the VMs in. Must already exist. |
|
||||
|
||||
___
|
||||
|
||||
## Annex - vSphere Permissions
|
||||
|
||||
The following table lists the permissions required for the vSphere user account configured in the node templates:
|
||||
|
||||
| Privilege Group | Operations |
|
||||
|:----------------------|:-----------------------------------------------------------------------|
|
||||
| Datastore | AllocateSpace </br> Browse </br> FileManagement (Low level file operations) </br> UpdateVirtualMachineFiles </br> UpdateVirtualMachineMetadata |
|
||||
| Network | Assign |
|
||||
| Resource | AssignVMToPool |
|
||||
| Virtual Machine | Config (All) </br> GuestOperations (All) </br> Interact (All) </br> Inventory (All) </br> Provisioning (All) |
|
||||
{{< youtube id="dPIwg6x1AlU">}}
|
||||
|
||||
+305
@@ -0,0 +1,305 @@
|
||||
---
|
||||
title: Provisioning Kubernetes Clusters in vSphere
|
||||
weight: 1
|
||||
---
|
||||
|
||||
This section explains how to configure Rancher with vSphere credentials, provision nodes in vSphere, and set up Kubernetes clusters on those nodes.
|
||||
|
||||
# Prerequisites
|
||||
|
||||
This section describes the requirements for setting up vSphere so that Rancher can provision VMs and clusters.
|
||||
|
||||
The node templates are documented and tested with the vSphere Web Services API version 6.5.
|
||||
|
||||
- [Create credentials in vSphere](#create-credentials-in-vsphere)
|
||||
- [Network permissions](#network-permissions)
|
||||
- [Valid ESXi License for vSphere API Access](#valid-esxi-license-for-vsphere-api-access)
|
||||
|
||||
### Create Credentials in vSphere
|
||||
|
||||
Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. When you set up a node template, the template will need to use these vSphere credentials.
|
||||
|
||||
Refer to this [how-to guide]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials) for instructions on how to create a user in vSphere with the required permissions. These steps result in a username and password that you will need to provide to Rancher, which allows Rancher to provision resources in vSphere.
|
||||
|
||||
### Network Permissions
|
||||
|
||||
There needs to be two-way communication between Rancher and the vSphere API.
|
||||
|
||||
You must ensure that the hosts running Rancher servers are able to establish network connections to the following network endpoints:
|
||||
|
||||
- vCenter server (usually port 443/TCP)
|
||||
- Every ESXi host that is part of the datacenter to be used to provision virtual machines for your clusters (port 443/TCP).
|
||||
|
||||
By default, Rancher uses port 443 to communicate with vSphere.
|
||||
|
||||
The vSphere API websocket port will be 84453 by default.
|
||||
|
||||
### Valid ESXi License for vSphere API Access
|
||||
|
||||
The free ESXi license does not support API access. The vSphere servers must have a valid or evaluation ESXi license.
|
||||
|
||||
# Creating Clusters in vSphere with Rancher
|
||||
|
||||
This section describes how to set up vSphere credentials, node templates, and vSphere clusters using the Rancher UI.
|
||||
|
||||
You will need to do the following:
|
||||
|
||||
1. [Create a node template using vSphere credentials](#1-create-a-node-template-using-vsphere-credentials)
|
||||
2. [Create a Kubernetes cluster using the node template](#2-create-a-kubernetes-cluster-using-the-node-template)
|
||||
3. [Optional: Provision storage](#3-optional-provision-storage)
|
||||
- [Enable the vSphere cloud provider for the cluster](#enable-the-vsphere-cloud-provider-for-the-cluster)
|
||||
|
||||
### Configuration References
|
||||
|
||||
For details on configuring the node template, refer to the [node template configuration reference.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/)
|
||||
|
||||
Rancher uses the RKE library to provision Kubernetes clusters. For details on configuring clusters in vSphere, refer to the [cluster configuration reference in the RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/)
|
||||
|
||||
Note that the vSphere cloud provider must be [enabled](#enable-the-vsphere-cloud-provider-for-the-cluster) to allow dynamic provisioning of volumes.
|
||||
|
||||
# 1. Create a Node Template Using vSphere Credentials
|
||||
|
||||
To create a cluster, you need to create at least one vSphere [node template]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) that specifies how VMs are created in vSphere.
|
||||
|
||||
After you create a node template, it is saved, and you can re-use it whenever you create additional vSphere clusters.
|
||||
|
||||
To create a node template,
|
||||
|
||||
1. Log in with an administrator account to the Rancher UI.
|
||||
|
||||
1. From the user settings menu, select **Node Templates.**
|
||||
|
||||
1. Click **Add Template** and then click on the **vSphere** icon.
|
||||
|
||||
Then, configure your template:
|
||||
|
||||
- [A. Configure the vSphere credential](#a-configure-the-vsphere-credential)
|
||||
- [B. Configure node scheduling](#b-configure-node-scheduling)
|
||||
- [C. Configure instances and operating systems](#c-configure-instances-and-operating-systems)
|
||||
- [D. Add networks](#d-add-networks)
|
||||
- [E. If not already enabled, enable disk UUIDs](#e-if-not-already-enabled-enable-disk-uuids)
|
||||
- [F. Optional: Configure node tags and custom attributes](#f-optional-configure-node-tags-and-custom-attributes)
|
||||
- [G. Optional: Configure cloud-init](#g-optional-configure-cloud-init)
|
||||
- [H. Saving the node template](#h-saving-the-node-template)
|
||||
|
||||
### A. Configure the vSphere Credential
|
||||
|
||||
The steps for configuring your vSphere credentials for the cluster are different depending on your version of Rancher.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.2.0+" %}}
|
||||
|
||||
Your account access information is in a [cloud credential.]({{<baseurl>}}/rancher/v2.x/en/user-settings/cloud-credentials/) Cloud credentials are stored as Kubernetes secrets.
|
||||
|
||||
You can use an existing cloud credential or create a new one. To create a new cloud credential,
|
||||
|
||||
1. Click **Add New.**
|
||||
1. In the **Name** field, enter a name for your vSphere credentials.
|
||||
1. In the **vCenter or ESXi Server** field, enter the vCenter or ESXi hostname/IP. ESXi is the virtualization platform where you create and run virtual machines and virtual appliances. vCenter Server is the service through which you manage multiple hosts connected in a network and pool host resources.
|
||||
1. Optional: In the **Port** field, configure the port of the vCenter or ESXi server.
|
||||
1. In the **Username** and **Password** fields, enter your vSphere login username and password.
|
||||
1. Click **Create.**
|
||||
|
||||
**Result:** The node template has the credentials required to provision nodes in vSphere.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.2.0" %}}
|
||||
In the **Account Access** section, enter the vCenter FQDN or IP address and the credentials for the vSphere user account.
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
### B. Configure Node Scheduling
|
||||
|
||||
Choose what hypervisor the virtual machine will be scheduled to. The configuration options depend on your version of Rancher.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.3.3+" %}}
|
||||
|
||||
The fields in the **Scheduling** section should auto-populate with the data center and other scheduling options that are available to you in vSphere.
|
||||
|
||||
1. In the **Data Center** field, choose the data center where the VM will be scheduled.
|
||||
1. Optional: Select a **Resource Pool.** Resource pools can be used to partition available CPU and memory resources of a standalone host or cluster, and they can also be nested.
|
||||
1. If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk.
|
||||
1. Optional: Select a folder where the VM will be placed. The VM folders in this dropdown menu directly correspond to your VM folders in vSphere. Note: The folder name should be prefaced with `vm/` in your vSphere config file.
|
||||
1. Optional: Choose a specific host to create the VM on. Leave this field blank for a standalone ESXi or for a cluster with DRS (Distributed Resource Scheduler). If specified, the host system's pool will be used and the **Resource Pool** parameter will be ignored.
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.3.3" %}}
|
||||
|
||||
In the **Scheduling** section, enter:
|
||||
|
||||
- The name/path of the **Data Center** to create the VMs in
|
||||
- The name of the **VM Network** to attach to
|
||||
- The name/path of the **Datastore** to store the disks in
|
||||
|
||||
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
### C. Configure Instances and Operating Systems
|
||||
|
||||
The instances are configured differently depending on your Rancher version.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.3.3+" %}}
|
||||
|
||||
In this section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.
|
||||
|
||||
In the **Creation method** field, you will configure the method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template.
|
||||
|
||||
[VM templates](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html) are useful for setting up the operating system and other software, because they allow you to save time. For example, you could use a VM template to automatically install Kubernetes and Docker on each node. You can choose ISOs defined from templates in a vSphere data center or content library.
|
||||
|
||||
The node can be created with any operating system that supports `cloud-init`.
|
||||
|
||||
Choose the way that the VM will be created:
|
||||
|
||||
- **Deploy from template: Data Center:** Choose a template that exists in the data center that you selected.
|
||||
- **Deploy from template: Content Library:** In the two fields that appear when you select this option, choose the [content library](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-254B2CE8-20A8-43F0-90E8-3F6776C2C896.html). Then select the [VM template](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html) from the list of templates within the content library. This template will be used to create the new VM.
|
||||
- **Clone an existing virtual machine:** In the **Virtual machine** field, choose an existing VM that the new VM will be cloned from.
|
||||
- **Install from boot2docker ISO:** Ensure that the OS ISO URL contains the URL of a VMware ISO release for RancherOS (rancheros-vmware.iso).
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.3.3" %}}
|
||||
|
||||
In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.
|
||||
|
||||
Only RancherOS VMs are supported.
|
||||
|
||||
Ensure that the [OS ISO URL](#instance-options) contains the URL of the VMware ISO release for RancherOS: `rancheros-vmware.iso`.
|
||||
|
||||
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
### D. Add Networks
|
||||
|
||||
_Available as of v2.3.3_
|
||||
|
||||
The node template now allows a VM to be provisioned with multiple networks. In the **Networks** field, you can now click **Add Network** to add any networks available to you in vSphere.
|
||||
|
||||
### E. If Not Already Enabled, Enable Disk UUIDs
|
||||
|
||||
In order to provision nodes with RKE, all nodes must be configured with disk UUIDs.
|
||||
|
||||
As of Rancher v2.0.4, disk UUIDs are enabled in vSphere node templates by default.
|
||||
|
||||
If you are using Rancher prior to v2.0.4, refer to these [instructions]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#enabling-disk-uuids-with-a-node-template) for details on how to enable a UUID with a Rancher node template.
|
||||
|
||||
### F. Optional: Configure Node Tags and Custom Attributes
|
||||
|
||||
The way to attach metadata to the VM is different depending on your Rancher version.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.3.3+" %}}
|
||||
|
||||
**Optional:** Add vSphere tags and custom attributes. Tags allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.
|
||||
|
||||
For tags, all your vSphere tags will show up as options to select from in your node template.
|
||||
|
||||
In the custom attributes, Rancher will let you select all the custom attributes you have already set up in vSphere. The custom attributes are keys and you can enter values for each one.
|
||||
|
||||
> **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere. These attributes allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.3.3" %}}
|
||||
|
||||
**Optional:**
|
||||
|
||||
- Provide a set of configuration parameters (instance-options) for the VMs.
|
||||
- Assign labels to the VMs that can be used as a base for scheduling rules in the cluster.
|
||||
- Customize the configuration of the Docker daemon on the VMs that will be created.
|
||||
|
||||
> **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere. These attributes allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
### G. Optional: Configure cloud-init
|
||||
|
||||
[Cloud-init](https://cloud-init.io/) is a tool that applies user data to your nodes when they boot for the first time.
|
||||
|
||||
The configuration file for `cloud-init` is named `cloud-config.yml.` In the **Cloud Init** field, it is optional to enter a file name or URL pointing to a `cloud-config.yml` file. Only YAML format is supported for the cloud config.
|
||||
|
||||
You can use `cloud-init` to automate tasks that should happen when the instance boots, such as creating users, running shell commands, adding a load balancer, or preinstalling Kubernetes on the VM.
|
||||
|
||||
For examples of how to write a `cloud-config` file, refer to the [cloud-init documentation.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html)
|
||||
|
||||
### H. Saving the Node Template
|
||||
|
||||
Assign a descriptive **Name** for this template and click **Create.**
|
||||
|
||||
### Node Template Configuration Reference
|
||||
|
||||
Refer to [this section]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/) for a reference on the configuration options available for vSphere node templates.
|
||||
|
||||
# 2. Create a Kubernetes Cluster Using the Node Template
|
||||
|
||||
After you've created a template, you can use it to stand up the vSphere cluster itself.
|
||||
|
||||
To install Kubernetes on vSphere nodes, you will need to enable the vSphere cloud provider by modifying the cluster YAML file. This requirement applies to both pre-created [custom nodes]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/) and for nodes created in Rancher using the vSphere node driver.
|
||||
|
||||
To create the cluster and enable the vSphere provider for cluster, follow these steps:
|
||||
|
||||
- [A. Set up the cluster name and member roles](#a-set-up-the-cluster-name-and-member-roles)
|
||||
- [B. Configure Kubernetes options](#b-configure-kubernetes-options)
|
||||
- [C. Add node pools to the cluster](#c-add-node-pools-to-the-cluster)
|
||||
- [D. Optional: Add a self-healing node pool](#d-optional-add-a-self-healing-node-pool)
|
||||
- [E. Create the cluster](#e-create-the-cluster)
|
||||
|
||||
### A. Set up the Cluster Name and Member Roles
|
||||
|
||||
1. Log in to the Rancher UI as an administrator.
|
||||
2. Navigate to **Clusters** in the **Global** view.
|
||||
3. Click **Add Cluster** and select the **vSphere** infrastructure provider.
|
||||
4. Assign a **Cluster Name.**
|
||||
5. Assign **Member Roles** as required. {{< step_create-cluster_member-roles >}}
|
||||
|
||||
> **Note:**
|
||||
>
|
||||
> If you have a cluster with DRS enabled, setting up [VM-VM Affinity Rules](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.resmgmt.doc/GUID-7297C302-378F-4AF2-9BD6-6EDB1E0A850A.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.
|
||||
|
||||
|
||||
### B. Configure Kubernetes Options
|
||||
{{<step_create-cluster_cluster-options>}}
|
||||
|
||||
### C. Add Node Pools to the Cluster
|
||||
{{<step_create-cluster_node-pools>}}
|
||||
|
||||
### D. Optional: Add a Self-Healing Node Pool
|
||||
|
||||
To make a node pool self-healing, enter a number greater than zero in the **Auto Replace** column. Rancher will use the node template for the given node pool to recreate the node if it becomes inactive for that number of minutes.
|
||||
|
||||
> **Note:** Self-healing node pools are designed to help you replace worker nodes for stateless applications. It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications.
|
||||
|
||||
### E. Create the Cluster
|
||||
|
||||
Click **Create** to start provisioning the VMs and Kubernetes services.
|
||||
|
||||
{{< result_create-cluster >}}
|
||||
|
||||
# 3. Optional: Provision Storage
|
||||
|
||||
For an example of how to provision storage in vSphere using Rancher, refer to the
|
||||
[cluster administration section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere)
|
||||
|
||||
In order to provision storage in vSphere, the vSphere provider must be enabled.
|
||||
|
||||
### Enable the vSphere Cloud Provider for the Cluster
|
||||
|
||||
1. Set **Cloud Provider** option to `Custom`.
|
||||
|
||||
|
||||
|
||||
1. Click on **Edit as YAML**
|
||||
1. Insert the following structure to the pre-populated cluster YAML. As of Rancher v2.3+, this structure must be placed under `rancher_kubernetes_engine_config`. In versions prior to v2.3, it has to be defined as a top-level field. Note that the `name` *must* be set to `vsphere`.
|
||||
|
||||
```yaml
|
||||
rancher_kubernetes_engine_config: # Required as of Rancher v2.3+
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
[Insert provider configuration]
|
||||
```
|
||||
|
||||
Rancher uses RKE (the Rancher Kubernetes Engine) to provision Kubernetes clusters. Refer to the [vSphere configuration reference in the RKE documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/) for details about the properties of the `vsphereCloudProvider` directive.
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: Creating Credentials in the vSphere Console
|
||||
weight: 1
|
||||
---
|
||||
|
||||
This section describes how to create a vSphere username and password. You will need to provide these vSphere credentials to Rancher, which allows Rancher to provision resources in vSphere.
|
||||
|
||||
The following table lists the permissions required for the vSphere user account:
|
||||
|
||||
| Privilege Group | Operations |
|
||||
|:----------------------|:-----------------------------------------------------------------------|
|
||||
| Datastore | AllocateSpace </br> Browse </br> FileManagement (Low level file operations) </br> UpdateVirtualMachineFiles </br> UpdateVirtualMachineMetadata |
|
||||
| Network | Assign |
|
||||
| Resource | AssignVMToPool |
|
||||
| Virtual Machine | Config (All) </br> GuestOperations (All) </br> Interact (All) </br> Inventory (All) </br> Provisioning (All) |
|
||||
|
||||
The following steps create a role with the required privileges and then assign it to a new user in the vSphere console:
|
||||
|
||||
1. From the **vSphere** console, go to the **Administration** page.
|
||||
|
||||
2. Go to the **Roles** tab.
|
||||
|
||||
3. Create a new role. Give it a name and select the privileges listed in the permissions table above.
|
||||
|
||||
|
||||
|
||||
4. Go to the **Users and Groups** tab.
|
||||
|
||||
5. Create a new user. Fill out the form and then click **OK**. Make sure to note the username and password, because you will need it when configuring node templates in Rancher.
|
||||
|
||||
|
||||
|
||||
6. Go to the **Global Permissions** tab.
|
||||
|
||||
7. Create a new Global Permission. Add the user you created earlier and assign it the role you created earlier. Click **OK**.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**Result:** You now have credentials that Rancher can use to manipulate vSphere resources.
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
---
|
||||
title: Enabling Disk UUIDs in Node Templates
|
||||
weight: 3
|
||||
---
|
||||
|
||||
As of Rancher v2.0.4, disk UUIDs are enabled in vSphere node templates by default.
|
||||
|
||||
For Rancher prior to v2.0.4, we recommend configuring a vSphere node template to automatically enable disk UUIDs because they are required for Rancher to manipulate vSphere resources.
|
||||
|
||||
To enable disk UUIDs for all VMs created for a cluster,
|
||||
|
||||
1. Navigate to the **Node Templates** in the Rancher UI while logged in as an administrator.
|
||||
|
||||
2. Add or edit an existing vSphere node template.
|
||||
|
||||
3. Under **Instance Options** click on **Add Parameter**.
|
||||
|
||||
4. Enter `disk.enableUUID` as key with a value of **TRUE**.
|
||||
|
||||
|
||||
|
||||
5. Click **Create** or **Save**.
|
||||
|
||||
**Result:** The disk UUID is enabled in the vSphere node template.
|
||||
+93
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: vSphere Node Template Configuration Reference
|
||||
weight: 4
|
||||
---
|
||||
|
||||
The tables below describe the configuration options available in the vSphere node template:
|
||||
|
||||
- [Account access](#account-access)
|
||||
- [Instance options](#instance-options)
|
||||
- [Scheduling options](#scheduling-options)
|
||||
|
||||
# Account Access
|
||||
|
||||
The account access parameters are different based on the Rancher version.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.2.0+" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:----------------------|:--------:|:-----|
|
||||
| Cloud Credentials | * | Your vSphere account access information, stored in a [cloud credential.]({{<baseurl>}}/rancher/v2.x/en/user-settings/cloud-credentials/) |
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.2.0" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------|:--------:|:------------------------------------------------------------|
|
||||
| vCenter or ESXi Server | * | IP or FQDN of the vCenter or ESXi server used for managing VMs. |
|
||||
| Port | * | Port to use when connecting to the server. Defaults to `443`. |
|
||||
| Username | * | vCenter/ESXi user to authenticate with the server. |
|
||||
| Password | * | User's password. |
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# Instance Options
|
||||
|
||||
The options for creating and configuring an instance are different depending on your Rancher version.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.3.3+" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:----------------|:--------:|:-----------|
|
||||
| CPUs | * | Number of vCPUS to assign to VMs. |
|
||||
| Memory | * | Amount of memory to assign to VMs. |
|
||||
| Disk | * | Size of the disk (in MB) to attach to the VMs. |
|
||||
| Creation method | * | The method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template. Depending on the creation method, you will also have to specify a VM template, content library, existing VM, or ISO. For more information on creation methods, refer to the section on [configuring instances.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#c-configure-instances-and-operating-systems) |
|
||||
| Cloud Init | | URL of a `cloud-config.yml` file or URL to provision VMs with. This file allows further customization of the operating system, such as network configuration, DNS servers, or system daemons. The operating system must support `cloud-init`. |
|
||||
| Networks | | Name(s) of the network to attach the VM to. |
|
||||
| Configuration Parameters used for guestinfo | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). |
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.3.3" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------|:--------:|:------------------------------------------------------------|
|
||||
| CPUs | * | Number of vCPUS to assign to VMs. |
|
||||
| Memory | * | Amount of memory to assign to VMs. |
|
||||
| Disk | * | Size of the disk (in MB) to attach to the VMs. |
|
||||
| Cloud Init | | URL of a [RancherOS cloud-config]({{< baseurl >}}/os/v1.x/en/installation/configuration/) file to provision VMs with. This file allows further customization of the RancherOS operating system, such as network configuration, DNS servers, or system daemons.|
|
||||
| OS ISO URL | * | URL of a RancherOS vSphere ISO file to boot the VMs from. You can find URLs for specific versions in the [Rancher OS GitHub Repo](https://github.com/rancher/os). |
|
||||
| Configuration Parameters | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). |
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# Scheduling Options
|
||||
The options for scheduling VMs to a hypervisor are different depending on your Rancher version.
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher v2.3.3+" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------|:--------:|:-------|
|
||||
| Data Center | * | Name/path of the datacenter to create VMs in. |
|
||||
| Resource Pool | | Name of the resource pool to schedule the VMs in. Leave blank for standalone ESXi. If not specified, the default resource pool is used. |
|
||||
| Data Store | * | If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk. |
|
||||
| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The folder name should be prefaced with `vm/` in your vSphere config file. |
|
||||
| Host | | The IP of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. |
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher prior to v2.3.3" %}}
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|:------------------------|:--------:|:------------------------------------------------------------|
|
||||
| Data Center | * | Name/path of the datacenter to create VMs in. |
|
||||
| Pool | | Name/path of the resource pool to schedule the VMs in. If not specified, the default resource pool is used. |
|
||||
| Host | | Name/path of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. |
|
||||
| Network | * | Name of the VM network to attach VMs to. |
|
||||
| Data Store | * | Datastore to store the VM disks. |
|
||||
| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The folder name should be prefaced with `vm/` in your vSphere config file. |
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
@@ -71,7 +71,7 @@ See the [RKE documentation on private registries]({{< baseurl >}}/rke/latest/en/
|
||||
|
||||
_Available as of v2.2.0_
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher. This is enabled by default, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates. It is recommended to create an FQDN pointing to a load balancer which load balances across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate which will be included in the generated kubeconfig to validate the certificate chain. See the [Kubeconfig Files]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubeconfig/) and [API Keys]({{< baseurl >}}/v2.x/en/user-settings/api-keys/#creating-an-api-key) documentation for more information.
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher. This is enabled by default, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates. It is recommended to create an FQDN pointing to a load balancer which load balances across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate which will be included in the generated kubeconfig to validate the certificate chain. See the [Kubeconfig Files]({{<baseurl>}}/rancher/v2.x/en/k8s-in-rancher/kubeconfig/) and [API Keys]({{<baseurl>}}/rancher/v2.x/en/user-settings/api-keys/#creating-an-api-key) documentation for more information.
|
||||
|
||||
### Advanced Cluster Options
|
||||
|
||||
@@ -349,4 +349,4 @@ There are two ways that you can specify an add-on:
|
||||
- [In-line Add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [Referencing YAML Files for Add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
|
||||
+4
-10
@@ -42,16 +42,10 @@ For a custom cluster, the general node requirements for networking, operating sy
|
||||
|
||||
### OS and Docker Requirements
|
||||
|
||||
In order to add Windows worker nodes to a cluster, the node must be running one of the following Windows Server versions and the corresponding version of Docker:
|
||||
In order to add Windows worker nodes to a cluster, the node must be running one of the following Windows Server versions and the corresponding version of Docker Engine - Enterprise Edition (EE):
|
||||
|
||||
- Windows Server core version 1809 and Docker 18.09
|
||||
- Windows server core version 1903 and Docker 19.03
|
||||
|
||||
The nodes must run Docker Engine - Enterprise Edition (EE).
|
||||
|
||||
Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09.
|
||||
|
||||
Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03.
|
||||
- Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09 or Docker EE-basic 19.03.
|
||||
- Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03.
|
||||
|
||||
> **Notes:**
|
||||
>
|
||||
@@ -160,7 +154,7 @@ Windows support only be enabled if the cluster uses Kubernetes v1.15+ and the Fl
|
||||
|
||||
1. Click **Next**.
|
||||
|
||||
> **Important:** For **Host Gateway (L2bridge)** networking, it's best to use the same Layer 2 network for all nodes. Otherwise, you need to configure the route rules for them. For details, refer to the [documentation on configuring cloud-hosted VM routes.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#cloud-hosted-vm-routes-configuration) You will also need to [disable private IP address checks]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#disabling-private-ip-address-checks) if you are using Amazon EC2, Google GCE, or Azure VM.
|
||||
> **Important:** For <b>Host Gateway (L2bridge)</b> networking, it's best to use the same Layer 2 network for all nodes. Otherwise, you need to configure the route rules for them. For details, refer to the [documentation on configuring cloud-hosted VM routes.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#cloud-hosted-vm-routes-configuration) You will also need to [disable private IP address checks]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#disabling-private-ip-address-checks) if you are using Amazon EC2, Google GCE, or Azure VM.
|
||||
|
||||
# 3. Add Nodes to the Cluster
|
||||
|
||||
|
||||
@@ -3,12 +3,12 @@ title: Technical
|
||||
weight: 8006
|
||||
---
|
||||
|
||||
### How can I reset the admin password?
|
||||
### How can I reset the administrator password?
|
||||
|
||||
Single node install:
|
||||
```
|
||||
$ docker exec -ti <container_id> reset-password
|
||||
New password for default admin user (user-xxxxx):
|
||||
New password for default administrator (user-xxxxx):
|
||||
<new_password>
|
||||
```
|
||||
|
||||
@@ -16,7 +16,7 @@ High Availability install (Helm):
|
||||
```
|
||||
$ KUBECONFIG=./kube_config_rancher-cluster.yml
|
||||
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password
|
||||
New password for default admin user (user-xxxxx):
|
||||
New password for default administrator (user-xxxxx):
|
||||
<new_password>
|
||||
```
|
||||
|
||||
@@ -24,7 +24,7 @@ High Availability install (RKE add-on):
|
||||
```
|
||||
$ KUBECONFIG=./kube_config_rancher-cluster.yml
|
||||
$ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- reset-password
|
||||
New password for default admin user (user-xxxxx):
|
||||
New password for default administrator (user-xxxxx):
|
||||
<new_password>
|
||||
```
|
||||
|
||||
@@ -33,8 +33,8 @@ New password for default admin user (user-xxxxx):
|
||||
Single node install:
|
||||
```
|
||||
$ docker exec -ti <container_id> ensure-default-admin
|
||||
New default admin user (user-xxxxx)
|
||||
New password for default admin user (user-xxxxx):
|
||||
New default administrator (user-xxxxx)
|
||||
New password for default administrator (user-xxxxx):
|
||||
<new_password>
|
||||
```
|
||||
|
||||
@@ -42,7 +42,7 @@ High Availability install (Helm):
|
||||
```
|
||||
$ KUBECONFIG=./kube_config_rancher-cluster.yml
|
||||
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- ensure-default-admin
|
||||
New password for default admin user (user-xxxxx):
|
||||
New password for default administrator (user-xxxxx):
|
||||
<new_password>
|
||||
```
|
||||
|
||||
|
||||
@@ -29,4 +29,4 @@ If Telemetry is not enabled, the process that collects the data is not running,
|
||||
|
||||
### How do I turn it on or off?
|
||||
|
||||
After initial setup, an admin user can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
|
||||
After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
|
||||
|
||||
@@ -5,11 +5,11 @@ weight: 275
|
||||
|
||||
For production environments, we recommend installing Rancher in a high-availability configuration so that your user base can always access Rancher Server. When installed in a Kubernetes cluster, Rancher will integrate with the cluster's etcd database and take advantage of Kubernetes scheduling for high-availability.
|
||||
|
||||
This procedure walks you through setting up a 3-node cluster with RKE and installing the Rancher chart with the Helm package manager.
|
||||
This procedure walks you through setting up a 3-node cluster with Rancher Kubernetes Engine (RKE) and installing the Rancher chart with the Helm package manager.
|
||||
|
||||
> **Important:** It is not supported, nor generally a good idea, to run Rancher on top of hosted Kubernetes solutions such as Amazon's EKS, or Google's GKE. These hosted Kubernetes solutions do not expose etcd to a degree that is manageable for Rancher, and their customizations can interfere with Rancher operations. It is strongly recommended to use hosted infrastructure such as EC2 or GCE instead.
|
||||
> **Important:** The Rancher management server can only be run on an RKE-managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported.
|
||||
|
||||
> **Important:** For the best performance, we recommend this Kubernetes cluster to be dedicated only to run Rancher. After the Kubernetes cluster to run Rancher is setup, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads.
|
||||
> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads.
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
@@ -34,7 +34,6 @@ The following CLI tools are required for this install. Please make sure these to
|
||||
|
||||
- [Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/)
|
||||
- [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/)
|
||||
- [Initialize Helm (tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
|
||||
- [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
|
||||
|
||||
## Additional Install Options
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
---
|
||||
title: "3. Initialize Helm (Install Tiller)"
|
||||
weight: 195
|
||||
---
|
||||
|
||||
Helm is the package management tool of choice for Kubernetes. Helm "charts" provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh/). To be able to use Helm, the server-side component `tiller` needs to be installed on your cluster.
|
||||
|
||||
For systems without direct internet access, see [Helm - Air Gap]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#helm) for install details.
|
||||
|
||||
Refer to the [Helm version requirements]({{<baseurl>}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher.
|
||||
|
||||
### Install Tiller on the Cluster
|
||||
|
||||
> **Important:** Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher.
|
||||
|
||||
Helm installs the `tiller` service on your cluster to manage charts. Since RKE enables RBAC by default we will need to use `kubectl` to create a `serviceaccount` and `clusterrolebinding` so `tiller` has permission to deploy to the cluster.
|
||||
|
||||
* Create the `ServiceAccount` in the `kube-system` namespace.
|
||||
* Create the `ClusterRoleBinding` to give the `tiller` account access to the cluster.
|
||||
* Finally use `helm` to install the `tiller` service
|
||||
|
||||
```plain
|
||||
kubectl -n kube-system create serviceaccount tiller
|
||||
|
||||
kubectl create clusterrolebinding tiller \
|
||||
--clusterrole=cluster-admin \
|
||||
--serviceaccount=kube-system:tiller
|
||||
|
||||
helm init --service-account tiller
|
||||
|
||||
# Users in China: You will need to specify a specific tiller-image in order to initialize tiller.
|
||||
# The list of tiller image tags are available here: https://dev.aliyun.com/detail.html?spm=5176.1972343.2.18.ErFNgC&repoId=62085.
|
||||
# When initializing tiller, you'll need to pass in --tiller-image
|
||||
|
||||
helm init --service-account tiller \
|
||||
--tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:<tag>
|
||||
```
|
||||
|
||||
> **Note:** This`tiller`install has full cluster access, which should be acceptable if the cluster is dedicated to Rancher server. Check out the [helm docs](https://docs.helm.sh/using_helm/#role-based-access-control) for restricting `tiller` access to suit your security requirements.
|
||||
|
||||
### Test your Tiller installation
|
||||
|
||||
Run the following command to verify the installation of `tiller` on your cluster:
|
||||
|
||||
```
|
||||
kubectl -n kube-system rollout status deploy/tiller-deploy
|
||||
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
|
||||
deployment "tiller-deploy" successfully rolled out
|
||||
```
|
||||
|
||||
And run the following command to validate Helm can talk to the `tiller` service:
|
||||
|
||||
```
|
||||
helm version
|
||||
Client: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
|
||||
Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
|
||||
```
|
||||
|
||||
### Issues or errors?
|
||||
|
||||
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/) page.
|
||||
|
||||
### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
title: Troubleshooting
|
||||
weight: 276
|
||||
---
|
||||
|
||||
### Helm commands show forbidden
|
||||
|
||||
When Helm is initiated in the cluster without specifying the correct `ServiceAccount`, the command `helm init` will succeed but you won't be able to execute most of the other `helm` commands. The following error will be shown:
|
||||
|
||||
```
|
||||
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
|
||||
```
|
||||
|
||||
To resolve this, the server component (`tiller`) needs to be removed and added with the correct `ServiceAccount`. You can use `helm reset --force` to remove the `tiller` from the cluster. Please check if it is removed using `helm version --server`.
|
||||
|
||||
```
|
||||
helm reset --force
|
||||
Tiller (the Helm server-side component) has been uninstalled from your Kubernetes Cluster.
|
||||
helm version --server
|
||||
Error: could not find tiller
|
||||
```
|
||||
|
||||
When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) to install `tiller` with the correct `ServiceAccount`.
|
||||
@@ -1,14 +1,20 @@
|
||||
---
|
||||
title: "4. Install Rancher"
|
||||
title: "3. Install Rancher"
|
||||
weight: 200
|
||||
---
|
||||
|
||||
Rancher installation is managed using the Helm package manager for Kubernetes. Use `helm` to install the prerequisite and charts to install Rancher.
|
||||
Rancher installation is managed using the Helm package manager for Kubernetes. Helm “charts” provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at https://helm.sh/.
|
||||
|
||||
For systems without direct internet access, see [Air Gap: High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/).
|
||||
|
||||
Refer to the [Helm version requirements]({{<baseurl>}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher.
|
||||
|
||||
> **Note:** The installation instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 Migration Docs](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/)
|
||||
|
||||
### Install Helm
|
||||
|
||||
Helm requires a simple CLI tool to be installed. Refer to the [instructions provided by the Helm project](https://helm.sh/docs/intro/install/) for your specific platofrm.
|
||||
|
||||
### Add the Helm Chart Repository
|
||||
|
||||
Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories).
|
||||
@@ -19,6 +25,13 @@ Use `helm repo add` command to add the Helm chart repository that contains chart
|
||||
helm repo add rancher-<CHART_REPO> https://releases.rancher.com/server-charts/<CHART_REPO>
|
||||
```
|
||||
|
||||
### Create a Namespace for Rancher
|
||||
We'll need to define a namespace where the resources created by the Chart should be installed. This should always be `cattle-system`:
|
||||
|
||||
```
|
||||
kubectl create namespace cattle-system
|
||||
```
|
||||
|
||||
### Choose your SSL Configuration
|
||||
|
||||
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
|
||||
@@ -75,10 +88,9 @@ These instructions are adapted from the [official cert-manager documentation](ht
|
||||
1. Install the cert-manager Helm chart
|
||||
```plain
|
||||
helm install \
|
||||
--name cert-manager \
|
||||
cert-manager jetstack/cert-manager \
|
||||
--namespace cert-manager \
|
||||
--version v0.9.1 \
|
||||
jetstack/cert-manager
|
||||
--version v0.9.1
|
||||
```
|
||||
|
||||
Once you’ve installed cert-manager, you can verify it is deployed correctly by checking the cert-manager namespace for running pods:
|
||||
@@ -105,8 +117,7 @@ The default is for Rancher to generate a CA and uses `cert-manager` to issue the
|
||||
- Set the `hostname` to the DNS name you pointed at your load balancer.
|
||||
|
||||
```
|
||||
helm install rancher-<CHART_REPO>/rancher \
|
||||
--name rancher \
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org
|
||||
```
|
||||
@@ -128,8 +139,7 @@ This option uses `cert-manager` to automatically request and renew [Let's Encryp
|
||||
- Set `hostname` to the public DNS record, set `ingress.tls.source` to `letsEncrypt` and `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices)
|
||||
|
||||
```
|
||||
helm install rancher-<CHART_REPO>/rancher \
|
||||
--name rancher \
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=letsEncrypt \
|
||||
@@ -155,8 +165,7 @@ Create Kubernetes secrets from your own certificates for Rancher to use.
|
||||
- If you are using a Private CA signed certificate , add `--set privateCA=true` to the command shown below.
|
||||
|
||||
```
|
||||
helm install rancher-<CHART_REPO>/rancher \
|
||||
--name rancher \
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=secret
|
||||
|
||||
@@ -127,4 +127,4 @@ Save a copy of the following files in a secure location:
|
||||
|
||||
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/troubleshooting/) page.
|
||||
|
||||
### [Next: Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
|
||||
### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
|
||||
|
||||
@@ -5,6 +5,8 @@ weight: 400
|
||||
|
||||
This section contains the requirements for Helm, which is the tool used to install Rancher on a high-availability Kubernetes cluster.
|
||||
|
||||
- Helm v2.15.1 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using.
|
||||
> **Note:** The installation instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 Migration Docs](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/)
|
||||
|
||||
- Helm v2.16.0 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using.
|
||||
- Helm v2.15.0 should not be used, because of an issue with converting/comparing numbers.
|
||||
- Helm v2.12.0 should not be used, because of an issue with `cert-manager`.
|
||||
- Helm v2.12.0 should not be used, because of an issue with `cert-manager`.
|
||||
|
||||
@@ -28,33 +28,18 @@ Rancher is tested on the following operating systems and their subsequent non-ma
|
||||
|
||||
For details on which OS and Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
|
||||
|
||||
- Ubuntu 16.04 (64-bit x86)
|
||||
- Docker 17.03.x, 18.06.x, 18.09.x
|
||||
- Ubuntu 18.04 (64-bit x86)
|
||||
- Docker 18.06.x, 18.09.x
|
||||
- Red Hat Enterprise Linux (RHEL)/CentOS 7.6 (64-bit x86)
|
||||
- RHEL Docker 1.13
|
||||
- Oracle Linux 7 update 6* (64-bit x86)
|
||||
- Docker 17.03.x, 18.06.x, 18.09.x
|
||||
- RancherOS 1.5.1 (64-bit x86)
|
||||
- Docker 17.03.x, 18.06.x, 18.09.x
|
||||
- Windows Server 2019 (64-bit x86)
|
||||
- Requires Docker Engine - Enterprise Edition (EE)
|
||||
- Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09
|
||||
- Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03
|
||||
- Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/)
|
||||
Operating System | Tested Docker Versions
|
||||
---------------------|--------------------------
|
||||
Ubuntu 16.04 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x
|
||||
Ubuntu 18.04 (64-bit x86) | Docker 18.06.x, 18.09.x, 19.03.x
|
||||
Red Hat Enterprise Linux (RHEL)/CentOS 7.7 (64-bit x86) | RHEL Docker 1.13
|
||||
Oracle Linux 7 update 6* (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x
|
||||
RancherOS 1.5.4 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x
|
||||
Windows Server 2019 (64-bit x86) | Requires Docker Engine - Enterprise Edition (EE).**
|
||||
|
||||
\* Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. This [how-to guide]({{<baseurl>}}/rancher/v2.x/en/installation/options/firewall) shows how to check the default firewall rules and how to open the ports with `firewalld` if necessary.
|
||||
|
||||
If you are using RancherOS, make sure you switch the Docker engine to a supported version using:
|
||||
|
||||
```
|
||||
# Look up available versions
|
||||
sudo ros engine list
|
||||
|
||||
# Switch to a supported version
|
||||
sudo ros engine switch docker-18.09.2
|
||||
```
|
||||
\** Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09 or Docker EE-basic 19.03. Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/)
|
||||
|
||||
If you plan to run Rancher on ARM64, see [Running on ARM64 (Experimental)]({{<baseurl>}}/rancher/v2.x/en/installation/arm64-platform/)
|
||||
|
||||
|
||||
@@ -3,13 +3,13 @@ title: Load Balancing and Ingresses
|
||||
weight: 3040
|
||||
---
|
||||
|
||||
Within Rancher, you can setup load balancers and ingress controllers to redirect service requests.
|
||||
Within Rancher, you can set up load balancers and ingress controllers to redirect service requests.
|
||||
|
||||
## Load Balancers
|
||||
|
||||
After you launch an application, the app is only available within the cluster. It can't be reached from outside the cluster.
|
||||
|
||||
If you want your applications to be externally accessible, you must add a load balancer to your cluster. Load balancers create a gateway for external connections to access your cluster, provided that the user knows the load balancer's IP address and the application's port number.
|
||||
If you want your applications to be externally accessible, you must add a load balancer or ingress to your cluster. Load balancers create a gateway for external connections to access your cluster, provided that the user knows the load balancer's IP address and the application's port number.
|
||||
|
||||
Rancher supports two types of load balancers:
|
||||
|
||||
@@ -33,18 +33,28 @@ Load Balancers have a couple of limitations you should be aware of:
|
||||
|
||||
## Ingress
|
||||
|
||||
As mentioned in the limitations above, using a load balancer per service can be expensive. You can get around this issue using an ingress.
|
||||
As mentioned in the limitations above, the disadvantages of using a load balancer are:
|
||||
|
||||
Ingress is a set or rules that act as a load balancer. Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster program the load balancer to direct the request to the correct service based on service subdomains or path rules that you've configured.
|
||||
- Load Balancers can only handle one IP address per service.
|
||||
- If you run multiple services in your cluster, you must have a load balancer for each service.
|
||||
- It can be expensive to have a load balancer for every service.
|
||||
|
||||
Your load balancer can either reside within your cluster or externally. Ingress and ingress controllers residing in RKE-launcher clusters are powered by [Nginx](https://www.nginx.com/).
|
||||
In contrast, when an ingress is used as the entrypoint into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service.
|
||||
|
||||
Therefore, it is useful to have an ingress if you want multiple services to be exposed with the same IP address, the same Layer 7 protocol, or the same privileged node-ports: 80 and 443.
|
||||
|
||||
Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster direct the request to the correct service based on service subdomains or path rules that you've configured.
|
||||
|
||||
Each Kubernetes Ingress resource corresponds roughly to a file in `/etc/nginx/sites-available/` containing a `server{}` configuration block, where requests for specific files and folders are configured.
|
||||
|
||||
Your ingress, which creates a port of entry to your cluster similar to a load balancer, can reside within your cluster or externally. Ingress and ingress controllers residing in RKE-launcher clusters are powered by [Nginx](https://www.nginx.com/).
|
||||
|
||||
Ingress can provide other functionality as well, such as SSL termination, name-based virtual hosting, and more.
|
||||
|
||||
>**Using Rancher in a High Availability Configuration?**
|
||||
>
|
||||
>Refrain from adding an Ingress to the `local` cluster. The Nginx Ingress Controller that Rancher uses acts as a global load balancer for _all_ clusters managed by Rancher, including the `local` cluster. Therefore, when users try to access an application, your Rancher connection may drop due to the Nginx configuration being reloaded. We recommend working around this issue by deploying applications only in clusters that you launch using Rancher.
|
||||
>Refrain from adding an Ingress to the `local` cluster. The Nginx Ingress Controller that Rancher uses acts as a global entry point for _all_ clusters managed by Rancher, including the `local` cluster. Therefore, when users try to access an application, your Rancher connection may drop due to the Nginx configuration being reloaded. We recommend working around this issue by deploying applications only in clusters that you launch using Rancher.
|
||||
|
||||
- For more information on how to setup ingress in Rancher, see [Ingress]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress).
|
||||
- For more information on how to set up ingress in Rancher, see [Ingress]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress).
|
||||
- For complete information about ingress and ingress controllers, see the [Kubernetes Ingress Documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/)
|
||||
- When using ingresses in a project, you can program the ingress hostname to an external DNS by setting up a Global DNS entry, see [Global DNS]({{< baseurl >}}/rancher/v2.x/en/catalog/globaldns/).
|
||||
|
||||
+15
-8
@@ -8,7 +8,11 @@ Kubernetes supports load balancing in two ways: Layer-4 Load Balancing and Layer
|
||||
|
||||
## Layer-4 Load Balancer
|
||||
|
||||
Layer-4 load balancer (or the external load balancer) forwards traffic to Nodeports. Layer-4 load balancer allows you to forward both HTTP and TCP traffic. Layer-4 load balancer is supported by the underlying cloud provider. As a result, when you deploy RKE clusters on bare metal servers and vSphere clusters, layer-4 load balancer is not supported.
|
||||
Layer-4 load balancer (or the external load balancer) forwards traffic to Nodeports. Layer-4 load balancer allows you to forward both HTTP and TCP traffic.
|
||||
|
||||
Often, the Layer-4 load balancer is supported by the underlying cloud provider, so when you deploy RKE clusters on bare-metal servers and vSphere clusters, Layer-4 load balancer is not supported. However, a single [globally managed config-map](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) can be used to expose services on NGINX or third-party ingress.
|
||||
|
||||
> **Note:** It is possible to deploy a cluster with a non-cloud load balancer, such as [MetalLB.](https://metallb.universe.tf/) However, that use case is more advanced than the Layer-4 load balancer supported by a cloud provider, and it is not configurable in Rancher or RKE.
|
||||
|
||||
### Support for Layer-4 Load Balancing
|
||||
|
||||
@@ -16,13 +20,16 @@ Support for layer-4 load balancer varies based on the underlying cloud provider.
|
||||
|
||||
Cluster Deployment | Layer-4 Load Balancer Support
|
||||
----------------------------------------------|--------------------------------
|
||||
Amazon EKS | Supported by AWS cloud provider
|
||||
Google GKE | Supported by GCE cloud provider
|
||||
Azure AKS | Supported by Azure cloud provider
|
||||
RKE on EC2 | Supported by AWS cloud provider
|
||||
RKE on DigitalOcean | Not Supported
|
||||
RKE on vSphere | Not Supported
|
||||
RKE on Custom Hosts<br/>(e.g. bare-metal servers) | Not Supported
|
||||
Amazon EKS | Supported by AWS cloud provider
|
||||
Google GKE | Supported by GCE cloud provider
|
||||
Azure AKS | Supported by Azure cloud provider
|
||||
RKE on EC2 | Supported by AWS cloud provider
|
||||
RKE on DigitalOcean | Limited NGINX or third-party Ingress*
|
||||
RKE on vSphere | Limited NGINX or third party-Ingress*
|
||||
RKE on Custom Hosts<br/>(e.g. bare-metal servers) | Limited NGINX or third-party Ingress*
|
||||
Third-party MetalLB | Limited NGINX or third-party Ingress*
|
||||
|
||||
\* Services can be exposed through a single [globally managed config-map.](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/)
|
||||
|
||||
## Layer-7 Load Balancer
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ Rancher ships with several example repositories that you can use to familiarize
|
||||
- Maven
|
||||
- php
|
||||
|
||||
> **Note**: The example repositories are only available if you have not [configured a version control provider]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/pipelines).
|
||||
> **Note:** The example repositories are only available if you have not [configured a version control provider]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/pipelines).
|
||||
|
||||
## Configure Repositories
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ Resource quotas in Rancher include the same functionality as the [native version
|
||||
|
||||
In a standard Kubernetes deployment, resource quotas are applied to individual namespaces. However, you cannot apply the quota to your namespaces simultaneously with a single action. Instead, the resource quota must be applied multiple times.
|
||||
|
||||
In the following diagram, a Kubernetes admin is trying to enforce a resource quota without Rancher. The admin wants to apply a resource quota that sets the same CPU and memory limit to every namespace in his cluster (`Namespace 1-4`) . However, in the base version of Kubernetes, each namespace requires a unique resource quota. The admin has to create four different resource quotas that have the same specs configured (`Resource Quota 1-4`) and apply them individually.
|
||||
In the following diagram, a Kubernetes administrator is trying to enforce a resource quota without Rancher. The administrator wants to apply a resource quota that sets the same CPU and memory limit to every namespace in his cluster (`Namespace 1-4`) . However, in the base version of Kubernetes, each namespace requires a unique resource quota. The administrator has to create four different resource quotas that have the same specs configured (`Resource Quota 1-4`) and apply them individually.
|
||||
|
||||
<sup>Base Kubernetes: Unique Resource Quotas Being Applied to Each Namespace</sup>
|
||||
|
||||
@@ -33,7 +33,7 @@ The resource quota includes two limits, which you set while creating or editing
|
||||
|
||||
This value is the default resource limit available for each namespace. When the resource quota is set on the project level, this limit is automatically propagated to each namespace in the project. Each namespace is bound to this default limit unless you [override it](#namespace-default-limit-overrides).
|
||||
|
||||
In the following diagram, a Rancher admin wants to apply a resource quota that sets the same CPU and memory limit for every namespace in their project (`Namespace 1-4`). However, in Rancher, the admin can set a resource quota for the project (`Project Resource Quota`) rather than individual namespaces. This quota includes resource limits for both the entire project (`Project Limit`) and individual namespaces (`Namespace Default Limit`). Rancher then propagates the `Namespace Default Limit` quotas to each namespace (`Namespace Resource Quota`).
|
||||
In the following diagram, a Rancher administrator wants to apply a resource quota that sets the same CPU and memory limit for every namespace in their project (`Namespace 1-4`). However, in Rancher, the administrator can set a resource quota for the project (`Project Resource Quota`) rather than individual namespaces. This quota includes resource limits for both the entire project (`Project Limit`) and individual namespaces (`Namespace Default Limit`). Rancher then propagates the `Namespace Default Limit` quotas to each namespace (`Namespace Resource Quota`).
|
||||
|
||||
<sup>Rancher: Resource Quotas Propagating to Each Namespace</sup>
|
||||
|
||||
@@ -80,7 +80,7 @@ When you create a resource quota, you are configuring the pool of resources avai
|
||||
|
||||
Although the **Namespace Default Limit** propagates from the project to each namespace, in some cases, you may need to increase (or decrease) the performance for a specific namespace. In this situation, you can override the default limits by editing the namespace.
|
||||
|
||||
In the diagram below, the Rancher admin has a resource quota in effect for their project. However, the admin wants to override the namespace limits for `Namespace 3` so that it performs better. Therefore, the admin [raises the namespace limits]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/#editing-namespace-resource-quotas) for `Namespace 3` so that the namespace can access more resources.
|
||||
In the diagram below, the Rancher administrator has a resource quota in effect for their project. However, the administrator wants to override the namespace limits for `Namespace 3` so that it performs better. Therefore, the administrator [raises the namespace limits]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/#editing-namespace-resource-quotas) for `Namespace 3` so that the namespace can access more resources.
|
||||
|
||||
<sup>Namespace Default Limit Override</sup>
|
||||
|
||||
|
||||
@@ -165,7 +165,7 @@ If you enable [project monitoring]({{< baseurl >}}/rancher/v2.x/en/project-admin
|
||||
|
||||
1. Continue adding more **Alert Rule** to the group.
|
||||
|
||||
1. Finally, choose the [notifiers]({{< baseurl >}}//rancher/v2.x/en/cluster-admin/tools/notifiers/) that send you alerts.
|
||||
1. Finally, choose the [notifiers]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) that send you alerts.
|
||||
|
||||
- You can set up multiple notifiers.
|
||||
- You can change notifier recipients on the fly.
|
||||
@@ -180,4 +180,4 @@ To manage project alerts, browse to the project that alerts you want to manage.
|
||||
- Edit alert settings
|
||||
- Delete unnecessary alerts
|
||||
- Mute firing alerts
|
||||
- Unmute muted alerts
|
||||
- Unmute muted alerts
|
||||
|
||||
@@ -7,10 +7,6 @@ aliases:
|
||||
- /rancher/v2.x/en/tools/pipelines/
|
||||
- /rancher/v2.x/en/tools/pipelines/configurations/
|
||||
---
|
||||
>**Notes:**
|
||||
>
|
||||
>- Pipelines are new and improved for Rancher v2.1! Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1.
|
||||
>- Still using v2.0.x? See the pipeline documentation for [previous versions]({{< baseurl >}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x).
|
||||
|
||||
A _pipeline_ is a software delivery process that is broken into different stages and steps. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects.
|
||||
|
||||
@@ -30,6 +26,12 @@ Typically, pipeline stages include:
|
||||
|
||||
Only [administrators]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), [cluster owners or members]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or [project owners]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) can [configure version control providers](#version-control-providers) and [manage global pipeline execution settings](#managing-global-pipeline-execution-settings). Project members can only configure [repositories]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#configuring-repositories) and [pipelines]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration).
|
||||
|
||||
|
||||
> **Notes:**
|
||||
>
|
||||
> - Pipelines were improved in Rancher v2.1. Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1.
|
||||
> - Still using v2.0.x? See the pipeline documentation for [previous versions]({{< baseurl >}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x).
|
||||
|
||||
## Overview
|
||||
|
||||
Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users.
|
||||
|
||||
@@ -45,7 +45,7 @@ You can set up your pipeline to run a series of stages and steps to test your co
|
||||
|
||||
- Only the branch {BRANCH NAME}: Only events triggered by changes to this branch will be built.
|
||||
|
||||
- Evertyhing but {BRANCH NAME}: Build any branch that triggered an event EXCEPT events from this branch.
|
||||
- Everything but {BRANCH NAME}: Build any branch that triggered an event EXCEPT events from this branch.
|
||||
|
||||
- All branches: Regardless of the branch that triggered the event always build.
|
||||
|
||||
|
||||
@@ -51,5 +51,5 @@ Rancher is committed to informing the community of security issues in our produc
|
||||
| [CVE-2019-12274](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12274) | Nodes using the built-in node drivers using a file path option allows the machine to read arbitrary files including sensitive ones from inside the Rancher server container. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
|
||||
| [CVE-2019-12303](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12303) | Project owners can inject extra fluentd logging configurations that makes it possible to read files or execute arbitrary commands inside the fluentd container. Reported by Tyler Welton from Untamed Theory. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
|
||||
| [CVE-2019-13209](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-13209) | The vulnerability is known as a [Cross-Site Websocket Hijacking attack](https://www.christian-schneider.net/CrossSiteWebSocketHijacking.html). This attack allows an exploiter to gain access to clusters managed by Rancher with the roles/permissions of a victim. It requires that a victim to be logged into a Rancher server and then access a third-party site hosted by the exploiter. Once that is accomplished, the exploiter is able to execute commands against the Kubernetes API with the permissions and identity of the victim. Reported by Matt Belisle and Alex Stevenson from Workiva. | 15 Jul 2019 | [Rancher v2.2.5](https://github.com/rancher/rancher/releases/tag/v2.2.5), [Rancher v2.1.11](https://github.com/rancher/rancher/releases/tag/v2.1.11) and [Rancher v2.0.16](https://github.com/rancher/rancher/releases/tag/v2.0.16) |
|
||||
| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them admin access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
|
||||
| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them administrator access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
|
||||
| [CVE-2019-14435](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14435) | This vulnerability allows authenticated users to potentially extract otherwise private data out of IPs reachable from system service containers used by Rancher. This can include but not only limited to services such as cloud provider metadata services. Although Rancher allow users to configure whitelisted domains for system service access, this flaw can still be exploited by a carefully crafted HTTP request. The issue was found and reported by Matt Belisle and Alex Stevenson at Workiva. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
|
||||
|
||||
@@ -870,7 +870,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
## 3.2 - Rancher Management Control Plane Authentication
|
||||
|
||||
### 3.2.1 - Change the local admin password from the default value
|
||||
### 3.2.1 - Change the local administrator password from the default value
|
||||
|
||||
**Profile Applicability**
|
||||
|
||||
@@ -878,11 +878,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
**Description**
|
||||
|
||||
The local admin password should be changed from the default.
|
||||
The local administrator password should be changed from the default.
|
||||
|
||||
**Rationale**
|
||||
|
||||
The default admin password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
The default administrator password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
|
||||
**Audit**
|
||||
|
||||
|
||||
@@ -913,7 +913,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
## 3.2 - Rancher Management Control Plane Authentication
|
||||
|
||||
### 3.2.1 - Change the local admin password from the default value
|
||||
### 3.2.1 - Change the local administrator password from the default value
|
||||
|
||||
**Profile Applicability**
|
||||
|
||||
@@ -921,11 +921,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
**Description**
|
||||
|
||||
The local admin password should be changed from the default.
|
||||
The local administrator password should be changed from the default.
|
||||
|
||||
**Rationale**
|
||||
|
||||
The default admin password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
The default administrator password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
|
||||
**Audit**
|
||||
|
||||
|
||||
@@ -1001,7 +1001,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
## 3.2 - Rancher Management Control Plane Authentication
|
||||
|
||||
### 3.2.1 - Change the local admin password from the default value
|
||||
### 3.2.1 - Change the local administrator password from the default value
|
||||
|
||||
**Profile Applicability**
|
||||
|
||||
@@ -1009,11 +1009,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log
|
||||
|
||||
**Description**
|
||||
|
||||
The local admin password should be changed from the default.
|
||||
The local administrator password should be changed from the default.
|
||||
|
||||
**Rationale**
|
||||
|
||||
The default admin password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
The default administrator password is common across all Rancher installations and should be changed immediately upon startup.
|
||||
|
||||
**Audit**
|
||||
|
||||
|
||||
@@ -45,7 +45,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
|
||||
|
||||
2. Launch it using `kubectl create -f ds-overlaytest.yml`
|
||||
3. Wait until `kubectl rollout status ds/overlaytest -w` returns: `daemon set "overlaytest" successfully rolled out`.
|
||||
4. Run the following command to let each container on every host ping each other (it's a single line command).
|
||||
4. Run the following command, from the same location, to let each container on every host ping each other (it's a single line bash command).
|
||||
|
||||
```
|
||||
echo "=> Start network overlay test"; kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.nodeName}{"\n"}{end}' | while read spod shost; do kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.status.podIP}{" "}{@.spec.nodeName}{"\n"}{end}' | while read tip thost; do kubectl --request-timeout='10s' exec $spod -- /bin/sh -c "ping -c2 $tip > /dev/null 2>&1"; RC=$?; if [ $RC -ne 0 ]; then echo $shost cannot reach $thost; fi; done; done; echo "=> End network overlay test"
|
||||
|
||||
@@ -105,5 +105,5 @@ addons: |-
|
||||
|
||||
From here follow the standard install steps.
|
||||
|
||||
* [3 - Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
|
||||
* [3 - Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
|
||||
* [4 - Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
|
||||
|
||||
@@ -67,6 +67,11 @@ kubernetes_version: "v1.11.6-rancher1-1"
|
||||
|
||||
In case both `kubernetes_version` and [system images]({{< baseurl >}}/rke/latest/en/config-options/system-images/) are defined, the system images configuration will take precedence over `kubernetes_version`.
|
||||
|
||||
> **Note:** In RKE, `kubernetes_version` is used to map the version of Kubernetes to the default services, parameters, and options:
|
||||
|
||||
> - For RKE v0.3.0+, the service defaults are located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go).
|
||||
> - For RKE prior to v0.3.0, the service defaults are located [here](https://github.com/rancher/types/blob/release/v2.2/apis/management.cattle.io/v3/k8s_defaults.go). Note: The version in the path of the service defaults file corresponds to a Rancher version. Therefore, for Rancher v2.1.x, [this file](https://github.com/rancher/types/blob/release/v2.1/apis/management.cattle.io/v3/k8s_defaults.go) should be used.
|
||||
|
||||
#### Listing Supported Kubernetes Versions
|
||||
|
||||
Please refer to the [release notes](https://github.com/rancher/rke/releases) of the RKE version that you are running, to find the list of supported Kubernetes versions as well as the default Kubernetes version.
|
||||
|
||||
@@ -3,256 +3,27 @@ title: vSphere Cloud Provider
|
||||
weight: 254
|
||||
---
|
||||
|
||||
In order to provision Kubernetes clusters in vSphere with the RKE CLI, you must enable the vSphere cloud provider.
|
||||
|
||||
The vSphere cloud provider must also be enabled in order to provision clusters with Rancher, which uses RKE as a library when provisioning [RKE clusters.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/)
|
||||
|
||||
The [vSphere Cloud Provider](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) interacts with VMware infrastructure (vCenter or standalone ESXi server) to provision and manage storage for persistent volumes in a Kubernetes cluster.
|
||||
|
||||
When provisioning Kubernetes using RKE CLI or using [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) in Rancher, the vSphere Cloud Provider can be enabled by configuring the `cloud_provider` directive in the cluster YAML file.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. You'll need to have credentials of a vCenter/ESXi user account with privileges allowing the cloud provider to interact with the vSphere infrastructure to provision storage. Refer to [this document](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) to create and assign a role with the required permissions in vCenter.
|
||||
2. VMware Tools must be running in the Guest OS for all nodes in the cluster.
|
||||
3. All nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly. See [Enabling Disk UUIDs](#enabling-disk-uuids-for-vsphere-vms).
|
||||
|
||||
## Clusters provisioned with RKE CLI
|
||||
|
||||
To enable the vSphere Cloud Provider in the cluster, you must add the top-level `cloud_provider` directive to the cluster configuration file, set the `name` property to `vsphere` and add the `vsphereCloudProvider` directive containing the configuration matching your infrastructure. See the [configuration reference](#configuration-reference) for the gory details.
|
||||
|
||||
## Clusters provisioned with Rancher
|
||||
|
||||
When provisioning clusters in Rancher using the [vSphere node driver]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/) or on pre-created [custom nodes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/) the cluster YAML file must be modified in order to enable the cloud provider.
|
||||
|
||||
1. Log in to the Rancher UI as admin user.
|
||||
2. Navigate to **Clusters** in the **Global** view.
|
||||
3. Click **Add Cluster** and select the **vSphere** infrastructure provider.
|
||||
4. Assign a **Cluster Name**.
|
||||
5. Assign **Member Roles** as required.
|
||||
6. Expand **Cluster Options** and configure as required.
|
||||
7. Set **Cloud Provider** option to `Custom`.
|
||||
|
||||
{{< img "/img/rancher/vsphere-node-driver-cloudprovider.png" "vsphere-node-driver-cloudprovider">}}
|
||||
|
||||
8. Click on **Edit as YAML**
|
||||
9. Insert the following top-level structure to the pre-populated cluster YAML. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive.
|
||||
|
||||
```yaml
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
[Insert provider configuration]
|
||||
```
|
||||
|
||||
10. Configure the **Node Pools** per your requirements while ensuring to use a node template that enables disk UUIDs for the VMs (See [Annex - Enable disk UUIDs for vSphere VMs]).
|
||||
11. Click on **Create** to start provisioning the VMs and Kubernetes services.
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
> **Note:** This documentation reflects the new vSphere Cloud Provider configuration schema introduced in Kubernetes v1.9 which differs from previous versions.
|
||||
|
||||
The vSphere configuration options are divided into 5 groups:
|
||||
|
||||
* global
|
||||
* virtual_center
|
||||
* workspace
|
||||
* disk
|
||||
* network
|
||||
|
||||
### global
|
||||
|
||||
The main purpose of global options is to be able to define a common set of configuration parameters that will be inherited by all vCenters defined under the `virtual_center` directive unless explicitly defined there.
|
||||
|
||||
Accordingly, the `global` directive accepts the same configuration options that are available under the `virtual_center` directive. Additionally it accepts a single parameter that can only be specified here:
|
||||
|
||||
| global Options | Type | Required | Description |
|
||||
|:---------------:|:-------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| insecure-flag | boolean | | Set to **true** if the vCenter/ESXi uses a self-signed certificate. |
|
||||
|
||||
___
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
global:
|
||||
insecure-flag: true
|
||||
```
|
||||
|
||||
### virtual_center
|
||||
|
||||
This configuration directive specifies the vCenters that are managing the nodes in the cluster. You must define at least one vCenter/ESXi server. If the nodes span multiple vCenters then all must be defined.
|
||||
|
||||
Each vCenter is defined by adding a new entry under the `virtual_center` directive with the vCenter IP or FQDN as the name. All required parameters must be provided for each vCenter unless they are already defined under the `global` directive.
|
||||
|
||||
| virtual_center Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| user | string | * | vCenter/ESXi user used to authenticate with this server. |
|
||||
| password | string | * | User's password. |
|
||||
| port | string | | Port to use to connect to this server. Defaults to 443. |
|
||||
| datacenters | string | * | Comma-separated list of all datacenters in which cluster nodes are running in.|
|
||||
| soap-roundtrip-count | uint | | Round tripper count for API requests to the vCenter (num retries = value - 1).|
|
||||
|
||||
|
||||
> The following additional options (introduced in Kubernetes v1.11) are not yet supported in RKE.
|
||||
|
||||
| virtual_center Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| secret-name | string | | Name of secret resource containing credential key/value pairs. Can be specified in lieu of user/password parameters.|
|
||||
| secret-namespace | string | | Namespace in which the secret resource was created in. |
|
||||
| ca-file | string | | Path to CA cert file used to verify the vCenter certificate. |
|
||||
|
||||
___
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
virtual_center:
|
||||
172.158.111.1: {} # This vCenter inherits all it's properties from global options
|
||||
172.158.110.2: # All required options are set explicitly
|
||||
user: vc-user
|
||||
password: othersecret
|
||||
datacenters: eu-west-2
|
||||
```
|
||||
|
||||
### workspace
|
||||
|
||||
This configuration group specifies how storage for volumes is created in vSphere.
|
||||
The following configuration options are available:
|
||||
|
||||
| workspace Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| server | string | * | IP or FQDN of the vCenter/ESXi that should be used for creating the volumes. Must match one of the vCenters defined under the `virtual_center` directive.|
|
||||
| datacenter | string | * | Name of the datacenter that should be used for creating volumes. For ESXi enter *ha-datacenter*.|
|
||||
| folder | string | * | Path of folder in which to create dummy VMs used for volume provisioning (relative from the root folder in vCenter), e.g. "kubernetes".|
|
||||
| default-datastore | string | | Name of default datastore to place VMDKs if neither datastore or storage policy are specified in the volume options of a PVC. If datastore is located in a storage folder or is a member of a datastore cluster, specify the full path. |
|
||||
| resourcepool-path | string | | Absolute or relative path to the resource pool where the dummy VMs for [Storage policy based provisioning](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html) should be created. If a relative path is specified, it is resolved with respect to the datacenter's *host* folder. Examples: `/<dataCenter>/host/<hostOrClusterName>/Resources/<poolName>`, `Resources/<poolName>`. For standalone ESXi specify `Resources`.|
|
||||
|
||||
___
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
workspace:
|
||||
server: 172.158.111.1 # matches IP of vCenter defined in the virtual_center block
|
||||
datacenter: eu-west-1
|
||||
folder: kubernetes
|
||||
default-datastore: ds-1
|
||||
```
|
||||
|
||||
### disk
|
||||
|
||||
The following configuration options are available under the disk directive:
|
||||
|
||||
| disk Options | Type | Required | Description |
|
||||
|:--------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| scsicontrollertype | string | | SCSI controller type to use when attaching block storage to VMs. Must be one of: *lsilogic-sas* or *pvscsi*. Default: *pvscsi*.|
|
||||
|
||||
___
|
||||
|
||||
### network
|
||||
|
||||
The following configuration options are available under the network directive:
|
||||
|
||||
| network Options | Type | Required | Description |
|
||||
|:-------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:|
|
||||
| public-network | string | | Name of public **VM Network** to which the VMs in the cluster are connected. Used to determine public IP addresses of VMs.|
|
||||
|
||||
|
||||
## Configuration Example
|
||||
|
||||
Given the following:
|
||||
|
||||
- VMs in the cluster are running in the same datacenter `eu-west-1` managed by the vCenter `vc.example.com`.
|
||||
- The vCenter has a user `provisioner` with password `secret` with the required roles assigned, see [Prerequisites](#prerequisites).
|
||||
- The vCenter has a datastore named `ds-1` which should be used to store the VMDKs for volumes.
|
||||
- A `kubernetes` folder exists in vCenter.
|
||||
|
||||
The corresponding configuration for the provider would then be as follows:
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
virtual_center:
|
||||
vc.example.com:
|
||||
user: provisioner
|
||||
password: secret
|
||||
datacenters: eu-west-1
|
||||
workspace:
|
||||
server: vc.example.com
|
||||
folder: kubernetes
|
||||
default-datastore: ds-1
|
||||
datacenter: eu-west-1
|
||||
|
||||
```
|
||||
|
||||
## Annex
|
||||
|
||||
### Enabling disk UUIDs for vSphere VMs
|
||||
|
||||
Depending on whether you are provisioning the VMs using the [vSphere node driver]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) in Rancher or using your own scripts or third-party tools, there are different methods available to enable disk UUIDs for VMs.
|
||||
|
||||
#### Using the Vsphere Console
|
||||
|
||||
The required property can be set while creating or modifying VMs in the vSphere Console:
|
||||
|
||||
1. For each VM navigate to the tab **VM Options** and click on **Edit Configuration**.
|
||||
2. Add the parameter `disk.EnableUUID` with a value of **TRUE**.
|
||||
|
||||
{{< img "/img/rke/vsphere-advanced-parameters.png" "vsphere-advanced-parameters">}}
|
||||
|
||||
#### Using the GOVC CLI tool
|
||||
|
||||
You can also modify properties of VMs with the [govc](https://github.com/vmware/govmomi/tree/master/govc) command-line tool to enable disk UUIDs:
|
||||
|
||||
```sh
|
||||
$ govc vm.change -vm <vm-path> -e disk.enableUUID=TRUE
|
||||
```
|
||||
|
||||
#### Using Rancher node template
|
||||
|
||||
When creating new clusters in Rancher using vSphere node templates, you can configure the template to automatically enable disk UUIDs for all VMs created for a cluster:
|
||||
|
||||
1. Navigate to the **Node Templates** in the Rancher UI while logged in as admin user.
|
||||
|
||||
2. Add or edit an existing vSphere node template.
|
||||
|
||||
3. Under **Instance Options** click on **Add Parameter**.
|
||||
|
||||
4. Enter `disk.enableUUID` as key with a value of **TRUE**.
|
||||
|
||||
{{< img "/img/rke/vsphere-nodedriver-enable-uuid.png" "vsphere-nodedriver-enable-uuid">}}
|
||||
|
||||
5. Click **Create** or **Save**.
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
If you are experiencing issues while provisioning a cluster with enabled vSphere Cloud Provider or while creating vSphere volumes for your workloads, you should inspect the logs of the following K8s services:
|
||||
|
||||
- controller-manager (Manages volumes in vCenter)
|
||||
- kubelet: (Mounts vSphere volumes to pods)
|
||||
|
||||
If your cluster is not configured with external [Cluster Logging]({{< baseurl >}}/rancher/v2.x/en/tools/logging/), you will need to SSH into nodes to get the logs of the `kube-controller-manager` (running on one of the control plane nodes) and the `kubelet` (pertaining to the node where the stateful pod has been scheduled).
|
||||
|
||||
The easiest way to create a SSH session with a node is the Rancher CLI tool.
|
||||
|
||||
1. [Configure the Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/) for your cluster.
|
||||
2. Run the following command to get a shell to the corresponding nodes:
|
||||
|
||||
```sh
|
||||
$ rancher ssh <nodeName>
|
||||
```
|
||||
|
||||
3. Inspect the logs of the controller-manager and kubelet containers looking for errors related to the vSphere cloud provider:
|
||||
|
||||
```sh
|
||||
$ docker logs --since 15m kube-controller-manager
|
||||
$ docker logs --since 15m kubelet
|
||||
```
|
||||
|
||||
This section describes how to enable the vSphere cloud provider. You will need to use the `cloud_provider` directive in the cluster YAML file.
|
||||
|
||||
### Related Links
|
||||
|
||||
- [vSphere Storage for Kubernetes](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/)
|
||||
- [Kubernetes Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
|
||||
- **Configuration:** For details on vSphere configuration in RKE, refer to the [configuration reference.]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference)
|
||||
- **Troubleshooting:** For guidance on troubleshooting a cluster with the vSphere cloud provider enabled, refer to the [troubleshooting section.]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting)
|
||||
- **Storage:** If you are setting up storage, see the [official vSphere documentation on storage for Kubernetes,](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) or the [official Kubernetes documentation on persistent volumes.](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) If you are using Rancher, refer to the [Rancher documentation on provisioning storage in vSphere.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere)
|
||||
- **For Rancher users:** Refer to the Rancher documentation on [creating vSphere Kubernetes clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) and [provisioning storage.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere)
|
||||
|
||||
# Prerequisites
|
||||
|
||||
- **Credentials:** You'll need to have credentials of a vCenter/ESXi user account with privileges allowing the cloud provider to interact with the vSphere infrastructure to provision storage. Refer to [this document](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) to create and assign a role with the required permissions in vCenter.
|
||||
- **VMware Tools** must be running in the Guest OS for all nodes in the cluster.
|
||||
- **Disk UUIDs:** All nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly. See the section on [enabling disk UUIDs.]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid)
|
||||
|
||||
# Enabling the vSphere Provider with the RKE CLI
|
||||
|
||||
To enable the vSphere Cloud Provider in the cluster, you must add the top-level `cloud_provider` directive to the cluster configuration file, set the `name` property to `vsphere` and add the `vsphereCloudProvider` directive containing the configuration matching your infrastructure. See the [configuration reference]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference) for the gory details.
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
---
|
||||
title: vSphere Configuration Reference
|
||||
weight: 3
|
||||
---
|
||||
|
||||
This section shows an example of how to configure the vSphere cloud provider.
|
||||
|
||||
The vSphere cloud provider must be enabled to allow dynamic provisioning of volumes.
|
||||
|
||||
For more details on deploying a Kubernetes cluster on vSphere, refer to the [official cloud provider documentation.](https://cloud-provider-vsphere.sigs.k8s.io/tutorials/kubernetes-on-vsphere-with-kubeadm.html)
|
||||
|
||||
> **Note:** This documentation reflects the new vSphere Cloud Provider configuration schema introduced in Kubernetes v1.9 which differs from previous versions.
|
||||
|
||||
# vSphere Configuration Example
|
||||
|
||||
Given the following:
|
||||
|
||||
- VMs in the cluster are running in the same datacenter `eu-west-1` managed by the vCenter `vc.example.com`.
|
||||
- The vCenter has a user `provisioner` with password `secret` with the required roles assigned, see [Prerequisites](#prerequisites).
|
||||
- The vCenter has a datastore named `ds-1` which should be used to store the VMDKs for volumes.
|
||||
- A `vm/kubernetes` folder exists in vCenter.
|
||||
|
||||
The corresponding configuration for the provider would then be as follows:
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
virtual_center:
|
||||
vc.example.com:
|
||||
user: provisioner
|
||||
password: secret
|
||||
port: 443
|
||||
datacenters: /us-west-1
|
||||
workspace:
|
||||
server: vc.example.com
|
||||
folder: /us-west-1/folder/myvmfolder
|
||||
default-datastore: /us-west-1/datastore/ds-1
|
||||
datacenter: /us-west-1
|
||||
resourcepool-path: /us-west-1/host/hn1/resources/myresourcepool
|
||||
|
||||
```
|
||||
# Configuration Options
|
||||
|
||||
The vSphere configuration options are divided into 5 groups:
|
||||
|
||||
* [global](#global)
|
||||
* [virtual_center](#virtual_center)
|
||||
* [workspace](#workspace)
|
||||
* [disk](#disk)
|
||||
* [network](#network)
|
||||
|
||||
### global
|
||||
|
||||
The main purpose of global options is to be able to define a common set of configuration parameters that will be inherited by all vCenters defined under the `virtual_center` directive unless explicitly defined there.
|
||||
|
||||
Accordingly, the `global` directive accepts the same configuration options that are available under the `virtual_center` directive. Additionally it accepts a single parameter that can only be specified here:
|
||||
|
||||
| global Options | Type | Required | Description |
|
||||
|:---------------:|:-------:|:---------:|:---------|
|
||||
| insecure-flag | boolean | | Set to **true** if the vCenter/ESXi uses a self-signed certificate. |
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
global:
|
||||
insecure-flag: true
|
||||
```
|
||||
|
||||
### virtual_center
|
||||
|
||||
This configuration directive specifies the vCenters that are managing the nodes in the cluster. You must define at least one vCenter/ESXi server. If the nodes span multiple vCenters then all must be defined.
|
||||
|
||||
Each vCenter is defined by adding a new entry under the `virtual_center` directive with the vCenter IP or FQDN as the name. All required parameters must be provided for each vCenter unless they are already defined under the `global` directive.
|
||||
|
||||
| virtual_center Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:-----------|
|
||||
| user | string | * | vCenter/ESXi user used to authenticate with this server. |
|
||||
| password | string | * | User's password. |
|
||||
| port | string | | Port to use to connect to this server. Defaults to 443. |
|
||||
| datacenters | string | * | Comma-separated list of all datacenters in which cluster nodes are running in. |
|
||||
| soap-roundtrip-count | uint | | Round tripper count for API requests to the vCenter (num retries = value - 1). |
|
||||
|
||||
> The following additional options (introduced in Kubernetes v1.11) are not yet supported in RKE.
|
||||
|
||||
| virtual_center Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:-------|
|
||||
| secret-name | string | | Name of secret resource containing credential key/value pairs. Can be specified in lieu of user/password parameters.|
|
||||
| secret-namespace | string | | Namespace in which the secret resource was created in. |
|
||||
| ca-file | string | | Path to CA cert file used to verify the vCenter certificate. |
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
virtual_center:
|
||||
172.158.111.1: {} # This vCenter inherits all it's properties from global options
|
||||
172.158.110.2: # All required options are set explicitly
|
||||
user: vc-user
|
||||
password: othersecret
|
||||
datacenters: eu-west-2
|
||||
```
|
||||
|
||||
### workspace
|
||||
|
||||
This configuration group specifies how storage for volumes is created in vSphere.
|
||||
The following configuration options are available:
|
||||
|
||||
| workspace Options | Type | Required | Description |
|
||||
|:----------------------:|:--------:|:---------:|:---------|
|
||||
| server | string | * | IP or FQDN of the vCenter/ESXi that should be used for creating the volumes. Must match one of the vCenters defined under the `virtual_center` directive.|
|
||||
| datacenter | string | * | Name of the datacenter that should be used for creating volumes. For ESXi enter *ha-datacenter*.|
|
||||
| folder | string | * | Path of folder in which to create dummy VMs used for volume provisioning (relative from the root folder in vCenter), e.g. "vm/kubernetes".|
|
||||
| default-datastore | string | | Name of default datastore to place VMDKs if neither datastore or storage policy are specified in the volume options of a PVC. If datastore is located in a storage folder or is a member of a datastore cluster, specify the full path. |
|
||||
| resourcepool-path | string | | Absolute or relative path to the resource pool where the dummy VMs for [Storage policy based provisioning](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html) should be created. If a relative path is specified, it is resolved with respect to the datacenter's *host* folder. Examples: `/<dataCenter>/host/<hostOrClusterName>/Resources/<poolName>`, `Resources/<poolName>`. For standalone ESXi specify `Resources`. |
|
||||
|
||||
**Example:**
|
||||
|
||||
```yaml
|
||||
(...)
|
||||
workspace:
|
||||
server: 172.158.111.1 # matches IP of vCenter defined in the virtual_center block
|
||||
datacenter: eu-west-1
|
||||
folder: vm/kubernetes
|
||||
default-datastore: ds-1
|
||||
```
|
||||
|
||||
### disk
|
||||
|
||||
The following configuration options are available under the disk directive:
|
||||
|
||||
| disk Options | Type | Required | Description |
|
||||
|:--------------------:|:--------:|:---------:|:----------------|
|
||||
| scsicontrollertype | string | | SCSI controller type to use when attaching block storage to VMs. Must be one of: *lsilogic-sas* or *pvscsi*. Default: *pvscsi*. |
|
||||
|
||||
### network
|
||||
|
||||
The following configuration options are available under the network directive:
|
||||
|
||||
| network Options | Type | Required | Description |
|
||||
|:-------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------|
|
||||
| public-network | string | | Name of public **VM Network** to which the VMs in the cluster are connected. Used to determine public IP addresses of VMs.|
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Enabling Disk UUIDs for vSphere VMs
|
||||
weight: 2
|
||||
---
|
||||
|
||||
In order to provision nodes with RKE, all nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly.
|
||||
|
||||
Depending on whether you are provisioning the VMs using the [vSphere node driver]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) in Rancher or using your own scripts or third-party tools, there are different methods available to enable disk UUIDs for VMs:
|
||||
|
||||
- [Using the vSphere console](#using-the-vsphere-console)
|
||||
- [Using the GOVC CLI tool](#using-the-govc-cli-tool)
|
||||
- [Using a Rancher node template](#using-a-rancher-node-template)
|
||||
|
||||
### Using the vSphere Console
|
||||
|
||||
The required property can be set while creating or modifying VMs in the vSphere Console:
|
||||
|
||||
1. For each VM navigate to the tab **VM Options** and click on **Edit Configuration**.
|
||||
2. Add the parameter `disk.EnableUUID` with a value of **TRUE**.
|
||||
|
||||
|
||||
|
||||
### Using the GOVC CLI tool
|
||||
|
||||
You can also modify properties of VMs with the [govc](https://github.com/vmware/govmomi/tree/master/govc) command-line tool to enable disk UUIDs:
|
||||
|
||||
```sh
|
||||
$ govc vm.change -vm <vm-path> -e disk.enableUUID=TRUE
|
||||
```
|
||||
|
||||
### Using a Rancher Node Template
|
||||
|
||||
In Rancher v2.0.4+, disk UUIDs are enabled in vSphere node templates by default.
|
||||
|
||||
If you are using Rancher prior to v2.0.4, refer to the [Rancher documentation.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/#enabling-disk-uuids-with-a-node-template) for details on how to enable a UUID with a Rancher node template.
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
---
|
||||
title: Troubleshooting vSphere Clusters
|
||||
weight: 4
|
||||
---
|
||||
|
||||
If you are experiencing issues while provisioning a cluster with enabled vSphere Cloud Provider or while creating vSphere volumes for your workloads, you should inspect the logs of the following K8s services:
|
||||
|
||||
- controller-manager (Manages volumes in vCenter)
|
||||
- kubelet: (Mounts vSphere volumes to pods)
|
||||
|
||||
If your cluster is not configured with external [Cluster Logging]({{< baseurl >}}/rancher/v2.x/en/tools/logging/), you will need to SSH into nodes to get the logs of the `kube-controller-manager` (running on one of the control plane nodes) and the `kubelet` (pertaining to the node where the stateful pod has been scheduled).
|
||||
|
||||
The easiest way to create a SSH session with a node is the Rancher CLI tool.
|
||||
|
||||
1. [Configure the Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/) for your cluster.
|
||||
2. Run the following command to get a shell to the corresponding nodes:
|
||||
|
||||
```sh
|
||||
$ rancher ssh <nodeName>
|
||||
```
|
||||
|
||||
3. Inspect the logs of the controller-manager and kubelet containers looking for errors related to the vSphere cloud provider:
|
||||
|
||||
```sh
|
||||
$ docker logs --since 15m kube-controller-manager
|
||||
$ docker logs --since 15m kubelet
|
||||
```
|
||||
@@ -5,6 +5,31 @@ weight: 210
|
||||
|
||||
The `nodes` directive is the only required section in the `cluster.yml` file. It's used by RKE to specify cluster node(s), ssh credentials used to access the node(s) and which roles these nodes will be in the Kubernetes cluster.
|
||||
|
||||
This section covers the following topics:
|
||||
|
||||
- [Node configuration example](#node-configuration-example)
|
||||
- [Kubernetes roles](#kubernetes-roles)
|
||||
- [etcd](#etcd)
|
||||
- [Controlplane](#controlplane)
|
||||
- [Worker](#worker)
|
||||
- [Node options](#node-options)
|
||||
- [Address](#address)
|
||||
- [Internal address](#internal-address)
|
||||
- [Overriding the hostname](#overriding-the-hostname)
|
||||
- [SSH port](#ssh-port)
|
||||
- [SSH users](#ssh-users)
|
||||
- [SSH key path](#ssh-key-path)
|
||||
- [SSH key](#ssh-key)
|
||||
- [SSH certificate path](#ssh-certificate-path)
|
||||
- [SSH certificate](#ssh-certificate)
|
||||
- [Docker socket](#docker-socket)
|
||||
- [Labels](#labels)
|
||||
- [Taints](#taints)
|
||||
|
||||
# Node Configuration Example
|
||||
|
||||
The following example shows node configuration in an example `cluster.yml`:
|
||||
|
||||
```yaml
|
||||
nodes:
|
||||
- address: 1.1.1.1
|
||||
@@ -49,7 +74,33 @@ nodes:
|
||||
app: ingress
|
||||
```
|
||||
|
||||
## Node Options
|
||||
# Kubernetes Roles
|
||||
|
||||
You can specify the list of roles that you want the node to be as part of the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process.
|
||||
|
||||
> **Note:** Prior to v0.1.8, workloads/pods might have run on any nodes with `worker` or `controlplane` roles, but as of v0.1.8, they will only be deployed to any `worker` nodes.
|
||||
|
||||
### etcd
|
||||
|
||||
With this role, the `etcd` container will be run on these nodes. Etcd keeps the state of your cluster and is the most important component in your cluster, single source of truth of your cluster. Although you can run etcd on just one node, it typically takes 3, 5 or more nodes to create an HA configuration. Etcd is a distributed reliable key-value store which stores all Kubernetes state. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **etcd** role is shown below:
|
||||
|
||||
Taint Key | Taint Value | Taint Effect
|
||||
---------------------------------------|--------------|--------------
|
||||
`node-role.kubernetes.io/etcd` | `true` | `NoExecute`
|
||||
|
||||
### Controlplane
|
||||
|
||||
With this role, the stateless components that are used to deploy Kubernetes will run on these nodes. These components are used to run the API server, scheduler, and controllers. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **controlplane** role is shown below:
|
||||
|
||||
Taint Key | Taint Value | Taint Effect
|
||||
---------------------------------------|--------------|--------------
|
||||
`node-role.kubernetes.io/controlplane` | `true` | `NoSchedule`
|
||||
|
||||
### Worker
|
||||
|
||||
With this role, any workloads or pods that are deployed will land on these nodes.
|
||||
|
||||
# Node Options
|
||||
|
||||
Within each node, there are multiple directives that can be used.
|
||||
|
||||
@@ -93,32 +144,6 @@ For each node, you can specify the path, i.e. `ssh_cert_path`, for the signed SS
|
||||
|
||||
Instead of setting the path to the signed SSH certificate, you can alternatively specify the actual certificate, i.e. `ssh_cert`, to be used to connect to the node.
|
||||
|
||||
### Kubernetes Roles
|
||||
|
||||
You can specify the list of roles that you want the node to be as part of the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process.
|
||||
|
||||
> **Note:** Prior to v0.1.8, workloads/pods might have run on any nodes with `worker` or `controlplane` roles, but as of v0.1.8, they will only be deployed to any `worker` nodes.
|
||||
|
||||
* **etcd**
|
||||
|
||||
With this role, the `etcd` container will be run on these nodes. Etcd keeps the state of your cluster and is the most important component in your cluster, single source of truth of your cluster. Although you can run etcd on just one node, it typically takes 3, 5 or more nodes to create an HA configuration. Etcd is a distributed reliable key-value store which stores all Kubernetes state. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **etcd** role is shown below:
|
||||
|
||||
Taint Key | Taint Value | Taint Effect
|
||||
---------------------------------------|--------------|--------------
|
||||
`node-role.kubernetes.io/etcd` | `true` | `NoExecute`
|
||||
|
||||
* **controlplane**
|
||||
|
||||
With this role, the stateless components that are used to deploy Kubernetes will run on these nodes. These components are used to run the API server, scheduler, and controllers. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **controlplane** role is shown below:
|
||||
|
||||
Taint Key | Taint Value | Taint Effect
|
||||
---------------------------------------|--------------|--------------
|
||||
`node-role.kubernetes.io/controlplane` | `true` | `NoSchedule`
|
||||
|
||||
* **worker**
|
||||
|
||||
With this role, any workloads or pods that are deployed will land on these nodes.
|
||||
|
||||
### Docker Socket
|
||||
|
||||
If the Docker socket is different than the default, you can set the `docker_socket`. The default is `/var/run/docker.sock`
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
---
|
||||
title: Encrypting Secret Data at Rest
|
||||
weight: 230
|
||||
---
|
||||
|
||||
As of version `v0.3.1` RKE adds the support for managing secret data encryption at rest, which is [supported by Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#before-you-begin) since version `v1.13`.
|
||||
|
||||
At-rest data encryption is required for:
|
||||
|
||||
- Compliance requirements
|
||||
- Additional layer of security
|
||||
- Reduce security impact of etcd node compromise
|
||||
- Reduce security impact of etcd backups compromise
|
||||
- Ability to use external Key Management Systems
|
||||
|
||||
RKE provides users with two paths of configuration to enable at-rest data encryption:
|
||||
|
||||
- Managed at-rest data encryption
|
||||
- Custom configuration for at-rest data encryption
|
||||
|
||||
Both configuration options can be added during initial cluster provisioning or by updating an exsiting cluster.
|
||||
|
||||
To utilize this feature, a new field `secrets_encryption_config` is added to the [Kubernetes API service configuration]({{<baseurl>}}//rke/latest/en/config-options/services/#kubernetes-api-server). A full custom configuration looks like this:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
custom_config:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: k-fw5hn
|
||||
secret: RTczRjFDODMwQzAyMDVBREU4NDJBMUZFNDhCNzM5N0I=
|
||||
- identity: {}
|
||||
|
||||
```
|
||||
# Managed At-Rest Data Encryption
|
||||
|
||||
Enabling and disabling at-rest data encryption in Kubernetes is a relatively complex process that requires several steps to be performed by the Kubernetes cluster administrator. The managed configuration aims to reduce this overhead and provides a simple abstraction layer to manage the process.
|
||||
|
||||
### Enable Encryption
|
||||
Managed at-rest data encryption is disabled by default and can be enabled by using the following configuration:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
```
|
||||
Once enabled, RKE will perform the following [actions](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#encrypting-your-data) to enable at-rest data encryption:
|
||||
|
||||
- Generate a new random 32-bit encryption key
|
||||
- Generate an encryption provider configuration file using the new key The default [provider](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#providers) used is `aescbc`
|
||||
- Deploy the provider configuration file to all nodes with `controlplane` role
|
||||
- Update the `kube-apiserver` container arguments to point to the provider configuration file.
|
||||
- Restart the `kube-apiserver` container.
|
||||
|
||||
After the `kube-api server` is restarted, data encryption is enabled. However, all existing secrets are still stored in plain text. RKE will [rewrite](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#ensure-all-secrets-are-encrypted) all secrets to ensure encryption is fully in effect.
|
||||
|
||||
### Disable Encryption
|
||||
To disable encryption, you can either set the `enabled` flag to `false`, or simply remove the `secrets_encryption_config` block entirely from cluster.yml.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
Once encryption is disabled in `cluster.yml`, RKE will perform the following [actions](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#encrypting-your-data) to disable encryption in your cluster:
|
||||
|
||||
- Generate a new provider configuration file with the no-encryption `identity{}` provider as the first provider, and the previous `aescbc` set in the second place. This will allow Kubernetes to use the first entry to write the secrets, and the second one to decrypt them.
|
||||
- Deploy the new provider configuration and restart `kube-apiserver`.
|
||||
- Rewrite all secrets. This is required because, at this point, new data will be written to disk in plain text, but the existing data is still encrypted using the old provider. By rewriting all secrets, RKE ensures that all stored data is decrypted.
|
||||
- Update `kube-apiserver` arguments to remove the encryption provider configuration and restart the `kube-apiserver`.
|
||||
- Remove the provider configuration file.
|
||||
|
||||
|
||||
# Key Rotation
|
||||
Sometimes there is a need to rotate encryption config in your cluster. For example, the key is compromised. There are two ways to rotate the keys: with an RKE CLI command, or by disabling and re-enabling encryption in `cluster.yml`.
|
||||
|
||||
### Rotating Keys with the RKE CLI
|
||||
|
||||
With managed configuration, RKE CLI has the ability to perform the key rotation process documented [here](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#rotating-a-decryption-key) with one command. To perform this operation, the following subcommand is used:
|
||||
```bash
|
||||
$ ./rke encrypt rotate-key --help
|
||||
NAME:
|
||||
rke encrypt rotate-key - Rotate cluster encryption provider key
|
||||
|
||||
USAGE:
|
||||
rke encrypt rotate-key [command options] [arguments...]
|
||||
|
||||
OPTIONS:
|
||||
--config value Specify an alternate cluster YAML file (default: "cluster.yml") [$RKE_CONFIG]
|
||||
--ssh-agent-auth Use SSH Agent Auth defined by SSH_AUTH_SOCK
|
||||
--ignore-docker-version Disable Docker version check
|
||||
|
||||
```
|
||||
This command will perform the following actions:
|
||||
|
||||
- Generate a new random 32-bit encryption key
|
||||
- Generate a new provider configuration with the new key as the first provider and the second key as the second provider. When the secrets are rewritten, the first key will be used to encrypt the data on the write operation, while the second key (the old key) will be used to decrypt the stored data during the the read operation
|
||||
- Deploy the new provider configuration to all `controlplane` nodes and restart the `kube-apiserver`
|
||||
- Rewrite all secrets. This process will re-encrypt all the secrets with the new key.
|
||||
- Update the configuration to remove the old key and restart the `kube-apiserver`
|
||||
|
||||
### Rotating Keys by Disabling and Re-enabling Encryption in cluster.yml
|
||||
|
||||
For a cluster with encryption enabled, you can rotate the encryption keys by updating `cluster.yml`. If you enable and re-enable the data encryption in the `cluster.yml`, RKE will not reuse old keys. Instead, it will generate new keys every time, yielding the same result as a key rotation with the RKE CLI.
|
||||
|
||||
# Custom At-Rest Data Encryption Configuration
|
||||
With managed configuration, RKE provides the user with a very simple way to enable and disable encryption with minimal interaction and configuration. However, it doesn't allow for any customization to the configuration.
|
||||
|
||||
With custom encryption configuration, RKE allows the user to provide their own configuration. Although RKE will help the user to deploy the configuration and rewrite the secrets if needed, it doesn't provide a configuration validation on user's behalf. It's the user responsibility to make sure their configuration is valid.
|
||||
|
||||
>**Warning:** Using invalid Encryption Provider Configuration could cause several issues with your cluster, ranging from crashing the Kubernetes API service, `kube-api`, to completely losing access to encrypted data.
|
||||
|
||||
### Example: Using Custom Encryption Configuration with Amazon KMS
|
||||
|
||||
An example for custom configuration would be enabling an external key management system like [Amazon KMS](https://aws.amazon.com/kms/). The following is an example of the configuration for AWS KMS:
|
||||
|
||||
```yaml
|
||||
|
||||
services:
|
||||
kube-api:
|
||||
extra_binds:
|
||||
- "/var/run/kmsplugin/:/var/run/kmsplugin/"
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
custom_config:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- kms:
|
||||
name: aws-encryption-provider
|
||||
endpoint: unix:///var/run/kmsplugin/socket.sock
|
||||
cachesize: 1000
|
||||
timeout: 3s
|
||||
- identity: {}
|
||||
```
|
||||
|
||||
Documentation for AWS KMS can be found [here](https://github.com/kubernetes-sigs/aws-encryption-provider). When Custom Configuration is set to to enable the AWS KMS provider, you should consider the following points:
|
||||
|
||||
- Since RKE runs the `kube-api` service in a container, it's required that you use the `extra_binds` feature to bind-mount the KMS provider socket location inside the `kube-api` container.
|
||||
- The AWS KMS provider runs as a pod in the cluster. Therefor, the proper way to enable it is to:
|
||||
1. Deploy your cluster with at-rest encryption disabled.
|
||||
2. Deploy the KMS pod and make sure it's working correctly.
|
||||
3. Update your cluster with the custom encryption configuration to utilize the KMS provider.
|
||||
- Kube API connects to the KMS provider using a Unix socket. You should configure your KMS deployment to run pods on all `controlplane` nodes in the cluster.
|
||||
- Your `controlplane` node should be configured with an AMI profile that has access to the KMS key you used in your configuration.
|
||||
|
||||
### How to Prevent Restore Failures after Rotating Keys
|
||||
It's important to understand that enabling encryption for you cluster means that you can no longer access encrypted data in your etcd database and/or etcd database backups without using your encryption keys.
|
||||
|
||||
The encryption configuration is stored in the cluster state file `cluster.rkestate`, which is decoupled from the etcd backups. For example, in any of the following backup cases, the restore process will fail:
|
||||
|
||||
- The snapshot is taken while encryption is enabled and restored when it's disabled. In this case, the encryption keys are no longer stored in the cluster state.
|
||||
- The snapshot is taken before the keys are rotated and restore is attempted after. In this case, the old keys used for encryption at the time of the snapshot no longer exist in the cluster state file.
|
||||
|
||||
Therefore, we recommend that when you enable or disable encryption, or when you rotate keys, you should [create a snapshot]({{<baseurl>}}/rke/latest/en/etcd-snapshots/one-time-snapshots/) so that your backup requires the same keys that you have access to.
|
||||
|
||||
This also means you should not rotate the keys during the restore process, because you would lose the encryption keys in `cluster.rkestate`.
|
||||
|
||||
The same applies to the custom configuration use case, however in this case it will depend on the user-provided encryption configuration.
|
||||
@@ -22,7 +22,7 @@ Kubernetes uses [etcd](https://etcd.io/) as a store for cluster state and data.
|
||||
|
||||
RKE supports running etcd in a single node mode or in HA cluster mode. It also supports adding and removing etcd nodes to the cluster.
|
||||
|
||||
You can enable etcd to [take recurring snapshots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#recurring-snapshots). These snapshots can be used to [restore etcd]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-disaster-recovery).
|
||||
You can enable etcd to [take recurring snapshots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#recurring-snapshots). These snapshots can be used to [restore etcd]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-disaster-recovery).
|
||||
|
||||
By default, RKE will deploy a new etcd service, but you can also run Kubernetes with an [external etcd service]({{< baseurl >}}/rke/latest/en/config-options/services/external-etcd/).
|
||||
|
||||
@@ -44,6 +44,8 @@ services:
|
||||
# Enable AlwaysPullImages Admission controller plugin
|
||||
# Available as of v0.2.0
|
||||
always_pull_images: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Kubernetes API Server Options
|
||||
@@ -55,7 +57,7 @@ RKE supports the following options for the `kube-api` service :
|
||||
- **Pod Security Policy** (`pod_security_policy`) - An option to enable the [Kubernetes Pod Security Policy](https://kubernetes.io/docs/concepts/policy/pod-security-policy/). By default, we do not enable pod security policies as it is set to `false`.
|
||||
> **Note:** If you set `pod_security_policy` value to `true`, RKE will configure an open policy to allow any pods to work on the cluster. You will need to configure your own policies to fully utilize PSP.
|
||||
- **Always Pull Images** (`always_pull_images`) - Enable `AlwaysPullImages` Admission controller plugin. Enabling `AlwaysPullImages` is a security best practice. It forces Kubernetes to validate the image and pull credentials with the remote image registry. Local image layer cache will still be used, but it does add a small bit of overhead when launching containers to pull and compare image hashes. _Note: Available as of v0.2.0_
|
||||
|
||||
- **Secrets Encryption Config** (`secrets_encryption_config`) - Manage Kubernetes at-rest data encryption. Documented [here]({{< baseurl >}}//rke/latest/en/config-options/secrets-encryption)
|
||||
## Kubernetes Controller Manager
|
||||
|
||||
> **Note for Rancher 2 users** If you are configuring Cluster Options using a [Config File]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file) when creating [Rancher Launched Kubernetes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), the names of services should contain underscores only: `kube_controller`. This only applies to Rancher v2.0.5 and v2.0.6.
|
||||
@@ -92,6 +94,8 @@ services:
|
||||
cluster_dns_server: 10.43.0.10
|
||||
# Fail if swap is on
|
||||
fail_swap_on: false
|
||||
# Generate per node serving certificate
|
||||
generate_serving_certificate: false
|
||||
```
|
||||
|
||||
### Kubelet Options
|
||||
@@ -101,6 +105,15 @@ RKE supports the following options for the `kubelet` service:
|
||||
- **Cluster Domain** (`cluster_domain`) - The [base domain](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/) for the cluster. All services and DNS records created on the cluster. By default, the domain is set to `cluster.local`.
|
||||
- **Cluster DNS Server** (`cluster_dns_server`) - The IP address assigned to the DNS service endpoint within the cluster. DNS queries will be sent to this IP address which is used by KubeDNS. The default value for this option is `10.43.0.10`
|
||||
- **Fail if Swap is On** (`fail_swap_on`) - In Kubernetes, the default behavior for the kubelet is to **fail** if swap is enabled on the node. RKE does **not** follow this default and allows deployments on nodes with swap enabled. By default, the value is `false`. If you'd like to revert to the default kubelet behavior, set this option to `true`.
|
||||
- **Generate Serving Certificate** (`generate_serving_certificate`) - Generate a certificate signed by the `kube-ca` Certificate Authority for the kubelet to use as a server certificate. The default value for this option is `false`. Before enabling this option, please read [the requirements](#kubelet-serving-certificate-requirements)
|
||||
|
||||
### Kubelet Serving Certificate Requirements
|
||||
|
||||
If `hostname_override` is configured for one or more nodes in `cluster.yml`, please make sure the correct IP address is configured in `address` (and the internal address in `internal_address`) to make sure the generated certificate contains the correct IP address(es).
|
||||
|
||||
An example of an error situation is an EC2 instance where the the public IP address is configured in `address`, and `hostname_override` is used, the connection between `kube-apiserver` and `kubelet` will fail because the `kubelet` will be contacted on the private IP address and the generated certificate will not be valid (the error `x509: certificate is valid for value_in_address, not private_ip` will be seen). The resolution is to provide the internal IP address in `internal_address`.
|
||||
|
||||
For more information on host overrides, refer to the [node configuration page.]({{<baseurl>}}/rke/latest/en/config-options/nodes/#overriding-the-hostname)
|
||||
|
||||
## Kubernetes Scheduler
|
||||
|
||||
|
||||
@@ -9,10 +9,16 @@ RKE supports additional service arguments, volume binds and environment variable
|
||||
|
||||
For any of the Kubernetes services, you can update the `extra_args` to change the existing defaults.
|
||||
|
||||
As of `v0.1.3`, using `extra_args` will add new arguments and **override** any existing defaults. For example, if you need to modify the default admission controllers list, you need to include the default list and edit it with your changes so all changes are included.
|
||||
As of `v0.1.3`, using `extra_args` will add new arguments and **override** any existing defaults. For example, if you need to modify the default admission plugins list, you need to include the default list and edit it with your changes so all changes are included.
|
||||
|
||||
Prior to `v0.1.3`, using `extra_args` would only add new arguments to the list and there was no ability to change the default list.
|
||||
|
||||
All service defaults and parameters are defined per [`kubernetes_version`]({{<baseurl>}}/rke/latest/en/config-options/#kubernetes-version):
|
||||
|
||||
- For RKE v0.3.0+, the service defaults and parameters are defined per [`kubernetes_version`]({{<baseurl>}}/rke/latest/en/config-options/#kubernetes-version). The service defaults are located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go). The default list of admissions plugins is the same for all Kubernetes versions and is located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go#L11).
|
||||
|
||||
- For RKE prior to v0.3.0, the service defaults and admission plugins are defined per [`kubernetes_version`]({{<baseurl>}}/rke/latest/en/config-options/#kubernetes-version) and located [here](https://github.com/rancher/types/blob/release/v2.2/apis/management.cattle.io/v3/k8s_defaults.go).
|
||||
|
||||
```yaml
|
||||
services:
|
||||
kube-controller:
|
||||
|
||||
@@ -8,7 +8,9 @@ The details of restoring your cluster from backup are different depending on you
|
||||
{{% tabs %}}
|
||||
{{% tab "RKE v0.2.0+"%}}
|
||||
|
||||
If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts the etcd to a specific snapshot. The following actions are included in the command:
|
||||
If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot and should be run on an etcd node of the the specific cluster that has suffered the disaster.
|
||||
|
||||
The following actions will be performed when you run the command:
|
||||
|
||||
- Syncs the snapshot or downloads the snapshot from S3, if necessary.
|
||||
- Checks snapshot checksum across etcd nodes to make sure they are identical.
|
||||
@@ -71,9 +73,9 @@ $ rke etcd snapshot-restore \
|
||||
{{% /tab %}}
|
||||
{{% tab "RKE prior to v0.2.0"%}}
|
||||
|
||||
If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot.
|
||||
If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot and should be run on an etcd node of the the specific cluster that has suffered the disaster.
|
||||
|
||||
The following actions are included in `rke etcd snapshot-restore`:
|
||||
The following actions will be performed when you run the command:
|
||||
|
||||
- Removes the old etcd cluster
|
||||
- Rebuilds the etcd cluster using the local snapshot
|
||||
|
||||
+43
@@ -2,6 +2,49 @@ map_hash_bucket_size 256;
|
||||
map $request_uri $redirect_uri {
|
||||
~^/docs/rancher/v2.0(/?.*)$ /docs/rancher/v2.x$1;
|
||||
~^/docs/rke/v0.1.x(/?.*)$ /docs/rke/latest$1;
|
||||
~^/docs/os/quick-start-guide/?$ /rancher-os;
|
||||
~^/docs/rancher/v1.0/zh/rancher-compose/?$ /docs/rancher/v2.x/en/k8s-in-rancher/workloads/add-a-sidecar/;
|
||||
~^/docs/rancher/v1.0/zh/rancher-compose/?$ /docs/rancher/v1.6/en/cattle/rancher-compose/;
|
||||
~^/docs/rancher/rancher-ui/applications/stacks/adding-balancers/?$ /docs/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/;
|
||||
~^/docs/os/running-rancheros/server/install-to-disk/?$ /os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/overview/;
|
||||
~^/docs/rancher/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/;
|
||||
~^/docs/os/running-rancheros/server/install-to-disk/?$ /docs/os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/os/running-rancheros/cloud/gce/?$ /docs/os/v1.x/en/installation/running-rancheros/cloud/gce/;
|
||||
~^/docs/os/amazon-ecs/?$ /docs/os/v1.x/en/installation/running-rancheros/cloud/aws/;
|
||||
~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/overview/architecture/;
|
||||
~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/overview/architecture/;
|
||||
~^/docs/rancher/rancher-compose/?$ /docs/rancher/v2.x/en/;
|
||||
~^/docs/os/networking/interfaces/?$ /docs/os/v1.x/en/installation/networking/interfaces/;
|
||||
~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/;
|
||||
~^/docs/rancher/rancher-services/storage-service/?$ /docs/rancher/v2.x/en/cluster-admin/volumes-and-storage/;
|
||||
~^/docs/rancher/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/installation/ha/;
|
||||
~^/docs/rancher/upgrading/?$ /docs/rancher/v2.x/en/upgrades/;
|
||||
~^/docs/rancher/configuration/access-control/?$ /docs/rancher/v2.x/en/admin-settings/rbac/;
|
||||
~^/docs/os/running-rancheros/server/install-to-disk/?$ /docs/os/v1.x/en/installation/running-rancheros/;
|
||||
~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/os/configuration/custom-rancheros-iso/?$ /docs/os/v1.x/en/installation/configuration/;
|
||||
~^/docs/rancher/rancher-compose/?$ /docs/rancher/v2.x/en/;
|
||||
~^/docs/os/running-rancheros/server/raspberry-pi/?$ /docs/os/v1.x/en/installation/running-rancheros/server/raspberry-pi/;
|
||||
~^/docs/rancher/v1.0/zh/installing-rancher/installing-server/?$ /docs/rancher/v1.6/en/installing-rancher/installing-server/;
|
||||
~^/docs/rancher/rancher-services/metadata-service/?$ /docs/rancher/v1.6/en/rancher-services/metadata-service/;
|
||||
~^/docs/rancher/api/?$ /docs/rancher/v2.x/en/api/;
|
||||
~^/docs/os/running-rancheros/server/raspberry-pi/?$ /docs/os/v1.x/en/installation/running-rancheros/server/raspberry-pi/;
|
||||
~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/;
|
||||
~^/docs/os/configuration/switching-consoles/?$ /docs/os/v1.x/en/about/recovery-console/;
|
||||
~^/docs/os/running-rancheros/server/install-to-disk//?$ /docs/os/v1.x/en/installation/running-rancheros/server/install-to-disk/;
|
||||
~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/;
|
||||
~^/docs/rancher/v1.2/en/installing-rancher/installing-server/multi-nodes//?$ /docs/rancher/v2.x/en/installation/ha/;
|
||||
~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/;
|
||||
~^/docs/os/system-services/adding-system-services/?$ /docs/os/v1.x/en/installation/system-services/adding-system-services/;
|
||||
~^/docs/rancher/installing-rancher/installing-server/?$ /docs/rancher/v2.x/en/installation/;
|
||||
~^/docs/rancher/latest/en/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/installation/ha/;
|
||||
~^/docs/rancher/rancher-services/metadata-service/?$ /docs/rancher/v1.6/en/rancher-services/metadata-service/;
|
||||
~^/docs/rancher/rancher-services/health-checks/?$ /docs/rancher/v1.0/en/rancher-services/health-checks/;
|
||||
~^/docs/os/cloud-config/?$ /docs/os/v1.x/en/installation/configuration/;
|
||||
~^/docs/rancher/api/?$ /docs/rancher/v2.x/en/api/;
|
||||
}
|
||||
|
||||
server {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 103 KiB |
Reference in New Issue
Block a user