Merge branch 'master' into master

This commit is contained in:
Catherine Luse
2020-01-09 21:30:58 -07:00
committed by GitHub
186 changed files with 1994 additions and 1397 deletions
+1
View File
@@ -5,3 +5,4 @@ package-lock.json
*.tern-port
*/**/.tern-port
.DS_Store
.vscode/settings.json
-2
View File
@@ -1,2 +0,0 @@
{
}
+2 -3
View File
@@ -14,13 +14,12 @@ Great for:
* ARM
* Situations where a PhD in k8s clusterology is infeasible
What is this?
---
# What is K3s?
K3s is a fully compliant Kubernetes distribution with the following enhancements:
* An embedded SQLite database has replaced etcd as the default datastore. External datastores such as PostgreSQL, MySQL, and etcd are also supported.
* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a helm controller, and the Traefik ingress controller.
* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller.
* Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates.
* In-tree cloud providers and storage plugins have been removed.
* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including:
+216 -22
View File
@@ -1,23 +1,209 @@
---
title: "Advanced Options"
title: "Advanced Options and Configuration"
weight: 40
aliases:
- /k3s/latest/en/running/
- /k3s/latest/en/configuration/
---
This section contains advanced information describing the different ways you can run and manage K3s.
This section contains advanced information describing the different ways you can run and manage K3s:
Starting the Server
------------------
- [Setting Up the kubeconfig File](#setting-up-the-kubeconfig-file)
- [Using Helm 3](#using-helm-3)
- [Auto-deploying manifests](#auto-deploying-manifests)
- [Using the Helm CRD](#using-the-helm-crd)
- [Accessing the cluster from outside with kubectl](#accessing-the-cluster-from-outside-with-kubectl)
- [Using Docker as the container runtime](#using-docker-as-the-container-runtime)
- [Running K3s with RootlessKit (Experimental)](#running-k3s-with-rootlesskit-experimental)
- [Node labels and taints](#node-labels-and-taints)
- [Starting the server with the installation script](#starting-the-server-with-the-installation-script)
- [Additional preparation for Alpine Linux setup](#additional-preparation-for-alpine-linux-setup)
- [Running K3d (K3s in Docker) and docker-compose](#running-k3d-k3s-in-docker-and-docker-compose)
# Setting Up the kubeconfig File
The kubeconfig file is used to configure access to the Kubernetes cluster. It is required to be set up properly in order to access the Kubernetes API such as with kubectl or for installing applications with Helm. You may set the kubeconfig by either exporting the KUBECONFIG environment variable or by specifying a flag for kubectl and helm. Refer to the examples below for details.
Leverage the KUBECONFIG environment variable:
```
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
kubectl get pods --all-namespaces
helm ls --all-namespaces
```
Or specify the location of the kubeconfig file per command:
```
kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get pods --all-namespaces
helm --kubeconfig /etc/rancher/k3s/k3s.yaml ls --all-namespaces
```
# Using Helm 3
K3s release _v1.17.0+k3s.1_ added support for Helm 3. You can access the Helm 3 documentation [here](https://helm.sh/docs/intro/quickstart/).
Note that Helm 3 no longer requires tiller and the helm init command. Refer to the official documentation for details.
K3s does not require any special configuration to start using Helm 3. Just be sure you have properly set up your kubeconfig as per the [Setting Up the kubeconfig File](#setting-up-the-kubeconfig-file) section above.
### Upgrading
If you were using Helm v2 in previous versions of K3s, you may upgrade to v1.17.0+k3s.1 or newer and Helm 2 will still function. If you wish to migrate to Helm 3, [this](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) blog post by Helm explains how to use a plugin to successfully migrate. Refer to the official Helm 3 documentation [here](https://helm.sh/docs/) for more information. K3s will handle either Helm v2 or Helm v3 as of v1.17.0+k3s.1. Just be sure you have properly set your kubeconfig as per the examples above in the [Setting Up the kubeconfig File](#setting-up-the-kubeconfig-file) section.
# Auto-Deploying Manifests
Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to
Kubernetes in a manner similar to `kubectl apply`.
It is also possible to deploy Helm charts. K3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`):
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: traefik
namespace: kube-system
spec:
chart: stable/traefik
set:
rbac.enabled: "true"
ssl.enabled: "true"
```
Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual Helm release, you can do that using `targetNamespace` key under the `spec` directive, as shown in the configuration example below.
> **Note:** In order for the Helm Controller to know which version of Helm to use to Auto-Deploy a helm app, please specify the `helmVersion` in the spec of your YAML file.
Also note that besides `set`, you can use `valuesContent` under the `spec` directive. And it's okay to use both of them:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: grafana
namespace: kube-system
spec:
chart: stable/grafana
targetNamespace: monitoring
set:
adminPassword: "NotVerySafePassword"
valuesContent: |-
image:
tag: master
env:
GF_EXPLORE_ENABLED: true
adminUser: admin
sidecar:
datasources:
enabled: true
```
K3s versions `<= v0.5.0` used `k3s.cattle.io` for the API group of HelmCharts. This has been changed to `helm.cattle.io` for later versions.
# Using the Helm CRD
You can deploy a 3rd party Helm chart using an example like this:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: nginx
namespace: kube-system
spec:
chart: nginx
repo: https://charts.bitnami.com/bitnami
targetNamespace: default
```
You can install a specific version of a Helm chart using an example like this:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: stable/nginx-ingress
namespace: kube-system
spec:
chart: nginx-ingress
version: 1.24.4
targetNamespace: default
```
# Accessing the Cluster from Outside with kubectl
Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace "localhost" with the IP or name of your K3s server. `kubectl` can now manage your K3s cluster.
# Using Docker as the Container Runtime
K3s includes and defaults to [containerd,](https://containerd.io/) an industry-standard container runtime. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag.
K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`. For advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template, the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32
# Running K3s with RootlessKit (Experimental)
> **Warning:** This feature is experimental.
RootlessKit is a kind of Linux-native "fake root" utility, made for mainly [running Docker and Kubernetes as an unprivileged user,](https://github.com/rootless-containers/usernetes) so as to protect the real root on the host from potential container-breakout attacks.
Initial rootless support has been added but there are a series of significant usability issues surrounding it.
We are releasing the initial support for those interested in rootless and hopefully some people can help to improve the usability. First, ensure you have a proper setup and support for user namespaces. Refer to the [requirements section](https://github.com/rootless-containers/rootlesskit#setup) in RootlessKit for instructions.
In short, latest Ubuntu is your best bet for this to work.
### Known Issues with RootlessKit
* **Ports**
When running rootless a new network namespace is created. This means that K3s instance is running with networking fairly detached from the host. The only way to access services run in K3s from the host is to set up port forwards to the K3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000.
That means service port 80 will become 10080 on the host, but 8080 will become 8080 without any offset.
Currently, only `LoadBalancer` services are automatically bound.
* **Daemon lifecycle**
Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left
with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace.
The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65
* **Cgroups**
Cgroups are not supported.
### Running Servers and Agents with Rootless
Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message
`Wrote kubeconfig [SOME PATH]` for where your kubeconfig to access you cluster is.
> Be careful, if you use `-o` to write
the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different
mount namespace.
# Node Labels and Taints
K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints [at registration time,]({{<baseurl>}}/k3s/latest/en/installation/install-options/#node-labels-and-taints-for-agents) so they can only be added once and not changed after that again by running K3s commands.
If you want to change node labels and taints after node registration you should use `kubectl`. Refer to the official Kubernetes documentation for details on how to add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) and [node labels.](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node)
# Starting the Server with the Installation Script
The installation script will auto-detect if your OS is using systemd or openrc and start the service.
When running with openrc logs will be created at `/var/log/k3s.log`, or with systemd in `/var/log/syslog` and viewed using `journalctl -u k3s`. An example of installing and auto-starting with the install script:
When running with openrc, logs will be created at `/var/log/k3s.log`.
When running with systemd, logs will be created in `/var/log/syslog` and viewed using `journalctl -u k3s`.
An example of installing and auto-starting with the install script:
```bash
curl -sfL https://get.k3s.io | sh -
```
When running the server manually you should get an output similar to:
When running the server manually you should get an output similar to the following:
```
$ k3s server
@@ -38,10 +224,9 @@ INFO[2019-01-22T15:16:20.541049100-07:00] Run: k3s kubectl
The output will likely be much longer as the agent will create a lot of logs. By default the server
will register itself as a node (run the agent).
Alpine Linux
------------
# Additional Preparation for Alpine Linux Setup
In order to pre-setup Alpine Linux you have to go through the following steps:
In order to set up Alpine Linux, you have to go through the following preparation:
```bash
echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab
@@ -75,19 +260,26 @@ reboot
After rebooting:
- download **k3s** to **/usr/local/bin/k3s**
- create an openrc file in **/etc/init.d**
- Download **k3s** to **/usr/local/bin/k3s**
- Create an openrc file in **/etc/init.d**
Running in Docker (and docker-compose)
-----------------
# Running K3d (K3s in Docker) and docker-compose
[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS.
[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker.
`rancher/k3s` images are also available to run K3s server and agent from Docker. A `docker-compose.yml` is in the root of the K3s repo that
serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo run:
It can be installed via the the [brew](https://brew.sh/) utility on MacOS:
```
brew install k3d
```
`rancher/k3s` images are also available to run the K3s server and agent from Docker.
A `docker-compose.yml` is in the root of the K3s repo that serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo, run:
docker-compose up --scale node=3
# kubeconfig is written to current dir
kubectl --kubeconfig kubeconfig.yaml get node
NAME STATUS ROLES AGE VERSION
@@ -95,12 +287,14 @@ serves as an example of how to run K3s from Docker. To run from `docker-compose
d54c8b17c055 Ready <none> 11s v1.13.2-k3s2
db7a5a5a5bdd Ready <none> 12s v1.13.2-k3s2
To run the agent only in Docker, use `docker-compose up node`. Alternatively the Docker run command can also be used;
To run the agent only in Docker, use `docker-compose up node`.
Alternatively the `docker run` command can also be used:
sudo docker run \
-d --tmpfs /run \
--tmpfs /var/run \
-e K3S_URL=${SERVER_URL} \
-e K3S_TOKEN=${NODE_TOKEN} \
--privileged rancher/k3s:vX.Y.Z
-d --tmpfs /run \
--tmpfs /var/run \
-e K3S_URL=${SERVER_URL} \
-e K3S_TOKEN=${NODE_TOKEN} \
--privileged rancher/k3s:vX.Y.Z
@@ -0,0 +1,52 @@
---
title: Architecture
weight: 1
---
This page describes the architecture of a high-availability K3s server cluster and how it differs from a single-node server cluster.
It also describes how agent nodes are registered with K3s servers.
A server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command.
This page covers the following topics:
- [Single-server setup with an embedded database](#single-server-setup-with-an-embedded-db)
- [High-availability K3s server with an external database](#high-availability-k3s-server-with-an-external-db)
- [Fixed registration address for agent nodes](#fixed-registration-address-for-agent-nodes)
- [How agent node registration works](#how-agent-node-registration-works)
# Single-server Setup with an Embedded DB
The following diagram shows an example of a cluster that has a single-node K3s server with an embedded SQLite database.
In this configuration, each agent node is registered to the same server node. A K3s user can manipulate Kubernetes resources by calling the K3s API on the server node.
![Architecture]({{<baseurl>}}/img/rancher/k3s-single-node-server-architecture.svg)
# High-Availability K3s Server with an External DB
Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of:
* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services
* An **external datastore** (as opposed to the embedded SQLite datastore used in single-server setups)
![Architecture]({{< baseurl >}}/img/rancher/k3s-ha-architecture.svg)
### Fixed Registration Address for Agent Nodes
In the high-availability server configuration, each node must also register with the Kubernetes API by using a fixed registration address, as shown in the diagram below.
After registration, the agent nodes establish a connection directly to one of the server nodes.
![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg)
# How Agent Node Registration Works
Agent nodes are registered with a websocket connection initiated by the `k3s agent` process, and the connection is maintained by a client-side load balancer running as part of the agent process.
Agents will register with the server using the node cluster secret along with a randomly generated password for the node, stored at `/etc/rancher/node/password`. The server will store the passwords for individual nodes at `/var/lib/rancher/k3s/server/cred/node-passwd`, and any subsequent attempts must use the same password.
If the `/etc/rancher/node` directory of an agent is removed, the password file should be recreated for the agent, or the entry removed from the server.
A unique node ID can be appended to the hostname by launching K3s servers or agents using the `--with-node-id` flag.
@@ -1,165 +0,0 @@
---
title: "Configuration Info"
weight: 50
---
This section contains information on using K3s with various configurations.
Auto-Deploying Manifests
------------------------
Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to
Kubernetes in a manner similar to `kubectl apply`.
It is also possible to deploy Helm charts. K3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`):
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: traefik
namespace: kube-system
spec:
chart: stable/traefik
set:
rbac.enabled: "true"
ssl.enabled: "true"
```
Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section:
```
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: grafana
namespace: kube-system
spec:
chart: stable/grafana
targetNamespace: monitoring
set:
adminPassword: "NotVerySafePassword"
valuesContent: |-
image:
tag: master
env:
GF_EXPLORE_ENABLED: true
adminUser: admin
sidecar:
datasources:
enabled: true
```
Also note that besides `set` you can use `valuesContent` in the spec section. And it's okay to use both of them.
K3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions.
Using the helm CRD
---------------------
You can deploy a 3rd party helm chart using an example like this:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: nginx
namespace: kube-system
spec:
chart: nginx
repo: https://charts.bitnami.com/bitnami
targetNamespace: default
```
You can install a specific version of a helm chart using an example like this:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
name: stable/nginx-ingress
namespace: kube-system
spec:
chart: nginx-ingress
version: 1.24.4
targetNamespace: default
```
Accessing Cluster from Outside
-----------------------------
Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace
"localhost" with the IP or name of your K3s server. `kubectl` can now manage your K3s cluster.
Node Registration
-----------------
Agents will register with the server using the node cluster secret along with a randomly generated
password for the node, stored at `/etc/rancher/node/password`. The server will
store the passwords for individual nodes at `/var/lib/rancher/k3s/server/cred/node-passwd`, and any
subsequent attempts must use the same password. If the `/etc/rancher/node` directory of an agent is removed the
password file should be recreated for the agent, or the entry removed from the server. A unique node
id can be appended to the hostname by launching k3s servers or agents using the `--with-node-id` flag.
Containerd and Docker
----------
K3s includes and defaults to containerd. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag.
K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template, the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32
Rootless (Experimental)
--------
_**WARNING**:_ Experimental feature
Initial rootless support has been added but there are a series of significant usability issues surrounding it.
We are releasing the initial support for those interested in rootless and hopefully some people can help to
improve the usability. First ensure you have proper setup and support for user namespaces. Refer to the
[requirements section](https://github.com/rootless-containers/rootlesskit#setup) in RootlessKit for instructions.
In short, latest Ubuntu is your best bet for this to work.
**Issues w/ Rootless**:
* **Ports**
When running rootless a new network namespace is created. This means that K3s instance is running with networking
fairly detached from the host. The only way to access services run in K3s from the host is to setup port forwards
to the K3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000.
That means service port 80 will become 10080 on the host, but 8080 will become 8080 without any offset.
Currently, only `LoadBalancer` services are automatically bound.
* **Daemon lifecycle**
Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left
with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace.
The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65
* **Cgroups**
Cgroups are not supported
**Running w/ Rootless**:
Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message
`Wrote kubeconfig [SOME PATH]` for where your kubeconfig to access you cluster is. Be careful, if you use `-o` to write
the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different
mount namespace.
Node Labels and Taints
----------------------
K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s. If you want to change node labels and taints after node registration you should use `kubectl`. Below is an example showing how to add labels and a taint:
```
--node-label foo=bar \
--node-label hello=world \
--node-taint key1=value1:NoExecute
```
+4 -4
View File
@@ -8,12 +8,12 @@ This section contains instructions for installing K3s in various environments. P
[Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) provides guidance on the options available to you when installing K3s.
[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to setup an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd.
[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to set up an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd.
[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to setup an HA K3s cluster that leverages a built-in distributed database.
[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to set up an HA K3s cluster that leverages a built-in distributed database.
[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to setup K3s in environments that do not have direct access to the Internet.
[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to set up K3s in environments that do not have direct access to the Internet.
### Uninstalling
If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`).
If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation. The script is created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`).
@@ -5,11 +5,11 @@ weight: 60
In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server.
Installation Outline
--------------------
1. Prepare Images Directory
2. Create Registry YAML
3. Install K3s
# Installation Outline
1. [Prepare Images Directory](#prepare-images-directory)
2. [Create Registry YAML](#create-registry-YAML)
3. [Install K3s](#install-k3s)
### Prepare Images Directory
Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running.
@@ -94,4 +94,4 @@ k3s server
```
### Embedded DQLite for HA (Experimental)
K3s's use of DQLite is similar to its use of SQLite. It is simple to setup and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option.
K3s's use of DQLite is similar to its use of SQLite. It is simple to set up and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option.
+19 -20
View File
@@ -1,36 +1,35 @@
---
title: "High Availability with an External DB"
title: High Availability with an External DB
weight: 30
---
>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release.
This section describes how to install a high-availability K3s cluster with an external database.
Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of:
* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services
* An **external datastore** (as opposed to the embedded SQLite datastore used in single server setups)
* A **fixed registration address** placed in front of the server nodes to allow worker nodes to register with the cluster
* An **external datastore** (as opposed to the embedded SQLite datastore used in single-server setups)
* A **fixed registration address** that is placed in front of the server nodes to allow worker nodes to register with the cluster
The following diagram illustrates the above configuration:
![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg)
For more details on how these components work together, refer to the [architecture section.]({{<baseurl>}}/k3s/latest/en/architecture/#high-availability-with-an-external-db)
In this architecture a server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command.
Workers register through the fixed registration address, but after registration they establish a connection directly to one of the server nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process.
Workers register through the fixed registration address, but after registration they establish a connection directly to one of the sever nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process.
# Installation Outline
Installation Outline
--------------------
Setting up an HA cluster requires the following steps:
1. Create an external datastore
2. Launch server nodes
3. Configure fixed registration address
4. Join worker nodes
1. [Create an external datastore](#1-create-an-external-datastore)
2. [Launch server nodes](#2-launch-server-nodes)
3. [Configure the fixed registration address](#3-configure-the-fixed-registration-address)
4. [Join worker nodes](#4-join-worker-nodes)
### Create an External Datastore
### 1. Create an External Datastore
You will first need to create an external datastore for the cluster. See the [Cluster Datastore Options]({{< baseurl >}}/k3s/latest/en/installation/datastore/) documentation for more details.
### Launch Server Nodes
### 2. Launch Server Nodes
K3s requires two or more server nodes for this HA configuration. See the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) guide for minimum machine requirements.
When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. Please see the [datastore configuration guide]({{< baseurl >}}/k3s/latest/en/installation/datastore/#external-datastore-configuration-parameters) for information on configuring this parameter.
@@ -41,16 +40,16 @@ By default, server nodes will be schedulable and thus your workloads can get lau
Once you've launched the `k3s server` process on all server nodes, you can ensure that the cluster has come up properly by checking that the nodes are in the Ready state with `k3s kubectl get nodes`.
### Configure the Fixed Registration Address
Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be setup using any number approaches, such as:
### 3. Configure the Fixed Registration Address
Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be set up using any number approaches, such as:
* A layer-4 (TCP) load balancer
* Round-robin DNS
* A virtual or elastic IP addresses
* Virtual or elastic IP addresses
This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your kubeconfig file to point to it instead of a specific node.
This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file to point to it instead of a specific node.
### Join Worker Nodes
### 4. Join Worker Nodes
Joining worker nodes in an HA cluster is the same as joining worker nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use.
```
K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443
@@ -1,9 +1,18 @@
---
title: "Installation and Configuration Options"
title: "Installation Options"
weight: 20
---
### Installation script options
This page focuses on the options that can be used when you set up K3s for the first time:
- [Installation script options](#installation-script-options)
- [Installing K3s from the binary](#installing-k3s-from-the-binary)
- [Registration options for the K3s server](#registration-options-for-the-k3s-server)
- [Registration options for the K3s agent](#registration-options-for-the-k3s-agent)
For more advanced options, refer to [this page.]({{<baseurl>}}/k3s/latest/en/advanced)
# Installation Script Options
As mentioned in the [Quick-Start Guide]({{< baseurl >}}/k3s/latest/en/quick-start/), you can use the installation script available at https://get.k3s.io to install K3s as a service on systemd and openrc based systems.
@@ -36,7 +45,7 @@ When using this method to install K3s, the following environment variables can b
- `INSTALL_K3S_BIN_DIR_READ_ONLY`
If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting INSTALL_K3S_SKIP_DOWNLOAD=true.
If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting `INSTALL_K3S_SKIP_DOWNLOAD=true`.
- `INSTALL_K3S_SYSTEMD_DIR`
@@ -44,7 +53,9 @@ When using this method to install K3s, the following environment variables can b
- `INSTALL_K3S_EXEC`
Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior:
Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set.
The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior of registering a server without flannel:
```sh
curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s -
curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s -
@@ -65,7 +76,8 @@ When using this method to install K3s, the following environment variables can b
Environment variables which begin with `K3S_` will be preserved for the systemd and openrc services to use. Setting `K3S_URL` without explicitly setting an exec command will default the command to "agent". When running the agent `K3S_TOKEN` must also be set.
### Beyond the Installation Script
# Installing K3s from the Binary
As stated, the installation script is primarily concerned with configuring K3s to run as a service. If you choose to not use the script, you can run K3s simply by downloading the binary from our [release page](https://github.com/rancher/k3s/releases/latest), placing it on your path, and executing it. The K3s binary supports the following commands:
Command | Description
@@ -79,7 +91,7 @@ Command | Description
The `k3s server` and `k3s agent` commands have additional configuration options that can be viewed with <span class='nowrap'>`k3s server --help`</span> or <span class='nowrap'>`k3s agent --help`</span>. For convenience, that help text is presented here:
### `k3s server`
# Registration Options for the K3s Server
```
NAME:
k3s server - Run management server
@@ -145,7 +157,7 @@ OPTIONS:
--cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET]
```
### `k3s agent`
# Registration Options for the K3s Agent
```
NAME:
k3s agent - Run node agent
@@ -181,3 +193,16 @@ OPTIONS:
--no-flannel (deprecated) use --flannel-backend=none
--cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET]
```
### Node Labels and Taints for Agents
K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s commands.
Below is an example showing how to add labels and a taint:
```
--node-label foo=bar \
--node-label hello=world \
--node-taint key1=value1:NoExecute
```
If you want to change node labels and taints after node registration you should use `kubectl`. Refer to the official Kubernetes documentation for details on how to add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) and [node labels.](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node)
@@ -34,7 +34,7 @@ Visit the [Project Calico Docs](https://docs.projectcalico.org/) website. Follow
}
```
Applyl the Canal YAML.
Apply the Canal YAML.
Ensure the settings were applied by running the following command on the host:
@@ -68,4 +68,3 @@ You should see that IP forwarding is set to true.
{{% /tab %}}
{{% /tabs %}}
@@ -18,6 +18,8 @@ K3s should run on just about any flavor of Linux. However, K3s is tested on the
* Ubuntu 18.04 (amd64)
* Raspbian Buster (armhf)
> If you are using Alpine Linux, follow [these steps]({{<baseurl>}}/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup) for additional setup.
## Hardware
Hardware requirements scale based on the size of your deployments. Minimum recommendations are outlined here.
@@ -11,3 +11,7 @@ If you plan to use K3s with docker, Docker installed via a snap package is not r
**Iptables**
If you are running iptables in nftables mode instead of legacy you might encounter issues. We recommend utilizing newer iptables (such as 1.6.1+) to avoid issues.
**RootlessKit**
Running K3s with RootlessKit is experimental and has several [known issues.]({{<baseurl>}}/k3s/latest/en/advanced/#known-issues-with-rootlesskit)
+8 -9
View File
@@ -12,27 +12,26 @@ Please reference the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installat
CoreDNS
-------
CoreDNS is deployed on start of the agent, to disable run each server with the `--no-deploy coredns` option.
CoreDNS is deployed on start of the agent. To disable, run each server with the `--no-deploy coredns` option.
If you don't install CoreDNS you will need to install a cluster DNS provider yourself.
If you don't install CoreDNS, you will need to install a cluster DNS provider yourself.
Traefik Ingress Controller
--------------------------
Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/configuration/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`.
[Traefik](https://traefik.io/) is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. It simplifies networking complexity while designing, deploying, and running applications.
Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/advanced/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`.
The Traefik ingress controller will use ports 80, 443, and 8080 on the host (i.e. these will not be usable for HostPort or NodePort).
You can tweak traefik to meet your needs by setting options in the traefik.yaml file.
Reference the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information.
You can tweak traefik to meet your needs by setting options in the traefik.yaml file. Refer to the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information.
To disable it, start each server with the `--no-deploy traefik` option.
Service Load Balancer
---------------------
K3s includes a basic service load balancer that uses available host ports. If you try to create
a load balancer that listens on port 80, for example, it will try to find a free host in the cluster
for port 80. If no port is available the load balancer will stay in Pending.
K3s includes a basic service load balancer that uses available host ports. If you try to create a load balancer that listens on port 80, for example, it will try to find a free host in the cluster for port 80. If no port is available, the load balancer will stay in Pending.
To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB.
To disable the embedded load balancer, run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB.
+4 -2
View File
@@ -3,7 +3,9 @@ title: "Quick-Start Guide"
weight: 10
---
>**Note:** This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up.
This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up.
For information on how K3s components work together, refer to the [architecture section.]({{<baseurl>}}/k3s/latest/en/architecture/#high-availability-with-an-external-db)
> New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/).
@@ -18,7 +20,7 @@ After running this installation:
* The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed
* Additional utilities will be installed, including `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`
* A kubeconfig file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it
* A [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it
To install on worker nodes and add them to the cluster, run the installation script with the `K3S_URL` and `K3S_TOKEN` environment variables. Here is an example showing how to join a worker node:
+39 -9
View File
@@ -5,7 +5,11 @@ weight: 30
When deploying an application that needs to retain data, youll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the applications pod fails.
# Local Storage Provider
A persistent volume (PV) is a piece of storage in the Kubernetes cluster, while a persistent volume claim (PVC) is a request for storage. For details on how PVs and PVCs work, refer to the official Kubernetes documentation on [storage.](https://kubernetes.io/docs/concepts/storage/volumes/)
This page describes how to set up persistent storage with a local storage provider, or with [Longhorn.](#setting-up-longhorn)
# Setting up the Local Storage Provider
K3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage).
Create a hostPath backed persistent volume claim and a pod to utilize it:
@@ -51,19 +55,33 @@ spec:
claimName: local-path-pvc
```
Apply the yaml `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml`
Apply the yaml:
Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each.
```
kubectl create -f pvc.yaml
kubectl create -f pod.yaml
```
# Longhorn
Confirm the PV and PVC are created:
```
kubectl get pv
kubectl get pvc
```
The status should be Bound for each.
# Setting up Longhorn
[comment]: <> (pending change - longhorn may support arm64 and armhf in the future.)
> **Note:** At this time Longhorn only supports amd64.
K3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md).
K3s supports [Longhorn](https://github.com/longhorn/longhorn). Longhorn is an open-source distributed block storage system for Kubernetes.
Apply the longhorn.yaml to install Longhorn.
Below we cover a simple example. For more information, refer to the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md).
Apply the longhorn.yaml to install Longhorn:
```
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
@@ -71,13 +89,18 @@ kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/depl
Longhorn will be installed in the namespace `longhorn-system`.
Before we create a PVC, we will create a storage class for longhorn with this yaml.
Before we create a PVC, we will create a storage class for Longhorn with this yaml:
```
kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/storageclass.yaml
```
Now, apply the following yaml to create the PVC and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml`
Apply the yaml to create the PVC and pod:
```
kubectl create -f pvc.yaml
kubectl create -f pod.yaml
```
### pvc.yaml
@@ -119,4 +142,11 @@ spec:
claimName: longhorn-volv-pvc
```
Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each.
Confirm the PV and PVC are created:
```
kubectl get pv
kubectl get pvc
```
The status should be Bound for each.
+9 -1
View File
@@ -3,7 +3,11 @@ title: "Upgrades"
weight: 25
---
>**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes.
You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version.
>**Note:** When upgrading, upgrade server nodes first one at a time, then any worker nodes.
### Upgrade K3s Using the Installation Script
To upgrade K3s from an older version you can re-run the installation script using the same flags, for example:
@@ -17,6 +21,8 @@ If you want to upgrade to specific version you can run the following command:
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh -
```
### Manually Upgrade K3s Using the Binary
Or to manually upgrade K3s:
1. Download the desired version of K3s from [releases](https://github.com/rancher/k3s/releases/latest)
@@ -24,6 +30,8 @@ Or to manually upgrade K3s:
3. Stop the old version
4. Start the new version
### Restarting K3s
Restarting K3s is supported by the installation script for systemd and openrc.
To restart manually for systemd use:
```sh
+1 -1
View File
@@ -14,7 +14,7 @@ weight: 303
<p style="padding: 8px">Please submit possible security issues by emailing <a href="mailto:security@rancher.com">security@rancher.com</a></p>
</td>
<td width="30%" style="border: none;">
<h4>Announcments</h4>
<h4>Announcements</h4>
<p style="padding: 8px">Subscribe to the <a href="https://forums.rancher.com/c/announcements">Rancher announcements forum</a> for release updates.</p>
</td>
</tr>
@@ -62,21 +62,21 @@ Latest Release: [v1.5.4](https://github.com/rancher/os/releases/tag/v1.5.4)
Region | Type | AMI
---|--- | ---
eu-north-1 | HVM - ECS enabled | [ami-0d0ea423c99b99528](https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#launchInstanceWizard:ami=ami-0d0ea423c99b99528)
ap-south-1 | HVM - ECS enabled | [ami-026d573feb2b40494](https://ap-south-1.console.aws.amazon.com/ec2/home?region=ap-south-1#launchInstanceWizard:ami=ami-026d573feb2b40494)
eu-west-3 | HVM - ECS enabled | [ami-0b00515ac5791981a](https://eu-west-3.console.aws.amazon.com/ec2/home?region=eu-west-3#launchInstanceWizard:ami=ami-0b00515ac5791981a)
eu-west-2 | HVM - ECS enabled | [ami-017c58d5ed26b91f4](https://eu-west-2.console.aws.amazon.com/ec2/home?region=eu-west-2#launchInstanceWizard:ami=ami-017c58d5ed26b91f4)
eu-west-1 | HVM - ECS enabled | [ami-00863d13761ef3724](https://eu-west-1.console.aws.amazon.com/ec2/home?region=eu-west-1#launchInstanceWizard:ami=ami-00863d13761ef3724)
ap-northeast-2 | HVM - ECS enabled | [ami-04e09c2c9c4b59d54](https://ap-northeast-2.console.aws.amazon.com/ec2/home?region=ap-northeast-2#launchInstanceWizard:ami=ami-04e09c2c9c4b59d54)
ap-northeast-1 | HVM - ECS enabled | [ami-0e5d74499b8bd1119](https://ap-northeast-1.console.aws.amazon.com/ec2/home?region=ap-northeast-1#launchInstanceWizard:ami=ami-0e5d74499b8bd1119)
sa-east-1 | HVM - ECS enabled | [ami-033aa9f2a4ea1c2ab](https://sa-east-1.console.aws.amazon.com/ec2/home?region=sa-east-1#launchInstanceWizard:ami=ami-033aa9f2a4ea1c2ab)
ca-central-1 | HVM - ECS enabled | [ami-0002d992ae120cc94](https://ca-central-1.console.aws.amazon.com/ec2/home?region=ca-central-1#launchInstanceWizard:ami=ami-0002d992ae120cc94)
ap-southeast-1 | HVM - ECS enabled | [ami-0524bbc4fb51f2190](https://ap-southeast-1.console.aws.amazon.com/ec2/home?region=ap-southeast-1#launchInstanceWizard:ami=ami-0524bbc4fb51f2190)
ap-southeast-2 | HVM - ECS enabled | [ami-09be0a7f78760ed49](https://ap-southeast-2.console.aws.amazon.com/ec2/home?region=ap-southeast-2#launchInstanceWizard:ami=ami-09be0a7f78760ed49)
eu-central-1 | HVM - ECS enabled | [ami-08b437124d5650e75](https://eu-central-1.console.aws.amazon.com/ec2/home?region=eu-central-1#launchInstanceWizard:ami=ami-08b437124d5650e75)
us-east-1 | HVM - ECS enabled | [ami-047c2cc8ce83362d5](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#launchInstanceWizard:ami=ami-047c2cc8ce83362d5)
us-east-2 | HVM - ECS enabled | [ami-022fc798437fb846a](https://us-east-2.console.aws.amazon.com/ec2/home?region=us-east-2#launchInstanceWizard:ami=ami-022fc798437fb846a)
us-west-1 | HVM - ECS enabled | [ami-00d236646389e14d0](https://us-west-1.console.aws.amazon.com/ec2/home?region=us-west-1#launchInstanceWizard:ami=ami-00d236646389e14d0)
us-west-2 | HVM - ECS enabled | [ami-0c12fa819f3adc03d](https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#launchInstanceWizard:ami=ami-0c12fa819f3adc03d)
cn-north-1 | HVM - ECS enabled | [ami-092d12a2396dc0822](https://cn-north-1.console.amazonaws.cn/ec2/home?region=cn-north-1#launchInstanceWizard:ami=ami-092d12a2396dc0822)
cn-northwest-1 | HVM - ECS enabled | [ami-0d63e8f32b5873368](https://cn-northwest-1.console.amazonaws.cn/ec2/home?region=cn-northwest-1#launchInstanceWizard:ami=ami-0d63e8f32b5873368)
eu-north-1 | HVM - ECS enabled | [ami-0c46c1da6468aa948](https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#launchInstanceWizard:ami=ami-0c46c1da6468aa948)
ap-south-1 | HVM - ECS enabled | [ami-097e5fa868c46e925](https://ap-south-1.console.aws.amazon.com/ec2/home?region=ap-south-1#launchInstanceWizard:ami=ami-097e5fa868c46e925)
eu-west-3 | HVM - ECS enabled | [ami-016e7d630d7f608e4](https://eu-west-3.console.aws.amazon.com/ec2/home?region=eu-west-3#launchInstanceWizard:ami=ami-016e7d630d7f608e4)
eu-west-2 | HVM - ECS enabled | [ami-00aacd261ab72302e](https://eu-west-2.console.aws.amazon.com/ec2/home?region=eu-west-2#launchInstanceWizard:ami=ami-00aacd261ab72302e)
eu-west-1 | HVM - ECS enabled | [ami-0812b3f8aec8d2d81](https://eu-west-1.console.aws.amazon.com/ec2/home?region=eu-west-1#launchInstanceWizard:ami=ami-0812b3f8aec8d2d81)
ap-northeast-2 | HVM - ECS enabled | [ami-0d9d77df6579e618a](https://ap-northeast-2.console.aws.amazon.com/ec2/home?region=ap-northeast-2#launchInstanceWizard:ami=ami-0d9d77df6579e618a)
ap-northeast-1 | HVM - ECS enabled | [ami-09e957ac11ef430a3](https://ap-northeast-1.console.aws.amazon.com/ec2/home?region=ap-northeast-1#launchInstanceWizard:ami=ami-09e957ac11ef430a3)
sa-east-1 | HVM - ECS enabled | [ami-09c22f3ce89280ed4](https://sa-east-1.console.aws.amazon.com/ec2/home?region=sa-east-1#launchInstanceWizard:ami=ami-09c22f3ce89280ed4)
ca-central-1 | HVM - ECS enabled | [ami-016ac80225e649cf9](https://ca-central-1.console.aws.amazon.com/ec2/home?region=ca-central-1#launchInstanceWizard:ami=ami-016ac80225e649cf9)
ap-southeast-1 | HVM - ECS enabled | [ami-06cdfc80bdbd6f419](https://ap-southeast-1.console.aws.amazon.com/ec2/home?region=ap-southeast-1#launchInstanceWizard:ami=ami-06cdfc80bdbd6f419)
ap-southeast-2 | HVM - ECS enabled | [ami-0335f7bb1c51c0a74](https://ap-southeast-2.console.aws.amazon.com/ec2/home?region=ap-southeast-2#launchInstanceWizard:ami=ami-0335f7bb1c51c0a74)
eu-central-1 | HVM - ECS enabled | [ami-0af71ec7ee8b729be](https://eu-central-1.console.aws.amazon.com/ec2/home?region=eu-central-1#launchInstanceWizard:ami=ami-0af71ec7ee8b729be)
us-east-1 | HVM - ECS enabled | [ami-07209d7ec9e7545b4](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#launchInstanceWizard:ami=ami-07209d7ec9e7545b4)
us-east-2 | HVM - ECS enabled | [ami-046358fe356dd0e35](https://us-east-2.console.aws.amazon.com/ec2/home?region=us-east-2#launchInstanceWizard:ami=ami-046358fe356dd0e35)
us-west-1 | HVM - ECS enabled | [ami-031bcb65b47cb0a77](https://us-west-1.console.aws.amazon.com/ec2/home?region=us-west-1#launchInstanceWizard:ami=ami-031bcb65b47cb0a77)
us-west-2 | HVM - ECS enabled | [ami-0d92d296ecb13ea45](https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#launchInstanceWizard:ami=ami-0d92d296ecb13ea45)
cn-north-1 | HVM - ECS enabled | [ami-04f1668aaf990acf6](https://cn-north-1.console.amazonaws.cn/ec2/home?region=cn-north-1#launchInstanceWizard:ami=ami-04f1668aaf990acf6)
cn-northwest-1 | HVM - ECS enabled | [ami-0771f259ffce58280](https://cn-northwest-1.console.amazonaws.cn/ec2/home?region=cn-northwest-1#launchInstanceWizard:ami=ami-0771f259ffce58280)
@@ -49,7 +49,7 @@ On desktop systems the Syslinux boot menu can be switched to graphical mode by a
#### Recovery console
`rancher.recovery=true` will start a single user `root` bash session as easily in the boot process, with no network, or persitent filesystem mounted. This can be used to fix disk problems, or to debug your system.
`rancher.recovery=true` will start a single user `root` bash session as easily in the boot process, with no network, or persistent filesystem mounted. This can be used to fix disk problems, or to debug your system.
#### Enable/Disable sshd
@@ -61,7 +61,7 @@ On desktop systems the Syslinux boot menu can be switched to graphical mode by a
#### Autologin console
`rancher.autologin=<tty...>` will automatically log in the sepcified console - common values are `tty1`, `ttyS0` and `ttyAMA0` - depending on your platform.
`rancher.autologin=<tty...>` will automatically log in the specified console - common values are `tty1`, `ttyS0` and `ttyAMA0` - depending on your platform.
#### Enable/Disable hypervisor service auto-enable
@@ -119,7 +119,7 @@ $ ros config set rancher.system_docker.bip 172.19.0.0/16
_Available as of v1.4.x_
The default path of system-docker logs is `/var/log/system-docker.log`. If you want to write the system-docker logs to a separate partition,
The default path of system-docker logs is `/var/log/system-docker.log`. If you want to write the system-docker logs to a separate partition,
e.g. [RANCHER_OEM partition]({{< baseurl >}}/os/v1.x/en/about/custom-partition-layout/#use-rancher-oem-partition), you can try `rancher.defaults.system_docker_logs`:
```
@@ -170,11 +170,11 @@ Status: Downloaded newer image for alpine:latest
_Available as of v1.5.0_
When RancherOS is booted, you start with a User Docker service that is running in System Docker. With v1.5.0, RancherOS has the ability to create additional User Docker services that can run at the same time.
When RancherOS is booted, you start with a User Docker service that is running in System Docker. With v1.5.0, RancherOS has the ability to create additional User Docker services that can run at the same time.
#### Terminology
Throughout the rest of this documentation, we may simplify to use these terms when describing Docker.
Throughout the rest of this documentation, we may simplify to use these terms when describing Docker.
| Terminology | Definition |
|-----------------------|--------------------------------------------------|
@@ -184,13 +184,13 @@ Throughout the rest of this documentation, we may simplify to use these terms wh
#### Pre-Requisites
User Docker must be set as Docker 17.12.1 or earlier. If it's a later Docker version, it will produce errors when creating a user defined network in System Docker.
User Docker must be set as Docker 17.12.1 or earlier. If it's a later Docker version, it will produce errors when creating a user defined network in System Docker.
```
$ ros engine switch docker-17.12.1-ce
```
You will need to create a user-defined network, which will be used when creating the Other User Docker.
You will need to create a user-defined network, which will be used when creating the Other User Docker.
```
$ system-docker network create --subnet=172.20.0.0/16 dind
@@ -204,7 +204,7 @@ In order to create another User Docker, you will use `ros engine create`. Curren
$ ros engine create otheruserdockername --network=dind --fixed-ip=172.20.0.2
```
After the Other User Docker service is created, users can query this service like other services.
After the Other User Docker service is created, users can query this service like other services.
```
$ ros service list
@@ -215,13 +215,13 @@ disabled volume-nfs
enabled otheruserdockername
```
You can use `ros service up` to start the Other User Docker service.
You can use `ros service up` to start the Other User Docker service.
```
$ ros service up otheruserdockername
```
After the Other User Docker service is running, you can interact with it just like you can use the built-in User Docker. You would need to append `-<SERVICE_NAME>` to `docker`.
After the Other User Docker service is running, you can interact with it just like you can use the built-in User Docker. You would need to append `-<SERVICE_NAME>` to `docker`.
```
$ docker-otheruserdockername ps -a
@@ -229,7 +229,7 @@ $ docker-otheruserdockername ps -a
#### SSH into the Other User Docker container
When creating the Other User Docker, you can set an external SSH port so you can SSH into the Other User Docker container in System Docker. By using `--ssh-port` and adding ssh keys with `--authorized-keys`, you can set up this optional SSH port.
When creating the Other User Docker, you can set an external SSH port so you can SSH into the Other User Docker container in System Docker. By using `--ssh-port` and adding ssh keys with `--authorized-keys`, you can set up this optional SSH port.
```
$ ros engine create --help
@@ -248,7 +248,7 @@ When using `--authorized-keys`, you will need to put the key file in one of the
/home/
```
RancherOS will generate a random password for each Other User Docker container, which can be viewed in the container logs. If you do not set any SSH keys, the password can be used.
RancherOS will generate a random password for each Other User Docker container, which can be viewed in the container logs. If you do not set any SSH keys, the password can be used.
```
$ system-docker logs otheruserdockername
@@ -259,7 +259,7 @@ password: xCrw6fEG
======================================
```
In System Docker, you can SSH into any Other Uesr Docker Container using `ssh`.
In System Docker, you can SSH into any Other User Docker Container using `ssh`.
```
$ system-docker ps
@@ -274,7 +274,7 @@ $ ssh root@<OTHERUSERDOCKER_CONTAINER_IP>
#### Removing any Other User Docker Service
We recommend using `ros engine rm` to remove any Other User Docker service.
We recommend using `ros engine rm` to remove any Other User Docker service.
```
$ ros engine rm otheruserdockername
@@ -12,7 +12,7 @@ runcmd:
- echo "test" > /home/rancher/test2
```
Commands specified using `runcmd` will be executed within the context of the `console` container. More details on the ordering of commands run in the `console` container can be found [here]({{< baseurl >}}/os/v1.x/en/installation/boot-process/built-in-system-services/#console).
Commands specified using `runcmd` will be executed within the context of the `console` container.
### Running Docker commands
@@ -232,7 +232,7 @@ rancher:
scan_ssid: 1
```
When adding in WiFi access, you do not need a system reboot, you only need to restart the `network` service in System Docker.
When adding in WiFi access, you do not need a system reboot, you only need to restart the `network` service in System Docker.
```
$ sudo system-docker restart network
@@ -244,13 +244,13 @@ $ sudo system-docker restart network
_Available as of v1.5_
In order to support 4G-LTE, 4G-LTE module will need to be connected to the motherboard and to get a good signal, an external atenna will need to be added. You can assemble such a device, which supports USB interface and SIM cards slot:
In order to support 4G-LTE, 4G-LTE module will need to be connected to the motherboard and to get a good signal, an external antenna will need to be added. You can assemble such a device, which supports USB interface and SIM cards slot:
![](https://ws1.sinaimg.cn/bmiddle/006tNc79ly1fzcuvhu6zpj30k80qwag1.jpg)
In order to use RancherOS, you will need to use the ISO built for 4G-LTE support. This ISO has a built-in `modem-manager` service and is available with each release.
In order to use RancherOS, you will need to use the ISO built for 4G-LTE support. This ISO has a built-in `modem-manager` service and is available with each release.
After booting the ISO, there will be a 4G NIC, such as `wwan0`. Use the following `cloud-config` to set the APN parameter.
After booting the ISO, there will be a 4G NIC, such as `wwan0`. Use the following `cloud-config` to set the APN parameter.
```yaml
rancher:
@@ -266,4 +266,4 @@ After any configuration changes, restart the `modem-manager` service to apply th
$ sudo system-docker restart modem-manager
```
> **Note:** Currently, RancherOS has some built-in rules in `udev` rules to allow RancherOS to recognize specific 4G devices, but there are additional vendors that may be missing. If you need to add these in, please file an issue.
> **Note:** Currently, RancherOS has some built-in rules in `udev` rules to allow RancherOS to recognize specific 4G devices, but there are additional vendors that may be missing. If you need to add these in, please file an issue.
@@ -3,7 +3,7 @@ title: Digital Ocean
weight: 107
---
RancherOS is avaliable in the Digital Ocean portal. RancherOS is a member of container distributions and you can find it easily.
RancherOS is available in the Digital Ocean portal. RancherOS is a member of container distributions and you can find it easily.
>**Note**
>Deploying to Digital Ocean will incur charges.
+5 -1
View File
@@ -1,5 +1,9 @@
---
shortTitle: Rancher 2.x
title: "Rancher 2.x"
shortTitle: "Rancher 2.x"
description: "Rancher adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
metaTitle: "Rancher 2.x Docs: What is New?"
metaDescription: "Rancher 2 adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
insertOneSix: true
weight: 1
ctaBanner: intro-k8s-rancher-online-training
@@ -46,16 +46,16 @@ For more information, see [Provisioning Drivers]({{< baseurl >}}/rancher/v2.x/en
_Available as of v2.3.0_
With this feature, you can upgrade to the latest version of Kubernetes as soon as it is released, without upgrading Rancher. This feature allows you to easily upgrade Kubernetes patch versions (i.e. `v1.15.X`), but not intended to upgrade Kubernetes minor versions (i.e. `v1.X.0`) as Kubernetes tends to deprecate or add APIs between minor versions.
With this feature, you can upgrade to the latest version of Kubernetes as soon as it is released, without upgrading Rancher. This feature allows you to easily upgrade Kubernetes patch versions (i.e. `v1.15.X`), but not intended to upgrade Kubernetes minor versions (i.e. `v1.X.0`) as Kubernetes tends to deprecate or add APIs between minor versions.
The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) is now located in the Rancher Kubernetes Metadata. For details on metadata configuration and how to change the Kubernetes version used for provisioning RKE clusters, see [Rancher Kubernetes Metadata]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rke-metadata/).
The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) is now located in the Rancher Kubernetes Metadata. For details on metadata configuration and how to change the Kubernetes version used for provisioning RKE clusters, see [Rancher Kubernetes Metadata.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata/)
Rancher Kubernetes Metadata contains Kubernetes version information which Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata).
For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata/).
## Enabling Experimental Features
_Available as of v2.3.0_
Rancher includes some features that are experimental and disabled by default. Feature flags were introduced to allow you to try these features. For more information, refer to the section about [feature flags.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags)
Rancher includes some features that are experimental and disabled by default. Feature flags were introduced to allow you to try these features. For more information, refer to the section about [feature flags.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags/)
@@ -5,19 +5,19 @@ aliases:
- /rancher/v2.x/en/tasks/global-configuration/authentication/active-directory/
---
If your organization uses Microsoft Active Directory as central user repository, you can configure Rancher to communicate with an Active Directory server to authenticate users. This allows Rancher admins to control access to clusters and projects based on users and groups managed externally in the Active Directory, while allowing end-users to authenticate with their AD credentials when logging in to the Rancher UI.
If your organization uses Microsoft Active Directory as central user repository, you can configure Rancher to communicate with an Active Directory server to authenticate users. This allows Rancher admins to control access to clusters and projects based on users and groups managed externally in the Active Directory, while allowing end-users to authenticate with their AD credentials when logging in to the Rancher UI.
Rancher uses LDAP to communicate with the Active Directory server. The authentication flow for Active Directory is therefore the same as for the [OpenLDAP authentication]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/openldap) integration.
> **Note:**
>
>
> Before you start, please familiarise yourself with the concepts of [External Authentication Configuration and Principal Users]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/#external-authentication-configuration-and-principal-users).
## Prerequisites
You'll need to create or obtain from your AD administrator a new AD user to use as service account for Rancher. This user must have sufficient permissions to perform LDAP searches and read attributes of users and groups under your AD domain.
Usually a (non-admin) **Domain User** account should be used for this purpose, as by default such user has read-only privileges for most objects in the domain partition.
Usually a (non-admin) **Domain User** account should be used for this purpose, as by default such user has read-only privileges for most objects in the domain partition.
Note however, that in some locked-down Active Directory configurations this default behaviour may not apply. In such case you will need to ensure that the service account user has at least **Read** and **List Content** permissions granted either on the Base OU (enclosing users and groups) or globally for the domain.
@@ -126,7 +126,7 @@ Once you have completed the configuration, proceed by testing the connection to
## Annex: Identify Search Base and Schema using ldapsearch
In order to successfully configure AD authentication it is crucial that you provide the correct configuration pertaining to the hirarchy and schema of your AD server.
In order to successfully configure AD authentication it is crucial that you provide the correct configuration pertaining to the hierarchy and schema of your AD server.
The [`ldapsearch`](http://manpages.ubuntu.com/manpages/artful/man1/ldapsearch.1.html) tool allows you to query your AD server to learn about the schema used for user and group objects.
@@ -165,7 +165,7 @@ The output of the above `ldapsearch` query also allows to determine the correct
> **Note:**
>
> If the AD users in our organisation were to authenticate with their UPN (e.g. jdoe@acme.com) instead of the short logon name, then we would have to set the `Login Attribute` to **userPrincipalName** instead.
> If the AD users in our organisation were to authenticate with their UPN (e.g. jdoe@acme.com) instead of the short logon name, then we would have to set the `Login Attribute` to **userPrincipalName** instead.
We'll also set the `Search Attribute` parameter to **sAMAccountName|name**. That way users can be added to clusters/projects in the Rancher UI either by entering their username or full name.
@@ -15,7 +15,7 @@ Within Rancher, only administrators or users with the **Manage Authentication**
- You must have the Admin SDK API enabled for your G Suite domain. You can enable it using the steps on [this page.](https://support.google.com/a/answer/60757?hl=en)
After the Admin SDK API is enabled, your G Suite domain's API screen should look like this:
![Enable Admin APIs]({{<baseurl>}}/img/rancher/Google-Enable-APIs-Screen.png)
![Enable Admin APIs]({{<baseurl>}}/img/rancher/Google-Enable-APIs-Screen.png)
# Setting up G Suite for OAuth with Rancher
Before you can set up Google OAuth in Rancher, you need to log in to your G Suite account and do the following:
@@ -27,8 +27,8 @@ Before you can set up Google OAuth in Rancher, you need to log in to your G Suit
### 1. Adding Rancher as an Authorized Domain
1. Click [here](https://console.developers.google.com/apis/credentials) to go to credentials page of your Google domain.
1. Select your project and click **OAuth consent screen.**
![OAuth Consent Screen]({{<baseurl>}}/img/rancher/Google-OAuth-consent-screen-tab.png)
1. Select your project and click **OAuth consent screen.**
![OAuth Consent Screen]({{<baseurl>}}/img/rancher/Google-OAuth-consent-screen-tab.png)
1. Go to **Authorized Domains** and enter the top private domain of your Rancher server URL in the list. The top private domain is the rightmost superdomain. So for example, www.foo.co.uk a top private domain of foo.co.uk. For more information on top-level domains, refer to [this article.](https://github.com/google/guava/wiki/InternetDomainNameExplained#public-suffixes-and-private-domains)
1. Go to **Scopes for Google APIs** and make sure **email,** **profile** and **openid** are enabled.
@@ -36,14 +36,14 @@ Before you can set up Google OAuth in Rancher, you need to log in to your G Suit
### 2. Creating OAuth2 Credentials for the Rancher Server
1. Go to the Google API console, select your project, and go to the [credentials page.](https://console.developers.google.com/apis/credentials)
![Credentials]({{<baseurl>}}/img/rancher/Google-Credentials-tab.png)
1. On the **Create Credentials** dropdown, select **OAuth client ID.**
![Credentials]({{<baseurl>}}/img/rancher/Google-Credentials-tab.png)
1. On the **Create Credentials** dropdown, select **OAuth client ID.**
1. Click **Web application.**
1. Provide a name.
1. Fill out the **Authorized JavaScript origins** and **Authorized redirect URIs.** Note: The Rancher UI page for setting up Google OAuth (available from the Global view under **Security > Authentication > Google**) provides you the exact links to enter for this step.
- Under **Authorized JavaScript origins,** enter your Rancher server URL.
- Under **Authorized redirect URIs,** enter your Rancher server URL appended with the path `verify-auth`. For example, if your URI is `https://rancherServer`, you will enter `https://rancherServer/verify-auth`.
1. Click on **Create.**
1. Provide a name.
1. Fill out the **Authorized JavaScript origins** and **Authorized redirect URIs.** Note: The Rancher UI page for setting up Google OAuth (available from the Global view under **Security > Authentication > Google**) provides you the exact links to enter for this step.
- Under **Authorized JavaScript origins,** enter your Rancher server URL.
- Under **Authorized redirect URIs,** enter your Rancher server URL appended with the path `verify-auth`. For example, if your URI is `https://rancherServer`, you will enter `https://rancherServer/verify-auth`.
1. Click on **Create.**
1. After the credential is created, you will see a screen with a list of your credentials. Choose the credential you just created, and in that row on rightmost side, click **Download JSON.** Save the file so that you can provide these credentials to Rancher.
**Result:** Your OAuth credentials have been successfully created.
@@ -58,16 +58,16 @@ As a workaround to get this capability, G Suite recommends creating a service ac
This section describes how to:
- Create a service account
- Create a key for the service account and download the credenials as JSON
- Create a key for the service account and download the credentials as JSON
1. Click [here](https://console.developers.google.com/iam-admin/serviceaccounts) and select your project for which you generated OAuth credentials.
1. Click on **Create Service Account.**
1. Enter a name and click **Create.**
![Service account creation Step 1]({{<baseurl>}}/img/rancher/Google-svc-acc-step1.png)
1. Don't provide any roles on the **Service account permissions** page and click **Continue**
![Service account creation Step 2]({{<baseurl>}}/img/rancher/Google-svc-acc-step2.png)
1. Click on **Create Service Account.**
1. Enter a name and click **Create.**
![Service account creation Step 1]({{<baseurl>}}/img/rancher/Google-svc-acc-step1.png)
1. Don't provide any roles on the **Service account permissions** page and click **Continue**
![Service account creation Step 2]({{<baseurl>}}/img/rancher/Google-svc-acc-step2.png)
1. Click on **Create Key** and select the JSON option. Download the JSON file and save it so that you can provide it as the service account credentials to Rancher.
![Service account creation Step 3]({{<baseurl>}}/img/rancher/Google-svc-acc-step3-key-creation.png)
![Service account creation Step 3]({{<baseurl>}}/img/rancher/Google-svc-acc-step3-key-creation.png)
**Result:** Your service account is created.
@@ -79,13 +79,13 @@ Using the Unique ID of the service account key, register it as an Oauth Client u
1. Get the Unique ID of the key you just created. If it's not displayed in the list of keys right next to the one you created, you will have to enable it. To enable it, click **Unique ID** and click **OK.** This will add a **Unique ID** column to the list of service account keys. Save the one listed for the service account you created. NOTE: This is a numeric key, not to be confused with the alphanumeric field **Key ID.**
![Service account Unique ID]({{<baseurl>}}/img/rancher/Google-Select-UniqueID-column.png)
1. Go to the [**Manage OAuth Client Access** page.](https://admin.google.com/AdminHome?chromeless=1#OGX:ManageOauthClients)
![Service account Unique ID]({{<baseurl>}}/img/rancher/Google-Select-UniqueID-column.png)
1. Go to the [**Manage OAuth Client Access** page.](https://admin.google.com/AdminHome?chromeless=1#OGX:ManageOauthClients)
1. Add the Unique ID obtained in the previous step in the **Client Name** field.
1. In the **One or More API Scopes** field, add the following scopes:
1. In the **One or More API Scopes** field, add the following scopes:
```
openid,profile,email,https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.group.readonly
```
```
1. Click **Authorize.**
**Result:** The service account is registered as an OAuth client in your G Suite account.
@@ -101,6 +101,6 @@ Using the Unique ID of the service account key, register it as an Oauth Client u
- For **Step Two,** provide the OAuth credentials JSON that you downloaded after completing [this section.](#2-creating-oauth2-credentials-for-the-rancher-server) You can upload the file or paste the contents into the **OAuth Credentials** field.
- For **Step Three,** provide the service account credentials JSON that downloaded at the end of [this section.](#3-creating-service-account-credentials) The credentials will only work if you successfully [registered the service account key](#4-register-the-service-account-key-as-an-oauth-client) as an OAuth client in your G Suite account.
1. Click **Authenticate with Google**.
1. Click **Save**.
1. Click **Save**.
**Result:** Google authentication is successfully configured.
@@ -1,5 +1,6 @@
---
title: Configuring Keycloak (SAML)
description: Create a Keycloak SAML client and configure Rancher to work with Keycloak. By the end your users will be able to sign into Rancher using their Keycloak logins
weight: 1200
---
_Available as of v2.1.0_
@@ -11,7 +12,7 @@ If your organization uses Keycloak Identity Provider (IdP) for user authenticati
- You must have a [Keycloak IdP Server](https://www.keycloak.org/docs/latest/server_installation/) configured.
- In Keycloak, create a [new SAML client](https://www.keycloak.org/docs/latest/server_admin/#saml-clients), with the settings below. See the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/#saml-clients) for help.
Setting | Value
Setting | Value
------------|------------
`Sign Documents` | `ON` <sup>1</sup>
`Sign Assertions` | `ON` <sup>1</sup>
@@ -22,7 +23,7 @@ If your organization uses Keycloak Identity Provider (IdP) for user authenticati
`Valid Redirect URI` | `https://yourRancherHostURL/v1-saml/keycloak/saml/acs`
><sup>1</sup>: Optionally, you can enable either one or both of these settings.
- Export a `metadata.xml` file from your Keycloak client:
- Export a `metadata.xml` file from your Keycloak client:
From the `Installation` tab, choose the `SAML Metadata IDPSSODescriptor` format option and download your file.
@@ -47,7 +48,7 @@ If your organization uses Keycloak Identity Provider (IdP) for user authenticati
| IDP-metadata | The `metadata.xml` file that you exported from your IdP server. |
>**Tip:** You can generate a key/certificate pair using an openssl command. For example:
>
>
> openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout myservice.key -out myservice.cert
@@ -63,7 +64,7 @@ If your organization uses Keycloak Identity Provider (IdP) for user authenticati
## Annex: Troubleshooting
If you are experiencing issues while testing the connection to the Keycloak server, first double-check the confiuration option of your SAML client. You may also inspect the Rancher logs to help pinpointing the problem cause. Debug logs may contain more detailed information about the error. Please refer to [How can I enable debug logging]({{< baseurl >}}/rancher/v2.x/en/faq/technical/#how-can-i-enable-debug-logging) in this documentation.
If you are experiencing issues while testing the connection to the Keycloak server, first double-check the configuration option of your SAML client. You may also inspect the Rancher logs to help pinpointing the problem cause. Debug logs may contain more detailed information about the error. Please refer to [How can I enable debug logging]({{< baseurl >}}/rancher/v2.x/en/faq/technical/#how-can-i-enable-debug-logging) in this documentation.
### You are not redirected to Keycloak
@@ -89,10 +90,10 @@ You are correctly redirected to your IdP login page and you are able to enter yo
* Check your Keycloak log.
* If the log displays `request validation failed: org.keycloak.common.VerificationException: SigAlg was null`, set `Client Signature Required` to `OFF` in your Keycloak client.
### Keycloak 6.0.0+: IDPSSODescriptor missing from options
Keycloak versions 6.0.0 and up no longer provide the IDP metadata under the `Installation` tab.
Keycloak versions 6.0.0 and up no longer provide the IDP metadata under the `Installation` tab.
You can still get the XML from the following url:
`https://{KEYCLOAK-URL}/auth/realms/{REALM-NAME}/protocol/saml/descriptor`
@@ -111,4 +112,3 @@ You are left with something similar as the example below:
</EntityDescriptor>
```
@@ -9,9 +9,9 @@ If your organization uses Ping Identity Provider (IdP) for user authentication,
>**Prerequisites:**
>
>- You must have a [Ping IdP Server](https://www.pingidentity.com/) configured.
>- Following are the Rancher Service Provider URLs needed for configuration:
Metadata URL: `https://<rancher-server>/v1-saml/ping/saml/metadata`
Assertion Consure Service (ACS) URL: `https://<rancher-server>/v1-saml/ping/saml/acs`
>- Following are the Rancher Service Provider URLs needed for configuration:
Metadata URL: `https://<rancher-server>/v1-saml/ping/saml/metadata`
Assertion Consumer Service (ACS) URL: `https://<rancher-server>/v1-saml/ping/saml/acs`
>- Export a `metadata.xml` file from your IdP Server. For more information, see the [PingIdentity documentation](https://documentation.pingidentity.com/pingfederate/pf83/index.shtml#concept_exportingMetadata.html).
1. From the **Global** view, select **Security > Authentication** from the main menu.
@@ -29,7 +29,7 @@ Rancher will periodically refresh the user information even before a user logs i
- **`auth-user-info-max-age-seconds`**
This setting controls how old a user's information can be before Rancher refreshes it. If a user makes an API call (either directly or by using the Rancher CLI or kubectl) and the time since the user's last refresh is greater than this setting, then Rancher will trigger a refresh. This settting defaults to `3600` seconds, i.e. 1 hour.
This setting controls how old a user's information can be before Rancher refreshes it. If a user makes an API call (either directly or by using the Rancher CLI or kubectl) and the time since the user's last refresh is greater than this setting, then Rancher will trigger a refresh. This setting defaults to `3600` seconds, i.e. 1 hour.
- **`auth-user-info-resync-cron`**
@@ -89,7 +89,7 @@ _Project roles_ are roles that can be used to grant users access to a project. T
These users can view everything in the project but cannot create, update, or delete anything.
>**Caveat:**
>
>
>Users assigned the `Owner` or `Member` role for a project automatically inherit the `namespace creation` role. However, this role is a [Kubernetes ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole), meaning its scope extends to all projects in the cluster. Therefore, users explicitly assigned the `owner` or `member` role for a project can create namespaces in other projects they're assigned to, even with only the `Read Only` role assigned.
@@ -126,7 +126,7 @@ The following table lists each built-in custom project role available in Rancher
> **Notes:**
>
>- Each project role listed above, including `Owner`, `Member`, and `Read Only`, is comprised of multiple rules granting access to various resources. You can view the roles and their rules on the Global > Security > Roles page.
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
>- The `Manage Project Members` role allows the project owner to manage any members of the project **and** grant them any project scoped role regardless of their access to the project resources. Be cautious when assigning this role out individually.
### Defining Custom Roles
@@ -161,12 +161,12 @@ You can change the cluster or project role(s) that are automatically assigned to
1. Enable the role as default.
{{% accordion id="cluster" label="For Clusters" %}}
1. From **Clustor Creator Default**, choose **Yes: Default role for new cluster creation**.
1. From **Cluster Creator Default**, choose **Yes: Default role for new cluster creation**.
1. Click **Save**.
{{% /accordion %}}
{{% accordion id="project" label="For Projects" %}}
1. From **Project Creator Default**, choose **Yes: Default role for new project creation**.
1. Click **Save**.
1. Click **Save**.
{{% /accordion %}}
1. If you want to remove a default role, edit the permission and select **No** from the default roles option.
@@ -181,4 +181,3 @@ When you revoke the cluster membership for a standard user that's explicitly ass
- Exercise any [individual project roles](#project-role-reference) they are assigned.
If you want to completely revoke a user's access within a cluster, revoke both their cluster and project memberships.
@@ -2,70 +2,166 @@
title: Custom Roles
weight: 1128
aliases:
- /rancher/v2.x/en/tasks/global-configuration/roles/
- /rancher/v2.x/en/tasks/global-configuration/roles/
---
Within Rancher, _roles_ determine what actions a user can make within a cluster or project.
Note that _roles_ are different from _permissions_, which determine what clusters and projects you can access.
>**Prerequisites:**
>
>To complete the tasks on this page, the following permissions are required:
>
>- [Administrator Global Permissions]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/).
>- [Custom Global Permissions]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/#custom-global-permissions) with the [Manage Roles]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/#global-permissions-reference) role assigned.
This section covers the following topics:
## Adding A Custom Role
- [Prerequisites](#prerequisites)
- [Creating a custom role for a cluster or project](#creating-a-custom-role-for-a-cluster-or-project)
- [Creating a custom global role that copies rules from an existing role](#creating-a-custom-global-role-that-copies-rules-from-an-existing-role)
- [Creating a custom global role that does not copy rules from another role](#creating-a-custom-global-role-that-does-not-copy-rules-from-another-role)
- [Deleting a custom global role](#deleting-a-custom-global-role)
- [Assigning a custom global role to a group](#assigning-a-custom-global-role-to-a-group)
## Prerequisites
To complete the tasks on this page, one of the following permissions are required:
- [Administrator Global Permissions]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/).
- [Custom Global Permissions]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/#custom-global-permissions) with the [Manage Roles]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/#global-permissions-reference) role assigned.
## Creating A Custom Role for a Cluster or Project
While Rancher comes out-of-the-box with a set of default user roles, you can also create default custom roles to provide users with very specific permissions within Rancher.
1. From the **Global** view, select **Security > Roles** from the main menu.
The steps to add custom roles differ depending on the version of Rancher.
1. **v2.0.7 and later only:** Select a tab to determine the scope of the roles you're adding. The tabs are:
{{% tabs %}}
{{% tab "Rancher v2.0.7+" %}}
- **Cluster**
1. From the **Global** view, select **Security > Roles** from the main menu.
The role is valid for assignment when adding/managing members to _only_ clusters.
1. Select a tab to determine the scope of the roles you're adding. The tabs are:
- **Project**
- **Cluster:** The role is valid for assignment when adding/managing members to _only_ clusters.
- **Project:** The role is valid for assignment when adding/managing members to _only_ projects.
The role is valid for assignment when adding/managing members to _only_ projects.
1. Click **Add Cluster/Project Role.**
>**Note:** You cannot edit the Global tab.
1. **Name** the role.
1. Click **Add Cluster/Project Role**.
1. Optional: Choose the **Cluster/Project Creator Default** option to assign this role to a user when they create a new cluster or project. Using this feature, you can expand or restrict the default roles for cluster/project creators.
1. **Name** the role.
> Out of the box, the Cluster Creator Default and the Project Creator Default roles are `Cluster Owner` and `Project Owner` respectively.
1. Choose whether to set the role to a status of [locked]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles/).
1. Use the **Grant Resources** options to assign individual [Kubernetes API endpoints](https://kubernetes.io/docs/reference/) to the role.
Locked roles cannot be assigned to users.
> When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
1. **v2.0.7 and later only:** Choose a **Cluster/Project Creator Default** option setting. Use this option to set if the role is assigned to a user when they create a new cluster or project. Using this feature, you can expand or restrict the default roles for cluster/project creators.
You can also choose the individual cURL methods (`Create`, `Delete`, `Get`, etc.) available for use with each endpoint you assign.
>**Note:** Out of the box, the Cluster Creator Default and the Project Creator Default roles are `Cluster Owner` and `Project Owner` respectively.
1. Use the **Inherit from a Role** options to assign individual Rancher roles to your custom roles. Note: When a custom role inherits from a parent role, the parent role cannot be deleted until the child role is deleted.
1. **v2.0.6 and earlier only:** Assign the role a **Context**. Context determines the scope of role assigned to the user. The contexts are:
1. Click **Create**.
- **All**
{{% /tab %}}
{{% tab "Rancher prior to v2.0.7" %}}
The user can use their assigned role regardless of context. This role is valid for assignment when adding/managing members to clusters or projects.
1. From the **Global** view, select **Security > Roles** from the main menu.
- **Cluster**
1. Click **Add Role**.
This role is valid for assignment when adding/managing members to _only_ clusters.
1. **Name** the role.
- **Project**
1. Choose whether to set the role to a status of [locked]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles/).
This role is valid for assignment when adding/managing members to _only_ projects.
> **Note:** Locked roles cannot be assigned to users.
6. Use the **Grant Resources** options to assign individual [Kubernetes API endpoints](https://kubernetes.io/docs/reference/) to the role.
1. In the **Context** dropdown menu, choose the scope of the role assigned to the user. The contexts are:
>**Note:** When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
- **All:** The user can use their assigned role regardless of context. This role is valid for assignment when adding/managing members to clusters or projects.
You can also choose the individual cURL methods (`Create`, `Delete`, `Get`, etc.) available for use with each endpoint you assign.
- **Cluster:** This role is valid for assignment when adding/managing members to _only_ clusters.
7. Use the **Inherit from a Role** options to assign individual Rancher roles to your custom roles.
- **Project:** This role is valid for assignment when adding/managing members to _only_ projects.
8. Click **Create**.
1. Use the **Grant Resources** options to assign individual [Kubernetes API endpoints](https://kubernetes.io/docs/reference/) to the role.
> When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
You can also choose the individual cURL methods (`Create`, `Delete`, `Get`, etc.) available for use with each endpoint you assign.
1. Use the **Inherit from a Role** options to assign individual Rancher roles to your custom roles. Note: When a custom role inherits from a parent role, the parent role cannot be deleted until the child role is deleted.
1. Click **Create**.
{{% /tab %}}
{{% /tabs %}}
## Creating a Custom Global Role that Copies Rules from an Existing Role
_Available as of v2.4_
If you have a group of individuals that need the same level of access in Rancher, it can save time to create a custom global role in which all of the rules from another role, such as the administrator role, are copied into a new role. This allows you to only configure the variations between the existing role and the new role.
The custom global role can then be assigned to a user or group so that the custom global role takes effect the first time the user or users sign into Rancher.
To create a custom global role based on an existing role,
1. Go to the **Global** view and click **Security > Roles.**
1. On the **Global** tab, go to the role that the custom global role will be based on. Click **Ellipsis (…) > Clone.**
1. Enter a name for the role.
1. Optional: To assign the custom role default for new users, go to the **New User Default** section and click **Yes: Default role for new users.**
1. In the **Grant Resources** section, select the Kubernetes resource operations that will be enabled for users with the custom role.
1. Click **Save.**
## Creating a Custom Global Role that Does Not Copy Rules from Another Role
_Available as of v2.4_
Custom global roles don't have to be based on existing roles. To create a custom global role by choosing the specific Kubernetes resource operations that should be allowed for the role, follow these steps:
1. Go to the **Global** view and click **Security > Roles.**
1. On the **Global** tab, click **Add Global Role.**
1. Enter a name for the role.
1. Optional: To assign the custom role default for new users, go to the **New User Default** section and click **Yes: Default role for new users.**
1. In the **Grant Resources** section, select the Kubernetes resource operations that will be enabled for users with the custom role.
1. Click **Save.**
## Deleting a Custom Global Role
_Available as of v2.4_
When deleting a custom global role, all global role bindings with this custom role are deleted.
If a user is only assigned one custom global role, and the role is deleted, the user would lose access to Rancher. For the user to regain access, an administrator would need to edit the user and apply new global permissions.
Custom global roles can be deleted, but built-in roles cannot be deleted.
To delete a custom global role,
1. Go to the **Global** view and click **Security > Roles.**
2. On the **Global** tab, go to the custom global role that should be deleted and click **Ellipsis (…) > Delete.**
3. Click **Delete.**
## Assigning a Custom Global Role to a Group
_Available as of v2.4_
If you have a group of individuals that need the same level of access in Rancher, it can save time to create a custom global role. When the role is assigned to a group, the users in the group have the appropriate level of access the first time they sign into Rancher.
When a user in the group logs in, they get the built-in Standard User global role by default. They will also get the permissions assigned to their groups.
If a user is removed from the external authentication provider group, they would lose their permissions from the custom global role that was assigned to the group. They would continue to have their individual Standard User role.
> **Prerequisites:** You can only assign a global role to a group if:
>
> * You have set up an [external authentication provider]({{<baseurl>}}/rancher/v2.x/en/admin-settings/authentication/#external-vs-local-authentication)
> * The external authentication provider supports [user groups]({{<baseurl>}}/rancher/v2.x/en/admin-settings/authentication/user-groups/)
> * You have already set up at least one user group with the authentication provider
To assign a custom global role to a group, follow these steps:
1. From the **Global** view, go to **Security > Groups.**
1. Click **Assign Global Role.**
1. In the **Select Group To Add** field, choose the existing group that will be assigned the custom global role.
1. In the **Custom** section, choose any custom global role that will be assigned to the group.
1. Optional: In the **Global Permissions** or **Built-in** sections, select any additional permissions that the group should have.
1. Click **Create.**
**Result:** The custom global role will take effect when the users in the group log into Rancher.
@@ -7,43 +7,59 @@ _Permissions_ are individual access rights that you can assign when selecting a
Global Permissions define user authorization outside the scope of any particular cluster. Out-of-the-box, there are two default global permissions: `Administrator` and `Standard User`.
- **Administrator:**
- **Administrator:** These users have full control over the entire Rancher system and all clusters within it.
These users have full control over the entire Rancher system and all clusters within it.
- <a id="user"></a>**Standard User:** These users can create new clusters and use them. Standard users can also assign other users permissions to their clusters.
- <a id="user"></a>**Standard User:**
You cannot update or delete the built-in Global Permissions.
These users can create new clusters and use them. Standard users can also assign other users permissions to their clusters.
This section covers the following topics:
>**Note:** You cannot create, update, or delete Global Permissions.
- [Global permission assignment](#global-permission-assignment)
- [Global permissions for new local users](#global-permissions-for-new-local-users)
- [Global permissions for users with external authentication](#global-permissions-for-users-with-external-authentication)
- [Custom global permissions](#custom-global-permissions)
- [Custom global permissions reference](#custom-global-permissions-reference)
- [Configuring default global permissions for new users](#configuring-default-global-permissions)
- [Configuring global permissions for existing individual users](#configuring-global-permissions-for-existing-individual-users)
- [Configuring global permissions for groups](#configuring-global-permissions-for-groups)
- [Refreshing group memberships](#refreshing-group-memberships)
# Global Permission Assignment
Assignment of global permissions to a user depends on their authentication source: external or local.
Global permissions for local users are assigned differently than users who log in to Rancher using external authentication.
- **External Authentication**
### Global Permissions for New Local Users
When a user logs into Rancher using an external authentication provider for the first time, they are automatically assigned the `Standard User` global permission.
When you create a new local user, you assign them a global permission as you complete the **Add User** form.
- **Local Authentication**
To see the default permissions for new users, go to the **Global** view and click **Security > Roles.** On the **Global** tab, there is a column named **New User Default.** When adding a new local user, the user receives all default global permissions that are marked as checked in this column. You can [change the default global permissions to meet your needs.](#configuring-default-global-permissions)
When you create a new local user, you assign them a global permission as you complete the **Add User** form.
### Global Permissions for Users with External Authentication
When a user logs into Rancher using an external authentication provider for the first time, they are automatically assigned the **New User Default** global permissions. By default, Rancher assigns the **Standard User** permission for new users.
To see the default permissions for new users, go to the **Global** view and click **Security > Roles.** On the **Global** tab, there is a column named **New User Default.** When adding a new local user, the user receives all default global permissions that are marked as checked in this column, and you can [change them to meet your needs.](#configuring-default-global-permissions)
Permissions can be assigned to an individual user with [these steps.](#configuring-global-permissions-for-existing-individual-users)
As of Rancher v2.4, you can [assign a role to everyone in the group at the same time](#configuring-global-permissions-for-groups) if the external authentication provider supports groups.
# Custom Global Permissions
Using custom permissions is convenient for providing users with narrow or specialized access to Rancher.
When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, after a user logs in from the first time, they are created as a user and assigned the default `user` permission. The standard `user` permission allows users to login and create clusters.
When a user from an [external authentication source]({{<baseurl>}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, after a user logs in for the first time, they are created as a user and assigned the default `user` permission. The standard `user` permission allows users to login and create clusters.
However, in some organizations, these permissions may extend too much access. Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a more restrictive set of custom global permissions.
The default roles, Administrator and Standard User, each come with multiple global permissions built into them. The Administrator role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role.
Administrators can enforce custom global permissions in two ways:
Administrators can enforce custom global permissions in multiple ways:
- Changing the [default permissions for new users](#configuring-default-global-permissions)
- Editing the [permissions of an existing user](#configuring-global-permissions-for-individual-users)
- [Changing the default permissions for new users](#configuring-default-global-permissions)
- [Editing the permissions of an existing user](#configuring-global-permissions-for-individual-users)
- [Assigning a custom global permission to a group](#assigning-a-custom-global-permission-to-a-group)
### Custom Global Permissions Reference
@@ -62,25 +78,25 @@ The following table lists each custom global permission available and whether it
| Manage Settings | ✓ | |
| Manage Users | ✓ | |
| Use Catalog Templates | ✓ | ✓ |
| User Base* (Basic log-in access) | ✓ | ✓ |
| User Base\* (Basic log-in access) | ✓ | ✓ |
> *This role has two names:
> \*This role has two names:
>
> - When you go to the <b>Users</b> tab and edit a user's global role, this role is called <b>Login Access</b> in the custom global permissions list.
> - When you go to the <b>Security</b> tab and edit the roles from the roles page, this role is called <b>User Base.</b>
For details on which Kubernetes resources correspond to each global permission, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Global** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role.
> **Notes:**
> **Notes:**
>
>- Each permission listed above is comprised of multiple individual permissions not listed in the Rancher UI. For a full list of these permissions and the rules they are comprised of, access through the API at `/v3/globalRoles`.
>- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
> - Each permission listed above is comprised of multiple individual permissions not listed in the Rancher UI. For a full list of these permissions and the rules they are comprised of, access through the API at `/v3/globalRoles`.
> - When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource.
### Configuring Default Global Permissions
If you want to restrict the default permissions for new users, you can remove the `user` permission as default role and then assign multiple individual permissions as default instead. Conversely, you can also add administrative permissions on top of a set of other standard permissions.
>**Note:** Default roles are only assigned to users added from an external authentication provider. For local users, you must explicitly assign global permissions when adding a user to Rancher. You can customize these global permissions when adding the user.
> **Note:** Default roles are only assigned to users added from an external authentication provider. For local users, you must explicitly assign global permissions when adding a user to Rancher. You can customize these global permissions when adding the user.
To change the default global permissions that are assigned to external users upon their first log in, follow these steps:
@@ -94,7 +110,7 @@ To change the default global permissions that are assigned to external users upo
**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column.
### Configuring Global Permissions for Individual Users
### Configuring Global Permissions for Existing Individual Users
To configure permission for a user,
@@ -109,3 +125,48 @@ To configure permission for a user,
1. Click **Save.**
> **Result:** The user's global permissions have been updated.
### Configuring Global Permissions for Groups
_Available as of v2.4_
If you have a group of individuals that need the same level of access in Rancher, it can save time to assign permissions to the entire group at once, so that the users in the group have the appropriate level of access the first time they sign into Rancher.
After you assign a custom global role to a group, the custom global role will be assigned to a user in the group when they log in to Rancher.
For existing users, the new permissions will take effect when the users log out of Rancher and back in again, or when an administrator [refreshes the group memberships.](#refreshing-group-memberships)
For new users, the new permissions take effect when the users log in to Rancher for the first time. New users from this group will receive the permissions from the custom global role in addition to the **New User Default** global permissions. By default, the **New User Default** permissions are equivalent to the **Standard User** global role, but the default permissions can be [configured.](#configuring-default-global-permissions)
If a user is removed from the external authentication provider group, they would lose their permissions from the custom global role that was assigned to the group. They would continue to have any remaining roles that were assigned to them, which would typically include the roles marked as **New User Default.** Rancher will remove the permissions that are associated with the group when the user logs out, or when an administrator [refreshes group memberships,]((#refreshing-group-memberships)) whichever comes first.
> **Prerequisites:** You can only assign a global role to a group if:
>
> * You have set up an [external authentication provider]({{<baseurl>}}/rancher/v2.x/en/admin-settings/authentication/#external-vs-local-authentication)
> * The external authentication provider supports [user groups]({{<baseurl>}}/rancher/v2.x/en/admin-settings/authentication/user-groups/)
> * You have already set up at least one user group with the authentication provider
To assign a custom global role to a group, follow these steps:
1. From the **Global** view, go to **Security > Groups.**
1. Click **Assign Global Role.**
1. In the **Select Group To Add** field, choose the existing group that will be assigned the custom global role.
1. In the **Global Permissions,** **Custom,** and/or **Built-in** sections, select the permissions that the group should have.
1. Click **Create.**
**Result:** The custom global role will take effect when the users in the group log into Rancher.
### Refreshing Group Memberships
When an administrator updates the global permissions for a group, the changes take effect for individual group members after they log out of Rancher and log in again.
To make the changes take effect immediately, an administrator or cluster owner can refresh group memberships.
An administrator might also want to refresh group memberships if a user is removed from a group in the external authentication service. In that case, the refresh makes Rancher aware that the user was removed from the group.
To refresh group memberships,
1. From the **Global** view, click **Security > Users.**
1. Click **Refresh Group Memberships.**
**Result:** Any changes to the group members' permissions will take effect.
@@ -47,7 +47,7 @@ The [add-on section](#add-ons) of an RKE template is especially powerful because
# Scope of RKE Templates
RKE templates are supported for Rancher-provisioned clusters. The templates can be used to provision custom clusters or clusters that are launched by an infrastructure provider.
RKE templates are for defining Kubernetes and Rancher settings. Node templates are responsible for configuring nodes. For tips on how to use RKE templates in conjunction with hardware, refer to [RKE Templates and Hardware]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware).
RKE templates can be created from scratch to pre-define cluster configuration. They can be applied to launch new clusters, or templates can also be exported from existing running clusters.
@@ -102,7 +102,7 @@ As of Rancher v2.3.3, you can [save the configuration of an existing cluster as
# Standardizing Hardware
RKE templates are designed to standardize Kubernetes and Rancher settings. If you want to standardize your infrastructure as well, you use RKE templates [in conjuction with other tools]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware).
RKE templates are designed to standardize Kubernetes and Rancher settings. If you want to standardize your infrastructure as well, you use RKE templates [in conjunction with other tools]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware).
# YAML Customization
@@ -15,11 +15,11 @@ These example scenarios describe how an organization could use templates to stan
Let's say there is an organization in which the administrators decide that all new clusters should be created with Kubernetes version 1.14.
1. First, an administrator creates a template which specifies the Kubernetes version as 1.14 and marks all other settings as **Allow User Override**.
1. First, an administrator creates a template which specifies the Kubernetes version as 1.14 and marks all other settings as **Allow User Override**.
1. The administrator makes the template public.
1. The administrator turns on template enforcement.
**Results:**
**Results:**
- All Rancher users in the organization have access to the template.
- All new clusters created by [standard users]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) with this template will use Kubernetes 1.14 and they are unable to use a different Kubernetes version. By default, standard users don't have permission to create templates, so this template will be the only template they can use unless more templates are shared with them.
@@ -29,10 +29,10 @@ In this way, the administrators enforce the Kubernetes version across the organi
# Templates for Basic and Advanced Users
Let's say an organization has both basic and advanced users. Adminstrators want the basic users to be required to use a template, while the advanced users and administrators create their clusters however they want.
Let's say an organization has both basic and advanced users. Administrators want the basic users to be required to use a template, while the advanced users and administrators create their clusters however they want.
1. First, an administrator turns on [RKE template enforcement.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rke-templates/enforcement/#requiring-new-clusters-to-use-a-cluster-template) This means that every [standard user]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) in Rancher will need to use an RKE template when they create a cluster.
1. The administrator then creates two templates:
1. The administrator then creates two templates:
- One template for basic users, with almost every option specified except for access keys
- One template for advanced users, which has most or all options has **Allow User Override** turned on
@@ -44,7 +44,7 @@ Let's say an organization has both basic and advanced users. Adminstrators want
# Updating Templates and Clusters Created with Them
Let's say an organization has a template that requires clusters to use Kubernetes v1.14. However, as time goes on, the adminstrators change their minds. They decide they want users to be able to upgrade their clusters to use newer versions of Kubernetes.
Let's say an organization has a template that requires clusters to use Kubernetes v1.14. However, as time goes on, the administrators change their minds. They decide they want users to be able to upgrade their clusters to use newer versions of Kubernetes.
In this organization, many clusters were created with a template that requires Kubernetes v1.14. Because the template does not allow that setting to be overridden, the users who created the cluster cannot directly edit that setting.
@@ -32,11 +32,11 @@ When possible, use a non-privileged user when running processes within your cont
### Define Resource Limits
Apply CPU and memory limits to your pods. This can help manage the resources on your worker nodes and avoid a malfunctioning microservice from impacting other microservices.
In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the [Rancher docs]({{<baseurl>}}/rancher/v2.x/en/project-admin/resource-quotas/).
In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the Rancher docs.
When setting resource quotas, if you set anything related to CPU or Memory (i.e. limits or reservations) on a project or namespace, all containers will require a respective CPU or Memory field set during creation. To avoid setting these limits on each and every container during workload creation, a default container resource limit can be specified on the namespace.
The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the [namespace level](https://kubernetes.io/docs/concepts/policy/resource-quotas/).
The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the namespace level.
### Define Resource Requirements
You should apply CPU and memory requirements to your pods. This is crucial for informing the scheduler which type of compute node your pod needs to be placed on, and ensuring it does not over-provision that node. In Kubernetes, you can set a resource requirement by defining `resources.requests` in the resource requests field in a pod's container spec. For details, refer to the [Kubernetes docs](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container).
@@ -15,7 +15,7 @@ Rancher is container-based and can potentially run on any Linux-based operating
### Upgrade Your Kubernetes Version
Keep your Kubernetes cluster up to date with a recent and supported version. Typically the Kubernetes community will support the current version and previous three minor releases (for example, 1.14.x, 1.13.x, 1.12.x, and 1.11.x). After a new version is released, the third-oldest supported version reaches EOL (End of Life) status. Running on an EOL release can be a risk if a security issues are found and patches are not available. The community typically makes minor releases every quarter (every three months).
Ranchers SLAs are not community dependent, but as Kubernetes is a community-driven software, the quality of experience will degrade as you get farther away from the community's supported target.
Ranchers SLAs are not community dependent, but as Kubernetes is a community-driven software, the quality of experience will degrade as you get farther away from the community's supported target.
### Kill Pods Randomly During Testing
Run chaoskube or a similar mechanism to randomly kill pods in your test environment. This will test the resiliency of your infrastructure and the ability of Kubernetes to self-heal. It's not recommended to run this in your production environment.
@@ -26,7 +26,7 @@ Rancher's "Add Cluster" UI is preferable for getting started with Kubernetes clu
Rancher [maintains a Terraform provider](https://rancher.com/blog/2019/rancher-2-terraform-provider/) for working with Rancher 2.0 Kubernetes. It is called the [Rancher2 Provider.](https://www.terraform.io/docs/providers/rancher2/index.html)
### Upgrade Rancher in a Staging Environment
All upgrades, both patch and feature upgrades, should be first tested on a staging environment before production is upgraded. The more closely the staging environment mirrors production, the higher chance your production upgrade will be successful.
All upgrades, both patch and feature upgrades, should be first tested on a staging environment before production is upgraded. The more closely the staging environment mirrors production, the higher chance your production upgrade will be successful.
### Renew Certificates Before they Expire
Multiple people in your organization should set up calendar reminders for certificate renewal. Consider renewing the certificate two weeks to one month in advance. If you have multiple certificates to track, consider using [monitoring and alerting mechanisms]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/) to track certificate expiration.
@@ -48,7 +48,7 @@ When installing or upgrading a non-production environment to an early release, a
Make sure the feature version you are upgrading to is considered "stable" as determined by Rancher. Use the beta, release candidate, and "latest" versions in a testing, development, or demo environment to try out new features. Feature version upgrades, for example 2.1.x to 2.2.x, should be considered as and when they are released. Some bug fixes and most features are not back ported into older versions.
Keep in mind that Rancher does End of Life support for old versions, so you will eventually want to upgrade if you want to continue to receive patches.
Keep in mind that Rancher does End of Life support for old versions, so you will eventually want to upgrade if you want to continue to receive patches.
For more detail on what happens during the Rancher product lifecycle, refer to the [Support Maintenance Terms](https://rancher.com/support-maintenance-terms/).
@@ -99,7 +99,7 @@ In addition to Rancher software updates, closely monitor security fixes for rela
# Tips for Multi-Tenant Clusters
### Namespaces
Each tenant should have their own unique namespaces within the cluster. This avoids naming conflicts and allows resources to be only visible to their owner through use of RBAC policy
Each tenant should have their own unique namespaces within the cluster. This avoids naming conflicts and allows resources to be only visible to their owner through use of RBAC policy
### Project Isolation
Use Rancher's Project Isolation to automatically generate Network Policy between Projects (sets of Namespaces). This further protects workloads from interference
@@ -108,18 +108,18 @@ Use Rancher's Project Isolation to automatically generate Network Policy between
Enforce use of sane resource limit definitions for every deployment in your cluster. This not only protects the owners of the deployment, but the neighboring resources from other tenants as well. Remember, namespaces do not isolate at the node level, so over-consumption of resources on a node affects other namespace deployments. Admission controllers can be written to require resource limit definitions
### Resource Requirements
Enforce use of resource requirement definitions for each deployment in your cluster. This enables the scheduler to appropriately schedule workloads. Otherwise you will eventually end up with over-provisioned nodes.
Enforce use of resource requirement definitions for each deployment in your cluster. This enables the scheduler to appropriately schedule workloads. Otherwise you will eventually end up with over-provisioned nodes.
# Class of Service and Kubernetes Clusters
A class of service describes the expectations around cluster uptime, durability, and duration of maintenance windows. Typically organizations group these characteristics into labels such as "dev" or "prod"
### Consider fault domains
Kubernetes clusters can span multiple classes of service, however it is important to consider the ability for one workload to affect another. Without proper deployment practices such as resource limits, requirements, etc, a deployment that is not behaving well has the potential to impact the health of the cluster. In a "dev" environment it is common for end-users to exercise less caution with deployments, thus increasing the chance of such behavior. Sharing this behavior with your production workload increases risk.
Kubernetes clusters can span multiple classes of service, however it is important to consider the ability for one workload to affect another. Without proper deployment practices such as resource limits, requirements, etc, a deployment that is not behaving well has the potential to impact the health of the cluster. In a "dev" environment it is common for end-users to exercise less caution with deployments, thus increasing the chance of such behavior. Sharing this behavior with your production workload increases risk.
### Upgrade risks
Upgrades of Kuberentes are not without risk, the best way to predict the outcome of an upgrade is try it on a cluster of similar load and use case as your production cluster. This is where having non-prod class of service clusters can be advantageous.
Upgrades of Kubernetes are not without risk, the best way to predict the outcome of an upgrade is try it on a cluster of similar load and use case as your production cluster. This is where having non-prod class of service clusters can be advantageous.
### Resource Efficiency
### Resource Efficiency
Clusters can be built with varying degrees of redundancy. In a class of service with low expectations for uptime, resources and cost can be conserved by building clusters without redundant Kubernetes control components. This approach may also free up more budget/resources to increase the redundancy at the production level
# Network Security
+2 -1
View File
@@ -1,5 +1,6 @@
---
title: Catalogs and Apps
title: Catalogs, Helm Charts and Apps
description: Rancher enables the use of catalogs to repeatedly deploy applications easily. Catalogs are GitHub or Helm Chart repositories filled with deployment-ready apps.
weight: 4000
aliases:
- /rancher/v2.x/en/concepts/global-configuration/catalog/
@@ -35,7 +35,7 @@ By default, only [global administrators]({{< baseurl >}}/rancher/v2.x/en/admin-s
1. From the **Global View**, select **Tools > Global DNS Providers**.
1. To add a provider, choose from the available provider options and configure the Global DNS Provider with necessary credentials and an optional domain.
1. (Optional) Add additional users so they could use the provider when creating Globel DNS entries as well as manage the Global DNS provider.
1. (Optional) Add additional users so they could use the provider when creating Global DNS entries as well as manage the Global DNS provider.
{{% accordion id="route53" label="Route53" %}}
1. Enter a **Name** for the provider.
@@ -81,7 +81,7 @@ By default, only [global administrators]({{< baseurl >}}/rancher/v2.x/en/admin-s
In order for Global DNS entries to be programmed, you will need to add a specific annotation on an ingress in your application or target project and this ingress needs to use a specific `hostname` and an annotation that should match the FQDN of the Global DNS entry.
1. For any application that you want targetted for your Global DNS entry, find an ingress associated with the application.
1. For any application that you want targeted for your Global DNS entry, find an ingress associated with the application.
1. In order for the DNS to be programmed, the following requirements must be met:
* The ingress routing rule must be set to use a `hostname` that matches the FQDN of the Global DNS entry.
* The ingress must have an annotation (`rancher.io/globalDNS.hostname`) and the value of this annotation should match the FQDN of the Global DNS entry.
@@ -18,7 +18,7 @@ After creating a multi-cluster application, you can program a [Global DNS entry]
3. (Optional) Review the detailed descriptions, which are derived from the Helm chart's `README`.
4. Under **Configuration Options** enter a **Name** for the multi-cluster application. By default, this name is also used to create a Kubernetes namespace in each [target project](#targets) for the multi-cluster application. The namespace is named as `<MULTI-CLUSTER_APPLICTION_NAME>-<PROJECT_ID>`.
4. Under **Configuration Options** enter a **Name** for the multi-cluster application. By default, this name is also used to create a Kubernetes namespace in each [target project](#targets) for the multi-cluster application. The namespace is named as `<MULTI-CLUSTER_APPLICATION_NAME>-<PROJECT_ID>`.
5. Select a **Template Version**.
@@ -50,13 +50,13 @@ In the **Upgrades** section, select the upgrade strategy to use, when you decide
#### Roles
In the **Roles** section, you define the role of the multi-cluster application. Typically, when a user [launches catalog applications]({{< baseurl >}}/rancher/v2.x/en/catalog/apps/#launching-catalog-applications), that specific users's permissions are used for creation of all workloads/resources that is required by the app.
In the **Roles** section, you define the role of the multi-cluster application. Typically, when a user [launches catalog applications]({{< baseurl >}}/rancher/v2.x/en/catalog/apps/#launching-catalog-applications), that specific user's permissions are used for creation of all workloads/resources that is required by the app.
For multi-cluster applications, the application is deployed by a _system user_ and is assigned as the creator of all underlying resources. A _system user_ is used instead of the actual user due to the fact that the actual user could be removed from one of the target projects. If the actual user was removed from one of the projects, then that user would no longer be able to manage the application for the other projects.
Rancher will let you select from two options for Roles, **Project** and **Cluster**. Rancher will allow creation using any of these roles based on the user's permissions.
- **Project** - This is the equivalent of a [project member]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles). If you select this role, Rancher will check that in all the target projects, the user has minimally the [project member]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) role. While the user might not be explicitly granted the _project member_ role, if the user is an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), a [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or a [project owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles), then the user is considered to have the appropriate level of permissions.
- **Project** - This is the equivalent of a [project member]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles). If you select this role, Rancher will check that in all the target projects, the user has minimally the [project member]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) role. While the user might not be explicitly granted the _project member_ role, if the user is an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), a [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or a [project owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles), then the user is considered to have the appropriate level of permissions.
- **Cluster** - This is the equivalent of a [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles). If you select this role, Rancher will check that in all the target projects, the user has minimally the [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) role. While the user might not be explicitly granted the _cluster owner_ role, if the user is an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), then the user is considered to have the appropriate level of permissions.
@@ -68,7 +68,7 @@ When launching the application, Rancher will confirm if you have these permissio
For each Helm chart, there are a list of desired answers that must be entered in order to successfully deploy the chart. When entering answers, you must format them using the syntax rules found in [Using Helm: The format and limitations of set](https://github.com/helm/helm/blob/master/docs/using_helm.md#the-format-and-limitations-of---set), as Rancher passes them as `--set` flags to Helm.
> For example, when entering an answer that includes two values separated by a comma (i.e. `abc, bcd`), it is reuired to wrap the values with double quotes (i.e., ``"abc, bcd"``).
> For example, when entering an answer that includes two values separated by a comma (i.e. `abc, bcd`), it is required to wrap the values with double quotes (i.e., ``"abc, bcd"``).
#### Using a `questions.yml` file
@@ -84,7 +84,7 @@ By default, multi-cluster applications can only be managed by the user who creat
1. Find the user that you want to add by typing in the member's name in the **Member** search box.
2. Select the **Access Type** for that member. There are three access types for a multi-cluster project, but due to how the permissions of a multi-cluster application are launched, please read carefully to understand what these access types mean.
2. Select the **Access Type** for that member. There are three access types for a multi-cluster project, but due to how the permissions of a multi-cluster application are launched, please read carefully to understand what these access types mean.
- **Owner**: This access type can manage any configuration part of the multi-cluster application including the template version, the [multi-cluster applications specific configuration options](#configuration-options-to-make-a-multi-cluster-app), the [application specific configuration options](#application-configuration-options), the [members who can interact with the multi-cluster application](#members) and the [custom application configuration answers](#overriding-application-configuration-options-for-specific-projects). Since a multi-cluster application is created with a different set of permissions from the user, any _owner_ of the multi-cluster application can manage/remove applications in [target projects](#targets) without explicitly having access to these project(s). Only trusted users should be provided with this access type.
@@ -108,7 +108,7 @@ The ability to use the same configuration to deploy the same application across
- **Answer**: Enter the answer that you want to be used instead.
## Upgrading Multi-Cluster App Roles and Projects
## Upgrading Multi-Cluster App Roles and Projects
- **Changing Roles on an existing Multi-Cluster app**
The creator and any users added with the access-type "owner" to a multi-cluster app, can upgrade its Roles. When adding a new Role, we check if the user has that exact role in all current target projects. These checks allow the same relaxations for global admins, cluster owners and project-owners as described in the installation section for the field `Roles`.
@@ -120,7 +120,7 @@ The creator and any users added with the access-type "owner" to a multi-cluster
## Multi-Cluster Application Management
One of the benefits of using a multi-cluster application as opposed to multiple individual applications of the same type, is the ease of manangement.Multi-cluster applications can be cloned, upgraded or rolled back.
One of the benefits of using a multi-cluster application as opposed to multiple individual applications of the same type, is the ease of management. Multi-cluster applications can be cloned, upgraded or rolled back.
1. From the **Global** view, choose **Apps** in the navigation bar.
+4 -1
View File
@@ -1,5 +1,8 @@
---
title: CLI
title: Using the Rancher Command Line Interface
description: The Rancher CLI is a unified tool that you can use to interact with Rancher. With it, you can operate Rancher using a command line interface rather than the GUI
metaTitle: "Using the Rancher Command Line Interface "
metaDescription: "The Rancher CLI is a unified tool that you can use to interact with Rancher. With it, you can operate Rancher using a command line interface rather than the GUI"
weight: 6000
aliases:
- /rancher/v2.x/en/concepts/cli-configuration/
@@ -3,66 +3,15 @@ title: Cluster Administration
weight: 2005
---
## What's a Kubernetes Cluster?
A cluster is a group of computers that work together as a single system.
A _Kubernetes Cluster_ is a cluster that uses the [Kubernetes container-orchestration system](https://kubernetes.io/) to deploy, maintain, and scale Docker containers, allowing your organization to automate application operations.
### Kubernetes Cluster Node Components
Each computing resource in a Kubernetes Cluster is called a _node_. Nodes can be either bare-metal servers or virtual machines. Kubernetes classifies nodes into three types: _etcd_ nodes, _control plane_ nodes, and _worker_ nodes.
#### etcd Nodes
[etcd](https://kubernetes.io/docs/concepts/overview/components/#etcd) nodes run the etcd database. The etcd database component is a key value store used as Kubernetes storage for all cluster data, such as cluster coordination and state management.
etcd is a distributed key value store, meaning it runs on multiple nodes so that there's always a backup available for fail over. Even though you can run etcd on a single node, you should run it on multiple nodes. We recommend 3, 5, or 7 etcd nodes for redundancy.
#### Control Plane Nodes
[Control plane](https://kubernetes.io/docs/concepts/#kubernetes-control-plane) nodes run the Kubernetes API server, scheduler, and controller manager. These nodes take care of routine tasks to ensure that your cluster maintains your configuration. Because all cluster data is stored on your etcd nodes, control plane nodes are stateless. You can run control plane on a single node, although two or more nodes are recommended for redundancy. Additionally, a single node can share the control plane and etcd roles.
#### Worker Nodes
[Worker nodes](https://kubernetes.io/docs/concepts/architecture/nodes/) run:
- _Kubelets_: An agent that monitors the state of the node, ensuring your containers are healthy.
- _Workloads_: The containers and pods that hold your apps, as well as other types of deployments.
Worker nodes also run storage and networking drivers, and ingress controllers when required. You create as many worker nodes as necessary to run your workloads.
After you provision a cluster in Rancher, you can begin using powerful Kubernetes features to deploy and scale your containerized applications in development, testing, or production environments.
## Interacting with Clusters
This page covers the following topics:
- **Rancher UI**
- [Switching between clusters](#switching-between-clusters)
- [Managing clusters in Rancher](#managing-clusters-in-rancher)
- [Configuring tools](#configuring-tools)
Rancher provides an intuitive user interface for interacting with your clusters. All options available in the UI use the Rancher API. Therefore any action possible in the UI is also possible in the Rancher CLI or Rancher API.
- **kubectl**
You can use the Kubernetes command-line tool, [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/), to manage your clusters. You have two options for using kubectl:
- **Rancher kubectl shell**
Interact with your clusters by launching a kubectl shell available in the Rancher UI. This option requires no configuration actions on your part.
For more information, see [Accessing Clusters with kubectl Shell]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubectl/#accessing-clusters-with-kubectl-shell).
- **Terminal remote connection**
You can also interact with your clusters by installing [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your local desktop and then copying the cluster's kubeconfig file to your local `~/.kube/config` directory.
For more information, see [Accessing Clusters with kubectl and a kubeconfig File]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubectl/#accessing-clusters-with-kubectl-and-a-kubeconfig-file).
- **Rancher CLI**
You can control your clusters by downloading Rancher's own command-line interface, [Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/). This CLI tool can interact directly with different clusters and projects or pass them `kubectl` commands.
- **Rancher API**
Finally, you can interact with your clusters over the Rancher API. Before you use the API, you must obtain an [API key]({{< baseurl >}}/rancher/v2.x/en/user-settings/api-keys/). To view the different resource fields and actions for an API object, open the API UI, which can be accessed by clicking on **View in API** for any Rancher UI object.
> This section assumes a basic familiarity with Docker and Kubernetes. For a brief explanation of how Kubernetes components work together, refer to the [concepts]({{<baseurl>}}/rancher/v2.x/en/overview/concepts) page.
## Switching between Clusters
@@ -76,9 +25,9 @@ After clusters have been [provisioned into Rancher]({{< baseurl >}}/rancher/v2.x
| Action | [Rancher launched Kubernetes Clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) | [Hosted Kubernetes Clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/) | [Imported Clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters) |
| --- | --- | ---| ---|
| [Using kubeconfig file to access a Cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/kubeconfig/) | * | * | * |
| [Using kubectl to Access a Cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/kubectl/) | * | * | * |
| [Adding Cluster Members]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/cluster-members/) | * | * | * |
| [Using kubeconfig file to access a Cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/cluster-access/kubeconfig/) | * | * | * |
| [Using kubectl to Access a Cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/cluster-access/kubectl/) | * | * | * |
| [Adding Cluster Members]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/cluster-access/cluster-members/) | * | * | * |
| [Editing Clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/editing-clusters/) | * | * | * |
| [Managing Nodes]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/nodes) | * | * | * |
| [Managing Persistent Volumes and Storage Classes]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/) | * | * | * |
@@ -7,7 +7,7 @@ _Available as of v2.2.0_
In the Rancher UI, etcd backup and recovery for [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) can be easily performed. Snapshots of the etcd database are taken and saved either [locally onto the etcd nodes](#local-backup-target) or to a [S3 compatible target](#s3-backup-target). The advantages of configuring S3 is that if all etcd nodes are lost, your snapshot is saved remotely and can be used to restore the cluster.
Rancher recommends configuring recurrent `etcd` snapshots for all production clusters. Additonally, one-time snapshots can easily be taken as well.
Rancher recommends configuring recurrent `etcd` snapshots for all production clusters. Additionally, one-time snapshots can easily be taken as well.
>**Note:** If you have any Rancher launched Kubernetes clusters that were created prior to v2.2.0, after upgrading Rancher, you must [edit the cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/editing-clusters/) and _save_ it, in order to enable the updated snapshot features. Even if you were already creating snapshots prior to v2.2.0, you must do this step as the older snapshots will not be available to use to [back up and restore etcd through the UI]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/restoring-etcd/).
@@ -55,7 +55,7 @@ By default, the `local` backup target is selected. The benefits of this option i
_Available as of v2.3.0_
As of v2.2.6, snapshot files are timestamped to simplify processing the files using external tools and scripts, but in some S3 compatible backends, these timestamps were unusable. As of Rancher v2.3.0, the option `safe_timestamp` is added to support compatiable file names. When this flag is set to `true`, all special characters in the snapshot filename timestamp are replaced.
As of v2.2.6, snapshot files are timestamped to simplify processing the files using external tools and scripts, but in some S3 compatible backends, these timestamps were unusable. As of Rancher v2.3.0, the option `safe_timestamp` is added to support compatible file names. When this flag is set to `true`, all special characters in the snapshot filename timestamp are replaced.
>>**Note:** This option is not available directly in the UI, and is only available through the `Edit as Yaml` interface.
@@ -1,13 +1,17 @@
---
title: Cleaning up Clusters
title: Removing Kubernetes Components from Nodes
description: Learn about cluster cleanup when removing nodes from your Rancher-launched Kubernetes cluster. What is removed, how to do it manually
weight: 2055
aliases:
- /rancher/v2.x/en/faq/cleaning-cluster-nodes/
- /rancher/v2.x/en/admin-settings/removing-rancher/user-cluster-nodes/
---
This section describes how to disconnect a node from a Rancher-launched Kubernetes cluster and remove all of the Kubernetes components from the node. This process allows you to use the node for other purposes.
When you use Rancher to [launch nodes for a cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher), resources (containers/virtual network interfaces) and configuration items (certificates/configuration files) are created.
When removing nodes from your Rancher launched Kubernetes cluster (provided that they are in `Active` state), those resources automatically cleaned, and the only action needed is to restart the node. When a node has become unreachable and the automatic cleanup process cannot be used, we describe the steps that need to be executed before the node can be added to a cluster again.
When removing nodes from your Rancher launched Kubernetes cluster (provided that they are in `Active` state), those resources are automatically cleaned, and the only action needed is to restart the node. When a node has become unreachable and the automatic cleanup process cannot be used, we describe the steps that need to be executed before the node can be added to a cluster again.
## What Gets Removed?
@@ -0,0 +1,34 @@
---
title: Cluster Access
weight: 1
---
There are many ways you can interact with Kubernetes clusters that are managed by Rancher:
- **Rancher UI**
Rancher provides an intuitive user interface for interacting with your clusters. All options available in the UI use the Rancher API. Therefore any action possible in the UI is also possible in the Rancher CLI or Rancher API.
- **kubectl**
You can use the Kubernetes command-line tool, [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/), to manage your clusters. You have two options for using kubectl:
- **Rancher kubectl shell**
Interact with your clusters by launching a kubectl shell available in the Rancher UI. This option requires no configuration actions on your part.
For more information, see [Accessing Clusters with kubectl Shell]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubectl/#accessing-clusters-with-kubectl-shell).
- **Terminal remote connection**
You can also interact with your clusters by installing [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your local desktop and then copying the cluster's kubeconfig file to your local `~/.kube/config` directory.
For more information, see [Accessing Clusters with kubectl and a kubeconfig File]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubectl/#accessing-clusters-with-kubectl-and-a-kubeconfig-file).
- **Rancher CLI**
You can control your clusters by downloading Rancher's own command-line interface, [Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/). This CLI tool can interact directly with different clusters and projects or pass them `kubectl` commands.
- **Rancher API**
Finally, you can interact with your clusters over the Rancher API. Before you use the API, you must obtain an [API key]({{< baseurl >}}/rancher/v2.x/en/user-settings/api-keys/). To view the different resource fields and actions for an API object, open the API UI, which can be accessed by clicking on **View in API** for any Rancher UI object.
@@ -0,0 +1,58 @@
---
title: Adding Users to Clusters
weight: 2020
aliases:
- /rancher/v2.x/en/tasks/clusters/adding-managing-cluster-members/
- /rancher/v2.x/en/cluster-provisioning/cluster-members/
- /rancher/v2.x/en/k8s-in-rancher/cluster-members/
- /rancher/v2.x/en/cluster-admin/cluster-members
---
If you want to provide a user with access and permissions to _all_ projects, nodes, and resources within a cluster, assign the user a cluster membership.
>**Tip:** Want to provide a user with access to a _specific_ project within a cluster? See [Adding Project Members]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/project-members/) instead.
There are two contexts where you can add cluster members:
- Adding Members to a New Cluster
You can add members to a cluster as you create it (recommended if possible).
- [Adding Members to an Existing Cluster](#editing-cluster-membership)
You can always add members to a cluster after a cluster is provisioned.
## Editing Cluster Membership
Cluster administrators can edit the membership for a cluster, controlling which Rancher users can access the cluster and what features they can use.
1. From the **Global** view, open the cluster that you want to add members to.
2. From the main menu, select **Members**. Then click **Add Member**.
3. Search for the user or group that you want to add to the cluster.
If external authentication is configured:
- Rancher returns users from your [external authentication]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) source as you type.
>**Using AD but can't find your users?**
>There may be an issue with your search attribute configuration. See [Configuring Active Directory Authentication: Step 5]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/ad/).
- A drop-down allows you to add groups instead of individual users. The drop-down only lists groups that you, the logged in user, are part of.
>**Note:** If you are logged in as a local user, external users do not display in your search results. For more information, see [External Authentication Configuration and Principal Users]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/#external-authentication-configuration-and-principal-users).
4. Assign the user or group **Cluster** roles.
[What are Cluster Roles?]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/)
>**Tip:** For Custom Roles, you can modify the list of individual roles available for assignment.
>
> - To add roles to the list, [Add a Custom Role]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/default-custom-roles/).
> - To remove roles from the list, [Lock/Unlock Roles]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles).
**Result:** The chosen users are added to the cluster.
- To revoke cluster membership, select the user and click **Delete**. This action deletes membership, not the user.
- To modify a user's roles in the cluster, delete them from the cluster, and then re-add them with modified roles.
@@ -1,9 +1,11 @@
---
title: Kubeconfig File
title: Configuring Access to Kubernetes Using A Kubeconfig File
description: A kubeconfig file is used to configure access to Kubernetes. When you create a cluster with Rancher, it automatically creates a kubeconfig for your cluster
weight: 2010
aliases:
- /rancher/v2.x/en/concepts/clusters/kubeconfig-files/
- /rancher/v2.x/en/k8s-in-rancher/kubeconfig/
- /rancher/2.x/en/cluster-admin/kubeconfig
---
A _kubeconfig file_ is a file used to configure access to Kubernetes when used in conjunction with the kubectl commandline tool (or other clients).
@@ -40,7 +42,7 @@ CURRENT NAME CLUSTER AUTHINFO N
* my-cluster my-cluster user-46tmn
my-cluster-controlplane-1 my-cluster-controlplane-1 user-46tmn
```
For more information on how the authorized cluster endpoint works, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/4-authorized-cluster-endpoint)
For more information on how the authorized cluster endpoint works, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/#4-authorized-cluster-endpoint)
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture-recommendations/#architecture-for-an-authorized-cluster-endpoint)
@@ -1,9 +1,11 @@
---
title: Using kubectl to Access a Cluster
title: "Access a Cluster with Kubectl Shell or Kubectl CLI"
description: "Lean how you can access and manage your Kubernetes clusters using kubectl in two ways: with kubectl Shell or with kubectl CLI and kubeconfig file"
weight: 2015
aliases:
- /rancher/v2.x/en/tasks/clusters/using-kubectl-to-access-a-cluster/
- /rancher/v2.x/en/k8s-in-rancher/kubectl/
- /rancher/v2.x/en/cluster-admin/kubectl
---
You can access and manage your Kubernetes clusters using kubectl in two ways:
@@ -1,22 +0,0 @@
---
title: Adding Users to Clusters
weight: 2020
aliases:
- /rancher/v2.x/en/tasks/clusters/adding-managing-cluster-members/
- /rancher/v2.x/en/cluster-provisioning/cluster-members/
- /rancher/v2.x/en/k8s-in-rancher/cluster-members/
---
If you want to provide a user with access and permissions to _all_ projects, nodes, and resources within a cluster, assign the user a cluster membership.
>**Tip:** Want to provide a user with access to a _specific_ project within a cluster? See [Adding Project Members]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/project-members/) instead.
There are two contexts where you can add cluster members:
- Adding Members to a New Cluster
You can add members to a cluster as you create it (recommended if possible).
- [Adding Members to an Existing Cluster]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/editing-clusters/)
You can always add members to a cluster after a cluster is provisioned.
@@ -1,5 +1,5 @@
---
title: Editing Clusters
title: Cluster Configuration
weight: 2025
aliases:
- /rancher/v2.x/en/k8s-in-rancher/editing-clusters/
@@ -22,92 +22,19 @@ The following table lists the options and settings available for each cluster ty
## Editing Cluster Membership
Cluster administrators can edit the membership for a cluster, controlling which Rancher users can access the cluster and what features they can use.
1. From the **Global** view, open the cluster that you want to add members to.
2. From the main menu, select **Members**. Then click **Add Member**.
3. Search for the user or group that you want to add to the cluster.
If external authentication is configured:
- Rancher returns users from your [external authentication]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) source as you type.
>**Using AD but can't find your users?**
>There may be an issue with your search attribute configuration. See [Configuring Active Directory Authentication: Step 5]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/ad/).
- A drop-down allows you to add groups instead of individual users. The drop-down only lists groups that you, the logged in user, are part of.
>**Note:** If you are logged in as a local user, external users do not display in your search results. For more information, see [External Authentication Configuration and Principal Users]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/#external-authentication-configuration-and-principal-users).
4. Assign the user or group **Cluster** roles.
[What are Cluster Roles?]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/)
>**Tip:** For Custom Roles, you can modify the list of individual roles available for assignment.
>
> - To add roles to the list, [Add a Custom Role]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/default-custom-roles/).
> - To remove roles from the list, [Lock/Unlock Roles]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles).
**Result:** The chosen users are added to the cluster.
- To revoke cluster membership, select the user and click **Delete**. This action deletes membership, not the user.
- To modify a user's roles in the cluster, delete them from the cluster, and then re-add them with modified roles.
Cluster administrators can [edit the membership for a cluster,]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/cluster-access/cluster-members) controlling which Rancher users can access the cluster and what features they can use.
## Cluster Options
When editing clusters, clusters that are [launched using RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) feature more options than clusters that are imported or hosted by a Kubernetes provider. The headings that follow document options available only for RKE clusters.
### Upgrading Kubernetes
### Updating ingress-nginx
Following an upgrade to the latest version of Rancher, you can update your existing clusters to use the latest supported version of Kubernetes.
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For example, Rancher v2.3.0 is was tested with Kubernetes v1.15.4, v1.14.7, and v1.13.11. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
As of Rancher v2.3.0, the Kubernetes metadata feature was added, which allows Rancher to ship Kubernetes patch versions without upgrading Rancher. For details, refer to the [section on Kubernetes metadata.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata)
>**Recommended:** Before upgrading Kubernetes, [backup your cluster]({{< baseurl >}}/rancher/v2.x/en/backups).
1. From the **Global** view, find the cluster for which you want to upgrade Kubernetes. Select **Vertical Ellipsis (...) > Edit**.
1. Expand **Cluster Options**.
1. From the **Kubernetes Version** drop-down, choose the version of Kubernetes that you want to use for the cluster.
1. Click **Save**.
**Result:** Kubernetes begins upgrading for the cluster. During the upgrade, your cluster is unavailable.
> **Note:** The `ingress-nginx` pods are set to only upgrade on delete. After upgrading your cluster, you need to delete these pods to get the correct version for your deployment.
### Adding a Pod Security Policy
When your cluster is running pods with security-sensitive configurations, assign it a [pod security policy]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/), which is a set of rules that monitors the conditions and settings in your pods. If a pod doesn't meet the rules specified in your policy, the policy stops it from running.
You can assign a pod security policy when you provision a cluster. However, if you need to relax or restrict security for your pods later, you can update the policy while editing your cluster.
1. From the **Global** view, find the cluster to which you want to apply a pod security policy. Select **Vertical Ellipsis (...) > Edit**.
2. Expand **Cluster Options**.
3. From **Pod Security Policy Support**, select **Enabled**.
>**Note:** This option is only available for clusters [provisioned by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
4. From the **Default Pod Security Policy** drop-down, select the policy you want to apply to the cluster.
Rancher ships with [policies]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/#default-pod-security-policies) of `restricted` and `unrestricted`, although you can [create custom policies]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/#default-pod-security-policies) as well.
5. Click **Save**.
**Result:** The pod security policy is applied to the cluster and any projects within the cluster.
>**Note:** Workloads already running before assignment of a pod security policy are grandfathered in. Even if they don't meet your pod security policy, workloads running before assignment of the policy continue to run.
>
>To check if a running workload passes your pod security policy, clone or upgrade it.
### Editing Other Cluster Options
# Editing Other Cluster Options
In [clusters launched by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), you can edit any of the remaining options that follow.
@@ -119,7 +46,7 @@ In [clusters launched by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioni
Option | Description |
---------|----------|
Kubernetes Version | The version of Kubernetes installed on each cluster node. For more detail, see [Upgrading Kubernetes](#upgrading-kubernetes). |
Kubernetes Version | The version of Kubernetes installed on each cluster node. For more detail, see [Upgrading Kubernetes]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/upgrading-kubernetes). |
Network Provider | The [container networking interface]({{< baseurl >}}/rancher/v2.x/en/faq/networking/#cni-providers) that powers networking for your cluster.<br/><br/>**Note:** You can only choose this option while provisioning your cluster. It cannot be edited later. |
Project Network Isolation | As of Rancher v2.0.7, if you're using the Canal network provider, you can choose whether to enable or disable inter-project communication. |
Nginx Ingress | If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use Nginx ingress within the cluster. |
@@ -130,7 +57,8 @@ Option | Description |
Default Pod Security Policy | If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster. |
Cloud Provider | If you're using a cloud provider to host cluster nodes launched by RKE, enable [this option]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers/) so that you can use the cloud provider's native features. If you want to store persistent data for your cloud-hosted cluster, this option is required. |
<br/>
#### Editing Cluster as YAML
# Editing Cluster as YAML
>**Note:** In Rancher v2.0.5 and v2.0.6, the names of services in the Config File (YAML) should contain underscores only: `kube_api` and `kube_controller`.
@@ -142,19 +70,3 @@ Instead of using the Rancher UI to choose Kubernetes options for the cluster, ad
![image]({{< baseurl >}}/img/rancher/cluster-options-yaml.png)
For an example of RKE config file syntax, see the [RKE documentation]({{< baseurl >}}/rke/latest/en/example-yamls/).
## Managing Node Pools
In clusters [launched by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), you can:
- Add new [pools of nodes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/) to your cluster. The nodes added to the pool are provisioned according to the [node template]({{< baseurl >}}/rancher/v2.x/en/user-settings/node-templates/) that you use.
- Click **+** and follow the directions on screen to create a new template.
- You can also reuse existing templates by selecting one from the **Template** drop-down.
- Redistribute Kubernetes roles amongst your node pools by making different checkbox selections
- Scale the number of nodes in a pool up or down (although, if you simply want to maintain your node scale, we recommend using the cluster's [Nodes tab]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/nodes/#nodes-provisioned-by-node-pool) instead.)
>**Note:** The Node Pools section is not available for imported clusters or clusters hosted by a Kubernetes provider.
@@ -1,5 +1,5 @@
---
title: Nodes
title: Nodes and Node Pools
weight: 2030
aliases:
- /rancher/v2.x/en/k8s-in-rancher/nodes/
@@ -7,11 +7,23 @@ aliases:
After you launch a Kubernetes cluster in Rancher, you can manage individual nodes from the cluster's **Node** tab. Depending on the [option used]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) to provision the cluster, there are different node options available.
This page covers the following topics:
- [Node options for each type of cluster](#node-options-for-each-type-of-cluster)
- [Cordoning and draining nodes](#cordoning-and-draining-nodes)
- [Editing a node](#editing-a-node)
- [Viewing a node API](#viewing-a-node-api)
- [Deleting a node](#deleting-a-node)
- [Scaling nodes](#scaling-nodes)
- [SSH into a node hosted by an infrastructure provider](#ssh-into-a-node-hosted-by-an-infrastructure-provider)
- [Managing node pools](#managing-node-pools)
To manage individual nodes, browse to the cluster that you want to manage and then select **Nodes** from the main menu. You can open the options menu for a node by clicking its **Ellipsis** icon (**...**).
>**Note:** If you want to manage the _cluster_ and not individual nodes, see [Editing Clusters]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/editing-clusters).
# Node Options for Each Type of Cluster
The following table lists which node options are available for each [type of cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-options) in Rancher. Click the links in the **Option** column for more detailed information about each feature.
| Option | [Nodes Hosted by an Infrastructure Provider][1] | [Custom Node][2] | [Hosted Cluster][3] | [Imported Nodes][4] | Description |
@@ -29,14 +41,26 @@ The following table lists which node options are available for each [type of clu
[3]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/
[4]: {{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/
## Cordoning a Node
### Notes for Node Pool Nodes
Clusters provisioned using [one of the node pool options]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-pools) automatically maintain the node scale that's set during the initial cluster provisioning. This scale determines the number of active nodes that Rancher maintains for the cluster.
### Notes for Nodes Provisioned by Hosted Kubernetes Providers
Options for managing nodes [hosted by a Kubernetes provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/) are somewhat limited in Rancher. Rather than using the Rancher UI to make edits such as scaling the number of nodes up or down, edit the cluster directly.
### Notes for Imported Nodes
Although you can deploy workloads to an [imported cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/) using Rancher, you cannot manage individual cluster nodes. All management of imported cluster nodes must take place outside of Rancher.
# Cordoning and Draining Nodes
_Cordoning_ a node marks it as unschedulable. This feature is useful for performing short tasks on the node during small maintenance windows, like reboots, upgrades, or decommissions. When you're done, power back on and make the node schedulable again by uncordoning it.
## Draining a Node
_Draining_ is the process of first cordoning the node, and then evicting all its pods. This feature is useful for performing node maintenance (like kernel upgrades or hardware maintenance). It prevents new pods from deploying to the node while redistributing existing pods so that users don't experience service interruption.
When nodes are drained, pods are handled with the following rules:
- For pods with a replica set, the pod is replaced by a new pod that will be scheduled to a new node. Additionally, if the pod is part of a service, then clients will automatically be redirected to the new pod.
- For pods with no replica set, you need to bring up a new copy of the pod, and assuming it is not part of a service, redirect clients to it.
@@ -99,7 +123,7 @@ Once drain successfully completes, the node will be in a state of `drained`. You
>**Want to know more about cordon and drain?** See the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#maintenance-on-a-node).
## Editing a Node
# Editing a Node
Editing a node lets you:
@@ -109,23 +133,23 @@ Editing a node lets you:
* Add/Remove [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
## Viewing a Node API
# Viewing a Node API
Select this option to view the node's [API endpoints]({{< baseurl >}}/rancher/v2.x/en/api/).
## Deleting a Node
# Deleting a Node
Use **Delete** to remove defective nodes from the cloud provider. When you the delete a defective node, Rancher automatically replaces it with an identically provisioned node.
>**Tip:** If your cluster is hosted by an infrastructure provider, and you want to scale your cluster down instead of deleting a defective node, [scale down](#scaling-nodes) rather than delete.
## Scaling Nodes
# Scaling Nodes
For nodes hosted by an infrastructure provider, you can scale the number of nodes in each node pool by using the scale controls. This option isn't available for other cluster types.
## SSH into a Node Hosted by an Infrastructure Provider
# SSH into a Node Hosted by an Infrastructure Provider
For [nodes hosted by an infrastructure provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/), you have the option of downloading its SSH key so that you can connect to it remotely from your desktop.
@@ -145,17 +169,19 @@ For [nodes hosted by an infrastructure provider]({{< baseurl >}}/rancher/v2.x/en
```
ssh -i id_rsa root@<IP_OF_HOST>
```
# Managing Node Pools
## Notes for Node Pool Nodes
> **Prerequisite:** The options below are available only for clusters that are [launched using RKE.]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) The node pool features are not available for imported clusters or clusters hosted by a Kubernetes provider.
Clusters provisioned using [one of the node pool options]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-pools) automatically maintain the node scale that's set during the initial cluster provisioning. This scale determines the number of active nodes that Rancher maintains for the cluster.
In clusters [launched by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), you can:
- Add new [pools of nodes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/) to your cluster. The nodes added to the pool are provisioned according to the [node template]({{< baseurl >}}/rancher/v2.x/en/user-settings/node-templates/) that you use.
## Notes for Nodes Provisioned by Hosted Kubernetes Providers
- Click **+** and follow the directions on screen to create a new template.
Options for managing nodes [hosted by a Kubernetes provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/) are somewhat limited in Rancher. Rather than using the Rancher UI to make edits such as scaling the number of nodes up or down, edit the cluster directly.
- You can also reuse existing templates by selecting one from the **Template** drop-down.
- Redistribute Kubernetes roles amongst your node pools by making different checkbox selections
## Notes for Imported Nodes
Although you can deploy workloads to an [imported cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/) using Rancher, you cannot manage individual cluster nodes. All management of imported cluster nodes must take place outside of Rancher.
- Scale the number of nodes in a pool up or down (although, if you simply want to maintain your node scale, we recommend using the cluster's [Nodes tab]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/nodes/#nodes-provisioned-by-node-pool) instead.)
@@ -0,0 +1,30 @@
---
title: Adding a Pod Security Policy
weight: 80
---
> **Prerequisite:** The options below are available only for clusters that are [launched using RKE.]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/)
When your cluster is running pods with security-sensitive configurations, assign it a [pod security policy]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/), which is a set of rules that monitors the conditions and settings in your pods. If a pod doesn't meet the rules specified in your policy, the policy stops it from running.
You can assign a pod security policy when you provision a cluster. However, if you need to relax or restrict security for your pods later, you can update the policy while editing your cluster.
1. From the **Global** view, find the cluster to which you want to apply a pod security policy. Select **Vertical Ellipsis (...) > Edit**.
2. Expand **Cluster Options**.
3. From **Pod Security Policy Support**, select **Enabled**.
>**Note:** This option is only available for clusters [provisioned by RKE]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
4. From the **Default Pod Security Policy** drop-down, select the policy you want to apply to the cluster.
Rancher ships with [policies]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/#default-pod-security-policies) of `restricted` and `unrestricted`, although you can [create custom policies]({{< baseurl >}}/rancher/v2.x/en/admin-settings/pod-security-policies/#default-pod-security-policies) as well.
5. Click **Save**.
**Result:** The pod security policy is applied to the cluster and any projects within the cluster.
>**Note:** Workloads already running before assignment of a pod security policy are grandfathered in. Even if they don't meet your pod security policy, workloads running before assignment of the policy continue to run.
>
>To check if a running workload passes your pod security policy, clone or upgrade it.
@@ -1,25 +1,38 @@
---
title: Projects and Namespaces
title: Projects and Kubernetes Namespaces with Rancher
description: Rancher Projects ease the administrative burden of your cluster and support multi-tenancy. Learn to create projects and divide projects into Kubernetes namespaces
weight: 2032
aliases:
- /rancher/v2.x/en/concepts/projects/
- /rancher/v2.x/en/tasks/projects/
- /rancher/v2.x/en/tasks/projects/create-project/
- /rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/
- /rancher/v2.x/en/tasks/projects/create-project/
---
## Projects
This section describes how projects and namespaces work with Rancher. It covers the following topics:
_Projects_ are organizational objects introduced in Rancher that ease the administrative burden of your cluster. You can use projects to support multi-tenancy.
- [About projects](#about-projects)
- [The cluster's default project](#the-cluster-s-default-project)
- [The system project](#the-system-project)
- [Project authorization](#project-authorization)
- [Pod security policies](#pod-security-policies)
- [Creating projects](#creating-projects)
- [Switching between clusters and projects](#switching-between-clusters-and-projects)
- [Namespaces](#namespaces)
Projects provide an extra level of organization in your Kubernetes clusters beyond [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/). In terms of hierarchy:
# About Projects
- Clusters contain projects.
- Projects contain namespaces.
A project is a group of multiple [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) and access control policies within a cluster. A project is a concept introduced by Rancher, not Kubernetes, which allows you manage multiple namespaces as a group and perform Kubernetes operations in them. The Rancher UI provides features for [project administration]({{<baseurl>}}/rancher/v2.x/en/project-admin/) and for [managing applications within projects.]({{<baseurl>}}/rancher/v2.x/en/k8s-in-rancher/)
Within Rancher, projects allow you manage multiple namespaces as a single entity. In the base version of Kubernetes, which does not include projects, features like role-based access rights or cluster resources are assigned to individual namespaces. In clusters where multiple namespaces require the same set of access rights, assigning these rights to each individual namespace can become tedious. Even though all namespaces require the same rights, there's no way to apply those rights to all of your namespaces in a single action. You'd have to repetitively assign these rights to each namespace!
In terms of hierarchy:
Rancher projects resolve this issue by allowing you to apply resources and access rights at the project level. Each namespace in the project then inherits these resources and policies, so you only have to assign them to the project once, rather than assigning them to each namespace.
- Clusters contain projects
- Projects contain namespaces
You can use projects to support multi-tenancy, so that a team can access a project within a cluster without having access to other projects in the same cluster.
In the base version of Kubernetes, features like role-based access rights or cluster resources are assigned to individual namespaces. A project allows you to save time by giving an individual or a team access to multiple namespaces simultaneously.
You can use projects to perform actions like:
@@ -28,18 +41,16 @@ You can use projects to perform actions like:
- Assign resources to the project.
- Assign Pod Security Policies.
When you create a cluster, two project are automatically created within it:
When you create a cluster, two projects are automatically created within it:
- [Default Project](#default-project)
- [System Project](#system-project)
### The Cluster's Default Project
### Default Project
When you provision a cluster with Rancher, it automatically creates a `default` project for the cluster. This is a project you can use to get started with your cluster, but you can always delete it and replace it with projects that have more descriptive names.
When you provision a cluster, it automatically creates a `default` project for the cluster. This is a project you can use to get started with your cluster, but you can always delete it and replace it with projects that have more descriptive names.
### System Project
### The System Project
_Available as of v2.0.7_
@@ -61,18 +72,18 @@ The `system` project:
>
>The `system` project overrides the Project Network Isolation option so that it can communicate with other projects, collect logs, and check health.
### Authorization
# Project Authorization
Non-administrative users are only authorized for project access after an administrator, cluster owner or cluster member explicitly adds them to the project's **Members** tab.
>**Exception:**
> Non-administrative users can access projects that they create themselves.
### Pod Security Policies
# Pod Security Policies
Rancher extends Kubernetes to allow the application of [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) at the project level in addition to the cluster level. However, as a best practice, we recommend applying Pod Security Policies at the cluster level.
Rancher extends Kubernetes to allow the application of [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) at the [project level]({{<baseurl>}}/rancher/v2.x/en/project-admin/pod-security-policies) in addition to the [cluster level.](../pod-security-policy) However, as a best practice, we recommend applying Pod Security Policies at the cluster level.
### Creating Projects
# Creating Projects
1. From the **Global** view, choose **Clusters** from the main menu. From the **Clusters** page, open the cluster from which you want to create a project.
@@ -129,7 +140,7 @@ Rancher extends Kubernetes to allow the application of [Pod Security Policies](h
1. **Optional:** Repeat these substeps to add more quotas.
1. **Optional:** Specify **Container Default Resource Limit**, which will be applied to every container started in the project. The parameter is recommended if you have CPU or Memory limits set by the Resource Quota. It can be overridden on per an individual namespace or a container level. For more information, see [Container Default Resource Limit]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/resource-quotas/#setting-container-default-resource-limit)
1. **Optional:** Specify **Container Default Resource Limit**, which will be applied to every container started in the project. The parameter is recommended if you have CPU or Memory limits set by the Resource Quota. It can be overridden on per an individual namespace or a container level. For more information, see [Container Default Resource Limit]({{< baseurl >}}/rancher/v2.x/en/project-admin/resource-quotas/#setting-container-default-resource-limit)
>**Note:** This option is available as of v2.2.0.
@@ -137,7 +148,7 @@ Rancher extends Kubernetes to allow the application of [Pod Security Policies](h
**Result:** Your project is created. You can view it from the cluster's **Projects/Namespaces** view.
## Switching between Clusters/Projects
# Switching between Clusters and Projects
To switch between clusters and projects, use the **Global** drop-down available in the main menu.
@@ -148,7 +159,7 @@ Alternatively, you can switch between projects and clusters using the main menu.
- To switch between clusters, open the **Global** view and select **Clusters** from the main menu. Then open a cluster.
- To switch between projects, open a cluster, and then select **Projects/Namespaces** from the main menu. Select the link for the project that you want to open.
## Namespaces
# Namespaces
Within Rancher, you can further divide projects into different [namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/), which are virtual clusters within a project backed by a physical cluster. Should you require another level of organization beyond projects and the `default` namespace, you can use multiple namespaces to isolate applications and resources.
@@ -1,5 +1,5 @@
---
title: Configuring Tools
title: Tools for Logging, Monitoring, and Visibility
weight: 2033
aliases:
- /rancher/v2.x/en/tools/
@@ -10,86 +10,41 @@ Rancher contains a variety of tools that aren't included in Kubernetes to assist
<!-- TOC -->
- [Notifiers](#notifiers)
- [Alerts](#alerts)
- [Notifiers and Alerts](#notifiers-and-alerts)
- [Logging](#logging)
- [Monitoring](#monitoring)
- [Istio](#istio)
<!-- /TOC -->
## Notifiers and Alerts
Notifiers and alerts are two features that work together to inform you of events in the Rancher system. Notifiers are objects that you configure to leverage popular IT services, which send you notification of Rancher events. Alerts are rule sets that trigger when those notifications are sent.
Notifiers and alerts are two features that work together to inform you of events in the Rancher system.
Notifiers and alerts are built on top of the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/alertmanager/). Leveraging these tools, Rancher can notify [cluster owners]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles) and [project owners]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) of events they need to address.
When you create a cluster, some alert rules are predefined. For details, refer to the [documentation on default alerts.]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/alerts/default-alerts)
### Notifiers
Before you can receive alerts, you must configure one or more notifier in Rancher.
_Notifiers_ are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Rancher integrates with a variety of popular IT services, including:
- Slack: Send alert notifications to your Slack channels.
- Email: Choose email recipients for alert notifications.
- PagerDuty: Route notifications to staff by phone, SMS, or personal email.
- Webhooks: Update a webpage with alert notifications.
- WeChat: Send alert notifications to your Enterprise WeChat contacts.
For more information, see [Notifiers]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/notifiers/).
### Alerts
To keep your clusters and applications healthy and driving your organizational productivity forward, you need stay informed of events occurring in your clusters, both planned and unplanned. To help you stay informed of these events, Rancher allows you to configure alerts.
_Alerts_ are sets of rules, chosen by you, to monitor for specific events. The scope for alerts can be set at either the cluster or project level.
Some examples of alert events are:
- A Kubernetes [master component]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#kubernetes-cluster-node-components) entering an unhealthy state.
- A node or [workload]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/workloads/) error occurring.
- A scheduled deployment taking place as planned.
- A node's hardware resources becoming overstressed.
When an event occurs, your alert is triggered, and you are sent a notification. You can then, if necessary, follow up with corrective actions.
Additionally, you can set an urgency level for each alert. This urgency appears in the notification you receive, helping you to prioritize your response actions. For example, if you have an alert configured to inform you of a routine deployment, no action is required. These alerts can be assigned a low priority level. However, if a deployment fails, it can critically impact your organization, and you need to react quickly. Assign these alerts a high priority level.
You can configure alerts at either the [cluster level]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/alerts/) or [project level]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/alerts/).
[Notifiers]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/notifiers) are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Notifications can be sent with Slack, email, PagerDuty, WeChat, and webhooks.
[Alerts]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/alerts) are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
## Logging
Rancher can integrate with popular external services used for event streams, telemetry, or search. Rancher can integrate with the following services:
Logging is helpful because it allows you to:
- Elasticsearch
- splunk
- kafka
- syslog
- fluentd
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
These services collect container log events, which are saved to the `/var/log/containers` directory on each of your nodes. The service collects both standard and error events. You can then log into your services to review the events collected, leveraging each service's unique features.
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
When configuring Rancher to integrate with these services, you'll have to point Rancher toward the service's endpoint and provide authentication information. Additionally, you'll have the opportunity to enter key value pairs to filter the log events collected. The service will only collect events for containers marked with your configured key value pairs.
### Logging Advantages
Setting up a logging service to collect logs from your cluster or project is helpful several ways:
- Logs errors and warnings in your Kubernetes infrastructure to a stream. The stream informs you of events like a container crashing, a pod eviction, or a node dying.
- Allows you to capture and analyze the state of your cluster and look for trends in your environment using the log stream.
- Helps you when troubleshooting or debugging.
- Saves your logs to a safe location outside of your cluster, so that you can still access them even if your cluster encounters issues.
You can configure these services to collect logs at either the [cluster level]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/logging/) or [project level]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/logging/).
For details, refer to the [logging section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/logging)
## Monitoring
_Available as of v2.2.0_
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. Prometheus provides a _time series_ of your data, which is a stream of timestamped values belonging to the same metric and the same set of labeled dimensions, along with comprehensive statistics and metrics of the monitored cluster.
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/monitoring)
In other words, Prometheus let's you view metrics from your different Rancher and Kubernetes objects. Using timestamps, you can query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or [Grafana](https://grafana.com/), which is an analytics viewing platform deployed along with Prometheus. By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc. Multi-tenancy support in terms of cluster and project-only Prometheus instances are also supported.
## Istio
You can configure these services to collect logs at either the [cluster level]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/monitoring/) or [project level]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/monitoring/).
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices. For details on how to enable Istio in Rancher, refer to the [Istio section.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio)
@@ -3,15 +3,38 @@ title: Alerts
weight: 2
---
To keep your clusters and applications healthy and driving your organizational productivity forward, you need to stay informed of events occurring in your clusters and projects, both planned and unplanned. To help you stay informed of these events, you can configure alerts.
To keep your clusters and applications healthy and driving your organizational productivity forward, you need to stay informed of events occurring in your clusters and projects, both planned and unplanned. When an event occurs, your alert is triggered, and you are sent a notification. You can then, if necessary, follow up with corrective actions.
Alerts are sets of rules, chosen by you, to monitor for specific events.
Notifiers and alerts are built on top of the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/alertmanager/). Leveraging these tools, Rancher can notify [cluster owners]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles) and [project owners]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) of events they need to address.
Before you can receive alerts, you must configure one or more notifier in Rancher.
When you create a cluster, some alert rules are predefined. You can receive these alerts if you configure a [notifier]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/notifiers) for them.
For details about what triggers the predefined alerts, refer to the [documentation on default alerts.]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/alerts/default-alerts)
## Alerts Scope
This section covers the following topics:
- [Alert event examples](#alert-event-examples)
- [Urgency levels](#urgency-levels)
- [Scope of alerts](#scope-of-alerts)
- [Adding cluster alerts](#adding-cluster-alerts)
- [Managing cluster alerts](#managing-cluster-alerts)
# Alert Event Examples
Some examples of alert events are:
- A Kubernetes [master component]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#kubernetes-cluster-node-components) entering an unhealthy state.
- A node or [workload]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/workloads/) error occurring.
- A scheduled deployment taking place as planned.
- A node's hardware resources becoming overstressed.
# Urgency Levels
You can set an urgency level for each alert. This urgency appears in the notification you receive, helping you to prioritize your response actions. For example, if you have an alert configured to inform you of a routine deployment, no action is required. These alerts can be assigned a low priority level. However, if a deployment fails, it can critically impact your organization, and you need to react quickly. Assign these alerts a high priority level.
# Scope of Alerts
The scope for alerts can be set at either the cluster level or [project level]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/alerts/).
@@ -22,7 +45,7 @@ At the cluster level, Rancher monitors components in your Kubernetes cluster, an
- The resource events from specific system services.
- The Prometheus expression cross the thresholds
## Adding Cluster Alerts
# Adding Cluster Alerts
As a [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), you can configure Rancher to send you alerts for cluster events.
@@ -202,7 +225,7 @@ This alert type monitors for the overload from Prometheus expression querying, i
**Result:** Your alert is configured. A notification is sent when the alert is triggered.
## Managing Cluster Alerts
# Managing Cluster Alerts
After you set up cluster alerts, you can manage each alert object. To manage alerts, browse to the cluster containing the alerts, and then select **Tools > Alerts** that you want to manage. You can:
@@ -210,4 +233,4 @@ After you set up cluster alerts, you can manage each alert object. To manage ale
- Edit alert settings
- Delete unnecessary alerts
- Mute firing alerts
- Unmute muted alerts
- Unmute muted alerts
@@ -30,7 +30,7 @@ Rancher's Istio integration comes with comprehensive visualization aids:
- **Trace the root cause of errors with Jaeger.** [Jaeger](https://www.jaegertracing.io/) is an open-source tool that provides a UI for a distributed tracing system, which is useful for root cause analysis and for determining what causes poor performance. Distributed tracing allows you to view an entire chain of calls, which might originate with a user request and traverse dozens of microservices.
- **Get the full picture of your microservice architecture with Kiali.** [Kiali](https://www.kiali.io/) provides a diagram that shows the services within a service mesh and how they are connected, including the traffic rates and latencies between them. You can check the health of the service mesh, or drill down to see the incoming and outgoing requests to a single component.
- **Gain insights from time series analytics with Grafana dashboards.** [Grafana](https://grafana.com/) is an analytics platform that allows you to query, visualize, alert on and understand the data gathered by Prometheus.
- **Write custom queries for time series data with the Promethus UI.** [Prometheus](https://prometheus.io/) is a systems monitoring and alerting toolkit. Prometheus scrapes data from your cluster, which is then used by Grafana. A Prometheus UI is also integrated into Rancher, and lets you write custom queries for time series data and see the results in the UI.
- **Write custom queries for time series data with the Prometheus UI.** [Prometheus](https://prometheus.io/) is a systems monitoring and alerting toolkit. Prometheus scrapes data from your cluster, which is then used by Grafana. A Prometheus UI is also integrated into Rancher, and lets you write custom queries for time series data and see the results in the UI.
# Prerequisites
@@ -42,7 +42,7 @@ Refer to the [setup guide]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/ist
# Disabling Istio
To remove Istio components from a cluster, namepace, or workload, refer to the section on [disabling Istio.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/disabling-istio)
To remove Istio components from a cluster, namespace, or workload, refer to the section on [disabling Istio.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/disabling-istio)
# Accessing Visualizations
@@ -7,7 +7,7 @@ This section describes the permissions required to access Istio features and how
# Cluster-level Access
By default, only cluster adminstrators can:
By default, only cluster administrators can:
- Enable Istio for the cluster
- Configure resource allocations for Istio
@@ -33,7 +33,7 @@ By default, the Kiali and Jaeger visualizations are restricted to the cluster o
Rancher supports giving groups permission to access Kiali and Jaeger, but not individuals.
To configure who has permission to access the Kiali and Jaeger UI,
To configure who has permission to access the Kiali and Jaeger UI,
1. Go to the cluster view and click **Tools > Istio.**
1. Then go to the **Member Access** section. If you want to restrict access to certain groups, choose **Allow cluster owner and specified members to access Kiali and Jaeger UI.** Search for the groups that you want to have access to Kiali and Jaeger. If you want all members to have access to the tools, click **Allow all members to access Kiali and Jaeger UI.**
@@ -50,7 +50,7 @@ To configure the resources allocated to an Istio component,
## Pilot
[Pilot](https://istio.io/docs/concepts/what-is-istio/#pilot) provides the following:
[Pilot](https://istio.io/docs/ops/deployment/architecture/#pilot) provides the following:
- Authentication configuration
- Service discovery for the Envoy sidecars
@@ -70,7 +70,7 @@ Pilot Selector | Ability to select the nodes in which istio-pilot pod is deploye
## Mixer
[Mixer](https://istio.io/docs/concepts/what-is-istio/#mixer) enforces access control and usage policies across the service mesh. It also integrates with plugins for monitoring tools such as Prometheus. The Envoy sidecar proxy passes telemetry data and monitoring data to Mixer, and Mixer passes the monitoring data to Prometheus.
[Mixer](https://istio.io/docs/ops/deployment/architecture/#mixer) enforces access control and usage policies across the service mesh. It also integrates with plugins for monitoring tools such as Prometheus. The Envoy sidecar proxy passes telemetry data and monitoring data to Mixer, and Mixer passes the monitoring data to Prometheus.
For more information on Mixer, policies and telemetry, refer to the [documentation](https://istio.io/docs/concepts/policies-and-telemetry/).
@@ -149,4 +149,4 @@ Enable Persistent Storage for Grafana | Enable Persistent Storage for Grafana |
Source | Use a Storage Class to provision a new persistent volume or Use an existing persistent volume claim | Yes, when Grafana enabled and enabled PV | Use SC
Storage Class | Storage Class for provisioning PV for Grafana | Yes, when Grafana enabled, enabled PV and use storage class | Use the default class
Persistent Volume Size | The size for the PV you would like to provision for Grafana | Yes, when Grafana enabled, enabled PV and use storage class | 5Gi
Existing Claim | Use existing PVC for Grafana | Yes, when Grafana enabled, enabled PV and use existing PVC | n/a
Existing Claim | Use existing PVC for Grafana | Yes, when Grafana enabled, enabled PV and use existing PVC | n/a
@@ -9,7 +9,7 @@ A Rancher [administrator]({{<baseurl>}}/rancher/v2.x/en/admin-settings/rbac/glob
1. From the **Global** view, navigate to the **cluster** where you want to enable Istio.
1. Click **Tools > Istio.**
1. Optional: Configure member access and [resource limits]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio.
1. Optional: Configure member access and [resource limits]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/resources/) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio.
1. Click **Enable**.
1. Click **Save**.
@@ -1,12 +1,20 @@
---
title: Logging
title: Rancher Integration with Logging Services
description: Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster.
metaDescription: "Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster."
weight: 3
aliases:
- /rancher/v2.x/en/tasks/logging/
- /rancher/v2.x/en/tools/logging/
---
Rancher can integrate with a variety of popular logging services and tools that exist outside of your Kubernetes clusters.
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debug and troubleshoot problems
Rancher supports integration with the following services:
@@ -16,9 +24,26 @@ Rancher supports integration with the following services:
- Syslog
- Fluentd
This section covers the following topics:
- [How logging integrations work](#how-logging-integrations-work)
- [Requirements](#requirements)
- [Logging scope](#logging-scope)
- [Enabling cluster logging](#enabling-cluster-logging)
# How Logging Integrations Work
Rancher can integrate with popular external services used for event streams, telemetry, or search. These services can log errors and warnings in your Kubernetes infrastructure to a stream.
These services collect container log events, which are saved to the `/var/log/containers` directory on each of your nodes. The service collects both standard and error events. You can then log into your services to review the events collected, leveraging each service's unique features.
When configuring Rancher to integrate with these services, you'll have to point Rancher toward the service's endpoint and provide authentication information.
Additionally, you'll have the opportunity to enter key-value pairs to filter the log events collected. The service will only collect events for containers marked with your configured key-value pairs.
>**Note:** You can only configure one logging service per cluster or per project.
## Requirements
# Requirements
The Docker daemon on each node in the cluster should be [configured](https://docs.docker.com/config/containers/logging/configure/) with the (default) log-driver: `json-file`. You can check the log-driver by running the following command:
@@ -27,16 +52,7 @@ $ docker info | grep 'Logging Driver'
Logging Driver: json-file
```
## Advantages
Setting up a logging service to collect logs from your cluster/project has several advantages:
- Logs errors and warnings in your Kubernetes infrastructure to a stream. The stream informs you of events like a container crashing, a pod eviction, or a node dying.
- Allows you to capture and analyze the state of your cluster and look for trends in your environment using the log stream.
- Helps you when troubleshooting or debugging.
- Saves your logs to a safe location outside of your cluster, so that you can still access them even if your cluster encounters issues.
## Logging Scope
# Logging Scope
You can configure logging at either cluster level or project level.
@@ -48,7 +64,7 @@ Logs that are sent to your logging service are from the following locations:
- Pod logs stored at `/var/log/containers`.
- Kubernetes system components logs stored at `/var/lib/rancher/rke/log/`.
## Enabling Cluster Logging
# Enabling Cluster Logging
As an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), you can configure Rancher to send Kubernetes logs to a logging service.
@@ -84,7 +100,7 @@ As an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global
```
openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 365 -nodes -subj "/CN=myservice.example.com"
```
2. If you are using a self-signed certificate, provide the **CA Certificate PEM**.
2. If you are using a self-signed certificate, provide the **CA Certificate PEM**.
1. (Optional) Complete the **Additional Logging Configuration** form.
@@ -1,17 +1,37 @@
---
title: Monitoring
title: Integrating Rancher and Prometheus for Cluster Monitoring
description: Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Learn about the scope of monitoring and how to enable cluster monitoring
weight: 4
---
_Available as of v2.2.0_
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. Prometheus provides a _time series_ of your data, which is, according to [Prometheus documentation](https://prometheus.io/docs/concepts/data_model/):
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
This section covers the following topics:
- [About Prometheus](#about-prometheus)
- [Monitoring scope](#monitoring-scope)
- [Enabling cluster monitoring](#enabling-cluster-monitoring)
- [Resource consumption](#resource-consumption)
- [Resource consumption of Prometheus pods](#resource-consumption-of-prometheus-pods)
- [Resource consumption of other pods](#resources-consumption-of-other-pods)
# About Prometheus
Prometheus provides a _time series_ of your data, which is, according to [Prometheus documentation](https://prometheus.io/docs/concepts/data_model/):
You can configure these services to collect logs at either the cluster level or the project level. This page describes how to enable monitoring for a cluster. For details on enabling monitoring for a project, refer to the [project administration section]({{<baseurl>}}/rancher/v2.x/en/project-admin/tools/monitoring/).
>A stream of timestamped values belonging to the same metric and the same set of labeled dimensions, along with comprehensive statistics and metrics of the monitored cluster.
In other words, Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or [Grafana](https://grafana.com/), which is an analytics viewing platform deployed along with Prometheus. By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc. Multi-tenancy support in terms of cluster and project-only Prometheus instances are also supported.
In other words, Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or [Grafana](https://grafana.com/), which is an analytics viewing platform deployed along with Prometheus.
## Monitoring Scope
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, restore crashed servers, etc.
Multi-tenancy support in terms of cluster-only and project-only Prometheus instances are also supported.
# Monitoring Scope
Using Prometheus, you can monitor Rancher at both the cluster level and [project level]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/monitoring/). For each cluster and project that is enabled for monitoring, Rancher deploys a Prometheus server.
@@ -23,7 +43,7 @@ Using Prometheus, you can monitor Rancher at both the cluster level and [project
- [Project monitoring]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/monitoring/) allows you to view the state of pods running in a given project. Prometheus collects metrics from the project's deployed HTTP and TCP/UDP workloads.
## Enabling Cluster Monitoring
# Enabling Cluster Monitoring
As an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), you can configure Rancher to deploy Prometheus to monitor your Kubernetes cluster.
@@ -35,13 +55,13 @@ As an [administrator]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global
1. Click **Save**.
**Result:** The Prometheus server will be deployed as well as two monitoring applications. The two monitoring applications, `cluster-monitoring` and `monitoring-operator`, are added as an [application]({{< baseurl >}}/rancher/v2.x/en/catalog/apps/) to the cluster's `system` project. After the applications are `active`, you can start viewing [cluster metrics]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/monitoring/cluster-metrics/) through the [Rancher dashboard]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/monitoring/viewing-metrics/#rancher-dashboard) or directly from [Grafana]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/monitoring/#grafana).
**Result:** The Prometheus server will be deployed as well as two monitoring applications. The two monitoring applications, `cluster-monitoring` and `monitoring-operator`, are added as an [application]({{< baseurl >}}/rancher/v2.x/en/catalog/apps/) to the cluster's `system` project. After the applications are `active`, you can start viewing [cluster metrics]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/monitoring/cluster-metrics/) through the [Rancher dashboard]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/monitoring/viewing-metrics/#rancher-dashboard) or directly from [Grafana]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/monitoring/#grafana).
### Resource Consumption
# Resource Consumption
When enabling cluster monitoring, you need to ensure your worker nodes and Prometheus pod have enough resources. The tables below provides a guide of how much resource consumption will be used. In larger deployments, it is strongly advised that the monitoring infrastructure be placed on dedicated nodes in the cluster.
#### Prometheus Pod Resource Consumption
### Resource Consumption of Prometheus Pods
This table is the resource consumption of the Prometheus pod, which is based on the number of all the nodes in the cluster. The count of nodes includes the worker, control plane and etcd nodes. Total disk space allocation should be approximated by the `rate * retention` period set at the cluster level. When enabling cluster level monitoring, you should adjust the CPU and Memory limits and reservation.
@@ -69,7 +89,7 @@ Additional pod resource requirements for cluster level monitoring.
| Operator | prometheus-operator | 100m | 50Mi | 200m | 100Mi | Y |
#### Other Pods Resource Consumption
### Resource Consumption of Other Pods
Besides the Prometheus pod, there are components that are deployed that require additional resources on the worker nodes.
@@ -78,4 +98,4 @@ Pod | CPU (milli CPU) | Memory (MB)
Node Exporter (Per Node) | 100 | 30
Kube State Cluster Monitor | 100 | 130
Grafana | 100 | 150
Prometheus Cluster Monitoring Nginx | 50 | 50
Prometheus Cluster Monitoring Nginx | 50 | 50
@@ -238,8 +238,8 @@ weight: 4
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"}) by (instance)`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"}) by (instance)`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"}) by (instance)`</td></tr><tr><td>accpeted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m]))) by (instance)`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m]))) by (instance)`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m]))) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"})`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"})`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"})`</td></tr><tr><td>accpeted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m])))`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m])))`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m])))`</td></tr></table> |
| Detail | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"}) by (instance)`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"}) by (instance)`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"}) by (instance)`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m]))) by (instance)`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m]))) by (instance)`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m]))) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"})`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"})`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"})`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m])))`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m])))`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m])))`</td></tr></table> |
- **Ingress Controller Request Process Time**
@@ -0,0 +1,24 @@
---
title: Upgrading Kubernetes
weight: 70
---
> **Prerequisite:** The options below are available only for clusters that are [launched using RKE.]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/)
Following an upgrade to the latest version of Rancher, you can update your existing clusters to use the latest supported version of Kubernetes.
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For example, Rancher v2.3.0 is was tested with Kubernetes v1.15.4, v1.14.7, and v1.13.11. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
As of Rancher v2.3.0, the Kubernetes metadata feature was added, which allows Rancher to ship Kubernetes patch versions without upgrading Rancher. For details, refer to the [section on Kubernetes metadata.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/k8s-metadata)
>**Recommended:** Before upgrading Kubernetes, [backup your cluster]({{< baseurl >}}/rancher/v2.x/en/backups).
1. From the **Global** view, find the cluster for which you want to upgrade Kubernetes. Select **Vertical Ellipsis (...) > Edit**.
1. Expand **Cluster Options**.
1. From the **Kubernetes Version** drop-down, choose the version of Kubernetes that you want to use for the cluster.
1. Click **Save**.
**Result:** Kubernetes begins upgrading for the cluster. During the upgrade, your cluster is unavailable.
@@ -1,5 +1,6 @@
---
title: Volumes and Storage
title: "Kubernetes Persistent Storage: Volumes and Storage Classes"
description: "Learn about the two ways with which you can create persistent storage in Kubernetes: persistent volumes and storage classes"
weight: 2031
aliases:
- /rancher/v2.x/en/concepts/volumes-and-storage/
@@ -42,7 +42,7 @@ In order to provision vSphere volumes in a cluster created with the [Rancher Kub
7. Assign a path in the **Mount Point** field. This is the full path where the volume will be mounted in the container file system, e.g. `/persistent`.
8. Click **Launch** to create the workload.
### Verifing Persistence of the Volume
### Verifying Persistence of the Volume
1. From the context menu of the workload you just created, click **Execute Shell**.
2. Note the directory at root where the volume has been mounted to (in this case `/persistent`).
@@ -50,7 +50,7 @@ In order to provision vSphere volumes in a cluster created with the [Rancher Kub
4. **Close** the shell window.
5. Click on the name of the workload to reveal detail information.
6. Open the context menu next to the Pod in the *Running* state.
7. Delete the Pod by selecting **Delete**.
7. Delete the Pod by selecting **Delete**.
8. Observe that the pod is deleted. Then a new pod is scheduled to replace it so that the workload maintains its configured scale of a single stateful pod.
9. Once the replacement pod is running, click **Execute Shell**.
10. Inspect the contents of the directory where the volume is mounted by entering `ls -l /<volumeMountPoint>`. Note that the file you created earlier is still present.
@@ -1,5 +1,6 @@
---
title: Provisioning Kubernetes Clusters
description: Provisioning Kubernetes Clusters
weight: 2000
aliases:
- /rancher/v2.x/en/concepts/clusters/
@@ -50,7 +50,7 @@ Huawei CCE service doesn't support the ability to create clusters with public ac
| Cluster Label | The labels for the cluster. |
| Highway Subnet | This option is only supported in `BareMetal` type. It requires you to select a VPC with high network speed for the bare metal machines. |
> **Note:** If you are editing the cluster in the `cluster.yml` instead of the Rancher UI, note that as of Rancher v2.3.0, cluster configuration directives must be nested under the `rancher_kubernetes_engine_config` directive in `cluster.yml`. For more information, refer to the section on [the config file structure in Rancher v2.3.0+.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file-structure-in-rancher-v2-3-0)
**Note:** If you are editing the cluster in the `cluster.yml` instead of the Rancher UI, note that as of Rancher v2.3.0, cluster configuration directives must be nested under the `rancher_kubernetes_engine_config` directive in `cluster.yml`. For more information, refer to the section on [the config file structure in Rancher v2.3.0+.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file-structure-in-rancher-v2-3-0)
7. Fill the following node configuration of the cluster:
@@ -48,7 +48,7 @@ You can use Rancher to create a cluster hosted in Tencent Kubernetes Engine (TKE
| VPC | Select the VPC name that you have created in the Tencent Cloud Console. |
| Container Network CIDR | Enter the CIDR range of your Kubernetes cluster, you may check the available range of the CIDR in the VPC service of the Tencent Cloud Console. Default to 172.16.0.0/16. |
> Note: If you are editing the cluster in the `cluster.yml` instead of the Rancher UI, note that as of Rancher v2.3.0, cluster configuration directives must be nested under the `rancher_kubernetes_engine_config` directive in `cluster.yml`. For more information, refer to the section on [the config file structure in Rancher v2.3.0+.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file-structure-in-rancher-v2-3-0)
**Note:** If you are editing the cluster in the `cluster.yml` instead of the Rancher UI, note that as of Rancher v2.3.0, cluster configuration directives must be nested under the `rancher_kubernetes_engine_config` directive in `cluster.yml`. For more information, refer to the section on [the config file structure in Rancher v2.3.0+.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file-structure-in-rancher-v2-3-0)
7. Click `Next: Select Instance Type` to choose the instance type that will use for your TKE cluster.
@@ -1,5 +1,8 @@
---
title: Importing Kubernetes Clusters
title: Import an Existing Cluster to Create a Cluster in Rancher
description: Learn how you can create a cluster in Rancher by importing an existing Kubernetes cluster. Then, you can manage it using Rancher
metaTitle: "Kubernetes Cluster Management"
metaDescription: "Learn how you can import an existing Kubernetes cluster and then manage it using Rancher"
weight: 2300
aliases:
- /rancher/v2.x/en/tasks/clusters/import-cluster/
@@ -23,9 +23,9 @@ Rancher works has been tested and is supported with downstream clusters running
All supported operating systems are 64-bit x86.
If you plan to use ARM64, see [Running on ARM64 (Experimental).]({{<baseurl>}}/rancher/v2.x/en/installation/options/arm64-platform/)
If you plan to use ARM64, see [Running on ARM64 (Experimental).]({{<baseurl>}}/rancher/v2.x/en/installation/options/arm64-platform/)
For information on how to install Docker, refer to the offical [Docker documentation.](https://docs.docker.com/)
For information on how to install Docker, refer to the official [Docker documentation.](https://docs.docker.com/)
> **Note:** Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. This [how-to guide]({{<baseurl>}}/rancher/v2.x/en/installation/options/firewall) shows how to check the default firewall rules and how to open the ports with `firewalld` if necessary.
@@ -41,7 +41,7 @@ Windows nodes can be used for worker nodes only. See [Configuring Custom Cluster
The hardware requirements for nodes with the `worker` role mostly depend on your workloads. The minimum to run the Kubernetes node components is 1 CPU (core) and 1GB of memory.
Regarding CPU and memory, it is recommended that the different planes of Kubernetes clusters (etcd, controplane, and workers) should be hosted on different nodes so that they can scale separately from each other.
Regarding CPU and memory, it is recommended that the different planes of Kubernetes clusters (etcd, controlplane, and workers) should be hosted on different nodes so that they can scale separately from each other.
For hardware recommendations for large Kubernetes clusters, refer to the official Kubernetes documentation on [building large clusters.](https://kubernetes.io/docs/setup/best-practices/cluster-large/)
@@ -106,7 +106,7 @@ The following table depicts the port requirements for [Rancher Launched Kubernet
### Port Requirements for Clusters Hosted by an Infrastructure Provider
If you are launching a Kubernetes cluster on nodes that are in an infastructure provider such as Amazon EC2, Google Container Engine, DigitalOcean, Azure, or vSphere, these port requirements apply.
If you are launching a Kubernetes cluster on nodes that are in an infrastructure provider such as Amazon EC2, Google Container Engine, DigitalOcean, Azure, or vSphere, these port requirements apply.
These required ports are automatically opened by Rancher during creation of clusters using cloud providers.
@@ -32,7 +32,8 @@ For a full list of all the best practices that we recommend, refer to the [best
For more information on what each role is used for, refer to the [section on roles for nodes in Kubernetes.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/production/nodes-and-roles)
For more information about the recommended number of nodes for each Kubernetes role, refer to the [section on recommended architecture.]({{<baseurl>}}/rancher/v2.x/encluster-provisioning/recommended-architecture)
For more information about the
number of nodes for each Kubernetes role, refer to the section on [recommended architecture.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture-recommendations/)
### Logging and Monitoring
@@ -1,5 +1,7 @@
---
title: Creating a Cluster with Custom Nodes
description: To create a cluster with custom nodes, youll need to access servers in your cluster and provision them according to Rancher requirements
metaDescription: "To create a cluster with custom nodes, youll need to access servers in your cluster and provision them according to Rancher requirements"
shortTitle: Custom Nodes
weight: 2225
aliases:
@@ -1,6 +1,7 @@
---
title: Creating an Amazon EC2 Cluster
shortTitle: Amazon EC2
description: Learn the prerequisites and steps required in order for you to create an Amazon EC2 cluster using Rancher
weight: 2210
aliases:
- /rancher/v2.x/en/tasks/clusters/creating-a-cluster/create-cluster-amazon-ec2/
@@ -13,7 +14,7 @@ Use {{< product >}} to create a Kubernetes cluster in Amazon EC2.
- IAM Policy created to add to the user of the Access Key And Secret Key. See [Amazon Documentation: Creating IAM Policies (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start) how to create an IAM policy. See our three example JSON policies below:
- [Example IAM Policy](#example-iam-policy)
- [Example IAM Policy with PassRole](#example-iam-policy-with-passrole) (needed if you want to use [Kubernetes Cloud Provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers) or want to pass an IAM Profile to an instance)
- [Example IAM Policy to allow encrypted EBS volumes](#example-iam-policy-to-allow-encrypted-ebs-volumes)
- [Example IAM Policy to allow encrypted EBS volumes](#example-iam-policy-to-allow-encrypted-ebs-volumes)
- IAM Policy added as Permission to the user. See [Amazon Documentation: Adding Permissions to a User (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) how to attach it to an user.
@@ -35,7 +36,7 @@ Use {{< product >}} to create a Kubernetes cluster in Amazon EC2.
1. Complete each of the following forms using information available from the [EC2 Management Console](https://aws.amazon.com/ec2).
- **Account Access** is where you configure the region of the nodes, and the credentials (Access Key and Secret Key) used to create the machine. See [Prerequisistes](#prerequisites) how to create the Access Key and Secret Key and the needed permissions.
- **Account Access** is where you configure the region of the nodes, and the credentials (Access Key and Secret Key) used to create the machine. See [Prerequisites](#prerequisites) how to create the Access Key and Secret Key and the needed permissions.
{{< step_create-cloud-credential >}}
@@ -199,4 +200,4 @@ Use {{< product >}} to create a Kubernetes cluster in Amazon EC2.
}
]
}
```
```
@@ -1,6 +1,8 @@
---
title: Creating a vSphere Cluster
shortTitle: vSphere
description: Use Rancher to create a vSphere cluster. It may consist of groups of VMs with distinct properties which allow for fine-grained control over the sizing of nodes.
metaDescription: Use Rancher to create a vSphere cluster. It may consist of groups of VMs with distinct properties which allow for fine-grained control over the sizing of nodes.
weight: 2225
aliases:
- /rancher/v2.x/en/tasks/clusters/creating-a-cluster/create-cluster-vsphere/
@@ -1,7 +1,7 @@
---
title: Cluster Options
weight: 2250
---
---
As you configure a new cluster that's provisioned using [RKE]({{< baseurl >}}/rke/latest/en/), you can choose custom Kubernetes options.
@@ -60,12 +60,12 @@ _Available as of v2.2.0_
The registry configuration here is applied during the provisioning of the cluster. This option tells Rancher where to pull the [system images]({{<baseurl>}}/rke/latest/en/config-options/system-images/) or [addon images.]({{<baseurl>}}/rke/latest/en/config-options/add-ons/)
- **System images** are components needed to maintain the Kubernetes cluster.
- **System images** are components needed to maintain the Kubernetes cluster.
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
To deploy workloads that pull images from a private registry, you will need to [set up your own Kubernetes registry]({{<baseurl>}}/rancher/v2.x/en/k8s-in-rancher/registries/) for your project.
To deploy workloads that pull images from a private registry, you will need to set up your own Kubernetes registry for your project.
See the [RKE documentation on private registries]({{< baseurl >}}/rke/latest/en/config-options/private-registries/) for more information on the private registry for components applied during the provisioning of the cluster.
See the RKE documentation on private registries for more information on the private registry for components applied during the provisioning of the cluster.
### Authorized Cluster Endpoint
@@ -77,7 +77,7 @@ Authorized Cluster Endpoint can be used to directly access the Kubernetes API se
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/4-authorized-cluster-endpoint)
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/#4-authorized-cluster-endpoint)
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.]({{<baseurl>}}/rancher/v2.x/en/overview/architecture-recommendations/#architecture-for-an-authorized-cluster-endpoint)
@@ -109,7 +109,7 @@ If the nodes you are adding to the cluster have Docker configured with a non-def
#### Recurring etcd Snapshots
Option to enable or disable [recurring etcd snaphots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
Option to enable or disable [recurring etcd snapshots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
## Config File
@@ -131,63 +131,63 @@ RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kuber
{{% accordion id="v2.3.0-cluster-config-file" label="Example Cluster Config File for Rancher v2.3.0+" %}}
```yaml
#
#
# Cluster Config
#
#
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
enable_cluster_monitoring: false
enable_network_policy: false
local_cluster_auth_endpoint:
enabled: true
#
#
# Rancher Config
#
#
rancher_kubernetes_engine_config: # Your RKE template config goes here.
addon_job_timeout: 30
authentication:
strategy: x509
ignore_docker_version: true
#
#
# # Currently only nginx ingress provider is supported.
# # To disable ingress controller, set `provider: none`
# # To enable ingress on specific nodes, use the node_selector, eg:
# provider: nginx
# node_selector:
# app: ingress
#
#
ingress:
provider: nginx
kubernetes_version: v1.15.3-rancher3-1
monitoring:
provider: metrics-server
#
#
# If you are using calico on AWS
#
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
#
# # To specify flannel interface
#
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
#
# # To specify flannel interface for canal plugin
#
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
#
network:
options:
flannel_backend_type: vxlan
plugin: canal
#
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
@@ -197,7 +197,7 @@ rancher_kubernetes_engine_config: # Your RKE template config goes here.
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
#
services:
etcd:
backup_config:
@@ -232,46 +232,46 @@ addon_job_timeout: 30
authentication:
strategy: x509
ignore_docker_version: true
#
#
# # Currently only nginx ingress provider is supported.
# # To disable ingress controller, set `provider: none`
# # To enable ingress on specific nodes, use the node_selector, eg:
# provider: nginx
# node_selector:
# app: ingress
#
#
ingress:
provider: nginx
kubernetes_version: v1.15.3-rancher3-1
monitoring:
provider: metrics-server
#
#
# If you are using calico on AWS
#
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
#
# # To specify flannel interface
#
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
#
# # To specify flannel interface for canal plugin
#
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
#
network:
options:
flannel_backend_type: vxlan
plugin: canal
#
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
@@ -281,7 +281,7 @@ network:
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
#
services:
etcd:
backup_config:
@@ -5,7 +5,7 @@ aliases:
- /rancher/v2.x/en/faq/contributing/
---
This section explains the respositories used for Rancher, how to build the repositories, and what information to include when you file an issue.
This section explains the repositories used for Rancher, how to build the repositories, and what information to include when you file an issue.
For more detailed information on how to contribute to the development of Rancher projects, refer to the [Rancher Developer Wiki](https://github.com/rancher/rancher/wiki). The wiki has resources on many topics, including the following:
@@ -1,5 +1,6 @@
---
title: CNI Providers
title: Container Network Interface (CNI) Providers
description: Learn about Container Network Interface (CNI), the CNI providers Rancher provides, the features they offer, and how to choose a provider for you
weight: 2300
---
@@ -126,7 +127,7 @@ The following table summarizes the different features available for each CNI net
- External Datastore: CNI network providers with this feature need an external datastore for its data.
- Encyption: This feature allows cyphered and secure network control and data planes.
- Encryption: This feature allows cyphered and secure network control and data planes.
- Ingress/Egress Policies: This feature allows you to manage routing control for both Kubernetes and non-Kubernetes communications.
@@ -1,5 +1,6 @@
---
title: Installation
title: Installing Rancher
description: Learn how to install Rancher in development and production environments. Read about single node and high availability installation
weight: 50
---
@@ -9,16 +10,16 @@ Before installing Rancher, make sure that your nodes fulfill all of the [install
### Overview of Installation Options
We recommend using [Helm,]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/concepts/#about-helm) a Kubernetes package manager, to install Rancher on a dedicated Kubernetes cluster. This is called a high-availability (HA) installation because increased avaialability is achieved by running Rancher on multiple nodes.
We recommend using [Helm,]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/concepts/#about-helm) a Kubernetes package manager, to install Rancher on a dedicated Kubernetes cluster. This is called a high-availability (HA) installation because increased availability is achieved by running Rancher on multiple nodes.
For testing and demonstration purposes, Rancher can also be installed with Docker on a single node. However, there is no migration path from a single-node Docker installation to an HA installation on a Kubernetes cluster. Therefore, you may want to use an HA installation from the start.
There are also separate instructions for installing Rancher in an air gap environment or behind an HTTP proxy:
| Level of Internet Access | Installing on a Kubernetes Cluster - Strongly Recommended | Installing in a Single Docker Container |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| With direct access to the Internet | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node) |
| Behind an HTTP proxy | These [docs,]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node) plus this [configuration]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-nodeproxy/) | These [docs,]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) plus this [configuration]({{<baseurl>}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#http-proxy) |
| Level of Internet Access | Installing on a Kubernetes Cluster - Strongly Recommended | Installing in a Single Docker Container |
| ---------------------------------- | ------------------------------ | ---------- |
| With direct access to the Internet | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node) |
| Behind an HTTP proxy | These [docs,]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) plus this [configuration]({{<baseurl>}}/rancher/v2.x/en/installation/options/chart-options/#http-proxy) | These [docs,]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node) plus this [configuration]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/single-node/proxy/) |
| In an air gap environment | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/air-gap) | [Docs]({{<baseurl>}}/rancher/v2.x/en/installation/other-installation-methods/air-gap) |
> For the best performance and greater security, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads.
@@ -1,6 +1,7 @@
---
title: How to Install Rancher on a High-availability Kubernetes Cluster
weight: 3
description: For production environments, install Rancher in a high-availability configuration. Read the guide for setting up a 3-node cluster and still install Rancher using a Helm chart.
---
For production environments, we recommend installing Rancher in a high-availability configuration so that your user base can always access Rancher Server. When installed in a Kubernetes cluster, Rancher will integrate with the cluster's etcd database and take advantage of Kubernetes scheduling for high-availability.
@@ -45,7 +46,7 @@ The following CLI tools are required for this install. Please make sure these to
## Previous Methods
[RKE add-on install]({{< baseurl >}}/rancher/v2.x/en/installation/ha/rke-add-on/)
[RKE add-on install]({{<baseurl>}}/rancher/v2.x/en/installation/options/rke-add-on/)
> **Important: RKE add-on install is only supported up to Rancher v2.0.8**
>
@@ -22,7 +22,7 @@ After installing NGINX, you need to update the NGINX configuration file, `nginx.
1. Copy and paste the code sample below into your favorite text editor. Save it as `nginx.conf`.
2. From `nginx.conf`, replace both occurences (port 80 and port 443) of `<IP_NODE_1>`, `<IP_NODE_2>`, and `<IP_NODE_3>` with the IPs of your [nodes]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/).
2. From `nginx.conf`, replace both occurrences (port 80 and port 443) of `<IP_NODE_1>`, `<IP_NODE_2>`, and `<IP_NODE_3>` with the IPs of your [nodes]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/).
> **Note:** See [NGINX Documentation: TCP and UDP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/) for all configuration options.
@@ -1,5 +1,6 @@
---
title: '3. Install Rancher'
title: 3. Installing Rancher Using the Helm Kubernetes Package Manager
description: Rancher installation is managed using the Helm Kubernetes package manager. Use Helm to install the prerequisites and charts to install Rancher
weight: 200
---
@@ -1,5 +1,6 @@
---
title: '2. Install Kubernetes with RKE'
title: 2. Install Kubernetes with RKE
description: Learn how to use Rancher Kubernetes Engine (RKE) to install Kubernetes with a high availability etcd configuration.
weight: 190
---
@@ -33,6 +34,13 @@ services:
snapshot: true
creation: 6h
retention: 24h
# Required for external TLS termination with
# ingress-nginx v0.22+
ingress:
provider: nginx
options:
use-forwarded-headers: "true"
```
#### Common RKE Nodes Options
@@ -3,7 +3,7 @@ title: About High-availability Installations
weight: 2
---
We recommend using [Helm,]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/concepts/#about-helm) a Kubernetes package manager, to install Rancher on a dedicated Kubernetes cluster. This is called a high-availability (HA) installation because increased avaialability is achieved by running Rancher on multiple nodes.
We recommend using [Helm,]({{<baseurl>}}/rancher/v2.x/en/overview/architecture/concepts/#about-helm) a Kubernetes package manager, to install Rancher on a dedicated Kubernetes cluster. This is called a high-availability (HA) installation because increased availability is achieved by running Rancher on multiple nodes.
In a typical HA Rancher installation, Kubernetes is first installed on three nodes that are hosted in an infrastructure provider such as Amazon's EC2 or Google Compute Engine.
@@ -192,7 +192,7 @@ This NGINX configuration is tested on NGINX 1.14.
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
- Replace `IP_NODE1`, `IP_NODE2` and `IP_NODE3` with the IP addresses of the nodes in your cluster.
- Replace both occurences of `FQDN` to the DNS name for Rancher.
- Replace both occurrences of `FQDN` to the DNS name for Rancher.
- Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
@@ -25,7 +25,7 @@ You can follow the recommendations from [the etcd docs](https://etcd.io/docs/v3.
Additionally, to reduce IO contention on the disks for etcd, you can use a dedicated device for the data and wal directory. Based on etcd best practices, mirroring RAID configurations are unnecessary because etcd replicates data between the nodes in the cluster. You can use stripping RAID configurations to increase available IOPS.
To implement this solution in an RKE cluster, the `/var/lib/etcd/data` and `/var/lib/etc/wal` directories will need to have disks mounted and formmated on the underlying host. In the `extra_args` directive of the `etcd` service, you must include the `wal_dir` directory. Without specifying the `wal_dir`, etcd process will try to manipulate the underlying `wal` mount with insufficient permissions.
To implement this solution in an RKE cluster, the `/var/lib/etcd/data` and `/var/lib/etc/wal` directories will need to have disks mounted and formatted on the underlying host. In the `extra_args` directive of the `etcd` service, you must include the `wal_dir` directory. Without specifying the `wal_dir`, etcd process will try to manipulate the underlying `wal` mount with insufficient permissions.
```yaml
# RKE cluster.yml
@@ -1,11 +1,13 @@
---
title: Enabling Experimental Features
weight: 8000
aliases:
- /rancher/v2.x/en/admin-settings/feature-flags
---
_Available as of v2.3.0_
Rancher includes some features that are experimental and disabled by default. You might want to enable these features, for example, if you decide that the benefits of using an [unsupported storage type]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers) outweighs the risk of using an untested feature. Feature flags were introduced to allow you to try these features that are not enabled by default.
Rancher includes some features that are experimental and disabled by default. You might want to enable these features, for example, if you decide that the benefits of using an [unsupported storage type]({{<baseurl>}}/rancher/v2.x/en/installation/options/feature-flags/enable-not-default-storage-drivers) outweighs the risk of using an untested feature. Feature flags were introduced to allow you to try these features that are not enabled by default.
The features can be enabled in three ways:
@@ -26,8 +28,8 @@ For example, if you install Rancher, then set a feature flag to true with the Ra
The following is a list of the feature flags available in Rancher:
- `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers) In other words, it enables types for storage providers and provisioners that are not enabled by default.
- `istio-virtual-service-ui`: This feature enables a [UI to create, read, update, and delete Istio virtual services and destination rules]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags/istio-virtual-service-ui), which are traffic management features of Istio.
- `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{<baseurl>}}/rancher/v2.x/en/installation/options/feature-flags/able-not-default-storage-drivers) In other words, it enables types for storage providers and provisioners that are not enabled by default.
- `istio-virtual-service-ui`: This feature enables a [UI to create, read, update, and delete Istio virtual services and destination rules]({{<baseurl>}}/rancher/v2.x/en/installation/options/feature-flags/istio-virtual-service-ui), which are traffic management features of Istio.
The below table shows the availability and default value for feature flags in Rancher:
@@ -1,12 +1,14 @@
---
title: Allow Unsupported Storage Drivers
weight: 1
aliases:
- /rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers
---
_Available as of v2.3.0_
This feature allows you to use types for storage providers and provisioners that are not enabled by default.
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags)
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.]({{<baseurl>}}/rancher/v2.x/en/installation/options/feature-flags/)
Environment Variable Key | Default Value | Description
---|---|---
@@ -1,6 +1,8 @@
---
title: UI for Istio Virtual Services and Destination Rules
weight: 2
aliases:
- /rancher/v2.x/en/admin-settings/feature-flags/istio-virtual-service-ui
---
_Available as of v2.3.0_
@@ -8,7 +10,7 @@ This feature enables a UI that lets you create, read, update and delete virtual
> **Prerequisite:** Turning on this feature does not enable Istio. A cluster administrator needs to [enable Istio for the cluster]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/tools/istio/setup) in order to use the feature.
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.]({{<baseurl>}}/rancher/v2.x/en/admin-settings/feature-flags)
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.]({{<baseurl>}}/rancher/v2.x/en/installation/options/feature-flags/)
Environment Variable Key | Default Value | Status | Available as of
---|---|---|---
@@ -38,10 +38,10 @@ The following CLI tools are required for this install. Please make sure these to
## Installation Outline
- [Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/)
- [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/)
- [Initialize Helm (tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
- [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
- [Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/)
- [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/)
- [Initialize Helm (tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-init/)
- [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/)
## Additional Install Options
@@ -49,10 +49,10 @@ The following CLI tools are required for this install. Please make sure these to
## Previous Methods
[RKE add-on install]({{< baseurl >}}/rancher/v2.x/en/installation/ha/rke-add-on/)
[RKE add-on install]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/)
> **Important: RKE add-on install is only supported up to Rancher v2.0.8**
>
> Please use the Rancher helm chart to install HA Rancher. For details, see the [HA Install - Installation Outline]({{< baseurl >}}/rancher/v2.x/en/installation/ha/#installation-outline).
> Please use the Rancher helm chart to install HA Rancher. For details, see the [HA Install - Installation Outline]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/#installation-outline).
>
> If you are currently using the RKE add-on install method, see [Migrating from an HA RKE Add-on Install]({{< baseurl >}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart.
@@ -24,7 +24,7 @@ Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configurat
#### Examples
* [Nginx]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/nginx/)
* [Amazon NLB]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/nlb/)
* [Nginx]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nginx/)
* [Amazon NLB]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nlb/)
### [Next: Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/)
### [Next: Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/)
@@ -21,7 +21,7 @@ After installing NGINX, you need to update the NGINX configuration file, `nginx.
1. Copy and paste the code sample below into your favorite text editor. Save it as `nginx.conf`.
2. From `nginx.conf`, replace both occurences (port 80 and port 443) of `<IP_NODE_1>`, `<IP_NODE_2>`, and `<IP_NODE_3>` with the IPs of your [nodes]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/).
2. From `nginx.conf`, replace both occurrences (port 80 and port 443) of `<IP_NODE_1>`, `<IP_NODE_2>`, and `<IP_NODE_3>` with the IPs of your [nodes]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/).
>**Note:** See [NGINX Documentation: TCP and UDP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/) for all configuration options.
@@ -1,5 +1,6 @@
---
title: "3. Initialize Helm (Install Tiller)"
title: "Initialize Helm: Install the Tiller Service"
description: "With Helm, you can create configurable deployments instead of using static files. In order to use Helm, the Tiller service needs to be installed on your cluster."
weight: 195
---
@@ -60,6 +61,6 @@ Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b
### Issues or errors?
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/) page.
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-init/troubleshooting/) page.
### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/)
### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/)
@@ -20,4 +20,4 @@ helm version --server
Error: could not find tiller
```
When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) to install `tiller` with the correct `ServiceAccount`.
When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-init/) to install `tiller` with the correct `ServiceAccount`.
@@ -27,7 +27,7 @@ Rancher Server is designed to be secure by default and requires SSL/TLS configur
There are three recommended options for the source of the certificate.
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#external-tls-termination).
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#external-tls-termination).
| Configuration | Chart option | Description | Requires cert-manager |
|-----|-----|-----|-----|
@@ -37,7 +37,7 @@ There are three recommended options for the source of the certificate.
### Optional: Install cert-manager
**Note:** cert-manager is only required for certificates issued by Rancher's generated CA (`ingress.tls.source=rancher`) and Let's Encrypt issued certificates (`ingress.tls.source=letsEncrypt`). You should skip this step if you are using your own certificate files (option `ingress.tls.source=secret`) or if you use [TLS termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#external-tls-termination).
**Note:** cert-manager is only required for certificates issued by Rancher's generated CA (`ingress.tls.source=rancher`) and Let's Encrypt issued certificates (`ingress.tls.source=letsEncrypt`). You should skip this step if you are using your own certificate files (option `ingress.tls.source=secret`) or if you use [TLS termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#external-tls-termination).
> **Important:**
> Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher.
@@ -164,7 +164,7 @@ helm install rancher-<CHART_REPO>/rancher \
--set ingress.tls.source=secret
```
Now that Rancher is deployed, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them.
Now that Rancher is deployed, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them.
After adding the secrets, check if Rancher was rolled out successfully:
@@ -188,11 +188,11 @@ It should show the same count for `DESIRED` and `AVAILABLE`.
The Rancher chart configuration has many options for customizing the install to suit your specific environment. Here are some common advanced scenarios.
* [HTTP Proxy]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#http-proxy)
* [Private Docker Image Registry]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#private-registry-and-air-gap-installs)
* [TLS Termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#external-tls-termination)
* [HTTP Proxy]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#http-proxy)
* [Private Docker Image Registry]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#private-registry-and-air-gap-installs)
* [TLS Termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#external-tls-termination)
See the [Chart Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/) for the full list of options.
See the [Chart Options]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/) for the full list of options.
### Save your options
@@ -202,4 +202,4 @@ Make sure you save the `--set` options you used. You will need to use the same o
That's it you should have a functional Rancher server. Point a browser at the hostname you picked and you should be greeted by the colorful login page.
Doesn't work? Take a look at the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/troubleshooting/) Page
Doesn't work? Take a look at the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/troubleshooting/) Page
@@ -152,7 +152,7 @@ We recommend configuring your load balancer as a Layer 4 balancer, forwarding pl
You may terminate the SSL/TLS on a L7 load balancer external to the Rancher cluster (ingress). Use the `--set tls=external` option and point your load balancer at port http 80 on all of the Rancher cluster nodes. This will expose the Rancher interface on http port 80. Be aware that clients that are allowed to connect directly to the Rancher cluster will not be encrypted. If you choose to do this we recommend that you restrict direct access at the network level to just your load balancer.
> **Note:** If you are using a Private CA signed certificate, add `--set privateCA=true` and see [Adding TLS Secrets - Using a Private CA Signed Certificate]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/#using-a-private-ca-signed-certificate) to add the CA cert for Rancher.
> **Note:** If you are using a Private CA signed certificate, add `--set privateCA=true` and see [Adding TLS Secrets - Using a Private CA Signed Certificate]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/tls-secrets/#using-a-private-ca-signed-certificate) to add the CA cert for Rancher.
Your load balancer must support long lived websocket connections and will need to insert proxy headers so Rancher can route links correctly.
@@ -192,7 +192,7 @@ This NGINX configuration is tested on NGINX 1.14.
>**Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
* Replace `IP_NODE1`, `IP_NODE2` and `IP_NODE3` with the IP addresses of the nodes in your cluster.
* Replace both occurences of `FQDN` to the DNS name for Rancher.
* Replace both occurrences of `FQDN` to the DNS name for Rancher.
* Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
@@ -1,5 +1,6 @@
---
title: Adding TLS Secrets
title: Adding Kubernetes TLS Secrets
description: Read about how to populate the Kubernetes TLS secret for a Rancher installation
weight: 276
---
@@ -124,10 +124,10 @@ W0705 23:04:58.240571 7 backend_ssl.go:49] error obtaining PEM from secret
### no matches for kind "Issuer"
The [SSL configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) option you have chosen requires [cert-manager]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#optional-install-cert-manager) to be installed before installing Rancher or else the following error is shown:
The [SSL configuration]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#choose-your-ssl-configuration) option you have chosen requires [cert-manager]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#optional-install-cert-manager) to be installed before installing Rancher or else the following error is shown:
```
Error: validation failed: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1"
```
Install [cert-manager]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#optional-install-cert-manager) and try installing Rancher again.
Install [cert-manager]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#optional-install-cert-manager) and try installing Rancher again.
@@ -125,6 +125,6 @@ Save a copy of the following files in a secure location:
### Issues or errors?
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/troubleshooting/) page.
See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/troubleshooting/) page.
### [Next: Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/)
### [Next: Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/options/helm2/helm-init/)

Some files were not shown because too many files have changed in this diff Show More