mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-25 04:28:15 +00:00
Merge branch 'master' into PNI-Imported
This commit is contained in:
Jen Travinski
@@ -0,0 +1,39 @@
|
||||
# Configuration for probot-stale - https://github.com/probot/stale
|
||||
|
||||
# Number of days of inactivity before an Issue or Pull Request becomes stale
|
||||
daysUntilStale: 90
|
||||
|
||||
# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
|
||||
# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
|
||||
daysUntilClose: 30
|
||||
|
||||
# Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled)
|
||||
onlyLabels: []
|
||||
|
||||
# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable
|
||||
exemptLabels: []
|
||||
|
||||
# Set to true to ignore issues in a project (defaults to false)
|
||||
exemptProjects: true
|
||||
|
||||
# Set to true to ignore issues in a milestone (defaults to false)
|
||||
exemptMilestones: true
|
||||
|
||||
# Set to true to ignore issues with an assignee (defaults to false)
|
||||
exemptAssignees: true
|
||||
|
||||
# Label to use when marking as stale
|
||||
staleLabel: status/stale
|
||||
|
||||
# Comment to post when marking as stale. Set to `false` to disable
|
||||
markComment: >
|
||||
This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label)
|
||||
for 90 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the
|
||||
issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the
|
||||
latest release), the bot will automatically close the issue in 30 days. Thank you for your contributions.
|
||||
|
||||
# Limit the number of actions per hour, from 1-30. Default is 30
|
||||
limitPerRun: 30
|
||||
|
||||
# Limit to only `issues`
|
||||
only: issues
|
||||
@@ -11,3 +11,5 @@ package-lock.json
|
||||
/scripts/converters/results_to_markdown/.terraform
|
||||
/scripts/converters/results_to_markdown/terraform.tfstate*
|
||||
/scripts/converters/results_to_markdown/*.tfvars
|
||||
|
||||
.idea/
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
FROM node:8-alpine
|
||||
FROM node:16-alpine
|
||||
|
||||
RUN apk update && apk add py-pygments bash git asciidoc gcompat && rm -rf /var/cache/apk/*
|
||||
|
||||
|
||||
+1
-1
@@ -209,4 +209,4 @@ pre = "<i class='material-icons'>keyboard_arrow_down</i>"
|
||||
[[menu.main]]
|
||||
name = "Partners"
|
||||
url = "https://rancher.com/partners/"
|
||||
parent = "about"
|
||||
parent = "about"
|
||||
|
||||
@@ -26,7 +26,7 @@ K3s is a fully compliant Kubernetes distribution with the following enhancements
|
||||
* Secure by default with reasonable defaults for lightweight environments.
|
||||
* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a Helm controller, and the Traefik ingress controller.
|
||||
* Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates.
|
||||
* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including:
|
||||
* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages the required dependencies, including:
|
||||
* containerd
|
||||
* Flannel
|
||||
* CoreDNS
|
||||
@@ -38,4 +38,4 @@ K3s is a fully compliant Kubernetes distribution with the following enhancements
|
||||
|
||||
# What's with the name?
|
||||
|
||||
We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
|
||||
We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
|
||||
|
||||
@@ -13,18 +13,18 @@ This section contains advanced information describing the different ways you can
|
||||
- [Using Docker as the container runtime](#using-docker-as-the-container-runtime)
|
||||
- [Using etcdctl](#using-etcdctl)
|
||||
- [Configuring containerd](#configuring-containerd)
|
||||
- [Secrets Encryption Config (Experimental)](#secrets-encryption-config-experimental)
|
||||
- [Running K3s with Rootless mode (Experimental)](#running-k3s-with-rootless-mode-experimental)
|
||||
- [Node labels and taints](#node-labels-and-taints)
|
||||
- [Starting the server with the installation script](#starting-the-server-with-the-installation-script)
|
||||
- [Additional preparation for Alpine Linux setup](#additional-preparation-for-alpine-linux-setup)
|
||||
- [Additional preparation for (Red Hat/CentOS) Enterprise Linux](#additional-preparation-for-red-hat/centos-enterprise-linux)
|
||||
- [Additional preparation for Raspberry Pi OS Setup](#additional-preparation-for-raspberry-pi-os-setup)
|
||||
- [Enabling vxlan on Ubuntu 21.10+ on Raspberry Pi](#enabling-vxlan-on-ubuntu-21.10+-on-raspberry-pi)
|
||||
- [Running K3d (K3s in Docker) and docker-compose](#running-k3d-k3s-in-docker-and-docker-compose)
|
||||
- [Enabling legacy iptables on Raspbian Buster](#enabling-legacy-iptables-on-raspbian-buster)
|
||||
- [Enabling cgroups for Raspbian Buster](#enabling-cgroups-for-raspbian-buster)
|
||||
- [SELinux Support](#selinux-support)
|
||||
- [Additional preparation for (Red Hat/CentOS) Enterprise Linux](#additional-preparation-for-red-hat-centos-enterprise-linux)
|
||||
- [Enabling Lazy Pulling of eStargz (Experimental)](#enabling-lazy-pulling-of-estargz-experimental)
|
||||
- [Additional Logging Sources](#additional-logging-sources)
|
||||
- [Server and agent tokens](#server-and-agent-tokens)
|
||||
|
||||
# Certificate Rotation
|
||||
|
||||
@@ -127,8 +127,8 @@ If you would like to use etcdctl after installing K3s with embedded etcd, instal
|
||||
|
||||
```
|
||||
$ VERSION="v3.5.0"
|
||||
$ curl -L https://github.com/etcd-io/etcd/releases/download/${VERSION}/etcd-${VERSION}-linux-amd64.tar.gz --output etcdctl-${VERSION}-linux-amd64.tar.gz
|
||||
$ sudo tar -zxvf etcdctl-${VERSION}-linux-amd64.tar.gz -C /usr/local/bin
|
||||
$ curl -L https://github.com/etcd-io/etcd/releases/download/${VERSION}/etcd-${VERSION}-linux-amd64.tar.gz --output etcdctl-linux-amd64.tar.gz
|
||||
$ sudo tar -zxvf etcdctl-linux-amd64.tar.gz --strip-components=1 -C /usr/local/bin etcd-${VERSION}-linux-amd64/etcdctl
|
||||
```
|
||||
|
||||
Then start using etcdctl commands with the appropriate K3s flags:
|
||||
@@ -143,46 +143,8 @@ K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/
|
||||
|
||||
For advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
|
||||
|
||||
The `config.toml.tmpl` will be treated as a Go template file, and the `config.Node` structure is being passed to the template. [This template](https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32) example on how to use the structure to customize the configuration file.
|
||||
The `config.toml.tmpl` will be treated as a Go template file, and the `config.Node` structure is being passed to the template. See [this folder](https://github.com/k3s-io/k3s/blob/master/pkg/agent/templates) for Linux and Windows examples on how to use the structure to customize the configuration file.
|
||||
|
||||
# Secrets Encryption Config (Experimental)
|
||||
As of v1.17.4+k3s1, K3s added the experimental feature of enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server, this flag will do the following automatically:
|
||||
|
||||
- Generate an AES-CBC key
|
||||
- Generate an encryption config file with the generated key
|
||||
|
||||
```
|
||||
{
|
||||
"kind": "EncryptionConfiguration",
|
||||
"apiVersion": "apiserver.config.k8s.io/v1",
|
||||
"resources": [
|
||||
{
|
||||
"resources": [
|
||||
"secrets"
|
||||
],
|
||||
"providers": [
|
||||
{
|
||||
"aescbc": {
|
||||
"keys": [
|
||||
{
|
||||
"name": "aescbckey",
|
||||
"secret": "xxxxxxxxxxxxxxxxxxx"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"identity": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- Pass the config to the KubeAPI as encryption-provider-config
|
||||
|
||||
Once enabled any created secret will be encrypted with this key. Note that if you disable encryption then any encrypted secrets will not be readable until you enable encryption again.
|
||||
|
||||
# Running K3s with Rootless mode (Experimental)
|
||||
|
||||
@@ -294,6 +256,39 @@ Then update the config and reboot:
|
||||
update-extlinux
|
||||
reboot
|
||||
```
|
||||
# Additional preparation for (Red Hat/CentOS) Enterprise Linux
|
||||
|
||||
It is recommended to turn off firewalld:
|
||||
```
|
||||
systemctl disable firewalld --now
|
||||
```
|
||||
|
||||
If enabled, it is required to disable nm-cloud-setup and reboot the node:
|
||||
```
|
||||
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
|
||||
reboot
|
||||
```
|
||||
|
||||
# Additional preparation for Raspberry Pi OS Setup
|
||||
## Enabling legacy iptables on Raspberry Pi OS
|
||||
Raspberry Pi OS (formerly Raspbian) defaults to using `nftables` instead of `iptables`. **K3S** networking features require `iptables` and do not work with `nftables`. Follow the steps below to switch configure **Buster** to use `legacy iptables`:
|
||||
```
|
||||
sudo iptables -F
|
||||
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
|
||||
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
## Enabling cgroups for Raspberry Pi OS
|
||||
|
||||
Standard Raspberry Pi OS installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`.
|
||||
|
||||
# Enabling vxlan on Ubuntu 21.10+ on Raspberry Pi
|
||||
|
||||
Starting with Ubuntu 21.10, vxlan support on Raspberry Pi has been moved into a seperate kernel module.
|
||||
```
|
||||
sudo apt install linux-modules-extra-raspi
|
||||
```
|
||||
|
||||
# Running K3d (K3s in Docker) and docker-compose
|
||||
|
||||
@@ -331,20 +326,6 @@ Alternatively the `docker run` command can also be used:
|
||||
--privileged rancher/k3s:vX.Y.Z
|
||||
|
||||
|
||||
# Enabling legacy iptables on Raspbian Buster
|
||||
|
||||
Raspbian Buster defaults to using `nftables` instead of `iptables`. **K3S** networking features require `iptables` and do not work with `nftables`. Follow the steps below to switch configure **Buster** to use `legacy iptables`:
|
||||
```
|
||||
sudo iptables -F
|
||||
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
|
||||
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
# Enabling cgroups for Raspbian Buster
|
||||
|
||||
Standard Raspbian Buster installations do not start with `cgroups` enabled. **K3S** needs `cgroups` to start the systemd service. `cgroups`can be enabled by appending `cgroup_memory=1 cgroup_enable=memory` to `/boot/cmdline.txt`.
|
||||
|
||||
### example of /boot/cmdline.txt
|
||||
```
|
||||
console=serial0,115200 console=tty1 root=PARTUUID=58b06195-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_memory=1 cgroup_enable=memory
|
||||
@@ -403,19 +384,6 @@ Using a custom `--data-dir` under SELinux is not supported. To customize it, you
|
||||
{{%/tab%}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# Additional preparation for (Red Hat/CentOS) Enterprise Linux
|
||||
|
||||
It is recommended to turn off firewalld:
|
||||
```
|
||||
systemctl disable firewalld --now
|
||||
```
|
||||
|
||||
If enabled, it is required to disable nm-cloud-setup and reboot the node:
|
||||
```
|
||||
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
|
||||
reboot
|
||||
```
|
||||
|
||||
# Enabling Lazy Pulling of eStargz (Experimental)
|
||||
|
||||
### What's lazy pulling and eStargz?
|
||||
@@ -479,4 +447,14 @@ helm repo add rancher-charts https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install --create-namespace -n cattle-logging-system rancher-logging-crd rancher-charts/rancher-logging-crd
|
||||
helm install --create-namespace -n cattle-logging-system rancher-logging --set additionalLoggingSources.k3s.enabled=true rancher-charts/rancher-logging
|
||||
```
|
||||
```
|
||||
|
||||
# Server and agent tokens
|
||||
|
||||
In K3s, there are two types of tokens: K3S_TOKEN and K3S_AGENT_TOKEN.
|
||||
|
||||
K3S_TOKEN: Defines the key required by the server to offer the HTTP config resources. These resources are requested by the other servers before joining the K3s HA cluster. If the K3S_AGENT_TOKEN is not defined, the agents use this token as well to access the required HTTP resources to join the cluster. Note that this token is also used to generate the encryption key for important content in the database (e.g., bootstrap data).
|
||||
|
||||
K3S_AGENT_TOKEN: Optional. Defines the key required by the server to offer HTTP config resources to the agents. If not defined, agents will require K3S_TOKEN. Defining K3S_AGENT_TOKEN is encouraged to avoid agents having to know K3S_TOKEN, which is also used to encrypt data.
|
||||
|
||||
If no K3S_TOKEN is defined, the first K3s server will generate a random one. The result is part of the content in `/var/lib/rancher/k3s/server/token`. For example, `K1070878408e06a827960208f84ed18b65fa10f27864e71a57d9e053c4caff8504b::server:df54383b5659b9280aa1e73e60ef78fc`, where `df54383b5659b9280aa1e73e60ef78fc` is the K3S_TOKEN.
|
||||
|
||||
@@ -18,7 +18,7 @@ For details on taking database snapshots and restoring your database from them,
|
||||
|
||||
- [Official MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-snapshot-method.html)
|
||||
- [Official PostgreSQL documentation](https://www.postgresql.org/docs/8.3/backup-dump.html)
|
||||
- [Official etcd documentation](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/recovery.md)
|
||||
- [Official etcd documentation](https://etcd.io/docs/latest/op-guide/recovery/)
|
||||
|
||||
# Backup and Restore with Embedded etcd Datastore (Experimental)
|
||||
|
||||
@@ -57,7 +57,7 @@ These options can be passed in with the command line, or in the [configuration f
|
||||
| Options | Description |
|
||||
| ----------- | --------------- |
|
||||
| `--etcd-disable-snapshots` | Disable automatic etcd snapshots |
|
||||
| `--etcd-snapshot-schedule-cron` value | Snapshot interval time in cron spec. eg. every 5 hours `* */5 * * *`(default: `0 */12 * * *`) |
|
||||
| `--etcd-snapshot-schedule-cron` value | Snapshot interval time in cron spec. eg. every 5 hours `0 */5 * * *`(default: `0 */12 * * *`) |
|
||||
| `--etcd-snapshot-retention` value | Number of snapshots to retain (default: 5) |
|
||||
| `--etcd-snapshot-dir` value | Directory to save db snapshots. (Default location: `${data-dir}/db/snapshots`) |
|
||||
| `--cluster-reset` | Forget all peers and become sole member of a new cluster. This can also be set with the environment variable `[$K3S_CLUSTER_RESET]`.
|
||||
|
||||
@@ -96,8 +96,9 @@ metadata:
|
||||
namespace: kube-system
|
||||
spec:
|
||||
valuesContent: |-
|
||||
image: traefik
|
||||
imageTag: v1.7.26-alpine
|
||||
image:
|
||||
name: traefik
|
||||
tag: v2.6.1
|
||||
proxyProtocol:
|
||||
enabled: true
|
||||
trustedIPs:
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "Air-Gap Install"
|
||||
weight: 60
|
||||
---
|
||||
|
||||
You can install K3s in an air-gapped environment using two different methods. You can either deploy a private registry and mirror docker.io or you can manually deploy images such as for small clusters.
|
||||
You can install K3s in an air-gapped environment using two different methods. An air-gapped environment is any environment that is not directly connected to the Internet. You can either deploy a private registry and mirror docker.io, or you can manually deploy images such as for small clusters.
|
||||
|
||||
# Private Registry Method
|
||||
|
||||
@@ -39,22 +39,23 @@ Follow the steps in the next section to install K3s.
|
||||
|
||||
# Install K3s
|
||||
|
||||
Only after you have completed either the [Private Registry Method](#private-registry-method) or the [Manually Deploy Images Method](#manually-deploy-images-method) above should you install K3s.
|
||||
### Prerequisites
|
||||
|
||||
Obtain the K3s binary from the [releases](https://github.com/rancher/k3s/releases) page, matching the same version used to get the airgap images.
|
||||
Obtain the K3s install script at https://get.k3s.io
|
||||
- Before installing K3s, complete the the [Private Registry Method](#private-registry-method) or the [Manually Deploy Images Method](#manually-deploy-images-method) above to prepopulate the images that K3s needs to install.
|
||||
- Download the K3s binary from the [releases](https://github.com/rancher/k3s/releases) page, matching the same version used to get the airgap images. Place the binary in `/usr/local/bin` on each air-gapped node and ensure it is executable.
|
||||
- Download the K3s install script at https://get.k3s.io. Place the install script anywhere on each air-gapped node, and name it `install.sh`.
|
||||
|
||||
Place the binary in `/usr/local/bin` on each node and ensure it is executable.
|
||||
Place the install script anywhere on each node, and name it `install.sh`.
|
||||
When running the K3s script with the `INSTALL_K3S_SKIP_DOWNLOAD` environment variable, K3s will use the local version of the script and binary.
|
||||
|
||||
|
||||
### Install Options
|
||||
### Installing K3s in an Air-Gapped Environment
|
||||
|
||||
You can install K3s on one or more servers as described below.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Single Server Configuration" %}}
|
||||
|
||||
To install K3s on a single server simply do the following on the server node.
|
||||
To install K3s on a single server, simply do the following on the server node:
|
||||
|
||||
```
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
|
||||
@@ -75,13 +76,13 @@ For example, step two of the High Availability with an External DB guide mention
|
||||
|
||||
```
|
||||
curl -sfL https://get.k3s.io | sh -s - server \
|
||||
--datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name"
|
||||
--datastore-endpoint='mysql://username:password@tcp(hostname:3306)/database-name'
|
||||
```
|
||||
|
||||
Instead, you would modify such examples like below:
|
||||
|
||||
```
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name"' ./install.sh
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_EXEC='server' K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/database-name' ./install.sh
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
@@ -23,7 +23,7 @@ If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you
|
||||
|
||||
CLI Flag | Environment Variable | Description
|
||||
------------|-------------|------------------
|
||||
<span style="white-space: nowrap">`--datastore-endpoint`</span> | `K3S_DATASTORE_ENDPOINT` | Specify a PostgresSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below.
|
||||
<span style="white-space: nowrap">`--datastore-endpoint`</span> | `K3S_DATASTORE_ENDPOINT` | Specify a PostgreSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below.
|
||||
<span style="white-space: nowrap">`--datastore-cafile`</span> | `K3S_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate. |
|
||||
| <span style="white-space: nowrap">`--datastore-certfile`</span> | `K3S_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. |
|
||||
| <span style="white-space: nowrap">`--datastore-keyfile`</span> | `K3S_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. |
|
||||
@@ -81,7 +81,7 @@ The above assumes a typical three node etcd cluster. The parameter can accept on
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
<br/>Based on the above, the following example command could be used to launch a server instance that connects to a PostgresSQL database named k3s:
|
||||
<br/>Based on the above, the following example command could be used to launch a server instance that connects to a PostgreSQL database named k3s:
|
||||
```
|
||||
K3S_DATASTORE_ENDPOINT='postgres://username:password@hostname:5432/k3s' k3s server
|
||||
```
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "Disable Components Flags"
|
||||
weight: 60
|
||||
---
|
||||
|
||||
When starting K3s server with --cluster-init it will run all control plane components that includes (api server, controller manager, scheduler, and etcd). However you can run server nodes with certain components and execlude others, the following sections will explain how to do that.
|
||||
Starting the K3s server with `--cluster-init` will run all control plane components, including the api server, controller manager, scheduler, and etcd. However, you can run server nodes with certain components and exclude others; the following sections will explain how to do that.
|
||||
|
||||
# ETCD Only Nodes
|
||||
|
||||
|
||||
@@ -23,3 +23,14 @@ K3S_TOKEN=SECRET k3s server --server https://<ip or hostname of server1>:6443
|
||||
```
|
||||
|
||||
Now you have a highly available control plane. Joining additional worker nodes to the cluster follows the same procedure as a single server cluster.
|
||||
|
||||
There are a few config flags that must be the same in all server nodes:
|
||||
|
||||
* Network related flags: `--cluster-dns`, `--cluster-domain`, `--cluster-cidr`, `--service-cidr`
|
||||
* Flags controlling the deployment of certain components: `--disable-helm-controller`, `--disable-kube-proxy`, `--disable-network-policy` and any component passed to `--disable`
|
||||
* Feature related flags: `--secrets-encryption`
|
||||
|
||||
## Existing clusters
|
||||
If you have an existing cluster using the default embedded SQLite database, you can convert it to etcd by simply restarting your K3s server with the `--cluster-init` flag. Once you've done that, you'll be able to add additional instances as described above.
|
||||
|
||||
>**Important:** K3s v1.22.2 and newer support migration from SQLite to etcd. Older versions will create a new empty datastore if you add `--cluster-init` to an existing server.
|
||||
|
||||
@@ -33,12 +33,13 @@ You will first need to create an external datastore for the cluster. See the [Cl
|
||||
### 2. Launch Server Nodes
|
||||
K3s requires two or more server nodes for this HA configuration. See the [Installation Requirements]({{<baseurl>}}/k3s/latest/en/installation/installation-requirements/) guide for minimum machine requirements.
|
||||
|
||||
When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore.
|
||||
When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. The `token` parameter can also be used to set a deterministic token when adding nodes. When empty, this token will be generated automatically for further use.
|
||||
|
||||
For example, a command like the following could be used to install the K3s server with a MySQL database as the external datastore:
|
||||
For example, a command like the following could be used to install the K3s server with a MySQL database as the external datastore and [set a token]({{<baseurl>}}/k3s/latest/en/installation/install-options/server-config/#cluster-options):
|
||||
|
||||
```
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | sh -s - server \
|
||||
--token=SECRET \
|
||||
--datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name"
|
||||
```
|
||||
|
||||
@@ -62,12 +63,37 @@ Agent nodes need a URL to register against. This can be the IP or hostname of an
|
||||
|
||||
This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file to point to it instead of a specific node. To avoid certificate errors in such a configuration, you should install the server with the `--tls-san YOUR_IP_OR_HOSTNAME_HERE` option. This option adds an additional hostname or IP as a Subject Alternative Name in the TLS cert, and it can be specified multiple times if you would like to access via both the IP and the hostname.
|
||||
|
||||
### 4. Optional: Join Agent Nodes
|
||||
### 4. Optional: Join Additional Server Nodes
|
||||
|
||||
The same example command in Step 2 can be used to join additional server nodes, where the token from the first node needs to be used.
|
||||
|
||||
If the first server node was started without the `--token` CLI flag or `K3S_TOKEN` variable, the token value can be retrieved from any server already joined to the cluster:
|
||||
```bash
|
||||
cat /var/lib/rancher/k3s/server/token
|
||||
```
|
||||
|
||||
Additional server nodes can then be added [using the token]({{<baseurl>}}/k3s/latest/en/installation/install-options/server-config/#cluster-options):
|
||||
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | sh -s - server \
|
||||
--token=SECRET \
|
||||
--datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name"
|
||||
```
|
||||
|
||||
There are a few config flags that must be the same in all server nodes:
|
||||
|
||||
* Network related flags: `--cluster-dns`, `--cluster-domain`, `--cluster-cidr`, `--service-cidr`
|
||||
* Flags controlling the deployment of certain components: `--disable-helm-controller`, `--disable-kube-proxy`, `--disable-network-policy` and any component passed to `--disable`
|
||||
* Feature related flags: `--secrets-encryption`
|
||||
|
||||
> **Note:** Ensure that you retain a copy of this token as it is required when restoring from backup and adding nodes. Previously, K3s did not enforce the use of a token when using external SQL datastores.
|
||||
|
||||
### 5. Optional: Join Agent Nodes
|
||||
|
||||
Because K3s server nodes are schedulable by default, the minimum number of nodes for an HA K3s server cluster is two server nodes and zero agent nodes. To add nodes designated to run your apps and services, join agent nodes to your cluster.
|
||||
|
||||
Joining agent nodes in an HA cluster is the same as joining agent nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use.
|
||||
|
||||
```
|
||||
```bash
|
||||
K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443
|
||||
```
|
||||
|
||||
@@ -27,9 +27,23 @@ If this command is not specified as a server or agent command, it will default t
|
||||
The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior of registering a server without flannel:
|
||||
|
||||
```bash
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-flannel" sh -s -
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --no-flannel" sh -s -
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --no-flannel
|
||||
curl -sfL https://get.k3s.io | sh -s - server --no-flannel
|
||||
curl -sfL https://get.k3s.io | sh -s - --no-flannel
|
||||
```
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend none" sh -s -
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --flannel-backend none" sh -s -
|
||||
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --flannel-backend none
|
||||
curl -sfL https://get.k3s.io | sh -s - server --flannel-backend none
|
||||
curl -sfL https://get.k3s.io | sh -s - --flannel-backend none
|
||||
```
|
||||
|
||||
### Example C: CONFIG FILE
|
||||
|
||||
Before installing k3s, you can create a file called `config.yaml` containing fields that match CLI flags. That file needs to be in the path: `/etc/rancher/k3s/config.yaml` for k3s to consume it.
|
||||
|
||||
The fields in the config file drop the starting `--` from the matching CLI flag. For example:
|
||||
|
||||
```
|
||||
write-kubeconfig-mode: 644
|
||||
token: "secret"
|
||||
node-ip: 10.0.10.22,2a05:d012:c6f:4655:d73c:c825:a184:1b75
|
||||
cluster-cidr: 10.42.0.0/16,2001:cafe:42:0::/56
|
||||
service-cidr: 10.43.0.0/16,2001:cafe:42:1::/112
|
||||
```
|
||||
|
||||
@@ -152,6 +152,7 @@ the agent options are there because the server has the agent process embedded wi
|
||||
|
||||
| Flag | Description |
|
||||
|------|--------------|
|
||||
| `--etcd-arg` value | Customized flag for etcd process |
|
||||
| `--kube-apiserver-arg` value | Customized flag for kube-apiserver process |
|
||||
| `--kube-scheduler-arg` value | Customized flag for kube-scheduler process |
|
||||
| `--kube-controller-manager-arg` value | Customized flag for kube-controller-manager process |
|
||||
@@ -234,6 +235,7 @@ OPTIONS:
|
||||
--token-file value (cluster) File containing the cluster-secret/token [$K3S_TOKEN_FILE]
|
||||
--write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT]
|
||||
--write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE]
|
||||
--etcd-arg value (flags) Customized flag for etcd process
|
||||
--kube-apiserver-arg value (flags) Customized flag for kube-apiserver process
|
||||
--kube-scheduler-arg value (flags) Customized flag for kube-scheduler process
|
||||
--kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process
|
||||
|
||||
@@ -44,13 +44,14 @@ K3s performance depends on the performance of the database. To ensure optimal sp
|
||||
|
||||
The K3s server needs port 6443 to be accessible by all nodes.
|
||||
|
||||
The nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. However, if you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s.
|
||||
The nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used or over UDP ports 51820 and 51821 (when using IPv6) when Flannel Wireguard backend is used. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. However, if you do not use Flannel and provide your own custom CNI, then the ports needed by Flannel are not needed by K3s.
|
||||
|
||||
If you wish to utilize the metrics server, you will need to open port 10250 on each node.
|
||||
|
||||
If you plan on achieving high availability with embedded etcd, server nodes must be accessible to each other on ports 2379 and 2380.
|
||||
|
||||
> **Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472.
|
||||
> **Warning:** Flannel relies on the [Bridge CNI plugin](https://www.cni.dev/plugins/current/main/bridge/) to create a L2 network that switches traffic. Rogue pods with NET_RAW capabilities can abuse that L2 network to launch attacks such as [ARP spoofing](https://static.sched.com/hosted_files/kccncna19/72/ARP%20DNS%20spoof.pdf). Therefore, as documented in the [kubernetes docs](https://kubernetes.io/docs/concepts/security/pod-security-standards/), please set a restricted profile that disables NET_RAW on non-trustable pods.
|
||||
|
||||
<figcaption>Inbound Rules for K3s Server Nodes</figcaption>
|
||||
|
||||
@@ -58,6 +59,8 @@ If you plan on achieving high availability with embedded etcd, server nodes must
|
||||
|-----|-----|----------------|---|
|
||||
| TCP | 6443 | K3s agent nodes | Kubernetes API Server
|
||||
| UDP | 8472 | K3s server and agent nodes | Required only for Flannel VXLAN
|
||||
| UDP | 51820 | K3s server and agent nodes | Required only for Flannel Wireguard backend
|
||||
| UDP | 51821 | K3s server and agent nodes | Required only for Flannel Wireguard backend with IPv6
|
||||
| TCP | 10250 | K3s server and agent nodes | Kubelet metrics
|
||||
| TCP | 2379-2380 | K3s server nodes | Required only for HA with embedded etcd
|
||||
|
||||
|
||||
@@ -19,10 +19,11 @@ If you wish to use WireGuard as your flannel backend it may require additional k
|
||||
<span style="white-space: nowrap">`--flannel-backend=ipsec`</span> | Uses the IPSEC backend which encrypts network traffic. |
|
||||
<span style="white-space: nowrap">`--flannel-backend=host-gw`</span> | Uses the host-gw backend. |
|
||||
<span style="white-space: nowrap">`--flannel-backend=wireguard`</span> | Uses the WireGuard backend which encrypts network traffic. May require additional kernel modules and configuration. |
|
||||
<span style="white-space: nowrap">`--flannel-ipv6-masq`</span> | Apply masquerading rules to IPv6 traffic (default for IPv4). Only applies on dual-stack or IPv6-only clusters |
|
||||
|
||||
### Custom CNI
|
||||
|
||||
Run K3s with `--flannel-backend=none` and install your CNI of choice. IP Forwarding should be enabled for Canal and Calico. Please reference the steps below.
|
||||
Run K3s with `--flannel-backend=none` and install your CNI of choice. Most CNI plugins come with their own network policy engine, so it is recommended to set `--disable-network-policy` as well to avoid conflicts. IP Forwarding should be enabled for Canal and Calico. Please reference the steps below.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Canal" %}}
|
||||
@@ -74,15 +75,22 @@ You should see that IP forwarding is set to true.
|
||||
|
||||
Dual-stack networking must be configured when the cluster is first created. It cannot be enabled on an existing single-stack cluster.
|
||||
|
||||
To enable dual-stack in k3s, you must provide valid dual-stack `cluster-cidr` and `service-cidr`, and set `disable-network-policy` on all server nodes. Both servers and agents must provide valid dual-stack `node-ip` settings. Node address auto-detection and network policy enforcement are not supported on dual-stack clusters when using the default flannel CNI. Besides, only vxlan backend is supported at the moment. This is an example of a valid configuration:
|
||||
Dual-stack is supported on k3s v1.21 or above.
|
||||
|
||||
To enable dual-stack in K3s, you must provide valid dual-stack `cluster-cidr` and `service-cidr` on all server nodes. Both servers and agents must provide valid dual-stack `node-ip` settings. Node address auto-detection is not supported on dual-stack clusters, because kubelet fetches only the first IP address that it finds. Additionally, only vxlan backend is supported currently. This is an example of a valid configuration:
|
||||
|
||||
```
|
||||
node-ip: 10.0.10.7,2a05:d012:c6f:4611:5c2:5602:eed2:898c
|
||||
cluster-cidr: 10.42.0.0/16,2001:cafe:42:0::/56
|
||||
service-cidr: 10.43.0.0/16,2001:cafe:42:1::/112
|
||||
disable-network-policy: true
|
||||
k3s server --node-ip 10.0.10.7,2a05:d012:c6f:4611:5c2:5602:eed2:898c --cluster-cidr 10.42.0.0/16,2001:cafe:42:0::/56 --service-cidr 10.43.0.0/16,2001:cafe:42:1::/112
|
||||
```
|
||||
|
||||
Note that you can choose whatever `cluster-cidr` and `service-cidr` value, however the `node-ip` values must correspond to the ip addresses of your main interface. Remember to allow ipv6 traffic if you are deploying in a public cloud.
|
||||
|
||||
If you are using a custom cni plugin, i.e. a cni plugin different from flannel, the previous configuration might not be enough to enable dual-stack in the cni plugin. Please check how to enable dual-stack in its documentation and verify if network policies can be enabled.
|
||||
|
||||
### IPv6 only installation
|
||||
|
||||
IPv6 only setup is supported on k3s v1.22 or above. Note that network policy enforcement is not supported on IPv6-only clusters when using the default flannel CNI. This is an example of a valid configuration:
|
||||
|
||||
```
|
||||
k3s server --disable-network-policy
|
||||
```
|
||||
|
||||
@@ -32,22 +32,43 @@ mirrors:
|
||||
|
||||
Each mirror must have a name and set of endpoints. When pulling an image from a registry, containerd will try these endpoint URLs one by one, and use the first working one.
|
||||
|
||||
#### Rewrites
|
||||
|
||||
Each mirror can have a set of rewrites. Rewrites can change the tag of an image based on a regular expression. This is useful if the organization/project structure in the mirror registry is different to the upstream one.
|
||||
|
||||
For example, the following configuration would transparently pull the image `docker.io/rancher/coredns-coredns:1.6.3` from `registry.example.com:5000/mirrorproject/rancher-images/coredns-coredns:1.6.3`:
|
||||
|
||||
```
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoint:
|
||||
- "https://registry.example.com:5000"
|
||||
rewrite:
|
||||
"^rancher/(.*)": "mirrorproject/rancher-images/$1"
|
||||
```
|
||||
|
||||
The image will still be stored under the original name so that a `crictl image ls` will show `docker.io/rancher/coredns-coredns:1.6.3` as available on the node, even though the image was pulled from the mirrored registry with a different name.
|
||||
|
||||
### Configs
|
||||
|
||||
The configs section defines the TLS and credential configuration for each mirror. For each mirror you can define `auth` and/or `tls`. The TLS part consists of:
|
||||
The `configs` section defines the TLS and credential configuration for each mirror. For each mirror you can define `auth` and/or `tls`.
|
||||
|
||||
Directive | Description
|
||||
----------|------------
|
||||
`cert_file` | The client certificate path that will be used to authenticate with the registry
|
||||
`key_file` | The client key path that will be used to authenticate with the registry
|
||||
`ca_file` | Defines the CA certificate path to be used to verify the registry's server cert file
|
||||
`insecure_skip_verify` | Boolean that defines if TLS verification should be skipped for the registry
|
||||
The `tls` part consists of:
|
||||
|
||||
The credentials consist of either username/password or authentication token:
|
||||
| Directive | Description |
|
||||
|------------------------|--------------------------------------------------------------------------------------|
|
||||
| `cert_file` | The client certificate path that will be used to authenticate with the registry |
|
||||
| `key_file` | The client key path that will be used to authenticate with the registry |
|
||||
| `ca_file` | Defines the CA certificate path to be used to verify the registry's server cert file |
|
||||
| `insecure_skip_verify` | Boolean that defines if TLS verification should be skipped for the registry |
|
||||
|
||||
- username: user name of the private registry basic auth
|
||||
- password: user password of the private registry basic auth
|
||||
- auth: authentication token of the private registry basic auth
|
||||
The `auth` part consists of either username/password or authentication token:
|
||||
|
||||
| Directive | Description |
|
||||
|------------|---------------------------------------------------------|
|
||||
| `username` | user name of the private registry basic auth |
|
||||
| `password` | user password of the private registry basic auth |
|
||||
| `auth` | authentication token of the private registry basic auth |
|
||||
|
||||
Below are basic examples of using private registries in different modes:
|
||||
|
||||
@@ -142,4 +163,4 @@ Example: `docker tag coredns-coredns:1.6.3 mycustomreg:5000/coredns-coredns`
|
||||
|
||||
Last, push the images to the private registry.
|
||||
|
||||
Example: `docker push mycustomreg:5000/coredns-coredns`
|
||||
Example: `docker push mycustomreg.com:5000/coredns-coredns`
|
||||
|
||||
@@ -42,7 +42,7 @@ To migrate from an older Traefik v1 instance please refer to the [Traefik docume
|
||||
|
||||
# Service Load Balancer
|
||||
|
||||
Any service load balancer (LB) can be leveraged in your Kubernetes cluster. K3s provides a load balancer known as [Klipper Load Balancer](https://github.com/rancher/klipper-lb) that uses available host ports.
|
||||
Any service load balancer (LB) can be leveraged in your Kubernetes cluster. K3s provides a load balancer known as [Klipper Load Balancer](https://github.com/k3s-io/klipper-lb) that uses available host ports.
|
||||
|
||||
Upstream Kubernetes allows a Service of type LoadBalancer to be created, but doesn't include the implementation of the LB. Some LB services require a cloud provider such as Amazon EC2 or Microsoft Azure. By contrast, the K3s service LB makes it possible to use an LB service without a cloud provider.
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ weight: 90
|
||||
|
||||
This section describes the methodology and means of securing a K3s cluster. It's broken into 2 sections. These guides assume k3s is running with embedded etcd.
|
||||
|
||||
The documents below apply to both CIS 1.5 & 1.6.
|
||||
The documents below apply to CIS Kubernetes Benchmark v1.6.
|
||||
|
||||
* [Hardening Guide](./hardening_guide/)
|
||||
* [CIS Benchmark Self-Assessment Guide](./self_assessment/)
|
||||
|
||||
@@ -3,12 +3,12 @@ title: "CIS Hardening Guide"
|
||||
weight: 80
|
||||
---
|
||||
|
||||
This document provides prescriptive guidance for hardening a production installation of K3s. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
This document provides prescriptive guidance for hardening a production installation of K3s. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Internet Security (CIS).
|
||||
|
||||
K3s has a number of security mitigations applied and turned on by default and will pass a number of the Kubernetes CIS controls without modification. There are some notable exceptions to this that require manual intervention to fully comply with the CIS Benchmark:
|
||||
|
||||
1. K3s will not modify the host operating system. Any host-level modifications will need to be done manually.
|
||||
2. Certain CIS policy controls for PodSecurityPolicies and NetworkPolicies will restrict the functionality of this cluster. You must opt into having K3s configure these by adding the appropriate options (enabling of admission plugins) to your command-line flags or configuration file as well as manually applying appropriate policies. Further detail in the sections below.
|
||||
2. Certain CIS policy controls for `PodSecurityPolicies` and `NetworkPolicies` will restrict the functionality of the cluster. You must opt into having K3s configure these by adding the appropriate options (enabling of admission plugins) to your command-line flags or configuration file as well as manually applying appropriate policies. Further details are presented in the sections below.
|
||||
|
||||
The first section (1.1) of the CIS Benchmark concerns itself primarily with pod manifest permissions and ownership. K3s doesn't utilize these for the core components since everything is packaged into a single binary.
|
||||
|
||||
@@ -31,23 +31,24 @@ vm.panic_on_oom=0
|
||||
vm.overcommit_memory=1
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
```
|
||||
|
||||
## Kubernetes Runtime Requirements
|
||||
|
||||
The runtime requirements to comply with the CIS Benchmark are centered around pod security (PSPs) and network policies. These are outlined in this section. K3s doesn't apply any default PSPs or network policies however K3s ships with a controller that is meant to apply a given set of network policies. By default, K3s runs with the "NodeRestriction" admission controller. To enable PSPs, add the following to the K3s start command: `--kube-apiserver-arg="enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ServiceAccount"`. This will have the effect of maintaining the "NodeRestriction" plugin as well as enabling the "PodSecurityPolicy".
|
||||
The runtime requirements to comply with the CIS Benchmark are centered around pod security (PSPs), network policies and API Server auditing logs. These are outlined in this section. K3s doesn't apply any default PSPs or network policies. However, K3s ships with a controller that is meant to apply a given set of network policies. By default, K3s runs with the `NodeRestriction` admission controller. To enable PSPs, add the following to the K3s start command: `--kube-apiserver-arg="enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ServiceAccount"`. This will have the effect of maintaining the `NodeRestriction` plugin as well as enabling the `PodSecurityPolicy`. The same happens with the API Server auditing logs, K3s doesn't enable them by default, so audit log configuration and audit policy must be created manually.
|
||||
|
||||
### PodSecurityPolicies
|
||||
### Pod Security Policies
|
||||
|
||||
When PSPs are enabled, a policy can be applied to satisfy the necessary controls described in section 5.2 of the CIS Benchmark.
|
||||
|
||||
Here's an example of a compliant PSP.
|
||||
Here is an example of a compliant PSP.
|
||||
|
||||
```yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: cis1.5-compliant-psp
|
||||
name: restricted-psp
|
||||
spec:
|
||||
privileged: false # CIS - 5.2.1
|
||||
allowPrivilegeEscalation: false # CIS - 5.2.5
|
||||
@@ -59,7 +60,9 @@ spec:
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false # CIS - 5.2.4
|
||||
hostIPC: false # CIS - 5.2.3
|
||||
hostPID: false # CIS - 5.2.2
|
||||
@@ -80,7 +83,7 @@ spec:
|
||||
readOnlyRootFilesystem: false
|
||||
```
|
||||
|
||||
Before the above PSP to be effective, we need to create a couple ClusterRoles and ClusterRole. We also need to include a "system unrestricted policy" which is needed for system-level pods that require additional privileges.
|
||||
For the above PSP to be effective, we need to create a ClusterRole and a ClusterRoleBinding. We also need to include a "system unrestricted policy" which is needed for system-level pods that require additional privileges.
|
||||
|
||||
These can be combined with the PSP yaml above and NetworkPolicy yaml below into a single file and placed in the `/var/lib/rancher/k3s/server/manifests` directory. Below is an example of a `policy.yaml` file.
|
||||
|
||||
@@ -88,7 +91,7 @@ These can be combined with the PSP yaml above and NetworkPolicy yaml below into
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: cis1.5-compliant-psp
|
||||
name: restricted-psp
|
||||
spec:
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
@@ -100,7 +103,9 @@ spec:
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
@@ -123,7 +128,7 @@ spec:
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
name: psp:restricted-psp
|
||||
labels:
|
||||
addonmanager.kubernetes.io/mode: EnsureExists
|
||||
rules:
|
||||
@@ -131,62 +136,23 @@ rules:
|
||||
resources: ['podsecuritypolicies']
|
||||
verbs: ['use']
|
||||
resourceNames:
|
||||
- cis1.5-compliant-psp
|
||||
- restricted-psp
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: default:restricted
|
||||
name: default:restricted-psp
|
||||
labels:
|
||||
addonmanager.kubernetes.io/mode: EnsureExists
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
name: psp:restricted-psp
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: system:authenticated
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: kube-system
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-system
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: default
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: default
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: kube-public
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-public
|
||||
---
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
@@ -253,6 +219,45 @@ subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: kube-system
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-system
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: default
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: default
|
||||
---
|
||||
kind: NetworkPolicy
|
||||
apiVersion: networking.k8s.io/v1
|
||||
metadata:
|
||||
name: intra-namespace
|
||||
namespace: kube-public
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
name: kube-public
|
||||
```
|
||||
|
||||
> **Note:** The Kubernetes critical additions such as CNI, DNS, and Ingress are ran as pods in the `kube-system` namespace. Therefore, this namespace will have a policy that is less restrictive so that these components can run properly.
|
||||
@@ -263,7 +268,7 @@ subjects:
|
||||
|
||||
CIS requires that all namespaces have a network policy applied that reasonably limits traffic into namespaces and pods.
|
||||
|
||||
Here's an example of a compliant network policy.
|
||||
Here is an example of a compliant network policy.
|
||||
|
||||
```yaml
|
||||
kind: NetworkPolicy
|
||||
@@ -302,11 +307,25 @@ spec:
|
||||
- Ingress
|
||||
```
|
||||
|
||||
If you are using the default traefik ingress controller with k3s, it will also be blocked by default, so the following network policies must be added to allow traffic to both traefik pods and svclb pods in the kube-system namespace. For version 1.20 and below there is a different label `traefik` used than in 1.21 and above, so remove the one that is not associated with your Kubernetes version.
|
||||
The metrics-server and Traefik ingress controller will be blocked by default if network policies are not created to allow access. Traefik v1 as packaged in K3s version 1.20 and below uses different labels than Traefik v2. Ensure that you only use the sample yaml below that is associated with the version of Traefik present on your cluster.
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-all-metrics-server
|
||||
namespace: kube-system
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
k8s-app: metrics-server
|
||||
ingress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-all-svclbtraefik-ingress
|
||||
namespace: kube-system
|
||||
@@ -319,7 +338,7 @@ spec:
|
||||
policyTypes:
|
||||
- Ingress
|
||||
---
|
||||
# 1.20
|
||||
# Below is for 1.20 ONLY -- remove if on 1.21 or above
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
@@ -334,7 +353,7 @@ spec:
|
||||
policyTypes:
|
||||
- Ingress
|
||||
---
|
||||
# 1.21
|
||||
# Below is for 1.21 and above ONLY -- remove if on 1.20 or below
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
@@ -352,24 +371,67 @@ spec:
|
||||
|
||||
> **Note:** Operators must manage network policies as normal for additional namespaces that are created.
|
||||
|
||||
### API Server audit configuration
|
||||
|
||||
CIS requirements 1.2.22 to 1.2.25 are related to configuring audit logs for the API Server. K3s doesn't create by default the log directory and audit policy, as auditing requirements are specific to each user's policies and environment.
|
||||
|
||||
The log directory, ideally, must be created before starting K3s. A restrictive access permission is recommended to avoid leaking potential sensitive information.
|
||||
|
||||
```bash
|
||||
sudo mkdir -p -m 700 /var/lib/rancher/k3s/server/logs
|
||||
```
|
||||
|
||||
A starter audit policy to log request metadata is provided below. The policy should be written to a file named `audit.yaml` in `/var/lib/rancher/k3s/server` directory. Detailed information about policy configuration for the API server can be found in the Kubernetes [documentation](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/).
|
||||
|
||||
```yaml
|
||||
apiVersion: audit.k8s.io/v1
|
||||
kind: Policy
|
||||
rules:
|
||||
- level: Metadata
|
||||
```
|
||||
|
||||
Both configurations must be passed as arguments to the API Server as:
|
||||
|
||||
```bash
|
||||
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log'
|
||||
--kube-apiserver-arg='audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml'
|
||||
```
|
||||
|
||||
If the configurations are created after K3s is installed, they must be added to K3s' systemd service in `/etc/systemd/system/k3s.service`.
|
||||
|
||||
```bash
|
||||
ExecStart=/usr/local/bin/k3s \
|
||||
server \
|
||||
'--kube-apiserver-arg=audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log' \
|
||||
'--kube-apiserver-arg=audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml' \
|
||||
```
|
||||
|
||||
K3s must be restarted to load the new configuration.
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart k3s.service
|
||||
```
|
||||
|
||||
Additional information about CIS requirements 1.2.22 to 1.2.25 is presented below.
|
||||
|
||||
## Known Issues
|
||||
The following are controls that K3s currently does not pass by default. Each gap will be explained, along with a note clarifying whether it can be passed through manual operator intervention, or if it will be addressed in a future release of K3s.
|
||||
|
||||
|
||||
### Control 1.2.15
|
||||
Ensure that the admission control plugin `NamespaceLifecycle` is set.
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
Setting admission control policy to NamespaceLifecycle ensures that objects cannot be created in non-existent namespaces, and that namespaces undergoing termination are not used for creating the new objects. This is recommended to enforce the integrity of the namespace termination process and also for the availability of the newer objects.
|
||||
Setting admission control policy to `NamespaceLifecycle` ensures that objects cannot be created in non-existent namespaces, and that namespaces undergoing termination are not used for creating the new objects. This is recommended to enforce the integrity of the namespace termination process and also for the availability of the newer objects.
|
||||
|
||||
This can be remediated by passing this argument as a value to the `enable-admission-plugins=` and pass that to `--kube-apiserver-arg=` argument to `k3s server`. An example can be found below.
|
||||
</details>
|
||||
|
||||
### Control 1.2.16 (mentioned above)
|
||||
### Control 1.2.16
|
||||
Ensure that the admission control plugin `PodSecurityPolicy` is set.
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
A Pod Security Policy is a cluster-level resource that controls the actions that a pod can perform and what it has the ability to access. The PodSecurityPolicy objects define a set of conditions that a pod must run with in order to be accepted into the system. Pod Security Policies are comprised of settings and strategies that control the security features a pod has access to and hence this must be used to control pod access permissions.
|
||||
A Pod Security Policy is a cluster-level resource that controls the actions that a pod can perform and what it has the ability to access. The `PodSecurityPolicy` objects define a set of conditions that a pod must run with in order to be accepted into the system. Pod Security Policies are comprised of settings and strategies that control the security features a pod has access to and hence this must be used to control pod access permissions.
|
||||
|
||||
This can be remediated by passing this argument as a value to the `enable-admission-plugins=` and pass that to `--kube-apiserver-arg=` argument to `k3s server`. An example can be found below.
|
||||
</details>
|
||||
@@ -432,16 +494,18 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
|
||||
Ensure that the `--encryption-provider-config` argument is set as appropriate.
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
Where `etcd` encryption is used, it is important to ensure that the appropriate set of encryption providers is used. Currently, the aescbc, kms and secretbox are likely to be appropriate options.
|
||||
`etcd` is a highly available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.
|
||||
|
||||
Detailed steps on how to configure secrets encryption in K3s are available in [Secrets Encryption](../secrets_encryption/).
|
||||
</details>
|
||||
|
||||
### Control 1.2.34
|
||||
Ensure that encryption providers are appropriately configured.
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
`etcd` is a highly available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.
|
||||
Where `etcd` encryption is used, it is important to ensure that the appropriate set of encryption providers is used. Currently, the `aescbc`, `kms` and `secretbox` are likely to be appropriate options.
|
||||
|
||||
This can be remediated by passing a valid configuration to `k3s` as outlined above.
|
||||
This can be remediated by passing a valid configuration to `k3s` as outlined above. Detailed steps on how to configure secrets encryption in K3s are available in [Secrets Encryption](../secrets_encryption/).
|
||||
</details>
|
||||
|
||||
### Control 1.3.1
|
||||
@@ -454,7 +518,7 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
|
||||
</details>
|
||||
|
||||
### Control 3.2.1
|
||||
Ensure that a minimal audit policy is created (Scored)
|
||||
Ensure that a minimal audit policy is created.
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
Logging is an important detective control for all systems, to detect potential unauthorized access.
|
||||
@@ -462,7 +526,6 @@ Logging is an important detective control for all systems, to detect potential u
|
||||
This can be remediated by passing controls 1.2.22 - 1.2.25 and verifying their efficacy.
|
||||
</details>
|
||||
|
||||
|
||||
### Control 4.2.7
|
||||
Ensure that the `--make-iptables-util-chains` argument is set to true.
|
||||
<details>
|
||||
@@ -473,24 +536,23 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
|
||||
</details>
|
||||
|
||||
### Control 5.1.5
|
||||
Ensure that default service accounts are not actively used. (Scored)
|
||||
Ensure that default service accounts are not actively used
|
||||
<details>
|
||||
<summary>Rationale</summary>
|
||||
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod.
|
||||
Kubernetes provides a `default` service account which is used by cluster workloads where no specific service account is assigned to the pod.
|
||||
|
||||
Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account.
|
||||
|
||||
The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
</details>
|
||||
|
||||
The remediation for this is to update the `automountServiceAccountToken` field to `false` for the `default` service account in each namespace.
|
||||
This can be remediated by updating the `automountServiceAccountToken` field to `false` for the `default` service account in each namespace.
|
||||
|
||||
For `default` service accounts in the built-in namespaces (`kube-system`, `kube-public`, `kube-node-lease`, and `default`), K3s does not automatically do this. You can manually update this field on these service accounts to pass the control.
|
||||
</details>
|
||||
|
||||
## Control Plane Execution and Arguments
|
||||
|
||||
Listed below are the K3s control plane components and the arguments they're given at start, by default. Commented to their right is the CIS 1.5 control that they satisfy.
|
||||
Listed below are the K3s control plane components and the arguments they are given at start, by default. Commented to their right is the CIS 1.6 control that they satisfy.
|
||||
|
||||
```bash
|
||||
kube-apiserver
|
||||
@@ -590,13 +652,14 @@ kubelet
|
||||
--tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key # 4.2.10
|
||||
```
|
||||
|
||||
The command below is an example of how the outlined remediations can be applied.
|
||||
The command below is an example of how the outlined remediations can be applied to harden K3s.
|
||||
|
||||
```bash
|
||||
k3s server \
|
||||
--protect-kernel-defaults=true \
|
||||
--secrets-encryption=true \
|
||||
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit-log' \
|
||||
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log' \
|
||||
--kube-apiserver-arg='audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml' \
|
||||
--kube-apiserver-arg='audit-log-maxage=30' \
|
||||
--kube-apiserver-arg='audit-log-maxbackup=10' \
|
||||
--kube-apiserver-arg='audit-log-maxsize=100' \
|
||||
@@ -611,4 +674,4 @@ k3s server \
|
||||
|
||||
## Conclusion
|
||||
|
||||
If you have followed this guide, your K3s cluster will be configured to comply with the CIS Kubernetes Benchmark. You can review the [CIS Benchmark Self-Assessment Guide](../self_assessment/) to understand the expectations of each of the benchmarks and how you can do the same on your cluster.
|
||||
If you have followed this guide, your K3s cluster will be configured to comply with the CIS Kubernetes Benchmark. You can review the [CIS Benchmark Self-Assessment Guide](../self_assessment/) to understand the expectations of each of the benchmark's checks and how you can do the same on your cluster.
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
---
|
||||
title: Secrets Encryption
|
||||
weight: 26
|
||||
---
|
||||
|
||||
# Secrets Encryption Config
|
||||
_Available as of v1.17.4+k3s1_
|
||||
|
||||
K3s supports enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server; this flag will do the following automatically:
|
||||
|
||||
- Generate an AES-CBC key
|
||||
- Generate an encryption config file with the generated key
|
||||
- Pass the config to the KubeAPI as encryption-provider-config
|
||||
|
||||
Example of the encryption config file:
|
||||
```
|
||||
{
|
||||
"kind": "EncryptionConfiguration",
|
||||
"apiVersion": "apiserver.config.k8s.io/v1",
|
||||
"resources": [
|
||||
{
|
||||
"resources": [
|
||||
"secrets"
|
||||
],
|
||||
"providers": [
|
||||
{
|
||||
"aescbc": {
|
||||
"keys": [
|
||||
{
|
||||
"name": "aescbckey",
|
||||
"secret": "xxxxxxxxxxxxxxxxxxx"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"identity": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Secrets Encryption Tool
|
||||
_Available as of v1.21.8+k3s1_
|
||||
|
||||
K3s contains a utility tool `secrets-encrypt`, which enables automatic control over the following:
|
||||
|
||||
- Disabling/Enabling secrets encryption
|
||||
- Adding new encryption keys
|
||||
- Rotating and deleting encryption keys
|
||||
- Reencrypting secrets
|
||||
|
||||
>**Warning:** Failure to follow proper procedure for rotating encryption keys can leave your cluster permanently corrupted. Proceed with caution.
|
||||
|
||||
### Single-Server Encryption Key Rotation
|
||||
To rotate secrets encryption keys on a single-node cluster:
|
||||
|
||||
- Start the K3s server with the flag `--secrets-encryption`
|
||||
|
||||
>**Note:** Starting K3s without encryption and enabling it at a later time is currently *not* supported.
|
||||
|
||||
1. Prepare
|
||||
|
||||
```
|
||||
k3s secrets-encrypt prepare
|
||||
```
|
||||
|
||||
2. Kill and restart the K3s server with same arguments
|
||||
3. Rotate
|
||||
|
||||
```
|
||||
k3s secrets-encrypt rotate
|
||||
```
|
||||
|
||||
4. Kill and restart the K3s server with same arguments
|
||||
5. Reencrypt
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt
|
||||
```
|
||||
|
||||
### High-Availability Encryption Key Rotation
|
||||
The steps are the same for both embedded DB and external DB clusters.
|
||||
|
||||
To rotate secrets encryption keys on HA setups:
|
||||
|
||||
>**Notes:**
|
||||
>
|
||||
> - Starting K3s without encryption and enabling it at a later time is currently *not* supported.
|
||||
>
|
||||
> - While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands.
|
||||
|
||||
- Start up all three K3s servers with the `--secrets-encryption` flag. For brevity, the servers will be referred to as S1, S2, S3.
|
||||
|
||||
1. Prepare on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt prepare
|
||||
```
|
||||
|
||||
2. Kill and restart S1 with same arguments
|
||||
3. Once S1 is up, kill and restart the S2 and S3
|
||||
|
||||
4. Rotate on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt rotate
|
||||
```
|
||||
|
||||
5. Kill and restart S1 with same arguments
|
||||
6. Once S1 is up, kill and restart the S2 and S3
|
||||
|
||||
7. Reencrypt on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt
|
||||
```
|
||||
|
||||
8. Kill and restart S1 with same arguments
|
||||
9. Once S1 is up, kill and restart the S2 and S3
|
||||
|
||||
### Single-Server Secrets Encryption Disable/Enable
|
||||
After launching a server with `--secrets-encryption` flag, secrets encryption can be disabled.
|
||||
|
||||
To disable secrets encryption on a single-node cluster:
|
||||
|
||||
1. Disable
|
||||
|
||||
```
|
||||
k3s secrets-encrypt disable
|
||||
```
|
||||
|
||||
2. Kill and restart the K3s server with same arguments
|
||||
|
||||
3. Reencrypt with flags
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt --force --skip
|
||||
```
|
||||
|
||||
To re-enable secrets encryption on a single node cluster:
|
||||
|
||||
1. Enable
|
||||
|
||||
```
|
||||
k3s secrets-encrypt enable
|
||||
```
|
||||
|
||||
2. Kill and restart the K3s server with same arguments
|
||||
|
||||
3. Reencrypt with flags
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt --force --skip
|
||||
```
|
||||
|
||||
### High-Availability Secrets Encryption Disable/Enable
|
||||
After launching a HA cluster with `--secrets-encryption` flags, secrets encryption can be disabled.
|
||||
>**Note:** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands.
|
||||
|
||||
For brevity, the three servers used in this guide will be referred to as S1, S2, S3.
|
||||
|
||||
To disable secrets encryption on a HA cluster:
|
||||
|
||||
1. Disable on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt disable
|
||||
```
|
||||
|
||||
2. Kill and restart S1 with same arguments
|
||||
3. Once S1 is up, kill and restart the S2 and S3
|
||||
|
||||
|
||||
4. Reencrypt with flags on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt --force --skip
|
||||
```
|
||||
|
||||
To re-enable secrets encryption on a HA cluster:
|
||||
|
||||
1. Enable on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt enable
|
||||
```
|
||||
|
||||
2. Kill and restart S1 with same arguments
|
||||
3. Once S1 is up, kill and restart the S2 and S3
|
||||
|
||||
4. Reencrypt with flags on S1
|
||||
|
||||
```
|
||||
k3s secrets-encrypt reencrypt --force --skip
|
||||
```
|
||||
|
||||
|
||||
### Secrets Encryption Status
|
||||
The secrets-encrypt tool includes a `status` command that displays information about the current status of secrets encryption on the node.
|
||||
|
||||
An example of the command on a single-server node:
|
||||
```
|
||||
$ k3s secrets-encrypt status
|
||||
Encryption Status: Enabled
|
||||
Current Rotation Stage: start
|
||||
Server Encryption Hashes: All hashes match
|
||||
|
||||
Active Key Type Name
|
||||
------ -------- ----
|
||||
* AES-CBC aescbckey
|
||||
|
||||
```
|
||||
|
||||
Another example on HA cluster, after rotating the keys, but before restarting the servers:
|
||||
```
|
||||
$ k3s secrets-encrypt status
|
||||
Encryption Status: Enabled
|
||||
Current Rotation Stage: rotate
|
||||
Server Encryption Hashes: hash does not match between node-1 and node-2
|
||||
|
||||
Active Key Type Name
|
||||
------ -------- ----
|
||||
* AES-CBC aescbckey-2021-12-10T22:54:38Z
|
||||
AES-CBC aescbckey
|
||||
|
||||
```
|
||||
|
||||
Details on each section are as follows:
|
||||
|
||||
- __Encryption Status__: Displayed whether secrets encryption is disabled or enabled on the node
|
||||
- __Current Rotation Stage__: Indicates the current rotation stage on the node.
|
||||
Stages are: `start`, `prepare`, `rotate`, `reencrypt_request`, `reencrypt_active`, `reencrypt_finished`
|
||||
- __Server Encryption Hashes__: Useful for HA clusters, this indicates whether all servers are on the same stage with their local files. This can be used to identify whether a restart of servers is required before proceeding to the next stage. In the HA example above, node-1 and node-2 have different hashes, indicating that they currently do not have the same encryption configuration. Restarting the servers will sync up their configuration.
|
||||
- __Key Table__: Summarizes information about the secrets encryption keys found on the node.
|
||||
* __Active__: The "*" indicates which, if any, of the keys are currently used for secrets encryption. An active key is used by Kubernetes to encrypt any new secrets.
|
||||
* __Key Type__: All keys using this tool are `AES-CBC` type. See more info [here.](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#providers)
|
||||
* __Name__: Name of the encryption key.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -9,6 +9,26 @@ A persistent volume (PV) is a piece of storage in the Kubernetes cluster, while
|
||||
|
||||
This page describes how to set up persistent storage with a local storage provider, or with [Longhorn.](#setting-up-longhorn)
|
||||
|
||||
# What's changed in K3s storage?
|
||||
|
||||
K3s removes several optional volume plugins and all built-in (sometimes referred to as "in-tree") cloud providers. We do this in order to achieve a smaller binary size and to avoid dependence on third-party cloud or data center technologies and services, which may not be available in many K3s use cases. We are able to do this because their removal affects neither core Kubernetes functionality nor conformance.
|
||||
|
||||
The following volume plugins have been removed from K3s:
|
||||
|
||||
* cephfs
|
||||
* fc
|
||||
* flocker
|
||||
* git_repo
|
||||
* glusterfs
|
||||
* portworx
|
||||
* quobyte
|
||||
* rbd
|
||||
* storageos
|
||||
|
||||
Both components have out-of-tree alternatives that can be used with K3s: The Kubernetes [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec/blob/master/spec.md) and [Cloud Provider Interface (CPI)](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/).
|
||||
|
||||
Kubernetes maintainers are actively migrating in-tree volume plugins to CSI drivers. For more information on this migration, please refer [here](https://kubernetes.io/blog/2021/12/10/storage-in-tree-to-csi-migration-status-update/).
|
||||
|
||||
# Setting up the Local Storage Provider
|
||||
K3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage).
|
||||
|
||||
|
||||
@@ -3,12 +3,22 @@ title: "Upgrades"
|
||||
weight: 25
|
||||
---
|
||||
|
||||
This section describes how to upgrade your K3s cluster.
|
||||
### Upgrading your K3s cluster
|
||||
|
||||
[Upgrade basics]({{< baseurl >}}/k3s/latest/en/upgrades/basic/) describes several techniques for upgrading your cluster manually. It can also be used as a basis for upgrading through third-party Infrastructure-as-Code tools like [Terraform](https://www.terraform.io/).
|
||||
|
||||
[Automated upgrades]({{< baseurl >}}/k3s/latest/en/upgrades/automated/) describes how to perform Kubernetes-native automated upgrades using Rancher's [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller).
|
||||
|
||||
> If Traefik is not disabled K3s versions 1.20 and earlier will have installed Traefik v1, while K3s versions 1.21 and later will install Traefik v2 if v1 is not already present. To upgrade Traefik, please refer to the [Traefik documentation](https://doc.traefik.io/traefik/migration/v1-to-v2/) and use the [migration tool](https://github.com/traefik/traefik-migration-tool) to migrate from the older Traefik v1 to Traefik v2.
|
||||
### Version-specific caveats
|
||||
|
||||
> The experimental embedded Dqlite data store was deprecated in K3s v1.19.1. Please note that upgrades from experimental Dqlite to experimental embedded etcd are not supported. If you attempt an upgrade it will not succeed and data will be lost.
|
||||
- **Traefik:** If Traefik is not disabled, K3s versions 1.20 and earlier will install Traefik v1, while K3s versions 1.21 and later will install Traefik v2, if v1 is not already present. To upgrade from the older Traefik v1 to Traefik v2, please refer to the [Traefik documentation](https://doc.traefik.io/traefik/migration/v1-to-v2/) and use the [migration tool](https://github.com/traefik/traefik-migration-tool).
|
||||
|
||||
- **K3s bootstrap data:** If you are using K3s in an HA configuration with an external SQL datastore, and your server (control-plane) nodes were not started with the `--token` CLI flag, you will no longer be able to add additional K3s servers to the cluster without specifying the token. Ensure that you retain a copy of this token, as it is required when restoring from backup. Previously, K3s did not enforce the use of a token when using external SQL datastores.
|
||||
- The affected versions are <= v1.19.12+k3s1, v1.20.8+k3s1, v1.21.2+k3s1; the patched versions are v1.19.13+k3s1, v1.20.9+k3s1, v1.21.3+k3s1.
|
||||
|
||||
- You may retrieve the token value from any server already joined to the cluster as follows:
|
||||
```
|
||||
cat /var/lib/rancher/k3s/server/token
|
||||
```
|
||||
|
||||
- **Experimental Dqlite:** The experimental embedded Dqlite data store was deprecated in K3s v1.19.1. Please note that upgrades from experimental Dqlite to experimental embedded etcd are not supported. If you attempt an upgrade, it will not succeed, and data will be lost.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: v2.0-v2.4.x
|
||||
title: Rancher 2.0-2.4
|
||||
weight: 3
|
||||
showBreadcrumb: false
|
||||
---
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
title: "Rancher v2.0-v2.4"
|
||||
title: "Rancher 2.0-2.4"
|
||||
shortTitle: "Rancher 2.0-2.4"
|
||||
description: "Rancher adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
|
||||
metaTitle: "Rancher 2.x Docs: What is New?"
|
||||
metaTitle: "Rancher 2.0-2.4 Docs: What is New?"
|
||||
metaDescription: "Rancher 2 adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
|
||||
insertOneSix: true
|
||||
weight: 1
|
||||
@@ -18,4 +18,4 @@ Rancher adds significant value on top of Kubernetes, first by centralizing authe
|
||||
|
||||
It then enables detailed monitoring and alerting for clusters and their resources, ships logs to external providers, and integrates directly with Helm via the Application Catalog. If you have an external CI/CD system, you can plug it into Rancher, but if you don't, Rancher even includes a pipeline engine to help you automatically deploy and upgrade workloads.
|
||||
|
||||
Rancher is a _complete_ container management platform for Kubernetes, giving you the tools to successfully run Kubernetes anywhere.
|
||||
Rancher is a _complete_ container management platform for Kubernetes, giving you the tools to successfully run Kubernetes anywhere.
|
||||
|
||||
@@ -29,6 +29,8 @@ Note that these URLs will not return valid data until the authentication configu
|
||||
|
||||
1. **Groups Field**: Make entries for managing group memberships (example: `memberOf`).
|
||||
|
||||
1. **Entity ID Field** (optional): The published, protocol-dependent, unique identifier of your partner. This ID defines your organization as the entity operating the server for SAML 2.0 transactions. This ID may have been obtained out-of-band or via a SAML metadata file.
|
||||
|
||||
1. **Rancher API Host**: Enter the URL for your Rancher Server.
|
||||
|
||||
1. **Private Key** and **Certificate**: This is a key-certificate pair to create a secure shell between Rancher and your IdP.
|
||||
|
||||
@@ -4,7 +4,7 @@ weight: 400
|
||||
aliases:
|
||||
---
|
||||
|
||||
You might want to use a private Docker registry to share your custom base images within your organization. With a private registry, you can keep a private, consistent, and centralized source of truth for the Docker images that are used in your clusters.
|
||||
You might want to use a private container registry to share your custom base images within your organization. With a private registry, you can keep a private, consistent, and centralized source of truth for the container images that are used in your clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the global default registry through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
|
||||
@@ -84,6 +84,10 @@ _Project roles_ are roles that can be used to grant users access to a project. T
|
||||
|
||||
These users can manage project-scoped resources like namespaces and workloads, but cannot manage other project members.
|
||||
|
||||
>**Note:**
|
||||
>
|
||||
>By default, the Rancher role of `project-member` inherits from the `Kubernetes-edit` role, and the `project-owner` role inherits from the `Kubernetes-admin` role. As such, both `project-member` and `project-owner` roles will allow for namespace management, including the ability to create and delete namespaces.
|
||||
|
||||
- **Read Only:**
|
||||
|
||||
These users can view everything in the project but cannot create, update, or delete anything.
|
||||
|
||||
@@ -27,25 +27,6 @@ This section covers the following topics:
|
||||
- [Configuring global permissions for groups](#configuring-global-permissions-for-groups)
|
||||
- [Refreshing group memberships](#refreshing-group-memberships)
|
||||
|
||||
### List of `restricted-admin` Permissions
|
||||
|
||||
The `restricted-admin` permissions are as follows:
|
||||
|
||||
- Has full admin access to all downstream clusters managed by Rancher.
|
||||
- Has very limited access to the local Kubernetes cluster. Can access Rancher custom resource definitions, but has no access to any Kubernetes native types.
|
||||
- Can add other users and assign them to clusters outside of the local cluster.
|
||||
- Can create other restricted admins.
|
||||
- Cannot grant any permissions in the local cluster they don't currently have. (This is how Kubernetes normally operates)
|
||||
|
||||
|
||||
### Changing Global Administrators to Restricted Admins
|
||||
|
||||
If Rancher already has a global administrator, they should change all global administrators over to the new `restricted-admin` role.
|
||||
|
||||
This can be done through **Security > Users** and moving any Administrator role over to Restricted Administrator.
|
||||
|
||||
Signed-in users can change themselves over to the `restricted-admin` if they wish, but they should only do that as the last step, otherwise they won't have the permissions to do so.
|
||||
|
||||
# Global Permission Assignment
|
||||
|
||||
Global permissions for local users are assigned differently than users who log in to Rancher using external authentication.
|
||||
|
||||
@@ -28,7 +28,7 @@ Because of the changes necessary to address [CVE-2018-20321](https://cve.mitre.o
|
||||
|
||||
2. After executing the command a `tokens.json` file will be created. Important! Back up this file in a safe place.** You will need it to restore functionality to your clusters after rolling back Rancher. **If you lose this file, you may lose access to your clusters.**
|
||||
|
||||
3. Rollback Rancher following the [normal instructions]({{<baseurl>}}/rancher/v2.x/en/upgrades/rollbacks/).
|
||||
3. Rollback Rancher following the [normal instructions]({{<baseurl>}}/rancher/v2.0-v2.4/en/upgrades/rollbacks/).
|
||||
|
||||
4. Once Rancher comes back up, every cluster managed by Rancher (except for Imported clusters) will be in an `Unavailable` state.
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ weight: 101
|
||||
aliases:
|
||||
- /rancher/v2.0-v2.4/en/best-practices/management
|
||||
- /rancher/v2.0-v2.4/en/best-practices/v2.0-v2.4/management
|
||||
- /rancher/v2.x/en/best-practices/management/
|
||||
- /rancher/v2.x/en/best-practices/v2.0-v2.4/management/
|
||||
---
|
||||
|
||||
@@ -86,8 +87,8 @@ Run your etcd and control plane nodes on virtual machines where you can scale vC
|
||||
### Use at Least Three etcd Nodes
|
||||
Provision 3 or 5 etcd nodes. Etcd requires a quorum to determine a leader by the majority of nodes, therefore it is not recommended to have clusters of even numbers. Three etcd nodes is generally sufficient for smaller clusters and five etcd nodes for large clusters.
|
||||
|
||||
### Use at Least Two Control Plane Nodes
|
||||
Provision two or more control plane nodes. Some control plane components, such as the `kube-apiserver`, run in [active-active](https://www.jscape.com/blog/active-active-vs-active-passive-high-availability-cluster) mode and will give you more scalability. Other components such as kube-scheduler and kube-controller run in active-passive mode (leader elect) and give you more fault tolerance.
|
||||
### Use at Least Three Control Plane Nodes
|
||||
Provision three or more control plane nodes. Some control plane components, such as the `kube-apiserver`, run in [active-active](https://www.jscape.com/blog/active-active-vs-active-passive-high-availability-cluster) mode and will give you more scalability. Other components such as kube-scheduler and kube-controller run in active-passive mode (leader elect) and give you more fault tolerance.
|
||||
|
||||
### Monitor Your Cluster
|
||||
Closely monitor and scale your nodes as needed. You should [enable cluster monitoring]({{<baseurl>}}/rancher/v2.0-v2.4/en/monitoring-alerting/legacy/monitoring/cluster-monitoring/) and use the Prometheus metrics and Grafana visualization options as a starting point.
|
||||
|
||||
@@ -162,6 +162,9 @@ By default, your user is added as the project `Owner`.
|
||||
>**Notes on Permissions:**
|
||||
>
|
||||
>- Users assigned the `Owner` or `Member` role for a project automatically inherit the `namespace creation` role. However, this role is a [Kubernetes ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole), meaning its scope extends to all projects in the cluster. Therefore, users explicitly assigned the `Owner` or `Member` role for a project can create namespaces in other projects they're assigned to, even with only the `Read Only` role assigned.
|
||||
>
|
||||
>- By default, the Rancher role of `project-member` inherits from the `Kubernetes-edit` role, and the `project-owner` role inherits from the `Kubernetes-admin` role. As such, both `project-member` and `project-owner` roles will allow for namespace management, including the ability to create and delete namespaces.
|
||||
>
|
||||
>- Choose `Custom` to create a custom role on the fly: [Custom Project Roles]({{<baseurl>}}/rancher/v2.0-v2.4/en/admin-settings/rbac/cluster-project-roles/#custom-project-roles).
|
||||
|
||||
To add members:
|
||||
|
||||
@@ -128,4 +128,4 @@ This reference contains variables that you can use in `questions.yml` nested und
|
||||
|
||||
# Tutorial: Example Custom Chart Creation
|
||||
|
||||
For a tutorial on adding a custom Helm chart to a custom catalog, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/catalog/tutorial)
|
||||
For a tutorial on adding a custom Helm chart to a custom catalog, refer to [this page.]({{<baseurl>}}/rancher/v2.0-v2.4/en/catalog/tutorial)
|
||||
|
||||
@@ -50,20 +50,20 @@ You can fill your custom catalogs with either Helm Charts or Rancher Charts, alt
|
||||
- CMS
|
||||
questions:
|
||||
- variable: persistence.enabled
|
||||
default: "false"
|
||||
description: "Enable persistent volume for WordPress"
|
||||
type: boolean
|
||||
required: true
|
||||
label: WordPress Persistent Volume Enabled
|
||||
show_subquestion_if: true
|
||||
group: "WordPress Settings"
|
||||
subquestions:
|
||||
- variable: persistence.size
|
||||
default: "false"
|
||||
description: "Enable persistent volume for WordPress"
|
||||
type: boolean
|
||||
required: true
|
||||
label: WordPress Persistent Volume Enabled
|
||||
show_subquestion_if: true
|
||||
group: "WordPress Settings"
|
||||
subquestions:
|
||||
- variable: persistence.size
|
||||
default: "10Gi"
|
||||
description: "WordPress Persistent Volume Size"
|
||||
type: string
|
||||
label: WordPress Volume Size
|
||||
- variable: persistence.storageClass
|
||||
- variable: persistence.storageClass
|
||||
default: ""
|
||||
description: "If undefined or null, uses the default StorageClass. Default to null"
|
||||
type: storageclass
|
||||
|
||||
@@ -196,7 +196,7 @@ helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=letsEncrypt \
|
||||
--set letsEncrypt.email=me@example.org
|
||||
--set letsEncrypt.email=me@example.org \
|
||||
```
|
||||
|
||||
Wait for Rancher to be rolled out:
|
||||
@@ -231,7 +231,7 @@ helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
If you are using a Private CA signed certificate , add `--set privateCA=true` to the command:
|
||||
|
||||
```
|
||||
helm install rancher rancher-latest/rancher \
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=secret \
|
||||
|
||||
+1
-1
@@ -126,7 +126,7 @@ To customize or use a different ingress with Rancher server you can set your own
|
||||
Example on setting a custom certificate issuer:
|
||||
|
||||
```plain
|
||||
--set ingress.extraAnnotations.'certmanager\.k8s\.io/cluster-issuer'=ca-key-pair
|
||||
--set ingress.extraAnnotations.'cert-manager\.io/cluster-issuer'=issuer-name
|
||||
```
|
||||
|
||||
Example on setting a static proxy header with `ingress.configurationSnippet`. This value is parsed like a template so variables can be used.
|
||||
|
||||
@@ -168,7 +168,7 @@ helm upgrade rancher rancher-<CHART_REPO>/rancher \
|
||||
|
||||
### Option B: Reinstalling Rancher and cert-manager
|
||||
|
||||
If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manger due to the API change in cert-manger v0.11.
|
||||
If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manager due to the API change in cert-manager v0.11.
|
||||
|
||||
1. Uninstall Rancher
|
||||
|
||||
|
||||
+1
-1
@@ -114,7 +114,7 @@ helm upgrade --install rancher rancher-<CHART_REPO>/rancher \
|
||||
|
||||
{{% accordion label="Option B: Reinstalling Rancher chart" %}}
|
||||
|
||||
If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manger due to the API change in cert-manger v0.11.
|
||||
If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manager due to the API change in cert-manager v0.11.
|
||||
|
||||
1. Uninstall Rancher
|
||||
|
||||
|
||||
+1
-3
@@ -142,7 +142,7 @@ For a how-to guide for setting up a DNS record to route domain traffic to an Ama
|
||||
|
||||
Rancher supports air gap installs using a secure Docker private registry. You must have your own private registry or other means of distributing Docker images to your machines.
|
||||
|
||||
In a later step, when you set up your K3s Kubernetes cluster, you will create a [private registries configuration file]({{<baseurl>}}/k3s/latest/en/installation/private-registry/) with details from this registry.
|
||||
In a later step, when you set up your RKE Kubernetes cluster, you will create a [private registries configuration file]({{<baseurl>}}/rke/latest/en/config-options/private-registries/) with details from this registry.
|
||||
|
||||
If you need help with creating a private registry, please refer to the [official Docker documentation.](https://docs.docker.com/registry/deploying/#run-an-externally-accessible-registry)
|
||||
|
||||
@@ -164,8 +164,6 @@ For an example of one way to set up Linux nodes, refer to this [tutorial]({{<bas
|
||||
|
||||
Rancher supports air gap installs using a Docker private registry on your bastion server. You must have your own private registry or other means of distributing Docker images to your machines.
|
||||
|
||||
In a later step, when you set up your K3s Kubernetes cluster, you will create a [private registries configuration file]({{<baseurl>}}/k3s/latest/en/installation/private-registry/) with details from this registry.
|
||||
|
||||
If you need help with creating a private registry, please refer to the [official Docker documentation.](https://docs.docker.com/registry/)
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
+1
-1
@@ -86,7 +86,7 @@ During upgrade, you create a copy of the data from your current Rancher containe
|
||||
|
||||
|
||||
```
|
||||
docker run --volumes-from rancher-data -v $PWD:/backup busybox tar zcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
|
||||
docker run --volumes-from rancher-data -v "$PWD:/backup" --rm busybox tar zcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
|
||||
```
|
||||
|
||||
**Step Result:** When you enter this command, a series of commands should run.
|
||||
|
||||
@@ -78,7 +78,8 @@ Performance increased in Rancher v2.4.0. For the requirements of Rancher before
|
||||
| X-Large | Up to 1000 | Up to 10,000 | 16 | 64 GB |
|
||||
| XX-Large | Up to 2000 | Up to 20,000 | 32 | 128 GB |
|
||||
|
||||
[Contact Rancher](https://rancher.com/contact/) for more than 2000 clusters and/or 20,000 nodes.
|
||||
Every use case and environment is different. Please [contact Rancher](https://rancher.com/contact/) to review yours.
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab "K3s" %}}
|
||||
@@ -93,7 +94,7 @@ These requirements apply to each host in a [K3s Kubernetes cluster where the Ran
|
||||
| X-Large | Up to 1000 | Up to 10,000 | 16 | 64 GB | 2 cores, 4 GB + 1000 IOPS |
|
||||
| XX-Large | Up to 2000 | Up to 20,000 | 32 | 128 GB | 2 cores, 4 GB + 1000 IOPS |
|
||||
|
||||
[Contact Rancher](https://rancher.com/contact/) for more than 2000 clusters and/or 20,000 nodes.
|
||||
Every use case and environment is different. Please [contact Rancher](https://rancher.com/contact/) to review yours.
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ title: Installing Docker
|
||||
weight: 1
|
||||
---
|
||||
|
||||
For Helm CLI installs, Docker is required to be installed on any node that runs the Rancher server.
|
||||
Docker is required to be installed on nodes where the Rancher server will be installed with Helm or Docker.
|
||||
|
||||
There are a couple of options for installing Docker. One option is to refer to the [official Docker documentation](https://docs.docker.com/install/) about how to install Docker on Linux. The steps will vary based on the Linux distribution.
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ title: Opening Ports with firewalld
|
||||
weight: 1
|
||||
---
|
||||
|
||||
> We recommend disabling firewalld. For Kubernetes 1.19, firewalld must be turned off.
|
||||
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
|
||||
|
||||
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
|
||||
|
||||
|
||||
+8
-2
@@ -131,7 +131,12 @@ deployment "rancher" successfully rolled out
|
||||
|
||||
This option uses `cert-manager` to automatically request and renew [Let's Encrypt](https://letsencrypt.org/) certificates. This is a free service that provides you with a valid certificate as Let's Encrypt is a trusted CA. This configuration uses HTTP validation (`HTTP-01`) so the load balancer must have a public DNS record and be accessible from the internet.
|
||||
|
||||
- Set `hostname` to the public DNS record, set `ingress.tls.source` to `letsEncrypt` and `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices)
|
||||
In the following command,
|
||||
|
||||
- Set `hostname` to the public DNS record that resolves to your load balancer.
|
||||
- Set `ingress.tls.source` to `letsEncrypt`.
|
||||
- Set `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices).
|
||||
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
|
||||
```
|
||||
@@ -140,7 +145,8 @@ helm install rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=letsEncrypt \
|
||||
--set letsEncrypt.email=me@example.org
|
||||
--set letsEncrypt.email=me@example.org \
|
||||
--set letsEncrypt.ingress.class=nginx
|
||||
```
|
||||
|
||||
Wait for Rancher to be rolled out:
|
||||
|
||||
@@ -107,7 +107,7 @@ When installing Rancher with Docker, use the `--features` option. In the below e
|
||||
docker run -d -p 80:80 -p 443:443 \
|
||||
--restart=unless-stopped \
|
||||
rancher/rancher:rancher-latest \
|
||||
--features=<FEATURE-FLAG-NAME-1>=true,<FEATURE-NAME-2>=true # Available as of v2.3.0
|
||||
--features=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true # Available as of v2.3.0
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
title: Installing Docker
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.0-v2.4/en/installation/requirements/installing-docker
|
||||
---
|
||||
|
||||
Docker is required to be installed on nodes where the Rancher server will be installed with Helm or Docker.
|
||||
|
||||
There are a couple of options for installing Docker. One option is to refer to the [official Docker documentation](https://docs.docker.com/install/) about how to install Docker on Linux. The steps will vary based on the Linux distribution.
|
||||
|
||||
Another option is to use one of Rancher's Docker installation scripts, which are available for most recent versions of Docker.
|
||||
|
||||
For example, this command could be used to install Docker 19.03 on Ubuntu:
|
||||
|
||||
```
|
||||
curl https://releases.rancher.com/install-docker/19.03.sh | sh
|
||||
```
|
||||
|
||||
Rancher has installation scripts for every version of upstream Docker that Kubernetes supports. To find out whether a script is available for installing a certain Docker version, refer to this [GitHub repository,](https://github.com/rancher/install-docker) which contains all of Rancher's Docker installation scripts.
|
||||
@@ -35,4 +35,4 @@ kubectl -n cattle-system create secret generic tls-ca \
|
||||
|
||||
# Updating a Private CA Certificate
|
||||
|
||||
Follow the steps on [this page]({{<baseurl>}}/rancher/v2.x/en/installation/resources/update-ca-cert) to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/) or to switch from the default self-signed certificate to a custom certificate.
|
||||
Follow the steps on [this page]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/resources/update-ca-cert) to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/) or to switch from the default self-signed certificate to a custom certificate.
|
||||
+1
-1
@@ -19,7 +19,7 @@ To address these changes, this guide will do two things:
|
||||
1. Explain the cert-manager API changes and link to cert-manager's official documentation for migrating your data
|
||||
|
||||
> **Important:**
|
||||
> If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, you need to reinstall both of them:
|
||||
> If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, you need to reinstall both of them:
|
||||
|
||||
> 1. Take a one-time snapshot of your Kubernetes cluster running Rancher server
|
||||
> 2. Uninstall Rancher, cert-manager, and the CustomResourceDefinition for cert-manager
|
||||
|
||||
+1
-1
@@ -19,7 +19,7 @@ To address these changes, this guide will do two things:
|
||||
1. Explain the cert-manager API changes and link to cert-manager's offficial documentation for migrating your data
|
||||
|
||||
> **Important:**
|
||||
> If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, you need to reinstall both of them:
|
||||
> If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, you need to reinstall both of them:
|
||||
|
||||
> 1. Take a one-time snapshot of your Kubernetes cluster running Rancher server
|
||||
> 2. Uninstall Rancher, cert-manager, and the CustomResourceDefinition for cert-manager
|
||||
|
||||
@@ -15,7 +15,7 @@ Deploy a workload to run an application in one or more containers.
|
||||
|
||||
1. Enter a **Name** for the workload.
|
||||
|
||||
1. Select a [workload type]({{<baseurl>}}/rancher/v2.0-v2.4/en/k8s-in-rancher/workloads/). The workload defaults to a scalable deployment, by can change the workload type by clicking **More options.**
|
||||
1. Select a [workload type]({{<baseurl>}}/rancher/v2.0-v2.4/en/k8s-in-rancher/workloads/). The workload defaults to a scalable deployment, but you can change the workload type by clicking **More options.**
|
||||
|
||||
1. From the **Docker Image** field, enter the name of the Docker image that you want to deploy to the project, optionally prefacing it with the registry host (e.g. `quay.io`, `registry.gitlab.com`, etc.). During deployment, Rancher pulls this image from the specified public or private registry. If no registry host is provided, Rancher will pull the image from [Docker Hub](https://hub.docker.com/explore/). Enter the name exactly as it appears in the registry server, including any required path, and optionally including the desired tag (e.g. `registry.gitlab.com/user/path/image:tag`). If no tag is provided, the `latest` tag will be automatically used.
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ The Rancher API server is built on top of an embedded Kubernetes API server and
|
||||
|
||||
- **Provisioning Kubernetes clusters:** The Rancher API server can [provision Kubernetes]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-provisioning/) on existing nodes, or perform [Kubernetes upgrades.]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/upgrading-kubernetes)
|
||||
- **Catalog management:** Rancher provides the ability to use a [catalog of Helm charts]({{<baseurl>}}/rancher/v2.0-v2.4/en/catalog/) that make it easy to repeatedly deploy applications.
|
||||
- **Managing projects:** A project is a group of multiple namespaces and access control policies within a cluster. A project is a Rancher concept, not a Kubernetes concept, which allows you manage multiple namespaces as a group and perform Kubernetes operations in them. The Rancher UI provides features for [project administration]({{<baseurl>}}/rancher/v2.0-v2.4/en/project-admin/) and for [managing applications within projects.]({{<baseurl>}}/rancher/v2.0-v2.4/en/k8s-in-rancher/)
|
||||
- **Managing projects:** A project is a group of multiple namespaces and access control policies within a cluster. A project is a Rancher concept, not a Kubernetes concept, which allows you to manage multiple namespaces as a group and perform Kubernetes operations in them. The Rancher UI provides features for [project administration]({{<baseurl>}}/rancher/v2.0-v2.4/en/project-admin/) and for [managing applications within projects.]({{<baseurl>}}/rancher/v2.0-v2.4/en/k8s-in-rancher/)
|
||||
- **Pipelines:** Setting up a [pipeline]({{<baseurl>}}/rancher/v2.0-v2.4/en/project-admin/pipelines/) can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects.
|
||||
- **Istio:** Our [integration with Istio]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/tools/istio/) is designed so that a Rancher operator, such as an administrator or cluster owner, can deliver Istio to developers. Then developers can use Istio to enforce security policies, troubleshoot problems, or manage traffic for green/blue deployments, canary deployments, or A/B testing.
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ Three etcd nodes is generally sufficient for smaller clusters and five etcd node
|
||||
|
||||
### Controlplane Nodes
|
||||
|
||||
Controlplane nodes run the Kubernetes API server, scheduler, and controller manager. These nodes take care of routine tasks to ensure that your cluster maintains your configuration. Because all cluster data is stored on your etcd nodes, control plane nodes are stateless. You can run control plane on a single node, although two or more nodes are recommended for redundancy. Additionally, a single node can share the control plane and etcd roles.
|
||||
Controlplane nodes run the Kubernetes API server, scheduler, and controller manager. These nodes take care of routine tasks to ensure that your cluster maintains your configuration. Because all cluster data is stored on your etcd nodes, control plane nodes are stateless. You can run control plane on a single node, although three or more nodes are recommended for redundancy. Additionally, a single node can share the control plane and etcd roles.
|
||||
|
||||
### Worker Nodes
|
||||
|
||||
@@ -69,4 +69,4 @@ For high-availability installations of Rancher, Helm is the tool used to install
|
||||
|
||||
Helm is the package management tool of choice for Kubernetes. Helm charts provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh).
|
||||
|
||||
For more information on service accounts and cluster role binding, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
|
||||
For more information on service accounts and cluster role binding, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
|
||||
|
||||
@@ -42,6 +42,8 @@ Following project creation, you can add users as project members so that they ca
|
||||
>
|
||||
>- Users assigned the `Owner` or `Member` role for a project automatically inherit the `namespace creation` role. However, this role is a [Kubernetes ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole), meaning its scope extends to all projects in the cluster. Therefore, users explicitly assigned the `Owner` or `Member` role for a project can create namespaces in other projects they're assigned to, even with only the `Read Only` role assigned.
|
||||
>
|
||||
>- By default, the Rancher role of `project-member` inherits from the `Kubernetes-edit` role, and the `project-owner` role inherits from the `Kubernetes-admin` role. As such, both `project-member` and `project-owner` roles will allow for namespace management, including the ability to create and delete namespaces.
|
||||
>
|
||||
>- For `Custom` roles, you can modify the list of individual roles available for assignment.
|
||||
>
|
||||
> - To add roles to the list, [Add a Custom Role]({{<baseurl>}}/rancher/v2.0-v2.4/en/admin-settings/rbac/default-custom-roles).
|
||||
|
||||
@@ -45,4 +45,4 @@ Edit [resource quotas]({{<baseurl>}}/rancher/v2.0-v2.4/en/k8s-in-rancher/project
|
||||
|
||||
1. Click **Create**.
|
||||
|
||||
**Result:** The resource quota is applied to your project and namespaces. When you add more namespaces in the future, Rancher validates that the project can accommodate the namespace. If the project can't allocate the resources, Rancher won't let you save your changes.
|
||||
**Result:** The resource quota is applied to your project and namespaces. When you add more namespaces in the future, Rancher validates that the project can accommodate the namespace. If the project can't allocate the resources, you may still create namespaces, but they will be given a resource quota of 0. Subsequently, Rancher will not allow you to create any resources restricted by this quota.
|
||||
|
||||
@@ -43,4 +43,4 @@ For details on project-level logging, see [this section.](./project-logging)
|
||||
|
||||
_Available as of v2.2.0_
|
||||
|
||||
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/tools/monitoring)
|
||||
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/tools/cluster-monitoring)
|
||||
|
||||
@@ -5,6 +5,8 @@ weight: 100
|
||||
---
|
||||
The following steps will quickly deploy a Rancher Server on AWS with a single node cluster attached.
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
>**Note**
|
||||
|
||||
@@ -5,6 +5,8 @@ weight: 100
|
||||
---
|
||||
The following steps will quickly deploy a Rancher Server on DigitalOcean with a single node cluster attached.
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
>**Note**
|
||||
|
||||
@@ -5,6 +5,8 @@ weight: 100
|
||||
---
|
||||
The following steps will quickly deploy a Rancher server on GCP in a single-node RKE Kubernetes cluster, with a single-node downstream Kubernetes cluster attached.
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
>**Note**
|
||||
|
||||
@@ -6,6 +6,8 @@ weight: 100
|
||||
|
||||
The following steps will quickly deploy a Rancher server on Azure in a single-node RKE Kubernetes cluster, with a single-node downstream Kubernetes cluster attached.
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
>**Note**
|
||||
|
||||
+4
-2
@@ -8,6 +8,8 @@ Howdy Partner! This tutorial walks you through:
|
||||
- Creation of your first cluster
|
||||
- Deployment of an application, Nginx
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Quick Start Outline
|
||||
|
||||
This Quick Start Guide is divided into different tasks for easier consumption.
|
||||
@@ -96,13 +98,13 @@ In this task, you can use the versatile **Custom** option. This option lets you
|
||||
|
||||
11. When you finish running the command on your Linux host, click **Done**.
|
||||
|
||||
**Result:**
|
||||
**Result:**
|
||||
|
||||
Your cluster is created and assigned a state of **Provisioning.** Rancher is standing up your cluster.
|
||||
|
||||
You can access your cluster after its state is updated to **Active.**
|
||||
|
||||
**Active** clusters are assigned two Projects:
|
||||
**Active** clusters are assigned two Projects:
|
||||
|
||||
- `Default`, containing the `default` namespace
|
||||
- `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces
|
||||
|
||||
+3
-1
@@ -4,6 +4,8 @@ weight: 200
|
||||
---
|
||||
The following steps quickly deploy a Rancher Server with a single node cluster attached.
|
||||
|
||||
>**Note:** The intent of these guides is to quickly launch a sandbox that you can use to evaluate Rancher. These guides are not intended for production environments. For comprehensive setup instructions, see [Installation]({{<baseurl>}}/rancher/v2.0-v2.4/en/installation/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [Vagrant](https://www.vagrantup.com): Vagrant is required as this is used to provision the machine based on the Vagrantfile.
|
||||
@@ -14,7 +16,7 @@ The following steps quickly deploy a Rancher Server with a single node cluster a
|
||||
- Vagrant will require plugins to create VirtualBox VMs. Install them with the following commands:
|
||||
|
||||
`vagrant plugin install vagrant-vboxmanage`
|
||||
|
||||
|
||||
`vagrant plugin install vagrant-vbguest`
|
||||
|
||||
## Getting Started
|
||||
|
||||
@@ -158,7 +158,7 @@ kubectl run -i --restart=Never --rm test-${RANDOM} --image=ubuntu --overrides='{
|
||||
Enabling query logging can be done by enabling the [log plugin](https://coredns.io/plugins/log/) in the Corefile configuration in the configmap `coredns`. You can do so by using `kubectl -n kube-system edit configmap coredns` or use the command below to replace the configuration in place:
|
||||
|
||||
```
|
||||
kubectl get configmap -n kube-system coredns -o json | kubectl get configmap -n kube-system coredns -o json | sed -e 's_loadbalance_log\\n loadbalance_g' | kubectl apply -f -
|
||||
kubectl get configmap -n kube-system coredns -o json | sed -e 's_loadbalance_log\\n loadbalance_g' | kubectl apply -f -
|
||||
```
|
||||
|
||||
All queries will now be logged and can be checked using the command in [Check CoreDNS logging](#check-coredns-logging).
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: Rancher 2.5.7-2.5.9
|
||||
title: Rancher 2.5
|
||||
weight: 2
|
||||
showBreadcrumb: false
|
||||
---
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: "Rancher 2.5"
|
||||
shortTitle: "Rancher 2.5"
|
||||
description: "Rancher adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
|
||||
metaTitle: "Rancher 2.5.7-2.5.9 Docs: What is New?"
|
||||
metaTitle: "Rancher 2.5 Docs: What is New?"
|
||||
metaDescription: "Rancher 2 adds significant value on top of Kubernetes: managing hundreds of clusters from one interface, centralizing RBAC, enabling monitoring and alerting. Read more."
|
||||
insertOneSix: false
|
||||
weight: 2
|
||||
|
||||
@@ -101,7 +101,7 @@ The table below details the parameters for the group schema configuration.
|
||||
| Search Attribute | Attribute used to construct search filters when adding groups to clusters or projects. See description of user schema `Search Attribute`. |
|
||||
| Search Filter | This filter gets applied to the list of groups that is searched when Rancher attempts to add groups to a site access list or tries to add groups to clusters or projects. For example, a group search filter could be <code>(|(cn=group1)(cn=group2))</code>. Note: If the search filter does not use [valid AD search syntax,](https://docs.microsoft.com/en-us/windows/win32/adsi/search-filter-syntax) the list of groups will be empty. |
|
||||
| Group DN Attribute | The name of the group attribute whose format matches the values in the user attribute describing a the user's memberships. See `User Member Attribute`. |
|
||||
| Nested Group Membership | This settings defines whether Rancher should resolve nested group memberships. Use only if your organisation makes use of these nested memberships (ie. you have groups that contain other groups as members. We advise avoiding nested groups when possible). |
|
||||
| Nested Group Membership | This settings defines whether Rancher should resolve nested group memberships. Use only if your organization makes use of these nested memberships (i.e., you have groups that contain other groups as members. We advise avoiding nested groups when possible). |
|
||||
|
||||
---
|
||||
|
||||
@@ -153,7 +153,7 @@ This command performs an LDAP search with the search base set to the domain root
|
||||
|
||||
Since in this case the user's DN is `CN=John Doe,CN=Users,DC=acme,DC=com` [5], we should configure the **User Search Base** with the parent node DN `CN=Users,DC=acme,DC=com`.
|
||||
|
||||
Similarly, based on the DN of the group referenced in the **memberOf** attribute [4], the correct value for the **Group Search Base** would be the parent node of that value, ie. `OU=Groups,DC=acme,DC=com`.
|
||||
Similarly, based on the DN of the group referenced in the **memberOf** attribute [4], the correct value for the **Group Search Base** would be the parent node of that value, i.e., `OU=Groups,DC=acme,DC=com`.
|
||||
|
||||
### Identify User Schema
|
||||
|
||||
@@ -166,7 +166,7 @@ The output of the above `ldapsearch` query also allows to determine the correct
|
||||
|
||||
> **Note:**
|
||||
>
|
||||
> If the AD users in our organisation were to authenticate with their UPN (e.g. jdoe@acme.com) instead of the short logon name, then we would have to set the `Login Attribute` to **userPrincipalName** instead.
|
||||
> If the AD users in our organization were to authenticate with their UPN (e.g. jdoe@acme.com) instead of the short logon name, then we would have to set the `Login Attribute` to **userPrincipalName** instead.
|
||||
|
||||
We'll also set the `Search Attribute` parameter to **sAMAccountName|name**. That way users can be added to clusters/projects in the Rancher UI either by entering their username or full name.
|
||||
|
||||
|
||||
@@ -25,36 +25,87 @@ If your organization uses Keycloak Identity Provider (IdP) for user authenticati
|
||||
|
||||
><sup>1</sup>: Optionally, you can enable either one or both of these settings.
|
||||
><sup>2</sup>: Rancher SAML metadata won't be generated until a SAML provider is configured and saved.
|
||||
|
||||
|
||||
{{< img "/img/rancher/keycloak/keycloak-saml-client-configuration.png" "">}}
|
||||
|
||||
|
||||
- In the new SAML client, create Mappers to expose the users fields
|
||||
- Add all "Builtin Protocol Mappers"
|
||||
{{< img "/img/rancher/keycloak/keycloak-saml-client-builtin-mappers.png" "">}}
|
||||
- Create a new "Group list" mapper to map the member attribute to a user's groups
|
||||
{{< img "/img/rancher/keycloak/keycloak-saml-client-group-mapper.png" "">}}
|
||||
- Export a `metadata.xml` file from your Keycloak client:
|
||||
From the `Installation` tab, choose the `SAML Metadata IDPSSODescriptor` format option and download your file.
|
||||
|
||||
>**Note**
|
||||
> Keycloak versions 6.0.0 and up no longer provide the IDP metadata under the `Installation` tab.
|
||||
> You can still get the XML from the following url:
|
||||
>
|
||||
> `https://{KEYCLOAK-URL}/auth/realms/{REALM-NAME}/protocol/saml/descriptor`
|
||||
>
|
||||
> The XML obtained from this URL contains `EntitiesDescriptor` as the root element. Rancher expects the root element to be `EntityDescriptor` rather than `EntitiesDescriptor`. So before passing this XML to Rancher, follow these steps to adjust it:
|
||||
>
|
||||
> * Copy all the attributes from `EntitiesDescriptor` to the `EntityDescriptor` that are not present.
|
||||
> * Remove the `<EntitiesDescriptor>` tag from the beginning.
|
||||
> * Remove the `</EntitiesDescriptor>` from the end of the xml.
|
||||
>
|
||||
> You are left with something similar as the example below:
|
||||
>
|
||||
> ```
|
||||
> <EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" entityID="https://{KEYCLOAK-URL}/auth/realms/{REALM-NAME}">
|
||||
> ....
|
||||
> </EntityDescriptor>
|
||||
> ```
|
||||
{{< img "/img/rancher/keycloak/keycloak-saml-client-group-mapper.png" "">}}
|
||||
|
||||
## Getting the IDP Metadata
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Keycloak 5 and earlier" %}}
|
||||
To get the IDP metadata, export a `metadata.xml` file from your Keycloak client.
|
||||
From the **Installation** tab, choose the **SAML Metadata IDPSSODescriptor** format option and download your file.
|
||||
{{% /tab %}}
|
||||
{{% tab "Keycloak 6-13" %}}
|
||||
|
||||
1. From the **Configure** section, click the **Realm Settings** tab.
|
||||
1. Click the **General** tab.
|
||||
1. From the **Endpoints** field, click **SAML 2.0 Identity Provider Metadata**.
|
||||
|
||||
Verify the IDP metadata contains the following attributes:
|
||||
|
||||
```
|
||||
xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
|
||||
xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
|
||||
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
|
||||
```
|
||||
|
||||
Some browsers, such as Firefox, may render/process the document such that the contents appear to have been modified, and some attributes appear to be missing. In this situation, use the raw response data that can be found using your browser.
|
||||
|
||||
The following is an example process for Firefox, but will vary slightly for other browsers:
|
||||
|
||||
1. Press **F12** to access the developer console.
|
||||
1. Click the **Network** tab.
|
||||
1. From the table, click the row containing `descriptor`.
|
||||
1. From the details pane, click the **Response** tab.
|
||||
1. Copy the raw response data.
|
||||
|
||||
The XML obtained contains `EntitiesDescriptor` as the root element. Rancher expects the root element to be `EntityDescriptor` rather than `EntitiesDescriptor`. So before passing this XML to Rancher, follow these steps to adjust it:
|
||||
|
||||
1. Copy all the attributes from `EntitiesDescriptor` to the `EntityDescriptor` that are not present.
|
||||
1. Remove the `<EntitiesDescriptor>` tag from the beginning.
|
||||
1. Remove the `</EntitiesDescriptor>` from the end of the xml.
|
||||
|
||||
You are left with something similar as the example below:
|
||||
|
||||
```
|
||||
<EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" entityID="https://{KEYCLOAK-URL}/auth/realms/{REALM-NAME}">
|
||||
....
|
||||
</EntityDescriptor>
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Keycloak 14+" %}}
|
||||
|
||||
1. From the **Configure** section, click the **Realm Settings** tab.
|
||||
1. Click the **General** tab.
|
||||
1. From the **Endpoints** field, click **SAML 2.0 Identity Provider Metadata**.
|
||||
|
||||
Verify the IDP metadata contains the following attributes:
|
||||
|
||||
```
|
||||
xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
|
||||
xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
|
||||
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
|
||||
```
|
||||
|
||||
Some browsers, such as Firefox, may render/process the document such that the contents appear to have been modified, and some attributes appear to be missing. In this situation, use the raw response data that can be found using your browser.
|
||||
|
||||
The following is an example process for Firefox, but will vary slightly for other browsers:
|
||||
|
||||
1. Press **F12** to access the developer console.
|
||||
1. Click the **Network** tab.
|
||||
1. From the table, click the row containing `descriptor`.
|
||||
1. From the details pane, click the **Response** tab.
|
||||
1. Copy the raw response data.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
## Configuring Keycloak in Rancher
|
||||
|
||||
|
||||
@@ -30,6 +30,8 @@ Note that these URLs will not return valid data until the authentication configu
|
||||
|
||||
1. **Groups Field**: Make entries for managing group memberships (example: `memberOf`).
|
||||
|
||||
1. **Entity ID Field** (optional): The published, protocol-dependent, unique identifier of your partner. This ID defines your organization as the entity operating the server for SAML 2.0 transactions. This ID may have been obtained out-of-band or via a SAML metadata file.
|
||||
|
||||
1. **Rancher API Host**: Enter the URL for your Rancher Server.
|
||||
|
||||
1. **Private Key** and **Certificate**: This is a key-certificate pair to create a secure shell between Rancher and your IdP.
|
||||
|
||||
@@ -5,7 +5,7 @@ aliases:
|
||||
- /rancher/v2.x/en/admin-settings/config-private-registry/
|
||||
---
|
||||
|
||||
You might want to use a private Docker registry to share your custom base images within your organization. With a private registry, you can keep a private, consistent, and centralized source of truth for the Docker images that are used in your clusters.
|
||||
You might want to use a private container registry to share your custom base images within your organization. With a private registry, you can keep a private, consistent, and centralized source of truth for the container images that are used in your clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the global default registry through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
|
||||
@@ -86,6 +86,10 @@ _Project roles_ are roles that can be used to grant users access to a project. T
|
||||
|
||||
These users can manage project-scoped resources like namespaces and workloads, but cannot manage other project members.
|
||||
|
||||
>**Note:**
|
||||
>
|
||||
>By default, the Rancher role of `project-member` inherits from the `Kubernetes-edit` role, and the `project-owner` role inherits from the `Kubernetes-admin` role. As such, both `project-member` and `project-owner` roles will allow for namespace management, including the ability to create and delete namespaces.
|
||||
|
||||
- **Read Only:**
|
||||
|
||||
These users can view everything in the project but cannot create, update, or delete anything.
|
||||
@@ -94,7 +98,6 @@ _Project roles_ are roles that can be used to grant users access to a project. T
|
||||
>
|
||||
>Users assigned the `Owner` or `Member` role for a project automatically inherit the `namespace creation` role. However, this role is a [Kubernetes ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole), meaning its scope extends to all projects in the cluster. Therefore, users explicitly assigned the `owner` or `member` role for a project can create namespaces in other projects they're assigned to, even with only the `Read Only` role assigned.
|
||||
|
||||
|
||||
#### Custom Project Roles
|
||||
|
||||
Rancher lets you assign _custom project roles_ to a standard user instead of the typical `Owner`, `Member`, or `Read Only` roles. These roles can be either a built-in custom project role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a standard user within a project. See the table below for a list of built-in custom project roles.
|
||||
|
||||
@@ -52,3 +52,17 @@ Most collections can be sorted on the server-side by common fields using HTTP qu
|
||||
## Pagination
|
||||
|
||||
API responses are paginated with a limit of 100 resources per page by default. This can be changed with the `limit` query parameter, up to a maximum of 1000, e.g. `/v3/pods?limit=1000`. The `pagination` map in collection responses tells you whether or not you have the full result set and has a link to the next page if you do not.
|
||||
|
||||
## Capturing Rancher API Calls
|
||||
|
||||
You can use browser developer tools to capture how the Rancher API is called. For example, you could follow these steps to use the Chrome developer tools to get the API call for provisioning an RKE cluster:
|
||||
|
||||
1. In the Rancher UI, go to **Cluster Management** and click **Create.**
|
||||
1. Click one of the cluster types. This example uses Digital Ocean.
|
||||
1. Fill out the form with a cluster name and node template, but don't click **Create**.
|
||||
1. You will need to open the developer tools before the cluster creation to see the API call being recorded. To open the tools, right-click on the Rancher UI and click **Inspect.**
|
||||
1. In the developer tools, click the **Network** tab.
|
||||
1. On the **Network** tab, make sure **Fetch/XHR** is selected.
|
||||
1. In the Rancher UI, click **Create**. In the developer tools, you should see a new network request with the name `cluster?_replace=true`.
|
||||
1. Right-click `cluster?_replace=true` and click **Copy > Copy as cURL.**
|
||||
1. Paste the result into any text editor. You will be able to see the POST request, including the URL it was sent to, all of the headers, and the full body of the request. This command can be used to create a cluster from the command line. Note: The request should be stored in a safe place because it contains credentials.
|
||||
@@ -63,6 +63,8 @@ The Backup and Restore custom resources can be created in the Rancher UI, or by
|
||||
|
||||
The `rancher-backup` operator can be installed from the Rancher UI, or with the Helm CLI. In both cases, the `rancher-backup` Helm chart is installed on the Kubernetes cluster running the Rancher server. It is a cluster-admin only feature and available only for the **local** cluster. (*If you do not see `rancher-backup` in the Rancher UI, you may have selected the wrong cluster.*)
|
||||
|
||||
>**NOTE:** There are two known issues in Fleet that occur after performing a restoration using the backup-restore-operator: Fleet agents are inoperable and clientSecretName and helmSecretName are not included in Fleet gitrepos. Refer [here]({{<baseurl>}}rancher/v2.5/en/deploy-across-clusters/fleet/#troubleshooting) for workarounds.
|
||||
|
||||
### Installing rancher-backup with the Rancher UI
|
||||
|
||||
1. In the Rancher UI's Cluster Manager, choose the cluster named **local**
|
||||
|
||||
@@ -7,11 +7,13 @@ aliases:
|
||||
- /rancher/v2.x/en/backups/v2.5/back-up-rancher/
|
||||
---
|
||||
|
||||
In this section, you'll learn how to back up Rancher running on any Kubernetes cluster. To backup Rancher installed with Docker, refer the instructions for [single node backups]({{<baseurl>}}/rancher/v2.5/en/backups/v2.5/docker-installs/docker-backups)
|
||||
In this section, you'll learn how to back up Rancher running on any Kubernetes cluster. To backup Rancher installed with Docker, refer to the instructions for [single node backups]({{<baseurl>}}/rancher/v2.5/en/backups/v2.5/docker-installs/docker-backups).
|
||||
|
||||
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
|
||||
|
||||
> When restoring a backup into a new Rancher setup, the version of the new setup should be the same as the one where the backup is made.
|
||||
Note that the rancher-backup operator version 1.x.x is for Rancher v2.5.x.
|
||||
|
||||
> When restoring a backup into a new Rancher setup, the version of the new setup should be the same as the one where the backup is made. The Kubernetes version should also be considered when restoring a backup, since the supported apiVersion in the cluster and in the backup file could be different.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
@@ -28,6 +30,8 @@ Backups are created as .tar.gz files. These files can be pushed to S3 or Minio,
|
||||
1. Click **Rancher Backups.**
|
||||
1. Configure the default storage location. For help, refer to the [storage configuration section.](../configuration/storage-config)
|
||||
|
||||
>**NOTE:** There are two known issues in Fleet that occur after performing a restoration using the backup-restore-operator: Fleet agents are inoperable and clientSecretName and helmSecretName are not included in Fleet gitrepos. Refer [here]({{<baseurl>}}/rancher/v2.5/en/deploy-across-clusters/fleet/#troubleshooting) for workarounds.
|
||||
|
||||
### 2. Perform a Backup
|
||||
|
||||
To perform a backup, a custom resource of type Backup must be created.
|
||||
|
||||
@@ -21,13 +21,19 @@ Rancher can be installed on any Kubernetes cluster, including hosted Kubernetes
|
||||
- [K3s Kubernetes installation docs]({{<baseurl>}}/k3s/latest/en/installation/)
|
||||
|
||||
### 1. Install the rancher-backup Helm chart
|
||||
Install version 1.x.x of the rancher-backup chart.
|
||||
Install version 1.x.x of the rancher-backup chart. The following assumes a connected environment with access to DockerHub:
|
||||
|
||||
```
|
||||
helm repo add rancher-charts https://charts.rancher.io
|
||||
helm repo update
|
||||
helm install rancher-backup-crd rancher-charts/rancher-backup-crd -n cattle-resources-system --create-namespace --version $CHART_VERSION
|
||||
helm install rancher-backup rancher-charts/rancher-backup -n cattle-resources-system --version $CHART_VERSION
|
||||
```
|
||||
</br>
|
||||
For an **air-gapped environment**, use the option below to pull the `backup-restore-operator` image from your private registry when installing the rancher-backup-crd helm chart.
|
||||
```
|
||||
--set image.repository $REGISTRY/rancher/backup-restore-operator
|
||||
```
|
||||
|
||||
### 2. Restore from backup using a Restore custom resource
|
||||
|
||||
@@ -70,20 +76,19 @@ spec:
|
||||
endpoint: s3.us-west-2.amazonaws.com
|
||||
```
|
||||
|
||||
> **Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
|
||||
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
|
||||
>**Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
|
||||
|
||||
```
|
||||
kubectl create secret generic encryptionconfig \
|
||||
--from-file=./encryption-provider-config.yaml \
|
||||
-n cattle-resources-system
|
||||
```
|
||||
|
||||
Then apply the resource:
|
||||
|
||||
```
|
||||
kubectl apply -f migrationResource.yaml
|
||||
```
|
||||
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
|
||||
```
|
||||
kubectl create secret generic encryptionconfig \
|
||||
--from-file=./encryption-provider-config.yaml \
|
||||
-n cattle-resources-system
|
||||
```
|
||||
|
||||
1. Then apply the resource:
|
||||
```
|
||||
kubectl apply -f migrationResource.yaml
|
||||
```
|
||||
|
||||
### 3. Install cert-manager
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ When you set up your high-availability Rancher installation, consider the follow
|
||||
Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on.
|
||||
|
||||
### Make sure nodes are configured correctly for Kubernetes ###
|
||||
It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/performance.md)
|
||||
It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.4/op-guide/performance/).
|
||||
|
||||
### When using RKE: Back up the Statefile
|
||||
RKE keeps record of the cluster state in a file called `cluster.rkestate`. This file is important for the recovery of a cluster and/or the continued maintenance of the cluster through RKE. Because this file contains certificate material, we strongly recommend encrypting this file before backing up. After each run of `rke up` you should backup the state file.
|
||||
|
||||
@@ -53,7 +53,7 @@ https://rancher.com/docs/rancher/v2.5/en/installation/requirements/
|
||||
|
||||
### Leverage VM Templates to Construct the Environment
|
||||
|
||||
To facilitate consistency across the deployed Virtual Machines across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customisation options.
|
||||
To facilitate the consistency of Virtual Machines deployed across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customization options.
|
||||
|
||||
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Rancher Cluster Nodes Across ESXi Hosts
|
||||
|
||||
|
||||
@@ -1,13 +1,23 @@
|
||||
---
|
||||
title: Using the Rancher Command Line Interface
|
||||
description: The Rancher CLI is a unified tool that you can use to interact with Rancher. With it, you can operate Rancher using a command line interface rather than the GUI
|
||||
metaTitle: "Using the Rancher Command Line Interface "
|
||||
metaDescription: "The Rancher CLI is a unified tool that you can use to interact with Rancher. With it, you can operate Rancher using a command line interface rather than the GUI"
|
||||
title: CLI with Rancher
|
||||
description: Interact with Rancher using command line interface (CLI) tools from your workstation.
|
||||
weight: 21
|
||||
aliases:
|
||||
- /rancher/v2.5/en/cluster-admin/cluster-access/cli
|
||||
---
|
||||
|
||||
- [Rancher CLI](#rancher-cli)
|
||||
- [Download Rancher CLI](#download-rancher-cli)
|
||||
- [Requirements](#requirements)
|
||||
- [CLI Authentication](#cli-authentication)
|
||||
- [Project Selection](#project-selection)
|
||||
- [Commands](#commands)
|
||||
- [Rancher CLI Help](#rancher-cli-help)
|
||||
- [Limitations](#limitations)
|
||||
- [kubectl](#kubectl)
|
||||
- [kubectl Utility](#kubectl-utility)
|
||||
- [Authentication with kubectl and kubeconfig Tokens with TTL](#authentication-with-kubectl-and-kubeconfig-tokens-with-ttl)
|
||||
|
||||
# Rancher CLI
|
||||
|
||||
The Rancher CLI (Command Line Interface) is a unified tool that you can use to interact with Rancher. With this tool, you can operate Rancher using a command line rather than the GUI.
|
||||
|
||||
### Download Rancher CLI
|
||||
@@ -38,11 +48,11 @@ Before you can perform any commands, you must select a Rancher project to perfor
|
||||
**Example: `./rancher context switch` Output**
|
||||
```
|
||||
User:rancher-cli-directory user$ ./rancher context switch
|
||||
NUMBER CLUSTER NAME PROJECT ID PROJECT NAME
|
||||
1 cluster-2 c-7q96s:p-h4tmb project-2
|
||||
2 cluster-2 c-7q96s:project-j6z6d Default
|
||||
3 cluster-1 c-lchzv:p-xbpdt project-1
|
||||
4 cluster-1 c-lchzv:project-s2mch Default
|
||||
NUMBER CLUSTER NAME PROJECT ID PROJECT NAME
|
||||
1 cluster-2 c-7q96s:p-h4tmb project-2
|
||||
2 cluster-2 c-7q96s:project-j6z6d Default
|
||||
3 cluster-1 c-lchzv:p-xbpdt project-1
|
||||
4 cluster-1 c-lchzv:project-s2mch Default
|
||||
Select a Project:
|
||||
```
|
||||
|
||||
@@ -53,14 +63,16 @@ INFO[0005] Setting new context to project project-1
|
||||
INFO[0005] Saving config to /Users/markbishop/.rancher/cli2.json
|
||||
```
|
||||
|
||||
Ensure you can run `rancher kubectl get pods` successfully.
|
||||
|
||||
### Commands
|
||||
|
||||
The following commands are available for use in Rancher CLI.
|
||||
|
||||
| Command | Result |
|
||||
|---|---|
|
||||
| `apps, [app]` | Performs operations on catalog applications (i.e. individual [Helm charts](https://docs.helm.sh/developing_charts/) or Rancher charts. |
|
||||
| `catalog` | Performs operations on [catalogs]({{<baseurl>}}/rancher/v2.5/en/catalog/). |
|
||||
| `apps, [app]` | Performs operations on catalog applications (i.e., individual [Helm charts](https://docs.helm.sh/developing_charts/)) or Rancher charts. |
|
||||
| `catalog` | Performs operations on [catalogs]({{<baseurl>}}/rancher/v2.5/en/helm-charts/). |
|
||||
| `clusters, [cluster]` | Performs operations on your [clusters]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/). |
|
||||
| `context` | Switches between Rancher [projects]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/). For an example, see [Project Selection](#project-selection). |
|
||||
| `inspect [OPTIONS] [RESOURCEID RESOURCENAME]` | Displays details about [Kubernetes resources](https://kubernetes.io/docs/reference/kubectl/cheatsheet/#resource-types) or Rancher resources (i.e.: [projects]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/) and [workloads]({{<baseurl>}}/rancher/v2.5/en/k8s-in-rancher/workloads/)). Specify resources by name or ID. |
|
||||
@@ -80,3 +92,38 @@ The following commands are available for use in Rancher CLI.
|
||||
Once logged into Rancher Server using the CLI, enter `./rancher --help` for a list of commands.
|
||||
|
||||
All commands accept the `--help` flag, which documents each command's usage.
|
||||
|
||||
### Limitations
|
||||
|
||||
The Rancher CLI **cannot** be used to install [dashboard apps or Rancher feature charts](../helm-charts/).
|
||||
|
||||
# kubectl
|
||||
|
||||
Interact with Rancher using kubectl.
|
||||
|
||||
### kubectl Utility
|
||||
|
||||
Install the `kubectl` utility. See [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
|
||||
|
||||
Configure kubectl by visiting your cluster in the Rancher Web UI, clicking on `Kubeconfig`, copying contents, and putting them into your `~/.kube/config` file.
|
||||
|
||||
Run `kubectl cluster-info` or `kubectl get pods` successfully.
|
||||
|
||||
### Authentication with kubectl and kubeconfig Tokens with TTL
|
||||
|
||||
_Requirements_
|
||||
|
||||
If admins have [enforced TTL on kubeconfig tokens]({{<baseurl>}}/rancher/v2.5/en/api/api-tokens/#setting-ttl-on-kubeconfig-tokens), the kubeconfig file requires the [Rancher CLI](../cli) to be present in your PATH when you run `kubectl`. Otherwise, you’ll see an error like:
|
||||
`Unable to connect to the server: getting credentials: exec: exec: "rancher": executable file not found in $PATH`.
|
||||
|
||||
This feature enables kubectl to authenticate with the Rancher server and get a new kubeconfig token when required. The following auth providers are currently supported:
|
||||
|
||||
1. Local
|
||||
2. Active Directory (LDAP only)
|
||||
3. FreeIPA
|
||||
4. OpenLDAP
|
||||
5. SAML providers: Ping, Okta, ADFS, Keycloak, Shibboleth
|
||||
|
||||
When you first run kubectl, for example, `kubectl get pods`, it will ask you to pick an auth provider and log in with the Rancher server.
|
||||
The kubeconfig token is cached in the path where you run kubectl under `./.cache/token`. This token is valid until [it expires](../../api/api-tokens/#setting-ttl-on-kubeconfig-tokens-period), or [gets deleted from the Rancher server](../../api/api-tokens/#deleting-tokens).
|
||||
Upon expiration, the next `kubectl get pods` will ask you to log in with the Rancher server again.
|
||||
|
||||
@@ -12,9 +12,29 @@ By default, Kubernetes clusters require certificates and Rancher launched Kubern
|
||||
Certificates can be rotated for the following services:
|
||||
|
||||
- etcd
|
||||
- kubelet
|
||||
- kubelet (node certificate)
|
||||
- kubelet (serving certificate, if [enabled]({{<baseurl>}}/rke/latest/en/config-options/services/#kubelet-options))
|
||||
- kube-apiserver
|
||||
- kube-proxy
|
||||
- kube-scheduler
|
||||
- kube-controller-manager
|
||||
|
||||
|
||||
### Certificate Rotation
|
||||
|
||||
Rancher launched Kubernetes clusters have the ability to rotate the auto-generated certificates through the UI.
|
||||
|
||||
1. In the **Global** view, navigate to the cluster that you want to rotate certificates.
|
||||
|
||||
2. Select **⋮ > Rotate Certificates**.
|
||||
|
||||
3. Select which certificates that you want to rotate.
|
||||
|
||||
* Rotate all Service certificates (keep the same CA)
|
||||
* Rotate an individual service and choose one of the services from the drop-down menu
|
||||
|
||||
4. Click **Save**.
|
||||
|
||||
**Results:** The selected certificates will be rotated and the related services will be restarted to start using the new certificate.
|
||||
|
||||
> **Note:** Even though the RKE CLI can use custom certificates for the Kubernetes cluster components, Rancher currently doesn't allow the ability to upload these in Rancher launched Kubernetes clusters.
|
||||
|
||||
+2
-2
@@ -16,7 +16,7 @@ Complete each drop-down and field using the information obtained for your IAM po
|
||||
| Setting | Description |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| Region | From the drop-down choose the geographical region in which to build your cluster. |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/user-settings/cloud-credentials/) |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/user-settings/cloud-credentials/) |
|
||||
|
||||
### Service Role
|
||||
|
||||
@@ -186,7 +186,7 @@ Complete each drop-down and field using the information obtained for your IAM po
|
||||
| Setting | Description |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| Region | From the drop-down choose the geographical region in which to build your cluster. |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/user-settings/cloud-credentials/) |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/user-settings/cloud-credentials/) |
|
||||
|
||||
### Service Role
|
||||
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ If restricting outgoing internet access is not a concern for your organization,
|
||||
>**Note**
|
||||
>This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
|
||||
|
||||
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
|
||||
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.5/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
|
||||
|
||||
### Private Control Plane Endpoint
|
||||
|
||||
|
||||
@@ -42,11 +42,11 @@ The following table lists which node options are available for each type of clus
|
||||
| [Download Keys](#ssh-into-a-node-hosted-by-an-infrastructure-provider) | ✓ | | | | | Download SSH key in order to SSH into the node. |
|
||||
| [Node Scaling](#scaling-nodes) | ✓ | | | ✓ | | Scale the number of nodes in the node pool up or down. |
|
||||
|
||||
[1]: {{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/
|
||||
[2]: {{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/
|
||||
[3]: {{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/
|
||||
[4]: {{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/registered-clusters/
|
||||
[5]: {{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/registered-clusters/
|
||||
[1]: {{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/
|
||||
[2]: {{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/custom-nodes/
|
||||
[3]: {{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/
|
||||
[4]: {{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/registered-clusters/
|
||||
[5]: {{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/registered-clusters/
|
||||
|
||||
\* Delete option accessible via View API
|
||||
|
||||
|
||||
@@ -156,6 +156,9 @@ By default, your user is added as the project `Owner`.
|
||||
>**Notes on Permissions:**
|
||||
>
|
||||
>- Users assigned the `Owner` or `Member` role for a project automatically inherit the `namespace creation` role. However, this role is a [Kubernetes ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole), meaning its scope extends to all projects in the cluster. Therefore, users explicitly assigned the `Owner` or `Member` role for a project can create namespaces in other projects they're assigned to, even with only the `Read Only` role assigned.
|
||||
>
|
||||
>- By default, the Rancher role of `project-member` inherits from the `Kubernetes-edit` role, and the `project-owner` role inherits from the `Kubernetes-admin` role. As such, both `project-member` and `project-owner` roles will allow for namespace management, including the ability to create and delete namespaces.
|
||||
>
|
||||
>- Choose `Custom` to create a custom role on the fly: [Custom Project Roles]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/cluster-project-roles/#custom-project-roles).
|
||||
|
||||
To add members:
|
||||
|
||||
@@ -19,9 +19,12 @@ If you encounter this issue, you can work around it by installing the initiator
|
||||
|
||||
After installing the initiator tool on your nodes, edit the YAML for your cluster, editing the kubelet configuration to mount the iSCSI binary and configuration, as shown in the sample below.
|
||||
|
||||
>**Note:**
|
||||
>**Notes:**
|
||||
>
|
||||
>- Before updating your Kubernetes YAML to mount the iSCSI binary and configuration, make sure either the `open-iscsi` (deb) or `iscsi-initiator-utils` (yum) package is installed on your cluster nodes. If this package isn't installed _before_ the bind mounts are created in your Kubernetes YAML, Docker will automatically create the directories and files on each node and will not allow the package install to succeed.</br>
|
||||
></br>
|
||||
>
|
||||
>Before updating your Kubernetes YAML to mount the iSCSI binary and configuration, make sure either the `open-iscsi` (deb) or `iscsi-initiator-utils` (yum) package is installed on your cluster nodes. If this package isn't installed _before_ the bind mounts are created in your Kubernetes YAML, Docker will automatically create the directories and files on each node and will not allow the package install to succeed.
|
||||
>- The example YAML below does not apply to K3s, but only to RKE clusters. Since the K3s kubelet does not run in a container, adding extra binds is not necessary. However, all iSCSI tools must still be installed on your K3s nodes.
|
||||
|
||||
```
|
||||
services:
|
||||
|
||||
+57
-51
@@ -24,6 +24,7 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b
|
||||
"ec2:RunInstances",
|
||||
"ec2:RevokeSecurityGroupIngress",
|
||||
"ec2:RevokeSecurityGroupEgress",
|
||||
"ec2:DescribeRegions",
|
||||
"ec2:DescribeVpcs",
|
||||
"ec2:DescribeTags",
|
||||
"ec2:DescribeSubnets",
|
||||
@@ -123,31 +124,6 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b
|
||||
|
||||
### Service Role Permissions
|
||||
|
||||
Rancher will create a service role with the following trust policy:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Action": "sts:AssumeRole",
|
||||
"Principal": {
|
||||
"Service": "eks.amazonaws.com"
|
||||
},
|
||||
"Effect": "Allow",
|
||||
"Sid": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
This role will also have two role policy attachments with the following policies ARNs:
|
||||
|
||||
```
|
||||
arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
|
||||
arn:aws:iam::aws:policy/AmazonEKSServicePolicy
|
||||
```
|
||||
|
||||
Permissions required for Rancher to create service role on users behalf during the EKS cluster creation process.
|
||||
|
||||
```json
|
||||
@@ -182,36 +158,66 @@ Permissions required for Rancher to create service role on users behalf during t
|
||||
}
|
||||
```
|
||||
|
||||
When an EKS cluster is created, Rancher will create a service role with the following trust policy:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Action": "sts:AssumeRole",
|
||||
"Principal": {
|
||||
"Service": "eks.amazonaws.com"
|
||||
},
|
||||
"Effect": "Allow",
|
||||
"Sid": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
This role will also have two role policy attachments with the following policies ARNs:
|
||||
|
||||
```
|
||||
arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
|
||||
arn:aws:iam::aws:policy/AmazonEKSServicePolicy
|
||||
```
|
||||
|
||||
### VPC Permissions
|
||||
|
||||
Permissions required for Rancher to create VPC and associated resources.
|
||||
|
||||
```json
|
||||
{
|
||||
"Sid": "VPCPermissions",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"ec2:ReplaceRoute",
|
||||
"ec2:ModifyVpcAttribute",
|
||||
"ec2:ModifySubnetAttribute",
|
||||
"ec2:DisassociateRouteTable",
|
||||
"ec2:DetachInternetGateway",
|
||||
"ec2:DescribeVpcs",
|
||||
"ec2:DeleteVpc",
|
||||
"ec2:DeleteTags",
|
||||
"ec2:DeleteSubnet",
|
||||
"ec2:DeleteRouteTable",
|
||||
"ec2:DeleteRoute",
|
||||
"ec2:DeleteInternetGateway",
|
||||
"ec2:CreateVpc",
|
||||
"ec2:CreateSubnet",
|
||||
"ec2:CreateSecurityGroup",
|
||||
"ec2:CreateRouteTable",
|
||||
"ec2:CreateRoute",
|
||||
"ec2:CreateInternetGateway",
|
||||
"ec2:AttachInternetGateway",
|
||||
"ec2:AssociateRouteTable"
|
||||
],
|
||||
"Resource": "*"
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "VPCPermissions",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"ec2:ReplaceRoute",
|
||||
"ec2:ModifyVpcAttribute",
|
||||
"ec2:ModifySubnetAttribute",
|
||||
"ec2:DisassociateRouteTable",
|
||||
"ec2:DetachInternetGateway",
|
||||
"ec2:DescribeVpcs",
|
||||
"ec2:DeleteVpc",
|
||||
"ec2:DeleteTags",
|
||||
"ec2:DeleteSubnet",
|
||||
"ec2:DeleteRouteTable",
|
||||
"ec2:DeleteRoute",
|
||||
"ec2:DeleteInternetGateway",
|
||||
"ec2:CreateVpc",
|
||||
"ec2:CreateSubnet",
|
||||
"ec2:CreateSecurityGroup",
|
||||
"ec2:CreateRouteTable",
|
||||
"ec2:CreateRoute",
|
||||
"ec2:CreateInternetGateway",
|
||||
"ec2:AttachInternetGateway",
|
||||
"ec2:AssociateRouteTable"
|
||||
],
|
||||
"Resource": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
@@ -32,6 +32,13 @@ For information on how to install Docker, refer to the official [Docker document
|
||||
|
||||
Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. We recommend disabling firewalld. For Kubernetes 1.19, firewalld must be turned off.
|
||||
|
||||
>**Note:** In RHEL 8.4, two extra services are included on the NetworkManager: `nm-cloud-setup.service` and `nm-cloud-setup.timer`. These services add a routing table that interferes with the CNI plugin's configuration. If these services are enabled, you must disable them using the command below, and then reboot the node to restore connectivity:
|
||||
>
|
||||
> ```
|
||||
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
|
||||
reboot
|
||||
```
|
||||
|
||||
### SUSE Linux Nodes
|
||||
|
||||
SUSE Linux may have a firewall that blocks all ports by default. In that situation, follow [these steps]({{<baseurl>}}/rancher/v2.5/en/installation/requirements/ports/#opening-suse-linux-ports) to open the ports needed for adding a host to a custom cluster.
|
||||
|
||||
@@ -20,6 +20,17 @@ The control that Rancher has to manage a registered cluster depends on the type
|
||||
|
||||
# Prerequisites
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "v2.5.9+" %}}
|
||||
|
||||
### Kubernetes Node Roles
|
||||
|
||||
Registered RKE Kubernetes clusters must have all three node roles - etcd, controlplane and worker. A cluster with only controlplane components cannot be registered in Rancher.
|
||||
|
||||
For more information on RKE node roles, see the [best practices.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/production/#cluster-architecture)
|
||||
|
||||
### Permissions
|
||||
|
||||
If your existing Kubernetes cluster already has a `cluster-admin` role defined, you must have this `cluster-admin` privilege to register the cluster in Rancher.
|
||||
|
||||
In order to apply the privilege, you need to run:
|
||||
@@ -36,6 +47,38 @@ By default, GKE users are not given this privilege, so you will need to run the
|
||||
|
||||
If you are registering a K3s cluster, make sure the `cluster.yml` is readable. It is protected by default. For details, refer to [Configuring a K3s cluster to enable importation to Rancher.](#configuring-a-k3s-cluster-to-enable-registration-in-rancher)
|
||||
|
||||
### EKS Clusters
|
||||
|
||||
EKS clusters must have at least one managed node group to be imported into Rancher or provisioned from Rancher successfully.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab "Rancher before v2.5.9" %}}
|
||||
|
||||
### Permissions
|
||||
|
||||
If your existing Kubernetes cluster already has a `cluster-admin` role defined, you must have this `cluster-admin` privilege to register the cluster in Rancher.
|
||||
|
||||
In order to apply the privilege, you need to run:
|
||||
|
||||
```plain
|
||||
kubectl create clusterrolebinding cluster-admin-binding \
|
||||
--clusterrole cluster-admin \
|
||||
--user [USER_ACCOUNT]
|
||||
```
|
||||
|
||||
before running the `kubectl` command to register the cluster.
|
||||
|
||||
By default, GKE users are not given this privilege, so you will need to run the command before registering GKE clusters. To learn more about role-based access control for GKE, please click [here](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control).
|
||||
|
||||
If you are registering a K3s cluster, make sure the `cluster.yml` is readable. It is protected by default. For details, refer to [Configuring a K3s cluster to enable importation to Rancher.](#configuring-a-k3s-cluster-to-enable-registration-in-rancher)
|
||||
|
||||
### EKS Clusters
|
||||
|
||||
EKS clusters must have at least one managed node group to be imported into Rancher or provisioned from Rancher successfully.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% /tabs %}}
|
||||
|
||||
# Registering a Cluster
|
||||
|
||||
1. From the **Clusters** page, click **Add Cluster**.
|
||||
@@ -76,6 +119,34 @@ The option can also be specified using the environment variable `K3S_KUBECONFIG_
|
||||
$ curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s -
|
||||
```
|
||||
|
||||
### Configuring an Imported EKS Cluster with Terraform
|
||||
|
||||
You should define **only** the minimum fields that Rancher requires when importing an EKS cluster with Terraform. This is important as Rancher will overwrite what was in the EKS cluster with any config that the user has provided.
|
||||
|
||||
>**Warning:** Even a small difference between the current EKS cluster and a user-provided config could have unexpected results.
|
||||
|
||||
The minimum config fields required by Rancher to import EKS clusters with Terraform using `eks_config_v2` are as follows:
|
||||
|
||||
- cloud_credential_id
|
||||
- name
|
||||
- region
|
||||
- imported (this field should always be set to `true` for imported clusters)
|
||||
|
||||
Example YAML configuration for imported EKS clusters:
|
||||
|
||||
```
|
||||
resource "rancher2_cluster" "my-eks-to-import" {
|
||||
name = "my-eks-to-import"
|
||||
description = "Terraform EKS Cluster"
|
||||
eks_config_v2 {
|
||||
cloud_credential_id = rancher2_cloud_credential.aws.id
|
||||
name = var.aws_eks_name
|
||||
region = var.aws_region
|
||||
imported = true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
# Management Capabilities for Registered Clusters
|
||||
|
||||
The control that Rancher has to manage a registered cluster depends on the type of cluster.
|
||||
|
||||
@@ -6,7 +6,7 @@ aliases:
|
||||
- /rancher/v2.5/en/cluster-provisioning/rke-clusters/options/cloud-providers
|
||||
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/cloud-providers/
|
||||
---
|
||||
A _cloud provider_ is a module in Kubernetes that provides an interface for managing nodes, load balancers, and networking routes. For more information, refer to the [official Kubernetes documentation on cloud providers.](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/)
|
||||
A _cloud provider_ is a module in Kubernetes that provides an interface for managing nodes, load balancers, and networking routes.
|
||||
|
||||
When a cloud provider is set up in Rancher, the Rancher server can automatically provision new nodes, load balancers or persistent storage devices when launching Kubernetes definitions, if the cloud provider you're using supports such automation.
|
||||
|
||||
@@ -39,9 +39,9 @@ For details on enabling the vSphere cloud provider, refer to [this page.](./vsph
|
||||
|
||||
### Setting up a Custom Cloud Provider
|
||||
|
||||
The `Custom` cloud provider is available if you want to configure any [Kubernetes cloud provider](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/).
|
||||
The `Custom` cloud provider is available if you want to configure any Kubernetes cloud provider.
|
||||
|
||||
For the custom cloud provider option, you can refer to the [RKE docs]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to edit the yaml file for your specific cloud provider. There are specific cloud providers that have more detailed configuration :
|
||||
For the custom cloud provider option, you can refer to the [RKE docs]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to edit the yaml file for your specific cloud provider. There are specific cloud providers that have more detailed configuration:
|
||||
|
||||
* [vSphere]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/)
|
||||
* [OpenStack]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/openstack/)
|
||||
|
||||
+4
-1
@@ -13,7 +13,10 @@ This page covers how to install the Cloud Provider Interface (CPI) and Cloud Sto
|
||||
|
||||
# Prerequisites
|
||||
|
||||
The vSphere version must be 7.0u1 or higher.
|
||||
The vSphere versions supported:
|
||||
|
||||
* 6.7u3
|
||||
* 7.0u1 or higher.
|
||||
|
||||
The Kubernetes version must be 1.19 or higher.
|
||||
|
||||
|
||||
@@ -158,7 +158,7 @@ Option to enable or disable [recurring etcd snapshots]({{<baseurl>}}/rke/latest/
|
||||
|
||||
_Available as of v2.5.6_
|
||||
|
||||
Option to set environment variables for [rancher agents]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
Option to set environment variables for [rancher agents]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
|
||||
|
||||
# Cluster Config File
|
||||
|
||||
+1
-1
@@ -85,7 +85,7 @@ If you are configuring DHCP options sets for an AWS virtual private cloud, note
|
||||
|
||||
> Some Linux operating systems accept multiple domain names separated by spaces. However, other Linux operating systems and Windows treat the value as a single domain, which results in unexpected behavior. If your DHCP options set is associated with a VPC that has instances with multiple operating systems, specify only one domain name.
|
||||
|
||||
#### Rancher on vSphere with ESXi 6.7u2 and above
|
||||
### Rancher on vSphere with ESXi 6.7u2 and above
|
||||
|
||||
If you are using Rancher on VMware vSphere with ESXi 6.7u2 or later with Red Hat Enterprise Linux 8.3, CentOS 8.3, or SUSE Enterprise Linux 15 SP2 or later, it is necessary to disable the `vmxnet3` virtual network adapter hardware offloading feature. Failure to do so will result in all network connections between pods on different cluster nodes to fail with timeout errors. All connections from Windows pods to critical services running on Linux nodes, such as CoreDNS, will fail as well. It is also possible that external connections may fail. This issue is the result of Linux distributions enabling the hardware offloading feature in `vmxnet3` and a bug in the `vmxnet3` hardware offloading feature that results in the discarding of packets for guest overlay traffic. To address this issue, it is necessary disable the `vmxnet3` hardware offloading feature. This setting does not survive reboot, so it is necessary to disable on every boot. The recommended course of action is to create a systemd unit file at `/etc/systemd/system/disable_hw_offloading.service`, which disables the `vmxnet3` hardware offloading feature on boot. A sample systemd unit file which disables the `vmxnet3` hardware offloading feature is as follows. Note that `<VM network interface>` must be customized to the host `vmxnet3` network interface, e.g., `ens192`:
|
||||
|
||||
|
||||
@@ -16,6 +16,8 @@ Fleet is a separate project from Rancher, and can be installed on any Kubernetes
|
||||
- [Windows Support](#windows-support)
|
||||
- [GitHub Repository](#github-repository)
|
||||
- [Using Fleet Behind a Proxy](#using-fleet-behind-a-proxy)
|
||||
- [Helm Chart Dependencies](#helm-chart-dependencies)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Documentation](#documentation)
|
||||
|
||||
# Architecture
|
||||
@@ -24,7 +26,24 @@ For information about how Fleet works, see [this page.](./architecture)
|
||||
|
||||
# Accessing Fleet in the Rancher UI
|
||||
|
||||
Fleet comes preinstalled in Rancher v2.5. To access it, go to the **Cluster Explorer** in the Rancher UI. In the top left dropdown menu, click **Cluster Explorer > Continuous Delivery.** On this page, you can edit Kubernetes resources and cluster groups managed by Fleet.
|
||||
Fleet comes preinstalled in Rancher v2.5. Users can leverage continuous delivery to deploy their applications to the Kubernetes clusters in the git repository without any manual operation by following **gitops** practice. For additional information on Continuous Delivery and other Fleet troubleshooting tips, refer [here](https://fleet.rancher.io/troubleshooting/).
|
||||
|
||||
Follow the steps below to access Continuous Delivery in the Rancher UI:
|
||||
|
||||
1. Click **Cluster Explorer** in the Rancher UI.
|
||||
|
||||
1. In the top left dropdown menu, click **Cluster Explorer > Continuous Delivery.**
|
||||
|
||||
1. Select your namespace at the top of the menu, noting the following:
|
||||
- By default,`fleet-default` is selected which includes all downstream clusters that are registered through Rancher.
|
||||
- You may switch to `fleet-local`, which only contains the `local` cluster, or you may create your own workspace to which you may assign and move clusters.
|
||||
- You can then manage clusters by clicking on **Clusters** on the left navigation bar.
|
||||
|
||||
1. Click on **Gitrepos** on the left navigation bar to deploy the gitrepo into your clusters in the current workspace.
|
||||
|
||||
1. Select your [git repository](https://fleet.rancher.io/gitrepo-add/) and [target clusters/cluster group](https://fleet.rancher.io/gitrepo-structure/). You can also create the cluster group in the UI by clicking on **Cluster Groups** from the left navigation bar.
|
||||
|
||||
1. Once the gitrepo is deployed, you can monitor the application through the Rancher UI.
|
||||
|
||||
# Windows Support
|
||||
|
||||
@@ -44,6 +63,31 @@ _Available as of v2.5.8_
|
||||
|
||||
For details on using Fleet behind a proxy, see [this page.](./proxy)
|
||||
|
||||
# Helm Chart Dependencies
|
||||
|
||||
In order for Helm charts with dependencies to deploy successfully, you must run a manual command (as listed below), as it is up to the user to fulfill the dependency list. If you do not do this and proceed to clone your repository and run `helm install`, your installation will fail because the dependencies will be missing.
|
||||
|
||||
The Helm chart in the git repository must include its dependencies in the charts subdirectory. You must either manually run `helm dependencies update $chart` OR run `helm dependencies build $chart` locally, then commit the complete charts directory to your git repository. Note that you will update your commands with the applicable parameters.
|
||||
|
||||
# Troubleshooting
|
||||
---
|
||||
* **Known Issue:** Fleet becomes inoperable after a restore using the [backup-restore-operator]({{<baseurl>}}/rancher/v2.5/en/backups/back-up-rancher/#1-install-the-rancher-backup-operator). We will update the community once a permanent solution is in place.
|
||||
|
||||
* **Temporary Workaround:** </br>
|
||||
1. Find the two service account tokens listed in the fleet-controller and the fleet-controller-bootstrap service accounts. These are under the fleet-system namespace of the local cluster. </br>
|
||||
2. Remove the non-existent token secret. Doing so allows for only one entry to be present for the service account token secret that actually exists. </br>
|
||||
3. Delete the fleet-controller Pod in the fleet-system namespace to reschedule. </br>
|
||||
4. After the service account token issue is resolved, you can force redeployment of the fleet-agents. In the Rancher UI, go to **☰ > Cluster Management**, click on **Clusters** page, then click **Force Update**. </br>
|
||||
5. If the fleet-agent bundles remain in a `Modified` state after Step 4, update the field `spec.forceSyncGeneration` for the fleet-agent bundle to force re-creation.
|
||||
|
||||
---
|
||||
* **Known Issue:** clientSecretName and helmSecretName secrets for Fleet gitrepos are not included in the backup nor restore created by the [backup-restore-operator]({{<baseurl>}}/rancher/v2.5/en/backups/back-up-rancher/#1-install-the-rancher-backup-operator). We will update the community once a permanent solution is in place.
|
||||
|
||||
* **Temporary Workaround:** </br>
|
||||
By default, user-defined secrets are not backed up in Fleet. It is necessary to recreate secrets if performing a disaster recovery restore or migration of Rancher into a fresh cluster. To modify resourceSet to include extra resources you want to backup, refer to docs [here](https://github.com/rancher/backup-restore-operator#user-flow).
|
||||
|
||||
---
|
||||
|
||||
# Documentation
|
||||
|
||||
The Fleet documentation is at [https://fleet.rancher.io/.](https://fleet.rancher.io/)
|
||||
|
||||
@@ -27,7 +27,7 @@ The capability to access a downstream cluster without Rancher depends on the typ
|
||||
|
||||
- **Registered clusters:** The cluster will be unaffected and you can access the cluster using the same methods that you did before the cluster was registered into Rancher.
|
||||
- **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
|
||||
- **RKE clusters:** To access an [RKE cluster,]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/) the cluster must have the [authorized cluster endpoint]({{<baseurl>}}/rancher/v2.5/en/overview/architecture/#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.]({{<baseurl>}}/rancher/v2.5/en/overview/architecture/#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
|
||||
- **RKE clusters:** Please note that you will no longer be able to manage the individual Kubernetes components or perform any upgrades on them after the deletion of the Rancher server. However, you can still access the cluster to manage your workloads. To access an [RKE cluster,]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/) the cluster must have the [authorized cluster endpoint]({{<baseurl>}}/rancher/v2.5/en/overview/architecture/#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.]({{<baseurl>}}/rancher/v2.5/en/overview/architecture/#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
|
||||
|
||||
### What if I don't want Rancher anymore?
|
||||
|
||||
|
||||
@@ -173,3 +173,8 @@ In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. Se
|
||||
### Can I use keyboard shortcuts in the UI?
|
||||
|
||||
Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
|
||||
|
||||
|
||||
### What does `Unknown schema for type:` errors followed by something like `catalog.cattle.io.operation` mean when trying to modify an App?
|
||||
|
||||
This error occurs when Kubernetes can not find the CRD mentioned. The vast majority of the time these are a result of missing RBAC permissions. Try with an admin user and if this works, add permissions for the resource mentioned by the error (ie. `Get`, `List`, `Patch` as needed).
|
||||
|
||||
@@ -50,6 +50,39 @@ From the left sidebar select _"Repositories"_.
|
||||
|
||||
These items represent helm repositories, and can be either traditional helm endpoints which have an index.yaml, or git repositories which will be cloned and can point to a specific branch. In order to use custom charts, simply add your repository here and they will become available in the Charts tab under the name of the repository.
|
||||
|
||||
To add a private CA for Helm Chart repositories:
|
||||
|
||||
- **HTTP-based chart repositories**: You must add a base64 encoded copy of the CA certificate in DER format to the spec.caBundle field of the chart repo, such as `openssl x509 -outform der -in ca.pem | base64 -w0`. Click **Edit YAML** for the chart repo and set, as in the following example:</br>
|
||||
```
|
||||
[...]
|
||||
spec:
|
||||
caBundle:
|
||||
MIIFXzCCA0egAwIBAgIUWNy8WrvSkgNzV0zdWRP79j9cVcEwDQYJKoZIhvcNAQELBQAwPzELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRQwEgYDVQQKDAtNeU9yZywgSW5jLjENMAsGA1UEAwwEcm9vdDAeFw0yMTEyMTQwODMyMTdaFw0yNDEwMDMwODMyMT
|
||||
...
|
||||
nDxZ/tNXt/WPJr/PgEB3hQdInDWYMg7vGO0Oz00G5kWg0sJ0ZTSoA10ZwdjIdGEeKlj1NlPyAqpQ+uDnmx6DW+zqfYtLnc/g6GuLLVPamraqN+gyU8CHwAWPNjZonFN9Vpg0PIk1I2zuOc4EHifoTAXSpnjfzfyAxCaZsnTptimlPFJJqAMj+FfDArGmr4=
|
||||
[...]
|
||||
```
|
||||
|
||||
- **Git-based chart repositories**: It is not currently possible to add a private CA. For git-based chart repositories with a certificate signed by a private CA, you must disable TLS verification. Click **Edit YAML** for the chart repo, and add the key/value pair as follows:
|
||||
```
|
||||
[...]
|
||||
spec:
|
||||
insecureSkipTLSVerify: true
|
||||
[...]
|
||||
```
|
||||
|
||||
> **Note:** Helm chart repositories with authentication
|
||||
>
|
||||
> As of Rancher v2.5.12, a new value `disableSameOriginCheck` has been added to the Repo.Spec. This allows users to bypass the same origin checks, sending the repository Authentication information as a Basic Auth Header with all API calls. This is not recommended but can be used as a temporary solution in cases of non-standard Helm chart repositories such as those that have redirects to a different origin URL.
|
||||
>
|
||||
> To use this feature for an existing Helm chart repository, click <b>⋮ > Edit YAML</b>. On the `spec` portion of the YAML file, add `disableSameOriginCheck` and set it to `true`.
|
||||
>
|
||||
> ```yaml
|
||||
[...]
|
||||
spec:
|
||||
disableSameOriginCheck: true
|
||||
[...]
|
||||
```
|
||||
|
||||
### Helm Compatibility
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ In this section,
|
||||
- **RKE (Rancher Kubernetes Engine)** is a certified Kubernetes distribution and CLI/library which creates and manages a Kubernetes cluster.
|
||||
- **K3s (Lightweight Kubernetes)** is also a fully compliant Kubernetes distribution. It is newer than RKE, easier to use, and more lightweight, with a binary size of less than 100 MB.
|
||||
- **RKE2** is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.
|
||||
- **RancherD** is a new tool for installing Rancher, which is available as of Rancher v2.5.4. It is an experimental feature. RancherD is a single binary that first launches an RKE2 Kubernetes cluster, then installs the Rancher server Helm chart on the cluster.
|
||||
- **RancherD** was an experimental tool for installing Rancher; a single binary that first launched an RKE2 Kubernetes cluster, then installed the Rancher server Helm chart on the cluster. It was available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
|
||||
|
||||
# Changes to Installation in Rancher v2.5
|
||||
|
||||
@@ -37,9 +37,7 @@ We recommend using Helm, a Kubernetes package manager, to install Rancher on mul
|
||||
|
||||
### High-availability Kubernetes Install with RancherD
|
||||
|
||||
_Available as of v2.5.4_
|
||||
|
||||
> This is an experimental feature.
|
||||
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
|
||||
|
||||
RancherD is a single binary that first launches an RKE2 Kubernetes cluster, then installs the Rancher server Helm chart on the cluster.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ aliases:
|
||||
- /rancher/v2.5/en/installation/k8s-install/
|
||||
- /rancher/v2.5/en/installation/k8s-install/helm-rancher
|
||||
- /rancher/v2.5/en/installation/k8s-install/kubernetes-rke
|
||||
- /rancher/v2.5/en/installation/ha-server-install
|
||||
- /rancher/v2.5/en/installation/ha-server-install
|
||||
- /rancher/v2.5/en/installation/install-rancher-on-k8s/install
|
||||
- /rancher/v2.x/en/installation/install-rancher-on-k8s/
|
||||
---
|
||||
@@ -24,7 +24,7 @@ In this section, you'll learn how to deploy Rancher on a Kubernetes cluster usin
|
||||
|
||||
### Kubernetes Cluster
|
||||
|
||||
Set up the Rancher server's local Kubernetes cluster.
|
||||
Set up the Rancher server's local Kubernetes cluster.
|
||||
|
||||
Rancher can be installed on any Kubernetes cluster. This cluster can use upstream Kubernetes, or it can use one of Rancher's Kubernetes distributions, or it can be a managed Kubernetes cluster from a provider such as Amazon EKS.
|
||||
|
||||
@@ -113,7 +113,7 @@ There are three recommended options for the source of the certificate used for T
|
||||
|
||||
### 4. Install cert-manager
|
||||
|
||||
> You should skip this step if you are bringing your own certificate files (option `ingress.tls.source=secret`), or if you use [TLS termination on an external load balancer]({{<baseurl>}}/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/#external-tls-termination).
|
||||
> You should skip this step if you are bringing your own certificate files (option `ingress.tls.source=secret`), or if you use [TLS termination on an external load balancer]({{<baseurl>}}/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/#external-tls-termination).
|
||||
|
||||
This step is only required to use certificates issued by Rancher's generated CA (`ingress.tls.source=rancher`) or to request Let's Encrypt issued certificates (`ingress.tls.source=letsEncrypt`).
|
||||
|
||||
@@ -157,6 +157,8 @@ cert-manager-webhook-787858fcdb-nlzsq 1/1 Running 0 2m
|
||||
|
||||
The exact command to install Rancher differs depending on the certificate configuration.
|
||||
|
||||
However, irrespective of the certificate configuration, the name of the Rancher installation in the `cattle-system` namespace should always be `rancher`.
|
||||
|
||||
{{% tabs %}}
|
||||
{{% tab "Rancher-generated Certificates" %}}
|
||||
|
||||
@@ -168,7 +170,7 @@ Because `rancher` is the default option for `ingress.tls.source`, we are not spe
|
||||
- Set `hostname` to the DNS record that resolves to your load balancer.
|
||||
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
|
||||
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
|
||||
```
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
@@ -190,7 +192,7 @@ deployment "rancher" successfully rolled out
|
||||
|
||||
This option uses `cert-manager` to automatically request and renew [Let's Encrypt](https://letsencrypt.org/) certificates. This is a free service that provides you with a valid certificate as Let's Encrypt is a trusted CA.
|
||||
|
||||
>**Note:**: You need to have port 80 open as the HTTP-01 challenge can only be done on port 80.
|
||||
>**Note:** You need to have port 80 open as the HTTP-01 challenge can only be done on port 80.
|
||||
|
||||
In the following command,
|
||||
|
||||
@@ -198,8 +200,9 @@ In the following command,
|
||||
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
|
||||
- Set `ingress.tls.source` to `letsEncrypt`.
|
||||
- Set `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices).
|
||||
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
|
||||
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
|
||||
```
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
@@ -207,7 +210,8 @@ helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--set hostname=rancher.my.org \
|
||||
--set replicas=3 \
|
||||
--set ingress.tls.source=letsEncrypt \
|
||||
--set letsEncrypt.email=me@example.org
|
||||
--set letsEncrypt.email=me@example.org \
|
||||
--set letsEncrypt.ingress.class=nginx
|
||||
```
|
||||
|
||||
Wait for Rancher to be rolled out:
|
||||
@@ -232,7 +236,7 @@ Although an entry in the `Subject Alternative Names` is technically required, ha
|
||||
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
|
||||
- Set `ingress.tls.source` to `secret`.
|
||||
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
|
||||
|
||||
```
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
@@ -245,7 +249,7 @@ helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
If you are using a Private CA signed certificate , add `--set privateCA=true` to the command:
|
||||
|
||||
```
|
||||
helm install rancher rancher-latest/rancher \
|
||||
helm install rancher rancher-<CHART_REPO>/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=secret \
|
||||
|
||||
@@ -124,7 +124,7 @@ To customize or use a different ingress with Rancher server you can set your own
|
||||
Example on setting a custom certificate issuer:
|
||||
|
||||
```plain
|
||||
--set ingress.extraAnnotations.'certmanager\.k8s\.io/cluster-issuer'=ca-key-pair
|
||||
--set ingress.extraAnnotations.'cert-manager\.io/cluster-issuer'=issuer-name
|
||||
```
|
||||
|
||||
Example on setting a static proxy header with `ingress.configurationSnippet`. This value is parsed like a template so variables can be used.
|
||||
|
||||
@@ -165,7 +165,7 @@ helm upgrade rancher rancher-<CHART_REPO>/rancher \
|
||||
|
||||
### Option B: Reinstalling Rancher and cert-manager
|
||||
|
||||
If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manger due to the API change in cert-manger v0.11.
|
||||
If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manager due to the API change in cert-manager v0.11.
|
||||
|
||||
1. Uninstall Rancher
|
||||
|
||||
|
||||
+2
-1
@@ -115,7 +115,8 @@ helm template cert-manager ./cert-manager-v1.5.1.tgz --output-dir . \
|
||||
--namespace cert-manager \
|
||||
--set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller \
|
||||
--set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook \
|
||||
--set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector
|
||||
--set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector \
|
||||
--set startupapicheck.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-ctl
|
||||
```
|
||||
|
||||
### 4. Download the cert-manager CRD
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user