Merge pull request #3751 from btat/staging-to-master

Staging to master
This commit is contained in:
Jen Travinski
2021-12-21 16:40:10 -05:00
committed by GitHub
13 changed files with 133 additions and 34 deletions
@@ -11,7 +11,9 @@ In this section, you'll learn how to back up Rancher running on any Kubernetes c
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
> When restoring a backup into a new Rancher setup, the version of the new setup should be the same as the one where the backup is made.
Note that the rancher-backup operator version 1.x.x is for Rancher v2.5.x.
> When restoring a backup into a new Rancher setup, the version of the new setup should be the same as the one where the backup is made. The Kubernetes version should also be considered when restoring a backup, since the supported apiVersion in the cluster and in the backup file could be different.
### Prerequisites
@@ -70,20 +70,19 @@ spec:
endpoint: s3.us-west-2.amazonaws.com
```
> **Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
>**Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
```
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
Then apply the resource:
```
kubectl apply -f migrationResource.yaml
```
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
```
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
1. Then apply the resource:
```
kubectl apply -f migrationResource.yaml
```
### 3. Install cert-manager
@@ -33,6 +33,7 @@ On this page, we provide security-related documentation along with resources to
- [The CIS Benchmark and self-assessment](#the-cis-benchmark-and-self-assessment)
- [Third-party penetration test reports](#third-party-penetration-test-reports)
- [Rancher CVEs and resolutions](#rancher-cves-and-resolutions)
- [Kubernetes Security Best Practices](#kubernetes-security-best-practices)
### Running a CIS Security Scan on a Kubernetes Cluster
@@ -86,3 +87,7 @@ Results:
### Rancher CVEs and Resolutions
Rancher is committed to informing the community of security issues in our products. For the list of CVEs (Common Vulnerabilities and Exposures) for issues we have resolved, refer to [this page.](./cve)
### Kubernetes Security Best Practices
For recommendations on securing your Kubernetes cluster, refer to the [Best Practices](./best-practices) guide.
@@ -0,0 +1,8 @@
---
title: Kubernetes Security Best Practices
weight: 5
---
# Restricting cloud metadata API access
Cloud providers such as AWS, Azure, or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.
@@ -43,6 +43,8 @@ When a Restore custom resource is created, the operator accesses the backup .tar
The Backup and Restore custom resources can be created in the Rancher UI, or by using `kubectl apply`.
>**Note:** Refer [here]({{<baseurl>}}/rancher/v2.6/en/backups/migrating-rancher/#2-restore-from-backup-using-a-restore-custom-resource) for help on restoring an existing backup file into a v1.22 cluster in Rancher v2.6.3.
# Installing the rancher-backup Operator
The `rancher-backup` operator can be installed from the Rancher UI, or with the Helm CLI. In both cases, the `rancher-backup` Helm chart is installed on the Kubernetes cluster running the Rancher server. It is a cluster-admin only feature and available only for the **local** cluster. (*If you do not see `rancher-backup` in the Rancher UI, you may have selected the wrong cluster.*)
@@ -5,12 +5,18 @@ weight: 1
In this section, you'll learn how to back up Rancher running on any Kubernetes cluster. To backup Rancher installed with Docker, refer the instructions for [single node backups]({{<baseurl>}}/rancher/v2.6/en/backups/docker-installs/docker-backups)
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
Note that the rancher-backup operator version 2.x.x is for Rancher v2.6.x.
> When restoring a backup into a new Rancher setup, the version of the new setup should be the same as the one where the backup is made. The Kubernetes version should also be considered when restoring a backup, since the supported apiVersion in the cluster and in the backup file could be different.
### Prerequisites
The Rancher version must be v2.5.0 and up.
Refer [here]({{<baseurl>}}/rancher/v2.6/en/backups/migrating-rancher/#2-restore-from-backup-using-a-restore-custom-resource) for help on restoring an existing backup file into a v1.22 cluster in Rancher v2.6.3.
### 1. Install the Rancher Backups operator
The backup storage location is an operator-level setting, so it needs to be configured when the Rancher Backups application is installed or upgraded.
@@ -29,6 +29,13 @@ helm install rancher-backup rancher-charts/rancher-backup -n cattle-resources-sy
### 2. Restore from backup using a Restore custom resource
>**Important:** Kubernetes v1.22, available as an experimental feature of v2.6.3, does not support restoring from backup files containing CRDs with the apiVersion `apiextensions.k8s.io/v1beta1`. In v1.22, the default `resourceSet` in the rancher-backup app is updated to collect only CRDs that use `apiextensions.k8s.io/v1`. There are currently two ways to work around this issue:
>
1. Update the default `resourceSet` to collect the CRDs with the apiVersion v1.
1. Update the default `resourceSet` and the client to use the new APIs internally, with `apiextensions.k8s.io/v1` as the replacement.
>
> - Note that when making or restoring backups for v1.22, the Rancher version and the local cluster's Kubernetes version should be the same. The Kubernetes version should be considered when restoring a backup since the supported apiVersion in the cluster and in the backup file could be different.
If you are using an S3 store as the backup source, and need to use your S3 credentials for restore, create a secret in this cluster using your S3 credentials. The Secret data must have two keys, `accessKey` and `secretKey` containing the s3 credentials like this:
```yaml
@@ -68,20 +75,19 @@ spec:
endpoint: s3.us-west-2.amazonaws.com
```
> **Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
>**Important:** The field `encryptionConfigSecretName` must be set only if your backup was created with encryption enabled. Provide the name of the Secret containing the encryption config file. If you only have the encryption config file, but don't have a secret created with it in this cluster, use the following steps to create the secret:
```
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
Then apply the resource:
```
kubectl apply -f migrationResource.yaml
```
1. The encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret. So save your `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
```
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
1. Then apply the resource:
```
kubectl apply -f migrationResource.yaml
```
### 3. Install cert-manager
@@ -12,6 +12,7 @@ The control that Rancher has to manage a registered cluster depends on the type
- [Management Capabilities for Registered Clusters](#management-capabilities-for-registered-clusters)
- [Configuring K3s Cluster Upgrades](#configuring-k3s-cluster-upgrades)
- [Debug Logging and Troubleshooting for Registered K3s Clusters](#debug-logging-and-troubleshooting-for-registered-k3s-clusters)
- [Authorized Cluster Endpoint Support for RKE2 and K3s Clusters](#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters)
- [Annotating Registered Clusters](#annotating-registered-clusters)
# Prerequisites
@@ -155,8 +156,51 @@ If the cluster becomes stuck in upgrading, restart the `system-upgrade-controlle
To prevent issues when upgrading, the [Kubernetes upgrade best practices](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) should be followed.
# Authorized Cluster Endpoint Support for RKE2 and K3s Clusters
_Available as of v2.6.3_
Authorized Cluster Endpoint (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the authorized cluster endpoint, click [here]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/).
> **Note:**
>
> - These steps only need to be performed on the control plane nodes of the downstream cluster. You must configure each control plane node individually.
>
> - The following steps will work on both RKE2 and K3s clusters registered in v2.6.x as well as those registered (or imported) from a previous version of Rancher with an upgrade to v2.6.x.
>
> - These steps will alter the configuration of the downstream RKE2 and K3s clusters and deploy the `kube-api-authn-webhook`. If a future implementation of ACE requires an update to the `kube-api-authn-webhook`, then this would also have to be done manually. For more information on this webhook, click [here]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/#about-the-kube-api-auth-authentication-webhook).
###### **Manual steps to be taken on the control plane of each downstream cluster to enable ACE:**
1. Create a file at `/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml` with the following contents:
apiVersion: v1
kind: Config
clusters:
- name: Default
cluster:
insecure-skip-tls-verify: true
server: http://127.0.0.1:6440/v1/authenticate
users:
- name: Default
user:
insecure-skip-tls-verify: true
current-context: webhook
contexts:
- name: webhook
context:
user: Default
cluster: Default
1. Add the following to the config file (or create one if it doesn’t exist); note that the default location is `/etc/rancher/{rke2,k3s}/config.yaml`:
kube-apiserver-arg:
- authentication-token-webhook-config-file=/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml
1. Finally, run the following commands:
sudo systemctl stop {rke2,k3s}-server
sudo systemctl start {rke2,k3s}-server
# Annotating Registered Clusters
@@ -46,3 +46,13 @@ Most Rancher tools have additional pages located in the toolbar below the _"Apps
> * use the default upgrade option ( i.e do not use _"--force"_ option )
> * uninstall the existing chart and install the upgraded chart
> * delete the resources with immutable fields from the cluster before performing the _"--force"_ upgrade
#### Changes in Rancher v2.6.3
The upgrade button has been removed for legacy apps from the **Apps & Marketplace > Installed Apps** page.
If you have a legacy app installed and want to upgrade it:
- The legacy [feature flag]({{<baseurl>}}/rancher/v2.6/en/installation/resources/feature-flags/) must be turned on (if it's not turned on automatically because of having a legacy app before upgrading)
- You can upgrade the app from cluster explorer, from the left nav section **Legacy > Project > Apps**
- For multi-cluster apps, you can go to **≡ > Multi-cluster Apps** and upgrade the app from there
@@ -12,7 +12,7 @@ This section covers the following topics:
- [Recommended Load Balancer Configuration for Kubernetes Installations](#recommended-load-balancer-configuration-for-kubernetes-installations)
- [Environment for Kubernetes Installations](#environment-for-kubernetes-installations)
- [Recommended Node Roles for Kubernetes Installations](#recommended-node-roles-for-kubernetes-installations)
- [Architecture for an Authorized Cluster Endpoint](#architecture-for-an-authorized-cluster-endpoint)
- [Architecture for an Authorized Cluster Endpoint (ACE)](#architecture-for-an-authorized-cluster-endpoint-ace)
# Separation of Rancher and User Clusters
@@ -104,8 +104,10 @@ Because no additional workloads will be deployed on the Rancher server cluster,
For more best practices for downstream clusters, refer to the [production checklist]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/production) or our [best practices guide.]({{<baseurl>}}/rancher/v2.6/en/best-practices/)
# Architecture for an Authorized Cluster Endpoint
# Architecture for an Authorized Cluster Endpoint (ACE)
If you are using an [authorized cluster endpoint,]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using an [authorized cluster endpoint (ACE),]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/kubectl/) and [API keys]({{<baseurl>}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information.
As of Rancher v2.6.3, ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here]({{<baseurl>}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters).
@@ -20,7 +20,7 @@ This section covers the following topics:
- [The authentication proxy](#1-the-authentication-proxy)
- [Cluster controllers and cluster agents](#2-cluster-controllers-and-cluster-agents)
- [Node agents](#3-node-agents)
- [Authorized cluster endpoint](#4-authorized-cluster-endpoint)
- [Authorized cluster endpoint (ACE)](#4-authorized-cluster-endpoint-ace)
- [Important files](#important-files)
- [Tools for provisioning Kubernetes clusters](#tools-for-provisioning-kubernetes-clusters)
- [Rancher server components and source code](#rancher-server-components-and-source-code)
@@ -103,11 +103,13 @@ If the cluster agent (also called `cattle-cluster-agent`) is not available, one
The `cattle-node-agent` is deployed using a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) resource to make sure it runs on every node in a Rancher-launched Kubernetes cluster. It is used to interact with the nodes when performing cluster operations. Examples of cluster operations include upgrading the Kubernetes version and creating or restoring etcd snapshots.
### 4. Authorized Cluster Endpoint
### 4. Authorized Cluster Endpoint (ACE)
An authorized cluster endpoint allows users to connect to the Kubernetes API server of a downstream cluster without having to route their requests through the Rancher authentication proxy.
> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. It is not available for registered clusters, or for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. The ACE is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
> The [ACE is available for registered RKE2 and K3s clusters]({{<baseurl>}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) as of Rancher v2.6.3.
There are two main reasons why a user might need the authorized cluster endpoint:
@@ -30,6 +30,7 @@ On this page, we provide security-related documentation along with resources to
- [The CIS Benchmark and self-assessment](#the-cis-benchmark-and-self-assessment)
- [Third-party penetration test reports](#third-party-penetration-test-reports)
- [Rancher CVEs and resolutions](#rancher-cves-and-resolutions)
- [Kubernetes Security Best Practices](#kubernetes-security-best-practices)
### Running a CIS Security Scan on a Kubernetes Cluster
@@ -83,3 +84,7 @@ Results:
### Rancher CVEs and Resolutions
Rancher is committed to informing the community of security issues in our products. For the list of CVEs (Common Vulnerabilities and Exposures) for issues we have resolved, refer to [this page.](./cve)
### Kubernetes Security Best Practices
For recommendations on securing your Kubernetes cluster, refer to the [Best Practices](./best-practices) guide.
@@ -0,0 +1,8 @@
---
title: Kubernetes Security Best Practices
weight: 5
---
# Restricting cloud metadata API access
Cloud providers such as AWS, Azure, or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.