Merge pull request #3672 from jtravee/ACE-support

Added documentation for ACE support for imported RKE2/K3S clusters
This commit is contained in:
Billy Tat
2021-11-25 11:05:48 -08:00
committed by GitHub
3 changed files with 54 additions and 6 deletions
@@ -12,6 +12,7 @@ The control that Rancher has to manage a registered cluster depends on the type
- [Management Capabilities for Registered Clusters](#management-capabilities-for-registered-clusters)
- [Configuring K3s Cluster Upgrades](#configuring-k3s-cluster-upgrades)
- [Debug Logging and Troubleshooting for Registered K3s Clusters](#debug-logging-and-troubleshooting-for-registered-k3s-clusters)
- [Authorized Cluster Endpoint Support for RKE2 and K3s Clusters](#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters)
- [Annotating Registered Clusters](#annotating-registered-clusters)
# Prerequisites
@@ -147,8 +148,51 @@ If the cluster becomes stuck in upgrading, restart the `system-upgrade-controlle
To prevent issues when upgrading, the [Kubernetes upgrade best practices](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) should be followed.
# Authorized Cluster Endpoint Support for RKE2 and K3s Clusters
_Available as of v2.6.3_
Authorized Cluster Endpoint (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the authorized cluster endpoint, click [here]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/).
> **Note:**
>
> - These steps only need to be performed on the control plane nodes of the downstream cluster. You must configure each control plane node individually.
>
> - The following steps will work on both RKE2 and K3s clusters registered in v2.6.x as well as those registered (or imported) from a previous version of Rancher with an upgrade to v2.6.x.
>
> - These steps will alter the configuration of the downstream RKE2 and K3s clusters and deploy the `kube-api-authn-webhook`. If a future implementation of ACE requires an update to the `kube-api-authn-webhook`, then this would also have to be done manually. For more information on this webhook, click [here]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/#about-the-kube-api-auth-authentication-webhook).
###### **Manual steps to be taken on the control plane of each downstream cluster to enable ACE:**
1. Create a file at `/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml` with the following contents:
apiVersion: v1
kind: Config
clusters:
- name: Default
cluster:
insecure-skip-tls-verify: true
server: http://127.0.0.1:6440/v1/authenticate
users:
- name: Default
user:
insecure-skip-tls-verify: true
current-context: webhook
contexts:
- name: webhook
context:
user: Default
cluster: Default
1. Add the following to the config file (or create one if it doesnt exist); note that the default location is `/etc/rancher/{rke2,k3s}/config.yaml`:
kube-apiserver-arg:
- authentication-token-webhook-config-file=/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml
1. Finally, run the following commands:
sudo systemctl stop {rke2,k3s}-server
sudo systemctl start {rke2,k3s}-server
# Annotating Registered Clusters
@@ -12,7 +12,7 @@ This section covers the following topics:
- [Recommended Load Balancer Configuration for Kubernetes Installations](#recommended-load-balancer-configuration-for-kubernetes-installations)
- [Environment for Kubernetes Installations](#environment-for-kubernetes-installations)
- [Recommended Node Roles for Kubernetes Installations](#recommended-node-roles-for-kubernetes-installations)
- [Architecture for an Authorized Cluster Endpoint](#architecture-for-an-authorized-cluster-endpoint)
- [Architecture for an Authorized Cluster Endpoint (ACE)](#architecture-for-an-authorized-cluster-endpoint-ace)
# Separation of Rancher and User Clusters
@@ -104,8 +104,10 @@ Because no additional workloads will be deployed on the Rancher server cluster,
For more best practices for downstream clusters, refer to the [production checklist]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/production) or our [best practices guide.]({{<baseurl>}}/rancher/v2.6/en/best-practices/)
# Architecture for an Authorized Cluster Endpoint
# Architecture for an Authorized Cluster Endpoint (ACE)
If you are using an [authorized cluster endpoint,]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using an [authorized cluster endpoint (ACE),]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/kubectl/) and [API keys]({{<baseurl>}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information.
As of Rancher v2.6.3, ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here]({{<baseurl>}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters).
@@ -20,7 +20,7 @@ This section covers the following topics:
- [The authentication proxy](#1-the-authentication-proxy)
- [Cluster controllers and cluster agents](#2-cluster-controllers-and-cluster-agents)
- [Node agents](#3-node-agents)
- [Authorized cluster endpoint](#4-authorized-cluster-endpoint)
- [Authorized cluster endpoint (ACE)](#4-authorized-cluster-endpoint-ace)
- [Important files](#important-files)
- [Tools for provisioning Kubernetes clusters](#tools-for-provisioning-kubernetes-clusters)
- [Rancher server components and source code](#rancher-server-components-and-source-code)
@@ -103,11 +103,13 @@ If the cluster agent (also called `cattle-cluster-agent`) is not available, one
The `cattle-node-agent` is deployed using a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) resource to make sure it runs on every node in a Rancher-launched Kubernetes cluster. It is used to interact with the nodes when performing cluster operations. Examples of cluster operations include upgrading the Kubernetes version and creating or restoring etcd snapshots.
### 4. Authorized Cluster Endpoint
### 4. Authorized Cluster Endpoint (ACE)
An authorized cluster endpoint allows users to connect to the Kubernetes API server of a downstream cluster without having to route their requests through the Rancher authentication proxy.
> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. It is not available for registered clusters, or for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. The ACE is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
> The [ACE is available for registered RKE2 and K3s clusters]({{<baseurl>}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) as of Rancher v2.6.3.
There are two main reasons why a user might need the authorized cluster endpoint: