From 7cb5d34ff84608cb24bb5a1ad1279cf268b4d5d5 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Wed, 10 Nov 2021 17:55:38 -0500 Subject: [PATCH 01/10] Added new section for ACE support with K3s/RKE2 registered clusters --- .../registered-clusters/_index.md | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md index 7bfa2eff0cb..0a3f3b831dc 100644 --- a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md @@ -12,6 +12,7 @@ The control that Rancher has to manage a registered cluster depends on the type - [Management Capabilities for Registered Clusters](#management-capabilities-for-registered-clusters) - [Configuring K3s Cluster Upgrades](#configuring-k3s-cluster-upgrades) - [Debug Logging and Troubleshooting for Registered K3s Clusters](#debug-logging-and-troubleshooting-for-registered-k3s-clusters) +- [ACE Support for RKE2 and K3s Clusters](#ace-support-for-rke2-and-k3s-clusters) - [Annotating Registered Clusters](#annotating-registered-clusters) # Prerequisites @@ -147,8 +148,49 @@ If the cluster becomes stuck in upgrading, restart the `system-upgrade-controlle To prevent issues when upgrading, the [Kubernetes upgrade best practices](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) should be followed. +# ACE Support for RKE2 and K3s Clusters +_Available as of v2.6.3_ +**Authorized Cluster Endpoint** (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the ACE, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). + +> **Notice:** +> +> - The following steps will work on both RKE2 and K3s clusters registered in v2.6.x as well as those registered (or imported) from a previous version of Rancher with an upgrade to v2.6.x. +> +> - These steps will alter the configuration of the downstream RKE2 and K3s clusters and deploy the `kube-api-authn-webhook`. If a future implementation of ACE requires an update to the `kube-api-authn-webhook`, then this would also have to be done manually. For more information on this webhook, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/#about-the-kube-api-auth-authentication-webhook). + +###### **Manual steps to be taken on the downstream cluster to enable ACE:** + +1. Create a file at `/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml` with the following contents: + + apiVersion: v1 + kind: Config + clusters: + - name: Default + cluster: + insecure-skip-tls-verify: true + server: http://127.0.0.1:6440/v1/authenticate + users: + - name: Default + user: + insecure-skip-tls-verify: true + current-context: webhook + contexts: + - name: webhook + context: + user: Default + cluster: Default + +1. Add the following to the config file (or create one if it doesn’t exist); note that the default location is `/etc/rancher/{rke2,k3s}/config.yaml`: + + kube-apiserver-arg: + - authentication-token-webhook-config-file=/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml + +1. Finally, run the following commands: + + - `sudo systemctl stop {rke2,k3s}-server` + - `sudo systemctl start {rke2,k3s}-server` # Annotating Registered Clusters From a46cdf749b5cf2512a03fdf37b9ef51208d7e0cd Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Wed, 10 Nov 2021 18:04:39 -0500 Subject: [PATCH 02/10] Added link to new section in ACE architecture page --- .../v2.6/en/overview/architecture-recommendations/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md index b679b3a3f72..c0a22064086 100644 --- a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md +++ b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md @@ -109,3 +109,5 @@ For more best practices for downstream clusters, refer to the [production checkl If you are using an [authorized cluster endpoint,]({{}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/kubectl/) and [API keys]({{}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information. + +As of Rancher v2.6.3, ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here]({{}}/v2.6/en/cluster-provisioning/registered-clusters/#ace-support-for-rke2-and-k3s-clusters). From d9a5d4a5aa11c650033a085f4a3ca04463f6c884 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Wed, 10 Nov 2021 18:17:01 -0500 Subject: [PATCH 03/10] Updated ACE verbiage for new support in 2.6.3 --- content/rancher/v2.6/en/overview/architecture/_index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.6/en/overview/architecture/_index.md b/content/rancher/v2.6/en/overview/architecture/_index.md index 1e70eccdd6c..62946c0ff6e 100644 --- a/content/rancher/v2.6/en/overview/architecture/_index.md +++ b/content/rancher/v2.6/en/overview/architecture/_index.md @@ -103,11 +103,13 @@ If the cluster agent (also called `cattle-cluster-agent`) is not available, one The `cattle-node-agent` is deployed using a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) resource to make sure it runs on every node in a Rancher-launched Kubernetes cluster. It is used to interact with the nodes when performing cluster operations. Examples of cluster operations include upgrading the Kubernetes version and creating or restoring etcd snapshots. -### 4. Authorized Cluster Endpoint +### 4. Authorized Cluster Endpoint (ACE) An authorized cluster endpoint allows users to connect to the Kubernetes API server of a downstream cluster without having to route their requests through the Rancher authentication proxy. -> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. It is not available for registered clusters, or for clusters in a hosted Kubernetes provider, such as Amazon's EKS. +> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. The ACE is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS. + +> The [ACE is available for registered RKE2 and K3s clusters]({{}}/rancher/v2.6/en/cluster-provisioning/registered-clusters/#ace-support-for-rke2-and-k3s-clusters) as of Rancher v2.6.3. There are two main reasons why a user might need the authorized cluster endpoint: From 30a04e05a3ba7838f14463548ac01fa00cf97fa0 Mon Sep 17 00:00:00 2001 From: Jen Travinski Date: Mon, 15 Nov 2021 09:17:54 -0500 Subject: [PATCH 04/10] Update content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md Co-authored-by: Billy Tat --- .../v2.6/en/cluster-provisioning/registered-clusters/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md index 0a3f3b831dc..ee19218c369 100644 --- a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md @@ -152,7 +152,7 @@ To prevent issues when upgrading, the [Kubernetes upgrade best practices](https: _Available as of v2.6.3_ -**Authorized Cluster Endpoint** (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the ACE, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). +Authorized Cluster Endpoint (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the ACE, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). > **Notice:** > From efe58b223f04e0b6dfb8ae46a7acaa1e87aa5336 Mon Sep 17 00:00:00 2001 From: Jen Travinski Date: Mon, 15 Nov 2021 09:18:15 -0500 Subject: [PATCH 05/10] Update content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md Co-authored-by: Billy Tat --- .../v2.6/en/cluster-provisioning/registered-clusters/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md index ee19218c369..0f2a367fca1 100644 --- a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md @@ -148,7 +148,7 @@ If the cluster becomes stuck in upgrading, restart the `system-upgrade-controlle To prevent issues when upgrading, the [Kubernetes upgrade best practices](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/) should be followed. -# ACE Support for RKE2 and K3s Clusters +# Authorized Cluster Endpoint Support for RKE2 and K3s Clusters _Available as of v2.6.3_ From 2775dc4aa29a5fea6515197f1eb1acc90644cc46 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Mon, 15 Nov 2021 09:28:00 -0500 Subject: [PATCH 06/10] Updating per feedback --- .../en/cluster-provisioning/registered-clusters/_index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md index 0a3f3b831dc..b65e5a68e8d 100644 --- a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md @@ -12,7 +12,7 @@ The control that Rancher has to manage a registered cluster depends on the type - [Management Capabilities for Registered Clusters](#management-capabilities-for-registered-clusters) - [Configuring K3s Cluster Upgrades](#configuring-k3s-cluster-upgrades) - [Debug Logging and Troubleshooting for Registered K3s Clusters](#debug-logging-and-troubleshooting-for-registered-k3s-clusters) -- [ACE Support for RKE2 and K3s Clusters](#ace-support-for-rke2-and-k3s-clusters) +- [Authorized Cluster Endpoint Support for RKE2 and K3s Clusters](#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) - [Annotating Registered Clusters](#annotating-registered-clusters) # Prerequisites @@ -154,7 +154,7 @@ _Available as of v2.6.3_ **Authorized Cluster Endpoint** (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the ACE, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). -> **Notice:** +> **Note:** > > - The following steps will work on both RKE2 and K3s clusters registered in v2.6.x as well as those registered (or imported) from a previous version of Rancher with an upgrade to v2.6.x. > @@ -189,8 +189,8 @@ _Available as of v2.6.3_ 1. Finally, run the following commands: - - `sudo systemctl stop {rke2,k3s}-server` - - `sudo systemctl start {rke2,k3s}-server` + sudo systemctl stop {rke2,k3s}-server + sudo systemctl start {rke2,k3s}-server # Annotating Registered Clusters From 2fd998b3f482c548c2a68fb8f08133c360729807 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Mon, 15 Nov 2021 09:48:44 -0500 Subject: [PATCH 07/10] Clarified acronym 'ACE' --- .../v2.6/en/overview/architecture-recommendations/_index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md index c0a22064086..2b027ab2437 100644 --- a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md +++ b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md @@ -104,10 +104,10 @@ Because no additional workloads will be deployed on the Rancher server cluster, For more best practices for downstream clusters, refer to the [production checklist]({{}}/rancher/v2.6/en/cluster-provisioning/production) or our [best practices guide.]({{}}/rancher/v2.6/en/best-practices/) -# Architecture for an Authorized Cluster Endpoint +# Architecture for an Authorized Cluster Endpoint (ACE) -If you are using an [authorized cluster endpoint,]({{}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role. +If you are using an [authorized cluster endpoint (ACE),]({{}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/kubectl/) and [API keys]({{}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information. -As of Rancher v2.6.3, ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here]({{}}/v2.6/en/cluster-provisioning/registered-clusters/#ace-support-for-rke2-and-k3s-clusters). +As of Rancher v2.6.3, ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here]({{}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters). From d79ecfd2f60b6b659a48f09965fecbb28784fa25 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Tue, 23 Nov 2021 09:37:02 -0500 Subject: [PATCH 08/10] Added clarifying note about CP node setup --- .../en/cluster-provisioning/registered-clusters/_index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md index 20d4d78ca92..54fee0662cd 100644 --- a/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.6/en/cluster-provisioning/registered-clusters/_index.md @@ -152,15 +152,17 @@ To prevent issues when upgrading, the [Kubernetes upgrade best practices](https: _Available as of v2.6.3_ -Authorized Cluster Endpoint (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the ACE, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). +Authorized Cluster Endpoint (ACE) support has been added for registered RKE2 and K3s clusters. This support includes manual steps you will perform on the downstream cluster to enable the ACE. For additional information on the authorized cluster endpoint, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/). > **Note:** > +> - These steps only need to be performed on the control plane nodes of the downstream cluster. You must configure each control plane node individually. +> > - The following steps will work on both RKE2 and K3s clusters registered in v2.6.x as well as those registered (or imported) from a previous version of Rancher with an upgrade to v2.6.x. > > - These steps will alter the configuration of the downstream RKE2 and K3s clusters and deploy the `kube-api-authn-webhook`. If a future implementation of ACE requires an update to the `kube-api-authn-webhook`, then this would also have to be done manually. For more information on this webhook, click [here]({{}}/rancher/v2.6/en/cluster-admin/cluster-access/ace/#about-the-kube-api-auth-authentication-webhook). -###### **Manual steps to be taken on the downstream cluster to enable ACE:** +###### **Manual steps to be taken on the control plane of each downstream cluster to enable ACE:** 1. Create a file at `/var/lib/rancher/{rke2,k3s}/kube-api-authn-webhook.yaml` with the following contents: From 50a7292b4e29a6d222cd4065f35508f3b05264e8 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Tue, 23 Nov 2021 09:54:21 -0500 Subject: [PATCH 09/10] Updated links --- content/rancher/v2.6/en/overview/architecture/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.6/en/overview/architecture/_index.md b/content/rancher/v2.6/en/overview/architecture/_index.md index 62946c0ff6e..3fb4384308e 100644 --- a/content/rancher/v2.6/en/overview/architecture/_index.md +++ b/content/rancher/v2.6/en/overview/architecture/_index.md @@ -20,7 +20,7 @@ This section covers the following topics: - [The authentication proxy](#1-the-authentication-proxy) - [Cluster controllers and cluster agents](#2-cluster-controllers-and-cluster-agents) - [Node agents](#3-node-agents) - - [Authorized cluster endpoint](#4-authorized-cluster-endpoint) + - [Authorized cluster endpoint (ACE)](#4-authorized-cluster-endpoint-ace) - [Important files](#important-files) - [Tools for provisioning Kubernetes clusters](#tools-for-provisioning-kubernetes-clusters) - [Rancher server components and source code](#rancher-server-components-and-source-code) @@ -109,7 +109,7 @@ An authorized cluster endpoint allows users to connect to the Kubernetes API ser > The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{}}/rancher/v2.6/en/cluster-provisioning/rke-clusters) to provision the cluster. The ACE is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS. -> The [ACE is available for registered RKE2 and K3s clusters]({{}}/rancher/v2.6/en/cluster-provisioning/registered-clusters/#ace-support-for-rke2-and-k3s-clusters) as of Rancher v2.6.3. +> The [ACE is available for registered RKE2 and K3s clusters]({{}}/v2.6/en/cluster-provisioning/registered-clusters/#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) as of Rancher v2.6.3. There are two main reasons why a user might need the authorized cluster endpoint: From 00a81417d423ac64156322531d53b1bb0a22d324 Mon Sep 17 00:00:00 2001 From: Jennifer Travinski Date: Tue, 23 Nov 2021 09:58:05 -0500 Subject: [PATCH 10/10] Added acronym, updated link --- .../v2.6/en/overview/architecture-recommendations/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md index 2b027ab2437..e476540c0b4 100644 --- a/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md +++ b/content/rancher/v2.6/en/overview/architecture-recommendations/_index.md @@ -12,7 +12,7 @@ This section covers the following topics: - [Recommended Load Balancer Configuration for Kubernetes Installations](#recommended-load-balancer-configuration-for-kubernetes-installations) - [Environment for Kubernetes Installations](#environment-for-kubernetes-installations) - [Recommended Node Roles for Kubernetes Installations](#recommended-node-roles-for-kubernetes-installations) -- [Architecture for an Authorized Cluster Endpoint](#architecture-for-an-authorized-cluster-endpoint) +- [Architecture for an Authorized Cluster Endpoint (ACE)](#architecture-for-an-authorized-cluster-endpoint-ace) # Separation of Rancher and User Clusters