From d497641fa577633d688947268084982d83ef2e46 Mon Sep 17 00:00:00 2001 From: Silvio Moioli Date: Tue, 1 Apr 2025 18:51:58 +0200 Subject: [PATCH 1/4] Discourage third party software on the upstream cluster Signed-off-by: Silvio Moioli --- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 28 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +----------- 18 files changed, 168 insertions(+), 120 deletions(-) diff --git a/docs/reference-guides/best-practices/rancher-server/rancher-server.md b/docs/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/docs/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/docs/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/docs/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/docs/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 55deb80c8d3..a760bbebff1 100644 --- a/docs/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/docs/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts diff --git a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 07dd8265986..d7b6707b679 100644 --- a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts diff --git a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 55deb80c8d3..a760bbebff1 100644 --- a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts diff --git a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 07dd8265986..d7b6707b679 100644 --- a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts diff --git a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 07dd8265986..d7b6707b679 100644 --- a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts diff --git a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..ce612a63ba7 100644 --- a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,32 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. + +Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. + +The following applications and components generally do not interfere with Rancher or Kubernetes system performance: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs + * Cloud controller managers + * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) + * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + +Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. + +In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. + +In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. + +The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). diff --git a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 07dd8265986..d7b6707b679 100644 --- a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts From 674ef8b72573f5469b85c0e1550b6de978759d0c Mon Sep 17 00:00:00 2001 From: Silvio Moioli Date: Thu, 10 Apr 2025 12:36:37 +0200 Subject: [PATCH 2/4] Recommended Harbor for community uses, reworded for clarity Signed-off-by: Silvio Moioli --- .../tips-for-running-rancher.md | 20 ++++++++++--------- .../tips-for-running-rancher.md | 20 ++++++++++--------- .../tips-for-running-rancher.md | 20 ++++++++++--------- .../tips-for-running-rancher.md | 20 ++++++++++--------- .../tips-for-running-rancher.md | 20 ++++++++++--------- .../tips-for-running-rancher.md | 20 ++++++++++--------- 6 files changed, 66 insertions(+), 54 deletions(-) diff --git a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) diff --git a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) diff --git a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) diff --git a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) diff --git a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) diff --git a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index ce612a63ba7..f5e651016f6 100644 --- a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -16,26 +16,28 @@ When you set up your high-availability Rancher installation, consider the follow ### Minimize Third-Party Software on the Upstream Cluster -Running Rancher, especially as the number of managed clusters, nodes, and workloads increases, can place a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher itself, potentially causing issues. +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. -Every third-party application introduces a risk of interference. To minimize performance and incompatibility issues on the upstream cluster, avoid deploying any applications or components other than essential Kubernetes system components and Rancher. +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. -The following applications and components generally do not interfere with Rancher or Kubernetes system performance: +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: * Rancher internal components, such as Fleet * Rancher extensions * Cluster API components - * CNIs + * CNIs, CPIs, CSIs * Cloud controller managers * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [SUSE Private Registry](https://documentation.suse.com/cloudnative/suse-private-registry/html/private-registry/index.html) + * the [Harbor](https://goharbor.io/) container registry -Remember that each of these has its own minimum resource requirements, which must be met in addition to Rancher's requirements. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. -In particular, SUSE Private Registry can require significant bandwidth for serving images. Ensure sufficient bandwidth is available (and ideally, reserved using Quality of Service mechanisms) for Rancher. +Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. -In high-scale scenarios, consider dedicating separate nodes to non-Rancher software to minimize interference. +For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -The following software has been found to interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) From 2a9af4d13aa3960149959e5db30c55ef823487d4 Mon Sep 17 00:00:00 2001 From: Silvio Moioli Date: Wed, 4 Jun 2025 14:18:45 +0200 Subject: [PATCH 3/4] rewording after feedback from Support about registries Signed-off-by: Silvio Moioli --- .../tips-for-running-rancher.md | 25 ++++++++++++------- .../tips-for-running-rancher.md | 25 ++++++++++++------- .../tips-for-running-rancher.md | 25 ++++++++++++------- .../tips-for-running-rancher.md | 1 - .../tips-for-running-rancher.md | 25 ++++++++++++------- .../tips-for-running-rancher.md | 25 ++++++++++++------- .../tips-for-running-rancher.md | 25 ++++++++++++------- 7 files changed, 96 insertions(+), 55 deletions(-) diff --git a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/docs/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.10/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.11/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.6/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.6/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 5ff66fd8201..f70de4b2cce 100644 --- a/versioned_docs/version-2.6/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.6/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -40,4 +40,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.7/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.8/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index f5e651016f6..c8e12b81efe 100644 --- a/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.9/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -28,20 +28,28 @@ The following applications and components generally do not interfere with Ranche * Cluster API components * CNIs, CPIs, CSIs * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - * the [Harbor](https://goharbor.io/) container registry + * Observability and monitoring tools (except prometheus-rancher-exporter) -Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. -Container registries, in particular, can require significant bandwidth for serving images. Ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms, for Rancher. - -For high-scale deployments, consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. - -The following software can interfere with Rancher performance at scale and is therefore discouraged on the upstream cluster: +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: * [CrossPlane](https://www.crossplane.io/) * [Argo CD](https://argoproj.github.io/cd/) * [Flux](https://fluxcd.io/) * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -63,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - From c92719e7156dba45cf3628f42bec09779f7460fa Mon Sep 17 00:00:00 2001 From: Billy Tat Date: Fri, 4 Jul 2025 19:23:58 -0700 Subject: [PATCH 4/4] Apply to version-2.12 --- .../rancher-server/rancher-server.md | 2 +- .../tips-for-running-rancher.md | 39 +++++++++++++++++-- ...and-best-practices-for-rancher-at-scale.md | 18 +-------- 3 files changed, 38 insertions(+), 21 deletions(-) diff --git a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/rancher-server.md b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/rancher-server.md index 8babb9636fd..3ebebc02958 100644 --- a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/rancher-server.md +++ b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/rancher-server.md @@ -10,7 +10,7 @@ This guide contains our recommendations for running the Rancher server, and is i ### Recommended Architecture and Infrastructure -Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server on a high-availability Kubernetes cluster. +Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation. ### Deployment Strategies diff --git a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md index 9dbf6d903c6..c8e12b81efe 100644 --- a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md +++ b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md @@ -14,8 +14,42 @@ If you are installing Rancher in a vSphere environment, refer to the best practi When you set up your high-availability Rancher installation, consider the following: -### Run Rancher on a Separate Cluster -Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on. +### Minimize Third-Party Software on the Upstream Cluster + +We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues. + +Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability. + +Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster. + +The following applications and components generally do not interfere with Rancher or the Kubernetes system: + * Rancher internal components, such as Fleet + * Rancher extensions + * Cluster API components + * CNIs, CPIs, CSIs + * Cloud controller managers + * Observability and monitoring tools (except prometheus-rancher-exporter) + +Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference. + +The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster: + * [CrossPlane](https://www.crossplane.io/) + * [Argo CD](https://argoproj.github.io/cd/) + * [Flux](https://fluxcd.io/) + * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) + * Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images + +### Guidance for Container Registries + +Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server. + +Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms. + +Consider the following recommendations based on your needs: + +* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution. +* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry. +* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic. ### Make sure nodes are configured correctly for Kubernetes It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/). @@ -37,4 +71,3 @@ However, metrics-driven capacity planning analysis should be the ultimate guidan Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus. After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale. - diff --git a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md index 55deb80c8d3..a760bbebff1 100644 --- a/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md +++ b/versioned_docs/version-2.12/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md @@ -23,23 +23,7 @@ When scaling up Rancher, one typical bottleneck is resource growth in the upstre ### Minimizing Third-Party Software on the Upstream Cluster -Running Rancher at scale can put significant load on internal Kubernetes components, such as `etcd` or `kubeapiserver`. Issues may arise if third-party software interferes with the performance of those components or with Rancher. - -Every third-party piece of software carries a risk of interference. To prevent performance issues on the upstream cluster, you should avoid running any other apps or components, beyond Kubernetes system components and Rancher itself. - -Software in the following categories generally won't interfere with Rancher or Kubernetes system performance: - * Rancher internal components, such as Fleet - * Rancher extensions - * Cluster API components - * CNIs - * Cloud controller managers - * Observability and monitoring tools (with the exception of prometheus-rancher-exporter) - -On the other hand, the following software are found to interfere with Rancher performance at scale: - * [CrossPlane](https://www.crossplane.io/) - * [Argo CD](https://argoproj.github.io/cd/) - * [Flux](https://fluxcd.io/) - * [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33)) +Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context. ### Managing Your Object Counts