Merge branch 'v2.12.0' into gce

This commit is contained in:
Billy Tat
2025-07-25 16:34:02 -07:00
committed by GitHub
480 changed files with 3560 additions and 233657 deletions
@@ -199,7 +199,6 @@ Because `rancher` is the default option for `ingress.tls.source`, we are not spe
- Set the `hostname` to the DNS name you pointed at your load balancer.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.7.0`
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -240,7 +239,6 @@ In the following command,
- `ingress.tls.source` is set to `letsEncrypt`
- `letsEncrypt.email` is set to the email address used for communication about your certificate (for example, expiry notices)
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
:::warning
@@ -289,7 +287,6 @@ If you want to check if your certificates are correct, see [How do I check Commo
- Set the `hostname`.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- Set `ingress.tls.source` to `secret`.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -1,72 +0,0 @@
---
title: Upgrading a Hardened Custom/Imported Cluster to Kubernetes v1.25
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrade-a-hardened-cluster-to-k8s-v1-25"/>
</head>
Kubernetes v1.25 changes how clusters describe and implement security policies. From this version forward, [Pod Security Policies (PSPs)](https://kubernetes.io/docs/concepts/security/pod-security-policy/) are no longer available. Kubernetes v1.25 replaces them with new security objects: [Pod Security Standards (PSS)](https://kubernetes.io/docs/concepts/security/pod-security-standards/), and [Pod Security Admissions (PSAs)](https://kubernetes.io/docs/concepts/security/pod-security-admission/).
If you have custom or imported hardened clusters, you must take special preparations to ensure that the upgrade from an earlier version of Kubernetes to v1.25 or later goes smoothly.
:::note
After you upgrade to v1.25, add the necessary Rancher namespace exemptions. See [Pod Security Admission (PSA) Configuration Templates](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md#exempting-required-rancher-namespaces) for more details.
:::
## Upgrading Imported Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
Perform the following on each node in the cluster:
1. Save [`rancher-psact.yaml`](./rancher-psact.yaml) in `/etc/rancher/rke2`.
1. Edit the RKE2 configuration file:
1. Update the `profile` field to `cis-1.23`.
1. Specify the path for the configuration file that you just added: `pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml`.
</TabItem>
<TabItem value="K3s">
Perform the following on each node in the cluster:
Follow the official K3s instructions on [Upgrading Hardened Clusters from v1.24.x to v1.25.x](https://docs.k3s.io/known-issues#hardened-125), but use a [custom](./rancher-psact.yaml) Rancher PSA configuration template, instead of the configuration provided on the official K3s site.
</TabItem>
</Tabs>
After you perform these steps, you can upgrade the cluster's Kubernetes version through the Rancher UI:
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
## Upgrading Custom Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. Under **Basics > Security**, in the **CIS Profile** dropdown menu, select `cis-1.23`.
1. In the **Pod Security Admission Configuration Template** dropdown menu, select `rancher-restricted`.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
</TabItem>
<TabItem value="K3s">
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit YAML**.
1. Delete `PodSecurityPolicy` from `kube-apiserver-arg.enable-admission-plugins`
1. Add this line to the `spec` field: `defaultPodSecurityAdmissionConfigurationTemplateName: rancher-restricted`
1. Update `kubernetesVersion` to your chosen version (v1.25 or later).
1. Click **Save**.
</TabItem>
</Tabs>
@@ -152,7 +152,6 @@ Upgrade Rancher to the latest version with all your settings.
Take all the values from the previous step and append them to the command using `--set key=value`.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
@@ -186,8 +185,7 @@ Alternatively, it's possible to export the current values to a file and referenc
```
1. Update only the Rancher version:
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
@@ -24,7 +24,19 @@ The following is a list of feature flags available in Rancher. If you've upgrade
- `harvester`: Manages access to the Virtualization Management page, where users can navigate directly to Harvester clusters and access the Harvester UI. See [Harvester Integration Overview](../../../integrations-in-rancher/harvester/overview.md) for more information.
- `istio-virtual-service-ui`: Enables a [visual interface](../../../how-to-guides/advanced-user-guides/enable-experimental-features/istio-traffic-management-features.md) to create, read, update, and delete Istio virtual services and destination rules, which are Istio traffic management features.
- `legacy`: Enables a set of features from 2.5.x and earlier, that are slowly being phased out in favor of newer implementations. These are a mix of deprecated features as well as features that will eventually be available to newer versions. This flag is disabled by default on new Rancher installations. If you're upgrading from a previous version of Rancher, this flag is enabled.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream RKE2/K3s clusters, currently limited to imported clusters and the local cluster, with plans to expand support to node-driver clusters.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream imported RKE2/K3s clusters, as well as in the local cluster if it is an RKE2/K3s cluster.
:::note Important:
This `managed-system-upgrade-controller` flag is intended for **internal use only** and does not have an associated Feature CR. Use with caution.
To control whether Rancher should manage the Kubernetes version of imported RKE2/K3s clusters, it is recommended to use the **imported-cluster-version-management** feature that is available in Rancher v2.11.0 or newer.
For more details, please refer to the Rancher documentation for version v2.11 or newer.
:::
- `multi-cluster-management`: Allows multi-cluster provisioning and management of Kubernetes clusters. This flag can only be set at install time. It can't be enabled or disabled later.
- `rke1-custom-node-cleanup`: Enables cleanup of deleted RKE1 custom nodes. We recommend that you keep this flag enabled, to prevent removed nodes from attempting to rejoin the cluster.
- `rke2`: Enables provisioning RKE2 clusters. This flag is enabled by default.
@@ -63,8 +63,6 @@ For information on enabling experimental features, refer to [this page.](../../.
| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system container images, e.g., http://registry.example.com/ |
| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" |
| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. |
| `global.cattle.psp.enabled` | `true` | `bool` - select 'false' to disable PSPs for Kubernetes v1.25 and above when using Rancher v2.7.2-v2.7.4. When using Rancher v2.7.5 and above, Rancher attempts to detect if a cluster is running a Kubernetes version where PSPs are not supported, and will default it's usage of PSPs to false if it can determine that PSPs are not supported in the cluster. Users can still manually override this by explicitly providing `true` or `false` for this value. Rancher will still use PSPs by default in clusters which support PSPs (such as clusters running Kubernetes v1.24 or lower). |
### Bootstrap Password
@@ -172,7 +172,6 @@ kubectl create namespace cattle-system
Next, install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
Placeholder | Description
------------|-------------
@@ -203,7 +202,6 @@ Create Kubernetes secrets from your own certificates for Rancher to use. The com
Install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------- |
@@ -12,6 +12,14 @@ The steps to set up RKE, RKE2, or K3s are shown below.
For convenience, export the IP address and port of your proxy into an environment variable and set up the HTTP_PROXY variables for your current shell on every node:
:::caution
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable for Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
```
export proxy_host="10.0.0.5:8888"
export HTTP_PROXY=http://${proxy_host}
@@ -35,7 +35,7 @@ The Rancher API server is built on top of an embedded Kubernetes API server and
### Authorization and Role-Based Access Control
- **User management:** The Rancher API server [manages user identities](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md) that correspond to external authentication providers like Active Directory or GitHub, in addition to local users.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) policies.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) standards.
### Working with Kubernetes
@@ -128,7 +128,6 @@ The final command to install Rancher is below. The command requires a domain nam
To install a specific Rancher version, use the `--version` flag (e.g., `--version 2.6.6`). Otherwise, the latest Rancher is installed by default. Refer to [Choosing a Rancher Version](../../installation-and-upgrade/resources/choose-a-rancher-version.md).
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
See [Setting up the Bootstrap Password](../../installation-and-upgrade/resources/bootstrap-password.md#password-requirements) for password requirements.
@@ -13,9 +13,3 @@ title: Install Rancher CIS Benchmark
1. Click **Install**.
**Result:** The CIS scan application is deployed on the Kubernetes cluster.
:::note
If you are running Kubernetes v1.24 or earlier, and have a [Pod Security Policy](../../new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) (PSP) hardened cluster, CIS Benchmark 4.0.0 and later disable PSPs by default. To install CIS Benchmark on a PSP-hardened cluster, set `global.psp.enabled` to `true` in the values before installing the chart. [Pod Security Admission](../../new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) (PSA) hardened clusters aren't affected.
:::
@@ -39,7 +39,6 @@ Values set from the Rancher API will override the value passed in through the co
When installing Rancher with a Helm chart, use the `--set` option. In the below example, two features are enabled by passing the feature flag names in a comma separated list:
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-latest/rancher \
@@ -9,7 +9,6 @@ title: Enable Istio in the Cluster
:::note Prerequisites:
- Only a user with the `cluster-admin` [Kubernetes default role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) assigned can configure and install Istio in a Kubernetes cluster.
- If you have pod security policies, you will need to install Istio with the CNI enabled. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/pod-security-policies.md)
- To install Istio on an RKE2 cluster, additional steps are required. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/install-istio-on-rke2-cluster.md)
- To install Istio in a cluster where project network isolation is enabled, additional steps are required. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/project-network-isolation.md)
@@ -1,43 +0,0 @@
---
title: Applying Pod Security Policies to Projects
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/manage-projects/manage-pod-security-policies"/>
</head>
:::note
These cluster options are only available for [clusters in which Rancher has launched Kubernetes](../../new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
:::
You can always assign a pod security policy (PSP) to an existing project if you didn't assign one during creation.
### Prerequisites
- Create a Pod Security Policy within Rancher. Before you can assign a default PSP to an existing project, you must have a PSP available for assignment. For instruction, see [Creating Pod Security Policies](../../new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md).
- Assign a default Pod Security Policy to the project's cluster. You can't assign a PSP to a project until one is already applied to the cluster. For more information, see [the documentation about adding a pod security policy to a cluster](../../new-user-guides/manage-clusters/add-a-pod-security-policy.md).
### Applying a Pod Security Policy
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to move a namespace and click **Explore**.
1. Click **Cluster > Projects/Namespaces**.
1. Find the project that you want to add a PSP to. From that project, select **⋮ > Edit Config**.
1. From the **Pod Security Policy** drop-down, select the PSP you want to apply to the project.
Assigning a PSP to a project will:
- Override the cluster's default PSP.
- Apply the PSP to the project.
- Apply the PSP to any namespaces you add to the project later.
1. Click **Save**.
**Result:** The PSP is applied to the project and any namespaces added to the project.
:::note
Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked to determine if they comply with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP.
:::
@@ -24,7 +24,6 @@ You can use projects to perform actions like:
- [Set resource quotas](manage-project-resource-quotas/manage-project-resource-quotas.md)
- [Manage namespaces](../../new-user-guides/manage-namespaces.md)
- [Configure tools](../../../reference-guides/rancher-project-tools.md)
- [Configure pod security policies](manage-pod-security-policies.md)
### Authorization
@@ -38,12 +38,6 @@ The user retention feature is disabled by default.
For more information, see [Enabling User Retention](../../advanced-user-guides/enable-user-retention.md).
## Pod Security Policies
_Pod Security Policies_ (or PSPs) are objects that control security-sensitive aspects of pod specification, e.g. root privileges. If a pod does not meet the conditions specified in the PSP, Kubernetes will not allow it to start, and Rancher will display an error message.
For more information how to create and use PSPs, see [Pod Security Policies](create-pod-security-policies.md).
## Provisioning Drivers
Drivers in Rancher allow you to manage which providers can be used to provision [hosted Kubernetes clusters](../kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/set-up-clusters-from-hosted-kubernetes-providers.md) or [nodes in an infrastructure provider](../launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md) to allow Rancher to deploy and manage Kubernetes.
@@ -1,82 +0,0 @@
---
title: Creating Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies"/>
</head>
:::caution
Pod Security Policies are only available in Kubernetes until v1.24. [Pod Security Standards](pod-security-standards.md) are the built-in alternative.
:::
[Pod Security Policies (PSPs)](https://kubernetes.io/docs/concepts/security/pod-security-policy/) are objects that control security-sensitive aspects of the pod specification (such as root privileges).
If a pod doesn't meet the conditions specified in the PSP, Kubernetes won't allow it to start, and Rancher will display the following error message: `Pod <NAME> is forbidden: unable to validate...`.
## How PSPs Work
You can assign PSPs at the cluster or project level.
PSPs work through inheritance:
- By default, PSPs assigned to a cluster are inherited by its projects, as well as any namespaces added to those projects.
- **Exception:** Namespaces that are not assigned to projects do not inherit PSPs, regardless of whether the PSP is assigned to a cluster or project. Because these namespaces have no PSPs, workload deployments to these namespaces will fail, which is the default Kubernetes behavior.
- You can override the default PSP by assigning a different PSP directly to the project.
Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked if it complies with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP.
Read more about Pod Security Policies in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).
## Default PSPs
Rancher ships with three default Pod Security Policies (PSPs): the `restricted-noroot`, `restricted` and `unrestricted` policies.
### Restricted-NoRoot
This policy is based on the Kubernetes [example restricted policy](https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/policy/restricted-psp.yaml). It significantly restricts what types of pods can be deployed to a cluster or project. This policy:
- Prevents pods from running as a privileged user and prevents escalation of privileges.
- Validates that server-required security mechanisms are in place, such as restricting what volumes can be mounted to only the core volume types and preventing root supplemental groups from being added.
### Restricted
This policy is a relaxed version of the `restricted-noroot` policy, with almost all the restrictions in place, except for the fact that it allows running containers as a privileged user.
### Unrestricted
This policy is equivalent to running Kubernetes with the PSP controller disabled. It has no restrictions on what pods can be deployed into a cluster or project.
:::note important
When disabling PSPs, default PSPs are **not** automatically deleted from your cluster. You must manually delete them if they're no longer needed.
:::
## Creating PSPs
Using Rancher, you can create a Pod Security Policy using our GUI rather than creating a YAML file.
### Requirements
Rancher can only assign PSPs for clusters that are [launched using RKE](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
You must enable PSPs at the cluster level before you can assign them to a project. This can be configured by [editing the cluster](../../../reference-guides/cluster-configuration/cluster-configuration.md).
It is a best practice to set PSP at the cluster level.
We recommend adding PSPs during cluster and project creation instead of adding it to an existing one.
### Creating PSPs in the Rancher UI
1. In the upper left corner, click **☰ > Cluster Management**.
1. In the left navigation bar, click **Pod Security Policies**.
1. Click **Add Policy**.
1. Name the policy.
1. Complete each section of the form. Refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) for more information on what each policy does.
1. Click **Create**.
## Configuration
The Kubernetes documentation on PSPs is [here](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).
@@ -11,54 +11,6 @@ They became available and were turned on by default in Kubernetes v1.23, and rep
PSS define security levels for workloads. PSAs describe requirements for pod security contexts and related fields. PSAs reference PSS levels to define security restrictions.
## Upgrade to Pod Security Standards (PSS)
Ensure that you migrate all PSPs to another workload security mechanism. This includes mapping your current PSPs to Pod Security Standards for enforcement with the [PSA controller](https://kubernetes.io/docs/concepts/security/pod-security-admission/). If the PSA controller won't meet all of your organization's needs, we recommend that you use a policy engine, such as [Kubewarden](https://www.kubewarden.io/), [Kyverno](https://kyverno.io/), or [NeuVector](https://neuvector.com/). Refer to the documentation of your policy engine of choice for more information on how to migrate from PSPs.
:::caution
You must add your new policy enforcement mechanisms _before_ you remove the PodSecurityPolicy objects. If you don't, you may create an opportunity for privilege escalation attacks within the cluster.
:::
### Removing PodSecurityPolicies from Rancher-Maintained Apps & Marketplace Workloads
Rancher v2.7.2 offers a new major version of Rancher-maintained Helm charts. v102.x.y allows you to remove PSPs that were installed with previous versions of the chart. This new version replaces non-standard PSPs switches with the standardized `global.cattle.psp.enabled` switch, which is turned off by default.
You must perform the following steps _while still in Kubernetes v1.24_:
1. Configure the PSA controller to suit your needs. You can use one of Rancher's built-in [PSA Configuration Templates](#pod-security-admission-configuration-templates), or create a custom template and apply it to the clusters that you are migrating.
1. Map your active PSPs to Pod Security Standards:
1. See which PSPs are still active in your cluster:
:::caution
This strategy may miss workloads that aren't currently running, such as CronJobs, workloads currently scaled to zero, or workloads that haven't rolled out yet.
:::
```shell
kubectl get pods \
--all-namespaces \
--output jsonpath='{.items[*].metadata.annotations.kubernetes\.io\/psp}' \
| tr " " "\n" | sort -u
```
1. Follow the Kubernetes guide on [Mapping PSPs to Pod Security Standards](https://kubernetes.io/docs/reference/access-authn-authz/psp-to-pod-security-standards/) to apply PSSs to your workloads that were relying on PSPs. See [Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission controller](https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/) for more details.
1. To remove PSPs from Rancher charts, upgrade the charts to the latest v102.x.y version _before_ you upgrade to Kubernetes v1.25. Make sure that the **Enable PodSecurityPolicies** option is **disabled**. This will remove any PSPs that were installed with previous chart versions.
:::info important
If you want to upgrade your charts to v102.x.y, but don't plan on upgrading your clusters to Kubernetes v1.25 and moving away from PSPs, make sure that you select the option **Enable PodSecurityPolicies** for each chart that you are upgrading.
:::
### Cleaning Up Releases After a Kubernetes v1.25 Upgrade
If you experience problems while removing PSPs from your charts, or have charts that don't contain a built-in mechanism for removing PSPs, your chart upgrades or deletions might fail with an error message such as the following:
```console
Error: UPGRADE FAILED: resource mapping not found for name: "<object-name>" namespace: "<object-namespace>" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
ensure CRDs are installed first
```
This happens when Helm tries to query the cluster for objects that were stored in a previous release's data blob. To clean up these releases and avoid this error, use the `helm-mapkubeapis` Helm plugin. To learn more about `helm-mapkubeapis`, how it works, and how it can be fine-tuned for your use case, see the [official Helm documentation](https://github.com/helm/helm-mapkubeapis#readme).
Note that Helm plugin installation is local to the machine that you run the commands from. Therefore, make sure that you run both the installation and cleanup from the same machine.
#### Install `helm-mapkubeapis`
1. Open your terminal in the machine you intend to use `helm-mapkubeapis` from and install the plugin:
@@ -110,15 +62,6 @@ After you install the `helm-mapkubeapis` plugin, clean up the releases that beca
1. Finally, after reviewing the changes, perform a full run with `helm mapkubeapis <release-name> --namespace <release-namespace>`.
#### Upgrading Charts to a Version That Supports Kubernetes v1.25
You can proceed with your upgrade once any releases that had lingering PSPs are cleaned up. For Rancher-maintained workloads, follow the steps outlined in the [Removing PodSecurityPolicies from Rancher-maintained Apps & Marketplace workloads](#removing-podsecuritypolicies-from-rancher-maintained-apps--marketplace-workloads) section of this document.
For workloads not maintained by Rancher, refer to the vendor documentation.
:::caution
Do not skip this step. Applications incompatible with Kubernetes v1.25 aren't guaranteed to work after a cleanup.
:::
## Pod Security Admission Configuration Templates
Rancher offers PSA configuration templates. These are pre-defined security configurations that you can apply to a cluster. Rancher admins (or those with the right permissions) can [create, manage, and edit](./psa-config-templates.md) PSA templates.
@@ -168,7 +168,6 @@ Follow the steps to [install cert-manager](../../../getting-started/installation
Use the same version of Helm to install Rancher, that was used on the first cluster.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```bash
helm install rancher rancher-latest/rancher \
@@ -56,6 +56,15 @@ GKE Autopilot clusters aren't supported. See [Compare GKE Autopilot and Standard
9. If you are using self-signed certificates, you will receive the message `certificate signed by unknown authority`. To work around this validation, copy the command starting with `curl` displayed in Rancher to your clipboard. Then run the command on a node where kubeconfig is configured to point to the cluster you want to import.
10. When you finish running the command(s) on your node, click **Done**.
:::important
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable in Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
**Result:**
- Your cluster is registered and assigned a state of **Pending**. Rancher is deploying resources to manage your cluster.
@@ -248,15 +257,7 @@ Therefore, when Rancher registers a cluster, it assumes that several capabilitie
However, if the cluster has a certain capability, such as the ability to use a pod security policy, a user of that cluster might still want to select pod security policies for the cluster in the Rancher UI. In order to do that, the user will need to manually indicate to Rancher that pod security policies are enabled for the cluster.
By annotating a registered cluster, it is possible to indicate to Rancher that a cluster was given a pod security policy, or another capability, outside of Rancher.
This example annotation indicates that a pod security policy is enabled:
```json
"capabilities.cattle.io/pspEnabled": "true"
```
The following annotation indicates Ingress capabilities. Note that the values of non-primitive objects need to be JSON encoded, with quotations escaped.
By annotating a registered cluster, it is possible to indicate to Rancher that a cluster was given Ingress capabilities, or another capability, outside of Rancher. The following annotation indicates Ingress capabilities. Note that the values of non-primitive objects need to be JSON encoded, with quotations escaped.
```json
"capabilities.cattle.io/ingressCapabilities": "[
@@ -273,7 +274,6 @@ These capabilities can be annotated for the cluster:
- `loadBalancerCapabilities`
- `nodePoolScalingSupported`
- `nodePortRange`
- `pspEnabled`
- `taintSupport`
All the capabilities and their type definitions can be viewed in the Rancher API view, at `[Rancher Server URL]/v3/schemas/capabilities`.
@@ -304,4 +304,4 @@ This section lists some of the most common errors that may occur when importing
```sh
az aks update --resource-group <resource-group> --name <cluster-name> --enable-local-accounts
```
```
@@ -1,43 +0,0 @@
---
title: Adding a Pod Security Policy
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/add-a-pod-security-policy"/>
</head>
:::note Prerequisite:
The options below are available only for clusters that are [launched using RKE.](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md)
:::
When your cluster is running pods with security-sensitive configurations, assign it a [pod security policy](../authentication-permissions-and-global-configuration/create-pod-security-policies.md), which is a set of rules that monitors the conditions and settings in your pods. If a pod doesn't meet the rules specified in your policy, the policy stops it from running.
You can assign a pod security policy when you provision a cluster. However, if you need to relax or restrict security for your pods later, you can update the policy while editing your cluster.
1. Click **☰ > Cluster Management**.
1. Go to the cluster to which you want to apply a pod security policy and click **⋮ > Edit Config**.
1. From **Pod Security Policy Support**, select **Enabled**.
:::note
This option is only available for clusters [provisioned by RKE](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
:::
4. From the **Default Pod Security Policy** drop-down, select the policy you want to apply to the cluster.
Rancher ships with [policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md#default-psps) of `restricted` and `unrestricted`, although you can [create custom policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md#creating-psps) as well.
5. Click **Save**.
**Result:** The pod security policy is applied to the cluster and any projects within the cluster.
:::note
Workloads already running before assignment of a pod security policy are grandfathered in. Even if they don't meet your pod security policy, workloads running before assignment of the policy continue to run.
To check if a running workload passes your pod security policy, clone or upgrade it.
:::
@@ -1,29 +0,0 @@
---
title: Assigning Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/assign-pod-security-policies"/>
</head>
_Pod Security Policies_ are objects that control security-sensitive aspects of pod specification (like root privileges).
## Adding a Default Pod Security Policy
When you create a new cluster with RKE, you can configure it to apply a PSP immediately. As you create the cluster, use the **Cluster Options** to enable a PSP. The PSP assigned to the cluster will be the default PSP for projects within the cluster.
:::note Prerequisite:
Create a Pod Security Policy within Rancher. Before you can assign a default PSP to a new cluster, you must have a PSP available for assignment. For instruction, see [Creating Pod Security Policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md).
:::
:::note
For security purposes, we recommend assigning a PSP as you create your clusters.
:::
To enable a default Pod Security Policy, set the **Pod Security Policy Support** option to **Enabled**, and then make a selection from the **Default Pod Security Policy** drop-down.
When the cluster finishes provisioning, the PSP you selected is applied to all projects within the cluster.
@@ -126,12 +126,11 @@ Rancher extends Kubernetes to allow the application of [Pod Security Policies](h
This section describes how to create a new project with a name and with optional pod security policy, members, and resource quotas.
1. [Name a new project.](#1-name-a-new-project)
2. [Optional: Select a pod security policy.](#2-optional-select-a-pod-security-policy)
3. [Recommended: Add project members.](#3-recommended-add-project-members)
4. [Optional: Add resource quotas.](#4-optional-add-resource-quotas)
1. [Name a new project.](#name-a-new-project)
1. [Recommended: Add project members.](#recommended-add-project-members)
1. [Optional: Add resource quotas.](#optional-add-resource-quotas)
### 1. Name a New Project
### Name a New Project
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster you want to project in and click **Explore**.
@@ -139,17 +138,7 @@ This section describes how to create a new project with a name and with optional
1. Click **Create Project**.
1. Enter a **Project Name**.
### 2. Optional: Select a Pod Security Policy
This option is only available if you've already created a Pod Security Policy. For instruction, see [Creating Pod Security Policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md).
Assigning a PSP to a project will:
- Override the cluster's default PSP.
- Apply the PSP to the project.
- Apply the PSP to any namespaces you add to the project later.
### 3. Recommended: Add Project Members
### Recommended: Add Project Members
Use the **Members** section to provide other users with project access and roles.
@@ -179,7 +168,7 @@ To add members:
:::
1. In the **Project Permissions** section, choose a role. For more information, refer to the [documentation on project roles.](../authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md)
### 4. Optional: Add Resource Quotas
### Optional: Add Resource Quotas
Resource quotas limit the resources that a project (and its namespaces) can consume. For more information, see [Resource Quotas](../../advanced-user-guides/manage-projects/manage-project-resource-quotas/manage-project-resource-quotas.md).
@@ -22,6 +22,14 @@ For private nodes or private clusters, the environment variables need to be set
When adding Fleet agent environment variables for the proxy, replace <PROXY_IP> with your private proxy IP.
:::caution
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable in Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
| Variable Name | Value |
|------------------|--------|
| `HTTP_PROXY` | http://<PROXY_IP>:8888 |
@@ -30,10 +30,6 @@ If you would like to limit Prometheus to specific namespaces, set `prometheus.pr
For details, refer to [this section.](selectors-and-scrape-configurations.md)
### Enable Istio with Pod Security Policies
Refer to [this section.](pod-security-policies.md)
### Additional Steps for Installing Istio on an RKE2 Cluster
Refer to [this section.](install-istio-on-rke2-cluster.md)
@@ -1,57 +0,0 @@
---
title: Enable Istio with Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/pod-security-policies"/>
</head>
If you have restrictive Pod Security Policies enabled, then Istio may not be able to function correctly, because it needs certain permissions in order to install itself and manage pod infrastructure. In this section, we will configure a cluster with PSPs enabled for an Istio install, and also set up the Istio CNI plugin.
The Istio CNI plugin removes the need for each application pod to have a privileged `NET_ADMIN` container. For further information, see the [Istio CNI Plugin docs](https://istio.io/docs/setup/additional-setup/cni). Please note that the [Istio CNI Plugin is in alpha](https://istio.io/about/feature-stages/).
:::note Prerequisites:
- The cluster must be an RKE Kubernetes cluster.
- The cluster must have been created with a default PodSecurityPolicy.
To enable pod security policy support when creating a Kubernetes cluster in the Rancher UI, go to <b>Advanced Options.</b> In the <b>Pod Security Policy Support</b> section, click <b>Enabled.</b> Then select a default pod security policy.
:::
1. [Set the PodSecurityPolicy to unrestricted](#1-set-the-podsecuritypolicy-to-unrestricted)
2. [Enable the CNI](#2-enable-the-cni)
3. [Verify that the CNI is working.](#3-verify-that-the-cni-is-working)
### 1. Set the PodSecurityPolicy to unrestricted
An unrestricted PSP allows Istio to be installed.
Set the PSP to `unrestricted` in the project where is Istio is installed, or the project where you plan to install Istio.
1. Click **☰ > Cluster Management**.
1. Go to the cluster that you created and click **Explore**.
1. Click **Cluster > Projects/Namespaces**.
1. Find the **Project: System** and select the **⋮ > Edit Config**.
1. Change the Pod Security Policy option to be unrestricted, then click **Save**.
### 2. Enable the CNI
When installing or upgrading Istio through **Apps,**
1. Click **Components**.
2. Check the box next to **Enabled CNI**.
3. Finish installing or upgrading Istio.
The CNI can also be enabled by editing the `values.yaml`:
```
istio_cni.enabled: true
```
Istio should install successfully with the CNI enabled in the cluster.
### 3. Verify that the CNI is working
Verify that the CNI is working by deploying a [sample application](https://istio.io/latest/docs/examples/bookinfo/) or deploying one of your own applications.
@@ -151,10 +151,6 @@ Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
### Pod Security Policy Support
Enables [pod security policies](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. After enabling this option, choose a policy using the **Default Pod Security Policy** drop-down.
You must have an existing Pod Security Policy configured before you can use this option.
### Docker Version on Nodes
@@ -129,10 +129,6 @@ If the cloud provider you want to use is not listed as an option, you will need
:::
##### Default Pod Security Policy
The default [pod security policy](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. Please refer to the [RKE2 documentation](https://docs.rke2.io/security/pod_security_policies) on the specifications of each available policy.
##### Pod Security Admission Configuration Template
The default [pod security admission configuration template](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for the cluster.
@@ -26,27 +26,21 @@ Each self-assessment guide is accompanied by a hardening guide. These guides wer
| Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|--------------------|-----------------------|-----------------------|------------------|
| Kubernetes v1.23 | CIS v1.23 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
| Kubernetes v1.24 | CIS v1.24 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
| Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
### RKE2 Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|------|--------------------|-----------------------|-----------------------|------------------|
| Rancher provisioned RKE2 | Kubernetes v1.23 | CIS v1.23 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Rancher provisioned RKE2 | Kubernetes v1.24 | CIS v1.24 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Rancher provisioned RKE2 | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Standalone RKE2 | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](https://docs.rke2.io/security/cis_self_assessment123) | [Link](https://docs.rke2.io/security/hardening_guide) |
| Standalone RKE2 | Kubernetes v1.27-v1.32 | CIS v1.9 | [Link](https://docs.rke2.io/security/cis_self_assessment19) | [Link](https://docs.rke2.io/security/hardening_guide) |
### K3s Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|------|--------------------|-----------------------|-----------------------|------------------|
| Rancher provisioned K3s cluster | Kubernetes v1.23 | CIS v1.23 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Rancher provisioned K3s cluster | Kubernetes v1.24 | CIS v1.24 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Rancher provisioned K3s cluster | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Standalone K3s | Kubernetes v1.22 up to v1.24 | CIS v1.23 | [Link](https://docs.k3s.io/security/self-assessment-1.8) | [Link](https://docs.k3s.io/security/hardening-guide) |
| Standalone K3s | Kubernetes v1.26 up to v1.29 | CIS v1.8 | [Link](https://docs.k3s.io/security/self-assessment-1.8) | [Link](https://docs.k3s.io/security/hardening-guide) |
## Rancher with SELinux
@@ -79,7 +79,7 @@ This configuration needs to be done before setting the kubelet flag, otherwise K
## Kubernetes Runtime Requirements
The CIS Benchmark runtime requirements center around pod security (via PSP or PSA), network policies and API Server auditing logs.
The CIS Benchmark runtime requirements center around pod security (via PSA), network policies and API Server auditing logs.
By default, K3s does not include any pod security or network policies. However, K3s ships with a controller that enforces any network policies you create. By default, K3s enables both the `PodSecurity` and `NodeRestriction` admission controllers, among others.
@@ -153,7 +153,6 @@ Execute this script to apply the `default-allow-all.yaml` configuration with the
## Known Limitations
- Rancher **exec shell** and **view logs** for pods are **not** functional in a hardened setup when only a public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
- When setting `default_pod_security_policy_template_id:` to `restricted` or `restricted-noroot`, based on the pod security policies (PSP) [provided](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) by Rancher, Rancher creates `RoleBindings` and `ClusterRoleBindings` on the `default` service accounts. The CIS check 5.1.5 requires that the `default` service accounts have no roles or cluster roles bound to it apart from the defaults. In addition, the `default` service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
## Reference Hardened RKE `cluster.yml` Configuration
@@ -10,12 +10,11 @@ If you operate Rancher behind a proxy and you want to access services through th
Make sure `NO_PROXY` contains the network addresses, network address ranges and domains that should be excluded from using the proxy.
| Environment variable | Purpose |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s) |
| Environment variable | Purpose |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s). <br/><br/> The value must be a comma-delimited string which contains IP addresses, CIDR notation, domain names, or special DNS labels (*). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) |
:::note Important:
NO_PROXY must be in uppercase to use network range (CIDR) notation.
@@ -62,4 +61,4 @@ acl SSL_ports port 2376
acl Safe_ports port 22 # ssh
acl Safe_ports port 2376 # docker port
```
```
@@ -12,7 +12,6 @@
| [Ability to rotate certificates](../how-to-guides/new-user-guides/manage-clusters/rotate-certificates.md) | ✓ | ✓ | | |
| Ability to [backup](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher-launched-kubernetes-clusters.md) and [restore](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-rancher-launched-kubernetes-clusters-from-backup.md) Rancher-launched clusters | ✓ | ✓ | | ✓<sup>4</sup> |
| [Cleaning Kubernetes components when clusters are no longer reachable from Rancher](../how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md) | ✓ | | | |
| [Configuring Pod Security Policies](../how-to-guides/new-user-guides/manage-clusters/add-a-pod-security-policy.md) | ✓ | ✓ | ||
1. Registered EKS, GKE and AKS clusters have the same options available as EKS, GKE and AKS clusters created from the Rancher UI. The difference is that when a registered cluster is deleted from the Rancher UI, it is not destroyed.
@@ -199,7 +199,6 @@ Because `rancher` is the default option for `ingress.tls.source`, we are not spe
- Set the `hostname` to the DNS name you pointed at your load balancer.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.7.0`
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -240,7 +239,6 @@ In the following command,
- `ingress.tls.source` is set to `letsEncrypt`
- `letsEncrypt.email` is set to the email address used for communication about your certificate (for example, expiry notices)
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
:::warning
@@ -289,7 +287,6 @@ If you want to check if your certificates are correct, see [How do I check Commo
- Set the `hostname`.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- Set `ingress.tls.source` to `secret`.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -1,72 +0,0 @@
---
title: Upgrading a Hardened Custom/Imported Cluster to Kubernetes v1.25
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrade-a-hardened-cluster-to-k8s-v1-25"/>
</head>
Kubernetes v1.25 changes how clusters describe and implement security policies. From this version forward, [Pod Security Policies (PSPs)](https://kubernetes.io/docs/concepts/security/pod-security-policy/) are no longer available. Kubernetes v1.25 replaces them with new security objects: [Pod Security Standards (PSS)](https://kubernetes.io/docs/concepts/security/pod-security-standards/), and [Pod Security Admissions (PSAs)](https://kubernetes.io/docs/concepts/security/pod-security-admission/).
If you have custom or imported hardened clusters, you must take special preparations to ensure that the upgrade from an earlier version of Kubernetes to v1.25 or later goes smoothly.
:::note
After you upgrade to v1.25, add the necessary Rancher namespace exemptions. See [Pod Security Admission (PSA) Configuration Templates](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md#exempting-required-rancher-namespaces) for more details.
:::
## Upgrading Imported Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
Perform the following on each node in the cluster:
1. Save [`rancher-psact.yaml`](./rancher-psact.yaml) in `/etc/rancher/rke2`.
1. Edit the RKE2 configuration file:
1. Update the `profile` field to `cis-1.23`.
1. Specify the path for the configuration file that you just added: `pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml`.
</TabItem>
<TabItem value="K3s">
Perform the following on each node in the cluster:
Follow the official K3s instructions on [Upgrading Hardened Clusters from v1.24.x to v1.25.x](https://docs.k3s.io/known-issues#hardened-125), but use a [custom](./rancher-psact.yaml) Rancher PSA configuration template, instead of the configuration provided on the official K3s site.
</TabItem>
</Tabs>
After you perform these steps, you can upgrade the cluster's Kubernetes version through the Rancher UI:
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
## Upgrading Custom Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. Under **Basics > Security**, in the **CIS Profile** dropdown menu, select `cis-1.23`.
1. In the **Pod Security Admission Configuration Template** dropdown menu, select `rancher-restricted`.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
</TabItem>
<TabItem value="K3s">
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit YAML**.
1. Delete `PodSecurityPolicy` from `kube-apiserver-arg.enable-admission-plugins`
1. Add this line to the `spec` field: `defaultPodSecurityAdmissionConfigurationTemplateName: rancher-restricted`
1. Update `kubernetesVersion` to your chosen version (v1.25 or later).
1. Click **Save**.
</TabItem>
</Tabs>
@@ -152,7 +152,6 @@ Upgrade Rancher to the latest version with all your settings.
Take all the values from the previous step and append them to the command using `--set key=value`.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
@@ -186,8 +185,7 @@ Alternatively, it's possible to export the current values to a file and referenc
```
1. Update only the Rancher version:
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
@@ -26,7 +26,30 @@ The following is a list of feature flags available in Rancher. If you've upgrade
- `imperative-api-extension`: Enables Rancher's [extension API server](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) to register new APIs to Kubernetes. This flag is enabled by default. See the [Extension API Server](../../../api/extension-apiserver.md) page for more information.
- `istio-virtual-service-ui`: Enables a [visual interface](../../../how-to-guides/advanced-user-guides/enable-experimental-features/istio-traffic-management-features.md) to create, read, update, and delete Istio virtual services and destination rules, which are Istio traffic management features.
- `legacy`: Enables a set of features from 2.5.x and earlier, that are slowly being phased out in favor of newer implementations. These are a mix of deprecated features as well as features that will eventually be available to newer versions. This flag is disabled by default on new Rancher installations. If you're upgrading from a previous version of Rancher, this flag is enabled.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream RKE2/K3s clusters, currently limited to imported clusters and the local cluster, with plans to expand support to node-driver clusters.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream imported RKE2/K3s clusters, as well as in the local cluster if it is an RKE2/K3s cluster.
:::note Important:
This `managed-system-upgrade-controller` flag is intended for **internal use only** and does not have an associated Feature CR. Use with caution.
To control whether Rancher should manage the Kubernetes version of imported RKE2/K3s clusters, it is recommended to use the [imported-cluster-version-management](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters) feature that is available in Rancher v2.11.0 or newer.
:::
:::danger
If the `managed-system-upgrade-controller` flag was **disabled** in Rancher v2.10.x, and any imported RKE2/K3s clusters were upgraded **outside of Rancher**, follow the steps below to prevent the unexpected installation of the system-upgrade-controller app and to ensure the [imported-cluster-version-management](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters) feature works correctly:
1. Upgrade Rancher to v2.11.0 or newer, making sure to **retain** the `managed-system-upgrade-controller=false` feature flag in Helm values if it was set during the v2.10.x installation.
1. After Rancher is fully up and running, disable the `imported-cluster-version-management` setting. You can do this either through the Rancher UI by clicking **☰ > Global Settings > Settings > imported-cluster-version-management**, or by editing the corresponding `Setting.management.cattle.io/v3` custom resource via kubectl.
1. Perform a second Helm upgrade, this time omitting the `managed-system-upgrade-controller=false` feature flag.
Now, the imported cluster version management is disabled by default, and Rancher no longer installs the system-upgrade-controller app on imported clusters automatically.
You can enable this feature on a per-cluster basis. For more information, please refer to the [documentation](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters).
:::
- `multi-cluster-management`: Allows multi-cluster provisioning and management of Kubernetes clusters. This flag can only be set at install time. It can't be enabled or disabled later.
- `rke1-custom-node-cleanup`: Enables cleanup of deleted RKE1 custom nodes. We recommend that you keep this flag enabled, to prevent removed nodes from attempting to rejoin the cluster.
- `rke2`: Enables provisioning RKE2 clusters. This flag is enabled by default.
@@ -62,8 +62,6 @@ For information on enabling experimental features, refer to [this page.](../../.
| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system container images, e.g., http://registry.example.com/ |
| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" |
| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. |
| `global.cattle.psp.enabled` | `true` | `bool` - select 'false' to disable PSPs for Kubernetes v1.25 and above when using Rancher v2.7.2-v2.7.4. When using Rancher v2.7.5 and above, Rancher attempts to detect if a cluster is running a Kubernetes version where PSPs are not supported, and will default it's usage of PSPs to false if it can determine that PSPs are not supported in the cluster. Users can still manually override this by explicitly providing `true` or `false` for this value. Rancher will still use PSPs by default in clusters which support PSPs (such as clusters running Kubernetes v1.24 or lower). |
### Bootstrap Password
@@ -172,7 +172,6 @@ kubectl create namespace cattle-system
Next, install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
Placeholder | Description
------------|-------------
@@ -203,7 +202,6 @@ Create Kubernetes secrets from your own certificates for Rancher to use. The com
Install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------- |
@@ -12,6 +12,14 @@ The steps to set up RKE, RKE2, or K3s are shown below.
For convenience, export the IP address and port of your proxy into an environment variable and set up the HTTP_PROXY variables for your current shell on every node:
:::caution
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable for Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
```
export proxy_host="10.0.0.5:8888"
export HTTP_PROXY=http://${proxy_host}
@@ -53,6 +53,10 @@ You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by loggi
## Upgrade
:::danger
Rancher upgrades to version 2.12.0 and later will be blocked if any RKE1-related resources are detected, as the Rancher Kubernetes Engine (RKE/RKE1) is end of life as of **July 31, 2025**. For detailed cleanup and recovery steps, refer to the [RKE1 Resource Validation and Upgrade Requirements in Rancher v2.12](#rke1-resource-validation-and-upgrade-requirements-in-rancher-v212).
:::
During upgrade, you create a copy of the data from your current Rancher container and a backup in case something goes wrong. Then you deploy the new version of Rancher in a new container using your existing data.
### 1. Create a copy of the data from your Rancher server container
@@ -388,6 +392,45 @@ See [Restoring Cluster Networking](https://github.com/rancher/rancher-docs/tree/
Remove the previous Rancher server container. If you only stop the previous Rancher server container (and don't remove it), the container may restart after the next server reboot.
## RKE1 Resource Validation and Upgrade Requirements in Rancher v2.12
Rancher v2.12.0 and later has removed support for the Rancher Kubernetes Engine (RKE/RKE1). During upgrade, Rancher validates the cluster resources and blocks the upgrade if any RKE1-related resources are detected.
This validation affects the following resource types:
- Clusters with `rkeConfig` (`clusters.management.cattle.io`)
- NodeTemplates (`nodetemplates.management.cattle.io`)
- ClusterTemplates (`clustertemplates.management.cattle.io`)
This is particularly relevant for single-node Docker installations, where Rancher is not running during the upgrade. In such cases, controllers are not available to automatically clean up deprecated resources, and the upgrade process will fail early with an error listing the blocking resources.
### 1. Pre-Upgrade (Recommended)
Before upgrading, while Rancher is still running:
- Run the `pre-upgrade-hook` cleanup script to delete all RKE1 clusters and templates. You can find the script in the Rancher GitHub repository: [pre-upgrade-hook.sh](https://github.com/rancher/rancher/blob/v2.12.0/chart/scripts/pre-upgrade-hook.sh).
- This allows Rancher to clean up associated resources and finalizers.
### 2. Post-Upgrade Failure Due to Residual RKE1 Resources
If the upgrade to Rancher v2.12.0 or later is attempted without prior cleanup of RKE1 resources:
- The upgrade will fail and display an error listing the resource names that are preventing the upgrade.
- This occurs because Rancher includes validation to detect and block upgrades when unsupported RKE1 resources are still present.
- To proceed, [rollback](#rolling-back) to the previous Rancher version, delete the identified resources, and then retry after [manual cleanup](#manual-cleanup-after-rollback).
:::note Helm-based Rancher
Helm-based Rancher installations are not affected by this issue, as Rancher remains available during the upgrade and can perform resource cleanup as needed.
:::
### Manual Cleanup After Rollback
Users should perform the following steps after rolling back to a previous Rancher version:
- **Manually delete** the resources listed in the upgrade error message (e.g., RKE1 clusters, NodeTemplates, ClusterTemplates).
- If deletion is blocked due to **finalizers**, edit the resources and remove the `metadata.finalizers` field.
- If a **validating webhook** prevents deletion (e.g., for the `system-project`), please refer to the [Bypassing the Webhook](../../../../reference-guides/rancher-webhook.md#bypassing-the-webhook) documentation.
## Rolling Back
If your upgrade does not complete successfully, you can roll back Rancher server and its data back to its last healthy state. For more information, see [Docker Rollback](roll-back-docker-installed-rancher.md).
@@ -35,7 +35,7 @@ The Rancher API server is built on top of an embedded Kubernetes API server and
### Authorization and Role-Based Access Control
- **User management:** The Rancher API server [manages user identities](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md) that correspond to external authentication providers like Active Directory or GitHub, in addition to local users.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) policies.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) standards.
### Working with Kubernetes
@@ -128,7 +128,6 @@ The final command to install Rancher is below. The command requires a domain nam
To install a specific Rancher version, use the `--version` flag (e.g., `--version 2.6.6`). Otherwise, the latest Rancher is installed by default. Refer to [Choosing a Rancher Version](../../installation-and-upgrade/resources/choose-a-rancher-version.md).
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
See [Setting up the Bootstrap Password](../../installation-and-upgrade/resources/bootstrap-password.md#password-requirements) for password requirements.
@@ -13,9 +13,3 @@ title: Install Rancher CIS Benchmark
1. Click **Install**.
**Result:** The CIS scan application is deployed on the Kubernetes cluster.
:::note
If you are running Kubernetes v1.24 or earlier, and have a [Pod Security Policy](../../new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) (PSP) hardened cluster, CIS Benchmark 4.0.0 and later disable PSPs by default. To install CIS Benchmark on a PSP-hardened cluster, set `global.psp.enabled` to `true` in the values before installing the chart. [Pod Security Admission](../../new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) (PSA) hardened clusters aren't affected.
:::
@@ -39,7 +39,6 @@ Values set from the Rancher API will override the value passed in through the co
When installing Rancher with a Helm chart, use the `--set` option. In the below example, two features are enabled by passing the feature flag names in a comma separated list:
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-latest/rancher \
@@ -17,7 +17,6 @@ Detailed information can be found in [this announcement](https://forums.suse.com
:::note Prerequisites:
- Only a user with the `cluster-admin` [Kubernetes default role](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) assigned can configure and install Istio in a Kubernetes cluster.
- If you have pod security policies, you will need to install Istio with the CNI enabled. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/pod-security-policies.md)
- To install Istio on an RKE2 cluster, additional steps are required. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/install-istio-on-rke2-cluster.md)
- To install Istio in a cluster where project network isolation is enabled, additional steps are required. For details, see [this section.](../../../integrations-in-rancher/istio/configuration-options/project-network-isolation.md)
@@ -1,43 +0,0 @@
---
title: Applying Pod Security Policies to Projects
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/manage-projects/manage-pod-security-policies"/>
</head>
:::note
These cluster options are only available for [clusters in which Rancher has launched Kubernetes](../../new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
:::
You can always assign a pod security policy (PSP) to an existing project if you didn't assign one during creation.
### Prerequisites
- Create a Pod Security Policy within Rancher. Before you can assign a default PSP to an existing project, you must have a PSP available for assignment. For instruction, see [Creating Pod Security Policies](../../new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md).
- Assign a default Pod Security Policy to the project's cluster. You can't assign a PSP to a project until one is already applied to the cluster. For more information, see [the documentation about adding a pod security policy to a cluster](../../new-user-guides/manage-clusters/add-a-pod-security-policy.md).
### Applying a Pod Security Policy
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to move a namespace and click **Explore**.
1. Click **Cluster > Projects/Namespaces**.
1. Find the project that you want to add a PSP to. From that project, select **⋮ > Edit Config**.
1. From the **Pod Security Policy** drop-down, select the PSP you want to apply to the project.
Assigning a PSP to a project will:
- Override the cluster's default PSP.
- Apply the PSP to the project.
- Apply the PSP to any namespaces you add to the project later.
1. Click **Save**.
**Result:** The PSP is applied to the project and any namespaces added to the project.
:::note
Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked to determine if they comply with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP.
:::
@@ -24,7 +24,6 @@ You can use projects to perform actions like:
- [Set resource quotas](manage-project-resource-quotas/manage-project-resource-quotas.md)
- [Manage namespaces](../../new-user-guides/manage-namespaces.md)
- [Configure tools](../../../reference-guides/rancher-project-tools.md)
- [Configure pod security policies](manage-pod-security-policies.md)
### Authorization
@@ -38,12 +38,6 @@ The user retention feature is disabled by default.
For more information, see [Enabling User Retention](../../advanced-user-guides/enable-user-retention.md).
## Pod Security Policies
_Pod Security Policies_ (or PSPs) are objects that control security-sensitive aspects of pod specification, e.g. root privileges. If a pod does not meet the conditions specified in the PSP, Kubernetes will not allow it to start, and Rancher will display an error message.
For more information how to create and use PSPs, see [Pod Security Policies](create-pod-security-policies.md).
## Provisioning Drivers
Drivers in Rancher allow you to manage which providers can be used to provision [hosted Kubernetes clusters](../kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/set-up-clusters-from-hosted-kubernetes-providers.md) or [nodes in an infrastructure provider](../launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md) to allow Rancher to deploy and manage Kubernetes.
@@ -1,82 +0,0 @@
---
title: Creating Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies"/>
</head>
:::caution
Pod Security Policies are only available in Kubernetes until v1.24. [Pod Security Standards](pod-security-standards.md) are the built-in alternative.
:::
[Pod Security Policies (PSPs)](https://kubernetes.io/docs/concepts/security/pod-security-policy/) are objects that control security-sensitive aspects of the pod specification (such as root privileges).
If a pod doesn't meet the conditions specified in the PSP, Kubernetes won't allow it to start, and Rancher will display the following error message: `Pod <NAME> is forbidden: unable to validate...`.
## How PSPs Work
You can assign PSPs at the cluster or project level.
PSPs work through inheritance:
- By default, PSPs assigned to a cluster are inherited by its projects, as well as any namespaces added to those projects.
- **Exception:** Namespaces that are not assigned to projects do not inherit PSPs, regardless of whether the PSP is assigned to a cluster or project. Because these namespaces have no PSPs, workload deployments to these namespaces will fail, which is the default Kubernetes behavior.
- You can override the default PSP by assigning a different PSP directly to the project.
Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked if it complies with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP.
Read more about Pod Security Policies in the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).
## Default PSPs
Rancher ships with three default Pod Security Policies (PSPs): the `restricted-noroot`, `restricted` and `unrestricted` policies.
### Restricted-NoRoot
This policy is based on the Kubernetes [example restricted policy](https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/policy/restricted-psp.yaml). It significantly restricts what types of pods can be deployed to a cluster or project. This policy:
- Prevents pods from running as a privileged user and prevents escalation of privileges.
- Validates that server-required security mechanisms are in place, such as restricting what volumes can be mounted to only the core volume types and preventing root supplemental groups from being added.
### Restricted
This policy is a relaxed version of the `restricted-noroot` policy, with almost all the restrictions in place, except for the fact that it allows running containers as a privileged user.
### Unrestricted
This policy is equivalent to running Kubernetes with the PSP controller disabled. It has no restrictions on what pods can be deployed into a cluster or project.
:::note important
When disabling PSPs, default PSPs are **not** automatically deleted from your cluster. You must manually delete them if they're no longer needed.
:::
## Creating PSPs
Using Rancher, you can create a Pod Security Policy using our GUI rather than creating a YAML file.
### Requirements
Rancher can only assign PSPs for clusters that are [launched using RKE](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
You must enable PSPs at the cluster level before you can assign them to a project. This can be configured by [editing the cluster](../../../reference-guides/cluster-configuration/cluster-configuration.md).
It is a best practice to set PSP at the cluster level.
We recommend adding PSPs during cluster and project creation instead of adding it to an existing one.
### Creating PSPs in the Rancher UI
1. In the upper left corner, click **☰ > Cluster Management**.
1. In the left navigation bar, click **Pod Security Policies**.
1. Click **Add Policy**.
1. Name the policy.
1. Complete each section of the form. Refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) for more information on what each policy does.
1. Click **Create**.
## Configuration
The Kubernetes documentation on PSPs is [here](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).
@@ -11,54 +11,6 @@ They became available and were turned on by default in Kubernetes v1.23, and rep
PSS define security levels for workloads. PSAs describe requirements for pod security contexts and related fields. PSAs reference PSS levels to define security restrictions.
## Upgrade to Pod Security Standards (PSS)
Ensure that you migrate all PSPs to another workload security mechanism. This includes mapping your current PSPs to Pod Security Standards for enforcement with the [PSA controller](https://kubernetes.io/docs/concepts/security/pod-security-admission/). If the PSA controller won't meet all of your organization's needs, we recommend that you use a policy engine, such as [Kubewarden](https://www.kubewarden.io/), [Kyverno](https://kyverno.io/), or [NeuVector](https://neuvector.com/). Refer to the documentation of your policy engine of choice for more information on how to migrate from PSPs.
:::caution
You must add your new policy enforcement mechanisms _before_ you remove the PodSecurityPolicy objects. If you don't, you may create an opportunity for privilege escalation attacks within the cluster.
:::
### Removing PodSecurityPolicies from Rancher-Maintained Apps & Marketplace Workloads
Rancher v2.7.2 offers a new major version of Rancher-maintained Helm charts. v102.x.y allows you to remove PSPs that were installed with previous versions of the chart. This new version replaces non-standard PSPs switches with the standardized `global.cattle.psp.enabled` switch, which is turned off by default.
You must perform the following steps _while still in Kubernetes v1.24_:
1. Configure the PSA controller to suit your needs. You can use one of Rancher's built-in [PSA Configuration Templates](#pod-security-admission-configuration-templates), or create a custom template and apply it to the clusters that you are migrating.
1. Map your active PSPs to Pod Security Standards:
1. See which PSPs are still active in your cluster:
:::caution
This strategy may miss workloads that aren't currently running, such as CronJobs, workloads currently scaled to zero, or workloads that haven't rolled out yet.
:::
```shell
kubectl get pods \
--all-namespaces \
--output jsonpath='{.items[*].metadata.annotations.kubernetes\.io\/psp}' \
| tr " " "\n" | sort -u
```
1. Follow the Kubernetes guide on [Mapping PSPs to Pod Security Standards](https://kubernetes.io/docs/reference/access-authn-authz/psp-to-pod-security-standards/) to apply PSSs to your workloads that were relying on PSPs. See [Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission controller](https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/) for more details.
1. To remove PSPs from Rancher charts, upgrade the charts to the latest v102.x.y version _before_ you upgrade to Kubernetes v1.25. Make sure that the **Enable PodSecurityPolicies** option is **disabled**. This will remove any PSPs that were installed with previous chart versions.
:::info important
If you want to upgrade your charts to v102.x.y, but don't plan on upgrading your clusters to Kubernetes v1.25 and moving away from PSPs, make sure that you select the option **Enable PodSecurityPolicies** for each chart that you are upgrading.
:::
### Cleaning Up Releases After a Kubernetes v1.25 Upgrade
If you experience problems while removing PSPs from your charts, or have charts that don't contain a built-in mechanism for removing PSPs, your chart upgrades or deletions might fail with an error message such as the following:
```console
Error: UPGRADE FAILED: resource mapping not found for name: "<object-name>" namespace: "<object-namespace>" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
ensure CRDs are installed first
```
This happens when Helm tries to query the cluster for objects that were stored in a previous release's data blob. To clean up these releases and avoid this error, use the `helm-mapkubeapis` Helm plugin. To learn more about `helm-mapkubeapis`, how it works, and how it can be fine-tuned for your use case, see the [official Helm documentation](https://github.com/helm/helm-mapkubeapis#readme).
Note that Helm plugin installation is local to the machine that you run the commands from. Therefore, make sure that you run both the installation and cleanup from the same machine.
#### Install `helm-mapkubeapis`
1. Open your terminal in the machine you intend to use `helm-mapkubeapis` from and install the plugin:
@@ -110,15 +62,6 @@ After you install the `helm-mapkubeapis` plugin, clean up the releases that beca
1. Finally, after reviewing the changes, perform a full run with `helm mapkubeapis <release-name> --namespace <release-namespace>`.
#### Upgrading Charts to a Version That Supports Kubernetes v1.25
You can proceed with your upgrade once any releases that had lingering PSPs are cleaned up. For Rancher-maintained workloads, follow the steps outlined in the [Removing PodSecurityPolicies from Rancher-maintained Apps & Marketplace workloads](#removing-podsecuritypolicies-from-rancher-maintained-apps--marketplace-workloads) section of this document.
For workloads not maintained by Rancher, refer to the vendor documentation.
:::caution
Do not skip this step. Applications incompatible with Kubernetes v1.25 aren't guaranteed to work after a cleanup.
:::
## Pod Security Admission Configuration Templates
Rancher offers PSA configuration templates. These are pre-defined security configurations that you can apply to a cluster. Rancher admins (or those with the right permissions) can [create, manage, and edit](./psa-config-templates.md) PSA templates.
@@ -168,7 +168,6 @@ Follow the steps to [install cert-manager](../../../getting-started/installation
Use the same version of Helm to install Rancher, that was used on the first cluster.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```bash
helm install rancher rancher-latest/rancher \
@@ -57,6 +57,14 @@ GKE Autopilot clusters aren't supported. See [Compare GKE Autopilot and Standard
9. If you are using self-signed certificates, you will receive the message `certificate signed by unknown authority`. To work around this validation, copy the command starting with `curl` displayed in Rancher to your clipboard. Then run the command on a node where kubeconfig is configured to point to the cluster you want to import.
10. When you finish running the command(s) on your node, click **Done**.
:::important
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable in Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
**Result:**
- Your cluster is registered and assigned a state of **Pending**. Rancher is deploying resources to manage your cluster.
@@ -162,6 +170,13 @@ You can define the default behavior for newly created clusters or existing ones
Changes to the global **imported-cluster-version-management** setting take effect during the cluster’s next reconciliation cycle.
:::note
If version management is enabled for a cluster, Rancher will deploy the system-upgrade-controller app, along with the associated Plans and other required Kubernetes resources, to the cluster.
If version management is disabled, Rancher will remove these components from the cluster.
:::
## Configuring RKE2 and K3s Cluster Upgrades
:::tip
@@ -269,15 +284,7 @@ Therefore, when Rancher registers a cluster, it assumes that several capabilitie
However, if the cluster has a certain capability, such as the ability to use a pod security policy, a user of that cluster might still want to select pod security policies for the cluster in the Rancher UI. In order to do that, the user will need to manually indicate to Rancher that pod security policies are enabled for the cluster.
By annotating a registered cluster, it is possible to indicate to Rancher that a cluster was given a pod security policy, or another capability, outside of Rancher.
This example annotation indicates that a pod security policy is enabled:
```json
"capabilities.cattle.io/pspEnabled": "true"
```
The following annotation indicates Ingress capabilities. Note that the values of non-primitive objects need to be JSON encoded, with quotations escaped.
By annotating a registered cluster, it is possible to indicate to Rancher that a cluster was given Ingress capabilities, or another capability, outside of Rancher. The following annotation indicates Ingress capabilities. Note that the values of non-primitive objects need to be JSON encoded, with quotations escaped.
```json
"capabilities.cattle.io/ingressCapabilities": "[
@@ -294,7 +301,6 @@ These capabilities can be annotated for the cluster:
- `loadBalancerCapabilities`
- `nodePoolScalingSupported`
- `nodePortRange`
- `pspEnabled`
- `taintSupport`
All the capabilities and their type definitions can be viewed in the Rancher API view, at `[Rancher Server URL]/v3/schemas/capabilities`.
@@ -1,43 +0,0 @@
---
title: Adding a Pod Security Policy
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/add-a-pod-security-policy"/>
</head>
:::note Prerequisite:
The options below are available only for clusters that are [launched using RKE.](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md)
:::
When your cluster is running pods with security-sensitive configurations, assign it a [pod security policy](../authentication-permissions-and-global-configuration/create-pod-security-policies.md), which is a set of rules that monitors the conditions and settings in your pods. If a pod doesn't meet the rules specified in your policy, the policy stops it from running.
You can assign a pod security policy when you provision a cluster. However, if you need to relax or restrict security for your pods later, you can update the policy while editing your cluster.
1. Click **☰ > Cluster Management**.
1. Go to the cluster to which you want to apply a pod security policy and click **⋮ > Edit Config**.
1. From **Pod Security Policy Support**, select **Enabled**.
:::note
This option is only available for clusters [provisioned by RKE](../launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
:::
4. From the **Default Pod Security Policy** drop-down, select the policy you want to apply to the cluster.
Rancher ships with [policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md#default-psps) of `restricted` and `unrestricted`, although you can [create custom policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md#creating-psps) as well.
5. Click **Save**.
**Result:** The pod security policy is applied to the cluster and any projects within the cluster.
:::note
Workloads already running before assignment of a pod security policy are grandfathered in. Even if they don't meet your pod security policy, workloads running before assignment of the policy continue to run.
To check if a running workload passes your pod security policy, clone or upgrade it.
:::
@@ -1,29 +0,0 @@
---
title: Assigning Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/assign-pod-security-policies"/>
</head>
_Pod Security Policies_ are objects that control security-sensitive aspects of pod specification (like root privileges).
## Adding a Default Pod Security Policy
When you create a new cluster with RKE, you can configure it to apply a PSP immediately. As you create the cluster, use the **Cluster Options** to enable a PSP. The PSP assigned to the cluster will be the default PSP for projects within the cluster.
:::note Prerequisite:
Create a Pod Security Policy within Rancher. Before you can assign a default PSP to a new cluster, you must have a PSP available for assignment. For instruction, see [Creating Pod Security Policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md).
:::
:::note
For security purposes, we recommend assigning a PSP as you create your clusters.
:::
To enable a default Pod Security Policy, set the **Pod Security Policy Support** option to **Enabled**, and then make a selection from the **Default Pod Security Policy** drop-down.
When the cluster finishes provisioning, the PSP you selected is applied to all projects within the cluster.
@@ -126,12 +126,11 @@ Rancher extends Kubernetes to allow the application of [Pod Security Policies](h
This section describes how to create a new project with a name and with optional pod security policy, members, and resource quotas.
1. [Name a new project.](#1-name-a-new-project)
2. [Optional: Select a pod security policy.](#2-optional-select-a-pod-security-policy)
3. [Recommended: Add project members.](#3-recommended-add-project-members)
4. [Optional: Add resource quotas.](#4-optional-add-resource-quotas)
1. [Name a new project.](#name-a-new-project)
1. [Recommended: Add project members.](#recommended-add-project-members)
1. [Optional: Add resource quotas.](#optional-add-resource-quotas)
### 1. Name a New Project
### Name a New Project
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster you want to project in and click **Explore**.
@@ -139,17 +138,7 @@ This section describes how to create a new project with a name and with optional
1. Click **Create Project**.
1. Enter a **Project Name**.
### 2. Optional: Select a Pod Security Policy
This option is only available if you've already created a Pod Security Policy. For instruction, see [Creating Pod Security Policies](../authentication-permissions-and-global-configuration/create-pod-security-policies.md).
Assigning a PSP to a project will:
- Override the cluster's default PSP.
- Apply the PSP to the project.
- Apply the PSP to any namespaces you add to the project later.
### 3. Recommended: Add Project Members
### Recommended: Add Project Members
Use the **Members** section to provide other users with project access and roles.
@@ -179,7 +168,7 @@ To add members:
:::
1. In the **Project Permissions** section, choose a role. For more information, refer to the [documentation on project roles.](../authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md)
### 4. Optional: Add Resource Quotas
### Optional: Add Resource Quotas
Resource quotas limit the resources that a project (and its namespaces) can consume. For more information, see [Resource Quotas](../../advanced-user-guides/manage-projects/manage-project-resource-quotas/manage-project-resource-quotas.md).
@@ -22,6 +22,14 @@ For private nodes or private clusters, the environment variables need to be set
When adding Fleet agent environment variables for the proxy, replace <PROXY_IP> with your private proxy IP.
:::caution
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable in Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
| Variable Name | Value |
|------------------|--------|
| `HTTP_PROXY` | http://<PROXY_IP>:8888 |
@@ -38,10 +38,6 @@ If you would like to limit Prometheus to specific namespaces, set `prometheus.pr
For details, refer to [this section.](selectors-and-scrape-configurations.md)
### Enable Istio with Pod Security Policies
Refer to [this section.](pod-security-policies.md)
### Additional Steps for Installing Istio on an RKE2 Cluster
Refer to [this section.](install-istio-on-rke2-cluster.md)
@@ -1,65 +0,0 @@
---
title: Enable Istio with Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/pod-security-policies"/>
</head>
:::warning
[Rancher-Istio](https://github.com/rancher/charts/tree/release-v2.11/charts/rancher-istio) will be deprecated in Rancher v2.12.0; turn to the [SUSE Rancher Application Collection](https://apps.rancher.io) build of Istio for enhanced security (included in SUSE Rancher Prime subscriptions).
Detailed information can be found in [this announcement](https://forums.suse.com/t/deprecation-of-rancher-istio/45043).
:::
If you have restrictive Pod Security Policies enabled, then Istio may not be able to function correctly, because it needs certain permissions in order to install itself and manage pod infrastructure. In this section, we will configure a cluster with PSPs enabled for an Istio install, and also set up the Istio CNI plugin.
The Istio CNI plugin removes the need for each application pod to have a privileged `NET_ADMIN` container. For further information, see the [Istio CNI Plugin docs](https://istio.io/docs/setup/additional-setup/cni). Please note that the [Istio CNI Plugin is in alpha](https://istio.io/about/feature-stages/).
:::note Prerequisites:
- The cluster must be an RKE Kubernetes cluster.
- The cluster must have been created with a default PodSecurityPolicy.
To enable pod security policy support when creating a Kubernetes cluster in the Rancher UI, go to <b>Advanced Options.</b> In the <b>Pod Security Policy Support</b> section, click <b>Enabled.</b> Then select a default pod security policy.
:::
1. [Set the PodSecurityPolicy to unrestricted](#1-set-the-podsecuritypolicy-to-unrestricted)
2. [Enable the CNI](#2-enable-the-cni)
3. [Verify that the CNI is working.](#3-verify-that-the-cni-is-working)
### 1. Set the PodSecurityPolicy to unrestricted
An unrestricted PSP allows Istio to be installed.
Set the PSP to `unrestricted` in the project where is Istio is installed, or the project where you plan to install Istio.
1. Click **☰ > Cluster Management**.
1. Go to the cluster that you created and click **Explore**.
1. Click **Cluster > Projects/Namespaces**.
1. Find the **Project: System** and select the **⋮ > Edit Config**.
1. Change the Pod Security Policy option to be unrestricted, then click **Save**.
### 2. Enable the CNI
When installing or upgrading Istio through **Apps,**
1. Click **Components**.
2. Check the box next to **Enabled CNI**.
3. Finish installing or upgrading Istio.
The CNI can also be enabled by editing the `values.yaml`:
```
istio_cni.enabled: true
```
Istio should install successfully with the CNI enabled in the cluster.
### 3. Verify that the CNI is working
Verify that the CNI is working by deploying a [sample application](https://istio.io/latest/docs/examples/bookinfo/) or deploying one of your own applications.
@@ -151,10 +151,6 @@ Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
### Pod Security Policy Support
Enables [pod security policies](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. After enabling this option, choose a policy using the **Default Pod Security Policy** drop-down.
You must have an existing Pod Security Policy configured before you can use this option.
### Docker Version on Nodes
@@ -129,10 +129,6 @@ If the cloud provider you want to use is not listed as an option, you will need
:::
##### Default Pod Security Policy
The default [pod security policy](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. Please refer to the [RKE2 documentation](https://docs.rke2.io/security/pod_security_policies) on the specifications of each available policy.
##### Pod Security Admission Configuration Template
The default [pod security admission configuration template](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for the cluster.
@@ -26,27 +26,21 @@ Each self-assessment guide is accompanied by a hardening guide. These guides wer
| Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|--------------------|-----------------------|-----------------------|------------------|
| Kubernetes v1.23 | CIS v1.23 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
| Kubernetes v1.24 | CIS v1.24 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
| Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](rke1-hardening-guide/rke1-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](rke1-hardening-guide/rke1-hardening-guide.md) |
### RKE2 Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|------|--------------------|-----------------------|-----------------------|------------------|
| Rancher provisioned RKE2 | Kubernetes v1.23 | CIS v1.23 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Rancher provisioned RKE2 | Kubernetes v1.24 | CIS v1.24 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Rancher provisioned RKE2 | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](rke2-hardening-guide/rke2-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](rke2-hardening-guide/rke2-hardening-guide.md) |
| Standalone RKE2 | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](https://docs.rke2.io/security/cis_self_assessment123) | [Link](https://docs.rke2.io/security/hardening_guide) |
| Standalone RKE2 | Kubernetes v1.27-v1.32 | CIS v1.9 | [Link](https://docs.rke2.io/security/cis_self_assessment19) | [Link](https://docs.rke2.io/security/hardening_guide) |
### K3s Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides |
|------|--------------------|-----------------------|-----------------------|------------------|
| Rancher provisioned K3s cluster | Kubernetes v1.23 | CIS v1.23 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.23-k8s-v1.23.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Rancher provisioned K3s cluster | Kubernetes v1.24 | CIS v1.24 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.24-k8s-v1.24.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Rancher provisioned K3s cluster | Kubernetes v1.25/v1.26/v1.27 | CIS v1.7 | [Link](k3s-hardening-guide/k3s-self-assessment-guide-with-cis-v1.7-k8s-v1.25-v1.26-v1.27.md) | [Link](k3s-hardening-guide/k3s-hardening-guide.md) |
| Standalone K3s | Kubernetes v1.22 up to v1.24 | CIS v1.23 | [Link](https://docs.k3s.io/security/self-assessment-1.8) | [Link](https://docs.k3s.io/security/hardening-guide) |
| Standalone K3s | Kubernetes v1.26 up to v1.29 | CIS v1.8 | [Link](https://docs.k3s.io/security/self-assessment-1.8) | [Link](https://docs.k3s.io/security/hardening-guide) |
## Rancher with SELinux
@@ -79,7 +79,7 @@ This configuration needs to be done before setting the kubelet flag, otherwise K
## Kubernetes Runtime Requirements
The CIS Benchmark runtime requirements center around pod security (via PSP or PSA), network policies and API Server auditing logs.
The CIS Benchmark runtime requirements center around pod security (via PSA), network policies and API Server auditing logs.
By default, K3s does not include any pod security or network policies. However, K3s ships with a controller that enforces any network policies you create. By default, K3s enables both the `PodSecurity` and `NodeRestriction` admission controllers, among others.
@@ -153,7 +153,6 @@ Execute this script to apply the `default-allow-all.yaml` configuration with the
## Known Limitations
- Rancher **exec shell** and **view logs** for pods are **not** functional in a hardened setup when only a public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
- When setting `default_pod_security_policy_template_id:` to `restricted` or `restricted-noroot`, based on the pod security policies (PSP) [provided](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) by Rancher, Rancher creates `RoleBindings` and `ClusterRoleBindings` on the `default` service accounts. The CIS check 5.1.5 requires that the `default` service accounts have no roles or cluster roles bound to it apart from the defaults. In addition, the `default` service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
## Reference Hardened RKE `cluster.yml` Configuration
@@ -10,12 +10,11 @@ If you operate Rancher behind a proxy and you want to access services through th
Make sure `NO_PROXY` contains the network addresses, network address ranges and domains that should be excluded from using the proxy.
| Environment variable | Purpose |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s) |
| Environment variable | Purpose |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s). <br/><br/> The value must be a comma-delimited string which contains IP addresses, CIDR notation, domain names, or special DNS labels (*). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) |
:::note Important:
NO_PROXY must be in uppercase to use network range (CIDR) notation.
@@ -62,4 +61,4 @@ acl SSL_ports port 2376
acl Safe_ports port 22 # ssh
acl Safe_ports port 2376 # docker port
```
```
@@ -12,7 +12,6 @@
| [Ability to rotate certificates](../how-to-guides/new-user-guides/manage-clusters/rotate-certificates.md) | ✓ | ✓ | | |
| Ability to [backup](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher-launched-kubernetes-clusters.md) and [restore](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-rancher-launched-kubernetes-clusters-from-backup.md) Rancher-launched clusters | ✓ | ✓ | | ✓<sup>4</sup> |
| [Cleaning Kubernetes components when clusters are no longer reachable from Rancher](../how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md) | ✓ | | | |
| [Configuring Pod Security Policies](../how-to-guides/new-user-guides/manage-clusters/add-a-pod-security-policy.md) | ✓ | ✓ | ||
1. Registered EKS, GKE and AKS clusters have the same options available as EKS, GKE and AKS clusters created from the Rancher UI. The difference is that when a registered cluster is deleted from the Rancher UI, it is not destroyed.
@@ -0,0 +1,205 @@
---
title: Kubeconfigs
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/workflows/kubeconfigs"/>
</head>
## Kubeconfig Resource
Kubeconfig is a Rancher resource `kubeconfigs.ext.cattle.io` that allows generating `v1.Config` kubeconfig files for interacting with Rancher and clusters managed by Rancher.
```sh
kubectl api-resources --api-group=ext.cattle.io
```
To get a description of the fields and structure of the Kubeconfig resource, run:
```sh
kubectl explain kubeconfigs.ext.cattle.io
```
## Feature Flag
The Kubeconfigs Public API is available since Rancher v2.12.0 and is enabled by default. It can be disabled by setting the `ext-kubeconfigs` feature flag to `false`.
```sh
kubectl patch feature ext-kubeconfigs -p '{"spec":{"value":false}}'
```
## Creating a Kubeconfig
Admins can delete any Kubeconfig, while regular users can only delete their own. When a Kubeconfig is deleted, the kubeconfig tokens are also deleted.
E.g. using a service account `system:admin` will lead to the following error:
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Kubeconfig
EOF
Error from server (Forbidden): error when creating "STDIN": kubeconfigs.ext.cattle.io is forbidden: user system:admin is not a Rancher user
```
:::warning Important
The kubeconfig content is generated and returned in the `.status.value` field **only once** when the Kubeconfig is successfully created because it contains secret values for created tokens. Therefore it has to be captured by using an appropriate output option, such as `-o jsonpath='{.status.value}'`, or `-o yaml`.
:::
A kubeconfig can be created for more than one cluster at a time by specifying a list of cluster names in the `spec.clusters` field. You can look up cluster names by listing `clusters.management.cattle.io` resources.
```sh
kubectl get clusters.management.cattle.io -o=jsonpath="{.items[*]['metadata.name', 'spec.displayName']}{'\n'}"
local local
c-m-p66cdvlj downstream1
```
The `metadata.name` and `metadata.generateName` fields are ignored, and the name of the new Kubeconfig is automatically generated using the prefix `kubeconfig-`.
You can use the `spec.currentContext` field to set the cluster name, and it is used to set the current context in the kubeconfig. If you do not set the `spec.currentContext` field, then the first cluster in the `spec.clusters` list will be used as the current context. For ACE-enabled clusters that don't have an FQDN set, the first control plane node will be used as the current context.
For ACE-enabled clusters, if the FQDN is set, then that will be used as a cluster entry in the kubeconfig; otherwise, entries for all control plane nodes will be created.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Kubeconfig
spec:
clusters: [c-m-p66cdvlj, c-m-fcd3g5h]
description: My Kubeconfig
currentContext: c-m-p66cdvlj
EOF
```
If `"*"` is specified as the first item in the `spec.clusters` field, the kubeconfig will be created for all clusters that the user has access to, if any.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Kubeconfig
spec:
clusters: ["*"]
description: My Kubeconfig
EOF
```
If `spec.ttl` is not specified, the Kubeconfig's tokens will be created with the expiration time defined in the `kubeconfig-default-token-ttl-minutes` setting, which is 30 days by default. If `spec.ttl` is specified, it should be greater than 0 and less than or equal to the value of the `kubeconfig-default-token-ttl-minutes` setting expressed in seconds.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Kubeconfig
spec:
clusters: [c-m-p66cdvlj] # Downstream cluster
ttl: 7200 # 2 hours
EOF
```
## Listing Kubeconfigs
Listing previously generated Kubeconfigs can be useful for cleaning up backing tokens if the Kubeconfig is no longer needed (e.g., it was issued temporarily). Admins can list all Kubeconfigs, while regular users can only view their own.
```sh
kubectl get kubeconfig
NAME TTL TOKENS STATUS AGE
kubeconfig-zp786 30d 2/2 Complete 18d
kubeconfig-7zvzp 30d 1/1 Complete 12d
kubeconfig-jznml 30d 1/1 Complete 12d
```
Use `-o wide` to get more details:
```sh
kubectl get kubeconfig -o wide
NAME TTL TOKENS STATUS AGE USER CLUSTERS DESCRIPTION
kubeconfig-zp786 30d 2/2 Complete 18d user-w5gcf * all clusters
kubeconfig-7zvzp 30d 1/1 Complete 12d u-w7drc *
kubeconfig-jznml 30d 1/1 Complete 12d u-w7drc *
```
## Viewing a Kubeconfig
Admins can get any Kubeconfig, while regular users can only get their own.
```sh
kubectl get kubeconfig kubeconfig-zp786
NAME TTL TOKENS STATUS AGE
kubeconfig-zp786 30d 2/2 Complete 18d
```
Use `-o wide` to get more details:
```sh
kubectl get kubeconfig kubeconfig-zp786 -o wide
NAME TTL TOKENS STATUS AGE USER CLUSTERS DESCRIPTION
kubeconfig-zp786 30d 2/2 Complete 18d user-w5gcf * all clusters
```
## Deleting a Kubeconfig
Admins can delete any Kubeconfig, while regular users can only delete their own. When a Kubeconfig is deleted, the kubeconfig tokens are also deleted.
```sh
kubectl delete kubeconfig kubeconfig-zp786
kubeconfig.ext.cattle.io "kubeconfig-zp786" deleted
```
To delete a Kubeconfig using preconditions:
```sh
cat <<EOF | k delete --raw /apis/ext.cattle.io/v1/kubeconfigs/kubeconfig-zp786 -f -
{
"apiVersion": "v1",
"kind": "DeleteOptions",
"preconditions": {
"uid": "52183e05-d382-47d2-b4b9-d0735823ce90",
"resourceVersion": "31331505"
}
}
EOF
```
## Deleting a Collection of Kubeconfigs
Admins can delete any Kubeconfig, while regular users can only delete their own.
To delete all Kubeconfigs:
```sh
kubectl delete --raw /apis/ext.cattle.io/v1/kubeconfigs
```
To delete a collection of Kubeconfigs by label:
```sh
kubectl delete --raw /apis/ext.cattle.io/v1/kubeconfigs?labelSelector=foo%3Dbar
```
## Updating a Kubeconfig
Only the `metadata`, e.g. adding a label or an annotation, and the `spec.description` field can be updated. All other `spec` fields are immutable.
To edit a Kubeconfig:
```sh
kubectl edit kubeconfig kubeconfig-zp786
```
To patch a Kubeconfig and update its description:
```sh
kubectl patch kubeconfig kubeconfig-zp786 -type merge -p '{"spec":{"description":"Updated description"}}'
kubeconfig.ext.cattle.io/kubeconfig-zp786 patched
kubectl get kubeconfig kubeconfig-fdcpl -o jsonpath='{.spec.description}'
Updated description
```
To patch a Kubeconfig and add a label:
```sh
kubectl patch kubeconfig kubeconfig-zp786 -type merge -p '{"metadata":{"labels":{"foo":"bar"}}}'
kubeconfig.ext.cattle.io/kubeconfig-zp786 patched
kubectl get kubeconfig kubeconfig-zp786 -o jsonpath='{.metadata.labels.foo}'
bar
```
@@ -0,0 +1,114 @@
---
title: Tokens
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/workflows/tokens"/>
</head>
## Feature Flag
The Tokens Public API is available for Rancher v2.12.0 and later, and is enabled by default. You can disable the Tokens Public API by setting the `ext-tokens` feature flag to `false` as shown in the example `kubectl` command below:
```sh
kubectl patch feature ext-tokens -p '{"spec":{"value":false}}'
```
## Creating a Token
Only a **valid and active** Rancher user can create a Token. Otherwise, you will get an error displayed (`Error from server (Forbidden)...`) when attempting to create a Token.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Token
EOF
Error from server (Forbidden): error when creating "STDIN": tokens.ext.cattle.io is forbidden: user system:admin is not a Rancher user
```
A Token is always created for the user making the request. Attempting to create a Token for a different user, by specifying a different `spec.userID`, is forbidden and will fail.
- The `spec.description` field can be set to an arbitrary human-readable description of the Token's purpose. The default value is empty.
- The `spec.kind` field can be set to the kind of Token. The value `session` indicates a login Token. All other values, including the default empty string, indicate a kind of derived Token.
- The `metadata.name` and `metadata.generateName` fields are ignored, and the name of the new Token is automatically generated using the prefix `token-`.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Token
spec:
description: My Token
EOF
```
- If the `spec.ttl` is not specified, the Token is created with the expiration time defined in the `auth-token-max-ttl-minutes` setting. The default expiration time is 90 days. If `spec.ttl` is specified, it should be greater than 0 and less than or equal to the value of the `auth-token-max-ttl-minutes` setting expressed in milliseconds.
```bash
kubectl create -o jsonpath='{.status.value}' -f -<<EOF
apiVersion: ext.cattle.io/v1
kind: Token
spec:
ttl: 7200000 # 2 hours
EOF
```
## Listing Tokens
Listing previously generated Tokens can help clean up tokens that are no longer needed (e.g., they were issued temporarily). Admins can list all Tokens, while regular users can only see their own.
```sh
kubectl get tokens.ext.cattle.io
NAME KIND TTL AGE
token-chjc9 90d 18s
token-6fzgj 90d 16s
token-8nbrm 90d 14s
```
Use `-o wide` to get more details:
```sh
kubectl get tokens.ext.cattle.io -o wide
NAME USER KIND TTL AGE DESCRIPTION
token-chjc9 user-jtghh 90d 24s example
token-6fzgj user-jtghh 90d 22s box
token-8nbrm user-jtghh 90d 20s jinx
```
## Viewing a Token
Admins can get any Token, while regular users can only get their own.
```sh
kubectl get tokens.ext.cattle.io token-chjc9
NAME KIND TTL AGE
token-chjc9 90d 18s
```
Use `-o wide` to get more details:
```sh
kubectl get tokens.ext.cattle.io token-chjc9 -o wide
NAME USER KIND TTL AGE DESCRIPTION
token-chjc9 user-jtghh 90d 24s example
```
## Deleting a Token
Admins can delete any Token, while regular users can only delete their own.
```sh
kubectl delete tokens.ext.cattle.io token-chjc9
token.ext.cattle.io "token-chjc9" deleted
```
## Updating a Token
Only the metadata fields `spec.description`, `spec.ttl`, and `spec.enabled` can be updated. All other `spec` fields are immutable. Admins can extend the `spec.ttl` field, while regular users can only reduce the value.
An example `kubectl` command to edit a Token:
```sh
kubectl edit tokens.ext.cattle.io token-zp786
```
@@ -39,7 +39,6 @@ User Interface | https://github.com/rancher/dashboard/ | This repository is the
(Rancher) Docker Machine | https://github.com/rancher/machine | This repository is the source of the Docker Machine binary used when using Node Drivers. This is a fork of the `docker/machine` repository.
machine-package | https://github.com/rancher/machine-package | This repository is used to build the Rancher Docker Machine binary.
kontainer-engine | https://github.com/rancher/kontainer-engine | This repository is the source of kontainer-engine, the tool to provision hosted Kubernetes clusters.
RKE repository | https://github.com/rancher/rke | This repository is the source of Rancher Kubernetes Engine, the tool to provision Kubernetes clusters on any machine.
CLI | https://github.com/rancher/cli | This repository is the source code for the Rancher CLI used in Rancher 2.x.
(Rancher) Helm repository | https://github.com/rancher/helm | This repository is the source of the packaged Helm binary. This is a fork of the `helm/helm` repository.
loglevel repository | https://github.com/rancher/loglevel | This repository is the source of the loglevel binary, used to dynamically change log levels.
@@ -109,27 +108,6 @@ Please remove any sensitive data as it will be publicly viewable.
-l app=rancher \
--timestamps=true
```
- Docker install using `docker` on each of the nodes in the RKE cluster
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher@|rancher_rancher" | awk '{ print $1 }')
```
- Kubernetes Install with RKE Add-On
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if the Rancher server is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
--timestamps=true \
-f $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name="cattle-server") | .metadata.name')
```
- System logging (these might not all exist, depending on operating system)
- `/var/log/messages`
- `/var/log/syslog`
@@ -16,9 +16,7 @@ Rancher will publish deprecated features as part of the [release notes](https://
| Patch Version | Release Date |
|---------------|---------------|
| [2.11.2](https://github.com/rancher/rancher/releases/tag/v2.11.2) | May 22, 2025 |
| [2.11.1](https://github.com/rancher/rancher/releases/tag/v2.11.1) | Apr 24, 2025 |
| [2.11.0](https://github.com/rancher/rancher/releases/tag/v2.11.0) | Mar 31, 2025 |
| [2.12.0](https://github.com/rancher/rancher/releases/tag/v2.12.0) | July 31, 2025 |
## What can I expect when a feature is marked for deprecation?
@@ -199,7 +199,6 @@ Because `rancher` is the default option for `ingress.tls.source`, we are not spe
- Set the `hostname` to the DNS name you pointed at your load balancer.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.7.0`
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -240,7 +239,6 @@ In the following command,
- `ingress.tls.source` is set to `letsEncrypt`
- `letsEncrypt.email` is set to the email address used for communication about your certificate (for example, expiry notices)
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
:::warning
@@ -289,7 +287,6 @@ If you want to check if your certificates are correct, see [How do I check Commo
- Set the `hostname`.
- Set the `bootstrapPassword` to something unique for the `admin` user.
- Set `ingress.tls.source` to `secret`.
- For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm install rancher rancher-<CHART_REPO>/rancher \
@@ -1,72 +0,0 @@
---
title: Upgrading a Hardened Custom/Imported Cluster to Kubernetes v1.25
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrade-a-hardened-cluster-to-k8s-v1-25"/>
</head>
Kubernetes v1.25 changes how clusters describe and implement security policies. From this version forward, [Pod Security Policies (PSPs)](https://kubernetes.io/docs/concepts/security/pod-security-policy/) are no longer available. Kubernetes v1.25 replaces them with new security objects: [Pod Security Standards (PSS)](https://kubernetes.io/docs/concepts/security/pod-security-standards/), and [Pod Security Admissions (PSAs)](https://kubernetes.io/docs/concepts/security/pod-security-admission/).
If you have custom or imported hardened clusters, you must take special preparations to ensure that the upgrade from an earlier version of Kubernetes to v1.25 or later goes smoothly.
:::note
After you upgrade to v1.25, add the necessary Rancher namespace exemptions. See [Pod Security Admission (PSA) Configuration Templates](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md#exempting-required-rancher-namespaces) for more details.
:::
## Upgrading Imported Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
Perform the following on each node in the cluster:
1. Save [`rancher-psact.yaml`](./rancher-psact.yaml) in `/etc/rancher/rke2`.
1. Edit the RKE2 configuration file:
1. Update the `profile` field to `cis-1.23`.
1. Specify the path for the configuration file that you just added: `pod-security-admission-config-file: /etc/rancher/rke2/rancher-psact.yaml`.
</TabItem>
<TabItem value="K3s">
Perform the following on each node in the cluster:
Follow the official K3s instructions on [Upgrading Hardened Clusters from v1.24.x to v1.25.x](https://docs.k3s.io/known-issues#hardened-125), but use a [custom](./rancher-psact.yaml) Rancher PSA configuration template, instead of the configuration provided on the official K3s site.
</TabItem>
</Tabs>
After you perform these steps, you can upgrade the cluster's Kubernetes version through the Rancher UI:
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
## Upgrading Custom Hardened Clusters to Kubernetes v1.25 or Later
<Tabs groupId="k8s-distro">
<TabItem value="RKE2" default>
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit Config**.
1. Under **Basics > Security**, in the **CIS Profile** dropdown menu, select `cis-1.23`.
1. In the **Pod Security Admission Configuration Template** dropdown menu, select `rancher-restricted`.
1. In the **Kubernetes Version** dropdown menu, select the version that you would like to use.
1. Click **Save**.
</TabItem>
<TabItem value="K3s">
1. In the upper left corner, click **☰ > Cluster Management**.
1. Find the cluster you want to update in the **Clusters** table, and click the **⋮**.
1. Select **Edit YAML**.
1. Delete `PodSecurityPolicy` from `kube-apiserver-arg.enable-admission-plugins`
1. Add this line to the `spec` field: `defaultPodSecurityAdmissionConfigurationTemplateName: rancher-restricted`
1. Update `kubernetesVersion` to your chosen version (v1.25 or later).
1. Click **Save**.
</TabItem>
</Tabs>
@@ -152,7 +152,6 @@ Upgrade Rancher to the latest version with all your settings.
Take all the values from the previous step and append them to the command using `--set key=value`.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
@@ -186,8 +185,7 @@ Alternatively, it's possible to export the current values to a file and referenc
```
1. Update only the Rancher version:
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
@@ -26,7 +26,30 @@ The following is a list of feature flags available in Rancher. If you've upgrade
- `imperative-api-extension`: Enables Rancher's [extension API server](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) to register new APIs to Kubernetes. This flag is enabled by default. See the [Extension API Server](../../../api/extension-apiserver.md) page for more information.
- `istio-virtual-service-ui`: Enables a [visual interface](../../../how-to-guides/advanced-user-guides/enable-experimental-features/istio-traffic-management-features.md) to create, read, update, and delete Istio virtual services and destination rules, which are Istio traffic management features.
- `legacy`: Enables a set of features from 2.5.x and earlier, that are slowly being phased out in favor of newer implementations. These are a mix of deprecated features as well as features that will eventually be available to newer versions. This flag is disabled by default on new Rancher installations. If you're upgrading from a previous version of Rancher, this flag is enabled.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream RKE2/K3s clusters, currently limited to imported clusters and the local cluster, with plans to expand support to node-driver clusters.
- `managed-system-upgrade-controller`: Enables the installation of the system-upgrade-controller app in downstream imported RKE2/K3s clusters, as well as in the local cluster if it is an RKE2/K3s cluster.
:::note Important:
This `managed-system-upgrade-controller` flag is intended for **internal use only** and does not have an associated Feature CR. Use with caution.
To control whether Rancher should manage the Kubernetes version of imported RKE2/K3s clusters, it is recommended to use the [imported-cluster-version-management](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters) feature that is available in Rancher v2.11.0 or newer.
:::
:::danger
If the `managed-system-upgrade-controller` flag was **disabled** in Rancher v2.10.x, and any imported RKE2/K3s clusters were upgraded **outside of Rancher**, follow the steps below to prevent the unexpected installation of the system-upgrade-controller app and to ensure the [imported-cluster-version-management](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters) feature works correctly:
1. Upgrade Rancher to v2.11.0 or newer, making sure to **retain** the `managed-system-upgrade-controller=false` feature flag in Helm values if it was set during the v2.10.x installation.
1. After Rancher is fully up and running, disable the `imported-cluster-version-management` setting. You can do this either through the Rancher UI by clicking **☰ > Global Settings > Settings > imported-cluster-version-management**, or by editing the corresponding `Setting.management.cattle.io/v3` custom resource via kubectl.
1. Perform a second Helm upgrade, this time omitting the `managed-system-upgrade-controller=false` feature flag.
Now, the imported cluster version management is disabled by default, and Rancher no longer installs the system-upgrade-controller app on imported clusters automatically.
You can enable this feature on a per-cluster basis. For more information, please refer to the [documentation](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#configuring-version-management-for-rke2-and-k3s-clusters).
:::
- `multi-cluster-management`: Allows multi-cluster provisioning and management of Kubernetes clusters. This flag can only be set at install time. It can't be enabled or disabled later.
- `rke1-custom-node-cleanup`: Enables cleanup of deleted RKE1 custom nodes. We recommend that you keep this flag enabled, to prevent removed nodes from attempting to rejoin the cluster.
- `rke2`: Enables provisioning RKE2 clusters. This flag is enabled by default.
@@ -62,8 +62,6 @@ For information on enabling experimental features, refer to [this page.](../../.
| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system container images, e.g., http://registry.example.com/ |
| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" |
| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. |
| `global.cattle.psp.enabled` | `true` | `bool` - select 'false' to disable PSPs for Kubernetes v1.25 and above when using Rancher v2.7.2-v2.7.4. When using Rancher v2.7.5 and above, Rancher attempts to detect if a cluster is running a Kubernetes version where PSPs are not supported, and will default it's usage of PSPs to false if it can determine that PSPs are not supported in the cluster. Users can still manually override this by explicitly providing `true` or `false` for this value. Rancher will still use PSPs by default in clusters which support PSPs (such as clusters running Kubernetes v1.24 or lower). |
### Bootstrap Password
@@ -172,7 +172,6 @@ kubectl create namespace cattle-system
Next, install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
Placeholder | Description
------------|-------------
@@ -203,7 +202,6 @@ Create Kubernetes secrets from your own certificates for Rancher to use. The com
Install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------- |
@@ -10,7 +10,15 @@ Once the infrastructure is ready, you can continue with setting up a Kubernetes
The steps to set up RKE, RKE2, or K3s are shown below.
For convenience, export the IP address and port of your proxy into an environment variable and set up the HTTP_PROXY variables for your current shell on every node:
For convenience, export the IP address and port of your proxy into an environment variable and set up the `HTTP_PROXY` variables for your current shell on every node:
:::caution
The `NO_PROXY` environment variable is not standardized, and the accepted format of the value can differ between applications. When configuring the `NO_PROXY` variable for Rancher, the value must adhere to the format expected by Golang.
Specifically, the value should be a comma-delimited string which only contains IP addresses, CIDR notation, domain names, or special DNS labels (e.g. `*`). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config)
:::
```
export proxy_host="10.0.0.5:8888"
@@ -53,6 +53,10 @@ You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by loggi
## Upgrade
:::danger
Rancher upgrades to version 2.12.0 and later will be blocked if any RKE1-related resources are detected, as the Rancher Kubernetes Engine (RKE/RKE1) is end of life as of **July 31, 2025**. For detailed cleanup and recovery steps, refer to the [RKE1 Resource Validation and Upgrade Requirements in Rancher v2.12](#rke1-resource-validation-and-upgrade-requirements-in-rancher-v212).
:::
During upgrade, you create a copy of the data from your current Rancher container and a backup in case something goes wrong. Then you deploy the new version of Rancher in a new container using your existing data.
### 1. Create a copy of the data from your Rancher server container
@@ -388,6 +392,45 @@ See [Restoring Cluster Networking](https://github.com/rancher/rancher-docs/tree/
Remove the previous Rancher server container. If you only stop the previous Rancher server container (and don't remove it), the container may restart after the next server reboot.
## RKE1 Resource Validation and Upgrade Requirements in Rancher v2.12
Rancher v2.12.0 and later has removed support for the Rancher Kubernetes Engine (RKE/RKE1). During upgrade, Rancher validates the cluster resources and blocks the upgrade if any RKE1-related resources are detected.
This validation affects the following resource types:
- Clusters with `rkeConfig` (`clusters.management.cattle.io`)
- NodeTemplates (`nodetemplates.management.cattle.io`)
- ClusterTemplates (`clustertemplates.management.cattle.io`)
This is particularly relevant for single-node Docker installations, where Rancher is not running during the upgrade. In such cases, controllers are not available to automatically clean up deprecated resources, and the upgrade process will fail early with an error listing the blocking resources.
### 1. Pre-Upgrade (Recommended)
Before upgrading, while Rancher is still running:
- Run the `pre-upgrade-hook` cleanup script to delete all RKE1 clusters and templates. You can find the script in the Rancher GitHub repository: [pre-upgrade-hook.sh](https://github.com/rancher/rancher/blob/v2.12.0/chart/scripts/pre-upgrade-hook.sh).
- This allows Rancher to clean up associated resources and finalizers.
### 2. Post-Upgrade Failure Due to Residual RKE1 Resources
If the upgrade to Rancher v2.12.0 or later is attempted without prior cleanup of RKE1 resources:
- The upgrade will fail and display an error listing the resource names that are preventing the upgrade.
- This occurs because Rancher includes validation to detect and block upgrades when unsupported RKE1 resources are still present.
- To proceed, [rollback](#rolling-back) to the previous Rancher version, delete the identified resources, and then retry after [manual cleanup](#manual-cleanup-after-rollback).
:::note Helm-based Rancher
Helm-based Rancher installations are not affected by this issue, as Rancher remains available during the upgrade and can perform resource cleanup as needed.
:::
### Manual Cleanup After Rollback
Users should perform the following steps after rolling back to a previous Rancher version:
- **Manually delete** the resources listed in the upgrade error message (e.g., RKE1 clusters, NodeTemplates, ClusterTemplates).
- If deletion is blocked due to **finalizers**, edit the resources and remove the `metadata.finalizers` field.
- If a **validating webhook** prevents deletion (e.g., for the `system-project`), please refer to the [Bypassing the Webhook](../../../../reference-guides/rancher-webhook.md#bypassing-the-webhook) documentation.
## Rolling Back
If your upgrade does not complete successfully, you can roll back Rancher server and its data back to its last healthy state. For more information, see [Docker Rollback](roll-back-docker-installed-rancher.md).
@@ -35,7 +35,7 @@ The Rancher API server is built on top of an embedded Kubernetes API server and
### Authorization and Role-Based Access Control
- **User management:** The Rancher API server [manages user identities](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md) that correspond to external authentication providers like Active Directory or GitHub, in addition to local users.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) policies.
- **Authorization:** The Rancher API server manages [access control](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) and [security](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) standards.
### Working with Kubernetes
@@ -128,7 +128,6 @@ The final command to install Rancher is below. The command requires a domain nam
To install a specific Rancher version, use the `--version` flag (e.g., `--version 2.6.6`). Otherwise, the latest Rancher is installed by default. Refer to [Choosing a Rancher Version](../../installation-and-upgrade/resources/choose-a-rancher-version.md).
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
See [Setting up the Bootstrap Password](../../installation-and-upgrade/resources/bootstrap-password.md#password-requirements) for password requirements.
@@ -1,17 +0,0 @@
---
title: CIS Scan Guides
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides"/>
</head>
- [Install rancher-cis-benchmark](install-rancher-cis-benchmark.md)
- [Uninstall rancher-cis-benchmark](uninstall-rancher-cis-benchmark.md)
- [Run a Scan](run-a-scan.md)
- [Run a Scan Periodically on a Schedule](run-a-scan-periodically-on-a-schedule.md)
- [Skip Tests](skip-tests.md)
- [View Reports](view-reports.md)
- [Enable Alerting for rancher-cis-benchmark](enable-alerting-for-rancher-cis-benchmark.md)
- [Configure Alerts for Periodic Scan on a Schedule](configure-alerts-for-periodic-scan-on-a-schedule.md)
- [Create a Custom Benchmark Version to Run](create-a-custom-benchmark-version-to-run.md)
@@ -1,13 +0,0 @@
---
title: Create a Custom Benchmark Version for Running a Cluster Scan
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/create-a-custom-benchmark-version-to-run"/>
</head>
There could be some Kubernetes cluster setups that require custom configurations of the Benchmark tests. For example, the path to the Kubernetes config files or certs might be different than the standard location where the upstream CIS Benchmarks look for them.
It is now possible to create a custom Benchmark Version for running a cluster scan using the `rancher-cis-benchmark` application.
For details, see [this page.](../../../integrations-in-rancher/cis-scans/custom-benchmark.md)
@@ -1,24 +0,0 @@
---
title: Enable Alerting for Rancher CIS Benchmark
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/enable-alerting-for-rancher-cis-benchmark"/>
</head>
Alerts can be configured to be sent out for a scan that runs on a schedule.
:::note Prerequisite:
Before enabling alerts for `rancher-cis-benchmark`, make sure to install the `rancher-monitoring` application and configure the Receivers and Routes. For more information, see [this section.](../../../reference-guides/monitoring-v2-configuration/receivers.md)
While configuring the routes for `rancher-cis-benchmark` alerts, you can specify the matching using the key-value pair `job: rancher-cis-scan`. An example route configuration is [here.](../../../reference-guides/monitoring-v2-configuration/receivers.md#example-route-config-for-cis-scan-alerts)
:::
While installing or upgrading the `rancher-cis-benchmark` Helm chart, set the following flag to `true` in the `values.yaml`:
```yaml
alerts:
enabled: true
```
@@ -1,21 +0,0 @@
---
title: Install Rancher CIS Benchmark
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/install-rancher-cis-benchmark"/>
</head>
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to install CIS Benchmark and click **Explore**.
1. In the left navigation bar, click **Apps > Charts**.
1. Click **CIS Benchmark**
1. Click **Install**.
**Result:** The CIS scan application is deployed on the Kubernetes cluster.
:::note
If you are running Kubernetes v1.24 or earlier, and have a [Pod Security Policy](../../new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) (PSP) hardened cluster, CIS Benchmark 4.0.0 and later disable PSPs by default. To install CIS Benchmark on a PSP-hardened cluster, set `global.psp.enabled` to `true` in the values before installing the chart. [Pod Security Admission](../../new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) (PSA) hardened clusters aren't affected.
:::
@@ -1,26 +0,0 @@
---
title: Run a Scan
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/run-a-scan"/>
</head>
When a ClusterScan custom resource is created, it launches a new CIS scan on the cluster for the chosen ClusterScanProfile.
:::note
There is currently a limitation of running only one CIS scan at a time for a cluster. If you create multiple ClusterScan custom resources, they will be run one after the other by the operator, and until one scan finishes, the rest of the ClusterScan custom resources will be in the "Pending" state.
:::
To run a scan,
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to run a CIS scan and click **Explore**.
1. Click **CIS Benchmark > Scan**.
1. Click **Create**.
1. Choose a cluster scan profile. The profile determines which CIS Benchmark version will be used and which tests will be performed. If you choose the Default profile, then the CIS Operator will choose a profile applicable to the type of Kubernetes cluster it is installed on.
1. Click **Create**.
**Result:** A report is generated with the scan results. To see the results, click the name of the scan that appears.
@@ -1,38 +0,0 @@
---
title: Skip Tests
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/skip-tests"/>
</head>
CIS scans can be run using test profiles with user-defined skips.
To skip tests, you will create a custom CIS scan profile. A profile contains the configuration for the CIS scan, which includes the benchmark versions to use and any specific tests to skip in that benchmark.
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to run a CIS scan and click **Explore**.
1. Click **CIS Benchmark > Profile**.
1. From here, you can create a profile in multiple ways. To make a new profile, click **Create** and fill out the form in the UI. To make a new profile based on an existing profile, go to the existing profile and click **⋮ Clone**. If you are filling out the form, add the tests to skip using the test IDs, using the relevant CIS Benchmark as a reference. If you are creating the new test profile as YAML, you will add the IDs of the tests to skip in the `skipTests` directive. You will also give the profile a name:
```yaml
apiVersion: cis.cattle.io/v1
kind: ClusterScanProfile
metadata:
annotations:
meta.helm.sh/release-name: clusterscan-operator
meta.helm.sh/release-namespace: cis-operator-system
labels:
app.kubernetes.io/managed-by: Helm
name: "<example-profile>"
spec:
benchmarkVersion: cis-1.5
skipTests:
- "1.1.20"
- "1.1.21"
```
1. Click **Create**.
**Result:** A new CIS scan profile is created.
When you [run a scan](./run-a-scan.md) that uses this profile, the defined tests will be skipped during the scan. The skipped tests will be marked in the generated report as `Skip`.
@@ -1,13 +0,0 @@
---
title: Uninstall Rancher CIS Benchmark
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/uninstall-rancher-cis-benchmark"/>
</head>
1. From the **Cluster Dashboard,** go to the left navigation bar and click **Apps > Installed Apps**.
1. Go to the `cis-operator-system` namespace and check the boxes next to `rancher-cis-benchmark-crd` and `rancher-cis-benchmark`.
1. Click **Delete** and confirm **Delete**.
**Result:** The `rancher-cis-benchmark` application is uninstalled.
@@ -1,23 +0,0 @@
---
title: View Reports
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/cis-scan-guides/view-reports"/>
</head>
To view the generated CIS scan reports,
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to run a CIS scan and click **Explore**.
1. Click **CIS Benchmark > Scan**.
1. The **Scans** page will show the generated reports. To see a detailed report, go to a scan report and click the name.
One can download the report from the Scans list or from the scan detail page.
To get the verbose version of the CIS scan results, run the following command on the cluster that was scanned. Note that the scan must be completed before this can be done.
```console
export REPORT="scan-report-name"
kubectl get clusterscanreport $REPORT -o json |jq ".spec.reportJSON | fromjson" | jq -r ".actual_value_map_data" | base64 -d | gunzip | jq .
```
@@ -0,0 +1,16 @@
---
title: Compliance Scan Guides
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/compliance-scan-guides"/>
</head>
- [Install rancher-compliance](install-rancher-compliance.md)
- [Uninstall rancher-compliance](uninstall-rancher-compliance.md)
- [Run a Scan](run-a-scan.md)
- [Run a Scan Periodically on a Schedule](run-a-scan-periodically-on-a-schedule.md)
- [View Reports](view-reports.md)
- [Enable Alerting for rancher-compliance](enable-alerting-for-rancher-compliance.md)
- [Configure Alerts for Periodic Scan on a Schedule](configure-alerts-for-periodic-scan-on-a-schedule.md)
- [Create a Custom Benchmark Version to Run](create-a-custom-compliance-version-to-run.md)

Some files were not shown because too many files have changed in this diff Show More