mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-25 20:48:11 +00:00
Add back removed files
This commit is contained in:
LucasSaintarbor
+365
@@ -0,0 +1,365 @@
|
||||
---
|
||||
title: RKE Cluster Configuration Reference
|
||||
---
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration"/>
|
||||
</head>
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
When Rancher installs Kubernetes, it uses [RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) or [RKE2](https://docs.rke2.io/) as the Kubernetes distribution.
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE Kubernetes cluster.
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#rke-cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
The RKE cluster config options are nested under the `rancher_kubernetes_engine_config` directive. For more information, see the section about the [cluster config file.](#rke-cluster-config-file-reference)
|
||||
|
||||
In [clusters launched by RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md), you can edit any of the remaining options that follow.
|
||||
|
||||
For an example of RKE config file syntax, see the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
|
||||
|
||||
The forms in the Rancher UI don't include all advanced options for configuring RKE. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
## Editing Clusters with a Form in the Rancher UI
|
||||
|
||||
To edit your cluster,
|
||||
|
||||
1. In the upper left corner, click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
|
||||
|
||||
|
||||
## Editing Clusters with YAML
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
RKE clusters (also called RKE1 clusters) are edited differently than RKE2 and K3s clusters.
|
||||
|
||||
To edit an RKE config file directly from the Rancher UI,
|
||||
|
||||
1. Click **☰ > Cluster Management**.
|
||||
1. Go to the RKE cluster you want to configure. Click and click **⋮ > Edit Config**. This take you to the RKE configuration form. Note: Because cluster provisioning changed in Rancher 2.6, the **⋮ > Edit as YAML** can be used for configuring RKE2 clusters, but it can't be used for editing RKE1 configuration.
|
||||
1. In the configuration form, scroll down and click **Edit as YAML**.
|
||||
1. Edit the RKE options under the `rancher_kubernetes_engine_config` directive.
|
||||
|
||||
## Configuration Options in the Rancher UI
|
||||
|
||||
:::tip
|
||||
|
||||
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
:::
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
|
||||
|
||||
### Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses. For more details on the different networking providers, please view our [Networking FAQ](../../../faq/container-network-interface-providers.md).
|
||||
|
||||
:::caution
|
||||
|
||||
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
|
||||
|
||||
:::
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
<DeprecationWeave />
|
||||
|
||||
:::note Notes on Weave:
|
||||
|
||||
When Weave is selected as network provider, Rancher will automatically enable encryption by generating a random password. If you want to specify the password manually, please see how to configure your cluster using a [Config File](#rke-cluster-config-file-reference) and the [Weave Network Plug-in Options](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options).
|
||||
|
||||
:::
|
||||
|
||||
### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### Kubernetes Cloud Providers
|
||||
|
||||
You can configure a [Kubernetes cloud provider](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
:::note
|
||||
|
||||
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#rke-cluster-config-file-reference) to configure the cloud provider. Please reference the [RKE cloud provider documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
:::
|
||||
|
||||
### Private Registries
|
||||
|
||||
The cluster-level private registry configuration is only used for provisioning clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the [global default registry](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md) through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
If your private registry requires credentials, you need to pass the credentials to Rancher by editing the cluster options for each cluster that needs to pull images from the registry.
|
||||
|
||||
The private registry configuration option tells Rancher where to pull the [system images](https://rancher.com/docs/rke/latest/en/config-options/system-images/) or [addon images](https://rancher.com/docs/rke/latest/en/config-options/add-ons/) that will be used in your cluster.
|
||||
|
||||
- **System images** are components needed to maintain the Kubernetes cluster.
|
||||
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
|
||||
|
||||
For more information on setting up a private registry for components applied during the provisioning of the cluster, see the [RKE documentation on private registries](https://rancher.com/docs/rke/latest/en/config-options/private-registries/).
|
||||
|
||||
Rancher v2.6 introduced the ability to configure [ECR registries for RKE clusters](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup).
|
||||
|
||||
### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint (ACE) can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
:::note
|
||||
|
||||
ACE is available on RKE, RKE2, and K3s clusters that are provisioned or registered with Rancher. It's not available on clusters in a hosted Kubernetes provider, such as Amazon's EKS.
|
||||
|
||||
:::
|
||||
|
||||
ACE must be set up [manually](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) on RKE2 and K3s clusters. In RKE, ACE is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self-signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
|
||||
|
||||
### Node Pools
|
||||
|
||||
For information on using the Rancher UI to set up node pools in an RKE cluster, refer to [this page.](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md)
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster.
|
||||
|
||||
### Metrics Server Monitoring
|
||||
|
||||
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
You must have an existing Pod Security Policy configured before you can use this option.
|
||||
|
||||
### Docker Version on Nodes
|
||||
|
||||
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support.
|
||||
|
||||
If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
|
||||
|
||||
For details on which Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
|
||||
|
||||
### Docker Root Directory
|
||||
|
||||
If the nodes you are adding to the cluster have Docker configured with a non-default Docker Root Directory (default is `/var/lib/docker`), specify the correct Docker Root Directory in this option.
|
||||
|
||||
### Default Pod Security Policy
|
||||
|
||||
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
|
||||
|
||||
### Node Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). Default is `30000-32767`.
|
||||
|
||||
### Recurring etcd Snapshots
|
||||
|
||||
Option to enable or disable [recurring etcd snapshots](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
|
||||
|
||||
### Agent Environment Variables
|
||||
|
||||
Option to set environment variables for [rancher agents](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
|
||||
### Updating ingress-nginx
|
||||
|
||||
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
|
||||
|
||||
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
|
||||
|
||||
### Cluster Agent Configuration and Fleet Agent Configuration
|
||||
|
||||
You can configure the scheduling fields and resource limits for the Cluster Agent and the cluster's Fleet Agent. You can use these fields to customize tolerations, affinity rules, and resource requirements. Additional tolerations are appended to a list of default tolerations and control plane node taints. If you define custom affinity rules, they override the global default affinity setting. Defining resource requirements sets requests or limits where there previously were none.
|
||||
|
||||
:::note
|
||||
|
||||
With this option, it's possible to override or remove rules that are required for the functioning of the cluster. We strongly recommend against removing or overriding these and any other affinity rules, as this may cause unwanted side effects:
|
||||
|
||||
- `affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution` for `cattle-cluster-agent`
|
||||
- `cluster-agent-default-affinity` for `cattle-cluster-agent`
|
||||
- `fleet-agent-default-affinity` for `fleet-agent`
|
||||
|
||||
:::
|
||||
|
||||
If you downgrade Rancher to v2.7.4 or below, your changes will be lost and the agents will re-deploy without your customizations. The Fleet agent will fallback to using its built-in default values when it re-deploys. If the Fleet version doesn't change during the downgrade, the re-deploy won't be immediate.
|
||||
|
||||
|
||||
## RKE Cluster Config File Reference
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available](https://rancher.com/docs/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API.
|
||||
|
||||
For the complete reference for configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
### Config File Structure in Rancher
|
||||
|
||||
RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kubernetes clusters. Rancher's cluster config files used to have the same structure as [RKE config files,](https://rancher.com/docs/rke/latest/en/example-yamls/) but the structure changed so that in Rancher, RKE cluster config items are separated from non-RKE config items. Therefore, configuration for your cluster needs to be nested under the `rancher_kubernetes_engine_config` directive in the cluster config file. Cluster config files created with earlier versions of Rancher will need to be updated for this format. An example cluster config file is included below.
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>Example Cluster Config File</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # Currently only nginx ingress provider is supported.
|
||||
# # To disable ingress controller, set `provider: none`
|
||||
# # To enable ingress on specific nodes, use the node_selector, eg:
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### Default DNS provider
|
||||
|
||||
The table below indicates what DNS provider is deployed by default. See [RKE documentation on DNS provider](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/) for more information how to configure a different DNS provider. CoreDNS can only be used on Kubernetes v1.12.0 and higher.
|
||||
|
||||
| Rancher version | Kubernetes version | Default DNS provider |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 and higher | v1.14.0 and higher | CoreDNS |
|
||||
| v2.2.5 and higher | v1.13.x and lower | kube-dns |
|
||||
| v2.2.4 and lower | any | kube-dns |
|
||||
|
||||
## Rancher Specific Parameters in YAML
|
||||
|
||||
Besides the RKE config file options, there are also Rancher specific settings that can be configured in the Config File (YAML):
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
See [Docker Root Directory](#docker-root-directory).
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
Option to enable or disable [Cluster Monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md).
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
Option to enable or disable Project Network Isolation.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
See [Authorized Cluster Endpoint](#authorized-cluster-endpoint).
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### Custom Network Plug-in
|
||||
|
||||
You can add a custom network plug-in by using the [user-defined add-on functionality](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. You define any add-on that you want deployed after the Kubernetes cluster is deployed.
|
||||
|
||||
There are two ways that you can specify an add-on:
|
||||
|
||||
- [In-line Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [Referencing YAML Files for Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
+513
@@ -0,0 +1,513 @@
|
||||
---
|
||||
title: RKE Hardening Guides
|
||||
---
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/hardening-guides/rke1-hardening-guide"/>
|
||||
</head>
|
||||
|
||||
This document provides prescriptive guidance for how to harden an RKE cluster intended for production, before provisioning it with Rancher. It outlines the configurations and controls required for Center for Information Security (CIS) Kubernetes benchmark controls.
|
||||
|
||||
:::note
|
||||
This hardening guide describes how to secure the nodes in your cluster. We recommended that you follow this guide before you install Kubernetes.
|
||||
:::
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and is associated with the following versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
| Rancher Version | CIS Benchmark Version | Kubernetes Version |
|
||||
|-----------------|-----------------------|------------------------------|
|
||||
| Rancher v2.7 | Benchmark v1.23 | Kubernetes v1.23 |
|
||||
| Rancher v2.7 | Benchmark v1.24 | Kubernetes v1.24 |
|
||||
| Rancher v2.7 | Benchmark v1.7 | Kubernetes v1.25 up to v1.26 |
|
||||
|
||||
:::note
|
||||
- In Benchmark v1.24 and later, check id `4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Automated)` might fail, as `/etc/kubernetes/ssl/kube-ca.pem` is set to 644 by default.
|
||||
- In Benchmark v1.7, the `--protect-kernel-defaults` (`4.2.6`) parameter isn't required anymore, and was removed by CIS.
|
||||
:::
|
||||
|
||||
For more details on how to evaluate a hardened RKE cluster against the official CIS benchmark, refer to the RKE self-assessment guides for specific Kubernetes and CIS benchmark versions.
|
||||
|
||||
## Host-level requirements
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes types in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
|
||||
A user account and group for the **etcd** service is required to be set up before installing RKE.
|
||||
|
||||
#### Create `etcd` user and group
|
||||
|
||||
To create the **etcd** user and group run the following console commands.
|
||||
The commands below use `52034` for **uid** and **gid** for example purposes.
|
||||
Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd --shell /usr/sbin/nologin
|
||||
```
|
||||
|
||||
When deploying RKE through its cluster configuration `config.yml` file, update the `uid` and `gid` of the `etcd` user:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
## Kubernetes runtime requirements
|
||||
|
||||
### Configure `default` Service Account
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod.
|
||||
Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account.
|
||||
The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including `default` and `kube-system` on a standard RKE install, the `default` service account must include this value:
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following configuration to a file called `account_update.yaml`.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`.
|
||||
Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `account_update.yaml` configuration to `default` service account in all namespaces.
|
||||
|
||||
### Configure Network Policy
|
||||
|
||||
#### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one compromised application attacking a neighboring application. Network segmentation is important to ensure that containers can communicate only with those they are supposed to. A network policy is a specification of how selections of pods are allowed to communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given namespace, all traffic not allowed by the policy is denied. However, if there are no network policies in a namespace all traffic will be allowed into and out of the pods in that namespace. To enforce network policies, a container network interface (CNI) plugin must be enabled. This guide uses [Canal](https://github.com/projectcalico/canal) to provide the policy enforcement. Additional information about CNI providers can be found [here](https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/).
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a **permissive** example is provided below. If you want to allow all traffic to all pods in a namespace (even if policies are added that cause some pods to be treated as “isolated”), you can create a policy that explicitly allows all traffic in that namespace. Save the following configuration as `default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/) about network policies can be found on the Kubernetes site.
|
||||
|
||||
:::caution
|
||||
This network policy is just an example and is not recommended for production use.
|
||||
:::
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to `chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `default-allow-all.yaml` configuration with the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a hardened setup when only a public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
|
||||
## Reference Hardened RKE `cluster.yml` Configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened installation of RKE. RKE [documentation](https://rancher.com/docs/rke/latest/en/installation/) provides additional details about the configuration items. This reference `cluster.yml` does not include the required `nodes` directive which will vary depending on your environment. Documentation for node configuration in RKE can be found [here](https://rancher.com/docs/rke/latest/en/config-options/nodes/).
|
||||
|
||||
The example `cluster.yml` configuration file contains an Admission Configuration policy in the `services.kube-api.admission_configuration` field. This [sample](../../psa-restricted-exemptions.md) policy contains the namespace exemptions necessary for an imported RKE cluster to run properly in Rancher, similar to Rancher's pre-defined [`rancher-restricted`](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) policy.
|
||||
|
||||
If you prefer to use RKE's default `restricted` policy, then leave the `services.kube-api.admission_configuration` field empty and set `services.pod_security_configuration` to `restricted`. See [the RKE docs](https://rke.docs.rancher.com/config-options/services/pod-security-admission) for more information.
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 and Newer" default>
|
||||
|
||||
:::note
|
||||
If you intend to import an RKE cluster into Rancher, please consult the [documentation](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for how to configure the PSA to exempt Rancher system namespaces.
|
||||
:::
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
nodes: []
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
# Leave `pod_security_configuration` out if you are setting a
|
||||
# custom policy in `admission_configuration`. Otherwise set
|
||||
# it to `restricted` to use RKE's pre-defined restricted policy,
|
||||
# and remove everything inside `admission_configuration` field.
|
||||
#
|
||||
# pod_security_configuration: restricted
|
||||
#
|
||||
admission_configuration:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: PodSecurity
|
||||
configuration:
|
||||
apiVersion: pod-security.admission.config.k8s.io/v1
|
||||
kind: PodSecurityConfiguration
|
||||
defaults:
|
||||
enforce: "restricted"
|
||||
enforce-version: "latest"
|
||||
audit: "restricted"
|
||||
audit-version: "latest"
|
||||
warn: "restricted"
|
||||
warn-version: "latest"
|
||||
exemptions:
|
||||
usernames: []
|
||||
runtimeClasses: []
|
||||
namespaces: [calico-apiserver,
|
||||
calico-system,
|
||||
cattle-alerting,
|
||||
cattle-csp-adapter-system,
|
||||
cattle-elemental-system,
|
||||
cattle-epinio-system,
|
||||
cattle-externalip-system,
|
||||
cattle-fleet-local-system,
|
||||
cattle-fleet-system,
|
||||
cattle-gatekeeper-system,
|
||||
cattle-global-data,
|
||||
cattle-global-nt,
|
||||
cattle-impersonation-system,
|
||||
cattle-istio,
|
||||
cattle-istio-system,
|
||||
cattle-logging,
|
||||
cattle-logging-system,
|
||||
cattle-monitoring-system,
|
||||
cattle-neuvector-system,
|
||||
cattle-prometheus,
|
||||
cattle-provisioning-capi-system,
|
||||
cattle-resources-system,
|
||||
cattle-sriov-system,
|
||||
cattle-system,
|
||||
cattle-ui-plugin-system,
|
||||
cattle-windows-gmsa-system,
|
||||
cert-manager,
|
||||
cis-operator-system,
|
||||
fleet-default,
|
||||
ingress-nginx,
|
||||
istio-system,
|
||||
kube-node-lease,
|
||||
kube-public,
|
||||
kube-system,
|
||||
longhorn-system,
|
||||
rancher-alerting-drivers,
|
||||
security-scan,
|
||||
tigera-operator]
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 and Older">
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
nodes: []
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
# Upstream Kubernetes restricted PSP policy
|
||||
# https://github.com/kubernetes/website/blob/564baf15c102412522e9c8fc6ef2b5ff5b6e766c/content/en/examples/policy/restricted-psp.yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted-noroot
|
||||
spec:
|
||||
privileged: false
|
||||
# Required to prevent escalations to root.
|
||||
allowPrivilegeEscalation: false
|
||||
requiredDropCapabilities:
|
||||
- ALL
|
||||
# Allow core volume types.
|
||||
volumes:
|
||||
- 'configMap'
|
||||
- 'emptyDir'
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
# Assume that ephemeral CSI drivers & persistentVolumes set up by the cluster admin are safe to use.
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
runAsUser:
|
||||
# Require the container to run without root privileges.
|
||||
rule: 'MustRunAsNonRoot'
|
||||
seLinux:
|
||||
# This policy assumes the nodes are using AppArmor rather than SELinux.
|
||||
rule: 'RunAsAny'
|
||||
supplementalGroups:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
fsGroup:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
readOnlyRootFilesystem: false
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted-noroot
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted-noroot
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Reference Hardened RKE Cluster Template Configuration
|
||||
|
||||
The reference RKE cluster template provides the minimum required configuration to achieve a hardened installation of Kubernetes. RKE templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher [documentation](../../../../getting-started/installation-and-upgrade/installation-and-upgrade.md) for additional information about installing RKE and its template details.
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 and Newer" default>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_admission_configuration_template_name: rancher-restricted
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # Define cluster name
|
||||
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 and Older">
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted-noroot
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # Define cluster name
|
||||
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Conclusion
|
||||
|
||||
If you have followed this guide, your RKE custom cluster provisioned by Rancher will be configured to pass the CIS Kubernetes Benchmark. You can review our RKE self-assessment guides to understand how we verified each of the benchmarks and how you can do the same on your cluster.
|
||||
+2864
File diff suppressed because it is too large
Load Diff
+357
@@ -0,0 +1,357 @@
|
||||
---
|
||||
title: RKE 集群配置参考
|
||||
---
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
Rancher 安装 Kubernetes 时，它使用 [RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) 或 [RKE2](https://docs.rke2.io/) 作为 Kubernetes 发行版。
|
||||
|
||||
本文介绍 Rancher 中可用于新的或现有的 RKE Kubernetes 集群的配置选项。
|
||||
|
||||
|
||||
## 概述
|
||||
|
||||
你可以通过以下两种方式之一来配置 Kubernetes 选项：
|
||||
|
||||
- [Rancher UI](#rancher-ui-中的配置选项)：使用 Rancher UI 来选择设置 Kubernetes 集群时常用的自定义选项。
|
||||
- [集群配置文件](#rke-集群配置文件参考)：高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你使用 YAML 来指定 RKE 安装中可用的任何选项（除了 system_images 配置）。
|
||||
|
||||
RKE 集群配置选项嵌套在 `rancher_kubernetes_engine_config` 参数下。有关详细信息，请参阅[集群配置文件](#rke-集群配置文件参考)。
|
||||
|
||||
在 [RKE 启动的集群](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md)中，你可以编辑任何后续剩余的选项。
|
||||
|
||||
有关 RKE 配置文件语法的示例，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/example-yamls/)。
|
||||
|
||||
Rancher UI 中的表单不包括配置 RKE 的所有高级选项。有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
## 在 Rancher UI 中使用表单编辑集群
|
||||
|
||||
要编辑你的集群：
|
||||
|
||||
1. 在左上角，单击 **☰ > 集群管理**。
|
||||
1. 转到要配置的集群，然后单击 **⋮ > 编辑配置**。
|
||||
|
||||
|
||||
## 使用 YAML 编辑集群
|
||||
|
||||
高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你使用 YAML 来指定 RKE 安装中可用的任何选项（除了 system_images 配置）。
|
||||
|
||||
RKE 集群（也称为 RKE1 集群）的编辑方式与 RKE2 和 K3s 集群不同。
|
||||
|
||||
要直接从 Rancher UI 编辑 RKE 配置文件：
|
||||
|
||||
1. 点击 **☰ > 集群管理**。
|
||||
1. 转到要配置的 RKE 集群。单击并单击 **⋮ > 编辑配置**。你将会转到 RKE 配置表单。请注意，由于集群配置在 Rancher 2.6 中发生了变更，**⋮ > 以 YAML 文件编辑**可用于配置 RKE2 集群，但不能用于编辑 RKE1 配置。
|
||||
1. 在配置表单中，向下滚动并单击**以 YAML 文件编辑**。
|
||||
1. 编辑 `rancher_kubernetes_engine_config` 参数下的 RKE 选项。
|
||||
|
||||
## Rancher UI 中的配置选项
|
||||
|
||||
:::tip
|
||||
|
||||
一些高级配置选项没有在 Rancher UI 表单中开放，但你可以通过在 YAML 中编辑 RKE 集群配置文件来启用这些选项。有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
:::
|
||||
|
||||
### Kubernetes 版本
|
||||
|
||||
这指的是集群节点上安装的 Kubernetes 版本。Rancher 基于 [hyperkube](https://github.com/rancher/hyperkube) 打包了自己的 Kubernetes 版本。
|
||||
|
||||
有关更多详细信息，请参阅[升级 Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md)。
|
||||
|
||||
### 网络提供商
|
||||
|
||||
这指的是集群使用的[网络提供商](https://kubernetes.io/docs/concepts/cluster-administration/networking/)。有关不同网络提供商的更多详细信息，请查看我们的[网络常见问题解答](../../../faq/container-network-interface-providers.md)。
|
||||
|
||||
:::caution
|
||||
|
||||
启动集群后，你无法更改网络提供商。由于 Kubernetes 不允许在网络提供商之间切换，因此，请谨慎选择要使用的网络提供商。使用网络提供商创建集群后，如果你需要更改网络提供商，你将需要拆除整个集群以及其中的所有应用。
|
||||
|
||||
:::
|
||||
|
||||
Rancher 与以下开箱即用的网络提供商兼容：
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
:::note Weave 注意事项：
|
||||
|
||||
选择 Weave 作为网络提供商时，Rancher 将通过生成随机密码来自动启用加密。如果你想手动指定密码，请参阅使用[配置文件](#rke-集群配置文件参考)和 [Weave 网络插件选项](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options)来配置集群。
|
||||
|
||||
:::
|
||||
|
||||
### 项目网络隔离
|
||||
|
||||
如果你的网络提供商允许项目网络隔离，你可以选择启用或禁用项目间的通信。
|
||||
|
||||
如果你使用支持执行 Kubernetes 网络策略的 RKE 网络插件（例如 Canal 或 Cisco ACI 插件），则可以使用项目网络隔离。
|
||||
|
||||
### Kubernetes 云提供商
|
||||
|
||||
你可以配置 [Kubernetes 云提供商](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md)。如果你想在 Kubernetes 中使用动态配置的[卷和存储](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md)，你通常需要选择特定的云提供商。例如，如果你想使用 Amazon EBS，则需要选择 `aws` 云提供商。
|
||||
|
||||
:::note
|
||||
|
||||
如果你要使用的云提供商未作为选项列出，你需要使用[配置文件选项](#rke-集群配置文件参考)来配置云提供商。请参考 [RKE 云提供商文档](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/)来了解如何配置云提供商。
|
||||
|
||||
:::
|
||||
|
||||
### 私有镜像仓库
|
||||
|
||||
集群级别的私有镜像仓库配置仅能用于配置集群。
|
||||
|
||||
在 Rancher 中设置私有镜像仓库的主要方法有两种：通过[全局默认镜像仓库](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md)中的**设置**选项卡设置全局默认镜像仓库，以及在集群级别设置的高级选项中设置私有镜像仓库。全局默认镜像仓库可以用于离线设置，不需要凭证的镜像仓库。而集群级私有镜像仓库用于所有需要凭证的私有镜像仓库。
|
||||
|
||||
如果你的私有镜像仓库需要凭证，为了将凭证传递给 Rancher，你需要编辑每个需要从仓库中拉取镜像的集群的集群选项。
|
||||
|
||||
私有镜像仓库的配置选项能让 Rancher 知道要从哪里拉取用于集群的[系统镜像](https://rancher.com/docs/rke/latest/en/config-options/system-images/)或[附加组件镜像](https://rancher.com/docs/rke/latest/en/config-options/add-ons/)。
|
||||
|
||||
- **系统镜像**是维护 Kubernetes 集群所需的组件。
|
||||
- **附加组件**用于部署多个集群组件，包括网络插件、ingress controller、DNS 提供商或 metrics server。
|
||||
|
||||
有关为集群配置期间应用的组件设置私有镜像仓库的更多信息，请参阅[私有镜像仓库的 RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/private-registries/)。
|
||||
|
||||
Rancher v2.6 引入了[为 RKE 集群配置 ECR 镜像仓库](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup)的功能。
|
||||
|
||||
### 授权集群端点
|
||||
|
||||
授权集群端点（ACE）可用于直接访问 Kubernetes API server，而无需通过 Rancher 进行通信。
|
||||
|
||||
:::note
|
||||
|
||||
授权集群端点仅适用于 Rancher 启动的 Kubernetes 集群，即只适用于 Rancher [使用 RKE](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#配置-kubernetes-集群的工具) 来配置的集群。它不适用于托管在 Kubernetes 提供商中的集群，例如 Amazon 的 EKS。
|
||||
|
||||
:::
|
||||
|
||||
在 Rancher 启动的 Kubernetes 集群中，它默认启用，使用具有 `controlplane` 角色的节点的 IP 和默认的 Kubernetes 自签名证书。
|
||||
|
||||
有关授权集群端点的工作原理以及使用的原因，请参阅[架构介绍](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-授权集群端点)。
|
||||
|
||||
我们建议使用具有授权集群端点的负载均衡器。有关详细信息，请参阅[推荐的架构](../../rancher-manager-architecture/architecture-recommendations.md#授权集群端点架构)。
|
||||
|
||||
### 节点池
|
||||
|
||||
有关使用 Rancher UI 在 RKE 集群中设置节点池的信息，请参阅[此页面](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md)。
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
如果你想使用高可用性配置来发布应用，并且你使用没有原生负载均衡功能的云提供商来托管主机，请启用此选项以在集群中使用 NGINX Ingress。
|
||||
|
||||
### Metrics Server 监控
|
||||
|
||||
这是启用或禁用 [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/) 的选项。
|
||||
|
||||
每个能够使用 RKE 启动集群的云提供商都可以收集指标并监控你的集群节点。如果启用此选项，你可以从你的云提供商门户查看你的节点指标。
|
||||
|
||||
### 节点上的 Docker 版本
|
||||
|
||||
表示是否允许节点运行 Rancher 不正式支持的 Docker 版本。
|
||||
|
||||
如果你选择使用支持的 Docker 版本，Rancher 会禁止 pod 运行在安装了不支持的 Docker 版本的节点上。
|
||||
|
||||
如需了解各个 Rancher 版本通过了哪些 Docker 版本测试，请参见[支持和维护条款](https://rancher.com/support-maintenance-terms/)。
|
||||
|
||||
### Docker 根目录
|
||||
|
||||
如果要添加到集群的节点为 Docker 配置了非默认 Docker 根目录（默认为 `/var/lib/docker`），请在此选项中指定正确的 Docker 根目录。
|
||||
|
||||
### 默认 Pod 安全策略
|
||||
|
||||
如果你启用了 **Pod 安全策略支持**，请使用此下拉菜单选择应用于集群的 pod 安全策略。
|
||||
|
||||
### 节点端口范围
|
||||
|
||||
更改可用于 [NodePort 服务](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport)的端口范围的选项。默认为 `30000-32767`。
|
||||
|
||||
### 定期 etcd 快照
|
||||
|
||||
启用或禁用[定期 etcd 快照](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots)的选项。
|
||||
|
||||
### Agent 环境变量
|
||||
|
||||
为 [rancher agent](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md) 设置环境变量的选项。你可以使用键值对设置环境变量。如果 Rancher Agent 需要使用代理与 Rancher Server 通信，则可以使用 Agent 环境变量设置 `HTTP_PROXY`，`HTTPS_PROXY` 和 `NO_PROXY` 环境变量。
|
||||
|
||||
### 更新 ingress-nginx
|
||||
|
||||
使用 Kubernetes 1.16 之前版本创建的集群将具有 `OnDelete`的 `ingress-nginx` `updateStrategy`。使用 Kubernetes 1.16 或更高版本创建的集群将具有 `RollingUpdate`。
|
||||
|
||||
如果 `ingress-nginx` 的 `updateStrategy` 是 `OnDelete`，则需要删除这些 pod 以获得 deployment 正确的版本。
|
||||
|
||||
### Cluster Agent 配置和 Fleet Agent 配置
|
||||
|
||||
你可以为 Cluster Agent 和集群的 Fleet Agent 配置调度字段和资源限制。你可以使用这些字段来自定义容忍度、亲和性规则和资源要求。其他容忍度会被尾附到默认容忍度和 Control Plane 节点污点的列表中。如果你定义了自定义亲和性规则，它们将覆盖全局默认亲和性设置。定义资源要求会在以前没有的地方设置请求或限制。
|
||||
|
||||
:::note
|
||||
|
||||
有了这个选项，你可以覆盖或删除运行集群所需的规则。我们强烈建议你不要删除或覆盖这些规则和其他亲和性规则，因为这可能会导致不必要的影响：
|
||||
|
||||
- `affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution` 用于 `cattle-cluster-agent`
|
||||
- `cluster-agent-default-affinity` 用于 `cattle-cluster-agent`
|
||||
- `fleet-agent-default-affinity` 用于 `fleet-agent`
|
||||
|
||||
:::
|
||||
|
||||
如果将 Rancher 降级到 v2.7.4 或更低版本，你的更改将丢失，而且 Agent 将在没有你的自定义设置的情况下重新部署。重新部署时，Fleet Agent 将回退到使用内置默认值。如果降级期间 Fleet 版本没有更改，则不会立即重新部署。
|
||||
|
||||
|
||||
## RKE 集群配置文件参考
|
||||
|
||||
高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你在 RKE 安装中设置任何[可用选项](https://rancher.com/docs/rke/latest/en/config-options/)（`system_images` 配置除外）。使用 Rancher UI 或 API 创建集群时，不支持 `system_images` 选项。
|
||||
|
||||
有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
### Rancher 中的配置文件结构
|
||||
|
||||
RKE（Rancher Kubernetes Engine）是 Rancher 用来配置 Kubernetes 集群的工具。过去，Rancher 的集群配置文件与 [RKE 配置文件](https://rancher.com/docs/rke/latest/en/example-yamls/)的结构是一致的。但由于 Rancher 文件结构发生了变化，因此在 Rancher 中，RKE 集群配置项与非 RKE 配置项是分开的。所以，你的集群配置需要嵌套在集群配置文件中的 `rancher_kubernetes_engine_config` 参数下。使用早期版本的 Rancher 创建的集群配置文件需要针对这种格式进行更新。以下是一个集群配置文件示例：
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>集群配置文件示例</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # 目前仅支持 Nginx ingress provider
|
||||
# # 要禁用 Ingress controller，设置 `provider: none`
|
||||
# # 要在指定节点上禁用 Ingress，使用 node_selector，例如：
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### 默认 DNS 提供商
|
||||
|
||||
下表显示了默认部署的 DNS 提供商。有关如何配置不同 DNS 提供商的更多信息，请参阅 [DNS 提供商相关的 RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/)。CoreDNS 只能在 Kubernetes v1.12.0 及更高版本上使用。
|
||||
|
||||
| Rancher 版本 | Kubernetes 版本 | 默认 DNS 提供商 |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 及更高版本 | v1.14.0 及更高版本 | CoreDNS |
|
||||
| v2.2.5 及更高版本 | v1.13.x 及更低版本 | kube-dns |
|
||||
| v2.2.4 及更低版本 | 任意 | kube-dns |
|
||||
|
||||
## YAML 中的 Rancher 特定参数
|
||||
|
||||
除了 RKE 配置文件选项外，还有可以在配置文件 (YAML) 中配置的 Rancher 特定设置如下。
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
请参阅 [Docker 根目录](#docker-根目录)。
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
启用或禁用[集群监控](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md)的选项。
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
启用或禁用项目网络隔离的选项。
|
||||
|
||||
如果你使用支持执行 Kubernetes 网络策略的 RKE 网络插件（例如 Canal 或 Cisco ACI 插件），则可以使用项目网络隔离。
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
请参阅[授权集群端点](#授权集群端点)。
|
||||
|
||||
示例：
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### 自定义网络插件
|
||||
|
||||
你可以使用 RKE 的[用户定义的附加组件功能](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/)来添加自定义网络插件。部署 Kubernetes 集群之后，你可以定义要部署的任何附加组件。
|
||||
|
||||
有两种方法可以指定附加组件：
|
||||
|
||||
- [内嵌附加组件](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [为附加组件引用 YAML 文件](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
有关如何通过编辑 `cluster.yml` 来配置自定义网络插件的示例，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)。
|
||||
+516
@@ -0,0 +1,516 @@
|
||||
---
|
||||
title: RKE 加固指南
|
||||
---
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/zh/reference-guides/rancher-security/hardening-guides/rke1-hardening-guide"/>
|
||||
</head>
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
本文档提供了针对生产环境的 RKE 集群进行加固的具体指导，以便在使用 Rancher 部署之前进行配置。它概述了满足信息安全中心（Center for Information Security, CIS）Kubernetes benchmark controls 所需的配置和控制。
|
||||
|
||||
:::note
|
||||
这份加固指南描述了如何确保你集群中的节点安全。我们建议你在安装 Kubernetes 之前遵循本指南。
|
||||
:::
|
||||
|
||||
此加固指南适用于 RKE 集群，并与以下版本的 CIS Kubernetes Benchmark、Kubernetes 和 Rancher 相关联：
|
||||
|
||||
| Rancher 版本 | CIS Benchmark 版本 | Kubernetes 版本 |
|
||||
|-----------------|-----------------------|------------------------------|
|
||||
| Rancher v2.7 | Benchmark v1.23 | Kubernetes v1.23 |
|
||||
| Rancher v2.7 | Benchmark v1.24 | Kubernetes v1.24 |
|
||||
| Rancher v2.7 | Benchmark v1.7 | Kubernetes v1.25 至 v1.26 |
|
||||
|
||||
:::note
|
||||
- 在 Benchmark v1.24 及更高版本中，检查 id `4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Automated)` 可能会失败，因为 `/etc/kubernetes/ssl/kube-ca.pem` 默认设置为 644。
|
||||
- 在 Benchmark v1.7 中，不再需要 `--protect-kernel-defaults` (`4.2.6`) 参数，并已被 CIS 删除。
|
||||
:::
|
||||
|
||||
有关如何评估加固的 RKE 集群与官方 CIS benchmark 的更多细节，请参考特定 Kubernetes 和 CIS benchmark 版本的 RKE 自我评估指南。
|
||||
|
||||
## 主机级别要求
|
||||
|
||||
### 配置 Kernel 运行时参数
|
||||
|
||||
建议对群集中的所有节点类型使用以下 `sysctl` 配置。在 `/etc/sysctl.d/90-kubelet.conf` 中设置以下参数：
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
|
||||
运行 `sysctl -p /etc/sysctl.d/90-kubelet.conf` 以启用设置。
|
||||
|
||||
### 配置 `etcd` 用户和组
|
||||
|
||||
在安装 RKE 之前，需要设置 **etcd** 服务的用户帐户和组。
|
||||
|
||||
#### 创建 `etcd` 用户和组
|
||||
|
||||
要创建 **etcd** 用户和组，请运行以下控制台命令。
|
||||
下面的命令示例中使用 `52034` 作为 **uid** 和 **gid** 。
|
||||
任何有效且未使用的 **uid** 或 **gid** 都可以代替 `52034`。
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd --shell /usr/sbin/nologin
|
||||
```
|
||||
|
||||
在通过集群配置文件 `config.yml` 部署RKE时，请更新 `etcd` 用户的 `uid` 和 `gid`：
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
## Kubernetes 运行时要求
|
||||
|
||||
### 配置 `default` Service Account
|
||||
|
||||
#### 设置 `automountServiceAccountToken` 为 `false` 用于 `default` service accounts
|
||||
|
||||
Kubernetes 提供了一个 default service account，供集群工作负载使用，其中没有为 pod 分配特定的 service account。
|
||||
如果需要从 pod 访问 Kubernetes API，则应为该 pod 创建特定的 service account，并向该 service account 授予权限。
|
||||
应配置 default service account，使其不提供 service account 令牌，并且不应具有任何明确的权限分配。
|
||||
|
||||
对于标准 RKE 安装上的每个命名空间（包括 `default` 和 `kube-system`），`default` service account 必须包含以下值：
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
将以下配置保存到名为 `account_update.yaml` 的文件中。
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
创建一个名为 `account_update.yaml` 的 bash 脚本文件。
|
||||
确保执行 `chmod +x account_update.sh` 命令，以赋予脚本执行权限。
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
执行此脚本将 `account_update.yaml` 配置应用到所有命名空间中的 `default` service account。
|
||||
|
||||
### 配置网络策略
|
||||
|
||||
#### 确保所有命名空间都定义了网络策略
|
||||
|
||||
在同一个 Kubernetes 集群上运行不同的应用程序会带来风险，即某个受感染的应用程序可能会攻击相邻的应用程序。为确保容器只与其预期通信的容器进行通信，网络分段至关重要。网络策略规定了哪些 Pod 可以互相通信，以及与其他网络终端通信的方式。
|
||||
|
||||
网络策略是命名空间范围的。当在特定命名空间引入网络策略时，所有未被策略允许的流量将被拒绝。然而，如果在命名空间中没有网络策略，那么所有流量将被允许进入和离开该命名空间中的 Pod。要强制执行网络策略，必须启用容器网络接口（container network interface, CNI）插件。本指南使用 [Canal](https://github.com/projectcalico/canal) 来提供策略执行。有关 CNI 提供程序的其他信息可以在[这里](https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)找到。
|
||||
|
||||
一旦在集群上启用了 CNI 提供程序，就可以应用默认的网络策略。下面提供了一个 **permissive** 的示例供参考。如果你希望允许匹配某个命名空间中所有 Pod 的所有入站和出站流量（即使添加了策略导致某些 Pod 被视为”隔离”），你可以创建一个明确允许该命名空间中所有流量的策略。请将以下配置保存为 `default-allow-all.yaml`。有关网络策略的其他[文档](https://kubernetes.io/docs/concepts/services-networking/network-policies/)可以在 Kubernetes 站点上找到。
|
||||
|
||||
:::caution
|
||||
此网络策略只是一个示例，不建议用于生产用途。
|
||||
:::
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
创建一个名为 `apply_networkPolicy_to_all_ns.sh`的 Bash 脚本文件。
|
||||
|
||||
确保运行 `chmod +x apply_networkPolicy_to_all_ns.sh` 命令，以赋予脚本执行权限。
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
执行此脚本以将 `default-allow-all.yaml` 配置和 **permissive** 的 `NetworkPolicy` 应用于所有命名空间。
|
||||
|
||||
## 已知限制
|
||||
|
||||
- 当注册自定义节点仅提供公共 IP 时，Rancher **exec shell** 和 **查看 pod 日志** 在加固设置中**不起作用**。 此功能需要在注册自定义节点时提供私有 IP。
|
||||
|
||||
## 加固的 RKE `cluster.yml` 配置参考
|
||||
|
||||
参考的 `cluster.yml` 文件是由 RKE CLI 使用的，它提供了实现 RKE 加固安装所需的配置。
|
||||
RKE [文档](https://rancher.com/docs/rke/latest/en/installation/)提供了有关配置项的更多详细信息。这里参考的 `cluster.yml` 不包括必需的 `nodes` 指令，因为它取决于你的环境。在 RKE 中有关节点配置的文档可以在[这里](https://rancher.com/docs/rke/latest/en/config-options/nodes/)找到。
|
||||
|
||||
示例 `cluster.yml` 配置文件中包含了一个 Admission Configuration 策略，在 `services.kube-api.admission_configuration` 字段中指定。这个[示例](../../psa-restricted-exemptions.md)策略包含了命名空间的豁免规则，这对于在Rancher中正确运行导入的RKE集群非常必要，类似于Rancher预定义的 [`rancher-restricted`](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) 策略。
|
||||
|
||||
如果你希望使用 RKE 的默认 `restricted` 策略，则将 `services.kube-api.admission_configuration` 字段留空，并将 `services.pod_security_configuration` 设置为 `restricted`。你可以在 [RKE 文档](https://rke.docs.rancher.com/config-options/services/pod-security-admission)中找到更多信息。
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 及更新版本" default>
|
||||
|
||||
:::note
|
||||
如果你打算将一个 RKE 集群导入到 Rancher 中，请参考此[文档](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md)以了解如何配置 PSA 以豁免 Rancher 系统命名空间。
|
||||
:::
|
||||
|
||||
```yaml
|
||||
# 如果你打算在离线环境部署 Kubernetes，
|
||||
# 请查阅文档以了解如何配置自定义的 RKE 镜像。
|
||||
nodes: []
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
# 如果你在 `admission_configuration` 中设置了自定义策略，
|
||||
# 请将 `pod_security_configuration` 字段留空。
|
||||
# 否则，将其设置为 `restricted` 以使用 RKE 预定义的受限策略，
|
||||
# 并删除 `admission_configuration` 字段中的所有内容。
|
||||
#
|
||||
# pod_security_configuration: restricted
|
||||
#
|
||||
admission_configuration:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: PodSecurity
|
||||
configuration:
|
||||
apiVersion: pod-security.admission.config.k8s.io/v1
|
||||
kind: PodSecurityConfiguration
|
||||
defaults:
|
||||
enforce: "restricted"
|
||||
enforce-version: "latest"
|
||||
audit: "restricted"
|
||||
audit-version: "latest"
|
||||
warn: "restricted"
|
||||
warn-version: "latest"
|
||||
exemptions:
|
||||
usernames: []
|
||||
runtimeClasses: []
|
||||
namespaces: [calico-apiserver,
|
||||
calico-system,
|
||||
cattle-alerting,
|
||||
cattle-csp-adapter-system,
|
||||
cattle-elemental-system,
|
||||
cattle-epinio-system,
|
||||
cattle-externalip-system,
|
||||
cattle-fleet-local-system,
|
||||
cattle-fleet-system,
|
||||
cattle-gatekeeper-system,
|
||||
cattle-global-data,
|
||||
cattle-global-nt,
|
||||
cattle-impersonation-system,
|
||||
cattle-istio,
|
||||
cattle-istio-system,
|
||||
cattle-logging,
|
||||
cattle-logging-system,
|
||||
cattle-monitoring-system,
|
||||
cattle-neuvector-system,
|
||||
cattle-prometheus,
|
||||
cattle-provisioning-capi-system,
|
||||
cattle-resources-system,
|
||||
cattle-sriov-system,
|
||||
cattle-system,
|
||||
cattle-ui-plugin-system,
|
||||
cattle-windows-gmsa-system,
|
||||
cert-manager,
|
||||
cis-operator-system,
|
||||
fleet-default,
|
||||
ingress-nginx,
|
||||
istio-system,
|
||||
kube-node-lease,
|
||||
kube-public,
|
||||
kube-system,
|
||||
longhorn-system,
|
||||
rancher-alerting-drivers,
|
||||
security-scan,
|
||||
tigera-operator]
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 及更早版本">
|
||||
|
||||
```yaml
|
||||
# 如果你打算在离线环境部署 Kubernetes，
|
||||
# 请查阅文档以了解如何配置自定义的 RKE 镜像。
|
||||
nodes: []
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
# Upstream Kubernetes restricted PSP policy
|
||||
# https://github.com/kubernetes/website/blob/564baf15c102412522e9c8fc6ef2b5ff5b6e766c/content/en/examples/policy/restricted-psp.yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted-noroot
|
||||
spec:
|
||||
privileged: false
|
||||
# Required to prevent escalations to root.
|
||||
allowPrivilegeEscalation: false
|
||||
requiredDropCapabilities:
|
||||
- ALL
|
||||
# Allow core volume types.
|
||||
volumes:
|
||||
- 'configMap'
|
||||
- 'emptyDir'
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
# Assume that ephemeral CSI drivers & persistentVolumes set up by the cluster admin are safe to use.
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
runAsUser:
|
||||
# Require the container to run without root privileges.
|
||||
rule: 'MustRunAsNonRoot'
|
||||
seLinux:
|
||||
# This policy assumes the nodes are using AppArmor rather than SELinux.
|
||||
rule: 'RunAsAny'
|
||||
supplementalGroups:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
fsGroup:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
readOnlyRootFilesystem: false
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted-noroot
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted-noroot
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## 加固后的 RKE 集群模板配置参考
|
||||
|
||||
参考的 RKE 集群模板提供了实现 Kubernetes 加固安装所需的最低配置。RKE 模板用于提供 Kubernetes 并定义 Rancher 设置。有关安装 RKE 及其模板详情的其他信息，请参考 Rancher [文档](../../../../getting-started/installation-and-upgrade/installation-and-upgrade.md) 。
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 及更新版本" default>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# 集群配置
|
||||
#
|
||||
default_pod_security_admission_configuration_template_name: rancher-restricted
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # 定义集群名称
|
||||
|
||||
#
|
||||
# Rancher 配置
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 及更早版本">
|
||||
|
||||
```yaml
|
||||
#
|
||||
# 集群配置
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted-noroot
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # 定义集群名称
|
||||
|
||||
#
|
||||
# Rancher 配置
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## 结论
|
||||
|
||||
如果你按照本指南操作，由 Rancher 提供的 RKE 自定义集群将配置为通过 CIS Kubernetes Benchmark 测试。你可以查看我们的 RKE 自我评估指南，了解我们是如何验证每个 benchmarks 的，并且你可以在你的集群上执行相同的操作。
|
||||
+2863
File diff suppressed because it is too large
Load Diff
+357
@@ -0,0 +1,357 @@
|
||||
---
|
||||
title: RKE 集群配置参考
|
||||
---
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
Rancher 安装 Kubernetes 时，它使用 [RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) 或 [RKE2](https://docs.rke2.io/) 作为 Kubernetes 发行版。
|
||||
|
||||
本文介绍 Rancher 中可用于新的或现有的 RKE Kubernetes 集群的配置选项。
|
||||
|
||||
|
||||
## 概述
|
||||
|
||||
你可以通过以下两种方式之一来配置 Kubernetes 选项：
|
||||
|
||||
- [Rancher UI](#rancher-ui-中的配置选项)：使用 Rancher UI 来选择设置 Kubernetes 集群时常用的自定义选项。
|
||||
- [集群配置文件](#rke-集群配置文件参考)：高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你使用 YAML 来指定 RKE 安装中可用的任何选项（除了 system_images 配置）。
|
||||
|
||||
RKE 集群配置选项嵌套在 `rancher_kubernetes_engine_config` 参数下。有关详细信息，请参阅[集群配置文件](#rke-集群配置文件参考)。
|
||||
|
||||
在 [RKE 启动的集群](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md)中，你可以编辑任何后续剩余的选项。
|
||||
|
||||
有关 RKE 配置文件语法的示例，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/example-yamls/)。
|
||||
|
||||
Rancher UI 中的表单不包括配置 RKE 的所有高级选项。有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
## 在 Rancher UI 中使用表单编辑集群
|
||||
|
||||
要编辑你的集群：
|
||||
|
||||
1. 在左上角，单击 **☰ > 集群管理**。
|
||||
1. 转到要配置的集群，然后单击 **⋮ > 编辑配置**。
|
||||
|
||||
|
||||
## 使用 YAML 编辑集群
|
||||
|
||||
高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你使用 YAML 来指定 RKE 安装中可用的任何选项（除了 system_images 配置）。
|
||||
|
||||
RKE 集群（也称为 RKE1 集群）的编辑方式与 RKE2 和 K3s 集群不同。
|
||||
|
||||
要直接从 Rancher UI 编辑 RKE 配置文件：
|
||||
|
||||
1. 点击 **☰ > 集群管理**。
|
||||
1. 转到要配置的 RKE 集群。单击并单击 **⋮ > 编辑配置**。你将会转到 RKE 配置表单。请注意，由于集群配置在 Rancher 2.6 中发生了变更，**⋮ > 以 YAML 文件编辑**可用于配置 RKE2 集群，但不能用于编辑 RKE1 配置。
|
||||
1. 在配置表单中，向下滚动并单击**以 YAML 文件编辑**。
|
||||
1. 编辑 `rancher_kubernetes_engine_config` 参数下的 RKE 选项。
|
||||
|
||||
## Rancher UI 中的配置选项
|
||||
|
||||
:::tip
|
||||
|
||||
一些高级配置选项没有在 Rancher UI 表单中开放，但你可以通过在 YAML 中编辑 RKE 集群配置文件来启用这些选项。有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
:::
|
||||
|
||||
### Kubernetes 版本
|
||||
|
||||
这指的是集群节点上安装的 Kubernetes 版本。Rancher 基于 [hyperkube](https://github.com/rancher/hyperkube) 打包了自己的 Kubernetes 版本。
|
||||
|
||||
有关更多详细信息，请参阅[升级 Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md)。
|
||||
|
||||
### 网络提供商
|
||||
|
||||
这指的是集群使用的[网络提供商](https://kubernetes.io/docs/concepts/cluster-administration/networking/)。有关不同网络提供商的更多详细信息，请查看我们的[网络常见问题解答](../../../faq/container-network-interface-providers.md)。
|
||||
|
||||
:::caution
|
||||
|
||||
启动集群后，你无法更改网络提供商。由于 Kubernetes 不允许在网络提供商之间切换，因此，请谨慎选择要使用的网络提供商。使用网络提供商创建集群后，如果你需要更改网络提供商，你将需要拆除整个集群以及其中的所有应用。
|
||||
|
||||
:::
|
||||
|
||||
Rancher 与以下开箱即用的网络提供商兼容：
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
:::note Weave 注意事项：
|
||||
|
||||
选择 Weave 作为网络提供商时，Rancher 将通过生成随机密码来自动启用加密。如果你想手动指定密码，请参阅使用[配置文件](#rke-集群配置文件参考)和 [Weave 网络插件选项](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options)来配置集群。
|
||||
|
||||
:::
|
||||
|
||||
### 项目网络隔离
|
||||
|
||||
如果你的网络提供商允许项目网络隔离，你可以选择启用或禁用项目间的通信。
|
||||
|
||||
如果你使用支持执行 Kubernetes 网络策略的 RKE 网络插件（例如 Canal 或 Cisco ACI 插件），则可以使用项目网络隔离。
|
||||
|
||||
### Kubernetes 云提供商
|
||||
|
||||
你可以配置 [Kubernetes 云提供商](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md)。如果你想在 Kubernetes 中使用动态配置的[卷和存储](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md)，你通常需要选择特定的云提供商。例如，如果你想使用 Amazon EBS，则需要选择 `aws` 云提供商。
|
||||
|
||||
:::note
|
||||
|
||||
如果你要使用的云提供商未作为选项列出，你需要使用[配置文件选项](#rke-集群配置文件参考)来配置云提供商。请参考 [RKE 云提供商文档](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/)来了解如何配置云提供商。
|
||||
|
||||
:::
|
||||
|
||||
### 私有镜像仓库
|
||||
|
||||
集群级别的私有镜像仓库配置仅能用于配置集群。
|
||||
|
||||
在 Rancher 中设置私有镜像仓库的主要方法有两种：通过[全局默认镜像仓库](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md)中的**设置**选项卡设置全局默认镜像仓库，以及在集群级别设置的高级选项中设置私有镜像仓库。全局默认镜像仓库可以用于离线设置，不需要凭证的镜像仓库。而集群级私有镜像仓库用于所有需要凭证的私有镜像仓库。
|
||||
|
||||
如果你的私有镜像仓库需要凭证，为了将凭证传递给 Rancher，你需要编辑每个需要从仓库中拉取镜像的集群的集群选项。
|
||||
|
||||
私有镜像仓库的配置选项能让 Rancher 知道要从哪里拉取用于集群的[系统镜像](https://rancher.com/docs/rke/latest/en/config-options/system-images/)或[附加组件镜像](https://rancher.com/docs/rke/latest/en/config-options/add-ons/)。
|
||||
|
||||
- **系统镜像**是维护 Kubernetes 集群所需的组件。
|
||||
- **附加组件**用于部署多个集群组件，包括网络插件、ingress controller、DNS 提供商或 metrics server。
|
||||
|
||||
有关为集群配置期间应用的组件设置私有镜像仓库的更多信息，请参阅[私有镜像仓库的 RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/private-registries/)。
|
||||
|
||||
Rancher v2.6 引入了[为 RKE 集群配置 ECR 镜像仓库](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup)的功能。
|
||||
|
||||
### 授权集群端点
|
||||
|
||||
授权集群端点（ACE）可用于直接访问 Kubernetes API server，而无需通过 Rancher 进行通信。
|
||||
|
||||
:::note
|
||||
|
||||
授权集群端点仅适用于 Rancher 启动的 Kubernetes 集群，即只适用于 Rancher [使用 RKE](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#配置-kubernetes-集群的工具) 来配置的集群。它不适用于托管在 Kubernetes 提供商中的集群，例如 Amazon 的 EKS。
|
||||
|
||||
:::
|
||||
|
||||
在 Rancher 启动的 Kubernetes 集群中，它默认启用，使用具有 `controlplane` 角色的节点的 IP 和默认的 Kubernetes 自签名证书。
|
||||
|
||||
有关授权集群端点的工作原理以及使用的原因，请参阅[架构介绍](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-授权集群端点)。
|
||||
|
||||
我们建议使用具有授权集群端点的负载均衡器。有关详细信息，请参阅[推荐的架构](../../rancher-manager-architecture/architecture-recommendations.md#授权集群端点架构)。
|
||||
|
||||
### 节点池
|
||||
|
||||
有关使用 Rancher UI 在 RKE 集群中设置节点池的信息，请参阅[此页面](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md)。
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
如果你想使用高可用性配置来发布应用，并且你使用没有原生负载均衡功能的云提供商来托管主机，请启用此选项以在集群中使用 NGINX Ingress。
|
||||
|
||||
### Metrics Server 监控
|
||||
|
||||
这是启用或禁用 [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/) 的选项。
|
||||
|
||||
每个能够使用 RKE 启动集群的云提供商都可以收集指标并监控你的集群节点。如果启用此选项，你可以从你的云提供商门户查看你的节点指标。
|
||||
|
||||
### 节点上的 Docker 版本
|
||||
|
||||
表示是否允许节点运行 Rancher 不正式支持的 Docker 版本。
|
||||
|
||||
如果你选择使用支持的 Docker 版本，Rancher 会禁止 pod 运行在安装了不支持的 Docker 版本的节点上。
|
||||
|
||||
如需了解各个 Rancher 版本通过了哪些 Docker 版本测试，请参见[支持和维护条款](https://rancher.com/support-maintenance-terms/)。
|
||||
|
||||
### Docker 根目录
|
||||
|
||||
如果要添加到集群的节点为 Docker 配置了非默认 Docker 根目录（默认为 `/var/lib/docker`），请在此选项中指定正确的 Docker 根目录。
|
||||
|
||||
### 默认 Pod 安全策略
|
||||
|
||||
如果你启用了 **Pod 安全策略支持**，请使用此下拉菜单选择应用于集群的 pod 安全策略。
|
||||
|
||||
### 节点端口范围
|
||||
|
||||
更改可用于 [NodePort 服务](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport)的端口范围的选项。默认为 `30000-32767`。
|
||||
|
||||
### 定期 etcd 快照
|
||||
|
||||
启用或禁用[定期 etcd 快照](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots)的选项。
|
||||
|
||||
### Agent 环境变量
|
||||
|
||||
为 [rancher agent](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md) 设置环境变量的选项。你可以使用键值对设置环境变量。如果 Rancher Agent 需要使用代理与 Rancher Server 通信，则可以使用 Agent 环境变量设置 `HTTP_PROXY`，`HTTPS_PROXY` 和 `NO_PROXY` 环境变量。
|
||||
|
||||
### 更新 ingress-nginx
|
||||
|
||||
使用 Kubernetes 1.16 之前版本创建的集群将具有 `OnDelete`的 `ingress-nginx` `updateStrategy`。使用 Kubernetes 1.16 或更高版本创建的集群将具有 `RollingUpdate`。
|
||||
|
||||
如果 `ingress-nginx` 的 `updateStrategy` 是 `OnDelete`，则需要删除这些 pod 以获得 deployment 正确的版本。
|
||||
|
||||
### Cluster Agent 配置和 Fleet Agent 配置
|
||||
|
||||
你可以为 Cluster Agent 和集群的 Fleet Agent 配置调度字段和资源限制。你可以使用这些字段来自定义容忍度、亲和性规则和资源要求。其他容忍度会被尾附到默认容忍度和 Control Plane 节点污点的列表中。如果你定义了自定义亲和性规则，它们将覆盖全局默认亲和性设置。定义资源要求会在以前没有的地方设置请求或限制。
|
||||
|
||||
:::note
|
||||
|
||||
有了这个选项，你可以覆盖或删除运行集群所需的规则。我们强烈建议你不要删除或覆盖这些规则和其他亲和性规则，因为这可能会导致不必要的影响：
|
||||
|
||||
- `affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution` 用于 `cattle-cluster-agent`
|
||||
- `cluster-agent-default-affinity` 用于 `cattle-cluster-agent`
|
||||
- `fleet-agent-default-affinity` 用于 `fleet-agent`
|
||||
|
||||
:::
|
||||
|
||||
如果将 Rancher 降级到 v2.7.4 或更低版本，你的更改将丢失，而且 Agent 将在没有你的自定义设置的情况下重新部署。重新部署时，Fleet Agent 将回退到使用内置默认值。如果降级期间 Fleet 版本没有更改，则不会立即重新部署。
|
||||
|
||||
|
||||
## RKE 集群配置文件参考
|
||||
|
||||
高级用户可以创建一个 RKE 配置文件，而不是使用 Rancher UI 来为集群选择 Kubernetes 选项。配置文件可以让你在 RKE 安装中设置任何[可用选项](https://rancher.com/docs/rke/latest/en/config-options/)（`system_images` 配置除外）。使用 Rancher UI 或 API 创建集群时，不支持 `system_images` 选项。
|
||||
|
||||
有关 YAML 中 RKE Kubernetes 集群的可配置选项的完整参考，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/)。
|
||||
|
||||
### Rancher 中的配置文件结构
|
||||
|
||||
RKE（Rancher Kubernetes Engine）是 Rancher 用来配置 Kubernetes 集群的工具。过去，Rancher 的集群配置文件与 [RKE 配置文件](https://rancher.com/docs/rke/latest/en/example-yamls/)的结构是一致的。但由于 Rancher 文件结构发生了变化，因此在 Rancher 中，RKE 集群配置项与非 RKE 配置项是分开的。所以，你的集群配置需要嵌套在集群配置文件中的 `rancher_kubernetes_engine_config` 参数下。使用早期版本的 Rancher 创建的集群配置文件需要针对这种格式进行更新。以下是一个集群配置文件示例：
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>集群配置文件示例</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # 目前仅支持 Nginx ingress provider
|
||||
# # 要禁用 Ingress controller，设置 `provider: none`
|
||||
# # 要在指定节点上禁用 Ingress，使用 node_selector，例如：
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### 默认 DNS 提供商
|
||||
|
||||
下表显示了默认部署的 DNS 提供商。有关如何配置不同 DNS 提供商的更多信息，请参阅 [DNS 提供商相关的 RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/)。CoreDNS 只能在 Kubernetes v1.12.0 及更高版本上使用。
|
||||
|
||||
| Rancher 版本 | Kubernetes 版本 | 默认 DNS 提供商 |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 及更高版本 | v1.14.0 及更高版本 | CoreDNS |
|
||||
| v2.2.5 及更高版本 | v1.13.x 及更低版本 | kube-dns |
|
||||
| v2.2.4 及更低版本 | 任意 | kube-dns |
|
||||
|
||||
## YAML 中的 Rancher 特定参数
|
||||
|
||||
除了 RKE 配置文件选项外，还有可以在配置文件 (YAML) 中配置的 Rancher 特定设置如下。
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
请参阅 [Docker 根目录](#docker-根目录)。
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
启用或禁用[集群监控](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md)的选项。
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
启用或禁用项目网络隔离的选项。
|
||||
|
||||
如果你使用支持执行 Kubernetes 网络策略的 RKE 网络插件（例如 Canal 或 Cisco ACI 插件），则可以使用项目网络隔离。
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
请参阅[授权集群端点](#授权集群端点)。
|
||||
|
||||
示例：
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### 自定义网络插件
|
||||
|
||||
你可以使用 RKE 的[用户定义的附加组件功能](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/)来添加自定义网络插件。部署 Kubernetes 集群之后，你可以定义要部署的任何附加组件。
|
||||
|
||||
有两种方法可以指定附加组件：
|
||||
|
||||
- [内嵌附加组件](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [为附加组件引用 YAML 文件](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
有关如何通过编辑 `cluster.yml` 来配置自定义网络插件的示例，请参阅 [RKE 文档](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)。
|
||||
+516
@@ -0,0 +1,516 @@
|
||||
---
|
||||
title: RKE 加固指南
|
||||
---
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/zh/reference-guides/rancher-security/hardening-guides/rke1-hardening-guide"/>
|
||||
</head>
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
本文档提供了针对生产环境的 RKE 集群进行加固的具体指导，以便在使用 Rancher 部署之前进行配置。它概述了满足信息安全中心（Center for Information Security, CIS）Kubernetes benchmark controls 所需的配置和控制。
|
||||
|
||||
:::note
|
||||
这份加固指南描述了如何确保你集群中的节点安全。我们建议你在安装 Kubernetes 之前遵循本指南。
|
||||
:::
|
||||
|
||||
此加固指南适用于 RKE 集群，并与以下版本的 CIS Kubernetes Benchmark、Kubernetes 和 Rancher 相关联：
|
||||
|
||||
| Rancher 版本 | CIS Benchmark 版本 | Kubernetes 版本 |
|
||||
|-----------------|-----------------------|------------------------------|
|
||||
| Rancher v2.7 | Benchmark v1.23 | Kubernetes v1.23 |
|
||||
| Rancher v2.7 | Benchmark v1.24 | Kubernetes v1.24 |
|
||||
| Rancher v2.7 | Benchmark v1.7 | Kubernetes v1.25 至 v1.26 |
|
||||
|
||||
:::note
|
||||
- 在 Benchmark v1.24 及更高版本中，检查 id `4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Automated)` 可能会失败，因为 `/etc/kubernetes/ssl/kube-ca.pem` 默认设置为 644。
|
||||
- 在 Benchmark v1.7 中，不再需要 `--protect-kernel-defaults` (`4.2.6`) 参数，并已被 CIS 删除。
|
||||
:::
|
||||
|
||||
有关如何评估加固的 RKE 集群与官方 CIS benchmark 的更多细节，请参考特定 Kubernetes 和 CIS benchmark 版本的 RKE 自我评估指南。
|
||||
|
||||
## 主机级别要求
|
||||
|
||||
### 配置 Kernel 运行时参数
|
||||
|
||||
建议对群集中的所有节点类型使用以下 `sysctl` 配置。在 `/etc/sysctl.d/90-kubelet.conf` 中设置以下参数：
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
|
||||
运行 `sysctl -p /etc/sysctl.d/90-kubelet.conf` 以启用设置。
|
||||
|
||||
### 配置 `etcd` 用户和组
|
||||
|
||||
在安装 RKE 之前，需要设置 **etcd** 服务的用户帐户和组。
|
||||
|
||||
#### 创建 `etcd` 用户和组
|
||||
|
||||
要创建 **etcd** 用户和组，请运行以下控制台命令。
|
||||
下面的命令示例中使用 `52034` 作为 **uid** 和 **gid** 。
|
||||
任何有效且未使用的 **uid** 或 **gid** 都可以代替 `52034`。
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd --shell /usr/sbin/nologin
|
||||
```
|
||||
|
||||
在通过集群配置文件 `config.yml` 部署RKE时，请更新 `etcd` 用户的 `uid` 和 `gid`：
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
## Kubernetes 运行时要求
|
||||
|
||||
### 配置 `default` Service Account
|
||||
|
||||
#### 设置 `automountServiceAccountToken` 为 `false` 用于 `default` service accounts
|
||||
|
||||
Kubernetes 提供了一个 default service account，供集群工作负载使用，其中没有为 pod 分配特定的 service account。
|
||||
如果需要从 pod 访问 Kubernetes API，则应为该 pod 创建特定的 service account，并向该 service account 授予权限。
|
||||
应配置 default service account，使其不提供 service account 令牌，并且不应具有任何明确的权限分配。
|
||||
|
||||
对于标准 RKE 安装上的每个命名空间（包括 `default` 和 `kube-system`），`default` service account 必须包含以下值：
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
将以下配置保存到名为 `account_update.yaml` 的文件中。
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
创建一个名为 `account_update.yaml` 的 bash 脚本文件。
|
||||
确保执行 `chmod +x account_update.sh` 命令，以赋予脚本执行权限。
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
执行此脚本将 `account_update.yaml` 配置应用到所有命名空间中的 `default` service account。
|
||||
|
||||
### 配置网络策略
|
||||
|
||||
#### 确保所有命名空间都定义了网络策略
|
||||
|
||||
在同一个 Kubernetes 集群上运行不同的应用程序会带来风险，即某个受感染的应用程序可能会攻击相邻的应用程序。为确保容器只与其预期通信的容器进行通信，网络分段至关重要。网络策略规定了哪些 Pod 可以互相通信，以及与其他网络终端通信的方式。
|
||||
|
||||
网络策略是命名空间范围的。当在特定命名空间引入网络策略时，所有未被策略允许的流量将被拒绝。然而，如果在命名空间中没有网络策略，那么所有流量将被允许进入和离开该命名空间中的 Pod。要强制执行网络策略，必须启用容器网络接口（container network interface, CNI）插件。本指南使用 [Canal](https://github.com/projectcalico/canal) 来提供策略执行。有关 CNI 提供程序的其他信息可以在[这里](https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)找到。
|
||||
|
||||
一旦在集群上启用了 CNI 提供程序，就可以应用默认的网络策略。下面提供了一个 **permissive** 的示例供参考。如果你希望允许匹配某个命名空间中所有 Pod 的所有入站和出站流量（即使添加了策略导致某些 Pod 被视为”隔离”），你可以创建一个明确允许该命名空间中所有流量的策略。请将以下配置保存为 `default-allow-all.yaml`。有关网络策略的其他[文档](https://kubernetes.io/docs/concepts/services-networking/network-policies/)可以在 Kubernetes 站点上找到。
|
||||
|
||||
:::caution
|
||||
此网络策略只是一个示例，不建议用于生产用途。
|
||||
:::
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
创建一个名为 `apply_networkPolicy_to_all_ns.sh`的 Bash 脚本文件。
|
||||
|
||||
确保运行 `chmod +x apply_networkPolicy_to_all_ns.sh` 命令，以赋予脚本执行权限。
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
执行此脚本以将 `default-allow-all.yaml` 配置和 **permissive** 的 `NetworkPolicy` 应用于所有命名空间。
|
||||
|
||||
## 已知限制
|
||||
|
||||
- 当注册自定义节点仅提供公共 IP 时，Rancher **exec shell** 和 **查看 pod 日志** 在加固设置中**不起作用**。 此功能需要在注册自定义节点时提供私有 IP。
|
||||
|
||||
## 加固的 RKE `cluster.yml` 配置参考
|
||||
|
||||
参考的 `cluster.yml` 文件是由 RKE CLI 使用的，它提供了实现 RKE 加固安装所需的配置。
|
||||
RKE [文档](https://rancher.com/docs/rke/latest/en/installation/)提供了有关配置项的更多详细信息。这里参考的 `cluster.yml` 不包括必需的 `nodes` 指令，因为它取决于你的环境。在 RKE 中有关节点配置的文档可以在[这里](https://rancher.com/docs/rke/latest/en/config-options/nodes/)找到。
|
||||
|
||||
示例 `cluster.yml` 配置文件中包含了一个 Admission Configuration 策略，在 `services.kube-api.admission_configuration` 字段中指定。这个[示例](../../psa-restricted-exemptions.md)策略包含了命名空间的豁免规则，这对于在Rancher中正确运行导入的RKE集群非常必要，类似于Rancher预定义的 [`rancher-restricted`](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) 策略。
|
||||
|
||||
如果你希望使用 RKE 的默认 `restricted` 策略，则将 `services.kube-api.admission_configuration` 字段留空，并将 `services.pod_security_configuration` 设置为 `restricted`。你可以在 [RKE 文档](https://rke.docs.rancher.com/config-options/services/pod-security-admission)中找到更多信息。
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 及更新版本" default>
|
||||
|
||||
:::note
|
||||
如果你打算将一个 RKE 集群导入到 Rancher 中，请参考此[文档](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md)以了解如何配置 PSA 以豁免 Rancher 系统命名空间。
|
||||
:::
|
||||
|
||||
```yaml
|
||||
# 如果你打算在离线环境部署 Kubernetes，
|
||||
# 请查阅文档以了解如何配置自定义的 RKE 镜像。
|
||||
nodes: []
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
# 如果你在 `admission_configuration` 中设置了自定义策略，
|
||||
# 请将 `pod_security_configuration` 字段留空。
|
||||
# 否则，将其设置为 `restricted` 以使用 RKE 预定义的受限策略，
|
||||
# 并删除 `admission_configuration` 字段中的所有内容。
|
||||
#
|
||||
# pod_security_configuration: restricted
|
||||
#
|
||||
admission_configuration:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: PodSecurity
|
||||
configuration:
|
||||
apiVersion: pod-security.admission.config.k8s.io/v1
|
||||
kind: PodSecurityConfiguration
|
||||
defaults:
|
||||
enforce: "restricted"
|
||||
enforce-version: "latest"
|
||||
audit: "restricted"
|
||||
audit-version: "latest"
|
||||
warn: "restricted"
|
||||
warn-version: "latest"
|
||||
exemptions:
|
||||
usernames: []
|
||||
runtimeClasses: []
|
||||
namespaces: [calico-apiserver,
|
||||
calico-system,
|
||||
cattle-alerting,
|
||||
cattle-csp-adapter-system,
|
||||
cattle-elemental-system,
|
||||
cattle-epinio-system,
|
||||
cattle-externalip-system,
|
||||
cattle-fleet-local-system,
|
||||
cattle-fleet-system,
|
||||
cattle-gatekeeper-system,
|
||||
cattle-global-data,
|
||||
cattle-global-nt,
|
||||
cattle-impersonation-system,
|
||||
cattle-istio,
|
||||
cattle-istio-system,
|
||||
cattle-logging,
|
||||
cattle-logging-system,
|
||||
cattle-monitoring-system,
|
||||
cattle-neuvector-system,
|
||||
cattle-prometheus,
|
||||
cattle-provisioning-capi-system,
|
||||
cattle-resources-system,
|
||||
cattle-sriov-system,
|
||||
cattle-system,
|
||||
cattle-ui-plugin-system,
|
||||
cattle-windows-gmsa-system,
|
||||
cert-manager,
|
||||
cis-operator-system,
|
||||
fleet-default,
|
||||
ingress-nginx,
|
||||
istio-system,
|
||||
kube-node-lease,
|
||||
kube-public,
|
||||
kube-system,
|
||||
longhorn-system,
|
||||
rancher-alerting-drivers,
|
||||
security-scan,
|
||||
tigera-operator]
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 及更早版本">
|
||||
|
||||
```yaml
|
||||
# 如果你打算在离线环境部署 Kubernetes，
|
||||
# 请查阅文档以了解如何配置自定义的 RKE 镜像。
|
||||
nodes: []
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
# Upstream Kubernetes restricted PSP policy
|
||||
# https://github.com/kubernetes/website/blob/564baf15c102412522e9c8fc6ef2b5ff5b6e766c/content/en/examples/policy/restricted-psp.yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted-noroot
|
||||
spec:
|
||||
privileged: false
|
||||
# Required to prevent escalations to root.
|
||||
allowPrivilegeEscalation: false
|
||||
requiredDropCapabilities:
|
||||
- ALL
|
||||
# Allow core volume types.
|
||||
volumes:
|
||||
- 'configMap'
|
||||
- 'emptyDir'
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
# Assume that ephemeral CSI drivers & persistentVolumes set up by the cluster admin are safe to use.
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
runAsUser:
|
||||
# Require the container to run without root privileges.
|
||||
rule: 'MustRunAsNonRoot'
|
||||
seLinux:
|
||||
# This policy assumes the nodes are using AppArmor rather than SELinux.
|
||||
rule: 'RunAsAny'
|
||||
supplementalGroups:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
fsGroup:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
readOnlyRootFilesystem: false
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted-noroot
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted-noroot
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## 加固后的 RKE 集群模板配置参考
|
||||
|
||||
参考的 RKE 集群模板提供了实现 Kubernetes 加固安装所需的最低配置。RKE 模板用于提供 Kubernetes 并定义 Rancher 设置。有关安装 RKE 及其模板详情的其他信息，请参考 Rancher [文档](../../../../getting-started/installation-and-upgrade/installation-and-upgrade.md) 。
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 及更新版本" default>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# 集群配置
|
||||
#
|
||||
default_pod_security_admission_configuration_template_name: rancher-restricted
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # 定义集群名称
|
||||
|
||||
#
|
||||
# Rancher 配置
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 及更早版本">
|
||||
|
||||
```yaml
|
||||
#
|
||||
# 集群配置
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted-noroot
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # 定义集群名称
|
||||
|
||||
#
|
||||
# Rancher 配置
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # 定义 RKE 版本
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## 结论
|
||||
|
||||
如果你按照本指南操作，由 Rancher 提供的 RKE 自定义集群将配置为通过 CIS Kubernetes Benchmark 测试。你可以查看我们的 RKE 自我评估指南，了解我们是如何验证每个 benchmarks 的，并且你可以在你的集群上执行相同的操作。
|
||||
+2863
File diff suppressed because it is too large
Load Diff
+365
@@ -0,0 +1,365 @@
|
||||
---
|
||||
title: RKE Cluster Configuration Reference
|
||||
---
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration"/>
|
||||
</head>
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
When Rancher installs Kubernetes, it uses [RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) or [RKE2](https://docs.rke2.io/) as the Kubernetes distribution.
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE Kubernetes cluster.
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#rke-cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
The RKE cluster config options are nested under the `rancher_kubernetes_engine_config` directive. For more information, see the section about the [cluster config file.](#rke-cluster-config-file-reference)
|
||||
|
||||
In [clusters launched by RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md), you can edit any of the remaining options that follow.
|
||||
|
||||
For an example of RKE config file syntax, see the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
|
||||
|
||||
The forms in the Rancher UI don't include all advanced options for configuring RKE. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
## Editing Clusters with a Form in the Rancher UI
|
||||
|
||||
To edit your cluster,
|
||||
|
||||
1. In the upper left corner, click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
|
||||
|
||||
|
||||
## Editing Clusters with YAML
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
RKE clusters (also called RKE1 clusters) are edited differently than RKE2 and K3s clusters.
|
||||
|
||||
To edit an RKE config file directly from the Rancher UI,
|
||||
|
||||
1. Click **☰ > Cluster Management**.
|
||||
1. Go to the RKE cluster you want to configure. Click and click **⋮ > Edit Config**. This take you to the RKE configuration form. Note: Because cluster provisioning changed in Rancher 2.6, the **⋮ > Edit as YAML** can be used for configuring RKE2 clusters, but it can't be used for editing RKE1 configuration.
|
||||
1. In the configuration form, scroll down and click **Edit as YAML**.
|
||||
1. Edit the RKE options under the `rancher_kubernetes_engine_config` directive.
|
||||
|
||||
## Configuration Options in the Rancher UI
|
||||
|
||||
:::tip
|
||||
|
||||
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
:::
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
|
||||
|
||||
### Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses. For more details on the different networking providers, please view our [Networking FAQ](../../../faq/container-network-interface-providers.md).
|
||||
|
||||
:::caution
|
||||
|
||||
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
|
||||
|
||||
:::
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
<DeprecationWeave />
|
||||
|
||||
:::note Notes on Weave:
|
||||
|
||||
When Weave is selected as network provider, Rancher will automatically enable encryption by generating a random password. If you want to specify the password manually, please see how to configure your cluster using a [Config File](#rke-cluster-config-file-reference) and the [Weave Network Plug-in Options](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options).
|
||||
|
||||
:::
|
||||
|
||||
### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### Kubernetes Cloud Providers
|
||||
|
||||
You can configure a [Kubernetes cloud provider](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
:::note
|
||||
|
||||
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#rke-cluster-config-file-reference) to configure the cloud provider. Please reference the [RKE cloud provider documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
:::
|
||||
|
||||
### Private Registries
|
||||
|
||||
The cluster-level private registry configuration is only used for provisioning clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the [global default registry](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md) through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
If your private registry requires credentials, you need to pass the credentials to Rancher by editing the cluster options for each cluster that needs to pull images from the registry.
|
||||
|
||||
The private registry configuration option tells Rancher where to pull the [system images](https://rancher.com/docs/rke/latest/en/config-options/system-images/) or [addon images](https://rancher.com/docs/rke/latest/en/config-options/add-ons/) that will be used in your cluster.
|
||||
|
||||
- **System images** are components needed to maintain the Kubernetes cluster.
|
||||
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
|
||||
|
||||
For more information on setting up a private registry for components applied during the provisioning of the cluster, see the [RKE documentation on private registries](https://rancher.com/docs/rke/latest/en/config-options/private-registries/).
|
||||
|
||||
Rancher v2.6 introduced the ability to configure [ECR registries for RKE clusters](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup).
|
||||
|
||||
### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint (ACE) can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
:::note
|
||||
|
||||
ACE is available on RKE, RKE2, and K3s clusters that are provisioned or registered with Rancher. It's not available on clusters in a hosted Kubernetes provider, such as Amazon's EKS.
|
||||
|
||||
:::
|
||||
|
||||
ACE must be set up [manually](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) on RKE2 and K3s clusters. In RKE, ACE is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self-signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
|
||||
|
||||
### Node Pools
|
||||
|
||||
For information on using the Rancher UI to set up node pools in an RKE cluster, refer to [this page.](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md)
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster.
|
||||
|
||||
### Metrics Server Monitoring
|
||||
|
||||
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
You must have an existing Pod Security Policy configured before you can use this option.
|
||||
|
||||
### Docker Version on Nodes
|
||||
|
||||
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support.
|
||||
|
||||
If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
|
||||
|
||||
For details on which Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
|
||||
|
||||
### Docker Root Directory
|
||||
|
||||
If the nodes you are adding to the cluster have Docker configured with a non-default Docker Root Directory (default is `/var/lib/docker`), specify the correct Docker Root Directory in this option.
|
||||
|
||||
### Default Pod Security Policy
|
||||
|
||||
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
|
||||
|
||||
### Node Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). Default is `30000-32767`.
|
||||
|
||||
### Recurring etcd Snapshots
|
||||
|
||||
Option to enable or disable [recurring etcd snapshots](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
|
||||
|
||||
### Agent Environment Variables
|
||||
|
||||
Option to set environment variables for [rancher agents](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
|
||||
### Updating ingress-nginx
|
||||
|
||||
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
|
||||
|
||||
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
|
||||
|
||||
### Cluster Agent Configuration and Fleet Agent Configuration
|
||||
|
||||
You can configure the scheduling fields and resource limits for the Cluster Agent and the cluster's Fleet Agent. You can use these fields to customize tolerations, affinity rules, and resource requirements. Additional tolerations are appended to a list of default tolerations and control plane node taints. If you define custom affinity rules, they override the global default affinity setting. Defining resource requirements sets requests or limits where there previously were none.
|
||||
|
||||
:::note
|
||||
|
||||
With this option, it's possible to override or remove rules that are required for the functioning of the cluster. We strongly recommend against removing or overriding these and any other affinity rules, as this may cause unwanted side effects:
|
||||
|
||||
- `affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution` for `cattle-cluster-agent`
|
||||
- `cluster-agent-default-affinity` for `cattle-cluster-agent`
|
||||
- `fleet-agent-default-affinity` for `fleet-agent`
|
||||
|
||||
:::
|
||||
|
||||
If you downgrade Rancher to v2.7.4 or below, your changes will be lost and the agents will re-deploy without your customizations. The Fleet agent will fallback to using its built-in default values when it re-deploys. If the Fleet version doesn't change during the downgrade, the re-deploy won't be immediate.
|
||||
|
||||
|
||||
## RKE Cluster Config File Reference
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available](https://rancher.com/docs/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API.
|
||||
|
||||
For the complete reference for configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
### Config File Structure in Rancher
|
||||
|
||||
RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kubernetes clusters. Rancher's cluster config files used to have the same structure as [RKE config files,](https://rancher.com/docs/rke/latest/en/example-yamls/) but the structure changed so that in Rancher, RKE cluster config items are separated from non-RKE config items. Therefore, configuration for your cluster needs to be nested under the `rancher_kubernetes_engine_config` directive in the cluster config file. Cluster config files created with earlier versions of Rancher will need to be updated for this format. An example cluster config file is included below.
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>Example Cluster Config File</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # Currently only nginx ingress provider is supported.
|
||||
# # To disable ingress controller, set `provider: none`
|
||||
# # To enable ingress on specific nodes, use the node_selector, eg:
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### Default DNS provider
|
||||
|
||||
The table below indicates what DNS provider is deployed by default. See [RKE documentation on DNS provider](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/) for more information how to configure a different DNS provider. CoreDNS can only be used on Kubernetes v1.12.0 and higher.
|
||||
|
||||
| Rancher version | Kubernetes version | Default DNS provider |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 and higher | v1.14.0 and higher | CoreDNS |
|
||||
| v2.2.5 and higher | v1.13.x and lower | kube-dns |
|
||||
| v2.2.4 and lower | any | kube-dns |
|
||||
|
||||
## Rancher Specific Parameters in YAML
|
||||
|
||||
Besides the RKE config file options, there are also Rancher specific settings that can be configured in the Config File (YAML):
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
See [Docker Root Directory](#docker-root-directory).
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
Option to enable or disable [Cluster Monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md).
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
Option to enable or disable Project Network Isolation.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
See [Authorized Cluster Endpoint](#authorized-cluster-endpoint).
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### Custom Network Plug-in
|
||||
|
||||
You can add a custom network plug-in by using the [user-defined add-on functionality](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. You define any add-on that you want deployed after the Kubernetes cluster is deployed.
|
||||
|
||||
There are two ways that you can specify an add-on:
|
||||
|
||||
- [In-line Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [Referencing YAML Files for Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
+513
@@ -0,0 +1,513 @@
|
||||
---
|
||||
title: RKE Hardening Guides
|
||||
---
|
||||
|
||||
<EOLRKE1Warning />
|
||||
|
||||
<head>
|
||||
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/hardening-guides/rke1-hardening-guide"/>
|
||||
</head>
|
||||
|
||||
This document provides prescriptive guidance for how to harden an RKE cluster intended for production, before provisioning it with Rancher. It outlines the configurations and controls required for Center for Information Security (CIS) Kubernetes benchmark controls.
|
||||
|
||||
:::note
|
||||
This hardening guide describes how to secure the nodes in your cluster. We recommended that you follow this guide before you install Kubernetes.
|
||||
:::
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and is associated with the following versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
| Rancher Version | CIS Benchmark Version | Kubernetes Version |
|
||||
|-----------------|-----------------------|------------------------------|
|
||||
| Rancher v2.7 | Benchmark v1.23 | Kubernetes v1.23 |
|
||||
| Rancher v2.7 | Benchmark v1.24 | Kubernetes v1.24 |
|
||||
| Rancher v2.7 | Benchmark v1.7 | Kubernetes v1.25 up to v1.26 |
|
||||
|
||||
:::note
|
||||
- In Benchmark v1.24 and later, check id `4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Automated)` might fail, as `/etc/kubernetes/ssl/kube-ca.pem` is set to 644 by default.
|
||||
- In Benchmark v1.7, the `--protect-kernel-defaults` (`4.2.6`) parameter isn't required anymore, and was removed by CIS.
|
||||
:::
|
||||
|
||||
For more details on how to evaluate a hardened RKE cluster against the official CIS benchmark, refer to the RKE self-assessment guides for specific Kubernetes and CIS benchmark versions.
|
||||
|
||||
## Host-level requirements
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes types in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
|
||||
A user account and group for the **etcd** service is required to be set up before installing RKE.
|
||||
|
||||
#### Create `etcd` user and group
|
||||
|
||||
To create the **etcd** user and group run the following console commands.
|
||||
The commands below use `52034` for **uid** and **gid** for example purposes.
|
||||
Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd --shell /usr/sbin/nologin
|
||||
```
|
||||
|
||||
When deploying RKE through its cluster configuration `config.yml` file, update the `uid` and `gid` of the `etcd` user:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
## Kubernetes runtime requirements
|
||||
|
||||
### Configure `default` Service Account
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod.
|
||||
Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account.
|
||||
The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including `default` and `kube-system` on a standard RKE install, the `default` service account must include this value:
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following configuration to a file called `account_update.yaml`.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`.
|
||||
Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `account_update.yaml` configuration to `default` service account in all namespaces.
|
||||
|
||||
### Configure Network Policy
|
||||
|
||||
#### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one compromised application attacking a neighboring application. Network segmentation is important to ensure that containers can communicate only with those they are supposed to. A network policy is a specification of how selections of pods are allowed to communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given namespace, all traffic not allowed by the policy is denied. However, if there are no network policies in a namespace all traffic will be allowed into and out of the pods in that namespace. To enforce network policies, a container network interface (CNI) plugin must be enabled. This guide uses [Canal](https://github.com/projectcalico/canal) to provide the policy enforcement. Additional information about CNI providers can be found [here](https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/).
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a **permissive** example is provided below. If you want to allow all traffic to all pods in a namespace (even if policies are added that cause some pods to be treated as “isolated”), you can create a policy that explicitly allows all traffic in that namespace. Save the following configuration as `default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/) about network policies can be found on the Kubernetes site.
|
||||
|
||||
:::caution
|
||||
This network policy is just an example and is not recommended for production use.
|
||||
:::
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to `chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o=jsonpath="{.items[*]['metadata.name']}"); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `default-allow-all.yaml` configuration with the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a hardened setup when only a public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
|
||||
## Reference Hardened RKE `cluster.yml` Configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened installation of RKE. RKE [documentation](https://rancher.com/docs/rke/latest/en/installation/) provides additional details about the configuration items. This reference `cluster.yml` does not include the required `nodes` directive which will vary depending on your environment. Documentation for node configuration in RKE can be found [here](https://rancher.com/docs/rke/latest/en/config-options/nodes/).
|
||||
|
||||
The example `cluster.yml` configuration file contains an Admission Configuration policy in the `services.kube-api.admission_configuration` field. This [sample](../../psa-restricted-exemptions.md) policy contains the namespace exemptions necessary for an imported RKE cluster to run properly in Rancher, similar to Rancher's pre-defined [`rancher-restricted`](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) policy.
|
||||
|
||||
If you prefer to use RKE's default `restricted` policy, then leave the `services.kube-api.admission_configuration` field empty and set `services.pod_security_configuration` to `restricted`. See [the RKE docs](https://rke.docs.rancher.com/config-options/services/pod-security-admission) for more information.
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 and Newer" default>
|
||||
|
||||
:::note
|
||||
If you intend to import an RKE cluster into Rancher, please consult the [documentation](../../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for how to configure the PSA to exempt Rancher system namespaces.
|
||||
:::
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
nodes: []
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
# Leave `pod_security_configuration` out if you are setting a
|
||||
# custom policy in `admission_configuration`. Otherwise set
|
||||
# it to `restricted` to use RKE's pre-defined restricted policy,
|
||||
# and remove everything inside `admission_configuration` field.
|
||||
#
|
||||
# pod_security_configuration: restricted
|
||||
#
|
||||
admission_configuration:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: PodSecurity
|
||||
configuration:
|
||||
apiVersion: pod-security.admission.config.k8s.io/v1
|
||||
kind: PodSecurityConfiguration
|
||||
defaults:
|
||||
enforce: "restricted"
|
||||
enforce-version: "latest"
|
||||
audit: "restricted"
|
||||
audit-version: "latest"
|
||||
warn: "restricted"
|
||||
warn-version: "latest"
|
||||
exemptions:
|
||||
usernames: []
|
||||
runtimeClasses: []
|
||||
namespaces: [calico-apiserver,
|
||||
calico-system,
|
||||
cattle-alerting,
|
||||
cattle-csp-adapter-system,
|
||||
cattle-elemental-system,
|
||||
cattle-epinio-system,
|
||||
cattle-externalip-system,
|
||||
cattle-fleet-local-system,
|
||||
cattle-fleet-system,
|
||||
cattle-gatekeeper-system,
|
||||
cattle-global-data,
|
||||
cattle-global-nt,
|
||||
cattle-impersonation-system,
|
||||
cattle-istio,
|
||||
cattle-istio-system,
|
||||
cattle-logging,
|
||||
cattle-logging-system,
|
||||
cattle-monitoring-system,
|
||||
cattle-neuvector-system,
|
||||
cattle-prometheus,
|
||||
cattle-provisioning-capi-system,
|
||||
cattle-resources-system,
|
||||
cattle-sriov-system,
|
||||
cattle-system,
|
||||
cattle-ui-plugin-system,
|
||||
cattle-windows-gmsa-system,
|
||||
cert-manager,
|
||||
cis-operator-system,
|
||||
fleet-default,
|
||||
ingress-nginx,
|
||||
istio-system,
|
||||
kube-node-lease,
|
||||
kube-public,
|
||||
kube-system,
|
||||
longhorn-system,
|
||||
rancher-alerting-drivers,
|
||||
security-scan,
|
||||
tigera-operator]
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 and Older">
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
nodes: []
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
generate_serving_certificate: true
|
||||
addons: |
|
||||
# Upstream Kubernetes restricted PSP policy
|
||||
# https://github.com/kubernetes/website/blob/564baf15c102412522e9c8fc6ef2b5ff5b6e766c/content/en/examples/policy/restricted-psp.yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted-noroot
|
||||
spec:
|
||||
privileged: false
|
||||
# Required to prevent escalations to root.
|
||||
allowPrivilegeEscalation: false
|
||||
requiredDropCapabilities:
|
||||
- ALL
|
||||
# Allow core volume types.
|
||||
volumes:
|
||||
- 'configMap'
|
||||
- 'emptyDir'
|
||||
- 'projected'
|
||||
- 'secret'
|
||||
- 'downwardAPI'
|
||||
# Assume that ephemeral CSI drivers & persistentVolumes set up by the cluster admin are safe to use.
|
||||
- 'csi'
|
||||
- 'persistentVolumeClaim'
|
||||
- 'ephemeral'
|
||||
hostNetwork: false
|
||||
hostIPC: false
|
||||
hostPID: false
|
||||
runAsUser:
|
||||
# Require the container to run without root privileges.
|
||||
rule: 'MustRunAsNonRoot'
|
||||
seLinux:
|
||||
# This policy assumes the nodes are using AppArmor rather than SELinux.
|
||||
rule: 'RunAsAny'
|
||||
supplementalGroups:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
fsGroup:
|
||||
rule: 'MustRunAs'
|
||||
ranges:
|
||||
# Forbid adding the root group.
|
||||
- min: 1
|
||||
max: 65535
|
||||
readOnlyRootFilesystem: false
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted-noroot
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted-noroot
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted-noroot
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Reference Hardened RKE Cluster Template Configuration
|
||||
|
||||
The reference RKE cluster template provides the minimum required configuration to achieve a hardened installation of Kubernetes. RKE templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher [documentation](../../../../getting-started/installation-and-upgrade/installation-and-upgrade.md) for additional information about installing RKE and its template details.
|
||||
|
||||
<Tabs groupId="rke1-version">
|
||||
<TabItem value="v1.25 and Newer" default>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_admission_configuration_template_name: rancher-restricted
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # Define cluster name
|
||||
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="v1.24 and Older">
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted-noroot
|
||||
enable_network_policy: true
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
name: # Define cluster name
|
||||
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
authentication:
|
||||
strategy: x509|webhook
|
||||
kubernetes_version: # Define RKE version
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: true
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Conclusion
|
||||
|
||||
If you have followed this guide, your RKE custom cluster provisioned by Rancher will be configured to pass the CIS Kubernetes Benchmark. You can review our RKE self-assessment guides to understand how we verified each of the benchmarks and how you can do the same on your cluster.
|
||||
+2864
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user