mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-25 20:48:11 +00:00
Merge pull request #3761 from macedogm/rancher/35735-hardening-docs
Restructure v2.6 Security section
This commit is contained in:
Billy Tat
@@ -11,7 +11,7 @@ weight: 20
|
||||
</td>
|
||||
<td width="30%" style="border: none;">
|
||||
<h4>Reporting process</h4>
|
||||
<p style="padding: 8px">Please submit possible security issues by emailing <a href="mailto:security@rancher.com">security@rancher.com</a></p>
|
||||
<p style="padding: 8px">Please submit possible security issues by emailing <a href="mailto:security@rancher.com">security@rancher.com</a> .</p>
|
||||
</td>
|
||||
<td width="30%" style="border: none;">
|
||||
<h4>Announcements</h4>
|
||||
@@ -20,25 +20,25 @@ weight: 20
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
Security is at the heart of all Rancher features. From integrating with all the popular authentication tools and services, to an enterprise grade [RBAC capability,]({{<baseurl>}}/rancher/v2.6/en/admin-settings/rbac) Rancher makes your Kubernetes clusters even more secure.
|
||||
Security is at the heart of all Rancher features. From integrating with all the popular authentication tools and services, to an enterprise grade [RBAC capability]({{<baseurl>}}/rancher/v2.6/en/admin-settings/rbac), Rancher makes your Kubernetes clusters even more secure.
|
||||
|
||||
On this page, we provide security-related documentation along with resources to help you secure your Rancher installation and your downstream Kubernetes clusters:
|
||||
On this page, we provide security related documentation along with resources to help you secure your Rancher installation and your downstream Kubernetes clusters:
|
||||
|
||||
- [Running a CIS security scan on a Kubernetes cluster](#running-a-cis-security-scan-on-a-kubernetes-cluster)
|
||||
- [SELinux RPM](#selinux-rpm)
|
||||
- [Guide to hardening Rancher installations](#rancher-hardening-guide)
|
||||
- [The CIS Benchmark and self-assessment](#the-cis-benchmark-and-self-assessment)
|
||||
- [Third-party penetration test reports](#third-party-penetration-test-reports)
|
||||
- [Rancher CVEs and resolutions](#rancher-cves-and-resolutions)
|
||||
- [Rancher Security Advisories and CVEs](#rancher-security-advisories-and-cves)
|
||||
- [Kubernetes Security Best Practices](#kubernetes-security-best-practices)
|
||||
|
||||
### Running a CIS Security Scan on a Kubernetes Cluster
|
||||
|
||||
Rancher leverages [kube-bench](https://github.com/aquasecurity/kube-bench) to run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS (Center for Internet Security) Kubernetes Benchmark.
|
||||
Rancher leverages [kube-bench](https://github.com/aquasecurity/kube-bench) to run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the [CIS](https://www.cisecurity.org/cis-benchmarks/) (Center for Internet Security) Kubernetes Benchmark.
|
||||
|
||||
The CIS Kubernetes Benchmark is a reference document that can be used to establish a secure configuration baseline for Kubernetes.
|
||||
|
||||
The Center for Internet Security (CIS) is a 501(c\)(3) non-profit organization, formed in October 2000, with a mission to "identify, develop, validate, promote, and sustain best practice solutions for cyber defense and build and lead communities to enable an environment of trust in cyberspace."
|
||||
The Center for Internet Security (CIS) is a 501(c\)(3) non-profit organization, formed in October 2000, with a mission to "identify, develop, validate, promote, and sustain best practice solutions for cyber defense and build and lead communities to enable an environment of trust in cyberspace".
|
||||
|
||||
CIS Benchmarks are best practices for the secure configuration of a target system. CIS Benchmarks are developed through the generous volunteer efforts of subject matter experts, technology vendors, public and private community members, and the CIS Benchmark Development team.
|
||||
|
||||
@@ -46,13 +46,13 @@ The Benchmark provides recommendations of two types: Automated and Manual. We ru
|
||||
|
||||
When Rancher runs a CIS security scan on a cluster, it generates a report showing the results of each test, including a summary with the number of passed, skipped and failed tests. The report also includes remediation steps for any failed tests.
|
||||
|
||||
For details, refer to the section on [security scans.]({{<baseurl>}}/rancher/v2.6/en/cis-scans)
|
||||
For details, refer to the section on [security scans]({{<baseurl>}}/rancher/v2.6/en/cis-scans).
|
||||
|
||||
### SELinux RPM
|
||||
|
||||
[Security-Enhanced Linux (SELinux)](https://en.wikipedia.org/wiki/Security-Enhanced_Linux) is a security enhancement to Linux. After being historically used by government agencies, SELinux is now industry standard and is enabled by default on CentOS 7 and 8.
|
||||
|
||||
We provide two RPMs (Red Hat packages) that enable Rancher products to function properly on SELinux-enforcing hosts: `rancher-selinux` and `rke2-selinux`. For details, see [this page.]({{<baseurl>}}/rancher/v2.6/en/security/selinux)
|
||||
We provide two RPMs (Red Hat packages) that enable Rancher products to function properly on SELinux-enforcing hosts: `rancher-selinux` and `rke2-selinux`. For details, see [this page]({{<baseurl>}}/rancher/v2.6/en/security/selinux).
|
||||
|
||||
### Rancher Hardening Guide
|
||||
|
||||
@@ -78,13 +78,13 @@ Rancher periodically hires third parties to perform security audits and penetrat
|
||||
|
||||
Results:
|
||||
|
||||
- [Cure53 Pen Test - 7/2019](https://releases.rancher.com/documents/security/pen-tests/2019/RAN-01-cure53-report.final.pdf)
|
||||
- [Untamed Theory Pen Test- 3/2019](https://releases.rancher.com/documents/security/pen-tests/2019/UntamedTheory-Rancher_SecurityAssessment-20190712_v5.pdf)
|
||||
- [Cure53 Pen Test - July 2019](https://releases.rancher.com/documents/security/pen-tests/2019/RAN-01-cure53-report.final.pdf)
|
||||
- [Untamed Theory Pen Test- March 2019](https://releases.rancher.com/documents/security/pen-tests/2019/UntamedTheory-Rancher_SecurityAssessment-20190712_v5.pdf)
|
||||
|
||||
### Rancher CVEs and Resolutions
|
||||
### Rancher Security Advisories and CVEs
|
||||
|
||||
Rancher is committed to informing the community of security issues in our products. For the list of CVEs (Common Vulnerabilities and Exposures) for issues we have resolved, refer to [this page.](./cve)
|
||||
|
||||
### Kubernetes Security Best Practices
|
||||
|
||||
For recommendations on securing your Kubernetes cluster, refer to the [Best Practices](./best-practices) guide.
|
||||
For recommendations on securing your Kubernetes cluster, refer to the [Kubernetes Security Best Practices](./best-practices) guide.
|
||||
|
||||
@@ -3,6 +3,10 @@ title: Kubernetes Security Best Practices
|
||||
weight: 5
|
||||
---
|
||||
|
||||
# Restricting cloud metadata API access
|
||||
### Restricting cloud metadata API access
|
||||
|
||||
Cloud providers such as AWS, Azure, or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.
|
||||
Cloud providers such as AWS, Azure, DigitalOcean or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS, DigitalOcean Kubernetes or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.
|
||||
|
||||
It is advised to consult your cloud provider's security best practices for further recommendations and specific details on how to restrict access to cloud instance metadata API.
|
||||
|
||||
Further references: MITRE ATT&CK knowledge base on - [Unsecured Credentials: Cloud Instance Metadata API](https://attack.mitre.org/techniques/T1552/005/).
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Rancher CVEs and Resolutions
|
||||
title: Security Advisories and CVEs
|
||||
weight: 300
|
||||
---
|
||||
|
||||
Rancher is committed to informing the community of security issues in our products. Rancher will publish CVEs (Common Vulnerabilities and Exposures) for issues we have resolved.
|
||||
Rancher is committed to informing the community of security issues in our products. Rancher will publish security advisories and CVEs (Common Vulnerabilities and Exposures) for issues we have resolved. New security advisories are also published in Rancher's GitHub [security page](https://github.com/rancher/rancher/security/advisories).
|
||||
|
||||
| ID | Description | Date | Resolution |
|
||||
|----|-------------|------|------------|
|
||||
@@ -18,4 +18,4 @@ Rancher is committed to informing the community of security issues in our produc
|
||||
| [CVE-2019-12274](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12274) | Nodes using the built-in node drivers using a file path option allows the machine to read arbitrary files including sensitive ones from inside the Rancher server container. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
|
||||
| [CVE-2019-11202](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11202) | The default admin, that is shipped with Rancher, will be re-created upon restart of Rancher despite being explicitly deleted. | 16 Apr 2019 | [Rancher v2.2.2](https://github.com/rancher/rancher/releases/tag/v2.2.2), [Rancher v2.1.9](https://github.com/rancher/rancher/releases/tag/v2.1.9) and [Rancher v2.0.14](https://github.com/rancher/rancher/releases/tag/v2.0.14) |
|
||||
| [CVE-2019-6287](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6287) | Project members continue to get access to namespaces from projects that they were removed from if they were added to more than one project. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) |
|
||||
| [CVE-2018-20321](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20321) | Any project member with access to the `default` namespace can mount the `netes-default` service account in a pod and then use that pod to execute administrative privileged commands against the Kubernetes cluster. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) - Rolling back from these versions or greater have specific [instructions]({{<baseurl>}}/rancher/v2.6/en/installation/install-rancher-on-k8s/rollbacks). |
|
||||
| [CVE-2018-20321](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20321) | Any project member with access to the `default` namespace can mount the `netes-default` service account in a pod and then use that pod to execute administrative privileged commands against the Kubernetes cluster. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) - Rolling back from these versions or greater have specific [instructions]({{<baseurl>}}/rancher/v2.6/en/installation/install-rancher-on-k8s/rollbacks). |
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
title: Self-Assessment and Hardening Guides for Rancher v2.6
|
||||
shortTitle: Rancher v2.6 Guides
|
||||
weight: 1
|
||||
aliases:
|
||||
- /rancher/v2.6/en/security/rancher-2.5/
|
||||
- /rancher/v2.6/en/security/rancher-2.5/1.5-hardening-2.5/
|
||||
- /rancher/v2.6/en/security/rancher-2.5/1.5-benchmark-2.5/
|
||||
- /rancher/v2.6/en/security/rancher-2.5/1.6-hardening-2.5/
|
||||
- /rancher/v2.6/en/security/rancher-2.5/1.6-benchmark-2.5/
|
||||
---
|
||||
|
||||
Rancher v2.6 hardening guides are currently being updated. For the time being, please consult [Rancher v2.5 self-assessment and hardening guides]({{<baseurl>}}/rancher/v2.5/en/security/rancher-2.5) for more information.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,720 +0,0 @@
|
||||
---
|
||||
title: Hardening Guide with CIS 1.5 Benchmark
|
||||
weight: 200
|
||||
---
|
||||
|
||||
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
Rancher Version | CIS Benchmark Version | Kubernetes Version
|
||||
----------------|-----------------------|------------------
|
||||
Rancher v2.5 | Benchmark v1.5 | Kubernetes 1.15
|
||||
|
||||
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.5.pdf)
|
||||
|
||||
### Overview
|
||||
|
||||
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5 with Kubernetes v1.15 or provisioning a RKE cluster with Kubernetes 1.15 to be used within Rancher v2.5. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.5 Benchmark - Self-Assessment Guide - Rancher v2.5]({{< baseurl >}}/rancher/v2.6/en/security/rancher-2.5/1.5-benchmark-2.5/).
|
||||
|
||||
#### Known Issues
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.5 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.5 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
|
||||
|
||||
#### create `etcd` user and group
|
||||
To create the **etcd** group run the following console commands.
|
||||
|
||||
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
|
||||
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
|
||||
|
||||
``` yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
|
||||
|
||||
```
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following yaml to a file called `account_update.yaml`
|
||||
|
||||
``` yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one
|
||||
compromised application attacking a neighboring application. Network segmentation is
|
||||
important to ensure that containers can communicate only with those they are supposed
|
||||
to. A network policy is a specification of how selections of pods are allowed to
|
||||
communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given
|
||||
namespace, all traffic not allowed by the policy is denied. However, if there are no network
|
||||
policies in a namespace all traffic will be allowed into and out of the pods in that
|
||||
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
|
||||
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
|
||||
Additional information about CNI providers can be found
|
||||
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
|
||||
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
|
||||
(even if policies are added that cause some pods to be treated as “isolated”),
|
||||
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
|
||||
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
about network policies can be found on the Kubernetes site.
|
||||
|
||||
> This `NetworkPolicy` is not recommended for production use
|
||||
|
||||
``` yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
|
||||
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
### Reference Hardened RKE `cluster.yml` configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
|
||||
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
|
||||
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
|
||||
|
||||
``` yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
kubernetes_version: "v1.15.9-rancher1-1"
|
||||
enable_network_policy: true
|
||||
default_pod_security_policy_template_id: "restricted"
|
||||
# the nodes directive is required and will vary depending on your environment
|
||||
# documentation for node configuration can be found here:
|
||||
# https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
nodes:
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
admission_configuration:
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: "RotateKubeletServerCertificate=true"
|
||||
scheduler:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
kubelet:
|
||||
generate_serving_certificate: true
|
||||
extra_args:
|
||||
feature-gates: "RotateKubeletServerCertificate=true"
|
||||
protect-kernel-defaults: "true"
|
||||
tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
cluster_domain: ""
|
||||
infra_container_image: ""
|
||||
cluster_dns_server: ""
|
||||
fail_swap_on: false
|
||||
kubeproxy:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
network:
|
||||
plugin: ""
|
||||
options: {}
|
||||
mtu: 0
|
||||
node_selector: {}
|
||||
authentication:
|
||||
strategy: ""
|
||||
sans: []
|
||||
webhook: null
|
||||
addons: |
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: ingress-nginx
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: ingress-nginx
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cattle-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: cattle-system
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: cattle-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tiller
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
|
||||
addons_include: []
|
||||
system_images:
|
||||
etcd: ""
|
||||
alpine: ""
|
||||
nginx_proxy: ""
|
||||
cert_downloader: ""
|
||||
kubernetes_services_sidecar: ""
|
||||
kubedns: ""
|
||||
dnsmasq: ""
|
||||
kubedns_sidecar: ""
|
||||
kubedns_autoscaler: ""
|
||||
coredns: ""
|
||||
coredns_autoscaler: ""
|
||||
kubernetes: ""
|
||||
flannel: ""
|
||||
flannel_cni: ""
|
||||
calico_node: ""
|
||||
calico_cni: ""
|
||||
calico_controllers: ""
|
||||
calico_ctl: ""
|
||||
calico_flexvol: ""
|
||||
canal_node: ""
|
||||
canal_cni: ""
|
||||
canal_flannel: ""
|
||||
canal_flexvol: ""
|
||||
weave_node: ""
|
||||
weave_cni: ""
|
||||
pod_infra_container: ""
|
||||
ingress: ""
|
||||
ingress_backend: ""
|
||||
metrics_server: ""
|
||||
windows_pod_infra_container: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert_path: ""
|
||||
ssh_agent_auth: false
|
||||
authorization:
|
||||
mode: ""
|
||||
options: {}
|
||||
ignore_docker_version: false
|
||||
private_registries: []
|
||||
ingress:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
extra_args: {}
|
||||
dns_policy: ""
|
||||
extra_envs: []
|
||||
extra_volumes: []
|
||||
extra_volume_mounts: []
|
||||
cluster_name: ""
|
||||
prefix_path: ""
|
||||
addon_job_timeout: 0
|
||||
bastion_host:
|
||||
address: ""
|
||||
port: ""
|
||||
user: ""
|
||||
ssh_key: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert: ""
|
||||
ssh_cert_path: ""
|
||||
monitoring:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
restore:
|
||||
restore: false
|
||||
snapshot_name: ""
|
||||
dns: null
|
||||
```
|
||||
|
||||
### Reference Hardened RKE Template configuration
|
||||
|
||||
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
|
||||
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
|
||||
[documentaion](https://rancher.com/docs/rancher/v2.6/en/installation) for additional installation and RKE Template details.
|
||||
|
||||
``` yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 30
|
||||
addons: |-
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: ingress-nginx
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: ingress-nginx
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cattle-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: cattle-system
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: cattle-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tiller
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
ignore_docker_version: true
|
||||
kubernetes_version: v1.15.9-rancher1-1
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
mtu: 0
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: false
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: '5000'
|
||||
heartbeat-interval: '500'
|
||||
gid: 52034
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 52034
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
service_node_port_range: 30000-32767
|
||||
kube_controller:
|
||||
extra_args:
|
||||
address: 127.0.0.1
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
profiling: 'false'
|
||||
terminated-pod-gc-threshold: '1000'
|
||||
kubelet:
|
||||
extra_args:
|
||||
anonymous-auth: 'false'
|
||||
event-qps: '0'
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
make-iptables-util-chains: 'true'
|
||||
protect-kernel-defaults: 'true'
|
||||
streaming-connection-idle-timeout: 1800s
|
||||
tls-cipher-suites: >-
|
||||
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
address: 127.0.0.1
|
||||
profiling: 'false'
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
|
||||
### Hardened Reference Ubuntu 18.04 LTS **cloud-config**:
|
||||
|
||||
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
|
||||
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
|
||||
needed before installing kubernetes.
|
||||
|
||||
``` yaml
|
||||
#cloud-config
|
||||
packages:
|
||||
- curl
|
||||
- jq
|
||||
runcmd:
|
||||
- sysctl -w vm.overcommit_memory=1
|
||||
- sysctl -w kernel.panic=10
|
||||
- sysctl -w kernel.panic_on_oops=1
|
||||
- curl https://releases.rancher.com/install-docker/18.09.sh | sh
|
||||
- usermod -aG docker ubuntu
|
||||
- return=1; while [ $return != 0 ]; do sleep 2; docker ps; return=$?; done
|
||||
- addgroup --gid 52034 etcd
|
||||
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
write_files:
|
||||
- path: /etc/sysctl.d/kubelet.conf
|
||||
owner: root:root
|
||||
permissions: "0644"
|
||||
content: |
|
||||
vm.overcommit_memory=1
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,570 +0,0 @@
|
||||
---
|
||||
title: Hardening Guide with CIS 1.6 Benchmark
|
||||
weight: 100
|
||||
---
|
||||
|
||||
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5.4. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
Rancher Version | CIS Benchmark Version | Kubernetes Version
|
||||
----------------|-----------------------|------------------
|
||||
Rancher v2.5.4 | Benchmark 1.6 | Kubernetes v1.18
|
||||
|
||||
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.6.pdf)
|
||||
|
||||
### Overview
|
||||
|
||||
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5.4 with Kubernetes v1.18 or provisioning a RKE cluster with Kubernetes v1.18 to be used within Rancher v2.5.4. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.6 Benchmark - Self-Assessment Guide - Rancher v2.5.4]({{< baseurl >}}/rancher/v2.6/en/security/rancher-2.5/1.6-benchmark-2.5/).
|
||||
|
||||
#### Known Issues
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.6 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.6 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
Migration Rancher from 2.4 to 2.5. Addons were removed in HG 2.5, and therefore namespaces on migration may be not created on the downstream clusters. Pod may fail to run because of missing namesapce like ingress-nginx, cattle-system.
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
|
||||
|
||||
#### create `etcd` user and group
|
||||
To create the **etcd** group run the following console commands.
|
||||
|
||||
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
|
||||
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following yaml to a file called `account_update.yaml`
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one
|
||||
compromised application attacking a neighboring application. Network segmentation is
|
||||
important to ensure that containers can communicate only with those they are supposed
|
||||
to. A network policy is a specification of how selections of pods are allowed to
|
||||
communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given
|
||||
namespace, all traffic not allowed by the policy is denied. However, if there are no network
|
||||
policies in a namespace all traffic will be allowed into and out of the pods in that
|
||||
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
|
||||
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
|
||||
Additional information about CNI providers can be found
|
||||
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
|
||||
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
|
||||
(even if policies are added that cause some pods to be treated as “isolated”),
|
||||
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
|
||||
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
about network policies can be found on the Kubernetes site.
|
||||
|
||||
> This `NetworkPolicy` is not recommended for production use
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
|
||||
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
### Reference Hardened RKE `cluster.yml` configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
|
||||
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
|
||||
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
# https://rancher.com/docs/rke/latest/en/installation/
|
||||
|
||||
# the nodes directive is required and will vary depending on your environment
|
||||
# documentation for node configuration can be found here:
|
||||
# https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
nodes: []
|
||||
services:
|
||||
etcd:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
external_urls: []
|
||||
ca_cert: ""
|
||||
cert: ""
|
||||
key: ""
|
||||
path: ""
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
snapshot: false
|
||||
retention: ""
|
||||
creation: ""
|
||||
backup_config: null
|
||||
kube-api:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
service_cluster_ip_range: ""
|
||||
service_node_port_range: ""
|
||||
pod_security_policy: true
|
||||
always_pull_images: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
custom_config: null
|
||||
audit_log:
|
||||
enabled: true
|
||||
configuration: null
|
||||
admission_configuration: null
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
configuration: null
|
||||
kube-controller:
|
||||
image: ""
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
cluster_cidr: ""
|
||||
service_cluster_ip_range: ""
|
||||
scheduler:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
kubelet:
|
||||
image: ""
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: "true"
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
cluster_domain: cluster.local
|
||||
infra_container_image: ""
|
||||
cluster_dns_server: ""
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
kubeproxy:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
network:
|
||||
plugin: ""
|
||||
options: {}
|
||||
mtu: 0
|
||||
node_selector: {}
|
||||
update_strategy: null
|
||||
authentication:
|
||||
strategy: ""
|
||||
sans: []
|
||||
webhook: null
|
||||
addons: |
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
addons_include: []
|
||||
system_images:
|
||||
etcd: ""
|
||||
alpine: ""
|
||||
nginx_proxy: ""
|
||||
cert_downloader: ""
|
||||
kubernetes_services_sidecar: ""
|
||||
kubedns: ""
|
||||
dnsmasq: ""
|
||||
kubedns_sidecar: ""
|
||||
kubedns_autoscaler: ""
|
||||
coredns: ""
|
||||
coredns_autoscaler: ""
|
||||
nodelocal: ""
|
||||
kubernetes: ""
|
||||
flannel: ""
|
||||
flannel_cni: ""
|
||||
calico_node: ""
|
||||
calico_cni: ""
|
||||
calico_controllers: ""
|
||||
calico_ctl: ""
|
||||
calico_flexvol: ""
|
||||
canal_node: ""
|
||||
canal_cni: ""
|
||||
canal_controllers: ""
|
||||
canal_flannel: ""
|
||||
canal_flexvol: ""
|
||||
weave_node: ""
|
||||
weave_cni: ""
|
||||
pod_infra_container: ""
|
||||
ingress: ""
|
||||
ingress_backend: ""
|
||||
metrics_server: ""
|
||||
windows_pod_infra_container: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert_path: ""
|
||||
ssh_agent_auth: false
|
||||
authorization:
|
||||
mode: ""
|
||||
options: {}
|
||||
ignore_docker_version: false
|
||||
kubernetes_version: v1.18.12-rancher1-1
|
||||
private_registries: []
|
||||
ingress:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
extra_args: {}
|
||||
dns_policy: ""
|
||||
extra_envs: []
|
||||
extra_volumes: []
|
||||
extra_volume_mounts: []
|
||||
update_strategy: null
|
||||
http_port: 0
|
||||
https_port: 0
|
||||
network_mode: ""
|
||||
cluster_name:
|
||||
cloud_provider:
|
||||
name: ""
|
||||
prefix_path: ""
|
||||
win_prefix_path: ""
|
||||
addon_job_timeout: 0
|
||||
bastion_host:
|
||||
address: ""
|
||||
port: ""
|
||||
user: ""
|
||||
ssh_key: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert: ""
|
||||
ssh_cert_path: ""
|
||||
monitoring:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
update_strategy: null
|
||||
replicas: null
|
||||
restore:
|
||||
restore: false
|
||||
snapshot_name: ""
|
||||
dns: null
|
||||
upgrade_strategy:
|
||||
max_unavailable_worker: ""
|
||||
max_unavailable_controlplane: ""
|
||||
drain: null
|
||||
node_drain_input: null
|
||||
```
|
||||
|
||||
### Reference Hardened RKE Template configuration
|
||||
|
||||
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
|
||||
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
|
||||
[documentaion](https://rancher.com/docs/rancher/v2.6/en/installation) for additional installation and RKE Template details.
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
ignore_docker_version: true
|
||||
kubernetes_version: v1.18.12-rancher1-1
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
mtu: 0
|
||||
plugin: canal
|
||||
rotate_encryption_key: false
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: false
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: '5000'
|
||||
heartbeat-interval: '500'
|
||||
gid: 52034
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 52034
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
service_node_port_range: 30000-32767
|
||||
kube_controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: 'true'
|
||||
tls-cipher-suites: >-
|
||||
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
ssh_agent_auth: false
|
||||
upgrade_strategy:
|
||||
max_unavailable_controlplane: '1'
|
||||
max_unavailable_worker: 10%
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
|
||||
### Hardened Reference Ubuntu 20.04 LTS **cloud-config**:
|
||||
|
||||
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
|
||||
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
|
||||
needed before installing kubernetes.
|
||||
|
||||
```yaml
|
||||
#cloud-config
|
||||
apt:
|
||||
sources:
|
||||
docker.list:
|
||||
source: deb [arch=amd64] http://download.docker.com/linux/ubuntu $RELEASE stable
|
||||
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
|
||||
system_info:
|
||||
default_user:
|
||||
groups:
|
||||
- docker
|
||||
write_files:
|
||||
- path: "/etc/apt/preferences.d/docker"
|
||||
owner: root:root
|
||||
permissions: '0600'
|
||||
content: |
|
||||
Package: docker-ce
|
||||
Pin: version 5:19*
|
||||
Pin-Priority: 800
|
||||
- path: "/etc/sysctl.d/90-kubelet.conf"
|
||||
owner: root:root
|
||||
permissions: '0644'
|
||||
content: |
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
package_update: true
|
||||
packages:
|
||||
- docker-ce
|
||||
- docker-ce-cli
|
||||
- containerd.io
|
||||
runcmd:
|
||||
- sysctl -p /etc/sysctl.d/90-kubelet.conf
|
||||
- groupadd --gid 52034 etcd
|
||||
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
@@ -1,55 +0,0 @@
|
||||
---
|
||||
title: Self-Assessment and Hardening Guides for Rancher v2.5
|
||||
shortTitle: Rancher v2.5 Guides
|
||||
weight: 1
|
||||
---
|
||||
|
||||
Rancher v2.5 introduced the capability to deploy Rancher on any Kubernetes cluster. For that reason, we now provide separate security hardening guides for Rancher deployments on each of Rancher's Kubernetes distributions.
|
||||
|
||||
- [Rancher Kubernetes Distributions](#rancher-kubernetes-distributions)
|
||||
- [Hardening Guides and Benchmark Versions](#hardening-guides-and-benchmark-versions)
|
||||
- [RKE Guides](#rke-guides)
|
||||
- [RKE2 Guides](#rke2-guides)
|
||||
- [K3s Guides](#k3s)
|
||||
- [Rancher with SELinux](#rancher-with-selinux)
|
||||
|
||||
# Rancher Kubernetes Distributions
|
||||
|
||||
Rancher has the following Kubernetes distributions:
|
||||
|
||||
- [**RKE,**]({{<baseurl>}}/rke/latest/en/) Rancher Kubernetes Engine, is a CNCF-certified Kubernetes distribution that runs entirely within Docker containers.
|
||||
- [**K3s,**]({{<baseurl>}}/k3s/latest/en/) is a fully conformant, lightweight Kubernetes distribution. It is easy to install, with half the memory of upstream Kubernetes, all in a binary of less than 100 MB.
|
||||
- [**RKE2**](https://docs.rke2.io/) is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.
|
||||
|
||||
To harden a Kubernetes cluster outside of Rancher's distributions, refer to your Kubernetes provider docs.
|
||||
|
||||
# Hardening Guides and Benchmark Versions
|
||||
|
||||
These guides have been tested along with the Rancher v2.5 release. Each self-assessment guide is accompanied with a hardening guide and tested on a specific Kubernetes version and CIS benchmark version. If a CIS benchmark has not been validated for your Kubernetes version, you can choose to use the existing guides until a newer version is added.
|
||||
|
||||
### RKE Guides
|
||||
|
||||
Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides
|
||||
---|---|---|---
|
||||
Kubernetes v1.15+ | CIS v1.5 | [Link](./1.5-benchmark-2.5) | [Link](./1.5-hardening-2.5)
|
||||
Kubernetes v1.18+ | CIS v1.6 | [Link](./1.6-benchmark-2.5) | [Link](./1.6-hardening-2.5)
|
||||
|
||||
### RKE2 Guides
|
||||
|
||||
Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guides
|
||||
---|---|---|---
|
||||
Kubernetes v1.18 | CIS v1.5 | [Link](https://docs.rke2.io/security/cis_self_assessment15/) | [Link](https://docs.rke2.io/security/hardening_guide/)
|
||||
Kubernetes v1.20 | CIS v1.6 | [Link](https://docs.rke2.io/security/cis_self_assessment16/) | [Link](https://docs.rke2.io/security/hardening_guide/)
|
||||
|
||||
### K3s Guides
|
||||
|
||||
Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guide
|
||||
---|---|---|---
|
||||
Kubernetes v1.17, v1.18, & v1.19 | CIS v1.5 | [Link]({{<baseurl>}}/k3s/latest/en/security/self_assessment/) | [Link]({{<baseurl>}}/k3s/latest/en/security/hardening_guide/)
|
||||
|
||||
|
||||
# Rancher with SELinux
|
||||
|
||||
[Security-Enhanced Linux (SELinux)](https://en.wikipedia.org/wiki/Security-Enhanced_Linux) is a security enhancement to Linux. After being historically used by government agencies, SELinux is now industry standard and is enabled by default on CentOS 7 and 8.
|
||||
|
||||
To use Rancher with SELinux, we recommend installing the `rancher-selinux` RPM according to the instructions on [this page.]({{<baseurl>}}/rancher/v2.6/en/security/selinux/#installing-the-rancher-selinux-rpm)
|
||||
Reference in New Issue
Block a user