Merge pull request #3897 from macedogm/rancher/35735-update-hardening-docs-k3s

Update K3s hardening guide to CIS Kubernetes Benchmark 1.6
This commit is contained in:
Billy Tat
2022-03-16 15:11:05 -07:00
committed by GitHub
7 changed files with 2539 additions and 1874 deletions
+1 -1
View File
@@ -5,7 +5,7 @@ weight: 90
This section describes the methodology and means of securing a K3s cluster. It's broken into 2 sections. These guides assume k3s is running with embedded etcd.
The documents below apply to both CIS 1.5 & 1.6.
The documents below apply to CIS Kubernetes Benchmark v1.6.
* [Hardening Guide](./hardening_guide/)
* [CIS Benchmark Self-Assessment Guide](./self_assessment/)
@@ -3,12 +3,12 @@ title: "CIS Hardening Guide"
weight: 80
---
This document provides prescriptive guidance for hardening a production installation of K3s. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
This document provides prescriptive guidance for hardening a production installation of K3s. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Internet Security (CIS).
K3s has a number of security mitigations applied and turned on by default and will pass a number of the Kubernetes CIS controls without modification. There are some notable exceptions to this that require manual intervention to fully comply with the CIS Benchmark:
1. K3s will not modify the host operating system. Any host-level modifications will need to be done manually.
2. Certain CIS policy controls for PodSecurityPolicies and NetworkPolicies will restrict the functionality of this cluster. You must opt into having K3s configure these by adding the appropriate options (enabling of admission plugins) to your command-line flags or configuration file as well as manually applying appropriate policies. Further detail in the sections below.
2. Certain CIS policy controls for `PodSecurityPolicies` and `NetworkPolicies` will restrict the functionality of the cluster. You must opt into having K3s configure these by adding the appropriate options (enabling of admission plugins) to your command-line flags or configuration file as well as manually applying appropriate policies. Further details are presented in the sections below.
The first section (1.1) of the CIS Benchmark concerns itself primarily with pod manifest permissions and ownership. K3s doesn't utilize these for the core components since everything is packaged into a single binary.
@@ -31,23 +31,24 @@ vm.panic_on_oom=0
vm.overcommit_memory=1
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxbytes=25000000
```
## Kubernetes Runtime Requirements
The runtime requirements to comply with the CIS Benchmark are centered around pod security (PSPs) and network policies. These are outlined in this section. K3s doesn't apply any default PSPs or network policies however K3s ships with a controller that is meant to apply a given set of network policies. By default, K3s runs with the "NodeRestriction" admission controller. To enable PSPs, add the following to the K3s start command: `--kube-apiserver-arg="enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ServiceAccount"`. This will have the effect of maintaining the "NodeRestriction" plugin as well as enabling the "PodSecurityPolicy".
The runtime requirements to comply with the CIS Benchmark are centered around pod security (PSPs), network policies and API Server auditing logs. These are outlined in this section. K3s doesn't apply any default PSPs or network policies. However, K3s ships with a controller that is meant to apply a given set of network policies. By default, K3s runs with the `NodeRestriction` admission controller. To enable PSPs, add the following to the K3s start command: `--kube-apiserver-arg="enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ServiceAccount"`. This will have the effect of maintaining the `NodeRestriction` plugin as well as enabling the `PodSecurityPolicy`. The same happens with the API Server auditing logs, K3s doesn't enable them by default, so audit log configuration and audit policy must be created manually.
### PodSecurityPolicies
### Pod Security Policies
When PSPs are enabled, a policy can be applied to satisfy the necessary controls described in section 5.2 of the CIS Benchmark.
Here's an example of a compliant PSP.
Here is an example of a compliant PSP.
```yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: cis1.5-compliant-psp
name: restricted-psp
spec:
privileged: false # CIS - 5.2.1
allowPrivilegeEscalation: false # CIS - 5.2.5
@@ -59,7 +60,9 @@ spec:
- 'projected'
- 'secret'
- 'downwardAPI'
- 'csi'
- 'persistentVolumeClaim'
- 'ephemeral'
hostNetwork: false # CIS - 5.2.4
hostIPC: false # CIS - 5.2.3
hostPID: false # CIS - 5.2.2
@@ -80,7 +83,7 @@ spec:
readOnlyRootFilesystem: false
```
Before the above PSP to be effective, we need to create a couple ClusterRoles and ClusterRole. We also need to include a "system unrestricted policy" which is needed for system-level pods that require additional privileges.
For the above PSP to be effective, we need to create a ClusterRole and a ClusterRoleBinding. We also need to include a "system unrestricted policy" which is needed for system-level pods that require additional privileges.
These can be combined with the PSP yaml above and NetworkPolicy yaml below into a single file and placed in the `/var/lib/rancher/k3s/server/manifests` directory. Below is an example of a `policy.yaml` file.
@@ -88,7 +91,7 @@ These can be combined with the PSP yaml above and NetworkPolicy yaml below into
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: cis1.5-compliant-psp
name: restricted-psp
spec:
privileged: false
allowPrivilegeEscalation: false
@@ -100,7 +103,9 @@ spec:
- 'projected'
- 'secret'
- 'downwardAPI'
- 'csi'
- 'persistentVolumeClaim'
- 'ephemeral'
hostNetwork: false
hostIPC: false
hostPID: false
@@ -123,7 +128,7 @@ spec:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp:restricted
name: psp:restricted-psp
labels:
addonmanager.kubernetes.io/mode: EnsureExists
rules:
@@ -131,62 +136,23 @@ rules:
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames:
- cis1.5-compliant-psp
- restricted-psp
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: default:restricted
name: default:restricted-psp
labels:
addonmanager.kubernetes.io/mode: EnsureExists
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp:restricted
name: psp:restricted-psp
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: kube-system
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: kube-system
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: default
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: default
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: kube-public
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: kube-public
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
@@ -253,6 +219,45 @@ subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: kube-system
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: kube-system
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: default
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: default
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: intra-namespace
namespace: kube-public
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
name: kube-public
```
> **Note:** The Kubernetes critical additions such as CNI, DNS, and Ingress are ran as pods in the `kube-system` namespace. Therefore, this namespace will have a policy that is less restrictive so that these components can run properly.
@@ -263,7 +268,7 @@ subjects:
CIS requires that all namespaces have a network policy applied that reasonably limits traffic into namespaces and pods.
Here's an example of a compliant network policy.
Here is an example of a compliant network policy.
```yaml
kind: NetworkPolicy
@@ -302,7 +307,7 @@ spec:
- Ingress
```
The metrics-server and Traefik ingress controller will be blocked by default if network policies are not created to allow access. Traefik v1 as packaged in K3s version 1.20 and below uses different labels than Traefik v2; ensure that you only use the sample yaml below that is associated with the version of Traefik present on your cluster.
The metrics-server and Traefik ingress controller will be blocked by default if network policies are not created to allow access. Traefik v1 as packaged in K3s version 1.20 and below uses different labels than Traefik v2. Ensure that you only use the sample yaml below that is associated with the version of Traefik present on your cluster.
```yaml
apiVersion: networking.k8s.io/v1
@@ -366,24 +371,67 @@ spec:
> **Note:** Operators must manage network policies as normal for additional namespaces that are created.
### API Server audit configuration
CIS requirements 1.2.22 to 1.2.25 are related to configuring audit logs for the API Server. K3s doesn't create by default the log directory and audit policy, as auditing requirements are specific to each user's policies and environment.
The log directory, ideally, must be created before starting K3s. A restrictive access permission is recommended to avoid leaking potential sensitive information.
```bash
sudo mkdir -p -m 700 /var/lib/rancher/k3s/server/logs
```
A starter audit policy to log request metadata is provided below. The policy should be written to a file named `audit.yaml` in `/var/lib/rancher/k3s/server` directory. Detailed information about policy configuration for the API server can be found in the Kubernetes [documentation](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/).
```yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
```
Both configurations must be passed as arguments to the API Server as:
```bash
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log'
--kube-apiserver-arg='audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml'
```
If the configurations are created after K3s is installed, they must be added to K3s' systemd service in `/etc/systemd/system/k3s.service`.
```bash
ExecStart=/usr/local/bin/k3s \
server \
'--kube-apiserver-arg=audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log' \
'--kube-apiserver-arg=audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml' \
```
K3s must be restarted to load the new configuration.
```bash
sudo systemctl daemon-reload
sudo systemctl restart k3s.service
```
Additional information about CIS requirements 1.2.22 to 1.2.25 is presented below.
## Known Issues
The following are controls that K3s currently does not pass by default. Each gap will be explained, along with a note clarifying whether it can be passed through manual operator intervention, or if it will be addressed in a future release of K3s.
### Control 1.2.15
Ensure that the admission control plugin `NamespaceLifecycle` is set.
<details>
<summary>Rationale</summary>
Setting admission control policy to NamespaceLifecycle ensures that objects cannot be created in non-existent namespaces, and that namespaces undergoing termination are not used for creating the new objects. This is recommended to enforce the integrity of the namespace termination process and also for the availability of the newer objects.
Setting admission control policy to `NamespaceLifecycle` ensures that objects cannot be created in non-existent namespaces, and that namespaces undergoing termination are not used for creating the new objects. This is recommended to enforce the integrity of the namespace termination process and also for the availability of the newer objects.
This can be remediated by passing this argument as a value to the `enable-admission-plugins=` and pass that to `--kube-apiserver-arg=` argument to `k3s server`. An example can be found below.
</details>
### Control 1.2.16 (mentioned above)
### Control 1.2.16
Ensure that the admission control plugin `PodSecurityPolicy` is set.
<details>
<summary>Rationale</summary>
A Pod Security Policy is a cluster-level resource that controls the actions that a pod can perform and what it has the ability to access. The PodSecurityPolicy objects define a set of conditions that a pod must run with in order to be accepted into the system. Pod Security Policies are comprised of settings and strategies that control the security features a pod has access to and hence this must be used to control pod access permissions.
A Pod Security Policy is a cluster-level resource that controls the actions that a pod can perform and what it has the ability to access. The `PodSecurityPolicy` objects define a set of conditions that a pod must run with in order to be accepted into the system. Pod Security Policies are comprised of settings and strategies that control the security features a pod has access to and hence this must be used to control pod access permissions.
This can be remediated by passing this argument as a value to the `enable-admission-plugins=` and pass that to `--kube-apiserver-arg=` argument to `k3s server`. An example can be found below.
</details>
@@ -446,16 +494,18 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
Ensure that the `--encryption-provider-config` argument is set as appropriate.
<details>
<summary>Rationale</summary>
Where `etcd` encryption is used, it is important to ensure that the appropriate set of encryption providers is used. Currently, the aescbc, kms and secretbox are likely to be appropriate options.
`etcd` is a highly available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.
Detailed steps on how to configure secrets encryption in K3s are available in [Secrets Encryption](../secrets_encryption/).
</details>
### Control 1.2.34
Ensure that encryption providers are appropriately configured.
<details>
<summary>Rationale</summary>
`etcd` is a highly available key-value store used by Kubernetes deployments for persistent storage of all of its REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.
Where `etcd` encryption is used, it is important to ensure that the appropriate set of encryption providers is used. Currently, the `aescbc`, `kms` and `secretbox` are likely to be appropriate options.
This can be remediated by passing a valid configuration to `k3s` as outlined above.
This can be remediated by passing a valid configuration to `k3s` as outlined above. Detailed steps on how to configure secrets encryption in K3s are available in [Secrets Encryption](../secrets_encryption/).
</details>
### Control 1.3.1
@@ -468,7 +518,7 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
</details>
### Control 3.2.1
Ensure that a minimal audit policy is created (Scored)
Ensure that a minimal audit policy is created.
<details>
<summary>Rationale</summary>
Logging is an important detective control for all systems, to detect potential unauthorized access.
@@ -476,7 +526,6 @@ Logging is an important detective control for all systems, to detect potential u
This can be remediated by passing controls 1.2.22 - 1.2.25 and verifying their efficacy.
</details>
### Control 4.2.7
Ensure that the `--make-iptables-util-chains` argument is set to true.
<details>
@@ -487,24 +536,23 @@ This can be remediated by passing this argument as a value to the `--kube-apiser
</details>
### Control 5.1.5
Ensure that default service accounts are not actively used. (Scored)
Ensure that default service accounts are not actively used
<details>
<summary>Rationale</summary>
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod.
Kubernetes provides a `default` service account which is used by cluster workloads where no specific service account is assigned to the pod.
Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account.
The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
</details>
The remediation for this is to update the `automountServiceAccountToken` field to `false` for the `default` service account in each namespace.
This can be remediated by updating the `automountServiceAccountToken` field to `false` for the `default` service account in each namespace.
For `default` service accounts in the built-in namespaces (`kube-system`, `kube-public`, `kube-node-lease`, and `default`), K3s does not automatically do this. You can manually update this field on these service accounts to pass the control.
</details>
## Control Plane Execution and Arguments
Listed below are the K3s control plane components and the arguments they're given at start, by default. Commented to their right is the CIS 1.5 control that they satisfy.
Listed below are the K3s control plane components and the arguments they are given at start, by default. Commented to their right is the CIS 1.6 control that they satisfy.
```bash
kube-apiserver
@@ -604,13 +652,14 @@ kubelet
--tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key # 4.2.10
```
The command below is an example of how the outlined remediations can be applied.
The command below is an example of how the outlined remediations can be applied to harden K3s.
```bash
k3s server \
--protect-kernel-defaults=true \
--secrets-encryption=true \
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit-log' \
--kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log' \
--kube-apiserver-arg='audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml' \
--kube-apiserver-arg='audit-log-maxage=30' \
--kube-apiserver-arg='audit-log-maxbackup=10' \
--kube-apiserver-arg='audit-log-maxsize=100' \
@@ -625,4 +674,4 @@ k3s server \
## Conclusion
If you have followed this guide, your K3s cluster will be configured to comply with the CIS Kubernetes Benchmark. You can review the [CIS Benchmark Self-Assessment Guide](../self_assessment/) to understand the expectations of each of the benchmarks and how you can do the same on your cluster.
If you have followed this guide, your K3s cluster will be configured to comply with the CIS Kubernetes Benchmark. You can review the [CIS Benchmark Self-Assessment Guide](../self_assessment/) to understand the expectations of each of the benchmark's checks and how you can do the same on your cluster.
File diff suppressed because it is too large Load Diff
+36
View File
@@ -0,0 +1,36 @@
---
title: CIS v1.6 Benchmark - Self-Assessment Guide - Rancher v2.6
weight: 101
---
### CIS v1.6 Kubernetes Benchmark - Rancher v2.6 with Kubernetes v1.18 to v1.21
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.6/Rancher_v2-6_CIS_v1-6_Benchmark_Assessment.pdf).
#### Overview
This document is a companion to the [Rancher v2.6 security hardening guide]({{<baseurl>}}/rancher/v2.6/en/security/hardening-guides/). The hardening guide provides prescriptive guidance for hardening a production installation of Rancher, and this benchmark guide is meant to help you evaluate the level of security of the hardened cluster against each control in the benchmark.
This guide corresponds to specific versions of the hardening guide, Rancher, CIS Benchmark and Kubernetes:
| Hardening Guide Version | Rancher Version | CIS Benchmark Version | Kubernetes Version |
| ----------------------- | --------------- | --------------------- | ------------------- |
| Hardening Guide CIS v1.6 Benchmark | Rancher v2.6.3 | CIS v1.6 | Kubernetes v1.18, v1.19, v1.20 and v1.21 |
Because Rancher and RKE install Kubernetes services as Docker containers, many of the control verification checks in the CIS Kubernetes Benchmark do not apply and will have a result of \`Not Applicable\`. This guide will walk through the various controls and provide updated example commands to audit compliance in Rancher created clusters.
This document is to be used by Rancher operators, security teams, auditors and decision makers.
For more detail about each audit, including rationales and remediations for failing tests, you can refer to the corresponding section of the CIS Kubernetes Benchmark v1.6. You can download the benchmark, after creating a free account, in [Center for Internet Security (CIS)](https://www.cisecurity.org/benchmark/kubernetes/).
#### Testing controls methodology
Rancher and RKE install Kubernetes services via Docker containers. Configuration is defined by arguments passed to the container at the time of initialization, not via configuration files.
Where control audits differ from the original CIS benchmark, the audit commands specific to Rancher are provided for testing. When performing the tests, you will need access to the Docker command line on the hosts of all three RKE roles. The commands also make use of the [kubectl](https://kubernetes.io/docs/tasks/tools/) (with a valid configuration file) and [jq](https://stedolan.github.io/jq/) tools, which are required in the testing and evaluation of test results.
> NOTE: Only `automated` tests (previously called `scored`) are covered in this guide.
### Controls
---
+35
View File
@@ -0,0 +1,35 @@
---
title: CIS Self Assessment Guide
weight: 90
---
### CIS Kubernetes Benchmark v1.6 - K3s with Kubernetes v1.17 to v1.21
#### Overview
This document is a companion to the [K3s security hardening guide]({{<baseurl>}}/k3s/latest/en/security/hardening_guide/). The hardening guide provides prescriptive guidance for hardening a production installation of K3s, and this benchmark guide is meant to help you evaluate the level of security of the hardened cluster against each control in the CIS Kubernetes Benchmark. It is to be used by K3s operators, security teams, auditors, and decision-makers.
This guide is specific to the **v1.17**, **v1.18**, **v1.19**, **v1.20** and **v1.21** release line of K3s and the **v1.6** release of the CIS Kubernetes Benchmark.
For more information about each control, including detailed descriptions and remediations for failing tests, you can refer to the corresponding section of the CIS Kubernetes Benchmark v1.6. You can download the benchmark, after creating a free account, in [Center for Internet Security (CIS)](https://www.cisecurity.org/benchmark/kubernetes/).
#### Testing controls methodology
Each control in the CIS Kubernetes Benchmark was evaluated against a K3s cluster that was configured according to the accompanying hardening guide.
Where control audits differ from the original CIS benchmark, the audit commands specific to K3s are provided for testing.
These are the possible results for each control:
- **Pass** - The K3s cluster under test passed the audit outlined in the benchmark.
- **Not Applicable** - The control is not applicable to K3s because of how it is designed to operate. The remediation section will explain why this is so.
- **Warn** - The control is manual in the CIS benchmark and it depends on the cluster's use case or some other factor that must be determined by the cluster operator. These controls have been evaluated to ensure K3s does not prevent their implementation, but no further configuration or auditing of the cluster under test has been performed.
This guide makes the assumption that K3s is running as a Systemd unit. Your installation may vary and will require you to adjust the "audit" commands to fit your scenario.
> NOTE: Only `automated` tests (previously called `scored`) are covered in this guide.
### Controls
---
+2 -1
View File
@@ -2,8 +2,9 @@
results=${1:?path to kube-bench json results is a required argument}
test_helpers=${2:?path to kube-bench test_helpers scripts is a required argument}
header=${3:?path to header file is a required argument}
[ -f ${results} ] || (echo "file:'${results}' does not exist"; exit 1)
[ -d ${test_helpers} ] || (echo "dir: '${test_helpers}' not a valid directory"; exit 1)
docker run -v${results}:/source/results.json -v ${test_helpers}:/test_helpers -it --rm doc_converters:latest results_to_md
docker run -v ${results}:/source/results.json -v ${test_helpers}:/test_helpers -v ${header}:/headers/header.md -it --rm doc_converters:latest results_to_md
+4 -41
View File
@@ -1,48 +1,11 @@
#!/bin/bash
#results_file="${1:-/source/results.json}"
results_file="${1:-/home/paraglade/brain/projects/cis_benchmark/clusters/cis/csr.json}"
#test_helpers="${2:-/test_helpers}"
test_helpers="${2:-/home/paraglade/brain/repos/rancher-security-scan/package/helper_scripts}"
results_file="${1:-/source/results.json}"
test_helpers="${2:-/test_helpers}"
header_file="${3:-/headers/header.md}"
header() {
cat <<EOF
---
title: CIS 1.6 Benchmark - Self-Assessment Guide - Rancher v2.5
weight: 101
---
### CIS v1.6 Kubernetes Benchmark - Rancher v2.5 with Kubernetes v1.18
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_1.6_Benchmark_Assessment.pdf)
#### Overview
This document is a companion to the Rancher v2.5 security hardening guide. The hardening guide provides prescriptive guidance for hardening a production installation of Rancher, and this benchmark guide is meant to help you evaluate the level of security of the hardened cluster against each control in the benchmark.
This guide corresponds to specific versions of the hardening guide, Rancher, CIS Benchmark, and Kubernetes:
Hardening Guide Version | Rancher Version | CIS Benchmark Version | Kubernetes Version
---------------------------|----------|---------|-------
Hardening Guide with CIS 1.5 Benchmark | Rancher v2.5 | CIS v1.5| Kubernetes v1.15
Because Rancher and RKE install Kubernetes services as Docker containers, many of the control verification checks in the CIS Kubernetes Benchmark don't apply and will have a result of \`Not Applicable\`. This guide will walk through the various controls and provide updated example commands to audit compliance in Rancher-created clusters.
This document is to be used by Rancher operators, security teams, auditors and decision makers.
For more detail about each audit, including rationales and remediations for failing tests, you can refer to the corresponding section of the CIS Kubernetes Benchmark v1.5. You can download the benchmark after logging in to [CISecurity.org]( https://www.cisecurity.org/benchmark/kubernetes/).
#### Testing controls methodology
Rancher and RKE install Kubernetes services via Docker containers. Configuration is defined by arguments passed to the container at the time of initialization, not via configuration files.
Where control audits differ from the original CIS benchmark, the audit commands specific to Rancher Labs are provided for testing.
When performing the tests, you will need access to the Docker command line on the hosts of all three RKE roles. The commands also make use of the the [jq](https://stedolan.github.io/jq/) and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) (with valid config) tools to and are required in the testing and evaluation of test results.
> NOTE: only scored tests are covered in this guide.
### Controls
EOF
cat ${header_file}
}
get_ids() {