mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-26 21:18:04 +00:00
2.5: Apply Divio and update links
This commit is contained in:
+12
@@ -0,0 +1,12 @@
|
||||
---
|
||||
title: Kubernetes Security Best Practices
|
||||
weight: 5
|
||||
---
|
||||
|
||||
### Restricting cloud metadata API access
|
||||
|
||||
Cloud providers such as AWS, Azure, DigitalOcean or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS, DigitalOcean Kubernetes or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.
|
||||
|
||||
It is advised to consult your cloud provider's security best practices for further recommendations and specific details on how to restrict access to cloud instance metadata API.
|
||||
|
||||
Further references: MITRE ATT&CK knowledge base on - [Unsecured Credentials: Cloud Instance Metadata API](https://attack.mitre.org/techniques/T1552/005/).
|
||||
+724
@@ -0,0 +1,724 @@
|
||||
---
|
||||
title: Hardening Guide with CIS 1.5 Benchmark
|
||||
weight: 200
|
||||
aliases:
|
||||
- /rancher/v2.x/en/security/rancher-2.5/1.5-hardening-2.5/
|
||||
---
|
||||
|
||||
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
Rancher Version | CIS Benchmark Version | Kubernetes Version
|
||||
----------------|-----------------------|------------------
|
||||
Rancher v2.5 | Benchmark v1.5 | Kubernetes 1.15
|
||||
|
||||
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.5.pdf)
|
||||
|
||||
### Overview
|
||||
|
||||
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5 with Kubernetes v1.15 or provisioning a RKE cluster with Kubernetes 1.15 to be used within Rancher v2.5. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.5 Benchmark - Self-Assessment Guide - Rancher v2.5](self-assessment-guide-with-cis-v1.5-benchmark.md).
|
||||
|
||||
#### Known Issues
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.5 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.5 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
|
||||
|
||||
#### create `etcd` user and group
|
||||
To create the **etcd** group run the following console commands.
|
||||
|
||||
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
|
||||
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
|
||||
|
||||
``` yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
|
||||
|
||||
```
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following yaml to a file called `account_update.yaml`
|
||||
|
||||
``` yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one
|
||||
compromised application attacking a neighboring application. Network segmentation is
|
||||
important to ensure that containers can communicate only with those they are supposed
|
||||
to. A network policy is a specification of how selections of pods are allowed to
|
||||
communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given
|
||||
namespace, all traffic not allowed by the policy is denied. However, if there are no network
|
||||
policies in a namespace all traffic will be allowed into and out of the pods in that
|
||||
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
|
||||
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
|
||||
Additional information about CNI providers can be found
|
||||
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
|
||||
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
|
||||
(even if policies are added that cause some pods to be treated as “isolated”),
|
||||
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
|
||||
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
about network policies can be found on the Kubernetes site.
|
||||
|
||||
> This `NetworkPolicy` is not recommended for production use
|
||||
|
||||
``` yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
|
||||
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
### Reference Hardened RKE `cluster.yml` configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
|
||||
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
|
||||
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
|
||||
|
||||
``` yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
kubernetes_version: "v1.15.9-rancher1-1"
|
||||
enable_network_policy: true
|
||||
default_pod_security_policy_template_id: "restricted"
|
||||
# the nodes directive is required and will vary depending on your environment
|
||||
# documentation for node configuration can be found here:
|
||||
# https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
nodes:
|
||||
services:
|
||||
etcd:
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
kube-api:
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
audit_log:
|
||||
enabled: true
|
||||
admission_configuration:
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
kube-controller:
|
||||
extra_args:
|
||||
feature-gates: "RotateKubeletServerCertificate=true"
|
||||
scheduler:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
kubelet:
|
||||
generate_serving_certificate: true
|
||||
extra_args:
|
||||
feature-gates: "RotateKubeletServerCertificate=true"
|
||||
protect-kernel-defaults: "true"
|
||||
tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
cluster_domain: ""
|
||||
infra_container_image: ""
|
||||
cluster_dns_server: ""
|
||||
fail_swap_on: false
|
||||
kubeproxy:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
network:
|
||||
plugin: ""
|
||||
options: {}
|
||||
mtu: 0
|
||||
node_selector: {}
|
||||
authentication:
|
||||
strategy: ""
|
||||
sans: []
|
||||
webhook: null
|
||||
addons: |
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: ingress-nginx
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: ingress-nginx
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cattle-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: cattle-system
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: cattle-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tiller
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
|
||||
addons_include: []
|
||||
system_images:
|
||||
etcd: ""
|
||||
alpine: ""
|
||||
nginx_proxy: ""
|
||||
cert_downloader: ""
|
||||
kubernetes_services_sidecar: ""
|
||||
kubedns: ""
|
||||
dnsmasq: ""
|
||||
kubedns_sidecar: ""
|
||||
kubedns_autoscaler: ""
|
||||
coredns: ""
|
||||
coredns_autoscaler: ""
|
||||
kubernetes: ""
|
||||
flannel: ""
|
||||
flannel_cni: ""
|
||||
calico_node: ""
|
||||
calico_cni: ""
|
||||
calico_controllers: ""
|
||||
calico_ctl: ""
|
||||
calico_flexvol: ""
|
||||
canal_node: ""
|
||||
canal_cni: ""
|
||||
canal_flannel: ""
|
||||
canal_flexvol: ""
|
||||
weave_node: ""
|
||||
weave_cni: ""
|
||||
pod_infra_container: ""
|
||||
ingress: ""
|
||||
ingress_backend: ""
|
||||
metrics_server: ""
|
||||
windows_pod_infra_container: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert_path: ""
|
||||
ssh_agent_auth: false
|
||||
authorization:
|
||||
mode: ""
|
||||
options: {}
|
||||
ignore_docker_version: false
|
||||
private_registries: []
|
||||
ingress:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
extra_args: {}
|
||||
dns_policy: ""
|
||||
extra_envs: []
|
||||
extra_volumes: []
|
||||
extra_volume_mounts: []
|
||||
cluster_name: ""
|
||||
prefix_path: ""
|
||||
addon_job_timeout: 0
|
||||
bastion_host:
|
||||
address: ""
|
||||
port: ""
|
||||
user: ""
|
||||
ssh_key: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert: ""
|
||||
ssh_cert_path: ""
|
||||
monitoring:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
restore:
|
||||
restore: false
|
||||
snapshot_name: ""
|
||||
dns: null
|
||||
```
|
||||
|
||||
### Reference Hardened RKE Template configuration
|
||||
|
||||
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
|
||||
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
|
||||
[documentaion](https://rancher.com/docs/rancher/v2.5/en/installation) for additional installation and RKE Template details.
|
||||
|
||||
``` yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 30
|
||||
addons: |-
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress-nginx
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: ingress-nginx
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: ingress-nginx
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cattle-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: default-psp-role
|
||||
namespace: cattle-system
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- default-psp
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: default-psp-rolebinding
|
||||
namespace: cattle-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: default-psp-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tiller
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tiller
|
||||
namespace: kube-system
|
||||
ignore_docker_version: true
|
||||
kubernetes_version: v1.15.9-rancher1-1
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
mtu: 0
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: false
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: '5000'
|
||||
heartbeat-interval: '500'
|
||||
gid: 52034
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 52034
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
service_node_port_range: 30000-32767
|
||||
kube_controller:
|
||||
extra_args:
|
||||
bind-address: 127.0.0.1
|
||||
address: 127.0.0.1
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
profiling: 'false'
|
||||
terminated-pod-gc-threshold: '1000'
|
||||
kubelet:
|
||||
extra_args:
|
||||
anonymous-auth: 'false'
|
||||
event-qps: '0'
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
make-iptables-util-chains: 'true'
|
||||
protect-kernel-defaults: 'true'
|
||||
streaming-connection-idle-timeout: 1800s
|
||||
tls-cipher-suites: >-
|
||||
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
bind-address: 127.0.0.1
|
||||
address: 127.0.0.1
|
||||
profiling: 'false'
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
|
||||
### Hardened Reference Ubuntu 18.04 LTS **cloud-config**:
|
||||
|
||||
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
|
||||
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
|
||||
needed before installing kubernetes.
|
||||
|
||||
``` yaml
|
||||
#cloud-config
|
||||
packages:
|
||||
- curl
|
||||
- jq
|
||||
runcmd:
|
||||
- sysctl -w vm.overcommit_memory=1
|
||||
- sysctl -w kernel.panic=10
|
||||
- sysctl -w kernel.panic_on_oops=1
|
||||
- curl https://releases.rancher.com/install-docker/18.09.sh | sh
|
||||
- usermod -aG docker ubuntu
|
||||
- return=1; while [ $return != 0 ]; do sleep 2; docker ps; return=$?; done
|
||||
- addgroup --gid 52034 etcd
|
||||
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
write_files:
|
||||
- path: /etc/sysctl.d/kubelet.conf
|
||||
owner: root:root
|
||||
permissions: "0644"
|
||||
content: |
|
||||
vm.overcommit_memory=1
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
```
|
||||
+578
@@ -0,0 +1,578 @@
|
||||
---
|
||||
title: Hardening Guide with CIS 1.6 Benchmark
|
||||
weight: 100
|
||||
aliases:
|
||||
- /rancher/v2.x/en/security/rancher-2.5/1.6-hardening-2.5/
|
||||
---
|
||||
|
||||
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5.4. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
|
||||
|
||||
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
|
||||
|
||||
Rancher Version | CIS Benchmark Version | Kubernetes Version
|
||||
----------------|-----------------------|------------------
|
||||
Rancher v2.5.4 | Benchmark 1.6 | Kubernetes v1.18
|
||||
|
||||
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.6.pdf)
|
||||
|
||||
### Overview
|
||||
|
||||
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5.4 with Kubernetes v1.18 or provisioning a RKE cluster with Kubernetes v1.18 to be used within Rancher v2.5.4. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
|
||||
|
||||
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.6 Benchmark - Self-Assessment Guide - Rancher v2.5.4](self-assessment-guide-with-cis-v1.6-benchmark.md).
|
||||
|
||||
#### Known Issues
|
||||
|
||||
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.6 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
|
||||
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.6 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
Migration Rancher from 2.4 to 2.5. Addons were removed in HG 2.5, and therefore namespaces on migration may be not created on the downstream clusters. Pod may fail to run because of missing namesapce like ingress-nginx, cattle-system.
|
||||
|
||||
### Configure Kernel Runtime Parameters
|
||||
|
||||
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
|
||||
|
||||
```ini
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
```
|
||||
|
||||
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
|
||||
|
||||
### Configure `etcd` user and group
|
||||
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
|
||||
|
||||
#### create `etcd` user and group
|
||||
To create the **etcd** group run the following console commands.
|
||||
|
||||
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
|
||||
|
||||
```bash
|
||||
groupadd --gid 52034 etcd
|
||||
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
|
||||
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
etcd:
|
||||
gid: 52034
|
||||
uid: 52034
|
||||
```
|
||||
|
||||
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
|
||||
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
|
||||
|
||||
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
|
||||
|
||||
```yaml
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Save the following yaml to a file called `account_update.yaml`
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
```
|
||||
|
||||
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
|
||||
done
|
||||
```
|
||||
|
||||
### Ensure that all Namespaces have Network Policies defined
|
||||
|
||||
Running different applications on the same Kubernetes cluster creates a risk of one
|
||||
compromised application attacking a neighboring application. Network segmentation is
|
||||
important to ensure that containers can communicate only with those they are supposed
|
||||
to. A network policy is a specification of how selections of pods are allowed to
|
||||
communicate with each other and other network endpoints.
|
||||
|
||||
Network Policies are namespace scoped. When a network policy is introduced to a given
|
||||
namespace, all traffic not allowed by the policy is denied. However, if there are no network
|
||||
policies in a namespace all traffic will be allowed into and out of the pods in that
|
||||
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
|
||||
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
|
||||
Additional information about CNI providers can be found
|
||||
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
|
||||
|
||||
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
|
||||
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
|
||||
(even if policies are added that cause some pods to be treated as “isolated”),
|
||||
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
|
||||
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
about network policies can be found on the Kubernetes site.
|
||||
|
||||
> This `NetworkPolicy` is not recommended for production use
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
```
|
||||
|
||||
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
|
||||
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
|
||||
|
||||
```bash
|
||||
#!/bin/bash -e
|
||||
|
||||
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
|
||||
kubectl apply -f default-allow-all.yaml -n ${namespace}
|
||||
done
|
||||
```
|
||||
|
||||
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
|
||||
|
||||
### Reference Hardened RKE `cluster.yml` configuration
|
||||
|
||||
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
|
||||
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
|
||||
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
|
||||
|
||||
```yaml
|
||||
# If you intend to deploy Kubernetes in an air-gapped environment,
|
||||
# please consult the documentation on how to configure custom RKE images.
|
||||
# https://rancher.com/docs/rke/latest/en/installation/
|
||||
|
||||
# the nodes directive is required and will vary depending on your environment
|
||||
# documentation for node configuration can be found here:
|
||||
# https://rancher.com/docs/rke/latest/en/config-options/nodes
|
||||
nodes: []
|
||||
services:
|
||||
etcd:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
external_urls: []
|
||||
ca_cert: ""
|
||||
cert: ""
|
||||
key: ""
|
||||
path: ""
|
||||
uid: 52034
|
||||
gid: 52034
|
||||
snapshot: false
|
||||
retention: ""
|
||||
creation: ""
|
||||
backup_config: null
|
||||
kube-api:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
service_cluster_ip_range: ""
|
||||
service_node_port_range: ""
|
||||
pod_security_policy: true
|
||||
always_pull_images: false
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
custom_config: null
|
||||
audit_log:
|
||||
enabled: true
|
||||
configuration: null
|
||||
admission_configuration: null
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
configuration: null
|
||||
kube-controller:
|
||||
image: ""
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
cluster_cidr: ""
|
||||
service_cluster_ip_range: ""
|
||||
scheduler:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
kubelet:
|
||||
image: ""
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: "true"
|
||||
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
cluster_domain: cluster.local
|
||||
infra_container_image: ""
|
||||
cluster_dns_server: ""
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
kubeproxy:
|
||||
image: ""
|
||||
extra_args: {}
|
||||
extra_binds: []
|
||||
extra_env: []
|
||||
win_extra_args: {}
|
||||
win_extra_binds: []
|
||||
win_extra_env: []
|
||||
network:
|
||||
plugin: ""
|
||||
options: {}
|
||||
mtu: 0
|
||||
node_selector: {}
|
||||
update_strategy: null
|
||||
authentication:
|
||||
strategy: ""
|
||||
sans: []
|
||||
webhook: null
|
||||
addons: |
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
name: restricted
|
||||
spec:
|
||||
requiredDropCapabilities:
|
||||
- NET_RAW
|
||||
privileged: false
|
||||
allowPrivilegeEscalation: false
|
||||
defaultAllowPrivilegeEscalation: false
|
||||
fsGroup:
|
||||
rule: RunAsAny
|
||||
runAsUser:
|
||||
rule: MustRunAsNonRoot
|
||||
seLinux:
|
||||
rule: RunAsAny
|
||||
supplementalGroups:
|
||||
rule: RunAsAny
|
||||
volumes:
|
||||
- emptyDir
|
||||
- secret
|
||||
- persistentVolumeClaim
|
||||
- downwardAPI
|
||||
- configMap
|
||||
- projected
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
rules:
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resourceNames:
|
||||
- restricted
|
||||
resources:
|
||||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp:restricted
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: psp:restricted
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:serviceaccounts
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
name: system:authenticated
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-allow-all
|
||||
spec:
|
||||
podSelector: {}
|
||||
ingress:
|
||||
- {}
|
||||
egress:
|
||||
- {}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: default
|
||||
automountServiceAccountToken: false
|
||||
addons_include: []
|
||||
system_images:
|
||||
etcd: ""
|
||||
alpine: ""
|
||||
nginx_proxy: ""
|
||||
cert_downloader: ""
|
||||
kubernetes_services_sidecar: ""
|
||||
kubedns: ""
|
||||
dnsmasq: ""
|
||||
kubedns_sidecar: ""
|
||||
kubedns_autoscaler: ""
|
||||
coredns: ""
|
||||
coredns_autoscaler: ""
|
||||
nodelocal: ""
|
||||
kubernetes: ""
|
||||
flannel: ""
|
||||
flannel_cni: ""
|
||||
calico_node: ""
|
||||
calico_cni: ""
|
||||
calico_controllers: ""
|
||||
calico_ctl: ""
|
||||
calico_flexvol: ""
|
||||
canal_node: ""
|
||||
canal_cni: ""
|
||||
canal_controllers: ""
|
||||
canal_flannel: ""
|
||||
canal_flexvol: ""
|
||||
weave_node: ""
|
||||
weave_cni: ""
|
||||
pod_infra_container: ""
|
||||
ingress: ""
|
||||
ingress_backend: ""
|
||||
metrics_server: ""
|
||||
windows_pod_infra_container: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert_path: ""
|
||||
ssh_agent_auth: false
|
||||
authorization:
|
||||
mode: ""
|
||||
options: {}
|
||||
ignore_docker_version: false
|
||||
kubernetes_version: v1.18.12-rancher1-1
|
||||
private_registries: []
|
||||
ingress:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
extra_args: {}
|
||||
dns_policy: ""
|
||||
extra_envs: []
|
||||
extra_volumes: []
|
||||
extra_volume_mounts: []
|
||||
update_strategy: null
|
||||
http_port: 0
|
||||
https_port: 0
|
||||
network_mode: ""
|
||||
cluster_name:
|
||||
cloud_provider:
|
||||
name: ""
|
||||
prefix_path: ""
|
||||
win_prefix_path: ""
|
||||
addon_job_timeout: 0
|
||||
bastion_host:
|
||||
address: ""
|
||||
port: ""
|
||||
user: ""
|
||||
ssh_key: ""
|
||||
ssh_key_path: ""
|
||||
ssh_cert: ""
|
||||
ssh_cert_path: ""
|
||||
monitoring:
|
||||
provider: ""
|
||||
options: {}
|
||||
node_selector: {}
|
||||
update_strategy: null
|
||||
replicas: null
|
||||
restore:
|
||||
restore: false
|
||||
snapshot_name: ""
|
||||
dns: null
|
||||
upgrade_strategy:
|
||||
max_unavailable_worker: ""
|
||||
max_unavailable_controlplane: ""
|
||||
drain: null
|
||||
node_drain_input: null
|
||||
```
|
||||
|
||||
### Reference Hardened RKE Template configuration
|
||||
|
||||
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
|
||||
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
|
||||
[documentaion](https://rancher.com/docs/rancher/v2.5/en/installation) for additional installation and RKE Template details.
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
default_pod_security_policy_template_id: restricted
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config:
|
||||
addon_job_timeout: 45
|
||||
ignore_docker_version: true
|
||||
kubernetes_version: v1.18.12-rancher1-1
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
mtu: 0
|
||||
plugin: canal
|
||||
rotate_encryption_key: false
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: false
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: '5000'
|
||||
heartbeat-interval: '500'
|
||||
gid: 52034
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 52034
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
audit_log:
|
||||
enabled: true
|
||||
event_rate_limit:
|
||||
enabled: true
|
||||
pod_security_policy: true
|
||||
secrets_encryption_config:
|
||||
enabled: true
|
||||
service_node_port_range: 30000-32767
|
||||
kube_controller:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
bind-address: 127.0.0.1
|
||||
address: 127.0.0.1
|
||||
kubelet:
|
||||
extra_args:
|
||||
feature-gates: RotateKubeletServerCertificate=true
|
||||
protect-kernel-defaults: 'true'
|
||||
tls-cipher-suites: >-
|
||||
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
fail_swap_on: false
|
||||
generate_serving_certificate: true
|
||||
scheduler:
|
||||
extra_args:
|
||||
bind-address: 127.0.0.1
|
||||
address: 127.0.0.1
|
||||
ssh_agent_auth: false
|
||||
upgrade_strategy:
|
||||
max_unavailable_controlplane: '1'
|
||||
max_unavailable_worker: 10%
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
|
||||
### Hardened Reference Ubuntu 20.04 LTS **cloud-config**:
|
||||
|
||||
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
|
||||
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
|
||||
needed before installing kubernetes.
|
||||
|
||||
```yaml
|
||||
#cloud-config
|
||||
apt:
|
||||
sources:
|
||||
docker.list:
|
||||
source: deb [arch=amd64] http://download.docker.com/linux/ubuntu $RELEASE stable
|
||||
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
|
||||
system_info:
|
||||
default_user:
|
||||
groups:
|
||||
- docker
|
||||
write_files:
|
||||
- path: "/etc/apt/preferences.d/docker"
|
||||
owner: root:root
|
||||
permissions: '0600'
|
||||
content: |
|
||||
Package: docker-ce
|
||||
Pin: version 5:19*
|
||||
Pin-Priority: 800
|
||||
- path: "/etc/sysctl.d/90-kubelet.conf"
|
||||
owner: root:root
|
||||
permissions: '0644'
|
||||
content: |
|
||||
vm.overcommit_memory=1
|
||||
vm.panic_on_oom=0
|
||||
kernel.panic=10
|
||||
kernel.panic_on_oops=1
|
||||
kernel.keys.root_maxbytes=25000000
|
||||
package_update: true
|
||||
packages:
|
||||
- docker-ce
|
||||
- docker-ce-cli
|
||||
- containerd.io
|
||||
runcmd:
|
||||
- sysctl -p /etc/sysctl.d/90-kubelet.conf
|
||||
- groupadd --gid 52034 etcd
|
||||
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
|
||||
```
|
||||
+2267
File diff suppressed because it is too large
Load Diff
+3319
File diff suppressed because it is too large
Load Diff
+29
@@ -0,0 +1,29 @@
|
||||
---
|
||||
title: Security Advisories and CVEs
|
||||
weight: 300
|
||||
---
|
||||
|
||||
Rancher is committed to informing the community of security issues in our products. Rancher will publish security advisories and CVEs (Common Vulnerabilities and Exposures) for issues we have resolved. New security advisories are also published in Rancher's GitHub [security page](https://github.com/rancher/rancher/security/advisories).
|
||||
|
||||
| ID | Description | Date | Resolution |
|
||||
|----|-------------|------|------------|
|
||||
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
|
||||
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rbac/) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
|
||||
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
|
||||
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Fleet](https://rancher.com/docs/rancher/v2.6/en/deploy-across-clusters/fleet/) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
|
||||
| [CVE-2021-36778](https://github.com/rancher/rancher/security/advisories/GHSA-4fc7-hc63-7fjg) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.11 and from 2.6.0 up to and including 2.6.2, where an insufficient check of the same-origin policy when downloading Helm charts from a configured private repository can lead to exposure of the repository credentials to a third-party provider. This issue only happens when the user configures access credentials to a private repository in Rancher inside `Apps & Marketplace > Repositories`. The issue was found and reported by Martin Andreas Ullrich. | 14 Apr 2022 | [Rancher v2.6.3](https://github.com/rancher/rancher/releases/tag/v2.6.3) and [Rancher v2.5.12](https://github.com/rancher/rancher/releases/tag/v2.5.12) |
|
||||
| [GHSA-hwm2-4ph6-w6m5](https://github.com/rancher/rancher/security/advisories/GHSA-hwm2-4ph6-w6m5) | A vulnerability was discovered in versions of Rancher starting 2.0 up to and including 2.6.3. The `restricted` pod security policy (PSP) provided in Rancher deviated from the upstream `restricted` policy provided in Kubernetes on account of which Rancher's PSP had `runAsUser` set to `runAsAny`, while upstream had `runAsUser` set to `MustRunAsNonRoot`. This allowed containers to run as any user, including a privileged user (`root`), even when Rancher's `restricted` policy was enforced on a project or at the cluster level. | 31 Mar 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) |
|
||||
| [CVE-2021-36775](https://github.com/rancher/rancher/security/advisories/GHSA-28g7-896h-695v) | A vulnerability was discovered in Rancher versions up to and including 2.4.17, 2.5.11 and 2.6.2. After removing a `Project Role` associated with a group from the project, the bindings that granted access to cluster-scoped resources for those subjects were not deleted. This was due to an incomplete authorization logic check. A user who was a member of the affected group with authenticated access to Rancher could exploit this vulnerability to access resources they shouldn't have had access to. The exposure level would depend on the original permission level granted to the affected project role. This vulnerability only affected customers using group based authentication in Rancher. | 31 Mar 2022 | [Rancher v2.6.3](https://github.com/rancher/rancher/releases/tag/v2.6.3), [Rancher v2.5.12](https://github.com/rancher/rancher/releases/tag/v2.5.12) and [Rancher v2.4.18](https://github.com/rancher/rancher/releases/tag/v2.4.18) |
|
||||
| [CVE-2021-36776](https://github.com/rancher/rancher/security/advisories/GHSA-gvh9-xgrq-r8hw) | A vulnerability was discovered in Rancher versions starting 2.5.0 up to and including 2.5.9, that allowed an authenticated user to impersonate any user on a cluster through an API proxy, without requiring knowledge of the impersonated user's credentials. This was due to the API proxy not dropping the impersonation header before sending the request to the Kubernetes API. A malicious user with authenticated access to Rancher could use this to impersonate another user with administrator access in Rancher, thereby gaining administrator level access to the cluster. | 31 Mar 2022 | [Rancher v2.6.0](https://github.com/rancher/rancher/releases/tag/v2.6.0) and [Rancher v2.5.10](https://github.com/rancher/rancher/releases/tag/v2.5.10) |
|
||||
| [CVE-2021-25318](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25318) | A vulnerability was discovered in Rancher versions 2.0 through the aforementioned fixed versions, where users were granted access to resources regardless of the resource's API group. For example, Rancher should have allowed users access to `apps.catalog.cattle.io`, but instead incorrectly gave access to `apps.*`. Resources affected in the **Downstream clusters** and **Rancher management cluster** can be found [here](https://github.com/rancher/rancher/security/advisories/GHSA-f9xf-jq4j-vqw4). There is not a direct mitigation besides upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
|
||||
| [CVE-2021-31999](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-31999) | A vulnerability was discovered in Rancher 2.0.0 through the aforementioned patched versions, where a malicious Rancher user could craft an API request directed at the proxy for the Kubernetes API of a managed cluster to gain access to information they do not have access to. This is done by passing the "Impersonate-User" or "Impersonate-Group" header in the Connection header, which is then correctly removed by the proxy. At this point, instead of impersonating the user and their permissions, the request will act as if it was from the Rancher management server and incorrectly return the information. The vulnerability is limited to valid Rancher users with some level of permissions on the cluster. There is not a direct mitigation besides upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
|
||||
| [CVE-2021-25320](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25320) | A vulnerability was discovered in Rancher 2.2.0 through the aforementioned patched versions, where cloud credentials weren't being properly validated through the Rancher API. Specifically through a proxy designed to communicate with cloud providers. Any Rancher user that was logged-in and aware of a cloud-credential ID that was valid for a given cloud provider, could call that cloud provider's API through the proxy API, and the cloud-credential would be attached. The exploit is limited to valid Rancher users. There is not a direct mitigation outside of upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
|
||||
| [CVE-2021-25313](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25313) | A security vulnerability was discovered on all Rancher 2 versions. When accessing the Rancher API with a browser, the URL was not properly escaped, making it vulnerable to an XSS attack. Specially crafted URLs to these API endpoints could include JavaScript which would be embedded in the page and execute in a browser. There is no direct mitigation. Avoid clicking on untrusted links to your Rancher server. | 2 Mar 2021 | [Rancher v2.5.6](https://github.com/rancher/rancher/releases/tag/v2.5.6), [Rancher v2.4.14](https://github.com/rancher/rancher/releases/tag/v2.4.14), and [Rancher v2.3.11](https://github.com/rancher/rancher/releases/tag/v2.3.11) |
|
||||
| [CVE-2019-14435](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14435) | This vulnerability allows authenticated users to potentially extract otherwise private data out of IPs reachable from system service containers used by Rancher. This can include but not only limited to services such as cloud provider metadata services. Although Rancher allow users to configure whitelisted domains for system service access, this flaw can still be exploited by a carefully crafted HTTP request. The issue was found and reported by Matt Belisle and Alex Stevenson at Workiva. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
|
||||
| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them administrator access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
|
||||
| [CVE-2019-13209](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-13209) | The vulnerability is known as a [Cross-Site Websocket Hijacking attack](https://www.christian-schneider.net/CrossSiteWebSocketHijacking.html). This attack allows an exploiter to gain access to clusters managed by Rancher with the roles/permissions of a victim. It requires that a victim to be logged into a Rancher server and then access a third-party site hosted by the exploiter. Once that is accomplished, the exploiter is able to execute commands against the Kubernetes API with the permissions and identity of the victim. Reported by Matt Belisle and Alex Stevenson from Workiva. | 15 Jul 2019 | [Rancher v2.2.5](https://github.com/rancher/rancher/releases/tag/v2.2.5), [Rancher v2.1.11](https://github.com/rancher/rancher/releases/tag/v2.1.11) and [Rancher v2.0.16](https://github.com/rancher/rancher/releases/tag/v2.0.16) |
|
||||
| [CVE-2019-12303](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12303) | Project owners can inject extra fluentd logging configurations that makes it possible to read files or execute arbitrary commands inside the fluentd container. Reported by Tyler Welton from Untamed Theory. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
|
||||
| [CVE-2019-12274](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12274) | Nodes using the built-in node drivers using a file path option allows the machine to read arbitrary files including sensitive ones from inside the Rancher server container. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
|
||||
| [CVE-2019-11202](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11202) | The default admin, that is shipped with Rancher, will be re-created upon restart of Rancher despite being explicitly deleted. | 16 Apr 2019 | [Rancher v2.2.2](https://github.com/rancher/rancher/releases/tag/v2.2.2), [Rancher v2.1.9](https://github.com/rancher/rancher/releases/tag/v2.1.9) and [Rancher v2.0.14](https://github.com/rancher/rancher/releases/tag/v2.0.14) |
|
||||
| [CVE-2019-6287](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6287) | Project members continue to get access to namespaces from projects that they were removed from if they were added to more than one project. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) |
|
||||
| [CVE-2018-20321](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20321) | Any project member with access to the `default` namespace can mount the `netes-default` service account in a pod and then use that pod to execute administrative privileged commands against the Kubernetes cluster. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) - Rolling back from these versions or greater have specific [instructions]({{<baseurl>}}/rancher/v2.6/en/installation/install-rancher-on-k8s/rollbacks). |
|
||||
+1
@@ -0,0 +1 @@
|
||||
<!-- PLACEHOLDER -->
|
||||
+1
@@ -0,0 +1 @@
|
||||
<!-- PLACEHOLDER -->
|
||||
Reference in New Issue
Block a user