From 200579854e6a923c5838a088f15611f7dbf52020 Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Thu, 9 Dec 2021 13:22:28 -0800 Subject: [PATCH 1/5] PostgreSQL typo fix Signed-off-by: Derek Nola --- content/k3s/latest/en/installation/datastore/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/k3s/latest/en/installation/datastore/_index.md b/content/k3s/latest/en/installation/datastore/_index.md index 059d73e16fe..b5476b42f35 100644 --- a/content/k3s/latest/en/installation/datastore/_index.md +++ b/content/k3s/latest/en/installation/datastore/_index.md @@ -23,7 +23,7 @@ If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you CLI Flag | Environment Variable | Description ------------|-------------|------------------ - `--datastore-endpoint` | `K3S_DATASTORE_ENDPOINT` | Specify a PostgresSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below. + `--datastore-endpoint` | `K3S_DATASTORE_ENDPOINT` | Specify a PostgreSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below. `--datastore-cafile` | `K3S_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate. | | `--datastore-certfile` | `K3S_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. | | `--datastore-keyfile` | `K3S_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. | @@ -81,7 +81,7 @@ The above assumes a typical three node etcd cluster. The parameter can accept on {{% /tab %}} {{% /tabs %}} -
Based on the above, the following example command could be used to launch a server instance that connects to a PostgresSQL database named k3s: +
Based on the above, the following example command could be used to launch a server instance that connects to a PostgreSQL database named k3s: ``` K3S_DATASTORE_ENDPOINT='postgres://username:password@hostname:5432/k3s' k3s server ``` From de1df25b7e40cfc0b4410e01d479710df1606780 Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Thu, 9 Dec 2021 14:04:05 -0800 Subject: [PATCH 2/5] Initial secrets-encrypt guides Signed-off-by: Derek Nola --- content/k3s/latest/en/advanced/_index.md | 39 ---- .../en/security/secrets_encryption/_index.md | 195 ++++++++++++++++++ 2 files changed, 195 insertions(+), 39 deletions(-) create mode 100644 content/k3s/latest/en/security/secrets_encryption/_index.md diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 494650ebf82..bed44cebd97 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -13,7 +13,6 @@ This section contains advanced information describing the different ways you can - [Using Docker as the container runtime](#using-docker-as-the-container-runtime) - [Using etcdctl](#using-etcdctl) - [Configuring containerd](#configuring-containerd) -- [Secrets Encryption Config (Experimental)](#secrets-encryption-config-experimental) - [Running K3s with Rootless mode (Experimental)](#running-k3s-with-rootless-mode-experimental) - [Node labels and taints](#node-labels-and-taints) - [Starting the server with the installation script](#starting-the-server-with-the-installation-script) @@ -145,44 +144,6 @@ For advanced customization for this file you can create another file called `con The `config.toml.tmpl` will be treated as a Go template file, and the `config.Node` structure is being passed to the template. [This template](https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32) example on how to use the structure to customize the configuration file. -# Secrets Encryption Config (Experimental) -As of v1.17.4+k3s1, K3s added the experimental feature of enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server, this flag will do the following automatically: - -- Generate an AES-CBC key -- Generate an encryption config file with the generated key - -``` -{ - "kind": "EncryptionConfiguration", - "apiVersion": "apiserver.config.k8s.io/v1", - "resources": [ - { - "resources": [ - "secrets" - ], - "providers": [ - { - "aescbc": { - "keys": [ - { - "name": "aescbckey", - "secret": "xxxxxxxxxxxxxxxxxxx" - } - ] - } - }, - { - "identity": {} - } - ] - } - ] -} -``` - -- Pass the config to the KubeAPI as encryption-provider-config - -Once enabled any created secret will be encrypted with this key. Note that if you disable encryption then any encrypted secrets will not be readable until you enable encryption again. # Running K3s with Rootless mode (Experimental) diff --git a/content/k3s/latest/en/security/secrets_encryption/_index.md b/content/k3s/latest/en/security/secrets_encryption/_index.md new file mode 100644 index 00000000000..e6f626c7af4 --- /dev/null +++ b/content/k3s/latest/en/security/secrets_encryption/_index.md @@ -0,0 +1,195 @@ +--- +title: Secrets Encryption +weight: 26 +--- + +# Secrets Encryption Config +_Avaliable as of v1.17.4+k3s1_ + +K3s supports enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server, this flag will do the following automatically: + +- Generate an AES-CBC key +- Generate an encryption config file with the generated key +- Pass the config to the KubeAPI as encryption-provider-config + +Example of the encryption config file: +``` +{ + "kind": "EncryptionConfiguration", + "apiVersion": "apiserver.config.k8s.io/v1", + "resources": [ + { + "resources": [ + "secrets" + ], + "providers": [ + { + "aescbc": { + "keys": [ + { + "name": "aescbckey", + "secret": "xxxxxxxxxxxxxxxxxxx" + } + ] + } + }, + { + "identity": {} + } + ] + } + ] +} +``` + + +## Secrets Encryption Tool +_Avaliable as of v1.21.8+k3s1_ + +K3s contains a utility tool `secrets-encrypt`, which enable automatic control over: + +- Disabling/Enabling secrets encryption +- Adding new encryption keys +- Rotating and deleting encryption keys +- Reencrypting secrets + +>**Warning** Failure to follow proper procedure for rotating encryption keys can leave your cluster permanently corrupted. Proceed with caution. + +### Single-Server Encryption Key Rotation +To rotate secrets encryption keys on a single-node cluster: + +- Start the K3s server with the flag `--secrets-encryption` + +>**Note** Starting K3s without encryption and enabling it at a later time is currently *not* supported. + +1. Prepare + + ``` + k3s secrets-encrypt prepare + ``` + +2. Kill and restart the K3s server with same arguments +3. Rotate + + ``` + k3s secrets-encrypt rotate + ``` + +4. Kill and restart the K3s server with same arguments +5. Reencrypt + + ``` + k3s secrets-encrypt reencrypt + ``` + +### High-Availability Encryption Key Rotation +The steps are the same for both embedded DB and external DB clusters. + +To rotate secrets encryption keys on HA setups: + +>**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. + +- Start up 3 K3s servers, all with the `--secrets-encrytion` flag. For brevity, the servers will be referred to as S1, S2, S3. + +1. Prepare on S1 + + ``` + k3s secrets-encrypt prepare + ``` + +2. Kill and restart S1 with same arguments +3. Once S1 is up, kill and restart the S2 and S3 + +4. Rotate on S1 + + ``` + k3s secrets-encrypt rotate + ``` + +5. Kill and restart S1 with same arguments +6. Once S1 is up, kill and restart the S2 and S3 + +7. Reencrypt on S1 + + ``` + k3s secrets-encrypt reencrypt + ``` + +8. Kill and restart S1 with same arguments +9. Once S1 is up, kill and restart the S2 and S3 + +### Single-Server Secrets Encryption Disable/Enable +After launching a server with `--secrets-encryption` flag, secrets encryption can be disabled. + +To disable secrets encryption on a single-node cluster: + +1. Disable + + ``` + k3s secrets-encrypt disable + ``` + +2. Kill and restart the K3s server with same arguments + +3. Reencrypt with flags + + ``` + k3s secrets-encrypt reencrypt --force --skip + ``` + +To re-enable secrets encryption on a single node cluster: + +1. Enable + + ``` + k3s secrets-encrypt enable + ``` + +2. Kill and restart the K3s server with same arguments + +3. Reencrypt with flags + + ``` + k3s secrets-encrypt reencrypt --force --skip + ``` + +### High-Avaliability Secrets Encryption Disable/Enable +After launching a HA cluster with `--secrets-encryption` flags, secrets encryption can be disabled. +>**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. + +For brevity, the three servers used in this guide will be referred to as S1, S2, S3. + +To disable secrets encryption on a HA cluster: + +1. Disable on S1 + + ``` + k3s secrets-encrypt disable + ``` + +2. Kill and restart S1 with same arguments +3. Once S1 is up, kill and restart the S2 and S3 + + +4. Reencrypt with flags on S1 + + ``` + k3s secrets-encrypt reencrypt --force --skip + ``` + +To re-enable secrets encryption on a HA cluster: + +1. Enable on S1 + + ``` + k3s secrets-encrypt enable + ``` + +2. Kill and restart S1 with same arguments +3. Once S1 is up, kill and restart the S2 and S3 + +4. Reencrypt with flags on S1 + + ``` + k3s secrets-encrypt reencrypt --force --skip + ``` \ No newline at end of file From d6871c51beaf89f7107a66dbe4c844c2e80c723d Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Fri, 10 Dec 2021 15:19:10 -0800 Subject: [PATCH 3/5] Status info Signed-off-by: Derek Nola --- .../en/security/secrets_encryption/_index.md | 44 ++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/content/k3s/latest/en/security/secrets_encryption/_index.md b/content/k3s/latest/en/security/secrets_encryption/_index.md index e6f626c7af4..9259e4931a9 100644 --- a/content/k3s/latest/en/security/secrets_encryption/_index.md +++ b/content/k3s/latest/en/security/secrets_encryption/_index.md @@ -192,4 +192,46 @@ To re-enable secrets encryption on a HA cluster: ``` k3s secrets-encrypt reencrypt --force --skip - ``` \ No newline at end of file + ``` + + +### Secrets Encryption Status +The secrets-encrypt tool includes a `status` command that displays information about the current status of secrets encryption on the node. + +An example of the command on a single-server node: +``` +$ k3s secrets-encrypt status +Encryption Status: Enabled +Current Rotation Stage: start +Server Encryption Hashes: All hashes match + +Active Key Type Name +------ -------- ---- + * AES-CBC aescbckey + +``` + +Another example on HA cluster, after rotating the keys, but before restarting the servers: +``` +$ k3s secrets-encrypt status +Encryption Status: Enabled +Current Rotation Stage: rotate +Server Encryption Hashes: hash does not match between node-1 and node-2 + +Active Key Type Name +------ -------- ---- + * AES-CBC aescbckey-2021-12-10T22:54:38Z + AES-CBC aescbckey + +``` + +Details on each section are as followed: + +- __Encryption Status__: Displayed whether secrets encryption is disabled or enabled on the node +- __Current Rotation Stage__: Indicates the current rotation stage on the node. + Stages are: `start`, `prepare`, `rotate`, `reencrypt_request`, `reencrypt_active`, `reencrypt_finished` +- __Server Encryption Hashes__: Useful for HA clusters, this indicates whether all server are on the same stage with their local files. This can be used to identify whether a restart of servers is required before proceeding to the next stage. In the HA example above, node-1 and node-2 different hashes, indicating that they currently do not have the same encryption configuration. Restarting the servers will sync up their configuration. +- __Key Table__: Summarizes information about the secrets encryption keys found on the node. + * __Active__: The "*" indicates which, if any, of the keys are currently used for secrets encryption. An active key is used by kubernetes to encrypt any new secrets. + * __Key Type__: All keys using this tool are `AES-CBC` type. See more info [here.](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#providers) + * __Name__: Name of the encryption key. \ No newline at end of file From 1be00a06284d08e5866ff53ecf9077d3313ebac0 Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Mon, 13 Dec 2021 11:30:37 -0800 Subject: [PATCH 4/5] Review changes Signed-off-by: Derek Nola --- .../en/security/secrets_encryption/_index.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/content/k3s/latest/en/security/secrets_encryption/_index.md b/content/k3s/latest/en/security/secrets_encryption/_index.md index 9259e4931a9..0de4ce9b3d0 100644 --- a/content/k3s/latest/en/security/secrets_encryption/_index.md +++ b/content/k3s/latest/en/security/secrets_encryption/_index.md @@ -4,9 +4,9 @@ weight: 26 --- # Secrets Encryption Config -_Avaliable as of v1.17.4+k3s1_ +_Available as of v1.17.4+k3s1_ -K3s supports enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server, this flag will do the following automatically: +K3s supports enabling secrets encryption at rest by passing the flag `--secrets-encryption` on a server; this flag will do the following automatically: - Generate an AES-CBC key - Generate an encryption config file with the generated key @@ -44,9 +44,9 @@ Example of the encryption config file: ## Secrets Encryption Tool -_Avaliable as of v1.21.8+k3s1_ +_Available as of v1.21.8+k3s1_ -K3s contains a utility tool `secrets-encrypt`, which enable automatic control over: +K3s contains a utility tool `secrets-encrypt`, which enables automatic control over the following: - Disabling/Enabling secrets encryption - Adding new encryption keys @@ -89,7 +89,7 @@ To rotate secrets encryption keys on HA setups: >**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. -- Start up 3 K3s servers, all with the `--secrets-encrytion` flag. For brevity, the servers will be referred to as S1, S2, S3. +- Start up all three K3s servers with the `--secrets-encryption` flag. For brevity, the servers will be referred to as S1, S2, S3. 1. Prepare on S1 @@ -153,7 +153,7 @@ To re-enable secrets encryption on a single node cluster: k3s secrets-encrypt reencrypt --force --skip ``` -### High-Avaliability Secrets Encryption Disable/Enable +### High-Availability Secrets Encryption Disable/Enable After launching a HA cluster with `--secrets-encryption` flags, secrets encryption can be disabled. >**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. @@ -225,13 +225,13 @@ Active Key Type Name ``` -Details on each section are as followed: +Details on each section are as follows: - __Encryption Status__: Displayed whether secrets encryption is disabled or enabled on the node - __Current Rotation Stage__: Indicates the current rotation stage on the node. Stages are: `start`, `prepare`, `rotate`, `reencrypt_request`, `reencrypt_active`, `reencrypt_finished` -- __Server Encryption Hashes__: Useful for HA clusters, this indicates whether all server are on the same stage with their local files. This can be used to identify whether a restart of servers is required before proceeding to the next stage. In the HA example above, node-1 and node-2 different hashes, indicating that they currently do not have the same encryption configuration. Restarting the servers will sync up their configuration. +- __Server Encryption Hashes__: Useful for HA clusters, this indicates whether all servers are on the same stage with their local files. This can be used to identify whether a restart of servers is required before proceeding to the next stage. In the HA example above, node-1 and node-2 have different hashes, indicating that they currently do not have the same encryption configuration. Restarting the servers will sync up their configuration. - __Key Table__: Summarizes information about the secrets encryption keys found on the node. - * __Active__: The "*" indicates which, if any, of the keys are currently used for secrets encryption. An active key is used by kubernetes to encrypt any new secrets. + * __Active__: The "*" indicates which, if any, of the keys are currently used for secrets encryption. An active key is used by Kubernetes to encrypt any new secrets. * __Key Type__: All keys using this tool are `AES-CBC` type. See more info [here.](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#providers) * __Name__: Name of the encryption key. \ No newline at end of file From cfeae950a30f05a301843f5dcd02257b25a0162f Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Wed, 15 Dec 2021 09:00:19 -0800 Subject: [PATCH 5/5] Added missing colons Signed-off-by: Derek Nola --- .../k3s/latest/en/security/secrets_encryption/_index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/k3s/latest/en/security/secrets_encryption/_index.md b/content/k3s/latest/en/security/secrets_encryption/_index.md index 0de4ce9b3d0..b91bc4d081a 100644 --- a/content/k3s/latest/en/security/secrets_encryption/_index.md +++ b/content/k3s/latest/en/security/secrets_encryption/_index.md @@ -53,14 +53,14 @@ K3s contains a utility tool `secrets-encrypt`, which enables automatic control o - Rotating and deleting encryption keys - Reencrypting secrets ->**Warning** Failure to follow proper procedure for rotating encryption keys can leave your cluster permanently corrupted. Proceed with caution. +>**Warning:** Failure to follow proper procedure for rotating encryption keys can leave your cluster permanently corrupted. Proceed with caution. ### Single-Server Encryption Key Rotation To rotate secrets encryption keys on a single-node cluster: - Start the K3s server with the flag `--secrets-encryption` ->**Note** Starting K3s without encryption and enabling it at a later time is currently *not* supported. +>**Note:** Starting K3s without encryption and enabling it at a later time is currently *not* supported. 1. Prepare @@ -87,7 +87,7 @@ The steps are the same for both embedded DB and external DB clusters. To rotate secrets encryption keys on HA setups: ->**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. +>**Note:** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. - Start up all three K3s servers with the `--secrets-encryption` flag. For brevity, the servers will be referred to as S1, S2, S3. @@ -155,7 +155,7 @@ To re-enable secrets encryption on a single node cluster: ### High-Availability Secrets Encryption Disable/Enable After launching a HA cluster with `--secrets-encryption` flags, secrets encryption can be disabled. ->**Note** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. +>**Note:** While not required, it is recommended that you pick one server node from which to run the `secrets-encrypt` commands. For brevity, the three servers used in this guide will be referred to as S1, S2, S3.