public/rancher-docs
1
0
Fork 0
mirror of https://github.com/rancher/rancher-docs.git synced 2026-05-14 09:03:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add v2.14 preview docs (#2212)

Browse Source
This commit is contained in:
Lucas Saintarbor
2026-03-05 12:30:57 -08:00
committed by GitHub
parent 4a0d71b3f3
commit 2dcfa6f6b8
874 changed files with 92618 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
versioned_docs/version-2.14/reference-guides/backup-restore-configuration/backup-configuration.md
+208
View File
@@ -0,0 +1,208 @@
---
title: Backup Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/backup-restore-configuration/backup-configuration"/>
</head>
The **Backup: Create** page lets you configure a schedule, enable encryption and specify the storage location for your backups.
You must first [install](../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md) the `rancher-backup` operator. After you do so, you can access the **Backup: Create** page:
## Schedule
Select the first option to perform a one-time backup, or select the second option to schedule recurring backups. Selecting **Recurring Backups** lets you configure following two fields:
- **Schedule**: This field accepts
- Standard [cron expressions](https://en.wikipedia.org/wiki/Cron), such as `"0 * * * *"`
- Descriptors, such as `"@midnight"` or `"@every 1h30m"`
- **Retention Count**: This value specifies how many backup files must be retained. If files exceed the given retentionCount, the oldest files will be deleted. The default value is 10.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `schedule` | Provide the cron string for scheduling recurring backups. |
| `retentionCount` | Provide the number of backup files to be retained. |
## ResourceSet
While you can create your own ResourceSets to back up custom applications, two ResourceSets specifically for backing up Rancher are officially maintained and created by default by the `rancher-backup` operator. We refer to these as `rancher-resource-set-basic` and `rancher-resource-set-full`. The difference between them is whether they include secrets or not in the backups.
`rancher-resource-set-basic` does not include any secrets in the backup files to safeguard confidential information. You are responsible for recording any secrets and redeploying them safely.
`rancher-resource-set-full` includes all essential secrets in the backup files to ensure Rancher continues running smoothly after a restore or migration. To avoid storing sensitive information in plain text, we strongly advise you to enable encryption with a strong key.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `resourceSetName` | Provide the name of the ResourceSet to define which resources will be included in this backup. |
## Encryption
The rancher-backup gathers resources by making calls to the kube-apiserver. Objects returned by apiserver are decrypted, so even if [encryption at rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) is enabled, even the encrypted objects gathered by the backup will be in plaintext.
To avoid storing them in plaintext, you can use the same `EncryptionConfiguration` file that was used for at rest encryption, to encrypt certain resources in your backup.
:::note Important:
When encrypting objects in the backup you must save the `EncryptionConfiguration` file for future use, because it won’t be saved by the rancher-backup operator.
For example, when [migrating Rancher to a new cluster](../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md) the file is used to re-create the secret in the new cluster.
:::
The operator consumes the `EncryptionConfiguration` as a Kubernetes Secret in the `cattle-resources-system` namespace under the key named `encryption-provider-config.yaml` in the secret data.
For the `EncryptionConfiguration`, you can use the [sample file provided in the Kubernetes documentation.](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#understanding-the-encryption-at-rest-configuration)
To ensure the correct key is used in the secret, the encryption configuration file must be named `encryption-provider-config.yaml`. The below command uses the `--from-file` flag to create the secret with the correct key name.
Save the `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
```bash
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
This will ensure that the secret contains a key named `encryption-provider-config.yaml`, and the operator will use this key to get the encryption configuration.
The `Encryption Config Secret` dropdown will filter out and list only those Secrets that have this exact key.
![](/img/backup_restore/backup/encryption.png)
In the example command above, the name `encryptionconfig` can be changed to anything.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `encryptionConfigSecretName` | Provide the name of the Secret from `cattle-resources-system` namespace, that contains the encryption config file. |
## Storage Location
If the StorageLocation is specified in the Backup, the operator will retrieve the backup location from that particular S3 bucket. If not specified, the operator will try to find this file in the default operator-level S3 store, and in the operator-level PVC store. The default storage location is configured during the deployment of the `rancher-backup` operator.
Selecting the first option stores this backup in the storage location configured while installing the rancher-backup chart. The second option lets you configure a different S3 compatible storage provider for storing the backup.
### S3
:::caution
If you use an S3 backup target, make sure that every cluster has its own bucket or folder. Rancher populates snapshot information from any available snapshot listed in the S3 bucket or folder configured for that cluster.
:::
The S3 storage location contains the following configuration fields:
1. **Credential Secret** (optional): If you need an AWS access key or secret key to access an S3 bucket, [create a secret](../../how-to-guides/new-user-guides/kubernetes-resources-setup/secrets.md) using your credentials, with keys and directives named `accessKey` and `secretKey`. The secret can be in any namespace. An example secret is [here](#example-credentialsecret). This directive is unnecessary if the nodes running your operator are in EC2 and assigned [IAM permissions to access S3](#iam-permissions-for-ec2-nodes-to-access-s3). The **Credential Secret** dropdown lists secrets in all namespaces.
1. **Bucket Name**: The name of the S3 bucket where backup files will be stored.
1. **Region** (optional): The AWS [region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. This field isn't needed for configuring MinIO.
1. **Folder** (optional): The name of the folder in the S3 bucket where backup files will be stored. Nested folders (e.g., `rancher/cluster1`) are not supported. If this field is left empty, the default behavior is to store the backup files in the root folder of the S3 bucket.
1. **Endpoint**: The [endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) that is used to access S3 in the region of your bucket.
1. **Endpoint CA** (optional): This should be the Base64 encoded CA cert. For an example, refer to the [example S3 compatible configuration.](#example-s3-storage-configuration)
1. **Skip TLS Verifications** (optional): Set to true if you are not using TLS.
#### YAML Directive Fields
| YAML Directive Name | Description | Required |
| ---------------- | ---------------- | ------------ |
| `credentialSecretName` | If you need an AWS access key or secret key to access an S3 bucket, [create a secret](../../how-to-guides/new-user-guides/kubernetes-resources-setup/secrets.md) using your credentials, with keys and directives named `accessKey` and `secretKey`. The secret can be in any namespace. An example secret is [here](#example-credentialsecret). This directive is unnecessary if the nodes running your operator are in EC2 and assigned [IAM permissions to access S3](#iam-permissions-for-ec2-nodes-to-access-s3). | |
| `credentialSecretNamespace` | The namespace of the secret containing the credentials to access S3. This directive is unnecessary if the nodes running your operator are in EC2 and set up with IAM permissions that allow them to access S3, as described in [this section.](#iam-permissions-for-ec2-nodes-to-access-s3) | |
| `bucketName` | The name of the S3 bucket where backup files will be stored. | ✓ |
| `folder` | The name of the folder in the S3 bucket where backup files will be stored. Nested folders (e.g., `rancher/cluster1`) are not supported. If this field is left empty, the default behavior is to store the backup files in the root folder of the S3 bucket. | |
| `region` | The AWS [region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. | ✓ |
| `endpoint` | The [endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) that is used to access S3 in the region of your bucket. | ✓ |
| `endpointCA` | This should be the Base64 encoded CA cert. For an example, refer to the [example S3 compatible configuration.](#example-s3-storage-configuration) | |
| `insecureTLSSkipVerify` | Set to true if you are not using TLS. | |
### Example S3 Storage Configuration
```yaml
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: rancher
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
```
### Example MinIO Configuration
```yaml
s3:
credentialSecretName: minio-creds
bucketName: rancherbackups
endpoint: minio.35.202.130.254.xip.io
endpointCA: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURHakNDQWdLZ0F3SUJBZ0lKQUtpWFZpNEpBb0J5TUEwR0NTcUdTSWIzRFFFQkN3VUFNQkl4RURBT0JnTlYKQkFNTUIzUmxjM1F0WTJFd0hoY05NakF3T0RNd01UZ3lOVFE1V2hjTk1qQXhNREk1TVRneU5UUTVXakFTTVJBdwpEZ1lEVlFRRERBZDBaWE4wTFdOaE1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBCjA4dnV3Q2Y0SEhtR2Q2azVNTmozRW5NOG00T2RpS3czSGszd1NlOUlXQkwyVzY5WDZxenBhN2I2M3U2L05mMnkKSnZWNDVqeXplRFB6bFJycjlpbEpWaVZ1NFNqWlFjdG9jWmFCaVNsL0xDbEFDdkFaUlYvKzN0TFVTZSs1ZDY0QQpWcUhDQlZObU5xM3E3aVY0TE1aSVpRc3N6K0FxaU1Sd0pOMVVKQTZ6V0tUc2Yzc3ByQ0J2dWxJWmZsVXVETVAyCnRCTCt6cXZEc0pDdWlhNEEvU2JNT29tVmM2WnNtTGkwMjdub3dGRld3MnRpSkM5d0xMRE14NnJoVHQ4a3VvVHYKQXJpUjB4WktiRU45L1Uzb011eUVKbHZyck9YS2ZuUDUwbk8ycGNaQnZCb3pUTStYZnRvQ1d5UnhKUmI5cFNTRApKQjlmUEFtLzNZcFpMMGRKY2sxR1h3SURBUUFCbzNNd2NUQWRCZ05WSFE0RUZnUVU5NHU4WXlMdmE2MTJnT1pyCm44QnlFQ2NucVFjd1FnWURWUjBqQkRzd09ZQVU5NHU4WXlMdmE2MTJnT1pybjhCeUVDY25xUWVoRnFRVU1CSXgKRURBT0JnTlZCQU1NQjNSbGMzUXRZMkdDQ1FDb2wxWXVDUUtBY2pBTUJnTlZIUk1FQlRBREFRSC9NQTBHQ1NxRwpTSWIzRFFFQkN3VUFBNElCQVFER1JRZ1RtdzdVNXRQRHA5Q2psOXlLRW9Vd2pYWWM2UlAwdm1GSHpubXJ3dUVLCjFrTkVJNzhBTUw1MEpuS29CY0ljVDNEeGQ3TGdIbTNCRE5mVVh2anArNnZqaXhJYXR2UWhsSFNVaWIyZjJsSTkKVEMxNzVyNCtROFkzelc1RlFXSDdLK08vY3pJTGh5ei93aHRDUlFkQ29lS1dXZkFiby8wd0VSejZzNkhkVFJzNwpHcWlGNWZtWGp6S0lOcTBjMHRyZ0xtalNKd1hwSnU0ZnNGOEcyZUh4b2pOKzdJQ1FuSkg5cGRIRVpUQUtOL2ppCnIvem04RlZtd1kvdTBndEZneWVQY1ZWbXBqRm03Y0ZOSkc4Y2ZYd0QzcEFwVjhVOGNocTZGeFBHTkVvWFZnclMKY1VRMklaU0RJd1FFY3FvSzFKSGdCUWw2RXBaUVpWMW1DRklrdFBwSQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
```
### Example credentialSecret
```yaml
apiVersion: v1
kind: Secret
metadata:
name: creds
type: Opaque
data:
accessKey: <base64-encoded access key>
secretKey: <base64-encoded secret key>
```
:::note
To avoid encoding issues, the `credentialSecret` can be created with the below command, updating the values for `accessKey` and `secretKey`.
```bash
kubectl create secret generic s3-creds \
--from-literal=accessKey=<access key> \
--from-literal=secretKey=<secret key>
```
:::
### IAM Permissions for EC2 Nodes to Access S3
There are two ways to set up the `rancher-backup` operator to use S3 as the backup storage location.
One way is to configure the `credentialSecretName` in the Backup custom resource, which refers to AWS credentials that have access to S3.
If the cluster nodes are in Amazon EC2, the S3 access can also be set up by assigning IAM permissions to the EC2 nodes so that they can access S3.
To allow a node to access S3, follow the instructions in the [AWS documentation](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-access-s3-bucket/) to create an IAM role for EC2. When you add a custom policy to the role, add the following permissions, and replace the `Resource` with your bucket name:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::rancher-backups"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::rancher-backups/*"
]
}
]
}
```
After the role is created, and you have attached the corresponding instance profile to your EC2 instance(s), the `credentialSecretName` directive can be left empty in the Backup custom resource.
## Examples
For example Backup custom resources, refer to [this page.](examples.md#backup)
versioned_docs/version-2.14/reference-guides/backup-restore-configuration/backup-restore-configuration.md
+12
View File
@@ -0,0 +1,12 @@
---
title: Rancher Backup Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/backup-restore-configuration"/>
</head>
- [Backup configuration](backup-configuration.md)
- [Restore configuration](restore-configuration.md)
- [Storage location configuration](storage-configuration.md)
- [Example Backup and Restore Custom Resources](examples.md)
versioned_docs/version-2.14/reference-guides/backup-restore-configuration/examples.md
+297
View File
@@ -0,0 +1,297 @@
---
title: Backup and Restore Examples
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/backup-restore-configuration/examples"/>
</head>
This section contains examples of Backup and Restore custom resources.
The default backup storage location is configured when the `rancher-backup` operator is installed or upgraded.
Encrypted backups can only be restored if the Restore custom resource uses the same encryption configuration secret that was used to create the backup.
## Backup
This section contains example Backup custom resources.
>**Note:** Refer to the [backup config reference page](./backup-configuration.md) for more information on configuring the options below.
### Backup in the Default Location with Encryption
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-location-encrypted-backup
spec:
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
```
### Recurring Backup in the Default Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-location-recurring-backup
spec:
resourceSetName: rancher-resource-set-basic
schedule: "@every 1h"
retentionCount: 10
```
### Encrypted Recurring Backup in the Default Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-enc-recurring-backup
spec:
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
schedule: "@every 1h"
retentionCount: 3
```
### Encrypted Backup in Minio
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: minio-backup
spec:
storageLocation:
s3:
credentialSecretName: minio-creds
credentialSecretNamespace: default
bucketName: rancherbackups
endpoint: minio.xip.io
endpointCA: <base64-encoded-cert>
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
```
### Backup in S3 Using AWS Credential Secret
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-backup
spec:
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
```
### Recurring Backup in S3 Using AWS Credential Secret
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-recurring-backup
spec:
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
schedule: "@every 1h"
retentionCount: 10
```
### Backup from EC2 Nodes with IAM Permission to Access S3
This example shows that the AWS credential secret does not have to be provided to create a backup if the nodes running `rancher-backup` have [these permissions for access to S3.](backup-configuration.md#iam-permissions-for-ec2-nodes-to-access-s3)
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-iam-backup
spec:
storageLocation:
s3:
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set-full
encryptionConfigSecretName: encryptionconfig
```
## Restore
This section contains example Restore custom resources.
>**Note:** Refer to the [restore config reference page](./restore-configuration.md) for more information on configuring the options below.
### Restore Using the Default Backup File Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-default
spec:
backupFilename: default-location-recurring-backup-752ecd87-d958-4d20-8350-072f8d090045-2020-09-26T12-29-54-07-00.tar.gz
# encryptionConfigSecretName: test-encryptionconfig
```
### Restore for Rancher Migration
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-migration
spec:
backupFilename: backup-b0450532-cee1-4aa1-a881-f5f48a007b1c-2020-09-15T07-27-09Z.tar.gz
prune: false
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
```
### Restore from Encrypted Backup
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-encrypted
spec:
backupFilename: default-test-s3-def-backup-c583d8f2-6daf-4648-8ead-ed826c591471-2020-08-24T20-47-05Z.tar.gz
encryptionConfigSecretName: encryptionconfig
```
### Restore an Encrypted Backup from Minio
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-minio
spec:
backupFilename: default-minio-backup-demo-aa5c04b7-4dba-4c48-9ac4-ab7916812eaa-2020-08-30T13-18-17-07-00.tar.gz
storageLocation:
s3:
credentialSecretName: minio-creds
credentialSecretNamespace: default
bucketName: rancherbackups
endpoint: minio.xip.io
endpointCA: <base64-encoded-cert>
encryptionConfigSecretName: test-encryptionconfig
```
### Restore from Backup Using an AWS Credential Secret to Access S3
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-s3-demo
spec:
backupFilename: test-s3-recurring-backup-752ecd87-d958-4d20-8350-072f8d090045-2020-09-26T12-49-34-07-00.tar.gz.enc
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
encryptionConfigSecretName: test-encryptionconfig
```
### Restore from EC2 Nodes with IAM Permissions to Access S3
This example shows that the AWS credential secret does not have to be provided to restore from backup if the nodes running `rancher-backup` have [these permissions for access to S3.](backup-configuration.md#iam-permissions-for-ec2-nodes-to-access-s3)
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-s3-demo
spec:
backupFilename: default-test-s3-recurring-backup-84bf8dd8-0ef3-4240-8ad1-fc7ec308e216-2020-08-24T10#52#44-07#00.tar.gz
storageLocation:
s3:
bucketName: rajashree-backup-test
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
encryptionConfigSecretName: test-encryptionconfig
```
## Example EncryptionConfiguration
The snippets below demonstrate two different types of secrets and their relevance with respect to Backup and Restore of custom resources. Creating the secret can be done with the following command:
```plain
kubectl create secret generic example-encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
The first example is that of a secret used to encrypt the backup files. The backup operator will read the contents of the **encryption-provider-config.yaml** key, which contains the definition of an EncryptionConfiguration resource encoded as Base64.
The second example is that of the Kubernetes EncryptionConfiguration resource itself, being the plain-text form of the Base64-encoded content from the first example. This resource is also used to encrypt secrets when stored in etcd and general cases of Encryption at Rest in Kubernetes. More information on that can be found in the [upstream documentation](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). The keys defined in this resource are essential for accessing the contents of encrypted Backups, particularly during the Restore process. For that reason, these must be kept secure, confidential and easily retrievable.
```yaml
apiVersion: v1
data:
encryption-provider-config.yaml: YXBpVmVyc2lvbjogYXBpc2VydmVyLmNvbmZpZy5rOHMuaW8vdjEKa2luZDogRW5jcnlwdGlvbkNvbmZpZ3VyYXRpb24KcmVzb3VyY2VzOgogIC0gcmVzb3VyY2VzOgogICAgICAtIHNlY3JldHMKICAgIHByb3ZpZGVyczoKICAgICAgLSBhZXNnY206CiAgICAgICAgICBrZXlzOgogICAgICAgICAgICAtIG5hbWU6IGtleTEKICAgICAgICAgICAgICBzZWNyZXQ6IGMyVmpjbVYwSUdseklITmxZM1Z5WlE9PQogICAgICAgICAgICAtIG5hbWU6IGtleTIKICAgICAgICAgICAgICBzZWNyZXQ6IGRHaHBjeUJwY3lCd1lYTnpkMjl5WkE9PQogICAgICAtIGFlc2NiYzoKICAgICAgICAgIGtleXM6CiAgICAgICAgICAgIC0gbmFtZToga2V5MQogICAgICAgICAgICAgIHNlY3JldDogYzJWamNtVjBJR2x6SUhObFkzVnlaUT09CiAgICAgICAgICAgIC0gbmFtZToga2V5MgogICAgICAgICAgICAgIHNlY3JldDogZEdocGN5QnBjeUJ3WVhOemQyOXlaQT09CiAgICAgIC0gc2VjcmV0Ym94OgogICAgICAgICAga2V5czoKICAgICAgICAgICAgLSBuYW1lOiBrZXkxCiAgICAgICAgICAgICAgc2VjcmV0OiBZV0pqWkdWbVoyaHBhbXRzYlc1dmNIRnljM1IxZG5kNGVYb3hNak0wTlRZPQ==
kind: Secret
metadata:
name: example-encryptionconfig
namespace: cattle-resources-system
type: Opaque
```
```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aesgcm:
keys:
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
- name: key2
secret: dGhpcyBpcyBwYXNzd29yZA==
- aescbc:
keys:
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
- name: key2
secret: dGhpcyBpcyBwYXNzd29yZA==
- secretbox:
keys:
- name: key1
secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=
```
versioned_docs/version-2.14/reference-guides/backup-restore-configuration/restore-configuration.md
+85
View File
@@ -0,0 +1,85 @@
---
title: Restore Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/backup-restore-configuration/restore-configuration"/>
</head>
The Restore Create page lets you provide details of the backup to restore from
![](/img/backup_restore/restore/restore.png)
## Backup Source
Provide details of the backup file and its storage location, which the operator will then use to perform the restore. Select from the following options to provide these details
### An existing backup config
Selecting this option will populate the **Target Backup** dropdown with the Backups available in this cluster. Select the Backup from the dropdown, and that will fill out the **Backup Filename** field for you, and will also pass the backup source information from the selected Backup to the operator.
![](/img/backup_restore/restore/existing.png)
If the Backup custom resource does not exist in the cluster, you need to get the exact filename and provide the backup source details with the default storage target or an S3-compatible object store.
### The default storage target
Select this option if you are restoring from a backup file that exists in the default storage location configured at the operator-level. The operator-level configuration is the storage location that was configured when the `rancher-backup` operator was installed or upgraded. Provide the exact filename in the **Backup Filename** field.
![](/img/backup_restore/restore/default.png)
### An S3-compatible object store
Select this option if no default storage location is configured at the operator-level, OR if the backup file exists in a different S3 bucket than the one configured as the default storage location. Provide the exact filename in the **Backup Filename** field. Refer to [this section](#getting-the-backup-filename-from-s3) for exact steps on getting the backup filename from s3. Fill in all the details for the S3 compatible object store. Its fields are exactly same as ones for the `backup.StorageLocation` configuration in the [Backup custom resource.](backup-configuration.md#storage-location)
![](/img/backup_restore/restore/s3store.png)
## Encryption
If the backup was created with encryption enabled, its file will have `.enc` suffix. Choosing such a Backup, or providing a backup filename with `.enc` suffix will display another dropdown named **Encryption Config Secret**.
![](/img/backup_restore/restore/encryption.png)
The Secret selected from this dropdown must have the same contents as the one used for the Backup custom resource while performing the backup. If the encryption configuration doesn't match, the restore will fail
The `Encryption Config Secret` dropdown will filter out and list only those Secrets that have this exact key
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `encryptionConfigSecretName` | Provide the name of the Secret from `cattle-resources-system` namespace, that contains the encryption config file. |
:::note Important:
This field should only be set if the backup was created with encryption enabled. Providing the incorrect encryption config will cause the restore to fail.
:::
## Prune During Restore
* **Prune**: In order to fully restore Rancher from a backup, and to go back to the exact state it was at when the backup was performed, we need to delete any additional resources that were created by Rancher after the backup was taken. The operator does so if the **Prune** flag is enabled. Prune is enabled by default and it is recommended to keep it enabled.
* **Delete Timeout**: This is the amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `prune` | Delete the resources managed by Rancher that are not present in the backup (Recommended). |
| `deleteTimeoutSeconds` | Amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again. |
## Getting the Backup Filename from S3
This is the name of the backup file that the `rancher-backup` operator will use to perform the restore.
To obtain this file name from S3, go to your S3 bucket (and folder if it was specified while performing backup).
Copy the filename and store it in your Restore custom resource. So assuming the name of your backup file is `backupfile`,
- If your bucket name is `s3bucket` and no folder was specified, then the `backupFilename` to use will be `backupfile`.
- If your bucket name is `s3bucket` and the base folder is`s3folder`, the `backupFilename` to use is only `backupfile` .
- If there is a subfolder inside `s3Folder` called `s3sub`, and that has your backup file, then the `backupFilename` to use is `s3sub/backupfile`.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `backupFilename` | This is the name of the backup file that the `rancher-backup` operator will use to perform the restore. |
versioned_docs/version-2.14/reference-guides/backup-restore-configuration/storage-configuration.md
+59
View File
@@ -0,0 +1,59 @@
---
title: Backup Storage Location Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/backup-restore-configuration/storage-configuration"/>
</head>
Configure a storage location where all backups are saved by default. You will have the option to override this with each backup, but will be limited to using an S3-compatible object store.
Only one storage location can be configured at the operator level.
## Storage Location Configuration
### No Default Storage Location
You can choose to not have any operator-level storage location configured. If you select this option, you must configure an S3-compatible object store as the storage location for each individual backup.
### S3-compatible Object Store
| Parameter | Description |
| -------------- | -------------- |
| Credential Secret | Choose the credentials for S3 from your secrets in Rancher. [Example](backup-configuration.md#example-credentialsecret). |
| Bucket Name | Enter the name of the [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html) where the backups will be stored. Default: `rancherbackups`. |
| Region | The [AWS region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. |
| Folder | The [folder in the S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/using-folders.html) where the backups will be stored. If this field is left empty, the default behavior is to store the backup files in the root folder of the S3 bucket. |
| Endpoint | The [S3 endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) For example, `s3.us-west-2.amazonaws.com`. |
| Endpoint CA | The CA cert used to for the S3 endpoint. Default: base64 encoded CA cert |
| insecureTLSSkipVerify | Set to true if you are not using TLS. |
### Existing StorageClass
Installing the `rancher-backup` chart by selecting the StorageClass option will create a Persistent Volume Claim (PVC), and Kubernetes will in turn dynamically provision a Persistent Volume (PV) where all the backups will be saved by default.
For information about creating storage classes refer to [this section.](../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/manage-persistent-storage/dynamically-provision-new-storage.md)
:::note Important:
It is highly recommended to use a StorageClass with a reclaim policy of "Retain". Otherwise if the PVC created by the `rancher-backup` chart gets deleted (either during app upgrade, or accidentally), the PV will get deleted too, which means all backups saved in it will get deleted.
If no such StorageClass is available, after the PV is provisioned, make sure to edit its reclaim policy and set it to "Retain" before storing backups in it.
:::
### Existing Persistent Volume
Select an existing Persistent Volume (PV) that will be used to store your backups. For information about creating PersistentVolumes in Rancher, refer to [this section.](../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/manage-persistent-storage/set-up-existing-storage.md#2-add-a-persistentvolume-that-refers-to-the-persistent-storage)
:::note Important:
It is highly recommended to use a Persistent Volume with a reclaim policy of "Retain". Otherwise if the PVC created by the `rancher-backup` chart gets deleted (either during app upgrade, or accidentally), the PV will get deleted too, which means all backups saved in it will get deleted.
:::
## Example YAML File: Rancher Backup Helm Chart
The documented `values.yaml` file that can be used to configure the `backup-restore-operator` when the Helm CLI is used can be found in the [backup-restore-operator repository.](https://github.com/rancher/backup-restore-operator/blob/master/charts/rancher-backup/values.yaml)
For more information about `values.yaml` files and configuring Helm charts during installation, refer to the [Helm documentation.](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing)
versioned_docs/version-2.14/reference-guides/best-practices/best-practices.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Best Practice Guides
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices"/>
</head>
The purpose of this section is to consolidate best practices for Rancher implementations. This also includes recommendations for related technologies, such as Kubernetes, Docker, containers, and more. The objective is to improve the outcome of a Rancher implementation using the operational experience of Rancher and its customers.
If you have any questions about how these might apply to your use case, please contact your Customer Success Manager or Support.
Use the navigation bar on the left to find the current best practices for managing and deploying the Rancher Server.
For more guidance on best practices, you can consult these resources:
- [Security](../rancher-security/rancher-security.md)
- [Rancher Blog](https://www.suse.com/c/rancherblog/)
- [Rancher Forum](https://forums.rancher.com/)
- [Rancher Users Slack](https://slack.rancher.io/)
- [Rancher Labs YouTube Channel - Online Meetups, Demos, Training, and Webinars](https://www.youtube.com/channel/UCh5Xtp82q8wjijP8npkVTBA/featured)
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/disconnected-clusters.md
+19
View File
@@ -0,0 +1,19 @@
---
title: Best Practices for Disconnected Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/disconnected-clusters"/>
</head>
Rancher supports managing clusters that may not always be online due to network disruptions, control plane availability, or because all cluster nodes are down. At the moment there are no known issues with disconnected clusters in the latest released Rancher version.
While a managed cluster is disconnected from Rancher, management operations will be unavailable, and the Rancher UI will not allow navigation to the cluster. However, once the connection is reestablished, functionality is fully restored.
### Best Practices for Managing Disconnected Clusters
- **Cluster Availability During Rancher Upgrades**: It is recommended to have all, or at least most, managed clusters online during a Rancher upgrade. The reason is that upgrading Rancher automatically upgrades the Rancher agent software running on managed clusters. Keeping the agent and Rancher versions aligned ensures consistent functionality. Any clusters that are disconnected during the upgrade will have their agents updated as soon as they reconnect.
- **Cleaning Up Disconnected Clusters**: Regularly remove clusters that will no longer reconnect to Rancher (e.g., clusters that have been decommissioned or destroyed). Keeping such clusters in the Rancher management system consumes unnecessary resources, which could impact Rancher's performance over time.
- **Certificate Rotation Considerations**: When designing processes that involve regularly shutting down clusters, whether connected to Rancher or not, take into account certificate rotation policies. For example, RKE2/K3s clusters may rotate certificates on startup if they exceeded their lifetime.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/logging-best-practices.md
+90
View File
@@ -0,0 +1,90 @@
---
title: Logging Best Practices
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-managed-clusters/logging-best-practices"/>
</head>
In this guide, we recommend best practices for cluster-level logging and application logging.
- [Cluster-level Logging](#cluster-level-logging)
- [Application Logging](#application-logging)
- [General Best Practices](#general-best-practices)
Before Rancher v2.5, logging in Rancher has historically been a pretty static integration. There were a fixed list of aggregators to choose from (ElasticSearch, Splunk, Kafka, Fluentd and Syslog), and only two configuration points to choose (Cluster-level and Project-level).
Rancher provides a flexible experience for log aggregation. With the logging feature, administrators and users alike can deploy logging that meets fine-grained collection criteria while offering a wider array of destinations and configuration options.
"Under the hood", Rancher logging uses the [Logging operator](https://github.com/kube-logging/logging-operator). We provide manageability of this operator (and its resources), and tie that experience in with managing your Rancher clusters.
## Cluster-level Logging
### Cluster-wide Scraping
For some users, it is desirable to scrape logs from every container running in the cluster. This usually coincides with your security team's request (or requirement) to collect all logs from all points of execution.
In this scenario, it is recommended to create at least two _ClusterOutput_ objects - one for your security team (if you have that requirement), and one for yourselves, the cluster administrators. When creating these objects take care to choose an output endpoint that can handle the significant log traffic coming from the entire cluster. Also make sure to choose an appropriate index to receive all these logs.
Once you have created these _ClusterOutput_ objects, create a _ClusterFlow_ to collect all the logs. Do not define any _Include_ or _Exclude_ rules on this flow. This will ensure that all logs from across the cluster are collected. If you have two _ClusterOutputs_, make sure to send logs to both of them.
### Kubernetes Components
_ClusterFlows_ have the ability to collect logs from all containers on all hosts in the Kubernetes cluster. This works well in cases where those containers are part of a Kubernetes pod.
A future release of Rancher will include the source container name which will enable filtering of these component logs. Once that change is made, you will be able to customize a _ClusterFlow_ to retrieve **only** the Kubernetes component logs, and direct them to an appropriate output.
## Application Logging
Best practice not only in Kubernetes but in all container-based applications is to direct application logs to `stdout`/`stderr`. The container runtime will then trap these logs and do **something** with them - typically writing them to a file. Depending on the container runtime (and its configuration), these logs can end up in any number of locations.
In the case of writing the logs to a file, Kubernetes helps by creating a `/var/log/containers` directory on each host. This directory symlinks the log files to their actual destination (which can differ based on configuration or container runtime).
Rancher logging will read all log entries in `/var/log/containers`, ensuring that all log entries from all containers (assuming a default configuration) will have the opportunity to be collected and processed.
### Specific Log Files
Log collection only retrieves `stdout`/`stderr` logs from pods in Kubernetes. But what if we want to collect logs from other files that are generated by applications? Here, a log streaming sidecar (or two) may come in handy.
The goal of setting up a streaming sidecar is to take log files that are written to disk, and have their contents streamed to `stdout`. This way, the Logging Operator can pick up those logs and send them to your desired output.
To set this up, edit your workload resource (e.g. Deployment) and add the following sidecar definition:
```yaml
...
containers:
- args:
- -F
- /path/to/your/log/file.log
command:
- tail
image: busybox
name: stream-log-file-[name]
volumeMounts:
- mountPath: /path/to/your/log
name: mounted-log
...
```
This will add a container to your workload definition that will now stream the contents of (in this example) `/path/to/your/log/file.log` to `stdout`.
This log stream is then automatically collected according to any _Flows_ or _ClusterFlows_ you have setup. You may also wish to consider creating a _Flow_ specifically for this log file by targeting the name of the container. See example:
```yaml
...
spec:
match:
- select:
container_names:
- stream-log-file-name
...
```
## General Best Practices
- Where possible, output structured log entries (e.g. `syslog`, JSON). This makes handling of the log entry easier as there are already parsers written for these formats.
- Try to provide the name of the application that is creating the log entry, in the entry itself. This can make troubleshooting easier as Kubernetes objects do not always carry the name of the application as the object name. For instance, a pod ID may be something like `myapp-098kjhsdf098sdf98` which does not provide much information about the application running inside the container.
- Except in the case of collecting all logs cluster-wide, try to scope your _Flow_ and _ClusterFlow_ objects tightly. This makes it easier to troubleshoot when problems arise, and also helps ensure unrelated log entries do not show up in your aggregator. An example of tight scoping would be to constrain a _Flow_ to a single _Deployment_ in a namespace, or perhaps even a single container within a _Pod_.
- Keep the log verbosity down except when troubleshooting. High log verbosity poses a number of issues, chief among them being **noise**: significant events can be drowned out in a sea of `DEBUG` messages. This is somewhat mitigated with automated alerting and scripting, but highly verbose logging still places an inordinate amount of stress on the logging infrastructure.
- Where possible, try to provide a transaction or request ID with the log entry. This can make tracing application activity across multiple log sources easier, especially when dealing with distributed applications.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/monitoring-best-practices.md
+115
View File
@@ -0,0 +1,115 @@
---
title: Monitoring Best Practices
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-managed-clusters/monitoring-best-practices"/>
</head>
Configuring sensible monitoring and alerting rules is vital for running any production workloads securely and reliably. This is not different when using Kubernetes and Rancher. Fortunately the integrated monitoring and alerting functionality makes this whole process a lot easier.
The [Rancher monitoring documentation](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) describes how you can set up a complete Prometheus and Grafana stack. Out of the box this will scrape monitoring data from all system and Kubernetes components in your cluster and provide sensible dashboards and alerts for them to get started. But for a reliable setup, you also need to monitor your own workloads and adapt Prometheus and Grafana to your own specific use cases and cluster sizes. This document aims to give you best practices for this.
## What to Monitor
Kubernetes itself, as well as applications running inside of it, form a distributed system where different components interact with each other. For the whole system and each individual component, you have to ensure performance, availability, reliability and scalability. A good resource with more details and information is Google's free [Site Reliability Engineering Book](https://sre.google/sre-book/table-of-contents/), especially the chapter about [Monitoring distributed systems](https://sre.google/sre-book/monitoring-distributed-systems/).
## Configuring Prometheus Resource Usage
When installing the integrated monitoring stack, Rancher allows to configure several settings that are dependent on the size of your cluster and the workloads running in it. This chapter covers these in more detail.
### Storage and Data Retention
The amount of storage needed for Prometheus directly correlates to the amount of time series and labels that you store and the data retention you have configured. It is important to note that Prometheus is not meant to be used as a long-term metrics storage. Data retention time is usually only a couple of days and not weeks or months. The reason for this is that Prometheus does not perform any aggregation on its stored metrics. This is great because aggregation can dilute data, but it also means that the needed storage grows linearly over time without retention.
One way to calculate the necessary storage is to look at the average size of a storage chunk in Prometheus with this query
```
rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[1h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[1h])
```
Next, find out your data ingestion rate per second:
```
rate(prometheus_tsdb_head_samples_appended_total[1h])
```
and then multiply this with the retention time, adding a few percentage points as buffer:
```
average chunk size in bytes * ingestion rate per second * retention time in seconds * 1.1 = necessary storage in bytes
```
You can find more information about how to calculate the necessary storage in this [blog post](https://www.robustperception.io/how-much-disk-space-do-prometheus-blocks-use).
You can read more about the Prometheus storage concept in the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/storage).
### CPU and Memory Requests and Limits
In larger Kubernetes clusters Prometheus can consume quite a bit of memory. The amount of memory Prometheus needs directly correlates to the amount of time series and amount of labels it stores and the scrape interval in which these are filled.
You can find more information about how to calculate the necessary memory in this [blog post](https://www.robustperception.io/how-much-ram-does-prometheus-2-x-need-for-cardinality-and-ingestion).
The amount of necessary CPUs correlate with the amount of queries you are performing.
### Federation and Long-term Storage
Prometheus is not meant to store metrics for a long amount of time, but should only be used for short term storage.
In order to store some, or all metrics for a long time, you can leverage Prometheus' [remote read/write](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations) capabilities to connect it to storage systems like [Thanos](https://thanos.io/), [InfluxDB](https://www.influxdata.com/), [M3DB](https://www.m3db.io/), or others. You can find an example setup in this [blog post](https://rancher.com/blog/2020/prometheus-metric-federation).
## Scraping Custom Workloads
While the integrated Rancher Monitoring already scrapes system metrics from a cluster's nodes and system components, the custom workloads that you deploy on Kubernetes should also be scraped for data. For that you can configure Prometheus to do an HTTP request to an endpoint of your applications in a certain interval. These endpoints should then return their metrics in a Prometheus format.
In general, you want to scrape data from all the workloads running in your cluster so that you can use them for alerts or debugging issues. Often, you recognize that you need some data only when you actually need the metrics during an incident. It is good, if it is already scraped and stored. Since Prometheus is only meant to be a short-term metrics storage, scraping and keeping lots of data is usually not that expensive. If you are using a long-term storage solution with Prometheus, you can then still decide which data you are actually persisting and keeping there.
### About Prometheus Exporters
Many 3rd party workloads, such as databases, queues, and web-servers, already support exposing metrics in a Prometheus format, or offer exporters that translate between the tool's metrics and a format that Prometheus understands. You can usually add these exporters as additional sidecar containers to the workload's Pods. Many Helm charts already include options to deploy the correct exporter. You can find a curated list of exports on [ExporterHub](https://exporterhub.io/).
### Prometheus support in Programming Languages and Frameworks
To get your own custom application metrics into Prometheus, you have to collect and expose these metrics directly from your application's code. Fortunately, there are already libraries and integrations available to help with this for most popular programming languages and frameworks. One example for this is the Prometheus support in the [Spring Framework](https://docs.spring.io/spring-metrics/docs/current/public/prometheus).
### ServiceMonitors and PodMonitors
Once all of your workloads expose metrics in a Prometheus format, you must configure Prometheus to scrape them. Under the hood, Rancher uses the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). This makes it easy to add scraping targets with ServiceMonitors and PodMonitors. Many Helm charts already include an option to create these monitors directly. You can also find more information in the Rancher documentation.
### Prometheus Push Gateway
There are some workloads that are traditionally hard to scrape by Prometheus. Examples for these are short lived workloads like Jobs and CronJobs, or applications that do not allow sharing data between individual handled incoming requests, like PHP applications.
To still get metrics for these use cases, you can set up [prometheus-pushgateways](https://github.com/prometheus/pushgateway). The CronJob or PHP application would push metric updates to the pushgateway. The pushgateway aggregates and exposes them through an HTTP endpoint, which then can be scraped by Prometheus.
### Prometheus Blackbox Monitor
Sometimes it is useful to monitor workloads from the outside. For this, you can use the [Prometheus blackbox-exporter](https://github.com/prometheus/blackbox_exporter) which allows probing any kind of endpoint over HTTP, HTTPS, DNS, TCP and ICMP.
## Monitoring in a (Micro)Service Architecture
If you have a (micro)service architecture where multiple individual workloads within your cluster are communicating with each other, it is really important to have detailed metrics and traces about this traffic to understand how all these workloads are communicating with each other and where a problem or bottleneck may be.
Of course you can monitor all this internal traffic in all your workloads and expose these metrics to Prometheus. But this can quickly become quite work intensive. Service Meshes like Istio, which can be installed with [a click](../../../integrations-in-rancher/istio/istio.md) in Rancher, can do this automatically and provide rich telemetry about the traffic between all services.
## Real User Monitoring
Monitoring the availability and performance of all your internal workloads is vitally important to run stable, reliable and fast applications. But these metrics only show you parts of the picture. To get a complete view it is also necessary to know how your end users are actually perceiving it. For this you can look into various [Real user monitoring solutions](https://en.wikipedia.org/wiki/Real_user_monitoring).
## Security Monitoring
In addition to monitoring workloads to detect performance, availability or scalability problems, the cluster and the workloads running into it should also be monitored for potential security problems. A good starting point is to frequently run and alert on [Compliance Scans](../../../how-to-guides/advanced-user-guides/compliance-scan-guides/compliance-scan-guides.md) which check if the cluster is configured according to security best practices.
For the workloads, you can have a look at Kubernetes and Container security solutions like [NeuVector](https://www.suse.com/products/neuvector/), [Falco](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/products/kubernetes-security/), [SysDig](https://sysdig.com/).
## Setting up Alerts
Getting all the metrics into a monitoring systems and visualizing them in dashboards is great, but you also want to be pro-actively alerted if something goes wrong.
The integrated Rancher monitoring already configures a sensible set of alerts that make sense in any Kubernetes cluster. You should extend these to cover your specific workloads and use cases.
When setting up alerts, configure them for all the workloads that are critical to the availability of your applications. But also make sure that they are not too noisy. Ideally every alert you are receiving should be because of a problem that needs your attention and needs to be fixed. If you have alerts that are firing all the time but are not that critical, there is a danger that you start ignoring your alerts all together and then miss the real important ones. Less may be more here. Start to focus on the real important metrics first, for example alert if your application is offline. Fix all the problems that start to pop up and then start to create more detailed alerts.
If an alert starts firing, but there is nothing you can do about it at the moment, it's also fine to silence the alert for a certain amount of time, so that you can look at it later.
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md).
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/rancher-managed-clusters-in-vsphere.md
+62
View File
@@ -0,0 +1,62 @@
---
title: Best Practices for Rancher Managed VMware vSphere Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-managed-clusters/rancher-managed-clusters-in-vsphere"/>
</head>
This guide outlines a reference architecture for provisioning downstream Rancher clusters in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.
- [1. VM Considerations](#1-vm-considerations)
- [2. Network Considerations](#2-network-considerations)
- [3. Storage Considerations](#3-storage-considerations)
- [4. Backups and Disaster Recovery](#4-backups-and-disaster-recovery)
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/solution_overview.drawio.svg)
## 1. VM Considerations
### Leverage VM Templates to Construct the Environment
To facilitate consistency across the deployed Virtual Machines across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customisation options.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Downstream Cluster Nodes Across ESXi Hosts
Doing so will ensure node VM's are spread across multiple ESXi hosts - preventing a single point of failure at the host level.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Downstream Cluster Nodes Across Datastores
Doing so will ensure node VM's are spread across multiple datastores - preventing a single point of failure at the datastore level.
### Configure VM's as Appropriate for Kubernetes
It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.
## 2. Network Considerations
### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes
Deploy etcd members within a single data center where possible to avoid latency overheads and reduce the likelihood of network partitioning. For most setups, 1Gb connections will suffice. For large clusters, 10Gb connections can reduce the time taken to restore from backup.
### Consistent IP Addressing for VM's
Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
## 3. Storage Considerations
### Leverage SSD Drives for ETCD Nodes
ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible.
## 4. Backups and Disaster Recovery
### Perform Regular Downstream Cluster Backups
Kubernetes uses etcd to store all its data - from configuration, state and metadata. Backing this up is crucial in the event of disaster recovery.
### Back up Downstream Node VMs
Incorporate the Rancher downstream node VM's within a standard VM backup policy.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/rancher-managed-clusters.md
+27
View File
@@ -0,0 +1,27 @@
---
title: Best Practices for Rancher Managed Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-managed-clusters"/>
</head>
### Logging
Refer to [this guide](logging-best-practices.md) for our recommendations for cluster-level logging and application logging.
### Monitoring
Configuring sensible monitoring and alerting rules is vital for running any production workloads securely and reliably. Refer to this [guide](monitoring-best-practices.md) for our recommendations.
### Disconnected clusters
Rancher supports managing clusters that may not always be online due to network disruptions, control plane availability, or because all cluster nodes are down. Refer to this [guide](disconnected-clusters.md) for our recommendations.
### Tips for Setting Up Containers
Running well-built containers can greatly impact the overall performance and security of your environment. Refer to this [guide](tips-to-set-up-containers.md) for tips.
### Best Practices for Rancher Managed VMware vSphere Clusters
This [guide](rancher-managed-clusters-in-vsphere.md) outlines a reference architecture for provisioning downstream Rancher clusters in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-managed-clusters/tips-to-set-up-containers.md
+56
View File
@@ -0,0 +1,56 @@
---
title: Tips for Setting Up Containers
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-managed-clusters/tips-to-set-up-containers"/>
</head>
Running well-built containers can greatly impact the overall performance and security of your environment.
Below are a few tips for setting up your containers.
For a more detailed discussion of security for containers, you can also refer to Rancher's [Guide to Container Security.](https://rancher.com/complete-guide-container-security)
### Use a Common Container OS
When possible, you should try to standardize on a common container base OS.
Smaller distributions such as Alpine and BusyBox reduce container image size and generally have a smaller attack/vulnerability surface.
Popular distributions such as Ubuntu, Fedora, and CentOS are more field-tested and offer more functionality.
### Start with a FROM scratch container
If your microservice is a standalone static binary, you should use a FROM scratch container.
The FROM scratch container is an [official Docker image](https://hub.docker.com/_/scratch) that is empty so that you can use it to design minimal images.
This will have the smallest attack surface and smallest image size.
### Run Container Processes as Unprivileged
When possible, use a non-privileged user when running processes within your container. While container runtimes provide isolation, vulnerabilities and attacks are still possible. Inadvertent or accidental host mounts can also be impacted if the container is running as root. For details on configuring a security context for a pod or container, refer to the [Kubernetes docs](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
### Define Resource Limits
Apply CPU and memory limits to your pods. This can help manage the resources on your worker nodes and avoid a malfunctioning microservice from impacting other microservices.
In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the Rancher docs.
When setting resource quotas, if you set anything related to CPU or Memory (i.e. limits or reservations) on a project or namespace, all containers will require a respective CPU or Memory field set during creation. To avoid setting these limits on each and every container during workload creation, a default container resource limit can be specified on the namespace.
The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the namespace level.
### Define Resource Requirements
You should apply CPU and memory requirements to your pods. This is crucial for informing the scheduler which type of compute node your pod needs to be placed on, and ensuring it does not over-provision that node. In Kubernetes, you can set a resource requirement by defining `resources.requests` in the resource requests field in a pod's container spec. For details, refer to the [Kubernetes docs](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container).
:::note
If you set a resource limit for the namespace that the pod is deployed in, and the container doesn't have a specific resource request, the pod will not be allowed to start. To avoid setting these fields on each and every container during workload creation, a default container resource limit can be specified on the namespace.
:::
It is recommended to define resource requirements on the container level because otherwise, the scheduler makes assumptions that will likely not be helpful to your application when the cluster experiences load.
### Liveness and Readiness Probes
Set up liveness and readiness probes for your container. Unless your container completely crashes, Kubernetes will not know it's unhealthy unless you create an endpoint or mechanism that can report container status. Alternatively, make sure your container halts and crashes if unhealthy.
The Kubernetes docs show how to [configure liveness and readiness probes for containers.](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
versioned_docs/version-2.14/reference-guides/best-practices/rancher-server/on-premises-rancher-in-vsphere.md
+88
View File
@@ -0,0 +1,88 @@
---
title: Installing Rancher in a VMware vSphere Environment
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-server/on-premises-rancher-in-vsphere"/>
</head>
This guide outlines a reference architecture for installing Rancher on an RKE Kubernetes cluster in a VMware vSphere environment. It also desctibes standard vSphere best practices as documented by VMware.
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/rancher-on-prem-vsphere.svg)
## 1. Load Balancer Considerations
A load balancer is required to direct traffic to the Rancher workloads residing on the RKE nodes.
### Leverage Fault Tolerance and High Availability
Leverage the use of an external (hardware or software) load balancer that has inherit high-availability functionality (F5, NSX-T, Keepalived, etc).
### Back Up Load Balancer Configuration
In the event of a Disaster Recovery activity, availability of the Load balancer configuration will expedite the recovery process.
### Configure Health Checks
Configure the Load balancer to automatically mark nodes as unavailable if a health check is failed. For example, NGINX can facilitate this with:
`max_fails=3 fail_timeout=5s`
### Leverage an External Load Balancer
Avoid implementing a software load balancer within the management cluster.
### Secure Access to Rancher
Configure appropriate Firewall / ACL rules to only expose access to Rancher
## 2. VM Considerations
### Size the VM's According to Rancher Documentation
See [Installation Requirements](../../../getting-started/installation-and-upgrade/installation-requirements/installation-requirements.md).
### Leverage VM Templates to Construct the Environment
To facilitate the consistency of Virtual Machines deployed across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customization options.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Rancher Cluster Nodes Across ESXi Hosts
Doing so will ensure node VM's are spread across multiple ESXi hosts - preventing a single point of failure at the host level.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Rancher Cluster Nodes Across Datastores
Doing so will ensure node VM's are spread across multiple datastores - preventing a single point of failure at the datastore level.
### Configure VM's as Appropriate for Kubernetes
It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.
## 3. Network Considerations
### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes
Deploy etcd members within a single data center where possible to avoid latency overheads and reduce the likelihood of network partitioning. For most setups, 1Gb connections will suffice. For large clusters, 10Gb connections can reduce the time taken to restore from backup.
### Consistent IP Addressing for VM's
Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
## 4. Storage Considerations
### Leverage SSD Drives for ETCD Nodes
ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible.
## 5. Backups and Disaster Recovery
### Perform Regular Management Cluster Backups
Rancher stores its data in the ETCD datastore of the Kubernetes cluster it resides on. Like with any Kubernetes cluster, perform frequent, tested backups of this cluster.
### Back up Rancher Cluster Node VMs
Incorporate the Rancher management node VM's within a standard VM backup policy.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-server/rancher-deployment-strategy.md
+40
View File
@@ -0,0 +1,40 @@
---
title: Rancher Deployment Strategy
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-server/rancher-deployment-strategy"/>
</head>
There are two recommended deployment strategies for a Rancher instance that manages downstream Kubernetes clusters. Each one has its own pros and cons. Read more about which one would fit best for your use case.
## Hub & Spoke Strategy
---
In this deployment scenario, there is a single Rancher instance managing Kubernetes clusters across the globe. The Rancher instance would be run on a high-availability Kubernetes cluster, and there would be impact due to latencies.
### Pros
* Single control plane interface to view/see all regions and environments.
* Kubernetes does not require Rancher to operate and can tolerate losing connectivity to the Rancher instance.
### Cons
* Subject to network latencies.
* If Rancher goes down, global provisioning of new services is unavailable until it is restored. However, each Kubernetes cluster can continue to be managed individually.
## Regional Strategy
---
In the regional deployment model a Rancher instance is deployed in close proximity to the downstream Kubernetes clusters.
### Pros
* Rancher functionality in regions stay operational if a Rancher instance in another region goes down.
* Network latency between Rancher and downstream clusters is greatly reduced, improving the performance of functionality in Rancher.
* Upgrades of Rancher can be done independently per region.
### Cons
* Overhead of managing multiple Rancher installations.
* Visibility into Kubernetes clusters in different regions requires multiple interfaces/panes of glass.
* Deploying multi-cluster apps in Rancher requires repeating the process for each Rancher server.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-server/rancher-server.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Best Practices for the Rancher Server
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-server"/>
</head>
This guide contains our recommendations for running the Rancher server, and is intended to be used in situations in which Rancher manages downstream Kubernetes clusters.
### Recommended Architecture and Infrastructure
Refer to this [guide](tips-for-running-rancher.md) for our general advice for setting up the Rancher server for a production installation.
### Deployment Strategies
This [guide](rancher-deployment-strategy.md) is designed to help you choose whether a regional deployment strategy or a hub-and-spoke deployment strategy is better for a Rancher server that manages downstream Kubernetes clusters.
### Installing Rancher in a VMware vSphere Environment
This [guide](on-premises-rancher-in-vsphere.md) outlines a reference architecture for installing Rancher in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md
+70
View File
@@ -0,0 +1,70 @@
---
title: Tips for Running Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-server/tips-for-running-rancher"/>
</head>
This guide is geared toward use cases where Rancher is used to manage downstream Kubernetes clusters. The high-availability setup is intended to prevent losing access to downstream clusters if the Rancher server is not available.
A high-availability Kubernetes installation, defined as an installation of Rancher on a Kubernetes cluster with at least three nodes, should be used in any production installation of Rancher, as well as any installation deemed "important." Multiple Rancher instances running on multiple nodes ensure high availability that cannot be accomplished with a single node environment.
If you are installing Rancher in a vSphere environment, refer to the best practices documented [here.](on-premises-rancher-in-vsphere.md)
When you set up your high-availability Rancher installation, consider the following:
### Minimize Third-Party Software on the Upstream Cluster
We generally recommend running Rancher on a dedicated cluster, free of other workloads, to avoid potential performance and compatibility issues.
Rancher, especially when managing a growing number of clusters, nodes, and workloads, places a significant load on core Kubernetes components like `etcd` and `kube-apiserver` on the upstream cluster. Third-party software can interfere with the performance of these components and Rancher, potentially leading to instability.
Furthermore, third-party software can functionally interfere with Rancher. To minimize compatibility risks, deploy only essential Kubernetes system components and Rancher on the upstream cluster.
The following applications and components generally do not interfere with Rancher or the Kubernetes system:
* Rancher internal components, such as Fleet
* Rancher extensions
* Cluster API components
* CNIs, CPIs, CSIs
* Cloud controller managers
* Observability and monitoring tools (except prometheus-rancher-exporter)
Note that each of these components has its own minimum resource requirements, which must be met in addition to Rancher's. For high-scale deployments, also consider dedicating separate nodes to non-Rancher software using [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to minimize interference.
The following software can interfere with Rancher performance and is therefore discouraged on the upstream cluster:
* [CrossPlane](https://www.crossplane.io/)
* [Argo CD](https://argoproj.github.io/cd/)
* [Flux](https://fluxcd.io/)
* [prometheus-rancher-exporter](https://github.com/David-VTUK/prometheus-rancher-exporter) (see [issue 33](https://github.com/David-VTUK/prometheus-rancher-exporter/issues/33))
* Container registries such as [Harbor](https://goharbor.io/), which can require significant bandwidth for serving images
### Guidance for Container Registries
Container registries, such as [Harbor](https://goharbor.io/), can consume significant network bandwidth when serving images. This demand increases with the number of images, the frequency of image pulls, and the quantity of clusters and container runtimes they serve. Due to this potential for interference with Rancher UI and API traffic, we recommend against running container registries on the same cluster as the Rancher management server.
Regardless of your deployment strategy for a container registry, ensure sufficient bandwidth is available, ideally reserved using Quality of Service (QoS) mechanisms.
Consider the following recommendations based on your needs:
* **Simple Setups (HA Not a Primary Concern):** A container registry deployed as a single Virtual Machine (VM) can be a viable solution.
* **High Availability (HA) Requirements:** We recommend running the registry in a dedicated Kubernetes cluster. All other clusters should then be configured to pull images from this centralized, HA registry.
* **Very Large-Scale or Complex Network Topologies:** Multiple registry clusters might be necessary. These can be deployed in a hierarchical or federated model to efficiently distribute images and manage traffic.
### Make sure nodes are configured correctly for Kubernetes
It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.5/op-guide/performance/).
### Run All Nodes in the Cluster in the Same Datacenter
For best performance, run all three of your nodes in the same geographic datacenter. If you are running nodes in the cloud, such as AWS, run each node in a separate Availability Zone. For example, launch node 1 in us-west-2a, node 2 in us-west-2b, and node 3 in us-west-2c.
### Development and Production Environments Should be Similar
It's strongly recommended to have a "staging" or "pre-production" environment of the Kubernetes cluster that Rancher runs on. This environment should mirror your production environment as closely as possible in terms of software and hardware configuration.
### Monitor Your Clusters to Plan Capacity
The Rancher server's Kubernetes cluster should run within the [system and hardware requirements](../../../getting-started/installation-and-upgrade/installation-requirements/installation-requirements.md) as closely as possible. The more you deviate from the system and hardware requirements, the more risk you take.
However, metrics-driven capacity planning analysis should be the ultimate guidance for scaling Rancher, because the published requirements take into account a variety of workload types.
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus.
After you [enable monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md) in the cluster, you can set up alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale.
versioned_docs/version-2.14/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale.md
+125
View File
@@ -0,0 +1,125 @@
---
title: Tuning and Best Practices for Rancher at Scale
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/best-practices/rancher-server/tuning-and-best-practices-for-rancher-at-scale"/>
</head>
This guide describes the best practices and tuning approaches to scale Rancher setups and the associated challenges with doing so. As systems grow, performance will naturally reduce, but there are steps that can minimize the load put on Rancher and optimize Rancher's ability to manage larger infrastructures.
## Optimizing Rancher Performance
* Keep Rancher up to date with patch releases. We are continuously improving Rancher with performance enhancements and bug fixes. The latest Rancher release contains all accumulated improvements to performance and stability, plus updates based on developer experience and user feedback.
* Always scale up gradually, and monitor and observe any changes in behavior while doing do. It is usually easier to resolve performance problems as soon as they surface, before other problems obscure the root cause.
* Reduce network latency between the upstream Rancher cluster and downstream clusters to the extent possible. Note that latency is, among other factors, a function of geographic distance - if you require clusters or nodes spread across the world, consider multiple Rancher installations.
## Minimizing Load on the Upstream Cluster
When scaling up Rancher, one typical bottleneck is resource growth in the upstream (local) Kubernetes cluster. The upstream cluster contains information for all downstream clusters. Many operations that apply to downstream clusters create new objects in the upstream cluster and require computation from handlers running in the upstream cluster.
### Minimizing Third-Party Software on the Upstream Cluster
Recommendations outlined in the [general Rancher recommendations](./tips-for-running-rancher.md#minimize-third-party-software-on-the-upstream-cluster) are particularly important in a high scale context.
### Managing Your Object Counts
Etcd is the backing database for Kubernetes and for Rancher. The database may eventually encounter limitations to the number of a single Kubernetes resource type it can store. Exact limits vary and depend on a number of factors. However, experience indicates that performance issues frequently arise once a single resource type's object count exceeds 60,000. Often that type is `RoleBinding`.
This is typical in Rancher, as many operations create new `RoleBinding` objects in the upstream cluster as a side effect.
You can reduce the number of `RoleBindings` in the upstream cluster in the following ways:
* Only add users to clusters and projects when necessary.
* Remove clusters and projects when they are no longer needed.
* Only use custom roles if necessary.
* Use as few rules as possible in custom roles.
* Consider whether adding a role to a user is redundant.
* Consider using less, but more powerful, clusters.
* Kubernetes permissions are always "additive" (allow-list) rather than "subtractive" (deny-list). Try to minimize configurations that gives access to all but one aspect of a cluster, project, or namespace, as that will result in the creation of a high number of `RoleBinding` objects.
* Experiment to see if creating new projects or clusters manifests in fewer `RoleBindings` for your specific use case.
### Using External Authentication
If you have fifty or more users, you should configure an [external authentication provider](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md). This is necessary for better performance.
After you configure external authentication, make sure to assign permissions to groups instead of to individual users. This helps reduce the `RoleBinding` object count.
### RoleBinding Count Estimation
Predicting how many `RoleBinding` objects a given configuration will create is complicated. However, the following considerations can offer a rough estimate:
* For a minimum estimate, use the formula `32C + U + 2UaC + 8P + 5Pa`.
* `C` is the total number of clusters.
* `U` is the total number of users.
* `Ua` is the average number of users with a membership on a cluster.
* `P` is the total number of projects.
* `Pa` is the average number of users with a membership on a project.
* The number of `RoleBindings` increases linearly with the number of clusters, projects, and users.
### Using New Apps Over Legacy Apps
Rancher uses two Kubernetes app resources: `apps.projects.cattle.io` and `apps.cattle.cattle.io`. Legacy apps, represented by `apps.projects.cattle.io`, were introduced with the former Cluster Manager UI and are now outdated. Current apps, represented by `apps.catalog.cattle.io`, are found in the Cluster Explorer UI for their respective cluster. `Apps.cattle.cattle.io` apps are preferable because their data resides in downstream clusters, which frees up resources in the upstream cluster.
You should remove any remaining legacy apps that appear in the Cluster Manager UI, and replace them with apps in the Cluster Explorer UI. Create any new apps only in the Cluster Explorer UI.
### Using the Authorized Cluster Endpoint (ACE)
An [Authorized Cluster Endpoint](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) (ACE) provides access to the Kubernetes API of Rancher-provisioned RKE2 and K3s clusters. When enabled, the ACE adds a context to kubeconfig files generated for the cluster. The context uses a direct endpoint to the cluster, thereby bypassing Rancher. This reduces load on Rancher for cases where unmediated API access is acceptable or preferable. See [Authorized Cluster Endpoint](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) for more information and configuration instructions.
### Reducing Event Handler Executions
The bulk of Rancher's logic occurs on event handlers. These event handlers run on an object whenever the object is updated, and when Rancher is started. Additionally, they run every 10 hours when Rancher syncs caches. In scaled setups these scheduled runs come with huge performance costs because every handler is being run on every applicable object. However, the scheduled handler execution can be disabled with the `CATTLE_SYNC_ONLY_CHANGED_OBJECTS` environment variable. If resource allocation spikes are seen every 10 hours, this setting can help.
The value for `CATTLE_SYNC_ONLY_CHANGED_OBJECTS` can be a comma separated list of the following options. The values refer to types of handlers and controllers (the structures that contain and run handlers). Adding the controller types to the variable disables that set of controllers from running their handlers as part of cache resyncing.
* `mgmt` refers to management controllers which only run on one Rancher node.
* `user` refers to user controllers which run for every cluster. Some of these run on the same node as management controllers, while others run in the downstream cluster. This option targets the former.
* `scaled` refers to scaled controllers which run on every Rancher node. You should avoid setting this value, as the scaled handlers are responsible for critical functions and changes may disrupt cluster stability.
In short, if you notice CPU usage peaks every 10 hours, add the `CATTLE_SYNC_ONLY_CHANGED_OBJECTS` environment variable to your Rancher deployment (in the `spec.containers.env` list) with the value `mgmt,user`
## Optimizations Outside of Rancher
Important influencing factors are the underlying cluster's own performance and configuration. The upstream cluster, if misconfigured, can introduce a bottleneck Rancher software has no chance to resolve.
### Manage Upstream Cluster Nodes Directly with RKE2
As Rancher can be very demanding on the upstream cluster, especially at scale, you should have full administrative control of the cluster's configuration and nodes. To identify the root cause of excess resource consumption, use standard Linux troubleshooting techniques and tools. This can aid in distinguishing between whether Rancher, Kubernetes, or operating system components are causing issues.
Although managed Kubernetes services make it easier to deploy and run Kubernetes clusters, they are discouraged for the upstream cluster in high scale scenarios. Managed Kubernetes services typically limit access to configuration and insights on individual nodes and services.
Use RKE2 for large scale use cases.
### Keep all Upstream Cluster Nodes co-located
To provide high availability, Kubernetes is designed to run nodes and control components in different zones. However, if nodes and control plane components are located in different zones, network traffic may be slower.
Traffic between Rancher components and the Kubernetes API is especially sensitive to network latency, as is etcd traffic between nodes.
To improve performance, run all upstream node clusters in the same location. In particular, make sure that latency between etcd nodes and Rancher is as low as possible.
### Keeping Kubernetes Versions Up to Date
You should keep the local Kubernetes cluster up to date. This will ensure that your cluster has all available performance enhancements and bug fixes.
### Optimizing etcd
Etcd is the backend database for Kubernetes and for Rancher. It plays a very important role in Rancher performance.
The two main bottlenecks to [etcd performance](https://etcd.io/docs/v3.5/op-guide/performance/) are disk and network speed. Etcd should run on dedicated nodes with a fast network setup and with SSDs that have high input/output operations per second (IOPS). For more information regarding etcd performance, see [Slow etcd performance (performance testing and optimization)](https://www.suse.com/support/kb/doc/?id=000020100) and [Tuning etcd for Large Installations](../../../how-to-guides/advanced-user-guides/tune-etcd-for-large-installs.md). Information on disks can also be found in the [Installation Requirements](../../../getting-started/installation-and-upgrade/installation-requirements/installation-requirements.md#disks).
It's best to run etcd on exactly three nodes, as adding more nodes will reduce operation speed. This may be counter-intuitive to common scaling approaches, but it's due to etcd's [replication mechanisms](https://etcd.io/docs/v3.5/faq/#what-is-maximum-cluster-size).
Etcd performance will also be negatively affected by network latency between nodes as that will slow down network communication. Etcd nodes should be located together with Rancher nodes.
### Browser Requirements
At high scale, Rancher transfers more data from the upstream cluster to UI components running in the browser, and those components also need to perform more processing.
For best performance, ensure that the host running the hardware meets these requirements:
- 2020 i5 10th generation Intel (4 cores) or equivalent
- 8 GB RAM
- Total network bandwidth to the upstream cluster: 72 Mb/s (equivalent to a single 802.11n Wi-Fi 4 link stream, ~8 MB/s http download throughput)
- Round-trip time (ping time) from browser to upstream cluster: 150 ms or less
versioned_docs/version-2.14/reference-guides/cli-with-rancher/cli-with-rancher.md
+9
View File
@@ -0,0 +1,9 @@
---
title: CLI with Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cli-with-rancher"/>
</head>
Interact with Rancher using command line interface (CLI) tools from your workstation. The following docs will describe the [Rancher CLI](rancher-cli.md) and [kubectl Utility](kubectl-utility.md).
versioned_docs/version-2.14/reference-guides/cli-with-rancher/kubectl-utility.md
+37
View File
@@ -0,0 +1,37 @@
---
title: kubectl Utility
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cli-with-rancher/kubectl-utility"/>
</head>
## kubectl
Interact with Rancher using kubectl.
### kubectl Utility
Install the `kubectl` utility. See [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
Configure kubectl by visiting your cluster in the Rancher Web UI, clicking on `Kubeconfig`, copying contents, and putting them into your `~/.kube/config` file.
Run `kubectl cluster-info` or `kubectl get pods` successfully.
### Authentication with kubectl and kubeconfig Tokens with TTL
_Requirements_
If admins have [kubeconfig token generation turned off](../../api/api-tokens.md#disable-tokens-in-generated-kubeconfigs), the kubeconfig file requires the [Rancher CLI](./rancher-cli.md) to be present in your PATH when you run `kubectl`. Otherwise, you’ll see an error like:
`Unable to connect to the server: getting credentials: exec: exec: "rancher": executable file not found in $PATH`.
This feature enables kubectl to authenticate with the Rancher server and get a new kubeconfig token when required. The following auth providers are currently supported:
1. Local
2. Active Directory (LDAP only)
3. FreeIPA
4. OpenLDAP
5. SAML providers: Ping, Okta, ADFS, Keycloak, Shibboleth
6. Azure AD
When you first run kubectl, for example, `kubectl get pods`, you are prompted to pick an auth provider and log in with the Rancher server. The kubeconfig token is cached in the path where you run kubectl under `./.cache/token`. This token is valid until [it expires](../../api/api-tokens.md#disable-tokens-in-generated-kubeconfigs), or [gets deleted from the Rancher server](../../api/api-tokens.md#deleting-tokens). Upon expiration, you must log in with the Rancher server again to run the `kubectl get pods` command.
versioned_docs/version-2.14/reference-guides/cli-with-rancher/rancher-cli.md
+93
View File
@@ -0,0 +1,93 @@
---
title: Rancher CLI
description: Interact with Rancher using command line interface (CLI) tools from your workstation.
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cli-with-rancher/rancher-cli"/>
</head>
The Rancher CLI (Command Line Interface) is a unified tool that you can use to interact with Rancher. With this tool, you can operate Rancher using a command line rather than the GUI.
## Download Rancher CLI
The binary can be downloaded directly from the UI.
1. In the upper left corner, click **☰**.
1. At the bottom of the navigation sidebar menu, click the Rancher version number.
1. Under the **CLI Downloads section**, there are links to download the binaries for Windows, Mac, and Linux. You can also check the [releases page for our CLI](https://github.com/rancher/cli/releases) for direct downloads of the binary.
## Requirements
After you download the Rancher CLI, you need to make a few configurations. Rancher CLI requires:
- Your Rancher Server URL, which is used to connect to Rancher Server.
- An API Bearer Token, which is used to authenticate with Rancher. For more information about obtaining a Bearer Token, see [Creating an API Key](../user-settings/api-keys.md).
## CLI Authentication
Before you can use Rancher CLI to control your Rancher Server, you must authenticate using an API Bearer Token. Log in using the following command (replace `<BEARER_TOKEN>` and `<SERVER_URL>` with your information):
```bash
$ ./rancher login https://<SERVER_URL> --token <BEARER_TOKEN>
```
If Rancher Server uses a self-signed certificate, Rancher CLI prompts you to continue with the connection.
## Project Selection
Before you can perform any commands, you must select a Rancher project to perform those commands against. To select a [project](../../how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces.md) to work on, use the command `./rancher context switch`. When you enter this command, a list of available projects displays. Enter a number to choose your project.
**Example: `./rancher context switch` Output**
```
User:rancher-cli-directory user$ ./rancher context switch
NUMBER CLUSTER NAME PROJECT ID PROJECT NAME
1 cluster-2 c-7q96s:p-h4tmb project-2
2 cluster-2 c-7q96s:project-j6z6d Default
3 cluster-1 c-lchzv:p-xbpdt project-1
4 cluster-1 c-lchzv:project-s2mch Default
Select a Project:
```
After you enter a number, the console displays a message that you've changed projects.
```
INFO[0005] Setting new context to project project-1
INFO[0005] Saving config to /Users/markbishop/.ranchcli2.json
```
Ensure you can run `rancher kubectl get pods` successfully.
## Commands
The following commands are available for use in Rancher CLI.
| Command | Result |
|---|---|
| `clusters, [cluster]` | Performs operations on your [clusters](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md). |
| `context` | Switches between Rancher [projects](../../how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces.md). For an example, see [Project Selection](#project-selection). |
| `inspect [OPTIONS] [RESOURCEID RESOURCENAME]` | Displays details about [Kubernetes resources](https://kubernetes.io/docs/reference/kubectl/cheatsheet/#resource-types) or Rancher resources (i.e.: [projects](../../how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces.md) and [workloads](../../how-to-guides/new-user-guides/kubernetes-resources-setup/workloads-and-pods/workloads-and-pods.md)). Specify resources by name or ID. |
| `kubectl` | Runs [kubectl commands](https://kubernetes.io/docs/reference/kubectl/overview/#operations). |
| `login, [l]` | Logs into a Rancher Server. For an example, see [CLI Authentication](#cli-authentication). |
| `machines, [machine]` | Performs operations on machines. |
| `namespaces, [namespace]` | Performs operations on [namespaces](../../how-to-guides/new-user-guides/manage-namespaces.md). |
| `nodes, [node]` | Performs operations on [nodes](../../how-to-guides/new-user-guides/manage-clusters/nodes-and-node-pools.md). |
| `projects, [project]` | Performs operations on [projects](../../how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces.md). |
| `ps` | Displays [workloads](../../how-to-guides/new-user-guides/kubernetes-resources-setup/workloads-and-pods/workloads-and-pods.md) in a project. |
| `server` | Performs operations for the server. |
| `settings, [setting]` | Shows the current settings for your Rancher Server. |
| `ssh` | Connects to one of your cluster nodes using the SSH protocol. |
| `up` | Applies compose config. |
| `wait` | Waits for resources cluster, app, project, multiClusterApp. |
| `token` | Authenticates and generates new kubeconfig token. |
| `help, [h]` | Shows a list of commands or help for one command. |
## Rancher CLI Help
Once logged into Rancher Server using the CLI, enter `./rancher --help` for a list of commands.
All commands accept the `--help` flag, which documents each command's usage.
## Limitations
The Rancher CLI **cannot** be used to install [dashboard apps or Rancher feature charts](../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md).
versioned_docs/version-2.14/reference-guides/cluster-configuration/cluster-configuration.md
+32
View File
@@ -0,0 +1,32 @@
---
title: Cluster Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration"/>
</head>
After you provision a Kubernetes cluster using Rancher, you can still edit options and settings for the cluster.
For information on editing cluster membership, go to [this page.](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/add-users-to-clusters.md)
## Cluster Configuration References
The cluster configuration options depend on the type of Kubernetes cluster:
- [RKE2 Cluster Configuration](rancher-server-configuration/rke2-cluster-configuration.md)
- [K3s Cluster Configuration](rancher-server-configuration/k3s-cluster-configuration.md)
- [EKS Cluster Configuration](rancher-server-configuration/eks-cluster-configuration.md)
- [GKE Cluster Configuration](rancher-server-configuration/gke-cluster-configuration/gke-cluster-configuration.md)
- [AKS Cluster Configuration](rancher-server-configuration/aks-cluster-configuration.md)
## Cluster Management Capabilities by Cluster Type
The options and settings available for an existing cluster change based on the method that you used to provision it.
The following table summarizes the options and settings available for each cluster type:
import ClusterCapabilitiesTable from '../../shared-files/_cluster-capabilities-table.md';
<ClusterCapabilitiesTable />
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/downstream-cluster-configuration.md
+9
View File
@@ -0,0 +1,9 @@
---
title: Downstream Cluster Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration"/>
</head>
The following docs will discuss [machine configuration](machine-configuration/machine-configuration.md).
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/amazon-ec2.md
+97
View File
@@ -0,0 +1,97 @@
---
title: EC2 Machine Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/amazon-ec2"/>
</head>
For more details about EC2 nodes, refer to the official documentation for the [EC2 Management Console](https://aws.amazon.com/ec2).
### Region
The geographical [region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html) in which to build your cluster.
### Zone
The [zone](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-availability-zones), an isolated location within a region to build your cluster
### Instance Type
The [instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html), which determines the hardware characteristics, used to provision your cluster.
### Root Disk Size
Configure the size (in GB) for your [root device](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html).
### VPC/Subnet
The [VPC](https://docs.aws.amazon.com/vpc/latest/userguide/configure-your-vpc.html) or specific [subnet](https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html), an IP range in your VPC, to add your resources to.
### IAM Instance Profile Name
The name of the [instance profile](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) used to pass an IAM role to an EC2 instance.
## Advanced Options
### AMI ID
The [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) used for the nodes in your cluster.
### SSH Username for AMI
The username for connecting to your launched instances. Refer to [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connection-prereqs.html) for the default usernames to selected AMIs. For AMIs not listed, check with the AMI provider.
### Security Group
Choose the default security group or configure a security group.
Please refer to [Amazon EC2 security group when using Node Driver](../../../../getting-started/installation-and-upgrade/installation-requirements/port-requirements.md#rancher-aws-ec2-security-group) to see what rules are created in the `rancher-nodes` Security Group.
### EBS Root Volume Type
The [EBS volume type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html) to use for the root device.
### Encrypt EBS Volume
Enable [Amazon EBS Encryption](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html).
### Request Spot Instance
Enable option to [request spot instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html) and specify the maximum instance price per hour you're willing to pay.
### Use only private address
Enable option on use only [private addresses](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html).
### EBS-Optimized Instance
Use an [EBS-optimized instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html).
### Allow access to EC2 metadata
Enable access to [EC2 metadata](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).
### Use tokens for metadata
Use [Instance Metadata Service Version 2 (IMDSv2)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html), a token-based method to access metadata.
### Add Tag
Add metadata using [tags](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) to categorize resources.
### IPv6 Address Count
Specify how many IPv6 addresses to assign to the instance’s network interface.
### IPv6 Address Only
Enable this option if the instance should use IPv6 exclusively. IPv6-only VPCs or subnets require this. When enabled, the instance will have IPv6 as its sole address, and the IPv6 Address Count must be greater than zero.
### HTTP Protocol IPv6
Enable or disable IPv6 endpoints for the instance metadata service.
### Enable Primary IPv6
Enable this option to designate the first assigned IPv6 address as the primary address. This ensures a consistent, non-changing IPv6 address for the instance. It does not control whether IPv6 addresses are assigned.
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/azure.md
+124
View File
@@ -0,0 +1,124 @@
---
title: Azure Machine Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/azure"/>
</head>
For more information about Azure, refer to the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/?product=featured)
### Environment
Microsoft provides multiple [clouds](https://docs.microsoft.com/en-us/cli/azure/cloud?view=azure-cli-latest) for compliance with regional laws, which are available for your use:
- AzurePublicCloud
- AzureGermanCloud
- AzureChinaCloud
- AzureUSGovernmentCloud
### Location
Configure the cluster and node [location](https://docs.microsoft.com/en-us/azure/virtual-machines/regions).
### Resource Group
A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.
Use an existing resource group or enter a resource group name and one will be created for you.
For information on managing resource groups, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal)
### Availability Set (unmanaged)
Name or ID of an existing [availability set](https://docs.microsoft.com/en-us/azure/virtual-machines/availability-set-overview) to add the VM to.
### Image
The name of the operating system image provided as an ARM resource identifier. Requires using managed disk.
### VM Size
Choose a size for each VM in the node pool. For details about each VM size, see [this page.](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/)
## Advanced Options
### Fault Domain Count
Fault domains define the group of virtual machines that share a common power source and network switch. If the availability set has already been created, the fault domain count will be ignored.
For more information on fault domains, see [refer here](https://docs.microsoft.com/en-us/azure/virtual-machines/availability-set-overview#how-do-availability-sets-work).
### Update Domain Count
Update domains indicate groups of virtual machines and underlying physical hardware that can be rebooted at the same time. If the availability set has already been created, the update domain count will be ignored.
For more information on update domains, see [refer here](https://docs.microsoft.com/en-us/azure/virtual-machines/availability-set-overview#how-do-availability-sets-work).
### Purchase Plan
Some VM images in the Azure Marketplace require a plan. If applicable, select a purchase plan, formatted as `publisher:product:plan`, to use with your chosen image.
### Subnet
The name of the subnet when creating a new VNet or referencing an existing one.
Default: `docker-machine`
### Subnet Prefix
The subnet IP address prefix to use when creating a new VNet in CIDR format.
Default: `192.168.0.0/16`
### Virtual Network
The [virtual network](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview) to use or create if one does not exist. Formatted as `[resourcegroup:]name`.
### Public IP Options
#### No Public IP
Do not allocate a public IP address.
#### Static Public IP
Allocate a static public IP address.
### Use Private IP
Use a static private IP address.
### Private IP Address
Configure a static private IP address to use.
### Network Security Group
The [network security group](https://docs.microsoft.com/en-us/azure/virtual-network/network-security-groups-overview) to use. All nodes using this template will use the supplied network security group. If no network security group is provided, a new one will be created for each node.
### DNS Label
A unique DNS name label for the public IP address.
### Storage Type
The [storage account](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-overview) type to use with your VMs. Options include Standard LRS, Standard ZRS, Standard GRS, Standard RAGRS, and Premium LRS.
### Use Managed Disks
[Azure managed disks](https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview) are block-level storage volumes that are managed by Azure and used with Azure Virtual Machines. Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability.
### Managed Disk Size
The size in GB for the disk for each node.
### SSH Username
The username used to create an SSH connection to your nodes.
### Open Port
Opens inbound traffic on specified ports. When using an existing Network Security Group, Open Ports are ignored.
Default: `2379/tcp, 2380/tcp, 6443/tcp, 9796/tcp, 10250/tcp, 10251/tcp, 10252/tcp, 10256/tcp` and `8472/udp, 4789/udp`
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/digitalocean.md
+39
View File
@@ -0,0 +1,39 @@
---
title: DigitalOcean Machine Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/digitalocean"/>
</head>
For more details about DigitalOcean, Droplets, refer to the [official documentation](https://docs.digitalocean.com/products/compute/).
### Region
Configure the [region](https://docs.digitalocean.com/glossary/region/) where Droplets are created.
### Size
Configure the [size](https://docs.digitalocean.com/products/droplets/resources/choose-plan/) of Droplets.
### OS Image
Configure the operating system [image](https://docs.digitalocean.com/products/images/) Droplets are created from.
### Monitoring
Enable the DigitalOcean agent for additional [monitoring](https://docs.digitalocean.com/products/monitoring/).
### IPv6
Enable IPv6 for Droplets.
For more information, refer to the [Digital Ocean IPv6 documentation](https://docs.digitalocean.com/products/networking/ipv6).
### Private Networking
Enable private networking for Droplets.
### Droplet Tags
Apply a tag (label) to a Droplet. Tags may only contain letters, numbers, colons, dashes, and underscores. For example, `my_server`.
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/google-gce.md
+87
View File
@@ -0,0 +1,87 @@
---
title: GCE Machine Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/google-gce"/>
</head>
For more information about Google Cloud Platform (GCP) and the Google Compute Engine (GCE), refer to the official [GCP documentation](https://cloud.google.com/docs).
### Zone
The GCP Region and Zone that the VM will be deployed to. For example, `us-east1-b`.
### Machine Image Project
The image project that the desired image families belong to.
### Machine Image Family
The image family that the desired machine operating system belongs to.
### Machine Image
The operating system that will be installed onto the VM.
### Disk Type
The type of the disk attached to the VM. The available types may differ between regions.
### Disk Size
The size of the disk attached to the VM, in Gigabytes.
### Machine Type
The type of VM that will be deployed. Machine types determine the number of resources (vCPU, RAM, etc.) allocated for each node.
### Network
The VPC network that the VM will be created in. This value cannot be changed once the machine pool has been provisioned.
### Subnet
The VPC subnetwork tha the VM will be created in. This value cannot be changed once the machine pool has been provisioned.
### Username
A custom username set as the default user of the GCE VM.
### External Address
The desired external IP address for the GCE VM.
### Scopes
A list of OAuth2 scopes which allow the VM to access other GCP APIs.
### Allow Internal Communication
By default, a VPC firewall rule is automatically created to expose a fixed set of ports within the VPC to facilitate communication between cluster nodes. This behavior can be disabled on a per machine pool basis, when clicking the `Show Advanced` option and disabling the `Allow Internal Communication` checkbox.
### Expose External ports
A list of ports to be opened _externally_ to the wider internet. Open ports are defined at the machine pool level. Enabling this option will result in the automatic creation of a VPC firewall rule. This rule will be automatically deleted when the cluster or machine pool is deleted.
### Network Tags
Tags is a list of _network tags_, which can be used to associate preexisting Firewall Rules with all VMs within a machine pool.
### Labels
A comma separated list of custom labels to be attached to all VMs within a given machine pool. Unlike Tags, Labels do not influence networking behavior and only serve to organize cloud resources.
## Advanced Options
When creating clusters via the Rancher UI some options are automatically configured for you. However, when creating machine config objects manually, you must ensure you properly configure the below fields.
### external-firewall-rule-prefix
A prefix that will be used when creating the firewall rule to expose ports publicly. Ideally, this should be a concatenation the machine pool name and the cluster name. This field must be set if the machine pool is configured to expose ports publicly, otherwise it can be omitted.
### internal-firewall-rule-prefix
A prefix that will be used when creating the internal firewall rule which allows for communication between nodes within the cluster. If this field is omitted, no internal firewall rule will be created.
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration/machine-configuration.md
+9
View File
@@ -0,0 +1,9 @@
---
title: Machine Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/machine-configuration"/>
</head>
Machine configuration is the arrangement of resources assigned to a virtual machine. Please see the docs for [Amazon EC2](amazon-ec2.md), [DigitalOcean](digitalocean.md), [Google GCE](google-gce.md), and [Azure](azure.md) to learn more.
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/amazon-ec2.md
+51
View File
@@ -0,0 +1,51 @@
---
title: EC2 Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/amazon-ec2"/>
</head>
For more details about EC2, nodes, refer to the official documentation for the [EC2 Management Console](https://aws.amazon.com/ec2).
### Region
In the **Region** field, select the same region that you used when creating your cloud credentials.
### Cloud Credentials
Your AWS account access information, stored in a [cloud credential.](../../../user-settings/manage-cloud-credentials.md)
See [Amazon Documentation: Creating Access Keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) how to create an Access Key and Secret Key.
See [Amazon Documentation: Creating IAM Policies (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start) how to create an IAM policy.
See [Amazon Documentation: Adding Permissions to a User (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) how to attach an IAM
See our three example JSON policies:
- [Example IAM Policy](../../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy)
- [Example IAM Policy with PassRole](../../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy-with-passrole) (needed if you want to use [Kubernetes Cloud Provider](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md) or want to pass an IAM Profile to an instance)
- [Example IAM Policy to allow encrypted EBS volumes](../../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy-to-allow-encrypted-ebs-volumes) policy to an user.
### Authenticate & Configure Nodes
Choose an availability zone and network settings for your cluster.
### Security Group
Choose the default security group or configure a security group.
Please refer to [Amazon EC2 security group when using Node Driver](../../../../getting-started/installation-and-upgrade/installation-requirements/port-requirements.md#rancher-aws-ec2-security-group) to see what rules are created in the `rancher-nodes` Security Group.
If you provide your own security group for an EC2 instance, please note that Rancher will not modify it. As such, you will be responsible for ensuring that your security group is set to allow the [necessary ports for Rancher to provision the instance](../../../../getting-started/installation-and-upgrade/installation-requirements/port-requirements.md#ports-for-rancher-server-nodes-on-rke2). For more information on controlling inbound and outbound traffic to EC2 instances with security groups, refer [here](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#WorkingWithSecurityGroups).
### Instance Options
Configure the instances that will be created. Make sure you configure the correct **SSH User** for the configured AMI. It is possible that a selected region does not support the default instance type. In this scenario you must select an instance type that does exist, otherwise an error will occur stating the requested configuration is not supported.
If you need to pass an **IAM Instance Profile Name** (not ARN), for example, when you want to use a [Kubernetes Cloud Provider](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md), you will need an additional permission in your policy. See [Example IAM policy with PassRole](../../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy-with-passrole) for an example policy.
### Engine Options
In the **Engine Options** section of the node template, you can configure the container daemon. You may want to specify the container version or a container image registry mirror.
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/azure.md
+37
View File
@@ -0,0 +1,37 @@
---
title: Azure Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/azure"/>
</head>
For more information about Azure, refer to the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/?product=featured)
Account access information is stored as a cloud credential. Cloud credentials are stored as Kubernetes secrets. Multiple node templates can use the same cloud credential. You can use an existing cloud credential or create a new one.
- **Placement** sets the geographical region where your cluster is hosted and other location metadata.
- **Network** configures the networking used in your cluster.
- **Instance** customizes your VM configuration.
:::note
If using a VNet in a different Resource Group than the VMs, the VNet name should be prefixed with the Resource Group name. For example, `<resource group>:<vnet>`.
:::
If you use Docker, the [Docker daemon](https://docs.docker.com/engine/docker-overview/#the-docker-daemon) configuration options include:
- **Labels:** For information on labels, refer to the [Docker object label documentation.](https://docs.docker.com/config/labels-custom-metadata/).
- **Docker Engine Install URL:** Determines what Docker version will be installed on the instance.
:::note
If you're provisioning Red Hat Enterprise Linux (RHEL) or CentOS nodes, leave the **Docker Install URL** field as the default value, or select **none**. This will bypass a check for Docker installation, as Docker is already installed on these node types.
If you set **Docker Install URL** to a value other than the default or **none**, you might see an error message such as the following: `Error creating machine: RHEL ssh command error: command: sudo -E yum install -y curl err: exit status 1 output: Updating Subscription Management repositories.`
:::
- **Registry mirrors:** Docker Registry mirror to be used by the Docker daemon.
- **Other advanced options:** Refer to the [Docker daemon option reference](https://docs.docker.com/engine/reference/commandline/dockerd/).
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/digitalocean.md
+22
View File
@@ -0,0 +1,22 @@
---
title: DigitalOcean Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/digitalocean"/>
</head>
Account access information is stored as a cloud credential. Cloud credentials are stored as Kubernetes secrets. Multiple node templates can use the same cloud credential. You can use an existing cloud credential or create a new one.
## Droplet Options
The **Droplet Options** provision your cluster's geographical region and specifications.
## Docker Daemon
If you use Docker, the [Docker daemon](https://docs.docker.com/engine/docker-overview/#the-docker-daemon) configuration options include:
- **Labels:** For information on labels, refer to the [Docker object label documentation.](https://docs.docker.com/config/labels-custom-metadata/)
- **Docker Engine Install URL:** Determines what Docker version will be installed on the instance.
- **Registry mirrors:** Docker Registry mirror to be used by the Docker daemon
- **Other advanced options:** Refer to the [Docker daemon option reference](https://docs.docker.com/engine/reference/commandline/dockerd/).
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/node-template-configuration.md
+11
View File
@@ -0,0 +1,11 @@
---
title: Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration"/>
</head>
<EOLRKE1Warning />
To learn about node template config, refer to [EC2 Node Template Configuration](amazon-ec2.md), [DigitalOcean Node Template Configuration](digitalocean.md), [Azure Node Template Configuration](azure.md), [vSphere Node Template Configuration](vsphere.md), and [Nutanix Node Template Configuration](nutanix.md).
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/nutanix.md
+70
View File
@@ -0,0 +1,70 @@
---
title: Nutanix Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/nutanix"/>
</head>
## Account Access
| Parameter | Required | Description | Default
|:-----------------------------|:--------:|:-----------------------------------------------------------------|:-----
| Management Endpoint | ✓ | Hostname/IP address of Prism Central |
| Username | ✓ | Username of the Prism Central user |
| Password | ✓ | Password of the Prism Central user |
| Allow insecure communication | | Set to true to allow insecure SSL communication to Prism Central | False
## Scheduling
Choose what Nutanix cluster the virtual machine will be scheduled to.
| Parameter | Required | Description
|:----------|:--------:|:----------------------------------------------------------------------------
| Cluster | ✓ | Name of the Nutanix cluster where the VM should be deployed (case sensitive)
## Instance Options
In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.
| Parameter | Required | Description | Default
|:---------------------|:--------:|:--------------------------------------------------------------------------------------------|:-------
| CPUs | | Number of vCPUs allocated to the VM (cores) | 2
| Memory | | Amount of RAM allocated to the VM (MB) | 2 GB
| Template Image | ✓ | Name of the Disk Image template to clone as the VM's primary disk (must support cloud-init) |
| VM Disk Size | | New size of the VM's primary disk (in GiB) |
| Additional Disk Size | | Size of an additional disk to add to the VM (in GiB) |
| Storage Container | | Storage container _UUID_ in which to provision an additional disk |
| Cloud Config YAML | | Cloud-init to provide to the VM (will be patched with Rancher root user) |
| Network | ✓ | Name(s) of the network(s) to attach to the VM |
| VM Categories | | Name(s) of any categories to be applied to the VM |
The VM may use any modern Linux operating system that is configured with support for [cloud-init](https://cloudinit.readthedocs.io/en/latest/) using the [Config Drive v2 datasource](https://cloudinit.readthedocs.io/en/latest/reference/datasources/configdrive.html).
## Networks
The node template allows a VM to be provisioned with multiple networks. In the **Network** field, you can click **Add** to add any networks available to you in AOS.
## VM Categories
A category is a grouping of entities into a key value pair. Typically, VMs are assigned to a category based on some criteria. Policies can then be tied to those entities that are assigned (grouped by) a specific category value.
## cloud-init
[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) allows you to initialize your nodes by applying configuration on the first boot. This may involve things such as creating users or authorizing SSH keys.
To make use of cloud-init initialization, paste a cloud config using valid YAML syntax into the **Cloud Config YAML** field. Refer to the [cloud-init documentation](https://cloudinit.readthedocs.io/en/latest/topics/examples.html) for a commented set of examples of supported cloud config directives.
Note that cloud-init based network configuration is not recommended and only supported via user data `runcmd` rather than by NoCloud or other network configuration datasources.
Nutanix IP Address Management (IPAM) or another DHCP service is recommended.
## Engine Options
In the **Engine Options** section of the node template, you can configure the container daemon. You may want to specify the container version or a container image registry mirror.
:::note
If you're provisioning Red Hat Enterprise Linux (RHEL) or CentOS nodes, leave the **Docker Install URL** field as the default value, or select **none**. This will bypass a check for Docker installation, as Docker is already installed on these node types.
If you set **Docker Install URL** to a value other than the default or **none**, you might see an error message such as the following: `Error creating machine: RHEL ssh command error: command: sudo -E yum install -y curl err: exit status 1 output: Updating Subscription Management repositories.`
:::
versioned_docs/version-2.14/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/vsphere.md
+100
View File
@@ -0,0 +1,100 @@
---
title: VMware vSphere Node Template Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/vsphere"/>
</head>
## Account Access
| Parameter | Required | Description |
|:----------------------|:--------:|:-----|
| Cloud Credentials | * | Your vSphere account access information, stored in a [cloud credential.](../../../user-settings/manage-cloud-credentials.md) |
Your cloud credential has these fields:
| Credential Field | Description |
|-----------------|--------------|
| vCenter or ESXi Server | Enter the vCenter or ESXi hostname/IP. ESXi is the virtualization platform where you create and run virtual machines and virtual appliances. vCenter Server is the service through which you manage multiple hosts connected in a network and pool host resources. |
| Port | Configure the port of the vCenter or ESXi server. |
| Username and password | Enter your vSphere login username and password. |
## Scheduling
Choose what hypervisor the virtual machine will be scheduled to.
The fields in the **Scheduling** section should auto-populate with the data center and other scheduling options that are available to you in vSphere.
| Field | Required | Explanation |
|---------|---------------|-----------|
| Data Center | * | Choose the name/path of the data center where the VM will be scheduled. |
| Resource Pool | | Name of the resource pool to schedule the VMs in. Resource pools can be used to partition available CPU and memory resources of a standalone host or cluster, and they can also be nested. Leave blank for standalone ESXi. If not specified, the default resource pool is used. |
| Data Store | * | If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk. |
| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The VM folders in this dropdown menu directly correspond to your VM folders in vSphere. The folder name should be prefaced with `vm/` in your vSphere config file. |
| Host | | The IP of the host system to schedule VMs in. Leave this field blank for a standalone ESXi or for a cluster with DRS (Distributed Resource Scheduler). If specified, the host system's pool will be used and the **Resource Pool** parameter will be ignored. |
| Graceful Shutdown Timeout | | The amount of time, in seconds, that Rancher waits before deleting virtual machines on a cluster. If set to `0`, graceful shutdown is disabled. Only accepts integer values. |
## Instance Options
In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.
| Parameter | Required | Description |
|:----------------|:--------:|:-----------|
| CPUs | * | Number of vCPUS to assign to VMs. |
| Memory | * | Amount of memory to assign to VMs. |
| Disk | * | Size of the disk (in MB) to attach to the VMs. |
| Creation method | * | The method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template. Depending on the creation method, you will also have to specify a VM template, content library, existing VM, or ISO. For more information on creation methods, refer to [About VM Creation Methods.](#about-vm-creation-methods) |
| Cloud Init | | URL of a `cloud-config.yml` file or URL to provision VMs with. This file allows further customization of the operating system, such as network configuration, DNS servers, or system daemons. The operating system must support `cloud-init`. |
| Networks | | Name(s) of the network to attach the VM to. |
| Configuration Parameters used for guestinfo | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo](https://rancher.com/docs/os/v1.x/en/installation/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). |
### About VM Creation Methods
In the **Creation method** field, configure the method used to provision VMs in vSphere. Available options include creating VMs that boot from a RancherOS ISO or creating VMs by cloning from an existing virtual machine or [VM template](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html).
The existing VM or template may use any modern Linux operating system that is configured with support for [cloud-init](https://cloudinit.readthedocs.io/en/latest/) using the [NoCloud datasource](https://canonical-cloud-init.readthedocs-hosted.com/en/latest/reference/datasources/nocloud.html).
Choose the way that the VM will be created:
- **Deploy from template: Data Center:** Choose a VM template that exists in the data center that you selected.
- **Deploy from template: Content Library:** First, select the [Content Library](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-254B2CE8-20A8-43F0-90E8-3F6776C2C896.html) that contains your template, then select the template from the populated list **Library templates**.
- **Clone an existing virtual machine:** In the **Virtual machine** field, choose an existing VM that the new VM will be cloned from.
- **Install from boot2docker ISO:** Ensure that the **OS ISO URL** field contains the URL of a VMware ISO release for RancherOS (`rancheros-vmware.iso`). Note that this URL must be accessible from the nodes running your Rancher server installation.
## Networks
The node template now allows a VM to be provisioned with multiple networks. In the **Networks** field, you can now click **Add Network** to add any networks available to you in vSphere.
## Node Tags and Custom Attributes
Tags allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.
For tags, all your vSphere tags will show up as options to select from in your node template.
In the custom attributes, Rancher will let you select all the custom attributes you have already set up in vSphere. The custom attributes are keys and you can enter values for each one.
:::note
Custom attributes are a legacy feature that will eventually be removed from vSphere.
:::
## cloud-init
[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) allows you to initialize your nodes by applying configuration on the first boot. This may involve things such as creating users, authorizing SSH keys or setting up the network.
To make use of cloud-init initialization, create a cloud config file using valid YAML syntax and paste the file content in the the **Cloud Init** field. Refer to the [cloud-init documentation.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html) for a commented set of examples of supported cloud config directives.
Note that cloud-init is not supported when using the ISO creation method.
## Engine Options
In the **Engine Options** section of the node template, you can configure the container daemon. You may want to specify the container version or a container image registry mirror.
:::note
If you're provisioning Red Hat Enterprise Linux (RHEL) or CentOS nodes, leave the **Docker Install URL** field as the default value, or select **none**. This will bypass a check for Docker installation, as Docker is already installed on these node types.
If you set **Docker Install URL** to a value other than the default or **none**, you might see an error message such as the following: `Error creating machine: RHEL ssh command error: command: sudo -E yum install -y curl err: exit status 1 output: Updating Subscription Management repositories.`
:::
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/aks-cluster-configuration.md
+221
View File
@@ -0,0 +1,221 @@
---
title: AKS Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/aks-cluster-configuration"/>
</head>
## Role-based Access Control
When provisioning an AKS cluster in the Rancher UI, RBAC cannot be disabled. If RBAC is disabled in the AKS cluster, the cluster cannot be registered or imported into Rancher. In practice, this means that local accounts must be enabled in order to register or import an AKS cluster.
Rancher can configure member roles for AKS clusters in the same way as any other cluster. For more information, see the section on [role-based access control.](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md)
## Cloud Credentials
:::note
The configuration information in this section assumes you have already set up a service principal for Rancher. For step-by-step instructions for how to set up the service principal, see [this section.](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/aks.md#prerequisites-in-microsoft-azure)
:::
### Subscription ID
To get the subscription ID, click **All Services** in the left navigation bar. Then click **Subscriptions**. Go to the name of the subscription that you want to associate with your Kubernetes cluster and copy the **Subscription ID**.
### Client ID
To get the client ID, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal. The client ID is listed on the app registration detail page as **Application (client) ID**.
### Client Secret
You can't retrieve the client secret value after it is created, so if you don't already have a client secret value, you will need to create a new client secret.
To get a new client secret, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal.
Then click **Certificates & secrets** and click **New client secret**. Click **Add**. Then copy the **Value** of the new client secret.
### Environment
Microsoft provides multiple [clouds](https://docs.microsoft.com/en-us/cli/azure/cloud?view=azure-cli-latest) for compliance with regional laws, which are available for your use:
- AzurePublicCloud
- AzureGermanCloud
- AzureChinaCloud
- AzureUSGovernmentCloud
## Account Access
In this section you will need to select an existing Azure cloud credential or create a new one.
For help configuring your Azure cloud credential, see [this section.](#cloud-credentials)
## Cluster Location
Configure the cluster and node location. For more information on availability zones for AKS, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/availability-zones)
The high availability locations include multiple availability zones.
## Cluster Options
### Kubernetes Version
The available Kubernetes versions are dynamically fetched from the Azure API.
### Cluster Resource Group
A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.
Use an existing resource group or enter a resource group name and one will be created for you.
Using a resource group containing an existing AKS cluster will create a new resource group. Azure AKS only allows one AKS cluster per resource group.
For information on managing resource groups, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal)
### Linux Admin Username
The username used to create an SSH connection to the Linux nodes.
The default username for AKS nodes is `azureuser`.
### SSH Public Key
The key used to create an SSH connection to the Linux nodes.
### Tags
Cluster tags can be useful if your organization uses tags as a way to organize resources across multiple Azure services. These tags don't apply to resources within the cluster.
## Networking Options
### LoadBalancer SKU
Azure load balancers support both standard and basic SKUs (stock keeping units).
For a comparison of standard and basic load balancers, see the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/load-balancer/skus#skus) Microsoft recommends the Standard load balancer.
The Standard load balancer is required if you have selected one or more availability zones, or if you have more than one node pool.
### Network Policy
All pods in an AKS cluster can send and receive traffic without limitations, by default. To improve security, you can define rules that control the flow of traffic. The Network Policy feature in Kubernetes lets you define rules for ingress and egress traffic between pods in a cluster.
Azure provides two ways to implement network policy. You choose a network policy option when you create an AKS cluster. The policy option can't be changed after the cluster is created:
- Azure's own implementation, called Azure Network Policies. The Azure network policy requires the Azure CNI.
- Calico Network Policies, an open-source network and network security solution founded by [Tigera](https://www.tigera.io/).
You can also choose to have no network policy.
For more information about the differences between Azure and Calico network policies and their capabilities, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/use-network-policies#differences-between-azure-and-calico-policies-and-their-capabilities)
### DNS Prefix
Enter a unique DNS prefix for your cluster's Kubernetes API server FQDN.
### Network Plugin
There are two network plugins: kubenet and Azure CNI.
The [kubenet](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#kubenet) Kubernetes plugin is the default configuration for AKS cluster creation. When kubenet is used, each node in the cluster receives a routable IP address. The pods use NAT to communicate with other resources outside the AKS cluster. This approach reduces the number of IP addresses you need to reserve in your network space for pods to use.
With the Azure CNI (advanced) networking plugin, pods get full virtual network connectivity and can be directly reached via their private IP address from connected networks. This plugin requires more IP address space.
For more information on the differences between kubenet and Azure CNI, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/concepts-network#compare-network-models)
### HTTP Application Routing
When enabled, the HTTP application routing add-on makes it easier to access applications deployed to the AKS cluster. It deploys two components: a [Kubernetes Ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress/) and an [External-DNS](https://github.com/kubernetes-incubator/external-dns) controller.
For more information, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/http-application-routing)
### Set Authorized IP Ranges
You can secure access to the Kubernetes API server using [authorized IP address ranges.](https://docs.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges#overview-of-api-server-authorized-ip-ranges)
The Kubernetes API server exposes the Kubernetes API. This component provides the interaction for management tools, such as kubectl. AKS provides a single-tenant cluster control plane with a dedicated API server. By default, the API server is assigned a public IP address, and you should control access to it using Kubernetes-based or Azure-based RBAC.
To secure access to the otherwise publicly accessible AKS control plane and API server, you can enable and use authorized IP ranges. These authorized IP ranges only allow defined IP address ranges to communicate with the API server.
However, even if you use authorized IP address ranges, you should still use Kubernetes RBAC or Azure RBAC to authorize users and the actions they request.
### Container Monitoring
Container monitoring gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring, metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux. Metrics are written to the metrics store and log data is written to the logs store associated with your [Log Analytics](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/log-query-overview) workspace.
### Log Analytics Workspace Resource Group
The [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview#resource-groups) containing the Log Analytics Workspace. You must create at least one workspace to use Azure Monitor Logs.
### Log Analytics Workspace Name
Data collected by Azure Monitor Logs is stored in one or more [Log Analytics workspaces.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/design-logs-deployment) The workspace defines the geographic location of the data, access rights defining which users can access data, and configuration settings such as the pricing tier and data retention.
You must create at least one workspace to use Azure Monitor Logs. A single workspace may be suffxicient for all of your monitoring data, or may choose to create multiple workspaces depending on your requirements. For example, you might have one workspace for your production data and another for testing.
For more information about Azure Monitor Logs, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs)
### Support Private Kubernetes Service
Typically, AKS worker nodes do not get public IPs, regardless of whether the cluster is private. In a private cluster, the control plane does not have a public endpoint.
Rancher can connect to a private AKS cluster in one of two ways.
The first way to ensure that Rancher is running on the same [NAT](https://docs.microsoft.com/en-us/azure/virtual-network/nat-overview) as the AKS nodes.
The second way is to run a command to register the cluster with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster’s Kubernetes API. This command is displayed in a pop-up when you provision an AKS cluster with a private API endpoint enabled.
:::note
Please be aware that when registering an existing AKS cluster, the cluster might take some time, possibly hours, to appear in the `Cluster To register` dropdown list. This outcome will be based on region.
:::
For more information about connecting to an AKS private cluster, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/private-clusters#options-for-connecting-to-the-private-cluster)
## Node Pools
### Mode
The Azure interface allows users to specify whether a Primary Node Pool relies on either `system` (normally used for control planes) or `user` (what is most typically needed for Rancher).
For Primary Node Pools, you can specify Mode, OS, Count and Size.
System node pools always require running nodes, so they cannot be scaled below one node. At least one system node pool is required.
For subsequent node pools, the Rancher UI forces the default of User. User node pools allow you to scale to zero nodes. User node pools don't run any part of the Kubernetes controlplane.
AKS doesn't expose the nodes that run the Kubernetes controlplane components.
### Availability Zones
[Availability zones](https://docs.microsoft.com/en-us/azure/availability-zones/az-overview) are unique physical locations within a region. Each zone is made up of one or more data centers equipped with independent power, cooling, and networking.
Not all regions have support for availability zones. For a list of Azure regions with availability zones, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/availability-zones/az-region#azure-regions-with-availability-zones)
### VM Size
Choose a size for each VM in the node pool. For details about each VM size, see [this page.](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/)
### OS Disk Type
The nodes in the node pool can have either managed or ephemeral disks.
[Ephemeral OS disks](https://docs.microsoft.com/en-us/azure/virtual-machines/ephemeral-os-disks) are created on the local virtual machine storage and not saved to the remote Azure Storage. Ephemeral OS disks work well for stateless workloads, where applications are tolerant of individual VM failures, but are more affected by VM deployment time or reimaging the individual VM instances. With Ephemeral OS disk, you get lower read/write latency to the OS disk and faster VM reimage.
[Azure managed disks](https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview) are block-level storage volumes that are managed by Azure and used with Azure Virtual Machines. Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability.
### OS Disk Size
The size in GB for the disk for each node.
### Node Count
The number of nodes in the node pool. The maximum number of nodes may be limited by your [Azure subscription.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits)
### Max Pods Per Node
The maximum number of pods per node defaults to 110 with a maximum of 250.
### Enable Auto Scaling
When auto scaling is enabled, you will need to enter a minimum and maximum node count.
When Auto Scaling is enabled, you can't manually scale the node pool. The scale is controlled by the AKS autoscaler.
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/eks-cluster-configuration.md
+180
View File
@@ -0,0 +1,180 @@
---
title: EKS Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/eks-cluster-configuration"/>
</head>
### Account Access
Complete each drop-down and field using the information obtained for your IAM policy.
| Setting | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
| Region | From the drop-down choose the geographical region in which to build your cluster. |
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.](../../user-settings/manage-cloud-credentials.md) |
### Service Role
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
Service Role | Description
-------------|---------------------------
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
### Secrets Encryption
Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html)
### API Server Endpoint Access
Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Private-only API Endpoints
If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API.
There are two ways to avoid this extra manual step:
- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster.
- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html).
### Public Access Endpoints
Optionally limit access to the public endpoint via explicit CIDR blocks.
If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster.
One of the following is required to enable private access:
- Rancher's IP must be part of an allowed CIDR block
- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group
For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Subnet
| Option | Description |
| ------- | ------------ |
| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. |
| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). |
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
### Security Group
Amazon Documentation:
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
### Logging
Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters.
Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation.
For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
### Managed Node Groups
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)
#### User-provided Launch Templates
You can provide your own launch template ID and version to configure the EC2 instances in a node group. If you provide the launch template, none of the template settings will be configurable from Rancher. You must set all of the required options listed below in your launch template.
Also, if you provide the launch template, you can only update the template version, not the template ID. To use a new template ID, create a new managed node group.
| Option | Description | Required/Optional |
| ------ | ----------- | ----------------- |
| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required |
| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional |
| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required |
| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional |
| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional |
| Instance Resource Tags | Tag each EC2 instance and its volumes in the node group | Optional |
:::caution
You can't directly update a node group to a newer Kubernetes version if the node group was created from a custom launch template. You must create a new launch template with the proper Kubernetes version, and associate the node group with the new template.
:::
#### Rancher-managed Launch Templates
If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated.
#### Custom AMIs
If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative.
If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it.
:::note
The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template.
:::
#### Spot instances
Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types.
:::note
Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration.
:::
#### Node Group Settings
The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created.
| Option | Description |
| ------- | ------------ |
| Node Group Name | The name of the node group. |
| Desired ASG Size | The desired number of instances. |
| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
| Labels | Kubernetes labels applied to the nodes in the managed node group. |
| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. |
### Self-managed Amazon Linux Nodes
You can register an EKS cluster containing self-managed Amazon Linux nodes. You must configure this type of cluster according to the instructions in the official AWS documentation for [launching self-managed Amazon Linux nodes](https://docs.aws.amazon.com/eks/latest/userguide/launch-workers.html). EKS clusters containing self-managed Amazon Linux nodes are usually operated by the [Karpenter](https://karpenter.sh/docs/) project. After you provision an EKS cluster containing self-managed Amazon Linux nodes, [register the cluster](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md) so it can be managed by Rancher. However, the nodes won't be visible in the Rancher UI.
### IAM Roles for Service Accounts
An Applications Deployment running on an EKS cluster can make requests to AWS services via IAM permissions. These applications must sign their requests with AWS credentials. IAM roles for service accounts manage these credentials using an AWS OIDC endpoint. Rather than distributing AWS credentials to containers or relying on an EC2 instance's role, you can link an [IAM role to a Kubernetes service account](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) and configure your Pods to use this account.
:::note
Linking to an IAM role is not supported for Rancher pods in an EKS cluster.
:::
To enable IAM roles for service accounts:
1. [Create an IAM OIDC provider for your cluster](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html)
1. [Configure a Kubernetes service account to assume an IAM role](https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html)
1. [Configure Pods to use a Kubernetes service account](https://docs.aws.amazon.com/eks/latest/userguide/pod-configuration.html)
1. [Use a supported AWS SDK](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html)
### Configuring the Refresh Interval
The `eks-refresh-cron` setting is deprecated. It has been migrated to the `eks-refresh` setting, which is an integer representing seconds.
The default value is 300 seconds.
The syncing interval can be changed by running `kubectl edit setting eks-refresh`.
If the `eks-refresh-cron` setting was previously set, the migration will happen automatically.
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/gke-cluster-configuration/gke-cluster-configuration.md
+327
View File
@@ -0,0 +1,327 @@
---
title: GKE Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/gke-cluster-configuration"/>
</head>
## Cluster Location
| Value | Description |
|--------|--------------|
| Location Type | Zonal or Regional. With GKE, you can create a cluster tailored to the availability requirements of your workload and your budget. By default, a cluster's nodes run in a single compute zone. When multiple zones are selected, the cluster's nodes will span multiple compute zones, while the controlplane is located in a single zone. Regional clusters increase the availability of the controlplane as well. For help choosing the type of cluster availability, refer to [these docs.](https://cloud.google.com/kubernetes-engine/docs/best-practices/scalability#choosing_a_regional_or_zonal_control_plane) |
| Zone | Each region in Compute engine contains a number of zones. For more information about available regions and zones, refer to [these docs.](https://cloud.google.com/compute/docs/regions-zones#available) |
| Additional Zones | For zonal clusters, you can select additional zones to create a [multi-zone cluster.](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#multi-zonal_clusters) |
| Region | For [regional clusters,](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#regional_clusters) you can select a region. For more information about available regions and zones, refer to [this section](https://cloud.google.com/compute/docs/regions-zones#available). The first part of each zone name is the name of the region. |
## Cluster Options
### Kubernetes Version
_Mutable: yes_
For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)
### Container Address Range
_Mutable: no_
The IP address range for pods in the cluster. Must be a valid CIDR range, e.g. 10.42.0.0/16. If not specified, a random range is automatically chosen from 10.0.0.0/8 and will exclude ranges already allocated to VMs, other clusters, or routes. Automatically chosen ranges may conflict with reserved IP addresses, dynamic routes, or routes within VPCs peering with the cluster.
### Network
_Mutable: no_
The Compute Engine Network that the cluster connects to. Routes and firewalls will be created using this network. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC networks that are shared to your project will appear here. will be available to select in this field. For more information, refer to [this page](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets).
### Node Subnet / Subnet
_Mutable: no_
The Compute Engine subnetwork that the cluster connects to. This subnetwork must belong to the network specified in the **Network** field. Select an existing subnetwork, or select "Auto Create Subnetwork" to have one automatically created. If not using an existing network, **Subnetwork Name** is required to generate one. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC subnets that are shared to your project will appear here. If using a Shared VPC network, you cannot select "Auto Create Subnetwork". For more information, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
### Subnetwork Name
_Mutable: no_
Automatically create a subnetwork with the provided name. Required if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. For more information on subnetworks, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
### Ip Aliases
_Mutable: no_
Enable [alias IPs](https://cloud.google.com/vpc/docs/alias-ip). This enables VPC-native traffic routing. Required if using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc).
### Network Policy
_Mutable: yes_
Enable network policy enforcement on the cluster. A network policy defines the level of communication that can occur between pods and services in the cluster. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/network-policy)
### Project Network Isolation
_Mutable: yes_
Choose whether to enable or disable inter-project communication.
#### Imported Clusters
For imported clusters, Project Network Isolation (PNI) requires Kubernetes Network Policy to be enabled on the cluster beforehand.
For clusters created by Rancher, Rancher enables Kubernetes Network Policy automatically.
1. In GKE, enable Network Policy at the cluster level. Refer to the [official GKE guide](https://cloud.google.com/kubernetes-engine/docs/how-to/network-policy) for instructions.
1. After enabling Network Policy, import the cluster into Rancher and enable PNI for project-level isolation.
### Node Ipv4 CIDR Block
_Mutable: no_
The IP address range of the instance IPs in this cluster. Can be set if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. Must be a valid CIDR range, e.g. 10.96.0.0/14. For more information on how to determine the IP address range, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing)
### Cluster Secondary Range Name
_Mutable: no_
The name of an existing secondary range for Pod IP addresses. If selected, **Cluster Pod Address Range** will automatically be populated. Required if using a Shared VPC network.
### Cluster Pod Address Range
_Mutable: no_
The IP address range assigned to pods in the cluster. Must be a valid CIDR range, e.g. 10.96.0.0/11. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your pods, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_pods)
### Services Secondary Range Name
_Mutable: no_
The name of an existing secondary range for service IP addresses. If selected, **Service Address Range** will be automatically populated. Required if using a Shared VPC network.
### Service Address Range
_Mutable: no_
The address range assigned to the services in the cluster. Must be a valid CIDR range, e.g. 10.94.0.0/18. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your services, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_svcs)
### Private Cluster
_Mutable: no_
:::caution
Private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide](gke-private-clusters.md).
:::
Assign nodes only internal IP addresses. Private cluster nodes cannot access the public internet unless additional networking steps are taken in GCP.
### Enable Private Endpoint
:::caution
Private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide](gke-private-clusters.md).
:::
_Mutable: no_
Locks down external access to the control plane endpoint. Only available if **Private Cluster** is also selected. If selected, and if Rancher does not have direct access to the Virtual Private Cloud network the cluster is running in, Rancher will provide a registration command to run on the cluster to enable Rancher to connect to it.
### Master IPV4 CIDR Block
_Mutable: no_
The IP range for the control plane VPC.
### Master Authorized Network
_Mutable: yes_
Enable control plane authorized networks to block untrusted non-GCP source IPs from accessing the Kubernetes master through HTTPS. If selected, additional authorized networks may be added. If the cluster is created with a public endpoint, this option is useful for locking down access to the public endpoint to only certain networks, such as the network where your Rancher service is running. If the cluster only has a private endpoint, this setting is required.
## Additional Options
### Cluster Addons
Additional Kubernetes cluster components. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters#Cluster.AddonsConfig)
#### Horizontal Pod Autoscaling
_Mutable: yes_
The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler)
#### HTTP (L7) Load Balancing
_Mutable: yes_
HTTP (L7) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on GKE. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer)
#### Network Policy Config (master only)
_Mutable: yes_
Configuration for NetworkPolicy. This only tracks whether the addon is enabled or not on the master, it does not track whether network policy is enabled for the nodes.
### Cluster Features (Alpha Features)
_Mutable: no_
Turns on all Kubernetes alpha API groups and features for the cluster. When enabled, the cluster cannot be upgraded and will be deleted automatically after 30 days. Alpha clusters are not recommended for production use as they are not covered by the GKE SLA. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters)
### Logging Service
_Mutable: yes_
The logging service the cluster uses to write logs. Use either [Cloud Logging](https://cloud.google.com/logging) or no logging service in which case no logs are exported from the cluster.
### Monitoring Service
_Mutable: yes_
The monitoring service the cluster uses to write metrics. Use either [Cloud Monitoring](https://cloud.google.com/monitoring) or monitoring service in which case no metrics are exported from the cluster.
### Maintenance Window
_Mutable: yes_
Set the start time for a 4 hour maintenance window. The time is specified in the UTC time zone using the HH:MM format. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions)
## Node Pools
In this section, enter details describing the configuration of each node in the node pool.
### Kubernetes Version
_Mutable: yes_
The Kubernetes version for each node in the node pool. For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)
### Image Type
_Mutable: yes_
The node operating system image. For more information for the node image options that GKE offers for each OS, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#available_node_images)
:::note
The default option is "Container-Optimized OS with Docker". The read-only filesystem on GCP's Container-Optimized OS is not compatible with the [legacy logging](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/explanations/integrations-in-rancher/cluster-logging/cluster-logging.md) implementation in Rancher. If you need to use the legacy logging feature, select "Ubuntu with Docker" or "Ubuntu with Containerd". The [current logging feature](../../../../integrations-in-rancher/logging/logging.md) is compatible with the Container-Optimized OS image.
:::
:::note
If selecting "Windows Long Term Service Channel" or "Windows Semi-Annual Channel" for the node pool image type, you must also add at least one Container-Optimized OS or Ubuntu node pool.
:::
### Machine Type
_Mutable: no_
The virtualized hardware resources available to node instances. For more information on Google Cloud machine types, refer to [this page.](https://cloud.google.com/compute/docs/machine-types#machine_types)
### Root Disk Type
_Mutable: no_
Standard persistent disks are backed by standard hard disk drives (HDD), while SSD persistent disks are backed by solid state drives (SSD). For more information, refer to [this section.](https://cloud.google.com/compute/docs/disks)
### Local SSD Disks
_Mutable: no_
Configure each node's local SSD disk storage in GB. Local SSDs are physically attached to the server that hosts your VM instance. Local SSDs have higher throughput and lower latency than standard persistent disks or SSD persistent disks. The data that you store on a local SSD persists only until the instance is stopped or deleted. For more information, see [this section.](https://cloud.google.com/compute/docs/disks#localssds)
### Preemptible nodes (beta)
_Mutable: no_
Preemptible nodes, also called preemptible VMs, are Compute Engine VM instances that last a maximum of 24 hours in general, and provide no availability guarantees. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms)
### Taints
_Mutable: no_
When you apply a taint to a node, only Pods that tolerate the taint are allowed to run on the node. In a GKE cluster, you can apply a taint to a node pool, which applies the taint to all nodes in the pool.
### Node Labels
_Mutable: no_
You can apply labels to the node pool, which applies the labels to all nodes in the pool.
Invalid labels can prevent upgrades or can prevent Rancher from starting. For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)
### Network Tags
_Mutable: no_
You can add network tags to the node pool to make firewall rules and routes between subnets. Tags will apply to all nodes in the pool.
For details on tag syntax and requirements, see the [Kubernetes documentation](https://cloud.google.com/vpc/docs/add-remove-network-tags).
## Group Details
In this section, enter details describing the node pool.
### Name
_Mutable: no_
Enter a name for the node pool.
### Initial Node Count
_Mutable: yes_
Integer for the starting number of nodes in the node pool.
### Max Pod Per Node
_Mutable: no_
GKE has a hard limit of 110 Pods per node. For more information on the Kubernetes limits, see [this section.](https://cloud.google.com/kubernetes-engine/docs/best-practices/scalability#dimension_limits)
### Autoscaling
_Mutable: yes_
Node pool autoscaling dynamically creates or deletes nodes based on the demands of your workload. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler)
### Auto Repair
_Mutable: yes_
GKE's node auto-repair feature helps you keep the nodes in your cluster in a healthy, running state. When enabled, GKE makes periodic checks on the health state of each node in your cluster. If a node fails consecutive health checks over an extended time period, GKE initiates a repair process for that node. For more information, see the section on [auto-repairing nodes.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-repair)
### Auto Upgrade
_Mutable: yes_
When enabled, the auto-upgrade feature keeps the nodes in your cluster up-to-date with the cluster control plane (master) version when your control plane is [updated on your behalf.](https://cloud.google.com/kubernetes-engine/upgrades#automatic_cp_upgrades) For more information about auto-upgrading nodes, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-upgrades)
### Access Scopes
_Mutable: no_
Access scopes are the legacy method of specifying permissions for your nodes.
- **Allow default access:** The default access for new clusters is the [Compute Engine default service account.](https://cloud.google.com/compute/docs/access/service-accounts?hl=en_US#default_service_account)
- **Allow full access to all Cloud APIs:** Generally, you can just set the cloud-platform access scope to allow full access to all Cloud APIs, then grant the service account only relevant IAM roles. The combination of access scopes granted to the virtual machine instance and the IAM roles granted to the service account determines the amount of access the service account has for that instance.
- **Set access for each API:** Alternatively, you can choose to set specific scopes that permit access to the particular API methods that the service will call.
For more information, see the [section about enabling service accounts for a VM.](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances)
### Configuring the Refresh Interval
The refresh interval can be configured through the setting "gke-refresh", which is an integer representing seconds.
The default value is 300 seconds.
The syncing interval can be changed by running `kubectl edit setting gke-refresh`.
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for GCP APIs.
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/gke-cluster-configuration/gke-private-clusters.md
+54
View File
@@ -0,0 +1,54 @@
---
title: Private Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/gke-cluster-configuration/gke-private-clusters"/>
</head>
In GKE, [private clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept) are clusters whose nodes are isolated from inbound and outbound traffic by assigning them internal IP addresses only. Private clusters in GKE have the option of exposing the control plane endpoint as a publicly accessible address or as a private address. This is different from other Kubernetes providers, which may refer to clusters with private control plane endpoints as "private clusters" but still allow traffic to and from nodes. You may want to create a cluster with private nodes, with or without a public control plane endpoint, depending on your organization's networking and security requirements. A GKE cluster provisioned from Rancher can use isolated nodes by selecting "Private Cluster" in the Cluster Options (under "Show advanced options"). The control plane endpoint can optionally be made private by selecting "Enable Private Endpoint".
## Private Nodes
Because the nodes in a private cluster only have internal IP addresses, they will not be able to install the cluster agent and Rancher will not be able to fully manage the cluster. This can be overcome in a few ways.
### Cloud NAT
:::caution
Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
:::
If restricting outgoing internet access is not a concern for your organization, use Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service to allow nodes in the private network to access the internet, enabling them to download the required images from Docker Hub and contact the Rancher management server. This is the simplest solution.
### Private Registry
:::caution
This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
:::
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](../../../../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
## Private Control Plane Endpoint
If the cluster has a public endpoint exposed, Rancher will be able to reach the cluster, and no additional steps need to be taken. However, if the cluster has no public endpoint, then considerations must be made to ensure Rancher can access the cluster.
### Cloud NAT
:::caution
Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
:::
As above, if restricting outgoing internet access to the nodes is not a concern, then Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service can be used to allow the nodes to access the internet. While the cluster is provisioning, Rancher will provide a registration command to run on the cluster. Download the [kubeconfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) for the new cluster and run the provided kubectl command on the cluster. Gaining access
to the cluster in order to run this command can be done by creating a temporary node or using an existing node in the VPC, or by logging on to or creating an SSH tunnel through one of the cluster nodes.
### Direct Access
If the Rancher server is run on the same VPC as the cluster's control plane, it will have direct access to the control plane's private endpoint. The cluster nodes will need to have access to a [private registry](#private-registry) to download images as described above.
You can also use services from Google such as [Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or [Cloud Interconnect VLAN](https://cloud.google.com/network-connectivity/docs/interconnect) to facilitate connectivity between your organization's network and your Google VPC.
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/k3s-cluster-configuration.md
+551
View File
@@ -0,0 +1,551 @@
---
title: K3s Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/k3s-cluster-configuration"/>
</head>
This section covers the configuration options that are available in Rancher for a new or existing K3s Kubernetes cluster.
## Overview
You can configure the Kubernetes options one of two ways:
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
- [Cluster Config File](#cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create a K3s config file. Using a config file lets you set any of the [options](https://rancher.com/docs/k3s/latest/en/installation/install-options/) available during a K3s installation.
## Editing Clusters in the Rancher UI
The Rancher UI provides two ways to edit a cluster:
1. With a form.
1. With YAML.
### Editing Clusters with a Form
The form covers the most frequently needed options for clusters.
To edit your cluster,
1. Click **☰ > Cluster Management**.
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
### Editing Clusters in YAML
For a complete reference of configurable options for K3s clusters in YAML, see the [K3s documentation](https://docs.k3s.io/installation/configuration).
To edit your cluster with YAML:
1. Click **☰ > Cluster Management**.
1. Go to the cluster you want to configure and click **⋮ > Edit as YAML**.
1. Edit the RKE options under the `rkeConfig` directive.
## Configuration Options in the Rancher UI
### Machine Pool Configuration
This subsection covers generic machine pool configurations. For specific infrastructure provider configurations, refer to the following:
- [Azure](../downstream-cluster-configuration/machine-configuration/azure.md)
- [DigitalOcean](../downstream-cluster-configuration/machine-configuration/digitalocean.md)
- [Amazon EC2](../downstream-cluster-configuration/machine-configuration/amazon-ec2.md)
- [Google GCE](../downstream-cluster-configuration/machine-configuration/google-gce.md)
##### Pool Name
The name of the machine pool.
##### Machine Count
The number of machines in the pool.
##### Roles
Option to assign etcd, control plane, and worker roles to nodes.
#### Advanced
##### Auto Replace
The amount of time nodes can be unreachable before they are automatically deleted and replaced.
##### Drain Before Delete
Enables draining nodes by evicting all pods before the node is deleted.
##### Kubernetes Node Labels
Add [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to nodes to help with organization and object selection.
For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)
##### Taints
Add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to nodes, to prevent pods from being scheduled to or executed on the nodes, unless the pods have matching tolerations.
### Cluster Configuration
#### Basics
##### Kubernetes Version
The version of Kubernetes installed on your cluster nodes.
For details on upgrading or rolling back Kubernetes, refer to [this guide](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
##### Pod Security Admission Configuration Template
The default [pod security admission configuration template](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for the cluster.
##### Encrypt Secrets
Option to enable or disable secrets encryption. When enabled, secrets will be encrypted using a AES-CBC key. If disabled, any previously secrets will not be readable until encryption is enabled again. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/advanced/#secrets-encryption-config-experimental) for details.
##### Project Network Isolation
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
##### SELinux
Option to enable or disable [SELinux](https://rancher.com/docs/k3s/latest/en/advanced/#selinux-support) support.
##### CoreDNS
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#coredns) for details.
##### Klipper Service LB
Option to enable or disable the [Klipper](https://github.com/rancher/klipper-lb) service load balancer. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#service-load-balancer) for details.
##### Traefik Ingress
Option to enable or disable the [Traefik](https://traefik.io/) HTTP reverse proxy and load balancer. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#traefik-ingress-controller).
##### Local Storage
Option to enable or disable [local storage](https://rancher.com/docs/k3s/latest/en/storage/) on the node(s).
##### Metrics Server
Option to enable or disable the [metrics server](https://github.com/kubernetes-incubator/metrics-server). If enabled, ensure port 10250 is opened for inbound TCP traffic.
#### Add-On Config
Additional Kubernetes manifests, managed as a [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/helm/#automatically-deploying-manifests-and-helm-charts) for details.
#### Agent Environment Vars
Option to set environment variables for [K3s agents](https://rancher.com/docs/k3s/latest/en/architecture/). The environment variables can be set using key value pairs. Refer to the [K3 documentation](https://rancher.com/docs/k3s/latest/en/installation/install-options/agent-config/) for more details.
#### etcd
##### Automatic Snapshots
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [K3s documentation](https://docs.k3s.io/cli/etcd-snapshot#creating-snapshots).
##### Metrics
Option to choose whether to expose etcd metrics to the public or only within the cluster.
#### Networking
##### Cluster CIDR
IPv4/IPv6 network CIDRs to use for pod IPs (default: `10.42.0.0/16`).
Example values:
- IPv4-only: `10.42.0.0/16`
- IPv6-only: `2001:cafe:42::/56`
- Dual-stack: `10.42.0.0/16,2001:cafe:42::/56`
For additional requirements and limitations related to dual-stack or IPv6-only networking, see the following resources:
- [K3s documentation: Dual-stack (IPv4 + IPv6) Networking](https://docs.k3s.io/networking/basic-network-options#dual-stack-ipv4--ipv6-networking)
- [K3s documentation: Single-stack IPv6 Networking](https://docs.k3s.io/networking/basic-network-options#single-stack-ipv6-networking)
:::caution
You must configure the Service CIDR when you first create the cluster. You cannot enable the Service CIDR on an existing cluster after it starts.
:::
##### Service CIDR
IPv4/IPv6 network CIDRs to use for service IPs (default: `10.43.0.0/16`).
Example values:
- IPv4-only: `10.43.0.0/16`
- IPv6-only: `2001:cafe:43::/112`
- Dual-stack: `10.43.0.0/16,2001:cafe:43::/112`
For additional requirements and limitations related to dual-stack or IPv6-only networking, see the following resources:
- [K3s documentation: Dual-stack (IPv4 + IPv6) Networking](https://docs.k3s.io/networking/basic-network-options#dual-stack-ipv4--ipv6-networking)
- [K3s documentation: Single-stack IPv6 Networking](https://docs.k3s.io/networking/basic-network-options#single-stack-ipv6-networking)
:::caution
You must configure the Service CIDR when you first create the cluster. You cannot enable the Service CIDR on an existing cluster after it starts.
:::
##### Cluster DNS
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: `10.43.0.10`).
##### Cluster Domain
Select the domain for the cluster. The default is `cluster.local`.
##### NodePort Service Port Range
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
##### Truncate Hostnames
Option to truncate hostnames to 15 characters or fewer. You can only set this field during the initial creation of the cluster. You can't enable or disable the 15-character limit after cluster creation.
This setting only affects machine-provisioned clusters. Since custom clusters set hostnames during their own node creation process, which occurs outside of Rancher, this field doesn't restrict custom cluster hostname length.
Truncating hostnames in a cluster improves compatibility with Windows-based systems. Although Kubernetes allows hostnames up to 63 characters in length, systems that use NetBIOS restrict hostnames to 15 characters or fewer.
##### TLS Alternate Names
Add hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
##### Authorized Cluster Endpoint
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
##### Stack Preference
Choose the networking stack for the cluster. This option affects:
- The address used for health and readiness probes of components such as Calico, etcd, kube-apiserver, kube-scheduler, kube-controller-manager, and kubelet.
- The server URL in the `authentication-token-webhook-config-file` for the Authorized Cluster Endpoint.
- The `advertise-client-urls` setting for etcd during snapshot restoration.
Options are `ipv4`, `ipv6`, `dual`:
- When set to `ipv4`, the cluster uses `127.0.0.1`
- When set to `ipv6`, the cluster uses `[::1]`
- When set to `dual`, the cluster uses `localhost`
The stack preference must match the cluster’s networking configuration:
- Set to `ipv4` for IPv4-only clusters
- Set to `ipv6` for IPv6-only clusters
- Set to `dual` for dual-stack clusters
:::caution
Ensuring the loopback address configuration is correct is critical for successful cluster provisioning.
For more information, refer to the [Node Requirements](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md) page.
:::
#### Registries
Select the image repository to pull Rancher images from. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/installation/private-registry/).
#### Upgrade Strategy
##### Control Plane Concurrency
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
##### Worker Concurrency
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
##### Drain Nodes (Control Plane)
Option to remove all pods from the node prior to upgrading.
##### Drain Nodes (Worker Nodes)
Option to remove all pods from the node prior to upgrading.
#### Advanced
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
## Cluster Config File Reference
Editing clusters in YAML allows you to set configurations that are already listed in [Configuration Options in the Rancher UI](#configuration-options-in-the-rancher-ui), as well as set Rancher-specific parameters.
<details>
<summary>
<b>Example Cluster Config File Snippet</b>
</summary>
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
cloudCredentialSecretName: cattle-global-data:cc-fllv6
clusterAgentDeploymentCustomization: {}
fleetAgentDeploymentCustomization: {}
kubernetesVersion: v1.26.7+k3s1
localClusterAuthEndpoint: {}
rkeConfig:
additionalManifest: ""
chartValues: {}
etcd:
snapshotRetention: 5
snapshotScheduleCron: 0 */5 * * *
machineGlobalConfig:
disable-apiserver: false
disable-cloud-controller: false
disable-controller-manager: false
disable-etcd: false
disable-kube-proxy: false
disable-network-policy: false
disable-scheduler: false
etcd-expose-metrics: false
kube-apiserver-arg:
- audit-policy-file=/etc/rancher/k3s/user-audit-policy.yaml
- audit-log-path=/etc/rancher/k3s/user-audit.logs
profile: null
secrets-encryption: false
machinePools:
- controlPlaneRole: true
etcdRole: true
machineConfigRef:
kind: Amazonec2Config
name: nc-test-pool1-pwl5h
name: pool1
quantity: 1
unhealthyNodeTimeout: 0s
workerRole: true
machineSelectorConfig:
- config:
docker: false
protect-kernel-defaults: false
selinux: false
machineSelectorFiles:
- fileSources:
- configMap:
name: ''
secret:
name: audit-policy
items:
- key: audit-policy
path: /etc/rancher/k3s/user-audit-policy.yaml
machineLabelSelector:
matchLabels:
rke.cattle.io/control-plane-role: 'true'
registries: {}
upgradeStrategy:
controlPlaneConcurrency: '1'
controlPlaneDrainOptions:
deleteEmptyDirData: true
disableEviction: false
enabled: false
force: false
gracePeriod: -1
ignoreDaemonSets: true
ignoreErrors: false
postDrainHooks: null
preDrainHooks: null
skipWaitForDeleteTimeoutSeconds: 0
timeout: 120
workerConcurrency: '1'
workerDrainOptions:
deleteEmptyDirData: true
disableEviction: false
enabled: false
force: false
gracePeriod: -1
ignoreDaemonSets: true
ignoreErrors: false
postDrainHooks: null
preDrainHooks: null
skipWaitForDeleteTimeoutSeconds: 0
timeout: 120
```
</details>
### additionalManifest
Specify additional manifests to deliver to the control plane nodes.
The value is a String, and will be placed at the path `/var/lib/rancher/k3s/server/manifests/rancher/addons.yaml` on target nodes.
Example:
```yaml
additionalManifest: |-
apiVersion: v1
kind: Namespace
metadata:
name: name-xxxx
```
:::note
If you want to customize system charts, you should use the `chartValues` field as described below.
Alternatives, such as using a HelmChartConfig to customize the system charts via `additionalManifest`, can cause unexpected behavior, due to having multiple HelmChartConfigs for the same chart.
:::
### chartValues
Specify the values for the system charts installed by K3s.
For more information about how K3s manges packaged components, please refer to [K3s documentation](https://docs.k3s.io/installation/packaged-components).
Example:
```yaml
chartValues:
chart-name:
key: value
```
### machineGlobalConfig
Specify K3s configurations. Any configuration change made here will apply to every node. The configuration options available in the [standalone version of k3s](https://docs.k3s.io/cli/server) can be applied here.
Example:
```yaml
machineGlobalConfig:
etcd-arg:
- key1=value1
- key2=value2
```
To make it easier to put files on nodes beforehand, Rancher expects the following values to be included in the configuration, while K3s expects the values to be entered as file paths:
- private-registry
- flannel-conf
Rancher delivers the files to the path `/var/lib/rancher/k3s/etc/config-files/<option>` in target nodes, and sets the proper options in the K3s server.
Example:
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
rkeConfig:
machineGlobalConfig:
private-registry: |
mirrors:
docker.io:
endpoint:
- "http://mycustomreg.com:5000"
configs:
"mycustomreg:5000":
auth:
username: xxxxxx # this is the registry username
password: xxxxxx # this is the registry password
```
### machineSelectorConfig
`machineSelectorConfig` is the same as [`machineGlobalConfig`](#machineglobalconfig) except that a [label](#kubernetes-node-labels) selector can be specified with the configuration. The configuration will only be applied to nodes that match the provided label selector.
Multiple `config` entries are allowed, each specifying their own `machineLabelSelector`. A user can specify `matchExpressions`, `matchLabels`, both, or neither. Omitting the `machineLabelSelector` section of this field has the same effect as putting the config in the `machineGlobalConfig` section.
Example:
```yaml
machineSelectorConfig
- config:
config-key: config-value
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
```
### machineSelectorFiles
:::note
This feature is available in Rancher v2.7.2 and later.
:::
Deliver files to nodes, so that the files can be in place before initiating K3s server or agent processes.
The content of the file is retrieved from either a secret or a configmap. The target nodes are filtered by the `machineLabelSelector`.
Example :
```yaml
machineSelectorFiles:
- fileSources:
- secret:
items:
- key: example-key
path: path-to-put-the-file-on-nodes
permissions: 644 (optional)
hash: base64-encoded-hash-of-the-content (optional)
name: example-secret-name
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
- fileSources:
- configMap:
items:
- key: example-key
path: path-to-put-the-file-on-nodes
permissions: 644 (optional)
hash: base64-encoded-hash-of-the-content (optional)
name: example-configmap-name
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
```
The secret or configmap must meet the following requirements:
1. It must be in the `fleet-default` namespace where the Cluster object exists.
2. It must have the annotation `rke.cattle.io/object-authorized-for-clusters: cluster-name1,cluster-name2`, which permits the target clusters to use it.
:::tip
Rancher Dashboard provides an easy-to-use form for creating the secret or configmap.
:::
Example:
```yaml
apiVersion: v1
data:
audit-policy: >-
IyBMb2cgYWxsIHJlcXVlc3RzIGF0IHRoZSBNZXRhZGF0YSBsZXZlbC4KYXBpVmVyc2lvbjogYXVkaXQuazhzLmlvL3YxCmtpbmQ6IFBvbGljeQpydWxlczoKLSBsZXZlbDogTWV0YWRhdGE=
kind: Secret
metadata:
annotations:
rke.cattle.io/object-authorized-for-clusters: cluster1
name: name1
namespace: fleet-default
```
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/rancher-server-configuration.md
+15
View File
@@ -0,0 +1,15 @@
---
title: Rancher Server Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration"/>
</head>
- [RKE2 Cluster Configuration](rke2-cluster-configuration.md)
- [K3s Cluster Configuration](k3s-cluster-configuration.md)
- [EKS Cluster Configuration](eks-cluster-configuration.md)
- [AKS Cluster Configuration](aks-cluster-configuration.md)
- [GKE Cluster Configuration](gke-cluster-configuration/gke-cluster-configuration.md)
- [Use Existing Nodes](use-existing-nodes/use-existing-nodes.md)
- [Sync Clusters](sync-clusters.md)
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md
+365
View File
@@ -0,0 +1,365 @@
---
title: RKE Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration"/>
</head>
<EOLRKE1Warning />
When Rancher installs Kubernetes, it uses [RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) or [RKE2](https://docs.rke2.io/) as the Kubernetes distribution.
This section covers the configuration options that are available in Rancher for a new or existing RKE Kubernetes cluster.
## Overview
You can configure the Kubernetes options one of two ways:
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
- [Cluster Config File](#rke-cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
The RKE cluster config options are nested under the `rancher_kubernetes_engine_config` directive. For more information, see the section about the [cluster config file.](#rke-cluster-config-file-reference)
In [clusters launched by RKE](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md), you can edit any of the remaining options that follow.
For an example of RKE config file syntax, see the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
The forms in the Rancher UI don't include all advanced options for configuring RKE. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
## Editing Clusters with a Form in the Rancher UI
To edit your cluster,
1. In the upper left corner, click **☰ > Cluster Management**.
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
## Editing Clusters with YAML
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
RKE clusters (also called RKE1 clusters) are edited differently than RKE2 and K3s clusters.
To edit an RKE config file directly from the Rancher UI,
1. Click **☰ > Cluster Management**.
1. Go to the RKE cluster you want to configure. Click and click **⋮ > Edit Config**. This take you to the RKE configuration form. Note: Because cluster provisioning changed in Rancher 2.6, the **⋮ > Edit as YAML** can be used for configuring RKE2 clusters, but it can't be used for editing RKE1 configuration.
1. In the configuration form, scroll down and click **Edit as YAML**.
1. Edit the RKE options under the `rancher_kubernetes_engine_config` directive.
## Configuration Options in the Rancher UI
:::tip
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
:::
### Kubernetes Version
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
### Network Provider
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses. For more details on the different networking providers, please view our [Networking FAQ](../../../faq/container-network-interface-providers.md).
:::caution
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
:::
Out of the box, Rancher is compatible with the following network providers:
- [Canal](https://github.com/projectcalico/canal)
- [Flannel](https://github.com/coreos/flannel#flannel)
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
- [Weave](https://github.com/weaveworks/weave)
<DeprecationWeave />
:::note Notes on Weave:
When Weave is selected as network provider, Rancher will automatically enable encryption by generating a random password. If you want to specify the password manually, please see how to configure your cluster using a [Config File](#rke-cluster-config-file-reference) and the [Weave Network Plug-in Options](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options).
:::
### Project Network Isolation
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
### Kubernetes Cloud Providers
You can configure a [Kubernetes cloud provider](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
:::note
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#rke-cluster-config-file-reference) to configure the cloud provider. Please reference the [RKE cloud provider documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
:::
### Private Registries
The cluster-level private registry configuration is only used for provisioning clusters.
There are two main ways to set up private registries in Rancher: by setting up the [global default registry](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md) through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
If your private registry requires credentials, you need to pass the credentials to Rancher by editing the cluster options for each cluster that needs to pull images from the registry.
The private registry configuration option tells Rancher where to pull the [system images](https://rancher.com/docs/rke/latest/en/config-options/system-images/) or [addon images](https://rancher.com/docs/rke/latest/en/config-options/add-ons/) that will be used in your cluster.
- **System images** are components needed to maintain the Kubernetes cluster.
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
For more information on setting up a private registry for components applied during the provisioning of the cluster, see the [RKE documentation on private registries](https://rancher.com/docs/rke/latest/en/config-options/private-registries/).
Rancher v2.6 introduced the ability to configure [ECR registries for RKE clusters](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup).
### Authorized Cluster Endpoint
Authorized Cluster Endpoint (ACE) can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
:::note
ACE is available on RKE, RKE2, and K3s clusters that are provisioned or registered with Rancher. It's not available on clusters in a hosted Kubernetes provider, such as Amazon's EKS.
:::
ACE must be set up [manually](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters) on RKE2 and K3s clusters. In RKE, ACE is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self-signed certificates.
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
### Node Pools
For information on using the Rancher UI to set up node pools in an RKE cluster, refer to [this page.](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md)
### NGINX Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster.
### Metrics Server Monitoring
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
You must have an existing Pod Security Policy configured before you can use this option.
### Docker Version on Nodes
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support.
If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
For details on which Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
### Docker Root Directory
If the nodes you are adding to the cluster have Docker configured with a non-default Docker Root Directory (default is `/var/lib/docker`), specify the correct Docker Root Directory in this option.
### Default Pod Security Policy
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
### Node Port Range
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). Default is `30000-32767`.
### Recurring etcd Snapshots
Option to enable or disable [recurring etcd snapshots](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
### Agent Environment Variables
Option to set environment variables for [rancher agents](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
### Updating ingress-nginx
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
### Cluster Agent Configuration and Fleet Agent Configuration
You can configure the scheduling fields and resource limits for the Cluster Agent and the cluster's Fleet Agent. You can use these fields to customize tolerations, affinity rules, and resource requirements. Additional tolerations are appended to a list of default tolerations and control plane node taints. If you define custom affinity rules, they override the global default affinity setting. Defining resource requirements sets requests or limits where there previously were none.
:::note
With this option, it's possible to override or remove rules that are required for the functioning of the cluster. We strongly recommend against removing or overriding these and any other affinity rules, as this may cause unwanted side effects:
- `affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution` for `cattle-cluster-agent`
- `cluster-agent-default-affinity` for `cattle-cluster-agent`
- `fleet-agent-default-affinity` for `fleet-agent`
:::
If you downgrade Rancher to v2.7.4 or below, your changes will be lost and the agents will re-deploy without your customizations. The Fleet agent will fallback to using its built-in default values when it re-deploys. If the Fleet version doesn't change during the downgrade, the re-deploy won't be immediate.
## RKE Cluster Config File Reference
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available](https://rancher.com/docs/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API.
For the complete reference for configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
### Config File Structure in Rancher
RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kubernetes clusters. Rancher's cluster config files used to have the same structure as [RKE config files,](https://rancher.com/docs/rke/latest/en/example-yamls/) but the structure changed so that in Rancher, RKE cluster config items are separated from non-RKE config items. Therefore, configuration for your cluster needs to be nested under the `rancher_kubernetes_engine_config` directive in the cluster config file. Cluster config files created with earlier versions of Rancher will need to be updated for this format. An example cluster config file is included below.
<details id="v2.3.0-cluster-config-file">
<summary>Example Cluster Config File</summary>
```yaml
#
# Cluster Config
#
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
enable_cluster_monitoring: false
enable_network_policy: false
local_cluster_auth_endpoint:
enabled: true
#
# Rancher Config
#
rancher_kubernetes_engine_config: # Your RKE template config goes here.
addon_job_timeout: 30
authentication:
strategy: x509
ignore_docker_version: true
#
# # Currently only nginx ingress provider is supported.
# # To disable ingress controller, set `provider: none`
# # To enable ingress on specific nodes, use the node_selector, eg:
# provider: nginx
# node_selector:
# app: ingress
#
ingress:
provider: nginx
kubernetes_version: v1.15.3-rancher3-1
monitoring:
provider: metrics-server
#
# If you are using calico on AWS
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
# # To specify flannel interface
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
# # To specify flannel interface for canal plugin
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
network:
options:
flannel_backend_type: vxlan
plugin: canal
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
# kube-controller:
# cluster_cidr: 10.42.0.0/16
# service_cluster_ip_range: 10.43.0.0/16
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
services:
etcd:
backup_config:
enabled: true
interval_hours: 12
retention: 6
safe_timestamp: false
creation: 12h
extra_args:
election-timeout: 5000
heartbeat-interval: 500
gid: 0
retention: 72h
snapshot: false
uid: 0
kube_api:
always_pull_images: false
pod_security_policy: false
service_node_port_range: 30000-32767
ssh_agent_auth: false
windows_prefered_cluster: false
```
</details>
### Default DNS provider
The table below indicates what DNS provider is deployed by default. See [RKE documentation on DNS provider](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/) for more information how to configure a different DNS provider. CoreDNS can only be used on Kubernetes v1.12.0 and higher.
| Rancher version | Kubernetes version | Default DNS provider |
|-------------|--------------------|----------------------|
| v2.2.5 and higher | v1.14.0 and higher | CoreDNS |
| v2.2.5 and higher | v1.13.x and lower | kube-dns |
| v2.2.4 and lower | any | kube-dns |
## Rancher Specific Parameters in YAML
Besides the RKE config file options, there are also Rancher specific settings that can be configured in the Config File (YAML):
### docker_root_dir
See [Docker Root Directory](#docker-root-directory).
### enable_cluster_monitoring
Option to enable or disable [Cluster Monitoring](../../../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md).
### enable_network_policy
Option to enable or disable Project Network Isolation.
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
### local_cluster_auth_endpoint
See [Authorized Cluster Endpoint](#authorized-cluster-endpoint).
Example:
```yaml
local_cluster_auth_endpoint:
enabled: true
fqdn: "FQDN"
ca_certs: |-
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
```
### Custom Network Plug-in
You can add a custom network plug-in by using the [user-defined add-on functionality](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. You define any add-on that you want deployed after the Kubernetes cluster is deployed.
There are two ways that you can specify an add-on:
- [In-line Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
- [Referencing YAML Files for Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/rke2-cluster-configuration.md
+580
View File
@@ -0,0 +1,580 @@
---
title: RKE2 Cluster Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/rke2-cluster-configuration"/>
</head>
This section covers the configuration options that are available in Rancher for a new or existing RKE2 Kubernetes cluster.
## Overview
You can configure the Kubernetes options in one of the two following ways:
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
- [Cluster Config File](#cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE2 config file. Using a config file allows you to set many additional [options](https://docs.rke2.io/install/configuration) available for an RKE2 installation.
## Editing Clusters in the Rancher UI
The Rancher UI provides two ways to edit a cluster:
1. With a form.
1. With YAML.
### Editing Clusters with a Form
The form covers the most frequently needed options for clusters.
To edit your cluster,
1. Click **☰ > Cluster Management**.
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
### Editing Clusters in YAML
For a complete reference of configurable options for RKE2 clusters in YAML, see the [RKE2 documentation](https://docs.rke2.io/install/configuration).
To edit your cluster in YAML:
1. Click **☰ > Cluster Management**.
1. Go to the cluster you want to configure and click **⋮ > Edit as YAML**.
1. Edit the RKE options under the `rkeConfig` directive.
## Configuration Options in the Rancher UI
### Machine Pool Configuration
This subsection covers generic machine pool configurations. For specific infrastructure provider configurations, refer to the following:
- [Azure](../downstream-cluster-configuration/machine-configuration/azure.md)
- [DigitalOcean](../downstream-cluster-configuration/machine-configuration/digitalocean.md)
- [Amazon EC2](../downstream-cluster-configuration/machine-configuration/amazon-ec2.md)
- [Google GCE](../downstream-cluster-configuration/machine-configuration/google-gce.md)
##### Pool Name
The name of the machine pool.
##### Machine Count
The number of machines in the pool.
##### Roles
Option to assign etcd, control plane, and worker roles to nodes.
#### Advanced
##### Auto Replace
The amount of time nodes can be unreachable before they are automatically deleted and replaced.
##### Drain Before Delete
Enables draining nodes by evicting all pods before the node is deleted.
##### Kubernetes Node Labels
Add [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to nodes to help with organization and object selection.
For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)
##### Taints
Add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to nodes, to prevent pods from being scheduled to or executed on the nodes, unless the pods have matching tolerations.
### Cluster Configuration
#### Basics
##### Kubernetes Version
The version of Kubernetes installed on your cluster nodes.
For details on upgrading or rolling back Kubernetes, refer to [this guide](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
##### Container Network Provider
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses.
:::caution
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you to tear down the entire cluster and all its applications.
:::
Out of the box, Rancher is compatible with the following network providers:
- [Canal](https://github.com/projectcalico/canal)
- [Cilium](https://cilium.io/)*
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
- [Flannel](https://github.com/flannel-io/flannel)
- [Multus](https://github.com/k8snetworkplumbingwg/multus-cni)
\* When using [project network isolation](#project-network-isolation) in the [Cilium CNI](../../../faq/container-network-interface-providers.md#cilium), it is possible to enable cross-node ingress routing. Click the [CNI provider docs](../../../faq/container-network-interface-providers.md#ingress-routing-across-nodes-in-cilium) to learn more.
For more details on the different networking providers and how to configure them, please view our [RKE2 documentation](https://docs.rke2.io/networking/basic_network_options).
:::caution
When using `cilium` or `multus,cilium` as your container network interface provider, ensure the **Enable IPv6 Support** option is also enabled.
:::
##### Cloud Provider
You can configure a [Kubernetes cloud provider](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../how-to-guides/new-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
:::note
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file-reference) to configure the cloud provider. Please reference [this documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
:::
##### Pod Security Admission Configuration Template
The default [pod security admission configuration template](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/psa-config-templates.md) for the cluster.
##### Worker Compliance Profile
Select a [compliance benchmark](../../../how-to-guides/advanced-user-guides/compliance-scan-guides/compliance-scan-guides.md) to validate the system configuration against.
##### Project Network Isolation
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
Project network isolation is available if you are using any RKE2 network plugin that supports the enforcement of Kubernetes network policies, such as Canal.
##### CoreDNS
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#coredns) for additional CoreDNS configurations.
##### NGINX Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
##### Metrics Server
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
Each cloud provider capable of launching a cluster using RKE2 can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
#### Add-On Config
Additional Kubernetes manifests, managed as an [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [RKE2 documentation](https://docs.rke2.io/helm#automatically-deploying-manifests-and-helm-charts) for details.
#### Agent Environment Vars
Option to set environment variables for [Rancher agents](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md). The environment variables can be set using key value pairs. Refer to the [RKE2 documentation](https://docs.rke2.io/reference/linux_agent_config) for more details.
#### etcd
##### Automatic Snapshots
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [RKE2 documentation](https://docs.rke2.io/datastore/backup_restore#creating-snapshots). Note that with RKE2, snapshots are stored on each etcd node. This varies from RKE1 which only stores one snapshot per cluster.
##### Metrics
Option to choose whether to expose etcd metrics to the public or only within the cluster.
#### Networking
##### Cluster CIDR
IPv4 and/or IPv6 network CIDRs to use for pod IPs (default: `10.42.0.0/16`).
Example values:
- IPv4-only: `10.42.0.0/16`
- IPv6-only: `2001:cafe:42::/56`
- Dual-stack: `10.42.0.0/16,2001:cafe:42::/56`
For additional requirements and limitations related to dual-stack or IPv6-only networking, see the following resources:
- [RKE2 documentation: Dual-stack configuration](https://docs.rke2.io/networking/basic_network_options#dual-stack-configuration)
- [RKE2 documentation: IPv6-only setup](https://docs.rke2.io/networking/basic_network_options#ipv6-setup)
:::caution
You must configure the Service CIDR when you first create the cluster. You cannot enable the Service CIDR on an existing cluster after it starts.
:::
:::caution
When using `cilium` or `multus,cilium` as your container network interface provider, ensure the **Enable IPv6 Support** option is also enabled.
:::
##### Service CIDR
IPv4/IPv6 network CIDRs to use for service IPs (default: `10.43.0.0/16`).
Example values:
- IPv4-only: `10.43.0.0/16`
- IPv6-only: `2001:cafe:43::/112`
- Dual-stack: `10.43.0.0/16,2001:cafe:43::/112`
For additional requirements and limitations related to dual-stack or IPv6-only networking, see the following resources:
- [RKE2 documentation: Dual-stack configuration](https://docs.rke2.io/networking/basic_network_options#dual-stack-configuration)
- [RKE2 documentation: IPv6-only setup](https://docs.rke2.io/networking/basic_network_options#ipv6-setup)
:::caution
You must configure the Service CIDR when you first create the cluster. You cannot enable the Service CIDR on an existing cluster after it starts.
:::
:::caution
When using `cilium` or `multus,cilium` as your container network interface provider, ensure the **Enable IPv6 Support** option is also enabled.
:::
##### Cluster DNS
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: `10.43.0.10`).
##### Cluster Domain
Select the domain for the cluster. The default is `cluster.local`.
##### NodePort Service Port Range
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
##### Truncate Hostnames
Option to truncate hostnames to 15 characters or fewer. You can only set this field during the initial creation of the cluster. You can't enable or disable the 15-character limit after cluster creation.
This setting only affects machine-provisioned clusters. Since custom clusters set hostnames during their own node creation process, which occurs outside of Rancher, this field doesn't restrict custom cluster hostname length.
Truncating hostnames in a cluster improves compatibility with Windows-based systems. Although Kubernetes allows hostnames up to 63 characters in length, systems that use NetBIOS restrict hostnames to 15 characters or fewer.
##### TLS Alternate Names
Add hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
##### Authorized Cluster Endpoint
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
##### Stack Preference
Choose the networking stack for the cluster. This option affects:
- The address used for health and readiness probes of components such as Calico, etcd, kube-apiserver, kube-scheduler, kube-controller-manager, and kubelet.
- The server URL in the `authentication-token-webhook-config-file` for the Authorized Cluster Endpoint.
- The `advertise-client-urls` setting for etcd during snapshot restoration.
Options are `ipv4`, `ipv6`, `dual`:
- When set to `ipv4`, the cluster uses `127.0.0.1`
- When set to `ipv6`, the cluster uses `[::1]`
- When set to `dual`, the cluster uses `localhost`
The stack preference must match the cluster’s networking configuration:
- Set to `ipv4` for IPv4-only clusters
- Set to `ipv6` for IPv6-only clusters
- Set to `dual` for dual-stack clusters
:::caution
Ensuring the loopback address configuration is correct is critical for successful cluster provisioning.
For more information, refer to the [Node Requirements](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md) page.
:::
#### Registries
Select the image repository to pull Rancher images from. For more details and configuration options, see the [RKE2 documentation](https://docs.rke2.io/install/private_registry).
#### Upgrade Strategy
##### Control Plane Concurrency
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
##### Worker Concurrency
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
##### Drain Nodes (Control Plane)
Option to remove all pods from the node prior to upgrading.
##### Drain Nodes (Worker Nodes)
Option to remove all pods from the node prior to upgrading.
#### Advanced
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
## Cluster Config File Reference
Editing clusters in YAML allows you to set the [options available](https://docs.rke2.io/install/configuration) in an RKE2 installation, including those already listed in [Configuration Options in the Rancher UI](#configuration-options-in-the-rancher-ui), as well as set Rancher-specific parameters.
<details>
<summary>
<b>Example Cluster Config File Snippet</b>
</summary>
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
cloudCredentialSecretName: cattle-global-data:cc-s879v
kubernetesVersion: v1.25.12+rke2r1
localClusterAuthEndpoint: {}
rkeConfig:
additionalManifest: ""
chartValues:
rke2-calico: {}
etcd:
snapshotRetention: 5
snapshotScheduleCron: 0 */5 * * *
machineGlobalConfig:
cni: calico
disable-kube-proxy: false
etcd-expose-metrics: false
profile: null
kube-apiserver-arg:
- audit-policy-file=/etc/rancher/rke2/user-audit-policy.yaml
- audit-log-path=/etc/rancher/rke2/user-audit.logs
machinePools:
- controlPlaneRole: true
etcdRole: true
machineConfigRef:
kind: Amazonec2Config
name: nc-test-pool1-pwl5h
name: pool1
quantity: 1
unhealthyNodeTimeout: 0s
workerRole: true
machineSelectorConfig:
- config:
protect-kernel-defaults: false
machineSelectorFiles:
- fileSources:
- configMap:
name: ''
secret:
name: audit-policy
items:
- key: audit-policy
path: /etc/rancher/rke2/user-audit-policy.yaml
machineLabelSelector:
matchLabels:
rke.cattle.io/control-plane-role: 'true'
registries: {}
upgradeStrategy:
controlPlaneConcurrency: "1"
controlPlaneDrainOptions:
deleteEmptyDirData: true
enabled: true
gracePeriod: -1
ignoreDaemonSets: true
timeout: 120
workerConcurrency: "1"
workerDrainOptions:
deleteEmptyDirData: true
enabled: true
gracePeriod: -1
ignoreDaemonSets: true
timeout: 120
```
</details>
### additionalManifest
Specify additional manifests to deliver to the control plane nodes.
The value is a String, and will be placed at the path `/var/lib/rancher/rke2/server/manifests/rancher/addons.yaml` on target nodes.
Example:
```yaml
additionalManifest: |-
apiVersion: v1
kind: Namespace
metadata:
name: name-xxxx
```
:::note
If you want to customize system charts, you should use the `chartValues` field as described below.
Alternatives, such as using a HelmChartConfig to customize the system charts via `additionalManifest`, can cause unexpected behavior, due to having multiple HelmChartConfigs for the same chart.
:::
### chartValues
Specify the values for the system charts installed by RKE2.
For more information about how RKE2 manges packaged components, please refer to [RKE2 documentation](https://docs.rke2.io/helm).
Example:
```yaml
chartValues:
chart-name:
key: value
```
### machineGlobalConfig
Specify RKE2 configurations. Any configuration change made here will apply to every node. The configuration options available in the [standalone version of RKE2](https://docs.rke2.io/reference/server_config) can be applied here.
Example:
```yaml
machineGlobalConfig:
etcd-arg:
- key1=value1
- key2=value2
```
There are some configuration options that can't be changed when provisioning via Rancher:
- data-dir (folder to hold state), which defaults to `/var/lib/rancher/rke2`.
To make it easier to put files on nodes beforehand, Rancher expects the following values to be included in the configuration, while RKE2 expects the values to be entered as file paths:
- audit-policy-file
- cloud-provider-config
- private-registry
Rancher delivers the files to the path `/var/lib/rancher/rke2/etc/config-files/<option>` in target nodes, and sets the proper options in the RKE2 server.
Example:
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
rkeConfig:
machineGlobalConfig:
audit-policy-file:
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: RequestResponse
resources:
- group: ""
resources:
- pods
```
### machineSelectorConfig
`machineSelectorConfig` is the same as [`machineGlobalConfig`](#machineglobalconfig) except that a [label](#kubernetes-node-labels) selector can be specified with the configuration. The configuration will only be applied to nodes that match the provided label selector.
Multiple `config` entries are allowed, each specifying their own `machineLabelSelector`. A user can specify `matchExpressions`, `matchLabels`, both, or neither. Omitting the `machineLabelSelector` section of this field has the same effect as putting the config in the `machineGlobalConfig` section.
Example:
```yaml
machineSelectorConfig
- config:
config-key: config-value
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
```
### machineSelectorFiles
:::note
This feature is available in Rancher v2.7.2 and later.
:::
Deliver files to nodes, so that the files can be in place before initiating RKE2 server or agent processes.
The content of the file is retrieved from either a secret or a configmap. The target nodes are filtered by the `machineLabelSelector`.
Example :
```yaml
machineSelectorFiles:
- fileSources:
- secret:
items:
- key: example-key
path: path-to-put-the-file-on-nodes
permissions: 644 (optional)
hash: base64-encoded-hash-of-the-content (optional)
name: example-secret-name
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
- fileSources:
- configMap:
items:
- key: example-key
path: path-to-put-the-file-on-nodes
permissions: 644 (optional)
hash: base64-encoded-hash-of-the-content (optional)
name: example-configmap-name
machineLabelSelector:
matchExpressions:
- key: example-key
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
values:
- example-value1
- example-value2
matchLabels:
key1: value1
key2: value2
```
The secret or configmap must meet the following requirements:
1. It must be in the `fleet-default` namespace where the Cluster object exists.
2. It must have the annotation `rke.cattle.io/object-authorized-for-clusters: cluster-name1,cluster-name2`, which permits the target clusters to use it.
:::tip
Rancher Dashboard provides an easy-to-use form for creating the secret or configmap.
:::
Example:
```yaml
apiVersion: v1
data:
audit-policy: >-
IyBMb2cgYWxsIHJlcXVlc3RzIGF0IHRoZSBNZXRhZGF0YSBsZXZlbC4KYXBpVmVyc2lvbjogYXVkaXQuazhzLmlvL3YxCmtpbmQ6IFBvbGljeQpydWxlczoKLSBsZXZlbDogTWV0YWRhdGE=
kind: Secret
metadata:
annotations:
rke.cattle.io/object-authorized-for-clusters: cluster1
name: name1
namespace: fleet-default
```
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/sync-clusters.md
+49
View File
@@ -0,0 +1,49 @@
---
title: Syncing Hosted Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/sync-clusters"/>
</head>
Syncing allows Rancher to update cluster values so that they're up to date with the corresponding cluster object hosted in AKS, EKS or GKE. This enables sources other than Rancher to own a hosted cluster’s state.
:::warning
You may accidentally overwrite the state from one source if you simultaneously process an update from another source. This may also occur if you process an update from one source within 5 minutes of finishing an update from another source.
:::
### How it works
There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
1. The config object for the cluster, located on the Spec of the Cluster:
* For AKS, the field is called AKSConfig
* For EKS, the field is called EKSConfig
* For GKE, the field is called GKEConfig
2. The UpstreamSpec object
* For AKS, this is located on the AKSStatus field on the Status of the Cluster.
* For EKS, this is located on the EKSStatus field on the Status of the Cluster.
* For GKE, this is located on the GKEStatus field on the Status of the Cluster.
The struct types that define these objects can be found in their corresponding operator projects:
* [aks-operator](https://github.com/rancher/aks-operator/blob/master/pkg/apis/aks.cattle.io/v1/types.go)
* [eks-operator](https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go)
* [gke-operator](https://github.com/rancher/gke-operator/blob/master/pkg/apis/gke.cattle.io/v1/types.go)
All fields are nillable, except for the following: the cluster name, the location (region or zone), Imported, and the cloud credential reference.
The AKSConfig, EKSConfig or GKEConfig represents the desired state. Nil values are ignored. Fields that are non-nil in the config object can be thought of as managed. When a cluster is created in Rancher, all fields are non-nil and therefore managed. When a pre-existing cluster is registered in Rancher all nillable fields are set to nil and aren't managed. Those fields become managed once their value has been changed by Rancher.
UpstreamSpec represents the cluster as it is in the hosted Kubernetes provider. It's refreshed every 5 minutes. After the UpstreamSpec is refreshed, Rancher checks if the cluster has an update in progress. If it's currently updating, nothing further is done. If it is not currently updating, any managed fields on AKSConfig, EKSConfig or GKEConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
:::warning
When you import a cluster from a cloud provider into Rancher, UpstreamSpec represents the cluster state and Config is empty. If you then update the imported cluster through the Rancher UI, both UpstreamSpec and Config become non-null. Any further updates to the cluster should be applied through Rancher. This is because there is no safe way to determine if changes originating from UpstreamSpec represent the desired state or just a mismatch with Config. If you update the imported cluster through the cloud provider console after you apply any updates through the Rancher UI, the controller will deploy a rollback and the content of Config will be considered the desired state.
:::
The effective desired state can be thought of as the UpstreamSpec, plus all non-nil fields in the AKSConfig, EKSConfig or GKEConfig. This is what is displayed in the UI.
If Rancher and another source attempt to update a cluster at the same time, or within 5 minutes of an update finishing, any managed fields are likely to get caught in a race condition. To use EKS as an example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console at 11:01, and tags are updated from Rancher before 11:05, then the value is likely to be overwritten. This can also occur if tags are updated while the cluster is still processing the update. The issue described in this example shouldn't occur if the cluster is registered and the PrivateAccess fields are nil.
versioned_docs/version-2.14/reference-guides/cluster-configuration/rancher-server-configuration/use-existing-nodes/use-existing-nodes.md
+151
View File
@@ -0,0 +1,151 @@
---
title: Launching Kubernetes on Existing Custom Nodes
description: To create a cluster with custom nodes, you’ll need to access servers in your cluster and provision them according to Rancher requirements
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/use-existing-nodes"/>
</head>
When you create a custom cluster, Rancher can use RKE2/K3s to create a Kubernetes cluster in on-prem bare-metal servers, on-prem virtual machines, or in any node hosted by an infrastructure provider.
To use this option, you need access to the servers that will be part of your Kubernetes cluster. Provision each server according to the [requirements](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md). Then, run the command provided in the Rancher UI on each server to convert it into a Kubernetes node.
This section describes how to set up a custom cluster.
## Creating a Cluster with Custom Nodes
:::note Want to use Windows hosts as Kubernetes workers?
See [Configuring Custom Clusters for Windows](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/use-windows-clusters/use-windows-clusters.md) before you start.
:::
### 1. Provision a Linux Host
Begin creation of a custom cluster by provisioning a Linux host. Your host can be:
- A cloud-host virtual machine (VM)
- An on-prem VM
- A bare-metal server
If you want to reuse a node from a previous custom cluster, [clean the node](../../../../how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md) before using it in a cluster again. If you reuse a node that hasn't been cleaned, cluster provisioning may fail.
Provision the host according to the [installation requirements](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md) and the [checklist for production-ready clusters.](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/checklist-for-production-ready-clusters/checklist-for-production-ready-clusters.md)
:::note IPv6-only cluster
For an IPv6-only cluster, ensure that your operating system correctly configures the `/etc/hosts` file.
```
::1 localhost
```
:::
### 2. Create the Custom Cluster
1. Click **☰ > Cluster Management**.
1. On the **Clusters** page, click **Create**.
1. Click **Custom**.
1. Enter a **Cluster Name**.
1. Use the **Cluster Configuration** section to set up the cluster. For more information, see [RKE2 Cluster Configuration Reference](../rke2-cluster-configuration.md) and [K3s Cluster Configuration Reference](../k3s-cluster-configuration.md).
:::note Windows nodes
To learn more about using Windows nodes as Kubernetes workers, see [Launching Kubernetes on Windows Clusters](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/use-windows-clusters/use-windows-clusters.md).
:::
1. Click **Create**.
**Result:** The UI redirects to the **Registration** page, where you can generate the registration command for your nodes.
1. From **Node Role**, select the roles you want a cluster node to fill. You must provision at least one node for each role: etcd, worker, and control plane. A custom cluster requires all three roles to finish provisioning. For more information on roles, see [Roles for Nodes in Kubernetes Clusters](../../../kubernetes-concepts.md#roles-for-nodes-in-kubernetes-clusters).
:::note Bare-Metal Server
If you plan to dedicate bare-metal servers to each role, you must provision a bare-metal server for each role (i.e., provision multiple bare-metal servers).
:::note
1. **Optional**: Click **Show Advanced** to configure additional settings such as specifying the IP address(es), overriding the node hostname, or adding [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) or [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to the node
:::note
The **Node Public IP** and **Node Private IP** fields can accept either a single address or a comma-separated list of addresses (for example: `10.0.0.5,2001:db8::1`).
:::
:::note Ipv6-only or Dual-stack Cluster
In both IPv6-only and dual-stack clusters, you should specify the node’s **IPv6 address** as the **Node Private IP**.
:::
1. Copy the command displayed on screen to your clipboard.
1. Log in to your Linux host using your preferred shell, such as PuTTy or a remote Terminal connection. Run the command copied to your clipboard.
:::note
Repeat steps 7-10 if you want to dedicate specific hosts to specific node roles. Repeat the steps as many times as needed.
:::
**Result:**
The cluster is created and transitions to the **Updating** state while Rancher initializes and provisions cluster components.
You can access your cluster after its state is updated to **Active**.
**Active** clusters are assigned two Projects:
- `Default`, containing the `default` namespace
- `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces
### 3. Amazon Only: Tag Resources
If you have configured your cluster to use Amazon as **Cloud Provider**, tag your AWS resources with a cluster ID.
[Amazon Documentation: Tagging Your Amazon EC2 Resources](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html)
:::note
You can use Amazon EC2 instances without configuring a cloud provider in Kubernetes. You only have to configure the cloud provider if you want to use specific Kubernetes cloud provider functionality. For more information, see [Kubernetes Cloud Providers](https://github.com/kubernetes/website/blob/release-1.18/content/en/docs/concepts/cluster-administration/cloud-providers.md)
:::
The following resources need to be tagged with a `ClusterID`:
- **Nodes**: All hosts added in Rancher.
- **Subnet**: The subnet used for your cluster
- **Security Group**: The security group used for your cluster.
:::note
Do not tag multiple security groups. Tagging multiple groups generates an error when creating Elastic Load Balancer.
:::
The tag that should be used is:
```
Key=kubernetes.io/cluster/<CLUSTERID>, Value=owned
```
`<CLUSTERID>` can be any string you choose. However, the same string must be used on every resource you tag. Setting the tag value to `owned` informs the cluster that all resources tagged with the `<CLUSTERID>` are owned and managed by this cluster.
If you share resources between clusters, you can change the tag to:
```
Key=kubernetes.io/cluster/CLUSTERID, Value=shared
```
## Optional Next Steps
After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:
- **Access your cluster with the kubectl CLI:** Follow [these steps](../../../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#accessing-clusters-with-kubectl-from-your-workstation) to access clusters with kubectl on your workstation. In this case, you will be authenticated through the Rancher server’s authentication proxy, then Rancher will connect you to the downstream cluster. This method lets you manage the cluster without the Rancher UI.
- **Access your cluster with the kubectl CLI, using the authorized cluster endpoint:** Follow [these steps](../../../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) to access your cluster with kubectl directly, without authenticating through Rancher. We recommend setting up this alternative method to access your cluster so that in case you can’t connect to Rancher, you can still access the cluster.
versioned_docs/version-2.14/reference-guides/dual-stack.md
+122
View File
@@ -0,0 +1,122 @@
---
title: IPv4/IPv6 Dual-stack
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/dual-stack/"/>
</head>
Kubernetes supports IPv4-only, IPv6-only, and dual-stack networking configurations.
For more details, refer to the official [Kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/dual-stack/).
## Installing Rancher on IPv6-Only or Dual-Stack Clusters
Rancher can run on clusters using:
- IPv4-only
- IPv6-only
- Dual-stack (IPv4 + IPv6)
When you install Rancher on an **IPv6-only cluster**, it can communicate externally **only over IPv6**. This means it can provision:
- IPv6-only clusters
- Dual-stack clusters
_(IPv4-only downstream clusters are not possible in this case)_
When you install Rancher on a **dual-stack cluster**, it can communicate over both IPv4 and IPv6, and can therefore provision:
- IPv4-only clusters
- IPv6-only clusters
- Dual-stack clusters
For installation steps, see the guide: **[Installing and Upgrading Rancher](../getting-started/installation-and-upgrade/installation-and-upgrade.md)**.
### Requirement for the Rancher Server URL
When provisioning IPv6-only downstream clusters, the **Rancher Server URL must be reachable over IPv6** because downstream nodes connect back to the Rancher server using IPv6.
## Provisioning IPv6-Only or Dual-Stack Clusters
You can provision RKE2 and K3s **Node driver** (machine pools) or **Custom cluster** (existing hosts) clusters using IPv4-only, IPv6-only, or dual-stack networking.
### Network Configuration
To enable IPv6-only or dual-stack networking, you must configure:
- Cluster CIDR
- Service CIDR
- Stack Preference
Configuration references:
- [K3s Cluster Configuration Reference](cluster-configuration/rancher-server-configuration/k3s-cluster-configuration.md)
- [RKE2 Cluster Configuration Reference](cluster-configuration/rancher-server-configuration/rke2-cluster-configuration.md)
### Support for Windows
Kubernetes on Windows:
| Feature | Support Status |
|---------------------|-------------------------------|
| IPv6-only clusters | Not supported |
| Dual-stack clusters | Supported |
| Services | Limited to a single IP family |
For more information, see the [Kubernetes Documentation](https://kubernetes.io/docs/concepts/services-networking/dual-stack/#windows-support).
K3s does **not** support Windows ([FAQ](https://docs.k3s.io/faq#does-k3s-support-windows))
RKE2 supports Windows, but requires using either `Calico` or `Flannel` as the CNI.
Note that Windows installations of RKE2 do not support dual-stack clusters using BGP.
For more details, see [RKE2 Network Options](https://docs.rke2.io/networking/basic_network_options).
### Provisioning Node Driver Clusters
Rancher currently supports assigning IPv6 addresses in **node driver** clusters with:
- [Amazon EC2](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md)
- [DigitalOcean](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-a-digitalocean-cluster.md)
Support for additional providers will be introduced in future releases.
:::note DigitalOcean Limitation
Creating an **IPv6-only cluster** using the DigitalOcean node driver is currently **not supported**.
For more details, please see [rancher/rancher#52523](https://github.com/rancher/rancher/issues/52523#issuecomment-3457803572).
:::
#### Infrastructure Requirements
Cluster nodes must meet the requirements listed in the [Node Requirements for Rancher Managed Clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md).
Machine pool configuration guides:
- [Amazon EC2 Configuration](cluster-configuration/downstream-cluster-configuration/machine-configuration/amazon-ec2.md)
- [DigitalOcean Configuration](cluster-configuration/downstream-cluster-configuration/machine-configuration/digitalocean.md)
### Provisioning Custom Clusters
To provision on your own nodes, follow the instructions in [Provision Kubernetes on Existing Nodes](cluster-configuration/rancher-server-configuration/use-existing-nodes/use-existing-nodes.md).
:::note
- **Node Public IP** and **Node Private IP** fields accept IPv4, IPv6, or both (comma-separated).
> Example: `10.0.0.5,2001:db8::1`
- In **IPv6-only** and **dual-stack** clusters, specify the node’s **IPv6 address** as the **Private IP**.
:::
#### Infrastructure Requirements
Infrastructure requirements are the same as above for node-driver clusters.
## Other Limitations
### GitHub.com
GitHub.com does **not** support IPv6. As a result:
- Any application repositories ( `ClusterRepo.catalog.cattle.io/v1` CR) hosted on GitHub.com will **not be reachable** from IPv6-only clusters.
- Similarly, any **non-builtin node drivers** hosted on GitHub.com will also **not be accessible** in IPv6-only environments.
versioned_docs/version-2.14/reference-guides/kubernetes-concepts.md
+62
View File
@@ -0,0 +1,62 @@
---
title: Kubernetes Concepts
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/kubernetes-concepts"/>
</head>
This page explains concepts related to Kubernetes that are important for understanding how Rancher works. The descriptions below provide a simplified overview of Kubernetes components. For more details, refer to the [official documentation on Kubernetes components.](https://kubernetes.io/docs/concepts/overview/components/)
## About Docker
Docker is a container packaging and runtime standard. Developers can build container images from Dockerfiles and distribute container images from Docker registries. [Docker Hub](https://hub.docker.com) is a popular public registry. Many organizations also set up private Docker registries. Docker is primarily used to manage containers on individual nodes.
## About Kubernetes
Kubernetes is the container cluster management standard. YAML files specify containers and other resources that form an application. Kubernetes performs functions such as scheduling, scaling, service discovery, health check, secret management, and configuration management.
## What is a Kubernetes Cluster?
A cluster is a group of computers that work together as a single system.
A _Kubernetes Cluster_ is a cluster that uses the [Kubernetes container-orchestration system](https://kubernetes.io/) to deploy, maintain, and scale containers, allowing your organization to automate application operations.
## Roles for Nodes in Kubernetes Clusters
Each computing resource in a Kubernetes cluster is called a _node_. Nodes can be either bare-metal servers or virtual machines. Kubernetes classifies nodes into three types: _etcd_ nodes, _control plane_ nodes, and _worker_ nodes.
A Kubernetes cluster consists of at least one etcd, controlplane, and worker node.
### etcd Nodes
Rancher uses etcd as a data store in both single node and high-availability installations. In Kubernetes, etcd is also a role for nodes that store the cluster state.
The state of a Kubernetes cluster is maintained in [etcd.](https://kubernetes.io/docs/concepts/overview/components/#etcd) The etcd nodes run the etcd database.
The etcd database component is a distributed key-value store used as Kubernetes storage for all cluster data, such as cluster coordination and state management. It is recommended to run etcd on multiple nodes so that there's always a backup available for failover.
Although you can run etcd on just one node, etcd requires a majority of nodes, a quorum, to agree on updates to the cluster state. The cluster should always contain enough healthy etcd nodes to form a quorum. For a cluster with n members, a quorum is (n/2)+1. For any odd-sized cluster, adding one node will always increase the number of nodes necessary for a quorum.
Three etcd nodes is generally sufficient for smaller clusters and five etcd nodes for large clusters.
### Controlplane Nodes
Controlplane nodes run the Kubernetes API server, scheduler, and controller manager. These nodes take care of routine tasks to ensure that your cluster maintains your configuration. Because all cluster data is stored on your etcd nodes, control plane nodes are stateless. You can run control plane on a single node, although three or more nodes are recommended for redundancy. Additionally, a single node can share the control plane and etcd roles.
### Worker Nodes
Each [worker node](https://kubernetes.io/docs/concepts/architecture/nodes/) runs the following:
- **Kubelets:** An agent that monitors the state of the node, ensuring your containers are healthy.
- **Workloads:** The containers and pods that hold your apps, as well as other types of deployments.
Worker nodes also run storage and networking drivers, and ingress controllers when required. You create as many worker nodes as necessary to run your [workloads](../how-to-guides/new-user-guides/kubernetes-resources-setup/workloads-and-pods/workloads-and-pods.md).
## About Helm
For high-availability installations of Rancher, Helm is the tool used to install Rancher on a Kubernetes cluster.
Helm is the package management tool of choice for Kubernetes. Helm charts provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh).
For more information on service accounts and cluster role binding, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/examples.md
+27
View File
@@ -0,0 +1,27 @@
---
title: Monitoring Configuration Examples
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/examples"/>
</head>
## ServiceMonitor
See the official prometheus-operator GitHub repo for an example [ServiceMonitor](https://github.com/prometheus-operator/prometheus-operator/blob/main/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml) YAML.
## PodMonitor
See the [Prometheus Operator documentation](https://prometheus-operator.dev/docs/developer/getting-started/#using-podmonitors) for an example PodMonitor and an example Prometheus resource that refers to a PodMonitor.
## PrometheusRule
A PrometheusRule contains the alerting and recording rules that you would usually place in a [Prometheus rule file](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/).
For a more fine-grained approach, the `ruleSelector` field on a Prometheus resource can select which PrometheusRules should be loaded onto Prometheus, based on the labels attached to the PrometheusRules resources.
See the [Prometheus Operator documentation](https://prometheus-operator.dev/docs/developer/alerting/) for an example PrometheusRule.
## Alertmanager Config
See the Rancher docs page on Receivers for an example [Alertmanager config](./receivers.md#example-alertmanager-configs).
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/helm-chart-options.md
+70
View File
@@ -0,0 +1,70 @@
---
title: Helm Chart Options
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/helm-chart-options"/>
</head>
## Configuring Resource Limits and Requests
The resource requests and limits can be configured when installing `rancher-monitoring`.
The default values are in the [values.yaml](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/values.yaml) in the `rancher-monitoring` Helm chart.
The default values in the table below are the minimum required resource limits and requests.
| Resource Name | Memory Limit | CPU Limit | Memory Request | CPU Request |
| ------------- | ------------ | ----------- | ---------------- | ------------------ |
| alertmanager | 500Mi | 1000m | 100Mi | 100m |
| grafana | 200Mi | 200m | 100Mi | 100m |
| kube-state-metrics subchart | 200Mi | 100m | 130Mi | 100m |
| prometheus-node-exporter subchart | 50Mi | 200m | 30Mi | 100m |
| prometheusOperator | 500Mi | 200m | 100Mi | 100m |
| prometheus | 2500Mi | 1000m | 1750Mi | 750m |
| **Total** | **3950Mi** | **2700m** | **2210Mi** | **1250m** |
At least 50Gi storage is recommended.
## Trusted CA for Notifiers
If you need to add a trusted CA to your notifier, follow these steps:
1. Create the `cattle-monitoring-system` namespace.
1. Add your trusted CA secret to the `cattle-monitoring-system` namespace.
1. Deploy or upgrade the `rancher-monitoring` Helm chart. In the chart options, reference the secret in **Alerting > Additional Secrets**.
**Result:** The default Alertmanager custom resource will have access to your trusted CA.
## Additional Scrape Configurations
If the scrape configuration you want cannot be specified via a ServiceMonitor or PodMonitor at the moment, you can provide an `additionalScrapeConfigSecret` on deploying or upgrading `rancher-monitoring`.
A [scrape_config section](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) specifies a set of targets and parameters describing how to scrape them. In the general case, one scrape configuration specifies a single job.
An example of where this might be used is with Istio. For more information, see [this section.](../../integrations-in-rancher/istio/configuration-options/selectors-and-scrape-configurations.md)
## Configuring Applications Packaged within Monitoring v2
We deploy kube-state-metrics and node-exporter with monitoring v2. The node exporters are deployed as DaemonSets. Each of these entities are deployed as sub-charts through the monitoring v2 Helm chart, values.yaml.
We also deploy Grafana, which is not managed by Prometheus.
Many values aren’t exposed in the top level chart. However, you can add values to the top level chart to override values that exist in the sub-charts.
### Increase the Replicas of Alertmanager
As part of the chart deployment options, you can opt to increase the number of replicas of the Alertmanager deployed onto your cluster. The replicas can all be managed using the same underlying Alertmanager Config Secret. For more information on the Alertmanager Config Secret, refer to [this section.](../../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/alertmanager.md#multiple-alertmanager-replicas)
### Configuring the Namespace for a Persistent Grafana Dashboard
To specify that you would like Grafana to watch for ConfigMaps across all namespaces, set this value in the `rancher-monitoring` Helm chart:
```
grafana.sidecar.dashboards.searchNamespace=ALL
```
Note that the RBAC roles exposed by the Monitoring chart to add Grafana Dashboards are still restricted to giving permissions for users to add dashboards in the namespace defined in `grafana.dashboards.namespace`, which defaults to `cattle-dashboards`.
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/monitoring-v2-configuration.md
+15
View File
@@ -0,0 +1,15 @@
---
title: Monitoring Configuration Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration"/>
</head>
The following sections will explain important options essential to configuring Monitoring V2 in Rancher:
- [Receiver Configuration](receivers.md)
- [Route Configuration](routes.md)
- [ServiceMonitor and PodMonitor Configuration](servicemonitors-and-podmonitors.md)
- [Helm Chart Options](helm-chart-options.md)
- [Examples](examples.md)
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/receivers.md
+380
View File
@@ -0,0 +1,380 @@
---
title: Receiver Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/receivers"/>
</head>
The [Alertmanager Config](https://prometheus.io/docs/alerting/latest/configuration/#configuration-file) Secret contains the configuration of an Alertmanager instance that sends out notifications based on alerts it receives from Prometheus.
:::note
This section assumes familiarity with how monitoring components work together. For more information about Alertmanager, see [this section.](../../integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md#3-how-alertmanager-works)
:::
## Creating Receivers in the Rancher UI
:::note Prerequisites:
- The monitoring application needs to be installed.
- If you configured monitoring with an existing Alertmanager Secret, it must have a format that is supported by Rancher's UI. Otherwise you will only be able to make changes based on modifying the Alertmanager Secret directly. Note: We are continuing to make enhancements to what kinds of Alertmanager Configurations we can support using the Routes and Receivers UI, so please [file an issue](https://github.com/rancher/rancher/issues/new) if you have a request for a feature enhancement.
:::
<Tabs>
<TabItem value="Rancher v2.6.5+">
1. Go to the cluster where you want to create receivers. Click **Monitoring -> Alerting -> AlertManagerConfigs**.
1. Click **Create**.
1. Enter a **Name** for the new AlertmanagerConfig.
1. Click **Create**.
1. After creating the AlertManagerConfig, click it to add a receiver.
1. Click **Add Receiver**.
1. Enter a **Name** for the receiver.
1. Configure one or more providers for the receiver. For help filling out the forms, refer to the configuration options below.
1. Click **Create**.
</TabItem>
<TabItem value="Rancher before v2.6.5">
1. Go to the cluster where you want to create receivers. Click **Monitoring** and click **Receiver**.
2. Enter a name for the receiver.
3. Configure one or more providers for the receiver. For help filling out the forms, refer to the configuration options below.
4. Click **Create**.
</TabItem>
</Tabs>
**Result:** Alerts can be configured to send notifications to the receiver(s).
## Receiver Configuration
The notification integrations are configured with the `receiver`, which is explained in the [Prometheus documentation.](https://prometheus.io/docs/alerting/latest/configuration/#receiver)
### Native vs. Non-native Receivers
By default, AlertManager provides native integration with some receivers, which are listed in [this section.](https://prometheus.io/docs/alerting/latest/configuration/#receiver) All natively supported receivers are configurable through the Rancher UI.
For notification mechanisms, such as Telegram, that are not natively supported by AlertManager, integration is achieved using the [webhook receiver.](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config) A list of third-party drivers providing such integrations can be found [here.](https://prometheus.io/docs/operating/integrations/#alertmanager-webhook-receiver) Access to these drivers, and their associated integrations, is provided through the Alerting Drivers app. Once enabled, configuring non-native receivers can also be done through the Rancher UI.
Currently the Rancher Alerting Drivers app provides access to the following integrations:
- Microsoft Teams, based on the [prom2teams](https://github.com/idealista/prom2teams) driver.
- Telegram, based on the [Sachet](https://github.com/messagebird/sachet) driver.
The following types of receivers can be configured in the Rancher UI:
- <a href="#slack">Slack</a>
- <a href="#email">Email</a>
- <a href="#pagerduty">PagerDuty</a>
- <a href="#opsgenie">Opsgenie</a>
- <a href="#webhook">Webhook</a>
- <a href="#custom">Custom</a>
- <a href="#teams">Teams</a>
- <a href="#sms">SMS</a>
- <a href="#telegram">Telegram</a>
The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.
## Slack
| Field | Type | Description |
|------|--------------|------|
| URL | String | Enter your Slack webhook URL. For instructions to create a Slack webhook, see the [Slack documentation.](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack) |
| Default Channel | String | Enter the name of the channel that you want to send alert notifications in the following format: `#<channelname>`. |
| Proxy URL | String | Proxy for the webhook notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
## Email
| Field | Type | Description |
|------|--------------|------|
| Default Recipient Address | String | The email address that will receive notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
SMTP options:
| Field | Type | Description |
|------|--------------|------|
| Sender | String | Enter an email address available on your SMTP mail server that you want to send the notification from. |
| Host | String | Enter the IP address or hostname for your SMTP server. Example: `smtp.email.com`. |
| Use TLS | Bool | Use TLS for encryption. |
| Username | String | Enter a username to authenticate with the SMTP server. |
| Password | String | Enter a password to authenticate with the SMTP server. |
## PagerDuty
| Field | Type | Description |
|------|------|-------|
| Integration Type | String | `Events API v2` or `Prometheus`. |
| Default Integration Key | String | For instructions to get an integration key, see the [PagerDuty documentation.](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/) |
| Proxy URL | String | Proxy for the PagerDuty notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
## Opsgenie
| Field | Description |
|------|-------------|
| API Key | For instructions to get an API key, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/api-key-management) |
| Proxy URL | Proxy for the Opsgenie notifications. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
Opsgenie Responders:
| Field | Type | Description |
|-------|------|--------|
| Type | String | Schedule, Team, User, or Escalation. For more information on alert responders, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/alert-recipients-and-teams) |
| Send To | String | Id, Name, or Username of the Opsgenie recipient. |
## Webhook
| Field | Description |
|-------|--------------|
| URL | Webhook URL for the app of your choice. |
| Proxy URL | Proxy for the webhook notification. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
## Custom
The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.
## Teams
### Enabling the Teams Receiver for Rancher Managed Clusters
The Teams receiver is not a native receiver and must be enabled before it can be used. You can enable the Teams receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the Teams option selected.
1. In the Rancher UI, go to the cluster where you want to install rancher-alerting-drivers and click **Apps**.
1. Click the **Alerting Drivers** app.
1. Click the **Helm Deploy Options** tab.
1. Select the **Teams** option and click **Install**.
1. Take note of the namespace used as it will be required in a later step.
### Configuring the Teams Receiver
1. To configure the Teams receiver, update its ConfigMap. The following example is a minimal Teams receiver configuration:
```yaml
[Microsoft Teams]
connector: https://your-teams-webhook-url
```
2. After you update the configuration, follow the instructions in [Creating Receivers in the Rancher UI](#creating-receivers-in-the-rancher-ui) to add the receiver. Use the example below to form your URL. Make sure to replace `<namespace>` with the namespace of the `rancher-alerting-drivers` app:
```yaml
url: http://rancher-alerting-drivers-prom2teams.<namespace>.svc:8089/v2/connector
```
<!-- https://github.com/idealista/prom2teams -->
## SMS
### Enabling the SMS Receiver for Rancher Managed Clusters
The SMS receiver is not a native receiver and must be enabled before it can be used. You can enable the SMS receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the SMS option selected.
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to install `rancher-alerting-drivers` and click **Explore**.
1. In the left navigation bar, click
1. Click the **Alerting Drivers** app.
1. Click the **Helm Deploy Options** tab
1. Select the **SMS** option and click **Install**.
1. Take note of the namespace used as it will be required in a later step.
### Configuring the SMS Receiver
The SMS receiver can be configured by updating its ConfigMap. For example, the following is a minimal SMS receiver configuration.
```yaml
providers:
telegram:
token: 'your-token-from-telegram'
receivers:
- name: 'telegram-receiver-1'
provider: 'telegram'
to:
- '123456789'
```
When configuration is complete, add the receiver using the steps in [this section](#creating-receivers-in-the-rancher-ui).
Use the example below as the name and URL, where:
- the name assigned to the receiver, e.g. `telegram-receiver-1`, must match the name in the `receivers.name` field in the ConfigMap, e.g. `telegram-receiver-1`
- `ns-1` in the URL is replaced with the namespace where the `rancher-alerting-drivers` app is installed
```yaml
name: telegram-receiver-1
url http://rancher-alerting-drivers-sachet.ns-1.svc:9876/alert
```
<!-- https://github.com/messagebird/sachet -->
## Telegram
### Enabling the Telegram Receiver for Rancher Managed Clusters
The Telegram receiver is not a native receiver. You must enable it before it can be used. You can enable the Telegram receiver for a Rancher-managed cluster by going to the **Apps** page and installing the `rancher-alerting-drivers` app with the **Telegram** option selected:
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to install `rancher-alerting-drivers` and click **Explore**.
1. In the left navigation bar, click on **Apps**.
1. Click the **Alerting Drivers** app.
1. Click on **Install**.
1. In the page that opens next, make sure that **Enable SMS** checkbox is selected. Telegram notifications require you to enable SMS.
1. Take note of the namespace used as it will be required in a later step.
### Test the Configuration by Configuring a PrometheusRule
To test your Telegram setup, create a **PrometheusRule** that continuously raises alerts.
:::caution NOTE
This rule is intended only to test if Telegram alerts work as expected. Do not leave it on after testing is completed.
:::
1. In the left navigation menu, click **Monitoring**.
1. Click **Advanced**.
1. Click **PrometheusRules > Create**.
1. Select a namespace to place the rule in and name the rule appropriately.
1. Set the group name to `test`. Use this value later when you create a **Route** in the **AlertManagerConfig**.
1. Under **Alerting Rules** click **Add**.
1. Set an appropriate **Alert Name**.
1. To trigger the alert immediately and continuously, enter the following PromQL Expression: `vector(1)`.
1. Under **Labels**, click **Add Label**. Enter the key `test` and value `alert`. This key-value pair will also be used later.
#### Configure an AlertManagerConfig
Configure an **AlertManagerConfig** to contain the **Receiver** and **Route** configuration for the **PrometheusRule** created above:
1. Click **Monitoring > Alerting**, and open **AlertManagerConfigs**.
1. Click **Create**
#### Create a Receiver in AlertManagerConfig
1. Choose a namespace from the dropdown and set an appropriate name.
1. Click **Create**.
1. Open the newly created **AlertManagerConfig** and click **⋮ > Edit Config**.
1. Click **Add Receiver**.
1. Select **Webhook** from the list on the **Create Receiver in AlertmanagerConfig** page.
1. Name the webhook, and click **Add Webhook**.
1. In the **Select Webhook Type** dropdown, select **SMS**. This will automatically populate the **Target** field as `http://rancher-alerting-drivers-sachet.cattle-monitoring-system.svc:9876/alert`. If you installed the **Alerting Drivers** in a namespace other than `cattle-monitoring-system`, the target URL will reflect that.
1. Click **Create**.
#### Create a Route in AlertManagerConfig
1. Click **⋮ > Edit Config**.
1. Click **Route**.
1. In the dropdown, select the **Receiver** you just created.
1. In the **Labels to Group Alerts By** field, type `test`.
1. Under **Waiting and Intervals**, set **Group Wait** to `1s` and **Group Interval** to `10s`. This triggers frequent alerts. Change the values as appropriate.
1. Under **Matchers** click **Add Matcher**. Enter `test` in the **Name** field and `alert` in the **Value** field. From the **Match Type** dropdown, select `MatchEqual`.
1. Click **Save**.
### Configuring the Telegram Receiver
You can configure the Telegram receiver by updating the `rancher-alerting-drivers-sachet` ConfigMap in the `cattle-monitoring-system` namespace. For example, the following is a minimal Telegram receiver configuration:
```yaml
providers:
telegram:
token: <your-token-from-telegram>
receivers:
- name: 'cattle-monitoring-system/test-amc/prom2tel'
provider: 'telegram'
to:
- '123456789'
```
To obtain a Telegram token, setup a Telegram bot. Refer to the [official Telegram guide](https://core.telegram.org/bots/tutorial) for details.
After you finish configuring the receiver, [add](#creating-receivers-in-the-rancher-ui) it.
Name the receiver `<namespace>/<alertmanagerconfig-name>/<receiver-name>`. Enter `123456789` as a placeholder for the Telegram user ID to send the notifications to. To find your Telegram ID, check [the Telegram userinfo bot](https://telegram.me/userinfobot).
You should now receive Telegram notifications to the user ID. If you don't receive notifications, please check if there are any errors reported in the Pod for the Deployment `rancher-alerting-drivers-sachet` under the `cattle-monitoring-system` namespace.
## Configuring Multiple Receivers
By editing the forms in the Rancher UI, you can set up a Receiver resource with all the information Alertmanager needs to send alerts to your notification system.
It is also possible to send alerts to multiple notification systems. One way is to configure the Receiver using custom YAML, in which case you can add the configuration for multiple notification systems, as long as you are sure that both systems should receive the same messages.
You can also set up multiple receivers by using the `continue` option for a route, so that the alerts sent to a receiver continue being evaluated in the next level of the routing tree, which could contain another receiver.
## Example Alertmanager Configs
### Slack
To set up notifications via Slack, the following Alertmanager Config YAML can be placed into the `alertmanager.yaml` key of the Alertmanager Config Secret, where the `api_url` should be updated to use your Webhook URL from Slack:
```yaml
route:
group_by: ['job']
group_wait: 30s
group_interval: 5m
repeat_interval: 3h
receiver: 'slack-notifications'
receivers:
- name: 'slack-notifications'
slack_configs:
- send_resolved: true
text: '{{ template "slack.rancher.text" . }}'
api_url: <user-provided slack webhook url here>
templates:
- /etc/alertmanager/config/*.tmpl
```
### PagerDuty
To set up notifications via PagerDuty, use the example below from the [PagerDuty documentation](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/) as a guideline. This example sets up a route that captures alerts for a database service and sends them to a receiver linked to a service that will directly notify the DBAs in PagerDuty, while all other alerts will be directed to a default receiver with a different PagerDuty integration key.
The following Alertmanager Config YAML can be placed into the `alertmanager.yaml` key of the Alertmanager Config Secret. The `service_key` should be updated to use your PagerDuty integration key and can be found as per the "Integrating with Global Event Routing" section of the PagerDuty documentation. For the full list of configuration options, refer to the [Prometheus documentation](https://prometheus.io/docs/alerting/latest/configuration/#pagerduty_config).
```yaml
route:
group_by: [cluster]
receiver: 'pagerduty-notifications'
group_interval: 5m
routes:
- match:
service: database
receiver: 'database-notifcations'
receivers:
- name: 'pagerduty-notifications'
pagerduty_configs:
- service_key: 'primary-integration-key'
- name: 'database-notifcations'
pagerduty_configs:
- service_key: 'database-integration-key'
```
## Example Route Config for Compliance Scan Alerts
While configuring the routes for `rancher-compliance` alerts, you can specify the matching using the key-value pair `job: rancher-compliance-scan`.
For example, the following example route configuration could be used with a Slack receiver named `test-compliance`:
```yaml
spec:
receiver: test-compliance
group_by:
# - string
group_wait: 30s
group_interval: 30s
repeat_interval: 30s
match:
job: rancher-compliance-scan
# key: string
match_re:
{}
# key: string
```
For more information on enabling alerting for `rancher-compliance-benchmark`, see [this section.](../../how-to-guides/advanced-user-guides/compliance-scan-guides/enable-alerting-for-rancher-compliance.md)
## Trusted CA for Notifiers
If you need to add a trusted CA to your notifier, follow the steps in [this section.](helm-chart-options.md#trusted-ca-for-notifiers)
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/routes.md
+68
View File
@@ -0,0 +1,68 @@
---
title: Route Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/routes"/>
</head>
The route configuration is the section of the Alertmanager custom resource that controls how the alerts fired by Prometheus are grouped and filtered before they reach the receiver.
When a Route is changed, the Prometheus Operator regenerates the Alertmanager custom resource to reflect the changes.
For more information about configuring routes, refer to the [official Alertmanager documentation.](https://www.prometheus.io/docs/alerting/latest/configuration/#route)
:::note
This section assumes familiarity with how monitoring components work together. For more information, see [this section.](../../integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)
:::
## Route Restrictions
Alertmanager proxies alerts for Prometheus based on its receivers and a routing tree that filters alerts to certain receivers based on labels.
Alerting drivers proxy alerts for Alertmanager to non-native receivers, such as Microsoft Teams and SMS.
In the Rancher UI for configuring routes and receivers, you can configure routing trees with one root and then a depth of one more level, for a tree with a depth of two. But if you use a `continue` route when configuring Alertmanager directly, you can make the tree deeper.
Each receiver is for one or more notification providers. So if you know that every alert for Slack should also go to PagerDuty, you can configure both in the same receiver.
## Route Configuration
### Note on Labels and Annotations
Labels should be used for identifying information that can affect the routing of notifications. Identifying information about the alert could consist of a container name, or the name of the team that should be notified.
Annotations should be used for information that does not affect who receives the alert, such as a runbook url or error message.
### Receiver
The route needs to refer to a [receiver](./receivers.md) that has already been configured.
### Grouping
| Field | Default | Description |
|-------|--------------|---------|
| Group By | N/a | List of labels to group by. Labels must be unique. Special label "..." (aggregate by all possible labels), if provided, must be the only element in the list. Accepts a list of strings. See the [upstream documentation](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#route) for details. |
| Group Wait | 30s | How long to wait to buffer alerts of the same group before sending initially. |
| Group Interval | 5m | How long to wait before sending an alert that has been added to a group of alerts for which an initial notification has already been sent. |
| Repeat Interval | 4h | How long to wait before re-sending a given alert that has already been sent. |
### Matching
The **Match** field refers to a set of equality matchers used to identify which alerts to send to a given Route based on labels defined on that alert. When you add key-value pairs to the Rancher UI, they correspond to the YAML in this format:
```yaml
match:
[ <labelname>: <labelvalue>, ... ]
```
The **Match Regex** field refers to a set of regex-matchers used to identify which alerts to send to a given Route based on labels defined on that alert. When you add key-value pairs in the Rancher UI, they correspond to the YAML in this format:
```yaml
match_re:
[ <labelname>: <regex>, ... ]
```
versioned_docs/version-2.14/reference-guides/monitoring-v2-configuration/servicemonitors-and-podmonitors.md
+37
View File
@@ -0,0 +1,37 @@
---
title: ServiceMonitor and PodMonitor Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/servicemonitors-and-podmonitors"/>
</head>
ServiceMonitors and PodMonitors are both pseudo-CRDs that map the scrape configuration of the Prometheus custom resource.
These configuration objects declaratively specify the endpoints that Prometheus will scrape metrics from.
ServiceMonitors are more commonly used than PodMonitors, and we recommend them for most use cases.
:::note
This section assumes familiarity with how monitoring components work together. For more information about Alertmanager, see [this section.](../../integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)
:::
## ServiceMonitors
This pseudo-CRD maps to a section of the Prometheus custom resource configuration. It declaratively specifies how groups of Kubernetes services should be monitored.
When a ServiceMonitor is created, the Prometheus Operator updates the Prometheus scrape configuration to include the ServiceMonitor configuration. Then Prometheus begins scraping metrics from the endpoint defined in the ServiceMonitor.
Any Services in your cluster that match the labels located within the ServiceMonitor `selector` field will be monitored based on the `endpoints` specified on the ServiceMonitor. For more information on what fields can be specified, please look at the [spec](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#servicemonitor) provided by Prometheus Operator.
For more information about how ServiceMonitors work, refer to the [Prometheus Operator documentation.](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/running-exporters.md)
## PodMonitors
This pseudo-CRD maps to a section of the Prometheus custom resource configuration. It declaratively specifies how group of pods should be monitored.
When a PodMonitor is created, the Prometheus Operator updates the Prometheus scrape configuration to include the PodMonitor configuration. Then Prometheus begins scraping metrics from the endpoint defined in the PodMonitor.
Any Pods in your cluster that match the labels located within the PodMonitor `selector` field will be monitored based on the `podMetricsEndpoints` specified on the PodMonitor. For more information on what fields can be specified, please look at the [spec](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#podmonitor) provided by Prometheus Operator.
versioned_docs/version-2.14/reference-guides/prometheus-federator/prometheus-federator.md
+108
View File
@@ -0,0 +1,108 @@
---
title: Prometheus Federator
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/prometheus-federator"/>
</head>
Prometheus Federator, also referred to as Project Monitoring v2, deploys a Helm Project Operator (based on the [rancher/helm-project-operator](https://github.com/rancher/helm-project-operator)), an operator that manages deploying Helm charts each containing a Project Monitoring Stack, where each stack contains:
- [Prometheus](https://prometheus.io/) (managed externally by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator))
- [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) (managed externally by [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator))
- [Grafana](https://github.com/helm/charts/tree/master/stable/grafana) (deployed via an embedded Helm chart)
- Default PrometheusRules and Grafana dashboards based on the collection of community-curated resources from [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus/)
- Default ServiceMonitors that watch the deployed resources
:::note Important:
Prometheus Federator is designed to be deployed alongside an existing Prometheus Operator deployment in a cluster that has already installed the Prometheus Operator CRDs.
:::
## How does the operator work?
1. On deploying this chart, users can create ProjectHelmCharts CRs with `spec.helmApiVersion` set to `monitoring.cattle.io/v1alpha1` (also known as "Project Monitors" in the Rancher UI) in a **Project Registration Namespace (`cattle-project-<id>`)**.
2. On seeing each ProjectHelmChartCR, the operator will automatically deploy a Project Prometheus stack on the Project Owner's behalf in the **Project Release Namespace (`cattle-project-<id>-monitoring`)** based on a HelmChart CR and a HelmRelease CR automatically created by the ProjectHelmChart controller in the **Operator / System Namespace**.
3. RBAC will automatically be assigned in the Project Release Namespace to allow users to view the Prometheus, Alertmanager, and Grafana UIs of the Project Monitoring Stack deployed; this will be based on RBAC defined on the Project Registration Namespace against the [default Kubernetes user-facing roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles). For more information, see the section on [configuring RBAC](rbac.md).
## What is a Project?
In Prometheus Federator, a Project is a group of namespaces that can be identified by a `metav1.LabelSelector`. By default, the label used to identify projects is `field.cattle.io/projectId`, the label used to identify namespaces that are contained within a given Rancher Project.
## Configuring the Helm release created by a ProjectHelmChart
The `spec.values` of this ProjectHelmChart's resources will correspond to the `values.yaml` override to be supplied to the underlying Helm chart deployed by the operator on the user's behalf; to see the underlying chart's `values.yaml` spec, either:
- View the chart's definition located at [`rancher/prometheus-federator` under `charts/rancher-project-monitoring`](https://github.com/rancher/prometheus-federator/blob/main/charts/rancher-project-monitoring) (where the chart version will be tied to the version of this operator).
- Look for the ConfigMap named `monitoring.cattle.io.v1alpha1` that is automatically created in each Project Registration Namespace, which will contain both the `values.yaml` and `questions.yaml` that was used to configure the chart (which was embedded directly into the `prometheus-federator` binary).
## Namespaces
As a Project Operator based on [rancher/helm-project-operator](https://github.com/rancher/helm-project-operator), Prometheus Federator has three different classifications of namespaces that the operator looks out for:
1. **Operator / System Namespace**: The namespace that the operator is deployed into (e.g., `cattle-monitoring-system`). This namespace will contain all HelmCharts and HelmReleases for all ProjectHelmCharts watched by this operator. **Only Cluster Admins should have access to this namespace.**
2. **Project Registration Namespace (`cattle-project-<id>`)**: The set of namespaces that the operator watches for ProjectHelmCharts within. The RoleBindings and ClusterRoleBindings that apply to this namespace will also be the source of truth for the auto-assigned RBAC created in the Project Release Namespace. For details, refer to the [RBAC page](rbac.md). **Project Owners (admin), Project Members (edit), and Read-Only Members (view) should have access to this namespace.**
:::note Notes:
- Project Registration Namespaces will be auto-generated by the operator and imported into the Project it is tied to if `.Values.global.cattle.projectLabel` is provided, which is set to `field.cattle.io/projectId` by default. This indicates that a Project Registration Namespace should be created by the operator if at least one namespace is observed with that label. The operator will not let these namespaces be deleted unless either all namespaces with that label are gone (e.g., this is the last namespace in that project, in which case the namespace will be marked with the label `"helm.cattle.io/helm-project-operator-orphaned": "true"`, which signals that it can be deleted), or it is no longer watching that project because the project ID was provided under `.Values.helmProjectOperator.otherSystemProjectLabelValues`, which serves as a denylist for Projects. These namespaces will also never be auto-deleted to avoid destroying user data; it is recommended that users clean up these namespaces manually if desired on creating or deleting a project.
- If `.Values.global.cattle.projectLabel` is not provided, the Operator / System Namespace will also be the Project Registration Namespace.
:::
3. **Project Release Namespace (`cattle-project-<id>-monitoring`):** The set of namespaces that the operator deploys Project Monitoring Stacks within on behalf of a ProjectHelmChart; the operator will also automatically assign RBAC to Roles created in this namespace by the Project Monitoring Stack based on bindings found in the Project Registration Namespace. **Only Cluster Admins should have access to this namespace; Project Owners (admin), Project Members (edit), and Read-Only Members (view) will be assigned limited access to this namespace by the deployed Helm Chart and Prometheus Federator.**
:::note Notes:
- Project Release Namespaces are automatically deployed and imported into the project whose ID is specified under `.Values.helmProjectOperator.projectReleaseNamespaces.labelValue`, which defaults to the value of `.Values.global.cattle.systemProjectId` if not specified, whenever a ProjectHelmChart is specified in a Project Registration Namespace.
- Project Release Namespaces follow the same orphaning conventions as Project Registration Namespaces (see note above).
- If `.Values.projectReleaseNamespaces.enabled` is false, the Project Release Namespace will be the same as the Project Registration Namespace.
:::
## Helm Resources (HelmChart, HelmRelease)
On deploying a ProjectHelmChart, the Prometheus Federator will automatically create and manage two child custom resources that manage the underlying Helm resources in turn:
- A HelmChart CR (managed via an embedded [k3s-io/helm-contoller](https://github.com/k3s-io/helm-controller) in the operator): This custom resource automatically creates a Job in the same namespace that triggers a `helm install`, `helm upgrade`, or `helm uninstall` depending on the change applied to the HelmChart CR. This CR is automatically updated on changes to the ProjectHelmChart (e.g., modifying the values.yaml) or changes to the underlying Project definition (e.g., adding or removing namespaces from a project).
:::note Important:
If a ProjectHelmChart is not deploying or updating the underlying Project Monitoring Stack for some reason, the Job created by this resource in the Operator / System namespace should be the first place you check to see if there's something wrong with the Helm operation. However, this is generally only accessible by a **Cluster Admin.**
:::
- A HelmRelease CR (managed via an embedded [rancher/helm-locker](https://github.com/rancher/helm-locker) in the operator): This custom resource automatically locks a deployed Helm release in place and automatically overwrites updates to underlying resources unless the change happens via a Helm operation (`helm install`, `helm upgrade`, or `helm uninstall` performed by the HelmChart CR).
:::note
HelmRelease CRs emit Kubernetes Events that detect when an underlying Helm release is being modified and locks it back to place. To view these events, you can use `kubectl describe helmrelease <helm-release-name> -n <operator/system-namespace>`; you can also view the logs on this operator to see when changes are detected and which resources modifications were attempted on.
:::
Both of these resources are created for all Helm charts in the Operator / System namespaces to avoid escalation of privileges to underprivileged users.
### Advanced Helm Project Operator Configuration
For more information on advanced configurations, refer to [this page](https://github.com/rancher/prometheus-federator/blob/main/charts/prometheus-federator/README.md#advanced-helm-project-operator-configuration).
<!--
|Value|Configuration|
|---|---------------------------|
|`helmProjectOperator.valuesOverride`| Allows an Operator to override values that are set on each ProjectHelmChart deployment on an operator-level; user-provided options (specified on the `spec.values` of the ProjectHelmChart) are automatically overridden if operator-level values are provided. For an example, see how the default value overrides `federate.targets`. Note: When overriding list values like `federate.targets`, user-provided list values will **not** be concatenated. |
|`helmProjectOperator.projectReleaseNamespaces.labelValues`| The value of the Project that all Project Release Namespaces should be auto-imported into via label and annotation. Not recommended to be overridden on a Rancher setup. |
|`helmProjectOperator.otherSystemProjectLabelValues`| Other namespaces that the operator should treat as a system namespace that should not be monitored. By default, all namespaces that match `global.cattle.systemProjectId` will not be matched. `cattle-monitoring-system`, `cattle-dashboards`, and `kube-system` are explicitly marked as system namespaces as well, regardless of label or annotation. |
|`helmProjectOperator.releaseRoleBindings.aggregate`| Whether to automatically create RBAC resources in Project Release namespaces.
|`helmProjectOperator.releaseRoleBindings.clusterRoleRefs.<admin\|edit\|view>`| ClusterRoles to reference to discover subjects to create RoleBindings for in the Project Release Namespace for all corresponding Project Release Roles. See RBAC above for more information. |
|`helmProjectOperator.hardenedNamespaces.enabled`| Whether to automatically patch the default ServiceAccount with `automountServiceAccountToken: false` and create a default NetworkPolicy in all managed namespaces in the cluster; the default values ensure that the creation of the namespace does not break a CIS 1.16 hardened scan. |
|`helmProjectOperator.hardenedNamespaces.configuration`| The configuration to be supplied to the default ServiceAccount or auto-generated NetworkPolicy on managing a namespace. |
-->
## Prometheus Federator on the Local Cluster
Prometheus Federator is a resource intensive application. Installing it to the local cluster is possible, but **not recommended**.
versioned_docs/version-2.14/reference-guides/prometheus-federator/rbac.md
+35
View File
@@ -0,0 +1,35 @@
---
title: Role-Based Access Control
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/prometheus-federator/rbac"/>
</head>
This section describes the expectations for Role-Based Access Control (RBAC) for Prometheus Federator.
As described in the section on [namespaces](prometheus-federator.md#namespaces), Prometheus Federator expects that Project Owners, Project Members, and other users in the cluster with Project-level permissions (e.g. permissions in a certain set of namespaces identified by a single label selector) have minimal permissions in any namespaces except the Project Registration Namespace (which is imported into the project by default) and those that already comprise their projects. Therefore, in order to allow Project Owners to assign specific chart permissions to other users in their Project namespaces, the Helm Project Operator will automatically watch the following bindings:
- ClusterRoleBindings
- RoleBindings in the Project Release Namespace
On observing a change to one of those types of bindings, the Helm Project Operator will check whether the `roleRef` that the the binding points to matches a ClusterRole with the name provided under:
- `helmProjectOperator.releaseRoleBindings.clusterRoleRefs.admin`
- `helmProjectOperator.releaseRoleBindings.clusterRoleRefs.edit`
- `helmProjectOperator.releaseRoleBindings.clusterRoleRefs.view`
By default, these roleRefs will correspond to `admin`, `edit`, and `view` respectively, which are the [default Kubernetes user-facing roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles).
:::note
For Rancher RBAC users, these [default Kubernetes user-facing roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) directly correlate to the `Project Owner`, `Project Member`, and `Read-Only` default Project Role Templates.
:::
If the `roleRef` matches, the Helm Project Operator will filter the `subjects` of the binding for all Users and Groups and use that to automatically construct a RoleBinding for each Role in the Project Release Namespace with the same name as the role and the following labels:
- `helm.cattle.io/project-helm-chart-role: {{ .Release.Name }}`
- `helm.cattle.io/project-helm-chart-role-aggregate-from: <admin|edit|view>`
By default, `rancher-project-monitoring`, the underlying chart deployed by Prometheus Federator, creates three default Roles per Project Release Namespace that provide `admin`, `edit`, and `view` users to permissions to view the Prometheus, Alertmanager, and Grafana UIs of the Project Monitoring Stack to provide least privilege. However, if a Cluster Admin would like to assign additional permissions to certain users, they can either directly assign RoleBindings in the Project Release Namespace to certain users or create Roles with the above two labels on them to allow Project Owners to control assigning those RBAC roles to users in their Project Registration namespaces.
versioned_docs/version-2.14/reference-guides/rancher-cluster-tools.md
+49
View File
@@ -0,0 +1,49 @@
---
title: Cluster Tools for Logging, Monitoring, and Visibility
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-cluster-tools"/>
</head>
Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently. Tools are divided into following categories:
## Logging
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
For more information, refer to the logging documentation [here.](../integrations-in-rancher/logging/logging.md)
## Monitoring and Alerts
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
After monitoring is enabled, you can set up alerts and notifiers that provide the mechanism to receive them.
Notifiers are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Notifications can be sent with Slack, email, PagerDuty, WeChat, and webhooks.
Alerts are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
For more information, refer to the monitoring documentation [here.](../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md)
## Istio
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices.
Rancher's integration with Istio was improved in Rancher v2.5.
For more information, refer to the Istio documentation [here.](../integrations-in-rancher/istio/istio.md)
## Compliance Scans
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in most recognized Kubernetes Security Benchmarks, such as STIG.
For more information, refer to the Compliance scan documentation [here.](../how-to-guides/advanced-user-guides/compliance-scan-guides/compliance-scan-guides.md)
versioned_docs/version-2.14/reference-guides/rancher-manager-architecture/architecture-recommendations.md
+70
View File
@@ -0,0 +1,70 @@
---
title: Architecture Recommendations
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-manager-architecture/architecture-recommendations"/>
</head>
If you are installing Rancher on a single node, the main architecture recommendation that applies to your installation is that the node running Rancher should be [separate from downstream clusters.](#separation-of-rancher-and-user-clusters)
## Separation of Rancher and User Clusters
A user cluster is a downstream Kubernetes cluster that runs your apps and services.
If you have a Docker installation of Rancher, the node running the Rancher server should be separate from your downstream clusters.
If Rancher is intended to manage downstream Kubernetes clusters, the Kubernetes cluster that the Rancher server runs on should also be separate from the downstream user clusters.
![Separation of Rancher Server from User Clusters](/img/rancher-architecture-separation-of-rancher-server.svg)
## Why HA is Better for Rancher in Production
We recommend installing the Rancher server on a high-availability Kubernetes cluster, primarily because it protects the Rancher server data. In a high-availability installation, a load balancer serves as the single point of contact for clients, distributing network traffic across multiple servers in the cluster and helping to prevent any one server from becoming a point of failure.
We don't recommend installing Rancher in a single Docker container, because if the node goes down, there is no copy of the cluster data available on other nodes and you could lose the data on your Rancher server.
### K3s Kubernetes Cluster Installations
One option for the underlying Kubernetes cluster is to use K3s Kubernetes. K3s is Rancher's CNCF certified Kubernetes distribution. It is easy to install and uses half the memory of Kubernetes, all in a binary of less than 100 MB. Another advantage of K3s is that it allows an external datastore to hold the cluster data, allowing the K3s server nodes to be treated as ephemeral.
<figcaption>Architecture of a K3s Kubernetes Cluster Running the Rancher Management Server</figcaption>
![Architecture of a K3s Kubernetes Cluster Running the Rancher Management Server](/img/k3s-server-storage.svg)
## Recommended Load Balancer Configuration for Kubernetes Installations
We recommend the following configurations for the load balancer and Ingress controllers:
* The DNS for Rancher should resolve to a Layer 4 load balancer (TCP).
* The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster.
* The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443.
* The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment.
<figcaption>Rancher installed on a Kubernetes cluster with layer 4 load balancer, depicting SSL termination at Ingress controllers</figcaption>
![Rancher HA](/img/ha/rancher2ha.svg)
## Environment for Kubernetes Installations
It is strongly recommended to install Rancher on a Kubernetes cluster on hosted infrastructure such as Amazon's EC2 or Google Compute Engine.
For the best performance and greater security, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) for running your workloads.
## Recommended Node Roles for Kubernetes Installations
The below recommendations apply when Rancher is installed on a K3s Kubernetes cluster.
### K3s Cluster Roles
In K3s clusters, there are two types of nodes: server nodes and agent nodes. Both servers and agents can have workloads scheduled on them. Server nodes run the Kubernetes master.
For the cluster running the Rancher management server, we recommend using two server nodes. Agent nodes are not required.
## Architecture for an Authorized Cluster Endpoint (ACE)
If you are using an [authorized cluster endpoint (ACE),](../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md) and [API keys](../user-settings/api-keys.md#creating-an-api-key) for more information.
ACE support is available for registered RKE2 and K3s clusters. To view the manual steps to perform on the downstream cluster to enable the ACE, click [here](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md#authorized-cluster-endpoint-support-for-rke2-and-k3s-clusters).
versioned_docs/version-2.14/reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md
+222
View File
@@ -0,0 +1,222 @@
---
title: Communicating with Downstream User Clusters
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters"/>
</head>
This section describes how Rancher provisions and manages the downstream user clusters that run your apps and services.
The below diagram shows how the cluster controllers, cluster agents, and node agents allow Rancher to control downstream clusters.
<figcaption>Communicating with Downstream Clusters</figcaption>
![Rancher Components](/img/rancher-architecture-cluster-controller.svg)
The following descriptions correspond to the numbers in the diagram above:
1. [The Authentication Proxy](#1-the-authentication-proxy)
2. [Cluster Controllers and Cluster Agents](#2-cluster-controllers-and-cluster-agents)
3. [Node Agents](#3-node-agents)
4. [Authorized Cluster Endpoint](#4-authorized-cluster-endpoint)
## 1. The Authentication Proxy
In this diagram, a user named Bob wants to see all pods running on a downstream user cluster called User Cluster 1. From within Rancher, he can run a `kubectl` command to see
the pods. Bob is authenticated through Rancher's authentication proxy.
The authentication proxy forwards all Kubernetes API calls to downstream clusters. It integrates with authentication services like local authentication, Active Directory, and GitHub. On every Kubernetes API call, the authentication proxy authenticates the caller and sets the proper Kubernetes impersonation headers before forwarding the call to Kubernetes masters.
Rancher communicates with Kubernetes clusters using a [service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/). Each user account in Rancher correlates with an equivalent service account in the downstream cluster. Rancher uses the service account to [impersonate](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation) the user, which provides all the permissions the user is intended to have.
By default, Rancher generates a [kubeconfig file](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md) that contains credentials for proxying through the Rancher server to connect to the Kubernetes API server on a downstream user cluster. The kubeconfig file (`kube_config_rancher-cluster.yml`) contains full access to the cluster.
## 2. Cluster Controllers and Cluster Agents
Each downstream user cluster has a cluster agent, which opens a tunnel to the corresponding cluster controller within the Rancher server.
There is one cluster controller and one cluster agent for each downstream cluster. Each cluster controller:
- Watches for resource changes in the downstream cluster
- Brings the current state of the downstream cluster to the desired state
- Configures access control policies to clusters and projects
- Provisions clusters by calling the required Docker machine drivers and Kubernetes engines, such as GKE
By default, to enable Rancher to communicate with a downstream cluster, the cluster controller connects to the cluster agent. If the cluster agent is not available, the cluster controller can connect to a [node agent](#3-node-agents) instead.
The cluster agent, also called `cattle-cluster-agent`, is a component that runs in a downstream user cluster. It performs the following tasks:
- Connects to the Kubernetes API of Rancher-launched Kubernetes clusters
- Manages workloads, pod creation and deployment within each cluster
- Applies the roles and bindings defined in each cluster's global policies
- Communicates between the cluster and Rancher server (through a tunnel to the cluster controller) about events, stats, node info, and health
## 3. Node Agents
If the cluster agent (also called `cattle-cluster-agent`) is not available, one of the node agents creates a tunnel to the cluster controller to communicate with Rancher.
The `cattle-node-agent` is deployed using a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) resource to make sure it runs on every node in a Rancher-launched Kubernetes cluster. It is used to interact with the nodes when performing cluster operations. Examples of cluster operations include upgrading the Kubernetes version and creating or restoring etcd snapshots.
## 4. Authorized Cluster Endpoint
An authorized cluster endpoint (ACE) allows users to connect to the Kubernetes API server of a downstream cluster without having to route their requests through the Rancher authentication proxy.
> ACE is available on RKE2 and K3s clusters that are provisioned or registered with Rancher. It's not available on clusters in a hosted Kubernetes provider, such as Amazon's EKS.
There are two main reasons why a user might need the authorized cluster endpoint:
- To access a downstream user cluster while Rancher is down
- To reduce latency in situations where the Rancher server and downstream cluster are separated by a long distance
The `kube-api-auth` microservice is deployed to provide the user authentication functionality for the authorized cluster endpoint. When you access the user cluster using `kubectl`, the cluster's Kubernetes API server authenticates you by using the `kube-api-auth` service as a webhook.
Like the authorized cluster endpoint, the `kube-api-auth` authentication service is also only available for Rancher-launched Kubernetes clusters.
> **Example scenario:** Let's say that the Rancher server is located in the United States, and User Cluster 1 is located in Australia. A user, Alice, also lives in Australia. Alice can manipulate resources in User Cluster 1 by using the Rancher UI, but her requests will have to be sent from Australia to the Rancher server in the United States, then be proxied back to Australia, where the downstream user cluster is. The geographical distance may cause significant latency, which Alice can reduce by using the authorized cluster endpoint.
With this endpoint enabled for the downstream cluster, Rancher generates an extra Kubernetes context in the kubeconfig file in order to connect directly to the cluster. This file has the credentials for `kubectl` and `helm`.
:::note
To use the ACE context in your kubeconfig, run `kubectl use-context <ace context name>` after enabling it.
:::
For more information, refer to the section on accessing your cluster with [kubectl and the kubeconfig file](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md).
We recommend exporting the kubeconfig file so that if Rancher goes down, you can still use the credentials in the file to access your cluster.
## Impersonation
Users technically exist only on the upstream cluster. Rancher creates [RoleBindings and ClusterRoleBindings](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) that refer to Rancher users, even though there is [no actual User resource](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#users-in-kubernetes) on the downstream cluster.
When users interact with a downstream cluster through the authentication proxy, there needs to be some entity downstream to serve as the actor for those requests. Rancher creates service accounts to be that entity. Each service account is only granted one permission, which is to **impersonate** the user they belong to. If there was only one service account that could impersonate any user, then it would be possible for a malicious user to corrupt that account and escalate their privileges by impersonating another user. This issue was the basis for a [CVE](https://github.com/rancher/rancher/security/advisories/GHSA-pvxj-25m6-7vqr).
### Impersonation Troubleshooting
On the downstream cluster, five resources handle impersonation:
- namespace: `cattle-impersonation-system`
- service account: `cattle-impersonation-system/cattle-impersonation-<user ID>`
- account token secret: `cattle-impersonation-system/cattle-impersonation-<user ID>-token-<hash>`
- cluster role: `cattle-impersonation-<user ID>`
- cluster role binding: `cattle-impersonation-<user ID>`
In this example of a typical impersonation cluster role, the system is configured to use `github` as the auth provider:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: "2021-10-06T18:20:13Z"
labels:
authz.cluster.cattle.io/impersonator: "true"
cattle.io/creator: norman
name: cattle-impersonation-user-abcde
resourceVersion: "3528"
uid: a7478731-72a0-4343-b09f-c3bf12552d77
rules:
# allowed to impersonate user user-abcde
- apiGroups:
- ""
resourceNames:
- user-abcde
resources:
- users
verbs:
- impersonate
# allowed to impersonate listed groups
- apiGroups:
- ""
resourceNames:
- github_team://123 # group from GitHub auth provider
- system:authenticated # automatic group from Kubernetes
- system:cattle:authenticated # automatic group from Rancher
resources:
- groups
verbs:
- impersonate
# allowed to impersonate principal ID github_user://098
- apiGroups:
- authentication.k8s.io
resourceNames:
- github_user://098 # principal ID from GitHub auth provider
resources:
- userextras/principalid
verbs:
- impersonate
# allowed to impersonate username example
- apiGroups:
- authentication.k8s.io
resourceNames:
- example # username from GitHub auth provider
resources:
- userextras/username
verbs:
- impersonate
```
When you troubleshoot impersonation issues, check whether these resources exist for the user, and whether the rules in the cluster role look similar to the above. For example:
```bash
kubectl --namespace cattle-impersonation-system get serviceaccount cattle-impersonation-<user ID>
kubectl --namespace cattle-impersonation-system get secret cattle-impersonation-<user ID>-token-<hash>
kubectl get clusterrole cattle-impersonation-<user ID> --output yaml
kubectl get clusterrolebinding cattle-impersonation-<user ID>
```
If you see an error related to "impersonation" in the UI, pay close attention to the *end* of the error message, which should indicate the real reason that the request failed.
## Important Files
The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster:
- `config.yaml`: The RKE2 and K3s cluster configuration file.
- `rke2.yaml` or `k3s.yaml`: The Kubeconfig file for your RKE2 or K3s cluster. This file contains credentials for full access to the cluster. You can use this file to authenticate with a Rancher-launched Kubernetes cluster if Rancher goes down.
For more information on connecting to a cluster without the Rancher authentication proxy and other configuration options, refer to the [kubeconfig file](../../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md) documentation.
## Tools for Provisioning Kubernetes Clusters
The tools that Rancher uses to provision downstream user clusters depends on the type of cluster that is being provisioned.
### Rancher Launched Kubernetes for Nodes Hosted in an Infrastructure Provider
Rancher can dynamically provision nodes in a provider such as Amazon EC2, DigitalOcean, Azure, or vSphere, then install Kubernetes on them.
Rancher provisions this type of cluster using [docker-machine.](https://github.com/rancher/machine)
### Rancher Launched Kubernetes for Custom Nodes
When setting up this type of cluster, Rancher installs Kubernetes on existing nodes, which creates a custom cluster.
Rancher provisions this type of cluster using [RKE2](https://github.com/rancher/rke2) or [K3s](https://github.com/rancher/k3s).
### Hosted Kubernetes Providers
When setting up this type of cluster, Kubernetes is installed by providers such as Google Kubernetes Engine, Amazon Elastic Container Service for Kubernetes, or Azure Kubernetes Service.
Rancher provisions this type of cluster using [kontainer-engine.](https://github.com/rancher/kontainer-engine)
### Imported Kubernetes Clusters
In this type of cluster, Rancher connects to a Kubernetes cluster that has already been set up. Therefore, Rancher does not provision Kubernetes, but only sets up the Rancher agents to communicate with the cluster.
## Rancher Server Components and Source Code
This diagram shows each component that the Rancher server is composed of:
![Rancher Components](/img/rancher-architecture-rancher-components.svg)
The GitHub repositories for Rancher can be found at the following links:
- [Main Rancher server repository](https://github.com/rancher/rancher)
- [Rancher UI](https://github.com/rancher/ui)
- [Rancher API UI](https://github.com/rancher/api-ui)
- [Norman,](https://github.com/rancher/norman) Rancher's API framework
- [Types](https://github.com/rancher/types)
- [Rancher CLI](https://github.com/rancher/cli)
- [Catalog applications](https://github.com/rancher/helm)
This is a partial list of the most important Rancher repositories. For more details about Rancher source code, refer to the section on [contributing to Rancher.](../../contribute-to-rancher.md#rancher-repositories) To see all libraries and projects used in Rancher, see the [`go.mod` file](https://github.com/rancher/rancher/blob/master/go.mod) in the `rancher/rancher` repository.
versioned_docs/version-2.14/reference-guides/rancher-manager-architecture/rancher-manager-architecture.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Rancher Architecture
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-manager-architecture"/>
</head>
This section focuses on the [Rancher server and its components](rancher-server-and-components.md) and how [Rancher communicates with downstream Kubernetes clusters](communicating-with-downstream-user-clusters.md).
For information on the different ways that Rancher can be installed, refer to the [overview of installation options.](../../getting-started/installation-and-upgrade/installation-and-upgrade.md#overview-of-installation-options)
For a list of main features of the Rancher API server, refer to the [overview section.](../../getting-started/overview.md#features-of-the-rancher-api-server)
For guidance about setting up the underlying infrastructure for the Rancher server, refer to the [architecture recommendations.](architecture-recommendations.md)
:::note
This section assumes a basic familiarity with Docker and Kubernetes. For a brief explanation of how Kubernetes components work together, refer to the [concepts](../kubernetes-concepts.md) page.
:::
versioned_docs/version-2.14/reference-guides/rancher-manager-architecture/rancher-server-and-components.md
+39
View File
@@ -0,0 +1,39 @@
---
title: Rancher Server and Components
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-manager-architecture/rancher-server-and-components"/>
</head>
The majority of Rancher 2.x software runs on the Rancher Server. Rancher Server includes all the software components used to manage the entire Rancher deployment.
The diagram below illustrates the high-level architecture of Rancher 2.x. The figure depicts a Rancher Server installation that manages two downstream Kubernetes clusters: one created by RKE2 and another created by Amazon EKS (Elastic Kubernetes Service).
For the best performance and security, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) for running your workloads.
The diagram shows how users can manipulate both [Rancher-launched Kubernetes](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) clusters and [hosted Kubernetes](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/set-up-clusters-from-hosted-kubernetes-providers.md) clusters through Rancher's authentication proxy:
<figcaption>Managing Kubernetes Clusters through Rancher's Authentication Proxy</figcaption>
![Architecture](/img/rancher-architecture-rancher-api-server.svg)
You can install Rancher on a single node, or on a high-availability Kubernetes cluster.
A high-availability Kubernetes installation is recommended for production.
A Docker installation of Rancher is recommended only for development and testing purposes. The ability to migrate Rancher to a high-availability cluster depends on the Rancher version.
To see deployed resources, run the following command:
```bash
kubectl get all -n <namespace>
```
If you have administrator privileges, you can also see these resources listed in the Rancher UI:
1. Click **☰** and select a cluster.
1. Select **More Resources** from the side navigation menu to view deployed resources by type.
1. Select **Cluster** > **Projects/Namespaces** from the side navigation menu, then select a namespace, to view deployed resources by namespace.
The Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster](../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md).
The Rancher server, regardless of the installation method, should always run on nodes that are separate from the downstream user clusters that it manages. If Rancher is installed on a high-availability Kubernetes cluster, it should run on a separate cluster from the cluster(s) it manages.
versioned_docs/version-2.14/reference-guides/rancher-project-tools.md
+36
View File
@@ -0,0 +1,36 @@
---
title: Project Tools for Logging, Monitoring, and Visibility
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-project-tools"/>
</head>
Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently.
## Notifiers and Alerts
Notifiers and alerts are two features that work together to inform you of events in the Rancher system. Before they can be enabled, the monitoring application must be installed.
Notifiers are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Notifications can be sent with Slack, email, PagerDuty, WeChat, and webhooks.
Alerts are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
## Logging
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
For details, refer to the [logging section.](../integrations-in-rancher/logging/logging.md)
## Monitoring
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.](../integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md)
versioned_docs/version-2.14/reference-guides/rancher-security/psa-restricted-exemptions.md
+69
View File
@@ -0,0 +1,69 @@
---
title: Sample PodSecurityConfiguration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/psa-restricted-exemptions"/>
</head>
The following PodSecurityConfiguration contains the required Rancher namespace exemptions for a `rancher-restricted` cluster to run properly.
```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1
kind: PodSecurityConfiguration
defaults:
enforce: "restricted"
enforce-version: "latest"
audit: "restricted"
audit-version: "latest"
warn: "restricted"
warn-version: "latest"
exemptions:
usernames: []
runtimeClasses: []
namespaces: [calico-apiserver,
calico-system,
cattle-alerting,
cattle-csp-adapter-system,
cattle-elemental-system,
cattle-epinio-system,
cattle-externalip-system,
cattle-fleet-local-system,
cattle-fleet-system,
cattle-gatekeeper-system,
cattle-global-data,
cattle-global-nt,
cattle-impersonation-system,
cattle-istio,
cattle-istio-system,
cattle-logging,
cattle-logging-system,
cattle-monitoring-system,
cattle-neuvector-system,
cattle-prometheus,
cattle-provisioning-capi-system,
cattle-resources-system,
cattle-sriov-system,
cattle-system,
cattle-ui-plugin-system,
cattle-windows-gmsa-system,
cert-manager,
cis-operator-system,
fleet-default,
fleet-local,
ingress-nginx,
istio-system,
kube-node-lease,
kube-public,
kube-system,
longhorn-system,
rancher-alerting-drivers,
security-scan,
sr-operator-system,
tigera-operator]
```
versioned_docs/version-2.14/reference-guides/rancher-security/rancher-security-best-practices.md
+94
View File
@@ -0,0 +1,94 @@
---
title: Rancher Security Best Practices
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/rancher-security-best-practices"/>
</head>
## Restrict Public Access to /version and /rancherversion Path
The upstream (local) Rancher instance provides information about the Rancher version it is running and the Go version that was used to build it. That information is accessible via the `/version` path, which is used for tasks such as automating version bumps, or confirming that a deployment was successful. The upstream instance also provides Rancher version information accessible via the `/rancherversion` path.
Adversaries can misuse this information to identify the running Rancher version and cross-relate it with potential bugs to exploit. If your upstream Rancher instance is publicly available on the web, use a Layer 7 firewall to block `/version` and `/rancherversion`.
See [OWASP Web Application Security Testing - Enumerate Infrastructure and Application Admin Interfaces](https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/02-Configuration_and_Deployment_Management_Testing/05-Enumerate_Infrastructure_and_Application_Admin_Interfaces.html) for more information on protecting your server.
## Session Management
Some environments may require additional security controls for session management. For example, you may want to limit users' concurrent active sessions or restrict which geolocations those sessions can be initiated from. Such features are not supported by Rancher out of the box.
If you require such features, combine Layer 7 firewalls with [external authentication providers](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md#external-vs-local-authentication).
## Use External Load Balancers to Protect Vulnerable Ports
You should protect the following ports behind an [external load balancer](../../how-to-guides/new-user-guides/kubernetes-resources-setup/load-balancer-and-ingress-controller/layer-4-and-layer-7-load-balancing.md#layer-4-load-balancer) that has SSL offload enabled:
- **K3s:** Port 6443, used by the Kubernetes API.
- **RKE2:** Port 6443, used by the Kubernetes API, and port 9345, used for node registration.
These ports have TLS SAN certificates which list nodes' public IP addresses. An attacker could use that information to gain unauthorized access or monitor activity on the cluster. Protecting these ports helps mitigate against nodes' public IP addresses being disclosed to potential attackers.
## Rancher Username Policy
By default, Kubernetes does not provide enforcement mechanisms for baseline username policies. In Rancher, this means that any enforcement of username formats, naming conventions, or baseline policies is expected to be handled by the external identity provider's policies, if such policies are in place.
In Rancher, `admin` is the default username for the Administrator user, as highlighted [here](../../getting-started/installation-and-upgrade/resources/bootstrap-password.md)
Examples of potential baseline policies include:
- Requiring usernames to follow an organizational convention (e.g., `firstname.lastname`)
- Enforcing minimum or maximum length requirements
- Disallowing certain special characters
- Preventing impersonation by disallowing reserved names (e.g., `admin`, `root`)
Without these controls at the identity provider layer, there is a risk of inconsistent or insecure username practices, which can complicate access audits and lead to privilege escalation attempts.
:::note Important
Rancher currently enforces only a [minimum password length](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/manage-users-and-groups.md#minimum-password-length).
:::
**Recommendation:**
We strongly advice that customers:
- Review and configure username baseline policies directly in their external identity providers (e.g., LDAP, Active Directory, SAML, or OIDC).
- Ensure that those policies align with the organization’s security and compliance requirements.
- Regularly audit user accounts to detect naming inconsistencies or policy violations.
For more information, see:
- [OWASP Cheat Sheet: Authentication](https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html)
- [OWASP Identity Management](https://owasp.org/www-project-top-ten/2017/A2_2017-Broken_Authentication)
## WAF Rules
Rancher is designed to support a wide range of deployment scenarios, including environments where customers may programmatically automate the creation or provisioning of large numbers of clusters. Imposing strict application-level limits within Rancher itself could interfere with legitimate workloads that require dynamic scaling.
For example:
- CI/CD pipelines may create and tear down clusters frequently.
- Self-service portals could provision clusters on-demand for developers.
- Test environments may generate high volumes of API calls.
**Risk:**
Without appropriate rate limiting, adversaries could exploit unauthenticated or authenticated endpoints to:
- Exhaust resources (Denial of Service).
- Inflate storage costs.
- Degrade performance for legitimate users.
**Recommendation:**
The most effective way to mitigate this risk is to implement rate limiting and abuse protection at the infrastructure or Web Application Firewall (WAF) layer. This approach allows thresholds to be tuned for each environment's expected usage and scaling characteristics. Some examples of controls can be:
- Configuring a Web Application Firewall or API Gateway to enforce rate limits on sensitive operations, such as cluster creation and provisioning.
- Defining thresholds based on baseline workload expectations (e.g., max requests per minute per client).
- Monitoring logs and alerting on anomalies to detect potential abuse.
- Apply a resource quota, which is a Rancher feature that limits the resources available to a project or namespace.
For more information, see:
- [Project Resource Quotas](../../how-to-guides/advanced-user-guides/manage-projects/manage-project-resource-quotas/manage-project-resource-quotas.md)
- [OWASP API Security Top 10 - API4:2019 - Lack of Resources & Rate Limiting](https://owasp.org/API-Security/editions/2023/en/0xa4-lack-of-resources-rate-limiting/)
- [OWASP Cheat Sheet: Rate Limiting](https://cheatsheetseries.owasp.org/cheatsheets/Rate_Limiting_Cheat_Sheet.html)
versioned_docs/version-2.14/reference-guides/rancher-security/rancher-security.md
+116
View File
@@ -0,0 +1,116 @@
---
title: Rancher Security Guides
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security"/>
</head>
<table width="100%">
<tr style={{verticalAlign: 'top'}}>
<td width="30%" style={{border: 'none'}}>
<h4>Security policy</h4>
<p style={{padding: '8px'}}>Rancher Labs supports responsible disclosure, and endeavours to resolve all issues in a reasonable time frame. </p>
</td>
<td width="30%" style={{border: 'none'}}>
<h4>Reporting process</h4>
<p style={{padding: '8px'}}>Please submit possible security issues by emailing <a href="mailto:security-rancher@suse.com">security-rancher@suse.com</a> .</p>
</td>
<td width="30%" style={{border: 'none'}}>
<h4>Announcements</h4>
<p style={{padding:'8px'}}>Subscribe to the <a href="https://forums.rancher.com/c/announcements">Rancher announcements forum</a> for release updates.</p>
</td>
</tr>
</table>
Security is at the heart of all Rancher features. From integrating with all the popular authentication tools and services, to an enterprise grade [RBAC capability](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md), Rancher makes your Kubernetes clusters even more secure.
On this page, we provide security related documentation along with resources to help you secure your Rancher installation and your downstream Kubernetes clusters.
## NeuVector Integration with Rancher
NeuVector is an open-source, container-focused security application that is now integrated into Rancher. NeuVector provides production security, DevOps vulnerability protection, and a container firewall, et al. Please see the [Rancher docs](../../integrations-in-rancher/neuvector/neuvector.md) and the [NeuVector docs](https://open-docs.neuvector.com/) for more information.
## Running a Compliance Security Scan on a Kubernetes Cluster
Rancher leverages [kube-bench](https://github.com/aquasecurity/kube-bench) to run a security scan to check whether Kubernetes is deployed according to security best practices.
When Rancher runs a Compliance scan on a cluster, it generates a report showing the results of each test, including a summary with the number of passed, skipped and failed tests. The report also includes remediation steps for any failed tests.
For details, refer to the section on [security scans](../../how-to-guides/advanced-user-guides/compliance-scan-guides/compliance-scan-guides.md).
`
## SELinux RPM
We provide three RPMs (RPM Package Manager) that enable Rancher products to function properly on SELinux-enforcing hosts: `rancher-selinux`, `rke2-selinux` and `k3s-selinux`. For details, see [this page](selinux-rpm/selinux-rpm.md).
## Rancher Hardening Guide
The Rancher Hardening Guide is based on controls and best practices found in the <a href="https://www.cisecurity.org/benchmark/kubernetes/" target="_blank">CIS Kubernetes Benchmark</a> from the Center for Internet Security.
The hardening guides provide prescriptive guidance for hardening a production installation of Rancher. See Rancher's guides for [Self Assessment of the CIS Kubernetes Benchmark](#the-cis-benchmark-and-self-assessment) for the full list of security controls.
> The hardening guides describe how to secure the nodes in your cluster, and it is recommended to follow a hardening guide before installing Kubernetes.
Each version of the hardening guide is intended to be used with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher.
## The CIS Benchmark and Self-Assessment
The benchmark self-assessment is a companion to the Rancher security hardening guide. While the hardening guide shows you how to harden the cluster, the benchmark guide is meant to help you evaluate the level of security of the hardened cluster.
This guide walks through the various controls and provide updated example commands to audit compliance in Rancher created clusters. The original benchmark documents can be downloaded from the [CIS website](https://www.cisecurity.org/benchmark/kubernetes/).
Each version of Rancher's self-assessment guide corresponds to specific versions of the hardening guide, Rancher, Kubernetes, and the CIS Benchmark.
## Third-party Penetration Test Reports
Rancher periodically hires third parties to perform security audits and penetration tests of the Rancher software stack. The environments under test follow the Rancher provided hardening guides at the time of the testing. Previous penetration test reports are available below.
Results:
- [Cure53 Pen Test - July 2019](https://releases.rancher.com/documents/security/pen-tests/2019/RAN-01-cure53-report.final.pdf)
- [Untamed Theory Pen Test - March 2019](https://releases.rancher.com/documents/security/pen-tests/2019/UntamedTheory-Rancher_SecurityAssessment-20190712_v5.pdf)
Please note that new reports are no longer shared or made publicly available.
## Rancher Security Advisories and CVEs
Rancher is committed to informing the community of security issues in our products. For the list of CVEs (Common Vulnerabilities and Exposures) for issues we have resolved, refer to [this page.](security-advisories-and-cves.md)
## Kubernetes Security Best Practices
For recommendations on securing your Kubernetes cluster, refer to the [Kubernetes Cluster Security Best Practices](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/) guide.
## Rancher Security Best Practices
For recommendations on securing your Rancher Manager deployments, refer to the [Rancher Security Best Practices](rancher-security-best-practices.md) guide.
## Rancher Kubernetes Distributions (K3s/RKE2) Self-Assessment and Hardening Guides
Rancher uses the following Kubernetes distributions:
- [**RKE2**](https://docs.rke2.io/) is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.
- [**K3s**](https://docs.k3s.io/) is a fully conformant, lightweight Kubernetes distribution. It is easy to install, with half the memory requirement of upstream Kubernetes, all in a binary of less than 100 MB.
To harden a Kubernetes cluster that's running a distribution other than those listed, refer to your Kubernetes provider's docs.
### Hardening Guides and Benchmark Versions
Each self-assessment guide is accompanied by a hardening guide. These guides were tested alongside the listed Rancher releases. Each self-assessment guide was tested on a specific Kubernetes version and CIS benchmark version. If a CIS benchmark has not been validated for your Kubernetes version, you can use the existing guides until a guide for your version is added.
### RKE2 Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guide |
|------|--------------------|-----------------------|-----------------------|------------------|
| Standalone RKE2 | Kubernetes v1.26 | CIS v1.8 | [Link](https://docs.rke2.io/security/cis_self_assessment18) | [Link](https://docs.rke2.io/security/hardening_guide) |
| Standalone RKE2 | Kubernetes v1.27 | CIS v1.9 | [Link](https://docs.rke2.io/security/cis_self_assessment19) | [Link](https://docs.rke2.io/security/hardening_guide) |
| Standalone RKE2 | Kubernetes v1.28 | CIS v1.10 | [Link](https://docs.rke2.io/security/cis_self_assessment110) | [Link](https://docs.rke2.io/security/hardening_guide) |
| Standalone RKE2 | Kubernetes v1.29 and above | CIS v1.11 | [Link](https://docs.rke2.io/security/cis_self_assessment111) | [Link](https://docs.rke2.io/security/hardening_guide) |
### K3s Guides
| Type | Kubernetes Version | CIS Benchmark Version | Self Assessment Guide | Hardening Guide |
|------|--------------------|-----------------------|-----------------------|------------------|
| Standalone K3s | Kubernetes v1.26 | CIS v1.8 | [Link](https://docs.k3s.io/security/self-assessment-1.8) | [Link](https://docs.k3s.io/security/hardening-guide) |
| Standalone K3s | Kubernetes v1.27 | CIS v1.9 | [Link](https://docs.k3s.io/security/self-assessment-1.9) | [Link](https://docs.k3s.io/security/hardening-guide) |
| Standalone K3s | Kubernetes v1.28 | CIS v1.10 | [Link](https://docs.k3s.io/security/self-assessment-1.10) | [Link](https://docs.k3s.io/security/hardening-guide) |
| Standalone K3s | Kubernetes v1.29 and above | CIS v1.11 | [Link](https://docs.k3s.io/security/self-assessment-1.11) | [Link](https://docs.k3s.io/security/hardening-guide) |
versioned_docs/version-2.14/reference-guides/rancher-security/rancher-webhook-hardening.md
+137
View File
@@ -0,0 +1,137 @@
---
title: Hardening the Rancher Webhook
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/rancher-webhook-hardening"/>
</head>
Rancher Webhook is an important component within Rancher, playing a role in enforcing security requirements for Rancher and its workloads. To decrease its attack surface, access to it should be limited to the only valid caller it has: the Kubernetes API server. This can be done by using network policies and authentication independently or in conjunction with each other to harden the webhook against attacks.
## Block External Traffic Using Network Policies
The webhook is only expected to accept requests from the Kubernetes API server. By default, however, the webhook can accept traffic from any source. If you are using a CNI that supports Network Policies, you can create a policy that blocks traffic that doesn't originate from the API server.
The built-in NetworkPolicy resource in Kubernetes can't block or admit traffic from the cluster hosts, and the `kube-apiserver` process is always running on the host network. Therefore, you must use the advanced network policy resources from the CNI in use. Examples for Calico and Cilium follow. Consult the documentation for your CNI for more details.
### Calico
Use the NetworkPolicy resource in the `crd.projectcalico.org/v1` API group. Use the selector `app == 'rancher-webhook'` to create a rule for the webhook, and set the CIDR of the control plane hosts as the ingress source:
```yaml
apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
name: allow-k8s
namespace: cattle-system
spec:
selector: app == 'rancher-webhook'
types:
- Ingress
ingress:
- action: Allow
protocol: TCP
source:
nets:
- 192.168.42.0/24 # CIDR of the control plane host. May list more than 1 if the hosts are in different subnets.
destination:
selector:
app == 'rancher-webhook'
```
### Cilium
Use the CiliumNetworkPolicy resource in the `cilium.io/v2` API group. Add the `host` and `remote-node` keys to the `fromEntities` ingress rule. This blocks in-cluster and external traffic while allowing traffic from the hosts.
```yaml
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
name: allow-k8s
namespace: cattle-system
spec:
endpointSelector:
matchLabels:
app: rancher-webhook
ingress:
- fromEntities:
- host
- remote-node
```
## Require the Kubernetes API Server to Authenticate to the Webhook
The webhook should only accept requests from the Kubernetes API server. By default, the webhook doesn't require clients to authenticate to it. It will accept any request. You can configure the webhook to require credentials so that only the API server can access it. More information can be found in the [Kubernetes documentation](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#authenticate-apiservers).
1. Configure the API server to present a client certificate to the webhook, pointing to an AdmissionConfiguration file to configure the ValidatingAdmissionWebhook and MutatingAdmissionWebhook plugins:
```yaml
# /etc/rancher/admission/admission.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ValidatingAdmissionWebhook
configuration:
apiVersion: apiserver.config.k8s.io/v1
kind: WebhookAdmissionConfiguration
kubeConfigFile: "/etc/rancher/admission/kubeconfig"
- name: MutatingAdmissionWebhook
configuration:
apiVersion: apiserver.config.k8s.io/v1
kind: WebhookAdmissionConfiguration
kubeConfigFile: "/etc/rancher/admission/kubeconfig"
```
This is also the same config file where other admission plugins are configured, such as PodSecurity. If your distro or your setup uses additional admission plugins, configure those as well. For example, add [RKE2's PodSecurity configuration](https://docs.rke2.io/security/pod_security_standards) to this file.
2. Create the kubeconfig file that the admission plugins refer to. Rancher Webhook only supports client certificate authentication, so generate a TLS key pair, and set the kubeconfig to use either `client-certificate` and `client-key` or `client-certificate-data` and `client-key-data`. For example:
```yaml
# /etc/rancher/admission/kubeconfig
apiVersion: v1
kind: Config
users:
- name: 'rancher-webhook.cattle-system.svc'
user:
client-certificate: /path/to/client/cert
client-key: /path/to/client/key
```
3. Start the kube-apiserver binary with the flag `--admission-control-config-file` pointing to your AdmissionConfiguration file. The way to do this varies by distro, and it isn't supported universally, such as in hosted Kubernetes providers. Consult the documentation for your Kubernetes distribution.
For RKE2, `rke2-server` can be started with a config file like so:
```yaml
# /etc/rancher/rke2/config.yaml
kube-apiserver-arg:
- admission-control-config-file=/etc/rancher/admission/admission.yaml
kube-apiserver-extra-mount:
- /etc/rancher/admission:/etc/rancher/admission:ro
```
:::danger
Some distros set this flag by default. If your distro provisions its own AdmissionConfiguration, you must include it in your custom admission control config file. For example, RKE2 installs an AdmissionConfiguration file at `/etc/rancher/rke2/rke2-pss.yaml`, which configures the PodSecurity admission plugin. Setting `admission-control-config-file` in config.yaml will override this essential security setting. To include both plugins, consult [the Default Pod Security Standards documentation](https://docs.rke2.io/security/pod_security_standards) and copy the appropriate plugin configuration to your admission.yaml.
:::
4. If you're using Rancher to provision your cluster using existing nodes, create these files on the node before you provision them.
If you're using Rancher to provision your cluster on new nodes, allow the provisioning to complete, then use the provided SSH key and IP address to connect to the nodes, and place the RKE2 config file in the `/etc/rancher/rke2/config.yaml.d/` directory.
5. After the cluster is configured with these credentials, configure the Rancher cluster agent to enable authentication in the webhook. Create a file containing these chart values:
```yaml
# values.yaml
auth:
clientCA: <base64-encoded certificate authority that signed the kube-apiserver's client certificates>
allowedCNs:
- <list of Common Names identifying the kube-apiserver's client certificates.>
- <if not provided, any cert signed by the given CA will be accepted.>
```
6. Create a configmap in the `cattle-system` namespace on the provisioned cluster with these values:
```
kubectl --namespace cattle-system create configmap rancher-config --from-file=rancher-webhook=values.yaml
```
The webhook will restart with these values.
versioned_docs/version-2.14/reference-guides/rancher-security/security-advisories-and-cves.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Security Advisories and CVEs
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/security-advisories-and-cves"/>
</head>
Rancher is committed to informing the community of security issues in our products. Rancher will publish security advisories and CVEs (Common Vulnerabilities and Exposures) for issues we have resolved. New security advisories are also published in Rancher's GitHub [security page](https://github.com/rancher/rancher/security/advisories).
| ID | Description | Date | Resolution |
|----|-------------|------|------------|
| [CVE-2025-62879](https://github.com/rancher/backup-restore-operator/security/advisories/GHSA-wj3p-5h3x-c74q) | Rancher now provides new versions of the Rancher Backup chart which prevent the leak of secret S3 credentials via the Rancher Backup pod log. | 29 Jan 2026 | Rancher [v2.13.2](https://github.com/rancher/rancher/releases/tag/v2.13.2), [v2.12.6](https://github.com/rancher/rancher/releases/tag/v2.12.6), [v2.11.10](https://github.com/rancher/rancher/releases/tag/v2.11.10), and [v2.10.11](https://github.com/rancher/rancher/releases/tag/v2.10.11) |
| [CVE-2025-67601](https://github.com/rancher/rancher/security/advisories/GHSA-mc24-7m59-4q5p) | Rancher now removes the ability to fetch CA certificates stored in Rancher’s setting `cacerts` when using the `login` command. | 29 Jan 2026 | Rancher [v2.13.2](https://github.com/rancher/rancher/releases/tag/v2.13.2), [v2.12.6](https://github.com/rancher/rancher/releases/tag/v2.12.6), [v2.11.10](https://github.com/rancher/rancher/releases/tag/v2.11.10), and [v2.10.11](https://github.com/rancher/rancher/releases/tag/v2.10.11) |
| [CVE-2023-32199](https://github.com/rancher/rancher/security/advisories/GHSA-j4vr-pcmw-hx59) | Rancher now removes the corresponding ClusterRoleBindings whenever the admin GlobalRole or its GlobalRoleBindings are deleted. Previously orphaned ClusterRoleBindings were marked with the annotation `authz.cluster.cattle.io/admin-globalrole-missing=true`. | 23 Oct 2025 | Rancher [v2.12.3](https://github.com/rancher/rancher/releases/tag/v2.12.3) and [v2.11.7](https://github.com/rancher/rancher/releases/tag/v2.11.7) |
| [CVE-2024-58269](https://github.com/rancher/rancher/security/advisories/GHSA-mw39-9qc2-f7mg) | The Rancher audit log redaction process has changed to the following: <br/><br/><ul><li> It now redacts `kubectl.kubernetes.io/last-applied-configuration` annotations on both Response and Request body contents. Previously it did not redact Response body content.</li><li> It now redacts Cluster Import URLs on both Request URLs and Referer headers. Previously it did not redact Referer headers.</li></ul> | 23 Oct 2025 | Rancher [v2.12.3](https://github.com/rancher/rancher/releases/tag/v2.12.3) |
| [CVE-2024-58260](https://github.com/rancher/rancher/security/advisories/GHSA-q82v-h4rq-5c86) | Setting the username of one user as the same username of another user causes an error when either user attempts to log in. Therefore, a user with the `Manage Users` permission could potentially deny any user, including admins, from logging in. To prevent this, usernames have been made immutable once set, and it is not possible to update or create a user with a username that is already in use. | 25 Sep 2025 | Rancher [v2.12.2](https://github.com/rancher/rancher/releases/tag/v2.12.2), [v2.11.6](https://github.com/rancher/rancher/releases/tag/v2.11.6), [v2.10.10](https://github.com/rancher/rancher/releases/tag/v2.10.10), and [v2.9.12](https://github.com/rancher/rancher/releases/tag/v2.9.12) |
| [CVE-2024-58267](https://github.com/rancher/rancher/security/advisories/GHSA-v3vj-5868-2ch2) | The Rancher CLI is modified to print the `requestId` more visibly than as part of the login URL. It also adds a `cli=true` origin marker to the URL. The dashboard is modified to recognize the presence of the `requestId` and uses that to show a warning message to the user, asking for verification that they initiated a CLI login with the related Id. The non-presence of the origin marker enables the dashboard to distinguish between the modified CLI and older CLI’s, and adjust the message accordingly. | 25 Sep 2025 | Rancher [v2.12.2](https://github.com/rancher/rancher/releases/tag/v2.12.2), [v2.11.6](https://github.com/rancher/rancher/releases/tag/v2.11.6), [v2.10.10](https://github.com/rancher/rancher/releases/tag/v2.10.10), and [v2.9.12](https://github.com/rancher/rancher/releases/tag/v2.9.12) |
| [CVE-2025-54468](https://github.com/rancher/rancher/security/advisories/GHSA-mjcp-rj3c-36fr) | `Impersonate-*` headers are removed for requests made through the `/meta/proxy` Rancher endpoint (e.g. when cloud credentials are being created) as the headers may contain identifiable and/or sensitive information. | 25 Sep 2025 | Rancher [v2.12.2](https://github.com/rancher/rancher/releases/tag/v2.12.2), [v2.11.6](https://github.com/rancher/rancher/releases/tag/v2.11.6), [v2.10.10](https://github.com/rancher/rancher/releases/tag/v2.10.10), and [v2.9.12](https://github.com/rancher/rancher/releases/tag/v2.9.12) |
| [CVE-2024-58259](https://github.com/rancher/rancher/security/advisories/GHSA-4h45-jpvh-6p5j) | POSTs to the Rancher API endpoints are now limited to 1 Mi; this is configurable through the settings if you need a larger limit. The Rancher authentication endpoints are configured independently of the main public API (as you might need bigger payloads in the other API endpoints). Suppose you need to increase the maximum allowed payload for authentication. In that case, you can set the environment variable `CATTLE_AUTH_API_BODY_LIMIT` to a quantity, e.g., 2 Mi, which would allow larger payloads for the authentication endpoints. | 28 Aug 2025 | Rancher [v2.12.1](https://github.com/rancher/rancher/releases/tag/v2.12.1), [v2.11.5](https://github.com/rancher/rancher/releases/tag/v2.11.5), [v2.10.9](https://github.com/rancher/rancher/releases/tag/v2.10.9) and [v2.9.11](https://github.com/rancher/rancher/releases/tag/v2.9.11) |
| [CVE-2024-52284](https://github.com/rancher/fleet/security/advisories/GHSA-6h9x-9j5v-7w9h) | Following a recent [change](https://github.com/rancher/fleet/pull/3403) excluding Helm values files from bundles, an edge case subsisted where the values files referenced in `fleet.yaml` with your directory name (e.g., `my-dir/values.yaml` instead of `values.yaml`) would not be excluded, which would potentially expose confidential data in bundle resources. Helm values files are now excluded from bundle resources regardless of how you reference them. | 28 Aug 2025 | Rancher [v2.12.1](https://github.com/rancher/rancher/releases/tag/v2.12.1), [v2.11.5](https://github.com/rancher/rancher/releases/tag/v2.11.5) and [v2.10.9](https://github.com/rancher/rancher/releases/tag/v2.10.9) |
versioned_docs/version-2.14/reference-guides/rancher-security/selinux-rpm/about-k3s-selinux.md
+13
View File
@@ -0,0 +1,13 @@
---
title: About k3s-selinux
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/selinux-rpm/about-k3s-selinux"/>
</head>
`k3s-selinux` provides policies for K3s. It is installed automatically when the K3s installer script detects that it is running on an RPM-based distro.
The `k3s-selinux` GitHub repository is [here](https://github.com/rancher/k3s-selinux).
For more information about installing K3s on SELinux-enabled hosts, see the [K3s documentation](https://docs.k3s.io/advanced#selinux-support).
versioned_docs/version-2.14/reference-guides/rancher-security/selinux-rpm/about-rancher-selinux.md
+71
View File
@@ -0,0 +1,71 @@
---
title: About rancher-selinux
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/selinux-rpm/about-rancher-selinux"/>
</head>
To allow Rancher to work with SELinux, some functionality has to be manually enabled for the SELinux nodes. To help with that, Rancher provides an SELinux RPM.
The `rancher-selinux` RPM contains a set of SELinux policies designed to grant the necessary privileges to various Rancher components running on Linux systems with SELinux enabled.
The `rancher-selinux` GitHub repository is [here.](https://github.com/rancher/rancher-selinux)
## Installing the rancher-selinux RPM
:::note Requirement:
The `rancher-selinux` RPM was tested on openSUSE Tumbleweed and RHEL-based distributions including Centos/RockyLinux 8 and 9.
:::
### 1. Set up the yum repo
Set up the yum repo to install `rancher-selinux` directly on all hosts in the cluster.
In order to use the RPM repository, on a CentOS 8 or RHEL 8 system, run the following bash snippet:
```
# cat << EOF > /etc/yum.repos.d/rancher.repo
[rancher]
name=Rancher
baseurl=https://rpm.rancher.io/rancher/production/centos/8/noarch
enabled=1
gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key
EOF
```
In order to use the RPM repository, on a CentOS 9 or RHEL 9 system, run the following bash snippet:
```
# cat << EOF > /etc/yum.repos.d/rancher.repo
[rancher]
name=Rancher
baseurl=https://rpm.rancher.io/rancher/production/centos/9/noarch
enabled=1
gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key
EOF
```
### 2. Installing the RPM
Install the RPM:
```
yum -y install rancher-selinux
```
## Configuring the Logging and Monitoring Applications to Work with SELinux
:::note Requirement:
Logging v2 and Monitoring v2 were tested with SELinux on RHEL/CentOS 8, 9, and Tumbleweed.
:::
Applications do not automatically work once the `rancher-selinux` RPM is installed on the host. They need to be configured to run in an allowed SELinux container domain provided by the RPM.
To configure the `rancher-logging` or the `rancher-monitoring` chart to be SELinux aware, change `global.seLinux.enabled` to true in the `values.yaml` when installing the charts.
versioned_docs/version-2.14/reference-guides/rancher-security/selinux-rpm/about-rke2-selinux.md
+13
View File
@@ -0,0 +1,13 @@
---
title: About rke2-selinux
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/selinux-rpm/about-rke2-selinux"/>
</head>
`rke2-selinux` provides policies for RKE2. It is installed automatically when the RKE2 installer script detects that it is running on an RPM-based distro.
The `rke2-selinux` GitHub repository is [here.](https://github.com/rancher/rke2-selinux)
For more information about installing RKE2 on SELinux-enabled hosts, see the [RKE2 documentation.](https://docs.rke2.io/install/methods#rpm)
versioned_docs/version-2.14/reference-guides/rancher-security/selinux-rpm/selinux-rpm.md
+23
View File
@@ -0,0 +1,23 @@
---
title: SELinux RPM
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-security/selinux-rpm"/>
</head>
[Security-Enhanced Linux (SELinux)](https://en.wikipedia.org/wiki/Security-Enhanced_Linux) is a security enhancement to Linux.
Developed by Red Hat, it is an implementation of mandatory access controls (MAC) on Linux. Mandatory access controls allow an administrator of a system to define how applications and users can access different resources such as files, devices, networks and inter-process communication. SELinux also enhances security by making an OS restrictive by default.
After being historically used by government agencies, SELinux is now an industry standard and enabled by default on SUSE distributions such as SUSE Linux Enterprise 16, openSUSE distributions such as openSUSE Tumbleweed, and RHEL-based OS such as Rocky Linux. To check whether SELinux is enabled and enforced on your system, use `getenforce`:
```
# getenforce
Enforcing
```
We provide three RPMs (RPM Package Manager) that enable Rancher products to function properly on SELinux-enforcing hosts:
- [`rancher-selinux`](about-rancher-selinux.md)
- [`rke2-selinux`](about-rke2-selinux.md)
- [`k3s-selinux`](about-k3s-selinux.md)
versioned_docs/version-2.14/reference-guides/rancher-webhook.md
+142
View File
@@ -0,0 +1,142 @@
---
title: Rancher Webhook
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rancher-webhook"/>
</head>
Rancher-Webhook is an essential component of Rancher that works in conjunction with Kubernetes to enhance security and enable critical features for Rancher-managed clusters.
It integrates with Kubernetes' extensible admission controllers, as described in the [Kubernetes documentation](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/), which allows Rancher-Webhook to inspect specific requests sent to the Kubernetes API server, and add custom validations and mutations to the requests that are specific to Rancher. Rancher-Webhook manages the resources to be validated using the `rancher.cattle.io` `ValidatingWebhookConfiguration` and the `rancher.cattle.io` `MutatingWebhookConfiguration` objects, and will override any manual edits.
Rancher deploys Rancher-Webhook as a separate deployment and service in both local and downstream clusters. Rancher manages Rancher-Webhook using Helm. It's important to note that Rancher may override modifications made by users to the Helm release. To safely modify these values see [Customizing Rancher-Webhook Configuration](#customizing-rancher-webhook-configuration).
Each Rancher version is designed to be compatible with a single version of the webhook. The compatible versions are provided below for convenience.
**Note:** Rancher manages deployment and upgrade of the webhook. Under most circumstances, no user intervention should be needed to ensure that the webhook version is compatible with the version of Rancher that you are running.
<!-- releaseTask -->
| Rancher Version | Webhook Version | Availability in Prime | Availability in Community |
|-----------------|-----------------|-----------------------|---------------------------|
| v2.13.3 | v0.9.3 | &check; | &check; |
| v2.13.2 | v0.9.2 | &check; | &check; |
| v2.13.1 | v0.9.1 | &check; | &check; |
| v2.13.0 | v0.9.0 | &cross; | &check; |
## Why Do We Need It?
Rancher-Webhook is crucial for Rancher to protect clusters against malicious attacks and enable various features.
Rancher relies on the Rancher-Webhook as an integral part of its functionality. Without the webhook, Rancher would not be a complete product.
It provides essential protection for Rancher-managed clusters, preventing security vulnerabilities and ensuring the consistency and stability of the cluster.
## What Resources Does the Webhook Validate?
You can find an in-progress list of the resources that the webhook validates in the [webhook's repo](https://github.com/rancher/webhook/blob/release/v0.4/docs.md). These docs are organized by group/version and resource (top-level header is group/version, next level header is resource). Checks specific to one version can be found by viewing the `docs.md` file associated with a particular tag (note that webhook versions prior to `v0.3.6` won't have this file).
## Bypassing the Webhook
Sometimes, you must bypass Rancher's webhook validation to perform emergency restore operations or fix other critical issues. The bypass operation is exhaustive, meaning no webhook validations or mutations apply when you use it. It is not possible to bypass some validations or mutations and have others still apply - they are either all bypassed or all active.
:::danger
Rancher's webhook provides critical security protections. Bypassing the webhook should only be done by administrators in specific scenarios, after all other options have been exhausted. In addition, permission to bypass the webhook should be carefully controlled, and never given to users who are not admins.
:::
To bypass the webhook, impersonate both the `rancher-webhook-sudo` service account and the `system:masters` group (both are required):
```bash
kubectl create -f example.yaml --as=system:serviceaccount:cattle-system:rancher-webhook-sudo --as-group=system:masters
```
## Customizing Rancher-Webhook Configuration
You can add custom Helm values when you install Rancher-Webhook via Helm. During a Helm install of the Rancher-Webhook chart, Rancher checks for custom Helm values. These custom values must be defined in a ConfigMap named `rancher-config`, in the `cattle-system` namespace, under the data key, `rancher-webhook`. The value of this key must be valid YAML.
``` yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: rancher-config
namespace: cattle-system
labels:
app.kubernetes.io/part-of: "rancher"
data:
rancher-webhook: '{"port": 9553, "priorityClassName": "system-node-critical"}'
```
Rancher redeploys the Rancher-Webhook chart when changes to the ConfigMap values are detected.
### Customizing Rancher-Webhook During Rancher Installation
When you use Helm to install the Rancher chart, you can add custom Helm values to the Rancher-Webhook of the local cluster. All values in the Rancher-Webhook chart are accessible as nested variables under the `webhook` name.
These values are synced to the `rancher-config` ConfigMap during installation.
```bash
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
...
--set webhook.port=9553 \
--set webhook.priorityClassName="system-node-critical"
```
## Common Issues
### EKS Cluster with Calico CNI
Users running an EKS cluster with Calico CNI may run into errors when the Kubernetes API server attempts to contact the Rancher-Webhook.
One workaround for this issue, as [documented by Calico](https://docs.tigera.io/calico/latest/getting-started/kubernetes/managed-public-cloud/eks#install-eks-with-calico-networking) involves setting `hostNetwork=true` for the webhook deployment. You can change this value by adding the Helm value `global.hostNetwork=true` to the `rancher-config` ConfigMap. See [Customizing Rancher-Webhook Configuration](#customizing-rancher-webhook-configuration) for more info.
``` bash
apiVersion: v1
kind: ConfigMap
metadata:
name: rancher-config
namespace: cattle-system
labels:
app.kubernetes.io/part-of: "rancher"
data:
rancher-webhook: '{"global": {"hostNetwork": true}}'
```
**Note:** This temporary workaround may violate an environment's security policy. This workaround also requires that port 9443 is unused on the host network.
**Note:** By default, Helm stores information as secrets. Secrets are a resource that some webhook versions validate. In these cases, directly update the deployment with the `hostNetwork=true` value using kubectl, then update the webhook configuration as specified above.
### Private GKE Cluster
When using a private GKE cluster, errors may occur that prevent the Kubernetes API server from communicating with the webhook. The following error message may appear:
```
Internal error occurred: failed calling webhook "rancher.cattle.io.namespaces.create-non-kubesystem": failed to call webhook: Post "https://rancher-webhook.cattle-system.svc:443/v1/webhook/validation/namespaces?timeout=10s": context deadline exceeded
```
This issue occurs because firewall rules restrict communication between the API server and the private cluster. To resolve this communication problem, users must add firewall rules to allow the GKE control plane to communicate with the Rancher-Webhook on port 9443. Please refer to the [GKE documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#add_firewall_rules) for detailed information and steps on updating the firewall rules.
### Application Fails to Deploy Due to rancher-webhook Blocking Access
The webhook provides extra validations on [namespaces](https://github.com/rancher/webhook/blob/release/v0.4/docs.md#psa-label-validation). One of these validations ensures that users can only update PSA relevant labels if they have the proper permissions (`updatepsa` for `projects` in `management.cattle.io`). This can result in specific operators, such as Tigera or Trident, failing when they attempt to deploy namespaces with PSA labels. There are several ways to resolve this issue:
- Configure the application to create a namespace with no PSA labels. If users wish to apply a PSA to these namespaces, they can add them to a project with the desired PSA after configuration. See the [docs on PSS and PSA resources](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/pod-security-standards.md) for instructions on how.
- This is the preferred option, though not all applications can be configured in this fashion.
- Manually grant the operator permissions to manage PSAs for namespaces.
- This option will introduce security risks, since the operator will now be able to set the PSA for the namespaces it has access to. This could allow the operator to deploy a privileged pod, or effect cluster takeover through other means.
- A user account with the proper permissions can pre-create the namespace with the appropriate configuration.
- This option depends on the ability of the application to handle existing resources.
Another one of these validations ensures that the user has the proper permissions to update the `field.cattle.io/projectId` annotation on a namespace. This is the `manage-namespaces` permission for `projects` in `management.cattle.io`.
## Issues on Specific Versions
**Note:** The following is an incomplete list of high-severity issues affecting specific Rancher/webhook versions. In most cases, these issues can be resolved by upgrading to a more recent Rancher version.
### Incompatible Webhook Version on Rollback
**Note:** This affects rolling back to Rancher v2.7.5 or earlier.
If you roll back to Rancher v2.7.5 or earlier, you may see webhook versions that are too recent to be compatible with downstream clusters running pre-v2.7.5 version of Rancher. This may cause various incompatibility issues. For example, project members may be unable to create namespaces. In addition, when you roll back to versions before the webhook was installed in downstream clusters, the webhook may remain installed, which can result in similar incompatibility issues.
To help alleviate these issues, you can run the [adjust-downstream-webhook](https://github.com/rancherlabs/support-tools/tree/master/adjust-downstream-webhook) shell script after roll back. This script selects and installs the proper webhook version (or removes the webhook entirely) for the corresponding Rancher version.
versioned_docs/version-2.14/reference-guides/rke1-template-example-yaml.md
+117
View File
@@ -0,0 +1,117 @@
---
title: RKE1 Example YAML
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/rke1-template-example-yaml"/>
</head>
<EOLRKE1Warning />
Below is an example RKE template configuration file for reference.
The YAML in the RKE template uses the same customization that is used when you create an RKE cluster. However, since the YAML is within the context of a Rancher provisioned RKE cluster, the customization from the RKE docs needs to be nested under the `rancher_kubernetes_engine` directive.
```yaml
#
# Cluster Config
#
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
# This setting is not enforced. Clusters
# created with this sample template
# would have alerting turned off by default,
# but end users could still turn alerting
# on or off.
enable_cluster_monitoring: true
# This setting is not enforced. Clusters
# created with this sample template
# would have monitoring turned on
# by default, but end users could still
# turn monitoring on or off.
enable_network_policy: false
local_cluster_auth_endpoint:
enabled: true
#
# Rancher Config
#
rancher_kubernetes_engine_config: # Your RKE template config goes here.
addon_job_timeout: 30
authentication:
strategy: x509
ignore_docker_version: true
#
# # Currently only nginx ingress provider is supported.
# # To disable ingress controller, set `provider: none`
# # To enable ingress on specific nodes, use the node_selector, eg:
# provider: nginx
# node_selector:
# app: ingress
#
ingress:
provider: nginx
kubernetes_version: v1.15.3-rancher3-1
monitoring:
provider: metrics-server
#
# If you are using calico on AWS
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
# # To specify flannel interface
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
# # To specify flannel interface for canal plugin
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
network:
options:
flannel_backend_type: vxlan
plugin: canal
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
# kube-controller:
# cluster_cidr: 10.42.0.0/16
# service_cluster_ip_range: 10.43.0.0/16
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
services:
etcd:
backup_config:
enabled: true
interval_hours: 12
retention: 6
safe_timestamp: false
creation: 12h
extra_args:
election-timeout: 5000
heartbeat-interval: 500
gid: 0
retention: 72h
snapshot: false
uid: 0
kube_api:
always_pull_images: false
pod_security_policy: false
service_node_port_range: 30000-32767
ssh_agent_auth: false
windows_prefered_cluster: false
```
versioned_docs/version-2.14/reference-guides/single-node-rancher-in-docker/advanced-options.md
+110
View File
@@ -0,0 +1,110 @@
---
title: Advanced Options for Docker Installs
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/single-node-rancher-in-docker/advanced-options"/>
</head>
## Custom CA Certificate
If you want to configure Rancher to use a CA root certificate to be used when validating services, you would start the Rancher container sharing the directory that contains the CA root certificate.
Use the command example to start a Rancher container with your private CA certificates mounted.
- The volume flag (`-v`) should specify the host directory containing the CA root certificates.
- The environment variable flag (`-e`) in combination with `SSL_CERT_DIR` and directory declares an environment variable that specifies the mounted CA root certificates directory location inside the container.
- Passing environment variables to the Rancher container can be done using `-e KEY=VALUE` or `--env KEY=VALUE`.
- Mounting a host directory inside the container can be done using `-v host-source-directory:container-destination-directory` or `--volume host-source-directory:container-destination-directory`.
The example below is based on having the CA root certificates in the `/host/certs` directory on the host and mounting this directory on `/container/certs` inside the Rancher container.
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /host/certs:/container/certs \
-e SSL_CERT_DIR="/container/certs" \
--privileged \
rancher/rancher:latest
```
## API Audit Log
The API Audit Log records all the user and system transactions made through Rancher server.
The API Audit Log writes to `/var/log/auditlog` inside the rancher container by default. Share that directory as a volume and set your `AUDIT_LEVEL` to enable the log.
See [API Audit Log](../../how-to-guides/advanced-user-guides/enable-api-audit-log.md) for more information and options.
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /var/log/rancher/auditlog:/var/log/auditlog \
-e AUDIT_LOG_ENABLED=true \
-e AUDIT_LEVEL=1 \
--privileged \
rancher/rancher:latest
```
## TLS settings
To set a different TLS configuration, you can use the `CATTLE_TLS_MIN_VERSION` and `CATTLE_TLS_CIPHERS` environment variables. For example, to configure TLS 1.0 as minimum accepted TLS version:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e CATTLE_TLS_MIN_VERSION="1.0" \
--privileged \
rancher/rancher:latest
```
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
See [TLS settings](../../getting-started/installation-and-upgrade/installation-references/tls-settings.md) for more information and options.
## Air Gap
If you are visiting this page to complete an air gap installation, you must prepend your private registry URL to the server tag when running the installation command in the option that you choose. Add `<REGISTRY.DOMAIN.COM:PORT>` with your private registry URL in front of `rancher/rancher:latest`.
**Example:**
<REGISTRY.DOMAIN.COM:PORT>/rancher/rancher:latest
## Persistent Data
Rancher uses etcd as a datastore. When Rancher is installed with Docker, the embedded etcd is being used. The persistent data is at the following path in the container: `/var/lib/rancher`.
You can bind mount a host volume to this location to preserve data on the host it is running on:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /opt/rancher:/var/lib/rancher \
--privileged \
rancher/rancher:latest
```
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
## Running `rancher/rancher` and `rancher/rancher-agent` on the Same Node
In the situation where you want to use a single node to run Rancher and to be able to add the same node to a cluster, you have to adjust the host ports mapped for the `rancher/rancher` container.
If a node is added to a cluster, it deploys the nginx ingress controller which will use port 80 and 443. This will conflict with the default ports we advise to expose for the `rancher/rancher` container.
Please note that this setup is not recommended for production use, but can be convenient for development/demo purposes.
To change the host ports mapping, replace the following part `-p 80:80 -p 443:443` with `-p 8080:80 -p 8443:443`:
```
docker run -d --restart=unless-stopped \
-p 8080:80 -p 8443:443 \
--privileged \
rancher/rancher:latest
```
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
versioned_docs/version-2.14/reference-guides/single-node-rancher-in-docker/http-proxy-configuration.md
+65
View File
@@ -0,0 +1,65 @@
---
title: HTTP Proxy Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/single-node-rancher-in-docker/http-proxy-configuration"/>
</head>
If you operate Rancher behind a proxy and you want to access services through the proxy (such as retrieving catalogs), you must provide Rancher information about your proxy. As Rancher is written in Go, it uses the common proxy environment variables as shown below.
Make sure `NO_PROXY` contains the network addresses, network address ranges and domains that should be excluded from using the proxy.
| Environment variable | Purpose |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s). <br/><br/> The value must be a comma-delimited string which contains IP addresses, CIDR notation, domain names, or special DNS labels (*). For a full description of the expected value format, refer to the [**upstream Golang documentation**](https://pkg.go.dev/golang.org/x/net/http/httpproxy#Config) |
:::note Important:
NO_PROXY must be in uppercase to use network range (CIDR) notation.
:::
## Docker Installation
Passing environment variables to the Rancher container can be done using `-e KEY=VALUE` or `--env KEY=VALUE`. Required values for `NO_PROXY` in a [Docker Installation](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) are:
- `localhost`
- `127.0.0.1`
- `0.0.0.0`
- `10.0.0.0/8`
- `cattle-system.svc`
- `.svc`
- `.cluster.local`
The example below is based on a proxy server accessible at `http://192.168.0.1:3128`, and excluding usage the proxy when accessing network range `192.168.10.0/24` and every hostname under the domain `example.com`.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e HTTP_PROXY="http://192.168.10.1:3128" \
-e HTTPS_PROXY="http://192.168.10.1:3128" \
-e NO_PROXY="localhost,127.0.0.1,0.0.0.0,10.0.0.0/8,cattle-system.svc,192.168.10.0/24,.svc,.cluster.local,example.com" \
--privileged \
rancher/rancher:latest
```
Privileged access is [required.](../../getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher)
### Air-gapped proxy configuration
You can now provision node driver clusters from an air-gapped cluster configured to use a proxy for outbound connections.
In addition to setting the default rules for a proxy server as shown above, you must also add the following rules to provision node driver clusters from a proxied Rancher environment:
You will configure your filepath according to your setup, e.g., `/etc/apt/apt.conf.d/proxy.conf`:
```
acl SSL_ports port 22
acl SSL_ports port 2376
acl Safe_ports port 22 # ssh
acl Safe_ports port 2376 # docker port
```
versioned_docs/version-2.14/reference-guides/single-node-rancher-in-docker/single-node-rancher-in-docker.md
+9
View File
@@ -0,0 +1,9 @@
---
title: Single Node Rancher in Docker
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/single-node-rancher-in-docker"/>
</head>
The following docs will discuss [HTTP proxy configuration](http-proxy-configuration.md) and [advanced options](advanced-options.md) for Docker installs.
versioned_docs/version-2.14/reference-guides/system-tools.md
+35
View File
@@ -0,0 +1,35 @@
---
title: System Tools
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/system-tools"/>
</head>
:::note
System Tools has been deprecated since June 2022.
:::
## Logs
Please use [logs-collector](https://github.com/rancherlabs/support-tools/tree/master/collection/rancher/v2.x/logs-collector) to collect logs from your cluster.
## Stats
If you want to replicate the stats command, you can run the following command on your cluster nodes:
:::note
This command below requires the package `sysstat` on the cluster node.
:::
```
/usr/bin/sar -u -r -F 1 1
```
## Remove
Please use the [Rancher Cleanup](https://github.com/rancher/rancher-cleanup) tool.
versioned_docs/version-2.14/reference-guides/user-settings/api-keys.md
+63
View File
@@ -0,0 +1,63 @@
---
title: API Keys
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/user-settings/api-keys"/>
</head>
## API Keys and User Authentication
If you want to access your Rancher clusters, projects, or other objects using external applications, you can do so using the Rancher API. However, before your application can access the API, you must provide the app with a key used to authenticate with Rancher. You can obtain a key using the Rancher UI.
An API key is also required for using Rancher CLI.
API Keys are composed of four components:
- **Endpoint:** This is the IP address and path that other applications use to send requests to the Rancher API.
- **Access Key:** The token's username.
- **Secret Key:** The token's password. For applications that prompt you for two different strings for API authentication, you usually enter the two keys together.
- **Bearer Token:** The token username and password concatenated together. Use this string for applications that prompt you for one authentication string.
:::note
Users may opt to enable [token hashing](../../api/api-tokens.md).
:::
## Creating an API Key
1. Select **User Avatar > Account & API Keys** from upper right corner.
2. Click **Create API Key**.
3. **Optional:** Enter a description for the API key and select an expiration period or a scope. We recommend setting an expiration date.
The API key won't be valid after expiration. Shorter expiration periods are more secure.
Expiration period is bound by `v3/settings/auth-token-max-ttl-minutes`. If it exceeds the max-ttl, the API key is created with max-ttl as the expiration period.
A scope limits the API key so that it only works against the Kubernetes API of the specified cluster. If the cluster is configured with an Authorized Cluster Endpoint, you are able to use a scoped token directly against the cluster's API without proxying through the Rancher server. See [Authorized Cluster Endpoints](../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) for more information.
4. Click **Create**.
**Step Result:** Your API Key is created. Your API **Endpoint**, **Access Key**, **Secret Key**, and **Bearer Token** are displayed.
Use the **Bearer Token** to authenticate with Rancher CLI.
5. Copy the information displayed to a secure location. This information is only displayed once, so if you lose your key, you'll have to make a new one.
## What's Next?
- Enter your API key information into the application that sends requests to the Rancher API.
- Learn more about the Rancher endpoints and parameters by selecting **View in API** for an object in the Rancher UI.
- API keys are used for API calls and [Rancher CLI](../cli-with-rancher/cli-with-rancher.md).
## Deleting API Keys
If you need to revoke an API key, delete it. You should delete API keys:
- That may have been compromised.
- That have expired.
To delete an API, select the stale key and click **Delete**.
versioned_docs/version-2.14/reference-guides/user-settings/manage-cloud-credentials.md
+55
View File
@@ -0,0 +1,55 @@
---
title: Managing Cloud Credentials
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/user-settings/manage-cloud-credentials"/>
</head>
When you create a cluster [hosted by an infrastructure provider](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md), [node templates](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#node-templates) are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node.
Node templates can use cloud credentials to access the credential information required to provision nodes in the infrastructure providers. The same cloud credential can be used by multiple node templates. By using a cloud credential, you do not have to re-enter access keys for the same cloud provider. Cloud credentials are stored as Kubernetes secrets.
Cloud credentials are only used by node templates if there are fields marked as `password`. The default `active` node drivers have their account access fields marked as `password`, but there may be some `inactive` node drivers, which are not using them yet. These node drivers will not use cloud credentials.
You can create cloud credentials in two contexts:
- [During creation of a node template](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#node-templates) for a cluster.
- In the **User Settings**
Cloud credentials are bound to their creator's user profile. They **cannot** be shared between non-admin users. However, admins can view and manage the cloud credentials of other users.
## Creating a Cloud Credential from User Settings
1. Click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
1. Click **Create**.
1. Click a cloud credential type. The values of this dropdown is based on the `active` [node drivers](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-node-drivers.md) in Rancher.
1. Enter a name for the cloud credential.
1. Based on the selected cloud credential type, enter the required values to authenticate with the infrastructure provider.
1. Click **Create**.
**Result:** The cloud credential is created and can immediately be used to [create node templates](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#node-templates).
## Updating a Cloud Credential
When access credentials are changed or compromised, updating a cloud credential allows you to rotate those credentials while keeping the same node template.
1. Click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
1. Choose the cloud credential you want to edit and click the **⋮ > Edit Config**.
1. Update the credential information and click **Save**.
**Result:** The cloud credential is updated with the new access credentials. All existing node templates using this cloud credential will automatically use the updated information whenever [new nodes are added](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md).
## Deleting a Cloud Credential
In order to delete cloud credentials, there must not be any node template associated with it. If you are unable to delete the cloud credential, [delete any node templates](manage-node-templates.md#deleting-a-node-template) that are still associated to that cloud credential.
1. Click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
1. You can either individually delete a cloud credential or bulk delete.
- To individually delete one, choose the cloud credential you want to edit and click the **⋮ > Delete**.
- To bulk delete cloud credentials, select one or more cloud credentials from the list. Click **Delete**.
1. Confirm that you want to delete these cloud credentials.
versioned_docs/version-2.14/reference-guides/user-settings/manage-node-templates.md
+58
View File
@@ -0,0 +1,58 @@
---
title: Managing Node Templates
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/user-settings/manage-node-templates"/>
</head>
When you provision a cluster [hosted by an infrastructure provider](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md), [node templates](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#node-templates) are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node. You can create node templates in two contexts:
- While [provisioning a node pool cluster](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md).
- At any time, from your [user settings](user-settings.md).
When you create a node template, it is bound to your user profile. Node templates cannot be shared among users. You can delete stale node templates that you no longer user from your user settings.
## Creating a Node Template
1. Click **☰ > Cluster Management**.
1. Click **RKE1 Configuration > Node Templates**.
1. Click **Add Template**.
1. Select one of the cloud providers available. Then follow the instructions on screen to configure the template.
**Result:** The template is configured. You can use the template later when you [provision a node pool cluster](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md).
## Updating a Node Template
1. Click **☰ > Cluster Management**.
1. Click **RKE1 Configuration > Node Templates**.
1. Choose the node template that you want to edit and click the **⋮ > Edit**.
:::note
The default `active` [node drivers](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-node-drivers.md) and any node driver, that has fields marked as `password`, are required to use [cloud credentials](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#cloud-credentials).
:::
1. Edit the required information and click **Save**.
**Result:** The node template is updated. All node pools using this node template will automatically use the updated information when new nodes are added.
## Cloning Node Templates
When creating new node templates from your user settings, you can clone an existing template and quickly update its settings rather than creating a new one from scratch. Cloning templates saves you the hassle of re-entering access keys for the cloud provider.
1. Click **☰ > Cluster Management**.
1. Click **RKE1 Configuration > Node Templates**.
1. Find the template you want to clone. Then select **⋮ > Clone**.
1. Complete the rest of the form.
**Result:** The template is cloned and configured. You can use the template later when you [provision a node pool cluster](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md).
## Deleting a Node Template
When you no longer use a node template, you can delete it from your user settings.
1. Click **☰ > Cluster Management**.
1. Click **RKE1 Configuration > Node Templates**.
1. Select one or more template from the list. Then click **Delete**. Confirm the delete when prompted.
versioned_docs/version-2.14/reference-guides/user-settings/user-preferences.md
+64
View File
@@ -0,0 +1,64 @@
---
title: User Preferences
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/user-settings/user-preferences"/>
</head>
You can set preferences to personalize your Rancher experience. To change preference settings:
1. Click on your user avatar in the upper right corner.
1. Click **Preferences**.
## Language
Choose your language for the Rancher UI. Options include:
- English
- 简体中文 (Simplified Chinese)
## Theme
Choose your background color for the Rancher UI. If you choose **Auto**, the background color changes from light to dark at 6 PM, and then changes back at 6 AM.
## Login Landing Page
Choose the page displayed after logging in. Options include:
- The home page.
- The area you last visited.
- A specific cluster of your choosing.
## Display Settings
Choose how certain information is displayed:
- Date format
- Time format
- Table rows per page
- Number of clusters to show in side menu
## Confirmation Setting
Choose whether to ask for confirmation when scaling down node pools.
## Advanced Features
- Enable "View in API".
- Show system Namespaces managed by Rancher (not intended for editing or deletion).
- Enable Dark/Light Theme keyboard shortcut toggle (shift+T).
- Hide All Type Descriptions.
- Enable Extension developer features.
![](/img/user-preferences-2-7.png)
## YAML Editor Key Mapping
- Normal human (Sublime)
- Emacs
- Vim
## Helm Charts
Choose whether to display released Helm charts only or to include prerelease versions as well. A version is considered to be a prerelease if it adheres to [the specification](https://semver.org/#spec-item-9) defined by [Semantic Versioning 2.0.0](https://semver.org/). For example, a Helm chart with a version of `0.1.3-dev.12ab4f` requires `Include Prerelease Versions` to be selected in order to be displayed.
versioned_docs/version-2.14/reference-guides/user-settings/user-settings.md
+19
View File
@@ -0,0 +1,19 @@
---
title: User Settings
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/user-settings"/>
</head>
Within Rancher, each user has a number of settings associated with their login: personal preferences, API keys, etc. You can configure these settings by choosing from the **User Settings** menu. You can open this menu by clicking your avatar, located within the main menu.
![User Settings Menu](/img/user-settings.png)
The available user settings are:
- [API & Keys](api-keys.md): If you want to interact with Rancher programmatically, you need an API key. Follow the directions in this section to obtain a key.
- [Cloud Credentials](manage-cloud-credentials.md): Manage cloud credentials [used by node templates](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md#node-templates) to [provision nodes for clusters](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
- [Node Templates](manage-node-templates.md): Manage templates [used by Rancher to provision nodes for clusters](../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md).
- [Preferences](user-preferences.md): Sets superficial preferences for the Rancher UI.
- Log Out: Ends your user session.