public/rancher-docs
1
0
Fork 0
mirror of https://github.com/rancher/rancher-docs.git synced 2026-09-25 20:48:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

2.5: Apply Divio and update links

Browse Source
This commit is contained in:
Billy Tat
2022-08-19 15:48:28 -07:00
parent 81f5cfdf72
commit 81cead2c90
410 changed files with 1672 additions and 1635 deletions
Download Patch File Download Diff File Expand all files Collapse all files
versioned_docs/version-2.5/reference-guides/about-the-api/api-tokens.md
+50
View File
@@ -0,0 +1,50 @@
---
title: API Tokens
weight: 1
aliases:
- /rancher/v2.5/en/cluster-admin/api/api-tokens/
- /rancher/v2.x/en/api/api-tokens/
---
By default, some cluster-level API tokens are generated with infinite time-to-live (`ttl=0`). In other words, API tokens with `ttl=0` never expire unless you invalidate them. Tokens are not invalidated by changing a password.
You can deactivate API tokens by deleting them or by deactivating the user account.
### Deleting tokens
To delete a token,
1. Go to the list of all tokens in the Rancher API view at `https://<Rancher-Server-IP>/v3/tokens`.
1. Access the token you want to delete by its ID. For example, `https://<Rancher-Server-IP>/v3/tokens/kubectl-shell-user-vqkqt`
1. Click **Delete.**
Here is the complete list of tokens that are generated with `ttl=0`:
| Token | Description |
|-------|-------------|
| `kubeconfig-*` | Kubeconfig token |
| `kubectl-shell-*` | Access to `kubectl` shell in the browser |
| `agent-*` | Token for agent deployment |
| `compose-token-*` | Token for compose |
| `helm-token-*` | Token for Helm chart deployment |
| `*-pipeline*` | Pipeline token for project |
| `telemetry-*` | Telemetry token |
| `drain-node-*` | Token for drain (we use `kubectl` for drain because there is no native Kubernetes API) |
### Setting TTL on Kubeconfig Tokens
Admins can set a global TTL on Kubeconfig tokens. Once the token expires the kubectl command will require the user to authenticate to Rancher.
1. Disable the kubeconfig-generate-token setting in the Rancher API view at `https://<Rancher-Server-IP/v3/settings/kubeconfig-generate-token`. This setting instructs Rancher to no longer automatically generate a token when a user clicks on download a kubeconfig file. The kubeconfig file will now provide a command to login to Rancher.
2. Edit the setting and set the value to `false`.
3. Go to setting kubeconfig-token-ttl-minutes in the Rancher API view at `https://<Rancher-Server-IP/v3/settings/kubeconfig-token-ttl-minutes`. By default, kubeconfig-token-ttl-minutes is 960 (16 hours).
4. Edit the setting and set the value to desired duration in minutes.
_**Note:**_ This value cannot exceed max-ttl of API tokens.(`https://<Rancher-Server-IP/v3/settings/auth-token-max-ttl-minutes`). `auth-token-max-ttl-minutes` is set to 1440 (24 hours) by default. `auth-token-max-ttl-minutes would default to 0 allowing tokens to never expire`.
versioned_docs/version-2.5/reference-guides/amazon-eks-permissions/minimum-eks-permissions.md
+223
View File
@@ -0,0 +1,223 @@
---
title: Minimum EKS Permissions
weight: 1
---
Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. Additional permissions are required for Rancher to provision the `Service Role` and `VPC` resources. Optionally these resources can be created **before** the cluster creation and will be selectable when defining the cluster configuration.
Resource | Description
---------|------------
Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions](#service-role-permissions).
VPC | Provides isolated network resources utilised by EKS and worker nodes. Rancher can create the VPC resources with the following [VPC Permissions](#vpc-permissions).
Resource targeting uses `*` as the ARN of many of the resources created cannot be known before creating the EKS cluster in Rancher.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2Permisssions",
"Effect": "Allow",
"Action": [
"ec2:RunInstances",
"ec2:RevokeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:DescribeRegions",
"ec2:DescribeVpcs",
"ec2:DescribeTags",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeRouteTables",
"ec2:DescribeLaunchTemplateVersions",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeKeyPairs",
"ec2:DescribeInternetGateways",
"ec2:DescribeImages",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeAccountAttributes",
"ec2:DeleteTags",
"ec2:DeleteSecurityGroup",
"ec2:DeleteKeyPair",
"ec2:CreateTags",
"ec2:CreateSecurityGroup",
"ec2:CreateLaunchTemplateVersion",
"ec2:CreateLaunchTemplate",
"ec2:CreateKeyPair",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:AuthorizeSecurityGroupEgress"
],
"Resource": "*"
},
{
"Sid": "CloudFormationPermisssions",
"Effect": "Allow",
"Action": [
"cloudformation:ListStacks",
"cloudformation:ListStackResources",
"cloudformation:DescribeStacks",
"cloudformation:DescribeStackResources",
"cloudformation:DescribeStackResource",
"cloudformation:DeleteStack",
"cloudformation:CreateStackSet",
"cloudformation:CreateStack"
],
"Resource": "*"
},
{
"Sid": "IAMPermissions",
"Effect": "Allow",
"Action": [
"iam:PassRole",
"iam:ListRoles",
"iam:ListRoleTags",
"iam:ListInstanceProfilesForRole",
"iam:ListInstanceProfiles",
"iam:ListAttachedRolePolicies",
"iam:GetRole",
"iam:GetInstanceProfile",
"iam:DetachRolePolicy",
"iam:DeleteRole",
"iam:CreateRole",
"iam:AttachRolePolicy"
],
"Resource": "*"
},
{
"Sid": "KMSPermisssions",
"Effect": "Allow",
"Action": "kms:ListKeys",
"Resource": "*"
},
{
"Sid": "EKSPermisssions",
"Effect": "Allow",
"Action": [
"eks:UpdateNodegroupVersion",
"eks:UpdateNodegroupConfig",
"eks:UpdateClusterVersion",
"eks:UpdateClusterConfig",
"eks:UntagResource",
"eks:TagResource",
"eks:ListUpdates",
"eks:ListTagsForResource",
"eks:ListNodegroups",
"eks:ListFargateProfiles",
"eks:ListClusters",
"eks:DescribeUpdate",
"eks:DescribeNodegroup",
"eks:DescribeFargateProfile",
"eks:DescribeCluster",
"eks:DeleteNodegroup",
"eks:DeleteFargateProfile",
"eks:DeleteCluster",
"eks:CreateNodegroup",
"eks:CreateFargateProfile",
"eks:CreateCluster"
],
"Resource": "*"
}
]
}
```
### Service Role Permissions
Permissions required for Rancher to create service role on users behalf during the EKS cluster creation process.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "IAMPermisssions",
"Effect": "Allow",
"Action": [
"iam:AddRoleToInstanceProfile",
"iam:AttachRolePolicy",
"iam:CreateInstanceProfile",
"iam:CreateRole",
"iam:CreateServiceLinkedRole",
"iam:DeleteInstanceProfile",
"iam:DeleteRole",
"iam:DetachRolePolicy",
"iam:GetInstanceProfile",
"iam:GetRole",
"iam:ListAttachedRolePolicies",
"iam:ListInstanceProfiles",
"iam:ListInstanceProfilesForRole",
"iam:ListRoles",
"iam:ListRoleTags",
"iam:PassRole",
"iam:RemoveRoleFromInstanceProfile"
],
"Resource": "*"
}
]
}
```
When an EKS cluster is created, Rancher will create a service role with the following trust policy:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "eks.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
```
This role will also have two role policy attachments with the following policies ARNs:
```
arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
arn:aws:iam::aws:policy/AmazonEKSServicePolicy
```
### VPC Permissions
Permissions required for Rancher to create VPC and associated resources.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VPCPermissions",
"Effect": "Allow",
"Action": [
"ec2:ReplaceRoute",
"ec2:ModifyVpcAttribute",
"ec2:ModifySubnetAttribute",
"ec2:DisassociateRouteTable",
"ec2:DetachInternetGateway",
"ec2:DescribeVpcs",
"ec2:DeleteVpc",
"ec2:DeleteTags",
"ec2:DeleteSubnet",
"ec2:DeleteRouteTable",
"ec2:DeleteRoute",
"ec2:DeleteInternetGateway",
"ec2:CreateVpc",
"ec2:CreateSubnet",
"ec2:CreateSecurityGroup",
"ec2:CreateRouteTable",
"ec2:CreateRoute",
"ec2:CreateInternetGateway",
"ec2:AttachInternetGateway",
"ec2:AssociateRouteTable"
],
"Resource": "*"
}
]
}
```
versioned_docs/version-2.5/reference-guides/backup-restore-configuration/backup-configuration.md
+186
View File
@@ -0,0 +1,186 @@
---
title: Backup Configuration
shortTitle: Backup
weight: 1
aliases:
- /rancher/v2.5/en/backups/v2.5/configuration/backup-config
- /rancher/v2.x/en/backups/v2.5/configuration/backup-config/
---
The Backup Create page lets you configure a schedule, enable encryption and specify the storage location for your backups.
![](/img/backup_restore/backup/backup.png)
- [Schedule](#schedule)
- [Encryption](#encryption)
- [Storage Location](#storage-location)
- [S3](#s3)
- [Example S3 Storage Configuration](#example-s3-storage-configuration)
- [Example MinIO Configuration](#example-minio-configuration)
- [Example credentialSecret](#example-credentialsecret)
- [IAM Permissions for EC2 Nodes to Access S3](#iam-permissions-for-ec2-nodes-to-access-s3)
- [Examples](#examples)
# Schedule
Select the first option to perform a one-time backup, or select the second option to schedule recurring backups. Selecting **Recurring Backups** lets you configure following two fields:
- **Schedule**: This field accepts
- Standard [cron expressions](https://en.wikipedia.org/wiki/Cron), such as `"0 * * * *"`
- Descriptors, such as `"@midnight"` or `"@every 1h30m"`
- **Retention Count**: This value specifies how many backup files must be retained. If files exceed the given retentionCount, the oldest files will be deleted. The default value is 10.
![](/img/backup_restore/backup/schedule.png)
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `schedule` | Provide the cron string for scheduling recurring backups. |
| `retentionCount` | Provide the number of backup files to be retained. |
# Encryption
The rancher-backup gathers resources by making calls to the kube-apiserver. Objects returned by apiserver are decrypted, so even if [encryption At rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) is enabled, even the encrypted objects gathered by the backup will be in plaintext.
To avoid storing them in plaintext, you can use the same encryptionConfig file that was used for at-rest encryption, to encrypt certain resources in your backup.
> **Important:** You must save the encryptionConfig file, because it won’t be saved by the rancher-backup operator.
The same encryptionFile needs to be used when performing a restore.
The operator consumes this encryptionConfig as a Kubernetes Secret, and the Secret must be in the operator’s namespace. Rancher installs the `rancher-backup` operator in the `cattle-resources-system` namespace, so create this encryptionConfig secret in that namespace.
For the `EncryptionConfiguration`, you can use the [sample file provided in the Kubernetes documentation.](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#understanding-the-encryption-at-rest-configuration)
To create the Secret, the encryption configuration file must be named `encryption-provider-config.yaml`, and the `--from-file` flag must be used to create this secret.
Save the `EncryptionConfiguration` in a file called `encryption-provider-config.yaml` and run this command:
```
kubectl create secret generic encryptionconfig \
--from-file=./encryption-provider-config.yaml \
-n cattle-resources-system
```
This will ensure that the secret contains a key named `encryption-provider-config.yaml`, and the operator will use this key to get the encryption configuration.
The `Encryption Config Secret` dropdown will filter out and list only those Secrets that have this exact key
![](/img/backup_restore/backup/encryption.png)
In the example command above, the name `encryptionconfig` can be changed to anything.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `encryptionConfigSecretName` | Provide the name of the Secret from `cattle-resources-system` namespace, that contains the encryption config file. |
# Storage Location
![](/img/backup_restore/backup/storageLocation.png)
If the StorageLocation is specified in the Backup, the operator will retrieve the backup location from that particular S3 bucket. If not specified, the operator will try to find this file in the default operator-level S3 store, and in the operator-level PVC store. The default storage location is configured during the deployment of the `rancher-backup` operator.
Selecting the first option stores this backup in the storage location configured while installing the rancher-backup chart. The second option lets you configure a different S3 compatible storage provider for storing the backup.
### S3
The S3 storage location contains the following configuration fields:
1. **Credential Secret** (optional): If you need to use the AWS Access keys Secret keys to access s3 bucket, create a secret with your credentials with keys and the directives `accessKey` and `secretKey`. It can be in any namespace. An example secret is [here.](#example-credentialsecret) This directive is unnecessary if the nodes running your operator are in EC2 and set up with IAM permissions that allow them to access S3, as described in [this section.](#iam-permissions-for-ec2-nodes-to-access-s3) The Credential Secret dropdown lists the secrets in all namespaces.
1. **Bucket Name**: The name of the S3 bucket where backup files will be stored.
1. **Region** (optional): The AWS [region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. This field isn't needed for configuring MinIO.
1. **Folder** (optional): The name of the folder in the S3 bucket where backup files will be stored. Nested folders (e.g., `rancher/cluster1`) are not supported.
1. **Endpoint**: The [endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) that is used to access S3 in the region of your bucket.
1. **Endpoint CA** (optional): This should be the Base64 encoded CA cert. For an example, refer to the [example S3 compatible configuration.](#example-s3-storage-configuration)
1. **Skip TLS Verifications** (optional): Set to true if you are not using TLS.
| YAML Directive Name | Description | Required |
| ---------------- | ---------------- | ------------ |
| `credentialSecretName` | If you need to use the AWS Access keys Secret keys to access s3 bucket, create a secret with your credentials with keys and the directives `accessKey` and `secretKey`. It can be in any namespace as long as you provide that namespace in `credentialSecretNamespace`. An example secret is [here.](#example-credentialsecret) This directive is unnecessary if the nodes running your operator are in EC2 and set up with IAM permissions that allow them to access S3, as described in [this section.](#iam-permissions-for-ec2-nodes-to-access-s3) | |
| `credentialSecretNamespace` | The namespace of the secret containing the credentials to access S3. This directive is unnecessary if the nodes running your operator are in EC2 and set up with IAM permissions that allow them to access S3, as described in [this section.](#iam-permissions-for-ec2-nodes-to-access-s3) | |
| `bucketName` | The name of the S3 bucket where backup files will be stored. | ✓ |
| `folder` | The name of the folder in the S3 bucket where backup files will be stored. Nested folders (e.g., `rancher/cluster1`) are not supported. | |
| `region` | The AWS [region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. | ✓ |
| `endpoint` | The [endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) that is used to access S3 in the region of your bucket. | ✓ |
| `endpointCA` | This should be the Base64 encoded CA cert. For an example, refer to the [example S3 compatible configuration.](#example-s3-storage-configuration) | |
| `insecureTLSSkipVerify` | Set to true if you are not using TLS. | |
### Example S3 Storage Configuration
```yaml
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: rancher
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
```
### Example MinIO Configuration
```yaml
s3:
credentialSecretName: minio-creds
bucketName: rancherbackups
endpoint: minio.35.202.130.254.sslip.io
endpointCA: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURHakNDQWdLZ0F3SUJBZ0lKQUtpWFZpNEpBb0J5TUEwR0NTcUdTSWIzRFFFQkN3VUFNQkl4RURBT0JnTlYKQkFNTUIzUmxjM1F0WTJFd0hoY05NakF3T0RNd01UZ3lOVFE1V2hjTk1qQXhNREk1TVRneU5UUTVXakFTTVJBdwpEZ1lEVlFRRERBZDBaWE4wTFdOaE1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBCjA4dnV3Q2Y0SEhtR2Q2azVNTmozRW5NOG00T2RpS3czSGszd1NlOUlXQkwyVzY5WDZxenBhN2I2M3U2L05mMnkKSnZWNDVqeXplRFB6bFJycjlpbEpWaVZ1NFNqWlFjdG9jWmFCaVNsL0xDbEFDdkFaUlYvKzN0TFVTZSs1ZDY0QQpWcUhDQlZObU5xM3E3aVY0TE1aSVpRc3N6K0FxaU1Sd0pOMVVKQTZ6V0tUc2Yzc3ByQ0J2dWxJWmZsVXVETVAyCnRCTCt6cXZEc0pDdWlhNEEvU2JNT29tVmM2WnNtTGkwMjdub3dGRld3MnRpSkM5d0xMRE14NnJoVHQ4a3VvVHYKQXJpUjB4WktiRU45L1Uzb011eUVKbHZyck9YS2ZuUDUwbk8ycGNaQnZCb3pUTStYZnRvQ1d5UnhKUmI5cFNTRApKQjlmUEFtLzNZcFpMMGRKY2sxR1h3SURBUUFCbzNNd2NUQWRCZ05WSFE0RUZnUVU5NHU4WXlMdmE2MTJnT1pyCm44QnlFQ2NucVFjd1FnWURWUjBqQkRzd09ZQVU5NHU4WXlMdmE2MTJnT1pybjhCeUVDY25xUWVoRnFRVU1CSXgKRURBT0JnTlZCQU1NQjNSbGMzUXRZMkdDQ1FDb2wxWXVDUUtBY2pBTUJnTlZIUk1FQlRBREFRSC9NQTBHQ1NxRwpTSWIzRFFFQkN3VUFBNElCQVFER1JRZ1RtdzdVNXRQRHA5Q2psOXlLRW9Vd2pYWWM2UlAwdm1GSHpubXJ3dUVLCjFrTkVJNzhBTUw1MEpuS29CY0ljVDNEeGQ3TGdIbTNCRE5mVVh2anArNnZqaXhJYXR2UWhsSFNVaWIyZjJsSTkKVEMxNzVyNCtROFkzelc1RlFXSDdLK08vY3pJTGh5ei93aHRDUlFkQ29lS1dXZkFiby8wd0VSejZzNkhkVFJzNwpHcWlGNWZtWGp6S0lOcTBjMHRyZ0xtalNKd1hwSnU0ZnNGOEcyZUh4b2pOKzdJQ1FuSkg5cGRIRVpUQUtOL2ppCnIvem04RlZtd1kvdTBndEZneWVQY1ZWbXBqRm03Y0ZOSkc4Y2ZYd0QzcEFwVjhVOGNocTZGeFBHTkVvWFZnclMKY1VRMklaU0RJd1FFY3FvSzFKSGdCUWw2RXBaUVpWMW1DRklrdFBwSQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t
```
### Example credentialSecret
```yaml
apiVersion: v1
kind: Secret
metadata:
name: creds
type: Opaque
data:
accessKey: <Enter your base64-encoded access key>
secretKey: <Enter your base64-encoded secret key>
```
### IAM Permissions for EC2 Nodes to Access S3
There are two ways to set up the `rancher-backup` operator to use S3 as the backup storage location.
One way is to configure the `credentialSecretName` in the Backup custom resource, which refers to AWS credentials that have access to S3.
If the cluster nodes are in Amazon EC2, the S3 access can also be set up by assigning IAM permissions to the EC2 nodes so that they can access S3.
To allow a node to access S3, follow the instructions in the [AWS documentation](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-access-s3-bucket/) to create an IAM role for EC2. When you add a custom policy to the role, add the following permissions, and replace the `Resource` with your bucket name:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::rancher-backups"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::rancher-backups/*"
]
}
]
}
```
After the role is created, and you have attached the corresponding instance profile to your EC2 instance(s), the `credentialSecretName` directive can be left empty in the Backup custom resource.
# Examples
For example Backup custom resources, refer to [this page.](examples.md#backup)
versioned_docs/version-2.5/reference-guides/backup-restore-configuration/examples.md
+301
View File
@@ -0,0 +1,301 @@
---
title: Examples
weight: 5
aliases:
- /rancher/v2.5/en/backups/v2.5/examples
- /rancher/v2.x/en/backups/v2.5/examples/
---
This section contains examples of Backup and Restore custom resources.
The default backup storage location is configured when the `rancher-backup` operator is installed or upgraded.
Encrypted backups can only be restored if the Restore custom resource uses the same encryption configuration secret that was used to create the backup.
- [Backup](#backup)
- [Backup in the default location with encryption](#backup-in-the-default-location-with-encryption)
- [Recurring backup in the default location](#recurring-backup-in-the-default-location)
- [Encrypted recurring backup in the default location](#encrypted-recurring-backup-in-the-default-location)
- [Encrypted backup in Minio](#encrypted-backup-in-minio)
- [Backup in S3 using AWS credential secret](#backup-in-s3-using-aws-credential-secret)
- [Recurring backup in S3 using AWS credential secret](#recurring-backup-in-s3-using-aws-credential-secret)
- [Backup from EC2 nodes with IAM permission to access S3](#backup-from-ec2-nodes-with-iam-permission-to-access-s3)
- [Restore](#restore)
- [Restore using the default backup file location](#restore-using-the-default-backup-file-location)
- [Restore for Rancher migration](#restore-for-rancher-migration)
- [Restore from encrypted backup](#restore-from-encrypted-backup)
- [Restore an encrypted backup from Minio](#restore-an-encrypted-backup-from-minio)
- [Restore from backup using an AWS credential secret to access S3](#restore-from-backup-using-an-aws-credential-secret-to-access-s3)
- [Restore from EC2 nodes with IAM permissions to access S3](#restore-from-ec2-nodes-with-iam-permissions-to-access-s3)
- [Example Credential Secret for Storing Backups in S3](#example-credential-secret-for-storing-backups-in-s3)
- [Example EncryptionConfiguration](#example-encryptionconfiguration)
# Backup
This section contains example Backup custom resources.
### Backup in the Default Location with Encryption
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-location-encrypted-backup
spec:
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
```
### Recurring Backup in the Default Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-location-recurring-backup
spec:
resourceSetName: rancher-resource-set
schedule: "@every 1h"
retentionCount: 10
```
### Encrypted Recurring Backup in the Default Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: default-enc-recurring-backup
spec:
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
schedule: "@every 1h"
retentionCount: 3
```
### Encrypted Backup in Minio
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: minio-backup
spec:
storageLocation:
s3:
credentialSecretName: minio-creds
credentialSecretNamespace: default
bucketName: rancherbackups
endpoint: minio.sslip.io
endpointCA: LS0tLS1CRUdJTi3VUFNQkl5UUT.....pbEpWaVzNkRS0tLS0t
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
```
### Backup in S3 Using AWS Credential Secret
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-backup
spec:
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
```
### Recurring Backup in S3 Using AWS Credential Secret
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-recurring-backup
spec:
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
schedule: "@every 1h"
retentionCount: 10
```
### Backup from EC2 Nodes with IAM Permission to Access S3
This example shows that the AWS credential secret does not have to be provided to create a backup if the nodes running `rancher-backup` have [these permissions for access to S3.](backup-configuration.md#iam-permissions-for-ec2-nodes-to-access-s3)
```yaml
apiVersion: resources.cattle.io/v1
kind: Backup
metadata:
name: s3-iam-backup
spec:
storageLocation:
s3:
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
resourceSetName: rancher-resource-set
encryptionConfigSecretName: encryptionconfig
```
# Restore
This section contains example Restore custom resources.
### Restore Using the Default Backup File Location
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-default
spec:
backupFilename: default-location-recurring-backup-752ecd87-d958-4d20-8350-072f8d090045-2020-09-26T12-29-54-07-00.tar.gz
# encryptionConfigSecretName: test-encryptionconfig
```
### Restore for Rancher Migration
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-migration
spec:
backupFilename: backup-b0450532-cee1-4aa1-a881-f5f48a007b1c-2020-09-15T07-27-09Z.tar.gz
prune: false
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
```
### Restore from Encrypted Backup
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-encrypted
spec:
backupFilename: default-test-s3-def-backup-c583d8f2-6daf-4648-8ead-ed826c591471-2020-08-24T20-47-05Z.tar.gz
encryptionConfigSecretName: encryptionconfig
```
### Restore an Encrypted Backup from Minio
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-minio
spec:
backupFilename: default-minio-backup-demo-aa5c04b7-4dba-4c48-9ac4-ab7916812eaa-2020-08-30T13-18-17-07-00.tar.gz
storageLocation:
s3:
credentialSecretName: minio-creds
credentialSecretNamespace: default
bucketName: rancherbackups
endpoint: minio.sslip.io
endpointCA: LS0tLS1CRUdJTi3VUFNQkl5UUT.....pbEpWaVzNkRS0tLS0t
encryptionConfigSecretName: test-encryptionconfig
```
### Restore from Backup Using an AWS Credential Secret to Access S3
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-s3-demo
spec:
backupFilename: test-s3-recurring-backup-752ecd87-d958-4d20-8350-072f8d090045-2020-09-26T12-49-34-07-00.tar.gz.enc
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
encryptionConfigSecretName: test-encryptionconfig
```
### Restore from EC2 Nodes with IAM Permissions to Access S3
This example shows that the AWS credential secret does not have to be provided to restore from backup if the nodes running `rancher-backup` have [these permissions for access to S3.](backup-configuration.md#iam-permissions-for-ec2-nodes-to-access-s3)
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-s3-demo
spec:
backupFilename: default-test-s3-recurring-backup-84bf8dd8-0ef3-4240-8ad1-fc7ec308e216-2020-08-24T10#52#44-07#00.tar.gz
storageLocation:
s3:
bucketName: rajashree-backup-test
folder: ecm1
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
encryptionConfigSecretName: test-encryptionconfig
```
# Example Credential Secret for Storing Backups in S3
```yaml
apiVersion: v1
kind: Secret
metadata:
name: creds
type: Opaque
data:
accessKey: <Enter your base64-encoded access key>
secretKey: <Enter your base64-encoded secret key>
```
# Example EncryptionConfiguration
```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aesgcm:
keys:
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
- name: key2
secret: dGhpcyBpcyBwYXNzd29yZA==
- aescbc:
keys:
- name: key1
secret: c2VjcmV0IGlzIHNlY3VyZQ==
- name: key2
secret: dGhpcyBpcyBwYXNzd29yZA==
- secretbox:
keys:
- name: key1
secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=
```
versioned_docs/version-2.5/reference-guides/backup-restore-configuration/restore-configuration.md
+90
View File
@@ -0,0 +1,90 @@
---
title: Restore Configuration
shortTitle: Restore
weight: 2
aliases:
- /rancher/v2.5/en/backups/v2.5/configuration/restore-config
- /rancher/v2.x/en/backups/v2.5/configuration/restore-config/
---
The Restore Create page lets you provide details of the backup to restore from
![](/img/backup_restore/restore/restore.png)
- [Backup Source](#backup-source)
- [An Existing Backup Config](#an-existing-backup-config)
- [The default storage target](#the-default-storage-target)
- [An S3-compatible object store](#an-s3-compatible-object-store)
- [Encryption](#encryption)
- [Prune during restore](#prune-during-restore)
- [Getting the Backup Filename from S3](#getting-the-backup-filename-from-s3)
# Backup Source
Provide details of the backup file and its storage location, which the operator will then use to perform the restore. Select from the following options to provide these details
### An existing backup config
Selecting this option will populate the **Target Backup** dropdown with the Backups available in this cluster. Select the Backup from the dropdown, and that will fill out the **Backup Filename** field for you, and will also pass the backup source information from the selected Backup to the operator.
![](/img/backup_restore/restore/existing.png)
If the Backup custom resource does not exist in the cluster, you need to get the exact filename and provide the backup source details with the default storage target or an S3-compatible object store.
### The default storage target
Select this option if you are restoring from a backup file that exists in the default storage location configured at the operator-level. The operator-level configuration is the storage location that was configured when the `rancher-backup` operator was installed or upgraded. Provide the exact filename in the **Backup Filename** field.
![](/img/backup_restore/restore/default.png)
### An S3-compatible object store
Select this option if no default storage location is configured at the operator-level, OR if the backup file exists in a different S3 bucket than the one configured as the default storage location. Provide the exact filename in the **Backup Filename** field. Refer to [this section](#getting-the-backup-filename-from-s3) for exact steps on getting the backup filename from s3. Fill in all the details for the S3 compatible object store. Its fields are exactly same as ones for the `backup.StorageLocation` configuration in the [Backup custom resource.](backup-configuration.md#storage-location)
![](/img/backup_restore/restore/s3store.png)
# Encryption
If the backup was created with encryption enabled, its file will have `.enc` suffix. Choosing such a Backup, or providing a backup filename with `.enc` suffix will display another dropdown named **Encryption Config Secret**.
![](/img/backup_restore/restore/encryption.png)
The Secret selected from this dropdown must have the same contents as the one used for the Backup custom resource while performing the backup. If the encryption configuration doesn't match, the restore will fail
The `Encryption Config Secret` dropdown will filter out and list only those Secrets that have this exact key
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `encryptionConfigSecretName` | Provide the name of the Secret from `cattle-resources-system` namespace, that contains the encryption config file. |
> **Important**
This field should only be set if the backup was created with encryption enabled. Providing the incorrect encryption config will cause the restore to fail.
# Prune During Restore
* **Prune**: In order to fully restore Rancher from a backup, and to go back to the exact state it was at when the backup was performed, we need to delete any additional resources that were created by Rancher after the backup was taken. The operator does so if the **Prune** flag is enabled. Prune is enabled by default and it is recommended to keep it enabled.
* **Delete Timeout**: This is the amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `prune` | Delete the resources managed by Rancher that are not present in the backup (Recommended). |
| `deleteTimeoutSeconds` | Amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again. |
# Getting the Backup Filename from S3
This is the name of the backup file that the `rancher-backup` operator will use to perform the restore.
To obtain this file name from S3, go to your S3 bucket (and folder if it was specified while performing backup).
Copy the filename and store it in your Restore custom resource. So assuming the name of your backup file is `backupfile`,
- If your bucket name is `s3bucket` and no folder was specified, then the `backupFilename` to use will be `backupfile`.
- If your bucket name is `s3bucket` and the base folder is`s3folder`, the `backupFilename` to use is only `backupfile` .
- If there is a subfolder inside `s3Folder` called `s3sub`, and that has your backup file, then the `backupFilename` to use is `s3sub/backupfile`.
| YAML Directive Name | Description |
| ---------------- | ---------------- |
| `backupFilename` | This is the name of the backup file that the `rancher-backup` operator will use to perform the restore. |
versioned_docs/version-2.5/reference-guides/backup-restore-configuration/storage-configuration.md
+62
View File
@@ -0,0 +1,62 @@
---
title: Backup Storage Location Configuration
shortTitle: Storage
weight: 3
aliases:
- /rancher/v2.5/en/backups/v2.5/configuration/storage-config
- /rancher/v2.x/en/backups/v2.5/configuration/storage-config/
---
Configure a storage location where all backups are saved by default. You will have the option to override this with each backup, but will be limited to using an S3-compatible object store.
Only one storage location can be configured at the operator level.
- [Storage Location Configuration](#storage-location-configuration)
- [No Default Storage Location](#no-default-storage-location)
- [S3-compatible Object Store](#s3-compatible-object-store)
- [Use an existing StorageClass](#existing-storageclass)
- [Use an existing PersistentVolume](#existing-persistent-volume)
- [Encryption](#encryption)
- [Example values.yaml for the rancher-backup Helm Chart](#example-values-yaml-for-the-rancher-backup-helm-chart)
# Storage Location Configuration
### No Default Storage Location
You can choose to not have any operator-level storage location configured. If you select this option, you must configure an S3-compatible object store as the storage location for each individual backup.
### S3-compatible Object Store
| Parameter | Description |
| -------------- | -------------- |
| Credential Secret | Choose the credentials for S3 from your secrets in Rancher. [Example](backups/v2.5/examples/#example-credential-secret-for-storing-backups-in-s3). |
| Bucket Name | Enter the name of the [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html) where the backups will be stored. Default: `rancherbackups`. |
| Region | The [AWS region](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) where the S3 bucket is located. |
| Folder | The [folder in the S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/using-folders.html) where the backups will be stored. |
| Endpoint | The [S3 endpoint](https://docs.aws.amazon.com/general/latest/gr/s3.html) For example, `s3.us-west-2.amazonaws.com`. |
| Endpoint CA | The CA cert used to for the S3 endpoint. Default: base64 encoded CA cert |
| insecureTLSSkipVerify | Set to true if you are not using TLS. |
### Existing StorageClass
Installing the `rancher-backup` chart by selecting the StorageClass option will create a Persistent Volume Claim (PVC), and Kubernetes will in turn dynamically provision a Persistent Volume (PV) where all the backups will be saved by default.
For information about creating storage classes refer to [this section.](../../how-to-guides/advanced-user-guides/manage-clusters/create-kubernetes-persistent-storage/manage-persistent-storage/dynamically-provision-new-storage.md)
> **Important**
It is highly recommended to use a StorageClass with a reclaim policy of "Retain". Otherwise if the PVC created by the `rancher-backup` chart gets deleted (either during app upgrade, or accidentally), the PV will get deleted too, which means all backups saved in it will get deleted.
If no such StorageClass is available, after the PV is provisioned, make sure to edit its reclaim policy and set it to "Retain" before storing backups in it.
### Existing Persistent Volume
Select an existing Persistent Volume (PV) that will be used to store your backups. For information about creating PersistentVolumes in Rancher, refer to [this section.](../../how-to-guides/advanced-user-guides/manage-clusters/create-kubernetes-persistent-storage/manage-persistent-storage/set-up-existing-storage.md#2-add-a-persistent-volume-that-refers-to-the-persistent-storage)
> **Important**
It is highly recommended to use a Persistent Volume with a reclaim policy of "Retain". Otherwise if the PVC created by the `rancher-backup` chart gets deleted (either during app upgrade, or accidentally), the PV will get deleted too, which means all backups saved in it will get deleted.
# Example values.yaml for the rancher-backup Helm Chart
The documented `values.yaml` file that can be used to configure `rancher-backup` operator when the Helm CLI is used can be found in the [backup-restore-operator repository.](https://github.com/rancher/backup-restore-operator/blob/release/v1.0/charts/rancher-backup/values.yaml)
For more information about `values.yaml` files and configuring Helm charts during installation, refer to the [Helm documentation.](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing)
versioned_docs/version-2.5/reference-guides/best-practices/rancher-managed-clusters/logging-best-practices.md
+94
View File
@@ -0,0 +1,94 @@
---
title: Logging Best Practices
weight: 1
aliases:
- /rancher/v2.5/en/best-practices/v2.5/rancher-managed/logging
- /rancher/v2.x/en/best-practices/v2.5/rancher-managed/logging/
---
In this guide, we recommend best practices for cluster-level logging and application logging.
- [Changes in Logging in Rancher v2.5](#changes-in-logging-in-rancher-v2-5)
- [Cluster-level Logging](#cluster-level-logging)
- [Application Logging](#application-logging)
- [General Best Practices](#general-best-practices)
# Changes in Logging in Rancher v2.5
Before Rancher v2.5, logging in Rancher has historically been a pretty static integration. There were a fixed list of aggregators to choose from (ElasticSearch, Splunk, Kafka, Fluentd and Syslog), and only two configuration points to choose (Cluster-level and Project-level).
Logging in 2.5 has been completely overhauled to provide a more flexible experience for log aggregation. With the new logging feature, administrators and users alike can deploy logging that meets fine-grained collection criteria while offering a wider array of destinations and configuration options.
"Under the hood", Rancher logging uses the Banzai Cloud logging operator. We provide manageability of this operator (and its resources), and tie that experience in with managing your Rancher clusters.
# Cluster-level Logging
### Cluster-wide Scraping
For some users, it is desirable to scrape logs from every container running in the cluster. This usually coincides with your security team's request (or requirement) to collect all logs from all points of execution.
In this scenario, it is recommended to create at least two _ClusterOutput_ objects - one for your security team (if you have that requirement), and one for yourselves, the cluster administrators. When creating these objects take care to choose an output endpoint that can handle the significant log traffic coming from the entire cluster. Also make sure to choose an appropriate index to receive all these logs.
Once you have created these _ClusterOutput_ objects, create a _ClusterFlow_ to collect all the logs. Do not define any _Include_ or _Exclude_ rules on this flow. This will ensure that all logs from across the cluster are collected. If you have two _ClusterOutputs_, make sure to send logs to both of them.
### Kubernetes Components
_ClusterFlows_ have the ability to collect logs from all containers on all hosts in the Kubernetes cluster. This works well in cases where those containers are part of a Kubernetes pod; however, RKE containers exist outside of the scope of Kubernetes.
Currently (as of v2.5.1) the logs from RKE containers are collected, but are not able to easily be filtered. This is because those logs do not contain information as to the source container (e.g. `etcd` or `kube-apiserver`).
A future release of Rancher will include the source container name which will enable filtering of these component logs. Once that change is made, you will be able to customize a _ClusterFlow_ to retrieve **only** the Kubernetes component logs, and direct them to an appropriate output.
# Application Logging
Best practice not only in Kubernetes but in all container-based applications is to direct application logs to `stdout`/`stderr`. The container runtime will then trap these logs and do **something** with them - typically writing them to a file. Depending on the container runtime (and its configuration), these logs can end up in any number of locations.
In the case of writing the logs to a file, Kubernetes helps by creating a `/var/log/containers` directory on each host. This directory symlinks the log files to their actual destination (which can differ based on configuration or container runtime).
Rancher logging will read all log entries in `/var/log/containers`, ensuring that all log entries from all containers (assuming a default configuration) will have the opportunity to be collected and processed.
### Specific Log Files
Log collection only retrieves `stdout`/`stderr` logs from pods in Kubernetes. But what if we want to collect logs from other files that are generated by applications? Here, a log streaming sidecar (or two) may come in handy.
The goal of setting up a streaming sidecar is to take log files that are written to disk, and have their contents streamed to `stdout`. This way, the Banzai Logging Operator can pick up those logs and send them to your desired output.
To set this up, edit your workload resource (e.g. Deployment) and add the following sidecar definition:
```
...
containers:
- args:
- -F
- /path/to/your/log/file.log
command:
- tail
image: busybox
name: stream-log-file-[name]
volumeMounts:
- mountPath: /path/to/your/log
name: mounted-log
...
```
This will add a container to your workload definition that will now stream the contents of (in this example) `/path/to/your/log/file.log` to `stdout`.
This log stream is then automatically collected according to any _Flows_ or _ClusterFlows_ you have setup. You may also wish to consider creating a _Flow_ specifically for this log file by targeting the name of the container. See example:
```
...
spec:
match:
- select:
container_names:
- stream-log-file-name
...
```
# General Best Practices
- Where possible, output structured log entries (e.g. `syslog`, JSON). This makes handling of the log entry easier as there are already parsers written for these formats.
- Try to provide the name of the application that is creating the log entry, in the entry itself. This can make troubleshooting easier as Kubernetes objects do not always carry the name of the application as the object name. For instance, a pod ID may be something like `myapp-098kjhsdf098sdf98` which does not provide much information about the application running inside the container.
- Except in the case of collecting all logs cluster-wide, try to scope your _Flow_ and _ClusterFlow_ objects tightly. This makes it easier to troubleshoot when problems arise, and also helps ensure unrelated log entries do not show up in your aggregator. An example of tight scoping would be to constrain a _Flow_ to a single _Deployment_ in a namespace, or perhaps even a single container within a _Pod_.
- Keep the log verbosity down except when troubleshooting. High log verbosity poses a number of issues, chief among them being **noise**: significant events can be drowned out in a sea of `DEBUG` messages. This is somewhat mitigated with automated alerting and scripting, but highly verbose logging still places an inordinate amount of stress on the logging infrastructure.
- Where possible, try to provide a transaction or request ID with the log entry. This can make tracing application activity across multiple log sources easier, especially when dealing with distributed applications.
versioned_docs/version-2.5/reference-guides/best-practices/rancher-managed-clusters/monitoring-best-practices.md
+123
View File
@@ -0,0 +1,123 @@
---
title: Monitoring Best Practices
weight: 2
aliases:
- /rancher/v2.5/en/best-practices/v2.5/rancher-managed/monitoring
- /rancher/v2.x/en/best-practices/v2.5/rancher-managed/monitoring/
---
Configuring sensible monitoring and alerting rules is vital for running any production workloads securely and reliably. This is not different when using Kubernetes and Rancher. Fortunately the integrated monitoring and alerting functionality makes this whole process a lot easier.
The [Rancher monitoring documentation](../../../pages-for-subheaders/monitoring-and-alerting.md) describes how you can set up a complete Prometheus and Grafana stack. Out of the box this will scrape monitoring data from all system and Kubernetes components in your cluster and provide sensible dashboards and alerts for them to get started. But for a reliable setup, you also need to monitor your own workloads and adapt Prometheus and Grafana to your own specific use cases and cluster sizes. This document aims to give you best practices for this.
- [What to Monitor](#what-to-monitor)
- [Configuring Prometheus Resource Usage](#configuring-prometheus-resource-usage)
- [Scraping Custom Workloads](#scraping-custom-workloads)
- [Monitoring in a (Micro)Service Architecture](#monitoring-in-a-micro-service-architecture)
- [Real User Monitoring](#real-user-monitoring)
- [Security Monitoring](#security-monitoring)
- [Setting up Alerts](#setting-up-alerts)
# What to Monitor
Kubernetes itself, as well as applications running inside of it, form a distributed system where different components interact with each other. For the whole system and each individual component, you have to ensure performance, availability, reliability and scalability. A good resource with more details and information is Google's free [Site Reliability Engineering Book](https://landing.google.com/sre/sre-book/), especially the chapter about [Monitoring distributed systems](https://landing.google.com/sre/sre-book/chapters/monitoring-distributed-systems/).
# Configuring Prometheus Resource Usage
When installing the integrated monitoring stack, Rancher allows to configure several settings that are dependent on the size of your cluster and the workloads running in it. This chapter covers these in more detail.
### Storage and Data Retention
The amount of storage needed for Prometheus directly correlates to the amount of time series and labels that you store and the data retention you have configured. It is important to note that Prometheus is not meant to be used as a long-term metrics storage. Data retention time is usually only a couple of days and not weeks or months. The reason for this is that Prometheus does not perform any aggregation on its stored metrics. This is great because aggregation can dilute data, but it also means that the needed storage grows linearly over time without retention.
One way to calculate the necessary storage is to look at the average size of a storage chunk in Prometheus with this query
```
rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[1h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[1h])
```
Next, find out your data ingestion rate per second:
```
rate(prometheus_tsdb_head_samples_appended_total[1h])
```
and then multiply this with the retention time, adding a few percentage points as buffer:
```
average chunk size in bytes * ingestion rate per second * retention time in seconds * 1.1 = necessary storage in bytes
```
You can find more information about how to calculate the necessary storage in this [blog post](https://www.robustperception.io/how-much-disk-space-do-prometheus-blocks-use).
You can read more about the Prometheus storage concept in the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/storage).
### CPU and Memory Requests and Limits
In larger Kubernetes clusters Prometheus can consume quite a bit of memory. The amount of memory Prometheus needs directly correlates to the amount of time series and amount of labels it stores and the scrape interval in which these are filled.
You can find more information about how to calculate the necessary memory in this [blog post](https://www.robustperception.io/how-much-ram-does-prometheus-2-x-need-for-cardinality-and-ingestion).
The amount of necessary CPUs correlate with the amount of queries you are performing.
### Federation and Long-term Storage
Prometheus is not meant to store metrics for a long amount of time, but should only be used for short term storage.
In order to store some, or all metrics for a long time, you can leverage Prometheus' [remote read/write](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations) capabilities to connect it to storage systems like [Thanos](https://thanos.io/), [InfluxDB](https://www.influxdata.com/), [M3DB](https://www.m3db.io/), or others. You can find an example setup in this [blog post](https://rancher.com/blog/2020/prometheus-metric-federation).
# Scraping Custom Workloads
While the integrated Rancher Monitoring already scrapes system metrics from a cluster's nodes and system components, the custom workloads that you deploy on Kubernetes should also be scraped for data. For that you can configure Prometheus to do an HTTP request to an endpoint of your applications in a certain interval. These endpoints should then return their metrics in a Prometheus format.
In general, you want to scrape data from all the workloads running in your cluster so that you can use them for alerts or debugging issues. Often, you recognize that you need some data only when you actually need the metrics during an incident. It is good, if it is already scraped and stored. Since Prometheus is only meant to be a short-term metrics storage, scraping and keeping lots of data is usually not that expensive. If you are using a long-term storage solution with Prometheus, you can then still decide which data you are actually persisting and keeping there.
### About Prometheus Exporters
A lot of 3rd party workloads like databases, queues or web-servers either already support exposing metrics in a Prometheus format, or there are so called exporters available that translate between the tool's metrics and the format that Prometheus understands. Usually you can add these exporters as additional sidecar containers to the workload's Pods. A lot of helm charts already include options to deploy the correct exporter. Additionally you can find a curated list of exports by SysDig on [promcat.io](https://promcat.io/) and on [ExporterHub](https://exporterhub.io/).
### Prometheus support in Programming Languages and Frameworks
To get your own custom application metrics into Prometheus, you have to collect and expose these metrics directly from your application's code. Fortunately, there are already libraries and integrations available to help with this for most popular programming languages and frameworks. One example for this is the Prometheus support in the [Spring Framework](https://docs.spring.io/spring-metrics/docs/current/public/prometheus).
### ServiceMonitors and PodMonitors
Once all your workloads expose metrics in a Prometheus format, you have to configure Prometheus to scrape it. Under the hood Rancher is using the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator). This makes it easy to add additional scraping targets with ServiceMonitors and PodMonitors. A lot of helm charts already include an option to create these monitors directly. You can also find more information in the Rancher documentation.
### Prometheus Push Gateway
There are some workloads that are traditionally hard to scrape by Prometheus. Examples for these are short lived workloads like Jobs and CronJobs, or applications that do not allow sharing data between individual handled incoming requests, like PHP applications.
To still get metrics for these use cases, you can set up [prometheus-pushgateways](https://github.com/prometheus/pushgateway). The CronJob or PHP application would push metric updates to the pushgateway. The pushgateway aggregates and exposes them through an HTTP endpoint, which then can be scraped by Prometheus.
### Prometheus Blackbox Monitor
Sometimes it is useful to monitor workloads from the outside. For this, you can use the [Prometheus blackbox-exporter](https://github.com/prometheus/blackbox_exporter) which allows probing any kind of endpoint over HTTP, HTTPS, DNS, TCP and ICMP.
# Monitoring in a (Micro)Service Architecture
If you have a (micro)service architecture where multiple individual workloads within your cluster are communicating with each other, it is really important to have detailed metrics and traces about this traffic to understand how all these workloads are communicating with each other and where a problem or bottleneck may be.
Of course you can monitor all this internal traffic in all your workloads and expose these metrics to Prometheus. But this can quickly become quite work intensive. Service Meshes like Istio, which can be installed with [a click](../../../pages-for-subheaders/istio.md) in Rancher, can do this automatically and provide rich telemetry about the traffic between all services.
# Real User Monitoring
Monitoring the availability and performance of all your internal workloads is vitally important to run stable, reliable and fast applications. But these metrics only show you parts of the picture. To get a complete view it is also necessary to know how your end users are actually perceiving it. For this you can look into various [Real user monitoring solutions](https://en.wikipedia.org/wiki/Real_user_monitoring).
# Security Monitoring
In addition to monitoring workloads to detect performance, availability or scalability problems, the cluster and the workloads running into it should also be monitored for potential security problems. A good starting point is to frequently run and alert on [CIS Scans](cis-scans/v2.5/) which check if the cluster is configured according to security best practices.
For the workloads, you can have a look at Kubernetes and Container security solutions like [Falco](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).
# Setting up Alerts
Getting all the metrics into a monitoring systems and visualizing them in dashboards is great, but you also want to be pro-actively alerted if something goes wrong.
The integrated Rancher monitoring already configures a sensible set of alerts that make sense in any Kubernetes cluster. You should extend these to cover your specific workloads and use cases.
When setting up alerts, configure them for all the workloads that are critical to the availability of your applications. But also make sure that they are not too noisy. Ideally every alert you are receiving should be because of a problem that needs your attention and needs to be fixed. If you have alerts that are firing all the time but are not that critical, there is a danger that you start ignoring your alerts all together and then miss the real important ones. Less may be more here. Start to focus on the real important metrics first, for example alert if your application is offline. Fix all the problems that start to pop up and then start to create more detailed alerts.
If an alert starts firing, but there is nothing you can do about it at the moment, it's also fine to silence the alert for a certain amount of time, so that you can look at it later.
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation](../../../pages-for-subheaders/monitoring-and-alerting.md).
versioned_docs/version-2.5/reference-guides/best-practices/rancher-managed-clusters/rancher-managed-clusters-in-vsphere.md
+62
View File
@@ -0,0 +1,62 @@
---
title: Best Practices for Rancher Managed vSphere Clusters
shortTitle: Rancher Managed Clusters in vSphere
aliases:
- /rancher/v2.5/en/best-practices/v2.5/rancher-managed/managed-vsphere
- /rancher/v2.x/en/best-practices/v2.5/rancher-managed/managed-vsphere/
---
This guide outlines a reference architecture for provisioning downstream Rancher clusters in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.
- [1. VM Considerations](#1-vm-considerations)
- [2. Network Considerations](#2-network-considerations)
- [3. Storage Considerations](#3-storage-considerations)
- [4. Backups and Disaster Recovery](#4-backups-and-disaster-recovery)
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/solution_overview.drawio.svg)
# 1. VM Considerations
### Leverage VM Templates to Construct the Environment
To facilitate consistency across the deployed Virtual Machines across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customisation options.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Downstream Cluster Nodes Across ESXi Hosts
Doing so will ensure node VM's are spread across multiple ESXi hosts - preventing a single point of failure at the host level.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Downstream Cluster Nodes Across Datastores
Doing so will ensure node VM's are spread across multiple datastores - preventing a single point of failure at the datastore level.
### Configure VM's as Appropriate for Kubernetes
It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.
# 2. Network Considerations
### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes
Deploy etcd members within a single data center where possible to avoid latency overheads and reduce the likelihood of network partitioning. For most setups, 1Gb connections will suffice. For large clusters, 10Gb connections can reduce the time taken to restore from backup.
### Consistent IP Addressing for VM's
Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
# 3. Storage Considerations
### Leverage SSD Drives for ETCD Nodes
ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible.
# 4. Backups and Disaster Recovery
### Perform Regular Downstream Cluster Backups
Kubernetes uses etcd to store all its data - from configuration, state and metadata. Backing this up is crucial in the event of disaster recovery.
### Back up Downstream Node VMs
Incorporate the Rancher downstream node VM's within a standard VM backup policy.
versioned_docs/version-2.5/reference-guides/best-practices/rancher-managed-clusters/tips-to-set-up-containers.md
+53
View File
@@ -0,0 +1,53 @@
---
title: Tips for Setting Up Containers
weight: 100
aliases:
- /rancher/v2.5/en/best-practices/containers
- /rancher/v2.5/en/best-practices/v2.5/rancher-managed/containers
- /rancher/v2.x/en/best-practices/v2.5/rancher-managed/containers/
---
Running well-built containers can greatly impact the overall performance and security of your environment.
Below are a few tips for setting up your containers.
For a more detailed discussion of security for containers, you can also refer to Rancher's [Guide to Container Security.](https://rancher.com/complete-guide-container-security)
### Use a Common Container OS
When possible, you should try to standardize on a common container base OS.
Smaller distributions such as Alpine and BusyBox reduce container image size and generally have a smaller attack/vulnerability surface.
Popular distributions such as Ubuntu, Fedora, and CentOS are more field-tested and offer more functionality.
### Start with a FROM scratch container
If your microservice is a standalone static binary, you should use a FROM scratch container.
The FROM scratch container is an [official Docker image](https://hub.docker.com/_/scratch) that is empty so that you can use it to design minimal images.
This will have the smallest attack surface and smallest image size.
### Run Container Processes as Unprivileged
When possible, use a non-privileged user when running processes within your container. While container runtimes provide isolation, vulnerabilities and attacks are still possible. Inadvertent or accidental host mounts can also be impacted if the container is running as root. For details on configuring a security context for a pod or container, refer to the [Kubernetes docs](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
### Define Resource Limits
Apply CPU and memory limits to your pods. This can help manage the resources on your worker nodes and avoid a malfunctioning microservice from impacting other microservices.
In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the Rancher docs.
When setting resource quotas, if you set anything related to CPU or Memory (i.e. limits or reservations) on a project or namespace, all containers will require a respective CPU or Memory field set during creation. To avoid setting these limits on each and every container during workload creation, a default container resource limit can be specified on the namespace.
The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the namespace level.
### Define Resource Requirements
You should apply CPU and memory requirements to your pods. This is crucial for informing the scheduler which type of compute node your pod needs to be placed on, and ensuring it does not over-provision that node. In Kubernetes, you can set a resource requirement by defining `resources.requests` in the resource requests field in a pod's container spec. For details, refer to the [Kubernetes docs](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container).
> **Note:** If you set a resource limit for the namespace that the pod is deployed in, and the container doesn't have a specific resource request, the pod will not be allowed to start. To avoid setting these fields on each and every container during workload creation, a default container resource limit can be specified on the namespace.
It is recommended to define resource requirements on the container level because otherwise, the scheduler makes assumptions that will likely not be helpful to your application when the cluster experiences load.
### Liveness and Readiness Probes
Set up liveness and readiness probes for your container. Unless your container completely crashes, Kubernetes will not know it's unhealthy unless you create an endpoint or mechanism that can report container status. Alternatively, make sure your container halts and crashes if unhealthy.
The Kubernetes docs show how to [configure liveness and readiness probes for containers.](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
versioned_docs/version-2.5/reference-guides/best-practices/rancher-server/on-premises-rancher-in-vsphere.md
+94
View File
@@ -0,0 +1,94 @@
---
title: Installing Rancher in a vSphere Environment
shortTitle: On-Premises Rancher in vSphere
weight: 3
aliases:
- /rancher/v2.5/en/best-practices/v2.5/rancher-server/rancher-in-vsphere
- /rancher/v2.x/en/best-practices/v2.5/rancher-server/rancher-in-vsphere/
---
This guide outlines a reference architecture for installing Rancher on an RKE Kubernetes cluster in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.
- [1. Load Balancer Considerations](#1-load-balancer-considerations)
- [2. VM Considerations](#2-vm-considerations)
- [3. Network Considerations](#3-network-considerations)
- [4. Storage Considerations](#4-storage-considerations)
- [5. Backups and Disaster Recovery](#5-backups-and-disaster-recovery)
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/rancher-on-prem-vsphere.svg)
# 1. Load Balancer Considerations
A load balancer is required to direct traffic to the Rancher workloads residing on the RKE nodes.
### Leverage Fault Tolerance and High Availability
Leverage the use of an external (hardware or software) load balancer that has inherit high-availability functionality (F5, NSX-T, Keepalived, etc).
### Back Up Load Balancer Configuration
In the event of a Disaster Recovery activity, availability of the Load balancer configuration will expedite the recovery process.
### Configure Health Checks
Configure the Load balancer to automatically mark nodes as unavailable if a health check is failed. For example, NGINX can facilitate this with:
`max_fails=3 fail_timeout=5s`
### Leverage an External Load Balancer
Avoid implementing a software load balancer within the management cluster.
### Secure Access to Rancher
Configure appropriate Firewall / ACL rules to only expose access to Rancher
# 2. VM Considerations
### Size the VM's According to Rancher Documentation
https://rancher.com/docs/rancher/v2.5/en/installation/requirements/
### Leverage VM Templates to Construct the Environment
To facilitate the consistency of Virtual Machines deployed across the environment, consider the use of "Golden Images" in the form of VM templates. Packer can be used to accomplish this, adding greater customization options.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Rancher Cluster Nodes Across ESXi Hosts
Doing so will ensure node VM's are spread across multiple ESXi hosts - preventing a single point of failure at the host level.
### Leverage DRS Anti-Affinity Rules (Where Possible) to Separate Rancher Cluster Nodes Across Datastores
Doing so will ensure node VM's are spread across multiple datastores - preventing a single point of failure at the datastore level.
### Configure VM's as Appropriate for Kubernetes
It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.
# 3. Network Considerations
### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes
Deploy etcd members within a single data center where possible to avoid latency overheads and reduce the likelihood of network partitioning. For most setups, 1Gb connections will suffice. For large clusters, 10Gb connections can reduce the time taken to restore from backup.
### Consistent IP Addressing for VM's
Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
# 4. Storage Considerations
### Leverage SSD Drives for ETCD Nodes
ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible.
# 5. Backups and Disaster Recovery
### Perform Regular Management Cluster Backups
Rancher stores its data in the ETCD datastore of the Kubernetes cluster it resides on. Like with any Kubernetes cluster, perform frequent, tested backups of this cluster.
### Back up Rancher Cluster Node VMs
Incorporate the Rancher management node VM's within a standard VM backup policy.
versioned_docs/version-2.5/reference-guides/best-practices/rancher-server/rancher-deployment-strategy.md
+48
View File
@@ -0,0 +1,48 @@
---
title: Rancher Deployment Strategy
weight: 100
aliases:
- /rancher/v2.5/en/best-practices/v2.5/rancher-server/deployment-strategies
- /rancher/v2.x/en/best-practices/v2.5/rancher-server/deployment-strategies/
---
There are two recommended deployment strategies for a Rancher server that manages downstream Kubernetes clusters. Each one has its own pros and cons. Read more about which one would fit best for your use case:
* [Hub and Spoke](#hub-and-spoke-strategy)
* [Regional](#regional-strategy)
# Hub & Spoke Strategy
---
In this deployment scenario, there is a single Rancher control plane managing Kubernetes clusters across the globe. The control plane would be run on a high-availability Kubernetes cluster, and there would be impact due to latencies.
![](/img/bpg/hub-and-spoke.png)
### Pros
* Environments could have nodes and network connectivity across regions.
* Single control plane interface to view/see all regions and environments.
* Kubernetes does not require Rancher to operate and can tolerate losing connectivity to the Rancher control plane.
### Cons
* Subject to network latencies.
* If the control plane goes out, global provisioning of new services is unavailable until it is restored. However, each Kubernetes cluster can continue to be managed individually.
# Regional Strategy
---
In the regional deployment model a control plane is deployed in close proximity to the compute nodes.
![](/img/bpg/regional.png)
### Pros
* Rancher functionality in regions stay operational if a control plane in another region goes down.
* Network latency is greatly reduced, improving the performance of functionality in Rancher.
* Upgrades of the Rancher control plane can be done independently per region.
### Cons
* Overhead of managing multiple Rancher installations.
* Visibility across global Kubernetes clusters requires multiple interfaces/panes of glass.
* Deploying multi-cluster apps in Rancher requires repeating the process for each Rancher server.
versioned_docs/version-2.5/reference-guides/best-practices/rancher-server/tips-for-running-rancher.md
+40
View File
@@ -0,0 +1,40 @@
---
title: Tips for Running Rancher
weight: 100
aliases:
- /rancher/v2.5/en/best-practices/deployment-types
- /rancher/v2.5/en/best-practices/v2.5/rancher-server/deployment-types
- /rancher/v2.x/en/best-practices/v2.5/rancher-server/deployment-types/
---
This guide is geared toward use cases where Rancher is used to manage downstream Kubernetes clusters. The high-availability setup is intended to prevent losing access to downstream clusters if the Rancher server is not available.
A high-availability Kubernetes installation, defined as an installation of Rancher on a Kubernetes cluster with at least three nodes, should be used in any production installation of Rancher, as well as any installation deemed "important." Multiple Rancher instances running on multiple nodes ensure high availability that cannot be accomplished with a single node environment.
If you are installing Rancher in a vSphere environment, refer to the best practices documented [here.](on-premises-rancher-in-vsphere.md)
When you set up your high-availability Rancher installation, consider the following:
### Run Rancher on a Separate Cluster
Don't run other workloads or microservices in the Kubernetes cluster that Rancher is installed on.
### Make sure nodes are configured correctly for Kubernetes ###
It's important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd. More details can be found in the [kubernetes docs](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) and [etcd's performance op guide](https://etcd.io/docs/v3.4/op-guide/performance/).
### When using RKE: Back up the Statefile
RKE keeps record of the cluster state in a file called `cluster.rkestate`. This file is important for the recovery of a cluster and/or the continued maintenance of the cluster through RKE. Because this file contains certificate material, we strongly recommend encrypting this file before backing up. After each run of `rke up` you should backup the state file.
### Run All Nodes in the Cluster in the Same Datacenter
For best performance, run all three of your nodes in the same geographic datacenter. If you are running nodes in the cloud, such as AWS, run each node in a separate Availability Zone. For example, launch node 1 in us-west-2a, node 2 in us-west-2b, and node 3 in us-west-2c.
### Development and Production Environments Should be Similar
It's strongly recommended to have a "staging" or "pre-production" environment of the Kubernetes cluster that Rancher runs on. This environment should mirror your production environment as closely as possible in terms of software and hardware configuration.
### Monitor Your Clusters to Plan Capacity
The Rancher server's Kubernetes cluster should run within the [system and hardware requirements](../../../pages-for-subheaders/installation-requirements.md) as closely as possible. The more you deviate from the system and hardware requirements, the more risk you take.
However, metrics-driven capacity planning analysis should be the ultimate guidance for scaling Rancher, because the published requirements take into account a variety of workload types.
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with Prometheus, a leading open-source monitoring solution, and Grafana, which lets you visualize the metrics from Prometheus.
After you [enable monitoring](../../../pages-for-subheaders/monitoring-and-alerting.md) in the cluster, you can set up [a notification channel](../../../pages-for-subheaders/monitoring-and-alerting.md) and alerts to let you know if your cluster is approaching its capacity. You can also use the Prometheus and Grafana monitoring framework to establish a baseline for key metrics as you scale.
versioned_docs/version-2.5/reference-guides/cli-with-rancher/kubectl-utility.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/cli-with-rancher/rancher-cli.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/amazon-ec2.md
+47
View File
@@ -0,0 +1,47 @@
---
title: EC2 Node Template Configuration
weight: 1
aliases:
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/ec2/ec2-node-template-config/
---
For more details about EC2, nodes, refer to the official documentation for the [EC2 Management Console](https://aws.amazon.com/ec2).
### Region
In the **Region** field, select the same region that you used when creating your cloud credentials.
### Cloud Credentials
Your AWS account access information, stored in a [cloud credential.](../../../user-settings/manage-cloud-credentials.md)
See [Amazon Documentation: Creating Access Keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) how to create an Access Key and Secret Key.
See [Amazon Documentation: Creating IAM Policies (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start) how to create an IAM policy.
See [Amazon Documentation: Adding Permissions to a User (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) how to attach an IAM
See our three example JSON policies:
- [Example IAM Policy](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy)
- [Example IAM Policy with PassRole](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy-with-passrole) (needed if you want to use [Kubernetes Cloud Provider](cluster-provisioning/rke-clusters/options/cloud-providers) or want to pass an IAM Profile to an instance)
- [Example IAM Policy to allow encrypted EBS volumes](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md#example-iam-policy-to-allow-encrypted-ebs-volumes) policy to an user.
### Authenticate & Configure Nodes
Choose an availability zone and network settings for your cluster.
### Security Group
Choose the default security group or configure a security group.
Please refer to [Amazon EC2 security group when using Node Driver](../../../../getting-started/installation-and-upgrade/installation-requirements/port-requirements.md#rancher-aws-ec2-security-group) to see what rules are created in the `rancher-nodes` Security Group.
### Instance Options
Configure the instances that will be created. Make sure you configure the correct **SSH User** for the configured AMI. It is possible that a selected region does not support the default instance type. In this scenario you must select an instance type that does exist, otherwise an error will occur stating the requested configuration is not supported.
If you need to pass an **IAM Instance Profile Name** (not ARN), for example, when you want to use a [Kubernetes Cloud Provider](cluster-provisioning/rke-clusters/options/cloud-providers), you will need an additional permission in your policy. See [Example IAM policy with PassRole](#example-iam-policy-with-passrole) for an example policy.
### Engine Options
In the **Engine Options** section of the node template, you can configure the Docker daemon. You may want to specify the docker version or a Docker registry mirror.
versioned_docs/version-2.5/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/azure.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Azure Node Template Configuration
weight: 1
aliases:
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/azure/azure-node-template-config/
---
For more information about Azure, refer to the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/?product=featured)
Account access information is stored as a cloud credential. Cloud credentials are stored as Kubernetes secrets. Multiple node templates can use the same cloud credential. You can use an existing cloud credential or create a new one.
- **Placement** sets the geographical region where your cluster is hosted and other location metadata.
- **Network** configures the networking used in your cluster.
- **Instance** customizes your VM configuration.
The [Docker daemon](https://docs.docker.com/engine/docker-overview/#the-docker-daemon) configuration options include:
- **Labels:** For information on labels, refer to the [Docker object label documentation.](https://docs.docker.com/config/labels-custom-metadata/)
- **Docker Engine Install URL:** Determines what Docker version will be installed on the instance.
- **Registry mirrors:** Docker Registry mirror to be used by the Docker daemon
- **Other advanced options:** Refer to the [Docker daemon option reference](https://docs.docker.com/engine/reference/commandline/dockerd/)
versioned_docs/version-2.5/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/digitalocean.md
+21
View File
@@ -0,0 +1,21 @@
---
title: DigitalOcean Node Template Configuration
weight: 1
aliases:
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/digital-ocean/do-node-template-config/
----
Account access information is stored as a cloud credential. Cloud credentials are stored as Kubernetes secrets. Multiple node templates can use the same cloud credential. You can use an existing cloud credential or create a new one.
### Droplet Options
The **Droplet Options** provision your cluster's geographical region and specifications.
### Docker Daemon
The [Docker daemon](https://docs.docker.com/engine/docker-overview/#the-docker-daemon) configuration options include:
- **Labels:** For information on labels, refer to the [Docker object label documentation.](https://docs.docker.com/config/labels-custom-metadata/)
- **Docker Engine Install URL:** Determines what Docker version will be installed on the instance.
- **Registry mirrors:** Docker Registry mirror to be used by the Docker daemon
- **Other advanced options:** Refer to the [Docker daemon option reference](https://docs.docker.com/engine/reference/commandline/dockerd/)
versioned_docs/version-2.5/reference-guides/cluster-configuration/downstream-cluster-configuration/node-template-configuration/vsphere.md
+95
View File
@@ -0,0 +1,95 @@
---
title: VSphere Node Template Configuration
weight: 2
aliases:
- /rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference
- /rancher/v2.5/en/cluster-provisionin/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/vsphere-node-template-config/
---
The following node template configuration reference applies to Rancher v2.3.3+.
- [Account Access](#account-access)
- [Scheduling](#scheduling)
- [Instance Options](#instance-options)
- [Networks](#networks)
- [Node tags and custom attributes](#node-tags-and-custom-attributes)
- [cloud-init](#cloud-init)
# Account Access
| Parameter | Required | Description |
|:----------------------|:--------:|:-----|
| Cloud Credentials | * | Your vSphere account access information, stored in a [cloud credential.](../../../user-settings/manage-cloud-credentials.md) |
Your cloud credential has these fields:
| Credential Field | Description |
|-----------------|--------------|
| vCenter or ESXi Server | Enter the vCenter or ESXi hostname/IP. ESXi is the virtualization platform where you create and run virtual machines and virtual appliances. vCenter Server is the service through which you manage multiple hosts connected in a network and pool host resources. |
| Port | Optional: configure configure the port of the vCenter or ESXi server. |
| Username and password | Enter your vSphere login username and password. |
# Scheduling
Choose what hypervisor the virtual machine will be scheduled to.
The fields in the **Scheduling** section should auto-populate with the data center and other scheduling options that are available to you in vSphere.
| Field | Required | Explanation |
|---------|---------------|-----------|
| Data Center | * | Choose the name/path of the data center where the VM will be scheduled. |
| Resource Pool | | Name of the resource pool to schedule the VMs in. Resource pools can be used to partition available CPU and memory resources of a standalone host or cluster, and they can also be nested. Leave blank for standalone ESXi. If not specified, the default resource pool is used. |
| Data Store | * | If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk. |
| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The VM folders in this dropdown menu directly correspond to your VM folders in vSphere. The folder name should be prefaced with `vm/` in your vSphere config file. |
| Host | | The IP of the host system to schedule VMs in. Leave this field blank for a standalone ESXi or for a cluster with DRS (Distributed Resource Scheduler). If specified, the host system's pool will be used and the **Resource Pool** parameter will be ignored. |
# Instance Options
In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.
| Parameter | Required | Description |
|:----------------|:--------:|:-----------|
| CPUs | * | Number of vCPUS to assign to VMs. |
| Memory | * | Amount of memory to assign to VMs. |
| Disk | * | Size of the disk (in MB) to attach to the VMs. |
| Creation method | * | The method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template. Depending on the creation method, you will also have to specify a VM template, content library, existing VM, or ISO. For more information on creation methods, refer to [About VM Creation Methods.](#about-vm-creation-methods) |
| Cloud Init | | URL of a `cloud-config.yml` file or URL to provision VMs with. This file allows further customization of the operating system, such as network configuration, DNS servers, or system daemons. The operating system must support `cloud-init`. |
| Networks | | Name(s) of the network to attach the VM to. |
| Configuration Parameters used for guestinfo | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{<baseurl>}}/os/v1.x/en/installation/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). |
### About VM Creation Methods
In the **Creation method** field, configure the method used to provision VMs in vSphere. Available options include creating VMs that boot from a RancherOS ISO or creating VMs by cloning from an existing virtual machine or [VM template](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html).
The existing VM or template may use any modern Linux operating system that is configured with support for [cloud-init](https://cloudinit.readthedocs.io/en/latest/) using the [NoCloud datasource](https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html).
Choose the way that the VM will be created:
- **Deploy from template: Data Center:** Choose a VM template that exists in the data center that you selected.
- **Deploy from template: Content Library:** First, select the [Content Library](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-254B2CE8-20A8-43F0-90E8-3F6776C2C896.html) that contains your template, then select the template from the populated list **Library templates.**
- **Clone an existing virtual machine:** In the **Virtual machine** field, choose an existing VM that the new VM will be cloned from.
- **Install from boot2docker ISO:** Ensure that the **OS ISO URL** field contains the URL of a VMware ISO release for RancherOS (`rancheros-vmware.iso`). Note that this URL must be accessible from the nodes running your Rancher server installation.
# Networks
The node template now allows a VM to be provisioned with multiple networks. In the **Networks** field, you can now click **Add Network** to add any networks available to you in vSphere.
# Node Tags and Custom Attributes
Tags allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.
For tags, all your vSphere tags will show up as options to select from in your node template.
In the custom attributes, Rancher will let you select all the custom attributes you have already set up in vSphere. The custom attributes are keys and you can enter values for each one.
> **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere.
# cloud-init
[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) allows you to initialize your nodes by applying configuration on the first boot. This may involve things such as creating users, authorizing SSH keys or setting up the network.
To make use of cloud-init initialization, create a cloud config file using valid YAML syntax and paste the file content in the the **Cloud Init** field. Refer to the [cloud-init documentation.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html) for a commented set of examples of supported cloud config directives.
Note that cloud-init is not supported when using the ISO creation method.
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/eks-cluster-configuration.md
+424
View File
@@ -0,0 +1,424 @@
---
title: EKS Cluster Configuration Reference
shortTitle: EKS Cluster Configuration
weight: 2
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
<TabItem value="Rancher v2.5.6+">
### Account Access
<a id="account-access-2-5-6"></a>
Complete each drop-down and field using the information obtained for your IAM policy.
| Setting | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
| Region | From the drop-down choose the geographical region in which to build your cluster. |
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.](../../user-settings/manage-cloud-credentials.md) |
### Service Role
<a id="service-role-2-5-6"></a>
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
Service Role | Description
-------------|---------------------------
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
### Secrets Encryption
<a id="secrets-encryption-2-5-6"></a>
Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html)
### API Server Endpoint Access
<a id="api-server-endpoint-access-2-5-6"></a>
Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Private-only API Endpoints
If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API.
There are two ways to avoid this extra manual step:
- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster.
- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html).
### Public Access Endpoints
<a id="public-access-endpoints-2-5-6"></a>
Optionally limit access to the public endpoint via explicit CIDR blocks.
If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster.
One of the following is required to enable private access:
- Rancher's IP must be part of an allowed CIDR block
- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group
For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Subnet
<a id="subnet-2-5-6"></a>
| Option | Description |
| ------- | ------------ |
| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. |
| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). |
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
### Security Group
<a id="security-group-2-5-6"></a>
Amazon Documentation:
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
### Logging
<a id="logging-2-5-6"></a>
Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters.
Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation.
For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
### Managed Node Groups
<a id="managed-node-groups-2-5-6"></a>
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)
#### Bring your own launch template
A launch template ID and version can be provided in order to easily configure the EC2 instances in a node group. If a launch template is provided, then none of the settings below will be configurable in Rancher. Therefore, using a launch template would require that all the necessary and desired settings from the list below would need to be specified in the launch template. Also note that if a launch template ID and version is provided, then only the template version can be updated. Using a new template ID would require creating a new managed node group.
| Option | Description | Required/Optional |
| ------ | ----------- | ----------------- |
| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required |
| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional |
| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required |
| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional |
| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional |
| Instance Resource Tags | Tag each EC2 instance in the node group | Optional |
#### Rancher-managed launch templates
If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated.
#### Custom AMIs
If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative.
If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it.
>**Note**
>The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template.
#### Spot instances
Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types.
>**Note**
>Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration.
#### Node Group Settings
The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created.
| Option | Description |
| ------- | ------------ |
| Node Group Name | The name of the node group. |
| Desired ASG Size | The desired number of instances. |
| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
| Labels | Kubernetes labels applied to the nodes in the managed node group. Note: Invalid labels can prevent upgrades or can prevent Rancher from starting. For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) |
| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. |
</TabItem>
<TabItem value="Rancher v2.5.0-v2.5.5">
### Changes in Rancher v2.5
More EKS options can be configured when you create an EKS cluster in Rancher, including the following:
- Managed node groups
- Desired size, minimum size, maximum size (requires the Cluster Autoscaler to be installed)
- Control plane logging
- Secrets encryption with KMS
The following capabilities have been added for configuring EKS clusters in Rancher:
- GPU support
- Exclusively use managed nodegroups that come with the most up-to-date AMIs
- Add new nodes
- Upgrade nodes
- Add and remove node groups
- Disable and enable private access
- Add restrictions to public access
- Use your cloud credentials to create the EKS cluster instead of passing in your access key and secret key
Due to the way that the cluster data is synced with EKS, if the cluster is modified from another source, such as in the EKS console, and in Rancher within five minutes, it could cause some changes to be overwritten. For information about how the sync works and how to configure it, refer to [this section](#syncing).
### Account Access
<a id="account-access-2-5"></a>
Complete each drop-down and field using the information obtained for your IAM policy.
| Setting | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
| Region | From the drop-down choose the geographical region in which to build your cluster. |
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.](../../user-settings/manage-cloud-credentials.md) |
### Service Role
<a id="service-role-2-5"></a>
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
Service Role | Description
-------------|---------------------------
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
### Secrets Encryption
<a id="secrets-encryption-2-5"></a>
Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html)
### API Server Endpoint Access
<a id="api-server-endpoint-access-2-5"></a>
Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Private-only API Endpoints
If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API.
There are two ways to avoid this extra manual step:
- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster.
- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html).
### Public Access Endpoints
<a id="public-access-endpoints-2-5"></a>
Optionally limit access to the public endpoint via explicit CIDR blocks.
If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster.
One of the following is required to enable private access:
- Rancher's IP must be part of an allowed CIDR block
- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group
For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
### Subnet
<a id="subnet-2-5"></a>
| Option | Description |
| ------- | ------------ |
| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. |
| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). |
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
### Security Group
<a id="security-group-2-5"></a>
Amazon Documentation:
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
### Logging
<a id="logging-2-5"></a>
Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters.
Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation.
For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
### Managed Node Groups
<a id="managed-node-groups-2-5"></a>
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)
Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version. You can configure whether the AMI has GPU enabled.
| Option | Description |
| ------- | ------------ |
| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. |
| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
</TabItem>
<TabItem value="Rancher prior to v2.5">
### Account Access
<a id="account-access-2-4"></a>
Complete each drop-down and field using the information obtained for your IAM policy.
| Setting | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
| Region | From the drop-down choose the geographical region in which to build your cluster. |
| Access Key | Enter the access key that you created for your IAM policy. |
| Secret Key | Enter the secret key that you created for your IAM policy. |
### Service Role
<a id="service-role-2-4"></a>
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
Service Role | Description
-------------|---------------------------
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
### Public IP for Worker Nodes
<a id="public-ip-for-worker-nodes-2-4"></a>
Your selection for this option determines what options are available for **VPC & Subnet**.
Option | Description
-------|------------
Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address.
No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.<br/><br/>If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane.
### VPC & Subnet
<a id="vpc-and-subnet-2-4"></a>
The available options depend on the [public IP for worker nodes.](#public-ip-for-worker-nodes)
Option | Description
-------|------------
Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet.
Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below.
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you've already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case.
<details id="yes">
<summary>Click to expand</summary>
If you're using **Custom: Choose from your existing VPC and Subnets**:
(If you're using **Standard**, skip to the [instance options.)](#select-instance-options-2-4)
1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected.
1. From the drop-down that displays, choose a VPC.
1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays.
1. Click **Next: Select Security Group**.
</details>
If your worker nodes have Private IPs only, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane.
<details id="no">
<summary>Click to expand</summary>
Follow the steps below.
>**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html).
1. From the drop-down that displays, choose a VPC.
1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays.
</details>
### Security Group
<a id="security-group-2-4"></a>
Amazon Documentation:
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
### Instance Options
<a id="select-instance-options-2-4"></a>
Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information.
Option | Description
-------|------------
Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning.
Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose.
Desired ASG Size | The number of instances that your cluster will provision.
User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_
</TabItem>
</Tabs>
### Configuring the Refresh Interval
<Tabs>
<TabItem value="Rancher v2.5.8+">
The `eks-refresh-cron` setting is deprecated. It has been migrated to the `eks-refresh` setting, which is an integer representing seconds.
The default value is 300 seconds.
The syncing interval can be changed by running `kubectl edit setting eks-refresh`.
If the `eks-refresh-cron` setting was previously set, the migration will happen automatically.
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
</TabItem>
<TabItem value="Before v2.5.8">
It is possible to change the refresh interval through the setting `eks-refresh-cron`. This setting accepts values in the Cron format. The default is `*/5 * * * *`.
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
</TabItem>
</Tabs>
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/gke-cluster-configuration/gke-private-clusters.md
+46
View File
@@ -0,0 +1,46 @@
---
title: Private Clusters
weight: 2
aliases:
- /rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/private-clusters
---
_Available as of v2.5.8_
In GKE, [private clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept) are clusters whose nodes are isolated from inbound and outbound traffic by assigning them internal IP addresses only. Private clusters in GKE have the option of exposing the control plane endpoint as a publicly accessible address or as a private address. This is different from other Kubernetes providers, which may refer to clusters with private control plane endpoints as "private clusters" but still allow traffic to and from nodes. You may want to create a cluster with private nodes, with or without a public control plane endpoint, depending on your organization's networking and security requirements. A GKE cluster provisioned from Rancher can use isolated nodes by selecting "Private Cluster" in the Cluster Options (under "Show advanced options"). The control plane endpoint can optionally be made private by selecting "Enable Private Endpoint".
### Private Nodes
Because the nodes in a private cluster only have internal IP addresses, they will not be able to install the cluster agent and Rancher will not be able to fully manage the cluster. This can be overcome in a few ways.
#### Cloud NAT
>**Note**
>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
If restricting outgoing internet access is not a concern for your organization, use Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service to allow nodes in the private network to access the internet, enabling them to download the required images from Dockerhub and contact the Rancher management server. This is the simplest solution.
#### Private registry
>**Note**
>This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.5/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
### Private Control Plane Endpoint
If the cluster has a public endpoint exposed, Rancher will be able to reach the cluster, and no additional steps need to be taken. However, if the cluster has no public endpoint, then considerations must be made to ensure Rancher can access the cluster.
#### Cloud NAT
>**Note**
>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
As above, if restricting outgoing internet access to the nodes is not a concern, then Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service can be used to allow the nodes to access the internet. While the cluster is provisioning, Rancher will provide a registration command to run on the cluster. Download the [kubeconfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) for the new cluster and run the provided kubectl command on the cluster. Gaining access
to the cluster in order to run this command can be done by creating a temporary node or using an existing node in the VPC, or by logging on to or creating an SSH tunnel through one of the cluster nodes.
#### Direct access
If the Rancher server is run on the same VPC as the cluster's control plane, it will have direct access to the control plane's private endpoint. The cluster nodes will need to have access to a [private registry](#private-registry) to download images as described above.
You can also use services from Google such as [Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or [Cloud Interconnect VLAN](https://cloud.google.com/network-connectivity/docs/interconnect) to facilitate connectivity between your organization's network and your Google VPC.
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/rancherd-configuration-reference.md
+321
View File
@@ -0,0 +1,321 @@
---
title: RancherD Configuration Reference
weight: 1
aliases:
- /rancher/v2.5/en/installation/install-rancher-on-linux/rancherd-configuration
- /rancher/v2.x/en/installation/install-rancher-on-linux/rancherd-configuration/
---
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
In RancherD, a server node is defined as a machine (bare-metal or virtual) running the `rancherd server` command. The server runs the Kubernetes API as well as Kubernetes workloads.
An agent node is defined as a machine running the `rancherd agent` command. They don't run the Kubernetes API. To add nodes designated to run your apps and services, join agent nodes to your cluster.
In the RancherD installation instructions, we recommend running three server nodes in the Rancher server cluster. Agent nodes are not required.
- [Certificates for the Rancher Server](#certificates-for-the-rancher-server)
- [Node Taints](#node-taints)
- [Customizing the RancherD Helm Chart](#customizing-the-rancherd-helm-chart)
- [RancherD Server CLI Options](#rancherd-server-cli-options)
- [RancherD Agent CLI Options](#rancherd-agent-cli-options)
# Certificates for the Rancher Server
Rancherd does not use cert-manager to provision certs. Instead RancherD allows you to bring your own self-signed or trusted certs by storing the .pem files in `/etc/rancher/ssl/`. When doing this you should also set the `publicCA` parameter to `true` in your HelmChartConfig. For more information on the HelmChartConfig, refer to the section about [customizing the RancherD Helm chart.](#customizing-the-rancherd-helm-chart)
Private key: `/etc/rancher/ssl/key.pem`
Certificate: `/etc/rancher/ssl/cert.pem`
CA Certificate(self-signed): `/etc/rancher/ssl/cacerts.pem`
Additional CA Certificate: `/etc/ssl/certs/ca-additional.pem`
# Node Taints
By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use taints. The node-taint parameter will allow you to configure nodes with taints. Here is an example of adding a node taint to the `config.yaml`:
```
node-taint:
- "CriticalAddonsOnly=true:NoExecute"
```
# Customizing the RancherD Helm Chart
Rancher is launched as a [Helm](https://helm.sh/) chart using the cluster’s [Helm integration.](https://docs.rke2.io/helm/) This means that you can easily customize the application through a manifest file describing your custom parameters.
The RancherD chart provisions Rancher in a daemonset. It exposes hostport `8080/8443` down to the container port (`80/443`), and uses hostpath to mount certs if needed.
RancherD uses `helm-controller` to bootstrap the RancherD chart. To provide a customized `values.yaml` file, the configuration options must be passed in through the `helm-controller` custom resource definition.
Here is an example of the manifest:
```yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rancher
namespace: kube-system
spec:
valuesContent: |
publicCA: true
```
Put this manifest on your host in `/var/lib/rancher/rke2/server/manifests` before running RancherD.
### Common Options
| Parameter | Default Value | Description |
| ------------------------------ | ----------------------------------------------------- | -------------------------------------------- |
| `addLocal` | "auto" | ***string*** - Have Rancher detect and import the local Rancher server cluster |
| `auditLog.destination` | "sidecar" | ***string*** - Stream to sidecar container console or hostPath volume - *"sidecar, hostPath"* |
| `auditLog.hostPath` | "/var/log/rancher/audit" | ***string*** - log file destination on host (only applies when **auditLog.destination** is set to **hostPath**) |
| `auditLog.level` | 0 | ***int*** - set the [API Audit Log level](https://rancher.com/docs/rancher/v2.5/en/installation/api-auditing). 0 is off. [0-3] |
| `auditLog.maxAge` | 1 | ***int*** - maximum number of days to retain old audit log files (only applies when **auditLog.destination** is set to **hostPath**) |
| `auditLog.maxBackups` | 1 | int - maximum number of audit log files to retain (only applies when **auditLog.destination** is set to **hostPath**) |
| `auditLog.maxSize` | 100 | ***int*** - maximum size in megabytes of the audit log file before it gets rotated (only applies when **auditLog.destination** is set to **hostPath**) |
| `debug` | false | ***bool*** - set debug flag on rancher server |
| `extraEnv` | [] | ***list*** - set additional environment variables for Rancher |
| `imagePullSecrets` | [] | ***list*** - list of names of Secret resource containing private registry credentials |
| `proxy` | " " | ***string** - HTTP[S] proxy server for Rancher |
| `noProxy` | "127.0.0.0/8,10.0.0.0/8,cattle-system.svc,172.16.0.0/12,192.168.0.0/16" | ***string*** - comma separated list of hostnames or ip address not to use the proxy |
| `resources` | {} | ***map*** - rancher pod resource requests & limits |
| `rancherImage` | "rancher/rancher" | ***string*** - rancher image source |
| `rancherImageTag` | same as chart version | ***string*** - rancher/rancher image tag |
| `rancherImagePullPolicy` | "IfNotPresent" | ***string*** - Override imagePullPolicy for rancher server images - *"Always", "Never", "IfNotPresent"* |
| `systemDefaultRegistry` | "" | ***string*** - private registry to be used for all system Docker images, e.g., [http://registry.example.com/] |
| `useBundledSystemChart` | false | ***bool*** - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. |
| `publicCA` | false | ***bool*** - Set to true if your cert is signed by a public CA |
# RancherD Server CLI Options
The command to run the Rancher management server is:
```
rancherd server [OPTIONS]
```
It can be run with the following options:
### Config
| Option | Description |
|--------|-------------|
| `--config FILE, -c FILE` | Load configuration from FILE (default: "/etc/rancher/rke2/config.yaml") |
### Logging
| Option | Description |
|--------|-------------|
| `--debug` | Turn on debug logs |
### Listener
| Option | Description |
|--------|-------------|
| `--bind-address value` | RancherD bind address (default: 0.0.0.0) |
| `--advertise-address value` | IP address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip) |
| `--tls-san value` | Add additional hostname or IP as a Subject Alternative Name in the TLS cert |
### Data
| Option | Description |
|--------|-------------|
| `--data-dir value, -d value` | Folder to hold state (default: "/var/lib/rancher/rancherd") |
### Networking
| Option | Description |
|--------|-------------|
| `--cluster-cidr value` | Network CIDR to use for pod IPs (default: "10.42.0.0/16") |
| `--service-cidr value` | Network CIDR to use for services IPs (default: "10.43.0.0/16") |
| `--cluster-dns value` | Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10) |
| `--cluster-domain value` | Cluster Domain (default: "cluster.local") |
### Cluster
| Option | Description |
|--------|-------------|
| `--token value, -t value` | Shared secret used to join a server or agent to a cluster |
| `--token-file value` | File containing the cluster-secret/token |
### Client
| Option | Description |
|--------|-------------|
| `--write-kubeconfig value, -o value` | Write kubeconfig for admin client to this file |
| `--write-kubeconfig-mode value` | Write kubeconfig with this mode |
### Flags
| Option | Description |
|--------|-------------|
| `--kube-apiserver-arg value` | Customized flag for kube-apiserver process |
| `--kube-scheduler-arg value` | Customized flag for kube-scheduler process |
| `--kube-controller-manager-arg value` | Customized flag for kube-controller-manager process |
### Database
| Option | Description |
|--------|-------------|
| `--etcd-disable-snapshots` | Disable automatic etcd snapshots |
| `--etcd-snapshot-schedule-cron value` | Snapshot interval time in cron spec. eg. every 5 hours '* */5 * * *' (default: "0 */12 * * *") |
| `--etcd-snapshot-retention value` | Number of snapshots to retain (default: 5) |
| `--etcd-snapshot-dir value` | Directory to save db snapshots. (Default location: ${data-dir}/db/snapshots) |
| `--cluster-reset-restore-path value` | Path to snapshot file to be restored |
### System Images Registry
| Option | Description |
|--------|-------------|
| `--system-default-registry value` | Private registry to be used for all system Docker images |
### Components
| Option | Description |
|--------|-------------|
| `--disable value` | Do not deploy packaged components and delete any deployed components (valid items: rancherd-canal, rancherd-coredns, rancherd-ingress, rancherd-kube-proxy, rancherd-metrics-server) |
### Cloud Provider
| Option | Description |
|--------|-------------|
| `--cloud-provider-name value` | Cloud provider name |
| `--cloud-provider-config value` | Cloud provider configuration file path |
### Security
| Option | Description |
|--------|-------------|
| `--profile value` | Validate system configuration against the selected benchmark (valid items: cis-1.5) |
### Agent Node
| Option | Description |
|--------|-------------|
| `--node-name value` | Node name |
| `--node-label value` | Registering and starting kubelet with set of labels |
| `--node-taint value` | Registering kubelet with set of taints |
| `--protect-kernel-defaults` | Kernel tuning behavior. If set, error if kernel tunables are different than kubelet defaults. |
| `--selinux` | Enable SELinux in containerd |
### Agent Runtime
| Option | Description |
|--------|-------------|
| `--container-runtime-endpoint value` | Disable embedded containerd and use alternative CRI implementation |
| `--snapshotter value` | Override default containerd snapshotter (default: "overlayfs") |
| `--private-registry value` | Private registry configuration file (default: "/etc/rancher/rke2/registries.yaml") |
### Agent Networking
| Option | Description |
|--------|-------------|
| `--node-ip value, -i value` | IP address to advertise for node |
| `--resolv-conf value` | Kubelet resolv.conf file |
### Agent Flags
| Option | Description |
|--------|-------------|
| `--kubelet-arg value` | Customized flag for kubelet process |
| `--kube-proxy-arg value` | Customized flag for kube-proxy process |
### Experimental
| Option | Description |
|--------|-------------|
| `--agent-token value` | Shared secret used to join agents to the cluster, but not servers |
| `--agent-token-file value` | File containing the agent secret |
| `--server value, -s value` | Server to connect to, used to join a cluster |
| `--cluster-reset` | Forget all peers and become sole member of a new cluster |
| `--secrets-encryption` | Enable Secret encryption at rest |
# RancherD Agent CLI Options
The following command is used to run the RancherD agent:
```
rancherd agent [OPTIONS]
```
The following options are available.
### Config
| Option | Description |
|--------|-------------|
| `--config FILE, -c FILE` | Load configuration from FILE (default: "/etc/rancher/rke2/config.yaml") |
### Data
| Option | Description |
|--------|-------------|
| `--data-dir value, -d value` | Folder to hold state (default: "/var/lib/rancher/rancherd") |
### Logging
| Option | Description |
|--------|-------------|
| `--debug` | Turn on debug logs |
### Cluster
| Option | Description |
|--------|-------------|
| `--token value, -t value` | Token to use for authentication |
| `--token-file value` | Token file to use for authentication |
| `--server value, -s value` | Server to connect to |
### Agent Node
| Option | Description |
|--------|-------------|
| `--node-name value` | Node name |
| `--node-label value` | Registering and starting kubelet with set of labels |
| `--node-taint value` | Registering kubelet with set of taints |
| `--selinux` | Enable SELinux in containerd |
| `--protect-kernel-defaults` | Kernel tuning behavior. If set, error if kernel tunables are different than kubelet defaults. |
### Agent Runtime
| Option | Description |
|--------|-------------|
| `--container-runtime-endpoint value` | Disable embedded containerd and use alternative CRI implementation |
| `--snapshotter value` | Override default containerd snapshotter (default: "overlayfs") |
| `--private-registry value` | Private registry configuration file (default: "/etc/rancher/rke2/registries.yaml") |
### Agent Networking
| Option | Description |
|--------|-------------|
| `--node-ip value, -i value` | IP address to advertise for node |
| `--resolv-conf value` | Kubelet resolv.conf file |
### Agent Flags
| Option | Description |
|--------|-------------|
| `--kubelet-arg value` | Customized flag for kubelet process |
| `--kube-proxy-arg value` | Customized flag for kube-proxy process |
### System Images Registry
| Option | Description |
|--------|-------------|
| `--system-default-registry value` | Private registry to be used for all system Docker images |
### Cloud Provider
| Option | Description |
|--------|-------------|
| `--cloud-provider-name value` | Cloud provider name |
| `--cloud-provider-config value` | Cloud provider configuration file path |
### Security
| Option | Description |
|--------|-------------|
| `--profile value` | Validate system configuration against the selected benchmark (valid items: cis-1.5) |
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md
+79
View File
@@ -0,0 +1,79 @@
---
title: RKE Cluster Configuration
weight: 1
---
In [clusters launched by RKE](../../../pages-for-subheaders/launch-kubernetes-with-rancher.md), you can edit any of the remaining options that follow.
- [Configuration Options in the Rancher UI](#configuration-options-in-the-rancher-ui)
- [Editing Clusters with YAML](#editing-clusters-with-yaml)
- [Updating ingress-nginx](#updating-ingress-nginx)
# Configuration Options in the Rancher UI
To edit your cluster, open the **Global** view, make sure the **Clusters** tab is selected, and then select **&#8942; > Edit** for the cluster that you want to edit.
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
### Kubernetes Version
The version of Kubernetes installed on each cluster node. For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
### Network Provider
The \container networking interface (CNI) that powers networking for your cluster.<br/><br/>**Note:** You can only choose this option while provisioning your cluster. It cannot be edited later.
### Project Network Isolation
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
Before Rancher v2.5.8, project network isolation is only available if you are using the Canal network plugin for RKE.
In v2.5.8+, project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
### Nginx Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use Nginx ingress within the cluster.
### Metrics Server Monitoring
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
### Pod Security Policy Support
Enables [pod security policies](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. After enabling this option, choose a policy using the **Default Pod Security Policy** drop-down.
### Docker version on nodes
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support. If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
### Docker Root Directory
The directory on your cluster nodes where you've installed Docker. If you install Docker on your nodes to a non-default directory, update this path.
### Default Pod Security Policy
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
### Cloud Provider
If you're using a cloud provider to host cluster nodes launched by RKE, enable [this option](cluster-provisioning/rke-clusters/options/cloud-providers/) so that you can use the cloud provider's native features. If you want to store persistent data for your cloud-hosted cluster, this option is required.
# Editing Clusters with YAML
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
- To edit an RKE config file directly from the Rancher UI, click **Edit as YAML**.
- To read from an existing RKE file, click **Read from File**.
![image](/img/cluster-options-yaml.png)
For an example of RKE config file syntax, see the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
# Updating ingress-nginx
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/sync-clusters.md
+37
View File
@@ -0,0 +1,37 @@
---
title: Syncing
weight: 10
aliases:
- /rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/syncing
---
Syncing is the feature for EKS and GKE clusters that causes Rancher to update the clusters' values so they are up to date with their corresponding cluster object in the hosted Kubernetes provider. This enables Rancher to not be the sole owner of a hosted cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other.
### How it works
There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
1. The config object for the cluster, located on the Spec of the Cluster:
* For EKS, the field is called EKSConfig
* For GKE, the field is called GKEConfig
2. The UpstreamSpec object
* For EKS, this is located on the EKSStatus field on the Status of the Cluster.
* For GKE, this is located on the GKEStatus field on the Status of the Cluster.
The struct types that define these objects can be found in their corresponding operator projects:
* [eks-operator](https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go)
* [gke-operator](https://github.com/rancher/gke-operator/blob/master/pkg/apis/gke.cattle.io/v1/types.go)
All fields with the exception of the cluster name, the location (region or zone), Imported, and the cloud credential reference, are nillable on this Spec object.
The EKSConfig or GKEConfig represents desired state for its non-nil values. Fields that are non-nil in the config object can be thought of as “managed". When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher.
UpstreamSpec represents the cluster as it is in the hosted Kubernetes provider and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed, Rancher checks if the cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig or GKEConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig or GKEConfig. This is what is displayed in the UI.
If Rancher and another source attempt to update a cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. To use EKS as an example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioned case.
versioned_docs/version-2.5/reference-guides/cluster-configuration/rancher-server-configuration/use-existing-nodes/rancher-agent-options.md
+58
View File
@@ -0,0 +1,58 @@
---
title: Rancher Agent Options
weight: 2500
aliases:
- /rancher/v2.5/en/admin-settings/agent-options/
- /rancher/v2.5/en/cluster-provisioning/custom-clusters/agent-options
- /rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/agent-options/
---
Rancher deploys an agent on each node to communicate with the node. This pages describes the options that can be passed to the agent. To use these options, you will need to [create a cluster with custom nodes](../../../../pages-for-subheaders/use-existing-nodes.md) and add the options to the generated `docker run` command when adding a node.
For an overview of how Rancher communicates with downstream clusters using node agents, refer to the [architecture section.](../../../../pages-for-subheaders/rancher-manager-architecture.md#3-node-agents)
## General options
| Parameter | Environment variable | Description |
| ---------- | -------------------- | ----------- |
| `--server` | `CATTLE_SERVER` | The configured Rancher `server-url` setting which the agent connects to |
| `--token` | `CATTLE_TOKEN` | Token that is needed to register the node in Rancher |
| `--ca-checksum` | `CATTLE_CA_CHECKSUM` | The SHA256 checksum of the configured Rancher `cacerts` setting to validate |
| `--node-name` | `CATTLE_NODE_NAME` | Override the hostname that is used to register the node (defaults to `hostname -s`) |
| `--label` | `CATTLE_NODE_LABEL` | Add node labels to the node. For multiple labels, pass additional `--label` options. (`--label key=value`) |
| `--taints` | `CATTLE_NODE_TAINTS` | Add node taints to the node. For multiple taints, pass additional `--taints` options. (`--taints key=value:effect`) |
## Role options
| Parameter | Environment variable | Description |
| ---------- | -------------------- | ----------- |
| `--all-roles` | `ALL=true` | Apply all roles (`etcd`,`controlplane`,`worker`) to the node |
| `--etcd` | `ETCD=true` | Apply the role `etcd` to the node |
| `--controlplane` | `CONTROL=true` | Apply the role `controlplane` to the node |
| `--worker` | `WORKER=true` | Apply the role `worker` to the node |
## IP address options
| Parameter | Environment variable | Description |
| ---------- | -------------------- | ----------- |
| `--address` | `CATTLE_ADDRESS` | The IP address the node will be registered with (defaults to the IP used to reach `8.8.8.8`) |
| `--internal-address` | `CATTLE_INTERNAL_ADDRESS` | The IP address used for inter-host communication on a private network |
### Dynamic IP address options
For automation purposes, you can't have a specific IP address in a command as it has to be generic to be used for every node. For this, we have dynamic IP address options. They are used as a value to the existing IP address options. This is supported for `--address` and `--internal-address`.
| Value | Example | Description |
| ---------- | -------------------- | ----------- |
| Interface name | `--address eth0` | The first configured IP address will be retrieved from the given interface |
| `ipify` | `--address ipify` | Value retrieved from `https://api.ipify.org` will be used |
| `awslocal` | `--address awslocal` | Value retrieved from `http://169.254.169.254/latest/meta-data/local-ipv4` will be used |
| `awspublic` | `--address awspublic` | Value retrieved from `http://169.254.169.254/latest/meta-data/public-ipv4` will be used |
| `doprivate` | `--address doprivate` | Value retrieved from `http://169.254.169.254/metadata/v1/interfaces/private/0/ipv4/address` will be used |
| `dopublic` | `--address dopublic` | Value retrieved from `http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address` will be used |
| `azprivate` | `--address azprivate` | Value retrieved from `http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2017-08-01&format=text` will be used |
| `azpublic` | `--address azpublic` | Value retrieved from `http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text` will be used |
| `gceinternal` | `--address gceinternal` | Value retrieved from `http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip` will be used |
| `gceexternal` | `--address gceexternal` | Value retrieved from `http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip` will be used |
| `packetlocal` | `--address packetlocal` | Value retrieved from `https://metadata.packet.net/2009-04-04/meta-data/local-ipv4` will be used |
| `packetpublic` | `--address packetlocal` | Value retrieved from `https://metadata.packet.net/2009-04-04/meta-data/public-ipv4` will be used |
versioned_docs/version-2.5/reference-guides/configure-openldap/openldap-config-reference.md
+88
View File
@@ -0,0 +1,88 @@
---
title: OpenLDAP Configuration Reference
weight: 2
aliases:
- /rancher/v2.x/en/admin-settings/authentication/openldap/openldap-config/
---
This section is intended to be used as a reference when setting up an OpenLDAP authentication provider in Rancher.
For further details on configuring OpenLDAP, refer to the [official documentation.](https://www.openldap.org/doc/)
> Before you proceed with the configuration, please familiarize yourself with the concepts of [External Authentication Configuration and Principal Users](../../pages-for-subheaders/about-authentication.md#external-authentication-configuration-and-principal-users).
- [Background: OpenLDAP Authentication Flow](#background-openldap-authentication-flow)
- [OpenLDAP server configuration](#openldap-server-configuration)
- [User/group schema configuration](#user-group-schema-configuration)
- [User schema configuration](#user-schema-configuration)
- [Group schema configuration](#group-schema-configuration)
## Background: OpenLDAP Authentication Flow
1. When a user attempts to login with his LDAP credentials, Rancher creates an initial bind to the LDAP server using a service account with permissions to search the directory and read user/group attributes.
2. Rancher then searches the directory for the user by using a search filter based on the provided username and configured attribute mappings.
3. Once the user has been found, he is authenticated with another LDAP bind request using the user's DN and provided password.
4. Once authentication succeeded, Rancher then resolves the group memberships both from the membership attribute in the user's object and by performing a group search based on the configured user mapping attribute.
# OpenLDAP Server Configuration
You will need to enter the address, port, and protocol to connect to your OpenLDAP server. `389` is the standard port for insecure traffic, `636` for TLS traffic.
> **Using TLS?**
>
> If the certificate used by the OpenLDAP server is self-signed or not from a recognized certificate authority, make sure have at hand the CA certificate (concatenated with any intermediate certificates) in PEM format. You will have to paste in this certificate during the configuration so that Rancher is able to validate the certificate chain.
If you are in doubt about the correct values to enter in the user/group Search Base configuration fields, consult your LDAP administrator or refer to the section [Identify Search Base and Schema using ldapsearch](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/about-authentication/authentication-config/configure-active-directory.md#annex-identify-search-base-and-schema-using-ldapsearch) in the Active Directory authentication documentation.
<figcaption>OpenLDAP Server Parameters</figcaption>
| Parameter | Description |
|:--|:--|
| Hostname | Specify the hostname or IP address of the OpenLDAP server |
| Port | Specify the port at which the OpenLDAP server is listening for connections. Unencrypted LDAP normally uses the standard port of 389, while LDAPS uses port 636.|
| TLS | Check this box to enable LDAP over SSL/TLS (commonly known as LDAPS). You will also need to paste in the CA certificate if the server uses a self-signed/enterprise-signed certificate. |
| Server Connection Timeout | The duration in number of seconds that Rancher waits before considering the server unreachable. |
| Service Account Distinguished Name | Enter the Distinguished Name (DN) of the user that should be used to bind, search and retrieve LDAP entries. |
| Service Account Password | The password for the service account. |
| User Search Base | Enter the Distinguished Name of the node in your directory tree from which to start searching for user objects. All users must be descendents of this base DN. For example: "ou=people,dc=acme,dc=com".|
| Group Search Base | If your groups live under a different node than the one configured under `User Search Base` you will need to provide the Distinguished Name here. Otherwise leave this field empty. For example: "ou=groups,dc=acme,dc=com".|
# User/Group Schema Configuration
If your OpenLDAP directory deviates from the standard OpenLDAP schema, you must complete the **Customize Schema** section to match it.
Note that the attribute mappings configured in this section are used by Rancher to construct search filters and resolve group membership. It is therefore always recommended to verify that the configuration here matches the schema used in your OpenLDAP.
If you are unfamiliar with the user/group schema used in the OpenLDAP server, consult your LDAP administrator or refer to the section [Identify Search Base and Schema using ldapsearch](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/about-authentication/authentication-config/configure-active-directory.md#annex-identify-search-base-and-schema-using-ldapsearch) in the Active Directory authentication documentation.
### User Schema Configuration
The table below details the parameters for the user schema configuration.
<figcaption>User Schema Configuration Parameters</figcaption>
| Parameter | Description |
|:--|:--|
| Object Class | The name of the object class used for user objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
| Username Attribute | The user attribute whose value is suitable as a display name. |
| Login Attribute | The attribute whose value matches the username part of credentials entered by your users when logging in to Rancher. This is typically `uid`. |
| User Member Attribute | The user attribute containing the Distinguished Name of groups a user is member of. Usually this is one of `memberOf` or `isMemberOf`. |
| Search Attribute | When a user enters text to add users or groups in the UI, Rancher queries the LDAP server and attempts to match users by the attributes provided in this setting. Multiple attributes can be specified by separating them with the pipe ("\|") symbol. |
| User Enabled Attribute | If the schema of your OpenLDAP server supports a user attribute whose value can be evaluated to determine if the account is disabled or locked, enter the name of that attribute. The default OpenLDAP schema does not support this and the field should usually be left empty. |
| Disabled Status Bitmask | This is the value for a disabled/locked user account. The parameter is ignored if `User Enabled Attribute` is empty. |
### Group Schema Configuration
The table below details the parameters for the group schema configuration.
<figcaption>Group Schema Configuration Parameters</figcaption>
| Parameter | Description |
|:--|:--|
| Object Class | The name of the object class used for group entries in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) |
| Name Attribute | The group attribute whose value is suitable for a display name. |
| Group Member User Attribute | The name of the **user attribute** whose format matches the group members in the `Group Member Mapping Attribute`. |
| Group Member Mapping Attribute | The name of the group attribute containing the members of a group. |
| Search Attribute | Attribute used to construct search filters when adding groups to clusters or projects in the UI. See description of user schema `Search Attribute`. |
| Group DN Attribute | The name of the group attribute whose format matches the values in the user's group membership attribute. See `User Member Attribute`. |
| Nested Group Membership | This settings defines whether Rancher should resolve nested group memberships. Use only if your organization makes use of these nested memberships (ie. you have groups that contain other groups as members). This option is disabled if you are using Shibboleth. |
versioned_docs/version-2.5/reference-guides/installation-references/feature-flags.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/installation-references/helm-chart-options.md
+263
View File
@@ -0,0 +1,263 @@
---
title: Rancher Helm Chart Options
weight: 1
aliases:
- /rancher/v2.5/en/installation/options/
- /rancher/v2.5/en/installation/options/chart-options/
- /rancher/v2.5/en/installation/options/helm2/helm-rancher/chart-options/
- /rancher/v2.5/en/installation/resources/chart-options
- /rancher/v2.x/en/installation/install-rancher-on-k8s/chart-options/
---
This page is a configuration reference for the Rancher Helm chart.
For help choosing a Helm chart version, refer to [this page.](../../getting-started/installation-and-upgrade/resources/choose-a-rancher-version.md)
For information on enabling experimental features, refer to [this page.](../../pages-for-subheaders/enable-experimental-features.md)
- [Common Options](#common-options)
- [Advanced Options](#advanced-options)
- [API Audit Log](#api-audit-log)
- [Setting Extra Environment Variables](#setting-extra-environment-variables)
- [TLS Settings](#tls-settings)
- [Customizing your Ingress](#customizing-your-ingress)
- [HTTP Proxy](#http-proxy)
- [Additional Trusted CAs](#additional-trusted-cas)
- [Private Registry and Air Gap Installs](#private-registry-and-air-gap-installs)
- [External TLS Termination](#external-tls-termination)
### Common Options
| Option | Default Value | Description |
| ------------------------- | ------------- | ---------------------------------------------------------------------------------- |
| `hostname` | " " | `string` - the Fully Qualified Domain Name for your Rancher Server |
| `ingress.tls.source` | "rancher" | `string` - Where to get the cert for the ingress. - "rancher, letsEncrypt, secret" |
| `letsEncrypt.email` | " " | `string` - Your email address |
| `letsEncrypt.environment` | "production" | `string` - Valid options: "staging, production" |
| `privateCA` | false | `bool` - Set to true if your cert is signed by a private CA |
<br/>
### Advanced Options
| Option | Default Value | Description |
| ------------------------------ | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| `additionalTrustedCAs` | false | `bool` - See [Additional Trusted CAs](#additional-trusted-cas) |
| `addLocal` | "true" | `string` - Have Rancher detect and import the "local" Rancher server cluster. _Note: This option is no longer available in v2.5.0. Consider using the `restrictedAdmin` option to prevent users from modifying the local cluster._ |
| `antiAffinity` | "preferred" | `string` - AntiAffinity rule for Rancher pods - "preferred, required" |
| `auditLog.destination` | "sidecar" | `string` - Stream to sidecar container console or hostPath volume - "sidecar, hostPath" |
| `auditLog.hostPath` | "/var/log/rancher/audit" | `string` - log file destination on host (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.level` | 0 | `int` - set the [API Audit Log](installation/api-auditing) level. 0 is off. [0-3] |
| `auditLog.maxAge` | 1 | `int` - maximum number of days to retain old audit log files (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.maxBackup` | 1 | `int` - maximum number of audit log files to retain (only applies when `auditLog.destination` is set to `hostPath`) |
| `auditLog.maxSize` | 100 | `int` - maximum size in megabytes of the audit log file before it gets rotated (only applies when `auditLog.destination` is set to `hostPath`) |
| `busyboxImage` | "busybox" | `string` - Image location for busybox image used to collect audit logs |
| `certmanager.version` | "" | `string` - set cert-manager compatibility |
| `debug` | false | `bool` - set debug flag on rancher server |
| `extraEnv` | [] | `list` - set additional environment variables for Rancher |
| `imagePullSecrets` | [] | `list` - list of names of Secret resource containing private registry credentials |
| `ingress.configurationSnippet` | "" | `string` - Add additional Nginx configuration. Can be used for proxy configuration. |
| `ingress.extraAnnotations` | {} | `map` - additional annotations to customize the ingress |
| `ingress.enabled` | true | When set to false, Helm will not install a Rancher ingress. Set the option to false to deploy your own ingress. _Available as of v2.5.6_ |
| `letsEncrypt.ingress.class` | "" | `string` - optional ingress class for the cert-manager acmesolver ingress that responds to the Let's Encrypt ACME challenges. Options: traefik, nginx. | |
| `noProxy` | "127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local,cattle-system.svc" | `string` - comma separated list of hostnames or ip address not to use the proxy | |
| `proxy` | "" | `string` - HTTP[S] proxy server for Rancher |
| `rancherImage` | "rancher/rancher" | `string` - rancher image source |
| `rancherImagePullPolicy` | "IfNotPresent" | `string` - Override imagePullPolicy for rancher server images - "Always", "Never", "IfNotPresent" |
| `rancherImageTag` | same as chart version | `string` - rancher/rancher image tag |
| `replicas` | 3 | `int` - Number of replicas of Rancher pods |
| `resources` | {} | `map` - rancher pod resource requests & limits |
| `restrictedAdmin` | `false` | _Available in Rancher v2.5_ `bool` - When this option is set to true, the initial Rancher user has restricted access to the local Kubernetes cluster to prevent privilege escalation. For more information, see the section about the [restricted-admin role.](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin) |
| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system Docker images, e.g., http://registry.example.com/ |
| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" |
| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. |
### API Audit Log
Enabling the [API Audit Log](installation/api-auditing/).
You can collect this log as you would any container log. Enable [logging](../../pages-for-subheaders/logging.md) for the `System` Project on the Rancher server cluster.
```plain
--set auditLog.level=1
```
By default enabling Audit Logging will create a sidecar container in the Rancher pod. This container (`rancher-audit-log`) will stream the log to `stdout`. You can collect this log as you would any container log. When using the sidecar as the audit log destination, the `hostPath`, `maxAge`, `maxBackups`, and `maxSize` options do not apply. It's advised to use your OS or Docker daemon's log rotation features to control disk space use. Enable [logging](../../pages-for-subheaders/logging.md) for the Rancher server cluster or System Project.
Set the `auditLog.destination` to `hostPath` to forward logs to volume shared with the host system instead of streaming to a sidecar container. When setting the destination to `hostPath` you may want to adjust the other auditLog parameters for log rotation.
### Setting Extra Environment Variables
You can set extra environment variables for Rancher server using `extraEnv`. This list uses the same `name` and `value` keys as the container manifest definitions. Remember to quote the values.
```plain
--set 'extraEnv[0].name=CATTLE_TLS_MIN_VERSION'
--set 'extraEnv[0].value=1.0'
```
### TLS Settings
When you install Rancher inside of a Kubernetes cluster, TLS is offloaded at the cluster's ingress controller. The possible TLS settings depend on the used ingress controller.
See [TLS settings](tls-settings.md) for more information and options.
### Import `local` Cluster
By default Rancher server will detect and import the `local` cluster it's running on. User with access to the `local` cluster will essentially have "root" access to all the clusters managed by Rancher server.
> **Important:** If you turn addLocal off, most Rancher v2.5 features won't work, including the EKS provisioner.
If this is a concern in your environment you can set this option to "false" on your initial install.
This option is only effective on the initial Rancher install. See [Issue 16522](https://github.com/rancher/rancher/issues/16522) for more information.
```plain
--set addLocal="false"
```
### Customizing your Ingress
To customize or use a different ingress with Rancher server you can set your own Ingress annotations.
Example on setting a custom certificate issuer:
```plain
--set ingress.extraAnnotations.'cert-manager\.io/cluster-issuer'=issuer-name
```
Example on setting a static proxy header with `ingress.configurationSnippet`. This value is parsed like a template so variables can be used.
```plain
--set ingress.configurationSnippet='more_set_input_headers X-Forwarded-Host {{ .Values.hostname }};'
```
### HTTP Proxy
Rancher requires internet access for some functionality (helm charts). Use `proxy` to set your proxy server.
Add your IP exceptions to the `noProxy` list. Make sure you add the Pod cluster IP range (default: `10.42.0.0/16`), Service cluster IP range (default: `10.43.0.0/16`), the internal cluster domains (default: `.svc,.cluster.local`) and any worker cluster `controlplane` nodes. Rancher supports CIDR notation ranges in this list.
```plain
--set proxy="http://<username>:<password>@<proxy_url>:<proxy_port>/"
--set noProxy="127.0.0.0/8\,10.0.0.0/8\,172.16.0.0/12\,192.168.0.0/16\,.svc\,.cluster.local"
```
### Additional Trusted CAs
If you have private registries, catalogs or a proxy that intercepts certificates, you may need to add additional trusted CAs to Rancher.
```plain
--set additionalTrustedCAs=true
```
Once the Rancher deployment is created, copy your CA certs in pem format into a file named `ca-additional.pem` and use `kubectl` to create the `tls-ca-additional` secret in the `cattle-system` namespace.
```plain
kubectl -n cattle-system create secret generic tls-ca-additional --from-file=ca-additional.pem=./ca-additional.pem
```
### Private Registry and Air Gap Installs
For details on installing Rancher with a private registry, see:
- [Air Gap: Docker Install](installation/air-gap-single-node/)
- [Air Gap: Kubernetes Install](installation/air-gap-high-availability/)
# External TLS Termination
We recommend configuring your load balancer as a Layer 4 balancer, forwarding plain 80/tcp and 443/tcp to the Rancher Management cluster nodes. The Ingress Controller on the cluster will redirect http traffic on port 80 to https on port 443.
You may terminate the SSL/TLS on a L7 load balancer external to the Rancher cluster (ingress). Use the `--set tls=external` option and point your load balancer at port http 80 on all of the Rancher cluster nodes. This will expose the Rancher interface on http port 80. Be aware that clients that are allowed to connect directly to the Rancher cluster will not be encrypted. If you choose to do this we recommend that you restrict direct access at the network level to just your load balancer.
> **Note:** If you are using a Private CA signed certificate, add `--set privateCA=true` and see [Adding TLS Secrets - Using a Private CA Signed Certificate](installation/resources/encryption/tls-secrets/) to add the CA cert for Rancher.
Your load balancer must support long lived websocket connections and will need to insert proxy headers so Rancher can route links correctly.
### Configuring Ingress for External TLS when Using NGINX v0.25
In NGINX v0.25, the behavior of NGINX has [changed](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0220) regarding forwarding headers and external TLS termination. Therefore, in the scenario that you are using external TLS termination configuration with NGINX v0.25, you must edit the `cluster.yml` to enable the `use-forwarded-headers` option for ingress:
```yaml
ingress:
provider: nginx
options:
use-forwarded-headers: 'true'
```
### Required Headers
- `Host`
- `X-Forwarded-Proto`
- `X-Forwarded-Port`
- `X-Forwarded-For`
### Recommended Timeouts
- Read Timeout: `1800 seconds`
- Write Timeout: `1800 seconds`
- Connect Timeout: `30 seconds`
### Health Checks
Rancher will respond `200` to health checks on the `/healthz` endpoint.
### Example NGINX config
This NGINX configuration is tested on NGINX 1.14.
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
- Replace `IP_NODE1`, `IP_NODE2` and `IP_NODE3` with the IP addresses of the nodes in your cluster.
- Replace both occurrences of `FQDN` to the DNS name for Rancher.
- Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
upstream rancher {
server IP_NODE_1:80;
server IP_NODE_2:80;
server IP_NODE_3:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name FQDN;
ssl_certificate /certs/fullchain.pem;
ssl_certificate_key /certs/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name FQDN;
return 301 https://$server_name$request_uri;
}
}
```
versioned_docs/version-2.5/reference-guides/installation-references/tls-settings.md
+27
View File
@@ -0,0 +1,27 @@
---
title: TLS Settings
weight: 3
aliases:
- /rancher/v2.5/en/installation/options/tls-settings/
- /rancher/v2.5/en/admin-settings/tls-settings
- /rancher/v2.5/en/installation/resources/encryption/tls-settings
- /rancher/v2.x/en/installation/resources/tls-settings/
---
Changing the default TLS settings depends on the chosen installation method.
# Running Rancher in a highly available Kubernetes cluster
When you install Rancher inside of a Kubernetes cluster, TLS is offloaded at the cluster's ingress controller. The possible TLS settings depend on the used ingress controller:
* nginx-ingress-controller (default for RKE1 and RKE2): [Default TLS Version and Ciphers](https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-tls-version-and-ciphers).
* traefik (default for K3s): [TLS Options](https://doc.traefik.io/traefik/https/tls/#tls-options).
# Running Rancher in a single Docker container
The default TLS configuration only accepts TLS 1.2 and secure TLS cipher suites. You can change this by setting the following environment variables:
| Parameter | Description | Default | Available options |
|-----|-----|-----|-----|
| `CATTLE_TLS_MIN_VERSION` | Minimum TLS version | `1.2` | `1.0`, `1.1`, `1.2`, `1.3` |
| `CATTLE_TLS_CIPHERS` | Allowed TLS cipher suites | `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`,<br/>`TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384`,<br/>`TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305`,<br/>`TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256`,<br/>`TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384`,<br/>`TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305` | See [Golang tls constants](https://golang.org/pkg/crypto/tls/#pkg-constants) |
versioned_docs/version-2.5/reference-guides/kubernetes-concepts.md
+74
View File
@@ -0,0 +1,74 @@
---
title: Kubernetes Concepts
weight: 4
aliases:
- /rancher/v2.x/en/overview/concepts/
---
This page explains concepts related to Kubernetes that are important for understanding how Rancher works. The descriptions below provide a simplified interview of Kubernetes components. For more details, refer to the [official documentation on Kubernetes components.](https://kubernetes.io/docs/concepts/overview/components/)
This section covers the following topics:
- [About Docker](#about-docker)
- [About Kubernetes](#about-kubernetes)
- [What is a Kubernetes Cluster?](#what-is-a-kubernetes-cluster)
- [Roles for Nodes in Kubernetes Clusters](#roles-for-nodes-in-kubernetes-clusters)
- [etcd Nodes](#etcd-nodes)
- [Controlplane Nodes](#controlplane-nodes)
- [Worker Nodes](#worker-nodes)
- [About Helm](#about-helm)
# About Docker
Docker is the container packaging and runtime standard. Developers build container images from Dockerfiles and distribute container images from Docker registries. [Docker Hub](https://hub.docker.com) is the most popular public registry. Many organizations also set up private Docker registries. Docker is primarily used to manage containers on individual nodes.
>**Note:** Although Rancher 1.6 supported Docker Swarm clustering technology, it is no longer supported in Rancher 2.x due to the success of Kubernetes.
# About Kubernetes
Kubernetes is the container cluster management standard. YAML files specify containers and other resources that form an application. Kubernetes performs functions such as scheduling, scaling, service discovery, health check, secret management, and configuration management.
# What is a Kubernetes Cluster?
A cluster is a group of computers that work together as a single system.
A _Kubernetes Cluster_ is a cluster that uses the [Kubernetes container-orchestration system](https://kubernetes.io/) to deploy, maintain, and scale Docker containers, allowing your organization to automate application operations.
# Roles for Nodes in Kubernetes Clusters
Each computing resource in a Kubernetes cluster is called a _node_. Nodes can be either bare-metal servers or virtual machines. Kubernetes classifies nodes into three types: _etcd_ nodes, _control plane_ nodes, and _worker_ nodes.
A Kubernetes cluster consists of at least one etcd, controlplane, and worker node.
### etcd Nodes
Rancher uses etcd as a data store in both single node and high-availability installations. In Kubernetes, etcd is also a role for nodes that store the cluster state.
The state of a Kubernetes cluster is maintained in [etcd.](https://kubernetes.io/docs/concepts/overview/components/#etcd) The etcd nodes run the etcd database.
The etcd database component is a distributed key-value store used as Kubernetes storage for all cluster data, such as cluster coordination and state management. It is recommended to run etcd on multiple nodes so that there's always a backup available for failover.
Although you can run etcd on just one node, etcd requires a majority of nodes, a quorum, to agree on updates to the cluster state. The cluster should always contain enough healthy etcd nodes to form a quorum. For a cluster with n members, a quorum is (n/2)+1. For any odd-sized cluster, adding one node will always increase the number of nodes necessary for a quorum.
Three etcd nodes is generally sufficient for smaller clusters and five etcd nodes for large clusters.
### Controlplane Nodes
Controlplane nodes run the Kubernetes API server, scheduler, and controller manager. These nodes take care of routine tasks to ensure that your cluster maintains your configuration. Because all cluster data is stored on your etcd nodes, control plane nodes are stateless. You can run control plane on a single node, although three or more nodes are recommended for redundancy. Additionally, a single node can share the control plane and etcd roles.
### Worker Nodes
Each [worker node](https://kubernetes.io/docs/concepts/architecture/nodes/) runs the following:
- **Kubelets:** An agent that monitors the state of the node, ensuring your containers are healthy.
- **Workloads:** The containers and pods that hold your apps, as well as other types of deployments.
Worker nodes also run storage and networking drivers, and ingress controllers when required. You create as many worker nodes as necessary to run your [workloads](../pages-for-subheaders/workloads-and-pods.md).
# About Helm
For high-availability installations of Rancher, Helm is the tool used to install Rancher on a Kubernetes cluster.
Helm is the package management tool of choice for Kubernetes. Helm charts provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh).
For more information on service accounts and cluster role binding, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
versioned_docs/version-2.5/reference-guides/monitoring-v2-configuration/examples.md
+24
View File
@@ -0,0 +1,24 @@
---
title: Examples
weight: 400
---
### ServiceMonitor
An example ServiceMonitor custom resource can be found [here.](https://github.com/prometheus-operator/prometheus-operator/blob/master/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml)
### PodMonitor
An example PodMonitor can be found [here.](https://github.com/prometheus-operator/prometheus-operator/blob/master/example/user-guides/getting-started/example-app-pod-monitor.yaml) An example Prometheus resource that refers to it can be found [here.](https://github.com/prometheus-operator/prometheus-operator/blob/master/example/user-guides/getting-started/prometheus-pod-monitor.yaml)
### PrometheusRule
For users who are familiar with Prometheus, a PrometheusRule contains the alerting and recording rules that you would normally place in a [Prometheus rule file](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/).
For a more fine-grained application of PrometheusRules within your cluster, the ruleSelector field on a Prometheus resource allows you to select which PrometheusRules should be loaded onto Prometheus based on the labels attached to the PrometheusRules resources.
An example PrometheusRule is on [this page.](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/user-guides/alerting.md)
### Alertmanager Config
For an example configuration, refer to [this section.](../../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/alertmanager.md#example-alertmanager-config)
versioned_docs/version-2.5/reference-guides/monitoring-v2-configuration/helm-chart-options.md
+77
View File
@@ -0,0 +1,77 @@
---
title: Helm Chart Options
weight: 8
---
- [Configuring Resource Limits and Requests](#configuring-resource-limits-and-requests)
- [Trusted CA for Notifiers](#trusted-ca-for-notifiers)
- [Additional Scrape Configurations](#additional-scrape-configurations)
- [Configuring Applications Packaged within Monitoring V2](#configuring-applications-packaged-within-monitoring-v2)
- [Increase the Replicas of Alertmanager](#increase-the-replicas-of-alertmanager)
- [Configuring the Namespace for a Persistent Grafana Dashboard](#configuring-the-namespace-for-a-persistent-grafana-dashboard)
# Configuring Resource Limits and Requests
The resource requests and limits can be configured when installing `rancher-monitoring`.
The default values are in the [values.yaml](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/values.yaml) in the `rancher-monitoring` Helm chart.
The default values in the table below are the minimum required resource limits and requests.
| Resource Name | Memory Limit | CPU Limit | Memory Request | CPU Request |
| ------------- | ------------ | ----------- | ---------------- | ------------------ |
| alertmanager | 500Mi | 1000m | 100Mi | 100m |
| grafana | 200Mi | 200m | 100Mi | 100m |
| kube-state-metrics subchart | 200Mi | 100m | 130Mi | 100m |
| prometheus-node-exporter subchart | 50Mi | 200m | 30Mi | 100m |
| prometheusOperator | 500Mi | 200m | 100Mi | 100m |
| prometheus | 2500Mi | 1000m | 1750Mi | 750m |
| **Total** | **3950Mi** | **2700m** | **2210Mi** | **1250m** |
At least 50Gi storage is recommended.
# Trusted CA for Notifiers
If you need to add a trusted CA to your notifier, follow these steps:
1. Create the `cattle-monitoring-system` namespace.
1. Add your trusted CA secret to the `cattle-monitoring-system` namespace.
1. Deploy or upgrade the `rancher-monitoring` Helm chart. In the chart options, reference the secret in **Alerting > Additional Secrets.**
**Result:** The default Alertmanager custom resource will have access to your trusted CA.
# Additional Scrape Configurations
If the scrape configuration you want cannot be specified via a ServiceMonitor or PodMonitor at the moment, you can provide an `additionalScrapeConfigSecret` on deploying or upgrading `rancher-monitoring`.
A [scrape_config section](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) specifies a set of targets and parameters describing how to scrape them. In the general case, one scrape configuration specifies a single job.
An example of where this might be used is with Istio. For more information, see [this section.](https://rancher.com/docs/rancher/v2.5/en/istio/configuration-reference/selectors-and-scrape)
# Configuring Applications Packaged within Monitoring v2
We deploy kube-state-metrics and node-exporter with monitoring v2. Node exporter are deployed as DaemonSets. In the monitoring v2 helm chart, in the values.yaml, each of the things are deployed as sub charts.
We also deploy grafana which is not managed by prometheus.
If you look at what the helm chart is doing like in kube-state-metrics, there are plenty more values that you can set that aren’t exposed in the top level chart.
But in the top level chart you can add values that override values that exist in the sub chart.
### Increase the Replicas of Alertmanager
As part of the chart deployment options, you can opt to increase the number of replicas of the Alertmanager deployed onto your cluster. The replicas can all be managed using the same underlying Alertmanager Config Secret. For more information on the Alertmanager Config Secret, refer to [this section]({{<baseurl>}}/monitoring-alerting/configuration/advanced/alertmanager/#multiple-alertmanager-replicas)
### Configuring the Namespace for a Persistent Grafana Dashboard
To specify that you would like Grafana to watch for ConfigMaps across all namespaces, set this value in the `rancher-monitoring` Helm chart:
```
grafana.sidecar.dashboards.searchNamespace=ALL
```
Note that the RBAC roles exposed by the Monitoring chart to add Grafana Dashboards are still restricted to giving permissions for users to add dashboards in the namespace defined in `grafana.dashboards.namespace`, which defaults to `cattle-dashboards`.
versioned_docs/version-2.5/reference-guides/monitoring-v2-configuration/receivers.md
+401
View File
@@ -0,0 +1,401 @@
---
title: Receiver Configuration
shortTitle: Receivers
weight: 1
aliases:
- /rancher/v2.5/en/monitoring-alerting/configuration/alertmanager
- rancher/v2.5/en/monitoring-alerting/legacy/notifiers/
- /rancher/v2.5/en/cluster-admin/tools/notifiers
- /rancher/v2.5/en/cluster-admin/tools/alerts
- /rancher/v2.5/en/monitoring-alerting/configuration/alertmanager
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
The [Alertmanager Config](https://prometheus.io/docs/alerting/latest/configuration/#configuration-file) Secret contains the configuration of an Alertmanager instance that sends out notifications based on alerts it receives from Prometheus.
> This section assumes familiarity with how monitoring components work together. For more information about Alertmanager, see [this section.](../../explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md#3-how-alertmanager-works)
- [Creating Receivers in the Rancher UI](#creating-receivers-in-the-rancher-ui)
- [Receiver Configuration](#receiver-configuration)
- [Slack](#slack)
- [Email](#email)
- [PagerDuty](#pagerduty)
- [Opsgenie](#opsgenie)
- [Webhook](#webhook)
- [Custom](#custom)
- [Teams](#teams)
- [SMS](#sms)
- [Route Configuration](#route-configuration)
- [Receiver](#receiver)
- [Grouping](#grouping)
- [Matching](#matching)
- [Configuring Multiple Receivers](#configuring-multiple-receivers)
- [Example Alertmanager Config](examples.md#example-alertmanager-config)
- [Example Route Config for CIS Scan Alerts](#example-route-config-for-cis-scan-alerts)
- [Trusted CA for Notifiers](#trusted-ca-for-notifiers)
# Creating Receivers in the Rancher UI
_Available as of v2.5.4_
> **Prerequisites:**
>
>- The monitoring application needs to be installed.
>- If you configured monitoring with an existing Alertmanager Secret, it must have a format that is supported by Rancher's UI. Otherwise you will only be able to make changes based on modifying the Alertmanager Secret directly. Note: We are continuing to make enhancements to what kinds of Alertmanager Configurations we can support using the Routes and Receivers UI, so please [file an issue](https://github.com/rancher/rancher/issues/new) if you have a request for a feature enhancement.
To create notification receivers in the Rancher UI,
1. Click **Cluster Explorer > Monitoring** and click **Receiver.**
2. Enter a name for the receiver.
3. Configure one or more providers for the receiver. For help filling out the forms, refer to the configuration options below.
4. Click **Create.**
**Result:** Alerts can be configured to send notifications to the receiver(s).
# Receiver Configuration
The notification integrations are configured with the `receiver`, which is explained in the [Prometheus documentation.](https://prometheus.io/docs/alerting/latest/configuration/#receiver)
### Native vs. Non-native Receivers
By default, AlertManager provides native integration with some receivers, which are listed in [this section.](https://prometheus.io/docs/alerting/latest/configuration/#receiver) All natively supported receivers are configurable through the Rancher UI.
For notification mechanisms not natively supported by AlertManager, integration is achieved using the [webhook receiver.](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config) A list of third-party drivers providing such integrations can be found [here.](https://prometheus.io/docs/operating/integrations/#alertmanager-webhook-receiver) Access to these drivers, and their associated integrations, is provided through the Alerting Drivers app. Once enabled, configuring non-native receivers can also be done through the Rancher UI.
Currently the Rancher Alerting Drivers app provides access to the following integrations:
- Microsoft Teams, based on the [prom2teams](https://github.com/idealista/prom2teams) driver
- SMS, based on the [Sachet](https://github.com/messagebird/sachet) driver
### Changes in Rancher v2.5.8
Rancher v2.5.8 added Microsoft Teams and SMS as configurable receivers in the Rancher UI.
### Changes in Rancher v2.5.4
Rancher v2.5.4 introduced the capability to configure receivers by filling out forms in the Rancher UI.
<Tabs>
<TabItem value="Rancher v2.5.8+">
The following types of receivers can be configured in the Rancher UI:
- <a href="#slack">Slack</a>
- <a href="#email">Email</a>
- <a href="#pagerduty">PagerDuty</a>
- <a href="#opsgenie">Opsgenie</a>
- <a href="#webhook">Webhook</a>
- <a href="#custom">Custom</a>
- <a href="#teams">Teams</a>
- <a href="#sms">SMS</a>
The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.
# Slack
| Field | Type | Description |
|------|--------------|------|
| URL | String | Enter your Slack webhook URL. For instructions to create a Slack webhook, see the [Slack documentation.](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack) |
| Default Channel | String | Enter the name of the channel that you want to send alert notifications in the following format: `#<channelname>`. |
| Proxy URL | String | Proxy for the webhook notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
# Email
| Field | Type | Description |
|------|--------------|------|
| Default Recipient Address | String | The email address that will receive notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
SMTP options:
| Field | Type | Description |
|------|--------------|------|
| Sender | String | Enter an email address available on your SMTP mail server that you want to send the notification from. |
| Host | String | Enter the IP address or hostname for your SMTP server. Example: `smtp.email.com`. |
| Use TLS | Bool | Use TLS for encryption. |
| Username | String | Enter a username to authenticate with the SMTP server. |
| Password | String | Enter a password to authenticate with the SMTP server. |
# PagerDuty
| Field | Type | Description |
|------|------|-------|
| Integration Type | String | `Events API v2` or `Prometheus`. |
| Default Integration Key | String | For instructions to get an integration key, see the [PagerDuty documentation.](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/) |
| Proxy URL | String | Proxy for the PagerDuty notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
# Opsgenie
| Field | Description |
|------|-------------|
| API Key | For instructions to get an API key, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/api-key-management) |
| Proxy URL | Proxy for the Opsgenie notifications. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
Opsgenie Responders:
| Field | Type | Description |
|-------|------|--------|
| Type | String | Schedule, Team, User, or Escalation. For more information on alert responders, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/alert-recipients-and-teams) |
| Send To | String | Id, Name, or Username of the Opsgenie recipient. |
# Webhook
| Field | Description |
|-------|--------------|
| URL | Webhook URL for the app of your choice. |
| Proxy URL | Proxy for the webhook notification. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
<!-- TODO add info on webhook for teams and sms and link to them -->
# Custom
The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.
# Teams
### Enabling the Teams Receiver for Rancher Managed Clusters
The Teams receiver is not a native receiver and must be enabled before it can be used. You can enable the Teams receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the Teams option selected.
1. In the Rancher UI, go to the cluster where you want to install rancher-alerting-drivers and click **Cluster Explorer**.
1. Click **Apps**.
1. Click the **Alerting Drivers** app.
1. Click the **Helm Deploy Options** tab
1. Select the **Teams** option and click **Install**.
1. Take note of the namespace used as it will be required in a later step.
### Configure the Teams Receiver
The Teams receiver can be configured by updating its ConfigMap. For example, the following is a minimal Teams receiver configuration.
```yaml
[Microsoft Teams]
teams-instance-1: https://your-teams-webhook-url
```
When configuration is complete, add the receiver using the steps in [this section](#creating-receivers-in-the-rancher-ui).
Use the example below as the URL where:
- `ns-1` is replaced with the namespace where the `rancher-alerting-drivers` app is installed
```yaml
url: http://rancher-alerting-drivers-prom2teams.ns-1.svc:8089/v2/teams-instance-1
```
<!-- https://github.com/idealista/prom2teams -->
# SMS
### Enabling the SMS Receiver for Rancher Managed Clusters
The SMS receiver is not a native receiver and must be enabled before it can be used. You can enable the SMS receiver for a Rancher managed cluster by going to the Apps page and installing the rancher-alerting-drivers app with the SMS option selected.
1. In the Rancher UI, go to the cluster where you want to install rancher-alerting-drivers and click **Cluster Explorer**.
1. Click **Apps**.
1. Click the **Alerting Drivers** app.
1. Click the **Helm Deploy Options** tab
1. Select the **SMS** option and click **Install**.
1. Take note of the namespace used as it will be required in a later step.
### Configure the SMS Receiver
The SMS receiver can be configured by updating its ConfigMap. For example, the following is a minimal SMS receiver configuration.
```yaml
providers:
telegram:
token: 'your-token-from-telegram'
receivers:
- name: 'telegram-receiver-1'
provider: 'telegram'
to:
- '123456789'
```
When configuration is complete, add the receiver using the steps in [this section](#creating-receivers-in-the-rancher-ui).
Use the example below as the name and URL, where:
- the name assigned to the receiver, e.g. `telegram-receiver-1`, must match the name in the `receivers.name` field in the ConfigMap, e.g. `telegram-receiver-1`
- `ns-1` in the URL is replaced with the namespace where the `rancher-alerting-drivers` app is installed
```yaml
name: telegram-receiver-1
url http://rancher-alerting-drivers-sachet.ns-1.svc:9876/alert
```
<!-- https://github.com/messagebird/sachet -->
</TabItem>
<TabItem value="Rancher v2.5.4-2.5.7">
The following types of receivers can be configured in the Rancher UI:
- <a href="#slack-254-257">Slack</a>
- <a href="#email-254-257">Email</a>
- <a href="#pagerduty-254-257">PagerDuty</a>
- <a href="#opsgenie-254-257">Opsgenie</a>
- <a href="#webhook-254-257">Webhook</a>
- <a href="#custom-254-257">Custom</a>
The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.
### Slack {#slack-254-257}
| Field | Type | Description |
|------|--------------|------|
| URL | String | Enter your Slack webhook URL. For instructions to create a Slack webhook, see the [Slack documentation.](https://get.slack.help/hc/en-us/articles/115005265063-Incoming-WebHooks-for-Slack) |
| Default Channel | String | Enter the name of the channel that you want to send alert notifications in the following format: `#<channelname>`. |
| Proxy URL | String | Proxy for the webhook notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
### Email {#email-254-257}
| Field | Type | Description |
|------|--------------|------|
| Default Recipient Address | String | The email address that will receive notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
SMTP options:
| Field | Type | Description |
|------|--------------|------|
| Sender | String | Enter an email address available on your SMTP mail server that you want to send the notification from. |
| Host | String | Enter the IP address or hostname for your SMTP server. Example: `smtp.email.com`. |
| Use TLS | Bool | Use TLS for encryption. |
| Username | String | Enter a username to authenticate with the SMTP server. |
| Password | String | Enter a password to authenticate with the SMTP server. |
### PagerDuty {#pagerduty-254-257}
| Field | Type | Description |
|------|------|-------|
| Integration Type | String | `Events API v2` or `Prometheus`. |
| Default Integration Key | String | For instructions to get an integration key, see the [PagerDuty documentation.](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/) |
| Proxy URL | String | Proxy for the PagerDuty notifications. |
| Enable Send Resolved Alerts | Bool | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
### Opsgenie {#opsgenie-254-257}
| Field | Description |
|------|-------------|
| API Key | For instructions to get an API key, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/api-key-management) |
| Proxy URL | Proxy for the Opsgenie notifications. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
Opsgenie Responders:
| Field | Type | Description |
|-------|------|--------|
| Type | String | Schedule, Team, User, or Escalation. For more information on alert responders, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/alert-recipients-and-teams) |
| Send To | String | Id, Name, or Username of the Opsgenie recipient. |
### Webhook {#webhook-1}
| Field | Description |
|-------|--------------|
| URL | Webhook URL for the app of your choice. |
| Proxy URL | Proxy for the webhook notification. |
| Enable Send Resolved Alerts | Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |
### Custom {#custom-254-257}
The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.
</TabItem>
<TabItem value="Rancher v2.5.0-2.5.3">
The Alertmanager must be configured in YAML, as shown in these [examples.](#example-alertmanager-configs)
</TabItem>
</Tabs>
# Configuring Multiple Receivers
By editing the forms in the Rancher UI, you can set up a Receiver resource with all the information Alertmanager needs to send alerts to your notification system.
It is also possible to send alerts to multiple notification systems. One way is to configure the Receiver using custom YAML, in which case you can add the configuration for multiple notification systems, as long as you are sure that both systems should receive the same messages.
You can also set up multiple receivers by using the `continue` option for a route, so that the alerts sent to a receiver continue being evaluated in the next level of the routing tree, which could contain another receiver.
# Example Alertmanager Configs
### Slack
To set up notifications via Slack, the following Alertmanager Config YAML can be placed into the `alertmanager.yaml` key of the Alertmanager Config Secret, where the `api_url` should be updated to use your Webhook URL from Slack:
```yaml
route:
group_by: ['job']
group_wait: 30s
group_interval: 5m
repeat_interval: 3h
receiver: 'slack-notifications'
receivers:
- name: 'slack-notifications'
slack_configs:
- send_resolved: true
text: '{{ template "slack.rancher.text" . }}'
api_url: <user-provided slack webhook url here>
templates:
- /etc/alertmanager/config/*.tmpl
```
### PagerDuty
To set up notifications via PagerDuty, use the example below from the [PagerDuty documentation](https://www.pagerduty.com/docs/guides/prometheus-integration-guide/) as a guideline. This example sets up a route that captures alerts for a database service and sends them to a receiver linked to a service that will directly notify the DBAs in PagerDuty, while all other alerts will be directed to a default receiver with a different PagerDuty integration key.
The following Alertmanager Config YAML can be placed into the `alertmanager.yaml` key of the Alertmanager Config Secret. The `service_key` should be updated to use your PagerDuty integration key and can be found as per the "Integrating with Global Event Routing" section of the PagerDuty documentation. For the full list of configuration options, refer to the [Prometheus documentation](https://prometheus.io/docs/alerting/latest/configuration/#pagerduty_config).
```yaml
route:
group_by: [cluster]
receiver: 'pagerduty-notifications'
group_interval: 5m
routes:
- match:
service: database
receiver: 'database-notifcations'
receivers:
- name: 'pagerduty-notifications'
pagerduty_configs:
- service_key: 'primary-integration-key'
- name: 'database-notifcations'
pagerduty_configs:
- service_key: 'database-integration-key'
```
# Example Route Config for CIS Scan Alerts
While configuring the routes for `rancher-cis-benchmark` alerts, you can specify the matching using the key-value pair `job: rancher-cis-scan`.
For example, the following example route configuration could be used with a Slack receiver named `test-cis`:
```yaml
spec:
receiver: test-cis
group_by:
# - string
group_wait: 30s
group_interval: 30s
repeat_interval: 30s
match:
job: rancher-cis-scan
# key: string
match_re:
{}
# key: string
```
For more information on enabling alerting for `rancher-cis-benchmark`, see [this section.](cis-scans/v2.5/#enabling-alerting-for-rancher-cis-benchmark)
# Trusted CA for Notifiers
If you need to add a trusted CA to your notifier, follow the steps in [this section.](helm-chart-options.md#trusted-ca-for-notifiers)
versioned_docs/version-2.5/reference-guides/monitoring-v2-configuration/routes.md
+79
View File
@@ -0,0 +1,79 @@
---
title: Route Configuration
shortTitle: Routes
weight: 5
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
The route configuration is the section of the Alertmanager custom resource that controls how the alerts fired by Prometheus are grouped and filtered before they reach the receiver.
When a Route is changed, the Prometheus Operator regenerates the Alertmanager custom resource to reflect the changes.
For more information about configuring routes, refer to the [official Alertmanager documentation.](https://www.prometheus.io/docs/alerting/latest/configuration/#route)
> This section assumes familiarity with how monitoring components work together. For more information, see [this section.](../../explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)
- [Route Restrictions](#route-restrictions)
- [Route Configuration](#route-configuration)
- [Receiver](#receiver)
- [Grouping](#grouping)
- [Matching](#matching)
# Route Restrictions
Alertmanager proxies alerts for Prometheus based on its receivers and a routing tree that filters alerts to certain receivers based on labels.
Alerting drivers proxy alerts for Alertmanager to non-native receivers, such as Microsoft Teams and SMS.
In the Rancher UI for configuring routes and receivers, you can configure routing trees with one root and then a depth of one more level, for a tree with a depth of two. But if you use a `continue` route when configuring Alertmanager directly, you can make the tree deeper.
Each receiver is for one or more notification providers. So if you know that every alert for Slack should also go to PagerDuty, you can configure both in the same receiver.
# Route Configuration
### Note on Labels and Annotations
Labels should be used for identifying information that can affect the routing of notifications. Identifying information about the alert could consist of a container name, or the name of the team that should be notified.
Annotations should be used for information that does not affect who receives the alert, such as a runbook url or error message.
<Tabs>
<TabItem value="Rancher v2.5.4+">
### Receiver
The route needs to refer to a [receiver](#receiver-configuration) that has already been configured.
### Grouping
| Field | Default | Description |
|-------|--------------|---------|
| Group By | N/a | The labels by which incoming alerts are grouped together. For example, `[ group_by: '[' <labelname>, ... ']' ]` Multiple alerts coming in for labels such as `cluster=A` and `alertname=LatencyHigh` can be batched into a single group. To aggregate by all possible labels, use the special value `'...'` as the sole label name, for example: `group_by: ['...']` Grouping by `...` effectively disables aggregation entirely, passing through all alerts as-is. This is unlikely to be what you want, unless you have a very low alert volume or your upstream notification system performs its own grouping. |
| Group Wait | 30s | How long to wait to buffer alerts of the same group before sending initially. |
| Group Interval | 5m | How long to wait before sending an alert that has been added to a group of alerts for which an initial notification has already been sent. |
| Repeat Interval | 4h | How long to wait before re-sending a given alert that has already been sent. |
### Matching
The **Match** field refers to a set of equality matchers used to identify which alerts to send to a given Route based on labels defined on that alert. When you add key-value pairs to the Rancher UI, they correspond to the YAML in this format:
```yaml
match:
[ <labelname>: <labelvalue>, ... ]
```
The **Match Regex** field refers to a set of regex-matchers used to identify which alerts to send to a given Route based on labels defined on that alert. When you add key-value pairs in the Rancher UI, they correspond to the YAML in this format:
```yaml
match_re:
[ <labelname>: <regex>, ... ]
```
</TabItem>
<TabItem value="Rancher v2.5.0-2.5.3">
The Alertmanager must be configured in YAML, as shown in this [example.](examples.md#alertmanager-config)
</TabItem>
</Tabs>
versioned_docs/version-2.5/reference-guides/monitoring-v2-configuration/servicemonitors-and-podmonitors.md
+31
View File
@@ -0,0 +1,31 @@
---
title: ServiceMonitor and PodMonitor Configuration
shortTitle: ServiceMonitors and PodMonitors
weight: 7
---
ServiceMonitors and PodMonitors are both pseudo-CRDs that map the scrape configuration of the Prometheus custom resource.
These configuration objects declaratively specify the endpoints that Prometheus will scrape metrics from.
ServiceMonitors are more commonly used than PodMonitors, and we recommend them for most use cases.
> This section assumes familiarity with how monitoring components work together. For more information, see [this section.](../../explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)
### ServiceMonitors
This pseudo-CRD maps to a section of the Prometheus custom resource configuration. It declaratively specifies how groups of Kubernetes services should be monitored.
When a ServiceMonitor is created, the Prometheus Operator updates the Prometheus scrape configuration to include the ServiceMonitor configuration. Then Prometheus begins scraping metrics from the endpoint defined in the ServiceMonitor.
Any Services in your cluster that match the labels located within the ServiceMonitor `selector` field will be monitored based on the `endpoints` specified on the ServiceMonitor. For more information on what fields can be specified, please look at the [spec](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#servicemonitor) provided by Prometheus Operator.
For more information about how ServiceMonitors work, refer to the [Prometheus Operator documentation.](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/user-guides/running-exporters.md)
### PodMonitors
This pseudo-CRD maps to a section of the Prometheus custom resource configuration. It declaratively specifies how group of pods should be monitored.
When a PodMonitor is created, the Prometheus Operator updates the Prometheus scrape configuration to include the PodMonitor configuration. Then Prometheus begins scraping metrics from the endpoint defined in the PodMonitor.
Any Pods in your cluster that match the labels located within the PodMonitor `selector` field will be monitored based on the `podMetricsEndpoints` specified on the PodMonitor. For more information on what fields can be specified, please look at the [spec](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#podmonitorspec) provided by Prometheus Operator.
versioned_docs/version-2.5/reference-guides/pipelines/concepts.md
+39
View File
@@ -0,0 +1,39 @@
---
title: Concepts
weight: 1
aliases:
- /rancher/v2.5/en/k8s-in-rancher/pipelines/concepts
- /rancher/v2.x/en/pipelines/concepts/
---
The purpose of this page is to explain common concepts and terminology related to pipelines.
- **Pipeline:**
A _pipeline_ is a software delivery process that is broken into different stages and steps. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects. A pipeline is based on a specific repository. It defines the process to build, test, and deploy your code. Rancher uses the [pipeline as code](https://jenkins.io/doc/book/pipeline-as-code/) model. Pipeline configuration is represented as a pipeline file in the source code repository, using the file name `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`.
- **Stages:**
A pipeline stage consists of multiple steps. Stages are executed in the order defined in the pipeline file. The steps in a stage are executed concurrently. A stage starts when all steps in the former stage finish without failure.
- **Steps:**
A pipeline step is executed inside a specified stage. A step fails if it exits with a code other than `0`. If a step exits with this failure code, the entire pipeline fails and terminates.
- **Workspace:**
The workspace is the working directory shared by all pipeline steps. In the beginning of a pipeline, source code is checked out to the workspace. The command for every step bootstraps in the workspace. During a pipeline execution, the artifacts from a previous step will be available in future steps. The working directory is an ephemeral volume and will be cleaned out with the executor pod when a pipeline execution is finished.
Typically, pipeline stages include:
- **Build:**
Each time code is checked into your repository, the pipeline automatically clones the repo and builds a new iteration of your software. Throughout this process, the software is typically reviewed by automated tests.
- **Publish:**
After the build is completed, either a Docker image is built and published to a Docker registry or a catalog template is published.
- **Deploy:**
After the artifacts are published, you would release your application so users could start using the updated product.
versioned_docs/version-2.5/reference-guides/pipelines/configure-persistent-data.md
+92
View File
@@ -0,0 +1,92 @@
---
title: Configuring Persistent Data for Pipeline Components
weight: 600
aliases:
- /rancher/v2.5/en/k8s-in-rancher/pipelines/storage
- /rancher/v2.x/en/pipelines/storage/
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
The pipelines' internal Docker registry and the Minio workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes.
This section assumes that you understand how persistent storage works in Kubernetes. For more information, refer to the section on [how storage works.](../../how-to-guides/advanced-user-guides/manage-clusters/create-kubernetes-persistent-storage/manage-persistent-storage/about-persistent-storage.md)
>**Prerequisites (for both parts A and B):**
>
>[Persistent volumes](../../pages-for-subheaders/create-kubernetes-persistent-storage.md) must be available for the cluster.
### A. Configuring Persistent Data for Docker Registry
1. From the project that you're configuring a pipeline for, and click **Resources > Workloads.**
1. Find the `docker-registry` workload and select **&#8942; > Edit**.
1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section:
- **Add Volume > Add a new persistent volume (claim)**
- **Add Volume > Use an existing persistent volume (claim)**
1. Complete the form that displays to choose a persistent volume for the internal Docker registry.
<Tabs>
<TabItem value="Add a new persistent volume">
1. Enter a **Name** for the volume claim.
1. Select a volume claim **Source**:
- If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
- If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
1. From the **Customize** section, choose the read/write access for the volume.
1. Click **Define**.
</TabItem>
<TabItem value="Use an existing persistent volume">
1. Enter a **Name** for the volume claim.
1. Choose a **Persistent Volume Claim** from the drop-down.
1. From the **Customize** section, choose the read/write access for the volume.
1. Click **Define**.
</TabItem>
</Tabs>
1. From the **Mount Point** field, enter `/var/lib/registry`, which is the data storage path inside the Docker registry container.
1. Click **Upgrade**.
### B. Configuring Persistent Data for Minio
1. From the project view, click **Resources > Workloads.** Find the `minio` workload and select **&#8942; > Edit**.
1. Scroll to the **Volumes** section and expand it. Make one of the following selections from the **Add Volume** menu, which is near the bottom of the section:
- **Add Volume > Add a new persistent volume (claim)**
- **Add Volume > Use an existing persistent volume (claim)**
1. Complete the form that displays to choose a persistent volume for the internal Docker registry.
<Tabs>
<TabItem value="Add a new persistent volume">
1. Enter a **Name** for the volume claim.
1. Select a volume claim **Source**:
- If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
- If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
1. From the **Customize** section, choose the read/write access for the volume.
1. Click **Define**.
</TabItem>
<TabItem value="Use an existing persistent volume">
1. Enter a **Name** for the volume claim.
1. Choose a **Persistent Volume Claim** from the drop-down.
1. From the **Customize** section, choose the read/write access for the volume.
1. Click **Define**.
</TabItem>
</Tabs>
1. From the **Mount Point** field, enter `/data`, which is the data storage path inside the Minio container.
1. Click **Upgrade**.
**Result:** Persistent storage is configured for your pipeline components.
versioned_docs/version-2.5/reference-guides/pipelines/example-repositories.md
+78
View File
@@ -0,0 +1,78 @@
---
title: Example Repositories
weight: 500
aliases:
- /rancher/v2.5/en/tools/pipelines/quick-start-guide/
- /rancher/v2.5/en/k8s-in-rancher/pipelines/example-repos
- /rancher/v2.x/en/pipelines/example-repos/
---
Rancher ships with several example repositories that you can use to familiarize yourself with pipelines. We recommend configuring and testing the example repository that most resembles your environment before using pipelines with your own repositories in a production environment. Use this example repository as a sandbox for repo configuration, build demonstration, etc. Rancher includes example repositories for:
- Go
- Maven
- php
> **Note:** The example repositories are only available if you have not [configured a version control provider](../../how-to-guides/advanced-user-guides/manage-projects/ci-cd-pipelines.md).
To start using these example repositories,
1. [Enable the example repositories](#1-enable-the-example-repositories)
2. [View the example pipeline](#2-view-the-example-pipeline)
3. [Run the example pipeline](#3-run-the-example-pipeline)
### 1. Enable the Example Repositories
By default, the example pipeline repositories are disabled. Enable one (or more) to test out the pipeline feature and see how it works.
1. From the **Global** view, navigate to the project that you want to test out pipelines.
1. Click **Resources > Pipelines.**
1. Click **Configure Repositories**.
**Step Result:** A list of example repositories displays.
>**Note:** Example repositories only display if you haven't fetched your own repos.
1. Click **Enable** for one of the example repos (e.g., `https://github.com/rancher/pipeline-example-go.git`). Then click **Done**.
**Results:**
- The example repository is enabled to work with a pipeline is available in the **Pipeline** tab.
- The following workloads are deployed to a new namespace:
- `docker-registry`
- `jenkins`
- `minio`
### 2. View the Example Pipeline
After enabling an example repository, review the pipeline to see how it is set up.
1. From the **Global** view, navigate to the project that you want to test out pipelines.
1. Click **Resources > Pipelines.**
1. Find the example repository, select the vertical **&#8942;**. There are two ways to view the pipeline:
* **Rancher UI**: Click on **Edit Config** to view the stages and steps of the pipeline.
* **YAML**: Click on View/Edit YAML to view the `./rancher-pipeline.yml` file.
### 3. Run the Example Pipeline
After enabling an example repository, run the pipeline to see how it works.
1. From the **Global** view, navigate to the project that you want to test out pipelines.
1. Click **Resources > Pipelines.**
1. Find the example repository, select the vertical **&#8942; > Run**.
>**Note:** When you run a pipeline the first time, it takes a few minutes to pull relevant images and provision necessary pipeline components.
**Result:** The pipeline runs. You can see the results in the logs.
### What's Next?
For detailed information about setting up your own pipeline for your repository, [configure a version control provider](../../how-to-guides/advanced-user-guides/manage-projects/ci-cd-pipelines.md), enable a repository and finally configure your pipeline.
versioned_docs/version-2.5/reference-guides/pipelines/example-yaml.md
+76
View File
@@ -0,0 +1,76 @@
---
title: Example YAML File
weight: 501
aliases:
- /rancher/v2.5/en/tools/pipelines/reference/
- /rancher/v2.5/en/k8s-in-rancher/pipelines/example
- /rancher/v2.x/en/pipelines/example/
---
Pipelines can be configured either through the UI or using a yaml file in the repository, i.e. `.rancher-pipeline.yml` or `.rancher-pipeline.yaml`.
In the [pipeline configuration reference](k8s-in-rancher/pipelines/config), we provide examples of how to configure each feature using the Rancher UI or using YAML configuration.
Below is a full example `rancher-pipeline.yml` for those who want to jump right in.
```yaml
# example
stages:
- name: Build something
# Conditions for stages
when:
branch: master
event: [ push, pull_request ]
# Multiple steps run concurrently
steps:
- runScriptConfig:
image: busybox
shellScript: echo ${FIRST_KEY} && echo ${ALIAS_ENV}
# Set environment variables in container for the step
env:
FIRST_KEY: VALUE
SECOND_KEY: VALUE2
# Set environment variables from project secrets
envFrom:
- sourceName: my-secret
sourceKey: secret-key
targetKey: ALIAS_ENV
- runScriptConfig:
image: busybox
shellScript: date -R
# Conditions for steps
when:
branch: [ master, dev ]
event: push
- name: Publish my image
steps:
- publishImageConfig:
dockerfilePath: ./Dockerfile
buildContext: .
tag: rancher/rancher:v2.0.0
# Optionally push to remote registry
pushRemote: true
registry: reg.example.com
- name: Deploy some workloads
steps:
- applyYamlConfig:
path: ./deployment.yaml
# branch conditions for the pipeline
branch:
include: [ master, feature/*]
exclude: [ dev ]
# timeout in minutes
timeout: 30
notification:
recipients:
- # Recipient
recipient: "#mychannel"
# ID of Notifier
notifier: "c-wdcsr:n-c9pg7"
- recipient: "test@example.com"
notifier: "c-wdcsr:n-lkrhd"
# Select which statuses you want the notification to be sent
condition: ["Failed", "Success", "Changed"]
# Ability to override the default message (Optional)
message: "my-message"
```
versioned_docs/version-2.5/reference-guides/pipelines/pipeline-configuration.md
+648
View File
@@ -0,0 +1,648 @@
---
title: Pipeline Configuration Reference
weight: 1
aliases:
- /rancher/v2.5/en/k8s-in-rancher/pipelines/config
- /rancher/v2.x/en/pipelines/config/
---
In this section, you'll learn how to configure pipelines.
- [Step Types](#step-types)
- [Step Type: Run Script](#step-type-run-script)
- [Step Type: Build and Publish Images](#step-type-build-and-publish-images)
- [Step Type: Publish Catalog Template](#step-type-publish-catalog-template)
- [Step Type: Deploy YAML](#step-type-deploy-yaml)
- [Step Type: Deploy Catalog App](#step-type-deploy-catalog-app)
- [Notifications](#notifications)
- [Timeouts](#timeouts)
- [Triggers and Trigger Rules](#triggers-and-trigger-rules)
- [Environment Variables](#environment-variables)
- [Secrets](#secrets)
- [Pipeline Variable Substitution Reference](#pipeline-variable-substitution-reference)
- [Global Pipeline Execution Settings](#global-pipeline-execution-settings)
- [Executor Quota](#executor-quota)
- [Resource Quota for Executors](#resource-quota-for-executors)
- [Custom CA](#custom-ca)
- [Persistent Data for Pipeline Components](#persistent-data-for-pipeline-components)
- [Example rancher-pipeline.yml](#example-rancher-pipeline-yml)
# Step Types
Within each stage, you can add as many steps as you'd like. When there are multiple steps in one stage, they run concurrently.
Step types include:
- [Run Script](#step-type-run-script)
- [Build and Publish Images](#step-type-build-and-publish-images)
- [Publish Catalog Template](#step-type-publish-catalog-template)
- [Deploy YAML](#step-type-deploy-yaml)
- [Deploy Catalog App](#step-type-deploy-catalog-app)
<!--
### Clone
The first stage is preserved to be a cloning step that checks out source code from your repo. Rancher handles the cloning of the git repository. This action is equivalent to `git clone <repository_link> <workspace_dir>`.
-->
### Configuring Steps By UI
If you haven't added any stages, click **Configure pipeline for this branch** to configure the pipeline through the UI.
1. Add stages to your pipeline execution by clicking **Add Stage**.
1. Enter a **Name** for each stage of your pipeline.
1. For each stage, you can configure [trigger rules](#triggers-and-trigger-rules) by clicking on **Show Advanced Options**. Note: this can always be updated at a later time.
1. After you've created a stage, start [adding steps](#step-types) by clicking **Add a Step**. You can add multiple steps to each stage.
### Configuring Steps by YAML
For each stage, you can add multiple steps. Read more about each [step type](#step-types) and the advanced options to get all the details on how to configure the YAML. This is only a small example of how to have multiple stages with a singular step in each stage.
```yaml
# example
stages:
- name: Build something
# Conditions for stages
when:
branch: master
event: [ push, pull_request ]
# Multiple steps run concurrently
steps:
- runScriptConfig:
image: busybox
shellScript: date -R
- name: Publish my image
steps:
- publishImageConfig:
dockerfilePath: ./Dockerfile
buildContext: .
tag: rancher/rancher:v2.0.0
# Optionally push to remote registry
pushRemote: true
registry: reg.example.com
```
# Step Type: Run Script
The **Run Script** step executes arbitrary commands in the workspace inside a specified container. You can use it to build, test and do more, given whatever utilities the base image provides. For your convenience, you can use variables to refer to metadata of a pipeline execution. Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables.
### Configuring Script by UI
1. From the **Step Type** drop-down, choose **Run Script** and fill in the form.
1. Click **Add**.
### Configuring Script by YAML
```yaml
# example
stages:
- name: Build something
steps:
- runScriptConfig:
image: golang
shellScript: go build
```
# Step Type: Build and Publish Images
The **Build and Publish Image** step builds and publishes a Docker image. This process requires a Dockerfile in your source code's repository to complete successfully.
The option to publish an image to an insecure registry is not exposed in the UI, but you can specify an environment variable in the YAML that allows you to publish an image insecurely.
### Configuring Building and Publishing Images by UI
1. From the **Step Type** drop-down, choose **Build and Publish**.
1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**.
Field | Description |
---------|----------|
Dockerfile Path | The relative path to the Dockerfile in the source code repo. By default, this path is `./Dockerfile`, which assumes the Dockerfile is in the root directory. You can set it to other paths in different use cases (`./path/to/myDockerfile` for example). |
Image Name | The image name in `name:tag` format. The registry address is not required. For example, to build `example.com/repo/my-image:dev`, enter `repo/my-image:dev`. |
Push image to remote repository | An option to set the registry that publishes the image that's built. To use this option, enable it and choose a registry from the drop-down. If this option is disabled, the image is pushed to the internal registry. |
Build Context <br/><br/> (**Show advanced options**)| By default, the root directory of the source code (`.`). For more details, see the Docker [build command documentation](https://docs.docker.com/engine/reference/commandline/build/).
### Configuring Building and Publishing Images by YAML
You can use specific arguments for Docker daemon and the build. They are not exposed in the UI, but they are available in pipeline YAML format, as indicated in the example below. Available environment variables include:
Variable Name | Description
------------------------|------------------------------------------------------------
PLUGIN_DRY_RUN | Disable docker push
PLUGIN_DEBUG | Docker daemon executes in debug mode
PLUGIN_MIRROR | Docker daemon registry mirror
PLUGIN_INSECURE | Docker daemon allows insecure registries
PLUGIN_BUILD_ARGS | Docker build args, a comma separated list
<br/>
```yaml
# This example shows an environment variable being used
# in the Publish Image step. This variable allows you to
# publish an image to an insecure registry:
stages:
- name: Publish Image
steps:
- publishImageConfig:
dockerfilePath: ./Dockerfile
buildContext: .
tag: repo/app:v1
pushRemote: true
registry: example.com
env:
PLUGIN_INSECURE: "true"
```
# Step Type: Publish Catalog Template
The **Publish Catalog Template** step publishes a version of a catalog app template (i.e. Helm chart) to a git hosted chart repository. It generates a git commit and pushes it to your chart repository. This process requires a chart folder in your source code's repository and a pre-configured secret in the dedicated pipeline namespace to complete successfully. Any variables in the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) is supported for any file in the chart folder.
### Configuring Publishing a Catalog Template by UI
1. From the **Step Type** drop-down, choose **Publish Catalog Template**.
1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**.
Field | Description |
---------|----------|
Chart Folder | The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located. |
Catalog Template Name | The name of the template. For example, wordpress. |
Catalog Template Version | The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file. |
Protocol | You can choose to publish via HTTP(S) or SSH protocol. |
Secret | The secret that stores your Git credentials. You need to create a secret in dedicated pipeline namespace in the project before adding this step. If you use HTTP(S) protocol, store Git username and password in `USERNAME` and `PASSWORD` key of the secret. If you use SSH protocol, store Git deploy key in `DEPLOY_KEY` key of the secret. After the secret is created, select it in this option. |
Git URL | The Git URL of the chart repository that the template will be published to. |
Git Branch | The Git branch of the chart repository that the template will be published to. |
Author Name | The author name used in the commit message. |
Author Email | The author email used in the commit message. |
### Configuring Publishing a Catalog Template by YAML
You can add **Publish Catalog Template** steps directly in the `.rancher-pipeline.yml` file.
Under the `steps` section, add a step with `publishCatalogConfig`. You will provide the following information:
* Path: The relative path to the chart folder in the source code repo, where the `Chart.yaml` file is located.
* CatalogTemplate: The name of the template.
* Version: The version of the template you want to publish, it should be consistent with the version defined in the `Chart.yaml` file.
* GitUrl: The git URL of the chart repository that the template will be published to.
* GitBranch: The git branch of the chart repository that the template will be published to.
* GitAuthor: The author name used in the commit message.
* GitEmail: The author email used in the commit message.
* Credentials: You should provide Git credentials by referencing secrets in dedicated pipeline namespace. If you publish via SSH protocol, inject your deploy key to the `DEPLOY_KEY` environment variable. If you publish via HTTP(S) protocol, inject your username and password to `USERNAME` and `PASSWORD` environment variables.
```yaml
# example
stages:
- name: Publish Wordpress Template
steps:
- publishCatalogConfig:
path: ./charts/wordpress/latest
catalogTemplate: wordpress
version: ${CICD_GIT_TAG}
gitUrl: git@github.com:myrepo/charts.git
gitBranch: master
gitAuthor: example-user
gitEmail: user@example.com
envFrom:
- sourceName: publish-keys
sourceKey: DEPLOY_KEY
```
# Step Type: Deploy YAML
This step deploys arbitrary Kubernetes resources to the project. This deployment requires a Kubernetes manifest file to be present in the source code repository. Pipeline variable substitution is supported in the manifest file. You can view an example file at [GitHub](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml). Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables.
### Configure Deploying YAML by UI
1. From the **Step Type** drop-down, choose **Deploy YAML** and fill in the form.
1. Enter the **YAML Path**, which is the path to the manifest file in the source code.
1. Click **Add**.
### Configure Deploying YAML by YAML
```yaml
# example
stages:
- name: Deploy
steps:
- applyYamlConfig:
path: ./deployment.yaml
```
# Step Type :Deploy Catalog App
The **Deploy Catalog App** step deploys a catalog app in the project. It will install a new app if it is not present, or upgrade an existing one.
### Configure Deploying Catalog App by UI
1. From the **Step Type** drop-down, choose **Deploy Catalog App**.
1. Fill in the rest of the form. Descriptions for each field are listed below. When you're done, click **Add**.
Field | Description |
---------|----------|
Catalog | The catalog from which the app template will be used. |
Template Name | The name of the app template. For example, wordpress. |
Template Version | The version of the app template you want to deploy. |
Namespace | The target namespace where you want to deploy the app. |
App Name | The name of the app you want to deploy. |
Answers | Key-value pairs of answers used to deploy the app. |
### Configure Deploying Catalog App by YAML
You can add **Deploy Catalog App** steps directly in the `.rancher-pipeline.yml` file.
Under the `steps` section, add a step with `applyAppConfig`. You will provide the following information:
* CatalogTemplate: The ID of the template. This can be found by clicking `Launch app` and selecting `View details` for the app. It is the last part of the URL.
* Version: The version of the template you want to deploy.
* Answers: Key-value pairs of answers used to deploy the app.
* Name: The name of the app you want to deploy.
* TargetNamespace: The target namespace where you want to deploy the app.
```yaml
# example
stages:
- name: Deploy App
steps:
- applyAppConfig:
catalogTemplate: cattle-global-data:library-mysql
version: 0.3.8
answers:
persistence.enabled: "false"
name: testmysql
targetNamespace: test
```
# Timeouts
By default, each pipeline execution has a timeout of 60 minutes. If the pipeline execution cannot complete within its timeout period, the pipeline is aborted.
### Configuring Timeouts by UI
Enter a new value in the **Timeout** field.
### Configuring Timeouts by YAML
In the `timeout` section, enter the timeout value in minutes.
```yaml
# example
stages:
- name: Build something
steps:
- runScriptConfig:
image: busybox
shellScript: ls
# timeout in minutes
timeout: 30
```
# Notifications
You can enable notifications to any notifiers based on the build status of a pipeline. Before enabling notifications, Rancher recommends [setting up notifiers](monitoring-alerting/legacy/notifiers/) so it will be easy to add recipients immediately.
### Configuring Notifications by UI
1. Within the **Notification** section, turn on notifications by clicking **Enable**.
1. Select the conditions for the notification. You can select to get a notification for the following statuses: `Failed`, `Success`, `Changed`. For example, if you want to receive notifications when an execution fails, select **Failed**.
1. If you don't have any existing notifiers, Rancher will provide a warning that no notifiers are set up and provide a link to be able to go to the notifiers page. Follow the [instructions](monitoring-alerting/legacy/notifiers/) to add a notifier. If you already have notifiers, you can add them to the notification by clicking the **Add Recipient** button.
> **Note:** Notifiers are configured at a cluster level and require a different level of permissions.
1. For each recipient, select which notifier type from the dropdown. Based on the type of notifier, you can use the default recipient or override the recipient with a different one. For example, if you have a notifier for _Slack_, you can update which channel to send the notification to. You can add additional notifiers by clicking **Add Recipient**.
### Configuring Notifications by YAML
In the `notification` section, you will provide the following information:
* **Recipients:** This will be the list of notifiers/recipients that will receive the notification.
* **Notifier:** The ID of the notifier. This can be found by finding the notifier and selecting **View in API** to get the ID.
* **Recipient:** Depending on the type of the notifier, the "default recipient" can be used or you can override this with a different recipient. For example, when configuring a slack notifier, you select a channel as your default recipient, but if you wanted to send notifications to a different channel, you can select a different recipient.
* **Condition:** Select which conditions of when you want the notification to be sent.
* **Message (Optional):** If you want to change the default notification message, you can edit this in the yaml. Note: This option is not available in the UI.
```yaml
# Example
stages:
- name: Build something
steps:
- runScriptConfig:
image: busybox
shellScript: ls
notification:
recipients:
- # Recipient
recipient: "#mychannel"
# ID of Notifier
notifier: "c-wdcsr:n-c9pg7"
- recipient: "test@example.com"
notifier: "c-wdcsr:n-lkrhd"
# Select which statuses you want the notification to be sent
condition: ["Failed", "Success", "Changed"]
# Ability to override the default message (Optional)
message: "my-message"
```
# Triggers and Trigger Rules
After you configure a pipeline, you can trigger it using different methods:
- **Manually:**
After you configure a pipeline, you can trigger a build using the latest CI definition from Rancher UI. When a pipeline execution is triggered, Rancher dynamically provisions a Kubernetes pod to run your CI tasks and then remove it upon completion.
- **Automatically:**
When you enable a repository for a pipeline, webhooks are automatically added to the version control system. When project users interact with the repo by pushing code, opening pull requests, or creating a tag, the version control system sends a webhook to Rancher Server, triggering a pipeline execution.
To use this automation, webhook management permission is required for the repository. Therefore, when users authenticate and fetch their repositories, only those on which they have webhook management permission will be shown.
Trigger rules can be created to have fine-grained control of pipeline executions in your pipeline configuration. Trigger rules come in two types:
- **Run this when:** This type of rule starts the pipeline, stage, or step when a trigger explicitly occurs.
- **Do Not Run this when:** This type of rule skips the pipeline, stage, or step when a trigger explicitly occurs.
If all conditions evaluate to `true`, then the pipeline/stage/step is executed. Otherwise it is skipped. When a pipeline is skipped, none of the pipeline is executed. When a stage/step is skipped, it is considered successful and follow-up stages/steps continue to run.
Wildcard character (`*`) expansion is supported in `branch` conditions.
This section covers the following topics:
- [Configuring pipeline triggers](#configuring-pipeline-triggers)
- [Configuring stage triggers](#configuring-stage-triggers)
- [Configuring step triggers](#configuring-step-triggers)
- [Configuring triggers by YAML](#configuring-triggers-by-yaml)
### Configuring Pipeline Triggers
1. From the **Global** view, navigate to the project that you want to configure a pipeline trigger rule.
1. Click **Resources > Pipelines.**
1. From the repository for which you want to manage trigger rules, select the vertical **&#8942; > Edit Config**.
1. Click on **Show Advanced Options**.
1. In the **Trigger Rules** section, configure rules to run or skip the pipeline.
1. Click **Add Rule**. In the **Value** field, enter the name of the branch that triggers the pipeline.
1. **Optional:** Add more branches that trigger a build.
1. Click **Done.**
### Configuring Stage Triggers
1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule.
1. Click **Resources > Pipelines.**
1. From the repository for which you want to manage trigger rules, select the vertical **&#8942; > Edit Config**.
1. Find the **stage** that you want to manage trigger rules, click the **Edit** icon for that stage.
1. Click **Show advanced options**.
1. In the **Trigger Rules** section, configure rules to run or skip the stage.
1. Click **Add Rule**.
1. Choose the **Type** that triggers the stage and enter a value.
| Type | Value |
| ------ | -------------------------------------------------------------------- |
| Branch | The name of the branch that triggers the stage. |
| Event | The type of event that triggers the stage. Values are: `Push`, `Pull Request`, `Tag` |
1. Click **Save**.
### Configuring Step Triggers
1. From the **Global** view, navigate to the project that you want to configure a stage trigger rule.
1. Click **Resources > Pipelines.**
1. From the repository for which you want to manage trigger rules, select the vertical **&#8942; > Edit Config**.
1. Find the **step** that you want to manage trigger rules, click the **Edit** icon for that step.
1. Click **Show advanced options**.
1. In the **Trigger Rules** section, configure rules to run or skip the step.
1. Click **Add Rule**.
1. Choose the **Type** that triggers the step and enter a value.
| Type | Value |
| ------ | -------------------------------------------------------------------- |
| Branch | The name of the branch that triggers the step. |
| Event | The type of event that triggers the step. Values are: `Push`, `Pull Request`, `Tag` |
1. Click **Save**.
### Configuring Triggers by YAML
```yaml
# example
stages:
- name: Build something
# Conditions for stages
when:
branch: master
event: [ push, pull_request ]
# Multiple steps run concurrently
steps:
- runScriptConfig:
image: busybox
shellScript: date -R
# Conditions for steps
when:
branch: [ master, dev ]
event: push
# branch conditions for the pipeline
branch:
include: [ master, feature/*]
exclude: [ dev ]
```
# Environment Variables
When configuring a pipeline, certain [step types](#step-types) allow you to use environment variables to configure the step's script.
### Configuring Environment Variables by UI
1. From the **Global** view, navigate to the project that you want to configure pipelines.
1. Click **Resources > Pipelines.**
1. From the pipeline for which you want to edit build triggers, select **&#8942; > Edit Config**.
1. Within one of the stages, find the **step** that you want to add an environment variable for, click the **Edit** icon.
1. Click **Show advanced options**.
1. Click **Add Variable**, and then enter a key and value in the fields that appear. Add more variables if needed.
1. Add your environment variable(s) into either the script or file.
1. Click **Save**.
### Configuring Environment Variables by YAML
```yaml
# example
stages:
- name: Build something
steps:
- runScriptConfig:
image: busybox
shellScript: echo ${FIRST_KEY} && echo ${SECOND_KEY}
env:
FIRST_KEY: VALUE
SECOND_KEY: VALUE2
```
# Secrets
If you need to use security-sensitive information in your pipeline scripts (like a password), you can pass them in using Kubernetes [secrets](../../how-to-guides/new-user-guides/kubernetes-resources-setup/secrets.md).
### Prerequisite
Create a secret in the same project as your pipeline, or explicitly in the namespace where pipeline build pods run.
<br/>
>**Note:** Secret injection is disabled on [pull request events](#triggers-and-trigger-rules).
### Configuring Secrets by UI
1. From the **Global** view, navigate to the project that you want to configure pipelines.
1. Click **Resources > Pipelines.**
1. From the pipeline for which you want to edit build triggers, select **&#8942; > Edit Config**.
1. Within one of the stages, find the **step** that you want to use a secret for, click the **Edit** icon.
1. Click **Show advanced options**.
1. Click **Add From Secret**. Select the secret file that you want to use. Then choose a key. Optionally, you can enter an alias for the key.
1. Click **Save**.
### Configuring Secrets by YAML
```yaml
# example
stages:
- name: Build something
steps:
- runScriptConfig:
image: busybox
shellScript: echo ${ALIAS_ENV}
# environment variables from project secrets
envFrom:
- sourceName: my-secret
sourceKey: secret-key
targetKey: ALIAS_ENV
```
# Pipeline Variable Substitution Reference
For your convenience, the following variables are available for your pipeline configuration scripts. During pipeline executions, these variables are replaced by metadata. You can reference them in the form of `${VAR_NAME}`.
Variable Name | Description
------------------------|------------------------------------------------------------
`CICD_GIT_REPO_NAME` | Repository name (Github organization omitted).
`CICD_GIT_URL` | URL of the Git repository.
`CICD_GIT_COMMIT` | Git commit ID being executed.
`CICD_GIT_BRANCH` | Git branch of this event.
`CICD_GIT_REF` | Git reference specification of this event.
`CICD_GIT_TAG` | Git tag name, set on tag event.
`CICD_EVENT` | Event that triggered the build (`push`, `pull_request` or `tag`).
`CICD_PIPELINE_ID` | Rancher ID for the pipeline.
`CICD_EXECUTION_SEQUENCE` | Build number of the pipeline.
`CICD_EXECUTION_ID` | Combination of `{CICD_PIPELINE_ID}-{CICD_EXECUTION_SEQUENCE}`.
`CICD_REGISTRY` | Address for the Docker registry for the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step.
`CICD_IMAGE` | Name of the image built from the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. It does not contain the image tag.<br/><br/> [Example](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml)
# Global Pipeline Execution Settings
After configuring a version control provider, there are several options that can be configured globally on how pipelines are executed in Rancher. These settings can be edited by selecting **Tools > Pipelines** in the navigation bar.
- [Executor Quota](#executor-quota)
- [Resource Quota for Executors](#resource-quota-for-executors)
- [Custom CA](#custom-ca)
### Executor Quota
Select the maximum number of pipeline executors. The _executor quota_ decides how many builds can run simultaneously in the project. If the number of triggered builds exceeds the quota, subsequent builds will queue until a vacancy opens. By default, the quota is `2`. A value of `0` or less removes the quota limit.
### Resource Quota for Executors
Configure compute resources for Jenkins agent containers. When a pipeline execution is triggered, a build pod is dynamically provisioned to run your CI tasks. Under the hood, A build pod consists of one Jenkins agent container and one container for each pipeline step. You can [manage compute resources](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) for every containers in the pod.
Edit the **Memory Reservation**, **Memory Limit**, **CPU Reservation** or **CPU Limit**, then click **Update Limit and Reservation**.
To configure compute resources for pipeline-step containers:
You can configure compute resources for pipeline-step containers in the `.rancher-pipeline.yml` file.
In a step, you will provide the following information:
* **CPU Reservation (`CpuRequest`)**: CPU request for the container of a pipeline step.
* **CPU Limit (`CpuLimit`)**: CPU limit for the container of a pipeline step.
* **Memory Reservation (`MemoryRequest`)**: Memory request for the container of a pipeline step.
* **Memory Limit (`MemoryLimit`)**: Memory limit for the container of a pipeline step.
```yaml
# example
stages:
- name: Build something
steps:
- runScriptConfig:
image: busybox
shellScript: ls
cpuRequest: 100m
cpuLimit: 1
memoryRequest:100Mi
memoryLimit: 1Gi
- publishImageConfig:
dockerfilePath: ./Dockerfile
buildContext: .
tag: repo/app:v1
cpuRequest: 100m
cpuLimit: 1
memoryRequest:100Mi
memoryLimit: 1Gi
```
>**Note:** Rancher sets default compute resources for pipeline steps except for `Build and Publish Images` and `Run Script` steps. You can override the default value by specifying compute resources in the same way.
### Custom CA
If you want to use a version control provider with a certificate from a custom/internal CA root, the CA root certificates need to be added as part of the version control provider configuration in order for the pipeline build pods to succeed.
1. Click **Edit cacerts**.
1. Paste in the CA root certificates and click **Save cacerts**.
**Result:** Pipelines can be used and new pods will be able to work with the self-signed-certificate.
# Persistent Data for Pipeline Components
The internal Docker registry and the Minio workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes.
For details on setting up persistent storage for pipelines, refer to [this page.](k8s-in-rancher/pipelines/storage)
# Example rancher-pipeline.yml
An example pipeline configuration file is on [this page.](k8s-in-rancher/pipelines/example)
versioned_docs/version-2.5/reference-guides/rancher-cluster-tools.md
+62
View File
@@ -0,0 +1,62 @@
---
title: Tools for Logging, Monitoring, and Visibility
weight: 2033
aliases:
- /rancher/v2.5/en/tools/notifiers-and-alerts/
- /rancher/v2.x/en/cluster-admin/tools/
---
Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently. Tools are divided into following categories:
<!-- TOC -->
- [Logging](#logging)
- [Monitoring and Alerts](#monitoring-and-alerts)
- [Istio](#istio)
- [OPA Gatekeeper](#opa-gatekeeper)
- [CIS Scans](#cis-scans)
<!-- /TOC -->
# Logging
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
For more information, refer to the logging documentation [here.](../pages-for-subheaders/logging.md)
# Monitoring and Alerts
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution.
After monitoring is enabled, you can set up alerts and notifiers that provide the mechanism to receive them.
Notifiers are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Notifications can be sent with Slack, email, PagerDuty, WeChat, and webhooks.
Alerts are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
For more information, refer to the monitoring documentation [here.](../pages-for-subheaders/monitoring-and-alerting.md)
# Istio
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, control, troubleshoot, and secure the traffic within a complex network of microservices.
Rancher's integration with Istio was improved in Rancher v2.5.
For more information, refer to the Istio documentation [here.](../pages-for-subheaders/istio.md)
# OPA Gatekeeper
[OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is an open-source project that provides integration between OPA and Kubernetes to provide policy control via admission controller webhooks. For details on how to enable Gatekeeper in Rancher, refer to the [OPA Gatekeeper section.](../explanations/integrations-in-rancher/opa-gatekeeper.md)
# CIS Scans
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark.
For more information, refer to the CIS scan documentation [here.](../pages-for-subheaders/cis-scan-guides.md)
versioned_docs/version-2.5/reference-guides/rancher-manager-architecture/architecture-recommendations.md
+113
View File
@@ -0,0 +1,113 @@
---
title: Architecture Recommendations
weight: 3
aliases:
- /rancher/v2.x/en/overview/architecture-recommendations/
---
Kubernetes cluster. If you are installing Rancher on a single node, the main architecture recommendation that applies to your installation is that the node running Rancher should be [separate from downstream clusters.](#separation-of-rancher-and-user-clusters)
This section covers the following topics:
- [Separation of Rancher and User Clusters](#separation-of-rancher-and-user-clusters)
- [Why HA is Better for Rancher in Production](#why-ha-is-better-for-rancher-in-production)
- [Recommended Load Balancer Configuration for Kubernetes Installations](#recommended-load-balancer-configuration-for-kubernetes-installations)
- [Environment for Kubernetes Installations](#environment-for-kubernetes-installations)
- [Recommended Node Roles for Kubernetes Installations](#recommended-node-roles-for-kubernetes-installations)
- [Architecture for an Authorized Cluster Endpoint](#architecture-for-an-authorized-cluster-endpoint)
# Separation of Rancher and User Clusters
A user cluster is a downstream Kubernetes cluster that runs your apps and services.
If you have a Docker installation of Rancher, the node running the Rancher server should be separate from your downstream clusters.
If Rancher is intended to manage downstream Kubernetes clusters, the Kubernetes cluster that the Rancher server runs on should also be separate from the downstream user clusters.
![Separation of Rancher Server from User Clusters](/img/rancher-architecture-separation-of-rancher-server.svg)
# Why HA is Better for Rancher in Production
We recommend installing the Rancher server on a high-availability Kubernetes cluster, primarily because it protects the Rancher server data. In a high-availability installation, a load balancer serves as the single point of contact for clients, distributing network traffic across multiple servers in the cluster and helping to prevent any one server from becoming a point of failure.
We don't recommend installing Rancher in a single Docker container, because if the node goes down, there is no copy of the cluster data available on other nodes and you could lose the data on your Rancher server.
### K3s Kubernetes Cluster Installations
One option for the underlying Kubernetes cluster is to use K3s Kubernetes. K3s is Rancher's CNCF certified Kubernetes distribution. It is easy to install and uses half the memory of Kubernetes, all in a binary of less than 100 MB. Another advantage of K3s is that it allows an external datastore to hold the cluster data, allowing the K3s server nodes to be treated as ephemeral.
<figcaption>Architecture of a K3s Kubernetes Cluster Running the Rancher Management Server</figcaption>
![Architecture of a K3s Kubernetes Cluster Running the Rancher Management Server](/img/k3s-server-storage.svg)
### RKE Kubernetes Cluster Installations
In an RKE installation, the cluster data is replicated on each of three etcd nodes in the cluster, providing redundancy and data duplication in case one of the nodes fails.
<figcaption>Architecture of an RKE Kubernetes Cluster Running the Rancher Management Server</figcaption>
![Architecture of an RKE Kubernetes cluster running the Rancher management server](/img/rke-server-storage.svg)
# Recommended Load Balancer Configuration for Kubernetes Installations
We recommend the following configurations for the load balancer and Ingress controllers:
* The DNS for Rancher should resolve to a Layer 4 load balancer (TCP)
* The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster.
* The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443.
* The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment.
<figcaption>Rancher installed on a Kubernetes cluster with layer 4 load balancer, depicting SSL termination at Ingress controllers</figcaption>
![Rancher HA](/img/ha/rancher2ha.svg)
# Environment for Kubernetes Installations
It is strongly recommended to install Rancher on a Kubernetes cluster on hosted infrastructure such as Amazon's EC2 or Google Compute Engine.
For the best performance and greater security, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../pages-for-subheaders/kubernetes-clusters-in-rancher-setup.md) for running your workloads.
# Recommended Node Roles for Kubernetes Installations
The below recommendations apply when Rancher is installed on a K3s Kubernetes cluster or an RKE Kubernetes cluster.
### K3s Cluster Roles
In K3s clusters, there are two types of nodes: server nodes and agent nodes. Both servers and agents can have workloads scheduled on them. Server nodes run the Kubernetes master.
For the cluster running the Rancher management server, we recommend using two server nodes. Agent nodes are not required.
### RKE Cluster Roles
If Rancher is installed on an RKE Kubernetes cluster, the cluster should have three nodes, and each node should have all three Kubernetes roles: etcd, controlplane, and worker.
### Contrasting RKE Cluster Architecture for Rancher Server and for Downstream Kubernetes Clusters
Our recommendation for RKE node roles on the Rancher server cluster contrasts with our recommendations for the downstream user clusters that run your apps and services.
Rancher uses RKE as a library when provisioning downstream Kubernetes clusters. Note: The capability to provision downstream K3s clusters will be added in a future version of Rancher.
For downstream Kubernetes clusters, we recommend that each node in a user cluster should have a single role for stability and scalability.
![Kubernetes Roles for Nodes in Rancher Server Cluster vs. User Clusters](/img/rancher-architecture-node-roles.svg)
RKE only requires at least one node with each role and does not require nodes to be restricted to one role. However, for the clusters that run your apps, we recommend separate roles for each node so that workloads on worker nodes don't interfere with the Kubernetes master or cluster data as your services scale.
We recommend that downstream user clusters should have at least:
- **Three nodes with only the etcd role** to maintain a quorum if one node is lost, making the state of your cluster highly available
- **Two nodes with only the controlplane role** to make the master component highly available
- **One or more nodes with only the worker role** to run the Kubernetes node components, as well as the workloads for your apps and services
With that said, it is safe to use all three roles on three nodes when setting up the Rancher server because:
* It allows one `etcd` node failure.
* It maintains multiple instances of the master components by having multiple `controlplane` nodes.
* No other workloads than Rancher itself should be created on this cluster.
Because no additional workloads will be deployed on the Rancher server cluster, in most cases it is not necessary to use the same architecture that we recommend for the scalability and reliability of downstream clusters.
For more best practices for downstream clusters, refer to the [production checklist](../../pages-for-subheaders/checklist-for-production-ready-clusters.md) or our [best practices guide.](best-practices/v2.5/)
# Architecture for an Authorized Cluster Endpoint
If you are using an [authorized cluster endpoint,](../../pages-for-subheaders/rancher-manager-architecture.md#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files](k8s-in-rancher/kubeconfig/) and [API keys](../user-settings/api-keys.md#creating-an-api-key) for more information.
versioned_docs/version-2.5/reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/rancher-manager-architecture/rancher-server-and-components.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/rancher-project-tools.md
+41
View File
@@ -0,0 +1,41 @@
---
title: Tools for Logging, Monitoring, and Visibility
weight: 2525
aliases:
- /rancher/v2.x/en/project-admin/tools/
---
Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently. Tools are divided into following categories:
<!-- TOC -->
- [Notifiers and Alerts](#notifiers-and-alerts)
- [Logging](#logging)
- [Monitoring](#monitoring)
<!-- /TOC -->
## Notifiers and Alerts
Notifiers and alerts are two features that work together to inform you of events in the Rancher system. Before they can be enabled, the monitoring application must be installed.
Notifiers are services that inform you of alert events. You can configure notifiers to send alert notifications to staff best suited to take corrective action. Notifications can be sent with Slack, email, PagerDuty, WeChat, and webhooks.
Alerts are rules that trigger those notifications. Before you can receive alerts, you must configure one or more notifier in Rancher. The scope for alerts can be set at either the cluster or project level.
## Logging
Logging is helpful because it allows you to:
- Capture and analyze the state of your cluster
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
For details, refer to the [logging section.](../pages-for-subheaders/logging.md)
## Monitoring
Using Rancher, you can monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments through integration with [Prometheus](https://prometheus.io/), a leading open-source monitoring solution. For details, refer to the [monitoring section.](../pages-for-subheaders/monitoring-and-alerting.md)
versioned_docs/version-2.5/reference-guides/rancher-security/kubernetes-security-best-practices.md
+12
View File
@@ -0,0 +1,12 @@
---
title: Kubernetes Security Best Practices
weight: 5
---
### Restricting cloud metadata API access
Cloud providers such as AWS, Azure, DigitalOcean or GCP often expose metadata services locally to instances. By default, this endpoint is accessible by pods running on a cloud instance, including pods in hosted Kubernetes providers such as EKS, AKS, DigitalOcean Kubernetes or GKE, and can contain cloud credentials for that node, provisioning data such as kubelet credentials, or other sensitive data. To mitigate this risk when running on a cloud platform, follow the [Kubernetes security recommendations](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#restricting-cloud-metadata-api-access): limit permissions given to instance credentials, use network policies to restrict pod access to the metadata API, and avoid using provisioning data to deliver secrets.
It is advised to consult your cloud provider's security best practices for further recommendations and specific details on how to restrict access to cloud instance metadata API.
Further references: MITRE ATT&CK knowledge base on - [Unsecured Credentials: Cloud Instance Metadata API](https://attack.mitre.org/techniques/T1552/005/).
versioned_docs/version-2.5/reference-guides/rancher-security/rancher-v2.5-hardening-guides/hardening-guide-with-cis-v1.5-benchmark.md
+724
View File
@@ -0,0 +1,724 @@
---
title: Hardening Guide with CIS 1.5 Benchmark
weight: 200
aliases:
- /rancher/v2.x/en/security/rancher-2.5/1.5-hardening-2.5/
---
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
Rancher Version | CIS Benchmark Version | Kubernetes Version
----------------|-----------------------|------------------
Rancher v2.5 | Benchmark v1.5 | Kubernetes 1.15
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.5.pdf)
### Overview
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5 with Kubernetes v1.15 or provisioning a RKE cluster with Kubernetes 1.15 to be used within Rancher v2.5. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.5 Benchmark - Self-Assessment Guide - Rancher v2.5](self-assessment-guide-with-cis-v1.5-benchmark.md).
#### Known Issues
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.5 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.5 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
### Configure Kernel Runtime Parameters
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
```
vm.overcommit_memory=1
vm.panic_on_oom=0
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxbytes=25000000
```
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
### Configure `etcd` user and group
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
#### create `etcd` user and group
To create the **etcd** group run the following console commands.
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
```
groupadd --gid 52034 etcd
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
```
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
``` yaml
services:
etcd:
gid: 52034
uid: 52034
```
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
```
automountServiceAccountToken: false
```
Save the following yaml to a file called `account_update.yaml`
``` yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
automountServiceAccountToken: false
```
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
```
#!/bin/bash -e
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
done
```
### Ensure that all Namespaces have Network Policies defined
Running different applications on the same Kubernetes cluster creates a risk of one
compromised application attacking a neighboring application. Network segmentation is
important to ensure that containers can communicate only with those they are supposed
to. A network policy is a specification of how selections of pods are allowed to
communicate with each other and other network endpoints.
Network Policies are namespace scoped. When a network policy is introduced to a given
namespace, all traffic not allowed by the policy is denied. However, if there are no network
policies in a namespace all traffic will be allowed into and out of the pods in that
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
Additional information about CNI providers can be found
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
(even if policies are added that cause some pods to be treated as “isolated”),
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
about network policies can be found on the Kubernetes site.
> This `NetworkPolicy` is not recommended for production use
``` yaml
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-allow-all
spec:
podSelector: {}
ingress:
- {}
egress:
- {}
policyTypes:
- Ingress
- Egress
```
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
```
#!/bin/bash -e
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
kubectl apply -f default-allow-all.yaml -n ${namespace}
done
```
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
### Reference Hardened RKE `cluster.yml` configuration
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
``` yaml
# If you intend to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
kubernetes_version: "v1.15.9-rancher1-1"
enable_network_policy: true
default_pod_security_policy_template_id: "restricted"
# the nodes directive is required and will vary depending on your environment
# documentation for node configuration can be found here:
# https://rancher.com/docs/rke/latest/en/config-options/nodes
nodes:
services:
etcd:
uid: 52034
gid: 52034
kube-api:
pod_security_policy: true
secrets_encryption_config:
enabled: true
audit_log:
enabled: true
admission_configuration:
event_rate_limit:
enabled: true
kube-controller:
extra_args:
feature-gates: "RotateKubeletServerCertificate=true"
scheduler:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
kubelet:
generate_serving_certificate: true
extra_args:
feature-gates: "RotateKubeletServerCertificate=true"
protect-kernel-defaults: "true"
tls-cipher-suites: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
extra_binds: []
extra_env: []
cluster_domain: ""
infra_container_image: ""
cluster_dns_server: ""
fail_swap_on: false
kubeproxy:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
network:
plugin: ""
options: {}
mtu: 0
node_selector: {}
authentication:
strategy: ""
sans: []
webhook: null
addons: |
---
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: default-psp-role
namespace: ingress-nginx
rules:
- apiGroups:
- extensions
resourceNames:
- default-psp
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-psp-rolebinding
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: default-psp-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: v1
kind: Namespace
metadata:
name: cattle-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: default-psp-role
namespace: cattle-system
rules:
- apiGroups:
- extensions
resourceNames:
- default-psp
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-psp-rolebinding
namespace: cattle-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: default-psp-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
requiredDropCapabilities:
- NET_RAW
privileged: false
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
fsGroup:
rule: RunAsAny
runAsUser:
rule: MustRunAsNonRoot
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- emptyDir
- secret
- persistentVolumeClaim
- downwardAPI
- configMap
- projected
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp:restricted
rules:
- apiGroups:
- extensions
resourceNames:
- restricted
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: psp:restricted
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
addons_include: []
system_images:
etcd: ""
alpine: ""
nginx_proxy: ""
cert_downloader: ""
kubernetes_services_sidecar: ""
kubedns: ""
dnsmasq: ""
kubedns_sidecar: ""
kubedns_autoscaler: ""
coredns: ""
coredns_autoscaler: ""
kubernetes: ""
flannel: ""
flannel_cni: ""
calico_node: ""
calico_cni: ""
calico_controllers: ""
calico_ctl: ""
calico_flexvol: ""
canal_node: ""
canal_cni: ""
canal_flannel: ""
canal_flexvol: ""
weave_node: ""
weave_cni: ""
pod_infra_container: ""
ingress: ""
ingress_backend: ""
metrics_server: ""
windows_pod_infra_container: ""
ssh_key_path: ""
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
mode: ""
options: {}
ignore_docker_version: false
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
dns_policy: ""
extra_envs: []
extra_volumes: []
extra_volume_mounts: []
cluster_name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
ssh_cert: ""
ssh_cert_path: ""
monitoring:
provider: ""
options: {}
node_selector: {}
restore:
restore: false
snapshot_name: ""
dns: null
```
### Reference Hardened RKE Template configuration
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
[documentaion](https://rancher.com/docs/rancher/v2.5/en/installation) for additional installation and RKE Template details.
``` yaml
#
# Cluster Config
#
default_pod_security_policy_template_id: restricted
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
enable_cluster_monitoring: false
enable_network_policy: true
#
# Rancher Config
#
rancher_kubernetes_engine_config:
addon_job_timeout: 30
addons: |-
---
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: default-psp-role
namespace: ingress-nginx
rules:
- apiGroups:
- extensions
resourceNames:
- default-psp
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-psp-rolebinding
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: default-psp-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: v1
kind: Namespace
metadata:
name: cattle-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: default-psp-role
namespace: cattle-system
rules:
- apiGroups:
- extensions
resourceNames:
- default-psp
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: default-psp-rolebinding
namespace: cattle-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: default-psp-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
requiredDropCapabilities:
- NET_RAW
privileged: false
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
fsGroup:
rule: RunAsAny
runAsUser:
rule: MustRunAsNonRoot
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- emptyDir
- secret
- persistentVolumeClaim
- downwardAPI
- configMap
- projected
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp:restricted
rules:
- apiGroups:
- extensions
resourceNames:
- restricted
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: psp:restricted
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
ignore_docker_version: true
kubernetes_version: v1.15.9-rancher1-1
#
# If you are using calico on AWS
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
# # To specify flannel interface
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
# # To specify flannel interface for canal plugin
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
network:
mtu: 0
plugin: canal
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
# kube-controller:
# cluster_cidr: 10.42.0.0/16
# service_cluster_ip_range: 10.43.0.0/16
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
services:
etcd:
backup_config:
enabled: false
interval_hours: 12
retention: 6
safe_timestamp: false
creation: 12h
extra_args:
election-timeout: '5000'
heartbeat-interval: '500'
gid: 52034
retention: 72h
snapshot: false
uid: 52034
kube_api:
always_pull_images: false
audit_log:
enabled: true
event_rate_limit:
enabled: true
pod_security_policy: true
secrets_encryption_config:
enabled: true
service_node_port_range: 30000-32767
kube_controller:
extra_args:
bind-address: 127.0.0.1
address: 127.0.0.1
feature-gates: RotateKubeletServerCertificate=true
profiling: 'false'
terminated-pod-gc-threshold: '1000'
kubelet:
extra_args:
anonymous-auth: 'false'
event-qps: '0'
feature-gates: RotateKubeletServerCertificate=true
make-iptables-util-chains: 'true'
protect-kernel-defaults: 'true'
streaming-connection-idle-timeout: 1800s
tls-cipher-suites: >-
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
fail_swap_on: false
generate_serving_certificate: true
scheduler:
extra_args:
bind-address: 127.0.0.1
address: 127.0.0.1
profiling: 'false'
ssh_agent_auth: false
windows_prefered_cluster: false
```
### Hardened Reference Ubuntu 18.04 LTS **cloud-config**:
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
needed before installing kubernetes.
``` yaml
#cloud-config
packages:
- curl
- jq
runcmd:
- sysctl -w vm.overcommit_memory=1
- sysctl -w kernel.panic=10
- sysctl -w kernel.panic_on_oops=1
- curl https://releases.rancher.com/install-docker/18.09.sh | sh
- usermod -aG docker ubuntu
- return=1; while [ $return != 0 ]; do sleep 2; docker ps; return=$?; done
- addgroup --gid 52034 etcd
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
write_files:
- path: /etc/sysctl.d/kubelet.conf
owner: root:root
permissions: "0644"
content: |
vm.overcommit_memory=1
kernel.panic=10
kernel.panic_on_oops=1
```
versioned_docs/version-2.5/reference-guides/rancher-security/rancher-v2.5-hardening-guides/hardening-guide-with-cis-v1.6-benchmark.md
+578
View File
@@ -0,0 +1,578 @@
---
title: Hardening Guide with CIS 1.6 Benchmark
weight: 100
aliases:
- /rancher/v2.x/en/security/rancher-2.5/1.6-hardening-2.5/
---
This document provides prescriptive guidance for hardening a production installation of a RKE cluster to be used with Rancher v2.5.4. It outlines the configurations and controls required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
> This hardening guide describes how to secure the nodes in your cluster, and it is recommended to follow this guide before installing Kubernetes.
This hardening guide is intended to be used for RKE clusters and associated with specific versions of the CIS Kubernetes Benchmark, Kubernetes, and Rancher:
Rancher Version | CIS Benchmark Version | Kubernetes Version
----------------|-----------------------|------------------
Rancher v2.5.4 | Benchmark 1.6 | Kubernetes v1.18
[Click here to download a PDF version of this document](https://releases.rancher.com/documents/security/2.5/Rancher_Hardening_Guide_CIS_1.6.pdf)
### Overview
This document provides prescriptive guidance for hardening a RKE cluster to be used for installing Rancher v2.5.4 with Kubernetes v1.18 or provisioning a RKE cluster with Kubernetes v1.18 to be used within Rancher v2.5.4. It outlines the configurations required to address Kubernetes benchmark controls from the Center for Information Security (CIS).
For more detail about evaluating a hardened cluster against the official CIS benchmark, refer to the [CIS 1.6 Benchmark - Self-Assessment Guide - Rancher v2.5.4](self-assessment-guide-with-cis-v1.6-benchmark.md).
#### Known Issues
- Rancher **exec shell** and **view logs** for pods are **not** functional in a CIS 1.6 hardened setup when only public IP is provided when registering custom nodes. This functionality requires a private IP to be provided when registering the custom nodes.
- When setting the `default_pod_security_policy_template_id:` to `restricted` Rancher creates **RoleBindings** and **ClusterRoleBindings** on the default service accounts. The CIS 1.6 5.1.5 check requires the default service accounts have no roles or cluster roles bound to it apart from the defaults. In addition the default service accounts should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
Migration Rancher from 2.4 to 2.5. Addons were removed in HG 2.5, and therefore namespaces on migration may be not created on the downstream clusters. Pod may fail to run because of missing namesapce like ingress-nginx, cattle-system.
### Configure Kernel Runtime Parameters
The following `sysctl` configuration is recommended for all nodes type in the cluster. Set the following parameters in `/etc/sysctl.d/90-kubelet.conf`:
```ini
vm.overcommit_memory=1
vm.panic_on_oom=0
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxbytes=25000000
```
Run `sysctl -p /etc/sysctl.d/90-kubelet.conf` to enable the settings.
### Configure `etcd` user and group
A user account and group for the **etcd** service is required to be setup before installing RKE. The **uid** and **gid** for the **etcd** user will be used in the RKE **config.yml** to set the proper permissions for files and directories during installation time.
#### create `etcd` user and group
To create the **etcd** group run the following console commands.
The commands below use `52034` for **uid** and **gid** are for example purposes. Any valid unused **uid** or **gid** could also be used in lieu of `52034`.
```bash
groupadd --gid 52034 etcd
useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
```
Update the RKE **config.yml** with the **uid** and **gid** of the **etcd** user:
```yaml
services:
etcd:
gid: 52034
uid: 52034
```
#### Set `automountServiceAccountToken` to `false` for `default` service accounts
Kubernetes provides a default service account which is used by cluster workloads where no specific service account is assigned to the pod. Where access to the Kubernetes API from a pod is required, a specific service account should be created for that pod, and rights granted to that service account. The default service account should be configured such that it does not provide a service account token and does not have any explicit rights assignments.
For each namespace including **default** and **kube-system** on a standard RKE install the **default** service account must include this value:
```yaml
automountServiceAccountToken: false
```
Save the following yaml to a file called `account_update.yaml`
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
automountServiceAccountToken: false
```
Create a bash script file called `account_update.sh`. Be sure to `chmod +x account_update.sh` so the script has execute permissions.
```bash
#!/bin/bash -e
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
kubectl patch serviceaccount default -n ${namespace} -p "$(cat account_update.yaml)"
done
```
### Ensure that all Namespaces have Network Policies defined
Running different applications on the same Kubernetes cluster creates a risk of one
compromised application attacking a neighboring application. Network segmentation is
important to ensure that containers can communicate only with those they are supposed
to. A network policy is a specification of how selections of pods are allowed to
communicate with each other and other network endpoints.
Network Policies are namespace scoped. When a network policy is introduced to a given
namespace, all traffic not allowed by the policy is denied. However, if there are no network
policies in a namespace all traffic will be allowed into and out of the pods in that
namespace. To enforce network policies, a CNI (container network interface) plugin must be enabled.
This guide uses [canal](https://github.com/projectcalico/canal) to provide the policy enforcement.
Additional information about CNI providers can be found
[here](https://rancher.com/blog/2019/2019-03-21-comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/)
Once a CNI provider is enabled on a cluster a default network policy can be applied. For reference purposes a
**permissive** example is provide below. If you want to allow all traffic to all pods in a namespace
(even if policies are added that cause some pods to be treated as “isolated”),
you can create a policy that explicitly allows all traffic in that namespace. Save the following `yaml` as
`default-allow-all.yaml`. Additional [documentation](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
about network policies can be found on the Kubernetes site.
> This `NetworkPolicy` is not recommended for production use
```yaml
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-allow-all
spec:
podSelector: {}
ingress:
- {}
egress:
- {}
policyTypes:
- Ingress
- Egress
```
Create a bash script file called `apply_networkPolicy_to_all_ns.sh`. Be sure to
`chmod +x apply_networkPolicy_to_all_ns.sh` so the script has execute permissions.
```bash
#!/bin/bash -e
for namespace in $(kubectl get namespaces -A -o json | jq -r '.items[].metadata.name'); do
kubectl apply -f default-allow-all.yaml -n ${namespace}
done
```
Execute this script to apply the `default-allow-all.yaml` the **permissive** `NetworkPolicy` to all namespaces.
### Reference Hardened RKE `cluster.yml` configuration
The reference `cluster.yml` is used by the RKE CLI that provides the configuration needed to achieve a hardened install
of Rancher Kubernetes Engine (RKE). Install [documentation](https://rancher.com/docs/rke/latest/en/installation/) is
provided with additional details about the configuration items. This reference `cluster.yml` does not include the required **nodes** directive which will vary depending on your environment. Documentation for node configuration can be found here: https://rancher.com/docs/rke/latest/en/config-options/nodes
```yaml
# If you intend to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
# https://rancher.com/docs/rke/latest/en/installation/
# the nodes directive is required and will vary depending on your environment
# documentation for node configuration can be found here:
# https://rancher.com/docs/rke/latest/en/config-options/nodes
nodes: []
services:
etcd:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
external_urls: []
ca_cert: ""
cert: ""
key: ""
path: ""
uid: 52034
gid: 52034
snapshot: false
retention: ""
creation: ""
backup_config: null
kube-api:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
service_cluster_ip_range: ""
service_node_port_range: ""
pod_security_policy: true
always_pull_images: false
secrets_encryption_config:
enabled: true
custom_config: null
audit_log:
enabled: true
configuration: null
admission_configuration: null
event_rate_limit:
enabled: true
configuration: null
kube-controller:
image: ""
extra_args:
feature-gates: RotateKubeletServerCertificate=true
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
cluster_cidr: ""
service_cluster_ip_range: ""
scheduler:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
kubelet:
image: ""
extra_args:
feature-gates: RotateKubeletServerCertificate=true
protect-kernel-defaults: "true"
tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
cluster_domain: cluster.local
infra_container_image: ""
cluster_dns_server: ""
fail_swap_on: false
generate_serving_certificate: true
kubeproxy:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
network:
plugin: ""
options: {}
mtu: 0
node_selector: {}
update_strategy: null
authentication:
strategy: ""
sans: []
webhook: null
addons: |
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
requiredDropCapabilities:
- NET_RAW
privileged: false
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
fsGroup:
rule: RunAsAny
runAsUser:
rule: MustRunAsNonRoot
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- emptyDir
- secret
- persistentVolumeClaim
- downwardAPI
- configMap
- projected
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp:restricted
rules:
- apiGroups:
- extensions
resourceNames:
- restricted
resources:
- podsecuritypolicies
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: psp:restricted
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:authenticated
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-allow-all
spec:
podSelector: {}
ingress:
- {}
egress:
- {}
policyTypes:
- Ingress
- Egress
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
automountServiceAccountToken: false
addons_include: []
system_images:
etcd: ""
alpine: ""
nginx_proxy: ""
cert_downloader: ""
kubernetes_services_sidecar: ""
kubedns: ""
dnsmasq: ""
kubedns_sidecar: ""
kubedns_autoscaler: ""
coredns: ""
coredns_autoscaler: ""
nodelocal: ""
kubernetes: ""
flannel: ""
flannel_cni: ""
calico_node: ""
calico_cni: ""
calico_controllers: ""
calico_ctl: ""
calico_flexvol: ""
canal_node: ""
canal_cni: ""
canal_controllers: ""
canal_flannel: ""
canal_flexvol: ""
weave_node: ""
weave_cni: ""
pod_infra_container: ""
ingress: ""
ingress_backend: ""
metrics_server: ""
windows_pod_infra_container: ""
ssh_key_path: ""
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
mode: ""
options: {}
ignore_docker_version: false
kubernetes_version: v1.18.12-rancher1-1
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
dns_policy: ""
extra_envs: []
extra_volumes: []
extra_volume_mounts: []
update_strategy: null
http_port: 0
https_port: 0
network_mode: ""
cluster_name:
cloud_provider:
name: ""
prefix_path: ""
win_prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
ssh_cert: ""
ssh_cert_path: ""
monitoring:
provider: ""
options: {}
node_selector: {}
update_strategy: null
replicas: null
restore:
restore: false
snapshot_name: ""
dns: null
upgrade_strategy:
max_unavailable_worker: ""
max_unavailable_controlplane: ""
drain: null
node_drain_input: null
```
### Reference Hardened RKE Template configuration
The reference RKE Template provides the configuration needed to achieve a hardened install of Kubenetes.
RKE Templates are used to provision Kubernetes and define Rancher settings. Follow the Rancher
[documentaion](https://rancher.com/docs/rancher/v2.5/en/installation) for additional installation and RKE Template details.
```yaml
#
# Cluster Config
#
default_pod_security_policy_template_id: restricted
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
enable_cluster_monitoring: false
enable_network_policy: true
#
# Rancher Config
#
rancher_kubernetes_engine_config:
addon_job_timeout: 45
ignore_docker_version: true
kubernetes_version: v1.18.12-rancher1-1
#
# If you are using calico on AWS
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
# # To specify flannel interface
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
# # To specify flannel interface for canal plugin
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
network:
mtu: 0
plugin: canal
rotate_encryption_key: false
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
# kube-controller:
# cluster_cidr: 10.42.0.0/16
# service_cluster_ip_range: 10.43.0.0/16
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
services:
etcd:
backup_config:
enabled: false
interval_hours: 12
retention: 6
safe_timestamp: false
creation: 12h
extra_args:
election-timeout: '5000'
heartbeat-interval: '500'
gid: 52034
retention: 72h
snapshot: false
uid: 52034
kube_api:
always_pull_images: false
audit_log:
enabled: true
event_rate_limit:
enabled: true
pod_security_policy: true
secrets_encryption_config:
enabled: true
service_node_port_range: 30000-32767
kube_controller:
extra_args:
feature-gates: RotateKubeletServerCertificate=true
bind-address: 127.0.0.1
address: 127.0.0.1
kubelet:
extra_args:
feature-gates: RotateKubeletServerCertificate=true
protect-kernel-defaults: 'true'
tls-cipher-suites: >-
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
fail_swap_on: false
generate_serving_certificate: true
scheduler:
extra_args:
bind-address: 127.0.0.1
address: 127.0.0.1
ssh_agent_auth: false
upgrade_strategy:
max_unavailable_controlplane: '1'
max_unavailable_worker: 10%
windows_prefered_cluster: false
```
### Hardened Reference Ubuntu 20.04 LTS **cloud-config**:
The reference **cloud-config** is generally used in cloud infrastructure environments to allow for
configuration management of compute instances. The reference config configures Ubuntu operating system level settings
needed before installing kubernetes.
```yaml
#cloud-config
apt:
sources:
docker.list:
source: deb [arch=amd64] http://download.docker.com/linux/ubuntu $RELEASE stable
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
system_info:
default_user:
groups:
- docker
write_files:
- path: "/etc/apt/preferences.d/docker"
owner: root:root
permissions: '0600'
content: |
Package: docker-ce
Pin: version 5:19*
Pin-Priority: 800
- path: "/etc/sysctl.d/90-kubelet.conf"
owner: root:root
permissions: '0644'
content: |
vm.overcommit_memory=1
vm.panic_on_oom=0
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxbytes=25000000
package_update: true
packages:
- docker-ce
- docker-ce-cli
- containerd.io
runcmd:
- sysctl -p /etc/sysctl.d/90-kubelet.conf
- groupadd --gid 52034 etcd
- useradd --comment "etcd service account" --uid 52034 --gid 52034 etcd
```
versioned_docs/version-2.5/reference-guides/rancher-security/rancher-v2.5-hardening-guides/self-assessment-guide-with-cis-v1.5-benchmark.md
+2267
View File
File diff suppressed because it is too large Load Diff
versioned_docs/version-2.5/reference-guides/rancher-security/rancher-v2.5-hardening-guides/self-assessment-guide-with-cis-v1.6-benchmark.md
+3319
View File
File diff suppressed because it is too large Load Diff
versioned_docs/version-2.5/reference-guides/rancher-security/security-advisories-and-cves.md
+29
View File
@@ -0,0 +1,29 @@
---
title: Security Advisories and CVEs
weight: 300
---
Rancher is committed to informing the community of security issues in our products. Rancher will publish security advisories and CVEs (Common Vulnerabilities and Exposures) for issues we have resolved. New security advisories are also published in Rancher's GitHub [security page](https://github.com/rancher/rancher/security/advisories).
| ID | Description | Date | Resolution |
|----|-------------|------|------------|
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rbac/) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Fleet](https://rancher.com/docs/rancher/v2.6/en/deploy-across-clusters/fleet/) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-36778](https://github.com/rancher/rancher/security/advisories/GHSA-4fc7-hc63-7fjg) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.11 and from 2.6.0 up to and including 2.6.2, where an insufficient check of the same-origin policy when downloading Helm charts from a configured private repository can lead to exposure of the repository credentials to a third-party provider. This issue only happens when the user configures access credentials to a private repository in Rancher inside `Apps & Marketplace > Repositories`. The issue was found and reported by Martin Andreas Ullrich. | 14 Apr 2022 | [Rancher v2.6.3](https://github.com/rancher/rancher/releases/tag/v2.6.3) and [Rancher v2.5.12](https://github.com/rancher/rancher/releases/tag/v2.5.12) |
| [GHSA-hwm2-4ph6-w6m5](https://github.com/rancher/rancher/security/advisories/GHSA-hwm2-4ph6-w6m5) | A vulnerability was discovered in versions of Rancher starting 2.0 up to and including 2.6.3. The `restricted` pod security policy (PSP) provided in Rancher deviated from the upstream `restricted` policy provided in Kubernetes on account of which Rancher's PSP had `runAsUser` set to `runAsAny`, while upstream had `runAsUser` set to `MustRunAsNonRoot`. This allowed containers to run as any user, including a privileged user (`root`), even when Rancher's `restricted` policy was enforced on a project or at the cluster level. | 31 Mar 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) |
| [CVE-2021-36775](https://github.com/rancher/rancher/security/advisories/GHSA-28g7-896h-695v) | A vulnerability was discovered in Rancher versions up to and including 2.4.17, 2.5.11 and 2.6.2. After removing a `Project Role` associated with a group from the project, the bindings that granted access to cluster-scoped resources for those subjects were not deleted. This was due to an incomplete authorization logic check. A user who was a member of the affected group with authenticated access to Rancher could exploit this vulnerability to access resources they shouldn't have had access to. The exposure level would depend on the original permission level granted to the affected project role. This vulnerability only affected customers using group based authentication in Rancher. | 31 Mar 2022 | [Rancher v2.6.3](https://github.com/rancher/rancher/releases/tag/v2.6.3), [Rancher v2.5.12](https://github.com/rancher/rancher/releases/tag/v2.5.12) and [Rancher v2.4.18](https://github.com/rancher/rancher/releases/tag/v2.4.18) |
| [CVE-2021-36776](https://github.com/rancher/rancher/security/advisories/GHSA-gvh9-xgrq-r8hw) | A vulnerability was discovered in Rancher versions starting 2.5.0 up to and including 2.5.9, that allowed an authenticated user to impersonate any user on a cluster through an API proxy, without requiring knowledge of the impersonated user's credentials. This was due to the API proxy not dropping the impersonation header before sending the request to the Kubernetes API. A malicious user with authenticated access to Rancher could use this to impersonate another user with administrator access in Rancher, thereby gaining administrator level access to the cluster. | 31 Mar 2022 | [Rancher v2.6.0](https://github.com/rancher/rancher/releases/tag/v2.6.0) and [Rancher v2.5.10](https://github.com/rancher/rancher/releases/tag/v2.5.10) |
| [CVE-2021-25318](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25318) | A vulnerability was discovered in Rancher versions 2.0 through the aforementioned fixed versions, where users were granted access to resources regardless of the resource's API group. For example, Rancher should have allowed users access to `apps.catalog.cattle.io`, but instead incorrectly gave access to `apps.*`. Resources affected in the **Downstream clusters** and **Rancher management cluster** can be found [here](https://github.com/rancher/rancher/security/advisories/GHSA-f9xf-jq4j-vqw4). There is not a direct mitigation besides upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
| [CVE-2021-31999](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-31999) | A vulnerability was discovered in Rancher 2.0.0 through the aforementioned patched versions, where a malicious Rancher user could craft an API request directed at the proxy for the Kubernetes API of a managed cluster to gain access to information they do not have access to. This is done by passing the "Impersonate-User" or "Impersonate-Group" header in the Connection header, which is then correctly removed by the proxy. At this point, instead of impersonating the user and their permissions, the request will act as if it was from the Rancher management server and incorrectly return the information. The vulnerability is limited to valid Rancher users with some level of permissions on the cluster. There is not a direct mitigation besides upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
| [CVE-2021-25320](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25320) | A vulnerability was discovered in Rancher 2.2.0 through the aforementioned patched versions, where cloud credentials weren't being properly validated through the Rancher API. Specifically through a proxy designed to communicate with cloud providers. Any Rancher user that was logged-in and aware of a cloud-credential ID that was valid for a given cloud provider, could call that cloud provider's API through the proxy API, and the cloud-credential would be attached. The exploit is limited to valid Rancher users. There is not a direct mitigation outside of upgrading to the patched Rancher versions. | 14 Jul 2021 | [Rancher v2.5.9](https://github.com/rancher/rancher/releases/tag/v2.5.9) and [Rancher v2.4.16](https://github.com/rancher/rancher/releases/tag/v2.4.16) |
| [CVE-2021-25313](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-25313) | A security vulnerability was discovered on all Rancher 2 versions. When accessing the Rancher API with a browser, the URL was not properly escaped, making it vulnerable to an XSS attack. Specially crafted URLs to these API endpoints could include JavaScript which would be embedded in the page and execute in a browser. There is no direct mitigation. Avoid clicking on untrusted links to your Rancher server. | 2 Mar 2021 | [Rancher v2.5.6](https://github.com/rancher/rancher/releases/tag/v2.5.6), [Rancher v2.4.14](https://github.com/rancher/rancher/releases/tag/v2.4.14), and [Rancher v2.3.11](https://github.com/rancher/rancher/releases/tag/v2.3.11) |
| [CVE-2019-14435](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14435) | This vulnerability allows authenticated users to potentially extract otherwise private data out of IPs reachable from system service containers used by Rancher. This can include but not only limited to services such as cloud provider metadata services. Although Rancher allow users to configure whitelisted domains for system service access, this flaw can still be exploited by a carefully crafted HTTP request. The issue was found and reported by Matt Belisle and Alex Stevenson at Workiva. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them administrator access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) |
| [CVE-2019-13209](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-13209) | The vulnerability is known as a [Cross-Site Websocket Hijacking attack](https://www.christian-schneider.net/CrossSiteWebSocketHijacking.html). This attack allows an exploiter to gain access to clusters managed by Rancher with the roles/permissions of a victim. It requires that a victim to be logged into a Rancher server and then access a third-party site hosted by the exploiter. Once that is accomplished, the exploiter is able to execute commands against the Kubernetes API with the permissions and identity of the victim. Reported by Matt Belisle and Alex Stevenson from Workiva. | 15 Jul 2019 | [Rancher v2.2.5](https://github.com/rancher/rancher/releases/tag/v2.2.5), [Rancher v2.1.11](https://github.com/rancher/rancher/releases/tag/v2.1.11) and [Rancher v2.0.16](https://github.com/rancher/rancher/releases/tag/v2.0.16) |
| [CVE-2019-12303](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12303) | Project owners can inject extra fluentd logging configurations that makes it possible to read files or execute arbitrary commands inside the fluentd container. Reported by Tyler Welton from Untamed Theory. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
| [CVE-2019-12274](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12274) | Nodes using the built-in node drivers using a file path option allows the machine to read arbitrary files including sensitive ones from inside the Rancher server container. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) |
| [CVE-2019-11202](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11202) | The default admin, that is shipped with Rancher, will be re-created upon restart of Rancher despite being explicitly deleted. | 16 Apr 2019 | [Rancher v2.2.2](https://github.com/rancher/rancher/releases/tag/v2.2.2), [Rancher v2.1.9](https://github.com/rancher/rancher/releases/tag/v2.1.9) and [Rancher v2.0.14](https://github.com/rancher/rancher/releases/tag/v2.0.14) |
| [CVE-2019-6287](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6287) | Project members continue to get access to namespaces from projects that they were removed from if they were added to more than one project. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) |
| [CVE-2018-20321](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20321) | Any project member with access to the `default` namespace can mount the `netes-default` service account in a pod and then use that pod to execute administrative privileged commands against the Kubernetes cluster. | 29 Jan 2019 | [Rancher v2.1.6](https://github.com/rancher/rancher/releases/tag/v2.1.6) and [Rancher v2.0.11](https://github.com/rancher/rancher/releases/tag/v2.0.11) - Rolling back from these versions or greater have specific [instructions]({{<baseurl>}}/rancher/v2.6/en/installation/install-rancher-on-k8s/rollbacks). |
versioned_docs/version-2.5/reference-guides/rancher-security/selinux-rpm/about-rancher-selinux.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/rancher-security/selinux-rpm/about-rke2-selinux.md
+1
View File
@@ -0,0 +1 @@
<!-- PLACEHOLDER -->
versioned_docs/version-2.5/reference-guides/rke1-template-example-yaml.md
+114
View File
@@ -0,0 +1,114 @@
---
title: Example YAML
weight: 60
aliases:
- /rancher/v2.x/en/admin-settings/rke-templates/example-yaml/
---
Below is an example RKE template configuration file for reference.
The YAML in the RKE template uses the same customization that is used when you create an RKE cluster. However, since the YAML is within the context of a Rancher provisioned RKE cluster, the customization from the RKE docs needs to be nested under the `rancher_kubernetes_engine` directive.
```yaml
#
# Cluster Config
#
docker_root_dir: /var/lib/docker
enable_cluster_alerting: false
# This setting is not enforced. Clusters
# created with this sample template
# would have alerting turned off by default,
# but end users could still turn alerting
# on or off.
enable_cluster_monitoring: true
# This setting is not enforced. Clusters
# created with this sample template
# would have monitoring turned on
# by default, but end users could still
# turn monitoring on or off.
enable_network_policy: false
local_cluster_auth_endpoint:
enabled: true
#
# Rancher Config
#
rancher_kubernetes_engine_config: # Your RKE template config goes here.
addon_job_timeout: 30
authentication:
strategy: x509
ignore_docker_version: true
#
# # Currently only nginx ingress provider is supported.
# # To disable ingress controller, set `provider: none`
# # To enable ingress on specific nodes, use the node_selector, eg:
# provider: nginx
# node_selector:
# app: ingress
#
ingress:
provider: nginx
kubernetes_version: v1.15.3-rancher3-1
monitoring:
provider: metrics-server
#
# If you are using calico on AWS
#
# network:
# plugin: calico
# calico_network_provider:
# cloud_provider: aws
#
# # To specify flannel interface
#
# network:
# plugin: flannel
# flannel_network_provider:
# iface: eth1
#
# # To specify flannel interface for canal plugin
#
# network:
# plugin: canal
# canal_network_provider:
# iface: eth1
#
network:
options:
flannel_backend_type: vxlan
plugin: canal
#
# services:
# kube-api:
# service_cluster_ip_range: 10.43.0.0/16
# kube-controller:
# cluster_cidr: 10.42.0.0/16
# service_cluster_ip_range: 10.43.0.0/16
# kubelet:
# cluster_domain: cluster.local
# cluster_dns_server: 10.43.0.10
#
services:
etcd:
backup_config:
enabled: true
interval_hours: 12
retention: 6
safe_timestamp: false
creation: 12h
extra_args:
election-timeout: 5000
heartbeat-interval: 500
gid: 0
retention: 72h
snapshot: false
uid: 0
kube_api:
always_pull_images: false
pod_security_policy: false
service_node_port_range: 30000-32767
ssh_agent_auth: false
windows_prefered_cluster: false
```
versioned_docs/version-2.5/reference-guides/single-node-rancher-in-docker/advanced-options.md
+117
View File
@@ -0,0 +1,117 @@
---
title: Advanced Options for Docker Installs
weight: 5
aliases:
- /rancher/v2.x/en/installation/other-installation-methods/single-node-docker/advanced/
---
When installing Rancher, there are several [advanced options](installation/options/) that can be enabled:
- [Custom CA Certificate](#custom-ca-certificate)
- [API Audit Log](#api-audit-log)
- [TLS Settings](#tls-settings)
- [Air Gap](#air-gap)
- [Persistent Data](#persistent-data)
- [Running `rancher/rancher` and `rancher/rancher-agent` on the Same Node](#running-rancher-rancher-and-rancher-rancher-agent-on-the-same-node)
### Custom CA Certificate
If you want to configure Rancher to use a CA root certificate to be used when validating services, you would start the Rancher container sharing the directory that contains the CA root certificate.
Use the command example to start a Rancher container with your private CA certificates mounted.
- The volume flag (`-v`) should specify the host directory containing the CA root certificates.
- The environment variable flag (`-e`) in combination with `SSL_CERT_DIR` and directory declares an environment variable that specifies the mounted CA root certificates directory location inside the container.
- Passing environment variables to the Rancher container can be done using `-e KEY=VALUE` or `--env KEY=VALUE`.
- Mounting a host directory inside the container can be done using `-v host-source-directory:container-destination-directory` or `--volume host-source-directory:container-destination-directory`.
The example below is based on having the CA root certificates in the `/host/certs` directory on the host and mounting this directory on `/container/certs` inside the Rancher container.
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /host/certs:/container/certs \
-e SSL_CERT_DIR="/container/certs" \
--privileged \
rancher/rancher:latest
```
### API Audit Log
The API Audit Log records all the user and system transactions made through Rancher server.
The API Audit Log writes to `/var/log/auditlog` inside the rancher container by default. Share that directory as a volume and set your `AUDIT_LEVEL` to enable the log.
See [API Audit Log](installation/api-auditing) for more information and options.
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /var/log/rancher/auditlog:/var/log/auditlog \
-e AUDIT_LEVEL=1 \
--privileged \
rancher/rancher:latest
```
### TLS settings
To set a different TLS configuration, you can use the `CATTLE_TLS_MIN_VERSION` and `CATTLE_TLS_CIPHERS` environment variables. For example, to configure TLS 1.0 as minimum accepted TLS version:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e CATTLE_TLS_MIN_VERSION="1.0" \
--privileged \
rancher/rancher:latest
```
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
See [TLS settings](admin-settings/tls-settings) for more information and options.
### Air Gap
If you are visiting this page to complete an air gap installation, you must prepend your private registry URL to the server tag when running the installation command in the option that you choose. Add `<REGISTRY.DOMAIN.COM:PORT>` with your private registry URL in front of `rancher/rancher:latest`.
**Example:**
<REGISTRY.DOMAIN.COM:PORT>/rancher/rancher:latest
### Persistent Data
Rancher uses etcd as a datastore. When Rancher is installed with Docker, the embedded etcd is being used. The persistent data is at the following path in the container: `/var/lib/rancher`.
You can bind mount a host volume to this location to preserve data on the host it is running on:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /opt/rancher:/var/lib/rancher \
--privileged \
rancher/rancher:latest
```
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
### Running `rancher/rancher` and `rancher/rancher-agent` on the Same Node
In the situation where you want to use a single node to run Rancher and to be able to add the same node to a cluster, you have to adjust the host ports mapped for the `rancher/rancher` container.
If a node is added to a cluster, it deploys the nginx ingress controller which will use port 80 and 443. This will conflict with the default ports we advise to expose for the `rancher/rancher` container.
Please note that this setup is not recommended for production use, but can be convenient for development/demo purposes.
To change the host ports mapping, replace the following part `-p 80:80 -p 443:443` with `-p 8080:80 -p 8443:443`:
```
docker run -d --restart=unless-stopped \
-p 8080:80 -p 8443:443 \
--privileged \
rancher/rancher:latest
```
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
versioned_docs/version-2.5/reference-guides/single-node-rancher-in-docker/http-proxy-configuration.md
+46
View File
@@ -0,0 +1,46 @@
---
title: HTTP Proxy Configuration
weight: 251
aliases:
- /rancher/v2.5/en/installation/proxy-configuration/
- /rancher/v2.5/en/installation/single-node/proxy
- /rancher/v2.x/en/installation/other-installation-methods/single-node-docker/proxy/
---
If you operate Rancher behind a proxy and you want to access services through the proxy (such as retrieving catalogs), you must provide Rancher information about your proxy. As Rancher is written in Go, it uses the common proxy environment variables as shown below.
Make sure `NO_PROXY` contains the network addresses, network address ranges and domains that should be excluded from using the proxy.
| Environment variable | Purpose |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) |
| HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) |
| NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s) |
> **Note** NO_PROXY must be in uppercase to use network range (CIDR) notation.
## Docker Installation
Passing environment variables to the Rancher container can be done using `-e KEY=VALUE` or `--env KEY=VALUE`. Required values for `NO_PROXY` in a [Docker Installation](installation/single-node-install/) are:
- `localhost`
- `127.0.0.1`
- `0.0.0.0`
- `10.0.0.0/8`
- `cattle-system.svc`
- `.svc`
- `.cluster.local`
The example below is based on a proxy server accessible at `http://192.168.0.1:3128`, and excluding usage the proxy when accessing network range `192.168.10.0/24` and every hostname under the domain `example.com`.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e HTTP_PROXY="http://192.168.10.1:3128" \
-e HTTPS_PROXY="http://192.168.10.1:3128" \
-e NO_PROXY="localhost,127.0.0.1,0.0.0.0,10.0.0.0/8,cattle-system.svc,192.168.10.0/24,.svc,.cluster.local,example.com" \
--privileged \
rancher/rancher:latest
```
As of Rancher v2.5, privileged access is [required.](../../pages-for-subheaders/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v2-5)
versioned_docs/version-2.5/reference-guides/system-tools.md
+118
View File
@@ -0,0 +1,118 @@
---
title: System Tools
weight: 22
aliases:
- /rancher/v2.x/en/system-tools/
---
System Tools is a tool to perform operational tasks on [Rancher Launched Kubernetes](../pages-for-subheaders/launch-kubernetes-with-rancher.md) clusters or [installations of Rancher on an RKE cluster.](../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md) The tasks include:
* Collect logging and system metrics from nodes.
* Remove Kubernetes resources created by Rancher.
The following commands are available:
| Command | Description
|---|---
| [logs](#logs) | Collect Kubernetes cluster component logs from nodes.
| [stats](#stats) | Stream system metrics from nodes.
| [remove](#remove) | Remove Kubernetes resources created by Rancher.
# Download System Tools
You can download the latest version of System Tools from the [GitHub releases page](https://github.com/rancher/system-tools/releases/latest). Download the version of `system-tools` for the OS that you are using to interact with the cluster.
Operating System | Filename
-----------------|-----
MacOS | `system-tools_darwin-amd64`
Linux | `system-tools_linux-amd64`
Windows | `system-tools_windows-amd64.exe`
After you download the tools, complete the following actions:
1. Rename the file to `system-tools`.
1. Give the file executable permissions by running the following command:
> **Using Windows?**
The file is already an executable, you can skip this step.
```
chmod +x system-tools
```
# Logs
The logs subcommand will collect log files of core Kubernetes cluster components from nodes in [Rancher-launched Kubernetes clusters](../pages-for-subheaders/launch-kubernetes-with-rancher.md) or nodes on an [RKE Kubernetes cluster that Rancher is installed on.](../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md). See [Troubleshooting]({{<baseurl>}}//rancher/v2.5/en/troubleshooting/) for a list of core Kubernetes cluster components.
System Tools will use the provided kubeconfig file to deploy a DaemonSet, that will copy all the logfiles from the core Kubernetes cluster components and add them to a single tar file (`cluster-logs.tar` by default). If you only want to collect logging from a single node, you can specify the node by using `--node NODENAME` or `-n NODENAME`.
### Usage
```
./system-tools_darwin-amd64 logs --kubeconfig <KUBECONFIG>
```
The following are the options for the logs command:
| Option | Description
| ------------------------------------------------------ | ------------------------------------------------------
| `--kubeconfig <KUBECONFIG_PATH>, -c <KUBECONFIG_PATH>` | The cluster's kubeconfig file.
| `--output <FILENAME>, -o cluster-logs.tar` | Name of the created tarball containing the logs. If no output filename is defined, the options defaults to `cluster-logs.tar`.
| `--node <NODENAME>, -n node1` | Specify the nodes to collect the logs from. If no node is specified, logs from all nodes in the cluster will be collected.
# Stats
The stats subcommand will display system metrics from nodes in [Rancher-launched Kubernetes clusters](../pages-for-subheaders/launch-kubernetes-with-rancher.md) or nodes in an [RKE Kubernetes cluster that Rancher is installed on.](../pages-for-subheaders/install-upgrade-on-a-kubernetes-cluster.md).
System Tools will deploy a DaemonSet, and run a predefined command based on `sar` (System Activity Report) to show system metrics.
### Usage
```
./system-tools_darwin-amd64 stats --kubeconfig <KUBECONFIG>
```
The following are the options for the stats command:
| Option | Description
| ------------------------------------------------------ | ------------------------------
| `--kubeconfig <KUBECONFIG_PATH>, -c <KUBECONFIG_PATH>` | The cluster's kubeconfig file.
| `--node <NODENAME>, -n node1` | Specify the nodes to display the system metrics from. If no node is specified, logs from all nodes in the cluster will be displayed.
| `--stats-command value, -s value` | The command to run to display the system metrics. If no command is defined, the options defaults to `/usr/bin/sar -u -r -F 1 1`.
# Remove
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md) before executing the command.
When you install Rancher on a Kubernetes cluster, it will create Kubernetes resources to run and to store configuration data. If you want to remove Rancher from your cluster, you can use the `remove` subcommand to remove the Kubernetes resources. When you use the `remove` subcommand, the following resources will be removed:
- The Rancher deployment namespace (`cattle-system` by default).
- Any `serviceAccount`, `clusterRoles`, and `clusterRoleBindings` that Rancher applied the `cattle.io/creator:norman` label to. Rancher applies this label to any resource that it creates.
- Labels, annotations, and finalizers.
- Rancher Deployment.
- Machines, clusters, projects, and user custom resource deployments (CRDs).
- All resources create under the `management.cattle.io` API Group.
- All CRDs created by Rancher v2.x.
>**Using 2.0.8 or Earlier?**
>
>These versions of Rancher do not automatically delete the `serviceAccount`, `clusterRole`, and `clusterRoleBindings` resources after the job runs. You'll have to delete them yourself.
### Usage
When you run the command below, all the resources listed [above](#remove) will be removed from the cluster.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd](../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md) before executing the command.
```
./system-tools remove --kubeconfig <KUBECONFIG> --namespace <NAMESPACE>
```
The following are the options for the `remove` command:
| Option | Description
| ---------------------------------------------- | ------------
| `--kubeconfig <KUBECONFIG_PATH>, -c <KUBECONFIG_PATH>` | The cluster's kubeconfig file
| `--namespace <NAMESPACE>, -n cattle-system` | Rancher 2.x deployment namespace (`<NAMESPACE>`). If no namespace is defined, the options defaults to `cattle-system`.
| `--force` | Skips the interactive removal confirmation and removes the Rancher deployment without prompt.
versioned_docs/version-2.5/reference-guides/user-settings/api-keys.md
+58
View File
@@ -0,0 +1,58 @@
---
title: API Keys
weight: 7005
aliases:
- /rancher/v2.5/en/concepts/api-keys/
- /rancher/v2.5/en/tasks/user-settings/api-keys/
- /rancher/v2.x/en/user-settings/api-keys/
---
## API Keys and User Authentication
If you want to access your Rancher clusters, projects, or other objects using external applications, you can do so using the Rancher API. However, before your application can access the API, you must provide the app with a key used to authenticate with Rancher. You can obtain a key using the Rancher UI.
An API key is also required for using Rancher CLI.
API Keys are composed of four components:
- **Endpoint:** This is the IP address and path that other applications use to send requests to the Rancher API.
- **Access Key:** The token's username.
- **Secret Key:** The token's password. For applications that prompt you for two different strings for API authentication, you usually enter the two keys together.
- **Bearer Token:** The token username and password concatenated together. Use this string for applications that prompt you for one authentication string.
## Creating an API Key
1. Select **User Avatar** > **API & Keys** from the **User Settings** menu in the upper-right.
2. Click **Add Key**.
3. **Optional:** Enter a description for the API key and select an expiration period or a scope. We recommend setting an expiration date.
The API key won't be valid after expiration. Shorter expiration periods are more secure.
Expiration period will be bound by `v3/settings/auth-token-max-ttl-minutes`. If it exceeds the max-ttl, API key will be created with max-ttl as the expiration period.
A scope will limit the API key so that it will only work against the Kubernetes API of the specified cluster. If the cluster is configured with an Authorized Cluster Endpoint, you will be able to use a scoped token directly against the cluster's API without proxying through the Rancher server. See [Authorized Cluster Endpoints](../../pages-for-subheaders/rancher-manager-architecture.md#4-authorized-cluster-endpoint) for more information.
4. Click **Create**.
**Step Result:** Your API Key is created. Your API **Endpoint**, **Access Key**, **Secret Key**, and **Bearer Token** are displayed.
Use the **Bearer Token** to authenticate with Rancher CLI.
5. Copy the information displayed to a secure location. This information is only displayed once, so if you lose your key, you'll have to make a new one.
## What's Next?
- Enter your API key information into the application that will send requests to the Rancher API.
- Learn more about the Rancher endpoints and parameters by selecting **View in API** for an object in the Rancher UI.
- API keys are used for API calls and [Rancher CLI](../../pages-for-subheaders/cli-with-rancher.md).
## Deleting API Keys
If you need to revoke an API key, delete it. You should delete API keys:
- That may have been compromised.
- That have expired.
To delete an API, select the stale key and click **Delete**.
versioned_docs/version-2.5/reference-guides/user-settings/manage-cloud-credentials.md
+51
View File
@@ -0,0 +1,51 @@
---
title: Managing Cloud Credentials
weight: 7011
aliases:
- /rancher/v2.x/en/user-settings/cloud-credentials/
---
When you create a cluster [hosted by an infrastructure provider](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md), [node templates](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md#node-templates) are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node.
Node templates can use cloud credentials to access the credential information required to provision nodes in the infrastructure providers. The same cloud credential can be used by multiple node templates. By using a cloud credential, you do not have to re-enter access keys for the same cloud provider. Cloud credentials are stored as Kubernetes secrets.
Cloud credentials are only used by node templates if there are fields marked as `password`. The default `active` node drivers have their account access fields marked as `password`, but there may be some `inactive` node drivers, which are not using them yet. These node drivers will not use cloud credentials.
You can create cloud credentials in two contexts:
- [During creation of a node template](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md#node-templates) for a cluster.
- In the **User Settings**
All cloud credentials are bound to the user profile of who created it. They **cannot** be shared across users.
## Creating a Cloud Credential from User Settings
1. From your user settings, select **User Avatar > Cloud Credentials**.
1. Click **Add Cloud Credential**.
1. Enter a name for the cloud credential.
1. Select a **Cloud Credential Type** from the drop down. The values of this dropdown is based on the `active` [node drivers](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-node-drivers.md) in Rancher.
1. Based on the selected cloud credential type, enter the required values to authenticate with the infrastructure provider.
1. Click **Create**.
**Result:** The cloud credential is created and can immediately be used to [create node templates](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md#node-templates).
## Updating a Cloud Credential
When access credentials are changed or compromised, updating a cloud credential allows you to rotate those credentials while keeping the same node template.
1. From your user settings, select **User Avatar > Cloud Credentials**.
1. Choose the cloud credential you want to edit and click the **&#8942; > Edit**.
1. Update the credential information and click **Save**.
**Result:** The cloud credential is updated with the new access credentials. All existing node templates using this cloud credential will automatically use the updated information whenever [new nodes are added](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md).
## Deleting a Cloud Credential
In order to delete cloud credentials, there must not be any node template associated with it. If you are unable to delete the cloud credential, [delete any node templates](manage-node-templates.md#deleting-a-node-template) that are still associated to that cloud credential.
1. From your user settings, select **User Avatar > Cloud Credentials**.
1. You can either individually delete a cloud credential or bulk delete.
- To individually delete one, choose the cloud credential you want to edit and click the **&#8942; > Delete**.
- To bulk delete cloud credentials, select one or more cloud credentials from the list. Click **Delete**.
1. Confirm that you want to delete these cloud credentials.
versioned_docs/version-2.5/reference-guides/user-settings/manage-node-templates.md
+49
View File
@@ -0,0 +1,49 @@
---
title: Managing Node Templates
weight: 7010
aliases:
- /rancher/v2.x/en/user-settings/node-templates/
---
When you provision a cluster [hosted by an infrastructure provider](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md), [node templates](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md#node-templates) are used to provision the cluster nodes. These templates use Docker Machine configuration options to define an operating system image and settings/parameters for the node. You can create node templates in two contexts:
- While [provisioning a node pool cluster](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md).
- At any time, from your [user settings](#creating-a-node-template-from-user-settings).
When you create a node template, it is bound to your user profile. Node templates cannot be shared among users. You can delete stale node templates that you no longer user from your user settings.
## Creating a Node Template from User Settings
1. From your user settings, select **User Avatar > Node Templates**.
1. Click **Add Template**.
1. Select one of the cloud providers available. Then follow the instructions on screen to configure the template.
**Result:** The template is configured. You can use the template later when you [provision a node pool cluster](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md).
## Updating a Node Template
1. From your user settings, select **User Avatar > Node Templates**.
1. Choose the node template that you want to edit and click the **&#8942; > Edit**.
> **Note:** As of v2.2.0, the default `active` [node drivers](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-node-drivers.md) and any node driver, that has fields marked as `password`, are required to use [cloud credentials](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md#cloud-credentials). If you have upgraded to v2.2.0, existing node templates will continue to work with the previous account access information, but when you edit the node template, you will be required to create a cloud credential and the node template will start using it.
1. Edit the required information and click **Save**.
**Result:** The node template is updated. All node pools using this node template will automatically use the updated information when new nodes are added.
## Cloning Node Templates
When creating new node templates from your user settings, you can clone an existing template and quickly update its settings rather than creating a new one from scratch. Cloning templates saves you the hassle of re-entering access keys for the cloud provider.
1. From your user settings, select **User Avatar > Node Templates**.
1. Find the template you want to clone. Then select **&#8942; > Clone**.
1. Complete the rest of the form.
**Result:** The template is cloned and configured. You can use the template later when you [provision a node pool cluster](../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md).
## Deleting a Node Template
When you no longer use a node template, you can delete it from your user settings.
1. From your user settings, select **User Avatar > Node Templates**.
1. Select one or more template from the list. Then click **Delete**. Confirm the delete when prompted.
versioned_docs/version-2.5/reference-guides/user-settings/user-preferences.md
+71
View File
@@ -0,0 +1,71 @@
---
title: User Preferences
weight: 7012
aliases:
- /rancher/v2.x/en/user-settings/preferences/
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Each user can choose preferences to personalize their Rancher experience. To change preference settings, open the **User Settings** menu and then select **Preferences**.
The preferences available will differ depending on whether the **User Settings** menu was accessed while on the Cluster Manager UI or the Cluster Explorer UI.
<Tabs>
<TabItem value="Cluster Manager">
## Theme
Choose your background color for the Rancher UI. If you choose **Auto**, the background color changes from light to dark at 6 PM, and then changes back at 6 AM.
## My Account
This section displays the **Name** (your display name) and **Username** (your login) used for your session. To change your login's current password, click the **Change Password** button.
## Table Row per Page
On pages that display system objects like clusters or deployments in a table, you can set the number of objects that display on the page before you must paginate. The default setting is `50`.
</TabItem>
<TabItem value="Cluster Explorer">
## Theme
Choose your background color for the Rancher UI. If you choose **Auto**, the background color changes from light to dark at 6 PM, and then changes back at 6 AM.
## Login Landing Page
Choose the default page to display after logging in.
## Date Format
Choose your preferred format to display dates. By default, dates are displayed in the form `Wed, Jun 9 2021`.
## Time Format
Choose your preferred format to display time. By default, the 12-hour format is used.
## Table Row per Page
On pages that display system objects like clusters or deployments in a table, you can set the number of objects that display on the page before you must paginate. The default setting is `50`.
## YAML Editor Key Mapping
Choose the editor used when editing YAML configurations. When Emacs or Vim is chosen, the editor's shortcut commands can also be used.
## Enable Developer Tools & Features
Enables developer tools and features to be used.
## Hide All Type Description Boxes
Hides all description boxes.
## Helm Charts
When deploying applications from the "Apps & Marketplace", choose whether to show only released versions of the Helm chart or to include prerelease versions as well.
</TabItem>
</Tabs>