mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-05-05 12:43:16 +00:00
Add version-2.7 docs
This commit is contained in:
Billy Tat
+224
@@ -0,0 +1,224 @@
|
||||
---
|
||||
title: AKS Cluster Configuration Reference
|
||||
---
|
||||
|
||||
## Changes in Rancher v2.6
|
||||
|
||||
- Support for adding more than one node pool
|
||||
- Support for private clusters
|
||||
- Enabled autoscaling node pools
|
||||
- The AKS permissions are now configured in cloud credentials
|
||||
|
||||
## Role-based Access Control
|
||||
|
||||
When provisioning an AKS cluster in the Rancher UI, RBAC cannot be disabled. If role-based access control is disabled for the cluster in AKS, the cluster cannot be registered or imported into Rancher.
|
||||
|
||||
Rancher can configure member roles for AKS clusters in the same way as any other cluster. For more information, see the section on [role-based access control.](../../../pages-for-subheaders/manage-role-based-access-control-rbac.md)
|
||||
|
||||
## Cloud Credentials
|
||||
|
||||
:::note
|
||||
|
||||
The configuration information in this section assumes you have already set up a service principal for Rancher. For step-by-step instructions for how to set up the service principal, see [this section.](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/aks.md#prerequisites-in-microsoft-azure)
|
||||
|
||||
:::
|
||||
|
||||
### Subscription ID
|
||||
|
||||
To get the subscription ID, click **All Services** in the left navigation bar. Then click **Subscriptions**. Go to the name of the subscription that you want to associate with your Kubernetes cluster and copy the **Subscription ID**.
|
||||
|
||||
### Client ID
|
||||
|
||||
To get the client ID, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal. The client ID is listed on the app registration detail page as **Application (client) ID**.
|
||||
|
||||
### Client Secret
|
||||
|
||||
You can't retrieve the client secret value after it is created, so if you don't already have a client secret value, you will need to create a new client secret.
|
||||
|
||||
To get a new client secret, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal.
|
||||
|
||||
Then click **Certificates & secrets** and click **New client secret**. Click **Add**. Then copy the **Value** of the new client secret.
|
||||
|
||||
### Environment
|
||||
|
||||
Microsoft provides multiple [clouds](https://docs.microsoft.com/en-us/cli/azure/cloud?view=azure-cli-latest) for compliance with regional laws, which are available for your use:
|
||||
|
||||
- AzurePublicCloud
|
||||
- AzureGermanCloud
|
||||
- AzureChinaCloud
|
||||
- AzureUSGovernmentCloud
|
||||
|
||||
## Account Access
|
||||
|
||||
In this section you will need to select an existing Azure cloud credential or create a new one.
|
||||
|
||||
For help configuring your Azure cloud credential, see [this section.](#cloud-credentials)
|
||||
|
||||
## Cluster Location
|
||||
|
||||
Configure the cluster and node location. For more information on availability zones for AKS, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/availability-zones)
|
||||
|
||||
The high availability locations include multiple availability zones.
|
||||
|
||||
## Cluster Options
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The available Kubernetes versions are dynamically fetched from the Azure API.
|
||||
|
||||
### Cluster Resource Group
|
||||
|
||||
A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.
|
||||
|
||||
Use an existing resource group or enter a resource group name and one will be created for you.
|
||||
|
||||
Using a resource group containing an existing AKS cluster will create a new resource group. Azure AKS only allows one AKS cluster per resource group.
|
||||
|
||||
For information on managing resource groups, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal)
|
||||
|
||||
### Linux Admin Username
|
||||
|
||||
The username used to create an SSH connection to the Linux nodes.
|
||||
|
||||
The default username for AKS nodes is `azureuser`.
|
||||
|
||||
### SSH Public Key
|
||||
|
||||
The key used to create an SSH connection to the Linux nodes.
|
||||
|
||||
### Tags
|
||||
|
||||
Cluster tags can be useful if your organization uses tags as a way to organize resources across multiple Azure services. These tags don't apply to resources within the cluster.
|
||||
|
||||
## Networking Options
|
||||
|
||||
### LoadBalancer SKU
|
||||
|
||||
Azure load balancers support both standard and basic SKUs (stock keeping units).
|
||||
|
||||
For a comparison of standard and basic load balancers, see the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/load-balancer/skus#skus) Microsoft recommends the Standard load balancer.
|
||||
|
||||
The Standard load balancer is required if you have selected one or more availability zones, or if you have more than one node pool.
|
||||
|
||||
### Network Policy
|
||||
|
||||
All pods in an AKS cluster can send and receive traffic without limitations, by default. To improve security, you can define rules that control the flow of traffic. The Network Policy feature in Kubernetes lets you define rules for ingress and egress traffic between pods in a cluster.
|
||||
|
||||
Azure provides two ways to implement network policy. You choose a network policy option when you create an AKS cluster. The policy option can't be changed after the cluster is created:
|
||||
|
||||
- Azure's own implementation, called Azure Network Policies. The Azure network policy requires the Azure CNI.
|
||||
- Calico Network Policies, an open-source network and network security solution founded by [Tigera](https://www.tigera.io/).
|
||||
|
||||
You can also choose to have no network policy.
|
||||
|
||||
For more information about the differences between Azure and Calico network policies and their capabilities, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/use-network-policies#differences-between-azure-and-calico-policies-and-their-capabilities)
|
||||
|
||||
### DNS Prefix
|
||||
Enter a unique DNS prefix for your cluster's Kubernetes API server FQDN.
|
||||
|
||||
### Network Plugin
|
||||
There are two network plugins: kubenet and Azure CNI.
|
||||
|
||||
The [kubenet](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#kubenet) Kubernetes plugin is the default configuration for AKS cluster creation. When kubenet is used, each node in the cluster receives a routable IP address. The pods use NAT to communicate with other resources outside the AKS cluster. This approach reduces the number of IP addresses you need to reserve in your network space for pods to use.
|
||||
|
||||
With the Azure CNI (advanced) networking plugin, pods get full virtual network connectivity and can be directly reached via their private IP address from connected networks. This plugin requires more IP address space.
|
||||
|
||||
For more information on the differences between kubenet and Azure CNI, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/concepts-network#compare-network-models)
|
||||
|
||||
### HTTP Application Routing
|
||||
|
||||
When enabled, the HTTP application routing add-on makes it easier to access applications deployed to the AKS cluster. It deploys two components: a [Kubernetes Ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress/) and an [External-DNS](https://github.com/kubernetes-incubator/external-dns) controller.
|
||||
|
||||
For more information, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/http-application-routing)
|
||||
|
||||
### Set Authorized IP Ranges
|
||||
|
||||
You can secure access to the Kubernetes API server using [authorized IP address ranges.](https://docs.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges#overview-of-api-server-authorized-ip-ranges)
|
||||
|
||||
The Kubernetes API server exposes the Kubernetes API. This component provides the interaction for management tools, such as kubectl. AKS provides a single-tenant cluster control plane with a dedicated API server. By default, the API server is assigned a public IP address, and you should control access to it using Kubernetes-based or Azure-based RBAC.
|
||||
|
||||
To secure access to the otherwise publicly accessible AKS control plane and API server, you can enable and use authorized IP ranges. These authorized IP ranges only allow defined IP address ranges to communicate with the API server.
|
||||
|
||||
However, even if you use authorized IP address ranges, you should still use Kubernetes RBAC or Azure RBAC to authorize users and the actions they request.
|
||||
|
||||
### Container Monitoring
|
||||
|
||||
Container monitoring gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring, metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux. Metrics are written to the metrics store and log data is written to the logs store associated with your [Log Analytics](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/log-query-overview) workspace.
|
||||
|
||||
### Log Analytics Workspace Resource Group
|
||||
|
||||
The [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview#resource-groups) containing the Log Analytics Workspace. You must create at least one workspace to use Azure Monitor Logs.
|
||||
|
||||
### Log Analytics Workspace Name
|
||||
|
||||
Data collected by Azure Monitor Logs is stored in one or more [Log Analytics workspaces.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/design-logs-deployment) The workspace defines the geographic location of the data, access rights defining which users can access data, and configuration settings such as the pricing tier and data retention.
|
||||
|
||||
You must create at least one workspace to use Azure Monitor Logs. A single workspace may be suffxicient for all of your monitoring data, or may choose to create multiple workspaces depending on your requirements. For example, you might have one workspace for your production data and another for testing.
|
||||
|
||||
For more information about Azure Monitor Logs, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs)
|
||||
|
||||
### Support Private Kubernetes Service
|
||||
|
||||
Typically, AKS worker nodes do not get public IPs, regardless of whether the cluster is private. In a private cluster, the control plane does not have a public endpoint.
|
||||
|
||||
Rancher can connect to a private AKS cluster in one of two ways.
|
||||
|
||||
The first way to ensure that Rancher is running on the same [NAT](https://docs.microsoft.com/en-us/azure/virtual-network/nat-overview) as the AKS nodes.
|
||||
|
||||
The second way is to run a command to register the cluster with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster’s Kubernetes API. This command is displayed in a pop-up when you provision an AKS cluster with a private API endpoint enabled.
|
||||
|
||||
:::note
|
||||
|
||||
Please be aware that when registering an existing AKS cluster, the cluster might take some time, possibly hours, to appear in the `Cluster To register` dropdown list. This outcome will be based on region.
|
||||
|
||||
:::
|
||||
|
||||
For more information about connecting to an AKS private cluster, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/private-clusters#options-for-connecting-to-the-private-cluster)
|
||||
|
||||
## Node Pools
|
||||
|
||||
### Mode
|
||||
|
||||
The Azure interface allows users to specify whether a Primary Node Pool relies on either `system` (normally used for control planes) or `user` (what is most typically needed for Rancher).
|
||||
|
||||
For Primary Node Pools, you can specify Mode, OS, Count and Size.
|
||||
|
||||
System node pools always require running nodes, so they cannot be scaled below one node. At least one system node pool is required.
|
||||
|
||||
For subsequent node pools, the Rancher UI forces the default of User. User node pools allow you to scale to zero nodes. User node pools don't run any part of the Kubernetes controlplane.
|
||||
|
||||
AKS doesn't expose the nodes that run the Kubernetes controlplane components.
|
||||
|
||||
### Availability Zones
|
||||
|
||||
[Availability zones](https://docs.microsoft.com/en-us/azure/availability-zones/az-overview) are unique physical locations within a region. Each zone is made up of one or more data centers equipped with independent power, cooling, and networking.
|
||||
|
||||
Not all regions have support for availability zones. For a list of Azure regions with availability zones, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/availability-zones/az-region#azure-regions-with-availability-zones)
|
||||
|
||||
### VM Size
|
||||
|
||||
Choose a size for each VM in the node pool. For details about each VM size, see [this page.](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/)
|
||||
|
||||
### OS Disk Type
|
||||
|
||||
The nodes in the node pool can have either managed or ephemeral disks.
|
||||
|
||||
[Ephemeral OS disks](https://docs.microsoft.com/en-us/azure/virtual-machines/ephemeral-os-disks) are created on the local virtual machine storage and not saved to the remote Azure Storage. Ephemeral OS disks work well for stateless workloads, where applications are tolerant of individual VM failures, but are more affected by VM deployment time or reimaging the individual VM instances. With Ephemeral OS disk, you get lower read/write latency to the OS disk and faster VM reimage.
|
||||
|
||||
[Azure managed disks](https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview) are block-level storage volumes that are managed by Azure and used with Azure Virtual Machines. Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability.
|
||||
|
||||
### OS Disk Size
|
||||
|
||||
The size in GB for the disk for each node.
|
||||
|
||||
### Node Count
|
||||
The number of nodes in the node pool. The maximum number of nodes may be limited by your [Azure subscription.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits)
|
||||
|
||||
### Max Pods Per Node
|
||||
The maximum number of pods per node defaults to 110 with a maximum of 250.
|
||||
|
||||
### Enable Auto Scaling
|
||||
|
||||
When auto scaling is enabled, you will need to enter a minimum and maximum node count.
|
||||
|
||||
When Auto Scaling is enabled, you can't manually scale the node pool. The scale is controlled by the AKS autoscaler.
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
---
|
||||
title: EKS Cluster Configuration Reference
|
||||
---
|
||||
|
||||
### Account Access
|
||||
|
||||
Complete each drop-down and field using the information obtained for your IAM policy.
|
||||
|
||||
| Setting | Description |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| Region | From the drop-down choose the geographical region in which to build your cluster. |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.](../../user-settings/manage-cloud-credentials.md) |
|
||||
|
||||
### Service Role
|
||||
|
||||
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
|
||||
|
||||
Service Role | Description
|
||||
-------------|---------------------------
|
||||
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
|
||||
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
|
||||
|
||||
### Secrets Encryption
|
||||
|
||||
Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html)
|
||||
|
||||
### API Server Endpoint Access
|
||||
|
||||
Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
|
||||
|
||||
### Private-only API Endpoints
|
||||
|
||||
If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API.
|
||||
|
||||
There are two ways to avoid this extra manual step:
|
||||
- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster.
|
||||
- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html).
|
||||
|
||||
### Public Access Endpoints
|
||||
|
||||
Optionally limit access to the public endpoint via explicit CIDR blocks.
|
||||
|
||||
If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster.
|
||||
|
||||
One of the following is required to enable private access:
|
||||
- Rancher's IP must be part of an allowed CIDR block
|
||||
- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group
|
||||
|
||||
For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
|
||||
|
||||
### Subnet
|
||||
|
||||
| Option | Description |
|
||||
| ------- | ------------ |
|
||||
| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. |
|
||||
| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). |
|
||||
|
||||
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
|
||||
|
||||
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
|
||||
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
|
||||
|
||||
### Security Group
|
||||
|
||||
Amazon Documentation:
|
||||
|
||||
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
|
||||
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
|
||||
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
|
||||
|
||||
### Logging
|
||||
|
||||
Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters.
|
||||
|
||||
Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation.
|
||||
|
||||
For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
|
||||
|
||||
### Managed Node Groups
|
||||
|
||||
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
|
||||
|
||||
For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)
|
||||
|
||||
#### Bring your own launch template
|
||||
|
||||
A launch template ID and version can be provided in order to easily configure the EC2 instances in a node group. If a launch template is provided, then none of the settings below will be configurable in Rancher. Therefore, using a launch template would require that all the necessary and desired settings from the list below would need to be specified in the launch template. Also note that if a launch template ID and version is provided, then only the template version can be updated. Using a new template ID would require creating a new managed node group.
|
||||
|
||||
| Option | Description | Required/Optional |
|
||||
| ------ | ----------- | ----------------- |
|
||||
| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required |
|
||||
| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional |
|
||||
| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required |
|
||||
| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional |
|
||||
| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional |
|
||||
| Instance Resource Tags | Tag each EC2 instance in the node group | Optional |
|
||||
|
||||
#### Rancher-managed launch templates
|
||||
|
||||
If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated.
|
||||
|
||||
#### Custom AMIs
|
||||
|
||||
If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative.
|
||||
|
||||
If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it.
|
||||
|
||||
:::note
|
||||
|
||||
The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template.
|
||||
|
||||
:::
|
||||
|
||||
#### Spot instances
|
||||
|
||||
Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types.
|
||||
|
||||
:::note
|
||||
|
||||
Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration.
|
||||
|
||||
:::
|
||||
|
||||
#### Node Group Settings
|
||||
|
||||
The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created.
|
||||
|
||||
| Option | Description |
|
||||
| ------- | ------------ |
|
||||
| Node Group Name | The name of the node group. |
|
||||
| Desired ASG Size | The desired number of instances. |
|
||||
| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
|
||||
| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
|
||||
| Labels | Kubernetes labels applied to the nodes in the managed node group. |
|
||||
| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. |
|
||||
|
||||
|
||||
### Configuring the Refresh Interval
|
||||
|
||||
The `eks-refresh-cron` setting is deprecated. It has been migrated to the `eks-refresh` setting, which is an integer representing seconds.
|
||||
|
||||
The default value is 300 seconds.
|
||||
|
||||
The syncing interval can be changed by running `kubectl edit setting eks-refresh`.
|
||||
|
||||
If the `eks-refresh-cron` setting was previously set, the migration will happen automatically.
|
||||
|
||||
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
|
||||
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: Private Clusters
|
||||
---
|
||||
|
||||
In GKE, [private clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept) are clusters whose nodes are isolated from inbound and outbound traffic by assigning them internal IP addresses only. Private clusters in GKE have the option of exposing the control plane endpoint as a publicly accessible address or as a private address. This is different from other Kubernetes providers, which may refer to clusters with private control plane endpoints as "private clusters" but still allow traffic to and from nodes. You may want to create a cluster with private nodes, with or without a public control plane endpoint, depending on your organization's networking and security requirements. A GKE cluster provisioned from Rancher can use isolated nodes by selecting "Private Cluster" in the Cluster Options (under "Show advanced options"). The control plane endpoint can optionally be made private by selecting "Enable Private Endpoint".
|
||||
|
||||
### Private Nodes
|
||||
|
||||
Because the nodes in a private cluster only have internal IP addresses, they will not be able to install the cluster agent and Rancher will not be able to fully manage the cluster. This can be overcome in a few ways.
|
||||
|
||||
#### Cloud NAT
|
||||
|
||||
:::caution
|
||||
|
||||
Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
|
||||
|
||||
:::
|
||||
|
||||
If restricting outgoing internet access is not a concern for your organization, use Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service to allow nodes in the private network to access the internet, enabling them to download the required images from Dockerhub and contact the Rancher management server. This is the simplest solution.
|
||||
|
||||
#### Private registry
|
||||
|
||||
:::caution
|
||||
|
||||
This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
|
||||
|
||||
:::
|
||||
|
||||
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.6/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
|
||||
|
||||
### Private Control Plane Endpoint
|
||||
|
||||
If the cluster has a public endpoint exposed, Rancher will be able to reach the cluster, and no additional steps need to be taken. However, if the cluster has no public endpoint, then considerations must be made to ensure Rancher can access the cluster.
|
||||
|
||||
#### Cloud NAT
|
||||
|
||||
:::caution
|
||||
|
||||
Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
|
||||
|
||||
:::
|
||||
|
||||
As above, if restricting outgoing internet access to the nodes is not a concern, then Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service can be used to allow the nodes to access the internet. While the cluster is provisioning, Rancher will provide a registration command to run on the cluster. Download the [kubeconfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) for the new cluster and run the provided kubectl command on the cluster. Gaining access
|
||||
to the cluster in order to run this command can be done by creating a temporary node or using an existing node in the VPC, or by logging on to or creating an SSH tunnel through one of the cluster nodes.
|
||||
|
||||
#### Direct access
|
||||
|
||||
If the Rancher server is run on the same VPC as the cluster's control plane, it will have direct access to the control plane's private endpoint. The cluster nodes will need to have access to a [private registry](#private-registry) to download images as described above.
|
||||
|
||||
You can also use services from Google such as [Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or [Cloud Interconnect VLAN](https://cloud.google.com/network-connectivity/docs/interconnect) to facilitate connectivity between your organization's network and your Google VPC.
|
||||
+145
@@ -0,0 +1,145 @@
|
||||
---
|
||||
title: K3s Cluster Configuration Reference
|
||||
---
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing K3s Kubernetes cluster.
|
||||
|
||||
## Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#cluster-config-file): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create a K3s config file. Using a config file allows you to set any of the [options](https://rancher.com/docs/k3s/latest/en/installation/install-options/) available in an K3s installation.
|
||||
|
||||
## Configuration Options in the Rancher UI
|
||||
|
||||
:::tip
|
||||
|
||||
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the K3s cluster configuration file in YAML. For the complete reference of configurable options for K3s clusters in YAML, see the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/install-options/)
|
||||
|
||||
:::
|
||||
|
||||
### Basics
|
||||
#### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
|
||||
|
||||
#### Encrypt Secrets
|
||||
|
||||
Option to enable or disable secrets encryption. When enabled, secrets will be encrypted using a AES-CBC key. If disabled, any previously secrets will not be readable until encryption is enabled again. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/advanced/#secrets-encryption-config-experimental) for details.
|
||||
|
||||
#### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
#### SELinux
|
||||
|
||||
Option to enable or disable [SELinux](https://rancher.com/docs/k3s/latest/en/advanced/#selinux-support) support.
|
||||
|
||||
#### CoreDNS
|
||||
|
||||
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#coredns) for details..
|
||||
|
||||
#### Klipper Service LB
|
||||
|
||||
Option to enable or disable the [Klipper](https://github.com/rancher/klipper-lb) service load balancer. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#service-load-balancer) for details.
|
||||
|
||||
#### Traefik Ingress
|
||||
|
||||
Option to enable or disable the [Traefik](https://traefik.io/) HTTP reverse proxy and load balancer. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#traefik-ingress-controller).
|
||||
|
||||
#### Local Storage
|
||||
|
||||
Option to enable or disable [local storage](https://rancher.com/docs/k3s/latest/en/storage/) on the node(s).
|
||||
|
||||
#### Metrics Server
|
||||
|
||||
Option to enable or disable the [metrics server](https://github.com/kubernetes-incubator/metrics-server). If enabled, ensure port 10250 is opened for inbound TCP traffic.
|
||||
|
||||
### Add-On Config
|
||||
|
||||
Additional Kubernetes manifests, managed as a [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/helm/#automatically-deploying-manifests-and-helm-charts) for details.
|
||||
|
||||
### Agent Environment Vars
|
||||
|
||||
Option to set environment variables for [K3s agents](https://rancher.com/docs/k3s/latest/en/architecture/). The environment variables can be set using key value pairs. Refer to the [K3 documentation](https://rancher.com/docs/k3s/latest/en/installation/install-options/agent-config/) for more details.
|
||||
|
||||
### etcd
|
||||
|
||||
#### Automatic Snapshots
|
||||
|
||||
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/backup-restore/#creating-snapshots).
|
||||
|
||||
#### Metrics
|
||||
|
||||
Option to choose whether to expose etcd metrics to the public or only within the cluster.
|
||||
|
||||
### Networking
|
||||
|
||||
#### Cluster CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16).
|
||||
|
||||
#### Service CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16).
|
||||
|
||||
#### Cluster DNS
|
||||
|
||||
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10).
|
||||
|
||||
#### Cluster Domain
|
||||
|
||||
Select the domain for the cluster. The default is `cluster.local`.
|
||||
|
||||
#### NodePort Service Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
|
||||
|
||||
#### TLS Alternate Names
|
||||
|
||||
Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
|
||||
|
||||
#### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
|
||||
|
||||
### Registries
|
||||
|
||||
Select the image repository to pull Rancher images from. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/installation/private-registry/).
|
||||
|
||||
### Upgrade Strategy
|
||||
|
||||
#### Controle Plane Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Worker Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Drain Nodes (Control Plane)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
#### Drain Nodes (Worker Nodes)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
### Advanced
|
||||
|
||||
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
|
||||
|
||||
## Cluster Config File
|
||||
|
||||
Instead of using the Rancher UI forms to choose Kubernetes options for the cluster, advanced users can create an K3s config file. Using a config file allows you to set any of the [options](https://rancher.com/docs/k3s/latest/en/installation/install-options/) available in an K3s installation.
|
||||
|
||||
To edit an K3s config file directly from the Rancher UI, click **Edit as YAML**.
|
||||
|
||||
|
||||
+346
@@ -0,0 +1,346 @@
|
||||
---
|
||||
title: RKE Cluster Configuration Reference
|
||||
---
|
||||
|
||||
When Rancher installs Kubernetes, it uses [RKE](../../../pages-for-subheaders/launch-kubernetes-with-rancher.md) or [RKE2](https://docs.rke2.io/) as the Kubernetes distribution.
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE Kubernetes cluster.
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#rke-cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
The RKE cluster config options are nested under the `rancher_kubernetes_engine_config` directive. For more information, see the section about the [cluster config file.](#rke-cluster-config-file-reference)
|
||||
|
||||
In [clusters launched by RKE](../../../pages-for-subheaders/launch-kubernetes-with-rancher.md), you can edit any of the remaining options that follow.
|
||||
|
||||
For an example of RKE config file syntax, see the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
|
||||
|
||||
The forms in the Rancher UI don't include all advanced options for configuring RKE. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
## Editing Clusters with a Form in the Rancher UI
|
||||
|
||||
To edit your cluster,
|
||||
|
||||
1. In the upper left corner, click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
|
||||
|
||||
|
||||
## Editing Clusters with YAML
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
RKE clusters (also called RKE1 clusters) are edited differently than RKE2 and K3s clusters.
|
||||
|
||||
To edit an RKE config file directly from the Rancher UI,
|
||||
|
||||
1. Click **☰ > Cluster Management**.
|
||||
1. Go to the RKE cluster you want to configure. Click and click **⋮ > Edit Config**. This take you to the RKE configuration form. Note: Because cluster provisioning changed in Rancher 2.6, the **⋮ > Edit as YAML** can be used for configuring RKE2 clusters, but it can't be used for editing RKE1 configuration.
|
||||
1. In the configuration form, scroll down and click **Edit as YAML**.
|
||||
1. Edit the RKE options under the `rancher_kubernetes_engine_config` directive.
|
||||
|
||||
## Configuration Options in the Rancher UI
|
||||
|
||||
:::tip
|
||||
|
||||
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
:::
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
|
||||
|
||||
### Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses. For more details on the different networking providers, please view our [Networking FAQ](../../../faq/container-network-interface-providers.md).
|
||||
|
||||
:::caution
|
||||
|
||||
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
|
||||
|
||||
:::
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
:::note Notes on Weave:
|
||||
|
||||
When Weave is selected as network provider, Rancher will automatically enable encryption by generating a random password. If you want to specify the password manually, please see how to configure your cluster using a [Config File](#rke-cluster-config-file-reference) and the [Weave Network Plug-in Options](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options).
|
||||
|
||||
:::
|
||||
|
||||
### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### Kubernetes Cloud Providers
|
||||
|
||||
You can configure a [Kubernetes cloud provider](../../../pages-for-subheaders/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../pages-for-subheaders/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
:::note
|
||||
|
||||
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#rke-cluster-config-file-reference) to configure the cloud provider. Please reference the [RKE cloud provider documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
:::
|
||||
|
||||
### Private Registries
|
||||
|
||||
The cluster-level private registry configuration is only used for provisioning clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the [global default registry](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/global-default-private-registry.md) through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
If your private registry requires credentials, you need to pass the credentials to Rancher by editing the cluster options for each cluster that needs to pull images from the registry.
|
||||
|
||||
The private registry configuration option tells Rancher where to pull the [system images](https://rancher.com/docs/rke/latest/en/config-options/system-images/) or [addon images](https://rancher.com/docs/rke/latest/en/config-options/add-ons/) that will be used in your cluster.
|
||||
|
||||
- **System images** are components needed to maintain the Kubernetes cluster.
|
||||
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
|
||||
|
||||
For more information on setting up a private registry for components applied during the provisioning of the cluster, see the [RKE documentation on private registries](https://rancher.com/docs/rke/latest/en/config-options/private-registries/).
|
||||
|
||||
Rancher v2.6 introduced the ability to configure [ECR registries for RKE clusters](https://rancher.com/docs/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup).
|
||||
|
||||
### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
:::note
|
||||
|
||||
The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#tools-for-provisioning-kubernetes-clusters) to provision the cluster. It is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
|
||||
|
||||
:::
|
||||
|
||||
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
|
||||
|
||||
### Node Pools
|
||||
|
||||
For information on using the Rancher UI to set up node pools in an RKE cluster, refer to [this page.](../../../pages-for-subheaders/use-new-nodes-in-an-infra-provider.md)
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster.
|
||||
|
||||
### Metrics Server Monitoring
|
||||
|
||||
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
### Pod Security Policy Support
|
||||
|
||||
Enables [pod security policies](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. After enabling this option, choose a policy using the **Default Pod Security Policy** drop-down.
|
||||
|
||||
You must have an existing Pod Security Policy configured before you can use this option.
|
||||
|
||||
### Docker Version on Nodes
|
||||
|
||||
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support.
|
||||
|
||||
If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
|
||||
|
||||
For details on which Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
|
||||
|
||||
### Docker Root Directory
|
||||
|
||||
If the nodes you are adding to the cluster have Docker configured with a non-default Docker Root Directory (default is `/var/lib/docker`), specify the correct Docker Root Directory in this option.
|
||||
|
||||
### Default Pod Security Policy
|
||||
|
||||
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
|
||||
|
||||
### Node Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). Default is `30000-32767`.
|
||||
|
||||
### Recurring etcd Snapshots
|
||||
|
||||
Option to enable or disable [recurring etcd snapshots](https://rancher.com/docs/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
|
||||
|
||||
### Agent Environment Variables
|
||||
|
||||
Option to set environment variables for [rancher agents](../../../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/about-rancher-agents.md). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
|
||||
### Updating ingress-nginx
|
||||
|
||||
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
|
||||
|
||||
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
|
||||
|
||||
|
||||
|
||||
## RKE Cluster Config File Reference
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available](https://rancher.com/docs/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API.
|
||||
|
||||
For the complete reference for configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/)
|
||||
|
||||
### Config File Structure in Rancher
|
||||
|
||||
RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kubernetes clusters. Rancher's cluster config files used to have the same structure as [RKE config files,](https://rancher.com/docs/rke/latest/en/example-yamls/) but the structure changed so that in Rancher, RKE cluster config items are separated from non-RKE config items. Therefore, configuration for your cluster needs to be nested under the `rancher_kubernetes_engine_config` directive in the cluster config file. Cluster config files created with earlier versions of Rancher will need to be updated for this format. An example cluster config file is included below.
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>Example Cluster Config File</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # Currently only nginx ingress provider is supported.
|
||||
# # To disable ingress controller, set `provider: none`
|
||||
# # To enable ingress on specific nodes, use the node_selector, eg:
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### Default DNS provider
|
||||
|
||||
The table below indicates what DNS provider is deployed by default. See [RKE documentation on DNS provider](https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/) for more information how to configure a different DNS provider. CoreDNS can only be used on Kubernetes v1.12.0 and higher.
|
||||
|
||||
| Rancher version | Kubernetes version | Default DNS provider |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 and higher | v1.14.0 and higher | CoreDNS |
|
||||
| v2.2.5 and higher | v1.13.x and lower | kube-dns |
|
||||
| v2.2.4 and lower | any | kube-dns |
|
||||
|
||||
## Rancher Specific Parameters in YAML
|
||||
|
||||
Besides the RKE config file options, there are also Rancher specific settings that can be configured in the Config File (YAML):
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
See [Docker Root Directory](#docker-root-directory).
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
Option to enable or disable [Cluster Monitoring](../../../pages-for-subheaders/monitoring-and-alerting.md).
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
Option to enable or disable Project Network Isolation.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
See [Authorized Cluster Endpoint](#authorized-cluster-endpoint).
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### Custom Network Plug-in
|
||||
|
||||
You can add a custom network plug-in by using the [user-defined add-on functionality](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. You define any add-on that you want deployed after the Kubernetes cluster is deployed.
|
||||
|
||||
There are two ways that you can specify an add-on:
|
||||
|
||||
- [In-line Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [Referencing YAML Files for Add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
+352
@@ -0,0 +1,352 @@
|
||||
---
|
||||
title: RKE2 Cluster Configuration Reference
|
||||
---
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE2 Kubernetes cluster.
|
||||
|
||||
## Overview
|
||||
|
||||
You can configure the Kubernetes options in one of the two following ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#cluster-config-file-reference): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE2 config file. Using a config file allows you to set many additional [options](https://docs.rke2.io/install/configuration) available for an RKE2 installation.
|
||||
|
||||
## Editing Clusters with a Form in the Rancher UI
|
||||
|
||||
To edit your cluster,
|
||||
|
||||
1. In the upper left corner, click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
|
||||
|
||||
## Editing Clusters with YAML
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE2 config file. Using a config file allows you to set any of the options available in an RKE2 installation by specifying them in YAML.
|
||||
|
||||
To edit an RKE2 config file directly from the Rancher UI,
|
||||
|
||||
1. Click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit as YAML**.
|
||||
1. Edit the RKE options under the `rkeConfig` directive.
|
||||
|
||||
## Configuration Options in the Rancher UI
|
||||
|
||||
:::tip
|
||||
|
||||
Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE2 cluster configuration file in YAML. For the complete reference of configurable options for RKE2 Kubernetes clusters in YAML, see the [RKE2 documentation.](https://docs.rke2.io/install/configuration)
|
||||
|
||||
:::
|
||||
|
||||
## Machine Pool
|
||||
|
||||
This subsection covers the generic machine pool configurations. For infrastructure provider specific, configurations refer to the following pages:
|
||||
|
||||
- [Azure](../downstream-cluster-configuration/machine-configuration/azure.md)
|
||||
- [DigitalOcean](../downstream-cluster-configuration/machine-configuration/digitalocean.md)
|
||||
- [EC2](../downstream-cluster-configuration/machine-configuration/amazon-ec2.md)
|
||||
|
||||
### Pool Name
|
||||
|
||||
The name of the machine pool.
|
||||
|
||||
### Machine Count
|
||||
|
||||
The number of machines in the pool.
|
||||
|
||||
### Roles
|
||||
|
||||
Option to assign etcd, control plane, and worker roles to nodes.
|
||||
|
||||
### Advanced
|
||||
|
||||
#### Auto Replace
|
||||
|
||||
The duration nodes can be unreachable before they are automatically deleted and replaced.
|
||||
|
||||
#### Drain Before Delete
|
||||
|
||||
Enables draining nodes by evicting all pods before the node is deleted.
|
||||
|
||||
#### Kubernetes Node Labels
|
||||
|
||||
Add [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to nodes to help with organization and object selection.
|
||||
|
||||
For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)
|
||||
|
||||
#### Taints
|
||||
|
||||
Add [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to nodes, which can be used to prevent pods from being scheduled to or executed on nodes, unless the pods have matching tolerations.
|
||||
|
||||
## Cluster Configuration
|
||||
|
||||
### Basics
|
||||
#### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes](../../../getting-started/installation-and-upgrade/upgrade-and-roll-back-kubernetes.md).
|
||||
|
||||
#### Container Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses.
|
||||
|
||||
:::caution
|
||||
|
||||
After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you to tear down the entire cluster and all its applications.
|
||||
|
||||
:::
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Cilium](https://cilium.io/)*
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Multus](https://github.com/k8snetworkplumbingwg/multus-cni)
|
||||
|
||||
\* When using [project network isolation](#project-network-isolation) in the [Cilium CNI](../../../faq/container-network-interface-providers.md#cilium), it is possible to enable cross-node ingress routing. Click the [CNI provider docs](../../../faq/container-network-interface-providers.md#ingress-routing-across-nodes-in-cilium) to learn more.
|
||||
|
||||
For more details on the different networking providers and how to configure them, please view our [RKE2 documentation](https://docs.rke2.io/install/network_options).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
[Dual-stack](https://docs.rke2.io/install/network_options#dual-stack-configuration) networking is supported for all CNI providers. To configure RKE2 in dual-stack mode, set valid IPv4/IPv6 CIDRs for your [Cluster CIDR](#cluster-cidr) and/or [Service CIDR](#service-cidr).
|
||||
|
||||
###### Additional Configuration {#dual-stack-additional-config}
|
||||
|
||||
When using `cilium` or `multus,cilium` as your container network interface provider, ensure the **Enable IPv6 Support** option is also enabled.
|
||||
|
||||
#### Cloud Provider
|
||||
|
||||
You can configure a [Kubernetes cloud provider](../../../pages-for-subheaders/set-up-cloud-providers.md). If you want to use dynamically provisioned [volumes and storage](../../../pages-for-subheaders/create-kubernetes-persistent-storage.md) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
:::note
|
||||
|
||||
If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file-reference) to configure the cloud provider. Please reference [this documentation](https://rancher.com/docs/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
:::
|
||||
|
||||
#### Default Pod Security Policy
|
||||
|
||||
Choose the default [pod security policy](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/create-pod-security-policies.md) for the cluster. Please refer to the [RKE2 documentation](https://docs.rke2.io/security/pod_security_policies) on the specifications of each available policy.
|
||||
|
||||
#### Worker CIS Profile
|
||||
|
||||
Select a [CIS benchmark](../../../pages-for-subheaders/cis-scan-guides.md) to validate the system configuration against.
|
||||
|
||||
#### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE2 network plugin that supports the enforcement of Kubernetes network policies, such as Canal.
|
||||
|
||||
#### CoreDNS
|
||||
|
||||
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#coredns) for additional CoreDNS configurations.
|
||||
|
||||
#### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
|
||||
|
||||
Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
|
||||
|
||||
#### Metrics Server
|
||||
|
||||
Option to enable or disable [Metrics Server](https://rancher.com/docs/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE2 can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
### Add-On Config
|
||||
|
||||
Additional Kubernetes manifests, managed as an [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [RKE2 documentation](https://docs.rke2.io/helm#automatically-deploying-manifests-and-helm-charts) for details.
|
||||
|
||||
### Agent Environment Vars
|
||||
|
||||
Option to set environment variables for [Rancher agents](https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. Refer to the [RKE2 documentation](https://docs.rke2.io/reference/linux_agent_config) for more details.
|
||||
|
||||
### etcd
|
||||
|
||||
#### Automatic Snapshots
|
||||
|
||||
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [RKE2 documentation](https://docs.rke2.io/backup_restore#creating-snapshots). Note that with RKE2, snapshots are stored on each etcd node. This varies from RKE1 which only stores one snapshot per cluster.
|
||||
|
||||
#### Metrics
|
||||
|
||||
Option to choose whether to expose etcd metrics to the public or only within the cluster.
|
||||
|
||||
### Networking
|
||||
|
||||
#### Cluster CIDR
|
||||
|
||||
IPv4 and/or IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
To configure [dual-stack](https://docs.rke2.io/install/network_options#dual-stack-configuration) mode, enter a valid IPv4/IPv6 CIDR. For example `10.42.0.0/16,2001:cafe:42:0::/56`.
|
||||
|
||||
[Additional configuration](#dual-stack-additional-config) is required when using `cilium` or `multus,cilium` as your [container network](#container-network-provider) interface provider.
|
||||
|
||||
#### Service CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
To configure [dual-stack](https://docs.rke2.io/install/network_options#dual-stack-configuration) mode, enter a valid IPv4/IPv6 CIDR. For example `10.42.0.0/16,2001:cafe:42:0::/56`.
|
||||
|
||||
[Additional configuration](#dual-stack-additional-config) is required when using `cilium ` or `multus,cilium` as your [container network](#container-network-provider) interface provider.
|
||||
|
||||
#### Cluster DNS
|
||||
|
||||
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10).
|
||||
|
||||
#### Cluster Domain
|
||||
|
||||
Select the domain for the cluster. The default is `cluster.local`.
|
||||
|
||||
#### NodePort Service Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
|
||||
|
||||
#### TLS Alternate Names
|
||||
|
||||
Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
|
||||
|
||||
#### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.](../../../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.](../../rancher-manager-architecture/architecture-recommendations.md#architecture-for-an-authorized-cluster-endpoint-ace)
|
||||
|
||||
### Registries
|
||||
|
||||
Select the image repository to pull Rancher images from. For more details and configuration options, see the [RKE2 documentation](https://docs.rke2.io/install/containerd_registry_configuration).
|
||||
|
||||
### Upgrade Strategy
|
||||
|
||||
#### Control Plane Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Worker Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Drain Nodes (Control Plane)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
#### Drain Nodes (Worker Nodes)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
### Advanced
|
||||
|
||||
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
|
||||
|
||||
## Cluster Config File Reference
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create a config file. Using a config file allows you to set the [options available](https://docs.rke2.io/install/configuration) in an RKE2 installation, including those already listed in [Configuration Options in the Rancher UI](#configuration-options-in-the-rancher-ui), as well as Rancher-specific parameters.
|
||||
|
||||
<details>
|
||||
<summary>
|
||||
<b>Example Cluster Config File Snippet</b>
|
||||
</summary>
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
cloudCredentialSecretName: cattle-global-data:cc-s879v
|
||||
kubernetesVersion: v1.23.6+rke2r2
|
||||
localClusterAuthEndpoint: {}
|
||||
rkeConfig:
|
||||
chartValues:
|
||||
rke2-calico: {}
|
||||
etcd:
|
||||
snapshotRetention: 5
|
||||
snapshotScheduleCron: 0 */5 * * *
|
||||
machineGlobalConfig:
|
||||
cni: calico
|
||||
disable-kube-proxy: false
|
||||
etcd-expose-metrics: false
|
||||
profile: null
|
||||
machinePools:
|
||||
- controlPlaneRole: true
|
||||
etcdRole: true
|
||||
machineConfigRef:
|
||||
kind: Amazonec2Config
|
||||
name: nc-test-pool1-pwl5h
|
||||
name: pool1
|
||||
quantity: 1
|
||||
unhealthyNodeTimeout: 0s
|
||||
workerRole: true
|
||||
machineSelectorConfig:
|
||||
- config:
|
||||
protect-kernel-defaults: false
|
||||
registries: {}
|
||||
upgradeStrategy:
|
||||
controlPlaneConcurrency: "1"
|
||||
controlPlaneDrainOptions:
|
||||
deleteEmptyDirData: true
|
||||
enabled: true
|
||||
gracePeriod: -1
|
||||
ignoreDaemonSets: true
|
||||
timeout: 120
|
||||
workerConcurrency: "1"
|
||||
workerDrainOptions:
|
||||
deleteEmptyDirData: true
|
||||
enabled: true
|
||||
gracePeriod: -1
|
||||
ignoreDaemonSets: true
|
||||
timeout: 120
|
||||
```
|
||||
</details>
|
||||
|
||||
### chartValues
|
||||
|
||||
Option to specify the values for the system charts installed by RKE2/k3s.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
chartValues:
|
||||
chart-name:
|
||||
key: value
|
||||
```
|
||||
### machineGlobalConfig
|
||||
|
||||
The RKE2/K3s configurations are nested under the `machineGlobalConfig` directive. Any configuration change made here will apply to every node. The configuration options available in the [standalone version of RKE2](https://docs.rke2.io/reference/server_config) can be applied here.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
machineGlobalConfig:
|
||||
etcd-arg:
|
||||
- key1=value1
|
||||
- key2=value2
|
||||
```
|
||||
|
||||
### machineSelectorConfig
|
||||
|
||||
This is the same as [`machineGlobalConfig`](#machineglobalconfig) except that a [label](#kubernetes-node-labels) selector can be specified with the configuration. The configuration will only be applied to nodes that match the provided label selector.
|
||||
|
||||
Multiple `config` entries are allowed, each specifying their own `machineLabelSelector`. A user can specify `matchExpressions`, `matchLabels`, both, or neither. Omitting the `machineLabelSelector` section of this has the same effect as putting the config in the `machineGlobalConfig` section.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
machineSelectorConfig
|
||||
- config:
|
||||
config-key: config-value
|
||||
machineLabelSelector:
|
||||
matchExpressions:
|
||||
- key: example-key
|
||||
operator: string # Valid operators are In, NotIn, Exists and DoesNotExist.
|
||||
values:
|
||||
- example-value1
|
||||
- example-value2
|
||||
matchLabels:
|
||||
key1: value1
|
||||
key2: value2
|
||||
```
|
||||
+41
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: Syncing
|
||||
---
|
||||
|
||||
Syncing allows Rancher to update cluster values so that they're up to date with the corresponding cluster object hosted in AKS, EKS or GKE. This enables sources other than Rancher to own a hosted cluster’s state.
|
||||
|
||||
:::warning
|
||||
You may accidentally overwrite the state from one source if you simultaneously process an update from another source. This may also occur if you process an update from one source within 5 minutes of finishing an update from another source.
|
||||
:::
|
||||
|
||||
### How it works
|
||||
|
||||
There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
|
||||
|
||||
1. The config object for the cluster, located on the Spec of the Cluster:
|
||||
|
||||
* For AKS, the field is called AKSConfig
|
||||
* For EKS, the field is called EKSConfig
|
||||
* For GKE, the field is called GKEConfig
|
||||
|
||||
2. The UpstreamSpec object
|
||||
|
||||
* For AKS, this is located on the AKSStatus field on the Status of the Cluster.
|
||||
* For EKS, this is located on the EKSStatus field on the Status of the Cluster.
|
||||
* For GKE, this is located on the GKEStatus field on the Status of the Cluster.
|
||||
|
||||
The struct types that define these objects can be found in their corresponding operator projects:
|
||||
|
||||
* [aks-operator](https://github.com/rancher/aks-operator/blob/master/pkg/apis/aks.cattle.io/v1/types.go)
|
||||
* [eks-operator](https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go)
|
||||
* [gke-operator](https://github.com/rancher/gke-operator/blob/master/pkg/apis/gke.cattle.io/v1/types.go)
|
||||
|
||||
All fields are nillable, except for the following: the cluster name, the location (region or zone), Imported, and the cloud credential reference.
|
||||
|
||||
The AKSConfig, EKSConfig or GKEConfig represents the desired state. Nil values are ignored. Fields that are non-nil in the config object can be thought of as managed. When a cluster is created in Rancher, all fields are non-nil and therefore managed. When a pre-existing cluster is registered in Rancher all nillable fields are set to nil and aren't managed. Those fields become managed once their value has been changed by Rancher.
|
||||
|
||||
UpstreamSpec represents the cluster as it is in the hosted Kubernetes provider. It's refreshed every 5 minutes. After the UpstreamSpec is refreshed, Rancher checks if the cluster has an update in progress. If it's currently updating, nothing further is done. If it is not currently updating, any managed fields on AKSConfig, EKSConfig or GKEConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
|
||||
|
||||
The effective desired state can be thought of as the UpstreamSpec, plus all non-nil fields in the AKSConfig, EKSConfig or GKEConfig. This is what is displayed in the UI.
|
||||
|
||||
If Rancher and another source attempt to update a cluster at the same time, or within 5 minutes of an update finishing, any managed fields are likely to get caught in a race condition. To use EKS as an example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console at 11:01, and tags are updated from Rancher before 11:05, then the value is likely to be overwritten. This can also occur if tags are updated while the cluster is still processing the update. The issue described in this example shouldn't occur if the cluster is registered and the PrivateAccess fields are nil.
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: Rancher Agent Options
|
||||
---
|
||||
|
||||
Rancher deploys an agent on each node to communicate with the node. This pages describes the options that can be passed to the agent. To use these options, you will need to [create a cluster with custom nodes](../../../../pages-for-subheaders/use-existing-nodes.md) and add the options to the generated `docker run` command when adding a node.
|
||||
|
||||
For an overview of how Rancher communicates with downstream clusters using node agents, refer to the [architecture section.](../../../rancher-manager-architecture/communicating-with-downstream-user-clusters.md#3-node-agents)
|
||||
|
||||
## General options
|
||||
|
||||
| Parameter | Environment variable | Description |
|
||||
| ---------- | -------------------- | ----------- |
|
||||
| `--server` | `CATTLE_SERVER` | The configured Rancher `server-url` setting which the agent connects to |
|
||||
| `--token` | `CATTLE_TOKEN` | Token that is needed to register the node in Rancher |
|
||||
| `--ca-checksum` | `CATTLE_CA_CHECKSUM` | The SHA256 checksum of the configured Rancher `cacerts` setting to validate |
|
||||
| `--node-name` | `CATTLE_NODE_NAME` | Override the hostname that is used to register the node (defaults to `hostname -s`) |
|
||||
| `--label` | `CATTLE_NODE_LABEL` | Add node labels to the node. For multiple labels, pass additional `--label` options. (`--label key=value`) |
|
||||
| `--taints` | `CATTLE_NODE_TAINTS` | Add node taints to the node. For multiple taints, pass additional `--taints` options. (`--taints key=value:effect`) |
|
||||
|
||||
## Role options
|
||||
|
||||
| Parameter | Environment variable | Description |
|
||||
| ---------- | -------------------- | ----------- |
|
||||
| `--all-roles` | `ALL=true` | Apply all roles (`etcd`,`controlplane`,`worker`) to the node |
|
||||
| `--etcd` | `ETCD=true` | Apply the role `etcd` to the node |
|
||||
| `--controlplane` | `CONTROL=true` | Apply the role `controlplane` to the node |
|
||||
| `--worker` | `WORKER=true` | Apply the role `worker` to the node |
|
||||
|
||||
## IP address options
|
||||
|
||||
| Parameter | Environment variable | Description |
|
||||
| ---------- | -------------------- | ----------- |
|
||||
| `--address` | `CATTLE_ADDRESS` | The IP address the node will be registered with (defaults to the IP used to reach `8.8.8.8`) |
|
||||
| `--internal-address` | `CATTLE_INTERNAL_ADDRESS` | The IP address used for inter-host communication on a private network |
|
||||
|
||||
### Dynamic IP address options
|
||||
|
||||
For automation purposes, you can't have a specific IP address in a command as it has to be generic to be used for every node. For this, we have dynamic IP address options. They are used as a value to the existing IP address options. This is supported for `--address` and `--internal-address`.
|
||||
|
||||
| Value | Example | Description |
|
||||
| ---------- | -------------------- | ----------- |
|
||||
| Interface name | `--address eth0` | The first configured IP address will be retrieved from the given interface |
|
||||
| `ipify` | `--address ipify` | Value retrieved from `https://api.ipify.org` will be used |
|
||||
| `awslocal` | `--address awslocal` | Value retrieved from `http://169.254.169.254/latest/meta-data/local-ipv4` will be used |
|
||||
| `awspublic` | `--address awspublic` | Value retrieved from `http://169.254.169.254/latest/meta-data/public-ipv4` will be used |
|
||||
| `doprivate` | `--address doprivate` | Value retrieved from `http://169.254.169.254/metadata/v1/interfaces/private/0/ipv4/address` will be used |
|
||||
| `dopublic` | `--address dopublic` | Value retrieved from `http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address` will be used |
|
||||
| `azprivate` | `--address azprivate` | Value retrieved from `http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2017-08-01&format=text` will be used |
|
||||
| `azpublic` | `--address azpublic` | Value retrieved from `http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text` will be used |
|
||||
| `gceinternal` | `--address gceinternal` | Value retrieved from `http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip` will be used |
|
||||
| `gceexternal` | `--address gceexternal` | Value retrieved from `http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip` will be used |
|
||||
| `packetlocal` | `--address packetlocal` | Value retrieved from `https://metadata.packet.net/2009-04-04/meta-data/local-ipv4` will be used |
|
||||
| `packetpublic` | `--address packetlocal` | Value retrieved from `https://metadata.packet.net/2009-04-04/meta-data/public-ipv4` will be used |
|
||||
Reference in New Issue
Block a user