mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-26 04:58:07 +00:00
Remove unneeded intermediate folders
This commit is contained in:
Billy Tat
+218
@@ -0,0 +1,218 @@
|
||||
---
|
||||
shortTitle: AKS Cluster Configuration
|
||||
title: AKS Cluster Configuration Reference
|
||||
weight: 4
|
||||
---
|
||||
|
||||
# Changes in Rancher v2.6
|
||||
|
||||
- Support for adding more than one node pool
|
||||
- Support for private clusters
|
||||
- Enabled autoscaling node pools
|
||||
- The AKS permissions are now configured in cloud credentials
|
||||
|
||||
# Role-based Access Control
|
||||
|
||||
When provisioning an AKS cluster in the Rancher UI, RBAC cannot be disabled. If role-based access control is disabled for the cluster in AKS, the cluster cannot be registered or imported into Rancher.
|
||||
|
||||
Rancher can configure member roles for AKS clusters in the same way as any other cluster. For more information, see the section on [role-based access control.]({{<baseurl>}}/rancher/v2.6/en/admin-settings/rbac)
|
||||
|
||||
# Cloud Credentials
|
||||
|
||||
> The configuration information in this section assumes you have already set up a service principal for Rancher. For step-by-step instructions for how to set up the service principal, see [this section.]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/hosted-kubernetes-clusters/aks/#prerequisites-in-microsoft-azure)
|
||||
|
||||
### Subscription ID
|
||||
|
||||
To get the subscription ID, click **All Services** in the left navigation bar. Then click **Subscriptions**. Go to the name of the subscription that you want to associate with your Kubernetes cluster and copy the **Subscription ID**.
|
||||
|
||||
### Client ID
|
||||
|
||||
To get the client ID, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal. The client ID is listed on the app registration detail page as **Application (client) ID**.
|
||||
|
||||
### Client Secret
|
||||
|
||||
You can't retrieve the client secret value after it is created, so if you don't already have a client secret value, you will need to create a new client secret.
|
||||
|
||||
To get a new client secret, go to the Azure Portal, then click **Azure Active Directory**, then click **App registrations,** then click the name of the service principal.
|
||||
|
||||
Then click **Certificates & secrets** and click **New client secret**. Click **Add**. Then copy the **Value** of the new client secret.
|
||||
|
||||
### Environment
|
||||
|
||||
Microsoft provides multiple [clouds](https://docs.microsoft.com/en-us/cli/azure/cloud?view=azure-cli-latest) for compliance with regional laws, which are available for your use:
|
||||
|
||||
- AzurePublicCloud
|
||||
- AzureGermanCloud
|
||||
- AzureChinaCloud
|
||||
- AzureUSGovernmentCloud
|
||||
|
||||
# Account Access
|
||||
|
||||
In this section you will need to select an existing Azure cloud credential or create a new one.
|
||||
|
||||
For help configuring your Azure cloud credential, see [this section.](#cloud-credentials)
|
||||
|
||||
# Cluster Location
|
||||
|
||||
Configure the cluster and node location. For more information on availability zones for AKS, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/availability-zones)
|
||||
|
||||
The high availability locations include multiple availability zones.
|
||||
|
||||
# Cluster Options
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The available Kubernetes versions are dynamically fetched from the Azure API.
|
||||
|
||||
### Cluster Resource Group
|
||||
|
||||
A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.
|
||||
|
||||
Use an existing resource group or enter a resource group name and one will be created for you.
|
||||
|
||||
Using a resource group containing an existing AKS cluster will create a new resource group. Azure AKS only allows one AKS cluster per resource group.
|
||||
|
||||
For information on managing resource groups, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal)
|
||||
|
||||
### Linux Admin Username
|
||||
|
||||
The username used to create an SSH connection to the Linux nodes.
|
||||
|
||||
The default username for AKS nodes is `azureuser`.
|
||||
|
||||
### SSH Public Key
|
||||
|
||||
The key used to create an SSH connection to the Linux nodes.
|
||||
|
||||
### Tags
|
||||
|
||||
Cluster tags can be useful if your organization uses tags as a way to organize resources across multiple Azure services. These tags don't apply to resources within the cluster.
|
||||
|
||||
# Networking Options
|
||||
|
||||
### LoadBalancer SKU
|
||||
|
||||
Azure load balancers support both standard and basic SKUs (stock keeping units).
|
||||
|
||||
For a comparison of standard and basic load balancers, see the official [Azure documentation.](https://docs.microsoft.com/en-us/azure/load-balancer/skus#skus) Microsoft recommends the Standard load balancer.
|
||||
|
||||
The Standard load balancer is required if you have selected one or more availability zones, or if you have more than one node pool.
|
||||
|
||||
### Network Policy
|
||||
|
||||
All pods in an AKS cluster can send and receive traffic without limitations, by default. To improve security, you can define rules that control the flow of traffic. The Network Policy feature in Kubernetes lets you define rules for ingress and egress traffic between pods in a cluster.
|
||||
|
||||
Azure provides two ways to implement network policy. You choose a network policy option when you create an AKS cluster. The policy option can't be changed after the cluster is created:
|
||||
|
||||
- Azure's own implementation, called Azure Network Policies. The Azure network policy requires the Azure CNI.
|
||||
- Calico Network Policies, an open-source network and network security solution founded by [Tigera](https://www.tigera.io/).
|
||||
|
||||
You can also choose to have no network policy.
|
||||
|
||||
For more information about the differences between Azure and Calico network policies and their capabilities, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/use-network-policies#differences-between-azure-and-calico-policies-and-their-capabilities)
|
||||
|
||||
### DNS Prefix
|
||||
Enter a unique DNS prefix for your cluster's Kubernetes API server FQDN.
|
||||
|
||||
### Network Plugin
|
||||
There are two network plugins: kubenet and Azure CNI.
|
||||
|
||||
The [kubenet](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#kubenet) Kubernetes plugin is the default configuration for AKS cluster creation. When kubenet is used, each node in the cluster receives a routable IP address. The pods use NAT to communicate with other resources outside the AKS cluster. This approach reduces the number of IP addresses you need to reserve in your network space for pods to use.
|
||||
|
||||
With the Azure CNI (advanced) networking plugin, pods get full virtual network connectivity and can be directly reached via their private IP address from connected networks. This plugin requires more IP address space.
|
||||
|
||||
For more information on the differences between kubenet and Azure CNI, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/concepts-network#compare-network-models)
|
||||
|
||||
### HTTP Application Routing
|
||||
|
||||
When enabled, the HTTP application routing add-on makes it easier to access applications deployed to the AKS cluster. It deploys two components: a [Kubernetes Ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress/) and an [External-DNS](https://github.com/kubernetes-incubator/external-dns) controller.
|
||||
|
||||
For more information, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/http-application-routing)
|
||||
|
||||
### Set Authorized IP Ranges
|
||||
|
||||
You can secure access to the Kubernetes API server using [authorized IP address ranges.](https://docs.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges#overview-of-api-server-authorized-ip-ranges)
|
||||
|
||||
The Kubernetes API server exposes the Kubernetes API. This component provides the interaction for management tools, such as kubectl. AKS provides a single-tenant cluster control plane with a dedicated API server. By default, the API server is assigned a public IP address, and you should control access to it using Kubernetes-based or Azure-based RBAC.
|
||||
|
||||
To secure access to the otherwise publicly accessible AKS control plane and API server, you can enable and use authorized IP ranges. These authorized IP ranges only allow defined IP address ranges to communicate with the API server.
|
||||
|
||||
However, even if you use authorized IP address ranges, you should still use Kubernetes RBAC or Azure RBAC to authorize users and the actions they request.
|
||||
|
||||
### Container Monitoring
|
||||
|
||||
Container monitoring gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring, metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux. Metrics are written to the metrics store and log data is written to the logs store associated with your [Log Analytics](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/log-query-overview) workspace.
|
||||
|
||||
### Log Analytics Workspace Resource Group
|
||||
|
||||
The [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/overview#resource-groups) containing the Log Analytics Workspace. You must create at least one workspace to use Azure Monitor Logs.
|
||||
|
||||
### Log Analytics Workspace Name
|
||||
|
||||
Data collected by Azure Monitor Logs is stored in one or more [Log Analytics workspaces.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/design-logs-deployment) The workspace defines the geographic location of the data, access rights defining which users can access data, and configuration settings such as the pricing tier and data retention.
|
||||
|
||||
You must create at least one workspace to use Azure Monitor Logs. A single workspace may be suffxicient for all of your monitoring data, or may choose to create multiple workspaces depending on your requirements. For example, you might have one workspace for your production data and another for testing.
|
||||
|
||||
For more information about Azure Monitor Logs, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs)
|
||||
|
||||
### Support Private Kubernetes Service
|
||||
|
||||
Typically, AKS worker nodes do not get public IPs, regardless of whether the cluster is private. In a private cluster, the control plane does not have a public endpoint.
|
||||
|
||||
Rancher can connect to a private AKS cluster in one of two ways.
|
||||
|
||||
The first way to ensure that Rancher is running on the same [NAT](https://docs.microsoft.com/en-us/azure/virtual-network/nat-overview) as the AKS nodes.
|
||||
|
||||
The second way is to run a command to register the cluster with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster’s Kubernetes API. This command is displayed in a pop-up when you provision an AKS cluster with a private API endpoint enabled.
|
||||
|
||||
> **Note:** Please be aware that when registering an existing AKS cluster, the cluster might take some time, possibly hours, to appear in the `Cluster To register` dropdown list. This outcome will be based on region.
|
||||
|
||||
For more information about connecting to an AKS private cluster, see the [AKS documentation.](https://docs.microsoft.com/en-us/azure/aks/private-clusters#options-for-connecting-to-the-private-cluster)
|
||||
|
||||
# Node Pools
|
||||
|
||||
### Mode
|
||||
|
||||
The Azure interface allows users to specify whether a Primary Node Pool relies on either `system` (normally used for control planes) or `user` (what is most typically needed for Rancher).
|
||||
|
||||
For Primary Node Pools, you can specify Mode, OS, Count and Size.
|
||||
|
||||
System node pools always require running nodes, so they cannot be scaled below one node. At least one system node pool is required.
|
||||
|
||||
For subsequent node pools, the Rancher UI forces the default of User. User node pools allow you to scale to zero nodes. User node pools don't run any part of the Kubernetes controlplane.
|
||||
|
||||
AKS doesn't expose the nodes that run the Kubernetes controlplane components.
|
||||
|
||||
### Availability Zones
|
||||
|
||||
[Availability zones](https://docs.microsoft.com/en-us/azure/availability-zones/az-overview) are unique physical locations within a region. Each zone is made up of one or more data centers equipped with independent power, cooling, and networking.
|
||||
|
||||
Not all regions have support for availability zones. For a list of Azure regions with availability zones, see the [Azure documentation.](https://docs.microsoft.com/en-us/azure/availability-zones/az-region#azure-regions-with-availability-zones)
|
||||
|
||||
### VM Size
|
||||
|
||||
Choose a size for each VM in the node pool. For details about each VM size, see [this page.](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/)
|
||||
|
||||
### OS Disk Type
|
||||
|
||||
The nodes in the node pool can have either managed or ephemeral disks.
|
||||
|
||||
[Ephemeral OS disks](https://docs.microsoft.com/en-us/azure/virtual-machines/ephemeral-os-disks) are created on the local virtual machine storage and not saved to the remote Azure Storage. Ephemeral OS disks work well for stateless workloads, where applications are tolerant of individual VM failures, but are more affected by VM deployment time or reimaging the individual VM instances. With Ephemeral OS disk, you get lower read/write latency to the OS disk and faster VM reimage.
|
||||
|
||||
[Azure managed disks](https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview) are block-level storage volumes that are managed by Azure and used with Azure Virtual Machines. Managed disks are designed for 99.999% availability. Managed disks achieve this by providing you with three replicas of your data, allowing for high durability.
|
||||
|
||||
### OS Disk Size
|
||||
|
||||
The size in GB for the disk for each node.
|
||||
|
||||
### Node Count
|
||||
The number of nodes in the node pool. The maximum number of nodes may be limited by your [Azure subscription.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits)
|
||||
|
||||
### Max Pods Per Node
|
||||
The maximum number of pods per node defaults to 110 with a maximum of 250.
|
||||
|
||||
### Enable Auto Scaling
|
||||
|
||||
When auto scaling is enabled, you will need to enter a minimum and maximum node count.
|
||||
|
||||
When Auto Scaling is enabled, you can't manually scale the node pool. The scale is controlled by the AKS autoscaler.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: Cluster Configuration
|
||||
weight: 2025
|
||||
---
|
||||
|
||||
After you provision a Kubernetes cluster using Rancher, you can still edit options and settings for the cluster.
|
||||
|
||||
For information on editing cluster membership, go to [this page.]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/cluster-members)
|
||||
|
||||
### Cluster Configuration References
|
||||
|
||||
The cluster configuration options depend on the type of Kubernetes cluster:
|
||||
|
||||
- [RKE Cluster Configuration](./rke-config-reference)
|
||||
- [RKE2 Cluster Configuration](./rke2-config-reference)
|
||||
- [K3s Cluster Configuration](./k3s-config-reference)
|
||||
- [EKS Cluster Configuration](./eks-config-reference)
|
||||
- [GKE Cluster Configuration](./gke-config-reference)
|
||||
- [AKS Cluster Configuration](./aks-config-reference)
|
||||
|
||||
### Cluster Management Capabilities by Cluster Type
|
||||
|
||||
The options and settings available for an existing cluster change based on the method that you used to provision it.
|
||||
|
||||
The following table summarizes the options and settings available for each cluster type:
|
||||
|
||||
{{% include file="/rancher/v2.6/en/cluster-provisioning/cluster-capabilities-table</summary>
|
||||
|
||||
+145
@@ -0,0 +1,145 @@
|
||||
---
|
||||
title: EKS Cluster Configuration Reference
|
||||
shortTitle: EKS Cluster Configuration
|
||||
weight: 2
|
||||
---
|
||||
|
||||
### Account Access
|
||||
|
||||
Complete each drop-down and field using the information obtained for your IAM policy.
|
||||
|
||||
| Setting | Description |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| Region | From the drop-down choose the geographical region in which to build your cluster. |
|
||||
| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{<baseurl>}}/rancher/v2.6/en/user-settings/cloud-credentials/) |
|
||||
|
||||
### Service Role
|
||||
|
||||
Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html).
|
||||
|
||||
Service Role | Description
|
||||
-------------|---------------------------
|
||||
Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster.
|
||||
Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role).
|
||||
|
||||
### Secrets Encryption
|
||||
|
||||
Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html)
|
||||
|
||||
### API Server Endpoint Access
|
||||
|
||||
Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
|
||||
|
||||
### Private-only API Endpoints
|
||||
|
||||
If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API.
|
||||
|
||||
There are two ways to avoid this extra manual step:
|
||||
- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster.
|
||||
- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html).
|
||||
|
||||
### Public Access Endpoints
|
||||
|
||||
Optionally limit access to the public endpoint via explicit CIDR blocks.
|
||||
|
||||
If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster.
|
||||
|
||||
One of the following is required to enable private access:
|
||||
- Rancher's IP must be part of an allowed CIDR block
|
||||
- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group
|
||||
|
||||
For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html)
|
||||
|
||||
### Subnet
|
||||
|
||||
| Option | Description |
|
||||
| ------- | ------------ |
|
||||
| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. |
|
||||
| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). |
|
||||
|
||||
For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step.
|
||||
|
||||
- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
|
||||
- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html)
|
||||
|
||||
### Security Group
|
||||
|
||||
Amazon Documentation:
|
||||
|
||||
- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html)
|
||||
- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
|
||||
- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group)
|
||||
|
||||
### Logging
|
||||
|
||||
Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters.
|
||||
|
||||
Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation.
|
||||
|
||||
For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html)
|
||||
|
||||
### Managed Node Groups
|
||||
|
||||
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
|
||||
|
||||
For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)
|
||||
|
||||
#### Bring your own launch template
|
||||
|
||||
A launch template ID and version can be provided in order to easily configure the EC2 instances in a node group. If a launch template is provided, then none of the settings below will be configurable in Rancher. Therefore, using a launch template would require that all the necessary and desired settings from the list below would need to be specified in the launch template. Also note that if a launch template ID and version is provided, then only the template version can be updated. Using a new template ID would require creating a new managed node group.
|
||||
|
||||
| Option | Description | Required/Optional |
|
||||
| ------ | ----------- | ----------------- |
|
||||
| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required |
|
||||
| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional |
|
||||
| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required |
|
||||
| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional |
|
||||
| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional |
|
||||
| Instance Resource Tags | Tag each EC2 instance in the node group | Optional |
|
||||
|
||||
#### Rancher-managed launch templates
|
||||
|
||||
If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated.
|
||||
|
||||
#### Custom AMIs
|
||||
|
||||
If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative.
|
||||
|
||||
If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it.
|
||||
|
||||
>**Note**
|
||||
>The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template.
|
||||
|
||||
#### Spot instances
|
||||
|
||||
Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types.
|
||||
|
||||
>**Note**
|
||||
>Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration.
|
||||
|
||||
#### Node Group Settings
|
||||
|
||||
The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created.
|
||||
|
||||
| Option | Description |
|
||||
| ------- | ------------ |
|
||||
| Node Group Name | The name of the node group. |
|
||||
| Desired ASG Size | The desired number of instances. |
|
||||
| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
|
||||
| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. |
|
||||
| Labels | Kubernetes labels applied to the nodes in the managed node group. |
|
||||
| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. |
|
||||
|
||||
|
||||
### Configuring the Refresh Interval
|
||||
|
||||
The `eks-refresh-cron` setting is deprecated. It has been migrated to the `eks-refresh` setting, which is an integer representing seconds.
|
||||
|
||||
The default value is 300 seconds.
|
||||
|
||||
The syncing interval can be changed by running `kubectl edit setting eks-refresh`.
|
||||
|
||||
If the `eks-refresh-cron` setting was previously set, the migration will happen automatically.
|
||||
|
||||
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs.
|
||||
|
||||
+307
@@ -0,0 +1,307 @@
|
||||
---
|
||||
title: GKE Cluster Configuration Reference
|
||||
shortTitle: GKE Cluster Configuration
|
||||
weight: 3
|
||||
---
|
||||
|
||||
# Changes in Rancher v2.6
|
||||
|
||||
- Support for additional configuration options:
|
||||
- Project network isolation
|
||||
- Network tags
|
||||
|
||||
# Cluster Location
|
||||
|
||||
| Value | Description |
|
||||
|--------|--------------|
|
||||
| Location Type | Zonal or Regional. With GKE, you can create a cluster tailored to the availability requirements of your workload and your budget. By default, a cluster's nodes run in a single compute zone. When multiple zones are selected, the cluster's nodes will span multiple compute zones, while the controlplane is located in a single zone. Regional clusters increase the availability of the controlplane as well. For help choosing the type of cluster availability, refer to [these docs.](https://cloud.google.com/kubernetes-engine/docs/best-practices/scalability#choosing_a_regional_or_zonal_control_plane) |
|
||||
| Zone | Each region in Compute engine contains a number of zones. For more information about available regions and zones, refer to [these docs.](https://cloud.google.com/compute/docs/regions-zones#available) |
|
||||
| Additional Zones | For zonal clusters, you can select additional zones to create a [multi-zone cluster.](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#multi-zonal_clusters) |
|
||||
| Region | For [regional clusters,](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#regional_clusters) you can select a region. For more information about available regions and zones, refer to [this section](https://cloud.google.com/compute/docs/regions-zones#available). The first part of each zone name is the name of the region. |
|
||||
|
||||
# Cluster Options
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)
|
||||
|
||||
### Container Address Range
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The IP address range for pods in the cluster. Must be a valid CIDR range, e.g. 10.42.0.0/16. If not specified, a random range is automatically chosen from 10.0.0.0/8 and will exclude ranges already allocated to VMs, other clusters, or routes. Automatically chosen ranges may conflict with reserved IP addresses, dynamic routes, or routes within VPCs peering with the cluster.
|
||||
|
||||
### Network
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The Compute Engine Network that the cluster connects to. Routes and firewalls will be created using this network. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC networks that are shared to your project will appear here. will be available to select in this field. For more information, refer to [this page](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets).
|
||||
|
||||
### Node Subnet / Subnet
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The Compute Engine subnetwork that the cluster connects to. This subnetwork must belong to the network specified in the **Network** field. Select an existing subnetwork, or select "Auto Create Subnetwork" to have one automatically created. If not using an existing network, **Subnetwork Name** is required to generate one. If using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc), the VPC subnets that are shared to your project will appear here. If using a Shared VPC network, you cannot select "Auto Create Subnetwork". For more information, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
|
||||
|
||||
### Subnetwork Name
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Automatically create a subnetwork with the provided name. Required if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. For more information on subnetworks, refer to [this page.](https://cloud.google.com/vpc/docs/vpc#vpc_networks_and_subnets)
|
||||
|
||||
### Ip Aliases
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Enable [alias IPs](https://cloud.google.com/vpc/docs/alias-ip). This enables VPC-native traffic routing. Required if using [Shared VPCs](https://cloud.google.com/vpc/docs/shared-vpc).
|
||||
|
||||
### Network Policy
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Enable network policy enforcement on the cluster. A network policy defines the level of communication that can occur between pods and services in the cluster. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/network-policy)
|
||||
|
||||
### Project Network Isolation
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
choose whether to enable or disable inter-project communication. Note that enabling Project Network Isolation will automatically enable Network Policy and Network Policy Config, but not vice versa.
|
||||
|
||||
### Node Ipv4 CIDR Block
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The IP address range of the instance IPs in this cluster. Can be set if "Auto Create Subnetwork" is selected for **Node Subnet** or **Subnet**. Must be a valid CIDR range, e.g. 10.96.0.0/14. For more information on how to determine the IP address range, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing)
|
||||
|
||||
### Cluster Secondary Range Name
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The name of an existing secondary range for Pod IP addresses. If selected, **Cluster Pod Address Range** will automatically be populated. Required if using a Shared VPC network.
|
||||
|
||||
### Cluster Pod Address Range
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The IP address range assigned to pods in the cluster. Must be a valid CIDR range, e.g. 10.96.0.0/11. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your pods, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_pods)
|
||||
|
||||
### Services Secondary Range Name
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The name of an existing secondary range for service IP addresses. If selected, **Service Address Range** will be automatically populated. Required if using a Shared VPC network.
|
||||
|
||||
### Service Address Range
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The address range assigned to the services in the cluster. Must be a valid CIDR range, e.g. 10.94.0.0/18. If not provided, will be created automatically. Must be provided if using a Shared VPC network. For more information on how to determine the IP address range for your services, refer to [this section.](https://cloud.google.com/kubernetes-engine/docs/concepts/alias-ips#cluster_sizing_secondary_range_svcs)
|
||||
|
||||
### Private Cluster
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.6/en/cluster-admin/editing-clusters/gke-config-reference/private-clusters/).
|
||||
|
||||
Assign nodes only internal IP addresses. Private cluster nodes cannot access the public internet unless additional networking steps are taken in GCP.
|
||||
|
||||
### Enable Private Endpoint
|
||||
|
||||
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.6/en/cluster-admin/editing-clusters/gke-config-reference/private-clusters/).
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Locks down external access to the control plane endpoint. Only available if **Private Cluster** is also selected. If selected, and if Rancher does not have direct access to the Virtual Private Cloud network the cluster is running in, Rancher will provide a registration command to run on the cluster to enable Rancher to connect to it.
|
||||
|
||||
### Master IPV4 CIDR Block
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The IP range for the control plane VPC.
|
||||
|
||||
### Master Authorized Network
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Enable control plane authorized networks to block untrusted non-GCP source IPs from accessing the Kubernetes master through HTTPS. If selected, additional authorized networks may be added. If the cluster is created with a public endpoint, this option is useful for locking down access to the public endpoint to only certain networks, such as the network where your Rancher service is running. If the cluster only has a private endpoint, this setting is required.
|
||||
|
||||
# Additional Options
|
||||
|
||||
### Cluster Addons
|
||||
|
||||
Additional Kubernetes cluster components. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters#Cluster.AddonsConfig)
|
||||
|
||||
#### Horizontal Pod Autoscaling
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
The Horizontal Pod Autoscaler changes the shape of your Kubernetes workload by automatically increasing or decreasing the number of Pods in response to the workload's CPU or memory consumption, or in response to custom metrics reported from within Kubernetes or external metrics from sources outside of your cluster. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler)
|
||||
|
||||
#### HTTP (L7) Load Balancing
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
HTTP (L7) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on GKE. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer)
|
||||
|
||||
#### Network Policy Config (master only)
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Configuration for NetworkPolicy. This only tracks whether the addon is enabled or not on the master, it does not track whether network policy is enabled for the nodes.
|
||||
|
||||
### Cluster Features (Alpha Features)
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Turns on all Kubernetes alpha API groups and features for the cluster. When enabled, the cluster cannot be upgraded and will be deleted automatically after 30 days. Alpha clusters are not recommended for production use as they are not covered by the GKE SLA. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters)
|
||||
|
||||
### Logging Service
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
The logging service the cluster uses to write logs. Use either [Cloud Logging](https://cloud.google.com/logging) or no logging service in which case no logs are exported from the cluster.
|
||||
|
||||
### Monitoring Service
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
The monitoring service the cluster uses to write metrics. Use either [Cloud Monitoring](https://cloud.google.com/monitoring) or monitoring service in which case no metrics are exported from the cluster.
|
||||
|
||||
|
||||
### Maintenance Window
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Set the start time for a 4 hour maintenance window. The time is specified in the UTC time zone using the HH:MM format. For more information, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions)
|
||||
|
||||
# Node Pools
|
||||
|
||||
In this section, enter details describing the configuration of each node in the node pool.
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
The Kubernetes version for each node in the node pool. For more information on GKE Kubernetes versions, refer to [these docs.](https://cloud.google.com/kubernetes-engine/versioning)
|
||||
|
||||
### Image Type
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
The node operating system image. For more information for the node image options that GKE offers for each OS, refer to [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/node-images#available_node_images)
|
||||
|
||||
> Note: the default option is "Container-Optimized OS with Docker". The read-only filesystem on GCP's Container-Optimized OS is not compatible with the [legacy logging]({{<baseurl>}}/rancher/v2.0-v2.4/en/cluster-admin/tools/logging) implementation in Rancher. If you need to use the legacy logging feature, select "Ubuntu with Docker" or "Ubuntu with Containerd". The [current logging feature]({{<baseurl>}}/rancher/v2.6/en/logging) is compatible with the Container-Optimized OS image.
|
||||
|
||||
> Note: if selecting "Windows Long Term Service Channel" or "Windows Semi-Annual Channel" for the node pool image type, you must also add at least one Container-Optimized OS or Ubuntu node pool.
|
||||
|
||||
### Machine Type
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
The virtualized hardware resources available to node instances. For more information on Google Cloud machine types, refer to [this page.](https://cloud.google.com/compute/docs/machine-types#machine_types)
|
||||
|
||||
### Root Disk Type
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Standard persistent disks are backed by standard hard disk drives (HDD), while SSD persistent disks are backed by solid state drives (SSD). For more information, refer to [this section.](https://cloud.google.com/compute/docs/disks)
|
||||
|
||||
### Local SSD Disks
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Configure each node's local SSD disk storage in GB. Local SSDs are physically attached to the server that hosts your VM instance. Local SSDs have higher throughput and lower latency than standard persistent disks or SSD persistent disks. The data that you store on a local SSD persists only until the instance is stopped or deleted. For more information, see [this section.](https://cloud.google.com/compute/docs/disks#localssds)
|
||||
|
||||
### Preemptible nodes (beta)
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Preemptible nodes, also called preemptible VMs, are Compute Engine VM instances that last a maximum of 24 hours in general, and provide no availability guarantees. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/preemptible-vms)
|
||||
|
||||
### Taints
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
When you apply a taint to a node, only Pods that tolerate the taint are allowed to run on the node. In a GKE cluster, you can apply a taint to a node pool, which applies the taint to all nodes in the pool.
|
||||
|
||||
### Node Labels
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
You can apply labels to the node pool, which applies the labels to all nodes in the pool.
|
||||
|
||||
Invalid labels can prevent upgrades or can prevent Rancher from starting. For details on label syntax requirements, see the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)
|
||||
|
||||
### Network Tags
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
You can add network tags to the node pool to make firewall rules and routes between subnets. Tags will apply to all nodes in the pool.
|
||||
|
||||
For details on tag syntax and requirements, see the [Kubernetes documentation](https://cloud.google.com/vpc/docs/add-remove-network-tags).
|
||||
|
||||
# Group Details
|
||||
|
||||
In this section, enter details describing the node pool.
|
||||
|
||||
### Name
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Enter a name for the node pool.
|
||||
|
||||
### Initial Node Count
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Integer for the starting number of nodes in the node pool.
|
||||
|
||||
### Max Pod Per Node
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
GKE has a hard limit of 110 Pods per node. For more information on the Kubernetes limits, see [this section.](https://cloud.google.com/kubernetes-engine/docs/best-practices/scalability#dimension_limits)
|
||||
|
||||
### Autoscaling
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
Node pool autoscaling dynamically creates or deletes nodes based on the demands of your workload. For more information, see [this page.](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler)
|
||||
|
||||
### Auto Repair
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
GKE's node auto-repair feature helps you keep the nodes in your cluster in a healthy, running state. When enabled, GKE makes periodic checks on the health state of each node in your cluster. If a node fails consecutive health checks over an extended time period, GKE initiates a repair process for that node. For more information, see the section on [auto-repairing nodes.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-repair)
|
||||
|
||||
### Auto Upgrade
|
||||
|
||||
_Mutable: yes_
|
||||
|
||||
When enabled, the auto-upgrade feature keeps the nodes in your cluster up-to-date with the cluster control plane (master) version when your control plane is [updated on your behalf.](https://cloud.google.com/kubernetes-engine/upgrades#automatic_cp_upgrades) For more information about auto-upgrading nodes, see [this page.](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-upgrades)
|
||||
|
||||
### Access Scopes
|
||||
|
||||
_Mutable: no_
|
||||
|
||||
Access scopes are the legacy method of specifying permissions for your nodes.
|
||||
|
||||
- **Allow default access:** The default access for new clusters is the [Compute Engine default service account.](https://cloud.google.com/compute/docs/access/service-accounts?hl=en_US#default_service_account)
|
||||
- **Allow full access to all Cloud APIs:** Generally, you can just set the cloud-platform access scope to allow full access to all Cloud APIs, then grant the service account only relevant IAM roles. The combination of access scopes granted to the virtual machine instance and the IAM roles granted to the service account determines the amount of access the service account has for that instance.
|
||||
- **Set access for each API:** Alternatively, you can choose to set specific scopes that permit access to the particular API methods that the service will call.
|
||||
|
||||
For more information, see the [section about enabling service accounts for a VM.](https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances)
|
||||
|
||||
|
||||
### Configuring the Refresh Interval
|
||||
|
||||
The refresh interval can be configured through the setting "gke-refresh", which is an integer representing seconds.
|
||||
|
||||
The default value is 300 seconds.
|
||||
|
||||
The syncing interval can be changed by running `kubectl edit setting gke-refresh`.
|
||||
|
||||
The shorter the refresh window, the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for GCP APIs.
|
||||
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: Private Clusters
|
||||
weight: 2
|
||||
---
|
||||
|
||||
In GKE, [private clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/private-cluster-concept) are clusters whose nodes are isolated from inbound and outbound traffic by assigning them internal IP addresses only. Private clusters in GKE have the option of exposing the control plane endpoint as a publicly accessible address or as a private address. This is different from other Kubernetes providers, which may refer to clusters with private control plane endpoints as "private clusters" but still allow traffic to and from nodes. You may want to create a cluster with private nodes, with or without a public control plane endpoint, depending on your organization's networking and security requirements. A GKE cluster provisioned from Rancher can use isolated nodes by selecting "Private Cluster" in the Cluster Options (under "Show advanced options"). The control plane endpoint can optionally be made private by selecting "Enable Private Endpoint".
|
||||
|
||||
### Private Nodes
|
||||
|
||||
Because the nodes in a private cluster only have internal IP addresses, they will not be able to install the cluster agent and Rancher will not be able to fully manage the cluster. This can be overcome in a few ways.
|
||||
|
||||
#### Cloud NAT
|
||||
|
||||
>**Note**
|
||||
>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
|
||||
|
||||
If restricting outgoing internet access is not a concern for your organization, use Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service to allow nodes in the private network to access the internet, enabling them to download the required images from Dockerhub and contact the Rancher management server. This is the simplest solution.
|
||||
|
||||
#### Private registry
|
||||
|
||||
>**Note**
|
||||
>This scenario is not officially supported, but is described for cases in which using the Cloud NAT service is not sufficient.
|
||||
|
||||
If restricting both incoming and outgoing traffic to nodes is a requirement, follow the air-gapped installation instructions to set up a private container image [registry](https://rancher.com/docs/rancher/v2.6/en/installation/other-installation-methods/air-gap/) on the VPC where the cluster is going to be, allowing the cluster nodes to access and download the images they need to run the cluster agent. If the control plane endpoint is also private, Rancher will need [direct access](#direct-access) to it.
|
||||
|
||||
### Private Control Plane Endpoint
|
||||
|
||||
If the cluster has a public endpoint exposed, Rancher will be able to reach the cluster, and no additional steps need to be taken. However, if the cluster has no public endpoint, then considerations must be made to ensure Rancher can access the cluster.
|
||||
|
||||
#### Cloud NAT
|
||||
|
||||
>**Note**
|
||||
>Cloud NAT will [incur charges](https://cloud.google.com/nat/pricing).
|
||||
|
||||
As above, if restricting outgoing internet access to the nodes is not a concern, then Google's [Cloud NAT](https://cloud.google.com/nat/docs/using-nat) service can be used to allow the nodes to access the internet. While the cluster is provisioning, Rancher will provide a registration command to run on the cluster. Download the [kubeconfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) for the new cluster and run the provided kubectl command on the cluster. Gaining access
|
||||
to the cluster in order to run this command can be done by creating a temporary node or using an existing node in the VPC, or by logging on to or creating an SSH tunnel through one of the cluster nodes.
|
||||
|
||||
#### Direct access
|
||||
|
||||
If the Rancher server is run on the same VPC as the cluster's control plane, it will have direct access to the control plane's private endpoint. The cluster nodes will need to have access to a [private registry](#private-registry) to download images as described above.
|
||||
|
||||
You can also use services from Google such as [Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview) or [Cloud Interconnect VLAN](https://cloud.google.com/network-connectivity/docs/interconnect) to facilitate connectivity between your organization's network and your Google VPC.
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
---
|
||||
title: K3s Cluster Configuration Reference
|
||||
shortTitle: K3s Cluster Configuration
|
||||
weight: 6
|
||||
---
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing K3s Kubernetes cluster.
|
||||
|
||||
# Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#cluster-config-file): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create a K3s config file. Using a config file allows you to set any of the [options](https://rancher.com/docs/k3s/latest/en/installation/install-options/) available in an K3s installation.
|
||||
|
||||
# Configuration Options in the Rancher UI
|
||||
|
||||
> Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the K3s cluster configuration file in YAML. For the complete reference of configurable options for K3s clusters in YAML, see the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/install-options/)
|
||||
|
||||
### Basics
|
||||
#### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/upgrading-kubernetes).
|
||||
|
||||
#### Encrypt Secrets
|
||||
|
||||
Option to enable or disable secrets encryption. When enabled, secrets will be encrypted using a AES-CBC key. If disabled, any previously secrets will not be readable until encryption is enabled again. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/advanced/#secrets-encryption-config-experimental) for details.
|
||||
|
||||
#### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
#### SELinux
|
||||
|
||||
Option to enable or disable [SELinux](https://rancher.com/docs/k3s/latest/en/advanced/#selinux-support) support.
|
||||
|
||||
#### CoreDNS
|
||||
|
||||
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#coredns) for details..
|
||||
|
||||
#### Klipper Service LB
|
||||
|
||||
Option to enable or disable the [Klipper](https://github.com/rancher/klipper-lb) service load balancer. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#service-load-balancer) for details.
|
||||
|
||||
#### Traefik Ingress
|
||||
|
||||
Option to enable or disable the [Traefik](https://traefik.io/) HTTP reverse proxy and load balancer. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/#traefik-ingress-controller).
|
||||
|
||||
#### Local Storage
|
||||
|
||||
Option to enable or disable [local storage](https://rancher.com/docs/k3s/latest/en/storage/) on the node(s).
|
||||
|
||||
#### Metrics Server
|
||||
|
||||
Option to enable or disable the [metrics server](https://github.com/kubernetes-incubator/metrics-server). If enabled, ensure port 10250 is opened for inbound TCP traffic.
|
||||
|
||||
### Add-On Config
|
||||
|
||||
Additional Kubernetes manifests, managed as a [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/helm/#automatically-deploying-manifests-and-helm-charts) for details.
|
||||
|
||||
### Agent Environment Vars
|
||||
|
||||
Option to set environment variables for [K3s agents](https://rancher.com/docs/k3s/latest/en/architecture/). The environment variables can be set using key value pairs. Refer to the [K3 documentation](https://rancher.com/docs/k3s/latest/en/installation/install-options/agent-config/) for more details.
|
||||
|
||||
### etcd
|
||||
|
||||
#### Automatic Snapshots
|
||||
|
||||
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/backup-restore/#creating-snapshots).
|
||||
|
||||
#### Metrics
|
||||
|
||||
Option to choose whether to expose etcd metrics to the public or only within the cluster.
|
||||
|
||||
### Networking
|
||||
|
||||
#### Cluster CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16).
|
||||
|
||||
#### Service CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16).
|
||||
|
||||
#### Cluster DNS
|
||||
|
||||
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10).
|
||||
|
||||
#### Cluster Domain
|
||||
|
||||
Select the domain for the cluster. The default is `cluster.local`.
|
||||
|
||||
#### NodePort Service Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
|
||||
|
||||
#### TLS Alternate Names
|
||||
|
||||
Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
|
||||
|
||||
#### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture-recommendations/#architecture-for-an-authorized-cluster-endpoint)
|
||||
|
||||
### Registries
|
||||
|
||||
Select the image repository to pull Rancher images from. For more details and configuration options, see the [K3s documentation](https://rancher.com/docs/k3s/latest/en/installation/private-registry/).
|
||||
|
||||
### Upgrade Strategy
|
||||
|
||||
#### Controle Plane Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Worker Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Drain Nodes (Control Plane)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
#### Drain Nodes (Worker Nodes)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
### Advanced
|
||||
|
||||
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
|
||||
|
||||
# Cluster Config File
|
||||
|
||||
Instead of using the Rancher UI forms to choose Kubernetes options for the cluster, advanced users can create an K3s config file. Using a config file allows you to set any of the [options](https://rancher.com/docs/k3s/latest/en/installation/install-options/) available in an K3s installation.
|
||||
|
||||
To edit an K3s config file directly from the Rancher UI, click **Edit as YAML**.
|
||||
|
||||
|
||||
+360
@@ -0,0 +1,360 @@
|
||||
---
|
||||
title: RKE Cluster Configuration Reference
|
||||
shortTitle: RKE Cluster Configuration
|
||||
weight: 1
|
||||
---
|
||||
|
||||
When Rancher installs Kubernetes, it uses [RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/) or [RKE2](https://docs.rke2.io/) as the Kubernetes distribution.
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE Kubernetes cluster.
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Editing Clusters with a Form in the Rancher UI](#editing-clusters-with-a-form-in-the-rancher-ui)
|
||||
- [Editing Clusters with YAML](#editing-clusters-with-yaml)
|
||||
- [Configuration Options in the Rancher UI](#configuration-options-in-the-rancher-ui)
|
||||
- [Kubernetes Version](#kubernetes-version)
|
||||
- [Network Provider](#network-provider)
|
||||
- [Project Network Isolation](#project-network-isolation)
|
||||
- [Kubernetes Cloud Providers](#kubernetes-cloud-providers)
|
||||
- [Private Registries](#private-registries)
|
||||
- [Authorized Cluster Endpoint](#authorized-cluster-endpoint)
|
||||
- [Node Pools](#node-pools)
|
||||
- [NGINX Ingress](#nginx-ingress)
|
||||
- [Metrics Server Monitoring](#metrics-server-monitoring)
|
||||
- [Pod Security Policy Support](#pod-security-policy-support)
|
||||
- [Docker Version on Nodes](#docker-version-on-nodes)
|
||||
- [Docker Root Directory](#docker-root-directory)
|
||||
- [Default Pod Security Policy](#default-pod-security-policy)
|
||||
- [Node Port Range](#node-port-range)
|
||||
- [Recurring etcd Snapshots](#recurring-etcd-snapshots)
|
||||
- [Agent Environment Variables](#agent-environment-variables)
|
||||
- [Updating ingress-nginx](#updating-ingress-nginx)
|
||||
- [RKE Cluster Config File Reference](#rke-cluster-config-file-reference)
|
||||
- [Config File Structure in Rancher](#config-file-structure-in-rancher)
|
||||
- [Default DNS Provider](#default-dns-provider)
|
||||
- [Rancher Specific Parameters in YAML](#rancher-specific-parameters-in-yaml)
|
||||
- [docker_root_dir](#docker_root_dir)
|
||||
- [enable_cluster_monitoring](#enable_cluster_monitoring)
|
||||
- [enable_network_policy](#enable_network_policy)
|
||||
- [local_cluster_auth_endpoint](#local_cluster_auth_endpoint)
|
||||
- [Custom Network Plug-in](#custom-network-plug-in)
|
||||
|
||||
# Overview
|
||||
|
||||
You can configure the Kubernetes options one of two ways:
|
||||
|
||||
- [Rancher UI](#rancher-ui-options): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#cluster-config-file): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
The RKE cluster config options are nested under the `rancher_kubernetes_engine_config` directive. For more information, see the section about the [cluster config file.](#cluster-config-file)
|
||||
|
||||
In [clusters launched by RKE]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/), you can edit any of the remaining options that follow.
|
||||
|
||||
For an example of RKE config file syntax, see the [RKE documentation]({{<baseurl>}}/rke/latest/en/example-yamls/).
|
||||
|
||||
The forms in the Rancher UI don't include all advanced options for configuring RKE. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/)
|
||||
|
||||
# Editing Clusters with a Form in the Rancher UI
|
||||
|
||||
To edit your cluster,
|
||||
|
||||
1. In the upper left corner, click **☰ > Cluster Management**.
|
||||
1. Go to the cluster you want to configure and click **⋮ > Edit Config**.
|
||||
|
||||
|
||||
# Editing Clusters with YAML
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the options available in an RKE installation, except for system_images configuration, by specifying them in YAML.
|
||||
|
||||
RKE clusters (also called RKE1 clusters) are edited differently than RKE2 and K3s clusters.
|
||||
|
||||
To edit an RKE config file directly from the Rancher UI,
|
||||
|
||||
1. Click **☰ > Cluster Management**.
|
||||
1. Go to the RKE cluster you want to configure. Click and click **⋮ > Edit Config**. This take you to the RKE configuration form. Note: Because cluster provisioning changed in Rancher 2.6, the **⋮ > Edit as YAML** can be used for configuring RKE2 clusters, but it can't be used for editing RKE1 configuration.
|
||||
1. In the configuration form, scroll down and click **Edit as YAML**.
|
||||
1. Edit the RKE options under the `rancher_kubernetes_engine_config` directive.
|
||||
|
||||
# Configuration Options in the Rancher UI
|
||||
|
||||
> Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE cluster configuration file in YAML. For the complete reference of configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/)
|
||||
|
||||
### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/upgrading-kubernetes).
|
||||
|
||||
### Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses. For more details on the different networking providers, please view our [Networking FAQ]({{<baseurl>}}/rancher/v2.6/en/faq/networking/cni-providers/).
|
||||
|
||||
> After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you tear down the entire cluster and all its applications.
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Flannel](https://github.com/coreos/flannel#flannel)
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Weave](https://github.com/weaveworks/weave)
|
||||
|
||||
**Notes on Weave:**
|
||||
|
||||
When Weave is selected as network provider, Rancher will automatically enable encryption by generating a random password. If you want to specify the password manually, please see how to configure your cluster using a [Config File]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/options/#cluster-config-file) and the [Weave Network Plug-in Options]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/#weave-network-plug-in-options).
|
||||
|
||||
### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### Kubernetes Cloud Providers
|
||||
|
||||
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
>**Note:** If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file) to configure the cloud provider. Please reference the [RKE cloud provider documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
### Private Registries
|
||||
|
||||
The cluster-level private registry configuration is only used for provisioning clusters.
|
||||
|
||||
There are two main ways to set up private registries in Rancher: by setting up the [global default registry]({{<baseurl>}}/rancher/v2.6/en/admin-settings/config-private-registry) through the **Settings** tab in the global view, and by setting up a private registry in the advanced options in the cluster-level settings. The global default registry is intended to be used for air-gapped setups, for registries that do not require credentials. The cluster-level private registry is intended to be used in all setups in which the private registry requires credentials.
|
||||
|
||||
If your private registry requires credentials, you need to pass the credentials to Rancher by editing the cluster options for each cluster that needs to pull images from the registry.
|
||||
|
||||
The private registry configuration option tells Rancher where to pull the [system images]({{<baseurl>}}/rke/latest/en/config-options/system-images/) or [addon images]({{<baseurl>}}/rke/latest/en/config-options/add-ons/) that will be used in your cluster.
|
||||
|
||||
- **System images** are components needed to maintain the Kubernetes cluster.
|
||||
- **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server.
|
||||
|
||||
For more information on setting up a private registry for components applied during the provisioning of the cluster, see the [RKE documentation on private registries]({{<baseurl>}}/rke/latest/en/config-options/private-registries/).
|
||||
|
||||
Rancher v2.6 introduced the ability to configure [ECR registries for RKE clusters]({{<baseurl>}}/rke/latest/en/config-options/private-registries/#amazon-elastic-container-registry-ecr-private-registry-setup).
|
||||
|
||||
### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
> The authorized cluster endpoint only works on Rancher-launched Kubernetes clusters. In other words, it only works in clusters where Rancher [used RKE]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#tools-for-provisioning-kubernetes-clusters) to provision the cluster. It is not available for clusters in a hosted Kubernetes provider, such as Amazon's EKS.
|
||||
|
||||
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture-recommendations/#architecture-for-an-authorized-cluster-endpoint)
|
||||
|
||||
### Node Pools
|
||||
|
||||
For information on using the Rancher UI to set up node pools in an RKE cluster, refer to [this page.]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/node-pools)
|
||||
|
||||
### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster.
|
||||
|
||||
### Metrics Server Monitoring
|
||||
|
||||
Option to enable or disable [Metrics Server]({{<baseurl>}}/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
### Pod Security Policy Support
|
||||
|
||||
Enables [pod security policies]({{<baseurl>}}/rancher/v2.6/en/admin-settings/pod-security-policies/) for the cluster. After enabling this option, choose a policy using the **Default Pod Security Policy** drop-down.
|
||||
|
||||
You must have an existing Pod Security Policy configured before you can use this option.
|
||||
|
||||
### Docker Version on Nodes
|
||||
|
||||
Configures whether nodes are allowed to run versions of Docker that Rancher doesn't officially support.
|
||||
|
||||
If you choose to require a supported Docker version, Rancher will stop pods from running on nodes that don't have a supported Docker version installed.
|
||||
|
||||
For details on which Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
|
||||
|
||||
### Docker Root Directory
|
||||
|
||||
If the nodes you are adding to the cluster have Docker configured with a non-default Docker Root Directory (default is `/var/lib/docker`), specify the correct Docker Root Directory in this option.
|
||||
|
||||
### Default Pod Security Policy
|
||||
|
||||
If you enable **Pod Security Policy Support**, use this drop-down to choose the pod security policy that's applied to the cluster.
|
||||
|
||||
### Node Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). Default is `30000-32767`.
|
||||
|
||||
### Recurring etcd Snapshots
|
||||
|
||||
Option to enable or disable [recurring etcd snapshots]({{<baseurl>}}/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots).
|
||||
|
||||
### Agent Environment Variables
|
||||
|
||||
Option to set environment variables for [rancher agents]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
|
||||
|
||||
### Updating ingress-nginx
|
||||
|
||||
Clusters that were created before Kubernetes 1.16 will have an `ingress-nginx` `updateStrategy` of `OnDelete`. Clusters that were created with Kubernetes 1.16 or newer will have `RollingUpdate`.
|
||||
|
||||
If the `updateStrategy` of `ingress-nginx` is `OnDelete`, you will need to delete these pods to get the correct version for your deployment.
|
||||
|
||||
|
||||
|
||||
# RKE Cluster Config File Reference
|
||||
|
||||
Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available]({{<baseurl>}}/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API.
|
||||
|
||||
For the complete reference for configurable options for RKE Kubernetes clusters in YAML, see the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/)
|
||||
|
||||
### Config File Structure in Rancher
|
||||
|
||||
RKE (Rancher Kubernetes Engine) is the tool that Rancher uses to provision Kubernetes clusters. Rancher's cluster config files used to have the same structure as [RKE config files,]({{<baseurl>}}/rke/latest/en/example-yamls/) but the structure changed so that in Rancher, RKE cluster config items are separated from non-RKE config items. Therefore, configuration for your cluster needs to be nested under the `rancher_kubernetes_engine_config` directive in the cluster config file. Cluster config files created with earlier versions of Rancher will need to be updated for this format. An example cluster config file is included below.
|
||||
|
||||
<details id="v2.3.0-cluster-config-file">
|
||||
<summary>Example Cluster Config File</summary>
|
||||
|
||||
```yaml
|
||||
#
|
||||
# Cluster Config
|
||||
#
|
||||
docker_root_dir: /var/lib/docker
|
||||
enable_cluster_alerting: false
|
||||
enable_cluster_monitoring: false
|
||||
enable_network_policy: false
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
#
|
||||
# Rancher Config
|
||||
#
|
||||
rancher_kubernetes_engine_config: # Your RKE template config goes here.
|
||||
addon_job_timeout: 30
|
||||
authentication:
|
||||
strategy: x509
|
||||
ignore_docker_version: true
|
||||
#
|
||||
# # Currently only nginx ingress provider is supported.
|
||||
# # To disable ingress controller, set `provider: none`
|
||||
# # To enable ingress on specific nodes, use the node_selector, eg:
|
||||
# provider: nginx
|
||||
# node_selector:
|
||||
# app: ingress
|
||||
#
|
||||
ingress:
|
||||
provider: nginx
|
||||
kubernetes_version: v1.15.3-rancher3-1
|
||||
monitoring:
|
||||
provider: metrics-server
|
||||
#
|
||||
# If you are using calico on AWS
|
||||
#
|
||||
# network:
|
||||
# plugin: calico
|
||||
# calico_network_provider:
|
||||
# cloud_provider: aws
|
||||
#
|
||||
# # To specify flannel interface
|
||||
#
|
||||
# network:
|
||||
# plugin: flannel
|
||||
# flannel_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
# # To specify flannel interface for canal plugin
|
||||
#
|
||||
# network:
|
||||
# plugin: canal
|
||||
# canal_network_provider:
|
||||
# iface: eth1
|
||||
#
|
||||
network:
|
||||
options:
|
||||
flannel_backend_type: vxlan
|
||||
plugin: canal
|
||||
#
|
||||
# services:
|
||||
# kube-api:
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kube-controller:
|
||||
# cluster_cidr: 10.42.0.0/16
|
||||
# service_cluster_ip_range: 10.43.0.0/16
|
||||
# kubelet:
|
||||
# cluster_domain: cluster.local
|
||||
# cluster_dns_server: 10.43.0.10
|
||||
#
|
||||
services:
|
||||
etcd:
|
||||
backup_config:
|
||||
enabled: true
|
||||
interval_hours: 12
|
||||
retention: 6
|
||||
safe_timestamp: false
|
||||
creation: 12h
|
||||
extra_args:
|
||||
election-timeout: 5000
|
||||
heartbeat-interval: 500
|
||||
gid: 0
|
||||
retention: 72h
|
||||
snapshot: false
|
||||
uid: 0
|
||||
kube_api:
|
||||
always_pull_images: false
|
||||
pod_security_policy: false
|
||||
service_node_port_range: 30000-32767
|
||||
ssh_agent_auth: false
|
||||
windows_prefered_cluster: false
|
||||
```
|
||||
</details>
|
||||
|
||||
### Default DNS provider
|
||||
|
||||
The table below indicates what DNS provider is deployed by default. See [RKE documentation on DNS provider]({{<baseurl>}}/rke/latest/en/config-options/add-ons/dns/) for more information how to configure a different DNS provider. CoreDNS can only be used on Kubernetes v1.12.0 and higher.
|
||||
|
||||
| Rancher version | Kubernetes version | Default DNS provider |
|
||||
|-------------|--------------------|----------------------|
|
||||
| v2.2.5 and higher | v1.14.0 and higher | CoreDNS |
|
||||
| v2.2.5 and higher | v1.13.x and lower | kube-dns |
|
||||
| v2.2.4 and lower | any | kube-dns |
|
||||
|
||||
# Rancher Specific Parameters in YAML
|
||||
|
||||
Besides the RKE config file options, there are also Rancher specific settings that can be configured in the Config File (YAML):
|
||||
|
||||
### docker_root_dir
|
||||
|
||||
See [Docker Root Directory](#docker-root-directory).
|
||||
|
||||
### enable_cluster_monitoring
|
||||
|
||||
Option to enable or disable [Cluster Monitoring]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting/).
|
||||
|
||||
### enable_network_policy
|
||||
|
||||
Option to enable or disable Project Network Isolation.
|
||||
|
||||
Project network isolation is available if you are using any RKE network plugin that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin.
|
||||
|
||||
### local_cluster_auth_endpoint
|
||||
|
||||
See [Authorized Cluster Endpoint](#authorized-cluster-endpoint).
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
local_cluster_auth_endpoint:
|
||||
enabled: true
|
||||
fqdn: "FQDN"
|
||||
ca_certs: |-
|
||||
-----BEGIN CERTIFICATE-----
|
||||
...
|
||||
-----END CERTIFICATE-----
|
||||
```
|
||||
|
||||
### Custom Network Plug-in
|
||||
|
||||
You can add a custom network plug-in by using the [user-defined add-on functionality]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/) of RKE. You define any add-on that you want deployed after the Kubernetes cluster is deployed.
|
||||
|
||||
There are two ways that you can specify an add-on:
|
||||
|
||||
- [In-line Add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons)
|
||||
- [Referencing YAML Files for Add-ons]({{<baseurl>}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons)
|
||||
|
||||
For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.]({{<baseurl>}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example)
|
||||
+188
@@ -0,0 +1,188 @@
|
||||
---
|
||||
title: RKE2 Cluster Configuration Reference
|
||||
shortTitle: RKE2 Cluster Configuration
|
||||
weight: 5
|
||||
---
|
||||
|
||||
This section covers the configuration options that are available in Rancher for a new or existing RKE2 Kubernetes cluster.
|
||||
|
||||
# Overview
|
||||
|
||||
You can configure the Kubernetes options in one of the two following ways:
|
||||
|
||||
- [Rancher UI](#configuration-options-in-the-rancher-ui): Use the Rancher UI to select options that are commonly customized when setting up a Kubernetes cluster.
|
||||
- [Cluster Config File](#cluster-config-file): Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE2 config file. Using a config file allows you to set any of the [options](https://docs.rke2.io/install/install_options/install_options) available in an RKE2 installation.
|
||||
|
||||
# Configuration Options in the Rancher UI
|
||||
|
||||
> Some advanced configuration options are not exposed in the Rancher UI forms, but they can be enabled by editing the RKE2 cluster configuration file in YAML. For the complete reference of configurable options for RKE2 Kubernetes clusters in YAML, see the [RKE2 documentation.](https://docs.rke2.io/install/install_options/install_options/)
|
||||
|
||||
### Basics
|
||||
#### Kubernetes Version
|
||||
|
||||
The version of Kubernetes installed on your cluster nodes. Rancher packages its own version of Kubernetes based on [hyperkube](https://github.com/rancher/hyperkube).
|
||||
|
||||
For more detail, see [Upgrading Kubernetes]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/upgrading-kubernetes).
|
||||
|
||||
#### Container Network Provider
|
||||
|
||||
The [Network Provider](https://kubernetes.io/docs/concepts/cluster-administration/networking/) that the cluster uses.
|
||||
|
||||
> After you launch the cluster, you cannot change your network provider. Therefore, choose which network provider you want to use carefully, as Kubernetes doesn't allow switching between network providers. Once a cluster is created with a network provider, changing network providers would require you to tear down the entire cluster and all its applications.
|
||||
|
||||
Out of the box, Rancher is compatible with the following network providers:
|
||||
|
||||
- [Canal](https://github.com/projectcalico/canal)
|
||||
- [Cilium](https://cilium.io/)*
|
||||
- [Calico](https://docs.projectcalico.org/v3.11/introduction/)
|
||||
- [Multus](https://github.com/k8snetworkplumbingwg/multus-cni)
|
||||
|
||||
\* When using [project network isolation](#project-network-isolation) in the [Cilium CNI]({{<baseurl>}}/rancher/v2.6/en/faq/networking/cni-providers/#cilium), it is possible to enable cross-node ingress routing. Click the [CNI provider docs]({{<baseurl>}}/rancher/v2.6/en/faq/networking/cni-providers/#ingress-routing-across-nodes-in-cilium) to learn more.
|
||||
|
||||
For more details on the different networking providers and how to configure them, please view our [RKE2 documentation](https://docs.rke2.io/install/network_options/).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
[Dual-stack](https://docs.rke2.io/install/network_options/#dual-stack-configuration) networking is supported for all CNI providers. To configure RKE2 in dual-stack mode, set valid IPv4/IPv6 CIDRs for your [Cluster CIDR](#cluster-cidr) and/or [Service CIDR](#service-cidr).
|
||||
|
||||
###### Additional Configuration {#dual-stack-additional-config}
|
||||
|
||||
When using `cilium` or `multus,cilium` as your container network interface provider, ensure the **Enable IPv6 Support** option is also enabled.
|
||||
|
||||
#### Cloud Provider
|
||||
|
||||
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
|
||||
|
||||
>**Note:** If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file) to configure the cloud provider. Please reference [this documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
|
||||
|
||||
#### Default Pod Security Policy
|
||||
|
||||
Choose the default [pod security policy]({{<baseurl>}}/rancher/v2.6/en/admin-settings/pod-security-policies/) for the cluster. Please refer to the [RKE2 documentation](https://docs.rke2.io/security/policies/) on the specifications of each available policy.
|
||||
|
||||
#### Worker CIS Profile
|
||||
|
||||
Select a [CIS benchmark]({{<baseurl>}}/rancher/v2.6/en/cis-scans/) to validate the system configuration against.
|
||||
|
||||
#### Project Network Isolation
|
||||
|
||||
If your network provider allows project network isolation, you can choose whether to enable or disable inter-project communication.
|
||||
|
||||
Project network isolation is available if you are using any RKE2 network plugin that supports the enforcement of Kubernetes network policies, such as Canal.
|
||||
|
||||
#### SELinux
|
||||
|
||||
Option to enable or disable [SELinux](https://docs.rke2.io/security/selinux) support.
|
||||
|
||||
#### CoreDNS
|
||||
|
||||
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/#coredns) for additional CoreDNS configurations.
|
||||
|
||||
#### NGINX Ingress
|
||||
|
||||
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/#nginx-ingress-controller) for additional configuration options.
|
||||
|
||||
Refer to the [RKE2 documentation](https://docs.rke2.io/networking/#nginx-ingress-controller) for additional configuration options.
|
||||
|
||||
#### Metrics Server
|
||||
|
||||
Option to enable or disable [Metrics Server]({{<baseurl>}}/rke/latest/en/config-options/add-ons/metrics-server/).
|
||||
|
||||
Each cloud provider capable of launching a cluster using RKE2 can collect metrics and monitor for your cluster nodes. Enable this option to view your node metrics from your cloud provider's portal.
|
||||
|
||||
### Add-On Config
|
||||
|
||||
Additional Kubernetes manifests, managed as an [Add-on](https://kubernetes.io/docs/concepts/cluster-administration/addons/), to apply to the cluster on startup. Refer to the [RKE2 documentation](https://docs.rke2.io/helm/#automatically-deploying-manifests-and-helm-charts) for details.
|
||||
|
||||
### Agent Environment Vars
|
||||
|
||||
Option to set environment variables for [Rancher agents](https://rancher.com/docs/rancher/v2.6/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. Refer to the [RKE2 documentation](https://docs.rke2.io/install/install_options/linux_agent_config/) for more details.
|
||||
|
||||
### etcd
|
||||
|
||||
#### Automatic Snapshots
|
||||
|
||||
Option to enable or disable recurring etcd snapshots. If enabled, users have the option to configure the frequency of snapshots. For details, refer to the [RKE2 documentation](https://docs.rke2.io/backup_restore/#creating-snapshots). Note that with RKE2, snapshots are stored on each etcd node. This varies from RKE1 which only stores one snapshot per cluster.
|
||||
|
||||
#### Metrics
|
||||
|
||||
Option to choose whether to expose etcd metrics to the public or only within the cluster.
|
||||
|
||||
### Networking
|
||||
|
||||
#### Cluster CIDR
|
||||
|
||||
IPv4 and/or IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
To configure [dual-stack](https://docs.rke2.io/install/network_options/#dual-stack-configuration) mode, enter a valid IPv4/IPv6 CIDR. For example `10.42.0.0/16,2001:cafe:42:0::/56`.
|
||||
|
||||
[Additional configuration](#dual-stack-additional-config) is required when using `cilium` or `multus,cilium` as your [container network](#container-network) interface provider.
|
||||
|
||||
#### Service CIDR
|
||||
|
||||
IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16).
|
||||
|
||||
##### Dual-stack Networking
|
||||
|
||||
To configure [dual-stack](https://docs.rke2.io/install/network_options/#dual-stack-configuration) mode, enter a valid IPv4/IPv6 CIDR. For example `10.42.0.0/16,2001:cafe:42:0::/56`.
|
||||
|
||||
[Additional configuration](#dual-stack-additional-config) is required when using `cilium ` or `multus,cilium` as your [container network](#container-network) interface provider.
|
||||
|
||||
#### Cluster DNS
|
||||
|
||||
IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10).
|
||||
|
||||
#### Cluster Domain
|
||||
|
||||
Select the domain for the cluster. The default is `cluster.local`.
|
||||
|
||||
#### NodePort Service Port Range
|
||||
|
||||
Option to change the range of ports that can be used for [NodePort services](https://kubernetes.io/docs/concepts/services-networking/service/#nodeport). The default is `30000-32767`.
|
||||
|
||||
#### TLS Alternate Names
|
||||
|
||||
Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert.
|
||||
|
||||
#### Authorized Cluster Endpoint
|
||||
|
||||
Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher.
|
||||
|
||||
This is enabled by default in Rancher-launched Kubernetes clusters, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates.
|
||||
|
||||
For more detail on how an authorized cluster endpoint works and why it is used, refer to the [architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint)
|
||||
|
||||
We recommend using a load balancer with the authorized cluster endpoint. For details, refer to the [recommended architecture section.]({{<baseurl>}}/rancher/v2.6/en/overview/architecture-recommendations/#architecture-for-an-authorized-cluster-endpoint)
|
||||
|
||||
### Registries
|
||||
|
||||
Select the image repository to pull Rancher images from. For more details and configuration options, see the [RKE2 documentation](https://docs.rke2.io/install/containerd_registry_configuration/).
|
||||
|
||||
### Upgrade Strategy
|
||||
|
||||
#### Control Plane Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Worker Concurrency
|
||||
|
||||
Select how many nodes can be upgraded at the same time. Can be a fixed number or percentage.
|
||||
|
||||
#### Drain Nodes (Control Plane)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
#### Drain Nodes (Worker Nodes)
|
||||
|
||||
Option to remove all pods from the node prior to upgrading.
|
||||
|
||||
### Advanced
|
||||
|
||||
Option to set kubelet options for different nodes. For available options, refer to the [Kubernetes documentation](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/).
|
||||
|
||||
# Cluster Config File
|
||||
|
||||
Instead of using the Rancher UI forms to choose Kubernetes options for the cluster, advanced users can create an RKE2 config file. Using a config file allows you to set any of the [options](https://docs.rke2.io/install/install_options/install_options) available in an RKE2 installation.
|
||||
|
||||
To edit an RKE2 config file directly from the Rancher UI, click **Edit as YAML**.
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Syncing
|
||||
weight: 10
|
||||
---
|
||||
|
||||
Syncing is the feature for EKS and GKE clusters that causes Rancher to update the clusters' values so they are up to date with their corresponding cluster object in the hosted Kubernetes provider. This enables Rancher to not be the sole owner of a hosted cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other.
|
||||
|
||||
### How it works
|
||||
|
||||
There are two fields on the Rancher Cluster object that must be understood to understand how syncing works:
|
||||
|
||||
1. The config object for the cluster, located on the Spec of the Cluster:
|
||||
|
||||
* For EKS, the field is called EKSConfig
|
||||
* For GKE, the field is called GKEConfig
|
||||
|
||||
2. The UpstreamSpec object
|
||||
|
||||
* For EKS, this is located on the EKSStatus field on the Status of the Cluster.
|
||||
* For GKE, this is located on the GKEStatus field on the Status of the Cluster.
|
||||
|
||||
The struct types that define these objects can be found in their corresponding operator projects:
|
||||
|
||||
* [eks-operator](https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go)
|
||||
* [gke-operator](https://github.com/rancher/gke-operator/blob/master/pkg/apis/gke.cattle.io/v1/types.go)
|
||||
|
||||
All fields with the exception of the cluster name, the location (region or zone), Imported, and the cloud credential reference, are nillable on this Spec object.
|
||||
|
||||
The EKSConfig or GKEConfig represents desired state for its non-nil values. Fields that are non-nil in the config object can be thought of as “managed". When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher.
|
||||
|
||||
UpstreamSpec represents the cluster as it is in the hosted Kubernetes provider and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed, Rancher checks if the cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig or GKEConfig are overwritten with their corresponding value from the recently updated UpstreamSpec.
|
||||
|
||||
The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig or GKEConfig. This is what is displayed in the UI.
|
||||
|
||||
If Rancher and another source attempt to update a cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. To use EKS as an example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioned case.
|
||||
Reference in New Issue
Block a user