Merge branch 'rancher:main' into rke2-restore

This commit is contained in:
Sunil Singh
2024-05-08 12:31:21 -07:00
committed by GitHub
332 changed files with 30315 additions and 1281 deletions
@@ -6,11 +6,11 @@ title: Node Drivers
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-node-drivers"/>
</head>
Node drivers are used to provision hosts, which Rancher uses to launch and manage Kubernetes clusters. A node driver is the same as a [Docker Machine driver](https://docs.docker.com/machine/drivers/). The availability of which node driver to display when creating node templates is defined based on the node driver's status. Only `active` node drivers will be displayed as an option for creating node templates. By default, Rancher is packaged with many existing Docker Machine drivers, but you can also create custom node drivers to add to Rancher.
A node driver is the same as a [Docker Machine driver](https://docs.docker.com/machine/drivers/). Node drivers are used to provision hosts, which Rancher uses to launch and manage Kubernetes clusters. By default, Rancher is packaged with many node drivers, but you can also create and add custom node drivers to Rancher.
If there are specific node drivers that you don't want to show to your users, you would need to de-activate these node drivers.
Only `Active` node drivers are displayed in the Rancher UI when you create node templates. If there are specific node drivers that you don't want to show your users, you must deactivate these node drivers.
#### Managing Node Drivers
## Managing Node Drivers
:::note Prerequisites:
@@ -21,17 +21,31 @@ To create, edit, or delete drivers, you need _one_ of the following permissions:
:::
## Activating/Deactivating Node Drivers
### Activating/Deactivating Node Drivers
By default, Rancher only activates drivers for the most popular cloud providers, Amazon EC2, Azure, DigitalOcean and vSphere. If you want to show or hide any node driver, you can change its status.
By default, Rancher only activates drivers for the most popular cloud providers, such as Amazon EC2, Azure, DigitalOcean, Linode and vSphere. If you want to show or hide any node driver, you can change its status.
1. In the upper left corner, click **☰ > Cluster Management**.
1. In the left navigation menu, click **Drivers**.
1. On the **Node Drivers** tab, select the driver that you wish to activate or deactivate and click **⋮ > Activate** or **⋮ > Deactivate**.
2. In the left navigation menu, click **Drivers**.
:::danger
2. On the **Node Drivers** tab, select the driver that you wish to activate or deactivate and click **⋮ > Activate** or **⋮ > Deactivate**.
You can lose access to clusters after deactivating a node driver.
## Adding Custom Node Drivers
Deactivating a node driver doesn't just affect its visibility in the Rancher UI. When you deactivate or delete a node driver, any nodes deployed with that driver become inaccessible.
For example, if you deactivate a vSphere node driver to hide it in the UI, and you have a vSphere cluster that was deployed with that driver, the initial node in the cluster will fail, and the entire cluster will become inaccessible. Attempts to delete the vSphere nodes will fail, with nodes stuck in an extended `Removing` state.
Before you deactivate a node driver, make sure that it has no associated clusters. One way to check is to see if the respective platform for a driver is listed among your clusters:
1. In the upper left corner, click **☰ > Cluster Management**.
1. Select **Clusters**.
1. Check the **Provider** column of the table for instances of the node driver you are deactivating.
:::
### Adding Custom Node Drivers
If you want to use a node driver that Rancher doesn't support out-of-the-box, you can add that provider's driver in order to start using them to create node templates and eventually node pools for your Kubernetes cluster.
@@ -40,6 +54,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
1. On **Node Drivers** tab, click **Add Node Driver**.
1. Complete the **Add Node Driver** form. Then click **Create**.
### Developing your own node driver
### Developing Your Own Node Drivers
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
@@ -1,12 +1,12 @@
---
title: Migrating vSphere In-tree to Out-of-tree
title: Migrating VMware vSphere In-tree to Out-of-tree
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/migrate-to-an-out-of-tree-cloud-provider/migrate-to-out-of-tree-vsphere"/>
</head>
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
Kubernetes is moving away from maintaining cloud providers in-tree. VMware vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
This page covers how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration.
@@ -1,12 +1,12 @@
---
title: Setting Up an In-tree vSphere Cloud Provider
title: Setting Up an In-tree VMware vSphere Cloud Provider
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-in-tree-vsphere"/>
</head>
To set up the in-tree vSphere cloud provider, follow these steps while creating the vSphere cluster in Rancher:
To set up the in-tree VMware vSphere cloud provider, follow these steps while creating the vSphere cluster in Rancher:
1. Set **Cloud Provider** option to `Custom` or `Custom (In-Tree)`.
1. Click on **Edit as YAML**
@@ -1,12 +1,12 @@
---
title: Setting Up an Out-of-tree vSphere Cloud Provider
title: Setting Up an Out-of-tree VMware vSphere Cloud Provider
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-out-of-tree-vsphere"/>
</head>
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
Kubernetes is moving away from maintaining cloud providers in-tree. VMware vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
This page covers how to install the Cloud Provider Interface (CPI) and Cloud Storage Interface (CSI) plugins after setting up a cluster.
@@ -25,7 +25,7 @@ Using the vSphere out-of-tree cloud provider requires Linux nodes and is not sup
The Cloud Provider Interface (CPI) should be installed first before installing the Cloud Storage Interface (CSI).
### 1. Create a vSphere cluster
### 1. Create a VMware vSphere cluster
1. Click **☰ > Cluster Management**.
1. On the **Clusters** page, click **Create**.
@@ -86,7 +86,7 @@ Some Linux operating systems accept multiple domain names separated by spaces. H
:::
### Rancher on vSphere with ESXi 6.7u2 and above
### Rancher on VMware vSphere with ESXi 6.7u2 and above
If you are using Rancher on VMware vSphere with ESXi 6.7u2 or later with Red Hat Enterprise Linux 8.3, CentOS 8.3, or SUSE Enterprise Linux 15 SP2 or later, it is necessary to disable the `vmxnet3` virtual network adapter hardware offloading feature. Failure to do so will result in all network connections between pods on different cluster nodes to fail with timeout errors. All connections from Windows pods to critical services running on Linux nodes, such as CoreDNS, will fail as well. It is also possible that external connections may fail. This issue is the result of Linux distributions enabling the hardware offloading feature in `vmxnet3` and a bug in the `vmxnet3` hardware offloading feature that results in the discarding of packets for guest overlay traffic. To address this issue, it is necessary disable the `vmxnet3` hardware offloading feature. This setting does not survive reboot, so it is necessary to disable on every boot. The recommended course of action is to create a systemd unit file at `/etc/systemd/system/disable_hw_offloading.service`, which disables the `vmxnet3` hardware offloading feature on boot. A sample systemd unit file which disables the `vmxnet3` hardware offloading feature is as follows. Note that `<VM network interface>` must be customized to the host `vmxnet3` network interface, e.g., `ens192`:
@@ -21,11 +21,62 @@ The `cattle-cluster-agent` is used to connect to the Kubernetes API of [Rancher
The `cattle-node-agent` is used to interact with nodes in a [Rancher Launched Kubernetes](launch-kubernetes-with-rancher.md) cluster when performing cluster operations. Examples of cluster operations are upgrading Kubernetes version and creating/restoring etcd snapshots. The `cattle-node-agent` is deployed using a DaemonSet resource to make sure it runs on every node. The `cattle-node-agent` is used as fallback option to connect to the Kubernetes API of [Rancher Launched Kubernetes](launch-kubernetes-with-rancher.md) clusters when `cattle-cluster-agent` is unavailable.
### Requests
The `cattle-cluster-agent` pod does not define the default CPU and memory request values. As a baseline, we recommend setting the CPU request at `50m` and memory request at `100Mi`. However, it is important that you assess your use case appropriately and that you allocate the correct resources to your cluster for your needs.
To configure request values through the UI:
<Tabs groupId="k8s-distro">
<TabItem value="RKE">
1. When you [create](./launch-kubernetes-with-rancher.md) or edit an existing cluster, go to the **Cluster Options** section.
1. Expand the **Cluster Configuration** subsection.
1. Configure your request values using the **CPU Requests** and **Memory Requests** fields as needed.
</TabItem>
<TabItem value="RKE2/K3s">
1. When you [create](./launch-kubernetes-with-rancher.md) or edit an existing cluster, go to the **Cluster Configuration**.
1. Select the **Cluster Agent** subsection.
1. Configure your request values using the **CPU Reservation** and **Memory Reservation** fields as needed.
</TabItem>
</Tabs>
If you prefer to configure via YAML, add the following snippet to your configuration file:
<Tabs groupId="k8s-distro">
<TabItem value="RKE">
```yaml
cluster_agent_deployment_customization:
override_resource_requirements:
requests:
cpu: 50m
memory: 100Mi
```
</TabItem>
<TabItem value="RKE2/K3s">
```yaml
spec:
clusterAgentDeploymentCustomization:
overrideResourceRequirements:
requests:
cpu: 50m
memory: 100Mi
```
</TabItem>
</Tabs>
### Scheduling rules
The `cattle-cluster-agent` uses either a fixed set of tolerations, or dynamically-added tolerations based on taints applied to the control plane nodes. This structure allows [Taint based Evictions](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#taint-based-evictions) to work properly for `cattle-cluster-agent`.
The `cattle-cluster-agent` uses either a fixed set of tolerations, or dynamically-added tolerations based on taints applied to the control plane nodes. This structure allows [Taint based Evictions](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#taint-based-evictions) to work properly for `cattle-cluster-agent`.
If control plane nodes are present in the cluster, the default tolerations will be replaced with tolerations matching the taints on the control plane nodes. The default set of tolerations are described below.
If control plane nodes are present in the cluster, the default tolerations will be replaced with tolerations matching the taints on the control plane nodes. The default set of tolerations are described below.
| Component | nodeAffinity nodeSelectorTerms | nodeSelector | Tolerations |
| ---------------------- | ------------------------------------------ | ------------ | ------------------------------------------------------------------------------ |
@@ -1,5 +1,5 @@
---
title: Creating a vSphere Virtual Machine Template
title: Creating a VMware vSphere Virtual Machine Template
---
<head>
@@ -1,12 +1,12 @@
---
title: Creating Credentials in the vSphere Console
title: Creating Credentials in the VMware vSphere Console
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/vsphere/create-credentials"/>
</head>
This section describes how to create a vSphere username and password. You will need to provide these vSphere credentials to Rancher, which allows Rancher to provision resources in vSphere.
This section describes how to create a VMware vSphere username and password. You must provide these vSphere credentials to Rancher, which allows Rancher to provision resources in vSphere.
The following table lists the permissions required for the vSphere user account:
@@ -1,12 +1,12 @@
---
title: Provisioning Kubernetes Clusters in vSphere
title: Provisioning Kubernetes Clusters in VMware vSphere
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/vsphere/provision-kubernetes-clusters-in-vsphere"/>
</head>
In this section, you'll learn how to use Rancher to install an [RKE](https://rancher.com/docs/rke/latest/en/) Kubernetes cluster in vSphere.
In this section, you'll learn how to use Rancher to install an [RKE](https://rancher.com/docs/rke/latest/en/) Kubernetes cluster in VMware vSphere.
First, you will set up your vSphere cloud credentials in Rancher. Then you will use your cloud credentials to create a node template, which Rancher will use to provision nodes in vSphere.
@@ -17,16 +17,16 @@ For details on configuring the vSphere node template, refer to the [vSphere node
For details on configuring RKE Kubernetes clusters in Rancher, refer to the [cluster configuration reference.](../../../../../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#rke-cluster-config-file-reference)
- [Preparation in vSphere](#preparation-in-vsphere)
- [Creating a vSphere Cluster](#creating-a-vsphere-cluster)
- [Preparation in vSphere](#preparation-in-vmware-vsphere)
- [Creating a vSphere Cluster](#creating-a-vmware-vsphere-cluster)
## Preparation in vSphere
## Preparation in VMware vSphere
This section describes the requirements for setting up vSphere so that Rancher can provision VMs and clusters.
The node templates are documented and tested with the vSphere Web Services API version 6.5.
### Create Credentials in vSphere
### Create Credentials in VMware vSphere
Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. When you set up a node template, the template will need to use these vSphere credentials.
@@ -42,7 +42,7 @@ It must be ensured that the hosts running the Rancher server are able to establi
See [Node Networking Requirements](../../../kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for a detailed list of port requirements applicable for creating nodes on an infrastructure provider.
### Valid ESXi License for vSphere API Access
### Valid ESXi License for VMware vSphere API Access
The free ESXi license does not support API access. The vSphere servers must have a valid or evaluation ESXi license.
@@ -50,7 +50,7 @@ The free ESXi license does not support API access. The vSphere servers must have
If you have a cluster with DRS enabled, setting up [VM-VM Affinity Rules](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.resmgmt.doc/GUID-7297C302-378F-4AF2-9BD6-6EDB1E0A850A.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.
## Creating a vSphere Cluster
## Creating a VMware vSphere Cluster
1. [Create your cloud credentials](#1-create-your-cloud-credentials)
2. [Create a node template with your cloud credentials](#2-create-a-node-template-with-your-cloud-credentials)
@@ -1,6 +1,6 @@
---
title: Creating a vSphere Cluster
description: Use Rancher to create a vSphere cluster. It may consist of groups of VMs with distinct properties which allow for fine-grained control over the sizing of nodes.
title: Creating a VMware vSphere Cluster
description: Use Rancher to create a VMware vSphere cluster. It may consist of groups of VMs with distinct properties which allow for fine-grained control over the sizing of nodes.
---
<head>
@@ -9,13 +9,13 @@ description: Use Rancher to create a vSphere cluster. It may consist of groups o
import YouTube from '@site/src/components/YouTube'
By using Rancher with vSphere, you can bring cloud operations on-premises.
By using Rancher with VMware vSphere, you can bring cloud operations on-premises.
Rancher can provision nodes in vSphere and install Kubernetes on them. When creating a Kubernetes cluster in vSphere, Rancher first provisions the specified number of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them.
A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for each Kubernetes role.
## vSphere Enhancements in Rancher v2.3
## VMware vSphere Enhancements in Rancher v2.3
The vSphere node templates have been updated, allowing you to bring cloud operations on-premises with the following enhancements:
@@ -33,7 +33,7 @@ It is not recommended to enable node auto-replace on a node pool of master nodes
Node templates for vSphere have been updated so that when you create a node template with your vSphere credentials, the template is automatically populated with the same options for provisioning VMs that you have access to in the vSphere console.
For the fields to be populated, your setup needs to fulfill the [prerequisites.](provision-kubernetes-clusters-in-vsphere.md#preparation-in-vsphere)
For the fields to be populated, your setup needs to fulfill the [prerequisites.](provision-kubernetes-clusters-in-vsphere.md#preparation-in-vmware-vsphere)
### More Supported Operating Systems
@@ -45,7 +45,7 @@ In this YouTube video, we demonstrate how to set up a node template with the new
<YouTube id="dPIwg6x1AlU"/>
## Creating a vSphere Cluster
## Creating a VMware vSphere Cluster
In [this section,](provision-kubernetes-clusters-in-vsphere.md) you'll learn how to use Rancher to install an [RKE](https://rancher.com/docs/rke/latest/en/) Kubernetes cluster in vSphere.
@@ -53,7 +53,7 @@ In [this section,](provision-kubernetes-clusters-in-vsphere.md) you'll learn how
For an example of how to provision storage in vSphere using Rancher, refer to [this section.](../../../manage-clusters/provisioning-storage-examples/vsphere-storage.md) In order to dynamically provision storage in vSphere, the vSphere provider must be enabled. Refer to [in-tree vSphere config](../../../kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-in-tree-vsphere.md) and [out-of-tree vSphere config](../../../kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-out-of-tree-vsphere.md).
## Enabling the vSphere Cloud Provider
## Enabling the VMware vSphere Cloud Provider
When a cloud provider is set up in Rancher, the Rancher server can automatically provision new infrastructure for the cluster, including new nodes or persistent storage devices.
@@ -67,7 +67,7 @@ In [Rancher Launched Kubernetes clusters](../../launch-kubernetes-with-rancher/l
### hostPath Volumes
Before you create a hostPath volume, you need to set up an [extra_bind](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/#extra-binds/) in your cluster configuration. This will mount the path as a volume in your kubelets, which can then be used for hostPath volumes in your workloads.
### Migrating vSphere Cloud Provider from In-tree to Out-of-tree
### Migrating VMware vSphere Cloud Provider from In-tree to Out-of-tree
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
@@ -1,12 +1,12 @@
---
title: vSphere Storage
title: VMware vSphere Storage
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/provisioning-storage-examples/vsphere-storage"/>
</head>
To provide stateful workloads with vSphere storage, we recommend creating a vSphereVolume StorageClass. This practice dynamically provisions vSphere storage when workloads request volumes through a PersistentVolumeClaim.
To provide stateful workloads with VMware vSphere storage, we recommend creating a vSphereVolume StorageClass. This practice dynamically provisions vSphere storage when workloads request volumes through a PersistentVolumeClaim.
In order to dynamically provision storage in vSphere, the vSphere provider must be enabled. See the following pages for more: [Out-of-tree vSphere](../../../new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-out-of-tree-vsphere.md) and [in-tree vSphere](../../../new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/configure-in-tree-vsphere.md).
@@ -35,7 +35,7 @@ The following steps can also be performed using the `kubectl` command line tool.
5. Optionally, specify additional properties for this storage class under **Parameters**. Refer to the [vSphere storage documentation](https://github.com/vmware-archive/vsphere-storage-for-kubernetes/blob/master/documentation/storageclass.md) for details.
5. Click **Create**.
### Creating a Workload with a vSphere Volume
### Creating a Workload with a VMware vSphere Volume
1. In the left navigation bar, click **Workload**.
1. Click **Create**.