mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-09-25 04:28:15 +00:00
Revise vSphere cloud provider docs in v2.x section #3090
This commit is contained in:
Catherine Luse
@@ -66,6 +66,13 @@ In [Rancher Launched Kubernetes clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-
|
||||
### hostPath Volumes
|
||||
Before you create a hostPath volume, you need to set up an [extra_bind]({{<baseurl>}}/rke/latest/en/config-options/services/services-extras/#extra-binds/) in your cluster configuration. This will mount the path as a volume in your kubelets, which can then be used for hostPath volumes in your workloads.
|
||||
|
||||
### Migrating vSphere Cloud Provider from In-tree to Out-of-tree
|
||||
|
||||
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
|
||||
|
||||
For instructions on how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/out-of-tree)
|
||||
|
||||
|
||||
### Related Links
|
||||
|
||||
- [Kubernetes Documentation: Storage](https://kubernetes.io/docs/concepts/storage/)
|
||||
|
||||
+6
-16
@@ -2,24 +2,14 @@
|
||||
title: Setting up the vSphere Cloud Provider
|
||||
weight: 4
|
||||
---
|
||||
In this section, you'll learn how to set up a vSphere cloud provider for a Rancher managed RKE Kubernetes cluster in vSphere.
|
||||
|
||||
In this section, you'll learn how to set up the vSphere cloud provider for a Rancher managed RKE Kubernetes cluster in vSphere.
|
||||
# In-tree Cloud Provider
|
||||
|
||||
Follow these steps while creating the vSphere cluster in Rancher:
|
||||
To use the in-tree vSphere cloud provider, you will need to use an RKE configuration option. For details, refer to [this page.](./in-tree)
|
||||
|
||||
1. Set **Cloud Provider** option to `Custom`.
|
||||
# Out-of-tree Cloud Provider
|
||||
|
||||
{{< img "/img/rancher/vsphere-node-driver-cloudprovider.png" "vsphere-node-driver-cloudprovider">}}
|
||||
_Available as of v2.5+_
|
||||
|
||||
1. Click on **Edit as YAML**
|
||||
1. Insert the following structure to the pre-populated cluster YAML. As of Rancher v2.3+, this structure must be placed under `rancher_kubernetes_engine_config`. In versions before v2.3, it has to be defined as a top-level field. Note that the `name` *must* be set to `vsphere`.
|
||||
|
||||
```yaml
|
||||
rancher_kubernetes_engine_config: # Required as of Rancher v2.3+
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
[Insert provider configuration]
|
||||
```
|
||||
|
||||
Rancher uses RKE (the Rancher Kubernetes Engine) to provision Kubernetes clusters. Refer to the [vSphere configuration reference in the RKE documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/) for details about the properties of the `vsphereCloudProvider` directive.
|
||||
To set up the out-of-tree vSphere cloud provider, you will need to install Helm charts from the Rancher marketplace. For details, refer to [this page.](./out-of-tree)
|
||||
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
---
|
||||
title: How to Configure In-tree vSphere Cloud Provider
|
||||
shortTitle: In-tree Cloud Provider
|
||||
weight: 10
|
||||
---
|
||||
|
||||
To set up the in-tree vSphere cloud provider, follow these steps while creating the vSphere cluster in Rancher:
|
||||
|
||||
1. Set **Cloud Provider** option to `Custom` or `Custom (In-Tree)`.
|
||||
|
||||
{{< img "/img/rancher/vsphere-node-driver-cloudprovider.png" "vsphere-node-driver-cloudprovider">}}
|
||||
|
||||
1. Click on **Edit as YAML**
|
||||
1. Insert the following structure to the pre-populated cluster YAML. This structure must be placed under `rancher_kubernetes_engine_config`. Note that the `name` *must* be set to `vsphere`.
|
||||
|
||||
```yaml
|
||||
rancher_kubernetes_engine_config:
|
||||
cloud_provider:
|
||||
name: vsphere
|
||||
vsphereCloudProvider:
|
||||
[Insert provider configuration]
|
||||
```
|
||||
|
||||
Rancher uses RKE (the Rancher Kubernetes Engine) to provision Kubernetes clusters. Refer to the [vSphere configuration reference in the RKE documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/) for details about the properties of the `vsphereCloudProvider` directive.
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
---
|
||||
title: How to Configure Out-of-tree vSphere Cloud Provider
|
||||
shortTitle: Out-of-tree Cloud Provider
|
||||
weight: 10
|
||||
---
|
||||
_Available as of v2.5+_
|
||||
|
||||
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
|
||||
|
||||
This page covers how to install the Cloud Provider Interface (CPI) and Cloud Storage Interface (CSI) plugins after bringing up a cluster.
|
||||
|
||||
# Prerequisites
|
||||
|
||||
The vSphere version must be 7.0u1 or higher.
|
||||
|
||||
The Kubernetes version must be 1.19 or higher.
|
||||
|
||||
Using the vSphere out-of-tree cloud provider requires Linux nodes and is not supported on Windows.
|
||||
|
||||
# Installation
|
||||
|
||||
The Cloud Provider Interface (CPI) should be installed first before installing the Cloud Storage Interface (CSI).
|
||||
|
||||
### 1. Create a vSphere cluster
|
||||
|
||||
1. On the Clusters page, click on **Add Cluster** and select the **vSphere** option or **Existing Nodes** option.
|
||||
1. Under **Cluster Options > Cloud Provider** select **External (Out-of-tree)**. This sets the cloud provider option on the Kubernetes cluster to `external` which sets your Kubernetes cluster up to be configured with an out-of-tree cloud provider.
|
||||
1. Finish creating your cluster.
|
||||
|
||||
### 2. Install the CPI plugin
|
||||
|
||||
1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
1. Select the **vSphere CPI** chart. Fill out the required vCenter details.
|
||||
1. vSphere CPI initializes all nodes with ProviderID which is needed by the vSphere CSI driver. Check if all nodes are initialized with the ProviderID before installing CSI driver with the following command:
|
||||
|
||||
```
|
||||
kubectl describe nodes | grep "ProviderID"
|
||||
```
|
||||
### 3. Installing the CSI plugin
|
||||
|
||||
1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
1. Select the **vSphere CSI** chart. Fill out the required vCenter details.
|
||||
2. Set **Enable CSI Migration** to **false**.
|
||||
3. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. Fill out the details for the StorageClass and launch the chart.
|
||||
|
||||
# Using the CSI driver for provisioning volumes
|
||||
The CSI chart by default creates a storageClass.
|
||||
|
||||
If that option was not selected while launching the chart, create a storageClass with the `csi.vsphere.vmware.com` as the provisioner.
|
||||
|
||||
All volumes provisioned using this StorageClass will get provisioned by the CSI driver.
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
---
|
||||
title: Migrating vSphere In-tree Volumes to CSI
|
||||
weight: 5
|
||||
---
|
||||
_Available as of v2.5+_
|
||||
|
||||
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
|
||||
|
||||
This page covers how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration.
|
||||
It follows the steps provided in the official [vSphere migration documentation](https://vsphere-csi-driver.sigs.k8s.io/features/vsphere_csi_migration.html) and provides the steps to be performed in Rancher.
|
||||
### Cloud-config Format Limitation
|
||||
Existing volumes that were provisioned using the following cloud-config format will NOT get migrated due to an existing bug in vsphere CSI.
|
||||
If the cloud-config has this format for datastore and resource pool path, vsphere CSI driver cannot recognize it:
|
||||
```yaml
|
||||
default-datastore: </datacenter>/datastore/<default-datastore-name>
|
||||
resourcepool-path: "</datacenter>/host/<cluster-name>/Resources/<resource-pool-name>"
|
||||
```
|
||||
Volumes provisioned with the in-tree provider using the following format will get migrated correctly:
|
||||
```yaml
|
||||
default-datastore: <default-datastore-name>
|
||||
resourcepool-path: "<cluster-name>/Resources/<resource-pool-name>"
|
||||
```
|
||||
Upstream bug: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/628
|
||||
Rancher issue tracking this bug: https://github.com/rancher/rancher/issues/31105
|
||||
# Prerequisites
|
||||
- vSphere CSI Migration requires vSphere 7.0u1. In order to be able to manage existing in-tree vSphere volumes, upgrade vSphere to 7.0u1.
|
||||
- The Kubernetes version must be 1.19 or higher.
|
||||
# Migration
|
||||
### 1. Install the CPI plugin
|
||||
Before installing CPI, we need to taint all nodes with `node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule`.
|
||||
This can be done by running the following commands:
|
||||
```
|
||||
curl -O https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh
|
||||
```
|
||||
Or:
|
||||
```
|
||||
wget https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh
|
||||
chmod +x taints.sh
|
||||
./taints.sh <path to kubeconfig if running the command outside the cluster>
|
||||
```
|
||||
|
||||
Once all nodes are tainted by the running the script, launch the Helm vSphere CPI chart.
|
||||
|
||||
1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
2. Select the **vSphere CPI** chart.
|
||||
3. Fill out the required vCenter details and click **Launch**.
|
||||
|
||||
vSphere CPI initializes all nodes with ProviderID, which is needed by the vSphere CSI driver.
|
||||
Check if all nodes are initialized with the ProviderID with the following command:
|
||||
```
|
||||
kubectl describe nodes | grep "ProviderID"
|
||||
```
|
||||
|
||||
### 2. Install the CSI driver
|
||||
|
||||
1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.**
|
||||
1. Select the **vSphere CSI** chart.
|
||||
1. Fill out the required vCenter details and click **Launch**.
|
||||
1. Set **Enable CSI Migration** to **true**.
|
||||
1. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. You can provide the URL of the datastore to be used for CSI volume provisioning while creating this StorageClass. The datastore URL can be found in the vSphere client by selecting the datastore and going to the Summary tab. Fill out the details for the StorageClass and click **Launch**.
|
||||
### 3. Edit the cluster to enable CSI migration feature flags
|
||||
1. While editing the cluster, if the Kubernetes version is less than 1.19, select Kubernetes version 1.19 or higher from the **Kubernetes Version** dropdown.
|
||||
2. For enabling feature flags, click on "Edit as YAML", and add the following under kube-controller and kubelet:
|
||||
```yaml
|
||||
extra_args:
|
||||
feature-gates: "CSIMigration=true,CSIMigrationvSphere=true"
|
||||
```
|
||||
### 4. Drain worker nodes
|
||||
Worker nodes must be drained during the upgrade before changing the kubelet and kube-controller-manager args.
|
||||
1. Click **Edit as Form** and then click on "Advanced Options."
|
||||
1. Set the field **Maximum Worker Nodes Unavailable** to count of 1.
|
||||
1. To drain the nodes during upgrade, select **Drain Nodes > Yes**.
|
||||
1. Set **Force** and **Delete Local Data** to **true**.
|
||||
1. Click **Save** to upgrade the cluster.
|
||||
-9
@@ -1,9 +0,0 @@
|
||||
---
|
||||
title: Migrating vSphere In-tree Volumes to CSI
|
||||
weight: 5
|
||||
---
|
||||
_Available as of v2.5.6_
|
||||
|
||||
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
|
||||
|
||||
For instructions on how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/vsphere-volume-migration)
|
||||
-9
@@ -1,9 +0,0 @@
|
||||
---
|
||||
title: vSphere Out-of-tree Cloud Provider
|
||||
weight: 4
|
||||
---
|
||||
_Available as of v2.5.6_
|
||||
|
||||
Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins.
|
||||
|
||||
For instructions on how to install the CPI and CSI plugins after bringing up a cluster, refer to [this page.]({{<baseurl>}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere)
|
||||
Reference in New Issue
Block a user