diff --git a/content/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/chart-options/_index.md b/content/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/chart-options/_index.md index ee179f1b218..2bf2f7681fe 100644 --- a/content/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/chart-options/_index.md +++ b/content/rancher/v2.0-v2.4/en/installation/install-rancher-on-k8s/chart-options/_index.md @@ -57,8 +57,8 @@ For information on enabling experimental features, refer to [this page.]({{ Edit as YAML** and restarting the worker nodes. + +```yaml +services: + kubelet: + extra_binds: + - '/lib/modules:/lib/modules:ro' +``` + +For more information about the `extra_binds` directive, refer to [this section.]({{}}/rke/latest/en/config-options/services/services-extras/#extra-binds) + +# Installing the ceph-csi driver on an RKE2 cluster + +> **Note:** These steps are needed for dynamic RBD provisioning only. + +For more information about the `ceph-csi-rbd` chart, refer to [this page.](https://github.com/ceph/ceph-csi/blob/devel/charts/ceph-csi-rbd/README.md) + +To get details about your SES cluster, run: + +``` +ceph mon dump +``` + +Read its output: + +``` +dumped monmap epoch 3 +epoch 3 +fsid 79179d9d-98d8-4976-ab2e-58635caa7235 +last_changed 2021-02-11T10:56:42.110184+0000 +created 2021-02-11T10:56:22.913321+0000 +min_mon_release 15 (octopus) +0: [v2:10.85.8.118:3300/0,v1:10.85.8.118:6789/0] mon.a +1: [v2:10.85.8.123:3300/0,v1:10.85.8.123:6789/0] mon.b +2: [v2:10.85.8.124:3300/0,v1:10.85.8.124:6789/0] mon.c +``` + +Later you'll need the fsid and mon addresses values. + +# Install the ceph-csi Driver Using Helm + +Run these commands: + +``` +helm repo add ceph-csi https://ceph.github.io/csi-charts +helm repo update +helm search repo ceph-csi -l +helm inspect values ceph-csi/ceph-csi-rbd > ceph-csi-rbd-values.yaml +``` + +Modify the `ceph-csi-rbd-values.yaml` file and keep there only the required changes: + +```yaml +# ceph-csi-rbd-values.yaml +csiConfig: + - clusterID: "79179d9d-98d8-4976-ab2e-58635caa7235" + monitors: + - "10.85.8.118:6789" + - "10.85.8.123:6789" + - "10.85.8.124:6789" +provisioner: + name: provisioner + replicaCount: 2 +``` + +Make sure the ceph monitors are reachable from the RKE2 cluster, for example, by ping. + +``` +kubectl create namespace ceph-csi-rbd +helm install --namespace ceph-csi-rbd ceph-csi-rbd ceph-csi/ceph-csi-rbd --values ceph-csi-rbd-values.yaml +kubectl rollout status deployment ceph-csi-rbd-provisioner -n ceph-csi-rbd +helm status ceph-csi-rbd -n ceph-csi-rbd +``` + +in case you'd like to modify the configuration directly via Helm, you may adapt the `ceph-csi-rbd-values.yaml` file and call: + +``` +helm upgrade \ + --namespace ceph-csi-rbd ceph-csi-rbd ceph-csi/ceph-csi-rbd --values ceph-csi-rbd-values.yaml +``` + +# Creating RBD Ceph Resources + +``` +# Create a ceph pool: +ceph osd pool create myPool 64 64 + +# Create a block device pool: +rbd pool init myPool + +# Create a block device image: +rbd create -s 2G myPool/image + +# Create a block device user and record the key: +ceph auth get-or-create-key client.myPoolUser mon "allow r" osd "allow class-read object_prefix rbd_children, allow rwx pool=myPool" | tr -d '\n' | base64 +QVFDZ0R5VmdyRk9KREJBQTJ5b2s5R1E2NUdSWExRQndhVVBwWXc9PQ== + +# Encode the ceph user myPoolUser into a bash64 hash: +echo "myPoolUser" | tr -d '\n' | base64 +bXlQb29sVXNlcg== + +# Create a block device admin user and record the key: +ceph auth get-or-create-key client.myPoolAdmin mds 'allow *' mgr 'allow *' mon 'allow *' osd 'allow * pool=myPool' | tr -d '\n' | base64 +QVFCK0hDVmdXSjQ1T0JBQXBrc0VtcVhlZFpjc0JwaStIcmU5M3c9PQ== + +# Encode the ceph user myPoolAdmin into a bash64 hash: +echo "myPoolAdmin" | tr -d '\n' | base64 +bXlQb29sQWRtaW4= +``` +# Configure RBD Ceph Access Secrets + +### User Account + +For static RBD provisioning (the image within the ceph pool must exist), run these commands: + +``` +cat > ceph-user-secret.yaml << EOF +apiVersion: v1 +kind: Secret +metadata: + name: ceph-user + namespace: default +type: kubernetes.io/rbd +data: + userID: bXlQb29sVXNlcg== + userKey: QVFDZ0R5VmdyRk9KREJBQTJ5b2s5R1E2NUdSWExRQndhVVBwWXc9PQ== +EOF + +kubectl apply -f ceph-user-secret.yaml +``` + +### Admin Account + +For dynamic RBD provisioning (used for automatic image creation within a given ceph pool), run these commands: + +``` +cat > ceph-admin-secret.yaml << EOF +apiVersion: v1 +kind: Secret +metadata: + name: ceph-admin + namespace: default +type: kubernetes.io/rbd +data: + userID: bXlQb29sQWRtaW4= + userKey: QVFCK0hDVmdXSjQ1T0JBQXBrc0VtcVhlZFpjc0JwaStIcmU5M3c9PQ== +EOF + +kubectl apply -f ceph-admin-secret.yaml +``` + +# Create RBD Testing Resources + +### Using RBD in Pods + +``` +# pod +cat > ceph-rbd-pod-inline.yaml << EOF +apiVersion: v1 +kind: Pod +metadata: + name: ceph-rbd-pod-inline +spec: + containers: + - name: ceph-rbd-pod-inline + image: busybox + command: ["sleep", "infinity"] + volumeMounts: + - mountPath: /mnt/ceph_rbd + name: volume + volumes: + - name: volume + rbd: + monitors: + - 10.85.8.118:6789 + - 10.85.8.123:6789 + - 10.85.8.124:6789 + pool: myPool + image: image + user: myPoolUser + secretRef: + name: ceph-user + fsType: ext4 + readOnly: false +EOF + +kubectl apply -f ceph-rbd-pod-inline.yaml +kubectl get pod +kubectl exec pod/ceph-rbd-pod-inline -- df -k | grep rbd +``` + +### Using RBD in Persistent Volumes + +``` +# pod-pvc-pv +cat > ceph-rbd-pod-pvc-pv-allinone.yaml << EOF +apiVersion: v1 +kind: PersistentVolume +metadata: + name: ceph-rbd-pv +spec: + capacity: + storage: 2Gi + accessModes: + - ReadWriteOnce + rbd: + monitors: + - 10.85.8.118:6789 + - 10.85.8.123:6789 + - 10.85.8.124:6789 + pool: myPool + image: image + user: myPoolUser + secretRef: + name: ceph-user + fsType: ext4 + readOnly: false +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: ceph-rbd-pvc +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi +--- +apiVersion: v1 +kind: Pod +metadata: + name: ceph-rbd-pod-pvc-pv +spec: + containers: + - name: ceph-rbd-pod-pvc-pv + image: busybox + command: ["sleep", "infinity"] + volumeMounts: + - mountPath: /mnt/ceph_rbd + name: volume + volumes: + - name: volume + persistentVolumeClaim: + claimName: ceph-rbd-pvc +EOF + +kubectl apply -f ceph-rbd-pod-pvc-pv-allinone.yaml +kubectl get pv,pvc,pod +kubectl exec pod/ceph-rbd-pod-pvc-pv -- df -k | grep rbd +``` + +### Using RBD in Storage Classes + +This example is for dynamic provisioning. The ceph-csi driver is needed. + +``` +# pod-pvc-sc +cat > ceph-rbd-pod-pvc-sc-allinone.yaml < /root/.bashrc << EOF +export PATH=$PATH:/var/lib/rancher/rke2/bin/ +export KUBECONFIG=/etc/rancher/rke2/rke2.yaml +EOF + +cat /var/lib/rancher/rke2/server/node-token +token: K10ca0c38d4ff90d8b80319ab34092e315a8b732622e6adf97bc9eb0536REDACTED::server:ec0308000b8a6b595da000efREDACTED +``` + +### RKE2 Agent/Worker provisioning + +``` +mkdir -p /etc/rancher/rke2/ + +cat > /etc/rancher/rke2/config.yaml << EOF +server: https://10.100.103.23:9345 +token: K10ca0c38d4ff90d8b80319ab34092e315a8b732622e6adf97bc9eb0536REDACTED::server:ec0308000b8a6b595da000efREDACTED +EOF + +curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh - +systemctl enable --now rke2-agent.service +``` + +The cluster can be imported into Rancher from the Rancher UI by clicking **Global/Add Cluster > Other Cluster.** Then run the provided kubectl command on the server/master node. + +# Tested Versions + +OS for running RKE2 nodes: JeOS SLE15-SP2 with installed kernel-default-5.3.18-24.49 + +``` +kubectl version +Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-22T12:00:00Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/amd64"} +Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7+rke2r1", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-20T01:50:52Z", GoVersion:"go1.15.5b5", Compiler:"gc", Platform:"linux/amd64"} + +helm version +version.BuildInfo{Version:"3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.12"} +``` + +Kubernetes version on RKE2 cluster: v1.19.7+rke2r1 + +# Troubleshooting + +In case you are using SUSE's ceph-rook based on SES7, it might be useful to expose the monitors on hostNetwork by editing `rook-1.4.5/ceph/cluster.yaml` and setting `spec.network.hostNetwork=true`. + +Also for operating the ceph-rook cluster, it is useful to deploy a toolbox on the Kubernetes cluster where ceph-rook is provisioned by `kubectl apply -f rook-1.4.5/ceph/toolbox.yaml` Then all the ceph related commands can be executed in the toolbox pod, for example, by running `kubectl exec -it -n rook-ceph rook-ceph-tools-686d8b8bfb-2nvqp -- bash` + +Operating with the ceph - basic commands: + +``` +ceph osd pool stats +ceph osd pool delete myPool myPool --yes-i-really-really-mean-it +rbd list -p myPool +> csi-vol-f5d3766c-7296-11eb-b32a-c2b045952d38 +> image +``` + +Delete the image: `rbd rm csi-vol-f5d3766c-7296-11eb-b32a-c2b045952d38 -p myPool` + +CephFS commands in rook toolbox: + +``` +ceph -s +ceph fs ls +ceph fs fail cephfs +ceph fs rm cephfs --yes-i-really-mean-it +ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it +ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it +``` + +To prepare a cephfs filesystem, you can run this command on a rook cluster: + +``` +kubectl apply -f rook-1.4.5/ceph/filesystem.yaml +``` \ No newline at end of file diff --git a/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md b/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md new file mode 100644 index 00000000000..c3f593e6549 --- /dev/null +++ b/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md @@ -0,0 +1,49 @@ +--- +title: vSphere Out-of-tree Cloud Provider +weight: 10 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +This page covers how to install the CPI and CSI plugins after bringing up a cluster. + +# Prerequisites + +The vSphere version must be 6.7U3 or higher. + +# Installation + +The Cloud Provider Interface (CPI) should be installed first before installing the Cloud Storage Interface (CSI). + +### 1. Create a vSphere cluster + +1. On the Clusters page, click on **Add Cluster** and select the **vSphere** option. +1. Under **Cluster Options > In-Tree Cloud Provider** select **External**. +1. Click **Create**. + +### 2. Install the CPI plugin + + 1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.** +1. Select the **vsphere-cpi** chart from the **helm3-library** catalog. Fill out the required vCenter details. +1. vSphere CPI initializes all nodes with ProviderID which is needed by the vSphere CSI driver. Check if all nodes are initialized with the ProviderID before installing CSI driver with the following command: + + ``` + kubectl describe nodes | grep "ProviderID" + ``` + +### 3. Installing the CSI plugin + + 1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.** +1. Select the **vsphere-csi** chart from the **helm3-library** catalog. Fill out the required vCenter details. +2. Set **Enable CSI Migration** to **false**. +3. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. Fill out the details for the StorageClass and launch the chart. + + +# Using the CSI driver for provisioning volumes + +The CSI chart by default creates a storageClass. + +If that option was not selected while launching the chart, create a storageClass with the `csi.vsphere.vmware.com` as the provisioner. + +All volumes provisioned using this StorageClass will get provisioned by the CSI driver. \ No newline at end of file diff --git a/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md b/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md new file mode 100644 index 00000000000..e024e44a252 --- /dev/null +++ b/content/rancher/v2.5/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md @@ -0,0 +1,99 @@ +--- +title: Migrating vSphere In-tree Volumes to CSI +weight: 5 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +This page covers how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration. + +It follows the steps provided in the official [vSphere migration documentation](https://vsphere-csi-driver.sigs.k8s.io/features/vsphere_csi_migration.html) and provides the steps to be performed in Rancher. + +### Cloud-config Format Limitation + +Existing volumes that were provisioned using the following cloud-config format will NOT get migrated due to an existing bug in vsphere CSI. + +If the cloud-config has this format for datastore and resource pool path, vsphere CSI driver cannot recognize it: + +```yaml +default-datastore: /datastore/ +resourcepool-path: "/host//Resources/" +``` + +Volumes provisioned with the in-tree provider using the following format will get migrated correctly: + +```yaml +default-datastore: +resourcepool-path: "/Resources/" +``` + +Upstream bug: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/628 + +Rancher issue tracking this bug: https://github.com/rancher/rancher/issues/31105 + +# Prerequisites + +- vSphere CSI Migration requires vSphere 7.0u1. In order to be able to manage existing in-tree vSphere volumes, upgrade vSphere to 7.0u1. +- The Kubernetes version must be 1.19 or higher. + +# Migration + +### 1. Install the CPI plugin + +Before installing CPI, we need to taint all nodes with `node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule`. + +This can be done by running the following commands: + +``` +curl -O https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh +``` + +Or: + +``` +wget https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh +chmod +x taints.sh +./taints.sh +``` + +Once all nodes are tainted by the running the script, launch the Helm vSphere CPI chart. + +1. Within a project, select **Apps > Launch.** +2. Select the **vsphere-cpi** chart from the **helm3-library** catalog. +3. Fill out the required vCenter details and click **Launch**. + +vSphere CPI initializes all nodes with ProviderID, which is needed by the vSphere CSI driver. + +Check if all nodes are initialized with the ProviderID with the following command: + +``` +kubectl describe nodes | grep "ProviderID" +``` + +### 2. Install the CSI driver + +1. Within a project, select **Apps > Launch** and select the **vsphere-csi** chart from the **helm3-library** catalog. +1. Fill out the required vCenter details and click **Launch**. +1. Set **Enable CSI Migration** to **true**. +1. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. You can provide the URL of the datastore to be used for CSI volume provisioning while creating this StorageClass. The datastore URL can be found in the vSphere client by selecting the datastore and going to the Summary tab. Fill out the details for the StorageClass and click **Launch**. + +### 3. Edit the cluster to enable CSI migration feature flags + +1. While editing the cluster, if the Kubernetes version is less than 1.19, select Kubernetes version 1.19 or higher from the **Kubernetes Version** dropdown. +2. For enabling feature flags, click on "Edit as YAML", and add the following under kube-controller and kubelet: + + ```yaml + extra_args: + feature-gates: "CSIMigration=true,CSIMigrationvSphere=true" + ``` + +### 4. Drain worker nodes + +Worker nodes must be drained during the upgrade before changing the kubelet and kube-controller-manager args. + +1. Click **Edit as Form** and then click on "Advanced Options." +1. Set the field **Maximum Worker Nodes Unavailable** to count of 1. +1. To drain the nodes during upgrade, select **Drain Nodes > Yes**. +1. Set **Force** and **Delete Local Data** to **true**. +1. Click **Save** to upgrade the cluster. \ No newline at end of file diff --git a/content/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md b/content/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md index 8318aa46342..103337c9c6e 100644 --- a/content/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md +++ b/content/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md @@ -5,7 +5,6 @@ weight: 2110 aliases: - /rancher/v2.5/en/tasks/clusters/creating-a-cluster/create-cluster-eks/ --- - Amazon EKS provides a managed control plane for your Kubernetes cluster. Amazon EKS runs the Kubernetes control plane instances across multiple Availability Zones to ensure high availability. Rancher provides an intuitive user interface for managing and deploying the Kubernetes clusters you run in Amazon EKS. With this guide, you will use Rancher to quickly and easily launch an Amazon EKS Kubernetes cluster in your AWS account. For more information on Amazon EKS, see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html). - [Prerequisites in Amazon Web Services](#prerequisites-in-amazon-web-services) @@ -32,7 +31,7 @@ To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Priva ### Amazon VPC -You need to set up an Amazon VPC to launch the EKS cluster. The VPC enables you to launch AWS resources into a virtual network that you've defined. For more information, refer to the [Tutorial: Creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html). +An Amazon VPC is required to launch the EKS cluster. The VPC enables you to launch AWS resources into a virtual network that you've defined. You can set one up yourself and provide it during cluster creation in Rancher. If you do not provide one during creation, Rancher will create one. For more information, refer to the [Tutorial: Creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html). ### IAM Policies @@ -108,6 +107,157 @@ The following capabilities have been added for configuring EKS clusters in Ranch Due to the way that the cluster data is synced with EKS, if the cluster is modified from another source, such as in the EKS console, and in Rancher within five minutes, it could cause some changes to be overwritten. For information about how the sync works and how to configure it, refer to [this section](#syncing). +{{% tabs %}} +{{% tab "Rancher v2.5.6+" %}} + +### Account Access + + + +Complete each drop-down and field using the information obtained for your IAM policy. + +| Setting | Description | +| ---------- | -------------------------------------------------------------------------------------------------------------------- | +| Region | From the drop-down choose the geographical region in which to build your cluster. | +| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | + +### Service Role + + + +Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + +Service Role | Description +-------------|--------------------------- +Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. +Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +### Secrets Encryption + + + +Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html) + +### API Server Endpoint Access + + + +Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Private-only API Endpoints + +If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API. + +There are two ways to avoid this extra manual step: +- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. +- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). + +### Public Access Endpoints + + + +Optionally limit access to the public endpoint via explicit CIDR blocks. + +If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster. + +One of the following is required to enable private access: +- Rancher's IP must be part of an allowed CIDR block +- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group + +For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Subnet + + + +| Option | Description | +| ------- | ------------ | +| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. | +| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). | + + For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + +- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) +- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + +### Security Group + + + +Amazon Documentation: + +- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) +- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) +- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +### Logging + + + +Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters. + +Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation. + +For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) + +### Managed Node Groups + + + +Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters. + +For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) + +#### Bring your own launch template + +A launch template ID and version can be provided in order to easily configure the EC2 instances in a node group. If a launch template is provided, then none of the settings below will be configurable in Rancher. Therefore, using a launch template would require that all the necessary and desired settings from the list below would need to be specified in the launch template. Also note that if a launch template ID and version is provided, then only the template version can be updated. Using a new template ID would require creating a new managed node group. + +| Option | Description | Required/Optional | +| ------ | ----------- | ----------------- | +| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required | +| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional | +| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required | +| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional | +| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional | +| Instance Resource Tags | Tag each EC2 instance in the node group | Optional | + +#### Rancher-managed launch templates + +If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated. + +#### Custom AMIs + +If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative. + +If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it. + +>**Note** +>The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template. + +#### Spot instances + +Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types. + +>**Note** +>Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration. + +#### Node Group Settings + +The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created. + +| Option | Description | +| ------- | ------------ | +| Node Group Name | The name of the node group. | +| Desired ASG Size | The desired number of instances. | +| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +| Labels | Kubernetes labels applied to the nodes in the managed node group. | +| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. | + + +{{% /tab %}} +{{% tab "Rancher v2.5.0-v2.5.5" %}} + ### Account Access @@ -117,7 +267,7 @@ Complete each drop-down and field using the information obtained for your IAM po | Setting | Description | | ---------- | -------------------------------------------------------------------------------------------------------------------- | | Region | From the drop-down choose the geographical region in which to build your cluster. | -| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.5/en/user-settings/cloud-credentials/) | +| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | ### Service Role @@ -147,8 +297,8 @@ Configuring Public/Private API access is an advanced use case. For details, refe If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API. There are two ways to avoid this extra manual step: -* You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. -* You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). +- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. +- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). ### Public Access Endpoints @@ -214,6 +364,117 @@ Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/u | Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | | Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +{{% /tab %}} +{{% tab "Rancher prior to v2.5" %}} + + +### Account Access + + + +Complete each drop-down and field using the information obtained for your IAM policy. + +| Setting | Description | +| ---------- | -------------------------------------------------------------------------------------------------------------------- | +| Region | From the drop-down choose the geographical region in which to build your cluster. | +| Access Key | Enter the access key that you created for your IAM policy. | +| Secret Key | Enter the secret key that you created for your IAM policy. | + +### Service Role + + + +Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + +Service Role | Description +-------------|--------------------------- +Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. +Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +### Public IP for Worker Nodes + + + +Your selection for this option determines what options are available for **VPC & Subnet**. + +Option | Description +-------|------------ +Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address. +No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.

If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. + +### VPC & Subnet + + + +The available options depend on the [public IP for worker nodes.](#public-ip-for-worker-nodes) + +Option | Description + -------|------------ + Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet. + Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below. + + For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + +- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) +- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + + +If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you've already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case. + +{{% accordion id="yes" label="Click to expand" %}} + +If you're using **Custom: Choose from your existing VPC and Subnets**: + +(If you're using **Standard**, skip to the [instance options.)](#select-instance-options-2-4) + +1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected. + +1. From the drop-down that displays, choose a VPC. + +1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + +1. Click **Next: Select Security Group**. +{{% /accordion %}} + +If your worker nodes have Private IPs only, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. +{{% accordion id="no" label="Click to expand" %}} +Follow the steps below. + +>**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html). + +1. From the drop-down that displays, choose a VPC. + +1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + +{{% /accordion %}} + +### Security Group + + + +Amazon Documentation: + +- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) +- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) +- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +### Instance Options + + + +Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information. + +Option | Description +-------|------------ +Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. +Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose. +Desired ASG Size | The number of instances that your cluster will provision. +User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_ + +{{% /tab %}} +{{% /tabs %}} + + # Troubleshooting If your changes were overwritten, it could be due to the way the cluster data is synced with EKS. Changes shouldn't be made to the cluster from another source, such as in the EKS console, and in Rancher within a five-minute span. For information on how this works and how to configure the refresh interval, refer to [Syncing.](#syncing) @@ -242,8 +503,8 @@ Documented here is a minimum set of permissions necessary to use all functionali Resource | Description ---------|------------ -Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions]({{}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#service-role-permissions). -VPC | Provides isolated network resources utilised by EKS and worker nodes. Rancher can create the VPC resources with the following [VPC Permissions]({{}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#vpc-permissions). +Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions]({{}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#service-role-permissions). +VPC | Provides isolated network resources utilised by EKS and worker nodes. Rancher can create the VPC resources with the following [VPC Permissions]({{}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#vpc-permissions). Resource targeting uses `*` as the ARN of many of the resources created cannot be known before creating the EKS cluster in Rancher. diff --git a/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md b/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md new file mode 100644 index 00000000000..ff491c7f33c --- /dev/null +++ b/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md @@ -0,0 +1,9 @@ +--- +title: Migrating vSphere In-tree Volumes to CSI +weight: 5 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +For instructions on how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration, refer to [this page.]({{}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/vsphere-volume-migration) \ No newline at end of file diff --git a/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md b/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md new file mode 100644 index 00000000000..9f85cd15c9e --- /dev/null +++ b/content/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md @@ -0,0 +1,9 @@ +--- +title: vSphere Out-of-tree Cloud Provider +weight: 4 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +For instructions on how to install the CPI and CSI plugins after bringing up a cluster, refer to [this page.]({{}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere) \ No newline at end of file diff --git a/content/rancher/v2.5/en/deploy-across-clusters/fleet/_index.md b/content/rancher/v2.5/en/deploy-across-clusters/fleet/_index.md index 6cf0f98eeb0..9af4a260e3d 100644 --- a/content/rancher/v2.5/en/deploy-across-clusters/fleet/_index.md +++ b/content/rancher/v2.5/en/deploy-across-clusters/fleet/_index.md @@ -3,6 +3,8 @@ title: Fleet - GitOps at Scale weight: 1 --- +_Available as of Rancher v2.5_ + Fleet is GitOps at scale. Fleet is designed to manage up to a million clusters. It's also lightweight enough that is works great for a [single cluster](https://fleet.rancher.io/single-cluster-install/) too, but it really shines when you get to a [large scale.](https://fleet.rancher.io/multi-cluster-install/) By large scale we mean either a lot of clusters, a lot of deployments, or a lot of teams in a single organization. Fleet is a separate project from Rancher, and can be installed on any Kubernetes cluster with Helm. @@ -15,6 +17,26 @@ Fleet can manage deployments from git of raw Kubernetes YAML, Helm charts, or Ku Fleet comes preinstalled in Rancher v2.5. To access it, go to the **Cluster Explorer** in the Rancher UI. In the top left dropdown menu, click **Cluster Explorer > Continuous Delivery.** On this page, you can edit Kubernetes resources and cluster groups managed by Fleet. +### Windows Support + +Prior to Rancher v2.5.6, the `agent` did not have native Windows manifests on downstream clusters with Windows nodes. +This would result in a failing `agent` pod for the cluster. +If you are upgrading from an older version of Rancher to v2.5.6+, you can deploy a working `agent` with the following workflow *in the downstream cluster*: + +1. Cordon all Windows nodes. +1. Apply the below toleration to the `agent` workload. +1. Uncordon all Windows nodes. +1. Delete all `agent` pods. New pods should be created with the new toleration. +1. Once the `agent` pods are running, and auto-update is enabled for Fleet, they should be updated to a Windows-compatible `agent` version. + +```yaml +tolerations: +- effect: NoSchedule + key: cattle.io/os + operator: Equal + value: linux +``` + ### GitHub Repository The Fleet Helm charts are available [here.](https://github.com/rancher/fleet/releases/latest) diff --git a/content/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/_index.md b/content/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/_index.md index 6fc48e7e6c9..e44c73d436d 100644 --- a/content/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/_index.md +++ b/content/rancher/v2.5/en/installation/install-rancher-on-k8s/chart-options/_index.md @@ -57,8 +57,8 @@ For information on enabling experimental features, refer to [this page.]({{}}/rancher/v2.5/en/deploy-across-clusters/fleet) preinstalled in v2.5+. - `istio-virtual-service-ui`: This feature enables a [UI to create, read, update, and delete Istio virtual services and destination rules]({{}}/rancher/v2.5/en/installation/options/feature-flags/istio-virtual-service-ui), which are traffic management features of Istio. -- `proxy`: This feature enables Rancher to use a new simplified code base for the proxy, which can help enhance performance and security. The proxy feature is known to have issues with Helm deployments, which prevents any catalog applications to be deployed which includes Rancher's tools like monitoring, logging, Istio, etc. - `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{}}/rancher/v2.5/en/installation/options/feature-flags/enable-not-default-storage-drivers) In other words, it enables types for storage providers and provisioners that are not enabled by default. The below table shows the availability and default value for feature flags in Rancher: @@ -38,10 +37,15 @@ The below table shows the availability and default value for feature flags in Ra | Feature Flag Name | Default Value | Status | Available as of | Rancher Restart Required? | | ----------------------------- | ------------- | ------------ | --------------- |---| | `dashboard` | `true` | Experimental | v2.4.0 | x | +| `dashboard` | `true` | GA* and no longer a feature flag | v2.5.0 | x | | `istio-virtual-service-ui` | `false` | Experimental | v2.3.0 | | -| `istio-virtual-service-ui` | `true` | GA | v2.3.2 | | +| `istio-virtual-service-ui` | `true` | GA* | v2.3.2 | | | `proxy` | `false` | Experimental | v2.4.0 | | +| `proxy` | N/A | Discontinued | v2.5.0 | | | `unsupported-storage-drivers` | `false` | Experimental | v2.3.0 | | +| `fleet` | `true` | GA* | v2.5.0 | | + +\* Generally Available. This feature is included in Rancher and it is not experimental. # Enabling Features when Starting Rancher diff --git a/content/rancher/v2.5/en/istio/resources/_index.md b/content/rancher/v2.5/en/istio/resources/_index.md index fba30f972b7..d4fbc4f3777 100644 --- a/content/rancher/v2.5/en/istio/resources/_index.md +++ b/content/rancher/v2.5/en/istio/resources/_index.md @@ -20,14 +20,32 @@ The table below shows a summary of the minimum recommended resource requests and In Kubernetes, the resource request indicates that the workload will not deployed on a node unless the node has at least the specified amount of memory and CPU available. If the workload surpasses the limit for CPU or memory, it can be terminated or evicted from the node. For more information on managing resource limits for containers, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) -Workload | CPU - Request | Mem - Request | CPU - Limit | Mem - Limit | Configurable +{{% tabs %}} +{{% tab "v2.5.6+" %}} + +| Workload | CPU - Request | Memory - Request | CPU - Limit | Memory - Limit | +|----------------------|---------------|------------|-----------------|-------------------| +| ingress gateway | 100m | 128mi | 2000m | 1024mi | +| egress gateway | 100m | 128mi | 2000m | 1024mi | +| istiod | 500m | 2048mi | No limit | No limit | +| proxy | 10m | 10mi | 2000m | 1024mi | +| **Totals:** | **710m** | **2314Mi** | **6000m** | **3072Mi** | + +{{% /tab %}} +{{% tab "v2.5.0-v2.5.5" %}} + +Workload | CPU - Request | Memory - Request | CPU - Limit | Mem - Limit | Configurable ---------:|---------------:|---------------:|-------------:|-------------:|-------------: -Istiod | 610m | 2186Mi | 4000m | 2048Mi | Y | Y -Istio-policy | 1000m | 1024Mi | 4800m | 4096Mi | Y -Istio-telemetry | 1000m | 10214Mi | 4800m | 4096Mi | Y -Istio-ingressgateway | 2000m | 1024Mi | 10m | 40Mi | Y -Others | 500m | 500Mi | - | - | Y -**Total** | **4500m** | **5620Mi** | **>12300m** | **>14848Mi** | **-** +Istiod | 500m | 2048Mi | No limit | No limit | Y | +Istio-Mixer | 1000m | 1000Mi | 4800m | 4000Mi | Y | +Istio-ingressgateway | 100m | 128Mi | 2000m | 1024Mi | Y | +Others | 10m | - | - | - | Y | +Totals: | 1710m | 3304Mi | >8800m | >6048Mi | - + +{{% /tab %}} +{{% /tabs %}} + + # Configuring Resource Allocations @@ -41,8 +59,23 @@ You can find more information about Istio configuration in the [official Istio d To configure the resources allocated to an Istio component, 1. In the Rancher **Cluster Explorer**, navigate to your Istio installation in **Apps & Marketplace** -1. Click **Upgrade** to edit the base components via changes the values.yaml or add an [overlay file]({{}}/rancher/v2.5/en/istio/v2.5/configuration-reference/#overlay-file). +1. Click **Upgrade** to edit the base components via changes the values.yaml or add an [overlay file]({{}}/rancher/v2.5/en/istio/v2.5/configuration-reference/#overlay-file). For more information about editing the overlay file, see [this section.](./#editing-the-overlay-file) 1. Change the CPU or memory allocations, the nodes where each component will be scheduled to, or the node tolerations. 1. Click **Upgrade.** to rollout changes -**Result:** The resource allocations for the Istio components are updated. \ No newline at end of file +**Result:** The resource allocations for the Istio components are updated. + +### Editing the Overlay File + +The overlay file can contain any of the values in the [Istio Operator spec.](https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioOperatorSpec) The overlay file included with the Istio application is just one example of a potential configuration of the overlay file. + +As long as the file contains `kind: IstioOperator` and the YAML options are valid, the file can be used as an overlay. + +In the example overlay file provided with the Istio application, the following section allows you to change Kubernetes resources: + +``` +# k8s: +# resources: +# requests: +# cpu: 200m +``` \ No newline at end of file diff --git a/content/rancher/v2.5/en/logging/_index.md b/content/rancher/v2.5/en/logging/_index.md index d8f42b26cf8..cf43e02b261 100644 --- a/content/rancher/v2.5/en/logging/_index.md +++ b/content/rancher/v2.5/en/logging/_index.md @@ -321,6 +321,13 @@ spec: In the above example, we ensure that our pod only runs on Linux nodes, and we add a `toleration` for the taint we have on all of our Linux nodes. You can do the same with Rancher's existing taints, or with your own custom ones. +# Working with a Custom Docker Root Directory + +_Applies to v2.5.6+_ + +If using a custom Docker root directory, you can set `global.dockerRootDirectory` in `values.yaml`. +This will ensure that the Logging CRs created will use your specified path rather than the default Docker `data-root` location. + ### Windows Support Clusters with Windows workers support exporting logs from Linux nodes, but Windows node logs are currently unable to be exported. diff --git a/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/ceph/_index.md b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/ceph/_index.md new file mode 100644 index 00000000000..fbc7451b5d2 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/ceph/_index.md @@ -0,0 +1,433 @@ +--- +title: Using an External Ceph Driver +weight: 10 +--- + +These instructions are about using the external Ceph driver in an RKE2 cluster. If you are using RKE, additional steps are required. For details, refer to [this section.](#using-the-ceph-driver-with-rke) + +- [Requirements](#requirements) +- [Using the Ceph Driver with RKE](#using-the-ceph-driver-with-rke) +- [Installing the ceph-csi driver on an RKE2 cluster](#installing-the-ceph-csi-driver-on-an-rke2-cluster) +- [Install the ceph-csi driver using Helm](#install-the-ceph-csi-driver-using-helm) +- [Creating RBD Ceph Resources](#creating-rbd-ceph-resources) +- [Configure RBD Ceph Access Secrets](#configure-rbd-ceph-access-secrets) + - [User Account](#user-account) + - [Admin Account](#admin-account) +- [Create RBD Testing Resources](#create-rbd-testing-resources) + - [Using RBD in Pods](#using-rbd-in-pods) + - [Using RBD in Persistent Volumes](#using-rbd-in-persistent-volumes) + - [Using RBD in Storage Classes](#using-rbd-in-storage-classes) + - [RKE2 Server/Master Provisioning](#rke2-server-master-provisioning) + - [RKE2 Agent/Worker provisioning](#rke2-agent-worker-provisioning) +- [Tested Versions](#tested-versions) +- [Troubleshooting](#troubleshooting) + +# Requirements + +Make sure ceph-common and xfsprogs packages are installed on SLE worker nodes. + +# Using the Ceph Driver with RKE + +The resources below are fully compatible with RKE based clusters, but there is a need to do an additional kubelet configuration for RKE. + +On RKE clusters, the kubelet component is running in a Docker container and doesn't have access to the host's kernel modules as rbd and libceph by default. + +To solve this limitation, you can either run `modprobe rbd` on worker nodes, or configure the kubelet containers to automatically mount the `/lib/modules` directory from the host into the container. + +For the kubelet configuration, put the following lines into the `cluster.yml` file prior to RKE cluster provisioning. You can also modify the `cluster.yml` later in the Rancher UI by clicking on **Edit Cluster > Edit as YAML** and restarting the worker nodes. + +```yaml +services: + kubelet: + extra_binds: + - '/lib/modules:/lib/modules:ro' +``` + +For more information about the `extra_binds` directive, refer to [this section.]({{}}/rke/latest/en/config-options/services/services-extras/#extra-binds) + +# Installing the ceph-csi driver on an RKE2 cluster + +> **Note:** These steps are needed for dynamic RBD provisioning only. + +For more information about the `ceph-csi-rbd` chart, refer to [this page.](https://github.com/ceph/ceph-csi/blob/devel/charts/ceph-csi-rbd/README.md) + +To get details about your SES cluster, run: + +``` +ceph mon dump +``` + +Read its output: + +``` +dumped monmap epoch 3 +epoch 3 +fsid 79179d9d-98d8-4976-ab2e-58635caa7235 +last_changed 2021-02-11T10:56:42.110184+0000 +created 2021-02-11T10:56:22.913321+0000 +min_mon_release 15 (octopus) +0: [v2:10.85.8.118:3300/0,v1:10.85.8.118:6789/0] mon.a +1: [v2:10.85.8.123:3300/0,v1:10.85.8.123:6789/0] mon.b +2: [v2:10.85.8.124:3300/0,v1:10.85.8.124:6789/0] mon.c +``` + +Later you'll need the fsid and mon addresses values. + +# Install the ceph-csi Driver Using Helm + +Run these commands: + +``` +helm repo add ceph-csi https://ceph.github.io/csi-charts +helm repo update +helm search repo ceph-csi -l +helm inspect values ceph-csi/ceph-csi-rbd > ceph-csi-rbd-values.yaml +``` + +Modify the `ceph-csi-rbd-values.yaml` file and keep there only the required changes: + +```yaml +# ceph-csi-rbd-values.yaml +csiConfig: + - clusterID: "79179d9d-98d8-4976-ab2e-58635caa7235" + monitors: + - "10.85.8.118:6789" + - "10.85.8.123:6789" + - "10.85.8.124:6789" +provisioner: + name: provisioner + replicaCount: 2 +``` + +Make sure the ceph monitors are reachable from the RKE2 cluster, for example, by ping. + +``` +kubectl create namespace ceph-csi-rbd +helm install --namespace ceph-csi-rbd ceph-csi-rbd ceph-csi/ceph-csi-rbd --values ceph-csi-rbd-values.yaml +kubectl rollout status deployment ceph-csi-rbd-provisioner -n ceph-csi-rbd +helm status ceph-csi-rbd -n ceph-csi-rbd +``` + +in case you'd like to modify the configuration directly via Helm, you may adapt the `ceph-csi-rbd-values.yaml` file and call: + +``` +helm upgrade \ + --namespace ceph-csi-rbd ceph-csi-rbd ceph-csi/ceph-csi-rbd --values ceph-csi-rbd-values.yaml +``` + +# Creating RBD Ceph Resources + +``` +# Create a ceph pool: +ceph osd pool create myPool 64 64 + +# Create a block device pool: +rbd pool init myPool + +# Create a block device image: +rbd create -s 2G myPool/image + +# Create a block device user and record the key: +ceph auth get-or-create-key client.myPoolUser mon "allow r" osd "allow class-read object_prefix rbd_children, allow rwx pool=myPool" | tr -d '\n' | base64 +QVFDZ0R5VmdyRk9KREJBQTJ5b2s5R1E2NUdSWExRQndhVVBwWXc9PQ== + +# Encode the ceph user myPoolUser into a bash64 hash: +echo "myPoolUser" | tr -d '\n' | base64 +bXlQb29sVXNlcg== + +# Create a block device admin user and record the key: +ceph auth get-or-create-key client.myPoolAdmin mds 'allow *' mgr 'allow *' mon 'allow *' osd 'allow * pool=myPool' | tr -d '\n' | base64 +QVFCK0hDVmdXSjQ1T0JBQXBrc0VtcVhlZFpjc0JwaStIcmU5M3c9PQ== + +# Encode the ceph user myPoolAdmin into a bash64 hash: +echo "myPoolAdmin" | tr -d '\n' | base64 +bXlQb29sQWRtaW4= +``` +# Configure RBD Ceph Access Secrets + +### User Account + +For static RBD provisioning (the image within the ceph pool must exist), run these commands: + +``` +cat > ceph-user-secret.yaml << EOF +apiVersion: v1 +kind: Secret +metadata: + name: ceph-user + namespace: default +type: kubernetes.io/rbd +data: + userID: bXlQb29sVXNlcg== + userKey: QVFDZ0R5VmdyRk9KREJBQTJ5b2s5R1E2NUdSWExRQndhVVBwWXc9PQ== +EOF + +kubectl apply -f ceph-user-secret.yaml +``` + +### Admin Account + +For dynamic RBD provisioning (used for automatic image creation within a given ceph pool), run these commands: + +``` +cat > ceph-admin-secret.yaml << EOF +apiVersion: v1 +kind: Secret +metadata: + name: ceph-admin + namespace: default +type: kubernetes.io/rbd +data: + userID: bXlQb29sQWRtaW4= + userKey: QVFCK0hDVmdXSjQ1T0JBQXBrc0VtcVhlZFpjc0JwaStIcmU5M3c9PQ== +EOF + +kubectl apply -f ceph-admin-secret.yaml +``` + +# Create RBD Testing Resources + +### Using RBD in Pods + +``` +# pod +cat > ceph-rbd-pod-inline.yaml << EOF +apiVersion: v1 +kind: Pod +metadata: + name: ceph-rbd-pod-inline +spec: + containers: + - name: ceph-rbd-pod-inline + image: busybox + command: ["sleep", "infinity"] + volumeMounts: + - mountPath: /mnt/ceph_rbd + name: volume + volumes: + - name: volume + rbd: + monitors: + - 10.85.8.118:6789 + - 10.85.8.123:6789 + - 10.85.8.124:6789 + pool: myPool + image: image + user: myPoolUser + secretRef: + name: ceph-user + fsType: ext4 + readOnly: false +EOF + +kubectl apply -f ceph-rbd-pod-inline.yaml +kubectl get pod +kubectl exec pod/ceph-rbd-pod-inline -- df -k | grep rbd +``` + +### Using RBD in Persistent Volumes + +``` +# pod-pvc-pv +cat > ceph-rbd-pod-pvc-pv-allinone.yaml << EOF +apiVersion: v1 +kind: PersistentVolume +metadata: + name: ceph-rbd-pv +spec: + capacity: + storage: 2Gi + accessModes: + - ReadWriteOnce + rbd: + monitors: + - 10.85.8.118:6789 + - 10.85.8.123:6789 + - 10.85.8.124:6789 + pool: myPool + image: image + user: myPoolUser + secretRef: + name: ceph-user + fsType: ext4 + readOnly: false +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: ceph-rbd-pvc +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi +--- +apiVersion: v1 +kind: Pod +metadata: + name: ceph-rbd-pod-pvc-pv +spec: + containers: + - name: ceph-rbd-pod-pvc-pv + image: busybox + command: ["sleep", "infinity"] + volumeMounts: + - mountPath: /mnt/ceph_rbd + name: volume + volumes: + - name: volume + persistentVolumeClaim: + claimName: ceph-rbd-pvc +EOF + +kubectl apply -f ceph-rbd-pod-pvc-pv-allinone.yaml +kubectl get pv,pvc,pod +kubectl exec pod/ceph-rbd-pod-pvc-pv -- df -k | grep rbd +``` + +### Using RBD in Storage Classes + +This example is for dynamic provisioning. The ceph-csi driver is needed. + +``` +# pod-pvc-sc +cat > ceph-rbd-pod-pvc-sc-allinone.yaml < /root/.bashrc << EOF +export PATH=$PATH:/var/lib/rancher/rke2/bin/ +export KUBECONFIG=/etc/rancher/rke2/rke2.yaml +EOF + +cat /var/lib/rancher/rke2/server/node-token +token: K10ca0c38d4ff90d8b80319ab34092e315a8b732622e6adf97bc9eb0536REDACTED::server:ec0308000b8a6b595da000efREDACTED +``` + +### RKE2 Agent/Worker provisioning + +``` +mkdir -p /etc/rancher/rke2/ + +cat > /etc/rancher/rke2/config.yaml << EOF +server: https://10.100.103.23:9345 +token: K10ca0c38d4ff90d8b80319ab34092e315a8b732622e6adf97bc9eb0536REDACTED::server:ec0308000b8a6b595da000efREDACTED +EOF + +curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh - +systemctl enable --now rke2-agent.service +``` + +The cluster can be imported into Rancher from the Rancher UI by clicking **Global/Add Cluster > Other Cluster.** Then run the provided kubectl command on the server/master node. + +# Tested Versions + +OS for running RKE2 nodes: JeOS SLE15-SP2 with installed kernel-default-5.3.18-24.49 + +``` +kubectl version +Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-22T12:00:00Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/amd64"} +Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7+rke2r1", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-20T01:50:52Z", GoVersion:"go1.15.5b5", Compiler:"gc", Platform:"linux/amd64"} + +helm version +version.BuildInfo{Version:"3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.12"} +``` + +Kubernetes version on RKE2 cluster: v1.19.7+rke2r1 + +# Troubleshooting + +In case you are using SUSE's ceph-rook based on SES7, it might be useful to expose the monitors on hostNetwork by editing `rook-1.4.5/ceph/cluster.yaml` and setting `spec.network.hostNetwork=true`. + +Also for operating the ceph-rook cluster, it is useful to deploy a toolbox on the Kubernetes cluster where ceph-rook is provisioned by `kubectl apply -f rook-1.4.5/ceph/toolbox.yaml` Then all the ceph related commands can be executed in the toolbox pod, for example, by running `kubectl exec -it -n rook-ceph rook-ceph-tools-686d8b8bfb-2nvqp -- bash` + +Operating with the ceph - basic commands: + +``` +ceph osd pool stats +ceph osd pool delete myPool myPool --yes-i-really-really-mean-it +rbd list -p myPool +> csi-vol-f5d3766c-7296-11eb-b32a-c2b045952d38 +> image +``` + +Delete the image: `rbd rm csi-vol-f5d3766c-7296-11eb-b32a-c2b045952d38 -p myPool` + +CephFS commands in rook toolbox: + +``` +ceph -s +ceph fs ls +ceph fs fail cephfs +ceph fs rm cephfs --yes-i-really-mean-it +ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it +ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it +``` + +To prepare a cephfs filesystem, you can run this command on a rook cluster: + +``` +kubectl apply -f rook-1.4.5/ceph/filesystem.yaml +``` \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md new file mode 100644 index 00000000000..c3f593e6549 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere/_index.md @@ -0,0 +1,49 @@ +--- +title: vSphere Out-of-tree Cloud Provider +weight: 10 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +This page covers how to install the CPI and CSI plugins after bringing up a cluster. + +# Prerequisites + +The vSphere version must be 6.7U3 or higher. + +# Installation + +The Cloud Provider Interface (CPI) should be installed first before installing the Cloud Storage Interface (CSI). + +### 1. Create a vSphere cluster + +1. On the Clusters page, click on **Add Cluster** and select the **vSphere** option. +1. Under **Cluster Options > In-Tree Cloud Provider** select **External**. +1. Click **Create**. + +### 2. Install the CPI plugin + + 1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.** +1. Select the **vsphere-cpi** chart from the **helm3-library** catalog. Fill out the required vCenter details. +1. vSphere CPI initializes all nodes with ProviderID which is needed by the vSphere CSI driver. Check if all nodes are initialized with the ProviderID before installing CSI driver with the following command: + + ``` + kubectl describe nodes | grep "ProviderID" + ``` + +### 3. Installing the CSI plugin + + 1. From the **Cluster Explorer** view, go to the top left dropdown menu and click **Apps & Marketplace.** +1. Select the **vsphere-csi** chart from the **helm3-library** catalog. Fill out the required vCenter details. +2. Set **Enable CSI Migration** to **false**. +3. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. Fill out the details for the StorageClass and launch the chart. + + +# Using the CSI driver for provisioning volumes + +The CSI chart by default creates a storageClass. + +If that option was not selected while launching the chart, create a storageClass with the `csi.vsphere.vmware.com` as the provisioner. + +All volumes provisioned using this StorageClass will get provisioned by the CSI driver. \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md new file mode 100644 index 00000000000..e024e44a252 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/vsphere-volume-migration/_index.md @@ -0,0 +1,99 @@ +--- +title: Migrating vSphere In-tree Volumes to CSI +weight: 5 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +This page covers how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration. + +It follows the steps provided in the official [vSphere migration documentation](https://vsphere-csi-driver.sigs.k8s.io/features/vsphere_csi_migration.html) and provides the steps to be performed in Rancher. + +### Cloud-config Format Limitation + +Existing volumes that were provisioned using the following cloud-config format will NOT get migrated due to an existing bug in vsphere CSI. + +If the cloud-config has this format for datastore and resource pool path, vsphere CSI driver cannot recognize it: + +```yaml +default-datastore: /datastore/ +resourcepool-path: "/host//Resources/" +``` + +Volumes provisioned with the in-tree provider using the following format will get migrated correctly: + +```yaml +default-datastore: +resourcepool-path: "/Resources/" +``` + +Upstream bug: https://github.com/kubernetes-sigs/vsphere-csi-driver/issues/628 + +Rancher issue tracking this bug: https://github.com/rancher/rancher/issues/31105 + +# Prerequisites + +- vSphere CSI Migration requires vSphere 7.0u1. In order to be able to manage existing in-tree vSphere volumes, upgrade vSphere to 7.0u1. +- The Kubernetes version must be 1.19 or higher. + +# Migration + +### 1. Install the CPI plugin + +Before installing CPI, we need to taint all nodes with `node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule`. + +This can be done by running the following commands: + +``` +curl -O https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh +``` + +Or: + +``` +wget https://raw.githubusercontent.com/rancher/helm3-charts/56b622f519728378abeddfe95074f1b87ab73b1e/charts/vsphere-cpi/taints.sh +chmod +x taints.sh +./taints.sh +``` + +Once all nodes are tainted by the running the script, launch the Helm vSphere CPI chart. + +1. Within a project, select **Apps > Launch.** +2. Select the **vsphere-cpi** chart from the **helm3-library** catalog. +3. Fill out the required vCenter details and click **Launch**. + +vSphere CPI initializes all nodes with ProviderID, which is needed by the vSphere CSI driver. + +Check if all nodes are initialized with the ProviderID with the following command: + +``` +kubectl describe nodes | grep "ProviderID" +``` + +### 2. Install the CSI driver + +1. Within a project, select **Apps > Launch** and select the **vsphere-csi** chart from the **helm3-library** catalog. +1. Fill out the required vCenter details and click **Launch**. +1. Set **Enable CSI Migration** to **true**. +1. This chart creates a StorageClass with the `csi.vsphere.vmware.com` as the provisioner. You can provide the URL of the datastore to be used for CSI volume provisioning while creating this StorageClass. The datastore URL can be found in the vSphere client by selecting the datastore and going to the Summary tab. Fill out the details for the StorageClass and click **Launch**. + +### 3. Edit the cluster to enable CSI migration feature flags + +1. While editing the cluster, if the Kubernetes version is less than 1.19, select Kubernetes version 1.19 or higher from the **Kubernetes Version** dropdown. +2. For enabling feature flags, click on "Edit as YAML", and add the following under kube-controller and kubelet: + + ```yaml + extra_args: + feature-gates: "CSIMigration=true,CSIMigrationvSphere=true" + ``` + +### 4. Drain worker nodes + +Worker nodes must be drained during the upgrade before changing the kubelet and kube-controller-manager args. + +1. Click **Edit as Form** and then click on "Advanced Options." +1. Set the field **Maximum Worker Nodes Unavailable** to count of 1. +1. To drain the nodes during upgrade, select **Drain Nodes > Yes**. +1. Set **Force** and **Delete Local Data** to **true**. +1. Click **Save** to upgrade the cluster. \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md index cb7c57d6109..1c793684ff9 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md @@ -32,7 +32,7 @@ To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Priva ### Amazon VPC -You need to set up an Amazon VPC to launch the EKS cluster. The VPC enables you to launch AWS resources into a virtual network that you've defined. For more information, refer to the [Tutorial: Creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html). +An Amazon VPC is required to launch the EKS cluster. The VPC enables you to launch AWS resources into a virtual network that you've defined. You can set one up yourself and provide it during cluster creation in Rancher. If you do not provide one during creation, Rancher will create one. For more information, refer to the [Tutorial: Creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html). ### IAM Policies @@ -109,7 +109,155 @@ The following capabilities have been added for configuring EKS clusters in Ranch Due to the way that the cluster data is synced with EKS, if the cluster is modified from another source, such as in the EKS console, and in Rancher within five minutes, it could cause some changes to be overwritten. For information about how the sync works and how to configure it, refer to [this section](#syncing). {{% tabs %}} -{{% tab "Rancher v2.5+" %}} +{{% tab "Rancher v2.5.6+" %}} + +### Account Access + + + +Complete each drop-down and field using the information obtained for your IAM policy. + +| Setting | Description | +| ---------- | -------------------------------------------------------------------------------------------------------------------- | +| Region | From the drop-down choose the geographical region in which to build your cluster. | +| Cloud Credentials | Select the cloud credentials that you created for your IAM policy. For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | + +### Service Role + + + +Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + +Service Role | Description +-------------|--------------------------- +Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. +Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +### Secrets Encryption + + + +Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html) + +### API Server Endpoint Access + + + +Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Private-only API Endpoints + +If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API. + +There are two ways to avoid this extra manual step: +- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. +- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). + +### Public Access Endpoints + + + +Optionally limit access to the public endpoint via explicit CIDR blocks. + +If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster. + +One of the following is required to enable private access: +- Rancher's IP must be part of an allowed CIDR block +- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group + +For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Subnet + + + +| Option | Description | +| ------- | ------------ | +| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. | +| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). | + + For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + +- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) +- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + +### Security Group + + + +Amazon Documentation: + +- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) +- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) +- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +### Logging + + + +Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters. + +Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation. + +For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) + +### Managed Node Groups + + + +Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters. + +For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) + +#### Bring your own launch template + +A launch template ID and version can be provided in order to easily configure the EC2 instances in a node group. If a launch template is provided, then none of the settings below will be configurable in Rancher. Therefore, using a launch template would require that all the necessary and desired settings from the list below would need to be specified in the launch template. Also note that if a launch template ID and version is provided, then only the template version can be updated. Using a new template ID would require creating a new managed node group. + +| Option | Description | Required/Optional | +| ------ | ----------- | ----------------- | +| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | Required | +| Image ID | Specify a custom AMI for the nodes. Custom AMIs used with EKS must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) | Optional | +| Node Volume Size | The launch template must specify an EBS volume with the desired size | Required | +| SSH Key | A key to be added to the instances to provide SSH access to the nodes | Optional | +| User Data | Cloud init script in [MIME multi-part format](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-user-data) | Optional | +| Instance Resource Tags | Tag each EC2 instance in the node group | Optional | + +#### Rancher-managed launch templates + +If you do not specify a launch template, then you will be able to configure the above options in the Rancher UI and all of them can be updated after creation. In order to take advantage of all of these options, Rancher will create and manage a launch template for you. Each cluster in Rancher will have one Rancher-managed launch template and each managed node group that does not have a specified launch template will have one version of the managed launch template. The name of this launch template will have the prefix "rancher-managed-lt-" followed by the display name of the cluster. In addition, the Rancher-managed launch template will be tagged with the key "rancher-managed-template" and value "do-not-modify-or-delete" to help identify it as Rancher-managed. It is important that this launch template and its versions not be modified, deleted, or used with any other clusters or managed node groups. Doing so could result in your node groups being "degraded" and needing to be destroyed and recreated. + +#### Custom AMIs + +If you specify a custom AMI, whether in a launch template or in Rancher, then the image must be [configured properly](https://aws.amazon.com/premiumsupport/knowledge-center/eks-custom-linux-ami/) and you must provide user data to [bootstrap the node](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html#launch-template-custom-ami). This is considered an advanced use case and understanding the requirements is imperative. + +If you specify a launch template that does not contain a custom AMI, then Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version and selected region. You can also select a [GPU enabled instance](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami) for workloads that would benefit from it. + +>**Note** +>The GPU enabled instance setting in Rancher is ignored if a custom AMI is provided, either in the dropdown or in a launch template. + +#### Spot instances + +Spot instances are now [supported by EKS](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types-spot). If a launch template is specified, Amazon recommends that the template not provide an instance type. Instead, Amazon recommends providing multiple instance types. If the "Request Spot Instances" checkbox is enabled for a node group, then you will have the opportunity to provide multiple instance types. + +>**Note** +>Any selection you made in the instance type dropdown will be ignored in this situation and you must specify at least one instance type to the "Spot Instance Types" section. Furthermore, a launch template used with EKS cannot request spot instances. Requesting spot instances must be part of the EKS configuration. + +#### Node Group Settings + +The following settings are also configurable. All of these except for the "Node Group Name" are editable after the node group is created. + +| Option | Description | +| ------- | ------------ | +| Node Group Name | The name of the node group. | +| Desired ASG Size | The desired number of instances. | +| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +| Labels | Kubernetes labels applied to the nodes in the managed node group. | +| Tags | These are tags for the managed node group and do not propagate to any of the associated resources. | + + +{{% /tab %}} +{{% tab "Rancher v2.5.0-v2.5.5" %}} ### Account Access @@ -150,8 +298,8 @@ Configuring Public/Private API access is an advanced use case. For details, refe If you enable private and disable public API endpoint access when creating a cluster, then there is an extra step you must take in order for Rancher to connect to the cluster successfully. In this case, a pop-up will be displayed with a command that you will run on the cluster to register it with Rancher. Once the cluster is provisioned, you can run the displayed command anywhere you can connect to the cluster's Kubernetes API. There are two ways to avoid this extra manual step: -* You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. -* You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). +- You can create the cluster with both private and public API endpoint access on cluster creation. You can disable public access after the cluster is created and in an active state and Rancher will continue to communicate with the EKS cluster. +- You can ensure that Rancher shares a subnet with the EKS cluster. Then security groups can be used to enable Rancher to communicate with the cluster's API endpoint. In this case, the command to register the cluster is not needed, and Rancher will be able to communicate with your cluster. For more information on configuring security groups, refer to the [security groups documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html). ### Public Access Endpoints diff --git a/content/rancher/v2.x/en/cluster-provisioning/registered-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/registered-clusters/_index.md index 7c040256ef7..fcf2605019b 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/registered-clusters/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/registered-clusters/_index.md @@ -45,11 +45,12 @@ If you are registering a K3s cluster, make sure the `cluster.yml` is readable. I 2. Choose **Register**. 3. Enter a **Cluster Name**. 4. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user. -5. Click **Create**. -6. The prerequisite for `cluster-admin` privileges is shown (see **Prerequisites** above), including an example command to fulfil the prerequisite. -7. Copy the `kubectl` command to your clipboard and run it on a node where kubeconfig is configured to point to the cluster you want to import. If you are unsure it is configured correctly, run `kubectl get nodes` to verify before running the command shown in Rancher. -8. If you are using self signed certificates, you will receive the message `certificate signed by unknown authority`. To work around this validation, copy the command starting with `curl` displayed in Rancher to your clipboard. Then run the command on a node where kubeconfig is configured to point to the cluster you want to import. -9. When you finish running the command(s) on your node, click **Done**. +5. Use **Agent Environment Variables** under **Cluster Options** to set environment variables for [rancher cluster agent]({{}}rancher/v2.x/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables. +6. Click **Create**. +7. The prerequisite for `cluster-admin` privileges is shown (see **Prerequisites** above), including an example command to fulfil the prerequisite. +8. Copy the `kubectl` command to your clipboard and run it on a node where kubeconfig is configured to point to the cluster you want to import. If you are unsure it is configured correctly, run `kubectl get nodes` to verify before running the command shown in Rancher. +9. If you are using self signed certificates, you will receive the message `certificate signed by unknown authority`. To work around this validation, copy the command starting with `curl` displayed in Rancher to your clipboard. Then run the command on a node where kubeconfig is configured to point to the cluster you want to import. +10. When you finish running the command(s) on your node, click **Done**. **Result:** diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md new file mode 100644 index 00000000000..0b50820e403 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/_index.md @@ -0,0 +1,9 @@ +--- +title: Migrating vSphere In-tree Volumes to CSI +weight: 5 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +For instructions on how to migrate from the in-tree vSphere cloud provider to out-of-tree, and manage the existing VMs post migration, refer to [this page.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/vsphere-volume-migration) \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md new file mode 100644 index 00000000000..d3a1f3a12f0 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/out-of-tree-vsphere/_index.md @@ -0,0 +1,9 @@ +--- +title: vSphere Out-of-tree Cloud Provider +weight: 4 +--- +_Available as of v2.5.6_ + +Kubernetes is moving away from maintaining cloud providers in-tree. vSphere has an out-of-tree cloud provider that can be used by installing the vSphere cloud provider and cloud storage plugins. + +For instructions on how to install the CPI and CSI plugins after bringing up a cluster, refer to [this page.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/out-of-tree-vsphere) \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md index b5bc0a0a5d6..267a8ba8faa 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md @@ -31,6 +31,7 @@ This section is a cluster configuration reference, covering the following topics - [Docker version on nodes](#docker-version-on-nodes) - [Docker root directory](#docker-root-directory) - [Recurring etcd snapshots](#recurring-etcd-snapshots) + - [Agent Environment Variables](#agent-environment-variables) - [Cluster config file](#cluster-config-file) - [Config file structure in Rancher v2.3.0+](#config-file-structure-in-rancher-v2-3-0) - [Config file structure in Rancher v2.0.0-v2.2.x](#config-file-structure-in-rancher-v2-0-0-v2-2-x) @@ -152,6 +153,12 @@ If the nodes you are adding to the cluster have Docker configured with a non-def Option to enable or disable [recurring etcd snapshots]({{}}/rke/latest/en/etcd-snapshots/#etcd-recurring-snapshots). +### Agent Environment Variables + +_Available as of v2.5.6_ + +Option to set environment variables for [rancher agents]({{}}rancher/v2.x/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables. + # Cluster Config File Instead of using the Rancher UI to choose Kubernetes options for the cluster, advanced users can create an RKE config file. Using a config file allows you to set any of the [options available]({{}}/rke/latest/en/config-options/) in an RKE installation, except for `system_images` configuration. The `system_images` option is not supported when creating a cluster with the Rancher UI or API. diff --git a/content/rancher/v2.x/en/deploy-across-clusters/fleet/_index.md b/content/rancher/v2.x/en/deploy-across-clusters/fleet/_index.md index d4827af0c9c..2b8ff6a6c7e 100644 --- a/content/rancher/v2.x/en/deploy-across-clusters/fleet/_index.md +++ b/content/rancher/v2.x/en/deploy-across-clusters/fleet/_index.md @@ -22,6 +22,7 @@ Fleet comes preinstalled in Rancher v2.5. To access it, go to the **Cluster Expl ### Windows Support Before Rancher v2.5.6, the `agent` did not have native Windows manifests on downstream clusters with Windows nodes. + This would result in a failing `agent` pod for the cluster. If you are upgrading from an older version of Rancher to v2.5.6+, you can deploy a working `agent` with the following workflow *in the downstream cluster*: diff --git a/content/rancher/v2.x/en/installation/install-rancher-on-k8s/chart-options/_index.md b/content/rancher/v2.x/en/installation/install-rancher-on-k8s/chart-options/_index.md index ec499f73d47..8186ad6918d 100644 --- a/content/rancher/v2.x/en/installation/install-rancher-on-k8s/chart-options/_index.md +++ b/content/rancher/v2.x/en/installation/install-rancher-on-k8s/chart-options/_index.md @@ -57,8 +57,9 @@ For information on enabling experimental features, refer to [this page.]({{}}/rancher/v2.x/en/installation/options/feature-flags/istio-virtual-service-ui), which are traffic management features of Istio. -- `proxy`: This feature enables Rancher to use a new simplified code base for the proxy, which can help enhance performance and security. The proxy feature is known to have issues with Helm deployments, which prevents any catalog applications to be deployed which includes Rancher's tools like monitoring, logging, Istio, etc. +- `proxy`: This feature enables Rancher to use a new simplified code base for the proxy, which can help enhance performance and security. The proxy feature is known to have issues with Helm deployments, which prevents any catalog applications to be deployed which includes Rancher's tools like monitoring, logging, Istio, etc. It is discontinued in Rancher v2.5. - `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{}}/rancher/v2.x/en/installation/options/feature-flags/enable-not-default-storage-drivers) In other words, it enables types for storage providers and provisioners that are not enabled by default. +- `fleet`: Rancher comes with [Fleet]({{}}/rancher/v2.x/en/deploy-across-clusters/fleet/) preinstalled in v2.5+. The below table shows the availability and default value for feature flags in Rancher: | Feature Flag Name | Default Value | Status | Available as of | Rancher Restart Required? | | ----------------------------- | ------------- | ------------ | --------------- |---| | `dashboard` | `true` | Experimental | v2.4.0 | x | +| `dashboard` | `true` | GA* and no longer a feature flag | v2.5.0 | x | | `istio-virtual-service-ui` | `false` | Experimental | v2.3.0 | | -| `istio-virtual-service-ui` | `true` | GA | v2.3.2 | | +| `istio-virtual-service-ui` | `true` | GA* | v2.3.2 | | | `proxy` | `false` | Experimental | v2.4.0 | | +| `proxy` | N/A | Discontinued | v2.5.0 | | | `unsupported-storage-drivers` | `false` | Experimental | v2.3.0 | | +| `fleet` | `true` | GA* | v2.5.0 | | + +\* Generally Available. This feature is included in Rancher and it is not experimental. # Enabling Features when Starting Rancher diff --git a/content/rancher/v2.x/en/istio/v2.5/resources/_index.md b/content/rancher/v2.x/en/istio/v2.5/resources/_index.md index 19dde46e0d2..1f82cfcaa10 100644 --- a/content/rancher/v2.x/en/istio/v2.5/resources/_index.md +++ b/content/rancher/v2.x/en/istio/v2.5/resources/_index.md @@ -7,7 +7,7 @@ aliases: - /rancher/v2.x/en/cluster-admin/tools/istio/resources - /rancher/v2.x/en/istio/resources --- -_This section applies to Istio in Rancher v2.5.0. If you are using Rancher v2.4.x, refer to [this section.]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/)_ +_This section applies to Istio in Rancher v2.5.x. If you are using Rancher v2.4.x, refer to [this section.]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/)_ This section describes the minimum recommended computing resources for the Istio components in a cluster. @@ -21,14 +21,32 @@ The table below shows a summary of the minimum recommended resource requests and In Kubernetes, the resource request indicates that the workload will not deployed on a node unless the node has at least the specified amount of memory and CPU available. If the workload surpasses the limit for CPU or memory, it can be terminated or evicted from the node. For more information on managing resource limits for containers, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) -Workload | CPU - Request | Mem - Request | CPU - Limit | Mem - Limit | Configurable +{{% tabs %}} +{{% tab "v2.5.6+" %}} + +| Workload | CPU - Request | Memory - Request | CPU - Limit | Memory - Limit | +|----------------------|---------------|------------|-----------------|-------------------| +| ingress gateway | 100m | 128mi | 2000m | 1024mi | +| egress gateway | 100m | 128mi | 2000m | 1024mi | +| istiod | 500m | 2048mi | No limit | No limit | +| proxy | 10m | 10mi | 2000m | 1024mi | +| **Totals:** | **710m** | **2314Mi** | **6000m** | **3072Mi** | + +{{% /tab %}} +{{% tab "v2.5.0-v2.5.5" %}} + +Workload | CPU - Request | Memory - Request | CPU - Limit | Mem - Limit | Configurable ---------:|---------------:|---------------:|-------------:|-------------:|-------------: -Istiod | 610m | 2186Mi | 4000m | 2048Mi | Y | Y -Istio-policy | 1000m | 1024Mi | 4800m | 4096Mi | Y -Istio-telemetry | 1000m | 10214Mi | 4800m | 4096Mi | Y -Istio-ingressgateway | 2000m | 1024Mi | 10m | 40Mi | Y -Others | 500m | 500Mi | - | - | Y -**Total** | **4500m** | **5620Mi** | **>12300m** | **>14848Mi** | **-** +Istiod | 500m | 2048Mi | No limit | No limit | Y | +Istio-Mixer | 1000m | 1000Mi | 4800m | 4000Mi | Y | +Istio-ingressgateway | 100m | 128Mi | 2000m | 1024Mi | Y | +Others | 10m | - | - | - | Y | +Totals: | 1710m | 3304Mi | >8800m | >6048Mi | - + +{{% /tab %}} +{{% /tabs %}} + + # Configuring Resource Allocations @@ -42,8 +60,24 @@ You can find more information about Istio configuration in the [official Istio d To configure the resources allocated to an Istio component, 1. In the Rancher **Cluster Explorer**, navigate to your Istio installation in **Apps & Marketplace** -1. Click **Upgrade** to edit the base components via changes the values.yaml or add an [overlay file]({{}}/rancher/v2.x/en/istio/v2.5/configuration-reference/#overlay-file). +1. Click **Upgrade** to edit the base components via changes the values.yaml or add an [overlay file]({{}}/rancher/v2.x/en/istio/v2.5/configuration-reference/#overlay-file). For more information about editing the overlay file, see [this section.](./#editing-the-overlay-file) 1. Change the CPU or memory allocations, the nodes where each component will be scheduled to, or the node tolerations. 1. Click **Upgrade.** to rollout changes -**Result:** The resource allocations for the Istio components are updated. \ No newline at end of file +**Result:** The resource allocations for the Istio components are updated. + + +### Editing the Overlay File + +The overlay file can contain any of the values in the [Istio Operator spec.](https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioOperatorSpec) The overlay file included with the Istio application is just one example of a potential configuration of the overlay file. + +As long as the file contains `kind: IstioOperator` and the YAML options are valid, the file can be used as an overlay. + +In the example overlay file provided with the Istio application, the following section allows you to change Kubernetes resources: + +``` +# k8s: +# resources: +# requests: +# cpu: 200m +``` \ No newline at end of file