From d6f9fdbbe2c0fdf32ddf6f30dc3feecac87e3deb Mon Sep 17 00:00:00 2001 From: "Juan J. Jimenez-Anca" Date: Sat, 10 Oct 2020 12:24:27 +0100 Subject: [PATCH 01/10] Update Istio docs for Network Isolation In clusters with namespace Network Isolation enabled, the default installation of Istio needs further tweaking for it to work --- .../setup/enable-istio-in-cluster/_index.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/_index.md b/content/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/_index.md index 06fc840f7d0..59df49bae16 100644 --- a/content/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/_index.md +++ b/content/rancher/v2.x/en/istio/setup/enable-istio-in-cluster/_index.md @@ -22,6 +22,22 @@ Only a user with the following [Kubernetes default roles](https://kubernetes.io/ Automatic sidecar injection is disabled by default. To enable this, set the `sidecarInjectorWebhook.enableNamespacesByDefault=true` in the values.yaml on install or upgrade. This automatically enables Istio sidecar injection into all new namespaces that are deployed. +>**Note:** In clusters where: +> +> - The [Canal network plug-in]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#canal) is in use. +> - The Project Network Isolation option is enabled. +> - You install the Istio Ingress module +> +>The Istio Ingress Gateway pod won't be able to redirect ingress traffic to the workloads by default. This is because all the namespaces will be innacessible from the namespace where Istio is installed. You have two options: +> 1. You add a new Network Policy in each of the namespaces where you intend to have ingress controlled by Istio. Your policy should include the following lines: +> ``` +> ... +> - podSelector: +> matchLabels: +> app: istio-ingressgateway +>``` +> 2. You move the `ingress-system` namespace to the `system` project, which by default is excluded from the network isolation + ## Additonal Config Options ### Overlay File @@ -147,4 +163,4 @@ This enables monitoring accross namespaces by giving prometheus additional scrap - source_labels: [__meta_kubernetes_pod_name] action: replace target_label: pod_name -``` \ No newline at end of file +``` From ae5cffacee5097f920f18fdbdaf0b3f1ef6ecca6 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Sun, 11 Oct 2020 12:10:48 -0700 Subject: [PATCH 02/10] Add changes from pull request #2741 (EKS IAM permissions) --- .../hosted-kubernetes-clusters/eks/_index.md | 641 +++++++----------- 1 file changed, 254 insertions(+), 387 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md index 74d9e9ef038..895e4a890d6 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md @@ -8,24 +8,13 @@ aliases: Amazon EKS provides a managed control plane for your Kubernetes cluster. Amazon EKS runs the Kubernetes control plane instances across multiple Availability Zones to ensure high availability. Rancher provides an intuitive user interface for managing and deploying the Kubernetes clusters you run in Amazon EKS. With this guide, you will use Rancher to quickly and easily launch an Amazon EKS Kubernetes cluster in your AWS account. For more information on Amazon EKS, see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html). -- [Prerequisites in Amazon Web Services](#prerequisites-in-amazon-web-services) - - [Amazon VPC](#amazon-vpc) - - [IAM Policies](#iam-policies) -- [Architecture](#architecture) -- [Create the EKS Cluster](#create-the-eks-cluster) -- [EKS Cluster Configuration Reference](#eks-cluster-configuration-reference) -- [Troubleshooting](#troubleshooting) -- [AWS Service Events](#aws-service-events) -- [Security and Compliance](#security-and-compliance) -- [Tutorial](#tutorial) -- [Minimum EKS Permissions](#minimum-eks-permissions) -# Prerequisites in Amazon Web Services +## Prerequisites in Amazon Web Services >**Note** >Deploying to Amazon AWS will incur charges. For more information, refer to the [EKS pricing page](https://aws.amazon.com/eks/pricing/). -To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Private Cloud). You will also need to make sure that the account you will be using to create the EKS cluster has the appropriate [permissions.](#minimum-eks-permissions) For details, refer to the official guide on [Amazon EKS Prerequisites](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html#eks-prereqs). +To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Private Cloud). You will also need to make sure that the account you will be using to create the EKS cluster has the appropriate permissions. For details, refer to the official guide on [Amazon EKS Prerequisites](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html#eks-prereqs). ### Amazon VPC @@ -37,7 +26,7 @@ Rancher needs access to your AWS account in order to provision and administer yo 1. Create a user with programmatic access by following the steps [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html). -2. Next, create an IAM policy that defines what this user has access to in your AWS account. It's important to only grant this user minimal access within your account. The minimum permissions required for an EKS cluster are listed [here.](#minimum-eks-permissions) Follow the steps [here](https://docs.aws.amazon.com/eks/latest/userguide/EKS_IAM_user_policies.html) to create an IAM policy and attach it to your user. +2. Next, create an IAM policy that defines what this user has access to in your AWS account. The required permissions are [here.]({{}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#appendix-minimum-eks-permissions) Follow the steps [here](https://docs.aws.amazon.com/eks/latest/userguide/EKS_IAM_user_policies.html) to create an IAM policy and attach it to your user. 3. Finally, follow the steps [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) to create an access key and secret key for this user. @@ -45,15 +34,13 @@ Rancher needs access to your AWS account in order to provision and administer yo For more detailed information on IAM policies for EKS, refer to the official [documentation on Amazon EKS IAM Policies, Roles, and Permissions](https://docs.aws.amazon.com/eks/latest/userguide/IAM_policies.html). -# Architecture +## Architecture The figure below illustrates the high-level architecture of Rancher 2.x. The figure depicts a Rancher Server installation that manages two Kubernetes clusters: one created by RKE and another created by EKS. -
Managing Kubernetes Clusters through Rancher's Authentication Proxy
+![Rancher architecture with EKS hosted cluster]({{}}/img/rancher/rancher-architecture.svg) -![Architecture]({{}}/img/rancher/rancher-architecture-rancher-api-server.svg) - -# Create the EKS Cluster +## Create the EKS Cluster Use Rancher to set up and configure your Kubernetes cluster. @@ -61,279 +48,120 @@ Use Rancher to set up and configure your Kubernetes cluster. 1. Choose **Amazon EKS**. -1. Enter a **Cluster Name.** +1. Enter a **Cluster Name**. 1. {{< step_create-cluster_member-roles >}} -1. Fill out the rest of the form. For help, refer to the [configuration reference.](#eks-cluster-configuration-reference) +1. Configure **Account Access** for the EKS cluster. Complete each drop-down and field using the information obtained in [2. Create Access Key and Secret Key](#prerequisites-in-amazon-web-services). + + | Setting | Description | + | ---------- | -------------------------------------------------------------------------------------------------------------------- | + | Region | From the drop-down choose the geographical region in which to build your cluster. | + | Access Key | Enter the access key that you created in [2. Create Access Key and Secret Key](#2-create-access-key-and-secret-key). | + | Secret Key | Enter the secret key that you created in [2. Create Access Key and Secret Key](#2-create-access-key-and-secret-key). | + +1. Click **Next: Select Service Role**. Then choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + + Service Role | Description + -------------|--------------------------- + Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. + Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +1. Click **Next: Select VPC and Subnet**. + +1. Choose an option for **Public IP for Worker Nodes**. Your selection for this option determines what options are available for **VPC & Subnet**. + + Option | Description + -------|------------ + Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address. + No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.

If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. + +1. Now choose a **VPC & Subnet**. For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + + - [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) + - [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + + {{% accordion id="yes" label="Public IP for Worker Nodes—Yes" %}} +If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you're already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case. + +1. Choose a **VPC and Subnet** option. + + Option | Description + -------|------------ + Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet. + Custom: Choose from your exiting VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below. + +1. If you're using **Custom: Choose from your existing VPC and Subnets**: + + (If you're using **Standard**, skip to [step 11](#select-instance-options)) + + 1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected. + + 1. From the drop-down that displays, choose a VPC. + + 1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + + 1. Click **Next: Select Security Group**. + {{% /accordion %}} + {{% accordion id="no" label="Public IP for Worker Nodes—No: Private IPs only" %}} +If you chose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. Follow the steps below. + +>**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html). + + 1. From the drop-down that displays, choose a VPC. + + 1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + + 1. Click **Next: Select Security Group**. + {{% /accordion %}} + +1. Choose a **Security Group**. See the documentation below on how to create one. + + Amazon Documentation: + - [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) + - [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) + - [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +1. Click **Select Instance Options**, and then edit the node options available. Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information. + + Option | Description + -------|------------ + Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. + Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose. + Desired ASG Size | The number of instances that your cluster will provision. + User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_ 1. Click **Create**. {{< result_create-cluster >}} -# EKS Cluster Configuration Reference - -### Changes in Rancher v2.5 - -More EKS options can be configured when you create an EKS cluster in Rancher, including the following: - -- Managed node groups -- Desired size, minimum size, maximum size (requires the Cluster Autoscaler to be installed) -- Control plane logging -- Secrets encryption with KMS - -The following capabilities have been added for configuring EKS clusters in Rancher: - -- GPU support -- Exclusively use managed nodegroups that come with the most up-to-date AMIs -- Add new nodes -- Upgrade nodes -- Add and remove node groups -- Disable and enable private access -- Add restrictions to public access -- Use your cloud credentials to create the EKS cluster instead of passing in your access key and secret key - -Due to the way that the cluster data is synced with EKS, if the cluster is modified from another source, such as in the EKS console, and in Rancher within five minutes, it could cause some changes to be overwritten. For information about how the sync works and how to configure it, refer to [this section](#syncing). - -{{% tabs %}} -{{% tab "Rancher v2.5+" %}} - -### Account Access - - - -Complete each drop-down and field using the information obtained for your [IAM policy.](#iam-policy) - -| Setting | Description | -| ---------- | -------------------------------------------------------------------------------------------------------------------- | -| Region | From the drop-down choose the geographical region in which to build your cluster. | -| Cloud Credentials | Select the cloud credentials that you created for your [IAM policy.](#iam-policy) For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | - -### Service Role - - - -Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). - -Service Role | Description --------------|--------------------------- -Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. -Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). - -### Secrets Encryption - - - -Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html) - -### API Server Endpoint Access - - - -Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) - -### Public Access Endpoints - - - -Optionally limit access to the public endpoint via explicit CIDR blocks. - -If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster. - -One of the following is required to enable private access: -- Rancher's IP must be part of an allowed CIDR block -- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group - -For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) - -### Subnet - - - -| Option | Description | -| ------- | ------------ | -| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. | -| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). | - - For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. - -- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) -- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) - -### Security Group - - - -Amazon Documentation: - -- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) -- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) -- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) - -### Logging - - - -Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters. - -Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation. - -For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) - -### Managed Node Groups - - - -Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters. - -For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) - -Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version. You can configure whether the AMI has GPU enabled. - -| Option | Description | -| ------- | ------------ | -| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | -| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | -| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | - -{{% /tab %}} -{{% tab "Rancher prior to v2.5" %}} - - -### Account Access - - - -Complete each drop-down and field using the information obtained for your [IAM policy.](#iam-policy) - -| Setting | Description | -| ---------- | -------------------------------------------------------------------------------------------------------------------- | -| Region | From the drop-down choose the geographical region in which to build your cluster. | -| Access Key | Enter the access key that you created for your [IAM policy.](#iam-policy) | -| Secret Key | Enter the secret key that you created for your [IAM policy.](#iam-policy) | - -### Service Role - - - -Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). - -Service Role | Description --------------|--------------------------- -Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. -Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). - -### Public IP for Worker Nodes - - - -Your selection for this option determines what options are available for **VPC & Subnet**. - -Option | Description --------|------------ -Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address. -No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.

If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. - -### VPC & Subnet - - - -The available options depend on the [public IP for worker nodes.](#public-ip-for-worker-nodes) - -Option | Description - -------|------------ - Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet. - Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below. - - For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. - -- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) -- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) - - -If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you've already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case. - -{{% accordion id="yes" label="Click to expand" %}} - -If you're using **Custom: Choose from your existing VPC and Subnets**: - -(If you're using **Standard**, skip to the [instance options.)](#select-instance-options-2-4) - -1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected. - -1. From the drop-down that displays, choose a VPC. - -1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. - -1. Click **Next: Select Security Group**. -{{% /accordion %}} - -If your worker nodes have Private IPs only, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. -{{% accordion id="no" label="Click to expand" %}} -Follow the steps below. - ->**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html). - -1. From the drop-down that displays, choose a VPC. - -1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. - -{{% /accordion %}} - -### Security Group - - - -Amazon Documentation: - -- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) -- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) -- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) - -### Instance Options - - - -Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information. - -Option | Description --------|------------ -Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. -Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose. -Desired ASG Size | The number of instances that your cluster will provision. -User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_ - -{{% /tab %}} -{{% /tabs %}} - - -# Troubleshooting - -If your changes were overwritten, it could be due to the way the cluster data is synced with EKS. Changes shouldn't be made to the cluster from another source, such as in the EKS console, and in Rancher within a five-minute span. For information on how this works and how to configure the refresh interval, refer to [Syncing.](#syncing) - -If an unauthorized error is returned while attempting to modify or register the cluster and the cluster was not created with the role or user that your credentials belong to, refer to [Security and Compliance.](#security-and-compliance) +## Troubleshooting For any issues or troubleshooting details for your Amazon EKS Kubernetes cluster, please see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/troubleshooting.html). -# AWS Service Events +## AWS Service Events To find information on any AWS Service events, please see [this page](https://status.aws.amazon.com/). -# Security and Compliance - -By default only the IAM user or role that created a cluster has access to it. Attempting to access the cluster with any other user or role without additional configuration will lead to an error. In Rancher, this means using a credential that maps to a user or role that was not used to create the cluster will cause an unauthorized error. For example, an EKSCtl cluster will not register in Rancher unless the credentials used to register the cluster match the role or user used by EKSCtl. Additional users and roles can be authorized to access a cluster by being added to the aws-auth configmap in the kube-system namespace. For a more in-depth explanation and detailed instructions, please see this [documentation](https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/). +## Security and Compliance For more information on security and compliance with your Amazon EKS Kubernetes cluster, please see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/shared-responsibilty.html). -# Tutorial +## Tutorial This [tutorial](https://aws.amazon.com/blogs/opensource/managing-eks-clusters-rancher/) on the AWS Open Source Blog will walk you through how to set up an EKS cluster with Rancher, deploy a publicly accessible app to test the cluster, and deploy a sample project to track real-time geospatial data using a combination of other open-source software such as Grafana and InfluxDB. -# Minimum EKS Permissions +## Appendix - Minimum EKS Permissions -Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. +Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. Additional permissions are required for Rancher to provision the `Service Role` and `VPC` resources. Optionally these resources can be created **before** the cluster creation and will be selectable when defining the cluster configuration. -Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. Some permissions (for example `ec2:CreateVpc`) are only used in situations where Rancher handles the creation of certain resources. +Resource | Description +---------|------------ +Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#service-role-permissions). +VPC | Provides isolated network resouces utilised by EKS and worker nodes. Rancher can create the VPC resouces with the follwoing [VPC Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#vpc-permissions). + + +Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. ```json { @@ -343,92 +171,26 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "EC2Permisssions", "Effect": "Allow", "Action": [ - "ec2:AuthorizeSecurityGroupIngress", - "ec2:DeleteSubnet", - "ec2:CreateKeyPair", - "ec2:AttachInternetGateway", - "ec2:ReplaceRoute", - "ec2:DeleteRouteTable", - "ec2:AssociateRouteTable", - "ec2:DescribeInternetGateways", - "ec2:CreateRoute", - "ec2:CreateInternetGateway", - "ec2:RevokeSecurityGroupEgress", - "ec2:DescribeAccountAttributes", - "ec2:DeleteInternetGateway", - "ec2:DescribeKeyPairs", - "ec2:CreateTags", - "ec2:CreateRouteTable", - "ec2:DescribeRouteTables", - "ec2:DetachInternetGateway", - "ec2:DisassociateRouteTable", "ec2:RevokeSecurityGroupIngress", - "ec2:DeleteVpc", - "ec2:CreateSubnet", - "ec2:DescribeSubnets", - "ec2:DeleteKeyPair", - "ec2:DeleteTags", - "ec2:CreateVpc", - "ec2:DescribeAvailabilityZones", - "ec2:CreateSecurityGroup", - "ec2:ModifyVpcAttribute", - "ec2:AuthorizeSecurityGroupEgress", - "ec2:DescribeTags", - "ec2:DeleteRoute", - "ec2:DescribeSecurityGroups", - "ec2:DescribeImages", + "ec2:RevokeSecurityGroupEgress", "ec2:DescribeVpcs", - "ec2:DeleteSecurityGroup" - ], - "Resource": "*" - }, - { - "Sid": "EKSPermissions", - "Effect": "Allow", - "Action": [ - "eks:DeleteFargateProfile", - "eks:DescribeFargateProfile", - "eks:ListTagsForResource", - "eks:UpdateClusterConfig", - "eks:DescribeNodegroup", - "eks:ListNodegroups", - "eks:DeleteCluster", - "eks:CreateFargateProfile", - "eks:DeleteNodegroup", - "eks:UpdateNodegroupConfig", - "eks:DescribeCluster", - "eks:ListClusters", - "eks:UpdateClusterVersion", - "eks:UpdateNodegroupVersion", - "eks:ListUpdates", - "eks:CreateCluster", - "eks:UntagResource", - "eks:CreateNodegroup", - "eks:ListFargateProfiles", - "eks:DescribeUpdate", - "eks:TagResource" - ], - "Resource": "*" - }, - { - "Sid": "IAMPermissions", - "Effect": "Allow", - "Action": [ - "iam:ListRoleTags", - "iam:RemoveRoleFromInstanceProfile", - "iam:CreateRole", - "iam:AttachRolePolicy", - "iam:AddRoleToInstanceProfile", - "iam:DetachRolePolicy", - "iam:GetRole", - "iam:DeleteRole", - "iam:CreateInstanceProfile", - "iam:ListInstanceProfilesForRole", - "iam:PassRole", - "iam:GetInstanceProfile", - "iam:ListRoles", - "iam:ListInstanceProfiles", - "iam:DeleteInstanceProfile" + "ec2:DescribeTags", + "ec2:DescribeSubnets", + "ec2:DescribeSecurityGroups", + "ec2:DescribeRouteTables", + "ec2:DescribeKeyPairs", + "ec2:DescribeInternetGateways", + "ec2:DescribeImages", + "ec2:DescribeAvailabilityZones", + "ec2:DescribeAccountAttributes", + "ec2:DeleteTags", + "ec2:DeleteSecurityGroup", + "ec2:DeleteKeyPair", + "ec2:CreateTags", + "ec2:CreateSecurityGroup", + "ec2:CreateKeyPair", + "ec2:AuthorizeSecurityGroupIngress", + "ec2:AuthorizeSecurityGroupEgress" ], "Resource": "*" }, @@ -436,30 +198,67 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "CloudFormationPermisssions", "Effect": "Allow", "Action": [ - "cloudformation:DescribeStackResource", - "cloudformation:ListStackResources", - "cloudformation:DescribeStackResources", - "cloudformation:DescribeStacks", "cloudformation:ListStacks", + "cloudformation:ListStackResources", + "cloudformation:DescribeStacks", + "cloudformation:DescribeStackResources", + "cloudformation:DescribeStackResource", + "cloudformation:DeleteStack", + "cloudformation:CreateStackSet", "cloudformation:CreateStack" ], "Resource": "*" }, { - "Sid": "AutoScalingPermissions", + "Sid": "IAMPermissions", "Effect": "Allow", "Action": [ - "autoscaling:DescribeAutoScalingGroups", - "autoscaling:UpdateAutoScalingGroup", - "autoscaling:TerminateInstanceInAutoScalingGroup", - "autoscaling:CreateOrUpdateTags", - "autoscaling:DeleteAutoScalingGroup", - "autoscaling:CreateAutoScalingGroup", - "autoscaling:DescribeAutoScalingInstances", - "autoscaling:DescribeLaunchConfigurations", - "autoscaling:DescribeScalingActivities", - "autoscaling:CreateLaunchConfiguration", - "autoscaling:DeleteLaunchConfiguration" + "iam:PassRole", + "iam:ListRoles", + "iam:ListRoleTags", + "iam:ListInstanceProfilesForRole", + "iam:ListInstanceProfiles", + "iam:ListAttachedRolePolicies", + "iam:GetRole", + "iam:GetInstanceProfile", + "iam:DetachRolePolicy", + "iam:DeleteRole", + "iam:CreateRole", + "iam:AttachRolePolicy" + ], + "Resource": "*" + }, + { + "Sid": "KMSPermisssions", + "Effect": "Allow", + "Action": "kms:ListKeys", + "Resource": "*" + }, + { + "Sid": "EKSPermisssions", + "Effect": "Allow", + "Action": [ + "eks:UpdateNodegroupVersion", + "eks:UpdateNodegroupConfig", + "eks:UpdateClusterVersion", + "eks:UpdateClusterConfig", + "eks:UntagResource", + "eks:TagResource", + "eks:ListUpdates", + "eks:ListTagsForResource", + "eks:ListNodegroups", + "eks:ListFargateProfiles", + "eks:ListClusters", + "eks:DescribeUpdate", + "eks:DescribeNodegroup", + "eks:DescribeFargateProfile", + "eks:DescribeCluster", + "eks:DeleteNodegroup", + "eks:DeleteFargateProfile", + "eks:DeleteCluster", + "eks:CreateNodegroup", + "eks:CreateFargateProfile", + "eks:CreateCluster" ], "Resource": "*" } @@ -467,29 +266,97 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b } ``` -# Syncing +### Service Role Permissions -Syncing is the feature that causes Rancher to update its EKS clusters' values so they are up to date with their corresponding cluster object in the EKS console. This enables Rancher to not be the sole owner of an EKS cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other. +Rancher will create a service role with the following trust policy: -### How it works +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Action": "sts:AssumeRole", + "Principal": { + "Service": "eks.amazonaws.com" + }, + "Effect": "Allow", + "Sid": "" + } + ] +} +``` -There are two fields on the Rancher Cluster object that must be understood to understand how syncing works: +This role will also have two role policy attachments with the following policies ARNs: -1. EKSConfig which is located on the Spec of the Cluster. -2. UpstreamSpec which is located on the EKSStatus field on the Status of the Cluster. +``` +arn:aws:iam::aws:policy/AmazonEKSClusterPolicy +arn:aws:iam::aws:policy/AmazonEKSServicePolicy +``` -Both of which are defined by the struct EKSClusterConfigSpec found in the eks-operator project: https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go +Permissions required for Rancher to create service role on users behalf during the EKS cluster creation process. -All fields with the exception of DisplayName, AmazonCredentialSecret, Region, and Imported are nillable on the EKSClusterConfigSpec. +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "IAMPermisssions", + "Effect": "Allow", + "Action": [ + "iam:AddRoleToInstanceProfile", + "iam:AttachRolePolicy", + "iam:CreateInstanceProfile", + "iam:CreateRole", + "iam:CreateServiceLinkedRole", + "iam:DeleteInstanceProfile", + "iam:DeleteRole", + "iam:DetachRolePolicy", + "iam:GetInstanceProfile", + "iam:GetRole", + "iam:ListAttachedRolePolicies", + "iam:ListInstanceProfiles", + "iam:ListInstanceProfilesForRole", + "iam:ListRoles", + "iam:ListRoleTags", + "iam:PassRole", + "iam:RemoveRoleFromInstanceProfile" + ], + "Resource": "*" + } + ] +} +``` -The EKSConfig represents desired state for its non-nil values. Fields that are non-nil in the EKSConfig can be thought of as “managed".When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher. +### VPC Permissions -UpstreamSpec represents the cluster as it is in EKS and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed rancher checks if the EKS cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig are overwritten with their corresponding value from the recently updated UpstreamSpec. +Permissions required for Rancher to create VPC and associated resources. -The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig. This is what is displayed in the UI. - -If Rancher and another source attempt to update an EKS cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. For example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioend case. - -### Configuring the Refresh Interval - -It is possible to change the refresh interval through the setting “eks-refresh-cron". This setting accepts values in the Cron format. The default is `*/5 * * * *`. The shorter the refresh window is the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs. +```json +{ + "Sid": "VPCPermissions", + "Effect": "Allow", + "Action": [ + "ec2:ReplaceRoute", + "ec2:ModifyVpcAttribute", + "ec2:ModifySubnetAttribute", + "ec2:DisassociateRouteTable", + "ec2:DetachInternetGateway", + "ec2:DescribeVpcs", + "ec2:DeleteVpc", + "ec2:DeleteTags", + "ec2:DeleteSubnet", + "ec2:DeleteRouteTable", + "ec2:DeleteRoute", + "ec2:DeleteInternetGateway", + "ec2:CreateVpc", + "ec2:CreateSubnet", + "ec2:CreateSecurityGroup", + "ec2:CreateRouteTable", + "ec2:CreateRoute", + "ec2:CreateInternetGateway", + "ec2:AttachInternetGateway", + "ec2:AssociateRouteTable" + ], + "Resource": "*" +} +``` From ec08f0fe9e9a9edb5770d7900c53d2f4e1bf2d1c Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Sun, 11 Oct 2020 12:18:32 -0700 Subject: [PATCH 03/10] Revert "Add changes from pull request #2741 (EKS IAM permissions)" This reverts commit ae5cffacee5097f920f18fdbdaf0b3f1ef6ecca6. --- .../hosted-kubernetes-clusters/eks/_index.md | 623 +++++++++++------- 1 file changed, 378 insertions(+), 245 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md index 895e4a890d6..74d9e9ef038 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md @@ -8,13 +8,24 @@ aliases: Amazon EKS provides a managed control plane for your Kubernetes cluster. Amazon EKS runs the Kubernetes control plane instances across multiple Availability Zones to ensure high availability. Rancher provides an intuitive user interface for managing and deploying the Kubernetes clusters you run in Amazon EKS. With this guide, you will use Rancher to quickly and easily launch an Amazon EKS Kubernetes cluster in your AWS account. For more information on Amazon EKS, see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html). +- [Prerequisites in Amazon Web Services](#prerequisites-in-amazon-web-services) + - [Amazon VPC](#amazon-vpc) + - [IAM Policies](#iam-policies) +- [Architecture](#architecture) +- [Create the EKS Cluster](#create-the-eks-cluster) +- [EKS Cluster Configuration Reference](#eks-cluster-configuration-reference) +- [Troubleshooting](#troubleshooting) +- [AWS Service Events](#aws-service-events) +- [Security and Compliance](#security-and-compliance) +- [Tutorial](#tutorial) +- [Minimum EKS Permissions](#minimum-eks-permissions) -## Prerequisites in Amazon Web Services +# Prerequisites in Amazon Web Services >**Note** >Deploying to Amazon AWS will incur charges. For more information, refer to the [EKS pricing page](https://aws.amazon.com/eks/pricing/). -To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Private Cloud). You will also need to make sure that the account you will be using to create the EKS cluster has the appropriate permissions. For details, refer to the official guide on [Amazon EKS Prerequisites](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html#eks-prereqs). +To set up a cluster on EKS, you will need to set up an Amazon VPC (Virtual Private Cloud). You will also need to make sure that the account you will be using to create the EKS cluster has the appropriate [permissions.](#minimum-eks-permissions) For details, refer to the official guide on [Amazon EKS Prerequisites](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html#eks-prereqs). ### Amazon VPC @@ -26,7 +37,7 @@ Rancher needs access to your AWS account in order to provision and administer yo 1. Create a user with programmatic access by following the steps [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html). -2. Next, create an IAM policy that defines what this user has access to in your AWS account. The required permissions are [here.]({{}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#appendix-minimum-eks-permissions) Follow the steps [here](https://docs.aws.amazon.com/eks/latest/userguide/EKS_IAM_user_policies.html) to create an IAM policy and attach it to your user. +2. Next, create an IAM policy that defines what this user has access to in your AWS account. It's important to only grant this user minimal access within your account. The minimum permissions required for an EKS cluster are listed [here.](#minimum-eks-permissions) Follow the steps [here](https://docs.aws.amazon.com/eks/latest/userguide/EKS_IAM_user_policies.html) to create an IAM policy and attach it to your user. 3. Finally, follow the steps [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) to create an access key and secret key for this user. @@ -34,13 +45,15 @@ Rancher needs access to your AWS account in order to provision and administer yo For more detailed information on IAM policies for EKS, refer to the official [documentation on Amazon EKS IAM Policies, Roles, and Permissions](https://docs.aws.amazon.com/eks/latest/userguide/IAM_policies.html). -## Architecture +# Architecture The figure below illustrates the high-level architecture of Rancher 2.x. The figure depicts a Rancher Server installation that manages two Kubernetes clusters: one created by RKE and another created by EKS. -![Rancher architecture with EKS hosted cluster]({{}}/img/rancher/rancher-architecture.svg) +
Managing Kubernetes Clusters through Rancher's Authentication Proxy
-## Create the EKS Cluster +![Architecture]({{}}/img/rancher/rancher-architecture-rancher-api-server.svg) + +# Create the EKS Cluster Use Rancher to set up and configure your Kubernetes cluster. @@ -48,120 +61,279 @@ Use Rancher to set up and configure your Kubernetes cluster. 1. Choose **Amazon EKS**. -1. Enter a **Cluster Name**. +1. Enter a **Cluster Name.** 1. {{< step_create-cluster_member-roles >}} -1. Configure **Account Access** for the EKS cluster. Complete each drop-down and field using the information obtained in [2. Create Access Key and Secret Key](#prerequisites-in-amazon-web-services). - - | Setting | Description | - | ---------- | -------------------------------------------------------------------------------------------------------------------- | - | Region | From the drop-down choose the geographical region in which to build your cluster. | - | Access Key | Enter the access key that you created in [2. Create Access Key and Secret Key](#2-create-access-key-and-secret-key). | - | Secret Key | Enter the secret key that you created in [2. Create Access Key and Secret Key](#2-create-access-key-and-secret-key). | - -1. Click **Next: Select Service Role**. Then choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). - - Service Role | Description - -------------|--------------------------- - Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. - Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). - -1. Click **Next: Select VPC and Subnet**. - -1. Choose an option for **Public IP for Worker Nodes**. Your selection for this option determines what options are available for **VPC & Subnet**. - - Option | Description - -------|------------ - Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address. - No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.

If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. - -1. Now choose a **VPC & Subnet**. For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. - - - [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) - - [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) - - {{% accordion id="yes" label="Public IP for Worker Nodes—Yes" %}} -If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you're already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case. - -1. Choose a **VPC and Subnet** option. - - Option | Description - -------|------------ - Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet. - Custom: Choose from your exiting VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below. - -1. If you're using **Custom: Choose from your existing VPC and Subnets**: - - (If you're using **Standard**, skip to [step 11](#select-instance-options)) - - 1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected. - - 1. From the drop-down that displays, choose a VPC. - - 1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. - - 1. Click **Next: Select Security Group**. - {{% /accordion %}} - {{% accordion id="no" label="Public IP for Worker Nodes—No: Private IPs only" %}} -If you chose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. Follow the steps below. - ->**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html). - - 1. From the drop-down that displays, choose a VPC. - - 1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. - - 1. Click **Next: Select Security Group**. - {{% /accordion %}} - -1. Choose a **Security Group**. See the documentation below on how to create one. - - Amazon Documentation: - - [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) - - [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) - - [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) - -1. Click **Select Instance Options**, and then edit the node options available. Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information. - - Option | Description - -------|------------ - Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. - Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose. - Desired ASG Size | The number of instances that your cluster will provision. - User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_ +1. Fill out the rest of the form. For help, refer to the [configuration reference.](#eks-cluster-configuration-reference) 1. Click **Create**. {{< result_create-cluster >}} -## Troubleshooting +# EKS Cluster Configuration Reference + +### Changes in Rancher v2.5 + +More EKS options can be configured when you create an EKS cluster in Rancher, including the following: + +- Managed node groups +- Desired size, minimum size, maximum size (requires the Cluster Autoscaler to be installed) +- Control plane logging +- Secrets encryption with KMS + +The following capabilities have been added for configuring EKS clusters in Rancher: + +- GPU support +- Exclusively use managed nodegroups that come with the most up-to-date AMIs +- Add new nodes +- Upgrade nodes +- Add and remove node groups +- Disable and enable private access +- Add restrictions to public access +- Use your cloud credentials to create the EKS cluster instead of passing in your access key and secret key + +Due to the way that the cluster data is synced with EKS, if the cluster is modified from another source, such as in the EKS console, and in Rancher within five minutes, it could cause some changes to be overwritten. For information about how the sync works and how to configure it, refer to [this section](#syncing). + +{{% tabs %}} +{{% tab "Rancher v2.5+" %}} + +### Account Access + + + +Complete each drop-down and field using the information obtained for your [IAM policy.](#iam-policy) + +| Setting | Description | +| ---------- | -------------------------------------------------------------------------------------------------------------------- | +| Region | From the drop-down choose the geographical region in which to build your cluster. | +| Cloud Credentials | Select the cloud credentials that you created for your [IAM policy.](#iam-policy) For more information on creating cloud credentials in Rancher, refer to [this page.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | + +### Service Role + + + +Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + +Service Role | Description +-------------|--------------------------- +Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. +Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +### Secrets Encryption + + + +Optional: To encrypt secrets, select or enter a key created in [AWS Key Management Service (KMS)](https://docs.aws.amazon.com/kms/latest/developerguide/overview.html) + +### API Server Endpoint Access + + + +Configuring Public/Private API access is an advanced use case. For details, refer to the EKS cluster endpoint access control [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Public Access Endpoints + + + +Optionally limit access to the public endpoint via explicit CIDR blocks. + +If you limit access to specific CIDR blocks, then it is recommended that you also enable the private access to avoid losing network communication to the cluster. + +One of the following is required to enable private access: +- Rancher's IP must be part of an allowed CIDR block +- Private access should be enabled, and Rancher must share a subnet with the cluster and have network access to the cluster, which can be configured with a security group + +For more information about public and private access to the cluster endpoint, refer to the [Amazon EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) + +### Subnet + + + +| Option | Description | +| ------- | ------------ | +| Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC with 3 public subnets. | +| Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your Control Plane and nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). | + + For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + +- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) +- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + +### Security Group + + + +Amazon Documentation: + +- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) +- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) +- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +### Logging + + + +Configure control plane logs to send to Amazon CloudWatch. You are charged the standard CloudWatch Logs data ingestion and storage costs for any logs sent to CloudWatch Logs from your clusters. + +Each log type corresponds to a component of the Kubernetes control plane. To learn more about these components, see [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) in the Kubernetes documentation. + +For more information on EKS control plane logging, refer to the official [documentation.](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) + +### Managed Node Groups + + + +Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters. + +For more information about how node groups work and how they are configured, refer to the [EKS documentation.](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) + +Amazon will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the Kubernetes version. You can configure whether the AMI has GPU enabled. + +| Option | Description | +| ------- | ------------ | +| Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. | +| Maximum ASG Size | The maximum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | +| Minimum ASG Size | The minimum number of instances. This setting won't take effect until the [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html) is installed. | + +{{% /tab %}} +{{% tab "Rancher prior to v2.5" %}} + + +### Account Access + + + +Complete each drop-down and field using the information obtained for your [IAM policy.](#iam-policy) + +| Setting | Description | +| ---------- | -------------------------------------------------------------------------------------------------------------------- | +| Region | From the drop-down choose the geographical region in which to build your cluster. | +| Access Key | Enter the access key that you created for your [IAM policy.](#iam-policy) | +| Secret Key | Enter the secret key that you created for your [IAM policy.](#iam-policy) | + +### Service Role + + + +Choose a [service role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). + +Service Role | Description +-------------|--------------------------- +Standard: Rancher generated service role | If you choose this role, Rancher automatically adds a service role for use with the cluster. +Custom: Choose from your existing service roles | If you choose this role, Rancher lets you choose from service roles that you're already created within AWS. For more information on creating a custom service role in AWS, see the [Amazon documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role). + +### Public IP for Worker Nodes + + + +Your selection for this option determines what options are available for **VPC & Subnet**. + +Option | Description +-------|------------ +Yes | When your cluster nodes are provisioned, they're assigned a both a private and public IP address. +No: Private IPs only | When your cluster nodes are provisioned, they're assigned only a private IP address.

If you choose this option, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. + +### VPC & Subnet + + + +The available options depend on the [public IP for worker nodes.](#public-ip-for-worker-nodes) + +Option | Description + -------|------------ + Standard: Rancher generated VPC and Subnet | While provisioning your cluster, Rancher generates a new VPC and Subnet. + Custom: Choose from your existing VPC and Subnets | While provisioning your cluster, Rancher configures your nodes to use a VPC and Subnet that you've already [created in AWS](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html). If you choose this option, complete the remaining steps below. + + For more information, refer to the AWS documentation for [Cluster VPC Considerations](https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html). Follow one of the sets of instructions below based on your selection from the previous step. + +- [What Is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) +- [VPCs and Subnets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) + + +If you choose to assign a public IP address to your cluster's worker nodes, you have the option of choosing between a VPC that's automatically generated by Rancher (i.e., **Standard: Rancher generated VPC and Subnet**), or a VPC that you've already created with AWS (i.e., **Custom: Choose from your existing VPC and Subnets**). Choose the option that best fits your use case. + +{{% accordion id="yes" label="Click to expand" %}} + +If you're using **Custom: Choose from your existing VPC and Subnets**: + +(If you're using **Standard**, skip to the [instance options.)](#select-instance-options-2-4) + +1. Make sure **Custom: Choose from your existing VPC and Subnets** is selected. + +1. From the drop-down that displays, choose a VPC. + +1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + +1. Click **Next: Select Security Group**. +{{% /accordion %}} + +If your worker nodes have Private IPs only, you must also choose a **VPC & Subnet** that allow your instances to access the internet. This access is required so that your worker nodes can connect to the Kubernetes control plane. +{{% accordion id="no" label="Click to expand" %}} +Follow the steps below. + +>**Tip:** When using only private IP addresses, you can provide your nodes internet access by creating a VPC constructed with two subnets, a private set and a public set. The private set should have its route tables configured to point toward a NAT in the public set. For more information on routing traffic from private subnets, please see the [official AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html). + +1. From the drop-down that displays, choose a VPC. + +1. Click **Next: Select Subnets**. Then choose one of the **Subnets** that displays. + +{{% /accordion %}} + +### Security Group + + + +Amazon Documentation: + +- [Cluster Security Group Considerations](https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html) +- [Security Groups for Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) +- [Create a Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/getting-started-ipv4.html#getting-started-create-security-group) + +### Instance Options + + + +Instance type and size of your worker nodes affects how many IP addresses each worker node will have available. See this [documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for more information. + +Option | Description +-------|------------ +Instance Type | Choose the [hardware specs](https://aws.amazon.com/ec2/instance-types/) for the instance you're provisioning. +Custom AMI Override | If you want to use a custom [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html#creating-an-ami) (AMI), specify it here. By default, Rancher will use the [EKS-optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for the EKS version that you chose. +Desired ASG Size | The number of instances that your cluster will provision. +User Data | Custom commands can to be passed to perform automated configuration tasks **WARNING: Modifying this may cause your nodes to be unable to join the cluster.** _Note: Available as of v2.2.0_ + +{{% /tab %}} +{{% /tabs %}} + + +# Troubleshooting + +If your changes were overwritten, it could be due to the way the cluster data is synced with EKS. Changes shouldn't be made to the cluster from another source, such as in the EKS console, and in Rancher within a five-minute span. For information on how this works and how to configure the refresh interval, refer to [Syncing.](#syncing) + +If an unauthorized error is returned while attempting to modify or register the cluster and the cluster was not created with the role or user that your credentials belong to, refer to [Security and Compliance.](#security-and-compliance) For any issues or troubleshooting details for your Amazon EKS Kubernetes cluster, please see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/troubleshooting.html). -## AWS Service Events +# AWS Service Events To find information on any AWS Service events, please see [this page](https://status.aws.amazon.com/). -## Security and Compliance +# Security and Compliance + +By default only the IAM user or role that created a cluster has access to it. Attempting to access the cluster with any other user or role without additional configuration will lead to an error. In Rancher, this means using a credential that maps to a user or role that was not used to create the cluster will cause an unauthorized error. For example, an EKSCtl cluster will not register in Rancher unless the credentials used to register the cluster match the role or user used by EKSCtl. Additional users and roles can be authorized to access a cluster by being added to the aws-auth configmap in the kube-system namespace. For a more in-depth explanation and detailed instructions, please see this [documentation](https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/). For more information on security and compliance with your Amazon EKS Kubernetes cluster, please see this [documentation](https://docs.aws.amazon.com/eks/latest/userguide/shared-responsibilty.html). -## Tutorial +# Tutorial This [tutorial](https://aws.amazon.com/blogs/opensource/managing-eks-clusters-rancher/) on the AWS Open Source Blog will walk you through how to set up an EKS cluster with Rancher, deploy a publicly accessible app to test the cluster, and deploy a sample project to track real-time geospatial data using a combination of other open-source software such as Grafana and InfluxDB. -## Appendix - Minimum EKS Permissions +# Minimum EKS Permissions -Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. Additional permissions are required for Rancher to provision the `Service Role` and `VPC` resources. Optionally these resources can be created **before** the cluster creation and will be selectable when defining the cluster configuration. +Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. -Resource | Description ----------|------------ -Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#service-role-permissions). -VPC | Provides isolated network resouces utilised by EKS and worker nodes. Rancher can create the VPC resouces with the follwoing [VPC Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#vpc-permissions). - - -Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. +Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. Some permissions (for example `ec2:CreateVpc`) are only used in situations where Rancher handles the creation of certain resources. ```json { @@ -171,41 +343,70 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "EC2Permisssions", "Effect": "Allow", "Action": [ - "ec2:RevokeSecurityGroupIngress", - "ec2:RevokeSecurityGroupEgress", - "ec2:DescribeVpcs", - "ec2:DescribeTags", - "ec2:DescribeSubnets", - "ec2:DescribeSecurityGroups", - "ec2:DescribeRouteTables", - "ec2:DescribeKeyPairs", - "ec2:DescribeInternetGateways", - "ec2:DescribeImages", - "ec2:DescribeAvailabilityZones", - "ec2:DescribeAccountAttributes", - "ec2:DeleteTags", - "ec2:DeleteSecurityGroup", - "ec2:DeleteKeyPair", - "ec2:CreateTags", - "ec2:CreateSecurityGroup", - "ec2:CreateKeyPair", "ec2:AuthorizeSecurityGroupIngress", - "ec2:AuthorizeSecurityGroupEgress" + "ec2:DeleteSubnet", + "ec2:CreateKeyPair", + "ec2:AttachInternetGateway", + "ec2:ReplaceRoute", + "ec2:DeleteRouteTable", + "ec2:AssociateRouteTable", + "ec2:DescribeInternetGateways", + "ec2:CreateRoute", + "ec2:CreateInternetGateway", + "ec2:RevokeSecurityGroupEgress", + "ec2:DescribeAccountAttributes", + "ec2:DeleteInternetGateway", + "ec2:DescribeKeyPairs", + "ec2:CreateTags", + "ec2:CreateRouteTable", + "ec2:DescribeRouteTables", + "ec2:DetachInternetGateway", + "ec2:DisassociateRouteTable", + "ec2:RevokeSecurityGroupIngress", + "ec2:DeleteVpc", + "ec2:CreateSubnet", + "ec2:DescribeSubnets", + "ec2:DeleteKeyPair", + "ec2:DeleteTags", + "ec2:CreateVpc", + "ec2:DescribeAvailabilityZones", + "ec2:CreateSecurityGroup", + "ec2:ModifyVpcAttribute", + "ec2:AuthorizeSecurityGroupEgress", + "ec2:DescribeTags", + "ec2:DeleteRoute", + "ec2:DescribeSecurityGroups", + "ec2:DescribeImages", + "ec2:DescribeVpcs", + "ec2:DeleteSecurityGroup" ], "Resource": "*" }, { - "Sid": "CloudFormationPermisssions", + "Sid": "EKSPermissions", "Effect": "Allow", "Action": [ - "cloudformation:ListStacks", - "cloudformation:ListStackResources", - "cloudformation:DescribeStacks", - "cloudformation:DescribeStackResources", - "cloudformation:DescribeStackResource", - "cloudformation:DeleteStack", - "cloudformation:CreateStackSet", - "cloudformation:CreateStack" + "eks:DeleteFargateProfile", + "eks:DescribeFargateProfile", + "eks:ListTagsForResource", + "eks:UpdateClusterConfig", + "eks:DescribeNodegroup", + "eks:ListNodegroups", + "eks:DeleteCluster", + "eks:CreateFargateProfile", + "eks:DeleteNodegroup", + "eks:UpdateNodegroupConfig", + "eks:DescribeCluster", + "eks:ListClusters", + "eks:UpdateClusterVersion", + "eks:UpdateNodegroupVersion", + "eks:ListUpdates", + "eks:CreateCluster", + "eks:UntagResource", + "eks:CreateNodegroup", + "eks:ListFargateProfiles", + "eks:DescribeUpdate", + "eks:TagResource" ], "Resource": "*" }, @@ -213,52 +414,52 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "IAMPermissions", "Effect": "Allow", "Action": [ - "iam:PassRole", - "iam:ListRoles", "iam:ListRoleTags", - "iam:ListInstanceProfilesForRole", - "iam:ListInstanceProfiles", - "iam:ListAttachedRolePolicies", - "iam:GetRole", - "iam:GetInstanceProfile", - "iam:DetachRolePolicy", - "iam:DeleteRole", + "iam:RemoveRoleFromInstanceProfile", "iam:CreateRole", - "iam:AttachRolePolicy" + "iam:AttachRolePolicy", + "iam:AddRoleToInstanceProfile", + "iam:DetachRolePolicy", + "iam:GetRole", + "iam:DeleteRole", + "iam:CreateInstanceProfile", + "iam:ListInstanceProfilesForRole", + "iam:PassRole", + "iam:GetInstanceProfile", + "iam:ListRoles", + "iam:ListInstanceProfiles", + "iam:DeleteInstanceProfile" ], "Resource": "*" }, { - "Sid": "KMSPermisssions", + "Sid": "CloudFormationPermisssions", "Effect": "Allow", - "Action": "kms:ListKeys", + "Action": [ + "cloudformation:DescribeStackResource", + "cloudformation:ListStackResources", + "cloudformation:DescribeStackResources", + "cloudformation:DescribeStacks", + "cloudformation:ListStacks", + "cloudformation:CreateStack" + ], "Resource": "*" }, { - "Sid": "EKSPermisssions", + "Sid": "AutoScalingPermissions", "Effect": "Allow", "Action": [ - "eks:UpdateNodegroupVersion", - "eks:UpdateNodegroupConfig", - "eks:UpdateClusterVersion", - "eks:UpdateClusterConfig", - "eks:UntagResource", - "eks:TagResource", - "eks:ListUpdates", - "eks:ListTagsForResource", - "eks:ListNodegroups", - "eks:ListFargateProfiles", - "eks:ListClusters", - "eks:DescribeUpdate", - "eks:DescribeNodegroup", - "eks:DescribeFargateProfile", - "eks:DescribeCluster", - "eks:DeleteNodegroup", - "eks:DeleteFargateProfile", - "eks:DeleteCluster", - "eks:CreateNodegroup", - "eks:CreateFargateProfile", - "eks:CreateCluster" + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:UpdateAutoScalingGroup", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "autoscaling:CreateOrUpdateTags", + "autoscaling:DeleteAutoScalingGroup", + "autoscaling:CreateAutoScalingGroup", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:CreateLaunchConfiguration", + "autoscaling:DeleteLaunchConfiguration" ], "Resource": "*" } @@ -266,97 +467,29 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b } ``` -### Service Role Permissions +# Syncing -Rancher will create a service role with the following trust policy: +Syncing is the feature that causes Rancher to update its EKS clusters' values so they are up to date with their corresponding cluster object in the EKS console. This enables Rancher to not be the sole owner of an EKS cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other. -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Action": "sts:AssumeRole", - "Principal": { - "Service": "eks.amazonaws.com" - }, - "Effect": "Allow", - "Sid": "" - } - ] -} -``` +### How it works -This role will also have two role policy attachments with the following policies ARNs: +There are two fields on the Rancher Cluster object that must be understood to understand how syncing works: -``` -arn:aws:iam::aws:policy/AmazonEKSClusterPolicy -arn:aws:iam::aws:policy/AmazonEKSServicePolicy -``` +1. EKSConfig which is located on the Spec of the Cluster. +2. UpstreamSpec which is located on the EKSStatus field on the Status of the Cluster. -Permissions required for Rancher to create service role on users behalf during the EKS cluster creation process. +Both of which are defined by the struct EKSClusterConfigSpec found in the eks-operator project: https://github.com/rancher/eks-operator/blob/master/pkg/apis/eks.cattle.io/v1/types.go -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "IAMPermisssions", - "Effect": "Allow", - "Action": [ - "iam:AddRoleToInstanceProfile", - "iam:AttachRolePolicy", - "iam:CreateInstanceProfile", - "iam:CreateRole", - "iam:CreateServiceLinkedRole", - "iam:DeleteInstanceProfile", - "iam:DeleteRole", - "iam:DetachRolePolicy", - "iam:GetInstanceProfile", - "iam:GetRole", - "iam:ListAttachedRolePolicies", - "iam:ListInstanceProfiles", - "iam:ListInstanceProfilesForRole", - "iam:ListRoles", - "iam:ListRoleTags", - "iam:PassRole", - "iam:RemoveRoleFromInstanceProfile" - ], - "Resource": "*" - } - ] -} -``` +All fields with the exception of DisplayName, AmazonCredentialSecret, Region, and Imported are nillable on the EKSClusterConfigSpec. -### VPC Permissions +The EKSConfig represents desired state for its non-nil values. Fields that are non-nil in the EKSConfig can be thought of as “managed".When a cluster is created in Rancher, all fields are non-nil and therefore “managed”. When a pre-existing cluster is registered in rancher all nillable fields are nil and are not “managed”. Those fields become managed once their value has been changed by Rancher. -Permissions required for Rancher to create VPC and associated resources. +UpstreamSpec represents the cluster as it is in EKS and is refreshed on an interval of 5 minutes. After the UpstreamSpec has been refreshed rancher checks if the EKS cluster has an update in progress. If it is updating, nothing further is done. If it is not currently updating, any “managed” fields on EKSConfig are overwritten with their corresponding value from the recently updated UpstreamSpec. -```json -{ - "Sid": "VPCPermissions", - "Effect": "Allow", - "Action": [ - "ec2:ReplaceRoute", - "ec2:ModifyVpcAttribute", - "ec2:ModifySubnetAttribute", - "ec2:DisassociateRouteTable", - "ec2:DetachInternetGateway", - "ec2:DescribeVpcs", - "ec2:DeleteVpc", - "ec2:DeleteTags", - "ec2:DeleteSubnet", - "ec2:DeleteRouteTable", - "ec2:DeleteRoute", - "ec2:DeleteInternetGateway", - "ec2:CreateVpc", - "ec2:CreateSubnet", - "ec2:CreateSecurityGroup", - "ec2:CreateRouteTable", - "ec2:CreateRoute", - "ec2:CreateInternetGateway", - "ec2:AttachInternetGateway", - "ec2:AssociateRouteTable" - ], - "Resource": "*" -} -``` +The effective desired state can be thought of as the UpstreamSpec + all non-nil fields in the EKSConfig. This is what is displayed in the UI. + +If Rancher and another source attempt to update an EKS cluster at the same time or within the 5 minute refresh window of an update finishing, then it is likely any “managed” fields can be caught in a race condition. For example, a cluster may have PrivateAccess as a managed field. If PrivateAccess is false and then enabled in EKS console, then finishes at 11:01, and then tags are updated from Rancher before 11:05 the value will likely be overwritten. This would also occur if tags were updated while the cluster was processing the update. If the cluster was registered and the PrivateAccess fields was nil then this issue should not occur in the aforementioend case. + +### Configuring the Refresh Interval + +It is possible to change the refresh interval through the setting “eks-refresh-cron". This setting accepts values in the Cron format. The default is `*/5 * * * *`. The shorter the refresh window is the less likely any race conditions will occur, but it does increase the likelihood of encountering request limits that may be in place for AWS APIs. From d274f391a457f46d44ea451fc3314bd19fe90342 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Sun, 11 Oct 2020 12:26:56 -0700 Subject: [PATCH 04/10] Add minimum EKS permissions from PR #2741 --- .../hosted-kubernetes-clusters/eks/_index.md | 280 +++++++++++------- 1 file changed, 178 insertions(+), 102 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md index 74d9e9ef038..d5e7e13ddf0 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/_index.md @@ -19,6 +19,9 @@ Amazon EKS provides a managed control plane for your Kubernetes cluster. Amazon - [Security and Compliance](#security-and-compliance) - [Tutorial](#tutorial) - [Minimum EKS Permissions](#minimum-eks-permissions) + - [Service Role Permissions](#service-role-permissions) + - [VPC Permissions](#vpc-permissions) +- [Syncing](#syncing) # Prerequisites in Amazon Web Services @@ -331,9 +334,15 @@ This [tutorial](https://aws.amazon.com/blogs/opensource/managing-eks-clusters-ra # Minimum EKS Permissions -Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. +Documented here is a minimum set of permissions necessary to use all functionality of the EKS driver in Rancher. Additional permissions are required for Rancher to provision the `Service Role` and `VPC` resources. Optionally these resources can be created **before** the cluster creation and will be selectable when defining the cluster configuration. -Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. Some permissions (for example `ec2:CreateVpc`) are only used in situations where Rancher handles the creation of certain resources. +Resource | Description +---------|------------ +Service Role | The service role provides Kubernetes the permissions it requires to manage resources on your behalf. Rancher can create the service role with the following [Service Role Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#service-role-permissions). +VPC | Provides isolated network resouces utilised by EKS and worker nodes. Rancher can create the VPC resouces with the follwoing [VPC Permissions](http://localhost:9001/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/eks/#vpc-permissions). + + +Resource targeting uses `*` as the ARN of many of the resources created cannot be known prior to creating the EKS cluster in Rancher. ```json { @@ -343,92 +352,26 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "EC2Permisssions", "Effect": "Allow", "Action": [ - "ec2:AuthorizeSecurityGroupIngress", - "ec2:DeleteSubnet", - "ec2:CreateKeyPair", - "ec2:AttachInternetGateway", - "ec2:ReplaceRoute", - "ec2:DeleteRouteTable", - "ec2:AssociateRouteTable", - "ec2:DescribeInternetGateways", - "ec2:CreateRoute", - "ec2:CreateInternetGateway", - "ec2:RevokeSecurityGroupEgress", - "ec2:DescribeAccountAttributes", - "ec2:DeleteInternetGateway", - "ec2:DescribeKeyPairs", - "ec2:CreateTags", - "ec2:CreateRouteTable", - "ec2:DescribeRouteTables", - "ec2:DetachInternetGateway", - "ec2:DisassociateRouteTable", "ec2:RevokeSecurityGroupIngress", - "ec2:DeleteVpc", - "ec2:CreateSubnet", - "ec2:DescribeSubnets", - "ec2:DeleteKeyPair", - "ec2:DeleteTags", - "ec2:CreateVpc", - "ec2:DescribeAvailabilityZones", - "ec2:CreateSecurityGroup", - "ec2:ModifyVpcAttribute", - "ec2:AuthorizeSecurityGroupEgress", - "ec2:DescribeTags", - "ec2:DeleteRoute", - "ec2:DescribeSecurityGroups", - "ec2:DescribeImages", + "ec2:RevokeSecurityGroupEgress", "ec2:DescribeVpcs", - "ec2:DeleteSecurityGroup" - ], - "Resource": "*" - }, - { - "Sid": "EKSPermissions", - "Effect": "Allow", - "Action": [ - "eks:DeleteFargateProfile", - "eks:DescribeFargateProfile", - "eks:ListTagsForResource", - "eks:UpdateClusterConfig", - "eks:DescribeNodegroup", - "eks:ListNodegroups", - "eks:DeleteCluster", - "eks:CreateFargateProfile", - "eks:DeleteNodegroup", - "eks:UpdateNodegroupConfig", - "eks:DescribeCluster", - "eks:ListClusters", - "eks:UpdateClusterVersion", - "eks:UpdateNodegroupVersion", - "eks:ListUpdates", - "eks:CreateCluster", - "eks:UntagResource", - "eks:CreateNodegroup", - "eks:ListFargateProfiles", - "eks:DescribeUpdate", - "eks:TagResource" - ], - "Resource": "*" - }, - { - "Sid": "IAMPermissions", - "Effect": "Allow", - "Action": [ - "iam:ListRoleTags", - "iam:RemoveRoleFromInstanceProfile", - "iam:CreateRole", - "iam:AttachRolePolicy", - "iam:AddRoleToInstanceProfile", - "iam:DetachRolePolicy", - "iam:GetRole", - "iam:DeleteRole", - "iam:CreateInstanceProfile", - "iam:ListInstanceProfilesForRole", - "iam:PassRole", - "iam:GetInstanceProfile", - "iam:ListRoles", - "iam:ListInstanceProfiles", - "iam:DeleteInstanceProfile" + "ec2:DescribeTags", + "ec2:DescribeSubnets", + "ec2:DescribeSecurityGroups", + "ec2:DescribeRouteTables", + "ec2:DescribeKeyPairs", + "ec2:DescribeInternetGateways", + "ec2:DescribeImages", + "ec2:DescribeAvailabilityZones", + "ec2:DescribeAccountAttributes", + "ec2:DeleteTags", + "ec2:DeleteSecurityGroup", + "ec2:DeleteKeyPair", + "ec2:CreateTags", + "ec2:CreateSecurityGroup", + "ec2:CreateKeyPair", + "ec2:AuthorizeSecurityGroupIngress", + "ec2:AuthorizeSecurityGroupEgress" ], "Resource": "*" }, @@ -436,30 +379,67 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b "Sid": "CloudFormationPermisssions", "Effect": "Allow", "Action": [ - "cloudformation:DescribeStackResource", - "cloudformation:ListStackResources", - "cloudformation:DescribeStackResources", - "cloudformation:DescribeStacks", "cloudformation:ListStacks", + "cloudformation:ListStackResources", + "cloudformation:DescribeStacks", + "cloudformation:DescribeStackResources", + "cloudformation:DescribeStackResource", + "cloudformation:DeleteStack", + "cloudformation:CreateStackSet", "cloudformation:CreateStack" ], "Resource": "*" }, { - "Sid": "AutoScalingPermissions", + "Sid": "IAMPermissions", "Effect": "Allow", "Action": [ - "autoscaling:DescribeAutoScalingGroups", - "autoscaling:UpdateAutoScalingGroup", - "autoscaling:TerminateInstanceInAutoScalingGroup", - "autoscaling:CreateOrUpdateTags", - "autoscaling:DeleteAutoScalingGroup", - "autoscaling:CreateAutoScalingGroup", - "autoscaling:DescribeAutoScalingInstances", - "autoscaling:DescribeLaunchConfigurations", - "autoscaling:DescribeScalingActivities", - "autoscaling:CreateLaunchConfiguration", - "autoscaling:DeleteLaunchConfiguration" + "iam:PassRole", + "iam:ListRoles", + "iam:ListRoleTags", + "iam:ListInstanceProfilesForRole", + "iam:ListInstanceProfiles", + "iam:ListAttachedRolePolicies", + "iam:GetRole", + "iam:GetInstanceProfile", + "iam:DetachRolePolicy", + "iam:DeleteRole", + "iam:CreateRole", + "iam:AttachRolePolicy" + ], + "Resource": "*" + }, + { + "Sid": "KMSPermisssions", + "Effect": "Allow", + "Action": "kms:ListKeys", + "Resource": "*" + }, + { + "Sid": "EKSPermisssions", + "Effect": "Allow", + "Action": [ + "eks:UpdateNodegroupVersion", + "eks:UpdateNodegroupConfig", + "eks:UpdateClusterVersion", + "eks:UpdateClusterConfig", + "eks:UntagResource", + "eks:TagResource", + "eks:ListUpdates", + "eks:ListTagsForResource", + "eks:ListNodegroups", + "eks:ListFargateProfiles", + "eks:ListClusters", + "eks:DescribeUpdate", + "eks:DescribeNodegroup", + "eks:DescribeFargateProfile", + "eks:DescribeCluster", + "eks:DeleteNodegroup", + "eks:DeleteFargateProfile", + "eks:DeleteCluster", + "eks:CreateNodegroup", + "eks:CreateFargateProfile", + "eks:CreateCluster" ], "Resource": "*" } @@ -467,6 +447,102 @@ Resource targeting uses `*` as the ARN of many of the resources created cannot b } ``` +### Service Role Permissions + +Rancher will create a service role with the following trust policy: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Action": "sts:AssumeRole", + "Principal": { + "Service": "eks.amazonaws.com" + }, + "Effect": "Allow", + "Sid": "" + } + ] +} +``` + +This role will also have two role policy attachments with the following policies ARNs: + +``` +arn:aws:iam::aws:policy/AmazonEKSClusterPolicy +arn:aws:iam::aws:policy/AmazonEKSServicePolicy +``` + +Permissions required for Rancher to create service role on users behalf during the EKS cluster creation process. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "IAMPermisssions", + "Effect": "Allow", + "Action": [ + "iam:AddRoleToInstanceProfile", + "iam:AttachRolePolicy", + "iam:CreateInstanceProfile", + "iam:CreateRole", + "iam:CreateServiceLinkedRole", + "iam:DeleteInstanceProfile", + "iam:DeleteRole", + "iam:DetachRolePolicy", + "iam:GetInstanceProfile", + "iam:GetRole", + "iam:ListAttachedRolePolicies", + "iam:ListInstanceProfiles", + "iam:ListInstanceProfilesForRole", + "iam:ListRoles", + "iam:ListRoleTags", + "iam:PassRole", + "iam:RemoveRoleFromInstanceProfile" + ], + "Resource": "*" + } + ] +} +``` + +### VPC Permissions + +Permissions required for Rancher to create VPC and associated resources. + +```json +{ + "Sid": "VPCPermissions", + "Effect": "Allow", + "Action": [ + "ec2:ReplaceRoute", + "ec2:ModifyVpcAttribute", + "ec2:ModifySubnetAttribute", + "ec2:DisassociateRouteTable", + "ec2:DetachInternetGateway", + "ec2:DescribeVpcs", + "ec2:DeleteVpc", + "ec2:DeleteTags", + "ec2:DeleteSubnet", + "ec2:DeleteRouteTable", + "ec2:DeleteRoute", + "ec2:DeleteInternetGateway", + "ec2:CreateVpc", + "ec2:CreateSubnet", + "ec2:CreateSecurityGroup", + "ec2:CreateRouteTable", + "ec2:CreateRoute", + "ec2:CreateInternetGateway", + "ec2:AttachInternetGateway", + "ec2:AssociateRouteTable" + ], + "Resource": "*" +} +``` + + # Syncing Syncing is the feature that causes Rancher to update its EKS clusters' values so they are up to date with their corresponding cluster object in the EKS console. This enables Rancher to not be the sole owner of an EKS cluster’s state. Its largest limitation is that processing an update from Rancher and another source at the same time or within 5 minutes of one finishing may cause the state from one source to completely overwrite the other. From 20796c3693e9441560e17abb295917c2061e4e45 Mon Sep 17 00:00:00 2001 From: catherineluse Date: Mon, 12 Oct 2020 14:45:16 -0700 Subject: [PATCH 05/10] Recommend Fleet on pipelines page --- content/rancher/v2.x/en/pipelines/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/rancher/v2.x/en/pipelines/_index.md b/content/rancher/v2.x/en/pipelines/_index.md index dbebec3695b..e15f4ff8724 100644 --- a/content/rancher/v2.x/en/pipelines/_index.md +++ b/content/rancher/v2.x/en/pipelines/_index.md @@ -5,6 +5,8 @@ aliases: - /rancher/v2.x/en/k8s-in-rancher/pipelines --- +> As of Rancher v2.5, Git-based deployment pipelines are now recommended to be handled with Rancher Continuous Delivery powered by [Fleet,]({{}}/rancher/v2.x/en/deploy-across-cluster) available in Cluster Explorer. + Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Using Rancher, you can integrate with a GitHub repository to setup a continuous integration (CI) pipeline. From e6ad02c6aa329452a0301391082d3f23aee42740 Mon Sep 17 00:00:00 2001 From: catherineluse Date: Mon, 12 Oct 2020 15:01:16 -0700 Subject: [PATCH 06/10] Update note on pipelines page --- content/rancher/v2.x/en/pipelines/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/pipelines/_index.md b/content/rancher/v2.x/en/pipelines/_index.md index e15f4ff8724..c1293be7518 100644 --- a/content/rancher/v2.x/en/pipelines/_index.md +++ b/content/rancher/v2.x/en/pipelines/_index.md @@ -5,7 +5,7 @@ aliases: - /rancher/v2.x/en/k8s-in-rancher/pipelines --- -> As of Rancher v2.5, Git-based deployment pipelines are now recommended to be handled with Rancher Continuous Delivery powered by [Fleet,]({{}}/rancher/v2.x/en/deploy-across-cluster) available in Cluster Explorer. +> As of Rancher v2.5, Git-based deployment pipelines are now recommended to be handled with Rancher Continuous Delivery powered by [Fleet,]({{}}/rancher/v2.x/en/deploy-across-clusters/fleet) available in Cluster Explorer. Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users. From 79b619f7cf951c8121f86b12b06bda32a861b320 Mon Sep 17 00:00:00 2001 From: mbookham7 <34167192+mbookham7@users.noreply.github.com> Date: Tue, 13 Oct 2020 17:00:10 +0100 Subject: [PATCH 07/10] Update _index.md changed 'both nodes' to 'any of the three nodes'. This is for the RKE install and there are three node not two. --- .../infrastructure-tutorials/infra-for-ha/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-ha/_index.md b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-ha/_index.md index 5949994f823..a6e267c4ac9 100644 --- a/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-ha/_index.md +++ b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-ha/_index.md @@ -29,7 +29,7 @@ For an example of one way to set up Linux nodes, refer to this [tutorial]({{ Date: Tue, 13 Oct 2020 09:58:55 -0700 Subject: [PATCH 08/10] Remove merge conflict error --- .../rancher/v2.x/en/monitoring-alerting/_index.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/content/rancher/v2.x/en/monitoring-alerting/_index.md b/content/rancher/v2.x/en/monitoring-alerting/_index.md index 3446b650c45..c9d75dae3b0 100644 --- a/content/rancher/v2.x/en/monitoring-alerting/_index.md +++ b/content/rancher/v2.x/en/monitoring-alerting/_index.md @@ -188,18 +188,6 @@ The default values in the table below are the minimum required resource limits a At least 50Gi storage is recommended. - -<<<<<<< HEAD -<<<<<<< HEAD -<<<<<<< HEAD -The data from Prometheus is used as the data source for the Grafana dashboard. Multiple data sources can be configured for Grafana. -======= -For more information about using the Promethus adapter, refer to this [documentation.](https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/docs/config-walkthrough.md) ->>>>>>> Update monitoring docs -======= ->>>>>>> Revise monitoring docs -======= # Known Issues There is a [known issue](https://github.com/rancher/rancher/issues/28787#issuecomment-693611821) that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend to setting `prometheus.prometheusSpec.resources.memory.limit` to 2500Mi` and `prometheus.prometheusSpec.resources.memory.request` to 1750Mi. ->>>>>>> Add Arvind's changes to monitoring docs From 1f7e1a1510dce7daa323a69af711d15a481d6e09 Mon Sep 17 00:00:00 2001 From: catherineluse Date: Tue, 13 Oct 2020 11:21:04 -0700 Subject: [PATCH 09/10] Restore Helm 2 installation docs --- .../advanced/air-gap-helm2/_index.md | 42 +- .../air-gap-helm2/install-rancher/_index.md | 42 +- .../air-gap-helm2/launch-kubernetes/_index.md | 83 ++++ .../populate-private-registry/_index.md | 275 ++++++++++++ .../air-gap-helm2/prepare-nodes/_index.md | 106 +++++ .../resources/advanced/helm2/_index.md | 58 ++- .../advanced/helm2/create-nodes-lb/_index.md | 32 ++ .../helm2/create-nodes-lb/nginx/_index.md | 81 ++++ .../helm2/create-nodes-lb/nlb/_index.md | 177 ++++++++ .../advanced/helm2/helm-init/_index.md | 2 +- .../helm2/helm-init/troubleshooting/_index.md | 25 ++ .../advanced/helm2/helm-rancher/_index.md | 10 +- .../helm-rancher/chart-options/_index.md | 247 +++++++++++ .../helm2/helm-rancher/tls-secrets/_index.md | 34 ++ .../helm-rancher/troubleshooting/_index.md | 135 ++++++ .../advanced/helm2/kubernetes-rke/_index.md | 134 ++++++ .../kubernetes-rke/troubleshooting/_index.md | 54 +++ .../advanced/helm2/rke-add-on/_index.md | 18 + .../helm2/rke-add-on/api-auditing/_index.md | 56 +++ .../helm2/rke-add-on/layer-4-lb/_index.md | 402 ++++++++++++++++++ .../helm2/rke-add-on/layer-4-lb/nlb/_index.md | 182 ++++++++ .../helm2/rke-add-on/layer-7-lb/_index.md | 289 +++++++++++++ .../helm2/rke-add-on/layer-7-lb/alb/_index.md | 104 +++++ .../rke-add-on/layer-7-lb/nginx/_index.md | 41 ++ .../advanced/helm2/rke-add-on/proxy/_index.md | 71 ++++ .../404-default-backend/_index.md | 49 +++ .../rke-add-on/troubleshooting/_index.md | 33 ++ .../generic-troubleshooting/_index.md | 162 +++++++ .../job-complete-status/_index.md | 63 +++ 29 files changed, 2975 insertions(+), 32 deletions(-) create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/launch-kubernetes/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/populate-private-registry/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/prepare-nodes/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nginx/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nlb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/troubleshooting/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/chart-options/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/tls-secrets/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/troubleshooting/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/troubleshooting/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/api-auditing/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/nlb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/alb/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/nginx/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/proxy/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/404-default-backend/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/generic-troubleshooting/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/job-complete-status/_index.md diff --git a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/_index.md index c85b3d857f9..720f5d24b3e 100644 --- a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/_index.md +++ b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/_index.md @@ -1,8 +1,44 @@ --- -title: Air Gap Install with Helm 2 -weight: 11 +title: Installing Rancher in an Air Gapped Environment with Helm 2 +weight: 2 aliases: + - /rancher/v2.x/en/installation/air-gap-installation/ + - /rancher/v2.x/en/installation/air-gap-high-availability/ + - /rancher/v2.x/en/installation/air-gap-single-node/ - /rancher/v2.x/en/installation/options/air-gap-helm2 --- -This page section preserves an older version of the Rancher installation documentation in which Helm 2 was used to install Rancher. \ No newline at end of file +> After Helm 3 was released, the Rancher installation instructions were updated to use Helm 3. +> +> If you are using Helm 2, we recommend [migrating to Helm 3](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) because it is simpler to use and more secure than Helm 2. +> +> This section provides a copy of the older instructions for installing Rancher on a Kubernetes cluster using Helm 2 in an air air gap environment, and it is intended to be used if upgrading to Helm 3 is not feasible. + +This section is about installations of Rancher server in an air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. + +Throughout the installations instructions, there will be _tabs_ for either a high availability Kubernetes installation or a single-node Docker installation. + +### Air Gapped Kubernetes Installations + +This section covers how to install Rancher on a Kubernetes cluster in an air gapped environment. + +A Kubernetes installation is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails. + +### Air Gapped Docker Installations + +These instructions also cover how to install Rancher on a single node in an air gapped environment. + +The Docker installation is for Rancher users that are wanting to test out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server. + +> **Important:** If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker Installation to a Kubernetes Installation. + +Instead of running the Docker installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation. + +# Installation Outline + +- [1. Prepare your Node(s)]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/prepare-nodes/) +- [2. Collect and Publish Images to your Private Registry]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/populate-private-registry/) +- [3. Launch a Kubernetes Cluster with RKE]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/launch-kubernetes/) +- [4. Install Rancher]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/) + +### [Next: Prepare your Node(s)]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/prepare-nodes/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/install-rancher/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/install-rancher/_index.md index 84c1154f0b9..251d448b9c0 100644 --- a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/install-rancher/_index.md @@ -34,7 +34,7 @@ From a system that has access to the internet, fetch the latest Helm chart and c helm init -c ``` -2. Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{}}/rancher/v2.x/en/installation/resources/chart-options/#helm-chart-repositories). +2. Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{}}/rancher/v2.x/en/installation/options/server-tags/#helm-chart-repositories). {{< release-channel >}} ``` helm repo add rancher- https://releases.rancher.com/server-charts/ @@ -45,13 +45,15 @@ From a system that has access to the internet, fetch the latest Helm chart and c helm fetch rancher-/rancher ``` +> Want additional options? Need help troubleshooting? See [Kubernetes Install: Advanced Options]({{}}/rancher/v2.x/en/installation/k8s-install/helm-rancher/#advanced-configurations). + ### B. Choose your SSL Configuration Rancher Server is designed to be secure by default and requires SSL/TLS configuration. When Rancher is installed on an air gapped Kubernetes cluster, there are two recommended options for the source of the certificate. -> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer]({{}}/rancher/v2.x/en/installation/resources/chart-options/#external-tls-termination). +> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer]({{}}/rancher/v2.x/en/installation/options/chart-options/#external-tls-termination). | Configuration | Chart option | Description | Requires cert-manager | | ------------------------------------------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | @@ -86,12 +88,12 @@ By default, Rancher generates a CA and uses cert-manager to issue the certificat 1. Fetch the latest cert-manager chart available from the [Helm chart repository](https://hub.helm.sh/charts/jetstack/cert-manager). ```plain - helm fetch jetstack/cert-manager --version v0.12.0 + helm fetch jetstack/cert-manager --version v0.14.2 ``` 1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. ```plain - helm template ./cert-manager-v0.12.0.tgz --output-dir . \ + helm template ./cert-manager-v0.14.2.tgz --output-dir . \ --name cert-manager --namespace cert-manager \ --set image.repository=/quay.io/jetstack/cert-manager-controller --set webhook.image.repository=/quay.io/jetstack/cert-manager-webhook @@ -100,7 +102,7 @@ By default, Rancher generates a CA and uses cert-manager to issue the certificat 1. Download the required CRD file for cert-manager ```plain - curl -L -o cert-manager/cert-manager-crd.yaml https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml + curl -L -o cert-manager/cert-manager-crd.yaml https://raw.githubusercontent.com/jetstack/cert-manager/release-0.14/deploy/manifests/00-crds.yaml ``` 1. Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools. @@ -162,7 +164,7 @@ If you are using a Private CA signed cert, add `--set privateCA=true` following --set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts ``` -Then refer to [Adding TLS Secrets]({{}}/rancher/v2.x/en/installation/resources/encryption/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +Then refer to [Adding TLS Secrets]({{}}/rancher/v2.x/en/installation/options/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. {{% /accordion %}} @@ -209,14 +211,14 @@ kubectl -n cattle-system apply -R -f ./rancher ### E. For Rancher versions prior to v2.3.0, Configure System Charts -If you are installing Rancher versions prior to v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in Github, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts]({{}}/rancher/v2.x/en/installation/resources/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0). +If you are installing Rancher versions prior to v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in Github, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts]({{}}/rancher/v2.x/en/installation/options/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0). ### Additional Resources These resources could be helpful when installing Rancher: -- [Rancher Helm chart options]({{}}/rancher/v2.x/en/installation/resources/chart-options/) -- [Adding TLS secrets]({{}}/rancher/v2.x/en/installation/resources/encryption/tls-secrets/) +- [Rancher Helm chart options]({{}}/rancher/v2.x/en/installation/options/chart-options/) +- [Adding TLS secrets]({{}}/rancher/v2.x/en/installation/options/tls-secrets/) - [Troubleshooting Rancher Kubernetes Installations]({{}}/rancher/v2.x/en/installation/options/troubleshooting/) {{% /tab %}} @@ -233,10 +235,10 @@ For security purposes, SSL (Secure Sockets Layer) is required when using Rancher > **Do you want to...** > -> - Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{}}/rancher/v2.x/en/installation/resources/chart-options/#additional-trusted-cas). -> - Record all transactions with the Rancher API? See [API Auditing]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/advanced/#api-audit-log). +> - Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{}}/rancher/v2.x/en/installation/options/chart-options/#additional-trusted-cas). +> - Record all transactions with the Rancher API? See [API Auditing]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/#api-audit-log). -- For Rancher prior to v2.3.0, you will need to mirror the `system-charts` repository to a location in your network that Rancher can reach. Then, after Rancher is installed, you will need to configure Rancher to use that repository. For details, refer to the documentation on [setting up the system charts for Rancher prior to v2.3.0.]({{}}/rancher/v2.x/en/installation/resources/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0) +- For Rancher prior to v2.3.0, you will need to mirror the `system-charts` repository to a location in your network that Rancher can reach. Then, after Rancher is installed, you will need to configure Rancher to use that repository. For details, refer to the documentation on [setting up the system charts for Rancher prior to v2.3.0.]({{}}/rancher/v2.x/en/installation/options/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0) Choose from the following options: @@ -249,7 +251,7 @@ Log into your Linux host, and then run the installation command below. When ente | Placeholder | Description | | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `` | Your private registry URL and port. | -| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/resources/chart-options/) that you want to install. | +| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/options/server-tags/) that you want to install. | ``` docker run -d --restart=unless-stopped \ @@ -267,8 +269,8 @@ In development or testing environments where your team will access your Rancher > **Prerequisites:** > From a computer with an internet connection, create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice. > -> - The certificate files must be in [PEM format]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/advanced/#pem). -> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/advanced/#cert-order). +> - The certificate files must be in [PEM format]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/#pem). +> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/#cert-order). After creating your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Use the `-v` flag and provide the path to your certificates to mount them in your container. @@ -277,9 +279,9 @@ After creating your certificate, log into your Linux host, and then run the inst | `` | The path to the directory containing your certificate files. | | `` | The path to your full certificate chain. | | `` | The path to the private key for your certificate. | -| `` | The path to the certificate authority's certificate. | +| `` | The path to the certificate authority's certificate. | | `` | Your private registry URL and port. | -| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/resources/chart-options/) that you want to install. | +| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/options/server-tags/) that you want to install. | ``` docker run -d --restart=unless-stopped \ @@ -297,7 +299,7 @@ docker run -d --restart=unless-stopped \ In development or testing environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings. -> **Prerequisite:** The certificate files must be in [PEM format]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/advanced/#pem). +> **Prerequisite:** The certificate files must be in [PEM format]({{}}/rancher/v2.x/en/installation/other-installation-methods/single-node-docker/#pem). After obtaining your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary. @@ -307,7 +309,7 @@ After obtaining your certificate, log into your Linux host, and then run the ins | `` | The path to your full certificate chain. | | `` | The path to the private key for your certificate. | | `` | Your private registry URL and port. | -| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/resources/chart-options/) that you want to install. | +| `` | The release tag of the [Rancher version]({{}}/rancher/v2.x/en/installation/options/server-tags/) that you want to install. | > **Note:** Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. @@ -326,7 +328,7 @@ docker run -d --restart=unless-stopped \ If you are installing Rancher v2.3.0+, the installation is complete. -If you are installing Rancher versions prior to v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in Github, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts]({{}}/rancher/v2.x/en/installation/resources/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0). +If you are installing Rancher versions prior to v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in Github, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts]({{}}/rancher/v2.x/en/installation/options/local-system-charts/#setting-up-system-charts-for-rancher-prior-to-v2-3-0). {{% /tab %}} {{% /tabs %}} diff --git a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/launch-kubernetes/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/launch-kubernetes/_index.md new file mode 100644 index 00000000000..8b8d2a8cb6a --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/launch-kubernetes/_index.md @@ -0,0 +1,83 @@ +--- +title: '3. Install Kubernetes with RKE (Kubernetes Installs Only)' +weight: 300 +aliases: + - /rancher/v2.x/en/installation/air-gap-high-availability/install-kube + - /rancher/v2.x/en/installation/options/air-gap-helm2/launch-kubernetes +--- + +This section is about how to prepare to launch a Kubernetes cluster which is used to deploy Rancher server for your air gapped environment. + +Since a Kubernetes Installation requires a Kubernetes cluster, we will create a Kubernetes cluster using [Rancher Kubernetes Engine]({{}}/rke/latest/en/) (RKE). Before being able to start your Kubernetes cluster, you'll need to [install RKE]({{}}/rke/latest/en/installation/) and create a RKE config file. + +- [A. Create an RKE Config File](#a-create-an-rke-config-file) +- [B. Run RKE](#b-run-rke) +- [C. Save Your Files](#c-save-your-files) + +### A. Create an RKE Config File + +From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create a new file named `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. + +Replace values in the code sample below with help of the _RKE Options_ table. Use the IP address or DNS names of the [3 nodes]({{}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts) you created. + +> **Tip:** For more details on the options available, see the RKE [Config Options]({{}}/rke/latest/en/config-options/). + +
RKE Options
+ +| Option | Required | Description | +| ------------------ | -------------------- | --------------------------------------------------------------------------------------- | +| `address` | ✓ | The DNS or IP address for the node within the air gap network. | +| `user` | ✓ | A user that can run docker commands. | +| `role` | ✓ | List of Kubernetes roles assigned to the node. | +| `internal_address` | optional1 | The DNS or IP address used for internal cluster traffic. | +| `ssh_key_path` | | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`). | + +> 1 Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls. + +```yaml +nodes: + - address: 10.10.3.187 # node air gap network IP + internal_address: 172.31.7.22 # node intra-cluster IP + user: rancher + role: ['controlplane', 'etcd', 'worker'] + ssh_key_path: /home/user/.ssh/id_rsa + - address: 10.10.3.254 # node air gap network IP + internal_address: 172.31.13.132 # node intra-cluster IP + user: rancher + role: ['controlplane', 'etcd', 'worker'] + ssh_key_path: /home/user/.ssh/id_rsa + - address: 10.10.3.89 # node air gap network IP + internal_address: 172.31.3.216 # node intra-cluster IP + user: rancher + role: ['controlplane', 'etcd', 'worker'] + ssh_key_path: /home/user/.ssh/id_rsa + +private_registries: + - url: # private registry url + user: rancher + password: '*********' + is_default: true +``` + +### B. Run RKE + +After configuring `rancher-cluster.yml`, bring up your Kubernetes cluster: + +``` +rke up --config ./rancher-cluster.yml +``` + +### C. Save Your Files + +> **Important** +> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster. + +Save a copy of the following files in a secure location: + +- `rancher-cluster.yml`: The RKE cluster configuration file. +- `kube_config_rancher-cluster.yml`: The [Kubeconfig file]({{}}/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster. +- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file]({{}}/rke/latest/en/installation/#kubernetes-cluster-state), this file contains credentials for full access to the cluster.

_The Kubernetes Cluster State file is only created when using RKE v0.2.0 or higher._ + +> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file. + +### [Next: Install Rancher]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/populate-private-registry/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/populate-private-registry/_index.md new file mode 100644 index 00000000000..3b52765be99 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/populate-private-registry/_index.md @@ -0,0 +1,275 @@ +--- +title: '2. Collect and Publish Images to your Private Registry' +weight: 200 +aliases: + - /rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/ + - /rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/ + - /rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/ + - /rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/ + - /rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/ + - /rancher/v2.x/en/installation/options/air-gap-helm2/populate-private-registry +--- + +> **Prerequisites:** You must have a [private registry](https://docs.docker.com/registry/deploying/) available to use. +> +> **Note:** Populating the private registry with images is the same process for HA and Docker installations, the differences in this section is based on whether or not you are planning to provision a Windows cluster or not. + +By default, all images used to [provision Kubernetes clusters]({{}}/rancher/v2.x/en/cluster-provisioning/) or launch any [tools]({{}}/rancher/v2.x/en/cluster-admin/tools/) in Rancher, e.g. monitoring, pipelines, alerts, are pulled from Docker Hub. In an air gap installation of Rancher, you will need a private registry that is located somewhere accessible by your Rancher server. Then, you will load the registry with all the images. + +This section describes how to set up your private registry so that when you install Rancher, Rancher will pull all the required images from this registry. + +By default, we provide the steps of how to populate your private registry assuming you are provisioning Linux only clusters, but if you plan on provisioning any [Windows clusters]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/), there are separate instructions to support the images needed for a Windows cluster. + +{{% tabs %}} +{{% tab "Linux Only Clusters" %}} + +For Rancher servers that will only provision Linux clusters, these are the steps to populate your private registry. + +A. Find the required assets for your Rancher version
+B. Collect all the required images
+C. Save the images to your workstation
+D. Populate the private registry + +### Prerequisites + +These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space. + +If you will use ARM64 hosts, the registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests. + +### A. Find the required assets for your Rancher version + +1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. Click **Assets*.* + +2. From the release's **Assets** section, download the following files: + +| Release File | Description | +| ---------------- | -------------- | +| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. | +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + +### B. Collect all the required images (For Kubernetes Installs using Rancher Generated Self-Signed Certificate) + +In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well. You skip this step if you are using you using your own certificates. + +1. Fetch the latest `cert-manager` Helm chart and parse the template for image details: + + > **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation]({{}}/rancher/v2.x/en/installation/options/upgrading-cert-manager/). + + ```plain + helm repo add jetstack https://charts.jetstack.io + helm repo update + helm fetch jetstack/cert-manager --version v0.14.2 + helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt + ``` + +2. Sort and unique the images list to remove any overlap between the sources: + + ```plain + sort -u rancher-images.txt -o rancher-images.txt + ``` + +### C. Save the images to your workstation + +1. Make `rancher-save-images.sh` an executable: + ``` + chmod +x rancher-save-images.sh + ``` + +1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images: + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + **Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory. + +### D. Populate the private registry + +Move the images in the `rancher-images.tar.gz` to your private registry using the scripts to load the images. The `rancher-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script. + +1. Log into your private registry if required: + ```plain + docker login + ``` +1. Make `rancher-load-images.sh` an executable: + ``` + chmod +x rancher-load-images.sh + ``` + +1. Use `rancher-load-images.sh` to extract, tag and push `rancher-images.txt` and `rancher-images.tar.gz` to your private registry: + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` +{{% /tab %}} +{{% tab "Linux and Windows Clusters" %}} + +_Available as of v2.3.0_ + +For Rancher servers that will provision Linux and Windows clusters, there are distinctive steps to populate your private registry for the Windows images and the Linux images. Since a Windows cluster is a mix of Linux and Windows nodes, the Linux images pushed into the private registry are manifests. + +### Windows Steps + +The Windows images need to be collected and pushed from a Windows server workstation. + +A. Find the required assets for your Rancher version
+B. Save the images to your Windows Server workstation
+C. Prepare the Docker daemon
+D. Populate the private registry + +{{% accordion label="Collecting and Populating Windows Images into the Private Registry"%}} + +### Prerequisites + +These steps expect you to use a Windows Server 1809 workstation that has internet access, access to your private registry, and at least 50 GB of disk space. + +The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters. + +Your registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests. + +### A. Find the required assets for your Rancher version + +1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. + +2. From the release's "Assets" section, download the following files: + +| Release File | Description | +|------------------------|-------------------| +| `rancher-windows-images.txt` | This file contains a list of Windows images needed to provision Windows clusters. | +| `rancher-save-images.ps1` | This script pulls all the images in the `rancher-windows-images.txt` from Docker Hub and saves all of the images as `rancher-windows-images.tar.gz`. | +| `rancher-load-images.ps1` | This script loads the images from the `rancher-windows-images.tar.gz` file and pushes them to your private registry. | + +### B. Save the images to your Windows Server workstation + +1. Using `powershell`, go to the directory that has the files that were downloaded in the previous step. + +1. Run `rancher-save-images.ps1` to create a tarball of all the required images: + + ```plain + ./rancher-save-images.ps1 + ``` + + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-windows-images.tar.gz`. Check that the output is in the directory. + +### C. Prepare the Docker daemon + +Append your private registry address to the `allow-nondistributable-artifacts` config field in the Docker daemon (`C:\ProgramData\Docker\config\daemon.json`). Since the base image of Windows images are maintained by the `mcr.microsoft.com` registry, this step is required as the layers in the Microsoft registry are missing from Docker Hub and need to be pulled into the private registry. + + ``` + { + ... + "allow-nondistributable-artifacts": [ + ... + "" + ] + ... + } + ``` + +### D. Populate the private registry + +Move the images in the `rancher-windows-images.tar.gz` to your private registry using the scripts to load the images. The `rancher-windows-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.ps1` script. + +1. Using `powershell`, log into your private registry if required: + ```plain + docker login + ``` + +1. Using `powershell`, use `rancher-load-images.ps1` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry: + ```plain + ./rancher-load-images.ps1 --registry + ``` + +{{% /accordion %}} + +### Linux Steps + +The Linux images needs to be collected and pushed from a Linux host, but _must be done after_ populating the Windows images into the private registry. These step are different from the Linux only steps as the Linux images that are pushed will actually manifests that support Windows and Linux images. + +A. Find the required assets for your Rancher version
+B. Collect all the required images
+C. Save the images to your Linux workstation
+D. Populate the private registry + +{{% accordion label="Collecting and Populating Linux Images into the Private Registry" %}} + +### Prerequisites + +You must populate the private registry with the Windows images before populating the private registry with Linux images. If you have already populated the registry with Linux images, you will need to follow these instructions again as they will publish manifests that support Windows and Linux images. + +These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space. + +The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters. + +### A. Find the required assets for your Rancher version + +1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. + +2. From the release's **Assets** section, download the following files, which are required to install Rancher in an air gap environment: + +| Release File | Description | +|----------------------------|------| +| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. | +| `rancher-windows-images.txt` | This file contains a list of images needed to provision Windows clusters. | +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + +### B. Collect all the required images + +**For Kubernetes Installs using Rancher Generated Self-Signed Certificate:** In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well. You skip this step if you are using you using your own certificates. + + 1. Fetch the latest `cert-manager` Helm chart and parse the template for image details: + > **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation]({{}}/rancher/v2.x/en/installation/options/upgrading-cert-manager/). + ```plain + helm repo add jetstack https://charts.jetstack.io + helm repo update + helm fetch jetstack/cert-manager --version v0.14.2 + helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt + ``` + + 2. Sort and unique the images list to remove any overlap between the sources: + ```plain + sort -u rancher-images.txt -o rancher-images.txt + ``` + +### C. Save the images to your workstation + +1. Make `rancher-save-images.sh` an executable: + ``` + chmod +x rancher-save-images.sh + ``` + +1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images: + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + + **Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory. + +### D. Populate the private registry + +Move the images in the `rancher-images.tar.gz` to your private registry using the `rancher-load-images.sh script` to load the images. The `rancher-images.txt` / `rancher-windows-images.txt` image list is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script. + +1. Log into your private registry if required: + ```plain + docker login + ``` + +1. Make `rancher-load-images.sh` an executable: + ``` + chmod +x rancher-load-images.sh + ``` + +1. Use `rancher-load-images.sh` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry: + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt \ + --windows-image-list ./rancher-windows-images.txt \ + --registry + ``` + +{{% /accordion %}} + +{{% /tab %}} +{{% /tabs %}} + +### [Next: Kubernetes Installs - Launch a Kubernetes Cluster with RKE]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/launch-kubernetes/) + +### [Next: Docker Installs - Install Rancher]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/install-rancher/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/prepare-nodes/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/prepare-nodes/_index.md new file mode 100644 index 00000000000..e6fd8736cf3 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/air-gap-helm2/prepare-nodes/_index.md @@ -0,0 +1,106 @@ +--- +title: '1. Prepare your Node(s)' +weight: 100 +aliases: + - /rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts + - /rancher/v2.x/en/installation/air-gap-single-node/provision-host + - /rancher/v2.x/en/installation/options/air-gap-helm2/prepare-nodes +--- + +This section is about how to prepare your node(s) to install Rancher for your air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. There are _tabs_ for either a high availability (recommended) or a Docker installation. + +# Prerequisites + +{{% tabs %}} +{{% tab "Kubernetes Install (Recommended)" %}} + +### OS, Docker, Hardware, and Networking + +Make sure that your node(s) fulfill the general [installation requirements.]({{}}/rancher/v2.x/en/installation/requirements/) + +### Private Registry + +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines. + +If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). + +### CLI Tools + +The following CLI tools are required for the Kubernetes Install. Make sure these tools are installed on your workstation and available in your `$PATH`. + +- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. +- [rke]({{}}/rke/latest/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. +- [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/options/helm-version) to choose a version of Helm to install Rancher. + +{{% /tab %}} +{{% tab "Docker Install" %}} + +### OS, Docker, Hardware, and Networking + +Make sure that your node(s) fulfill the general [installation requirements.]({{}}/rancher/v2.x/en/installation/requirements/) + +### Private Registry + +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines. + +If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). +{{% /tab %}} +{{% /tabs %}} + +# Set up Infrastructure + +{{% tabs %}} +{{% tab "Kubernetes Install (Recommended)" %}} + +Rancher recommends installing Rancher on a Kubernetes cluster. A highly available Kubernetes install is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails. + +### Recommended Architecture + +- DNS for Rancher should resolve to a layer 4 load balancer +- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. +- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. +- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. + +
Rancher installed on a Kubernetes cluster with layer 4 load balancer, depicting SSL termination at ingress controllers
+ +![Rancher HA]({{}}/img/rancher/ha/rancher2ha.svg) + +### A. Provision three air gapped Linux hosts according to our requirements + +These hosts will be disconnected from the internet, but require being able to connect with your private registry. + +View hardware and software requirements for each of your cluster nodes in [Requirements]({{}}/rancher/v2.x/en/installation/requirements). + +### B. Set up your Load Balancer + +When setting up the Kubernetes cluster that will run the Rancher server components, an Ingress controller pod will be deployed on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. + +You will need to configure a load balancer as a basic Layer 4 TCP forwarder to direct traffic to these ingress controller pods. The exact configuration will vary depending on your environment. + +> **Important:** +> Only use this load balancer (i.e, the `local` cluster Ingress) to load balance the Rancher server. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. + +**Load Balancer Configuration Samples:** + +- For an example showing how to set up an NGINX load balancer, refer to [this page.]({{}}/rancher/v2.x/en/installation/options/nginx) +- For an example showing how to set up an Amazon NLB load balancer, refer to [this page.]({{}}/rancher/v2.x/en/installation/options/nlb) + +{{% /tab %}} +{{% tab "Docker Install" %}} + +The Docker installation is for Rancher users that are wanting to test out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server. + +> **Important:** If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker installation to a Kubernetes Installation. + +Instead of running the Docker installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation. + +### A. Provision a single, air gapped Linux host according to our Requirements + +These hosts will be disconnected from the internet, but require being able to connect with your private registry. + +View hardware and software requirements for each of your cluster nodes in [Requirements]({{}}/rancher/v2.x/en/installation/requirements). + +{{% /tab %}} +{{% /tabs %}} + +### [Next: Collect and Publish Images to your Private Registry]({{}}/rancher/v2.x/en/installation/other-installation-methods/air-gap/populate-private-registry/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/_index.md index 72ac7485ce2..394cf2ca596 100644 --- a/content/rancher/v2.x/en/installation/resources/advanced/helm2/_index.md +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/_index.md @@ -1,8 +1,60 @@ --- -title: Helm 2 -weight: 10 +title: Kubernetes Installation Using Helm 2 +weight: 1 aliases: - /rancher/v2.x/en/installation/options/helm2 --- -This section preserves an older version of the Rancher installation documentation that used Helm 2. \ No newline at end of file +> After Helm 3 was released, the Rancher installation instructions were updated to use Helm 3. +> +> If you are using Helm 2, we recommend [migrating to Helm 3](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) because it is simpler to use and more secure than Helm 2. +> +> This section provides a copy of the older high-availability Kubernetes Rancher installation instructions that used Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible. + +For production environments, we recommend installing Rancher in a high-availability configuration so that your user base can always access Rancher Server. When installed in a Kubernetes cluster, Rancher will integrate with the cluster's etcd database and take advantage of Kubernetes scheduling for high-availability. + +This procedure walks you through setting up a 3-node cluster with Rancher Kubernetes Engine (RKE) and installing the Rancher chart with the Helm package manager. + +> **Important:** The Rancher management server can only be run on an RKE-managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported. + +> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads. + +## Recommended Architecture + +- DNS for Rancher should resolve to a Layer 4 load balancer (TCP) +- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. +- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. +- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. + +
Kubernetes Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
+![High-availability Kubernetes Install]({{}}/img/rancher/ha/rancher2ha.svg) +Kubernetes Rancher install with Layer 4 load balancer (TCP), depicting SSL termination at ingress controllers + +## Required Tools + +The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` + +- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. +- [rke]({{}}/rke/latest/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. +- [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/options/helm-version) to choose a version of Helm to install Rancher. + +## Installation Outline + +- [Create Nodes and Load Balancer]({{}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/) +- [Install Kubernetes with RKE]({{}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/) +- [Initialize Helm (tiller)]({{}}/rancher/v2.x/en/installation/options/helm2/helm-init/) +- [Install Rancher]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/) + +## Additional Install Options + +- [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) + +## Previous Methods + +[RKE add-on install]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/) + +> **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +> Please use the Rancher helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +> If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the Helm chart. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/_index.md new file mode 100644 index 00000000000..05ccaad000d --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/_index.md @@ -0,0 +1,32 @@ +--- +title: "1. Create Nodes and Load Balancer" +weight: 185 +aliases: + - /rancher/v2.x/en/installation/options/helm2/create-nodes-lb +--- + +Use your provider of choice to provision 3 nodes and a Load Balancer endpoint for your RKE install. + +> **Note:** These nodes must be in the same region/datacenter. You may place these servers in separate availability zones. + +### Node Requirements + +View the supported operating systems and hardware/software/networking requirements for nodes running Rancher at [Node Requirements]({{}}/rancher/v2.x/en/installation/requirements). + +View the OS requirements for RKE at [RKE Requirements]({{}}/rke/latest/en/os/) + +### Load Balancer + +RKE will configure an Ingress controller pod, on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. + +Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. + +>**Important:** +>Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications. + +#### Examples + +* [Nginx]({{}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nginx/) +* [Amazon NLB]({{}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nlb/) + +### [Next: Install Kubernetes with RKE]({{}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nginx/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nginx/_index.md new file mode 100644 index 00000000000..89cd1374eb3 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nginx/_index.md @@ -0,0 +1,81 @@ +--- +title: NGINX +weight: 270 +aliases: + - /rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nginx +--- +NGINX will be configured as Layer 4 load balancer (TCP) that forwards connections to one of your Rancher nodes. + +>**Note:** +> In this configuration, the load balancer is positioned in front of your nodes. The load balancer can be any host capable of running NGINX. +> +> One caveat: do not use one of your Rancher nodes as the load balancer. + +## Install NGINX + +Start by installing NGINX on the node you want to use as a load balancer. NGINX has packages available for all known operating systems. The versions tested are `1.14` and `1.15`. For help installing NGINX, refer to their [install documentation](https://www.nginx.com/resources/wiki/start/topics/tutorials/install/). + +The `stream` module is required, which is present when using the official NGINX packages. Please refer to your OS documentation on how to install and enable the NGINX `stream` module on your operating system. + +## Create NGINX Configuration + +After installing NGINX, you need to update the NGINX configuration file, `nginx.conf`, with the IP addresses for your nodes. + +1. Copy and paste the code sample below into your favorite text editor. Save it as `nginx.conf`. + +2. From `nginx.conf`, replace both occurrences (port 80 and port 443) of ``, ``, and `` with the IPs of your [nodes]({{}}/rancher/v2.x/en/installation/options/helm2/create-nodes-lb/). + + >**Note:** See [NGINX Documentation: TCP and UDP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/) for all configuration options. + +
Example NGINX config
+ ``` + worker_processes 4; + worker_rlimit_nofile 40000; + + events { + worker_connections 8192; + } + + stream { + upstream rancher_servers_http { + least_conn; + server :80 max_fails=3 fail_timeout=5s; + server :80 max_fails=3 fail_timeout=5s; + server :80 max_fails=3 fail_timeout=5s; + } + server { + listen 80; + proxy_pass rancher_servers_http; + } + + upstream rancher_servers_https { + least_conn; + server :443 max_fails=3 fail_timeout=5s; + server :443 max_fails=3 fail_timeout=5s; + server :443 max_fails=3 fail_timeout=5s; + } + server { + listen 443; + proxy_pass rancher_servers_https; + } + } + ``` + +3. Save `nginx.conf` to your load balancer at the following path: `/etc/nginx/nginx.conf`. + +4. Load the updates to your NGINX configuration by running the following command: + + ``` + # nginx -s reload + ``` + +## Option - Run NGINX as Docker container + +Instead of installing NGINX as a package on the operating system, you can rather run it as a Docker container. Save the edited **Example NGINX config** as `/etc/nginx.conf` and run the following command to launch the NGINX container: + +``` +docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + -v /etc/nginx.conf:/etc/nginx/nginx.conf \ + nginx:1.14 +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nlb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nlb/_index.md new file mode 100644 index 00000000000..70569e316a4 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/create-nodes-lb/nlb/_index.md @@ -0,0 +1,177 @@ +--- +title: Amazon NLB +weight: 277 +aliases: + - /rancher/v2.x/en/installation/options/helm2/create-nodes-lb/nlb +--- +## Objectives + +Configuring an Amazon NLB is a multistage process. We've broken it down into multiple tasks so that it's easy to follow. + +1. [Create Target Groups](#create-target-groups) + + Begin by creating two target groups for the **TCP** protocol, one regarding TCP port 443 and one regarding TCP port 80 (providing redirect to TCP port 443). You'll add your Linux nodes to these groups. + +2. [Register Targets](#register-targets) + + Add your Linux nodes to the target groups. + +3. [Create Your NLB](#create-your-nlb) + + Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in **1. Create Target Groups**. + +> **Note:** Rancher only supports using the Amazon NLB when terminating traffic in `tcp` mode for port 443 rather than `tls` mode. This is due to the fact that the NLB does not inject the correct headers into requests when terminated at the NLB. This means that if you want to use certificates managed by the Amazon Certificate Manager (ACM), you should use an ELB or ALB. + +## Create Target Groups + +Your first NLB configuration step is to create two target groups. Technically, only port 443 is needed to access Rancher, but its convenient to add a listener for port 80 which will be redirected to port 443 automatically. The NGINX ingress controller on the nodes will make sure that port 80 gets redirected to port 443. + +Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started, make sure to select the **Region** where your EC2 instances (Linux nodes) are created. + +The Target Groups configuration resides in the **Load Balancing** section of the **EC2** service. Select **Services** and choose **EC2**, find the section **Load Balancing** and open **Target Groups**. + +{{< img "/img/rancher/ha/nlb/ec2-loadbalancing.png" "EC2 Load Balancing section">}} + +Click **Create target group** to create the first target group, regarding TCP port 443. + +### Target Group (TCP port 443) + +Configure the first target group according to the table below. Screenshots of the configuration are shown just below the table. + +Option | Setting +--------------------------------------|------------------------------------ +Target Group Name | `rancher-tcp-443` +Protocol | `TCP` +Port | `443` +Target type | `instance` +VPC | Choose your VPC +Protocol
(Health Check) | `HTTP` +Path
(Health Check) | `/healthz` +Port (Advanced health check) | `override`,`80` +Healthy threshold (Advanced health) | `3` +Unhealthy threshold (Advanced) | `3` +Timeout (Advanced) | `6 seconds` +Interval (Advanced) | `10 second` +Success codes | `200-399` + +
+**Screenshot Target group TCP port 443 settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-443.png" "Target group 443">}} + +
+**Screenshot Target group TCP port 443 Advanced settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-443-advanced.png" "Target group 443 Advanced">}} + +
+ +Click **Create target group** to create the second target group, regarding TCP port 80. + +### Target Group (TCP port 80) + +Configure the second target group according to the table below. Screenshots of the configuration are shown just below the table. + +Option | Setting +--------------------------------------|------------------------------------ +Target Group Name | `rancher-tcp-80` +Protocol | `TCP` +Port | `80` +Target type | `instance` +VPC | Choose your VPC +Protocol
(Health Check) | `HTTP` +Path
(Health Check) | `/healthz` +Port (Advanced health check) | `traffic port` +Healthy threshold (Advanced health) | `3` +Unhealthy threshold (Advanced) | `3` +Timeout (Advanced) | `6 seconds` +Interval (Advanced) | `10 second` +Success codes | `200-399` + +
+**Screenshot Target group TCP port 80 settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-80.png" "Target group 80">}} + +
+**Screenshot Target group TCP port 80 Advanced settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-80-advanced.png" "Target group 80 Advanced">}} + +
+ +## Register Targets + +Next, add your Linux nodes to both target groups. + +Select the target group named **rancher-tcp-443**, click the tab **Targets** and choose **Edit**. + +{{< img "/img/rancher/ha/nlb/edit-targetgroup-443.png" "Edit target group 443">}} + +Select the instances (Linux nodes) you want to add, and click **Add to registered**. + +
+**Screenshot Add targets to target group TCP port 443**
+ +{{< img "/img/rancher/ha/nlb/add-targets-targetgroup-443.png" "Add targets to target group 443">}} + +
+**Screenshot Added targets to target group TCP port 443**
+ +{{< img "/img/rancher/ha/nlb/added-targets-targetgroup-443.png" "Added targets to target group 443">}} + +When the instances are added, click **Save** on the bottom right of the screen. + +Repeat those steps, replacing **rancher-tcp-443** with **rancher-tcp-80**. The same instances need to be added as targets to this target group. + +## Create Your NLB + +Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in [Create Target Groups](#create-target-groups). + +1. From your web browser, navigate to the [Amazon EC2 Console](https://console.aws.amazon.com/ec2/). + +2. From the navigation pane, choose **LOAD BALANCING** > **Load Balancers**. + +3. Click **Create Load Balancer**. + +4. Choose **Network Load Balancer** and click **Create**. + +5. Complete the **Step 1: Configure Load Balancer** form. + - **Basic Configuration** + + - Name: `rancher` + - Scheme: `internal` or `internet-facing` + + The Scheme that you choose for your NLB is dependent on the configuration of your instances/VPC. If your instances do not have public IPs associated with them, or you will only be accessing Rancher internally, you should set your NLB Scheme to `internal` rather than `internet-facing`. + - **Listeners** + + Add the **Load Balancer Protocols** and **Load Balancer Ports** below. + - `TCP`: `443` + + - **Availability Zones** + + - Select Your **VPC** and **Availability Zones**. + +6. Complete the **Step 2: Configure Routing** form. + + - From the **Target Group** drop-down, choose **Existing target group**. + + - From the **Name** drop-down, choose `rancher-tcp-443`. + + - Open **Advanced health check settings**, and configure **Interval** to `10 seconds`. + +7. Complete **Step 3: Register Targets**. Since you registered your targets earlier, all you have to do is click **Next: Review**. + +8. Complete **Step 4: Review**. Look over the load balancer details and click **Create** when you're satisfied. + +9. After AWS creates the NLB, click **Close**. + +## Add listener to NLB for TCP port 80 + +1. Select your newly created NLB and select the **Listeners** tab. + +2. Click **Add listener**. + +3. Use `TCP`:`80` as **Protocol** : **Port** + +4. Click **Add action** and choose **Forward to...** + +5. From the **Forward to** drop-down, choose `rancher-tcp-80`. + +6. Click **Save** in the top right of the screen. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/_index.md index 61b8073ee0b..565fbec8f27 100644 --- a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/_index.md +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/_index.md @@ -3,7 +3,7 @@ title: "Initialize Helm: Install the Tiller Service" description: "With Helm, you can create configurable deployments instead of using static files. In order to use Helm, the Tiller service needs to be installed on your cluster." weight: 195 aliases: - - /rancher/v2.x/en/installation/options/helm2/helm-init/ + - /rancher/v2.x/en/installation/options/helm2/helm-init --- Helm is the package management tool of choice for Kubernetes. Helm "charts" provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh/). To be able to use Helm, the server-side component `tiller` needs to be installed on your cluster. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/troubleshooting/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/troubleshooting/_index.md new file mode 100644 index 00000000000..112b87f3c63 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-init/troubleshooting/_index.md @@ -0,0 +1,25 @@ +--- +title: Troubleshooting +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/helm-init/troubleshooting +--- + +### Helm commands show forbidden + +When Helm is initiated in the cluster without specifying the correct `ServiceAccount`, the command `helm init` will succeed but you won't be able to execute most of the other `helm` commands. The following error will be shown: + +``` +Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system" +``` + +To resolve this, the server component (`tiller`) needs to be removed and added with the correct `ServiceAccount`. You can use `helm reset --force` to remove the `tiller` from the cluster. Please check if it is removed using `helm version --server`. + +``` +helm reset --force +Tiller (the Helm server-side component) has been uninstalled from your Kubernetes Cluster. +helm version --server +Error: could not find tiller +``` + +When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)]({{}}/rancher/v2.x/en/installation/options/helm2/helm-init/) to install `tiller` with the correct `ServiceAccount`. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/_index.md index 3c8a4f2bab4..3529904e3b6 100644 --- a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/_index.md @@ -2,7 +2,7 @@ title: "4. Install Rancher" weight: 200 aliases: - - /rancher/v2.x/en/installation/options/helm2/helm-rancher/ + - /rancher/v2.x/en/installation/options/helm2/helm-rancher --- Rancher installation is managed using the Helm package manager for Kubernetes. Use `helm` to install the prerequisite and charts to install Rancher. @@ -15,7 +15,7 @@ Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installat ### Add the Helm Chart Repository -Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{}}/rancher/v2.x/en/installation/resources/chart-options/#helm-chart-repositories). +Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{}}/rancher/v2.x/en/installation/options/server-tags/#helm-chart-repositories). {{< release-channel >}} @@ -53,7 +53,7 @@ These instructions are adapted from the [official cert-manager documentation](ht 1. Install the CustomResourceDefinition resources separately ```plain - kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml + kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.9/deploy/manifests/00-crds.yaml ``` 1. Create the namespace for cert-manager @@ -81,7 +81,7 @@ These instructions are adapted from the [official cert-manager documentation](ht helm install \ --name cert-manager \ --namespace cert-manager \ - --version v0.12.0 \ + --version v0.14.2 \ jetstack/cert-manager ``` @@ -179,7 +179,7 @@ helm install rancher-/rancher \ --set privateCA=true ``` -Now that Rancher is deployed, see [Adding TLS Secrets]({{}}/rancher/v2.x/en/installation/resources/encryption/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +Now that Rancher is deployed, see [Adding TLS Secrets]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. After adding the secrets, check if Rancher was rolled out successfully: diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/chart-options/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/chart-options/_index.md new file mode 100644 index 00000000000..0659c5619e5 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/chart-options/_index.md @@ -0,0 +1,247 @@ +--- +title: Chart Options +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options +--- + +### Common Options + +| Option | Default Value | Description | +| --- | --- | --- | +| `hostname` | " " | `string` - the Fully Qualified Domain Name for your Rancher Server | +| `ingress.tls.source` | "rancher" | `string` - Where to get the cert for the ingress. - "rancher, letsEncrypt, secret" | +| `letsEncrypt.email` | " " | `string` - Your email address | +| `letsEncrypt.environment` | "production" | `string` - Valid options: "staging, production" | +| `privateCA` | false | `bool` - Set to true if your cert is signed by a private CA | + +
+ +### Advanced Options + +| Option | Default Value | Description | +| --- | --- | --- | +| `additionalTrustedCAs` | false | `bool` - See [Additional Trusted CAs](#additional-trusted-cas) | +| `addLocal` | "auto" | `string` - Have Rancher detect and import the "local" Rancher server cluster [Import "local Cluster](#import-local-cluster) | +| `antiAffinity` | "preferred" | `string` - AntiAffinity rule for Rancher pods - "preferred, required" | +| `auditLog.destination` | "sidecar" | `string` - Stream to sidecar container console or hostPath volume - "sidecar, hostPath" | +| `auditLog.hostPath` | "/var/log/rancher/audit" | `string` - log file destination on host (only applies when `auditLog.destination` is set to `hostPath`) | +| `auditLog.level` | 0 | `int` - set the [API Audit Log]({{}}/rancher/v2.x/en/installation/api-auditing) level. 0 is off. [0-3] | +| `auditLog.maxAge` | 1 | `int` - maximum number of days to retain old audit log files (only applies when `auditLog.destination` is set to `hostPath`) | +| `auditLog.maxBackups` | 1 | `int` - maximum number of audit log files to retain (only applies when `auditLog.destination` is set to `hostPath`) | +| `auditLog.maxSize` | 100 | `int` - maximum size in megabytes of the audit log file before it gets rotated (only applies when `auditLog.destination` is set to `hostPath`) | +| `busyboxImage` | "busybox" | `string` - Image location for busybox image used to collect audit logs _Note: Available as of v2.2.0_ | +| `debug` | false | `bool` - set debug flag on rancher server | +| `extraEnv` | [] | `list` - set additional environment variables for Rancher _Note: Available as of v2.2.0_ | +| `imagePullSecrets` | [] | `list` - list of names of Secret resource containing private registry credentials | +| `ingress.extraAnnotations` | {} | `map` - additional annotations to customize the ingress | +| `ingress.configurationSnippet` | "" | `string` - Add additional Nginx configuration. Can be used for proxy configuration. _Note: Available as of v2.0.15, v2.1.10 and v2.2.4_ | +| `proxy` | "" | `string` - HTTP[S] proxy server for Rancher | +| `noProxy` | "127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16" | `string` - comma separated list of hostnames or ip address not to use the proxy | +| `resources` | {} | `map` - rancher pod resource requests & limits | +| `rancherImage` | "rancher/rancher" | `string` - rancher image source | +| `rancherImageTag` | same as chart version | `string` - rancher/rancher image tag | +| `tls` | "ingress" | `string` - See [External TLS Termination](#external-tls-termination) for details. - "ingress, external" | +| `systemDefaultRegistry` | "" | `string` - private registry to be used for all system Docker images, e.g., http://registry.example.com/ _Available as of v2.3.0_ | +| `useBundledSystemChart` | `false` | `bool` - select to use the system-charts packaged with Rancher server. This option is used for air gapped installations. _Available as of v2.3.0_ + +
+ +### API Audit Log + +Enabling the [API Audit Log]({{}}/rancher/v2.x/en/installation/api-auditing/). + +You can collect this log as you would any container log. Enable the [Logging service under Rancher Tools]({{}}/rancher/v2.x/en/cluster-admin/tools/logging/) for the `System` Project on the Rancher server cluster. + +```plain +--set auditLog.level=1 +``` + +By default enabling Audit Logging will create a sidecar container in the Rancher pod. This container (`rancher-audit-log`) will stream the log to `stdout`. You can collect this log as you would any container log. When using the sidecar as the audit log destination, the `hostPath`, `maxAge`, `maxBackups`, and `maxSize` options do not apply. It's advised to use your OS or Docker daemon's log rotation features to control disk space use. Enable the [Logging service under Rancher Tools]({{}}/rancher/v2.x/en/cluster-admin/tools/logging/) for the Rancher server cluster or System Project. + +Set the `auditLog.destination` to `hostPath` to forward logs to volume shared with the host system instead of streaming to a sidecar container. When setting the destination to `hostPath` you may want to adjust the other auditLog parameters for log rotation. + +### Setting Extra Environment Variables + +_Available as of v2.2.0_ + +You can set extra environment variables for Rancher server using `extraEnv`. This list uses the same `name` and `value` keys as the container manifest definitions. Remember to quote the values. + +```plain +--set 'extraEnv[0].name=CATTLE_TLS_MIN_VERSION' +--set 'extraEnv[0].value=1.0' +``` + +### TLS settings + +_Available as of v2.2.0_ + +To set a different TLS configuration, you can use the `CATTLE_TLS_MIN_VERSION` and `CATTLE_TLS_CIPHERS` environment variables. For example, to configure TLS 1.0 as minimum accepted TLS version: + +```plain +--set 'extraEnv[0].name=CATTLE_TLS_MIN_VERSION' +--set 'extraEnv[0].value=1.0' +``` + +See [TLS settings]({{}}/rancher/v2.x/en/admin-settings/tls-settings) for more information and options. + +### Import `local` Cluster + +By default Rancher server will detect and import the `local` cluster it's running on. User with access to the `local` cluster will essentially have "root" access to all the clusters managed by Rancher server. + +If this is a concern in your environment you can set this option to "false" on your initial install. + +> Note: This option is only effective on the initial Rancher install. See [Issue 16522](https://github.com/rancher/rancher/issues/16522) for more information. + +```plain +--set addLocal="false" +``` + +### Customizing your Ingress + +To customize or use a different ingress with Rancher server you can set your own Ingress annotations. + +Example on setting a custom certificate issuer: + +```plain +--set ingress.extraAnnotations.'certmanager\.k8s\.io/cluster-issuer'=ca-key-pair +``` + +_Available as of v2.0.15, v2.1.10 and v2.2.4_ + +Example on setting a static proxy header with `ingress.configurationSnippet`. This value is parsed like a template so variables can be used. + +```plain +--set ingress.configurationSnippet='more_set_input_headers X-Forwarded-Host {{ .Values.hostname }};' +``` + +### HTTP Proxy + +Rancher requires internet access for some functionality (helm charts). Use `proxy` to set your proxy server. + +Add your IP exceptions to the `noProxy` list. Make sure you add the Service cluster IP range (default: 10.43.0.1/16) and any worker cluster `controlplane` nodes. Rancher supports CIDR notation ranges in this list. + +```plain +--set proxy="http://:@:/" +--set noProxy="127.0.0.0/8\,10.0.0.0/8\,172.16.0.0/12\,192.168.0.0/16" +``` + +### Additional Trusted CAs + +If you have private registries, catalogs or a proxy that intercepts certificates, you may need to add additional trusted CAs to Rancher. + +```plain +--set additionalTrustedCAs=true +``` + +Once the Rancher deployment is created, copy your CA certs in pem format into a file named `ca-additional.pem` and use `kubectl` to create the `tls-ca-additional` secret in the `cattle-system` namespace. + +```plain +kubectl -n cattle-system create secret generic tls-ca-additional --from-file=ca-additional.pem +``` + +### Private Registry and Air Gap Installs + +For details on installing Rancher with a private registry, see: + +- [Air Gap: Docker Install]({{}}/rancher/v2.x/en/installation/air-gap-single-node/) +- [Air Gap: Kubernetes Install]({{}}/rancher/v2.x/en/installation/air-gap-high-availability/) + + +### External TLS Termination + +We recommend configuring your load balancer as a Layer 4 balancer, forwarding plain 80/tcp and 443/tcp to the Rancher Management cluster nodes. The Ingress Controller on the cluster will redirect http traffic on port 80 to https on port 443. + +You may terminate the SSL/TLS on a L7 load balancer external to the Rancher cluster (ingress). Use the `--set tls=external` option and point your load balancer at port http 80 on all of the Rancher cluster nodes. This will expose the Rancher interface on http port 80. Be aware that clients that are allowed to connect directly to the Rancher cluster will not be encrypted. If you choose to do this we recommend that you restrict direct access at the network level to just your load balancer. + +> **Note:** If you are using a Private CA signed certificate, add `--set privateCA=true` and see [Adding TLS Secrets - Using a Private CA Signed Certificate]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/tls-secrets/#using-a-private-ca-signed-certificate) to add the CA cert for Rancher. + +Your load balancer must support long lived websocket connections and will need to insert proxy headers so Rancher can route links correctly. + +#### Configuring Ingress for External TLS when Using NGINX v0.25 + +In NGINX v0.25, the behavior of NGINX has [changed](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0220) regarding forwarding headers and external TLS termination. Therefore, in the scenario that you are using external TLS termination configuration with NGINX v0.25, you must edit the `cluster.yml` to enable the `use-forwarded-headers` option for ingress: + +```yaml +ingress: + provider: nginx + options: + use-forwarded-headers: "true" +``` + +#### Required Headers + +* `Host` +* `X-Forwarded-Proto` +* `X-Forwarded-Port` +* `X-Forwarded-For` + +#### Recommended Timeouts + +* Read Timeout: `1800 seconds` +* Write Timeout: `1800 seconds` +* Connect Timeout: `30 seconds` + +#### Health Checks + +Rancher will respond `200` to health checks on the `/healthz` endpoint. + + +#### Example NGINX config + +This NGINX configuration is tested on NGINX 1.14. + + >**Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/). + +* Replace `IP_NODE1`, `IP_NODE2` and `IP_NODE3` with the IP addresses of the nodes in your cluster. +* Replace both occurrences of `FQDN` to the DNS name for Rancher. +* Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively. + +``` +worker_processes 4; +worker_rlimit_nofile 40000; + +events { + worker_connections 8192; +} + +http { + upstream rancher { + server IP_NODE_1:80; + server IP_NODE_2:80; + server IP_NODE_3:80; + } + + map $http_upgrade $connection_upgrade { + default Upgrade; + '' close; + } + + server { + listen 443 ssl http2; + server_name FQDN; + ssl_certificate /certs/fullchain.pem; + ssl_certificate_key /certs/privkey.pem; + + location / { + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Forwarded-Port $server_port; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_pass http://rancher; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection $connection_upgrade; + # This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close. + proxy_read_timeout 900s; + proxy_buffering off; + } + } + + server { + listen 80; + server_name FQDN; + return 301 https://$server_name$request_uri; + } +} +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/tls-secrets/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/tls-secrets/_index.md new file mode 100644 index 00000000000..25465c243cd --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/tls-secrets/_index.md @@ -0,0 +1,34 @@ +--- +title: Adding Kubernetes TLS Secrets +description: Read about how to populate the Kubernetes TLS secret for a Rancher installation +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/helm-rancher/tls-secrets +--- + +Kubernetes will create all the objects and services for Rancher, but it will not become available until we populate the `tls-rancher-ingress` secret in the `cattle-system` namespace with the certificate and key. + +Combine the server certificate followed by any intermediate certificate(s) needed into a file named `tls.crt`. Copy your certificate key into a file named `tls.key`. + +Use `kubectl` with the `tls` secret type to create the secrets. + +``` +kubectl -n cattle-system create secret tls tls-rancher-ingress \ + --cert=tls.crt \ + --key=tls.key +``` + +> **Note:** If you want to replace the certificate, you can delete the `tls-rancher-ingress` secret using `kubectl -n cattle-system delete secret tls-rancher-ingress` and add a new one using the command shown above. If you are using a private CA signed certificate, replacing the certificate is only possible if the new certificate is signed by the same CA as the certificate currently in use. + +### Using a Private CA Signed Certificate + +If you are using a private CA, Rancher requires a copy of the CA certificate which is used by the Rancher Agent to validate the connection to the server. + +Copy the CA certificate into a file named `cacerts.pem` and use `kubectl` to create the `tls-ca` secret in the `cattle-system` namespace. + +>**Important:** Make sure the file is called `cacerts.pem` as Rancher uses that filename to configure the CA certificate. + +``` +kubectl -n cattle-system create secret generic tls-ca \ + --from-file=cacerts.pem +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/troubleshooting/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/troubleshooting/_index.md new file mode 100644 index 00000000000..bf28fd0ed3e --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/helm-rancher/troubleshooting/_index.md @@ -0,0 +1,135 @@ +--- +title: Troubleshooting +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/helm-rancher/troubleshooting +--- + +### Where is everything + +Most of the troubleshooting will be done on objects in these 3 namespaces. + +* `cattle-system` - `rancher` deployment and pods. +* `ingress-nginx` - Ingress controller pods and services. +* `kube-system` - `tiller` and `cert-manager` pods. + +### "default backend - 404" + +A number of things can cause the ingress-controller not to forward traffic to your rancher instance. Most of the time its due to a bad ssl configuration. + +Things to check + +* [Is Rancher Running](#is-rancher-running) +* [Cert CN is "Kubernetes Ingress Controller Fake Certificate"](#cert-cn-is-kubernetes-ingress-controller-fake-certificate) + +### Is Rancher Running + +Use `kubectl` to check the `cattle-system` system namespace and see if the Rancher pods are in a Running state. + +``` +kubectl -n cattle-system get pods + +NAME READY STATUS RESTARTS AGE +pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m +``` + +If the state is not `Running`, run a `describe` on the pod and check the Events. + +``` +kubectl -n cattle-system describe pod + +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Scheduled 11m default-scheduler Successfully assigned rancher-784d94f59b-vgqzh to localhost + Normal SuccessfulMountVolume 11m kubelet, localhost MountVolume.SetUp succeeded for volume "rancher-token-dj4mt" + Normal Pulling 11m kubelet, localhost pulling image "rancher/rancher:v2.0.4" + Normal Pulled 11m kubelet, localhost Successfully pulled image "rancher/rancher:v2.0.4" + Normal Created 11m kubelet, localhost Created container + Normal Started 11m kubelet, localhost Started container +``` + +### Checking the rancher logs + +Use `kubectl` to list the pods. + +``` +kubectl -n cattle-system get pods + +NAME READY STATUS RESTARTS AGE +pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m +``` + +Use `kubectl` and the pod name to list the logs from the pod. + +``` +kubectl -n cattle-system logs -f rancher-784d94f59b-vgqzh +``` + +### Cert CN is "Kubernetes Ingress Controller Fake Certificate" + +Use your browser to check the certificate details. If it says the Common Name is "Kubernetes Ingress Controller Fake Certificate", something may have gone wrong with reading or issuing your SSL cert. + +> **Note:** if you are using LetsEncrypt to issue certs it can sometimes take a few minuets to issue the cert. + +#### cert-manager issued certs (Rancher Generated or LetsEncrypt) + +`cert-manager` has 3 parts. + +* `cert-manager` pod in the `kube-system` namespace. +* `Issuer` object in the `cattle-system` namespace. +* `Certificate` object in the `cattle-system` namespace. + +Work backwards and do a `kubectl describe` on each object and check the events. You can track down what might be missing. + +For example there is a problem with the Issuer: + +``` +kubectl -n cattle-system describe certificate +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning IssuerNotReady 18s (x23 over 19m) cert-manager Issuer rancher not ready +``` + +``` +kubectl -n cattle-system describe issuer +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning ErrInitIssuer 19m (x12 over 19m) cert-manager Error initializing issuer: secret "tls-rancher" not found + Warning ErrGetKeyPair 9m (x16 over 19m) cert-manager Error getting keypair for CA issuer: secret "tls-rancher" not found +``` + +#### Bring Your Own SSL Certs + +Your certs get applied directly to the Ingress object in the `cattle-system` namespace. + +Check the status of the Ingress object and see if its ready. + +``` +kubectl -n cattle-system describe ingress +``` + +If its ready and the SSL is still not working you may have a malformed cert or secret. + +Check the nginx-ingress-controller logs. Because the nginx-ingress-controller has multiple containers in its pod you will need to specify the name of the container. + +``` +kubectl -n ingress-nginx logs -f nginx-ingress-controller-rfjrq nginx-ingress-controller +... +W0705 23:04:58.240571 7 backend_ssl.go:49] error obtaining PEM from secret cattle-system/tls-rancher-ingress: error retrieving secret cattle-system/tls-rancher-ingress: secret cattle-system/tls-rancher-ingress was not found +``` + +### no matches for kind "Issuer" + +The [SSL configuration]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#choose-your-ssl-configuration) option you have chosen requires [cert-manager]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#optional-install-cert-manager) to be installed before installing Rancher or else the following error is shown: + +``` +Error: validation failed: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1" +``` + +Install [cert-manager]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/#optional-install-cert-manager) and try installing Rancher again. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/_index.md new file mode 100644 index 00000000000..c081ab68856 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/_index.md @@ -0,0 +1,134 @@ +--- +title: "2. Install Kubernetes with RKE" +weight: 190 +aliases: + - /rancher/v2.x/en/installation/options/helm2/kubernetes-rke +--- + +Use RKE to install Kubernetes with a high availability etcd configuration. + +>**Note:** For systems without direct internet access see [Air Gap: Kubernetes install]({{}}/rancher/v2.x/en/installation/air-gap-high-availability/) for install details. + +### Create the `rancher-cluster.yml` File + +Using the sample below create the `rancher-cluster.yml` file. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. + +> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls. + + +```yaml +nodes: + - address: 165.227.114.63 + internal_address: 172.16.22.12 + user: ubuntu + role: [controlplane,worker,etcd] + - address: 165.227.116.167 + internal_address: 172.16.32.37 + user: ubuntu + role: [controlplane,worker,etcd] + - address: 165.227.127.226 + internal_address: 172.16.42.73 + user: ubuntu + role: [controlplane,worker,etcd] + +services: + etcd: + snapshot: true + creation: 6h + retention: 24h +``` + +#### Common RKE Nodes Options + +| Option | Required | Description | +| --- | --- | --- | +| `address` | yes | The public DNS or IP address | +| `user` | yes | A user that can run docker commands | +| `role` | yes | List of Kubernetes roles assigned to the node | +| `internal_address` | no | The private DNS or IP address for internal cluster traffic | +| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | + +#### Advanced Configurations + +RKE has many configuration options for customizing the install to suit your specific environment. + +Please see the [RKE Documentation]({{}}/rke/latest/en/config-options/) for the full list of options and capabilities. + +For tuning your etcd cluster for larger Rancher installations see the [etcd settings guide]({{}}/rancher/v2.x/en/installation/options/etcd/). + +### Run RKE + +``` +rke up --config ./rancher-cluster.yml +``` + +When finished, it should end with the line: `Finished building Kubernetes cluster successfully`. + +### Testing Your Cluster + +RKE should have created a file `kube_config_rancher-cluster.yml`. This file has the credentials for `kubectl` and `helm`. + +> **Note:** If you have used a different file name from `rancher-cluster.yml`, then the kube config file will be named `kube_config_.yml`. + +You can copy this file to `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_rancher-cluster.yml`. + +``` +export KUBECONFIG=$(pwd)/kube_config_rancher-cluster.yml +``` + +Test your connectivity with `kubectl` and see if all your nodes are in `Ready` state. + +``` +kubectl get nodes + +NAME STATUS ROLES AGE VERSION +165.227.114.63 Ready controlplane,etcd,worker 11m v1.13.5 +165.227.116.167 Ready controlplane,etcd,worker 11m v1.13.5 +165.227.127.226 Ready controlplane,etcd,worker 11m v1.13.5 +``` + +### Check the Health of Your Cluster Pods + +Check that all the required pods and containers are healthy are ready to continue. + +* Pods are in `Running` or `Completed` state. +* `READY` column shows all the containers are running (i.e. `3/3`) for pods with `STATUS` `Running` +* Pods with `STATUS` `Completed` are run-once Jobs. For these pods `READY` should be `0/1`. + +``` +kubectl get pods --all-namespaces + +NAMESPACE NAME READY STATUS RESTARTS AGE +ingress-nginx nginx-ingress-controller-tnsn4 1/1 Running 0 30s +ingress-nginx nginx-ingress-controller-tw2ht 1/1 Running 0 30s +ingress-nginx nginx-ingress-controller-v874b 1/1 Running 0 30s +kube-system canal-jp4hz 3/3 Running 0 30s +kube-system canal-z2hg8 3/3 Running 0 30s +kube-system canal-z6kpw 3/3 Running 0 30s +kube-system kube-dns-7588d5b5f5-sf4vh 3/3 Running 0 30s +kube-system kube-dns-autoscaler-5db9bbb766-jz2k6 1/1 Running 0 30s +kube-system metrics-server-97bc649d5-4rl2q 1/1 Running 0 30s +kube-system rke-ingress-controller-deploy-job-bhzgm 0/1 Completed 0 30s +kube-system rke-kubedns-addon-deploy-job-gl7t4 0/1 Completed 0 30s +kube-system rke-metrics-addon-deploy-job-7ljkc 0/1 Completed 0 30s +kube-system rke-network-plugin-deploy-job-6pbgj 0/1 Completed 0 30s +``` + +### Save Your Files + +> **Important** +> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster. + +Save a copy of the following files in a secure location: + +- `rancher-cluster.yml`: The RKE cluster configuration file. +- `kube_config_rancher-cluster.yml`: The [Kubeconfig file]({{}}/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster. +- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file]({{}}/rke/latest/en/installation/#kubernetes-cluster-state), this file contains credentials for full access to the cluster.

_The Kubernetes Cluster State file is only created when using RKE v0.2.0 or higher._ + +> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file. + +### Issues or errors? + +See the [Troubleshooting]({{}}/rancher/v2.x/en/installation/options/helm2/kubernetes-rke/troubleshooting/) page. + +### [Next: Initialize Helm (Install tiller)]({{}}/rancher/v2.x/en/installation/options/helm2/helm-init/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/troubleshooting/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/troubleshooting/_index.md new file mode 100644 index 00000000000..741eecd1d84 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/kubernetes-rke/troubleshooting/_index.md @@ -0,0 +1,54 @@ +--- +title: Troubleshooting +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/kubernetes-rke/troubleshooting +--- + +### canal Pods show READY 2/3 + +The most common cause of this issue is port 8472/UDP is not open between the nodes. Check your local firewall, network routing or security groups. + +Once the network issue is resolved, the `canal` pods should timeout and restart to establish their connections. + +### nginx-ingress-controller Pods show RESTARTS + +The most common cause of this issue is the `canal` pods have failed to establish the overlay network. See [canal Pods show READY `2/3`](#canal-pods-show-ready-2-3) for troubleshooting. + +### Failed to set up SSH tunneling for host [xxx.xxx.xxx.xxx]: Can't retrieve Docker Info + +#### Failed to dial to /var/run/docker.sock: ssh: rejected: administratively prohibited (open failed) + +* User specified to connect with does not have permission to access the Docker socket. This can be checked by logging into the host and running the command `docker ps`: + +``` +$ ssh user@server +user@server$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +See [Manage Docker as a non-root user](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user) how to set this up properly. + +* When using RedHat/CentOS as operating system, you cannot use the user `root` to connect to the nodes because of [Bugzilla #1527565](https://bugzilla.redhat.com/show_bug.cgi?id=1527565). You will need to add a separate user and configure it to access the Docker socket. See [Manage Docker as a non-root user](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user) how to set this up properly. + +* SSH server version is not version 6.7 or higher. This is needed for socket forwarding to work, which is used to connect to the Docker socket over SSH. This can be checked using `sshd -V` on the host you are connecting to, or using netcat: +``` +$ nc xxx.xxx.xxx.xxx 22 +SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.10 +``` + +#### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: Error configuring SSH: ssh: no key found + +* The key file specified as `ssh_key_path` cannot be accessed. Make sure that you specified the private key file (not the public key, `.pub`), and that the user that is running the `rke` command can access the private key file. + +#### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain + +* The key file specified as `ssh_key_path` is not correct for accessing the node. Double-check if you specified the correct `ssh_key_path` for the node and if you specified the correct user to connect with. + +#### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: Error configuring SSH: ssh: cannot decode encrypted private keys + +* If you want to use encrypted private keys, you should use `ssh-agent` to load your keys with your passphrase. If the `SSH_AUTH_SOCK` environment variable is found in the environment where the `rke` command is run, it will be used automatically to connect to the node. + +#### Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? + +* The node is not reachable on the configured `address` and `port`. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/_index.md new file mode 100644 index 00000000000..17a94566846 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/_index.md @@ -0,0 +1,18 @@ +--- +title: RKE Add-On Install +weight: 276 +aliases: + - /rancher/v2.x/en/installation/options/helm2/rke-add-on +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + + +* [Kubernetes installation with External Load Balancer (TCP/Layer 4)]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb) +* [Kubernetes installation with External Load Balancer (HTTPS/Layer 7)]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-7-lb) +* [HTTP Proxy Configuration for a Kubernetes installation]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/proxy/) +* [Troubleshooting RKE Add-on Installs]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/troubleshooting/) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/api-auditing/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/api-auditing/_index.md new file mode 100644 index 00000000000..8ea13eb4b52 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/api-auditing/_index.md @@ -0,0 +1,56 @@ +--- +title: Enable API Auditing +weight: 300 +aliases: + - /rke/latest/en/config-options/add-ons/api-auditing/ + - /rancher/v2.x/en/installation/options/helm2/rke-add-on/api-auditing +--- + +>**Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +If you're using RKE to install Rancher, you can use directives to enable API Auditing for your Rancher install. You can know what happened, when it happened, who initiated it, and what cluster it affected. API auditing records all requests and responses to and from the Rancher API, which includes use of the Rancher UI and any other use of the Rancher API through programmatic use. + +## In-line Arguments + +Enable API Auditing using RKE by adding arguments to your Rancher container. + +To enable API auditing: + +- Add API Auditing arguments (`args`) to your Rancher container. +- Declare a `mountPath` in the `volumeMounts` directive of the container. +- Declare a `path` in the `volumes` directive. + +For more information about each argument, its syntax, and how to view API Audit logs, see [Rancher v2.0 Documentation: API Auditing]({{}}/rancher/v2.x/en/installation/api-auditing). + +```yaml +... +containers: + - image: rancher/rancher:latest + imagePullPolicy: Always + name: cattle-server + args: ["--audit-log-path", "/var/log/auditlog/rancher-api-audit.log", "--audit-log-maxbackup", "5", "--audit-log-maxsize", "50", "--audit-level", "2"] + ports: + - containerPort: 80 + protocol: TCP + - containerPort: 443 + protocol: TCP + volumeMounts: + - mountPath: /etc/rancher/ssl + name: cattle-keys-volume + readOnly: true + - mountPath: /var/log/auditlog + name: audit-log-dir + volumes: + - name: cattle-keys-volume + secret: + defaultMode: 420 + secretName: cattle-keys-server + - name: audit-log-dir + hostPath: + path: /var/log/rancher/auditlog + type: Directory +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/_index.md new file mode 100644 index 00000000000..2b119a1a6e7 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/_index.md @@ -0,0 +1,402 @@ +--- +title: Kubernetes Install with External Load Balancer (TCP/Layer 4) +weight: 275 +aliases: + - /rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a High-availability Kubernetes install with an RKE add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the Helm chart. + +This procedure walks you through setting up a 3-node cluster using the Rancher Kubernetes Engine (RKE). The cluster's sole purpose is running pods for Rancher. The setup is based on: + +- Layer 4 load balancer (TCP) +- [NGINX ingress controller with SSL termination (HTTPS)](https://kubernetes.github.io/ingress-nginx/) + +In a Kubernetes setup that uses a layer 4 load balancer, the load balancer accepts Rancher client connections over the TCP/UDP protocols (i.e., the transport level). The load balancer then forwards these connections to individual cluster nodes without reading the request itself. Because the load balancer cannot read the packets it's forwarding, the routing decisions it can make are limited. + +Kubernetes Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers +![High-availability Kubernetes installation of Rancher]({{}}/img/rancher/ha/rancher2ha.svg) + +## Installation Outline + +Installation of Rancher in a high-availability configuration involves multiple procedures. Review this outline to learn about each procedure you need to complete. + + + +- [1. Provision Linux Hosts](#1-provision-linux-hosts) +- [2. Configure Load Balancer](#2-configure-load-balancer) +- [3. Configure DNS](#3-configure-dns) +- [4. Install RKE](#4-install-rke) +- [5. Download RKE Config File Template](#5-download-rke-config-file-template) +- [6. Configure Nodes](#6-configure-nodes) +- [7. Configure Certificates](#7-configure-certificates) +- [8. Configure FQDN](#8-configure-fqdn) +- [9. Configure Rancher version](#9-configure-rancher-version) +- [10. Back Up Your RKE Config File](#10-back-up-your-rke-config-file) +- [11. Run RKE](#11-run-rke) +- [12. Back Up Auto-Generated Config File](#12-back-up-auto-generated-config-file) + + + +
+ +## 1. Provision Linux Hosts + +Provision three Linux hosts according to our [Requirements]({{}}/rancher/v2.x/en/installation/requirements). + +## 2. Configure Load Balancer + +We will be using NGINX as our Layer 4 Load Balancer (TCP). NGINX will forward all connections to one of your Rancher nodes. If you want to use Amazon NLB, you can skip this step and use [Amazon NLB configuration]({{}}/rancher/v2.x/en/installation/k8s-install-server-install/nlb/) + +>**Note:** +> In this configuration, the load balancer is positioned in front of your Linux hosts. The load balancer can be any host that you have available that's capable of running NGINX. +> +>One caveat: do not use one of your Rancher nodes as the load balancer. + +### A. Install NGINX + +Start by installing NGINX on your load balancer host. NGINX has packages available for all known operating systems. For help installing NGINX, refer to their [install documentation](https://www.nginx.com/resources/wiki/start/topics/tutorials/install/). + +The `stream` module is required, which is present when using the official NGINX packages. Please refer to your OS documentation how to install and enable the NGINX `stream` module on your operating system. + +### B. Create NGINX Configuration + +After installing NGINX, you need to update the NGINX config file, `nginx.conf`, with the IP addresses for your nodes. + +1. Copy and paste the code sample below into your favorite text editor. Save it as `nginx.conf`. + +2. From `nginx.conf`, replace `IP_NODE_1`, `IP_NODE_2`, and `IP_NODE_3` with the IPs of your [Linux hosts](#1-provision-linux-hosts). + + >**Note:** This Nginx configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - TCP and UDP Load Balancer](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/). + + **Example NGINX config:** + ``` + worker_processes 4; + worker_rlimit_nofile 40000; + + events { + worker_connections 8192; + } + + http { + server { + listen 80; + return 301 https://$host$request_uri; + } + } + + stream { + upstream rancher_servers { + least_conn; + server IP_NODE_1:443 max_fails=3 fail_timeout=5s; + server IP_NODE_2:443 max_fails=3 fail_timeout=5s; + server IP_NODE_3:443 max_fails=3 fail_timeout=5s; + } + server { + listen 443; + proxy_pass rancher_servers; + } + } + ``` + +3. Save `nginx.conf` to your load balancer at the following path: `/etc/nginx/nginx.conf`. + +4. Load the updates to your NGINX configuration by running the following command: + + ``` + # nginx -s reload + ``` + +### Option - Run NGINX as Docker container + +Instead of installing NGINX as a package on the operating system, you can rather run it as a Docker container. Save the edited **Example NGINX config** as `/etc/nginx.conf` and run the following command to launch the NGINX container: + +``` +docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + -v /etc/nginx.conf:/etc/nginx/nginx.conf \ + nginx:1.14 +``` + +## 3. Configure DNS + +Choose a fully qualified domain name (FQDN) that you want to use to access Rancher (e.g., `rancher.yourdomain.com`).

+ +1. Log into your DNS server a create a `DNS A` record that points to the IP address of your [load balancer](#2-configure-load-balancer). + +2. Validate that the `DNS A` is working correctly. Run the following command from any terminal, replacing `HOSTNAME.DOMAIN.COM` with your chosen FQDN: + + `nslookup HOSTNAME.DOMAIN.COM` + + **Step Result:** Terminal displays output similar to the following: + + ``` + $ nslookup rancher.yourdomain.com + Server: YOUR_HOSTNAME_IP_ADDRESS + Address: YOUR_HOSTNAME_IP_ADDRESS#53 + + Non-authoritative answer: + Name: rancher.yourdomain.com + Address: HOSTNAME.DOMAIN.COM + ``` + +
+ +## 4. Install RKE + +RKE (Rancher Kubernetes Engine) is a fast, versatile Kubernetes installer that you can use to install Kubernetes on your Linux hosts. We will use RKE to setup our cluster and run Rancher. + +1. Follow the [RKE Install]({{}}/rke/latest/en/installation) instructions. + +2. Confirm that RKE is now executable by running the following command: + + ``` + rke --version + ``` + +## 5. Download RKE Config File Template + +RKE uses a `.yml` config file to install and configure your Kubernetes cluster. There are 2 templates to choose from, depending on the SSL certificate you want to use. + +1. Download one of following templates, depending on the SSL certificate you're using. + + - [Template for self-signed certificate
`3-node-certificate.yml`](https://raw.githubusercontent.com/rancher/rancher/master/rke-templates/3-node-certificate.yml) + - [Template for certificate signed by recognized CA
`3-node-certificate-recognizedca.yml`](https://raw.githubusercontent.com/rancher/rancher/master/rke-templates/3-node-certificate-recognizedca.yml) + + >**Advanced Config Options:** + > + >- Want records of all transactions with the Rancher API? Enable the [API Auditing]({{}}/rancher/v2.x/en/installation/api-auditing) feature by editing your RKE config file. For more information, see how to enable it in [your RKE config file]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/api-auditing/). + >- Want to know the other config options available for your RKE template? See the [RKE Documentation: Config Options]({{}}/rke/latest/en/config-options/). + + +2. Rename the file to `rancher-cluster.yml`. + +## 6. Configure Nodes + +Once you have the `rancher-cluster.yml` config file template, edit the nodes section to point toward your Linux hosts. + +1. Open `rancher-cluster.yml` in your favorite text editor. + +1. Update the `nodes` section with the information of your [Linux hosts](#1-provision-linux-hosts). + + For each node in your cluster, update the following placeholders: `IP_ADDRESS_X` and `USER`. The specified user should be able to access the Docket socket, you can test this by logging in with the specified user and run `docker ps`. + + >**Note:** + > When using RHEL/CentOS, the SSH user can't be root due to https://bugzilla.redhat.com/show_bug.cgi?id=1527565. See [Operating System Requirements]({{}}/rke/latest/en/installation/os#redhat-enterprise-linux-rhel-centos) >for RHEL/CentOS specific requirements. + + nodes: + # The IP address or hostname of the node + - address: IP_ADDRESS_1 + # User that can login to the node and has access to the Docker socket (i.e. can execute `docker ps` on the node) + # When using RHEL/CentOS, this can't be root due to https://bugzilla.redhat.com/show_bug.cgi?id=1527565 + user: USER + role: [controlplane,etcd,worker] + # Path the SSH key that can be used to access to node with the specified user + ssh_key_path: ~/.ssh/id_rsa + - address: IP_ADDRESS_2 + user: USER + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + - address: IP_ADDRESS_3 + user: USER + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + +1. **Optional:** By default, `rancher-cluster.yml` is configured to take backup snapshots of your data. To disable these snapshots, change the `backup` directive setting to `false`, as depicted below. + + services: + etcd: + backup: false + + +## 7. Configure Certificates + +For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster. + +Choose from the following options: + +{{% accordion id="option-a" label="Option A—Bring Your Own Certificate: Self-Signed" %}} + +>**Prerequisites:** +>Create a self-signed certificate. +> +>- The certificate files must be in [PEM format](#pem). +>- The certificate files must be encoded in [base64](#base64). +>- In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Intermediate Certificates](#cert-order). + +1. In `kind: Secret` with `name: cattle-keys-ingress`: + + * Replace `` with the base64 encoded string of the Certificate file (usually called `cert.pem` or `domain.crt`) + * Replace `` with the base64 encoded string of the Certificate Key file (usually called `key.pem` or `domain.key`) + + >**Note:** + > The base64 encoded string should be on the same line as `tls.crt` or `tls.key`, without any newline at the beginning, in between or at the end. + + **Step Result:** After replacing the values, the file should look like the example below (the base64 encoded strings should be different): + + ```yaml + --- + apiVersion: v1 + kind: Secret + metadata: + name: cattle-keys-ingress + namespace: cattle-system + type: Opaque + data: + tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1RENDQWN5Z0F3SUJBZ0lKQUlHc25NeG1LeGxLTUEwR0NTcUdTSWIzRFFFQkN3VUFNQkl4RURBT0JnTlYKQkFNTUIzUmxjM1F0WTJFd0hoY05NVGd3TlRBMk1qRXdOREE1V2hjTk1UZ3dOekExTWpFd05EQTVXakFXTVJRdwpFZ1lEVlFRRERBdG9ZUzV5Ym1Ob2NpNXViRENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DCmdnRUJBTFJlMXdzekZSb2Rib2pZV05DSHA3UkdJaUVIMENDZ1F2MmdMRXNkUUNKZlcrUFEvVjM0NnQ3bSs3TFEKZXJaV3ZZMWpuY2VuWU5JSGRBU0VnU0ducWExYnhUSU9FaE0zQXpib3B0WDhjSW1OSGZoQlZETGdiTEYzUk0xaQpPM1JLTGdIS2tYSTMxZndjbU9zWGUwaElYQnpUbmxnM20vUzlXL3NTc0l1dDVwNENDUWV3TWlpWFhuUElKb21lCmpkS3VjSHFnMTlzd0YvcGVUalZrcVpuMkJHazZRaWFpMU41bldRV0pjcThTenZxTTViZElDaWlwYU9hWWQ3RFEKYWRTejV5dlF0YkxQNW4wTXpnOU43S3pGcEpvUys5QWdkWDI5cmZqV2JSekp3RzM5R3dRemN6VWtLcnZEb05JaQo0UFJHc01yclFNVXFSYjRSajNQOEJodEMxWXNDQXdFQUFhTTVNRGN3Q1FZRFZSMFRCQUl3QURBTEJnTlZIUThFCkJBTUNCZUF3SFFZRFZSMGxCQll3RkFZSUt3WUJCUVVIQXdJR0NDc0dBUVVGQndNQk1BMEdDU3FHU0liM0RRRUIKQ3dVQUE0SUJBUUNKZm5PWlFLWkowTFliOGNWUW5Vdi9NZkRZVEJIQ0pZcGM4MmgzUGlXWElMQk1jWDhQRC93MgpoOUExNkE4NGNxODJuQXEvaFZYYy9JNG9yaFY5WW9jSEg5UlcvbGthTUQ2VEJVR0Q1U1k4S292MHpHQ1ROaDZ6Ci9wZTNqTC9uU0pYSjRtQm51czJheHFtWnIvM3hhaWpYZG9kMmd3eGVhTklvRjNLbHB2aGU3ZjRBNmpsQTM0MmkKVVlCZ09iN1F5KytRZWd4U1diSmdoSzg1MmUvUUhnU2FVSkN6NW1sNGc1WndnNnBTUXhySUhCNkcvREc4dElSYwprZDMxSk1qY25Fb1Rhc1Jyc1NwVmNGdXZyQXlXN2liakZyYzhienBNcE1obDVwYUZRcEZzMnIwaXpZekhwakFsCk5ZR2I2OHJHcjBwQkp3YU5DS2ErbCtLRTk4M3A3NDYwCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K + tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdEY3WEN6TVZHaDF1aU5oWTBJZW50RVlpSVFmUUlLQkMvYUFzU3gxQUlsOWI0OUQ5ClhmanEzdWI3c3RCNnRsYTlqV09keDZkZzBnZDBCSVNCSWFlcHJWdkZNZzRTRXpjRE51aW0xZnh3aVkwZCtFRlUKTXVCc3NYZEV6V0k3ZEVvdUFjcVJjamZWL0J5WTZ4ZDdTRWhjSE5PZVdEZWI5TDFiK3hLd2k2M21uZ0lKQjdBeQpLSmRlYzhnbWlaNk4wcTV3ZXFEWDJ6QVgrbDVPTldTcG1mWUVhVHBDSnFMVTNtZFpCWWx5cnhMTytvemx0MGdLCktLbG81cGgzc05CcDFMUG5LOUMxc3MvbWZRek9EMDNzck1Xa21oTDcwQ0IxZmIydCtOWnRITW5BYmYwYkJETnoKTlNRcXU4T2cwaUxnOUVhd3l1dEF4U3BGdmhHUGMvd0dHMExWaXdJREFRQUJBb0lCQUJKYUErOHp4MVhjNEw0egpwUFd5bDdHVDRTMFRLbTNuWUdtRnZudjJBZXg5WDFBU2wzVFVPckZyTnZpK2xYMnYzYUZoSFZDUEN4N1RlMDVxClhPa2JzZnZkZG5iZFQ2RjgyMnJleVByRXNINk9TUnBWSzBmeDVaMDQwVnRFUDJCWm04eTYyNG1QZk1vbDdya2MKcm9Kd09rOEVpUHZZekpsZUd0bTAwUm1sRysyL2c0aWJsOTVmQXpyc1MvcGUyS3ZoN2NBVEtIcVh6MjlpUmZpbApiTGhBamQwcEVSMjNYU0hHR1ZqRmF3amNJK1c2L2RtbDZURDhrSzFGaUtldmJKTlREeVNXQnpPbXRTYUp1K01JCm9iUnVWWG4yZVNoamVGM1BYcHZRMWRhNXdBa0dJQWxOWjRHTG5QU2ZwVmJyU0plU3RrTGNzdEJheVlJS3BWZVgKSVVTTHM0RUNnWUVBMmNnZUE2WHh0TXdFNU5QWlNWdGhzbXRiYi9YYmtsSTdrWHlsdk5zZjFPdXRYVzkybVJneQpHcEhUQ0VubDB0Z1p3T081T1FLNjdFT3JUdDBRWStxMDJzZndwcmgwNFZEVGZhcW5QNTBxa3BmZEJLQWpmanEyCjFoZDZMd2hLeDRxSm9aelp2VkowV0lvR1ZLcjhJSjJOWGRTUVlUanZUZHhGczRTamdqNFFiaEVDZ1lFQTFBWUUKSEo3eVlza2EvS2V2OVVYbmVrSTRvMm5aYjJ1UVZXazRXSHlaY2NRN3VMQVhGY3lJcW5SZnoxczVzN3RMTzJCagozTFZNUVBzazFNY25oTTl4WE4vQ3ZDTys5b2t0RnNaMGJqWFh6NEJ5V2lFNHJPS1lhVEFwcDVsWlpUT3ZVMWNyCm05R3NwMWJoVDVZb2RaZ3IwUHQyYzR4U2krUVlEWnNFb2lFdzNkc0NnWUVBcVJLYWNweWZKSXlMZEJjZ0JycGkKQTRFalVLMWZsSjR3enNjbGFKUDVoM1NjZUFCejQzRU1YT0kvSXAwMFJsY3N6em83N3cyMmpud09mOEJSM0RBMwp6ZTRSWDIydWw4b0hGdldvdUZOTTNOZjNaNExuYXpVc0F0UGhNS2hRWGMrcEFBWGthUDJkZzZ0TU5PazFxaUNHCndvU212a1BVVE84b1ViRTB1NFZ4ZmZFQ2dZQUpPdDNROVNadUlIMFpSSitIV095enlOQTRaUEkvUkhwN0RXS1QKajVFS2Y5VnR1OVMxY1RyOTJLVVhITXlOUTNrSjg2OUZPMnMvWk85OGg5THptQ2hDTjhkOWN6enI5SnJPNUFMTApqWEtBcVFIUlpLTFgrK0ZRcXZVVlE3cTlpaHQyMEZPb3E5OE5SZDMzSGYxUzZUWDNHZ3RWQ21YSml6dDAxQ3ZHCmR4VnVnd0tCZ0M2Mlp0b0RLb3JyT2hvdTBPelprK2YwQS9rNDJBOENiL29VMGpwSzZtdmxEWmNYdUF1QVZTVXIKNXJCZjRVYmdVYndqa1ZWSFR6LzdDb1BWSjUvVUxJWk1Db1RUNFprNTZXWDk4ZE93Q3VTVFpZYnlBbDZNS1BBZApTZEpuVVIraEpnSVFDVGJ4K1dzYnh2d0FkbWErWUhtaVlPRzZhSklXMXdSd1VGOURLUEhHCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== + ``` + +2. In `kind: Secret` with `name: cattle-keys-server`, replace `` with the base64 encoded string of the CA Certificate file (usually called `ca.pem` or `ca.crt`). + + >**Note:** + > The base64 encoded string should be on the same line as `cacerts.pem`, without any newline at the beginning, in between or at the end. + + + **Step Result:** The file should look like the example below (the base64 encoded string should be different): + + ```yaml + --- + apiVersion: v1 + kind: Secret + metadata: + name: cattle-keys-server + namespace: cattle-system + type: Opaque + data: + cacerts.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNvRENDQVlnQ0NRRHVVWjZuMEZWeU16QU5CZ2txaGtpRzl3MEJBUXNGQURBU01SQXdEZ1lEVlFRRERBZDAKWlhOMExXTmhNQjRYRFRFNE1EVXdOakl4TURRd09Wb1hEVEU0TURjd05USXhNRFF3T1Zvd0VqRVFNQTRHQTFVRQpBd3dIZEdWemRDMWpZVENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNQmpBS3dQCndhRUhwQTdaRW1iWWczaTNYNlppVmtGZFJGckJlTmFYTHFPL2R0RUdmWktqYUF0Wm45R1VsckQxZUlUS3UzVHgKOWlGVlV4Mmo1Z0tyWmpwWitCUnFiZ1BNbk5hS1hocmRTdDRtUUN0VFFZdGRYMVFZS0pUbWF5NU45N3FoNTZtWQprMllKRkpOWVhHWlJabkdMUXJQNk04VHZramF0ZnZOdmJ0WmtkY2orYlY3aWhXanp2d2theHRUVjZlUGxuM2p5CnJUeXBBTDliYnlVcHlad3E2MWQvb0Q4VUtwZ2lZM1dOWmN1YnNvSjhxWlRsTnN6UjVadEFJV0tjSE5ZbE93d2oKaG41RE1tSFpwZ0ZGNW14TU52akxPRUc0S0ZRU3laYlV2QzlZRUhLZTUxbGVxa1lmQmtBZWpPY002TnlWQUh1dApuay9DMHpXcGdENkIwbkVDQXdFQUFUQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFHTCtaNkRzK2R4WTZsU2VBClZHSkMvdzE1bHJ2ZXdia1YxN3hvcmlyNEMxVURJSXB6YXdCdFJRSGdSWXVtblVqOGo4T0hFWUFDUEthR3BTVUsKRDVuVWdzV0pMUUV0TDA2eTh6M3A0MDBrSlZFZW9xZlVnYjQrK1JLRVJrWmowWXR3NEN0WHhwOVMzVkd4NmNOQQozZVlqRnRQd2hoYWVEQmdma1hXQWtISXFDcEsrN3RYem9pRGpXbi8walI2VDcrSGlaNEZjZ1AzYnd3K3NjUDIyCjlDQVZ1ZFg4TWpEQ1hTcll0Y0ZINllBanlCSTJjbDhoSkJqa2E3aERpVC9DaFlEZlFFVFZDM3crQjBDYjF1NWcKdE03Z2NGcUw4OVdhMnp5UzdNdXk5bEthUDBvTXl1Ty82Tm1wNjNsVnRHeEZKSFh4WTN6M0lycGxlbTNZQThpTwpmbmlYZXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + ``` + +{{% /accordion %}} + +{{% accordion id="option-b" label="Option B—Bring Your Own Certificate: Signed by Recognized CA" %}} +>**Note:** +> If you are using Self Signed Certificate, [click here](#option-a-bring-your-own-certificate-self-signed) to proceed. + +If you are using a Certificate Signed By A Recognized Certificate Authority, you will need to generate a base64 encoded string for the Certificate file and the Certificate Key file. Make sure that your certificate file includes all the [intermediate certificates](#cert-order) in the chain, the order of certificates in this case is first your own certificate, followed by the intermediates. Please refer to the documentation of your CSP (Certificate Service Provider) to see what intermediate certificate(s) need to be included. + +In the `kind: Secret` with `name: cattle-keys-ingress`: + +* Replace `` with the base64 encoded string of the Certificate file (usually called `cert.pem` or `domain.crt`) +* Replace `` with the base64 encoded string of the Certificate Key file (usually called `key.pem` or `domain.key`) + +After replacing the values, the file should look like the example below (the base64 encoded strings should be different): + +>**Note:** +> The base64 encoded string should be on the same line as `tls.crt` or `tls.key`, without any newline at the beginning, in between or at the end. + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: cattle-keys-ingress + namespace: cattle-system +type: Opaque +data: + tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1RENDQWN5Z0F3SUJBZ0lKQUlHc25NeG1LeGxLTUEwR0NTcUdTSWIzRFFFQkN3VUFNQkl4RURBT0JnTlYKQkFNTUIzUmxjM1F0WTJFd0hoY05NVGd3TlRBMk1qRXdOREE1V2hjTk1UZ3dOekExTWpFd05EQTVXakFXTVJRdwpFZ1lEVlFRRERBdG9ZUzV5Ym1Ob2NpNXViRENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DCmdnRUJBTFJlMXdzekZSb2Rib2pZV05DSHA3UkdJaUVIMENDZ1F2MmdMRXNkUUNKZlcrUFEvVjM0NnQ3bSs3TFEKZXJaV3ZZMWpuY2VuWU5JSGRBU0VnU0ducWExYnhUSU9FaE0zQXpib3B0WDhjSW1OSGZoQlZETGdiTEYzUk0xaQpPM1JLTGdIS2tYSTMxZndjbU9zWGUwaElYQnpUbmxnM20vUzlXL3NTc0l1dDVwNENDUWV3TWlpWFhuUElKb21lCmpkS3VjSHFnMTlzd0YvcGVUalZrcVpuMkJHazZRaWFpMU41bldRV0pjcThTenZxTTViZElDaWlwYU9hWWQ3RFEKYWRTejV5dlF0YkxQNW4wTXpnOU43S3pGcEpvUys5QWdkWDI5cmZqV2JSekp3RzM5R3dRemN6VWtLcnZEb05JaQo0UFJHc01yclFNVXFSYjRSajNQOEJodEMxWXNDQXdFQUFhTTVNRGN3Q1FZRFZSMFRCQUl3QURBTEJnTlZIUThFCkJBTUNCZUF3SFFZRFZSMGxCQll3RkFZSUt3WUJCUVVIQXdJR0NDc0dBUVVGQndNQk1BMEdDU3FHU0liM0RRRUIKQ3dVQUE0SUJBUUNKZm5PWlFLWkowTFliOGNWUW5Vdi9NZkRZVEJIQ0pZcGM4MmgzUGlXWElMQk1jWDhQRC93MgpoOUExNkE4NGNxODJuQXEvaFZYYy9JNG9yaFY5WW9jSEg5UlcvbGthTUQ2VEJVR0Q1U1k4S292MHpHQ1ROaDZ6Ci9wZTNqTC9uU0pYSjRtQm51czJheHFtWnIvM3hhaWpYZG9kMmd3eGVhTklvRjNLbHB2aGU3ZjRBNmpsQTM0MmkKVVlCZ09iN1F5KytRZWd4U1diSmdoSzg1MmUvUUhnU2FVSkN6NW1sNGc1WndnNnBTUXhySUhCNkcvREc4dElSYwprZDMxSk1qY25Fb1Rhc1Jyc1NwVmNGdXZyQXlXN2liakZyYzhienBNcE1obDVwYUZRcEZzMnIwaXpZekhwakFsCk5ZR2I2OHJHcjBwQkp3YU5DS2ErbCtLRTk4M3A3NDYwCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K + tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdEY3WEN6TVZHaDF1aU5oWTBJZW50RVlpSVFmUUlLQkMvYUFzU3gxQUlsOWI0OUQ5ClhmanEzdWI3c3RCNnRsYTlqV09keDZkZzBnZDBCSVNCSWFlcHJWdkZNZzRTRXpjRE51aW0xZnh3aVkwZCtFRlUKTXVCc3NYZEV6V0k3ZEVvdUFjcVJjamZWL0J5WTZ4ZDdTRWhjSE5PZVdEZWI5TDFiK3hLd2k2M21uZ0lKQjdBeQpLSmRlYzhnbWlaNk4wcTV3ZXFEWDJ6QVgrbDVPTldTcG1mWUVhVHBDSnFMVTNtZFpCWWx5cnhMTytvemx0MGdLCktLbG81cGgzc05CcDFMUG5LOUMxc3MvbWZRek9EMDNzck1Xa21oTDcwQ0IxZmIydCtOWnRITW5BYmYwYkJETnoKTlNRcXU4T2cwaUxnOUVhd3l1dEF4U3BGdmhHUGMvd0dHMExWaXdJREFRQUJBb0lCQUJKYUErOHp4MVhjNEw0egpwUFd5bDdHVDRTMFRLbTNuWUdtRnZudjJBZXg5WDFBU2wzVFVPckZyTnZpK2xYMnYzYUZoSFZDUEN4N1RlMDVxClhPa2JzZnZkZG5iZFQ2RjgyMnJleVByRXNINk9TUnBWSzBmeDVaMDQwVnRFUDJCWm04eTYyNG1QZk1vbDdya2MKcm9Kd09rOEVpUHZZekpsZUd0bTAwUm1sRysyL2c0aWJsOTVmQXpyc1MvcGUyS3ZoN2NBVEtIcVh6MjlpUmZpbApiTGhBamQwcEVSMjNYU0hHR1ZqRmF3amNJK1c2L2RtbDZURDhrSzFGaUtldmJKTlREeVNXQnpPbXRTYUp1K01JCm9iUnVWWG4yZVNoamVGM1BYcHZRMWRhNXdBa0dJQWxOWjRHTG5QU2ZwVmJyU0plU3RrTGNzdEJheVlJS3BWZVgKSVVTTHM0RUNnWUVBMmNnZUE2WHh0TXdFNU5QWlNWdGhzbXRiYi9YYmtsSTdrWHlsdk5zZjFPdXRYVzkybVJneQpHcEhUQ0VubDB0Z1p3T081T1FLNjdFT3JUdDBRWStxMDJzZndwcmgwNFZEVGZhcW5QNTBxa3BmZEJLQWpmanEyCjFoZDZMd2hLeDRxSm9aelp2VkowV0lvR1ZLcjhJSjJOWGRTUVlUanZUZHhGczRTamdqNFFiaEVDZ1lFQTFBWUUKSEo3eVlza2EvS2V2OVVYbmVrSTRvMm5aYjJ1UVZXazRXSHlaY2NRN3VMQVhGY3lJcW5SZnoxczVzN3RMTzJCagozTFZNUVBzazFNY25oTTl4WE4vQ3ZDTys5b2t0RnNaMGJqWFh6NEJ5V2lFNHJPS1lhVEFwcDVsWlpUT3ZVMWNyCm05R3NwMWJoVDVZb2RaZ3IwUHQyYzR4U2krUVlEWnNFb2lFdzNkc0NnWUVBcVJLYWNweWZKSXlMZEJjZ0JycGkKQTRFalVLMWZsSjR3enNjbGFKUDVoM1NjZUFCejQzRU1YT0kvSXAwMFJsY3N6em83N3cyMmpud09mOEJSM0RBMwp6ZTRSWDIydWw4b0hGdldvdUZOTTNOZjNaNExuYXpVc0F0UGhNS2hRWGMrcEFBWGthUDJkZzZ0TU5PazFxaUNHCndvU212a1BVVE84b1ViRTB1NFZ4ZmZFQ2dZQUpPdDNROVNadUlIMFpSSitIV095enlOQTRaUEkvUkhwN0RXS1QKajVFS2Y5VnR1OVMxY1RyOTJLVVhITXlOUTNrSjg2OUZPMnMvWk85OGg5THptQ2hDTjhkOWN6enI5SnJPNUFMTApqWEtBcVFIUlpLTFgrK0ZRcXZVVlE3cTlpaHQyMEZPb3E5OE5SZDMzSGYxUzZUWDNHZ3RWQ21YSml6dDAxQ3ZHCmR4VnVnd0tCZ0M2Mlp0b0RLb3JyT2hvdTBPelprK2YwQS9rNDJBOENiL29VMGpwSzZtdmxEWmNYdUF1QVZTVXIKNXJCZjRVYmdVYndqa1ZWSFR6LzdDb1BWSjUvVUxJWk1Db1RUNFprNTZXWDk4ZE93Q3VTVFpZYnlBbDZNS1BBZApTZEpuVVIraEpnSVFDVGJ4K1dzYnh2d0FkbWErWUhtaVlPRzZhSklXMXdSd1VGOURLUEhHCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== +``` + +{{% /accordion %}} + + + +## 8. Configure FQDN + +There are two references to `` in the config file (one in this step and one in the next). Both need to be replaced with the FQDN chosen in [Configure DNS](#3-configure-dns). + +In the `kind: Ingress` with `name: cattle-ingress-http`: + +* Replace `` with the FQDN chosen in [Configure DNS](#3-configure-dns). + +After replacing `` with the FQDN chosen in [Configure DNS](#3-configure-dns), the file should look like the example below (`rancher.yourdomain.com` is the FQDN used in this example): + +```yaml + --- + apiVersion: extensions/v1beta1 + kind: Ingress + metadata: + namespace: cattle-system + name: cattle-ingress-http + annotations: + nginx.ingress.kubernetes.io/proxy-connect-timeout: "30" + nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open + nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open + spec: + rules: + - host: rancher.yourdomain.com + http: + paths: + - backend: + serviceName: cattle-service + servicePort: 80 + tls: + - secretName: cattle-keys-ingress + hosts: + - rancher.yourdomain.com +``` + +Save the `.yml` file and close it. + +## 9. Configure Rancher version + +The last reference that needs to be replaced is ``. This needs to be replaced with a Rancher version which is marked as stable. The latest stable release of Rancher can be found in the [GitHub README](https://github.com/rancher/rancher/blob/master/README.md). Make sure the version is an actual version number, and not a named tag like `stable` or `latest`. The example below shows the version configured to `v2.0.6`. + +``` + spec: + serviceAccountName: cattle-admin + containers: + - image: rancher/rancher:v2.0.6 + imagePullPolicy: Always +``` + +## 10. Back Up Your RKE Config File + +After you close your `.yml` file, back it up to a secure location. You can use this file again when it's time to upgrade Rancher. + +## 11. Run RKE + +With all configuration in place, use RKE to launch Rancher. You can complete this action by running the `rke up` command and using the `--config` parameter to point toward your config file. + +1. From your workstation, make sure `rancher-cluster.yml` and the downloaded `rke` binary are in the same directory. + +2. Open a Terminal instance. Change to the directory that contains your config file and `rke`. + +3. Enter one of the `rke up` commands listen below. + +``` +rke up --config rancher-cluster.yml +``` + +**Step Result:** The output should be similar to the snippet below: + +``` +INFO[0000] Building Kubernetes cluster +INFO[0000] [dialer] Setup tunnel for host [1.1.1.1] +INFO[0000] [network] Deploying port listener containers +INFO[0000] [network] Pulling image [alpine:latest] on host [1.1.1.1] +... +INFO[0101] Finished building Kubernetes cluster successfully +``` + +## 12. Back Up Auto-Generated Config File + +During installation, RKE automatically generates a config file named `kube_config_rancher-cluster.yml` in the same directory as the RKE binary. Copy this file and back it up to a safe location. You'll use this file later when upgrading Rancher Server. + +## What's Next? + +You have a couple of options: + +- Create a backup of your Rancher Server in case of a disaster scenario: [High Availability Back Up and Restoration]({{}}/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration). +- Create a Kubernetes cluster: [Provisioning Kubernetes Clusters]({{}}/rancher/v2.x/en/cluster-provisioning/). + +
+ +## FAQ and Troubleshooting + +{{< ssl_faq_ha >}} diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/nlb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/nlb/_index.md new file mode 100644 index 00000000000..334253353c1 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-4-lb/nlb/_index.md @@ -0,0 +1,182 @@ +--- +title: Amazon NLB Configuration +weight: 277 +aliases: +- /rancher/v2.x/en/installation/ha-server-install/nlb/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-4-lb/nlb +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a High-availability Kubernetes install with an RKE add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +## Objectives + +Configuring an Amazon NLB is a multistage process. We've broken it down into multiple tasks so that it's easy to follow. + +1. [Create Target Groups](#create-target-groups) + + Begin by creating two target groups for the **TCP** protocol, one regarding TCP port 443 and one regarding TCP port 80 (providing redirect to TCP port 443). You'll add your Linux nodes to these groups. + +2. [Register Targets](#register-targets) + + Add your Linux nodes to the target groups. + +3. [Create Your NLB](#create-your-nlb) + + Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in **1. Create Target Groups**. + + +## Create Target Groups + +Your first NLB configuration step is to create two target groups. Technically, only port 443 is needed to access Rancher, but its convenient to add a listener for port 80 which will be redirected to port 443 automatically. The NGINX controller on the nodes will make sure that port 80 gets redirected to port 443. + +Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started, make sure to select the **Region** where your EC2 instances (Linux nodes) are created. + +The Target Groups configuration resides in the **Load Balancing** section of the **EC2** service. Select **Services** and choose **EC2**, find the section **Load Balancing** and open **Target Groups**. + +{{< img "/img/rancher/ha/nlb/ec2-loadbalancing.png" "EC2 Load Balancing section">}} + +Click **Create target group** to create the first target group, regarding TCP port 443. + +### Target Group (TCP port 443) + +Configure the first target group according to the table below. Screenshots of the configuration are shown just below the table. + +Option | Setting +--------------------------------------|------------------------------------ +Target Group Name | `rancher-tcp-443` +Protocol | `TCP` +Port | `443` +Target type | `instance` +VPC | Choose your VPC +Protocol
(Health Check) | `HTTP` +Path
(Health Check) | `/healthz` +Port (Advanced health check) | `override`,`80` +Healthy threshold (Advanced health) | `3` +Unhealthy threshold (Advanced) | `3` +Timeout (Advanced) | `6 seconds` +Interval (Advanced) | `10 second` +Success codes | `200-399` + +
+**Screenshot Target group TCP port 443 settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-443.png" "Target group 443">}} + +
+**Screenshot Target group TCP port 443 Advanced settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-443-advanced.png" "Target group 443 Advanced">}} + +
+ +Click **Create target group** to create the second target group, regarding TCP port 80. + +### Target Group (TCP port 80) + +Configure the second target group according to the table below. Screenshots of the configuration are shown just below the table. + +Option | Setting +--------------------------------------|------------------------------------ +Target Group Name | `rancher-tcp-80` +Protocol | `TCP` +Port | `80` +Target type | `instance` +VPC | Choose your VPC +Protocol
(Health Check) | `HTTP` +Path
(Health Check) | `/healthz` +Port (Advanced health check) | `traffic port` +Healthy threshold (Advanced health) | `3` +Unhealthy threshold (Advanced) | `3` +Timeout (Advanced) | `6 seconds` +Interval (Advanced) | `10 second` +Success codes | `200-399` + +
+**Screenshot Target group TCP port 80 settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-80.png" "Target group 80">}} + +
+**Screenshot Target group TCP port 80 Advanced settings**
+{{< img "/img/rancher/ha/nlb/create-targetgroup-80-advanced.png" "Target group 80 Advanced">}} + +
+ +## Register Targets + +Next, add your Linux nodes to both target groups. + +Select the target group named **rancher-tcp-443**, click the tab **Targets** and choose **Edit**. + +{{< img "/img/rancher/ha/nlb/edit-targetgroup-443.png" "Edit target group 443">}} + +Select the instances (Linux nodes) you want to add, and click **Add to registered**. + +
+**Screenshot Add targets to target group TCP port 443**
+ +{{< img "/img/rancher/ha/nlb/add-targets-targetgroup-443.png" "Add targets to target group 443">}} + +
+**Screenshot Added targets to target group TCP port 443**
+ +{{< img "/img/rancher/ha/nlb/added-targets-targetgroup-443.png" "Added targets to target group 443">}} + +When the instances are added, click **Save** on the bottom right of the screen. + +Repeat those steps, replacing **rancher-tcp-443** with **rancher-tcp-80**. The same instances need to be added as targets to this target group. + +## Create Your NLB + +Use Amazon's Wizard to create an Network Load Balancer. As part of this process, you'll add the target groups you created in [Create Target Groups](#create-target-groups). + +1. From your web browser, navigate to the [Amazon EC2 Console](https://console.aws.amazon.com/ec2/). + +2. From the navigation pane, choose **LOAD BALANCING** > **Load Balancers**. + +3. Click **Create Load Balancer**. + +4. Choose **Network Load Balancer** and click **Create**. + +5. Complete the **Step 1: Configure Load Balancer** form. + - **Basic Configuration** + + - Name: `rancher` + - Scheme: `internet-facing` + - **Listeners** + + Add the **Load Balancer Protocols** and **Load Balancer Ports** below. + - `TCP`: `443` + + - **Availability Zones** + + - Select Your **VPC** and **Availability Zones**. + +6. Complete the **Step 2: Configure Routing** form. + + - From the **Target Group** drop-down, choose **Existing target group**. + + - From the **Name** drop-down, choose `rancher-tcp-443`. + + - Open **Advanced health check settings**, and configure **Interval** to `10 seconds`. + +7. Complete **Step 3: Register Targets**. Since you registered your targets earlier, all you have to do is click **Next: Review**. + +8. Complete **Step 4: Review**. Look over the load balancer details and click **Create** when you're satisfied. + +9. After AWS creates the NLB, click **Close**. + +## Add listener to NLB for TCP port 80 + +1. Select your newly created NLB and select the **Listeners** tab. + +2. Click **Add listener**. + +3. Use `TCP`:`80` as **Protocol** : **Port** + +4. Click **Add action** and choose **Forward to...** + +5. From the **Forward to** drop-down, choose `rancher-tcp-80`. + +6. Click **Save** in the top right of the screen. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/_index.md new file mode 100644 index 00000000000..d4f5bab4941 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/_index.md @@ -0,0 +1,289 @@ +--- +title: Kubernetes Install with External Load Balancer (HTTPS/Layer 7) +weight: 276 +aliases: +- /rancher/v2.x/en/installation/ha-server-install-external-lb/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-7-lb +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +This procedure walks you through setting up a 3-node cluster using the Rancher Kubernetes Engine (RKE). The cluster's sole purpose is running pods for Rancher. The setup is based on: + +- Layer 7 Loadbalancer with SSL termination (HTTPS) +- [NGINX Ingress controller (HTTP)](https://kubernetes.github.io/ingress-nginx/) + +In an Kubernetes setup that uses a layer 7 load balancer, the load balancer accepts Rancher client connections over the HTTP protocol (i.e., the application level). This application-level access allows the load balancer to read client requests and then redirect to them to cluster nodes using logic that optimally distributes load. + +Kubernetes Rancher install with layer 7 load balancer, depicting SSL termination at load balancer +![Rancher HA]({{}}/img/rancher/ha/rancher2ha-l7.svg) + +## Installation Outline + +Installation of Rancher in a high-availability configuration involves multiple procedures. Review this outline to learn about each procedure you need to complete. + + + +- [1. Provision Linux Hosts](#1-provision-linux-hosts) +- [2. Configure Load Balancer](#2-configure-load-balancer) +- [3. Configure DNS](#3-configure-dns) +- [4. Install RKE](#4-install-rke) +- [5. Download RKE Config File Template](#5-download-rke-config-file-template) +- [6. Configure Nodes](#6-configure-nodes) +- [7. Configure Certificates](#7-configure-certificates) +- [8. Configure FQDN](#8-configure-fqdn) +- [9. Configure Rancher version](#9-configure-rancher-version) +- [10. Back Up Your RKE Config File](#10-back-up-your-rke-config-file) +- [11. Run RKE](#11-run-rke) +- [12. Back Up Auto-Generated Config File](#12-back-up-auto-generated-config-file) + + + +## 1. Provision Linux Hosts + +Provision three Linux hosts according to our [Requirements]({{}}/rancher/v2.x/en/installation/requirements). + +## 2. Configure Load Balancer + +When using a load balancer in front of Rancher, there's no need for the container to redirect port communication from port 80 or port 443. By passing the header `X-Forwarded-Proto: https`, this redirect is disabled. This is the expected configuration when terminating SSL externally. + +The load balancer has to be configured to support the following: + +* **WebSocket** connections +* **SPDY** / **HTTP/2** protocols +* Passing / setting the following headers: + +| Header | Value | Description | +|---------------------|----------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `Host` | FQDN used to reach Rancher. | To identify the server requested by the client. | +| `X-Forwarded-Proto` | `https` | To identify the protocol that a client used to connect to the load balancer.

**Note:** If this header is present, `rancher/rancher` does not redirect HTTP to HTTPS. | +| `X-Forwarded-Port` | Port used to reach Rancher. | To identify the protocol that client used to connect to the load balancer. | +| `X-Forwarded-For` | IP of the client connection. | To identify the originating IP address of a client. | + +Health checks can be executed on the `/healthz` endpoint of the node, this will return HTTP 200. + +We have example configurations for the following load balancers: + +* [Amazon ALB configuration](alb/) +* [NGINX configuration](nginx/) + +## 3. Configure DNS + +Choose a fully qualified domain name (FQDN) that you want to use to access Rancher (e.g., `rancher.yourdomain.com`).

+ +1. Log into your DNS server a create a `DNS A` record that points to the IP address of your [load balancer](#2-configure-load-balancer). + +2. Validate that the `DNS A` is working correctly. Run the following command from any terminal, replacing `HOSTNAME.DOMAIN.COM` with your chosen FQDN: + + `nslookup HOSTNAME.DOMAIN.COM` + + **Step Result:** Terminal displays output similar to the following: + + ``` + $ nslookup rancher.yourdomain.com + Server: YOUR_HOSTNAME_IP_ADDRESS + Address: YOUR_HOSTNAME_IP_ADDRESS#53 + + Non-authoritative answer: + Name: rancher.yourdomain.com + Address: HOSTNAME.DOMAIN.COM + ``` + +
+ +## 4. Install RKE + +RKE (Rancher Kubernetes Engine) is a fast, versatile Kubernetes installer that you can use to install Kubernetes on your Linux hosts. We will use RKE to setup our cluster and run Rancher. + +1. Follow the [RKE Install]({{}}/rke/latest/en/installation) instructions. + +2. Confirm that RKE is now executable by running the following command: + + ``` + rke --version + ``` + +## 5. Download RKE Config File Template + +RKE uses a YAML config file to install and configure your Kubernetes cluster. There are 2 templates to choose from, depending on the SSL certificate you want to use. + +1. Download one of following templates, depending on the SSL certificate you're using. + + - [Template for self-signed certificate
`3-node-externalssl-certificate.yml`](https://raw.githubusercontent.com/rancher/rancher/master/rke-templates/3-node-externalssl-certificate.yml) + - [Template for certificate signed by recognized CA
`3-node-externalssl-recognizedca.yml`](https://raw.githubusercontent.com/rancher/rancher/master/rke-templates/3-node-externalssl-recognizedca.yml) + + >**Advanced Config Options:** + > + >- Want records of all transactions with the Rancher API? Enable the [API Auditing]({{}}/rancher/v2.x/en/installation/api-auditing) feature by editing your RKE config file. For more information, see how to enable it in [your RKE config file]({{}}/rancher/v2.x/en/installation/options/helm2/rke-add-on/api-auditing/). + >- Want to know the other config options available for your RKE template? See the [RKE Documentation: Config Options]({{}}/rke/latest/en/config-options/). + + +2. Rename the file to `rancher-cluster.yml`. + +## 6. Configure Nodes + +Once you have the `rancher-cluster.yml` config file template, edit the nodes section to point toward your Linux hosts. + +1. Open `rancher-cluster.yml` in your favorite text editor. + +1. Update the `nodes` section with the information of your [Linux hosts](#1-provision-linux-hosts). + + For each node in your cluster, update the following placeholders: `IP_ADDRESS_X` and `USER`. The specified user should be able to access the Docket socket, you can test this by logging in with the specified user and run `docker ps`. + + >**Note:** + > + >When using RHEL/CentOS, the SSH user can't be root due to https://bugzilla.redhat.com/show_bug.cgi?id=1527565. See [Operating System Requirements]({{}}/rke/latest/en/installation/os#redhat-enterprise-linux-rhel-centos) for RHEL/CentOS specific requirements. + + nodes: + # The IP address or hostname of the node + - address: IP_ADDRESS_1 + # User that can login to the node and has access to the Docker socket (i.e. can execute `docker ps` on the node) + # When using RHEL/CentOS, this can't be root due to https://bugzilla.redhat.com/show_bug.cgi?id=1527565 + user: USER + role: [controlplane,etcd,worker] + # Path the SSH key that can be used to access to node with the specified user + ssh_key_path: ~/.ssh/id_rsa + - address: IP_ADDRESS_2 + user: USER + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + - address: IP_ADDRESS_3 + user: USER + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + +1. **Optional:** By default, `rancher-cluster.yml` is configured to take backup snapshots of your data. To disable these snapshots, change the `backup` directive setting to `false`, as depicted below. + + services: + etcd: + backup: false + +## 7. Configure Certificates + +For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster. + +Choose from the following options: + +{{% accordion id="option-a" label="Option A—Bring Your Own Certificate: Self-Signed" %}} +>**Prerequisites:** +>Create a self-signed certificate. +> +>- The certificate files must be in [PEM format](#pem). +>- The certificate files must be encoded in [base64](#base64). +>- In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting](#cert-order). + +In `kind: Secret` with `name: cattle-keys-ingress`, replace `` with the base64 encoded string of the CA Certificate file (usually called `ca.pem` or `ca.crt`) + +>**Note:** The base64 encoded string should be on the same line as `cacerts.pem`, without any newline at the beginning, in between or at the end. + +After replacing the values, the file should look like the example below (the base64 encoded strings should be different): + + --- + apiVersion: v1 + kind: Secret + metadata: + name: cattle-keys-server + namespace: cattle-system + type: Opaque + data: + cacerts.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNvRENDQVlnQ0NRRHVVWjZuMEZWeU16QU5CZ2txaGtpRzl3MEJBUXNGQURBU01SQXdEZ1lEVlFRRERBZDAKWlhOMExXTmhNQjRYRFRFNE1EVXdOakl4TURRd09Wb1hEVEU0TURjd05USXhNRFF3T1Zvd0VqRVFNQTRHQTFVRQpBd3dIZEdWemRDMWpZVENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFNQmpBS3dQCndhRUhwQTdaRW1iWWczaTNYNlppVmtGZFJGckJlTmFYTHFPL2R0RUdmWktqYUF0Wm45R1VsckQxZUlUS3UzVHgKOWlGVlV4Mmo1Z0tyWmpwWitCUnFiZ1BNbk5hS1hocmRTdDRtUUN0VFFZdGRYMVFZS0pUbWF5NU45N3FoNTZtWQprMllKRkpOWVhHWlJabkdMUXJQNk04VHZramF0ZnZOdmJ0WmtkY2orYlY3aWhXanp2d2theHRUVjZlUGxuM2p5CnJUeXBBTDliYnlVcHlad3E2MWQvb0Q4VUtwZ2lZM1dOWmN1YnNvSjhxWlRsTnN6UjVadEFJV0tjSE5ZbE93d2oKaG41RE1tSFpwZ0ZGNW14TU52akxPRUc0S0ZRU3laYlV2QzlZRUhLZTUxbGVxa1lmQmtBZWpPY002TnlWQUh1dApuay9DMHpXcGdENkIwbkVDQXdFQUFUQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFHTCtaNkRzK2R4WTZsU2VBClZHSkMvdzE1bHJ2ZXdia1YxN3hvcmlyNEMxVURJSXB6YXdCdFJRSGdSWXVtblVqOGo4T0hFWUFDUEthR3BTVUsKRDVuVWdzV0pMUUV0TDA2eTh6M3A0MDBrSlZFZW9xZlVnYjQrK1JLRVJrWmowWXR3NEN0WHhwOVMzVkd4NmNOQQozZVlqRnRQd2hoYWVEQmdma1hXQWtISXFDcEsrN3RYem9pRGpXbi8walI2VDcrSGlaNEZjZ1AzYnd3K3NjUDIyCjlDQVZ1ZFg4TWpEQ1hTcll0Y0ZINllBanlCSTJjbDhoSkJqa2E3aERpVC9DaFlEZlFFVFZDM3crQjBDYjF1NWcKdE03Z2NGcUw4OVdhMnp5UzdNdXk5bEthUDBvTXl1Ty82Tm1wNjNsVnRHeEZKSFh4WTN6M0lycGxlbTNZQThpTwpmbmlYZXc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== + +{{% /accordion %}} +{{% accordion id="option-b" label="Option B—Bring Your Own Certificate: Signed by Recognized CA" %}} +If you are using a Certificate Signed By A Recognized Certificate Authority, you don't need to perform any step in this part. +{{% /accordion %}} + +## 8. Configure FQDN + +There is one reference to `` in the RKE config file. Replace this reference with the FQDN you chose in [3. Configure DNS](#3-configure-dns). + +1. Open `rancher-cluster.yml`. + +2. In the `kind: Ingress` with `name: cattle-ingress-http:` + + Replace `` with the FQDN chosen in [3. Configure DNS](#3-configure-dns). + + **Step Result:** After replacing the values, the file should look like the example below (the base64 encoded strings should be different): + + ``` + apiVersion: extensions/v1beta1 + kind: Ingress + metadata: + namespace: cattle-system + name: cattle-ingress-http + annotations: + nginx.ingress.kubernetes.io/proxy-connect-timeout: "30" + nginx.ingress.kubernetes.io/proxy-read-timeout: "1800" # Max time in seconds for ws to remain shell window open + nginx.ingress.kubernetes.io/proxy-send-timeout: "1800" # Max time in seconds for ws to remain shell window open + spec: + rules: + - host: rancher.yourdomain.com + http: + paths: + - backend: + serviceName: cattle-service + servicePort: 80 + ``` + + +3. Save the file and close it. + +## 9. Configure Rancher version + +The last reference that needs to be replaced is ``. This needs to be replaced with a Rancher version which is marked as stable. The latest stable release of Rancher can be found in the [GitHub README](https://github.com/rancher/rancher/blob/master/README.md). Make sure the version is an actual version number, and not a named tag like `stable` or `latest`. The example below shows the version configured to `v2.0.6`. + +``` + spec: + serviceAccountName: cattle-admin + containers: + - image: rancher/rancher:v2.0.6 + imagePullPolicy: Always +``` + +## 10. Back Up Your RKE Config File + +After you close your RKE config file, `rancher-cluster.yml`, back it up to a secure location. You can use this file again when it's time to upgrade Rancher. + +## 11. Run RKE + +With all configuration in place, use RKE to launch Rancher. You can complete this action by running the `rke up` command and using the `--config` parameter to point toward your config file. + +1. From your workstation, make sure `rancher-cluster.yml` and the downloaded `rke` binary are in the same directory. + +2. Open a Terminal instance. Change to the directory that contains your config file and `rke`. + +3. Enter one of the `rke up` commands listen below. + + ``` + rke up --config rancher-cluster.yml + ``` + + **Step Result:** The output should be similar to the snippet below: + + ``` + INFO[0000] Building Kubernetes cluster + INFO[0000] [dialer] Setup tunnel for host [1.1.1.1] + INFO[0000] [network] Deploying port listener containers + INFO[0000] [network] Pulling image [alpine:latest] on host [1.1.1.1] + ... + INFO[0101] Finished building Kubernetes cluster successfully + ``` + +## 12. Back Up Auto-Generated Config File + +During installation, RKE automatically generates a config file named `kube_config_rancher-cluster.yml` in the same directory as the `rancher-cluster.yml` file. Copy this file and back it up to a safe location. You'll use this file later when upgrading Rancher Server. + +## What's Next? + +- **Recommended:** Review [Creating Backups—High Availability Back Up and Restoration]({{}}/rancher/v2.x/en/backups/backups/ha-backups/) to learn how to backup your Rancher Server in case of a disaster scenario. +- Create a Kubernetes cluster: [Creating a Cluster]({{}}/rancher/v2.x/en/tasks/clusters/creating-a-cluster/). + +
+ +## FAQ and Troubleshooting + +{{< ssl_faq_ha >}} diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/alb/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/alb/_index.md new file mode 100644 index 00000000000..603c1b2144b --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/alb/_index.md @@ -0,0 +1,104 @@ +--- +title: Amazon ALB Configuration +weight: 277 +aliases: +- /rancher/v2.x/en/installation/ha-server-install-external-lb/alb/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-7-lb/alb +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher helm chart to install Kubernetes Rancher. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +## Objectives + +Configuring an Amazon ALB is a multistage process. We've broken it down into multiple tasks so that it's easy to follow. + +1. [Create Target Group](#create-target-group) + + Begin by creating one target group for the http protocol. You'll add your Linux nodes to this group. + +2. [Register Targets](#register-targets) + + Add your Linux nodes to the target group. + +3. [Create Your ALB](#create-your-alb) + + Use Amazon's Wizard to create an Application Load Balancer. As part of this process, you'll add the target groups you created in **1. Create Target Groups**. + + +## Create Target Group + +Your first ALB configuration step is to create one target group for HTTP. + +Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started. + +The document below will guide you through this process. Use the data in the tables below to complete the procedure. + +[Amazon Documentation: Create a Target Group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-target-group.html) + +### Target Group (HTTP) + +Option | Setting +----------------------------|------------------------------------ +Target Group Name | `rancher-http-80` +Protocol | `HTTP` +Port | `80` +Target type | `instance` +VPC | Choose your VPC +Protocol
(Health Check) | `HTTP` +Path
(Health Check) | `/healthz` + +## Register Targets + +Next, add your Linux nodes to your target group. + +[Amazon Documentation: Register Targets with Your Target Group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-register-targets.html) + +### Create Your ALB + +Use Amazon's Wizard to create an Application Load Balancer. As part of this process, you'll add the target group you created in [Create Target Group](#create-target-group). + +1. From your web browser, navigate to the [Amazon EC2 Console](https://console.aws.amazon.com/ec2/). + +2. From the navigation pane, choose **LOAD BALANCING** > **Load Balancers**. + +3. Click **Create Load Balancer**. + +4. Choose **Application Load Balancer**. + +5. Complete the **Step 1: Configure Load Balancer** form. + - **Basic Configuration** + + - Name: `rancher-http` + - Scheme: `internet-facing` + - IP address type: `ipv4` + - **Listeners** + + Add the **Load Balancer Protocols** and **Load Balancer Ports** below. + - `HTTP`: `80` + - `HTTPS`: `443` + + - **Availability Zones** + + - Select Your **VPC** and **Availability Zones**. + +6. Complete the **Step 2: Configure Security Settings** form. + + Configure the certificate you want to use for SSL termination. + +7. Complete the **Step 3: Configure Security Groups** form. + +8. Complete the **Step 4: Configure Routing** form. + + - From the **Target Group** drop-down, choose **Existing target group**. + + - Add target group `rancher-http-80`. + +9. Complete **Step 5: Register Targets**. Since you registered your targets earlier, all you have to do it click **Next: Review**. + +10. Complete **Step 6: Review**. Look over the load balancer details and click **Create** when you're satisfied. + +11. After AWS creates the ALB, click **Close**. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/nginx/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/nginx/_index.md new file mode 100644 index 00000000000..20b5aa5f89e --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/layer-7-lb/nginx/_index.md @@ -0,0 +1,41 @@ +--- +title: NGINX Configuration +weight: 277 +aliases: +- /rancher/v2.x/en/installation/ha-server-install-external-lb/nginx/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/layer-7-lb/nginx +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +## Install NGINX + +Start by installing NGINX on your load balancer host. NGINX has packages available for all known operating systems. + +For help installing NGINX, refer to their [install documentation](https://www.nginx.com/resources/wiki/start/topics/tutorials/install/). + +## Create NGINX Configuration + +See [Example NGINX config]({{}}/rancher/v2.x/en/installation/options/helm2/helm-rancher/chart-options/#example-nginx-config). + +## Run NGINX + +* Reload or restart NGINX + + ```` + # Reload NGINX + nginx -s reload + + # Restart NGINX + # Depending on your Linux distribution + service nginx restart + systemctl restart nginx + ```` + +## Browse to Rancher UI + +You should now be to able to browse to `https://FQDN`. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/proxy/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/proxy/_index.md new file mode 100644 index 00000000000..454f98ac60b --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/proxy/_index.md @@ -0,0 +1,71 @@ +--- +title: HTTP Proxy Configuration +weight: 277 +aliases: + - /rancher/v2.x/en/installation/options/helm2/rke-add-on/proxy +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +If you operate Rancher behind a proxy and you want to access services through the proxy (such as retrieving catalogs), you must provide Rancher information about your proxy. As Rancher is written in Go, it uses the common proxy environment variables as shown below. + +Make sure `NO_PROXY` contains the network addresses, network address ranges and domains that should be excluded from using the proxy. + +Environment variable | Purpose +--------------------------|--------- +HTTP_PROXY | Proxy address to use when initiating HTTP connection(s) +HTTPS_PROXY | Proxy address to use when initiating HTTPS connection(s) +NO_PROXY | Network address(es), network address range(s) and domains to exclude from using the proxy when initiating connection(s) + +> **Note** NO_PROXY must be in uppercase to use network range (CIDR) notation. + +## Kubernetes installation + +When using Kubernetes installation, the environment variables need to be added to the RKE Config File template. + +* [Kubernetes Installation with External Load Balancer (TCP/Layer 4) RKE Config File Template]({{}}/rancher/v2.x/en/installation/k8s-install-server-install/#5-download-rke-config-file-template) +* [Kubernetes Installation with External Load Balancer (HTTPS/Layer 7) RKE Config File Template]({{}}/rancher/v2.x/en/installation/k8s-install-server-install-external-lb/#5-download-rke-config-file-template) + +The environment variables should be defined in the `Deployment` inside the RKE Config File Template. You only have to add the part starting with `env:` to (but not including) `ports:`. Make sure the indentation is identical to the preceding `name:`. Required values for `NO_PROXY` are: + +* `localhost` +* `127.0.0.1` +* `0.0.0.0` +* Configured `service_cluster_ip_range` (default: `10.43.0.0/16`) + +The example below is based on a proxy server accessible at `http://192.168.0.1:3128`, and excluding usage of the proxy when accessing network range `192.168.10.0/24`, the configured `service_cluster_ip_range` (`10.43.0.0/16`) and every hostname under the domain `example.com`. If you have changed the `service_cluster_ip_range`, you have to update the value below accordingly. + +```yaml +... +--- + kind: Deployment + apiVersion: extensions/v1beta1 + metadata: + namespace: cattle-system + name: cattle + spec: + replicas: 1 + template: + metadata: + labels: + app: cattle + spec: + serviceAccountName: cattle-admin + containers: + - image: rancher/rancher:latest + imagePullPolicy: Always + name: cattle-server + env: + - name: HTTP_PROXY + value: "http://192.168.10.1:3128" + - name: HTTPS_PROXY + value: "http://192.168.10.1:3128" + - name: NO_PROXY + value: "localhost,127.0.0.1,0.0.0.0,10.43.0.0/16,192.168.10.0/24,example.com" + ports: +... +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/404-default-backend/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/404-default-backend/_index.md new file mode 100644 index 00000000000..ed4dda0817c --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/404-default-backend/_index.md @@ -0,0 +1,49 @@ +--- +title: 404 - default backend +weight: 30 +aliases: +- /rancher/v2.x/en/installation/troubleshooting-ha/404-default-backend/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/troubleshooting/404-default-backend +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +To debug issues around this error, you will need to download the command-line tool `kubectl`. See [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) how to download `kubectl` for your platform. + +When you have made changes to `rancher-cluster.yml`, you will have to run `rke remove --config rancher-cluster.yml` to clean the nodes, so it cannot conflict with previous configuration errors. + +### Possible causes + +The nginx ingress controller is not able to serve the configured host in `rancher-cluster.yml`. This should be the FQDN you configured to access Rancher. You can check if it is properly configured by viewing the ingress that is created by running the following command: + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml get ingress -n cattle-system -o wide +``` + +Check if the `HOSTS` column is displaying the FQDN you configured in the template, and that the used nodes are listed in the `ADDRESS` column. If that is configured correctly, we can check the logging of the nginx ingress controller. + +The logging of the nginx ingress controller will show why it cannot serve the requested host. To view the logs, you can run the following command + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs -l app=ingress-nginx -n ingress-nginx +``` + +Errors + +* `x509: certificate is valid for fqdn, not your_configured_fqdn` + +The used certificates do not contain the correct hostname. Generate new certificates that contain the chosen FQDN to access Rancher and redeploy. + +* `Port 80 is already in use. Please check the flag --http-port` + +There is a process on the node occupying port 80, this port is needed for the nginx ingress controller to route requests to Rancher. You can find the process by running the command: `netstat -plant | grep \:80`. + +Stop/kill the process and redeploy. + +* `unexpected error creating pem file: no valid PEM formatted block found` + +The base64 encoded string configured in the template is not valid. Please check if you can decode the configured string using `base64 -D STRING`, this should return the same output as the content of the file you used to generate the string. If this is correct, please check if the base64 encoded string is placed directly after the key, without any newlines before, in between or after. (For example: `tls.crt: LS01..`) diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/_index.md new file mode 100644 index 00000000000..c4a25094cdd --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/_index.md @@ -0,0 +1,33 @@ +--- +title: Troubleshooting HA RKE Add-On Install +weight: 370 +aliases: +- /rancher/v2.x/en/installation/troubleshooting-ha/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/troubleshooting +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +This section contains common errors seen when setting up a Kubernetes installation. + +Choose from the following options: + +- [Generic troubleshooting](generic-troubleshooting/) + + In this section, you can find generic ways to debug your Kubernetes cluster. + +- [Failed to set up SSH tunneling for host]({{}}/rke/latest/en/troubleshooting/ssh-connectivity-errors/) + + In this section, you can find errors related to SSH tunneling when you run the `rke` command to setup your nodes. + +- [Failed to get job complete status](job-complete-status/) + + In this section, you can find errors related to deploying addons. + +- [404 - default backend](404-default-backend/) + + In this section, you can find errors related to the `404 - default backend` page that is shown when trying to access Rancher. diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/generic-troubleshooting/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/generic-troubleshooting/_index.md new file mode 100644 index 00000000000..af5f42679bb --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/generic-troubleshooting/_index.md @@ -0,0 +1,162 @@ +--- +title: Generic troubleshooting +weight: 5 +aliases: +- /rancher/v2.x/en/installation/troubleshooting-ha/generic-troubleshooting/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/troubleshooting/generic-troubleshooting +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +Below are steps that you can follow to determine what is wrong in your cluster. + +### Double check if all the required ports are opened in your (host) firewall + +Double check if all the [required ports]({{}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) are opened in your (host) firewall. + +### All nodes should be present and in **Ready** state + +To check, run the command: + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml get nodes +``` + +If a node is not shown in this output or a node is not in **Ready** state, you can check the logging of the `kubelet` container. Login to the node and run `docker logs kubelet`. + +### All pods/jobs should be in **Running**/**Completed** state + +To check, run the command: + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml get pods --all-namespaces +``` + +If a pod is not in **Running** state, you can dig into the root cause by running: + +#### Describe pod + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml describe pod POD_NAME -n NAMESPACE +``` + +#### Pod container logs + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs POD_NAME -n NAMESPACE +``` + +If a job is not in **Completed** state, you can dig into the root cause by running: + +#### Describe job + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml describe job JOB_NAME -n NAMESPACE +``` + +#### Logs from the containers of pods of the job + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs -l job-name=JOB_NAME -n NAMESPACE +``` + +### Check ingress + +Ingress should have the correct `HOSTS` (showing the configured FQDN) and `ADDRESS` (address(es) it will be routed to). + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml get ingress --all-namespaces +``` + +### List all Kubernetes cluster events + +Kubernetes cluster events are stored, and can be retrieved by running: + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml get events --all-namespaces +``` + +### Check Rancher container logging + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs -l app=cattle -n cattle-system +``` + +### Check NGINX ingress controller logging + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs -l app=ingress-nginx -n ingress-nginx +``` + +### Check if overlay network is functioning correctly + +The pod can be scheduled to any of the hosts you used for your cluster, but that means that the NGINX ingress controller needs to be able to route the request from `NODE_1` to `NODE_2`. This happens over the overlay network. If the overlay network is not functioning, you will experience intermittent TCP/HTTP connection failures due to the NGINX ingress controller not being able to route to the pod. + +To test the overlay network, you can launch the following `DaemonSet` definition. This will run an `alpine` container on every host, which we will use to run a `ping` test between containers on all hosts. + +1. Save the following file as `ds-alpine.yml` + + ``` + apiVersion: apps/v1 + kind: DaemonSet + metadata: + name: alpine + spec: + selector: + matchLabels: + name: alpine + template: + metadata: + labels: + name: alpine + spec: + tolerations: + - effect: NoExecute + key: "node-role.kubernetes.io/etcd" + value: "true" + - effect: NoSchedule + key: "node-role.kubernetes.io/controlplane" + value: "true" + containers: + - image: alpine + imagePullPolicy: Always + name: alpine + command: ["sh", "-c", "tail -f /dev/null"] + terminationMessagePath: /dev/termination-log + ``` + +2. Launch it using `kubectl --kubeconfig kube_config_rancher-cluster.yml create -f ds-alpine.yml` +3. Wait until `kubectl --kubeconfig kube_config_rancher-cluster.yml rollout status ds/alpine -w` returns: `daemon set "alpine" successfully rolled out`. +4. Run the following command to let each container on every host ping each other (it's a single line command). + + ``` + echo "=> Start"; kubectl --kubeconfig kube_config_rancher-cluster.yml get pods -l name=alpine -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.nodeName}{"\n"}{end}' | while read spod shost; do kubectl --kubeconfig kube_config_rancher-cluster.yml get pods -l name=alpine -o jsonpath='{range .items[*]}{@.status.podIP}{" "}{@.spec.nodeName}{"\n"}{end}' | while read tip thost; do kubectl --kubeconfig kube_config_rancher-cluster.yml --request-timeout='10s' exec $spod -- /bin/sh -c "ping -c2 $tip > /dev/null 2>&1"; RC=$?; if [ $RC -ne 0 ]; then echo $shost cannot reach $thost; fi; done; done; echo "=> End" + ``` + +5. When this command has finished running, the output indicating everything is correct is: + + ``` + => Start + => End + ``` + +If you see error in the output, that means that the [required ports]({{}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for overlay networking are not opened between the hosts indicated. + +Example error output of a situation where NODE1 had the UDP ports blocked. + +``` +=> Start +command terminated with exit code 1 +NODE2 cannot reach NODE1 +command terminated with exit code 1 +NODE3 cannot reach NODE1 +command terminated with exit code 1 +NODE1 cannot reach NODE2 +command terminated with exit code 1 +NODE1 cannot reach NODE3 +=> End +``` diff --git a/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/job-complete-status/_index.md b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/job-complete-status/_index.md new file mode 100644 index 00000000000..583d124da07 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/advanced/helm2/rke-add-on/troubleshooting/job-complete-status/_index.md @@ -0,0 +1,63 @@ +--- +title: Failed to get job complete status +weight: 20 +aliases: +- /rancher/v2.x/en/installation/troubleshooting-ha/job-complete-status/ +- /rancher/v2.x/en/installation/options/helm2/rke-add-on/troubleshooting/job-complete-status +--- + +> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +> +>Please use the Rancher Helm chart to install Rancher on a Kubernetes cluster. For details, see the [Kubernetes Install - Installation Outline]({{}}/rancher/v2.x/en/installation/options/helm2/#installation-outline). +> +>If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on]({{}}/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/) for details on how to move to using the helm chart. + +To debug issues around this error, you will need to download the command-line tool `kubectl`. See [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) how to download `kubectl` for your platform. + +When you have made changes to `rancher-cluster.yml`, you will have to run `rke remove --config rancher-cluster.yml` to clean the nodes, so it cannot conflict with previous configuration errors. + +### Failed to deploy addon execute job [rke-user-includes-addons]: Failed to get job complete status + +Something is wrong in the addons definitions, you can run the following command to get the root cause in the logging of the job: + +``` +kubectl --kubeconfig kube_config_rancher-cluster.yml logs -l job-name=rke-user-addon-deploy-job -n kube-system +``` + +#### error: error converting YAML to JSON: yaml: line 9: + +The structure of the addons definition in `rancher-cluster.yml` is wrong. In the different resources specified in the addons section, there is a error in the structure of the YAML. The pointer `yaml line 9` references to the line number of the addon that is causing issues. + +Things to check +
    +
      +
    • Is each of the base64 encoded certificate string placed directly after the key, for example: `tls.crt: LS01...`, there should be no newline/space before, in between or after.
      • +
    • Is the YAML properly formatted, each indentation should be 2 spaces as shown in the template files.
      • +
    • Verify the integrity of your certificate by running this command `cat MyCertificate | base64 -d` on Linux, `cat MyCertificate | base64 -D` on Mac OS . If any error exists, the command output will tell you. +
    +
+ +#### Error from server (BadRequest): error when creating "/etc/config/rke-user-addon.yaml": Secret in version "v1" cannot be handled as a Secret + +The base64 string of one of the certificate strings is wrong. The log message will try to show you what part of the string is not recognized as valid base64. + +Things to check +
    +
      +
    • Check if the base64 string is valid by running one of the commands below:
      • + +``` +# MacOS +echo BASE64_CRT | base64 -D +# Linux +echo BASE64_CRT | base64 -d +# Windows +certutil -decode FILENAME.base64 FILENAME.verify +``` + +
    +
+ +#### The Ingress "cattle-ingress-http" is invalid: spec.rules[0].host: Invalid value: "IP": must be a DNS name, not an IP address + +The host value can only contain a host name, as it is needed by the ingress controller to match the hostname and pass to the correct backend. From c5f85d5c43e772790e4dd8e10e63e18ee383576c Mon Sep 17 00:00:00 2001 From: catherineluse Date: Tue, 13 Oct 2020 11:26:38 -0700 Subject: [PATCH 10/10] Remove placeholder section for Rancher v2.5 hardening guide --- content/rancher/v2.x/en/security/rancher-2.5/_index.md | 6 ------ .../v2.x/en/security/rancher-2.5/benchmark-2.5/_index.md | 4 ---- .../v2.x/en/security/rancher-2.5/hardening-2.5/_index.md | 4 ---- 3 files changed, 14 deletions(-) delete mode 100644 content/rancher/v2.x/en/security/rancher-2.5/_index.md delete mode 100644 content/rancher/v2.x/en/security/rancher-2.5/benchmark-2.5/_index.md delete mode 100644 content/rancher/v2.x/en/security/rancher-2.5/hardening-2.5/_index.md diff --git a/content/rancher/v2.x/en/security/rancher-2.5/_index.md b/content/rancher/v2.x/en/security/rancher-2.5/_index.md deleted file mode 100644 index 02736328b26..00000000000 --- a/content/rancher/v2.x/en/security/rancher-2.5/_index.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -title: Rancher v2.5 -weight: 1 ---- - -This section contains the hardening and self-assessment guides for Rancher v2.5. \ No newline at end of file diff --git a/content/rancher/v2.x/en/security/rancher-2.5/benchmark-2.5/_index.md b/content/rancher/v2.x/en/security/rancher-2.5/benchmark-2.5/_index.md deleted file mode 100644 index 4631d4c984f..00000000000 --- a/content/rancher/v2.x/en/security/rancher-2.5/benchmark-2.5/_index.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -title: Self Assessment Guide -weight: 2 ---- \ No newline at end of file diff --git a/content/rancher/v2.x/en/security/rancher-2.5/hardening-2.5/_index.md b/content/rancher/v2.x/en/security/rancher-2.5/hardening-2.5/_index.md deleted file mode 100644 index cfb41e6e16a..00000000000 --- a/content/rancher/v2.x/en/security/rancher-2.5/hardening-2.5/_index.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -title: Hardening Guide -weight: 1 ---- \ No newline at end of file