From 444af5654ae41e90506bb5c75a3f4b02a304e08a Mon Sep 17 00:00:00 2001 From: galal-hussein Date: Wed, 24 Feb 2021 22:45:31 +0200 Subject: [PATCH] Add installation method for rke2 --- .../resources/k8s-tutorials/ha-RKE2/_index.md | 185 ++++++++++++++++++ .../infra-for-rke2-ha/_index.md | 54 +++++ 2 files changed, 239 insertions(+) create mode 100644 content/rancher/v2.x/en/installation/resources/k8s-tutorials/ha-RKE2/_index.md create mode 100644 content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-rke2-ha/_index.md diff --git a/content/rancher/v2.x/en/installation/resources/k8s-tutorials/ha-RKE2/_index.md b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/ha-RKE2/_index.md new file mode 100644 index 00000000000..8c962ba6a00 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/ha-RKE2/_index.md @@ -0,0 +1,185 @@ +--- +title: Setting up a High-availability RKE2 Kubernetes Cluster for Rancher +shortTitle: Set up RKE2 for Rancher +weight: 2 +--- + +This section describes how to install a Kubernetes cluster according to the [best practices for the Rancher server environment.]({{}}/rancher/v2.x/en/overview/architecture-recommendations/#environment-for-kubernetes-installations) + +For systems without direct internet access, refer to the air gap installation instructions. + +> **Single-node Installation Tip:** +> In a single-node Kubernetes cluster, the Rancher server does not have high availability, which is important for running Rancher in production. However, installing Rancher on a single-node cluster can be useful if you want to save resources by using a single node in the short term, while preserving a high-availability migration path. +> +> To set up a single-node RKE2 cluster, run the Rancher server installation command on just one node instead of two nodes. +> +> In both single-node setups, Rancher can be installed with Helm on the Kubernetes cluster in the same way that it would be installed on any other cluster. + +# Prerequisites + +These instructions assume you have set up three nodes, a load balancer, a DNS record, and an external MySQL database as described in [this section.]({{}}/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-ha-with-external-db/) + +Note that in order for rke2 to work correctly with the load balancer you need to set up two listeners one for the supervisor port on 9345 and one for kubernetes API port 6443. + +Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/) To specify the RKE2 version, use the INSTALL_RKE2_VERSION environment variable when running the RKE2 installation script. +# Installing Kubernetes + +### 1. Install Kubernetes and Set up the RKE2 Server + +RKE2 server runs with embedded etcd so you will not need to set up an external datastore to run in HA mode. + +1. On the first node, you should set up the configuration file with your own pre-shared secret as the token, set the token argument on startup. + +If you do not specify a pre-shared secret, RKE2 will generate one and place it at /var/lib/rancher/rke2/server/node-token. + +To avoid certificate errors with the fixed registration address, you should launch the server with the tls-san parameter set. This option adds an additional hostname or IP as a Subject Alternative Name in the server's TLS cert, and it can be specified as a list if you would like to access via both the IP and the hostname. + +Here is an example of what the RKE2 config file (at /etc/rancher/rke2/config.yaml) would look like if you are following this guide: + +``` +token: my-shared-secret +tls-san: + - my-kubernetes-domain.com + - another-kubernetes-domain.com +``` +After that you need to run installer command and enable and start rke2: +``` +curl -sfL https://get.rke2.io | sh - +systemctl enable rke2-server.service +systemctl start rke2-server.service +``` +1. To join the rest of the nodes you need to configure them with the same shared token or the one generated automatically, here is an example of the configuration: +``` +token: my-shared-secret +server: https://:9345 +tls-san: + - my-kubernetes-domain.com + - another-kubernetes-domain.com +``` +After that you need to run the installer and enable then start rke2 +``` +curl -sfL https://get.rke2.io | sh - +systemctl enable rke2-server.service +systemctl start rke2-server.service +``` + +1. Repeat the same command on your third RKE2 server node. + +### 2. Confirm that RKE2 is Running + +Once you've launched the rke2 server process on all server nodes, ensure that the cluster has come up properly with + +``` +/var/lib/rancher/rke2/bin/kubectl \ + --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes +You should see your server nodes in the Ready state. +``` + +Then test the health of the cluster pods: +``` +/var/lib/rancher/rke2/bin/kubectl \ + --kubeconfig /etc/rancher/rke2/rke2.yaml get pods --all-namespaces +``` + +**Result:** You have successfully set up a RKE2 Kubernetes cluster. + +### 3. Save and Start Using the kubeconfig File + +When you installed RKE2 on each Rancher server node, a `kubeconfig` file was created on the node at `/etc/rancher/rke2/rke2.yaml`. This file contains credentials for full access to the cluster, and you should save this file in a secure location. + +To use this `kubeconfig` file, + +1. Install [kubectl,](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) a Kubernetes command-line tool. +2. Copy the file at `/etc/rancher/rke2/rke2.yaml` and save it to the directory `~/.kube/config` on your local machine. +3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your load balancer, referring to port 6443. (The Kubernetes API server will be reached at port 6443, while the Rancher server will be reached at ports 80 and 443.) Here is an example `rke2.yaml`: + +```yml +apiVersion: v1 +clusters: +- cluster: + certificate-authority-data: [CERTIFICATE-DATA] + server: [LOAD-BALANCER-DNS]:6443 # Edit this line + name: default +contexts: +- context: + cluster: default + user: default + name: default +current-context: default +kind: Config +preferences: {} +users: +- name: default + user: + password: [PASSWORD] + username: admin +``` + +**Result:** You can now use `kubectl` to manage your RKE2 cluster. If you have more than one kubeconfig file, you can specify which one you want to use by passing in the path to the file when using `kubectl`: + +``` +kubectl --kubeconfig ~/.kube/config/krke23s.yaml get pods --all-namespaces +``` + +For more information about the `kubeconfig` file, refer to the [K3S documentation]({{}}/k3s/latest/en/cluster-access/) or the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) about organizing cluster access using `kubeconfig` files. + +### 4. Check the Health of Your Cluster Pods + +Now that you have set up the `kubeconfig` file, you can use `kubectl` to access the cluster from your local machine. + +Check that all the required pods and containers are healthy are ready to continue: + +``` + /var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get pods -A +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system etcd-ip-172-31-18-145 1/1 Running 0 4m37s +kube-system etcd-ip-172-31-25-73 1/1 Running 0 20m +kube-system etcd-ip-172-31-31-210 1/1 Running 0 9m12s +kube-system helm-install-rke2-canal-th9k9 0/1 Completed 0 21m +kube-system helm-install-rke2-coredns-6njr6 0/1 Completed 0 21m +kube-system helm-install-rke2-ingress-nginx-vztsd 0/1 Completed 0 21m +kube-system helm-install-rke2-kube-proxy-6std5 0/1 Completed 0 21m +kube-system helm-install-rke2-metrics-server-9sl7m 0/1 Completed 0 21m +kube-system kube-apiserver-ip-172-31-18-145 1/1 Running 0 4m22s +kube-system kube-apiserver-ip-172-31-25-73 1/1 Running 0 20m +kube-system kube-apiserver-ip-172-31-31-210 1/1 Running 0 9m8s +kube-system kube-controller-manager-ip-172-31-18-145 1/1 Running 0 4m8s +kube-system kube-controller-manager-ip-172-31-25-73 1/1 Running 0 21m +kube-system kube-controller-manager-ip-172-31-31-210 1/1 Running 0 8m55s +kube-system kube-proxy-57twm 1/1 Running 0 10m +kube-system kube-proxy-f7pc6 1/1 Running 0 5m24s +kube-system kube-proxy-rj4t5 1/1 Running 0 21m +kube-system kube-scheduler-ip-172-31-18-145 1/1 Running 0 4m15s +kube-system kube-scheduler-ip-172-31-25-73 1/1 Running 0 21m +kube-system kube-scheduler-ip-172-31-31-210 1/1 Running 0 8m48s +kube-system rke2-canal-4x972 2/2 Running 0 10m +kube-system rke2-canal-flh8m 2/2 Running 0 5m24s +kube-system rke2-canal-zfhkr 2/2 Running 0 21m +kube-system rke2-coredns-rke2-coredns-6cd96645d6-cmstq 1/1 Running 0 21m +kube-system rke2-ingress-nginx-controller-54946dd48f-6mp76 1/1 Running 0 20m +kube-system rke2-ingress-nginx-default-backend-5795954f8-p92xx 1/1 Running 0 20m +kube-system rke2-metrics-server-5f9b5757dc-k5sgh 1/1 Running 0 20m +``` + +**Result:** You have confirmed that you can access the cluster with `kubectl` and the RKE2 cluster is running successfully. Now the Rancher management server can be installed on the cluster. + +### 4. Configure nginx to be a daemonset + +Currently RKE2 deploys nginx-ingress as a deployment and thats can impact the rancher deployment where you can not use all servers to proxy reqeust to rancher pods, to rectify that you should drop the following file to /var/lib/rancher/rke2/server/manifests on any of the server nodes + +``` +apiVersion: helm.cattle.io/v1 +kind: HelmChartConfig +metadata: + name: rke2-ingress-nginx + namespace: kube-system +spec: + valuesContent: |- + controller: + kind: DaemonSet + daemonset: + useHostPort: true + image: + repository: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller + tag: "v0.34.1" +``` \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-rke2-ha/_index.md b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-rke2-ha/_index.md new file mode 100644 index 00000000000..5ecf2b93972 --- /dev/null +++ b/content/rancher/v2.x/en/installation/resources/k8s-tutorials/infrastructure-tutorials/infra-for-rke2-ha/_index.md @@ -0,0 +1,54 @@ +--- +title: 'Set up Infrastructure for a High Availability RKE2 Kubernetes Cluster' +weight: 1 +--- + +This tutorial is intended to help you provision the underlying infrastructure for a Rancher management server. + +The recommended infrastructure for the Rancher-only Kubernetes cluster differs depending on whether Rancher will be installed on a RKE2 Kubernetes cluster, an RKE Kubernetes cluster, or a single Docker container. + +For more information about each installation option, refer to [this page.]({{}}/rancher/v2.x/en/installation) + +> **Note:** These nodes must be in the same region. You may place these servers in separate availability zones (datacenter). + +To install the Rancher management server on a high-availability RKE2 cluster, we recommend setting up the following infrastructure: + +- **Three Linux nodes,** typically virtual machines, in the infrastructure provider of your choice. +- **A load balancer** to direct traffic to the two nodes. +- **A DNS record** to map a URL to the load balancer. This will become the Rancher server URL, and downstream Kubernetes clusters will need to reach it. + +### 1. Set up Linux Nodes + +Make sure that your nodes fulfill the general installation requirements for [OS, container runtime, hardware, and networking.]({{}}/rancher/v2.x/en/installation/requirements/) + +For an example of one way to set up Linux nodes, refer to this [tutorial]({{}}/rancher/v2.x/en/installation/options/ec2-node) for setting up nodes as instances in Amazon EC2. + +### 2. Set up the Load Balancer + +You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server. + +When Kubernetes gets set up in a later step, the RKE2 tool will deploy a Nginx Ingress controller. This controller will listen on ports 80 and 443 of the worker nodes, answering traffic destined for specific hostnames. + +When Rancher is installed (also in a later step), the Rancher system creates an Ingress resource. That Ingress tells the Nginx Ingress controller to listen for traffic destined for the Rancher hostname. The Nginx Ingress controller, when receiving traffic destined for the Rancher hostname, will forward that traffic to the running Rancher pods in the cluster. + +For your implementation, consider if you want or need to use a Layer-4 or Layer-7 load balancer: + +- **A layer-4 load balancer** is the simpler of the two choices, in which you are forwarding TCP traffic to your nodes. We recommend configuring your load balancer as a Layer 4 balancer, forwarding traffic to ports TCP/80 and TCP/443 to the Rancher management cluster nodes. The Ingress controller on the cluster will redirect HTTP traffic to HTTPS and terminate SSL/TLS on port TCP/443. The Ingress controller will forward traffic to port TCP/80 to the Ingress pod in the Rancher deployment. +- **A layer-7 load balancer** is a bit more complicated but can offer features that you may want. For instance, a layer-7 load balancer is capable of handling TLS termination at the load balancer, as opposed to Rancher doing TLS termination itself. This can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with. If you decide to terminate the SSL/TLS traffic on a layer-7 load balancer, you will need to use the `--set tls=external` option when installing Rancher in a later step. For more information, refer to the [Rancher Helm chart options.]({{}}/rancher/v2.x/en/installation/options/chart-options/#external-tls-termination) + +For an example showing how to set up an NGINX load balancer, refer to [this page.]({{}}/rancher/v2.x/en/installation/options/nginx/) + +For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to [this page.]({{}}/rancher/v2.x/en/installation/options/nlb/) + +> **Important:** +> Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications. + +### 4. Set up the DNS Record + +Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer. + +Depending on your environment, this may be an A record pointing to the load balancer IP, or it may be a CNAME pointing to the load balancer hostname. In either case, make sure this record is the hostname that you intend Rancher to respond on. + +You will need to specify this hostname in a later step when you install Rancher, and it is not possible to change it later. Make sure that your decision is a final one. + +For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)