public/rancher-docs
1
0
Fork 0
mirror of https://github.com/rancher/rancher-docs.git synced 2026-09-25 20:48:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Reorganize node requirements (#1920)

Browse Source 
* Reorganize node requirements

* Cherry pick 'Fix links' commit

* Cherry pick 'Link to RKE port requirements for each node in Rancher docs'

* Fix merge conflict errors

* Delete page that was moved elsewhere

* Delete section that was included elsewhere

* Re-add port requirements for custom nodes

* Rearrange pages

* Change wording of note in Rancher installation requirements

* Fix links

* Edit links

* Edit links

* Replace user cluster requirements page

* Fix typo

* Fix link to imported cluster requirements

* Revise node requirements

* Add mini TOC for port requirements

* Remove alias for port requirements page
This commit is contained in:
Catherine Luse
2019-11-05 11:23:09 -08:00
committed by Denise
parent 45d128bedb
commit 65e52ec86b
14 changed files with 406 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/rancher/v2.x/en/cluster-provisioning/node-requirements/_index.md
+160
View File
@@ -0,0 +1,160 @@
---
title: Node Requirements for User Clusters
weight: 1
---
This page describes the requirements for the nodes where your apps and services will be installed.
In this section, "user cluster" refers to a cluster running your apps, which should be separate from the cluster (or single node) running Rancher.
> It is important to note that if Rancher is installed on a high-availability Kubernetes cluster, the Rancher server cluster and user clusters have the same requirements for OS and Docker, but other requirements are different. For Rancher installation requirements, refer to the node requirements in the [installation section.]({{<baseurl>}}/rancher/v2.x/en/installation/requirements/)
Make sure the nodes for the Rancher server fulfill the following requirements:
- Operating systems and Docker requirements - same as the [requirements for Rancher installation]({{<baseurl>}}/rancher/v2.x/en/installation/requirements/#operating-systems-and-docker-requirements)
- [Hardware Requirements](#hardware-requirements)
- [Networking Requirements](#networking-requirements)
# Operating Systems and Docker Requirements
For the nodes in user clusters, the requirements for the operating system and Docker version are the same as the [OS and Docker requirements for the Rancher server cluster.]({{<baseurl>}}/rancher/v2.x/en/installation/requirements/#operating-systems-and-docker-requirements)
# Hardware Requirements
The hardware requirements for nodes with the `worker` role mostly depend on your workloads. The minimum to run the Kubernetes node components is 1 CPU (core) and 1GB of memory.
Regarding CPU and memory, it is recommended that the different planes of Kubernetes clusters (etcd, controplane, and workers) should be hosted on different nodes so that they can scale separately from each other.
For hardware recommendations for large Kubernetes clusters, refer to the official Kubernetes documentation on [building large clusters.](https://kubernetes.io/docs/setup/best-practices/cluster-large/)
For hardware recommendations for etcd clusters in production, refer to the official [etcd documentation.](https://etcd.io/docs/v3.4.0/op-guide/hardware/)
# Networking Requirements
For a production cluster, we recommend that you restrict traffic by opening only the ports defined in the port requirements below.
The ports required to be open are different depending on how the user cluster is launched. Each of the sections below list the ports that need to be opened for different [cluster creation options]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-options).
For a breakdown of the port requirements for etcd nodes, controlplane nodes, and worker nodes in a Kubernetes cluster, refer to the [port requirements for the Rancher Kubernetes Engine.]({{<baseurl>}}/rke/latest/en/os/#ports)
Details on which ports are used in each situation are found in the following sections:
- [Commonly used ports](#commonly-used-ports)
- [Port requirements for custom clusters](#port-requirements-for-custom-clusters)
- [Port requirements for clusters hosted by an infrastructure provider](#port-requirements-for-clusters-hosted-by-an-infrastructure-provider)
- [Security group for nodes on AWS EC2](#security-group-for-nodes-on-aws-ec2)
- [Port requirements for clusters hosted by a Kubernetes provider](#port-requirements-for-clusters-hosted-by-a-kubernetes-provider)
- [Port requirements for imported clusters](#port-requirements-for-imported-clusters)
- [Port requirements for local traffic](#port-requirements-for-local-traffic)
### Commonly Used Ports
If security isn't a large concern and you're okay with opening a few additional ports, you can use this table as your port reference instead of the comprehensive tables in the following sections.
These ports are typically opened on your Kubernetes nodes, regardless of what type of cluster it is.
{{% accordion id="common-ports" label="Click to Expand" %}}
<figcaption>Commonly Used Ports Reference</figcaption>
| Protocol | Port | Description |
|:--------: |:----------------: |------------------------------------------------- |
| TCP | 22 | Node driver SSH provisioning |
| TCP | 2376 | Node driver Docker daemon TLS port |
| TCP | 2379 | etcd client requests |
| TCP | 2380 | etcd peer communication |
| UDP | 8472 | Canal/Flannel VXLAN overlay networking |
| UDP | 4789 | Flannel VXLAN overlay networking on Windows cluster |
| TCP | 9099 | Canal/Flannel livenessProbe/readinessProbe |
| TCP | 6783 | Weave Port |
| UDP | 6783-6784 | Weave UDP Ports |
| TCP | 10250 | kubelet API |
| TCP | 10254 | Ingress controller livenessProbe/readinessProbe |
| TCP/UDP | 30000-</br>32767 | NodePort port range |
{{% /accordion %}}
### Port Requirements for Custom Clusters
If you are launching a Kubernetes cluster on your existing infrastructure, refer to these port requirements.
{{% accordion id="port-reqs-for-custom-clusters" label="Click to Expand" %}}
The following table depicts the port requirements for [Rancher Launched Kubernetes]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) with [custom nodes]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/).
{{< ports-custom-nodes >}}
{{% /accordion %}}
### Port Requirements for Clusters Hosted by an Infrastructure Provider
If you are launching a Kubernetes cluster on nodes that are in an infastructure provider such as Amazon EC2, Google Container Engine, DigitalOcean, Azure, or vSphere, [these port requirements]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/port-reqs-for-infrastructure-provider) apply.
These required ports are automatically opened by Rancher during creation of clusters using cloud providers.
{{% accordion id="port-reqs-for-infrastructure-providers" label="Click to Expand" %}}
The following table depicts the port requirements for [Rancher Launched Kubernetes]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) with nodes created in an [Infrastructure Provider]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/).
>**Note:**
>The required ports are automatically opened by Rancher during creation of clusters in cloud providers like Amazon EC2 or DigitalOcean.
{{< ports-iaas-nodes >}}
{{% /accordion %}}
#### Security Group for Nodes on AWS EC2
When using the [AWS EC2 node driver]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/ec2/) to provision cluster nodes in Rancher, you can choose to let Rancher create a security group called `rancher-nodes`. The following rules are automatically added to this security group.
| Type | Protocol | Port Range | Source/Destination | Rule Type |
|-----------------|:--------:|:-----------:|------------------------|:---------:|
| SSH | TCP | 22 | 0.0.0.0/0 | Inbound |
| HTTP | TCP | 80 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 443 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 2376 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 2379-2380 | sg-xxx (rancher-nodes) | Inbound |
| Custom UDP Rule | UDP | 4789 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 6443 | 0.0.0.0/0 | Inbound |
| Custom UDP Rule | UDP | 8472 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 10250-10252 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 10256 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 30000-32767 | 0.0.0.0/0 | Inbound |
| Custom UDP Rule | UDP | 30000-32767 | 0.0.0.0/0 | Inbound |
| All traffic | All | All | 0.0.0.0/0 | Outbound |
### Port Requirements for Clusters Hosted by a Kubernetes Provider
If you are launching a cluster with a hosted Kubernetes provider such as Google Kubernetes Engine, Amazon EKS, or Azure Kubernetes Service, refer to these port requirements.
{{% accordion id="port-reqs-for-hosted-kubernetes" label="Click to Expand" %}}
The following table depicts the port requirements for nodes in [hosted Kubernetes clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters).
{{< ports-imported-hosted >}}
{{% /accordion %}}
### Port Requirements for Imported Clusters
If you are importing an existing cluster, refer to these port requirements.
{{% accordion id="port-reqs-for-imported-clusters" label="Click to Expand" %}}
The following table depicts the port requirements for [imported clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/imported-clusters/).
{{< ports-imported-hosted >}}
{{% /accordion %}}
### Port Requirements for Local Traffic
Ports marked as `local traffic` (i.e., `9099 TCP`) in the port requirements are used for Kubernetes healthchecks (`livenessProbe` and`readinessProbe`).
These healthchecks are executed on the node itself. In most cloud environments, this local traffic is allowed by default.
However, this traffic may be blocked when:
- You have applied strict host firewall policies on the node.
- You are using nodes that have multiple interfaces (multihomed).
In these cases, you have to explicitly allow this traffic in your host firewall, or in case of public/private cloud hosted machines (i.e. AWS or OpenStack), in your security group configuration. Keep in mind that when using a security group as source or destination in your security group, explicitly opening ports only applies to the private interface of the nodes/instances.
content/rancher/v2.x/en/cluster-provisioning/production/_index.md
+23 -111
View File
@@ -1,137 +1,49 @@
---
title: Production Ready Cluster
title: Checklist for Production-Ready Clusters
weight: 2005
---
While Rancher makes it easy to create Kubernetes clusters, a production ready cluster takes more consideration and planning. There are three roles that can be assigned to nodes: `etcd`, `controlplane` and `worker`. In the next sections each of the roles will be described in more detail.
In this section, we recommend best practices for creating the production-ready Kubernetes clusters that will run your apps and services.
When designing your cluster(s), you have two options:
For a list of requirements for your cluster, including the requirements for OS/Docker, hardware, and networking, refer to the section on [node requirements.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements)
* Use dedicated nodes for each role. This ensures resource availability for the components needed for the specified role. It also strictly isolates network traffic between each of the roles according to the [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/).
* Assign the `etcd` and `controlplane` roles to the same nodes. These nodes must meet the hardware requirements for both roles.
This is a shortlist of best practices that we strongly recommend for all production clusters.
>**Note:** Do not add the `worker` role to any node configured with either the `etcd` or `controlplane` role. This will make the nodes schedulable for regular workloads, which could interfere with critical cluster components running on the nodes with the `etcd` or `controlplane` role.
For a full list of all the best practices that we recommend, refer to the [best practices section.]({{<baseurl>}}/rancher/v2.x/en/best-practices)
## etcd
### Node Requirements
Nodes with the `etcd` role run etcd, which is a consistent and highly available key value store used as Kubernetes’ backing store for all cluster data. etcd replicates the data to each node.
* Make sure your nodes fulfill all of the [node requirements,]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/) including the port requirements.
>**Note:** Nodes with the `etcd` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default.
### Back up etcd
### Hardware Requirements
* Enable etcd snapshots. Verify that snapshots are being created, and run a disaster recovery scenario to verify the snapshots are valid. etcd is the location where the state of your cluster is stored, and losing etcd data means losing your cluster. Make sure you configure [etcd Recurring Snapshots]({{<baseurl>}}/rancher/v2.x/en/backups/backups/ha-backups/#option-a-recurring-snapshots) for your cluster(s), and make sure the snapshots are stored externally (off the node) as well.
Please see [Kubernetes: Building Large Clusters](https://kubernetes.io/docs/setup/cluster-large/) and [etcd: Hardware Recommendations](https://coreos.com/etcd/docs/latest/op-guide/hardware.html) for the hardware requirements.
### Count of etcd Nodes
The number of nodes that you can lose at once while maintaining cluster availability is determined by the number of nodes assigned the `etcd` role. For a cluster with n members, the minimum is (n/2)+1. Therefore, we recommend creating an `etcd` node in 3 different availability zones to survive the loss of one availability zone within a region. If you use only two zones, you can only survive the loss of the zone where you don't lose the majority of nodes.
| Nodes with `etcd` role | Majority | Failure Tolerance |
|--------------|------------|-------------------|
| 1 | 1 | 0 |
| 2 | 2 | 0 |
| 3 | 2 | **1** |
| 4 | 3 | 1 |
| 5 | 3 | **2** |
| 6 | 4 | 2 |
| 7 | 4 | **3** |
| 8 | 5 | 3 |
| 9 | 5 | **4** |
References:
* [etcd cluster size](https://coreos.com/etcd/docs/latest/v2/admin_guide.html#optimal-cluster-size)
* [Operating etcd clusters for Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/)
### Network Latency
Rancher recommends minimizing latency between the etcd nodes. The default setting for `heartbeat-interval` is `500`, and the default setting for `election-timeout` is `5000`. These settings allow etcd to run in most networks (except really high latency networks).
References:
* [etcd Tuning](https://coreos.com/etcd/docs/latest/tuning.html)
### Backups
etcd is the location where the state of your cluster is stored. Losing etcd data means losing your cluster. Make sure you configure [etcd Recurring Snapshots]({{< baseurl >}}/rancher/v2.x/en/backups/backups/ha-backups/#option-a-recurring-snapshots) for your cluster(s), and make sure the snapshots are stored externally (off the node) as well.
## controlplane
Nodes with the `controlplane` role run the Kubernetes master components (excluding `etcd`, as it's a separate role). See [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components) for a detailed list of components.
>**Note:** Nodes with the `controlplane` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default.
References:
* [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components)
### Hardware Requirements
Please see [Kubernetes: Building Large Clusters](https://kubernetes.io/docs/setup/cluster-large/) for the hardware requirements.
### Count of controlplane Nodes
Adding more than one node with the `controlplane` role makes every master component highly available. See below for a breakdown of how high availability is achieved per component.
#### kube-apiserver
The Kubernetes API server (`kube-apiserver`) scales horizontally. Each node with the role `controlplane` will be added to the NGINX proxy on the nodes with components that need to access the Kubernetes API server. This means that if a node becomes unreachable, the local NGINX proxy on the node will forward the request to another Kubernetes API server in the list.
#### kube-controller-manager
The Kubernetes controller manager uses leader election using an endpoint in Kubernetes. One instance of the `kube-controller-manager` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive).
#### kube-scheduler
The Kubernetes scheduler uses leader election using an endpoint in Kubernetes. One instance of the `kube-scheduler` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive).
## worker
Nodes with the `worker` role run the Kubernetes node components. See [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components) for a detailed list of components.
References:
* [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components)
### Hardware Requirements
The hardware requirements for nodes with the `worker` role mostly depend on your workloads. The minimum to run the Kubernetes node components is 1 CPU (core) and 1GB of memory.
### Count of worker Nodes
Adding more than one node with the `worker` role will make sure your workloads can be rescheduled if a node fails.
## Networking
Cluster nodes should be located within a single region. Most cloud providers provide multiple availability zones within a region, which can be used to create higher availability for your cluster. Using multiple availability zones is fine for nodes with any role. If you are using [Kubernetes Cloud Provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers/) resources, consult the documentation for any restrictions (i.e. zone storage restrictions).
## Cluster Diagram
This diagram is applicable to Kubernetes clusters built using RKE or [Rancher Launched Kubernetes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
![Cluster diagram]({{< baseurl >}}/img/rancher/clusterdiagram.svg)<br/>
<sup>Lines show the traffic flow between components. Colors are used purely for visual aid</sup>
## Production checklist
### Cluster Architecture
* Nodes should have one of the following role configurations:
* `etcd`
* `controlplane`
* `etcd` and `controlplane`
* `worker` (the `worker` role should not be used or added on nodes with the `etcd` or `controlplane` role)
* Network traffic is only strictly allowed according to [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/).
* Have at least three nodes with the role `etcd` to survive losing one node. Increase this count for higher node fault toleration, and spread them across (availability) zones to provide even better fault tolerance.
* Assign two or more nodes the `controlplane` role for master component high availability.
* Assign two or more nodes the `worker` role for workload rescheduling upon node failure.
* Enable etcd snapshots. Verify that snapshots are being created, and run a disaster recovery scenario to verify the snapshots are valid.
* Perform load tests on your cluster to verify that its hardware can support your workloads.
For more information on what each role is used for, refer to the [section on roles for nodes in Kubernetes.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/production/nodes-and-roles)
For more information about the recommended number of nodes for each Kubernetes role, refer to the [section on recommended architecture.]({{<baseurl>}}/rancher/v2.x/en/cluster/provisioning/recommended-architecture)
### Logging and Monitoring
* Configure alerts/notifiers for Kubernetes components (System Service).
* Configure logging for cluster analysis and post-mortems.
## RKE cluster running Rancher HA
### Reliability
You may have noticed that our [High Availability (HA) Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha/) instructions do not meet our definition of a production-ready cluster, as there are no dedicated nodes for the `worker` role. However, for your Rancher installation, this three node cluster is valid, as:
* Perform load tests on your cluster to verify that its hardware can support your workloads.
* It allows one `etcd` node failure.
* It maintains multiple instances of the master components by having multiple `controlplane` nodes.
* No other workloads than Rancher itself should be created on this cluster.
### Networking
* Minimize network latency. Rancher recommends minimizing latency between the etcd nodes. The default setting for `heartbeat-interval` is `500`, and the default setting for `election-timeout` is `5000`. These [settings for etcd tuning](https://coreos.com/etcd/docs/latest/tuning.html) allow etcd to run in most networks (except really high latency networks).
* Cluster nodes should be located within a single region. Most cloud providers provide multiple availability zones within a region, which can be used to create higher availability for your cluster. Using multiple availability zones is fine for nodes with any role. If you are using [Kubernetes Cloud Provider]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers/) resources, consult the documentation for any restrictions (i.e. zone storage restrictions).
content/rancher/v2.x/en/cluster-provisioning/production/nodes-and-roles/_index.md
+43
View File
@@ -0,0 +1,43 @@
---
title: Roles for Nodes in Kubernetes
weight: 1
---
This section describes the roles for etcd nodes, controlplane nodes, and worker nodes in Kubernetes, and how the roles work together in a cluster.
This diagram is applicable to Kubernetes clusters built using RKE or [Rancher Launched Kubernetes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/).
![Cluster diagram]({{< baseurl >}}/img/rancher/clusterdiagram.svg)<br/>
<sup>Lines show the traffic flow between components. Colors are used purely for visual aid</sup>
# etcd
Nodes with the `etcd` role run etcd, which is a consistent and highly available key value store used as Kubernetes’ backing store for all cluster data. etcd replicates the data to each node.
>**Note:** Nodes with the `etcd` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default.
# controlplane
Nodes with the `controlplane` role run the Kubernetes master components (excluding `etcd`, as it's a separate role). See [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components) for a detailed list of components.
>**Note:** Nodes with the `controlplane` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default.
### kube-apiserver
The Kubernetes API server (`kube-apiserver`) scales horizontally. Each node with the role `controlplane` will be added to the NGINX proxy on the nodes with components that need to access the Kubernetes API server. This means that if a node becomes unreachable, the local NGINX proxy on the node will forward the request to another Kubernetes API server in the list.
### kube-controller-manager
The Kubernetes controller manager uses leader election using an endpoint in Kubernetes. One instance of the `kube-controller-manager` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive).
### kube-scheduler
The Kubernetes scheduler uses leader election using an endpoint in Kubernetes. One instance of the `kube-scheduler` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive).
# worker
Nodes with the `worker` role run the Kubernetes node components. See [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components) for a detailed list of components.
# References
* [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components)
content/rancher/v2.x/en/cluster-provisioning/production/recommended-architecture/_index.md
+74
View File
@@ -0,0 +1,74 @@
---
title: Recommended Cluster Architecture
weight: 1
---
There are three roles that can be assigned to nodes: `etcd`, `controlplane` and `worker`.
# Separating Worker Nodes from Nodes with Other Roles
When designing your cluster(s), you have two options:
* Use dedicated nodes for each role. This ensures resource availability for the components needed for the specified role. It also strictly isolates network traffic between each of the roles according to the [port requirements]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/).
* Assign the `etcd` and `controlplane` roles to the same nodes. These nodes must meet the hardware requirements for both roles.
In either case, the `worker` role should not be used or added to nodes with the `etcd` or `controlplane` role.
Therefore, each node should have one of the following role configurations:
* `etcd`
* `controlplane`
* Both `etcd` and `controlplane`
* `worker`
# Recommended Number of Nodes with Each Role
The cluster should have:
- At least three nodes with the role `etcd` to survive losing one node. Increase this count for higher node fault toleration, and spread them across (availability) zones to provide even better fault tolerance.
- At least two nodes with the role `controlplane` for master component high availability.
- At least two nodes with the role `worker` for workload rescheduling upon node failure.
For more information on what each role is used for, refer to the [section on roles for nodes in Kubernetes.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/production/nodes-and-roles)
### Number of Controlplane Nodes
Adding more than one node with the `controlplane` role makes every master component highly available.
### Number of etcd Nodes
The number of nodes that you can lose at once while maintaining cluster availability is determined by the number of nodes assigned the `etcd` role. For a cluster with n members, the minimum is (n/2)+1. Therefore, we recommend creating an `etcd` node in 3 different availability zones within a region to survive the loss of one availability zone. If you use only two zones, you can only survive the loss of the zone where you don't lose the majority of nodes.
| Nodes with `etcd` role | Majority | Failure Tolerance |
|--------------|------------|-------------------|
| 1 | 1 | 0 |
| 2 | 2 | 0 |
| 3 | 2 | **1** |
| 4 | 3 | 1 |
| 5 | 3 | **2** |
| 6 | 4 | 2 |
| 7 | 4 | **3** |
| 8 | 5 | 3 |
| 9 | 5 | **4** |
References:
* [Official etcd documentation on optimal etcd cluster size](https://github.com/bmizerany/etcd-team/blob/master/Documentation/optimal-cluster-size.md)
* [Official Kubernetes documentation on operating etcd clusters for Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/)
### Number of Worker Nodes
Adding more than one node with the `worker` role will make sure your workloads can be rescheduled if a node fails.
### Why Production Requirements are Different for the Rancher Cluster and the Clusters Running Your Applications
You may have noticed that our [High Availability (HA) Install]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) instructions do not meet our definition of a production-ready cluster, as there are no dedicated nodes for the `worker` role. However, for your Rancher installation, this three node cluster is valid, because:
* It allows one `etcd` node failure.
* It maintains multiple instances of the master components by having multiple `controlplane` nodes.
* No other workloads than Rancher itself should be created on this cluster.
# References
* [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components)
content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/ec2/_index.md
+1 -1
View File
@@ -40,7 +40,7 @@ Use {{< product >}} to create a Kubernetes cluster in Amazon EC2.
{{< step_create-cloud-credential >}}
- **Zone and Network** configures the availability zone and network settings for your cluster.
- **Security Groups** creates or configures the Security Groups applied to your nodes. Please refer to [Amazon EC2 security group when using Node Driver]({{< baseurl >}}/rancher/v2.x/en/installation/references/#amazonec2-securitygroup-nodedriver) to see what rules are created in the `rancher-nodes` Security Group.
- **Security Groups** creates or configures the Security Groups applied to your nodes. Please refer to [Amazon EC2 security group when using Node Driver]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#amazonec2-securitygroup-nodedriver) to see what rules are created in the `rancher-nodes` Security Group.
- **Instance** configures the instances that will be created. Make sure you configure the correct **SSH User** for the configured AMI.
<br><br>
If you need to pass an **IAM Instance Profile Name** (not ARN), for example, when you want to use a [Kubernetes Cloud Provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers), you will need an additional permission in your policy. See [Example IAM policy with PassRole](#example-iam-policy-with-passrole) for an example policy.
content/rancher/v2.x/en/faq/networking/cni-providers/_index.md
+5 -5
View File
@@ -53,11 +53,11 @@ Canal is a CNI network provider that gives you the best of Flannel and Calico. I
In Rancher, Canal is the default CNI network provider combined with Flannel and VXLAN encapsulation.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for more details.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). For details, refer to [the port requirements for user clusters.]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/)
{{< img "/img/rancher/canal-diagram.png" "Canal Diagram">}}
For more information, see the [Canal GitHub Page](https://github.com/projectcalico/canal).
For more information, see the [Canal GitHub Page.](https://github.com/projectcalico/canal)
#### Flannel
@@ -67,7 +67,7 @@ Flannel is a simple and easy way to configure L3 network fabric designed for Kub
Encapsulated traffic is unencrypted by default. Therefore, flannel provides an experimental backend for encryption, [IPSec](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#ipsec), which makes use of [strongSwan](https://www.strongswan.org/) to establish encrypted IPSec tunnels between Kubernetes workers.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for more details.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [the port requirements for user clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for more details.
![Flannel Diagram]({{< baseurl >}}/img/rancher/flannel-diagram.png)
@@ -81,7 +81,7 @@ Calico enables networking and network policy in Kubernetes clusters across the c
Calico also provides a stateless IP-in-IP encapsulation mode that can be used, if necessary. Calico also offers policy isolation, allowing you to secure and govern your Kubernetes workloads using advanced ingress and egress policies.
Kubernetes workers should open TCP port `179` (BGP). See [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for more details.
Kubernetes workers should open TCP port `179` (BGP). See [the port requirements for user clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for more details.
![Calico Diagram]({{< baseurl >}}/img/rancher/calico-diagram.svg)
@@ -99,7 +99,7 @@ _Available as of v2.2.0_
Weave enables networking and network policy in Kubernetes clusters across the cloud. Additionally, it support encrypting traffic between the peers.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for more details.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See the [port requirements for user clusters]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for more details.
For more information, see the following pages:
content/rancher/v2.x/en/installation/_index.md
+2 -10
View File
@@ -4,6 +4,8 @@ weight: 50
---
This section contains instructions for installing Rancher in development and production environments.
Before installing Rancher, make sure that your nodes fulfill all of the [installation requirements.]({{<baseurl>}}/rancher/v2.x/en/installation/requirements/)
### Installation Options
- [Single Node Installation]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/)
@@ -13,13 +15,3 @@ This section contains instructions for installing Rancher in development and pro
- [High Availability Installation]({{< baseurl >}}/rancher/v2.x/en/installation/ha/)
This install scenario creates a new Kubernetes cluster dedicated to running Rancher Server in a high-availability (HA) configuration, which runs Rancher Server on multiple hosts so that it's always accessible provided that one of your cluster nodes is running. We recommend high-availability installs in production environments, where your user base requires 24/7 access to your applications.
### Reference
- [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements/)
A reference of hardware and software requirements for the server(s) hosting Rancher.
- [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/)
List of required ports you must open to operate Rancher.
content/rancher/v2.x/en/installation/ha/rke-add-on/troubleshooting/generic-troubleshooting/_index.md
+2 -2
View File
@@ -15,7 +15,7 @@ Below are steps that you can follow to determine what is wrong in your cluster.
### Double check if all the required ports are opened in your (host) firewall
Double check if all the [required ports]({{< baseurl >}}/rancher/v2.x/en/installation/references/) are opened in your (host) firewall.
Double check if all the [required ports]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) are opened in your (host) firewall.
### All nodes should be present and in **Ready** state
@@ -143,7 +143,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
=> End
```
If you see error in the output, that means that the [required ports]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for overlay networking are not opened between the hosts indicated.
If you see error in the output, that means that the [required ports]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for overlay networking are not opened between the hosts indicated.
Example error output of a situation where NODE1 had the UDP ports blocked.
content/rancher/v2.x/en/installation/references/_index.md
-2
View File
@@ -1,8 +1,6 @@
---
title: Port Requirements
weight: 300
aliases:
- /rancher/v2.x/en/hosts/amazon/#required-ports-for-rancher-to-work/
---
To operate properly, Rancher requires a number of ports to be open on Rancher nodes and Kubernetes cluster nodes.
content/rancher/v2.x/en/installation/requirements/_index.md
+93 -38
View File
@@ -1,16 +1,33 @@
---
title: Node Requirements
title: Installation Requirements
weight: 1
aliases:
- /rancher/v2.x/en/hosts/amazon/#required-ports-for-rancher-to-work/
- /rancher/v2.x/en/installation/references
---
Whether you're configuring Rancher to run in a single-node or high-availability setup, each node running Rancher Server must meet the following requirements.
This page describes the software, hardware, and networking requirements for the nodes where the Rancher server will be installed. The Rancher server can be installed on a single node or a high-availability Kubernetes cluster.
> It is important to note that if you install Rancher on a Kubernetes cluster, the hardware and networking requirements for the Rancher cluster are different than the [node requirements for user clusters,]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/) which will run your apps and services.
Make sure the node(s) for the Rancher server fulfill the following requirements:
- [Operating Systems and Docker Requirements](#operating-systems-and-docker-requirements)
- [Hardware Requirements](#hardware-requirements)
- [CPU and Memory](#cpu-and-memory)
- [Disks](#disks)
- [Networking Requirements](#networking-requirements)
- [Node IP Addresses](#node-ip-addresses)
- [Port Requirements](#port-requirements)
For a list of best practices that we recommend for running the Rancher server in production, refer to the [best practices section.]({{<baseurl>}}/rancher/v2.x/en/best-practices/deployment-types/)
# Operating Systems and Docker Requirements
{{% tabs %}}
{{% tab "Operating Systems and Docker" %}}
<br>
Rancher is tested on the following operating systems and their subsequent non-major releases with a supported version of [Docker](https://www.docker.com/).
For details on which OS and Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
- Ubuntu 16.04 (64-bit x86)
- Docker 17.03.x, 18.06.x, 18.09.x
- Ubuntu 18.04 (64-bit x86)
@@ -39,18 +56,26 @@ sudo ros engine list
sudo ros engine switch docker-18.09.2
```
See [Running on ARM64 (Experimental)]({{< baseurl >}}/rancher/v2.x/en/installation/arm64-platform/) if you plan to run Rancher on ARM64.
<br>
<br>
[Docker Documentation: Installation Instructions](https://docs.docker.com/)
<br>
<br>
{{% /tab %}}
{{% tab "Hardware" %}}
<br>
Hardware requirements scale based on the size of your Rancher deployment. Provision each individual node according to the requirements.
If you plan to run Rancher on ARM64, see [Running on ARM64 (Experimental)]({{<baseurl>}}/rancher/v2.x/en/installation/arm64-platform/)
**[HA Node]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/) Requirements**
For information on how to install Docker, refer to the offical [Docker documentation.](https://docs.docker.com/)
# Hardware Requirements
This section describes the CPU, memory, and disk requirements for the nodes where the Rancher server is installed.
### CPU and Memory
Hardware requirements scale based on the size of your Rancher deployment. Provision each individual node according to the requirements. The requirements are different depending on if you are installing Rancher on a single node or on a high-availability (HA) cluster.
For production environments, the Rancher server should be installed on an HA cluster.
Rancher can also be installed on a single node in a development or testing environment.
{{% tabs %}}
{{% tab "HA Node Requirements" %}}
These requirements apply to [HA installations]({{<baseurl>}}/rancher/v2.x/en/installation/ha/) of Rancher.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | --------- | ---------- | ----------------------------------------------- | ----------------------------------------------- |
@@ -60,41 +85,71 @@ Hardware requirements scale based on the size of your Rancher deployment. Provis
| X-Large | Up to 100 | Up to 1000 | 32 | 128 GB |
| XX-Large | 100+ | 1000+ | [Contact Rancher](https://rancher.com/contact/) | [Contact Rancher](https://rancher.com/contact/) |
<br>
{{% /tab %}}
{{% tab "Single Node Requirements" %}}
**[Single Node]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/) Requirements**
These requirements apply to [single node]({{<baseurl>}}/rancher/v2.x/en/installation/single-node/) installations of Rancher.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | -------- | --------- | ----- | ---- |
| Small | Up to 5 | Up to 50 | 1 | 4 GB |
| Medium | Up to 15 | Up to 200 | 2 | 8 GB |
<br/>
**Disks**
Rancher performance depends on etcd in the cluster performance. To ensure optimal speed, we recommend always using SSD disks to back your Rancher management Kubernetes cluster. On cloud providers, you will also want to use the minimum size that allows the maximum IOPs. In larger clusters consider using dedicated storage devices for etcd data and wal directories.
<br/>
{{% /tab %}}
{{% tab "Networking" %}}
<br>
{{% /tabs %}}
### Node IP Address
### Disks
Each node used (either for the Single Node Install, High Availability (HA) Install or nodes that are used in clusters) should have a static IP configured. In case of DHCP, the nodes should have a DHCP reservation to make sure the node gets the same IP allocated.
Rancher performance depends on etcd in the cluster performance. To ensure optimal speed, we recommend always using SSD disks to back your Rancher management Kubernetes cluster. On cloud providers, you will also want to use the minimum size that allows the maximum IOPS. In larger clusters, consider using dedicated storage devices for etcd data and wal directories.
# Networking Requirements
This section describes the networking requirements for the node(s) where the Rancher server is installed.
### Node IP Addresses
Each node used should have a static IP configured, regardless of whether you are installing Rancher on a single node or on an HA cluster. In case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
### Port Requirements
When deploying Rancher in an HA cluster, certain ports on your nodes must be open to allow communication with Rancher. The ports that must be open change according to the type of machines hosting your cluster nodes. For example, if your are deploying Rancher on nodes hosted by an infrastructure, port `22` must be open for SSH. The following diagram depicts the ports that are opened for each [cluster type]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning).
This section describes the port requirements for nodes running the `rancher/rancher` container.
<figcaption>Cluster Type Port Requirements</figcaption>
The port requirements are different depending on whether you are installing Rancher on a single node or on a high-availability Kubernetes cluster. For a single node, you only need to open the [ports required to enable Rancher to communicate with user clusters.](#port-requirements-for-enabling-rancher-to-communicate-with-user-clusters) For a high-availability installation, the same ports need to be opened, as well as additional [ports required to set up the Kubernetes cluster](#additional-port-requirements-for-nodes-in-high-availability-rancher-installations) that Rancher is installed on.
![Basic Port Requirements]({{< baseurl >}}/img/rancher/port-communications.svg)
### Port Requirements for Enabling Rancher to Communicate with User Clusters
{{< requirements_ports_rancher >}}
{{< requirements_ports_rke >}}
{{< ports_aws_securitygroup_nodedriver >}}
{{% /tab %}}
{{% /tabs %}}
For a single-node installation, you only need to open the ports for the Rancher management plane. These ports are opened to allow the Rancher server to communicate with the Kubernetes clusters that will run your apps and services.
For a high-availability installation, these rules apply as well as the [port requirements to set up the Kubernetes cluster](#additional-port-requirements-for-nodes-in-high-availability-rancher-installations) that Rancher is installed on.
The port requirements are different based the infrastructure you are using. For example, if you are deploying Rancher on nodes hosted by an infrastructure provider, port `22` must be open for SSH. The following diagram depicts the ports that are opened for each [cluster type]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning).
<figcaption>Port Requirements for the Rancher Management Plane</figcaption>
![Basic Port Requirements]({{<baseurl>}}/img/rancher/port-communications.svg)
The following tables break down the port requirements for inbound and outbound traffic:
<figcaption>Inbound Rules for Rancher Nodes</figcaption>
| Protocol | Port | Source | Description |
|------------|-------|---------|----------------|
| TCP | 80 | Load balancer/proxy that does external SSL termination | Rancher UI/API when external SSL termination is used |
| TCP | 443 | <ul><li>etcd nodes</li><li>controlplane nodes</li><li>worker nodes</li><li>hosted/imported Kubernetes</li><li>any source that needs to be able to use the Rancher UI or API</li></ul> | Rancher agent, Rancher UI/API, kubectl |
<figcaption>Outbound Rules for Rancher Nodes</figcaption>
| Protocol | Port | Source | Description |
|------------|-------|---------|----------------|
| TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
| TCP | 443 | `35.160.43.145/32`, `35.167.242.46/32`, `52.33.59.17/32` | git.rancher.io (catalogs) |
| TCP | 2376 | Any node IP from a node created using Node driver | Docker daemon TLS port used by Docker Machine |
| TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |
**Note** Rancher nodes may also require additional outbound access for any external [authentication provider]({{< baseurl >}}rancher/v2.x/en/admin-settings/authentication/) which is configured (LDAP for example).
### Additional Port Requirements for Nodes in High-Availability Rancher Installations
You will need to open additional ports to the launch the Kubernetes cluster that is required for a high-availability installation of Rancher.
The ports that need to be opened for each node depend on the node's Kubernetes role: etcd, controlplane, or worker. For a breakdown of the port requirements for each role, refer to the [port requirements for the Rancher Kubernetes Engine.]({{<baseurl>}}/rke/latest/en/os/#ports)
content/rancher/v2.x/en/quick-start-guide/deployment/quickstart-manual-setup/_index.md
+1 -1
View File
@@ -36,7 +36,7 @@ This Quick Start Guide is divided into different tasks for easier consumption.
>**Note:**
> When using a cloud-hosted virtual machine you need to allow inbound TCP communication to ports 80 and 443. Please see your cloud-host's documentation for information regarding port configuration.
>
> For a full list of port requirements, refer to [Single Node Installation]({{< baseurl >}}/rancher/v2.x/en/installation/references).
> For a full list of port requirements, refer to [Single Node Installation]({{<baseurl>}}/rancher/v2.x/en/installation/node-requirements/).
Provision the host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements/).
content/rancher/v2.x/en/troubleshooting/networking/_index.md
+2 -2
View File
@@ -9,7 +9,7 @@ Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG
### Double check if all the required ports are opened in your (host) firewall
Double check if all the [required ports]({{< baseurl >}}/rancher/v2.x/en/installation/references/) are opened in your (host) firewall. The overlay network uses UDP in comparison to all other required ports which are TCP.
Double check if all the [required ports]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) are opened in your (host) firewall. The overlay network uses UDP in comparison to all other required ports which are TCP.
### Check if overlay network is functioning correctly
@@ -58,7 +58,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition
=> End network overlay test
```
If you see error in the output, that means that the [required ports]({{< baseurl >}}/rancher/v2.x/en/installation/references/) for overlay networking are not opened between the hosts indicated.
If you see error in the output, that means that the [required ports]({{<baseurl>}}/rancher/v2.x/en/cluster-provisioning/node-requirements/#networking-requirements/) for overlay networking are not opened between the hosts indicated.
Example error output of a situation where NODE1 had the UDP ports blocked.
content/rke/latest/en/os/_index.md
-1
View File
@@ -200,7 +200,6 @@ docker version --format '{{.Server.Version}}'
```
## Ports
{{< ports-rke-nodes >}}
{{< requirements_ports_rke >}}
layouts/shortcodes/requirements_ports_rancher.html
-59
View File
@@ -1,59 +0,0 @@
<div>
<p><strong>Rancher nodes:</strong><br/>Nodes running the <code>rancher/rancher</code> container</p>
<h3>Rancher nodes - Inbound rules</h3>
<table>
<tr>
<th>Protocol</th>
<th>Port</th>
<th align="left">Source</th>
<th align="left">Description</th>
</tr>
<tr>
<td>TCP</td>
<td>80</td>
<td><ul><li>Load balancer/proxy that does external SSL termination</li></ul></td>
<td>Rancher UI/API when external SSL termination is used</td>
</tr>
<tr>
<td>TCP</td>
<td>443</td>
<td><ul><li>etcd nodes</li><li>controlplane nodes</li><li>worker nodes</li><li>Hosted/Imported Kubernetes</li><li>any that needs to be able to use UI/API</li></ul></td>
<td>Rancher agent, Rancher UI/API, kubectl</td>
</tr>
</table>
<h3>Rancher nodes - Outbound rules</h3>
<table>
<tr>
<th>Protocol</th>
<th>Port</th>
<th align="left">Destination</th>
<th align="left">Description</th>
</tr>
<tr>
<td>TCP</td>
<td>22</td>
<td><ul><li>Any node IP from a node created using Node Driver</li></td>
<td>SSH provisioning of nodes using Node Driver</td>
</tr>
<tr>
<td>TCP</td>
<td>443</td>
<td><ul><li>35.160.43.145/32</li><li>35.167.242.46/32</li><li>52.33.59.17/32</li></td>
<td>git.rancher.io (catalogs)</td>
</tr>
<tr>
<td>TCP</td>
<td>2376</td>
<td><ul><li>Any node IP from a node created using Node Driver</li></td>
<td>Docker daemon TLS port used by Docker Machine</td>
</tr>
<tr>
<td>TCP</td>
<td>6443</td>
<td><ul><li>Hosted/Imported Kubernetes API</li></td>
<td>Kubernetes apiserver</td>
</tr>
</table>
<br/>
</div>