Merge pull request #324 from deniseschannon/rke-updates

Rke updates
This commit is contained in:
Denise
2018-06-21 15:57:35 -07:00
committed by GitHub
9 changed files with 165 additions and 116 deletions
+57 -32
View File
@@ -4,23 +4,46 @@ weight: 3000
draft: true
---
When setting up your cluster.yml for RKE, there are a lot of different options that can be configured to control RKE behavior.
When setting up your `cluster.yml` for RKE, there are a lot of different options that can be configured to control the behavior of how RKE launches Kubernetes.
## Cluster options
There are several options that can be configured in cluster configuration option. There are several [example yamls]({{< baseurl >}}/rke/v0.1.x/en/config-options/example-yamls/) that contain all the options.
#### Cluster Name
### Configuring Nodes
* [Nodes]({{< baseurl >}}/rke/v0.1.x/en/config-options/nodes/)
* [Ignoring unsupported Docker versions](#supported-docker-versions)
* [Private Registries]({{< baseurl >}}/rke/v0.1.x/en/config-options/private-registries/)
* [Cluster Level SSH Key Path](#cluster-level-ssh-key-path)
* [SSH Agent](#ssh-agent)
* [Bastion Host]({{< baseurl >}}/rke/v0.1.x/en/config-options/bastion-host/)
`cluster_name` is an optional value that you can set to identify your cluster. The name will be set in your cluster's generated kubeconfig file.
### Configuring Kubernetes Cluster
* [Cluster Name](#cluster-name)
* [Kubernetes Version](#kubernetes-version)
* [System Images]({{< baseurl >}}/rke/v0.1.x/en/config-options/system-images/)
* [Services]({{< baseurl >}}/rke/v0.1.x/en/config-options/services/)
* [External Etcd]({{< baseurl >}}/rke/v0.1.x/en/config-options/services/external-etcd/)
* [Extra Args and Binds]({{< baseurl >}}/rke/v0.1.x/en/config-options/services/extra-args-and-binds/)
* [Add-ons]({{< baseurl >}}/rke/v0.1.x/en/config-options/add-ons/)
* [Add-ons Jobs Timeout](#add-ons-jobs-timeout)
* [Authentication]({{< baseurl >}}/rke/v0.1.x/en/config-options/authentication/)
* [Authorization]({{< baseurl >}}/rke/v0.1.x/en/config-options/authorization/)
* [Cloud Providers]({{< baseurl >}}/rke/v0.1.x/en/config-options/cloud-providers/)
* [Network Plugins]({{< baseurl >}}/rke/v0.1.x/en/config-options/network-plugins/)
* [Ingress Controller]({{< baseurl >}}/rke/v0.1.x/en/config-options/ingress-controller/)
The default value for this option is `local`.
## Cluster Level Options
### Cluster Name
By default, the name of your cluster will be `local`. If you want a different name, you would use the `cluster_name` directive to change the name of your cluster. The name will be set in your cluster's generated kubeconfig file.
```yaml
cluster_name: mycluster
```
#### Supported Docker Versions
By default, RKE will check the installed Docker version on all hosts and fail with an error if the version is not supported by Kubernetes. To override this behavior, set this option to `true`.
### Supported Docker Versions
By default, RKE will check the installed Docker version on all hosts and fail with an error if the version is not supported by Kubernetes. The list of [supported Docker versions](https://github.com/rancher/rke/blob/master/docker/docker.go#L29) are set specifically for each Kubernetes version. To override this behavior, set this option to `true`.
The default value is `false`.
@@ -28,25 +51,9 @@ The default value is `false`.
ignore_docker_version: true
```
#### SSH Key Path
RKE connects to host(s) using `ssh`. Typically, each node will have an independent path for each ssh key, i.e. `ssh_key_path`) in the `nodes` section, but if you have a SSH key that is able to access **all** hosts in your cluster configuration file, you can set the path to that ssh key at the top level.
### Kubernetes Version
If both cluster-level and node-level ssh keys are defined, the node-level key will take precedence.
```yaml
ssh_key_path: ~/.ssh/test
```
#### SSH Agent
RKE supports using ssh connection configuration from a local ssh agent. The default value for this option is `false`.
```yaml
ssh_agent_auth: true
```
#### Kubernetes Version
Use this option to choose Kubernetes version to install. Since this option was added mainly to be used by Rancher v2.0, it has a limited number of supported tags:
You can select which version of Kubernetes to install for your cluster. These options are the Kubernetes versions made available in Rancher v2.x. The current default Kubernetes version used by RKE is `v1.10.3-rancher2-1`. If a version is defined in `kubernetes_version` and is not found in this list, the default is used.
|Kubernetes version|
|-----------------|
@@ -58,21 +65,39 @@ Use this option to choose Kubernetes version to install. Since this option was a
|v1.8.11-rancher2-1|
|v1.8.10-rancher1-1|
The current default Kubernetes version used by RKE is `v1.10.3-rancher2-1`. If a version is defined in `kubernetes_version` and is not found in this list, the default is used.
There are two ways to select a Kubernetes version:
- Using the kubernetes image defined in [System Images](#rke-system-images)
- Using the configuration option `kubernetes_version`
- Using the Kubernetes image defined in [system images](#rke-system-images)
- Using the configuration option `kubernetes_version` in
```yaml
kubernetes_version: "v1.10.3-rancher2-1"
```
In case both are defined, the system images configuration will take precedence over `kubernetes_version`.
### Cluster Level SSH Key Path
RKE connects to host(s) using `ssh`. Typically, each node will have an independent path for each ssh key, i.e. `ssh_key_path`, in the `nodes` section, but if you have a SSH key that is able to access **all** hosts in your cluster configuration file, you can set the path to that ssh key at the top level. Otherwise, you would set the ssh key path in the [nodes]({{< baseurl >}}/rke/v0.1.x/en/config-options/nodes/).
#### Addons Job Timeout
If ssh key paths are defined at the cluster level and at the node level, the node-level key will take precedence.
RKE kubernetes add-ons are deployed using kubernetes jobs. RKE will give up on trying to get the job status after this timeout in seconds. The default timeout value is `30` seconds.
```yaml
ssh_key_path: ~/.ssh/test
```
### SSH Agent
RKE supports using ssh connection configuration from a local ssh agent. The default value for this option is `false`. If you want to set using a local ssh agent, you would set this to `true`.
```yaml
ssh_agent_auth: true
```
### Add-ons Job Timeout
You can define [add-ons]({{< baseurl >}}/rke/v0.1.x/en/config-options/add-ons/) to be deployed after the Kubernetes cluster comes up, which uses Kubernetes [jobs](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/). RKE will stop attempting to retrieve the job status after the timeout, which is in seconds. The default timeout value is `30` seconds.
```yaml
addon_job_timeout: 30
```
<!--add in sections for each option with high level description and link-->
@@ -19,24 +19,24 @@ bastion_host:
# -----END RSA PRIVATE KEY-----
```
### Bastion Host Options
## Bastion Host Options
#### Address
### Address
The `address` directive will be used to set the hostname or IP address of the bastion host. RKE must be able to connect to this address.
#### SSH Port
### SSH Port
You specify which `port` to be used when connecting to the bastion host. The default port is `22`.
#### SSH Users
### SSH Users
You specify the `user` to be used when connecting to this node.
#### SSH Key Path
### SSH Key Path
You specify the path, i.e. `ssh_key_path`, for the SSH private key to be used when connecting to the bastion host.
#### SSH Key
### SSH Key
Instead of setting the path to the SSH key, you can specify the actual key, i.e. `ssh_key`, to be used to connect to the bastion host.
@@ -108,9 +108,11 @@ Besides the minimum set of options, there are many other options that are suppor
| Azure Configuration Options | Type | Required |
|:----------------------------: |:------: |:---------:|
| cloud | string | |
| tenantId | string | * |
| subscriptionId | string | * |
| aadClientId | string | * |
| aadClientSecret | string | * |
| cloud | string | |
| resourceGroup | string | |
| location | string | |
| vnetName | string | |
@@ -121,8 +123,6 @@ Besides the minimum set of options, there are many other options that are suppor
| primaryAvailabilitySetName | string | |
| vmType | string | |
| primaryScaleSetName | string | |
| aadClientId | string | * |
| aadClientSecret | string | * |
| aadClientCertPath | string | |
| aadClientCertPassword | string | |
| cloudProviderBackoff | bool | |
@@ -155,7 +155,7 @@ cloud_provider:
subnet-id: xxxxxxxxxxxxxx
block_storage:
ignore-volume-az: true
router:
route:
router-id: xxxxxxxxxxxxxx
metadata:
search-order: xxxxxxxxxxxxxx
@@ -172,7 +172,7 @@ The Openstack configuration options are divided into 5 groups.
* Global
* Load Balancer
* Block Storage
* Router
* Route
* Metadata
##### Global
@@ -205,11 +205,12 @@ These are the options that are available under the `load_balancer` directive.
| floating-network-id | string | |
| lb-method | string | |
| lb-provider | string | |
| manage-security-groups | bool | |
| create-monitor | bool | |
| monitor-delay | int | * if `create-monitor` is true |
| monitor-timeout | int | * if `create-monitor` is true |
| monitor-max-retries | int | * if `create-monitor` is true |
| manage-security-groups | bool | |
##### Block Storage
@@ -221,11 +222,11 @@ These are the options that are available under the `block_storage` directive.
| trust-device-path | bool | |
| ignore-volume-az | bool | |
##### Router
##### Route
This is the option that is available under the `router` directive.
This is the option that is available under the `route` directive.
| OpenStack's Router Configuration Option | Type | Required |
| OpenStack's Route Configuration Option | Type | Required |
|:--------------------: |:------: |:---------:|
| router-id | string | |
@@ -34,43 +34,43 @@ nodes:
app: ingress
```
### Node Options
## Node Options
Within each node, there are multiple directives that can be used.
#### Address
### Address
The `address` directive will be used to set the hostname or IP address of the node. RKE must be able to connect to this address.
#### Internal Address
### Internal Address
The `internal_address` provides the ability to have nodes with multiple addresses set a specific address to use for inter-host communication on a private network. If the `internal_address` is not set, the `address` is used for inter-host communication.
#### Overriding the Hostname
### Overriding the Hostname
The `hostname_override` is used to be able to provide a friendly name for RKE to use when registering the node in Kubernetes. This hostname doesn't need to be a routable address. If the `hostname_override` isn't set, then the `address` directive is used when registering the node in Kubernetes.
> **Note:** When [cloud providers]({{< baseurl >}}/rke/v0.1.x/en/config-options/cloud-providers/) are configured, you may need to override the hostname in order to use the cloud provider correctly.
> **Note:** When [cloud providers]({{< baseurl >}}/rke/v0.1.x/en/config-options/cloud-providers/) are configured, you may need to override the hostname in order to use the cloud provider correctly. There is an exception for the [AWS cloud provider](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/#aws), where the `hostname_override` field will be explicitly ignored.
#### SSH Port
### SSH Port
In each node, you specify which `port` to be used when connecting to this node. The default port is `22`.
#### SSH Users
### SSH Users
For each node, you specify the `user` to be used when connecting to this node. This user must be a member of the Docker group or allowed to write to the node's Docker socket.
#### SSH Key Path
### SSH Key Path
For each node, you specify the path, i.e. `ssh_key_path`, for the SSH private key to be used when connecting to this node.
> **Note:** If you have a private key that can be used across all nodes, you can set the [SSH key path at the cluster level]({{< baseurl >}}/rke/v0.1.x/en/config-options/#cluster-level-ssh-key-path). The SSH key path set in each node will always take precedence.
#### SSH Key
### SSH Key
Instead of setting the path to the SSH key, you can alternatively specify the actual key, i.e. `ssh_key`, to be used to connect to the node.
#### Kubernetes Roles
### Kubernetes Roles
You can specify the list of roles that you want the node to be as part of the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process.
@@ -89,11 +89,11 @@ With this role, any workloads or pods that are deployed will land on these nodes
> **Note:** Prior to v0.1.8, workloads/pods might have run on any nodes with `worker` or `controlplane` roles, but as of v0.1.8, they will only be deployed to any `worker` nodes.
#### Docker Socket
### Docker Socket
If the Docker socket is different than the default, you can set the `docker_socket`. The default is `/var/run/docker.sock`
#### Labels
### Labels
You have the ability to add an arbitrary map of labels for each node. It can be used when using the [ingress controller's]({{< baseurl >}}/rke/v0.1.x/en/config-options/ingress-controller/) `node_selector` option.
@@ -1,9 +1,11 @@
---
title: Private Registries
weight: 3000
weight: 3010
draft: true
---
RKE supports configuring multiple private/authenticated Docker registries. This is useful if you have private images to use or are deploying your cluster in an air-gapped environment.
RKE supports the ability to configure multiple private Docker registries. By passing in your registry and credentials, it allows the nodes to pull images from these private registries.
```yaml
private_registries:
- url: registry.com
@@ -13,3 +15,7 @@ private_registries:
user: myuser
password: mypassword
```
### Air-gapped Setups
If you are in an air-gapped setup, you will need to not only configure your private registry credentials, but you will need to also update all your [system images]({{< baseurl >}}/rke/v0.1.x/en/config-options/system-images/) so they are going to pull from the private registry. By default, these system images are pulling from `docker.io`.
@@ -1,6 +1,6 @@
---
title: Kubernetes Services
weight: 3000
weight: 3025
draft: true
---
@@ -1,11 +1,12 @@
---
title: External etcd
weight: 3000
weight: 3027
draft: true
---
By default, RKE will launch etcd servers, but RKE also supports being able to use an external etcd. RKE only supports connecting to a TLS enabled etcd setup.
RKE supports using external etcd instead of deploying etcd servers, to enable external etcd the following parameters should be populated:
> **Note:** RKE will not accept having external etcd servers in conjunction with [nodes]({{< baseurl >}}/rke/v0.1.x/en/config-options/nodes/) with the `etcd` role.
```
services:
@@ -27,4 +28,16 @@ services:
-----END PRIVATE KEY-----
```
Note that RKE only supports connecting to TLS enabled etcd setup, user can enable multiple endpoints in the `external_urls` field. RKE will not accept having external urls and nodes with `etcd` role at the same time, user should only specify either etcd role for servers or external etcd but not both.
## External etcd Options
### Path
The `path` defines the location of where the etcd cluster is on the endpoints.
### External URLs
The `external_urls` are the endpoints of where the etcd cluster is hosted. There can be multiple endpoints for the etcd cluster.
### CA Cert/Cert/KEY
The certificates and private keys used to authenticate and access the etcd service.
@@ -1,11 +1,18 @@
---
title: Extra Args and Binds
weight: 3000
weight: 3026
draft: true
---
RKE supports the ability to configure your Kubernetes components by adding in extra service arguments to these components.
RKE supports additional service arguments.
## Extra Args
For any of the Kubernetes services, you can update the `extra_args` to change the existing defaults.
As of `v0.1.3`, using `extra_args` will add new arguments and **override** any existing defaults. For example, if you need to modify the default admission controllers list, you need to include the default list and edit it with your changes so all changes are included.
Prior to `v0.1.3`, using `extra_args` would only add new arguments to the list and there was no ability to change the default list.
```yaml
services:
@@ -14,11 +21,10 @@ services:
extra_args:
cluster-name: "mycluster"
```
This will add/append `--cluster-name=mycluster` to the container list of arguments.
As of `v0.1.3-rc2` using `extra_args` will add new arguments and **override** existing defaults. For example, if you need to modify the default admission controllers list, you need to change the default list and add apply it using `extra_args`.
## Extra Binds
RKE also supports additional volume binds:
Additional volume binds can be made to services using the `extra_binds` arguments.
```yaml
services:
@@ -1,59 +1,57 @@
---
title: System Images
weight: 3000
weight: 3020
draft: true
---
RKE pulls several images during it's operation. Both for actual system components and to perform deployment actions.
When RKE is deploying Kubernetes, there are several images that are pulled. These images are used as Kubernetes system components as well as helping to deploy these system components.
As of `v0.1.6`, the functionality of a couple of the system images were consolidated into a single `rancher/rke-tools` image to simplify and speed the deployment process.
This is the example of the full list of system images used to deploy Kubernetes through RKE. The image tags are dependent on the [Kubernetes image/version used](https://github.com/rancher/types/blob/master/apis/management.cattle.io/v3/k8s_defaults.go).
> **Note:** As versions of RKE are released, the tags on these images will no longer be up to date. This list is specific for `v1.10.3-rancher2`.
```yaml
system_images:
kubernetes: rancher/hyperkube:v1.10.3-rancher2
etcd: rancher/coreos-etcd:v3.1.12
alpine: rancher/rke-tools:v0.1.9
nginx_proxy: rancher/rke-tools:v0.1.9
cert_downloader: rancher/rke-tools:v0.1.9
kubernetes_services_sidecar: rancher/rke-tools:v0.1.9
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.8
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.8
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.8
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
pod_infra_container: rancher/pause-amd64:3.1
# Networking Options
flannel: rancher/coreos-flannel:v0.9.1
flannel_cni: rancher/coreos-flannel-cni:v0.2.0
calico_node: rancher/calico-node:v3.1.1
calico_cni: rancher/calico-cni:v3.1.1
calico_ctl: rancher/calico-ctl:v2.0.0
canal_node: rancher/calico-node:v3.1.1
canal_cni: rancher/calico-cni:v3.1.1
canal_flannel: rancher/coreos-flannel:v0.9.1
weave_node: weaveworks/weave-kube:2.1.2
weave_cni: weaveworks/weave-npc:2.1.2
# Ingress OPTIONS
ingress: rancher/nginx-ingress-controller:0.10.2-rancher3
ingressBackend: rancher/nginx-ingress-controller-defaultbackend:1.4
```
Prior to `v0.1.6`, instead of using the `rancher/rke-tools` image, we used the following images:
Prior to version `0.1.6`, RKE used the following list of images for deployment and cluster configuration:
```yaml
system_images:
etcd: rancher/etcd:v3.0.17
kubernetes: rancher/k8s:v1.8.9-rancher1-1
alpine: alpine:latest
nginx_proxy: rancher/rke-nginx-proxy:v0.1.1
cert_downloader: rancher/rke-cert-deployer:v0.1.1
kubernetes_services_sidecar: rancher/rke-service-sidekick:v0.1.0
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.5
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.5
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.5
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
```
In addintion to the above list, images for the network plugin were also pulled. The exact network plugin images depend on the configured plugin and it's version.
### Air-gapped Setups
As of version `0.1.6`, the functionality of several system images were consolidated into a single `rke-tools` image to simplify and speed the deployment process.
The following images are no longer used. They are replaced by `rancher/rke-tools`:
- alpine:latest
- rancher/rke-nginx-proxy:v0.1.1
- rancher/rke-cert-deployer:v0.1.1
- rancher/rke-service-sidekick:v0.1.0
This as of version `v0.1.8`, this is a full list of images and values:
```yaml
etcd: rancher/coreos-etcd:v3.1.12
alpine: rancher/rke-tools:v0.1.9
nginx_proxy: rancher/rke-tools:v0.1.9
cert_downloader: rancher/rke-tools:v0.1.9
kubernetes_services_sidecar: rancher/rke-tools:v0.1.9
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.8
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.8
kubednsSidecar: rancher/k8s-dns-sidecar-amd64:1.14.8
kubednsAutoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
kubernetes: rancher/hyperkube:v1.10.3-rancher2
flannel: rancher/coreos-flannel:v0.9.1
flannelCNI: rancher/coreos-flannel-cni:v0.2.0
calico_node: rancher/calico-node:v3.1.1
calico_cni: rancher/calico-cni:v3.1.1
calico_ctl: rancher/calico-ctl:v2.0.0
canal_node: rancher/calico-node:v3.1.1
canal_cni: rancher/calico-cni:v3.1.1
Canalflannel: rancher/coreos-flannel:v0.9.1
wave_node: weaveworks/weave-kube:2.1.2
weave_cni: weaveworks/weave-npc:2.1.2
pod_infra_container: rancher/pause-amd64:3.1
ingress: rancher/nginx-ingress-controller:0.10.2-rancher3
ingressBackend: rancher/nginx-ingress-controller-defaultbackend:1.4
```
If you have an air-gapped setup and cannot access `docker.io`, you will need to set up your [private registry]({{< baseurl >}}/rke/v0.1.x/en/config-options/private-registries/) in your cluster configuration file. After you set up private registry, you will need to update these images to pull from your private registry.