Updates for RKE config-options

This commit is contained in:
moelsayed
2018-06-18 11:06:58 -07:00
committed by Denise
parent b569e21606
commit 3f1120fc9d
15 changed files with 276 additions and 125 deletions
+54 -31
View File
@@ -4,52 +4,75 @@ weight: 3000
draft: true
---
When setting up your cluster.yml for RKE, there are a lot of different options that can be configured.
When setting up your cluster.yml for RKE, there are a lot of different options that can be configured to control RKE behavior.
## Cluster options
## Using RKE to generate a cluster.ml
#### Cluster Name
RKE supports command `rke config` which generates a cluster config template for the user, to start using this command just write:
`cluster_name` is an optional value that you can set to identify your cluster. The name will be set in your cluster's generated kubeconfig file.
```bash
rke config --name mycluster.yml
```
The default value for this option is `local`.
RKE will ask some questions around the cluster file like number of the hosts, ips, ssh users, etc, `--empty` option will generate an empty cluster.yml file, also if you just want to print on the screen and not save it in a file you can use `--print`.
## Cluster Yaml Links
## Naming your Cluster
```
# If set, this is the cluster name that will be used in the kube config file
# Default value is "local"
```yaml
cluster_name: mycluster
```
## Docker Version check
#### Supported Docker Versions
By default, RKE will check the installed Docker version on all hosts and fail with an error if the version is not supported by Kubernetes. To override this behavior, set this option to `true`.
```
# If set to true, rke won't fail when unsupported Docker version is found
ignore_docker_version: false
The default value is `false`.
```yaml
ignore_docker_version: true
```
## Kubernetes authorization (RBAC)
#### SSH Key Path
RKE connects to host(s) using `ssh`. Typically, each node will have an independent path for each ssh key, i.e. `ssh_key_path`) in the `nodes` section, but if you have a SSH key that is able to access **all** hosts in your cluster configuration file, you can set the path to that ssh key at the top level.
```
# Kubernetes authorization mode
# Use `mode: rbac` to enable RBAC
# Use `mode: none` to disable authorization
authorization:
mode: rbac
```
## SSH Key path
```
If both cluster-level and node-level ssh keys are defined, the node-level key will take precedence.
```yaml
ssh_key_path: ~/.ssh/test
```
#### SSH Agent
RKE supports using ssh connection configuration from a local ssh agent. The default value for this option is `false`.
```yaml
ssh_agent_auth: true
```
#### Kubernetes Version
Use this option to choose Kubernetes version to install. Since this option was added mainly to be used by Rancher v2.0, it has a limited number of supported tags:
|Kubernetes version|
|-----------------|
|v1.10.3-rancher2-1|
|v1.10.1-rancher2-1|
|v1.10.0-rancher1-1|
|v1.9.7-rancher2-1|
|v1.9.5-rancher1-1|
|v1.8.11-rancher2-1|
|v1.8.10-rancher1-1|
The current default Kubernetes version used by RKE is `v1.10.3-rancher2-1`. If a version is defined in `kubernetes_version` and is not found in this list, the default is used.
There are two ways to select a Kubernetes version:
- Using the kubernetes image defined in [System Images](#rke-system-images)
- Using the configuration option `kubernetes_version`
In case both are defined, the system images configuration will take precedence over `kubernetes_version`.
#### Addons Job Timeout
RKE kubernetes add-ons are deployed using kubernetes jobs. RKE will give up on trying to get the job status after this timeout in seconds. The default timeout value is `30` seconds.
```yaml
addon_job_timeout: 30
```
<!--add in sections for each option with high level description and link-->
@@ -5,22 +5,36 @@ draft: true
---
RKE supports pluggable addons. Addons are used to deploy several cluster components including:
RKE supports pluggable add-ons. Addons are used to deploy several cluster components including:
- Network plugin
- KubeDNS
- Ingress controller
In addition, a user can specify the addon yaml in the cluster.yml file, and when running
In addition, a user can specify an addon yaml manifests in the cluster.yml file. RKE will deploy the user-defined add-ons after the cluster deployment is complete.
```yaml
rke up --config cluster.yml
```
RKE first uploads the yaml manifest as a configmap to the Kubernetes cluster. Next, it will run a kubernetes job that mounts this configmap and deploy the addon by running `kubectl apply -f`.
RKE will deploy the addons yaml after the cluster starts, RKE first uploads this yaml file as a configmap in kubernetes cluster and then run a kubernetes job that mounts this config map and deploy the addons.
> **Note: RKE doesn't support removal or update of cluster add-ons when doing `rke up` with a different list of add-ons. Please update the add-ons by using `kubectl edit`.**
> Note that RKE doesn't support yet removal or update of the addons, so once they are deployed the first time you can't change them using rke
To start using addons use `addons:` option in the `cluster.yml` file for example:
#### Critical and non-critical add-ons
As of version `0.1.7`, add-ons are split into two categories:
- **Critical add-ons:** If these add-ons fail to deploy for any reason, RKE will error out.
- **Non-critical add-ons:** If these add-ons fail to deploy, RKE will only log a warning and continue deploying other add-ons.
Currently, only the network plugin is considered critical. KubeDNS, Ingress and user-defined add-ons are considered non-critical.
## Defining User Addons
User defined add-ons can be added by either defining the yaml directly in the RKE config or pointing to a file. Adding a yaml directly into the file uses the `addons` directive, while pointing to a file uses the `addons_include:` directive.
> **Note**: When using user-defined add-ons, you *must* define a namespace for *all* your resources, otherwise they will end up in the `kube-system` namespace.
##### In-line Addons
To define an add-on directly in the yaml file, make sure to use the yaml's block indicator `|-` as the `addons` directive is a multi-line string option. It's possible to specify multiple yaml resource definitions by separating them using the `---` directive.
```yaml
addons: |-
@@ -38,9 +52,11 @@ addons: |-
- containerPort: 80
```
Note that we are using `|-` because the addons option is a multi line string option, where you can specify multiple yaml files and separate them with `---`
For `addons_include:` you may pass either http/https urls or file paths, for example:
#### Referencing files for Add-ons
User defined add-ons support referencing a local file or a URL.
```yaml
addons_include:
- https://raw.githubusercontent.com/rook/rook/master/cluster/examples/kubernetes/rook-operator.yaml
@@ -49,14 +65,10 @@ addons_include:
- ./nginx.yaml
```
#### Addon deployment jobs
RKE uses kubernetes Jobs to deploy addons. In some cases, addons deployment takes longer than expected. Starting with version `0.1.7-rc1`, RKE provides an option to controle the job check timeout in seconds:
RKE uses kubernetes Jobs to deploy add-ons. In some cases, add-ons deployment takes longer than expected. Starting with version `0.1.7-rc1`, RKE provides an option to control the job check timeout in seconds:
```yaml
addon_job_timeout: 30
```
#### Critical and noncritical addons
As of version `0.1.7-rc1`, addons are split into two categories: critical and noncritical.
Critical addons will cause RKE to error out if they fail to deploy for any reason. While noncritical addons will just log a warning and continue with the deployment. Currently only the network plugin is considered critical.
@@ -4,7 +4,7 @@ weight: 3000
draft: true
---
RKE Supports x509 authentication strategy. You can additionally define a list of SANs (Subject Alternative Names) to add to the Kubernetes API Server PKI certificates. This allows you to connect to your Kubernetes cluster API Server through a load balancer, for example, rather than a single node.
RKE supports x509 authentication strategy. You can additionally define a list of SANs (Subject Alternative Names) to add to the Kubernetes API Server PKI certificates. This allows you to connect to your Kubernetes cluster API Server through a load balancer, for example, rather than a single node.
```yaml
authentication:
@@ -0,0 +1,16 @@
---
title: Authorization
weight: 3000
draft: true
---
Kubernetes supports multiple [Authorization Modules](https://kubernetes.io/docs/reference/access-authn-authz/authorization/#authorization-modules). Currently, RKE provides support for the [RBAC module](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) only.
RBAC is enabled by default in RKE. It's possible to disable authorization support by seeting the authorization mode to `none`
```yaml
# Use `mode: none` to disable authorization
authorization:
mode: rbac
```
<!-- explain cluster authorization configuration-->
@@ -3,10 +3,8 @@ title: Bastion/Jump Host Configuration
weight: 3000
draft: true
---
```
# Bastion/Jump host configuration
Since RKE uses `ssh` to connect to hosts, you may have to use RKE with a bastion host.
```yaml
bastion_host:
address: x.x.x.x
user: ubuntu
@@ -177,7 +177,7 @@ ignore_docker_version: false
# The kubernetes version used. For now, this should match the version defined in rancher/types defaults map: https://github.com/rancher/types/blob/master/apis/management.cattle.io/v3/k8s_defaults.go#L14
kubernetes_version: v1.10.1-rancher1
# addons are deployed using kubernetes jobs. RKE will give up on trying to get the job status after this timeout in seconds..
# add-ons are deployed using kubernetes jobs. RKE will give up on trying to get the job status after this timeout in seconds..
addon_job_timeout: 30
# If set, this is the cluster name that will be used in the kube config file
# Default value is "local"
@@ -1,20 +0,0 @@
---
title: Kubernetes Versions
weight: 3000
draft: true
---
<!--where to find default for rke
how is default version determined
how to change it using configuration Options-->
The current default kubernetes version used by RKE is `v1.10.1-rancher1`.
There are two ways to select a kubernetes version:
- Using the kubernetes image defined in [System Images](#rke-system-images)
- Using the configuration option `kubernetes_version`
In case both are defined, the system images configuration will take precedence over `kubernetes_version`. Since the `kubernetes_version` options was added mainly to be used by Rancher v2.0, it has a limited number of supported tags that can be found [here](https://github.com/rancher/types/blob/master/apis/management.cattle.io/v3/k8s_defaults.go#L14).
If a version is defined in `kubernetes_version` and is not found in this map, the default is used.
@@ -4,19 +4,22 @@ weight: 3000
draft: true
---
RKE supports the following network plugins that are deployed as addons:
RKE supports the following network plugins that are deployed as add-ons:
- Flannel
- Calico
- Canal
- Weave
To use specific network plugin configure `cluster.yml` to include:
To use a specific network plugin configure `cluster.yml` to include:
```yaml
network:
plugin: flannel
```
Additionally, some plugins support additional options that can be used to customize their behavior.
If this section is not specified in the cluster configration, the default network plugin will be used. The default is `canal`.
### Network Options
@@ -24,25 +27,8 @@ There are extra options that can be specified for each network plugin:
#### Flannel
- **flannel_image**: Flannel daemon Docker image
- **flannel_cni_image**: Flannel CNI binary installer Docker image
- **flannel_iface**: Interface to use for inter-host communication
#### Calico
- **calico_node_image**: Calico Daemon Docker image
- **calico_cni_image**: Calico CNI binary installer Docker image
- **calico_controllers_image**: Calico Controller Docker image
- **calicoctl_image**: Calicoctl tool Docker image
- **calico_cloud_provider**: Cloud provider where Calico will operate, currently supported values are: `aws`, `gce`
#### Canal
- **canal_node_image**: Canal Node Docker image
- **canal_cni_image**: Canal CNI binary installer Docker image
- **canal_flannel_image**: Canal Flannel Docker image
#### Weave
- **weave_node_image**: Weave Node Docker image
- **weave_cni_image**: Weave CNI binary installer Docker image
@@ -3,5 +3,80 @@ title: Nodes
weight: 3000
draft: true
---
The `nodes` section is the only required section in the `cluster.yml` file. It's used by RKE to specify cluster node, ssh credentials used to access them and their roles in the RKE cluster.
The `nodes` section is a yaml list of node definitions:
``` yaml
nodes:
nodes:
- address: 1.1.1.1
user: ubuntu
role:
- controlplane
- etcd
ssh_key_path: /home/user/.ssh/id_rsa
port: 2222
- address: 2.2.2.2
user: ubuntu
role:
- worker
ssh_key: |-
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
- address: example.com
user: ubuntu
role:
- role
hostname_override: node3
internal_address: 192.168.1.6
labels:
app: ingress
```
The node definition supports the following options:
###### address
This is the hostname or the IP address of the node. RKE should be able to connect to this address.
###### internal_address
This option allows the user to use nodes with multiple addresses to keep inter-host communication on a private network. If this is not set, `address` is used instead.
###### hostname_override
This is a friendly name that RKE use to register the node with in Kubernetes. This doesn't need to be a routable address. If the `hostname_override` is not used, the `address` field is used instead.
> Note that this option is ignored by Kubernetes when Cloud Providers configuration is used. In that case, Kubernetes uses the hostname provided by the cloud provider.
###### port
SSH port to be used when connecting to this node. Default port is `22`
###### user
SSH user to be used when connecting to this node. Note that this user must be a member of the `docker` group or allowed to write to the node Docker socket.
###### ssh_key_path
Path for the ssh private key used to connect to this node.
###### ssh_key
An in-line option to set the private key used to connect to this node inside the cluster.yml file.
###### role
A yaml list of roles to specify the node role(s) in the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process.
A working cluster has to have at least 1 node with the `controlplane` and `etcd` roles.
###### docker_socket
An option to specify a different Docker socket location for the node. The default is `/var/run/docker.sock`
###### labels
An option to set an arbitrary map labels for nodes. It's also handy when used with the ingress controller `node_selector` option.
<!-- explain how to set up nodes in yaml with examples and what is required-->
@@ -3,12 +3,13 @@ title: Private Registries
weight: 3000
draft: true
---
```
# List of registry credentials, if you are using a Docker Hub registry,
# you can omit the `url` or set it to `docker.io`
RKE supports configuring multiple private/authenticated Docker registries. This is useful if you have private images to use or are deploying your cluster in an air-gapped environment.
```yaml
private_registries:
- url: registry.com
user: Username
password: password
- url: myregistry.com
user: myuser
password: mypassword
```
@@ -1,5 +1,5 @@
title: Default Services and images
title: Kubernetes Services
weight: 3000
draft: true
---
@@ -53,29 +53,3 @@ services:
kubeproxy:
```
## System Images
Prior to version `0.1.6`, RKE used the following list of images for deployment and cluster configuration:
```
system_images:
etcd: rancher/etcd:v3.0.17
kubernetes: rancher/k8s:v1.8.9-rancher1-1
alpine: alpine:latest
nginx_proxy: rancher/rke-nginx-proxy:v0.1.1
cert_downloader: rancher/rke-cert-deployer:v0.1.1
kubernetes_services_sidecar: rancher/rke-service-sidekick:v0.1.0
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.5
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.5
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.5
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
flannel: rancher/coreos-flannel:v0.9.1
flannel_cni: rancher/coreos-flannel-cni:v0.2.0
```
As of version `0.1.6`, we consolidated several of those images into a single image to simplify and speed the deployment process.
The following images are no longer required, and can be replaced by `rancher/rke-tools:v0.1.4`:
- alpine:latest
- rancher/rke-nginx-proxy:v0.1.1
- rancher/rke-cert-deployer:v0.1.1
- rancher/rke-service-sidekick:v0.1.0
@@ -0,0 +1,59 @@
---
title: System Images
weight: 3000
draft: true
---
RKE pulls several images during it's operation. Both for actual system components and to perform deployment actions.
Prior to version `0.1.6`, RKE used the following list of images for deployment and cluster configuration:
```yaml
system_images:
etcd: rancher/etcd:v3.0.17
kubernetes: rancher/k8s:v1.8.9-rancher1-1
alpine: alpine:latest
nginx_proxy: rancher/rke-nginx-proxy:v0.1.1
cert_downloader: rancher/rke-cert-deployer:v0.1.1
kubernetes_services_sidecar: rancher/rke-service-sidekick:v0.1.0
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.5
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.5
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.5
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
```
In addintion to the above list, images for the network plugin were also pulled. The exact network plugin images depend on the configured plugin and it's version.
As of version `0.1.6`, the functionality of several system images were consolidated into a single `rke-tools` image to simplify and speed the deployment process.
The following images are no longer used. They are replaced by `rancher/rke-tools`:
- alpine:latest
- rancher/rke-nginx-proxy:v0.1.1
- rancher/rke-cert-deployer:v0.1.1
- rancher/rke-service-sidekick:v0.1.0
This as of version `v0.1.8`, this is a full list of images and values:
```yaml
etcd: rancher/coreos-etcd:v3.1.12
alpine: rancher/rke-tools:v0.1.9
nginx_proxy: rancher/rke-tools:v0.1.9
cert_downloader: rancher/rke-tools:v0.1.9
kubernetes_services_sidecar: rancher/rke-tools:v0.1.9
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.8
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.8
kubednsSidecar: rancher/k8s-dns-sidecar-amd64:1.14.8
kubednsAutoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
kubernetes: rancher/hyperkube:v1.10.3-rancher2
flannel: rancher/coreos-flannel:v0.9.1
flannelCNI: rancher/coreos-flannel-cni:v0.2.0
calico_node: rancher/calico-node:v3.1.1
calico_cni: rancher/calico-cni:v3.1.1
calico_ctl: rancher/calico-ctl:v2.0.0
canal_node: rancher/calico-node:v3.1.1
canal_cni: rancher/calico-cni:v3.1.1
Canalflannel: rancher/coreos-flannel:v0.9.1
wave_node: weaveworks/weave-kube:2.1.2
weave_cni: weaveworks/weave-npc:2.1.2
pod_infra_container: rancher/pause-amd64:3.1
ingress: rancher/nginx-ingress-controller:0.10.2-rancher3
ingressBackend: rancher/nginx-ingress-controller-defaultbackend:1.4
```
+29 -2
View File
@@ -93,7 +93,34 @@ firewall-cmd --permanent --zone=public --add-rich-rule='
firewall-cmd --reload
```
### Creating `cluster.yml` and Kubernetes cluster
## Creating your cluster configuration
### Using RKE to generate a cluster.yml
RKE provides the command `rke config` to generate a cluster configuration template or to interactivity generate a working cluster configuration file.
Start by running the following command:
```bash
rke config --name cluster.yml
```
RKE will interactivity prompt the user for cluster configuration such as number of the hosts, IP addresses or hostnames, ssh users..etc.
You can also use the `--empty` flag, which will generate an empty cluster.yml template:
```bash
rke config --empty --name cluster.yml
```
Additionally, it's possible to print the generated configuration to stdout and skip creating a file using the `--print` flag:
```bash
rke config --print
```
### Creating your first Kubernetes cluster
For this quick start, we will be configuring one machine. There are 3 roles you can define on a machine:
@@ -159,7 +186,7 @@ There should be a `cluster.yml` created in the directory where you launched the
If you are satisfied with the configuration, you can start building your cluster by using `./rke_darwin-amd64 up` or `./rke_linux-amd64 up`.
### Using RKE
### Running RKE