From 6d57900705f24587242580e094a4622ea14892c7 Mon Sep 17 00:00:00 2001 From: Alena Prokharchyk Date: Thu, 10 Oct 2019 16:22:06 -0700 Subject: [PATCH 001/103] Fix for cloud provider section in cluster.yml for vSphere --- .../latest/en/config-options/cloud-providers/vsphere/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md index f517fe2d20d..88a70c231fc 100644 --- a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md @@ -32,7 +32,7 @@ When provisioning clusters in Rancher using the [vSphere node driver]({{< baseur ![vsphere-node-driver-cloudprovider]({{< baseurl >}}/img/rancher/vsphere-node-driver-cloudprovider.png) 8. Click on **Edit as YAML** -9. Insert the following top-level structure to the pre-populated cluster YAML. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive. +9. Insert the following structure to the pre-populated cluster YAML. In versions prior to v2.3.0, it has to be defined as a top level field. In versions v2.3.0 and higher, it has to be placed under `rancher_kubernetes_engine_config`. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive. ```yaml cloud_provider: From ca6a00873a634c46f1e503528a6f45ced5725443 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Thu, 7 Nov 2019 13:14:17 -0700 Subject: [PATCH 002/103] Update k3s main page Update the main page for #1017 and #1018 This tweaks the main page to update the description of what we remove and add to k3s that sets us apart from k8s. --- content/k3s/latest/en/_index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index d684241f29f..f144d04709f 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -20,14 +20,14 @@ What is this? k3s is intended to be a fully compliant Kubernetes distribution with the following changes: -1. Legacy, alpha, non-default features are removed. Hopefully, you shouldn't notice the +1. Removed legacy and non-default features. You lilkely won't notice the stuff that has been removed. 2. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced with out of tree addons. -3. Add sqlite3 as the default storage mechanism. etcd3 is still available, but not the default. -4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. -5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required - dependencies +3. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. +4. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. +5. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. +6. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required dependencies * containerd * Flannel * CoreDNS From 9016ebaa534e172e5c7ce3e213705c27c4b7b76f Mon Sep 17 00:00:00 2001 From: Sebastiaan van Steenis Date: Fri, 8 Nov 2019 16:25:57 +0100 Subject: [PATCH 003/103] Add docs on kubelet serving certificate --- content/rke/latest/en/config-options/services/_index.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/content/rke/latest/en/config-options/services/_index.md b/content/rke/latest/en/config-options/services/_index.md index 8446aeb52b4..50c4e9ea3b1 100644 --- a/content/rke/latest/en/config-options/services/_index.md +++ b/content/rke/latest/en/config-options/services/_index.md @@ -92,6 +92,8 @@ services: cluster_dns_server: 10.43.0.10 # Fail if swap is on fail_swap_on: false + # Generate per node serving certificate + generate_serving_certificate: false ``` ### Kubelet Options @@ -101,6 +103,13 @@ RKE supports the following options for the `kubelet` service: - **Cluster Domain** (`cluster_domain`) - The [base domain](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/) for the cluster. All services and DNS records created on the cluster. By default, the domain is set to `cluster.local`. - **Cluster DNS Server** (`cluster_dns_server`) - The IP address assigned to the DNS service endpoint within the cluster. DNS queries will be sent to this IP address which is used by KubeDNS. The default value for this option is `10.43.0.10` - **Fail if Swap is On** (`fail_swap_on`) - In Kubernetes, the default behavior for the kubelet is to **fail** if swap is enabled on the node. RKE does **not** follow this default and allows deployments on nodes with swap enabled. By default, the value is `false`. If you'd like to revert to the default kubelet behavior, set this option to `true`. +- **Generate Serving Certificate** (`generate_serving_certificate`) - Generate a certificate signed by the `kube-ca` Certificate Authority for the kubelet to use as serving certificate. The default value for this option is `false`. Before enabling this option, please read [the requirements](#kubelet-serving-certificate-requirements) + +### Kubelet serving certificate requirements + +If `hostname_override` is configured for one or more nodes in `cluster.yml`, please make sure the correct IP address is configured in `address` (and the internal address in `internal_address`) to make sure the generated certificate contains the correct IP address(es). + +An example of an error situation is an EC2 instance where the the public IP address is configured in `address`, and `hostname_override` is used, the connection between `kube-apiserver` and `kubelet` will fail because the `kubelet` will be contacted on the private IP address and the generated certificate will not be valid (the error `x509: certificate is valid for value_in_address, not private_ip` will be seen). The resolution is to provide the internal IP address in `internal_address`. ## Kubernetes Scheduler From 6bf76cf735ea3d5d96cf6e8c8ff900df7cec8518 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 8 Nov 2019 10:28:24 -0700 Subject: [PATCH 004/103] Improve ha install guide This adds an image representing a recommended configuration for HA k3s This includes some context around the image Later, we will tweak the HA page again before release to make some tweaks in a separate PR. --- .../k3s/latest/en/installation/ha/_index.md | 6 + static/img/k3s/k3s-production-setup-v3.svg | 1790 +++++++++++++++++ 2 files changed, 1796 insertions(+) create mode 100644 static/img/k3s/k3s-production-setup-v3.svg diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 0e700cdd6ac..f7d119f3ee2 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -8,6 +8,12 @@ weight: 30 For production environments, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1 +# Recommended Architecture +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v3.svg) +This image depicts a k3s HA install with load balancer to expose workloads and a load balancer for worker (agent) node registration and Kubernetes API access (e.g. kubectl). + +The HA database shown should be a single endpoint k3s can access such as a load balancer. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. + Installation Outline -------------------- 1. Create backend database (PostgreSQL) diff --git a/static/img/k3s/k3s-production-setup-v3.svg b/static/img/k3s/k3s-production-setup-v3.svg new file mode 100644 index 00000000000..cbaa695858c --- /dev/null +++ b/static/img/k3s/k3s-production-setup-v3.svg @@ -0,0 +1,1790 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From 657e0a172548924f151f5ff10aa17e16470191e6 Mon Sep 17 00:00:00 2001 From: Tejeev Date: Fri, 8 Nov 2019 20:02:54 +0000 Subject: [PATCH 005/103] Bolded cluster to make it more clear where step 1 is pointing --- .../tools/istio/setup/enable-istio-in-cluster/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md index 96b5d2590e0..8d6d64b099b 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md @@ -7,7 +7,7 @@ This cluster uses the default Nginx controller to allow traffic into the cluster A Rancher [administrator]({{}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/) or [cluster owner]({{}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles) can configure Rancher to deploy Istio in a Kubernetes cluster. -1. From the **Global** view, navigate to the cluster where you want to enable Istio. +1. From the **Global** view, navigate to the **cluster** where you want to enable Istio. 1. Click **Tools > Istio.** 1. Optional: Configure member access and [resource limits]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/config/) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio. 1. Click **Enable**. @@ -19,4 +19,4 @@ The Istio application, `cluster-istio`, is added as an [application]({{ When Istio is enabled in the cluster, the label for Istio sidecar auto injection,`istio-injection=enabled`, will be automatically added to each new namespace in this cluster. This automatically enables Istio sidecar injection in all new workloads that are deployed in those namespaces. You will need to manually enable Istio in preexisting namespaces and workloads. -### [Next: Enable Istio in a Namespace]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace) \ No newline at end of file +### [Next: Enable Istio in a Namespace]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace) From 36f2c9b39902a2d7fa7a7c3079609dd0b52271e1 Mon Sep 17 00:00:00 2001 From: Tejeev Date: Fri, 8 Nov 2019 20:41:06 +0000 Subject: [PATCH 006/103] kubectl get gateway needs a namespace --- .../v2.x/en/cluster-admin/tools/istio/setup/gateway/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway/_index.md index f297f525faa..771af212486 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway/_index.md @@ -86,7 +86,7 @@ spec: Confirm that the resource exists by running: ``` -kubectl get gateway +kubectl get gateway -A ``` The result should be something like this: From 5409ebd8d0e7bd76d93033ab46258380cb56c59f Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 8 Nov 2019 14:25:47 -0700 Subject: [PATCH 007/103] Update image subtext wording to resolve requested changes --- content/k3s/latest/en/installation/ha/_index.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index f7d119f3ee2..8058fb77a7d 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -10,7 +10,10 @@ For production environments, we recommend installing k3s in a high-availability # Recommended Architecture ![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v3.svg) -This image depicts a k3s HA install with load balancer to expose workloads and a load balancer for worker (agent) node registration and Kubernetes API access (e.g. kubectl). +This image depicts a k3s HA install with two load balancers: + +* A load balancer to expose workloads to external traffic +* A load balancer to expose the Kubernetes API for worker node registration and admin access via `kubectl` The HA database shown should be a single endpoint k3s can access such as a load balancer. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. From 42b2d02eccce2aa8c6b308313fbfb4238fdce156 Mon Sep 17 00:00:00 2001 From: Tejeev Date: Fri, 8 Nov 2019 21:34:49 +0000 Subject: [PATCH 008/103] Fixed 'CPU and memory requirements' link --- .../rancher/v2.x/en/cluster-admin/tools/istio/setup/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/_index.md index 4c3a2faa090..da1fbcacc7a 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/_index.md @@ -23,6 +23,6 @@ If you use Istio for traffic management, you will need to allow external traffic This guide assumes you have already [installed Rancher,]({{}}/rancher/v2.x/en/installation) and you have already [provisioned a separate Kubernetes cluster]({{}}/rancher/v2.x/en/cluster-provisioning) on which you will install Istio. -The nodes in your cluster must meet the [CPU and memory requirements.]({{}}/rancher/v2.x/en/cluster-admin/istio/#cpu-and-memory-requirements) +The nodes in your cluster must meet the [CPU and memory requirements.]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/resources/) The workloads and services that you want to be controlled by Istio must meet [Istio's requirements.](https://istio.io/docs/setup/additional-setup/requirements/) From fe52fadc157b2157b532dd21663560f36f673fa0 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 8 Nov 2019 14:48:53 -0700 Subject: [PATCH 009/103] Remove first line --- content/k3s/latest/en/_index.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index f144d04709f..39b66a7c45c 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -20,14 +20,12 @@ What is this? k3s is intended to be a fully compliant Kubernetes distribution with the following changes: -1. Removed legacy and non-default features. You lilkely won't notice the - stuff that has been removed. -2. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced +1. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced with out of tree addons. -3. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. -4. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. -5. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. -6. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required dependencies +2. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. +3. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. +4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. +5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required dependencies * containerd * Flannel * CoreDNS From b75e96b0330ba12ced713acffb0d1490ca10b0e2 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 8 Nov 2019 15:05:48 -0700 Subject: [PATCH 010/103] Update HA image --- .../k3s/latest/en/installation/ha/_index.md | 2 +- static/img/k3s/k3s-production-setup-v3.svg | 1790 ----------------- static/img/k3s/k3s-production-setup-v4.svg | 1 + 3 files changed, 2 insertions(+), 1791 deletions(-) delete mode 100644 static/img/k3s/k3s-production-setup-v3.svg create mode 100644 static/img/k3s/k3s-production-setup-v4.svg diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 8058fb77a7d..73f4a3676cc 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -9,7 +9,7 @@ weight: 30 For production environments, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1 # Recommended Architecture -![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v3.svg) +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v4.svg) This image depicts a k3s HA install with two load balancers: * A load balancer to expose workloads to external traffic diff --git a/static/img/k3s/k3s-production-setup-v3.svg b/static/img/k3s/k3s-production-setup-v3.svg deleted file mode 100644 index cbaa695858c..00000000000 --- a/static/img/k3s/k3s-production-setup-v3.svg +++ /dev/null @@ -1,1790 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/static/img/k3s/k3s-production-setup-v4.svg b/static/img/k3s/k3s-production-setup-v4.svg new file mode 100644 index 00000000000..c2fa54c57d7 --- /dev/null +++ b/static/img/k3s/k3s-production-setup-v4.svg @@ -0,0 +1 @@ +k3s-production-setup-v2 \ No newline at end of file From ea9d648aa1bad6df1798915d480b784e251386ec Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 8 Nov 2019 16:29:23 -0700 Subject: [PATCH 011/103] Normalize page weights and initial add storage section This tweaks the page weights so they make more sense. We now have these in multiples of ten so we can easily move up or down new sections as we go beetween these numbers. The Volumes and Storage section as added, intial only. A separate commit will have the content for the Volumes and Storage page. --- content/k3s/latest/en/advanced/_index.md | 2 +- content/k3s/latest/en/configuration/_index.md | 2 +- content/k3s/latest/en/faq/_index.md | 2 +- content/k3s/latest/en/installation/_index.md | 2 +- content/k3s/latest/en/known-issues/_index.md | 2 +- content/k3s/latest/en/quick-start/_index.md | 2 +- content/k3s/latest/en/storage/_index.md | 6 ++++++ 7 files changed, 12 insertions(+), 6 deletions(-) create mode 100644 content/k3s/latest/en/storage/_index.md diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 0d7d6e25696..700eb033c04 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -1,6 +1,6 @@ --- title: "Advanced Options" -weight: 3 +weight: 40 aliases: - /k3s/latest/en/running/ --- diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index c86b7c687d4..6591c51bd83 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -1,6 +1,6 @@ --- title: "Configuration Info" -weight: 4 +weight: 50 --- This section contains information on using k3s with various configurations. diff --git a/content/k3s/latest/en/faq/_index.md b/content/k3s/latest/en/faq/_index.md index 9546d22dd83..5b55183ed82 100644 --- a/content/k3s/latest/en/faq/_index.md +++ b/content/k3s/latest/en/faq/_index.md @@ -1,6 +1,6 @@ --- title: FAQ -weight: 8000 +weight: 60 --- The FAQ is updated periodically and designed to answer the questions our users most frequently ask about k3s. diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 677a27014fe..49c6c466dbe 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -1,6 +1,6 @@ --- title: "Installation Options" -weight: 2 +weight: 20 --- This section contains instructions for installing k3s in testing and production environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing k3s. diff --git a/content/k3s/latest/en/known-issues/_index.md b/content/k3s/latest/en/known-issues/_index.md index 7731c22dc4b..7592fcb1572 100644 --- a/content/k3s/latest/en/known-issues/_index.md +++ b/content/k3s/latest/en/known-issues/_index.md @@ -1,6 +1,6 @@ --- title: Known Issues -weight: 9000 +weight: 70 --- The Known Issues are updated periodically and designed to inform you about any issues that may not be immediately addressed in the next upcoming release. diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index 0535c3da2be..dba7ccd0253 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -1,6 +1,6 @@ --- title: "Quick-Start Guide" -weight: 1 +weight: 10 --- >**Note:** The intent of this guide is to quickly launch a cluster that you can use to evaluate k3s. This guide is not intended for production environments. Production environments should utilize a High-Availiability solution. The [installation options](../installation) section covers in greater detail how k3s can be setup. diff --git a/content/k3s/latest/en/storage/_index.md b/content/k3s/latest/en/storage/_index.md new file mode 100644 index 00000000000..0c89f8d4ef1 --- /dev/null +++ b/content/k3s/latest/en/storage/_index.md @@ -0,0 +1,6 @@ +--- +title: "Volumes and Storage" +weight: 30 +--- + +Placeholder From 81cdf7b7ff8601ff932f76a3bed5dcd9c3b20925 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 8 Nov 2019 17:25:33 -0700 Subject: [PATCH 012/103] Copy edit --- content/rke/latest/en/config-options/services/_index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/rke/latest/en/config-options/services/_index.md b/content/rke/latest/en/config-options/services/_index.md index 50c4e9ea3b1..2a629e1bc84 100644 --- a/content/rke/latest/en/config-options/services/_index.md +++ b/content/rke/latest/en/config-options/services/_index.md @@ -103,14 +103,16 @@ RKE supports the following options for the `kubelet` service: - **Cluster Domain** (`cluster_domain`) - The [base domain](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/) for the cluster. All services and DNS records created on the cluster. By default, the domain is set to `cluster.local`. - **Cluster DNS Server** (`cluster_dns_server`) - The IP address assigned to the DNS service endpoint within the cluster. DNS queries will be sent to this IP address which is used by KubeDNS. The default value for this option is `10.43.0.10` - **Fail if Swap is On** (`fail_swap_on`) - In Kubernetes, the default behavior for the kubelet is to **fail** if swap is enabled on the node. RKE does **not** follow this default and allows deployments on nodes with swap enabled. By default, the value is `false`. If you'd like to revert to the default kubelet behavior, set this option to `true`. -- **Generate Serving Certificate** (`generate_serving_certificate`) - Generate a certificate signed by the `kube-ca` Certificate Authority for the kubelet to use as serving certificate. The default value for this option is `false`. Before enabling this option, please read [the requirements](#kubelet-serving-certificate-requirements) +- **Generate Serving Certificate** (`generate_serving_certificate`) - Generate a certificate signed by the `kube-ca` Certificate Authority for the kubelet to use as a server certificate. The default value for this option is `false`. Before enabling this option, please read [the requirements](#kubelet-serving-certificate-requirements) -### Kubelet serving certificate requirements +### Kubelet Serving Certificate Requirements If `hostname_override` is configured for one or more nodes in `cluster.yml`, please make sure the correct IP address is configured in `address` (and the internal address in `internal_address`) to make sure the generated certificate contains the correct IP address(es). An example of an error situation is an EC2 instance where the the public IP address is configured in `address`, and `hostname_override` is used, the connection between `kube-apiserver` and `kubelet` will fail because the `kubelet` will be contacted on the private IP address and the generated certificate will not be valid (the error `x509: certificate is valid for value_in_address, not private_ip` will be seen). The resolution is to provide the internal IP address in `internal_address`. +For more information on host overrides, refer to the [node configuration page.]({{}}/rke/latest/en/config-options/nodes/#overriding-the-hostname) + ## Kubernetes Scheduler The [Kubernetes Scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) service is responsible for scheduling cluster workloads based on various configurations, metrics, resource requirements and workload-specific requirements. From 424140b7603495bab340b29513472277f92d9d22 Mon Sep 17 00:00:00 2001 From: Alexey Dushechkin Date: Sun, 10 Nov 2019 19:14:36 +0700 Subject: [PATCH 013/103] Minor typo Raspian -> Raspbian. https://www.raspberrypi.org/downloads/raspbian/. --- content/k3s/latest/en/installation/node-requirements/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index 565f6af8ba3..b52919459a8 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -16,7 +16,7 @@ k3s should run on just about any flavor of Linux. However, k3s is tested on the * Ubuntu 16.04 (amd64) * Ubuntu 18.04 (amd64) -* Raspian Buster (armhf) +* Raspbian Buster (armhf) ## Hardware From 87f68bbb3c3a7e57f8b37bd4e5492cdba4925dc7 Mon Sep 17 00:00:00 2001 From: Bill Maxwell Date: Fri, 8 Nov 2019 14:59:25 -0700 Subject: [PATCH 014/103] call out we only support sp initiated logins --- .../v2.x/en/admin-settings/authentication/azure-ad/_index.md | 2 ++ .../v2.x/en/admin-settings/authentication/okta/_index.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/content/rancher/v2.x/en/admin-settings/authentication/azure-ad/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/azure-ad/_index.md index 34f2f545d68..b4879220c29 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/azure-ad/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/azure-ad/_index.md @@ -9,6 +9,8 @@ _Available as of v2.0.3_ If you have an instance of Active Directory (AD) hosted in Azure, you can configure Rancher to allow your users to log in using their AD accounts. Configuration of Azure AD external authentication requires you to make configurations in both Azure and Rancher. +>**Note:** Azure AD integration only supports Service Provider initiated logins. + >**Prerequisite:** Have an instance of Azure AD configured. >**Note:** Most of this procedure takes place from the [Microsoft Azure Portal](https://portal.azure.com/). diff --git a/content/rancher/v2.x/en/admin-settings/authentication/okta/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/okta/_index.md index b0af27f7f12..517cd8f6975 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/okta/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/okta/_index.md @@ -7,6 +7,8 @@ _Available as of v2.2.0_ If your organization uses Okta Identity Provider (IdP) for user authentication, you can configure Rancher to allow your users to log in using their IdP credentials. +>**Note:** Okta integration only supports Service Provider initiated logins. + ## Prerequisites In Okta, create a SAML Application with the settings below. See the [Okta documentation](https://developer.okta.com/standards/SAML/setting_up_a_saml_application_in_okta) for help. From 6da6e0671eba14e9782b9329d34510e567418820 Mon Sep 17 00:00:00 2001 From: Tejeev Date: Fri, 8 Nov 2019 20:32:59 +0000 Subject: [PATCH 015/103] There is no import yaml button at the cluster level --- .../tools/istio/setup/deploy-workloads/_index.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads/_index.md index b0679a6ca1d..38bb20f588a 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/deploy-workloads/_index.md @@ -15,9 +15,10 @@ Wait a few minutes for the workload to upgrade to have the istio sidecar. Click Next we add the Kubernetes resources for the sample deployments and services for the BookInfo app in Istio's documentation. -1. Go to the cluster view and click **Import YAML.** +1. Go to the project inside the cluster you want to deploy the workload on. +1. In Workloads, click **Import YAML.** 1. Copy the below resources into the form. -1. Click **Import.** +1. Click **Import.** This will set up the following sample resources from Istio's example BookInfo app: @@ -318,4 +319,4 @@ spec: --- ``` -### [Next: Set up the Istio Gateway]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway) \ No newline at end of file +### [Next: Set up the Istio Gateway]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/gateway) From b875447404f33df9919c3fd773de9c2bc0c1b70a Mon Sep 17 00:00:00 2001 From: Eamon Bauman Date: Sun, 10 Nov 2019 11:16:28 -0500 Subject: [PATCH 016/103] Added clarity for objectClass fields in LDAP providers (#1980) * Made clear the objectClass field requirements * Made clear requirements for the objectClass field --- .../v2.x/en/admin-settings/authentication/ad/_index.md | 4 ++-- .../v2.x/en/admin-settings/authentication/openldap/_index.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md index 516f7b87fac..6725f47f482 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md @@ -74,7 +74,7 @@ The table below details the parameters for the user schema section configuration | Parameter | Description | |:--|:--| -| Object Class | The name of the object class used for user objects in your domain. | +| Object Class | The name of the object class used for user objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) | | Username Attribute | The user attribute whose value is suitable as a display name. | | Login Attribute | The attribute whose value matches the username part of credentials entered by your users when logging in to Rancher. If your users authenticate with their UPN (e.g. "jdoe@acme.com") as username then this field must normally be set to `userPrincipalName`. Otherwise for the old, NetBIOS-style logon names (e.g. "jdoe") it's usually `sAMAccountName`. | | User Member Attribute | The attribute containing the groups that a user is a member of. | @@ -93,7 +93,7 @@ The table below details the parameters for the group schema configuration. | Parameter | Description | |:--|:--| -| Object Class | The name of the object class used for group objects in your domain. | +| Object Class | The name of the object class used for group objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) | | Name Attribute | The group attribute whose value is suitable for a display name. | | Group Member User Attribute | The name of the **user attribute** whose format matches the group members in the `Group Member Mapping Attribute`. | | Group Member Mapping Attribute | The name of the group attribute containing the members of a group. | diff --git a/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md index 2777c006cac..0ffa2607e3c 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md @@ -75,7 +75,7 @@ The table below details the parameters for the user schema configuration. | Parameter | Description | |:--|:--| -| Object Class | The name of the object class used for user objects in your domain. | +| Object Class | The name of the object class used for user objects in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) | | Username Attribute | The user attribute whose value is suitable as a display name. | | Login Attribute | The attribute whose value matches the username part of credentials entered by your users when logging in to Rancher. This is typically `uid`. | | User Member Attribute | The user attribute containing the Distinguished Name of groups a user is member of. Usually this is one of `memberOf` or `isMemberOf`. | @@ -93,7 +93,7 @@ The table below details the parameters for the group schema configuration. | Parameter | Description | |:--|:--| -| Object Class | The name of the object class used for group entries in your domain. | +| Object Class | The name of the object class used for group entries in your domain. If defined, only specify the name of the object class - *don't* include it in an LDAP wrapper such as &(objectClass=xxxx) | | Name Attribute | The group attribute whose value is suitable for a display name. | | Group Member User Attribute | The name of the **user attribute** whose format matches the group members in the `Group Member Mapping Attribute`. | | Group Member Mapping Attribute | The name of the group attribute containing the members of a group. | From 41d4630a4d9dd743daf35548bcedd66af9332609 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 7 Nov 2019 12:32:18 -0700 Subject: [PATCH 017/103] Update installation requirements for 2.3 --- .../en/installation/requirements/_index.md | 33 +++++-------------- 1 file changed, 9 insertions(+), 24 deletions(-) diff --git a/content/rancher/v2.x/en/installation/requirements/_index.md b/content/rancher/v2.x/en/installation/requirements/_index.md index 50d31733859..97a10e00ff9 100644 --- a/content/rancher/v2.x/en/installation/requirements/_index.md +++ b/content/rancher/v2.x/en/installation/requirements/_index.md @@ -28,33 +28,18 @@ Rancher is tested on the following operating systems and their subsequent non-ma For details on which OS and Docker versions were tested with each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/) -- Ubuntu 16.04 (64-bit x86) -- Docker 17.03.x, 18.06.x, 18.09.x -- Ubuntu 18.04 (64-bit x86) -- Docker 18.06.x, 18.09.x -- Red Hat Enterprise Linux (RHEL)/CentOS 7.6 (64-bit x86) -- RHEL Docker 1.13 -- Oracle Linux 7 update 6* (64-bit x86) -- Docker 17.03.x, 18.06.x, 18.09.x -- RancherOS 1.5.1 (64-bit x86) -- Docker 17.03.x, 18.06.x, 18.09.x -- Windows Server 2019 (64-bit x86) - - Requires Docker Engine - Enterprise Edition (EE) - - Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09 - - Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03 - - Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/) +Operating System | Tested Docker Versions +---------------------|-------------------------- +Ubuntu 16.04 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x +Ubuntu 18.04 (64-bit x86) | Docker 18.06.x, 18.09.x +Red Hat Enterprise Linux (RHEL)/CentOS 7.6 (64-bit x86) | RHEL Docker 1.13 +Oracle Linux 7 update 6* (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x +RancherOS 1.5.4 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x +Windows Server 2019 (64-bit x86) | Requires Docker Engine - Enterprise Edition (EE).** \* Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. This [how-to guide]({{}}/rancher/v2.x/en/installation/options/firewall) shows how to check the default firewall rules and how to open the ports with `firewalld` if necessary. -If you are using RancherOS, make sure you switch the Docker engine to a supported version using: - -``` -# Look up available versions -sudo ros engine list - -# Switch to a supported version -sudo ros engine switch docker-18.09.2 -``` +\** Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09. Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/) If you plan to run Rancher on ARM64, see [Running on ARM64 (Experimental)]({{}}/rancher/v2.x/en/installation/arm64-platform/) From fb507d61927c621bee36afa3f7d1114c691ba98a Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 7 Nov 2019 12:19:26 -0700 Subject: [PATCH 018/103] Clean up headers in RKE node config page --- .../latest/en/config-options/nodes/_index.md | 79 ++++++++++++------- 1 file changed, 52 insertions(+), 27 deletions(-) diff --git a/content/rke/latest/en/config-options/nodes/_index.md b/content/rke/latest/en/config-options/nodes/_index.md index 7d9ee845340..75321c4c6b9 100644 --- a/content/rke/latest/en/config-options/nodes/_index.md +++ b/content/rke/latest/en/config-options/nodes/_index.md @@ -5,6 +5,31 @@ weight: 210 The `nodes` directive is the only required section in the `cluster.yml` file. It's used by RKE to specify cluster node(s), ssh credentials used to access the node(s) and which roles these nodes will be in the Kubernetes cluster. +This section covers the following topics: + +- [Node configuration example](#node-configuration-example) +- [Kubernetes roles](#kubernetes-roles) + - [etcd](#etcd) + - [Controlplane](#controlplane) + - [Worker](#worker) +- [Node options](#node-options) + - [Address](#address) + - [Internal address](#internal-address) + - [Overriding the hostname](#overriding-the-hostname) + - [SSH port](#ssh-port) + - [SSH users](#ssh-users) + - [SSH key path](#ssh-key-path) + - [SSH key](#ssh-key) + - [SSH certificate path](#ssh-certificate-path) + - [SSH certificate](#ssh-certificate) + - [Docker socket](#docker-socket) + - [Labels](#labels) + - [Taints](#taints) + +# Node Configuration Example + +The following example shows node configuration in an example `cluster.yml`: + ```yaml nodes: - address: 1.1.1.1 @@ -49,7 +74,33 @@ nodes: app: ingress ``` -## Node Options +# Kubernetes Roles + +You can specify the list of roles that you want the node to be as part of the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process. + +> **Note:** Prior to v0.1.8, workloads/pods might have run on any nodes with `worker` or `controlplane` roles, but as of v0.1.8, they will only be deployed to any `worker` nodes. + +### etcd + +With this role, the `etcd` container will be run on these nodes. Etcd keeps the state of your cluster and is the most important component in your cluster, single source of truth of your cluster. Although you can run etcd on just one node, it typically takes 3, 5 or more nodes to create an HA configuration. Etcd is a distributed reliable key-value store which stores all Kubernetes state. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **etcd** role is shown below: + +Taint Key | Taint Value | Taint Effect +---------------------------------------|--------------|-------------- +`node-role.kubernetes.io/etcd` | `true` | `NoExecute` + +### Controlplane + +With this role, the stateless components that are used to deploy Kubernetes will run on these nodes. These components are used to run the API server, scheduler, and controllers. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **controlplane** role is shown below: + +Taint Key | Taint Value | Taint Effect +---------------------------------------|--------------|-------------- +`node-role.kubernetes.io/controlplane` | `true` | `NoSchedule` + +### Worker + +With this role, any workloads or pods that are deployed will land on these nodes. + +# Node Options Within each node, there are multiple directives that can be used. @@ -93,32 +144,6 @@ For each node, you can specify the path, i.e. `ssh_cert_path`, for the signed SS Instead of setting the path to the signed SSH certificate, you can alternatively specify the actual certificate, i.e. `ssh_cert`, to be used to connect to the node. -### Kubernetes Roles - -You can specify the list of roles that you want the node to be as part of the Kubernetes cluster. Three roles are supported: `controlplane`, `etcd` and `worker`. Node roles are not mutually exclusive. It's possible to assign any combination of roles to any node. It's also possible to change a node's role using the upgrade process. - -> **Note:** Prior to v0.1.8, workloads/pods might have run on any nodes with `worker` or `controlplane` roles, but as of v0.1.8, they will only be deployed to any `worker` nodes. - -* **etcd** - -With this role, the `etcd` container will be run on these nodes. Etcd keeps the state of your cluster and is the most important component in your cluster, single source of truth of your cluster. Although you can run etcd on just one node, it typically takes 3, 5 or more nodes to create an HA configuration. Etcd is a distributed reliable key-value store which stores all Kubernetes state. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **etcd** role is shown below: - -Taint Key | Taint Value | Taint Effect ----------------------------------------|--------------|-------------- -`node-role.kubernetes.io/etcd` | `true` | `NoExecute` - -* **controlplane** - -With this role, the stateless components that are used to deploy Kubernetes will run on these nodes. These components are used to run the API server, scheduler, and controllers. [Taint set on nodes](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) with the **controlplane** role is shown below: - -Taint Key | Taint Value | Taint Effect ----------------------------------------|--------------|-------------- -`node-role.kubernetes.io/controlplane` | `true` | `NoSchedule` - -* **worker** - -With this role, any workloads or pods that are deployed will land on these nodes. - ### Docker Socket If the Docker socket is different than the default, you can set the `docker_socket`. The default is `/var/run/docker.sock` From 6a95af056e22f90ab7a9cedd422f00ddc612efe3 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 6 Nov 2019 11:20:51 -0700 Subject: [PATCH 019/103] Fix link in authorized cluster endpoint section --- .../v2.x/en/cluster-provisioning/rke-clusters/options/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md index d9972410fd3..8c3cfafb88e 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md @@ -71,7 +71,7 @@ See the [RKE documentation on private registries]({{< baseurl >}}/rke/latest/en/ _Available as of v2.2.0_ -Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher. This is enabled by default, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates. It is recommended to create an FQDN pointing to a load balancer which load balances across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate which will be included in the generated kubeconfig to validate the certificate chain. See the [Kubeconfig Files]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/kubeconfig/) and [API Keys]({{< baseurl >}}/v2.x/en/user-settings/api-keys/#creating-an-api-key) documentation for more information. +Authorized Cluster Endpoint can be used to directly access the Kubernetes API server, without requiring communication through Rancher. This is enabled by default, using the IP of the node with the `controlplane` role and the default Kubernetes self signed certificates. It is recommended to create an FQDN pointing to a load balancer which load balances across your nodes with the `controlplane` role. If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate which will be included in the generated kubeconfig to validate the certificate chain. See the [Kubeconfig Files]({{}}/rancher/v2.x/en/k8s-in-rancher/kubeconfig/) and [API Keys]({{}}/rancher/v2.x/en/user-settings/api-keys/#creating-an-api-key) documentation for more information. ### Advanced Cluster Options From 9c589b6e939a36f35972c086932a4d791f6760d3 Mon Sep 17 00:00:00 2001 From: Denise Date: Sun, 10 Nov 2019 08:32:50 -0800 Subject: [PATCH 020/103] Update _index.md --- .../config-options/cloud-providers/vsphere/_index.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md index 88a70c231fc..bc1ac20712d 100644 --- a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md @@ -32,13 +32,14 @@ When provisioning clusters in Rancher using the [vSphere node driver]({{< baseur ![vsphere-node-driver-cloudprovider]({{< baseurl >}}/img/rancher/vsphere-node-driver-cloudprovider.png) 8. Click on **Edit as YAML** -9. Insert the following structure to the pre-populated cluster YAML. In versions prior to v2.3.0, it has to be defined as a top level field. In versions v2.3.0 and higher, it has to be placed under `rancher_kubernetes_engine_config`. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive. +9. Insert the following structure to the pre-populated cluster YAML. As of Rancher v2.3+, this structure must be placed under `rancher_kubernetes_engine_config`. In versions prior to v2.3, it has to be defined as a top level field. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive. ```yaml - cloud_provider: - name: vsphere - vsphereCloudProvider: - [Insert provider configuration] + rancher_kubernetes_engine_config: # Required as of Rancher v2.3+ + cloud_provider: + name: vsphere + vsphereCloudProvider: + [Insert provider configuration] ``` 10. Configure the **Node Pools** per your requirements while ensuring to use a node template that enables disk UUIDs for the VMs (See [Annex - Enable disk UUIDs for vSphere VMs]). From f3d381d617e015ef7d35f467b365299aa1a5348e Mon Sep 17 00:00:00 2001 From: Tejeev Date: Wed, 6 Nov 2019 18:46:10 +0000 Subject: [PATCH 021/103] Minor clarifications on running overlay --- content/rancher/v2.x/en/troubleshooting/networking/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/troubleshooting/networking/_index.md b/content/rancher/v2.x/en/troubleshooting/networking/_index.md index 0943f53e626..a8c5f64a6a5 100644 --- a/content/rancher/v2.x/en/troubleshooting/networking/_index.md +++ b/content/rancher/v2.x/en/troubleshooting/networking/_index.md @@ -45,7 +45,7 @@ To test the overlay network, you can launch the following `DaemonSet` definition 2. Launch it using `kubectl create -f ds-overlaytest.yml` 3. Wait until `kubectl rollout status ds/overlaytest -w` returns: `daemon set "overlaytest" successfully rolled out`. -4. Run the following command to let each container on every host ping each other (it's a single line command). +4. Run the following command, from the same location, to let each container on every host ping each other (it's a single line bash command). ``` echo "=> Start network overlay test"; kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.nodeName}{"\n"}{end}' | while read spod shost; do kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.status.podIP}{" "}{@.spec.nodeName}{"\n"}{end}' | while read tip thost; do kubectl --request-timeout='10s' exec $spod -- /bin/sh -c "ping -c2 $tip > /dev/null 2>&1"; RC=$?; if [ $RC -ne 0 ]; then echo $shost cannot reach $thost; fi; done; done; echo "=> End network overlay test" From 6427f8aa8033c223a9ab95cad51ad959e35fb246 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Sun, 10 Nov 2019 16:16:25 -0700 Subject: [PATCH 022/103] Add main body to storage _index.md Adds content for the local path provisoner bundled with k3s and Longhorn. Each exaplains the essentials to create a pvc and a pod that utilizes it. --- content/k3s/latest/en/storage/_index.md | 118 +++++++++++++++++++++++- 1 file changed, 117 insertions(+), 1 deletion(-) diff --git a/content/k3s/latest/en/storage/_index.md b/content/k3s/latest/en/storage/_index.md index 0c89f8d4ef1..bec797d67de 100644 --- a/content/k3s/latest/en/storage/_index.md +++ b/content/k3s/latest/en/storage/_index.md @@ -3,4 +3,120 @@ title: "Volumes and Storage" weight: 30 --- -Placeholder +When deploying an application that needs to retain data, you’ll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the application’s pod fails. + +# Local Storage Provider +k3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage). + +Create a hostPath backed persistent volume and a pod to utilize it: + +### pvc.yaml + +``` +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: local-path-pvc + namespace: default +spec: + accessModes: + - ReadWriteOnce + storageClassName: local-path + resources: + requests: + storage: 2Gi +``` + +### pod.yaml + +``` +apiVersion: v1 +kind: Pod +metadata: + name: volume-test + namespace: default +spec: + containers: + - name: volume-test + image: nginx:stable-alpine + imagePullPolicy: IfNotPresent + volumeMounts: + - name: volv + mountPath: /data + ports: + - containerPort: 80 + volumes: + - name: volv + persistentVolumeClaim: + claimName: local-path-pvc +``` + +Apply the yaml `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` + +Confirm the pv and pvc are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. + +# Longhorn + +[comment]: <> (pending change - longhorn may support arm64 and armhf in the future.) + +> **Note:** At this time Longhorn only supports amd64. + +k3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md). + +Apply the longhorn.yaml to install Longhorn. + +``` +kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml +``` + +Longhorn will be installed in the namespace `longhorn-system` + +Before we create a PVC, we will create a storage class for longhorn with this yaml. + +``` +kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/storageclass.yaml +``` + +Now, apply the following yaml to create the pvc and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` + +### pvc.yaml + +``` +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: longhorn-volv-pvc +spec: + accessModes: + - ReadWriteOnce + storageClassName: longhorn + resources: + requests: + storage: 2Gi +``` + +### pod.yaml + +``` +apiVersion: v1 +kind: Pod +metadata: + name: volume-test + namespace: default +spec: + containers: + - name: volume-test + image: nginx:stable-alpine + imagePullPolicy: IfNotPresent + volumeMounts: + - name: volv + mountPath: /data + ports: + - containerPort: 80 + volumes: + - name: volv + persistentVolumeClaim: + claimName: longhorn-volv-pvc +``` + +Confirm the pv and pvc are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. From 0977996e8d90838fa0e6012a2d8695f0592572b5 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Sun, 10 Nov 2019 17:24:06 -0700 Subject: [PATCH 023/103] Misc fixes Fix pv to pv claim, capitzalize PV and PVC in all appropriate areas. Add a period to a line where it was missing. --- content/k3s/latest/en/storage/_index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/k3s/latest/en/storage/_index.md b/content/k3s/latest/en/storage/_index.md index bec797d67de..21021221c3b 100644 --- a/content/k3s/latest/en/storage/_index.md +++ b/content/k3s/latest/en/storage/_index.md @@ -8,7 +8,7 @@ When deploying an application that needs to retain data, you’ll need to create # Local Storage Provider k3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage). -Create a hostPath backed persistent volume and a pod to utilize it: +Create a hostPath backed persistent volume claim and a pod to utilize it: ### pvc.yaml @@ -53,7 +53,7 @@ spec: Apply the yaml `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` -Confirm the pv and pvc are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. +Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. # Longhorn @@ -69,7 +69,7 @@ Apply the longhorn.yaml to install Longhorn. kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml ``` -Longhorn will be installed in the namespace `longhorn-system` +Longhorn will be installed in the namespace `longhorn-system`. Before we create a PVC, we will create a storage class for longhorn with this yaml. @@ -77,7 +77,7 @@ Before we create a PVC, we will create a storage class for longhorn with this ya kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/storageclass.yaml ``` -Now, apply the following yaml to create the pvc and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` +Now, apply the following yaml to create the PVC and pod with `kubectl create -f pvc.yaml` and `kubectl create -f pod.yaml` ### pvc.yaml @@ -119,4 +119,4 @@ spec: claimName: longhorn-volv-pvc ``` -Confirm the pv and pvc are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. +Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The status should be Bound for each. From 373eae6e9060765a95da2e5fc2095f0e777da2f8 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Steenis Date: Mon, 11 Nov 2019 11:49:51 +0100 Subject: [PATCH 024/103] Add section on using GlusterFS volumes --- .../volumes-and-storage/_index.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/_index.md b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/_index.md index 0032cecead6..e8dea8adda3 100644 --- a/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/volumes-and-storage/_index.md @@ -189,6 +189,33 @@ services: - "/sbin/iscsiadm:/sbin/iscsiadm" ``` +## GlusterFS Volumes With Rancher Launched Kubernetes Clusters + +In [Rancher Launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) that store data on GlusterFS volumes, you may experience an issue where pods fail to mount volumes after restarting the `kubelet`. The logging of the `kubelet` will show: `transport endpoint is not connected`. To prevent this from happening, you can configure your cluster to mount the `systemd-run` binary in the `kubelet` container. There are two requirements before you can change the cluster configuration: + +- The node needs to have the `systemd-run` binary installed (this can be checked by using the command `which systemd-run` on each cluster node) +- The `systemd-run` binary needs to be compatible with Debian OS on which the hyperkube image is based (this can be checked using the following command on each cluster node, replacing the image tag with the Kubernetes version you want to use) + +``` +docker run -v /usr/bin/systemd-run:/usr/bin/systemd-run --entrypoint /usr/bin/systemd-run rancher/hyperkube:v1.16.2-rancher1 --version +``` + +>**Note:** +> +>Before updating your Kubernetes YAML to mount the `systemd-run` binary, make sure the `systemd` package is installed on your cluster nodes. If this package isn't installed _before_ the bind mounts are created in your Kubernetes YAML, Docker will automatically create the directories and files on each node and will not allow the package install to succeed. + +``` +services: + kubelet: + extra_binds: + - "/usr/bin/systemd-run:/usr/bin/systemd-run" +``` + +After the cluster has finished provisioning, you can check the `kubelet` container logging to see if the functionality is activated by looking for the following logline: + +``` +Detected OS with systemd +``` ## What's Next? From 83f3ecd221667cb96caf70dbc10a56e4e63e83ea Mon Sep 17 00:00:00 2001 From: David Nuzik <45179589+davidnuzik@users.noreply.github.com> Date: Mon, 11 Nov 2019 10:31:56 -0700 Subject: [PATCH 025/103] Improve k3s navigation / move content around (#1985) * Move airgap from advanced to install - move Air-Gap section in Advanced Options to Install Options -- small tweak to the air-gap content but looks like may need more tweaks later - update Installation Options _index.md so that it now lists "Air-Gap Installation" as an option with hyperlink * Move airgap part 2 (actually del airgap from advanced) Whoopsie. I forgot to remove the Air-Gap section from advanced in my last commit. This simply deletes the Air-Gap section that used to be in the advanced page since we have moved the content to the installation/airgap section. * Move ha backend options from configuration to install - move storage backend options from the Configuration Info page to the Install Options (HA) page -- small tweaks to content, very minor * Move Uninstallation notes from advanced to install - move note about Uninstallation from the Advanced Options page to Install Options main page -- small tweaks to fix content and wording, very minor * Fix and move upgrades content to main Install Options section - In a previous commit, "Upgrades" section under the advanced section was mistakenly moved to the new airgap page This fixes that and moves this content to the main install options section. -- Some content tweaks to make the wording better and fix things up. * Move networking info from Configuration Info to new Networking section - Move networking info from confiuration to networking (new section) * Move networking port info from configuration to networking - Move some networking/security port info from the configuration page to the new networking page. Note: This info is also intentionally already duplicated on the Node Registration page as it has important info on ports that need to be open and accessible. * REMOVE "Building from Source" section completely - Remove the Building from Source section - Add FAQ Q&A about this and link to the k3s README.md where this info will go Note: This was done after some discussion a while back. The Building from Source section is important, but not for the majority of users as release binaries / curl script should be utilized to install k3s. A separate PR will add this content into the main k3s README.md file which the FAQ question that was added references. * Fixes for comments, tweak ha install heading names - Resolves comments asking to change headings in 2 areas of ha install - Removes ambiguity in these areas --- content/k3s/latest/en/advanced/_index.md | 64 --------- content/k3s/latest/en/building/_index.md | 47 ------ content/k3s/latest/en/configuration/_index.md | 135 ------------------ content/k3s/latest/en/faq/_index.md | 4 + content/k3s/latest/en/installation/_index.md | 42 ++++++ .../latest/en/installation/airgap/_index.md | 28 ++++ .../k3s/latest/en/installation/ha/_index.md | 100 ++++++++++++- content/k3s/latest/en/networking/_index.md | 45 ++++++ 8 files changed, 217 insertions(+), 248 deletions(-) delete mode 100644 content/k3s/latest/en/building/_index.md create mode 100644 content/k3s/latest/en/installation/airgap/_index.md create mode 100644 content/k3s/latest/en/networking/_index.md diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 700eb033c04..62628f1e7f9 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -104,70 +104,6 @@ To run the agent only in Docker, use `docker-compose up node`. Alternatively the -e K3S_TOKEN=${NODE_TOKEN} \ --privileged rancher/k3s:vX.Y.Z -Air-Gap Support ---------------- - -k3s supports pre-loading of containerd images by placing them in the `images` directory for the agent before starting, for example: -```sh -sudo mkdir -p /var/lib/rancher/k3s/agent/images/ -sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/ -``` -Images needed for a base install are provided through the releases page, additional images can be created with the `docker save` command. - -Offline Helm charts are served from the `/var/lib/rancher/k3s/server/static` directory, and Helm chart manifests may reference the static files with a `%{KUBERNETES_API}%` templated variable. For example, the default traefik manifest chart installs from `https://%{KUBERNETES_API}%/static/charts/traefik-X.Y.Z.tgz`. - -If networking is completely disabled k3s may not be able to start (ie ethernet unplugged or wifi disconnected), in which case it may be necessary to add a default route. For example: -```sh -sudo ip -c address add 192.168.123.123/24 dev eno1 -sudo ip route add default via 192.168.123.1 -``` - -k3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks. - -Upgrades --------- - -To upgrade k3s from an older version you can re-run the installation script using the same flags, for example: - -```sh -curl -sfL https://get.k3s.io | sh - -``` - -If you want to upgrade to specific version you can run the following command: - -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - -``` - -Or to manually upgrade k3s: - -1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest) -2. Install to an appropriate location (normally `/usr/local/bin/k3s`) -3. Stop the old version -4. Start the new version - -Restarting k3s is supported by the installation script for systemd and openrc. -To restart manually for systemd use: -```sh -sudo systemctl restart k3s -``` - -To restart manually for openrc use: -```sh -sudo service k3s restart -``` - -Upgrading an air-gap environment can be accomplished in the following manner: - -1. Download air-gap images and install if changed -2. Install new k3s binary (from installer or manual download) -3. Restart k3s (if not restarted automatically by installer) - -Uninstalling ------------- - -If you installed k3s with the help of `install.sh` script an uninstall script is generated during installation, which will be created on your server node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). - Hyperkube --------- diff --git a/content/k3s/latest/en/building/_index.md b/content/k3s/latest/en/building/_index.md deleted file mode 100644 index 397010eb218..00000000000 --- a/content/k3s/latest/en/building/_index.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: "Building from Source" -weight: 99999 ---- - -This section provides information on building k3s from source. - -See the [release](https://github.com/rancher/k3s/releases/latest) page for pre-built releases. - -The clone will be much faster on this repo if you do - - git clone --depth 1 https://github.com/rancher/k3s.git - -This repo includes all of Kubernetes history so `--depth 1` will avoid most of that. - -To build the full release binary run `make` and that will create `./dist/artifacts/k3s`. - -Optionally to build the binaries without running linting or building docker images: -```sh -./scripts/download && ./scripts/build && ./scripts/package-cli -``` - -For development, you just need go 1.12 and a sane GOPATH. To compile the binaries run: -```bash -go build -o k3s -go build -o kubectl ./cmd/kubectl -go build -o hyperkube ./vendor/k8s.io/kubernetes/cmd/hyperkube -``` - -This will create the main executable, but it does not include the dependencies like containerd, CNI, -etc. To run a server and agent with all the dependencies for development run the following -helper scripts: -```bash -# Server -./scripts/dev-server.sh - -# Agent -./scripts/dev-agent.sh -``` - - -Kubernetes Source ------------------ - -The source code for Kubernetes is in `vendor/` and the location from which that is copied -is in `./vendor.conf`. Go to the referenced repo/tag and you'll find all the patches applied -to upstream Kubernetes. diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index 6591c51bd83..aefafb94156 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -92,19 +92,6 @@ Accessing Cluster from Outside Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace "localhost" with the IP or name of your k3s server. `kubectl` can now manage your k3s cluster. -Open Ports / Network Security ---------------------------- - -The server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach -other nodes over UDP port 8472. The nodes also need to be able to reach the server on UDP port 8472. This is used for flannel VXLAN. If you don't use flannel -and provide your own custom CNI, then 8472 is not needed by k3s. The node should not listen -on any other port. k3s uses reverse tunneling such that the nodes make outbound connections -to the server and all kubelet traffic runs through that tunnel. - -IMPORTANT. The VXLAN port on nodes should not be exposed to the world, it opens up your -cluster network to accessed by anyone. Run your nodes behind a firewall/security group that -disables access to port 8472. - Node Registration ----------------- @@ -175,34 +162,6 @@ k3s agents can be configured with options `--node-label` and `--node-taint` whic --node-taint key1=value1:NoExecute ``` -Flannel -------- - -Flannel is included by default, if you don't want flannel then run the agent with `--no-flannel` option. - -In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) - -CoreDNS -------- - -CoreDNS is deployed on start of the agent, to disable run the server with the `--no-deploy coredns` option. - -If you don't install CoreDNS you will need to install a cluster DNS provider yourself. - -Traefik -------- - -Traefik is deployed by default when starting the server; to disable it, start the server with the `--no-deploy traefik` option. The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. - -Service Load Balancer ---------------------- - -k3s includes a basic service load balancer that uses available host ports. If you try to create -a load balancer that listens on port 80, for example, it will try to find a free host in the cluster -for port 80. If no port is available the load balancer will stay in Pending. - -To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. - Metrics Server -------------- @@ -214,97 +173,3 @@ this should be edited as appropriate for your architecture. As of this writing m the following images relevant to k3s: `amd64:v0.3.3`, `arm64:v0.3.2`, and `arm:v0.3.2`. Further information on the images provided through gcr.io can be found at https://console.cloud.google.com/gcr/images/google-containers/GLOBAL. -Storage Backends (Experimental) ----------------- - -As of version 0.6.0, k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enhancement depends on the following arguments that can be passed to k3s server: - -* `--storage-endpoint` _value_ - - Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] - -* `--storage-cafile` _value_ - - SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] - -* `--storage-certfile` _value_ - - SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] - -* `--storage-keyfile` _value_ - - SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] - -### MySQL - -To use k3s with MySQL storage backend, you can specify the following for insecure connection: - -``` - --storage-endpoint="mysql://" -``` -By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" -``` - -This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name - -To connect to MySQL securely, you can use the following example: -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" \ - --storage-cafile ca.crt \ - --storage-certfile mysql.crt \ - --storage-keyfile mysql.key -``` -The above command will use these certificates to generate the tls config to communicate with mysql securely. - - -### Postgres - -Connection to postgres can be established using the following command: - -``` - --storage-endpoint="postgres://" -``` - -By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" -``` - -This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq - -To connect to Postgres securely, you can use the following example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" \ - --storage-certfile postgres.crt \ - --storage-keyfile postgres.key \ - --storage-cafile ca.crt -``` - -The above command will use these certificates to generate the tls config to communicate with postgres securely. - -### etcd - -Connection to etcd3 can be established using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" -``` -The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" \ - --storage-cafile ca.crt \ - --storage-certfile etcd.crt \ - --storage-keyfile etcd.key -``` - -The above command will use these certificates to generate the tls config to communicate with etcd securely. diff --git a/content/k3s/latest/en/faq/_index.md b/content/k3s/latest/en/faq/_index.md index 5b55183ed82..1d5cbb5df14 100644 --- a/content/k3s/latest/en/faq/_index.md +++ b/content/k3s/latest/en/faq/_index.md @@ -16,3 +16,7 @@ Simply start k3s server with `--no-deploy=traefik` and deploy your ingress. **Does k3s support Windows?** At this time k3s does not natively support Windows, however we are open to the idea in the future. + +**How can I build from source?** + +Please reference the k3s [README.md](https://github.com/rancher/k3s/blob/master/README.md) with instructions. diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 49c6c466dbe..210b1e8126e 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -14,3 +14,45 @@ This section contains instructions for installing k3s in testing and production * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments. + +* [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) + + Install k3s in an air-gapped environment. High Availability is recommended for production environments. + +### Upgrading + +>**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes. + +To upgrade k3s from an older version you can re-run the installation script using the same flags, for example: + +```sh +curl -sfL https://get.k3s.io | sh - +``` + +If you want to upgrade to specific version you can run the following command: + +```sh +curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - +``` + +Or to manually upgrade k3s: + +1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest) +2. Install to an appropriate location (normally `/usr/local/bin/k3s`) +3. Stop the old version +4. Start the new version + +Restarting k3s is supported by the installation script for systemd and openrc. +To restart manually for systemd use: +```sh +sudo systemctl restart k3s +``` + +To restart manually for openrc use: +```sh +sudo service k3s restart +``` + +### Uninstalling + +If you installed k3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). \ No newline at end of file diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md new file mode 100644 index 00000000000..8d0a91ebaa3 --- /dev/null +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -0,0 +1,28 @@ +--- +title: "Air-Gap Install" +weight: 40 +--- + +k3s supports pre-loading of containerd images by placing them in the `images` directory for the agent before starting, for example: +```sh +sudo mkdir -p /var/lib/rancher/k3s/agent/images/ +sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/ +``` +Images needed for a base install are provided through the releases page, additional images can be created with the `docker save` command. + +Offline Helm charts are served from the `/var/lib/rancher/k3s/server/static` directory, and Helm chart manifests may reference the static files with a `%{KUBERNETES_API}%` templated variable. For example, the default traefik manifest chart installs from `https://%{KUBERNETES_API}%/static/charts/traefik-X.Y.Z.tgz`. + +If networking is completely disabled k3s may not be able to start (ie ethernet unplugged or wifi disconnected), in which case it may be necessary to add a default route. For example: +```sh +sudo ip -c address add 192.168.123.123/24 dev eno1 +sudo ip route add default via 192.168.123.1 +``` + +k3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks. + +Upgrading an air-gap environment can be accomplished in the following manner: + +1. Download air-gap images and install if changed +2. Install new k3s binary (from installer or manual download) +3. Restart k3s (if not restarted automatically by installer) + diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 73f4a3676cc..9f08e206638 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -23,8 +23,8 @@ Installation Outline 2. Create master nodes 3. Join worker nodes -### Create Database -The first step for setting up High Availability (HA) is to create the database for the backend. As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1 +### Create Database for Cluster Datastore +The first step for setting up High Availability (HA) is to create the database for the backend. As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. ### Create Master Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. @@ -57,3 +57,99 @@ curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_TOKEN=XXX s Provide the IP or DNS in place of `` this can be any one master node. k3s automatically handles load balancing the master nodes. +# Cluster Datastore Options + +>**Note:** As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. + +k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enhancement depends on the following arguments that can be passed to k3s server: + +* `--storage-endpoint` _value_ + + Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] + +* `--storage-cafile` _value_ + + SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] + +* `--storage-certfile` _value_ + + SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] + +* `--storage-keyfile` _value_ + + SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] + +### MySQL + +To use k3s with MySQL storage backend, you can specify the following for insecure connection: + +``` + --storage-endpoint="mysql://" +``` +By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. + +To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: + +``` + --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" +``` + +This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name + +To connect to MySQL securely, you can use the following example: +``` + --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" \ + --storage-cafile ca.crt \ + --storage-certfile mysql.crt \ + --storage-keyfile mysql.key +``` +The above command will use these certificates to generate the tls config to communicate with mysql securely. + + +### Postgres + +Connection to postgres can be established using the following command: + +``` + --storage-endpoint="postgres://" +``` + +By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. + +To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: + +``` + --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" +``` + +This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq + +To connect to Postgres securely, you can use the following example: + +``` + --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" \ + --storage-certfile postgres.crt \ + --storage-keyfile postgres.key \ + --storage-cafile ca.crt +``` + +The above command will use these certificates to generate the tls config to communicate with postgres securely. + +### etcd + +Connection to etcd3 can be established using the following command: + +``` + --storage-endpoint="https://127.0.0.1:2379" +``` +The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command: + +``` + --storage-endpoint="https://127.0.0.1:2379" \ + --storage-cafile ca.crt \ + --storage-certfile etcd.crt \ + --storage-keyfile etcd.key +``` + +The above command will use these certificates to generate the tls config to communicate with etcd securely. + diff --git a/content/k3s/latest/en/networking/_index.md b/content/k3s/latest/en/networking/_index.md new file mode 100644 index 00000000000..265fdf601b5 --- /dev/null +++ b/content/k3s/latest/en/networking/_index.md @@ -0,0 +1,45 @@ +--- +title: "Networking" +weight: 35 +--- + +Open Ports / Network Security +--------------------------- + +The server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach +other nodes over UDP port 8472. The nodes also need to be able to reach the server on UDP port 8472. This is used for flannel VXLAN. If you don't use flannel +and provide your own custom CNI, then 8472 is not needed by k3s. The node should not listen +on any other port. k3s uses reverse tunneling such that the nodes make outbound connections +to the server and all kubelet traffic runs through that tunnel. + +IMPORTANT. The VXLAN port on nodes should not be exposed to the world, it opens up your +cluster network to accessed by anyone. Run your nodes behind a firewall/security group that +disables access to port 8472. + +Flannel +------- + +Flannel is included by default, if you don't want flannel then run the agent with `--no-flannel` option. + +In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) + +CoreDNS +------- + +CoreDNS is deployed on start of the agent, to disable run the server with the `--no-deploy coredns` option. + +If you don't install CoreDNS you will need to install a cluster DNS provider yourself. + +Traefik +------- + +Traefik is deployed by default when starting the server; to disable it, start the server with the `--no-deploy traefik` option. The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. + +Service Load Balancer +--------------------- + +k3s includes a basic service load balancer that uses available host ports. If you try to create +a load balancer that listens on port 80, for example, it will try to find a free host in the cluster +for port 80. If no port is available the load balancer will stay in Pending. + +To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. \ No newline at end of file From 85e94ef3ea2ad2828e86d0c454c656aff0536d02 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 10:52:02 -0700 Subject: [PATCH 026/103] Move "Upgrades" from Installation Options section to a new Upgrades section --- content/k3s/latest/en/installation/_index.md | 34 ------------------ content/k3s/latest/en/upgrades/_index.md | 36 ++++++++++++++++++++ 2 files changed, 36 insertions(+), 34 deletions(-) create mode 100644 content/k3s/latest/en/upgrades/_index.md diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 210b1e8126e..ca16b4cd5e8 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -19,40 +19,6 @@ This section contains instructions for installing k3s in testing and production Install k3s in an air-gapped environment. High Availability is recommended for production environments. -### Upgrading - ->**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes. - -To upgrade k3s from an older version you can re-run the installation script using the same flags, for example: - -```sh -curl -sfL https://get.k3s.io | sh - -``` - -If you want to upgrade to specific version you can run the following command: - -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - -``` - -Or to manually upgrade k3s: - -1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest) -2. Install to an appropriate location (normally `/usr/local/bin/k3s`) -3. Stop the old version -4. Start the new version - -Restarting k3s is supported by the installation script for systemd and openrc. -To restart manually for systemd use: -```sh -sudo systemctl restart k3s -``` - -To restart manually for openrc use: -```sh -sudo service k3s restart -``` - ### Uninstalling If you installed k3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). \ No newline at end of file diff --git a/content/k3s/latest/en/upgrades/_index.md b/content/k3s/latest/en/upgrades/_index.md new file mode 100644 index 00000000000..fbad260a38b --- /dev/null +++ b/content/k3s/latest/en/upgrades/_index.md @@ -0,0 +1,36 @@ +--- +title: "Upgrades" +weight: 25 +--- + +>**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes. + +To upgrade k3s from an older version you can re-run the installation script using the same flags, for example: + +```sh +curl -sfL https://get.k3s.io | sh - +``` + +If you want to upgrade to specific version you can run the following command: + +```sh +curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - +``` + +Or to manually upgrade k3s: + +1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest) +2. Install to an appropriate location (normally `/usr/local/bin/k3s`) +3. Stop the old version +4. Start the new version + +Restarting k3s is supported by the installation script for systemd and openrc. +To restart manually for systemd use: +```sh +sudo systemctl restart k3s +``` + +To restart manually for openrc use: +```sh +sudo service k3s restart +``` \ No newline at end of file From fdd5e010705c332ec9c827ad1708553aac16e049 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 14:59:51 -0700 Subject: [PATCH 027/103] Adjust quick-start and install pages regarding downtime - This makes changes to the quick-start and install guides that change wording around HA. - Prevoiusly we only suggested HA for ALL production enviornments this is no longer the case. Single-server is okay if some down time is tolerable. - This commit includes a couple of misc tweaks to fix wording in related areas. --- content/k3s/latest/en/installation/_index.md | 10 +++++----- content/k3s/latest/en/installation/ha/_index.md | 4 ++-- .../k3s/latest/en/installation/single-server/_index.md | 4 +--- content/k3s/latest/en/quick-start/_index.md | 3 ++- 4 files changed, 10 insertions(+), 11 deletions(-) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index ca16b4cd5e8..05f56d45eb5 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -3,22 +3,22 @@ title: "Installation Options" weight: 20 --- -This section contains instructions for installing k3s in testing and production environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing k3s. +This section contains instructions for installing k3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing k3s. ### Installation Options * [Single Master Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install k3s on a single Linux host. Single master installs are recommended for development and test environments, as setup is simple and the cluster doesn't have to be readily available for a user-base. + Install k3s on a single Linux host. Single master installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some down time is acceptible. * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) - Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments. + Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any down time. * [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) - Install k3s in an air-gapped environment. High Availability is recommended for production environments. + Install k3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any down time. ### Uninstalling -If you installed k3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). \ No newline at end of file +If you installed k3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 9f08e206638..d2af22d05c8 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -6,7 +6,7 @@ weight: 30 >**Important:** High-Availability (HA) was introduced in the v0.10.0 release of k3s and is _experimental_. Our v1.0 release plans to support HA in production environments. HA should currently only be used for testing purposes in non-production environments. >**Note:** k3s does not utilize etcd by default so only a 2-node cluster is needed for HA at a minimum. The following will guide you through setting up a 2-node cluster with PostgreSQL. You could optionally add one or more nodes for additional redundancy. In the future we plan to add support for additional database providers. -For production environments, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1 +For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1 # Recommended Architecture ![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v4.svg) @@ -24,7 +24,7 @@ Installation Outline 3. Join worker nodes ### Create Database for Cluster Datastore -The first step for setting up High Availability (HA) is to create the database for the backend. As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. +The first step for setting up High Availability (HA) is to create the database for the backend (cluster datastore). As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. ### Create Master Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 6bd6dd1ca4f..49ebd772060 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -3,9 +3,7 @@ title: "Single Master Install" weight: 20 --- ->**Note:** This section contains information on flags and environment variables used for starting a single-master -(non-HA) k3s cluster. A High-Availability (HA) k3s cluster is required for production. A single server install is -intended only for development and testing environments. +>**Note:** This section contains information on flags and environment variables used for starting a single-master (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. Installation ------------ diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index dba7ccd0253..43af73ff5ed 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -3,7 +3,8 @@ title: "Quick-Start Guide" weight: 10 --- ->**Note:** The intent of this guide is to quickly launch a cluster that you can use to evaluate k3s. This guide is not intended for production environments. Production environments should utilize a High-Availiability solution. The [installation options](../installation) section covers in greater detail how k3s can be setup. +>**Note:** The intent of this guide is to quickly luanch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. +The [installation options](../installation) section covers in greater detail how k3s can be setup. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). From f45ceddfe58dc7c8bc71ac53ae4c964ed07a0b8b Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 16:00:14 -0700 Subject: [PATCH 028/103] Modify quick-start to remove cluster-secret and add to ha install guide - This changes the quick-start guide so we just remove the K3S_CLUSTER_SECRET environemnt variable. It wasn't needed for a simple quick-start. - I changed the ha install guide so that we mention the K3S_CLUSTER_SECRET / --cluster-secret in this area as it would be likely relevant here i.e. for auto-scaling node pools. --- content/k3s/latest/en/installation/ha/_index.md | 12 ++++++++---- content/k3s/latest/en/quick-start/_index.md | 6 +++--- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index d2af22d05c8..bf7edf5bed3 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -33,10 +33,12 @@ On the first machine, run the following command to install k3s and connect it to >**Note:** You may wish to taint the master nodes. They will run the kubelet by default and be scheduleable. You can only add node labels and taints during the install process. If you wish to do this, use the `--node-taint` flag. For example `--node-taint key1=value1:NoExecute` the following examples do not include this flag. +>If your master node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The following examples include this optional flag. + ``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --bootstrap-save +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' --bootstrap-save ``` -Note: You may want to provide the password temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. +Note: You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. On the second machine, run the following command. Since we ran the first node with the `--bootstrap-save` flag the second and any additional machines will now automatically bootstrap HA. @@ -49,14 +51,16 @@ Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get ### Join Worker Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s). -Run the following command to join a worker node to the master nodes. You can get the node-token from any of the servers at `/var/lib/rancher/k3s/server/node-token` +Run the following command to join a worker node to the master nodes. We are leveraging the cluster-secret here. Since our master nodes were set up to use this, so too must any agent nodes. ``` -curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_TOKEN=XXX sh - +curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - ``` Provide the IP or DNS in place of `` this can be any one master node. k3s automatically handles load balancing the master nodes. +Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. + # Cluster Datastore Options >**Note:** As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index 43af73ff5ed..54eb41daa29 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -3,7 +3,7 @@ title: "Quick-Start Guide" weight: 10 --- ->**Note:** The intent of this guide is to quickly luanch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. +>**Note:** The intent of this guide is to quickly launch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. The [installation options](../installation) section covers in greater detail how k3s can be setup. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). @@ -19,10 +19,10 @@ curl -sfL https://get.k3s.io | sh - A kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` and the service is automatically started or restarted. The install script will install k3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`. -To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with `K3S_TOKEN` or `K3S_CLUSTER_SECRET` environment variables. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node: +To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with the `K3S_TOKEN` environment variable. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node: ```bash -curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=XXX sh - +curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh - ``` Note: Each machine must have a unique hostname. If your machines do not have unique hostnames, pass the `K3S_NODE_NAME` environment variable and provide a value with a valid and unique hostname for each node. From 4aa6e141790ba4bb30b1f92924ee0c09b49ba12b Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 16:25:52 -0700 Subject: [PATCH 029/103] Resolve nitpicks / wordage in a couple of areas - node-requirements "set" env var, not "pass" - single-server remove duplicate "specific" word - possibly some other minor tweaks --- content/k3s/latest/en/installation/node-requirements/_index.md | 2 +- content/k3s/latest/en/installation/single-server/_index.md | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index b52919459a8..6ef2d4e3d3e 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -8,7 +8,7 @@ k3s is very lightweight, but has some minimum requirements as outlined below. Whether you're configuring a k3s cluster to run in a single-node or high-availability (HA) setup, each node running k3s should meet the following minimum requirements. You may need more resources to fit your needs. ## Pre-requisites -* Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or `$K3S_NODE_NAME` with a unique name for each node you add to the cluster. +* Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or set `$K3S_NODE_NAME` with a unique name for each node you add to the cluster. ## Operating Systems diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 49ebd772060..942c566f2d6 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -14,8 +14,7 @@ k3s is easy to install. To install the latest version, simply run: curl -sfL https://get.k3s.io | sh - ``` -The install script will attempt to download the latest release. To specify a specific -version for download we can use the `INSTALL_K3S_VERSION` environment variable. For example: +The install script will attempt to download the latest release. To specify a version for download we can use the `INSTALL_K3S_VERSION` environment variable. For example: ```sh curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - ``` From fcb6b782d001160b300a666ea8a193655bffdd06 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 16:46:17 -0700 Subject: [PATCH 030/103] Remove hyperkube - We plan to eventually remove support for hyperkube so we will no longer document here. --- content/k3s/latest/en/advanced/_index.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 62628f1e7f9..6e6f488b7c3 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -104,10 +104,3 @@ To run the agent only in Docker, use `docker-compose up node`. Alternatively the -e K3S_TOKEN=${NODE_TOKEN} \ --privileged rancher/k3s:vX.Y.Z -Hyperkube ---------- - -k3s is bundled in a nice wrapper to remove the majority of the headache of running k8s. If -you don't want that wrapper and just want a smaller k8s distro, the releases includes -the `hyperkube` binary you can use. It's then up to you to know how to use `hyperkube`. If -you want individual binaries you will need to compile them yourself from source. From 503e0f44fd34d2495e06c8f82f5c06bf1c402623 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 17:05:18 -0700 Subject: [PATCH 031/103] Change Installation Options title to Installation --- content/k3s/latest/en/installation/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 05f56d45eb5..292fca0003a 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -1,5 +1,5 @@ --- -title: "Installation Options" +title: "Installation" weight: 20 --- From a51b259cc8cb3363636eb035d748ea585084cf88 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 17:12:51 -0700 Subject: [PATCH 032/103] HA fixes for comments in GH issue - Addresses 2 comments about the HA page: https://github.com/rancher/k3s/issues/1031#issuecomment-552672334 https://github.com/rancher/k3s/issues/1031#issuecomment-552673219 --- content/k3s/latest/en/installation/ha/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index bf7edf5bed3..bd6436ef948 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -13,9 +13,9 @@ For production environments that cannot tolerate down time, we recommend install This image depicts a k3s HA install with two load balancers: * A load balancer to expose workloads to external traffic -* A load balancer to expose the Kubernetes API for worker node registration and admin access via `kubectl` +* A load balancer to expose the Kubernetes API for clients such as kubectl and to expose a stable k3s worker registration endpoint -The HA database shown should be a single endpoint k3s can access such as a load balancer. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. +The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. Installation Outline -------------------- From bc9e3c80f5ee340d5757905df7f03855383c9558 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Mon, 11 Nov 2019 17:44:39 -0700 Subject: [PATCH 033/103] Update PGsql versions - Only support what we have tested 10.7 and 11.5 and remove the "R1" --- content/k3s/latest/en/installation/ha/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index bd6436ef948..183fba1178a 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -6,7 +6,7 @@ weight: 30 >**Important:** High-Availability (HA) was introduced in the v0.10.0 release of k3s and is _experimental_. Our v1.0 release plans to support HA in production environments. HA should currently only be used for testing purposes in non-production environments. >**Note:** k3s does not utilize etcd by default so only a 2-node cluster is needed for HA at a minimum. The following will guide you through setting up a 2-node cluster with PostgreSQL. You could optionally add one or more nodes for additional redundancy. In the future we plan to add support for additional database providers. -For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7-R1 thru 11.5-R1 +For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7 and 11.5 # Recommended Architecture ![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v4.svg) From 8a836f724b921eed490c803f292b6991122292cc Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 11:07:39 -0700 Subject: [PATCH 034/103] Update HA Recommended Architecture image - fix arrows - Fixes the direction of arrows in the iamge - Addresses https://github.com/rancher/k3s/issues/1031#issuecomment-552670275 --- .../k3s/latest/en/installation/ha/_index.md | 2 +- static/img/k3s/k3s-production-setup-v4.svg | 1 - static/img/k3s/k3s-production-setup-v5.svg | 2018 +++++++++++++++++ 3 files changed, 2019 insertions(+), 2 deletions(-) delete mode 100644 static/img/k3s/k3s-production-setup-v4.svg create mode 100644 static/img/k3s/k3s-production-setup-v5.svg diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 183fba1178a..03aed6c65cf 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -9,7 +9,7 @@ weight: 30 For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7 and 11.5 # Recommended Architecture -![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v4.svg) +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v5.svg) This image depicts a k3s HA install with two load balancers: * A load balancer to expose workloads to external traffic diff --git a/static/img/k3s/k3s-production-setup-v4.svg b/static/img/k3s/k3s-production-setup-v4.svg deleted file mode 100644 index c2fa54c57d7..00000000000 --- a/static/img/k3s/k3s-production-setup-v4.svg +++ /dev/null @@ -1 +0,0 @@ -k3s-production-setup-v2 \ No newline at end of file diff --git a/static/img/k3s/k3s-production-setup-v5.svg b/static/img/k3s/k3s-production-setup-v5.svg new file mode 100644 index 00000000000..cb50963c422 --- /dev/null +++ b/static/img/k3s/k3s-production-setup-v5.svg @@ -0,0 +1,2018 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From 20b777634a75a36ba1e875b808f38ca1faee05c1 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 12:16:39 -0700 Subject: [PATCH 035/103] Apply requested fixes - Resolve all four comments posted by Catherine. --- content/k3s/latest/en/installation/_index.md | 6 +++--- content/k3s/latest/en/installation/ha/_index.md | 5 ++++- .../k3s/latest/en/installation/node-requirements/_index.md | 2 +- content/k3s/latest/en/quick-start/_index.md | 2 +- 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 292fca0003a..4d60414d649 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -9,15 +9,15 @@ This section contains instructions for installing k3s in various environments. P * [Single Master Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install k3s on a single Linux host. Single master installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some down time is acceptible. + Install k3s on a single Linux host. Single master installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) - Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any down time. + Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any downtime. * [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) - Install k3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any down time. + Install k3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any downtime. ### Uninstalling diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 03aed6c65cf..7d6f5920a9b 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -38,7 +38,10 @@ On the first machine, run the following command to install k3s and connect it to ``` curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' --bootstrap-save ``` -Note: You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. + +The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. + +Note: You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. On the second machine, run the following command. Since we ran the first node with the `--bootstrap-save` flag the second and any additional machines will now automatically bootstrap HA. diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index 6ef2d4e3d3e..bc2d3bcc814 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -7,7 +7,7 @@ k3s is very lightweight, but has some minimum requirements as outlined below. Whether you're configuring a k3s cluster to run in a single-node or high-availability (HA) setup, each node running k3s should meet the following minimum requirements. You may need more resources to fit your needs. -## Pre-requisites +## Prerequisites * Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or set `$K3S_NODE_NAME` with a unique name for each node you add to the cluster. ## Operating Systems diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index 54eb41daa29..a81b5c72370 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -4,7 +4,7 @@ weight: 10 --- >**Note:** The intent of this guide is to quickly launch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. -The [installation options](../installation) section covers in greater detail how k3s can be setup. +The [installation options](../installation) section covers in greater detail how k3s can be set up. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). From 7a3bdd69f0ea7a8bb6ca178541e106bfda2d8a23 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 14:40:35 -0700 Subject: [PATCH 036/103] Remove metrics-server mention in configuration page - We now includ our own metrics-server and thus standard kubectl top nodes, etc functionality works out of the box. Since this works like standard k8s now with the top command, etc we don't need to document the custom-method to deploy metrics-server so we should simply remove this section. --- content/k3s/latest/en/configuration/_index.md | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index aefafb94156..5084f2c1c4e 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -162,14 +162,3 @@ k3s agents can be configured with options `--node-label` and `--node-taint` whic --node-taint key1=value1:NoExecute ``` -Metrics Server --------------- - -To add functionality for commands such as `k3s kubectl top nodes` metrics-server must be installed, -to install see the instructions located at https://github.com/kubernetes-incubator/metrics-server/. - -**NOTE** : By default the image used in `metrics-server-deployment.yaml` is valid only for **amd64** devices, -this should be edited as appropriate for your architecture. As of this writing metrics-server provides -the following images relevant to k3s: `amd64:v0.3.3`, `arm64:v0.3.2`, and `arm:v0.3.2`. Further information -on the images provided through gcr.io can be found at https://console.cloud.google.com/gcr/images/google-containers/GLOBAL. - From 4ac5a190265bee46bf8e6a9813a9764f9fa835c0 Mon Sep 17 00:00:00 2001 From: Mohamed El Sayed Date: Wed, 13 Nov 2019 00:51:01 +0200 Subject: [PATCH 037/103] Add rke documentaion for secrets encryption (#1884) * Add rke documentaiton for secrets encryption * Update _index.md * Copy edit and add paragraphs to section about encryption and backups --- .../secrets-encryption/_index.md | 174 ++++++++++++++++++ .../en/config-options/services/_index.md | 6 +- 2 files changed, 178 insertions(+), 2 deletions(-) create mode 100644 content/rke/latest/en/config-options/secrets-encryption/_index.md diff --git a/content/rke/latest/en/config-options/secrets-encryption/_index.md b/content/rke/latest/en/config-options/secrets-encryption/_index.md new file mode 100644 index 00000000000..9d447592c0a --- /dev/null +++ b/content/rke/latest/en/config-options/secrets-encryption/_index.md @@ -0,0 +1,174 @@ +--- +title: Encrypting Secret Data at Rest +weight: 230 +--- + +As of version `v0.3.1` RKE adds the support for managing secret data encryption at rest, which is [supported by Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#before-you-begin) since version `v1.13`. + +At-rest data encryption is required for: + +- Compliance requirements +- Additional layer of security +- Reduce security impact of etcd node compromise +- Reduce security impact of etcd backups compromise +- Ability to use external Key Management Systems + +RKE provides users with two paths of configuration to enable at-rest data encryption: + +- Managed at-rest data encryption +- Custom configuration for at-rest data encryption + +Both configuration options can be added during initial cluster provisioning or by updating an exsiting cluster. + +To utilize this feature, a new field `secrets_encryption_config` is added to the [Kubernetes API service configuration]({{}}//rke/latest/en/config-options/services/#kubernetes-api-server). A full custom configuration looks like this: + +```yaml +services: + kube-api: + secrets_encryption_config: + enabled: true + custom_config: + apiVersion: apiserver.config.k8s.io/v1 + kind: EncryptionConfiguration + resources: + - resources: + - secrets + providers: + - aescbc: + keys: + - name: k-fw5hn + secret: RTczRjFDODMwQzAyMDVBREU4NDJBMUZFNDhCNzM5N0I= + - identity: {} + +``` +# Managed At-Rest Data Encryption + +Enabling and disabling at-rest data encryption in Kubernetes is a relatively complex process that requires several steps to be performed by the Kubernetes cluster administrator. The managed configuration aims to reduce this overhead and provides a simple abstraction layer to manage the process. + +### Enable Encryption +Managed at-rest data encryption is disabled by default and can be enabled by using the following configuration: + +```yaml +services: + kube-api: + secrets_encryption_config: + enabled: true +``` +Once enabled, RKE will perform the following [actions](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#encrypting-your-data) to enable at-rest data encryption: + +- Generate a new random 32-bit encryption key +- Generate an encryption provider configuration file using the new key The default [provider](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#providers) used is `aescbc` +- Deploy the provider configuration file to all nodes with `controlplane` role +- Update the `kube-apiserver` container arguments to point to the provider configuration file. +- Restart the `kube-apiserver` container. + +After the `kube-api server` is restarted, data encryption is enabled. However, all existing secrets are still stored in plain text. RKE will [rewrite](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#ensure-all-secrets-are-encrypted) all secrets to ensure encryption is fully in effect. + +### Disable Encryption +To disable encryption, you can either set the `enabled` flag to `false`, or simply remove the `secrets_encryption_config` block entirely from cluster.yml. + +```yaml +services: + kube-api: + secrets_encryption_config: + enabled: false +``` + +Once encryption is disabled in `cluster.yml`, RKE will perform the following [actions](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#encrypting-your-data) to disable encryption in your cluster: + +- Generate a new provider configuration file with the no-encryption `identity{}` provider as the first provider, and the previous `aescbc` set in the second place. This will allow Kubernetes to use the first entry to write the secrets, and the second one to decrypt them. +- Deploy the new provider configuration and restart `kube-apiserver`. +- Rewrite all secrets. This is required because, at this point, new data will be written to disk in plain text, but the existing data is still encrypted using the old provider. By rewriting all secrets, RKE ensures that all stored data is decrypted. +- Update `kube-apiserver` arguments to remove the encryption provider configuration and restart the `kube-apiserver`. +- Remove the provider configuration file. + + +# Key Rotation +Sometimes there is a need to rotate encryption config in your cluster. For example, the key is compromised. There are two ways to rotate the keys: with an RKE CLI command, or by disabling and re-enabling encryption in `cluster.yml`. + +### Rotating Keys with the RKE CLI + +With managed configuration, RKE CLI has the ability to perform the key rotation process documented [here](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#rotating-a-decryption-key) with one command. To perform this operation, the following subcommand is used: +```bash +$ ./rke encrypt rotate-key --help +NAME: + rke encrypt rotate-key - Rotate cluster encryption provider key + +USAGE: + rke encrypt rotate-key [command options] [arguments...] + +OPTIONS: + --config value Specify an alternate cluster YAML file (default: "cluster.yml") [$RKE_CONFIG] + --ssh-agent-auth Use SSH Agent Auth defined by SSH_AUTH_SOCK + --ignore-docker-version Disable Docker version check + +``` +This command will perform the following actions: + +- Generate a new random 32-bit encryption key +- Generate a new provider configuration with the new key as the first provider and the second key as the second provider. When the secrets are rewritten, the first key will be used to encrypt the data on the write operation, while the second key (the old key) will be used to decrypt the stored data during the the read operation +- Deploy the new provider configuration to all `controlplane` nodes and restart the `kube-apiserver` +- Rewrite all secrets. This process will re-encrypt all the secrets with the new key. +- Update the configuration to remove the old key and restart the `kube-apiserver` + +### Rotating Keys by Disabling and Re-enabling Encryption in cluster.yml + +For a cluster with encryption enabled, you can rotate the encryption keys by updating `cluster.yml`. If you enable and re-enable the data encryption in the `cluster.yml`, RKE will not reuse old keys. Instead, it will generate new keys every time, yielding the same result as a key rotation with the RKE CLI. + +# Custom At-Rest Data Encryption Configuration +With managed configuration, RKE provides the user with a very simple way to enable and disable encryption with minimal interaction and configuration. However, it doesn't allow for any customization to the configuration. + +With custom encryption configuration, RKE allows the user to provide their own configuration. Although RKE will help the user to deploy the configuration and rewrite the secrets if needed, it doesn't provide a configuration validation on user's behalf. It's the user responsibility to make sure their configuration is valid. + +>**Warning:** Using invalid Encryption Provider Configuration could cause several issues with your cluster, ranging from crashing the Kubernetes API service, `kube-api`, to completely losing access to encrypted data. + +### Example: Using Custom Encryption Configuration with Amazon KMS + +An example for custom configuration would be enabling an external key management system like [Amazon KMS](https://aws.amazon.com/kms/). The following is an example of the configuration for AWS KMS: + +```yaml + +services: + kube-api: + extra_binds: + - "/var/run/kmsplugin/:/var/run/kmsplugin/" + secrets_encryption_config: + enabled: true + custom_config: + apiVersion: apiserver.config.k8s.io/v1 + kind: EncryptionConfiguration + resources: + - resources: + - secrets + providers: + - kms: + name: aws-encryption-provider + endpoint: unix:///var/run/kmsplugin/socket.sock + cachesize: 1000 + timeout: 3s + - identity: {} +``` + +Documentation for AWS KMS can be found [here](https://github.com/kubernetes-sigs/aws-encryption-provider). When Custom Configuration is set to to enable the AWS KMS provider, you should consider the following points: + +- Since RKE runs the `kube-api` service in a container, it's required that you use the `extra_binds` feature to bind-mount the KMS provider socket location inside the `kube-api` container. +- The AWS KMS provider runs as a pod in the cluster. Therefor, the proper way to enable it is to: + 1. Deploy your cluster with at-rest encryption disabled. + 2. Deploy the KMS pod and make sure it's working correctly. + 3. Update your cluster with the custom encryption configuration to utilize the KMS provider. +- Kube API connects to the KMS provider using a Unix socket. You should configure your KMS deployment to run pods on all `controlplane` nodes in the cluster. +- Your `controlplane` node should be configured with an AMI profile that has access to the KMS key you used in your configuration. + +### How to Prevent Restore Failures after Rotating Keys +It's important to understand that enabling encryption for you cluster means that you can no longer access encrypted data in your etcd database and/or etcd database backups without using your encryption keys. + +The encryption configuration is stored in the cluster state file `cluster.rkestate`, which is decoupled from the etcd backups. For example, in any of the following backup cases, the restore process will fail: + +- The snapshot is taken while encryption is enabled and restored when it's disabled. In this case, the encryption keys are no longer stored in the cluster state. +- The snapshot is taken before the keys are rotated and restore is attempted after. In this case, the old keys used for encryption at the time of the snapshot no longer exist in the cluster state file. + +Therefore, we recommend that when you enable or disable encryption, or when you rotate keys, you should [create a snapshot]({{}}/rke/latest/en/etcd-snapshots/one-time-snapshots/) so that your backup requires the same keys that you have access to. + +This also means you should not rotate the keys during the restore process, because you would lose the encryption keys in `cluster.rkestate`. + +The same applies to the custom configuration use case, however in this case it will depend on the user-provided encryption configuration. diff --git a/content/rke/latest/en/config-options/services/_index.md b/content/rke/latest/en/config-options/services/_index.md index 2a629e1bc84..17af63348bf 100644 --- a/content/rke/latest/en/config-options/services/_index.md +++ b/content/rke/latest/en/config-options/services/_index.md @@ -22,7 +22,7 @@ Kubernetes uses [etcd](https://etcd.io/) as a store for cluster state and data. RKE supports running etcd in a single node mode or in HA cluster mode. It also supports adding and removing etcd nodes to the cluster. -You can enable etcd to [take recurring snapshots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#recurring-snapshots). These snapshots can be used to [restore etcd]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-disaster-recovery). +You can enable etcd to [take recurring snapshots]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#recurring-snapshots). These snapshots can be used to [restore etcd]({{< baseurl >}}/rke/latest/en/etcd-snapshots/#etcd-disaster-recovery). By default, RKE will deploy a new etcd service, but you can also run Kubernetes with an [external etcd service]({{< baseurl >}}/rke/latest/en/config-options/services/external-etcd/). @@ -44,6 +44,8 @@ services: # Enable AlwaysPullImages Admission controller plugin # Available as of v0.2.0 always_pull_images: false + secrets_encryption_config: + enabled: true ``` ### Kubernetes API Server Options @@ -55,7 +57,7 @@ RKE supports the following options for the `kube-api` service : - **Pod Security Policy** (`pod_security_policy`) - An option to enable the [Kubernetes Pod Security Policy](https://kubernetes.io/docs/concepts/policy/pod-security-policy/). By default, we do not enable pod security policies as it is set to `false`. > **Note:** If you set `pod_security_policy` value to `true`, RKE will configure an open policy to allow any pods to work on the cluster. You will need to configure your own policies to fully utilize PSP. - **Always Pull Images** (`always_pull_images`) - Enable `AlwaysPullImages` Admission controller plugin. Enabling `AlwaysPullImages` is a security best practice. It forces Kubernetes to validate the image and pull credentials with the remote image registry. Local image layer cache will still be used, but it does add a small bit of overhead when launching containers to pull and compare image hashes. _Note: Available as of v0.2.0_ - +- **Secrets Encryption Config** (`secrets_encryption_config`) - Manage Kubernetes at-rest data encryption. Documented [here]({{< baseurl >}}//rke/latest/en/config-options/secrets-encryption) ## Kubernetes Controller Manager > **Note for Rancher 2 users** If you are configuring Cluster Options using a [Config File]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/#config-file) when creating [Rancher Launched Kubernetes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/), the names of services should contain underscores only: `kube_controller`. This only applies to Rancher v2.0.5 and v2.0.6. From e6e43cf381fbf11dbcb4a294f2999405b0e9670a Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 18:07:38 -0700 Subject: [PATCH 038/103] Add older-installations for ha prior to v1.0.0 and update main ha page - Add an older-installations page, a child of the Installation main section to address older k3s versions different method for bringing up servers - Modify the main HA install page so that we no longer list anything as experimental -- Also update the page so we remove the Cluster Datastore Options section, compact this into a modified section of step 2 for bring up the master (server) nodes -- Fix any wording, modify text so it's more readable and easier to follow. --- .../k3s/latest/en/installation/ha/_index.md | 221 +++++++++--------- .../older-installations/_index.md | 40 ++++ 2 files changed, 146 insertions(+), 115 deletions(-) create mode 100644 content/k3s/latest/en/installation/older-installations/_index.md diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 7d6f5920a9b..edd185e3e05 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -1,12 +1,13 @@ --- -title: "High Availability (HA) Install (Experimental)" +title: "High Availability (HA) Install" weight: 30 --- ->**Important:** High-Availability (HA) was introduced in the v0.10.0 release of k3s and is _experimental_. Our v1.0 release plans to support HA in production environments. HA should currently only be used for testing purposes in non-production environments. ->**Note:** k3s does not utilize etcd by default so only a 2-node cluster is needed for HA at a minimum. The following will guide you through setting up a 2-node cluster with PostgreSQL. You could optionally add one or more nodes for additional redundancy. In the future we plan to add support for additional database providers. +>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. HA may work on older releases but has limited support. Older versions of k3s bring up master nodes slightly differently. Reference [this]({{< baseurl >}}/k3s/latest/en/installation/older-installations/) page for more information. -For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-node cluster with k3s with an external PostgreSQL database. As of v0.10.0 release (Experimental HA) we are supporting PostgreSQL 10.7 and 11.5 +>k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. + +For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-server cluster with k3s with an external database for the cluster datastore. # Recommended Architecture ![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v5.svg) @@ -19,35 +20,121 @@ The external database shown should be a single endpoint k3s can access. The work Installation Outline -------------------- -1. Create backend database (PostgreSQL) -2. Create master nodes +1. Create backend database (PostgreSQL, MySQL, or etcd) +2. Create server (master) nodes 3. Join worker nodes ### Create Database for Cluster Datastore -The first step for setting up High Availability (HA) is to create the database for the backend (cluster datastore). As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. +You will first need to create the database for the backend (cluster datastore). k3s must have a single endpoint it can reach to talk to the database. + +Here is a list of supported backend databases as of our v1.0.0 release: + +* PostgreSQL 10.7 and 11.5 +* MySQL 5.7 +* etcd 3.3.15 ### Create Master Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. +The sections below will indicate what you need to run (depending on the type of database) to bring the server (master) nodes up. -On the first machine, run the following command to install k3s and connect it to the database. +>**Note:** You may wish to taint the server (master) nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. ->**Note:** You may wish to taint the master nodes. They will run the kubelet by default and be scheduleable. You can only add node labels and taints during the install process. If you wish to do this, use the `--node-taint` flag. For example `--node-taint key1=value1:NoExecute` the following examples do not include this flag. +>If your server (master) node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. ->If your master node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The following examples include this optional flag. + +{{% tabs %}} +{{% tab "PostgreSQL" %}} + +>**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='postgres://'` then the server will attempt to connect with the following: + +> * Connect to `localhost` using the `postgres` user. +> * Use `postgres` as the password. +> * Create a database with the name `kubernetes`. + +> If you specify a database name and it does not exist, then the server will attempt to create it. +> For more information about the Postgres driver DSN, please refer to https://godoc.org/github.com/lib/pq + +### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. ``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' --bootstrap-save -``` - -The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. - -Note: You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. - -On the second machine, run the following command. Since we ran the first node with the `--bootstrap-save` flag the second and any additional machines will now automatically bootstrap HA. +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile postgres.crt \ + --storage-keyfile postgres.key ``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' + +The above command will use these certificates to generate the tls config to communicate with postgres securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + ``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% tab "MySQL" %}} + +>**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='mysql://'` then the server will attempt to connect with the following: + +> * Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user. +> * Use no password. +> * Create a database with the name `kubernetes`. + +> If you specify a database name and it does not exist, then the server will attempt to create it. +> For more information about the MySQL driver DSN, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name + +### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile mysql.crt \ + --storage-keyfile mysql.key + + +``` +The above command will use these certificates to generate the tls config to communicate with mysql securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% tab "etcd" %}} + +### Secure Method + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostname:2379" \ + --cluster-secret='mysecret' \ + --storage-cafile ca.crt \ + --storage-certfile etcd.crt \ + --storage-keyfile etcd.key +``` +The above command will use these certificates to generate the tls config to communicate with etcd securely. + +### Insecure Method +Run the following command on each server to install k3s and connect it to the database insecurely. + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='http://hostname:2379' --cluster-secret='mysecret' +``` + +{{% /tab %}} +{{% /tabs %}} + + +>**Note:** The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. You should use a strong password. +>You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes` @@ -64,99 +151,3 @@ Provide the IP or DNS in place of `` this can be any one master nod Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. -# Cluster Datastore Options - ->**Note:** As of v0.10.0 release (Experimental HA) we are currently supporting PostgreSQL 10.7-R1 thru 11.5-R1. - -k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enhancement depends on the following arguments that can be passed to k3s server: - -* `--storage-endpoint` _value_ - - Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] - -* `--storage-cafile` _value_ - - SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] - -* `--storage-certfile` _value_ - - SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] - -* `--storage-keyfile` _value_ - - SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] - -### MySQL - -To use k3s with MySQL storage backend, you can specify the following for insecure connection: - -``` - --storage-endpoint="mysql://" -``` -By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" -``` - -This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name - -To connect to MySQL securely, you can use the following example: -``` - --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" \ - --storage-cafile ca.crt \ - --storage-certfile mysql.crt \ - --storage-keyfile mysql.key -``` -The above command will use these certificates to generate the tls config to communicate with mysql securely. - - -### Postgres - -Connection to postgres can be established using the following command: - -``` - --storage-endpoint="postgres://" -``` - -By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN. - -To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" -``` - -This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq - -To connect to Postgres securely, you can use the following example: - -``` - --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest" \ - --storage-certfile postgres.crt \ - --storage-keyfile postgres.key \ - --storage-cafile ca.crt -``` - -The above command will use these certificates to generate the tls config to communicate with postgres securely. - -### etcd - -Connection to etcd3 can be established using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" -``` -The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command: - -``` - --storage-endpoint="https://127.0.0.1:2379" \ - --storage-cafile ca.crt \ - --storage-certfile etcd.crt \ - --storage-keyfile etcd.key -``` - -The above command will use these certificates to generate the tls config to communicate with etcd securely. - diff --git a/content/k3s/latest/en/installation/older-installations/_index.md b/content/k3s/latest/en/installation/older-installations/_index.md new file mode 100644 index 00000000000..273566d5cdf --- /dev/null +++ b/content/k3s/latest/en/installation/older-installations/_index.md @@ -0,0 +1,40 @@ +--- +title: "Installations Prior to v1.0.0" +weight: 50 +--- + +>**Note:** Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. + +>In v1.0.0 the method for adding master nodes is easier. If you are running a version of k3s older than v1.0.0 use these instructions for adding additional servers. Otherwise, the process is unchanged such as for joining workers (agents) to the cluster. + +First, create your initial master, for example: + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' --cluster-secret='mysecret' --bootstrap-save +``` + +Then, add additional masters: + +``` +curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' +``` + +Substitute the value for `--storage-endpoint` with different strings for PostgreSQL, MySQL, or etcd accordingly. Below are examples of the `--storage-endpoint` flag for each type of database: + +### PostgreSQL + +``` +--storage-endpoint='postgres://username:password@hostname:5432/dbname' +``` + +### MySQL + +``` +--storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname'' +``` + +### etcd + +``` +--storage-endpoint='https://etcd_hostname:2379' +``` From 44428a2d22f490fba684cbeb511c8d097f4e6d8f Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 18:31:08 -0700 Subject: [PATCH 039/103] Add missing line to etcd tab --- content/k3s/latest/en/installation/ha/_index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index edd185e3e05..450ea638de1 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -112,6 +112,7 @@ curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://usernam {{% tab "etcd" %}} ### Secure Method +Run the following command on each server to install k3s and connect it to the database securely. ``` curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostname:2379" \ From eed95ec66033ea507f60e1adf15bd4a474205b7e Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 18:41:22 -0700 Subject: [PATCH 040/103] Add older-configuration section and put old metrics-server info there instead. - Addresses comments in the PR adding to keep this content but move it. - Added some context to this page and info about how v1.0.0 and newer has the metrics-server by default so this is clear to the user. --- .../older-configuration/_index.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 content/k3s/latest/en/configuration/older-configuration/_index.md diff --git a/content/k3s/latest/en/configuration/older-configuration/_index.md b/content/k3s/latest/en/configuration/older-configuration/_index.md new file mode 100644 index 00000000000..d4d4d76ed13 --- /dev/null +++ b/content/k3s/latest/en/configuration/older-configuration/_index.md @@ -0,0 +1,21 @@ +--- +title: "Configuration Info Prior to v1.0.0" +weight: 10 +--- + +>**Note:** Running k3s v1.0.0 or newer is recommended. + +This page outlines any configuration info that is relevant to versions older than v1.0.0 but was removed or no longer needed in v1.0.0 and newer. + +Metrics Server +-------------- + +>**Note:** k3s v1.0.0 includes the metrics-server by default. + +To add functionality for commands such as `k3s kubectl top nodes` metrics-server must be installed, +to install see the instructions located at https://github.com/kubernetes-incubator/metrics-server/. + +**NOTE** : By default the image used in `metrics-server-deployment.yaml` is valid only for **amd64** devices, +this should be edited as appropriate for your architecture. As of this writing metrics-server provides +the following images relevant to k3s: `amd64:v0.3.3`, `arm64:v0.3.2`, and `arm:v0.3.2`. Further information +on the images provided through gcr.io can be found at https://console.cloud.google.com/gcr/images/google-containers/GLOBAL. \ No newline at end of file From a725da986bc26ef8f351f8f2ac20e89e45e0c06b Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 18:45:25 -0700 Subject: [PATCH 041/103] Add Metrics Server back in to config page but link to older config page - Forgot about one comment Catherine had. I added a link to the older config page on the main config page. --- content/k3s/latest/en/configuration/_index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index 5084f2c1c4e..a5f1b996abf 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -162,3 +162,7 @@ k3s agents can be configured with options `--node-label` and `--node-taint` whic --node-taint key1=value1:NoExecute ``` +Metrics Server +-------------- + +>**Note:** v1.0.0 and newer includes the metrics-server by default. For older versions of k3s, please visit [Configuration Info Prior to v1.0.0]({{< baseurl >}}/k3s/latest/en/configuration/older-configuration/). From 1449325bca9ed4cdb84dd477d24ff30c234c49c0 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 19:08:28 -0700 Subject: [PATCH 042/103] Address comments - Remove the term "master" from everywhere in the docs. k3s servers are servers not "masters" - Remove an uneeded section from the ha install page - Misc small tweaks --- content/k3s/latest/en/installation/_index.md | 4 +-- .../k3s/latest/en/installation/ha/_index.md | 28 ++++++++----------- .../older-installations/_index.md | 6 ++-- .../en/installation/single-server/_index.md | 4 +-- 4 files changed, 19 insertions(+), 23 deletions(-) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 4d60414d649..051965c81b3 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -7,9 +7,9 @@ This section contains instructions for installing k3s in various environments. P ### Installation Options -* [Single Master Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) +* [Single Server Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install k3s on a single Linux host. Single master installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. + Install k3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 450ea638de1..b8ada60b048 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -3,7 +3,7 @@ title: "High Availability (HA) Install" weight: 30 --- ->**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. HA may work on older releases but has limited support. Older versions of k3s bring up master nodes slightly differently. Reference [this]({{< baseurl >}}/k3s/latest/en/installation/older-installations/) page for more information. +>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. HA may work on older releases but has limited support. Older versions of k3s bring up server nodes slightly differently. Reference [this]({{< baseurl >}}/k3s/latest/en/installation/older-installations/) page for more information. >k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. @@ -16,13 +16,13 @@ This image depicts a k3s HA install with two load balancers: * A load balancer to expose workloads to external traffic * A load balancer to expose the Kubernetes API for clients such as kubectl and to expose a stable k3s worker registration endpoint -The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the master nodepool will be auto scaling and thus master nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. +The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the server nodepool will be auto scaling and thus server nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. Installation Outline -------------------- -1. Create backend database (PostgreSQL, MySQL, or etcd) -2. Create server (master) nodes -3. Join worker nodes +1. Create Database for Cluster Datastore (PostgreSQL, MySQL, or etcd) +2. Create Server Nodes +3. Join Worker Nodes ### Create Database for Cluster Datastore You will first need to create the database for the backend (cluster datastore). k3s must have a single endpoint it can reach to talk to the database. @@ -33,13 +33,13 @@ Here is a list of supported backend databases as of our v1.0.0 release: * MySQL 5.7 * etcd 3.3.15 -### Create Master Nodes +### Create Server Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. -The sections below will indicate what you need to run (depending on the type of database) to bring the server (master) nodes up. +The sections below will indicate what you need to run (depending on the type of database) to bring the server nodes up. ->**Note:** You may wish to taint the server (master) nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. +>**Note:** You may wish to taint the server nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. ->If your server (master) node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. +>If your server node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. {{% tabs %}} @@ -133,22 +133,18 @@ curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='http://hostname {{% /tab %}} {{% /tabs %}} - ->**Note:** The cluster-secret can contain any Unicode, although you should avoid single and double quotes and make sure the contents are terminal-friendly. You should use a strong password. ->You may want to provide the database password and cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. - Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes` ### Join Worker Nodes Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s). -Run the following command to join a worker node to the master nodes. We are leveraging the cluster-secret here. Since our master nodes were set up to use this, so too must any agent nodes. +Run the following command to join a worker node to the server nodes. We are leveraging the cluster-secret here. Since our server nodes were set up to use this, so too must any agent nodes. ``` -curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - +curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - ``` -Provide the IP or DNS in place of `` this can be any one master node. k3s automatically handles load balancing the master nodes. +Provide the IP or DNS in place of `` this can be any one server node. k3s automatically handles load balancing the server nodes. Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. diff --git a/content/k3s/latest/en/installation/older-installations/_index.md b/content/k3s/latest/en/installation/older-installations/_index.md index 273566d5cdf..6de20c2a286 100644 --- a/content/k3s/latest/en/installation/older-installations/_index.md +++ b/content/k3s/latest/en/installation/older-installations/_index.md @@ -5,15 +5,15 @@ weight: 50 >**Note:** Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. ->In v1.0.0 the method for adding master nodes is easier. If you are running a version of k3s older than v1.0.0 use these instructions for adding additional servers. Otherwise, the process is unchanged such as for joining workers (agents) to the cluster. +>In v1.0.0 the method for adding server nodes is easier. If you are running a version of k3s older than v1.0.0 use these instructions for adding additional servers. Otherwise, the process is unchanged such as for joining workers (agents) to the cluster. -First, create your initial master, for example: +First, create your initial server, for example: ``` curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' --cluster-secret='mysecret' --bootstrap-save ``` -Then, add additional masters: +Then, add additional servers: ``` curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 942c566f2d6..75b4034dd09 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -1,9 +1,9 @@ --- -title: "Single Master Install" +title: "Single Server Install" weight: 20 --- ->**Note:** This section contains information on flags and environment variables used for starting a single-master (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. +>**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. Installation ------------ From 7532a043f9e689c2352fd4e449dd429c49b6053a Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Tue, 12 Nov 2019 19:12:10 -0700 Subject: [PATCH 043/103] Completely remove cli flags from single-server install page - As discussed via slack with Craig, completely remove cli flags from this page -- This content was not helpful. It's the same as running the --help flag we didn't provide any context or additional info. -- At a later date we may cherry pick some of those cli flags and add more detail about them where necessary. --- .../en/installation/single-server/_index.md | 305 ------------------ 1 file changed, 305 deletions(-) diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 75b4034dd09..38d0416a320 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -95,308 +95,3 @@ The full help text for the install script environment variables are as follows: Type of systemd service to create, will default from the k3s exec command if not specified. -Server Options --------------- - -The following information on server options is also available through `k3s server --help` : - -* `--bind-address` _value_ - - k3s bind address (default: localhost) - -* `--https-listen-port` _value_ - - HTTPS listen port (default: 6443) - -* `--http-listen-port` _value_ - - HTTP listen port (for /healthz, HTTPS redirect, and port for TLS terminating LB) (default: 0) - -* `--data-dir` _value_, `-d` _value_ - - Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root - -* `--log` _value_, `-l` _value_ - - Log to file - -* `--cluster-cidr` _value_ - - Network CIDR to use for pod IPs (default: "10.42.0.0/16") - -* `--cluster-secret` _value_ - - Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`] - -* `--service-cidr` _value_ - - Network CIDR to use for services IPs (default: "10.43.0.0/16") - -* `--cluster-dns` _value_ - - Cluster IP for coredns service. Should be in your service-cidr range - -* `--cluster-domain` _value_ - - Cluster Domain (default: "cluster.local") - -* `--no-deploy` _value_ - - Do not deploy packaged components (valid items: coredns, servicelb, traefik) - -* `--write-kubeconfig` _value_, `-o` _value_ - - Write kubeconfig for admin client to this file [$`K3S_KUBECONFIG_OUTPUT`] - -* `--write-kubeconfig-mode` _value_ - - Write kubeconfig with this mode [$`K3S_KUBECONFIG_MODE`] - -* `--tls-san` _value_ - - Add additional hostname or IP as a Subject Alternative Name in the TLS cert - -* `--kube-apiserver-arg` _value_ - - Customized flag for kube-apiserver process - -* `--kube-scheduler-arg` _value_ - - Customized flag for kube-scheduler process - -* `--kube-controller-arg` _value_ - - Customized flag for kube-controller-manager process - -* `--kube-cloud-controller-arg` _value_ - - Customized flag for kube-cloud-controller-manager process - -* `--rootless` - - (experimental) Run rootless - -* `--storage-endpoint` _value_ - - Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$`K3S_STORAGE_ENDPOINT`] - -* `--storage-cafile` _value_ - - SSL Certificate Authority file used to secure storage backend communication [$`K3S_STORAGE_CAFILE`] - -* `--storage-certfile` _value_ - - SSL certification file used to secure storage backend communication [$`K3S_STORAGE_CERTFILE`] - -* `--storage-keyfile` _value_ - - SSL key file used to secure storage backend communication [$`K3S_STORAGE_KEYFILE`] - -* `--advertise-address` _value_ - - IP address that apiserver uses to advertise to members of the cluster - -* `--advertise-port` _value_ - - Port that apiserver uses to advertise to members of the cluster (default: 0) - -* `--disable-scheduler` - - Disable Kubernetes default scheduler - -* `--disable-cloud-controller` - - Disable k3s default cloud controller manager - -* `--disable-network-policy` - - Disable k3s default network policy controller - -* `--default-local-storage-path` _value_ - - Default local storage path for local provisioner storage class - -* `--node-ip` _value_, `-i` _value_ - - (agent) IP address to advertise for node - -* `--node-name` _value_ - - (agent) Node name [$`K3S_NODE_NAME`] - -* `--docker` - - (agent) Use docker instead of containerd - -* `--no-flannel` - - (agent) Disable embedded flannel - -* `--flannel-iface` _value_ - - (agent) Override default flannel interface - -* `--flannel-conf` _value_ - - (agent) (experimental) Override default flannel config file - -* `--flannel-backend` _value_ - - (agent) Specify the flannel backend you would like to use: vxlan (default), ipsec, or wireguard - -* `--container-runtime-endpoint` _value_ - - (agent) Disable embedded containerd and use alternative CRI implementation - -* `--pause-image` _value_ - - (agent) Customized pause image for containerd sandbox - -* `--resolv-conf` _value_ - - (agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`] - -* `--kubelet-arg` _value_ - - (agent) Customized flag for kubelet process - -* `--kube-proxy-arg` _value_ - - (agent) Customized flag for kube-proxy process - -* `--node-label` _value_ - - (agent) Registering kubelet with set of labels - -* `--node-taint` _value_ - - (agent) Registering kubelet with set of taints - -* `--private-registry` _value_ - - (agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") - -* `--node-external-ip` _value_ - - (agent) External IP address to advertise for node - -Agent Options ------------------- - -The following information on agent options is also available through `k3s agent --help` : - -* `--token` _value_, `-t` _value_ - - Token to use for authentication [$`K3S_TOKEN`] - -* `--token-file` _value_ - - Token file to use for authentication [$`K3S_TOKEN_FILE`] - -* `--server` _value_, `-s` _value_ - - Server to connect to [$`K3S_URL`] - -* `--data-dir` _value_, `-d` _value_ - - Folder to hold state (default: "/var/lib/rancher/k3s") - -* `--cluster-secret` _value_ - - Shared secret used to bootstrap a cluster [$`K3S_CLUSTER_SECRET`] - -* `--rootless` - - (experimental) Run rootless - -* `--docker` - - (agent) Use docker instead of containerd - -* `--no-flannel` - - (agent) Disable embedded flannel - -* `--flannel-iface` _value_ - - (agent) Override default flannel interface - -* `--flannel-conf` _value_ - - (agent) (experimental) Override default flannel config file - -* `--node-name` _value_ - - (agent) Node name [$`K3S_NODE_NAME`] - -* `--node-ip` _value_, `-i` _value_ - - (agent) IP address to advertise for node - -* `--container-runtime-endpoint` _value_ - - (agent) Disable embedded containerd and use alternative CRI implementation - -* `--pause-image` _value_ - - (agent) Customized pause image for containerd sandbox - -* `--resolv-conf` _value_ - - (agent) Kubelet resolv.conf file [$`K3S_RESOLV_CONF`] - -* `--kubelet-arg` _value_ - - (agent) Customized flag for kubelet process - -* `--kube-proxy-arg` _value_ - - (agent) Customized flag for kube-proxy process - -* `--node-label` _value_ - - (agent) Registering kubelet with set of labels - -* `--node-taint` _value_ - - (agent) Registering kubelet with set of taints - -* `--private-registry` _value_ - - (agent) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") - -* `--node-external-ip` _value_ - - (agent) External IP address to advertise for node - -Customizing components ----------------------- - -As of v0.3.0 any of the following processes can be customized with extra flags: - -* `--kube-apiserver-arg` _value_ - - (server) [kube-apiserver options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) - -* `--kube-controller-arg` _value_ - - (server) [kube-controller-manager options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) - -* `--kube-scheduler-arg` _value_ - - (server) [kube-scheduler options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) - -* `--kubelet-arg` _value_ - - (agent) [kubelet options](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) - -* `--kube-proxy-arg` _value_ - - (agent) [kube-proxy options](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/) - -Adding extra arguments can be done by passing the following flags to server or agent. -For example to add the following arguments `-v=9` and `log-file=/tmp/kubeapi.log` to the kube-apiserver, you should add the following options to k3s server: - -``` ---kube-apiserver-arg v=9 --kube-apiserver-arg log-file=/tmp/kubeapi.log -``` From 81b586457c40ad778f14dfead755c2060898ec07 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Wed, 13 Nov 2019 11:51:57 -0700 Subject: [PATCH 044/103] Address comments - Address all of Catherine's comments. https://github.com/rancher/docs/pull/1996#pullrequestreview-316448418 --- content/k3s/latest/en/installation/ha/_index.md | 8 ++++---- .../latest/en/installation/older-installations/_index.md | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index b8ada60b048..e2ec3a3701d 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -66,7 +66,7 @@ curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://user ``` -The above command will use these certificates to generate the tls config to communicate with postgres securely. +The above command will use these certificates to generate the TLS config to communicate with PostgreSQL securely. ### Insecure Method Run the following command on each server to install k3s and connect it to the database insecurely. @@ -99,7 +99,7 @@ curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://usernam ``` -The above command will use these certificates to generate the tls config to communicate with mysql securely. +The above command will use these certificates to generate the TLS config to communicate with MySQL securely. ### Insecure Method Run the following command on each server to install k3s and connect it to the database insecurely. @@ -121,7 +121,7 @@ curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostnam --storage-certfile etcd.crt \ --storage-keyfile etcd.key ``` -The above command will use these certificates to generate the tls config to communicate with etcd securely. +The above command will use these certificates to generate the TLS config to communicate with etcd securely. ### Insecure Method Run the following command on each server to install k3s and connect it to the database insecurely. @@ -144,7 +144,7 @@ Run the following command to join a worker node to the server nodes. We are leve curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - ``` -Provide the IP or DNS in place of `` this can be any one server node. k3s automatically handles load balancing the server nodes. +Provide the IP or DNS in place of ``. This can be any one server node. k3s automatically handles load balancing the server nodes. Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. diff --git a/content/k3s/latest/en/installation/older-installations/_index.md b/content/k3s/latest/en/installation/older-installations/_index.md index 6de20c2a286..21525e7875c 100644 --- a/content/k3s/latest/en/installation/older-installations/_index.md +++ b/content/k3s/latest/en/installation/older-installations/_index.md @@ -3,9 +3,9 @@ title: "Installations Prior to v1.0.0" weight: 50 --- ->**Note:** Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. +Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. ->In v1.0.0 the method for adding server nodes is easier. If you are running a version of k3s older than v1.0.0 use these instructions for adding additional servers. Otherwise, the process is unchanged such as for joining workers (agents) to the cluster. +In v1.0.0 the method for adding server nodes is easier. If you are running a version of k3s older than v1.0.0, use these instructions for adding additional servers. Otherwise, the process is unchanged for joining workers. First, create your initial server, for example: From 582a63886a013f807541255a3a8cfcc1bcbb57d5 Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Wed, 13 Nov 2019 15:03:46 -0800 Subject: [PATCH 045/103] broken links --- .../installation/configuration/running-commands/_index.md | 2 +- content/rancher/v2.x/en/admin-settings/_index.md | 6 +++--- content/rancher/v2.x/en/best-practices/containers/_index.md | 4 ++-- content/rancher/v2.x/en/catalog/_index.md | 2 +- .../tools/istio/setup/enable-istio-in-cluster/_index.md | 4 ++-- .../en/cluster-provisioning/rke-clusters/options/_index.md | 6 +++--- content/rancher/v2.x/en/installation/references/_index.md | 2 +- .../rancher/v2.x/en/project-admin/tools/alerts/_index.md | 6 +++--- 8 files changed, 16 insertions(+), 16 deletions(-) diff --git a/content/os/v1.x/en/installation/configuration/running-commands/_index.md b/content/os/v1.x/en/installation/configuration/running-commands/_index.md index 01bc8047343..11b8d44d8be 100644 --- a/content/os/v1.x/en/installation/configuration/running-commands/_index.md +++ b/content/os/v1.x/en/installation/configuration/running-commands/_index.md @@ -12,7 +12,7 @@ runcmd: - echo "test" > /home/rancher/test2 ``` -Commands specified using `runcmd` will be executed within the context of the `console` container. More details on the ordering of commands run in the `console` container can be found [here]({{< baseurl >}}/os/v1.x/en/installation/boot-process/built-in-system-services/#console). +Commands specified using `runcmd` will be executed within the context of the `console` container. ### Running Docker commands diff --git a/content/rancher/v2.x/en/admin-settings/_index.md b/content/rancher/v2.x/en/admin-settings/_index.md index b39b1e6157e..c9b9ef8a460 100644 --- a/content/rancher/v2.x/en/admin-settings/_index.md +++ b/content/rancher/v2.x/en/admin-settings/_index.md @@ -48,14 +48,14 @@ _Available as of v2.3.0_ With this feature, you can upgrade to the latest version of Kubernetes as soon as it is released, without upgrading Rancher. This feature allows you to easily upgrade Kubernetes patch versions (i.e. `v1.15.X`), but not intended to upgrade Kubernetes minor versions (i.e. `v1.X.0`) as Kubernetes tends to deprecate or add APIs between minor versions. -The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) is now located in the [Rancher Kubernetes Metadata]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rke-metadata/). For details on metadata configuration and how to change the Kubernetes version used for provisioning RKE clusters, see [Rancher Kubernetes Metadata]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rke-metadata/). +The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) is now located in the Rancher Kubernetes Metadata. For details on metadata configuration and how to change the Kubernetes version used for provisioning RKE clusters, see [Rancher Kubernetes Metadata]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rke-metadata/). Rancher Kubernetes Metadata contains Kubernetes version information which Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/). -For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rke-metadata/). +For more information on how metadata works and how to configure metadata config, see Rancher Kubernetes Metadata. ## Enabling Experimental Features _Available as of v2.3.0_ -Rancher includes some features that are experimental and disabled by default. Feature flags were introduced to allow you to try these features. For more information, refer to the section about [feature flags.]({{}}/rancher/v2.x/en/admin-settings/feature-flags) \ No newline at end of file +Rancher includes some features that are experimental and disabled by default. Feature flags were introduced to allow you to try these features. For more information, refer to the section about [feature flags.]({{}}/rancher/v2.x/en/admin-settings/feature-flags) diff --git a/content/rancher/v2.x/en/best-practices/containers/_index.md b/content/rancher/v2.x/en/best-practices/containers/_index.md index ce67e87ef6d..410dfb2f437 100644 --- a/content/rancher/v2.x/en/best-practices/containers/_index.md +++ b/content/rancher/v2.x/en/best-practices/containers/_index.md @@ -32,11 +32,11 @@ When possible, use a non-privileged user when running processes within your cont ### Define Resource Limits Apply CPU and memory limits to your pods. This can help manage the resources on your worker nodes and avoid a malfunctioning microservice from impacting other microservices. -In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the [Rancher docs]({{}}/rancher/v2.x/en/project-admin/resource-quotas/). +In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the Rancher docs. When setting resource quotas, if you set anything related to CPU or Memory (i.e. limits or reservations) on a project or namespace, all containers will require a respective CPU or Memory field set during creation. To avoid setting these limits on each and every container during workload creation, a default container resource limit can be specified on the namespace. -The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the [namespace level](https://kubernetes.io/docs/concepts/policy/resource-quotas/). +The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the namespace level. ### Define Resource Requirements You should apply CPU and memory requirements to your pods. This is crucial for informing the scheduler which type of compute node your pod needs to be placed on, and ensuring it does not over-provision that node. In Kubernetes, you can set a resource requirement by defining `resources.requests` in the resource requests field in a pod's container spec. For details, refer to the [Kubernetes docs](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container). diff --git a/content/rancher/v2.x/en/catalog/_index.md b/content/rancher/v2.x/en/catalog/_index.md index c597a720a90..f548860fd9e 100644 --- a/content/rancher/v2.x/en/catalog/_index.md +++ b/content/rancher/v2.x/en/catalog/_index.md @@ -151,7 +151,7 @@ _Available as v2.2.0_ When creating applications that span multiple Kubernetes clusters, a Global DNS entry can be created to route traffic to the endpoints in all of the different clusters. An external DNS server will need be programmed to assign a fully qualified domain name (a.k.a FQDN) to your application. Rancher will use the FQDN you provide and the IP addresses where your application is running to program the DNS. Rancher will gather endpoints from all the Kubernetes clusters running your application and program the DNS. -For more information on how to use this feature, see [Global DNS]({{< baseurl >}}/rancher/v2.x/en/admin-settings/globaldns/). +For more information on how to use this feature, see Global DNS. ## Chart Compatibility with Rancher diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md index 96b5d2590e0..3079aafd2df 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-cluster/_index.md @@ -9,7 +9,7 @@ A Rancher [administrator]({{}}/rancher/v2.x/en/admin-settings/rbac/glob 1. From the **Global** view, navigate to the cluster where you want to enable Istio. 1. Click **Tools > Istio.** -1. Optional: Configure member access and [resource limits]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/config/) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio. +1. Optional: Configure member access and resource limits for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio. 1. Click **Enable**. 1. Click **Save**. @@ -19,4 +19,4 @@ The Istio application, `cluster-istio`, is added as an [application]({{ When Istio is enabled in the cluster, the label for Istio sidecar auto injection,`istio-injection=enabled`, will be automatically added to each new namespace in this cluster. This automatically enables Istio sidecar injection in all new workloads that are deployed in those namespaces. You will need to manually enable Istio in preexisting namespaces and workloads. -### [Next: Enable Istio in a Namespace]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace) \ No newline at end of file +### [Next: Enable Istio in a Namespace]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace) diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md index d9972410fd3..a6ae66d5dba 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md @@ -63,9 +63,9 @@ The registry configuration here is applied during the provisioning of the cluste - **System images** are components needed to maintain the Kubernetes cluster. - **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server. -To deploy workloads that pull images from a private registry, you will need to [set up your own Kubernetes registry]({{}}/rancher/v2.x/en/k8s-in-rancher/registries/) for your project. +To deploy workloads that pull images from a private registry, you will need to set up your own Kubernetes registry for your project. -See the [RKE documentation on private registries]({{< baseurl >}}/rke/latest/en/config-options/private-registries/) for more information on the private registry for components applied during the provisioning of the cluster. +See the RKE documentation on private registries for more information on the private registry for components applied during the provisioning of the cluster. ### Authorized Cluster Endpoint @@ -349,4 +349,4 @@ There are two ways that you can specify an add-on: - [In-line Add-ons]({{}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#in-line-add-ons) - [Referencing YAML Files for Add-ons]({{}}/rke/latest/en/config-options/add-ons/user-defined-add-ons/#referencing-yaml-files-for-add-ons) -For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.]({{}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example) \ No newline at end of file +For an example of how to configure a custom network plug-in by editing the `cluster.yml`, refer to the [RKE documentation.]({{}}/rke/latest/en/config-options/add-ons/network-plugins/custom-network-plugin-example) diff --git a/content/rancher/v2.x/en/installation/references/_index.md b/content/rancher/v2.x/en/installation/references/_index.md index 5c79484b9e1..a39ef0ca510 100644 --- a/content/rancher/v2.x/en/installation/references/_index.md +++ b/content/rancher/v2.x/en/installation/references/_index.md @@ -11,7 +11,7 @@ The following table lists the ports that need to be open to and from nodes that {{< ports-rancher-nodes >}} -**Note** Rancher nodes may also require additional outbound access for any external [authentication provider]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) which is configured (LDAP for example). +**Note** Rancher nodes may also require additional outbound access for any external authentication provider which is configured (LDAP for example). ## Kubernetes Cluster Nodes diff --git a/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md b/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md index fa2eb991c2c..401191ecc10 100644 --- a/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md +++ b/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md @@ -40,7 +40,7 @@ For information on other default alerts, refer to the section on [cluster-level ## Adding Project Alerts ->**Prerequisite:** Before you can receive project alerts, you must [add a notifier]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers). +>**Prerequisite:** Before you can receive project alerts, you must add a notifier. 1. From the **Global** view, navigate to the project that you want to configure project alerts for. Select **Tools > Alerts**. In versions prior to v2.2.0, you can choose **Resources > Alerts**. @@ -165,7 +165,7 @@ If you enable [project monitoring]({{< baseurl >}}/rancher/v2.x/en/project-admin 1. Continue adding more **Alert Rule** to the group. -1. Finally, choose the [notifiers]({{< baseurl >}}//rancher/v2.x/en/cluster-admin/tools/notifiers/) that send you alerts. +1. Finally, choose the notifiers that send you alerts. - You can set up multiple notifiers. - You can change notifier recipients on the fly. @@ -180,4 +180,4 @@ To manage project alerts, browse to the project that alerts you want to manage. - Edit alert settings - Delete unnecessary alerts - Mute firing alerts -- Unmute muted alerts \ No newline at end of file +- Unmute muted alerts From 50f05276aa46a1da90fccb8354df6b2019b6580a Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Wed, 13 Nov 2019 19:17:11 -0700 Subject: [PATCH 046/103] Tweaks to Traefik on Networking page - Modify the Traefik sectoin so it's clear it's the ingress controller - Make it clear what ports are use don the host (so can't use for HostPort or a strange NodePort setting) - Small misc tweaks --- content/k3s/latest/en/networking/_index.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/content/k3s/latest/en/networking/_index.md b/content/k3s/latest/en/networking/_index.md index 265fdf601b5..042090854d2 100644 --- a/content/k3s/latest/en/networking/_index.md +++ b/content/k3s/latest/en/networking/_index.md @@ -19,21 +19,28 @@ disables access to port 8472. Flannel ------- -Flannel is included by default, if you don't want flannel then run the agent with `--no-flannel` option. +Flannel is included by default, if you don't want flannel then run each agent with `--no-flannel` option. In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) CoreDNS ------- -CoreDNS is deployed on start of the agent, to disable run the server with the `--no-deploy coredns` option. +CoreDNS is deployed on start of the agent, to disable run each server with the `--no-deploy coredns` option. If you don't install CoreDNS you will need to install a cluster DNS provider yourself. -Traefik -------- +Traefik Ingress Controller +-------------------------- -Traefik is deployed by default when starting the server; to disable it, start the server with the `--no-deploy traefik` option. The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. +Traefik is deployed by default when starting the server. For more information see [Auto Deploying Manifests]({{< baseurl >}}/k3s/latest/en/configuration/#auto-deploying-manifests). The default config file is found in `/var/lib/rancher/k3s/server/manifests/traefik.yaml` and any changes made to this file will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. + +The Traefik ingress controller will use ports 80, 443, and 8080 on the host (i.e. these will not be usable for HostPort or NodePort). + +You can tweak traefik to meet your needs by setting options in the traefik.yaml file. +Reference the official [Traefik for Helm Configuration Parameters](https://github.com/helm/charts/tree/master/stable/traefik#configuration) readme for more information. + +To disable it, start each server with the `--no-deploy traefik` option. Service Load Balancer --------------------- From 0a2e829bfa51f6b617323f38c7f4b0e62c2dc0aa Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Wed, 13 Nov 2019 19:21:20 -0700 Subject: [PATCH 047/103] fix 1 char typo on ha install page --- content/k3s/latest/en/installation/ha/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index e2ec3a3701d..80030829b81 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -141,7 +141,7 @@ Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/nod Run the following command to join a worker node to the server nodes. We are leveraging the cluster-secret here. Since our server nodes were set up to use this, so too must any agent nodes. ``` -curl -sfL https://get.k3s.io | K3S_URL=https:/:6443 K3S_CLUSTER_SECRET='mysecret' sh - +curl -sfL https://get.k3s.io | K3S_URL=https://:6443 K3S_CLUSTER_SECRET='mysecret' sh - ``` Provide the IP or DNS in place of ``. This can be any one server node. k3s automatically handles load balancing the server nodes. From a27c913ce67ba1199b48f6a2a304b0b1e16f63ee Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Thu, 14 Nov 2019 15:58:12 -0700 Subject: [PATCH 048/103] Remove the "Prior v1.0.0" docs - Based on discussion with the K3s team we decided to remove these completely This is because everything prior to v1.0.0 is not GA and not officially supported. It's strongly recommended to use the v1.0.0 or newer version of K3s. --- content/k3s/latest/en/configuration/_index.md | 4 -- .../older-configuration/_index.md | 21 ---------- .../k3s/latest/en/installation/ha/_index.md | 2 +- .../older-installations/_index.md | 40 ------------------- 4 files changed, 1 insertion(+), 66 deletions(-) delete mode 100644 content/k3s/latest/en/configuration/older-configuration/_index.md delete mode 100644 content/k3s/latest/en/installation/older-installations/_index.md diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index a5f1b996abf..5084f2c1c4e 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -162,7 +162,3 @@ k3s agents can be configured with options `--node-label` and `--node-taint` whic --node-taint key1=value1:NoExecute ``` -Metrics Server --------------- - ->**Note:** v1.0.0 and newer includes the metrics-server by default. For older versions of k3s, please visit [Configuration Info Prior to v1.0.0]({{< baseurl >}}/k3s/latest/en/configuration/older-configuration/). diff --git a/content/k3s/latest/en/configuration/older-configuration/_index.md b/content/k3s/latest/en/configuration/older-configuration/_index.md deleted file mode 100644 index d4d4d76ed13..00000000000 --- a/content/k3s/latest/en/configuration/older-configuration/_index.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: "Configuration Info Prior to v1.0.0" -weight: 10 ---- - ->**Note:** Running k3s v1.0.0 or newer is recommended. - -This page outlines any configuration info that is relevant to versions older than v1.0.0 but was removed or no longer needed in v1.0.0 and newer. - -Metrics Server --------------- - ->**Note:** k3s v1.0.0 includes the metrics-server by default. - -To add functionality for commands such as `k3s kubectl top nodes` metrics-server must be installed, -to install see the instructions located at https://github.com/kubernetes-incubator/metrics-server/. - -**NOTE** : By default the image used in `metrics-server-deployment.yaml` is valid only for **amd64** devices, -this should be edited as appropriate for your architecture. As of this writing metrics-server provides -the following images relevant to k3s: `amd64:v0.3.3`, `arm64:v0.3.2`, and `arm:v0.3.2`. Further information -on the images provided through gcr.io can be found at https://console.cloud.google.com/gcr/images/google-containers/GLOBAL. \ No newline at end of file diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 80030829b81..5b5364420df 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -3,7 +3,7 @@ title: "High Availability (HA) Install" weight: 30 --- ->**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. HA may work on older releases but has limited support. Older versions of k3s bring up server nodes slightly differently. Reference [this]({{< baseurl >}}/k3s/latest/en/installation/older-installations/) page for more information. +>**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. >k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. diff --git a/content/k3s/latest/en/installation/older-installations/_index.md b/content/k3s/latest/en/installation/older-installations/_index.md deleted file mode 100644 index 21525e7875c..00000000000 --- a/content/k3s/latest/en/installation/older-installations/_index.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: "Installations Prior to v1.0.0" -weight: 50 ---- - -Running k3s v1.0.0 or newer is recommended as it has official support for PostgreSQL, MySQL, and etcd. v0.10.0 introduced support for PostgreSQL 10.7 and 11.5 only. Older versions did not have any official support for any external databases. - -In v1.0.0 the method for adding server nodes is easier. If you are running a version of k3s older than v1.0.0, use these instructions for adding additional servers. Otherwise, the process is unchanged for joining workers. - -First, create your initial server, for example: - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' --cluster-secret='mysecret' --bootstrap-save -``` - -Then, add additional servers: - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='value_here' -``` - -Substitute the value for `--storage-endpoint` with different strings for PostgreSQL, MySQL, or etcd accordingly. Below are examples of the `--storage-endpoint` flag for each type of database: - -### PostgreSQL - -``` ---storage-endpoint='postgres://username:password@hostname:5432/dbname' -``` - -### MySQL - -``` ---storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname'' -``` - -### etcd - -``` ---storage-endpoint='https://etcd_hostname:2379' -``` From e74c4645609cf2937015c4add3ccc4aff6159f83 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 09:11:45 -0700 Subject: [PATCH 049/103] Fix faq building question, point to correct url --- content/k3s/latest/en/faq/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/faq/_index.md b/content/k3s/latest/en/faq/_index.md index 1d5cbb5df14..1b4d8b8a57f 100644 --- a/content/k3s/latest/en/faq/_index.md +++ b/content/k3s/latest/en/faq/_index.md @@ -19,4 +19,4 @@ At this time k3s does not natively support Windows, however we are open to the i **How can I build from source?** -Please reference the k3s [README.md](https://github.com/rancher/k3s/blob/master/README.md) with instructions. +Please reference the k3s [BUILDING.md](https://github.com/rancher/k3s/blob/master/BUILDING.md) with instructions. From ef07352df19e410ce7c3dc717d3231e1b9008c7a Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 09:19:36 -0700 Subject: [PATCH 050/103] Update node req port info regarding metrics-server port 10250 - Also change the Networking page to instead link to the Networking seciton of the NOde Requirements page so we don't duplicate information in two areas. --- .../en/installation/node-requirements/_index.md | 2 ++ content/k3s/latest/en/networking/_index.md | 17 ++++------------- 2 files changed, 6 insertions(+), 13 deletions(-) diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index bc2d3bcc814..f3bb9ef42a8 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -34,3 +34,5 @@ k3s performance depends on the performance of the database. To ensure optimal sp The k3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use flannel and provide your own custom CNI, then port 8472 is not needed by k3s. The node should not listen on any other port. k3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. IMPORTANT: The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disabled access to port 8472. + +If you wish to utilize the metrics server, you will need to open port 10250 on each node. diff --git a/content/k3s/latest/en/networking/_index.md b/content/k3s/latest/en/networking/_index.md index 042090854d2..55182d17e63 100644 --- a/content/k3s/latest/en/networking/_index.md +++ b/content/k3s/latest/en/networking/_index.md @@ -3,18 +3,9 @@ title: "Networking" weight: 35 --- -Open Ports / Network Security ---------------------------- - -The server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach -other nodes over UDP port 8472. The nodes also need to be able to reach the server on UDP port 8472. This is used for flannel VXLAN. If you don't use flannel -and provide your own custom CNI, then 8472 is not needed by k3s. The node should not listen -on any other port. k3s uses reverse tunneling such that the nodes make outbound connections -to the server and all kubelet traffic runs through that tunnel. - -IMPORTANT. The VXLAN port on nodes should not be exposed to the world, it opens up your -cluster network to accessed by anyone. Run your nodes behind a firewall/security group that -disables access to port 8472. +Open Ports +---------- +Please reference the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/#networking) page for port information. Flannel ------- @@ -49,4 +40,4 @@ k3s includes a basic service load balancer that uses available host ports. If y a load balancer that listens on port 80, for example, it will try to find a free host in the cluster for port 80. If no port is available the load balancer will stay in Pending. -To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. \ No newline at end of file +To disable the embedded load balancer run the server with the `--no-deploy servicelb` option. This is necessary if you wish to run a different load balancer, such as MetalLB. From 72d10615c682e42c756f742f4212d995607afb8d Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 09:34:00 -0700 Subject: [PATCH 051/103] Capitalize k3s > K3s in all appropriate areas - Don't capitalize if is in code block, url, backticks, etc. --- content/k3s/latest/en/_index.md | 4 +-- content/k3s/latest/en/advanced/_index.md | 8 +++--- content/k3s/latest/en/configuration/_index.md | 26 +++++++++---------- content/k3s/latest/en/faq/_index.md | 14 +++++----- content/k3s/latest/en/installation/_index.md | 10 +++---- .../installation/node-requirements/_index.md | 10 +++---- .../en/installation/single-server/_index.md | 16 ++++++------ content/k3s/latest/en/known-issues/_index.md | 2 +- content/k3s/latest/en/networking/_index.md | 2 +- content/k3s/latest/en/quick-start/_index.md | 8 +++--- content/k3s/latest/en/storage/_index.md | 4 +-- content/k3s/latest/en/upgrades/_index.md | 8 +++--- 12 files changed, 56 insertions(+), 56 deletions(-) diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index 39b66a7c45c..02dae30dcfd 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -18,14 +18,14 @@ Great for: What is this? --- -k3s is intended to be a fully compliant Kubernetes distribution with the following changes: +K3s is intended to be a fully compliant Kubernetes distribution with the following changes: 1. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced with out of tree addons. 2. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. 3. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. 4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. -5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required dependencies +5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). K3s packages required dependencies * containerd * Flannel * CoreDNS diff --git a/content/k3s/latest/en/advanced/_index.md b/content/k3s/latest/en/advanced/_index.md index 6e6f488b7c3..426a40d4afd 100644 --- a/content/k3s/latest/en/advanced/_index.md +++ b/content/k3s/latest/en/advanced/_index.md @@ -5,7 +5,7 @@ aliases: - /k3s/latest/en/running/ --- -This section contains advanced information describing the different ways you can run and manage k3s. +This section contains advanced information describing the different ways you can run and manage K3s. Starting the Server ------------------ @@ -81,10 +81,10 @@ After rebooting: Running in Docker (and docker-compose) ----------------- -[k3d](https://github.com/rancher/k3d) is a utility designed to easily run k3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS. +[k3d](https://github.com/rancher/k3d) is a utility designed to easily run K3s in Docker. It can be installed via the [brew](https://brew.sh/) utility for MacOS. -`rancher/k3s` images are also available to run k3s server and agent from Docker. A `docker-compose.yml` is in the root of the k3s repo that -serves as an example of how to run k3s from Docker. To run from `docker-compose` from this repo run: +`rancher/k3s` images are also available to run K3s server and agent from Docker. A `docker-compose.yml` is in the root of the K3s repo that +serves as an example of how to run K3s from Docker. To run from `docker-compose` from this repo run: docker-compose up --scale node=3 # kubeconfig is written to current dir diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index 5084f2c1c4e..fbeb83ef1af 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -3,7 +3,7 @@ title: "Configuration Info" weight: 50 --- -This section contains information on using k3s with various configurations. +This section contains information on using K3s with various configurations. Auto-Deploying Manifests @@ -12,7 +12,7 @@ Auto-Deploying Manifests Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`. -It is also possible to deploy Helm charts. k3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`): +It is also possible to deploy Helm charts. K3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`): ```yaml apiVersion: helm.cattle.io/v1 @@ -27,7 +27,7 @@ spec: ssl.enabled: "true" ``` -Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because k3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section: +Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because the K3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section: ``` apiVersion: helm.cattle.io/v1 @@ -53,7 +53,7 @@ spec: Also note that besides `set` you can use `valuesContent` in the spec section. And it's okay to use both of them. -k3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions. +K3s versions `<= v0.5.0` used `k3s.cattle.io` for the api group of helmcharts, this has been changed to `helm.cattle.io` for later versions. Using the helm CRD --------------------- @@ -90,7 +90,7 @@ Accessing Cluster from Outside ----------------------------- Copy `/etc/rancher/k3s/k3s.yaml` on your machine located outside the cluster as `~/.kube/config`. Then replace -"localhost" with the IP or name of your k3s server. `kubectl` can now manage your k3s cluster. +"localhost" with the IP or name of your K3s server. `kubectl` can now manage your K3s cluster. Node Registration ----------------- @@ -104,9 +104,9 @@ password file should be recreated for the agent, or the entry removed from the s Containerd and Docker ---------- -k3s includes and defaults to containerd. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag. +K3s includes and defaults to containerd. If you want to use Docker instead of containerd then you simply need to run the agent with the `--docker` flag. -k3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead. +K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead. The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template, the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32 @@ -126,9 +126,9 @@ In short, latest Ubuntu is your best bet for this to work. * **Ports** - When running rootless a new network namespace is created. This means that k3s instance is running with networking - fairly detached from the host. The only way to access services run in k3s from the host is to setup port forwards - to the k3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000. + When running rootless a new network namespace is created. This means that K3s instance is running with networking + fairly detached from the host. The only way to access services run in K3s from the host is to setup port forwards + to the K3s network namespace. We have a controller that will automatically bind 6443 and service port below 1024 to the host with an offset of 10000. That means service port 80 will become 10080 on the host, but 8080 will become 8080 without any offset. @@ -136,7 +136,7 @@ In short, latest Ubuntu is your best bet for this to work. * **Daemon lifecycle** - Once you kill k3s and then start a new instance of k3s it will create a new network namespace, but it doesn't kill the old pods. So you are left + Once you kill K3s and then start a new instance of K3s it will create a new network namespace, but it doesn't kill the old pods. So you are left with a fairly broken setup. This is the main issue at the moment, how to deal with the network namespace. The issue is tracked in https://github.com/rootless-containers/rootlesskit/issues/65 @@ -149,13 +149,13 @@ In short, latest Ubuntu is your best bet for this to work. Just add `--rootless` flag to either server or agent. So run `k3s server --rootless` and then look for the message `Wrote kubeconfig [SOME PATH]` for where your kubeconfig to access you cluster is. Be careful, if you use `-o` to write -the kubeconfig to a different directory it will probably not work. This is because the k3s instance in running in a different +the kubeconfig to a different directory it will probably not work. This is because the K3s instance in running in a different mount namespace. Node Labels and Taints ---------------------- -k3s agents can be configured with options `--node-label` and `--node-taint` which adds set of Labels and Taints to kubelet, the two options only adds labels/taints at registration time, so they can only be added once and not changed after that, an example of options to add new label is: +K3s agents can be configured with options `--node-label` and `--node-taint` which adds set of Labels and Taints to kubelet, the two options only adds labels/taints at registration time, so they can only be added once and not changed after that, an example of options to add new label is: ``` --node-label foo=bar \ --node-label hello=world \ diff --git a/content/k3s/latest/en/faq/_index.md b/content/k3s/latest/en/faq/_index.md index 1b4d8b8a57f..9de75aa8645 100644 --- a/content/k3s/latest/en/faq/_index.md +++ b/content/k3s/latest/en/faq/_index.md @@ -3,20 +3,20 @@ title: FAQ weight: 60 --- -The FAQ is updated periodically and designed to answer the questions our users most frequently ask about k3s. +The FAQ is updated periodically and designed to answer the questions our users most frequently ask about K3s. -**Is k3s a suitable replacement for k8s?** +**Is K3s a suitable replacement for k8s?** -k3s is capable of nearly everything k8s can do. It is just a more lightweight version. See the [main]({{}}/k3s/latest/en/) docs page for more details. +K3s is capable of nearly everything k8s can do. It is just a more lightweight version. See the [main]({{}}/k3s/latest/en/) docs page for more details. **How can I use my own Ingress instead of Traefik?** -Simply start k3s server with `--no-deploy=traefik` and deploy your ingress. +Simply start K3s server with `--no-deploy=traefik` and deploy your ingress. -**Does k3s support Windows?** +**Does K3s support Windows?** -At this time k3s does not natively support Windows, however we are open to the idea in the future. +At this time K3s does not natively support Windows, however we are open to the idea in the future. **How can I build from source?** -Please reference the k3s [BUILDING.md](https://github.com/rancher/k3s/blob/master/BUILDING.md) with instructions. +Please reference the K3s [BUILDING.md](https://github.com/rancher/k3s/blob/master/BUILDING.md) with instructions. diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index 051965c81b3..b1c54edc481 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -3,22 +3,22 @@ title: "Installation" weight: 20 --- -This section contains instructions for installing k3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing k3s. +This section contains instructions for installing K3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing K3s. ### Installation Options * [Single Server Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install k3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. + Install K3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. * [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) - Install k3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any downtime. + Install K3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any downtime. * [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) - Install k3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any downtime. + Install K3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any downtime. ### Uninstalling -If you installed k3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). +If you installed K3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). diff --git a/content/k3s/latest/en/installation/node-requirements/_index.md b/content/k3s/latest/en/installation/node-requirements/_index.md index f3bb9ef42a8..25e2a8b3119 100644 --- a/content/k3s/latest/en/installation/node-requirements/_index.md +++ b/content/k3s/latest/en/installation/node-requirements/_index.md @@ -3,16 +3,16 @@ title: Node Requirements weight: 1 --- -k3s is very lightweight, but has some minimum requirements as outlined below. +K3s is very lightweight, but has some minimum requirements as outlined below. -Whether you're configuring a k3s cluster to run in a single-node or high-availability (HA) setup, each node running k3s should meet the following minimum requirements. You may need more resources to fit your needs. +Whether you're configuring a K3s cluster to run in a single-node or high-availability (HA) setup, each node running K3s should meet the following minimum requirements. You may need more resources to fit your needs. ## Prerequisites * Two nodes cannot have the same hostname. If all your nodes have the same hostname, pass `--node-name` or set `$K3S_NODE_NAME` with a unique name for each node you add to the cluster. ## Operating Systems -k3s should run on just about any flavor of Linux. However, k3s is tested on the following operating systems and their subsequent non-major releases. +K3s should run on just about any flavor of Linux. However, K3s is tested on the following operating systems and their subsequent non-major releases. * Ubuntu 16.04 (amd64) * Ubuntu 18.04 (amd64) @@ -27,11 +27,11 @@ Hardware requirements scale based on the size of your deployments. Minimum recom #### Disks -k3s performance depends on the performance of the database. To ensure optimal speed, we recommend using an SSD when possible. Disk performance will vary on ARM devices utilizing an SD card or eMMC. +K3s performance depends on the performance of the database. To ensure optimal speed, we recommend using an SSD when possible. Disk performance will vary on ARM devices utilizing an SD card or eMMC. ## Networking -The k3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use flannel and provide your own custom CNI, then port 8472 is not needed by k3s. The node should not listen on any other port. k3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. +The K3s server needs port 6443 to be accessible by the nodes. The nodes need to be able to reach other nodes over UDP port 8472 (Flannel VXLAN). If you do not use flannel and provide your own custom CNI, then port 8472 is not needed by K3s. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. IMPORTANT: The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disabled access to port 8472. diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md index 38d0416a320..17ea504c0d1 100644 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ b/content/k3s/latest/en/installation/single-server/_index.md @@ -3,12 +3,12 @@ title: "Single Server Install" weight: 20 --- ->**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) k3s cluster. A High-Availability (HA) k3s cluster is recommended for production environments that cannot tolerate down time. +>**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) K3s cluster. A High-Availability (HA) K3s cluster is recommended for production environments that cannot tolerate down time. Installation ------------ -k3s is easy to install. To install the latest version, simply run: +K3s is easy to install. To install the latest version, simply run: ```sh curl -sfL https://get.k3s.io | sh - @@ -47,7 +47,7 @@ The full help text for the install script environment variables are as follows: - `INSTALL_K3S_SKIP_DOWNLOAD` - If set to true will not download k3s hash or binary. + If set to true will not download K3s hash or binary. - `INSTALL_K3S_SYMLINK` @@ -56,12 +56,12 @@ The full help text for the install script environment variables are as follows: - `INSTALL_K3S_VERSION` - Version of k3s to download from github. Will attempt to download the + Version of K3s to download from github. Will attempt to download the latest version if not specified. - `INSTALL_K3S_BIN_DIR` - Directory to install k3s binary, links, and uninstall script to, or use + Directory to install K3s binary, links, and uninstall script to, or use /usr/local/bin as the default - `INSTALL_K3S_SYSTEMD_DIR` @@ -71,7 +71,7 @@ The full help text for the install script environment variables are as follows: - `INSTALL_K3S_EXEC` or script arguments - Command with flags to use for launching k3s in the systemd service, if + Command with flags to use for launching K3s in the systemd service, if the command is not specified will default to "agent" if `K3S_URL` is set or "server" if not. The final systemd command resolves to a combination of EXEC and script args ($@). @@ -87,11 +87,11 @@ The full help text for the install script environment variables are as follows: - `INSTALL_K3S_NAME` - Name of systemd service to create, will default from the k3s exec command + Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'. - `INSTALL_K3S_TYPE` - Type of systemd service to create, will default from the k3s exec command + Type of systemd service to create, will default from the K3s exec command if not specified. diff --git a/content/k3s/latest/en/known-issues/_index.md b/content/k3s/latest/en/known-issues/_index.md index 7592fcb1572..c79f84bc653 100644 --- a/content/k3s/latest/en/known-issues/_index.md +++ b/content/k3s/latest/en/known-issues/_index.md @@ -6,7 +6,7 @@ The Known Issues are updated periodically and designed to inform you about any i **Snap Docker** -If you plan to use k3s with docker, Docker installed via a snap package is not recommended as it has been known to cause issues running k3s. +If you plan to use K3s with docker, Docker installed via a snap package is not recommended as it has been known to cause issues running K3s. **Iptables** diff --git a/content/k3s/latest/en/networking/_index.md b/content/k3s/latest/en/networking/_index.md index 55182d17e63..4c6ae853e43 100644 --- a/content/k3s/latest/en/networking/_index.md +++ b/content/k3s/latest/en/networking/_index.md @@ -36,7 +36,7 @@ To disable it, start each server with the `--no-deploy traefik` option. Service Load Balancer --------------------- -k3s includes a basic service load balancer that uses available host ports. If you try to create +K3s includes a basic service load balancer that uses available host ports. If you try to create a load balancer that listens on port 80, for example, it will try to find a free host in the cluster for port 80. If no port is available the load balancer will stay in Pending. diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index a81b5c72370..fa3b36c2b72 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -4,20 +4,20 @@ weight: 10 --- >**Note:** The intent of this guide is to quickly launch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. -The [installation options](../installation) section covers in greater detail how k3s can be set up. +The [installation options](../installation) section covers in greater detail how K3s can be set up. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). Install Script -------------- -The k3s `install.sh` script provides a convenient way for installing to systemd or openrc, -to install k3s as a service just run: +The K3s `install.sh` script provides a convenient way for installing to systemd or openrc, +to install K3s as a service just run: ```bash curl -sfL https://get.k3s.io | sh - ``` A kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` and the service is automatically started or restarted. -The install script will install k3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`. +The install script will install K3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`. To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with the `K3S_TOKEN` environment variable. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node: diff --git a/content/k3s/latest/en/storage/_index.md b/content/k3s/latest/en/storage/_index.md index 21021221c3b..a567420905b 100644 --- a/content/k3s/latest/en/storage/_index.md +++ b/content/k3s/latest/en/storage/_index.md @@ -6,7 +6,7 @@ weight: 30 When deploying an application that needs to retain data, you’ll need to create persistent storage. Persistent storage allows you to store application data external from the pod running your application. This storage practice allows you to maintain application data, even if the application’s pod fails. # Local Storage Provider -k3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage). +K3s comes with Rancher's Local Path Provisioner and this enables the ability to create persistent volume claims out of the box using local storage on the respective node. Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage). Create a hostPath backed persistent volume claim and a pod to utilize it: @@ -61,7 +61,7 @@ Confirm the PV and PVC are created. `kubectl get pv` and `kubectl get pvc` The s > **Note:** At this time Longhorn only supports amd64. -k3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md). +K3s supports [Longhorn](https://github.com/longhorn/longhorn). Below we cover a simple example. For more information please reference the official documentation [here](https://github.com/longhorn/longhorn/blob/master/README.md). Apply the longhorn.yaml to install Longhorn. diff --git a/content/k3s/latest/en/upgrades/_index.md b/content/k3s/latest/en/upgrades/_index.md index fbad260a38b..62d3dd47cd8 100644 --- a/content/k3s/latest/en/upgrades/_index.md +++ b/content/k3s/latest/en/upgrades/_index.md @@ -5,7 +5,7 @@ weight: 25 >**Note:** When upgrading, upgrade server nodes first one at a time then any worker nodes. -To upgrade k3s from an older version you can re-run the installation script using the same flags, for example: +To upgrade K3s from an older version you can re-run the installation script using the same flags, for example: ```sh curl -sfL https://get.k3s.io | sh - @@ -17,14 +17,14 @@ If you want to upgrade to specific version you can run the following command: curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - ``` -Or to manually upgrade k3s: +Or to manually upgrade K3s: -1. Download the desired version of k3s from [releases](https://github.com/rancher/k3s/releases/latest) +1. Download the desired version of K3s from [releases](https://github.com/rancher/k3s/releases/latest) 2. Install to an appropriate location (normally `/usr/local/bin/k3s`) 3. Stop the old version 4. Start the new version -Restarting k3s is supported by the installation script for systemd and openrc. +Restarting K3s is supported by the installation script for systemd and openrc. To restart manually for systemd use: ```sh sudo systemctl restart k3s From c372765eba3550cd0a9a06076650926566833ea9 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 11:32:49 -0700 Subject: [PATCH 052/103] Change wording for node labels and taints - Make it more clear that you can still modify / add / remove labels and taints, just not with k3s after node registration You can still change these with the kubectl client, so the wording was changed to make this more clear to avoid confusion. --- content/k3s/latest/en/configuration/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index fbeb83ef1af..66f282ed3c4 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -155,7 +155,7 @@ mount namespace. Node Labels and Taints ---------------------- -K3s agents can be configured with options `--node-label` and `--node-taint` which adds set of Labels and Taints to kubelet, the two options only adds labels/taints at registration time, so they can only be added once and not changed after that, an example of options to add new label is: +K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s again. If you want to change node labels and taints after node registration you should use `kubectl`. Below is an example showing how to add labels and a taint: ``` --node-label foo=bar \ --node-label hello=world \ From af6c148e8daa77488047020f3d6efcf9ef443823 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 11:36:04 -0700 Subject: [PATCH 053/103] remove duplicate "again" --- content/k3s/latest/en/configuration/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index 66f282ed3c4..b1b5ece6f97 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -155,7 +155,7 @@ mount namespace. Node Labels and Taints ---------------------- -K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s again. If you want to change node labels and taints after node registration you should use `kubectl`. Below is an example showing how to add labels and a taint: +K3s agents can be configured with the options `--node-label` and `--node-taint` which adds a label and taint to the kubelet. The two options only add labels and/or taints at registration time, so they can only be added once and not changed after that again by running K3s. If you want to change node labels and taints after node registration you should use `kubectl`. Below is an example showing how to add labels and a taint: ``` --node-label foo=bar \ --node-label hello=world \ From 7c25e3d468bc7502833888bb656967a703bd35f1 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 15 Nov 2019 11:58:46 -0700 Subject: [PATCH 054/103] Document service defaults/admission plugins location (#1999) * Document service defaults/admission plugins location * Update _index.md * Update _index.md * Update _index.md * Update _index.md * Add note about types file and Rancher version --- content/rke/latest/en/config-options/_index.md | 5 +++++ .../en/config-options/services/services-extras/_index.md | 8 +++++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/content/rke/latest/en/config-options/_index.md b/content/rke/latest/en/config-options/_index.md index b942e5e6f3e..f8a920363e6 100644 --- a/content/rke/latest/en/config-options/_index.md +++ b/content/rke/latest/en/config-options/_index.md @@ -67,6 +67,11 @@ kubernetes_version: "v1.11.6-rancher1-1" In case both `kubernetes_version` and [system images]({{< baseurl >}}/rke/latest/en/config-options/system-images/) are defined, the system images configuration will take precedence over `kubernetes_version`. +> **Note:** In RKE, `kubernetes_version` is used to map the version of Kubernetes to the default services, parameters, and options: + +> - For RKE v0.3.0+, the service defaults are located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go). +> - For RKE prior to v0.3.0, the service defaults are located [here](https://github.com/rancher/types/blob/release/v2.2/apis/management.cattle.io/v3/k8s_defaults.go). Note: The version in the path of the service defaults file corresponds to a Rancher version. Therefore, for Rancher v2.1.x, [this file](https://github.com/rancher/types/blob/release/v2.1/apis/management.cattle.io/v3/k8s_defaults.go) should be used. + #### Listing Supported Kubernetes Versions Please refer to the [release notes](https://github.com/rancher/rke/releases) of the RKE version that you are running, to find the list of supported Kubernetes versions as well as the default Kubernetes version. diff --git a/content/rke/latest/en/config-options/services/services-extras/_index.md b/content/rke/latest/en/config-options/services/services-extras/_index.md index 867d6a98fa2..57f623800ab 100644 --- a/content/rke/latest/en/config-options/services/services-extras/_index.md +++ b/content/rke/latest/en/config-options/services/services-extras/_index.md @@ -9,10 +9,16 @@ RKE supports additional service arguments, volume binds and environment variable For any of the Kubernetes services, you can update the `extra_args` to change the existing defaults. -As of `v0.1.3`, using `extra_args` will add new arguments and **override** any existing defaults. For example, if you need to modify the default admission controllers list, you need to include the default list and edit it with your changes so all changes are included. +As of `v0.1.3`, using `extra_args` will add new arguments and **override** any existing defaults. For example, if you need to modify the default admission plugins list, you need to include the default list and edit it with your changes so all changes are included. Prior to `v0.1.3`, using `extra_args` would only add new arguments to the list and there was no ability to change the default list. +All service defaults and parameters are defined per [`kubernetes_version`]({{}}/rke/latest/en/config-options/#kubernetes-version): + +- For RKE v0.3.0+, the service defaults and parameters are defined per [`kubernetes_version`]({{}}/rke/latest/en/config-options/#kubernetes-version). The service defaults are located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go). The default list of admissions plugins is the same for all Kubernetes versions and is located [here](https://github.com/rancher/kontainer-driver-metadata/blob/master/rke/k8s_service_options.go#L11). + +- For RKE prior to v0.3.0, the service defaults and admission plugins are defined per [`kubernetes_version`]({{}}/rke/latest/en/config-options/#kubernetes-version) and located [here](https://github.com/rancher/types/blob/release/v2.2/apis/management.cattle.io/v3/k8s_defaults.go). + ```yaml services: kube-controller: From ef8f25e9ada34083b91428f61231eba537c9ff02 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 16:39:55 -0700 Subject: [PATCH 055/103] Air-Gap Improvements - Add additional steps such as for private registry - Improve readability by clearly outlining steps --- .../latest/en/installation/airgap/_index.md | 70 ++++++++++++++++--- 1 file changed, 59 insertions(+), 11 deletions(-) diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index 8d0a91ebaa3..578db199807 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -3,26 +3,74 @@ title: "Air-Gap Install" weight: 40 --- -k3s supports pre-loading of containerd images by placing them in the `images` directory for the agent before starting, for example: +In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server. + +Installation Outline +-------------------- +1. Prepare Images Directory +2. Create Registry YAML +3. Install K3s + +### Prepare Images Directory +Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. + +Place the tar file in the `images` directory before starting K3s on each node, for example: + ```sh sudo mkdir -p /var/lib/rancher/k3s/agent/images/ sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/ ``` -Images needed for a base install are provided through the releases page, additional images can be created with the `docker save` command. -Offline Helm charts are served from the `/var/lib/rancher/k3s/server/static` directory, and Helm chart manifests may reference the static files with a `%{KUBERNETES_API}%` templated variable. For example, the default traefik manifest chart installs from `https://%{KUBERNETES_API}%/static/charts/traefik-X.Y.Z.tgz`. +### Create Registry YAML +Create the registries.yaml file at `/etc/rancher/k3s/registries.yaml`. This will tell K3s the necessary details to connect to your private registry. +The registries.yaml file should look like this before plugging in the necessary information: -If networking is completely disabled k3s may not be able to start (ie ethernet unplugged or wifi disconnected), in which case it may be necessary to add a default route. For example: -```sh -sudo ip -c address add 192.168.123.123/24 dev eno1 -sudo ip route add default via 192.168.123.1 +``` +--- +mirrors: + customreg: + endpoint: + - "https://ip-to-server:5000" +configs: + customreg: + auth: + username: xxxxxx # this is the registry username + password: xxxxxx # this is the registry password + tls: + cert_file: + key_file: + ca_file: ``` -k3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks. +Note, at this time only secure registries are supported with K3s (SSL with custom CA) + +### Install K3s + +Obtain the K3s binary from the [releases](https://github.com/rancher/k3s/releases) page, matching the same version used to get the airgap images tar. +Also obtain the K3s install script at https://get.k3s.io + +Place the binary in `/usr/local/bin` on each node. +Place the install script anywhere on each node, name it `install.sh`. + +Install K3s on each node. The example below shows how to do this for a server or an agent (worker): + +``` +# K3s Server +INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh + +# K3s Agent +INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken ./install.sh +``` + +Note, take care to ensure you replace `myserver` with the IP or valid DNS of the server and replace `mynodetoken` with the node-token from the server. +The node-token is on the server at `/var/lib/rancher/k3s/server/node-token` + + +>**Note:** K3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks. Upgrading an air-gap environment can be accomplished in the following manner: -1. Download air-gap images and install if changed -2. Install new k3s binary (from installer or manual download) -3. Restart k3s (if not restarted automatically by installer) +1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file. +2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same enviornment variables. +3. Restart the K3s service (if not restarted automatically by installer). From 945a5bc419e82d7ad613df2228ec9883ce95f7b8 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Fri, 15 Nov 2019 16:43:16 -0700 Subject: [PATCH 056/103] Fix upgrading section make h1 header - h1 header separates out from the steps for normal install - easier to read and find special upgrading section for airgap --- content/k3s/latest/en/installation/airgap/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index 578db199807..d8b26320885 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -68,6 +68,8 @@ The node-token is on the server at `/var/lib/rancher/k3s/server/node-token` >**Note:** K3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks. +# Upgrading + Upgrading an air-gap environment can be accomplished in the following manner: 1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file. From ea2090300d7950fed3aa0643d8c88c1ea742ce43 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 14 Nov 2019 15:03:12 -0700 Subject: [PATCH 057/103] Fix error in vSphere docs --- .../latest/en/config-options/cloud-providers/vsphere/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md index 9b4884c19c5..2f37d14edb0 100644 --- a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md @@ -182,7 +182,7 @@ cloud_provider: datacenters: eu-west-1 workspace: server: vc.example.com - folder: kubernetes + folder: vm/kubernetes default-datastore: ds-1 datacenter: eu-west-1 From a170554bdffcf236c1b162237945d88cdcedbd7b Mon Sep 17 00:00:00 2001 From: Jonas Janz Date: Thu, 14 Nov 2019 16:52:43 +0100 Subject: [PATCH 058/103] fix typo in example-repos fixing a small typo in the docs --- .../v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md index 48adfeb3a6f..be9733f856a 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/pipelines/example-repos/_index.md @@ -11,7 +11,7 @@ Rancher ships with several example repositories that you can use to familiarize - Maven - php -> **Note**: The example repositories are only available if you have not [configured a version control provider]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/pipelines). +> **Note:** The example repositories are only available if you have not [configured a version control provider]({{< baseurl >}}/rancher/v2.x/en/project-admin/tools/pipelines). ## Configure Repositories From e5dcee0056c4eb8058fb9e8381cde388cf80ea57 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Mon, 11 Nov 2019 17:55:26 -0700 Subject: [PATCH 059/103] Finish updating installation requirements for Rancher 2.3 --- .../v2.x/en/installation/requirements/_index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/rancher/v2.x/en/installation/requirements/_index.md b/content/rancher/v2.x/en/installation/requirements/_index.md index 97a10e00ff9..47c1f17e9a7 100644 --- a/content/rancher/v2.x/en/installation/requirements/_index.md +++ b/content/rancher/v2.x/en/installation/requirements/_index.md @@ -30,11 +30,11 @@ For details on which OS and Docker versions were tested with each Rancher versio Operating System | Tested Docker Versions ---------------------|-------------------------- -Ubuntu 16.04 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x -Ubuntu 18.04 (64-bit x86) | Docker 18.06.x, 18.09.x -Red Hat Enterprise Linux (RHEL)/CentOS 7.6 (64-bit x86) | RHEL Docker 1.13 -Oracle Linux 7 update 6* (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x -RancherOS 1.5.4 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x +Ubuntu 16.04 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x +Ubuntu 18.04 (64-bit x86) | Docker 18.06.x, 18.09.x, 19.03.x +Red Hat Enterprise Linux (RHEL)/CentOS 7.7 (64-bit x86) | RHEL Docker 1.13 +Oracle Linux 7 update 6* (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x +RancherOS 1.5.4 (64-bit x86) | Docker 17.03.x, 18.06.x, 18.09.x, 19.03.x Windows Server 2019 (64-bit x86) | Requires Docker Engine - Enterprise Edition (EE).** \* Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. This [how-to guide]({{}}/rancher/v2.x/en/installation/options/firewall) shows how to check the default firewall rules and how to open the ports with `firewalld` if necessary. From 7493c0f5a8d3f550f067a84f07db2c4138918940 Mon Sep 17 00:00:00 2001 From: Craig Jellick Date: Fri, 15 Nov 2019 18:23:44 -0700 Subject: [PATCH 060/103] Overhaul installation docs --- content/k3s/latest/en/_index.md | 18 +- content/k3s/latest/en/installation/_index.md | 15 +- .../latest/en/installation/airgap/_index.md | 7 +- .../en/installation/datastore/_index.md | 97 + .../en/installation/ha-embedded/_index.md | 22 + .../k3s/latest/en/installation/ha/_index.md | 153 +- .../en/installation/install-options/_index.md | 183 ++ .../en/installation/single-server/_index.md | 97 - content/k3s/latest/en/quick-start/_index.md | 16 +- static/img/k3s/k3s-production-setup-v5.svg | 2018 ----------------- static/img/k3s/k3s-production-setup.svg | 1176 ++++++++++ 11 files changed, 1533 insertions(+), 2269 deletions(-) create mode 100644 content/k3s/latest/en/installation/datastore/_index.md create mode 100644 content/k3s/latest/en/installation/ha-embedded/_index.md create mode 100644 content/k3s/latest/en/installation/install-options/_index.md delete mode 100644 content/k3s/latest/en/installation/single-server/_index.md delete mode 100644 static/img/k3s/k3s-production-setup-v5.svg create mode 100644 static/img/k3s/k3s-production-setup.svg diff --git a/content/k3s/latest/en/_index.md b/content/k3s/latest/en/_index.md index 02dae30dcfd..85d522e5e37 100644 --- a/content/k3s/latest/en/_index.md +++ b/content/k3s/latest/en/_index.md @@ -1,6 +1,6 @@ --- -title: "k3s - 5 less than k8s" -shortTitle: k3s +title: "K3s - 5 less than K8s" +shortTitle: K3s date: 2019-02-05T09:52:46-07:00 name: "menu" --- @@ -18,16 +18,14 @@ Great for: What is this? --- -K3s is intended to be a fully compliant Kubernetes distribution with the following changes: +K3s is a fully compliant Kubernetes distribution with the following enhancements: -1. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced - with out of tree addons. -2. Added sqlite3 as the default storage mechanism and support for other external SQL databases such as PostgreSQL and MySQL. etcd3 is also supported as an external database. -3. Added local storage provider, service load balancer, helm-controller, and traefik ingress controller. -4. Wrapped in simple launcher that handles a lot of the complexity of TLS and options. -5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). K3s packages required dependencies +* An embedded SQLite database has replaced etcd as the default datastore. External datastores such as PostgreSQL, MySQL, and etcd are also supported. +* Simple but powerful "batteries-included" features have been added, such as: a local storage provider, a service load balancer, a helm controller, and the Traefik ingress controller. +* Operation of all Kubernetes control plane components is encapsulated in a single binary and process. This allows K3s to automate and manage complex cluster operations like distributing certificates. +* In-tree cloud providers and storage plugins have been removed. +* External dependencies have been minimized (just a modern kernel and cgroup mounts needed). K3s packages required dependencies, including: * containerd * Flannel * CoreDNS - * CNI * Host utilities (iptables, socat, etc) diff --git a/content/k3s/latest/en/installation/_index.md b/content/k3s/latest/en/installation/_index.md index b1c54edc481..f631fe9c543 100644 --- a/content/k3s/latest/en/installation/_index.md +++ b/content/k3s/latest/en/installation/_index.md @@ -5,20 +5,15 @@ weight: 20 This section contains instructions for installing K3s in various environments. Please ensure you have met the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) before you begin installing K3s. -### Installation Options +[Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) provides guidance on the options available to you when installing K3s. -* [Single Server Installation]({{< baseurl >}}/k3s/latest/en/installation/single-server/) - Install K3s on a single Linux host. Single server installs are recommended for development, test, or production environments where the cluster doesn't have to be readily available for a user-base and some downtime is acceptible. +[High Availability with an External DB]({{< baseurl >}}/k3s/latest/en/installation/ha/) details how to setup an HA K3s cluster backed by an external datastore such as MySQL, PostgreSQL, or etcd. -* [High Availability (HA) Installation]({{< baseurl >}}/k3s/latest/en/installation/ha/) +[High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) details how to setup an HA K3s cluster that leverages a built-in distributed database. - Install K3s on two or more Linux hosts. High Availability installs are recommended for production environments that cannot tolerate any downtime. - -* [Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) - - Install K3s in an air-gapped environment. High Availability is recommended for production environments that cannot tolerate any downtime. +[Air-Gap Installation]({{< baseurl >}}/k3s/latest/en/installation/airgap/) details how to setup K3s in environments that do not have direct access to the Internet. ### Uninstalling -If you installed K3s with the help of the `install.sh` script an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). +If you installed K3s with the help of the `install.sh` script, an uninstall script is generated during installation, which will be created on your node at `/usr/local/bin/k3s-uninstall.sh` (or as `k3s-agent-uninstall.sh`). diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index d8b26320885..f919b3768ce 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -1,6 +1,6 @@ --- title: "Air-Gap Install" -weight: 40 +weight: 60 --- In this guide, we are assuming you have created your nodes in your air-gap environment and have a secure Docker private registry on your bastion server. @@ -12,7 +12,7 @@ Installation Outline 3. Install K3s ### Prepare Images Directory -Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. +Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running. Place the tar file in the `images` directory before starting K3s on each node, for example: @@ -73,6 +73,5 @@ The node-token is on the server at `/var/lib/rancher/k3s/server/node-token` Upgrading an air-gap environment can be accomplished in the following manner: 1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file. -2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same enviornment variables. +2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same environment variables. 3. Restart the K3s service (if not restarted automatically by installer). - diff --git a/content/k3s/latest/en/installation/datastore/_index.md b/content/k3s/latest/en/installation/datastore/_index.md new file mode 100644 index 00000000000..6e5ac992ab3 --- /dev/null +++ b/content/k3s/latest/en/installation/datastore/_index.md @@ -0,0 +1,97 @@ +--- +title: "Cluster Datastore Options" +weight: 50 +--- + +The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to Kubernetes operators. The available datastore options allow you to select a datastore that best fits your use case. For example: + +* If your team doesn't have expertise in operating etcd, you can choose an enterprise-grade SQL database like MySQL or PostgreSQL +* If you need to run a simple, short-lived cluster in your CI/CD environment, you can use the embedded SQLite database +* If you wish to deploy Kubernetes on the edge and require a highly available solution but can't afford the operational overhead of managing a database at the edge, you can use K3s's embedded HA datastore built on top of DQLite (currently experimental) + +K3s supports the following datastore options: + +* Embedded [SQLite](https://www.sqlite.org/index.html) +* [PostgreSQL](https://www.postgresql.org/) (certified against versions 10.7 and 11.5) +* [MySQL](https://www.mysql.com/) (certified against version 5.7) +* [etcd](https://etcd.io/) (certified against version 3.3.15) +* Embedded [DQLite](https://dqlite.io/) for High Availability (experimental) + +### External Datastore Configuration Parameters +If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you must set the `datastore-endpoint` parameter so that K3s knows how to connect to it. You may also specify parameters to configure the authentication and encryption of the connection. The below table summarizes these parameters, which can be passed as either CLI flags or environment variables. + + CLI Flag | Environment Variable | Description + ------------|-------------|------------------ + `--datastore-endpoint` | `K3S_DATASTORE_ENDPOINT` | Specify a PostgresSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below. + `--datastore-cafile` | `K3S_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate. | +| `--datastore-certfile` | `K3S_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. | +| `--datastore-keyfile` | `K3S_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. | + +As a best practice we recommend setting these parameters as environment variables rather than command line arguments so that your database credentials or other sensitive information aren't exposed as part of the process info. + +### Datastore Endpoint Format and Functionality +As mentioned, the format of the value passed to the `datastore-endpoint` parameter is dependent upon the datastore backend. The following details this format and functionality for each supported external datastore. + +{{% tabs %}} +{{% tab "PostgreSQL" %}} + +In its most common form, the datastore-endpoint parameter for PostgreSQL has the following format: + +`postgres://username:password@hostname:port/database-name` + +More advanced configuration parameters are available. For more information on these, please see https://godoc.org/github.com/lib/pq. + +If you specify a database name and it does not exist, the server will attempt to create it. + +If you only supply `postgres://` as the endpoint, K3s will attempt to do the following: + +* Connect to localhost using `postgres` as the username and password +* Create a database named `kubernetes` + + +{{% /tab %}} +{{% tab "MySQL" %}} + +In its most common form, the `datastore-endpoint` parameter for MySQL has the following format: + +`mysql://username:password@tcp(hostname:3306)/database-name` + +More advanced configuration parameters are available. For more information on these, please see https://github.com/go-sql-driver/mysql#dsn-data-source-name + +Note that due to a [known issue](https://github.com/rancher/k3s/issues/1093) in K3s, you cannot set the `tls` parameter. TLS communication is supported, but you cannot, for example, set this parameter to "skip-verify" to cause K3s to skip certificate verification. + +If you specify a database name and it does not exist, the server will attempt to create it. + +If you only supply `mysql://` as the endpoint, K3s will attempt to do the following: + +* Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user and no password +* Create a database with the name `kubernetes` + + +{{% /tab %}} +{{% tab "etcd" %}} + +In its most common form, the `datastore-endpoint` parameter for etcd has the following format: + +`https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379` + +The above assumes a typical three node etcd cluster. The parameter can accept one more comma separated etcd URLs. + +{{% /tab %}} +{{% /tabs %}} + +
Based on the above, the following example command could be used to launch a server instance that connects to a PostgresSQL database named k3s-db: +``` +K3S_DATASTORE_ENDPOINT='postgres://username:password@hostname:5432/k3s-db' k3s server +``` + +And the following example could be used to connect to a MySQL database using client certificate authentication: +``` +K3S_DATASTORE_ENDPOINT='mysql://username:password@tcp(hostname:3306)/k3s-db' \ +K3S_DATASTORE_CERTFILE='/path/to/client.crt' \ +K3S_DATASTORE_KEYFILE='/path/to/client.key' \ +k3s server +``` + +### Embedded DQLite for HA (Experimental) +K3s's use of DQLite is similar to its use of SQLite. It is simple to setup and manage. As such, there is no external configuration or additional steps to take in order to use this option. Please see [High Availability with Embedded DB (Experimental)]({{< baseurl >}}/k3s/latest/en/installation/ha-embedded/) for instructions on how to run with this option. diff --git a/content/k3s/latest/en/installation/ha-embedded/_index.md b/content/k3s/latest/en/installation/ha-embedded/_index.md new file mode 100644 index 00000000000..58526b65280 --- /dev/null +++ b/content/k3s/latest/en/installation/ha-embedded/_index.md @@ -0,0 +1,22 @@ +--- +title: "High Availability with Embedded DB (Experimental)" +weight: 40 +--- + +As of v1.0.0, K3s is previewing support for running a highly available control plane without the need for an external database. This means there is no need to manage an external etcd or SQL datastore in order to run a reliable production-grade setup. While this feature is currently experimental, we expect it to be the primary architecture for running HA K3s clusters in the future. + +This architecture is achieved by embedding a dqlite database within the K3s server process. DQLite is short for "distributed SQLite." According to https://dqlite.io, it is "*a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices.*" This makes it a natural fit for K3s. + +To run K3s in this mode, you must have an odd number of server nodes. We recommend starting with three nodes. + +To get started, first launch a server node with the `cluster-init` flag to enable clustering and a token that will be used as a shared secret to join additional servers to the cluster. +``` +K3S_TOKEN=SECRET k3s server --cluster-init +``` + +After launching the first server, join the second and third servers to the cluster using the shared secret: +``` +K3S_TOKEN=SECRET k3s server --server https://:6443 +``` + +Now you have a highly available control plane. Joining additional worker nodes to the cluster follows the same procedure as a single server cluster. diff --git a/content/k3s/latest/en/installation/ha/_index.md b/content/k3s/latest/en/installation/ha/_index.md index 5b5364420df..81adf97d1a7 100644 --- a/content/k3s/latest/en/installation/ha/_index.md +++ b/content/k3s/latest/en/installation/ha/_index.md @@ -1,150 +1,57 @@ --- -title: "High Availability (HA) Install" +title: "High Availability with an External DB" weight: 30 --- >**Note:** Official support for High-Availability (HA) was introduced in our v1.0.0 release. ->k3s servers do not utilize a quorum for leader election and so only a 2-node cluster is needed for HA at a minimum. You could optionally add one or more server nodes for additional redundancy. +Single server clusters can meet a variety of use cases, but for environments where uptime of the Kubernetes control plane is critical, you can run K3s in an HA configuration. An HA K3s cluster is comprised of: -For production environments that cannot tolerate down time, we recommend installing k3s in a high-availability configuration so that you can always access your cluster. This procedure walks you through setting up a 2-server cluster with k3s with an external database for the cluster datastore. +* Two or more **server nodes** that will serve the Kubernetes API and run other control plane services +* An **external datastore** (as opposed to the embedded SQLite datastore used in single server setups) +* A **fixed registration address** placed in front of the server nodes to allow worker nodes to register with the cluster -# Recommended Architecture -![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup-v5.svg) -This image depicts a k3s HA install with two load balancers: +The following diagram illustrates the above configuration: +![k3s HA]({{< baseurl >}}/img/k3s/k3s-production-setup.svg) -* A load balancer to expose workloads to external traffic -* A load balancer to expose the Kubernetes API for clients such as kubectl and to expose a stable k3s worker registration endpoint +In this architecture a server node is defined as a machine (bare-metal or virtual) running the `k3s server` command. A worker node is defined as a machine running the `k3s agent` command. -The external database shown should be a single endpoint k3s can access. The worker registration / kubernetes API load balancer is needed if the server nodepool will be auto scaling and thus server nodes are ephemeral. Port 6443 is used for worker (agent) node registration and the Kubernetes API. +Workers register through the fixed registration address, but after registration they establish a connection directly to one of the sever nodes. This is a websocket connection initiated by the `k3s agent` process and it is maintained by a client-side load balancer running as part of the agent process. Installation Outline -------------------- -1. Create Database for Cluster Datastore (PostgreSQL, MySQL, or etcd) -2. Create Server Nodes -3. Join Worker Nodes +Setting up an HA cluster requires the following steps: -### Create Database for Cluster Datastore -You will first need to create the database for the backend (cluster datastore). k3s must have a single endpoint it can reach to talk to the database. +1. Create an external datastore +2. Launch server nodes +3. Configure fixed registration address +4. Join worker nodes -Here is a list of supported backend databases as of our v1.0.0 release: +### Create an External Datastore +You will first need to create an external datastore for the cluster. See the [Cluster Datastore Options]({{< baseurl >}}/k3s/latest/en/installation/datastore/) documentation for more details. -* PostgreSQL 10.7 and 11.5 -* MySQL 5.7 -* etcd 3.3.15 +### Launch Server Nodes +K3s requires two or more server nodes for this HA configuration. See the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) guide for minimum machine requirements. -### Create Server Nodes -Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision at least two machines. -The sections below will indicate what you need to run (depending on the type of database) to bring the server nodes up. +When running the `k3s server` command on these nodes, you must set the `datastore-endpoint` parameter so that K3s knows how to connect to the external datastore. Please see the [datastore configuration guide]({{< baseurl >}}/k3s/latest/en/installation/datastore/#external-datastore-configuration-parameters) for information on configuring this parameter. ->**Note:** You may wish to taint the server nodes. They will run the kubelet and be scheduleable. If you wish to do this, you can use the `--node-taint` flag. For example `--node-taint key=value:NoExecute`. +> **Note:** The same installation options available to single-server installs are also available for HA installs. For more details, see the [Installation and Configuration Options]({{< baseurl >}}/k3s/latest/en/installation/install-options/) documentation. ->If your server node pool will be auto-scaling, we recommend using the `--cluster-secret` flag instead of the default node-token. For example, this will make it easier to write user-data. The examples we provide use this flag. +By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use taints. The `node-taint` parameter will allow you to configure nodes with taints, for example `--node-taint k3s-controlplane=true:NoExecute`. +Once you've launched the `k3s server` process on all server nodes, you can ensure that the cluster has come up properly by checking that the nodes are in the Ready state with `k3s kubectl get nodes`. -{{% tabs %}} -{{% tab "PostgreSQL" %}} +### Configure the Fixed Registration Address +Worker nodes need a URL to register against. This can be the IP or hostname of any of the server nodes, but in many cases those may change over time. For example, if you are running your cluster in a cloud that supports scaling groups, you may scale the server node group up and down over time, causing nodes to be created and destroyed and thus having different IPs from the initial set of server nodes. Therefore, you should have a stable endpoint in front of the server nodes that will not change over time. This endpoint can be setup using any number approaches, such as: ->**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='postgres://'` then the server will attempt to connect with the following: +* A layer-4 (TCP) load balancer +* Round-robin DNS +* A virtual or elastic IP addresses -> * Connect to `localhost` using the `postgres` user. -> * Use `postgres` as the password. -> * Create a database with the name `kubernetes`. - -> If you specify a database name and it does not exist, then the server will attempt to create it. -> For more information about the Postgres driver DSN, please refer to https://godoc.org/github.com/lib/pq - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile postgres.crt \ - --storage-keyfile postgres.key - -``` - -The above command will use these certificates to generate the TLS config to communicate with PostgreSQL securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='postgres://username:password@hostname:5432/dbname' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% tab "MySQL" %}} - ->**Note:** If you do not specify a DSN (data source name), for example if you only supply `--storage-endpoint='mysql://'` then the server will attempt to connect with the following: - -> * Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user. -> * Use no password. -> * Create a database with the name `kubernetes`. - -> If you specify a database name and it does not exist, then the server will attempt to create it. -> For more information about the MySQL driver DSN, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile mysql.crt \ - --storage-keyfile mysql.key - - -``` -The above command will use these certificates to generate the TLS config to communicate with MySQL securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='mysql://username:password@tcp(hostname:3306)/dbname' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% tab "etcd" %}} - -### Secure Method -Run the following command on each server to install k3s and connect it to the database securely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint="https://hostname:2379" \ - --cluster-secret='mysecret' \ - --storage-cafile ca.crt \ - --storage-certfile etcd.crt \ - --storage-keyfile etcd.key -``` -The above command will use these certificates to generate the TLS config to communicate with etcd securely. - -### Insecure Method -Run the following command on each server to install k3s and connect it to the database insecurely. - -``` -curl -fL https://get.k3s.io | sh -s - server --storage-endpoint='http://hostname:2379' --cluster-secret='mysecret' -``` - -{{% /tab %}} -{{% /tabs %}} - -Ensure that both of the nodes are in a Ready state such as with `k3s kubectl get nodes` +This endpoint can also be used for accessing the Kubernetes API. So you can, for example, modify your kubeconfig file to point to it instead of a specific node. ### Join Worker Nodes -Following the [Node Requirements]({{< baseurl >}}/k3s/latest/en/installation/node-requirements/) page, provision one or more machines to fill the role of the worker node(s). - -Run the following command to join a worker node to the server nodes. We are leveraging the cluster-secret here. Since our server nodes were set up to use this, so too must any agent nodes. - +Joining worker nodes in an HA cluster is the same as joining worker nodes in a single server cluster. You just need to specify the URL the agent should register to and the token it should use. ``` -curl -sfL https://get.k3s.io | K3S_URL=https://:6443 K3S_CLUSTER_SECRET='mysecret' sh - +K3S_TOKEN=SECRET k3s agent --server https://fixed-registration-address:6443 ``` - -Provide the IP or DNS in place of ``. This can be any one server node. k3s automatically handles load balancing the server nodes. - -Note: You may want to provide the cluster-secret temporarily via a file or environment variable then destroy it or clear your bash history so the password is no longer exposed in plain text on the machine. - diff --git a/content/k3s/latest/en/installation/install-options/_index.md b/content/k3s/latest/en/installation/install-options/_index.md new file mode 100644 index 00000000000..b9c3aa859e2 --- /dev/null +++ b/content/k3s/latest/en/installation/install-options/_index.md @@ -0,0 +1,183 @@ +--- +title: "Installation and Configuration Options" +weight: 20 +--- + +### Installation script options + +As mentioned in the [Quick-Start Guide]({{< baseurl >}}/k3s/latest/en/quick-start/), you can use the installation script available at https://get.k3s.io to install K3s as a service on systemd and openrc based systems. + +The simplest form of this command is as follows: +```sh +curl -sfL https://get.k3s.io | sh - +``` + +When using this method to install K3s, the following environment variables can be used to configure the installation: + +- `INSTALL_K3S_SKIP_DOWNLOAD` + + If set to true will not download K3s hash or binary. + +- `INSTALL_K3S_SYMLINK` + + If set to 'skip' will not create symlinks, 'force' will overwrite, default will symlink if command does not exist in path. + +- `INSTALL_K3S_SKIP_START` + + If set to true will not start K3s service. + +- `INSTALL_K3S_VERSION` + + Version of K3s to download from github. Will attempt to download the latest version if not specified. + +- `INSTALL_K3S_BIN_DIR` + + Directory to install K3s binary, links, and uninstall script to, or use `/usr/local/bin` as the default. + +- `INSTALL_K3S_BIN_DIR_READ_ONLY` + + If set to true will not write files to `INSTALL_K3S_BIN_DIR`, forces setting INSTALL_K3S_SKIP_DOWNLOAD=true. + +- `INSTALL_K3S_SYSTEMD_DIR` + + Directory to install systemd service and environment files to, or use `/etc/systemd/system` as the default. + +- `INSTALL_K3S_EXEC` + + Command with flags to use for launching K3s in the service. If the command is not specified, it will default to "agent" if `K3S_URL` is set or "server" if it is not set. The final systemd command resolves to a combination of this environment variable and script args. To illustrate this, the following commands result in the same behavior: + ```sh + curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s - + curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s - + curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel + curl ... | sh -s - server --no-flannel + curl ... | sh -s - --no-flannel + ``` + + - `INSTALL_K3S_NAME` + + Name of systemd service to create, will default from the K3s exec command if not specified. If specified the name will be prefixed with 'k3s-'. + + - `INSTALL_K3S_TYPE` + + Type of systemd service to create, will default from the K3s exec command if not specified. + + +Environment variables which begin with `K3S_` will be preserved for the systemd and openrc services to use. Setting `K3S_URL` without explicitly setting an exec command will default the command to "agent". When running the agent `K3S_TOKEN` must also be set. + + +### Beyond the Installation Script +As stated, the installation script is primarily concerned with configuring K3s to run as a service. If you choose to not use the script, you can run K3s simply by downloading the binary from our [release page](https://github.com/rancher/k3s/releases/latest), placing it on your path, and executing it. The K3s binary supports the following commands: + +Command | Description +--------|------------------ +`k3s server` | Run the K3s management server, which will also launch Kubernetes control plane components such as the API server, controller-manager, and scheduler. +`k3s agent` | Run the K3s node agent. This will cause K3s to run as a worker node, launching the Kubernetes node services `kubelet` and `kube-proxy`. +`k3s kubectl` | Run an embedded [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) CLI. If the `KUBECONFIG` environment variable is not set, this will automatically attempt to use the config file that is created at `/etc/rancher/k3s/k3s.yaml` when launching a K3s server node. +`k3s crictl` | Run an embedded [crictl](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md). This is a CLI for interacting with Kubernetes's container runtime interface (CRI). Useful for debugging. +`k3s ctr` | Run an embedded [ctr](https://github.com/projectatomic/containerd/blob/master/docs/cli.md). This is a CLI for containerd, the container daemon used by K3s. Useful for debugging. +`k3s help` | Shows a list of commands or help for one command + +The `k3s server` and `k3s agent` commands have additional configuration options that can be viewed with `k3s server --help` or `k3s agent --help`. For convenience, that help text is presented here: + +### `k3s server` +``` +NAME: + k3s server - Run management server + +USAGE: + k3s server [OPTIONS] + +OPTIONS: + -v value (logging) Number for the log level verbosity (default: 0) + --vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging + --log value, -l value (logging) Log to file + --alsologtostderr (logging) Log to standard error as well as file (if set) + --bind-address value (listener) k3s bind address (default: 0.0.0.0) + --https-listen-port value (listener) HTTPS listen port (default: 6443) + --advertise-address value (listener) IP address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip) + --advertise-port value (listener) Port that apiserver uses to advertise to members of the cluster (default: listen-port) (default: 0) + --tls-san value (listener) Add additional hostname or IP as a Subject Alternative Name in the TLS cert + --data-dir value, -d value (data) Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root + --cluster-cidr value (networking) Network CIDR to use for pod IPs (default: "10.42.0.0/16") + --service-cidr value (networking) Network CIDR to use for services IPs (default: "10.43.0.0/16") + --cluster-dns value (networking) Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10) + --cluster-domain value (networking) Cluster Domain (default: "cluster.local") + --flannel-backend value (networking) One of 'none', 'vxlan', 'ipsec', or 'flannel' (default: "vxlan") + --token value, -t value (cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN] + --token-file value (cluster) File containing the cluster-secret/token [$K3S_TOKEN_FILE] + --write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT] + --write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE] + --kube-apiserver-arg value (flags) Customized flag for kube-apiserver process + --kube-scheduler-arg value (flags) Customized flag for kube-scheduler process + --kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process + --kube-cloud-controller-manager-arg value (flags) Customized flag for kube-cloud-controller-manager process + --datastore-endpoint value (db) Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$K3S_DATASTORE_ENDPOINT] + --datastore-cafile value (db) TLS Certificate Authority file used to secure datastore backend communication [$K3S_DATASTORE_CAFILE] + --datastore-certfile value (db) TLS certification file used to secure datastore backend communication [$K3S_DATASTORE_CERTFILE] + --datastore-keyfile value (db) TLS key file used to secure datastore backend communication [$K3S_DATASTORE_KEYFILE] + --default-local-storage-path value (storage) Default local storage path for local provisioner storage class + --no-deploy value (components) Do not deploy packaged components (valid items: coredns, servicelb, traefik, local-storage, metrics-server) + --disable-scheduler (components) Disable Kubernetes default scheduler + --disable-cloud-controller (components) Disable k3s default cloud controller manager + --disable-network-policy (components) Disable k3s default network policy controller + --node-name value (agent/node) Node name [$K3S_NODE_NAME] + --with-node-id (agent/node) Append id to node name + --node-label value (agent/node) Registering kubelet with set of labels + --node-taint value (agent/node) Registering kubelet with set of taints + --docker (agent/runtime) Use docker instead of containerd + --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation + --pause-image value (agent/runtime) Customized pause image for containerd sandbox + --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") + --node-ip value, -i value (agent/networking) IP address to advertise for node + --node-external-ip value (agent/networking) External IP address to advertise for node + --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] + --flannel-iface value (agent/networking) Override default flannel interface + --flannel-conf value (agent/networking) Override default flannel config file + --kubelet-arg value (agent/flags) Customized flag for kubelet process + --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process + --rootless (experimental) Run rootless + --agent-token value (experimental/cluster) Shared secret used to join agents to the cluster, but not servers [$K3S_AGENT_TOKEN] + --agent-token-file value (experimental/cluster) File containing the agent secret [$K3S_AGENT_TOKEN_FILE] + --server value, -s value (experimental/cluster) Server to connect to, used to join a cluster [$K3S_URL] + --cluster-init (experimental/cluster) Initialize new cluster master [$K3S_CLUSTER_INIT] + --cluster-reset (experimental/cluster) Forget all peers and become a single cluster new cluster master [$K3S_CLUSTER_RESET] + --no-flannel (deprecated) use --flannel-backend=none + --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] +``` + +### `k3s agent` +``` +NAME: + k3s agent - Run node agent + +USAGE: + k3s agent [OPTIONS] + +OPTIONS: + -v value (logging) Number for the log level verbosity (default: 0) + --vmodule value (logging) Comma-separated list of pattern=N settings for file-filtered logging + --log value, -l value (logging) Log to file + --alsologtostderr (logging) Log to standard error as well as file (if set) + --token value, -t value (cluster) Token to use for authentication [$K3S_TOKEN] + --token-file value (cluster) Token file to use for authentication [$K3S_TOKEN_FILE] + --server value, -s value (cluster) Server to connect to [$K3S_URL] + --data-dir value, -d value (agent/data) Folder to hold state (default: "/var/lib/rancher/k3s") + --node-name value (agent/node) Node name [$K3S_NODE_NAME] + --with-node-id (agent/node) Append id to node name + --node-label value (agent/node) Registering kubelet with set of labels + --node-taint value (agent/node) Registering kubelet with set of taints + --docker (agent/runtime) Use docker instead of containerd + --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use alternative CRI implementation + --pause-image value (agent/runtime) Customized pause image for containerd sandbox + --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") + --node-ip value, -i value (agent/networking) IP address to advertise for node + --node-external-ip value (agent/networking) External IP address to advertise for node + --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] + --flannel-iface value (agent/networking) Override default flannel interface + --flannel-conf value (agent/networking) Override default flannel config file + --kubelet-arg value (agent/flags) Customized flag for kubelet process + --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process + --rootless (experimental) Run rootless + --no-flannel (deprecated) use --flannel-backend=none + --cluster-secret value (deprecated) use --token [$K3S_CLUSTER_SECRET] +``` diff --git a/content/k3s/latest/en/installation/single-server/_index.md b/content/k3s/latest/en/installation/single-server/_index.md deleted file mode 100644 index 17ea504c0d1..00000000000 --- a/content/k3s/latest/en/installation/single-server/_index.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: "Single Server Install" -weight: 20 ---- - ->**Note:** This section contains information on flags and environment variables used for starting a single-server (non-HA) K3s cluster. A High-Availability (HA) K3s cluster is recommended for production environments that cannot tolerate down time. - -Installation ------------- - -K3s is easy to install. To install the latest version, simply run: - -```sh -curl -sfL https://get.k3s.io | sh - -``` - -The install script will attempt to download the latest release. To specify a version for download we can use the `INSTALL_K3S_VERSION` environment variable. For example: -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - -``` - -To install with a specific flag we can use the `INSTALL_K3S_EXEC` -environment variable. For example: -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-flannel" sh - -``` - -The installer can also be run without performing downloads by setting `INSTALL_K3S_SKIP_DOWNLOAD=true`. For example: -```sh -curl -sfL https://github.com/rancher/k3s/releases/download/vX.Y.Z/k3s -o /usr/local/bin/k3s -chmod 0755 /usr/local/bin/k3s - -curl -sfL https://get.k3s.io -o install-k3s.sh -chmod 0755 install-k3s.sh - -export INSTALL_K3S_SKIP_DOWNLOAD=true -./install-k3s.sh -``` - -The full help text for the install script environment variables are as follows: - - `K3S_*` - - Environment variables which begin with `K3S_` will be preserved for the - systemd service to use. Setting `K3S_URL` without explicitly setting - a systemd exec command will default the command to "agent", and we - enforce that `K3S_TOKEN` or `K3S_CLUSTER_SECRET` is also set. - - - `INSTALL_K3S_SKIP_DOWNLOAD` - - If set to true will not download K3s hash or binary. - - - `INSTALL_K3S_SYMLINK` - - If set to 'skip' will not create symlinks, 'force' will overwrite, - default will symlink if command does not exist in path. - - - `INSTALL_K3S_VERSION` - - Version of K3s to download from github. Will attempt to download the - latest version if not specified. - - - `INSTALL_K3S_BIN_DIR` - - Directory to install K3s binary, links, and uninstall script to, or use - /usr/local/bin as the default - - - `INSTALL_K3S_SYSTEMD_DIR` - - Directory to install systemd service and environment files to, or use - /etc/systemd/system as the default - - - `INSTALL_K3S_EXEC` or script arguments - - Command with flags to use for launching K3s in the systemd service, if - the command is not specified will default to "agent" if `K3S_URL` is set - or "server" if not. The final systemd command resolves to a combination - of EXEC and script args ($@). - - The following commands result in the same behavior: - ```sh - curl ... | INSTALL_K3S_EXEC="--no-flannel" sh -s - - curl ... | INSTALL_K3S_EXEC="server --no-flannel" sh -s - - curl ... | INSTALL_K3S_EXEC="server" sh -s - --no-flannel - curl ... | sh -s - server --no-flannel - curl ... | sh -s - --no-flannel - ``` - - - `INSTALL_K3S_NAME` - - Name of systemd service to create, will default from the K3s exec command - if not specified. If specified the name will be prefixed with 'k3s-'. - - - `INSTALL_K3S_TYPE` - - Type of systemd service to create, will default from the K3s exec command - if not specified. - diff --git a/content/k3s/latest/en/quick-start/_index.md b/content/k3s/latest/en/quick-start/_index.md index fa3b36c2b72..e4e8156bf34 100644 --- a/content/k3s/latest/en/quick-start/_index.md +++ b/content/k3s/latest/en/quick-start/_index.md @@ -3,26 +3,28 @@ title: "Quick-Start Guide" weight: 10 --- ->**Note:** The intent of this guide is to quickly launch a cluster with default options. It is suitable for production environments if you can accept some downtime in production (e.g. on the Edge). A High-Availiability solution should be utilized if down time is not acceptible. -The [installation options](../installation) section covers in greater detail how K3s can be set up. +>**Note:** This guide will help you quickly launch a cluster with default options. The [installation section](../installation) covers in greater detail how K3s can be set up. > New to Kubernetes? The official Kubernetes docs already have some great tutorials outlining the basics [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/). Install Script -------------- -The K3s `install.sh` script provides a convenient way for installing to systemd or openrc, -to install K3s as a service just run: +K3s provides an installation script that is a convenient way to install it as a service on systemd or openrc based systems. This script is available at https://get.k3s.io. To install K3s using this method, just run: ```bash curl -sfL https://get.k3s.io | sh - ``` -A kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` and the service is automatically started or restarted. -The install script will install K3s and additional utilities, such as `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh`. +After running this installation: -To install on worker nodes and add them to the cluster, we should pass `K3S_URL` along with the `K3S_TOKEN` environment variable. `K3S_TOKEN` is created at `/var/lib/rancher/k3s/server/node-token` on your server. Here is an example showing how to join a node: +* The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed +* Additional utilities will be installed, including `kubectl`, `crictl`, `ctr`, `k3s-killall.sh`, and `k3s-uninstall.sh` +* A kubeconfig file will be written to `/etc/rancher/k3s/k3s.yaml` and the kubectl installed by K3s will automatically use it + +To install on worker nodes and add them to the cluster, run the installation script with the `K3S_URL` and `K3S_TOKEN` environment variables. Here is an example showing how to join a worker node: ```bash curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh - ``` +Setting the `K3S_URL` parameter causes K3s to run in worker mode. The K3s agent will register with the K3s server listening at the supplied URL. The value to use for `K3S_TOKEN` is stored at `/var/lib/rancher/k3s/server/node-token` on your server node. Note: Each machine must have a unique hostname. If your machines do not have unique hostnames, pass the `K3S_NODE_NAME` environment variable and provide a value with a valid and unique hostname for each node. diff --git a/static/img/k3s/k3s-production-setup-v5.svg b/static/img/k3s/k3s-production-setup-v5.svg deleted file mode 100644 index cb50963c422..00000000000 --- a/static/img/k3s/k3s-production-setup-v5.svg +++ /dev/null @@ -1,2018 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/static/img/k3s/k3s-production-setup.svg b/static/img/k3s/k3s-production-setup.svg new file mode 100644 index 00000000000..2d132eb9566 --- /dev/null +++ b/static/img/k3s/k3s-production-setup.svg @@ -0,0 +1,1176 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From 53ed57efe36850ae77c88b860a1567b0408452f6 Mon Sep 17 00:00:00 2001 From: Erik Wilson Date: Wed, 20 Nov 2019 13:40:46 -0700 Subject: [PATCH 061/103] Update k3s v1.0.0 Node Registration information --- content/k3s/latest/en/configuration/_index.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/content/k3s/latest/en/configuration/_index.md b/content/k3s/latest/en/configuration/_index.md index b1b5ece6f97..1f3cc7d7bf8 100644 --- a/content/k3s/latest/en/configuration/_index.md +++ b/content/k3s/latest/en/configuration/_index.md @@ -96,10 +96,11 @@ Node Registration ----------------- Agents will register with the server using the node cluster secret along with a randomly generated -password for the node, stored at `/var/lib/rancher/k3s/agent/node-password.txt`. The server will +password for the node, stored at `/etc/rancher/node/password`. The server will store the passwords for individual nodes at `/var/lib/rancher/k3s/server/cred/node-passwd`, and any -subsequent attempts must use the same password. If the data directory of an agent is removed the -password file should be recreated for the agent, or the entry removed from the server. +subsequent attempts must use the same password. If the `/etc/rancher/node` directory of an agent is removed the +password file should be recreated for the agent, or the entry removed from the server. A unique node +id can be appended to the hostname by launching k3s servers or agents using the `--with-node-id` flag. Containerd and Docker ---------- From 4d6989d0797763b05be9b4a92bbe5f10bfc96c4a Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Wed, 20 Nov 2019 13:40:48 -0800 Subject: [PATCH 062/103] fix some of the removed links --- .../en/installation/configuration/running-commands/_index.md | 2 +- content/rancher/v2.x/en/best-practices/containers/_index.md | 4 ++-- content/rancher/v2.x/en/catalog/_index.md | 2 +- .../en/cluster-provisioning/rke-clusters/options/_index.md | 4 ++-- content/rancher/v2.x/en/installation/references/_index.md | 2 +- content/rancher/v2.x/en/project-admin/tools/alerts/_index.md | 4 ++-- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/content/os/v1.x/en/installation/configuration/running-commands/_index.md b/content/os/v1.x/en/installation/configuration/running-commands/_index.md index 11b8d44d8be..01bc8047343 100644 --- a/content/os/v1.x/en/installation/configuration/running-commands/_index.md +++ b/content/os/v1.x/en/installation/configuration/running-commands/_index.md @@ -12,7 +12,7 @@ runcmd: - echo "test" > /home/rancher/test2 ``` -Commands specified using `runcmd` will be executed within the context of the `console` container. +Commands specified using `runcmd` will be executed within the context of the `console` container. More details on the ordering of commands run in the `console` container can be found [here]({{< baseurl >}}/os/v1.x/en/installation/boot-process/built-in-system-services/#console). ### Running Docker commands diff --git a/content/rancher/v2.x/en/best-practices/containers/_index.md b/content/rancher/v2.x/en/best-practices/containers/_index.md index 410dfb2f437..ce67e87ef6d 100644 --- a/content/rancher/v2.x/en/best-practices/containers/_index.md +++ b/content/rancher/v2.x/en/best-practices/containers/_index.md @@ -32,11 +32,11 @@ When possible, use a non-privileged user when running processes within your cont ### Define Resource Limits Apply CPU and memory limits to your pods. This can help manage the resources on your worker nodes and avoid a malfunctioning microservice from impacting other microservices. -In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the Rancher docs. +In standard Kubernetes, you can set resource limits on the namespace level. In Rancher, you can set resource limits on the project level and they will propagate to all the namespaces within the project. For details, refer to the [Rancher docs]({{}}/rancher/v2.x/en/project-admin/resource-quotas/). When setting resource quotas, if you set anything related to CPU or Memory (i.e. limits or reservations) on a project or namespace, all containers will require a respective CPU or Memory field set during creation. To avoid setting these limits on each and every container during workload creation, a default container resource limit can be specified on the namespace. -The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the namespace level. +The Kubernetes docs have more information on how resource limits can be set at the [container level](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) and the [namespace level](https://kubernetes.io/docs/concepts/policy/resource-quotas/). ### Define Resource Requirements You should apply CPU and memory requirements to your pods. This is crucial for informing the scheduler which type of compute node your pod needs to be placed on, and ensuring it does not over-provision that node. In Kubernetes, you can set a resource requirement by defining `resources.requests` in the resource requests field in a pod's container spec. For details, refer to the [Kubernetes docs](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container). diff --git a/content/rancher/v2.x/en/catalog/_index.md b/content/rancher/v2.x/en/catalog/_index.md index f548860fd9e..6a954a02a01 100644 --- a/content/rancher/v2.x/en/catalog/_index.md +++ b/content/rancher/v2.x/en/catalog/_index.md @@ -151,7 +151,7 @@ _Available as v2.2.0_ When creating applications that span multiple Kubernetes clusters, a Global DNS entry can be created to route traffic to the endpoints in all of the different clusters. An external DNS server will need be programmed to assign a fully qualified domain name (a.k.a FQDN) to your application. Rancher will use the FQDN you provide and the IP addresses where your application is running to program the DNS. Rancher will gather endpoints from all the Kubernetes clusters running your application and program the DNS. -For more information on how to use this feature, see Global DNS. +For more information on how to use this feature, see [Global DNS]({{< baseurl >}}/rancher/v2.x/en/catalog/globaldns/). ## Chart Compatibility with Rancher diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md index a6ae66d5dba..27611cb1713 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/_index.md @@ -63,9 +63,9 @@ The registry configuration here is applied during the provisioning of the cluste - **System images** are components needed to maintain the Kubernetes cluster. - **Add-ons** are used to deploy several cluster components, including network plug-ins, the ingress controller, the DNS provider, or the metrics server. -To deploy workloads that pull images from a private registry, you will need to set up your own Kubernetes registry for your project. +To deploy workloads that pull images from a private registry, you will need to [set up your own Kubernetes registry]({{}}/rancher/v2.x/en/k8s-in-rancher/registries/) for your project. -See the RKE documentation on private registries for more information on the private registry for components applied during the provisioning of the cluster. +See the [RKE documentation on private registries]({{< baseurl >}}/rke/latest/en/config-options/private-registries/) for more information on the private registry for components applied during the provisioning of the cluster. ### Authorized Cluster Endpoint diff --git a/content/rancher/v2.x/en/installation/references/_index.md b/content/rancher/v2.x/en/installation/references/_index.md index a39ef0ca510..5c79484b9e1 100644 --- a/content/rancher/v2.x/en/installation/references/_index.md +++ b/content/rancher/v2.x/en/installation/references/_index.md @@ -11,7 +11,7 @@ The following table lists the ports that need to be open to and from nodes that {{< ports-rancher-nodes >}} -**Note** Rancher nodes may also require additional outbound access for any external authentication provider which is configured (LDAP for example). +**Note** Rancher nodes may also require additional outbound access for any external [authentication provider]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) which is configured (LDAP for example). ## Kubernetes Cluster Nodes diff --git a/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md b/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md index 401191ecc10..daaa5b1427b 100644 --- a/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md +++ b/content/rancher/v2.x/en/project-admin/tools/alerts/_index.md @@ -40,7 +40,7 @@ For information on other default alerts, refer to the section on [cluster-level ## Adding Project Alerts ->**Prerequisite:** Before you can receive project alerts, you must add a notifier. +>**Prerequisite:** Before you can receive project alerts, you must [add a notifier]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/notifiers/#adding-notifiers). 1. From the **Global** view, navigate to the project that you want to configure project alerts for. Select **Tools > Alerts**. In versions prior to v2.2.0, you can choose **Resources > Alerts**. @@ -165,7 +165,7 @@ If you enable [project monitoring]({{< baseurl >}}/rancher/v2.x/en/project-admin 1. Continue adding more **Alert Rule** to the group. -1. Finally, choose the notifiers that send you alerts. +1. Finally, choose the [notifiers]({{< baseurl >}}/rancher/v2.x/en/cluster-admin/tools/notifiers/) that send you alerts. - You can set up multiple notifiers. - You can change notifier recipients on the fly. From 00eab31f7e05a4e0cdc2264fbcecde14e9a6733c Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Wed, 20 Nov 2019 16:38:44 -0800 Subject: [PATCH 063/103] replace removed links with updated links --- content/rancher/v2.x/en/admin-settings/_index.md | 2 +- .../tools/istio/setup/enable-istio-in-cluster/_index.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/_index.md b/content/rancher/v2.x/en/admin-settings/_index.md index c9b9ef8a460..01bbb1343ff 100644 --- a/content/rancher/v2.x/en/admin-settings/_index.md +++ b/content/rancher/v2.x/en/admin-settings/_index.md @@ -52,7 +52,7 @@ The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/ra Rancher Kubernetes Metadata contains Kubernetes version information which Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/). -For more information on how metadata works and how to configure metadata config, see Rancher Kubernetes Metadata. +For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{}}/rancher/v2.x/en/admin-settings/rbac/glob 1. From the **Global** view, navigate to the cluster where you want to enable Istio. 1. Click **Tools > Istio.** -1. Optional: Configure member access and resource limits for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio. +1. Optional: Configure member access and [resource limits]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/resources) for the Istio components. Ensure you have enough resources on your worker nodes to enable Istio. 1. Click **Enable**. 1. Click **Save**. From 6e408a1165d342af543cb9d6f366e59c8cabc02d Mon Sep 17 00:00:00 2001 From: Tejeev Date: Fri, 22 Nov 2019 12:28:36 +0000 Subject: [PATCH 064/103] Clarification --- .../latest/en/etcd-snapshots/restoring-from-backup/_index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md b/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md index c6dae0f85fc..32f983ea64e 100644 --- a/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md +++ b/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md @@ -8,7 +8,9 @@ The details of restoring your cluster from backup are different depending on you {{% tabs %}} {{% tab "RKE v0.2.0+"%}} -If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts the etcd to a specific snapshot. The following actions are included in the command: +If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot and should be run on an etcd node of the the specific cluster that has suffered the disaster. + +The following actions will be performed when you run the command: - Syncs the snapshot or downloads the snapshot from S3, if necessary. - Checks snapshot checksum across etcd nodes to make sure they are identical. From 86adb6c102574d8d3981e8ff848d037c9deaf78a Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 22 Nov 2019 09:52:47 -0700 Subject: [PATCH 065/103] Making the same change to the other tab --- .../latest/en/etcd-snapshots/restoring-from-backup/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md b/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md index 32f983ea64e..a4e0ce38419 100644 --- a/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md +++ b/content/rke/latest/en/etcd-snapshots/restoring-from-backup/_index.md @@ -73,9 +73,9 @@ $ rke etcd snapshot-restore \ {{% /tab %}} {{% tab "RKE prior to v0.2.0"%}} -If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot. +If there is a disaster with your Kubernetes cluster, you can use `rke etcd snapshot-restore` to recover your etcd. This command reverts etcd to a specific snapshot and should be run on an etcd node of the the specific cluster that has suffered the disaster. -The following actions are included in `rke etcd snapshot-restore`: +The following actions will be performed when you run the command: - Removes the old etcd cluster - Rebuilds the etcd cluster using the local snapshot From 02d849b50c58d3da11087a501974d3a0ca2b92c1 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 22 Nov 2019 12:23:08 -0700 Subject: [PATCH 066/103] Document turning feature flags on/off from Rancher UI --- .../en/admin-settings/feature-flags/_index.md | 34 +++++++++++++++---- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/feature-flags/_index.md b/content/rancher/v2.x/en/admin-settings/feature-flags/_index.md index e3fac65abe9..fc9a9dbe0c9 100644 --- a/content/rancher/v2.x/en/admin-settings/feature-flags/_index.md +++ b/content/rancher/v2.x/en/admin-settings/feature-flags/_index.md @@ -6,19 +6,20 @@ _Available as of v2.3.0_ Rancher includes some features that are experimental and disabled by default. You might want to enable these features, for example, if you decide that the benefits of using an [unsupported storage type]({{}}/rancher/v2.x/en/admin-settings/feature-flags/enable-not-default-storage-drivers) outweighs the risk of using an untested feature. Feature flags were introduced to allow you to try these features that are not enabled by default. -The features can be enabled in two ways: +The features can be enabled in three ways: -- When installing Rancher with a CLI, you can use a feature flag to enable a feature by default -- After installing Rancher, you can turn on the features with the Rancher API +- [Enable features when starting Rancher.](#enabling-features-when-starting-rancher) When installing Rancher with a CLI, you can use a feature flag to enable a feature by default. +- [Enable features from the Rancher UI](#enabling-features-with-the-rancher-ui) in Rancher v2.3.3+ by going to the **Settings** page. +- [Enable features with the Rancher API](#enabling-features-with-the-rancher-api) after installing Rancher. Each feature has two values: - A default value, which can be configured with a flag or environment variable from the command line -- A set value, which can be configured with the Rancher API +- A set value, which can be configured with the Rancher API or UI If no value has been set, Rancher uses the default value. -Because the API sets the actual value and the command line sets the default value, that means that if you enable or disable a feature with the API, it will override any value set with the command line. +Because the API sets the actual value and the command line sets the default value, that means that if you enable or disable a feature with the API or UI, it will override any value set with the command line. For example, if you install Rancher, then set a feature flag to true with the Rancher API, then upgrade Rancher with a command that sets the feature flag to false, the default value will still be false, but the feature will still be enabled because it was set with the Rancher API. If you then deleted the set value (true) with the Rancher API, setting it to NULL, the default value (false) would take effect. @@ -76,6 +77,25 @@ docker run -d -p 80:80 -p 443:443 \ {{% /tab %}} {{% /tabs %}} +# Enabling Features with the Rancher UI + +_Available as of Rancher v2.3.3_ + +1. Go to the **Global** view and click **Settings.** +1. Click the **Feature Flags** tab. You will see a list of experimental features. +1. To enable a feature, go to the disabled feature you want to enable and click **Ellipsis (...) > Activate.** + +**Result:** The feature is enabled. + +### Disabling Features with the Rancher UI + +1. Go to the **Global** view and click **Settings.** +1. Click the **Feature Flags** tab. You will see a list of experimental features. +1. To disable a feature, go to the enabled feature you want to disable and click **Ellipsis (...) > Deactivate.** + +**Result:** The feature is disabled. + + # Enabling Features with the Rancher API 1. Go to `/v3/features`. @@ -88,7 +108,7 @@ docker run -d -p 80:80 -p 443:443 \ **Result:** The feature is enabled. -# Disabling Features with the Rancher API +### Disabling Features with the Rancher API 1. Go to `/v3/features`. 1. In the `data` section, you will see an array containing all of the features that can be turned on with feature flags. The name of the feature is in the `id` field. Click the name of the feature you want to enable. @@ -98,4 +118,4 @@ docker run -d -p 80:80 -p 443:443 \ 1. Click **Send Request.** 1. Click **Close.** -**Result:** The feature is disabled. +**Result:** The feature is disabled. \ No newline at end of file From 4c1b25651b7f6bd9f980e70bafb3e5ef532eac97 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 21 Nov 2019 17:23:31 -0700 Subject: [PATCH 067/103] Explain how to exclude workloads from Istio sidecar --- .../setup/enable-istio-in-namespace/_index.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace/_index.md b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace/_index.md index d99e00661f5..948d15c7c05 100644 --- a/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace/_index.md +++ b/content/rancher/v2.x/en/cluster-admin/tools/istio/setup/enable-istio-in-namespace/_index.md @@ -21,4 +21,25 @@ This namespace setting will only affect new workloads in the namespace. Any pree To verify that Istio is enabled, deploy a hello-world workload in the namespace. Go to the workload and click the pod name. In the **Containers** section, you should see the `istio-proxy` container. +### Excluding Workloads from Being Injected with the Istio Sidecar + +If you need to exclude a workload from getting injected with the Istio sidecar, use the following annotation on the workload: + +``` +sidecar.istio.io/inject: “false” +``` + +To add the annotation to a workload, + +1. From the **Global** view, open the project that has the workload that should not have the sidecar. +1. Click **Resources > Workloads.** +1. Go to the workload that should not have the sidecar and click **Ellipsis (...) > Edit.** +1. Click **Show Advanced Options.** Then expand the **Labels & Annotations** section. +1. Click **Add Annotation.** +1. In the **Key** field, enter `sidecar.istio.io/inject`. +1. In the **Value** field, enter `false`. +1. Click **Save.** + +**Result:** The Istio sidecar will not be injected into the workload. + ### [Next: Set up Taints and Tolerations]({{}}/rancher/v2.x/en/cluster-admin/tools/istio/setup/node-selectors) \ No newline at end of file From 7b6ac93f4481c106938b9f146d2652e6fc7ee3a9 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 20 Nov 2019 10:58:27 -0700 Subject: [PATCH 068/103] Change note that says Rancher v2.1 is new --- .../v2.x/en/project-admin/tools/pipelines/_index.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/project-admin/tools/pipelines/_index.md b/content/rancher/v2.x/en/project-admin/tools/pipelines/_index.md index 2ab36f06cb9..4628689ee68 100644 --- a/content/rancher/v2.x/en/project-admin/tools/pipelines/_index.md +++ b/content/rancher/v2.x/en/project-admin/tools/pipelines/_index.md @@ -7,10 +7,6 @@ aliases: - /rancher/v2.x/en/tools/pipelines/ - /rancher/v2.x/en/tools/pipelines/configurations/ --- ->**Notes:** -> ->- Pipelines are new and improved for Rancher v2.1! Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1. ->- Still using v2.0.x? See the pipeline documentation for [previous versions]({{< baseurl >}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x). A _pipeline_ is a software delivery process that is broken into different stages and steps. Setting up a pipeline can help developers deliver new software as quickly and efficiently as possible. Within Rancher, you can configure pipelines for each of your Rancher projects. @@ -30,6 +26,12 @@ Typically, pipeline stages include: Only [administrators]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/global-permissions/), [cluster owners or members]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#cluster-roles), or [project owners]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/#project-roles) can [configure version control providers](#version-control-providers) and [manage global pipeline execution settings](#managing-global-pipeline-execution-settings). Project members can only configure [repositories]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#configuring-repositories) and [pipelines]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/pipelines/#pipeline-configuration). + +> **Notes:** +> +> - Pipelines were improved in Rancher v2.1. Therefore, if you configured pipelines while using v2.0.x, you'll have to reconfigure them after upgrading to v2.1. +> - Still using v2.0.x? See the pipeline documentation for [previous versions]({{< baseurl >}}/rancher/v2.x/en/tools/pipelines/docs-for-v2.0.x). + ## Overview Rancher's pipeline provides a simple CI/CD experience. Use it to automatically checkout code, run builds or scripts, publish Docker images or catalog applications, and deploy the updated software to users. From d0993d9dbab6e3a3c640542d3550eca7fd34aa0b Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 19 Nov 2019 17:16:45 -0700 Subject: [PATCH 069/103] Explain behavior of enable_cluster_alerting and enable_cluster_monitoring --- .../rke-templates/example-yaml/_index.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/example-yaml/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/example-yaml/_index.md index 4fbe8a08a3c..3c85e86d616 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/example-yaml/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/example-yaml/_index.md @@ -12,8 +12,21 @@ The YAML in the RKE template uses the same customization that is used when you c # Cluster Config # docker_root_dir: /var/lib/docker + enable_cluster_alerting: false -enable_cluster_monitoring: false +# This setting is not enforced. Clusters +# created with this sample template +# would have alerting turned off by default, +# but end users could still turn alerting +# on or off. + +enable_cluster_monitoring: true +# This setting is not enforced. Clusters +# created with this sample template +# would have monitoring turned on +# by default, but end users could still +# turn monitoring on or off. + enable_network_policy: false local_cluster_auth_endpoint: enabled: true From c83c7a0de83c1540fccf2e7fdd6c7bb112744b40 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 31 Oct 2019 09:40:59 -0700 Subject: [PATCH 070/103] Say that Windows Server 1809 also works with Docker 19.03 --- .../rke-clusters/windows-clusters/_index.md | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md index 6dc198a6e99..e033ca87427 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md @@ -42,16 +42,10 @@ For a custom cluster, the general node requirements for networking, operating sy ### OS and Docker Requirements -In order to add Windows worker nodes to a cluster, the node must be running one of the following Windows Server versions and the corresponding version of Docker: +In order to add Windows worker nodes to a cluster, the node must be running one of the following Windows Server versions and the corresponding version of Docker Engine - Enterprise Edition (EE): -- Windows Server core version 1809 and Docker 18.09 -- Windows server core version 1903 and Docker 19.03 - -The nodes must run Docker Engine - Enterprise Edition (EE). - -Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09. - -Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. +- Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09 or Docker EE-basic 19.03. +- Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. > **Notes:** > From 36ce0a442340d93bd5b7a5d0bc8b0acc810ebaa8 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 19 Nov 2019 16:54:19 -0700 Subject: [PATCH 071/103] Update Docker requirements for Windows nodes --- content/rancher/v2.x/en/installation/requirements/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/requirements/_index.md b/content/rancher/v2.x/en/installation/requirements/_index.md index 47c1f17e9a7..722a465a8f3 100644 --- a/content/rancher/v2.x/en/installation/requirements/_index.md +++ b/content/rancher/v2.x/en/installation/requirements/_index.md @@ -39,7 +39,7 @@ Windows Server 2019 (64-bit x86) | Requires Docker Engine - Enterprise Edition ( \* Some distributions of Linux derived from RHEL, including Oracle Linux, may have default firewall rules that block communication with Helm. This [how-to guide]({{}}/rancher/v2.x/en/installation/options/firewall) shows how to check the default firewall rules and how to open the ports with `firewalld` if necessary. -\** Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09. Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/) +\** Nodes with Windows Server core version 1809 should use Docker EE-basic 18.09 or Docker EE-basic 19.03. Nodes with Windows Server core version 1903 should use Docker EE-basic 19.03. Supported for worker nodes only. See [Configuring Custom Clusters for Windows]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/) If you plan to run Rancher on ARM64, see [Running on ARM64 (Experimental)]({{}}/rancher/v2.x/en/installation/arm64-platform/) From 23b52f53d3e3bf308149c2f4f6af2a608fa5970a Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Sat, 23 Nov 2019 13:30:56 -0700 Subject: [PATCH 072/103] Clarify ingress and load balancer docs (#1964) * Clarify ingress and load balancer docs * Edit ingress and load balancer doc --- .../load-balancers-and-ingress/_index.md | 24 +++++++++++++------ .../load-balancers/_index.md | 23 +++++++++++------- 2 files changed, 32 insertions(+), 15 deletions(-) diff --git a/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/_index.md index c2821d5c62a..ef8e7c8d762 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/_index.md @@ -3,13 +3,13 @@ title: Load Balancing and Ingresses weight: 3040 --- -Within Rancher, you can setup load balancers and ingress controllers to redirect service requests. +Within Rancher, you can set up load balancers and ingress controllers to redirect service requests. ## Load Balancers After you launch an application, the app is only available within the cluster. It can't be reached from outside the cluster. -If you want your applications to be externally accessible, you must add a load balancer to your cluster. Load balancers create a gateway for external connections to access your cluster, provided that the user knows the load balancer's IP address and the application's port number. +If you want your applications to be externally accessible, you must add a load balancer or ingress to your cluster. Load balancers create a gateway for external connections to access your cluster, provided that the user knows the load balancer's IP address and the application's port number. Rancher supports two types of load balancers: @@ -33,18 +33,28 @@ Load Balancers have a couple of limitations you should be aware of: ## Ingress -As mentioned in the limitations above, using a load balancer per service can be expensive. You can get around this issue using an ingress. +As mentioned in the limitations above, the disadvantages of using a load balancer are: -Ingress is a set or rules that act as a load balancer. Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster program the load balancer to direct the request to the correct service based on service subdomains or path rules that you've configured. +- Load Balancers can only handle one IP address per service. +- If you run multiple services in your cluster, you must have a load balancer for each service. +- It can be expensive to have a load balancer for every service. -Your load balancer can either reside within your cluster or externally. Ingress and ingress controllers residing in RKE-launcher clusters are powered by [Nginx](https://www.nginx.com/). +In contrast, when an ingress is used as the entrypoint into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service. + +Therefore, it is useful to have an ingress if you want multiple services to be exposed with the same IP address, the same Layer 7 protocol, or the same privileged node-ports: 80 and 443. + +Ingress works in conjunction with one or more ingress controllers to dynamically route service requests. When the ingress receives a request, the ingress controller(s) in your cluster direct the request to the correct service based on service subdomains or path rules that you've configured. + +Each Kubernetes Ingress resource corresponds roughly to a file in `/etc/nginx/sites-available/` containing a `server{}` configuration block, where requests for specific files and folders are configured. + +Your ingress, which creates a port of entry to your cluster similar to a load balancer, can reside within your cluster or externally. Ingress and ingress controllers residing in RKE-launcher clusters are powered by [Nginx](https://www.nginx.com/). Ingress can provide other functionality as well, such as SSL termination, name-based virtual hosting, and more. >**Using Rancher in a High Availability Configuration?** > ->Refrain from adding an Ingress to the `local` cluster. The Nginx Ingress Controller that Rancher uses acts as a global load balancer for _all_ clusters managed by Rancher, including the `local` cluster. Therefore, when users try to access an application, your Rancher connection may drop due to the Nginx configuration being reloaded. We recommend working around this issue by deploying applications only in clusters that you launch using Rancher. +>Refrain from adding an Ingress to the `local` cluster. The Nginx Ingress Controller that Rancher uses acts as a global entry point for _all_ clusters managed by Rancher, including the `local` cluster. Therefore, when users try to access an application, your Rancher connection may drop due to the Nginx configuration being reloaded. We recommend working around this issue by deploying applications only in clusters that you launch using Rancher. -- For more information on how to setup ingress in Rancher, see [Ingress]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress). +- For more information on how to set up ingress in Rancher, see [Ingress]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress). - For complete information about ingress and ingress controllers, see the [Kubernetes Ingress Documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/) - When using ingresses in a project, you can program the ingress hostname to an external DNS by setting up a Global DNS entry, see [Global DNS]({{< baseurl >}}/rancher/v2.x/en/catalog/globaldns/). diff --git a/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/_index.md b/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/_index.md index 77afd316aa7..c00f76846db 100644 --- a/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/_index.md +++ b/content/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/_index.md @@ -8,7 +8,11 @@ Kubernetes supports load balancing in two ways: Layer-4 Load Balancing and Layer ## Layer-4 Load Balancer -Layer-4 load balancer (or the external load balancer) forwards traffic to Nodeports. Layer-4 load balancer allows you to forward both HTTP and TCP traffic. Layer-4 load balancer is supported by the underlying cloud provider. As a result, when you deploy RKE clusters on bare metal servers and vSphere clusters, layer-4 load balancer is not supported. +Layer-4 load balancer (or the external load balancer) forwards traffic to Nodeports. Layer-4 load balancer allows you to forward both HTTP and TCP traffic. + +Often, the Layer-4 load balancer is supported by the underlying cloud provider, so when you deploy RKE clusters on bare-metal servers and vSphere clusters, Layer-4 load balancer is not supported. However, a single [globally managed config-map](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) can be used to expose services on NGINX or third-party ingress. + +> **Note:** It is possible to deploy a cluster with a non-cloud load balancer, such as [MetalLB.](https://metallb.universe.tf/) However, that use case is more advanced than the Layer-4 load balancer supported by a cloud provider, and it is not configurable in Rancher or RKE. ### Support for Layer-4 Load Balancing @@ -16,13 +20,16 @@ Support for layer-4 load balancer varies based on the underlying cloud provider. Cluster Deployment | Layer-4 Load Balancer Support ----------------------------------------------|-------------------------------- -Amazon EKS | Supported by AWS cloud provider -Google GKE | Supported by GCE cloud provider -Azure AKS | Supported by Azure cloud provider -RKE on EC2 | Supported by AWS cloud provider -RKE on DigitalOcean | Not Supported -RKE on vSphere | Not Supported -RKE on Custom Hosts
(e.g. bare-metal servers) | Not Supported +Amazon EKS | Supported by AWS cloud provider +Google GKE | Supported by GCE cloud provider +Azure AKS | Supported by Azure cloud provider +RKE on EC2 | Supported by AWS cloud provider +RKE on DigitalOcean | Limited NGINX or third-party Ingress* +RKE on vSphere | Limited NGINX or third party-Ingress* +RKE on Custom Hosts
(e.g. bare-metal servers) | Limited NGINX or third-party Ingress* +Third-party MetalLB | Limited NGINX or third-party Ingress* + +\* Services can be exposed through a single [globally managed config-map.](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) ## Layer-7 Load Balancer From 5b37a78988bcc486cbeb07889f170787ca62513e Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 20 Nov 2019 12:07:50 -0700 Subject: [PATCH 073/103] Fix formatting issue on Windows page --- .../rke-clusters/windows-clusters/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md index e033ca87427..9042b9c9857 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/_index.md @@ -154,7 +154,7 @@ Windows support only be enabled if the cluster uses Kubernetes v1.15+ and the Fl 1. Click **Next**. -> **Important:** For **Host Gateway (L2bridge)** networking, it's best to use the same Layer 2 network for all nodes. Otherwise, you need to configure the route rules for them. For details, refer to the [documentation on configuring cloud-hosted VM routes.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#cloud-hosted-vm-routes-configuration) You will also need to [disable private IP address checks]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#disabling-private-ip-address-checks) if you are using Amazon EC2, Google GCE, or Azure VM. +> **Important:** For Host Gateway (L2bridge) networking, it's best to use the same Layer 2 network for all nodes. Otherwise, you need to configure the route rules for them. For details, refer to the [documentation on configuring cloud-hosted VM routes.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#cloud-hosted-vm-routes-configuration) You will also need to [disable private IP address checks]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/windows-clusters/host-gateway-requirements/#disabling-private-ip-address-checks) if you are using Amazon EC2, Google GCE, or Azure VM. # 3. Add Nodes to the Cluster From 83dd00ee7ce39e571bc6d8ca4e19e8a76716f368 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 12 Nov 2019 18:13:46 -0700 Subject: [PATCH 074/103] Describe how to convert existing clusters to use an RKE template --- .../en/admin-settings/rke-templates/_index.md | 20 +++++++--- .../applying-templates/_index.md | 34 ++++++++++++++++- .../creating-and-revising/_index.md | 38 ++++++++++++++++--- 3 files changed, 79 insertions(+), 13 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md index 6ebcedc3ac5..10dd222989f 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md @@ -17,7 +17,9 @@ Admins control which cluster options can be changed by end users. RKE templates If a cluster was created with an RKE template, you can't change it to a different RKE template. You can only update the cluster to a new revision of the same template. -To summarize, RKE templates allow DevOps and security teams to: +As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. + +The core features of RKE templates allow DevOps and security teams to: - Standardize cluster configuration and ensure that Rancher-provisioned clusters are created following best practices - Prevent less technical users from making uninformed choices when provisioning clusters @@ -48,7 +50,10 @@ RKE templates are supported for Rancher-provisioned clusters. The templates can RKE templates are for defining Kubernetes and Rancher settings. Node templates are responsible for configuring nodes. For tips on how to use RKE templates in conjunction with hardware, refer to [RKE Templates and Hardware]({{}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware). -RKE templates can be applied to new clusters, but not existing clusters. +RKE templates can be applied to new clusters. + +As of v2.3.3, the settings of an existing cluster can be [saved as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) This creates a new template and binds the cluster settings to the template, so that the cluster can only be upgraded if the [template is updated]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template), and the cluster is upgraded to [use a newer version of the template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) The new template can also be used to create new clusters. + # Example Scenarios When an organization has both basic and advanced Rancher users, administrators might want to give the advanced users more options for cluster creation, while restricting the options for basic users. @@ -58,13 +63,13 @@ These [example scenarios]({{}}/rancher/v2.x/en/admin-settings/rke-templ Some of the example scenarios include the following: - **Enforcing templates:** Administrators might want to [enforce one or more template settings for everyone]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#enforcing-a-template-setting-for-everyone) if they want all new Rancher-provisioned clusters to have those settings. -- **Sharing different templates with different users:** Administrators might give [different templates to basic and advanced users,]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#templates-for-basic-and-advanced-users) so that basic users can have more restricted options and advanced users can have more discretion when creating clusters. -- **Updating template settings:** If an organization's security and DevOps teams decide to embed best practices into the required settings for new clusters, those best practices could change over time. If the best practices change, [a template can be updated to a new revision]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#updating-templates-and-clusters-created-with-them) and clusters created from the template can upgrade to the new version of the template. +- **Sharing different templates with different users:** Administrators might give [different templates to basic and advanced users,]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#templates-for-basic-and-advanced-users) so that basic users can have more restricted options and advanced users can use more discretion when creating clusters. +- **Updating template settings:** If an organization's security and DevOps teams decide to embed best practices into the required settings for new clusters, those best practices could change over time. If the best practices change, [a template can be updated to a new revision]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#updating-templates-and-clusters-created-with-them) and clusters created from the template can [upgrade to the new version]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) of the template. - **Sharing ownership of a template:** When a template owner no longer wants to maintain a template, or wants to share ownership of the template, this scenario describes how [template ownership can be shared.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/example-scenarios/#allowing-other-users-to-control-and-share-a-template) # Template Management -When you create a RKE template, it is available in the Rancher UI from the **Global** view under **Tools > RKE Templates.** When you create a template, you become the template owner, which gives you permission to revise and share the template. You can share the RKE templates with specific users or groups, and you can also make it public. +When you create an RKE template, it is available in the Rancher UI from the **Global** view under **Tools > RKE Templates.** When you create a template, you become the template owner, which gives you permission to revise and share the template. You can share the RKE templates with specific users or groups, and you can also make it public. Administrators can turn on template enforcement to require users to always use RKE templates when creating a cluster. This allows administrators to guarantee that Rancher always provisions clusters with specific settings. @@ -91,9 +96,12 @@ You can [create a cluster from a template]({{}}/rancher/v2.x/en/admin-s If the RKE template owner creates a new revision of the template, you can [upgrade your cluster to that revision.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#updating-a-cluster-created-with-an-rke-template) -RKE templates can only be applied to new clusters, not existing clusters. +RKE templates can be applied to new clusters. + +As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. # Standardizing Hardware + RKE templates are designed to standardize Kubernetes and Rancher settings. If you want to standardize your infrastructure as well, you use RKE templates [in conjuction with other tools]({{}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware). # YAML Customization diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/_index.md index 68a5a8d29a5..1a3010aefe0 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/_index.md @@ -5,9 +5,17 @@ weight: 50 You can create a cluster from an RKE template that you created, or from a template that has been [shared with you.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/template-access-and-sharing) -RKE templates can only be applied to new clusters, not existing clusters. +RKE templates can be applied to new clusters. -You can't change the cluster to use a different RKE template. You can only update the cluster to a new revision of the same template. +As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.](#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. + +You can't change a cluster to use a different RKE template. You can only update the cluster to a new revision of the same template. + +This section covers the following topics: + +- [Creating a cluster from an RKE template](#creating-a-cluster-from-an-rke-template) +- [Updating a cluster created with an RKE template](#updating-a-cluster-created-with-an-rke-template) +- [Converting an existing cluster to use an RKE template](#converting-an-existing-cluster-to-use-an-rke-template) ### Creating a Cluster from an RKE Template @@ -30,4 +38,26 @@ When the template owner creates a template, each setting has a switch in the Ran If a cluster was created from an RKE template, you can edit the cluster to update the cluster to a new revision of the template. +As of Rancher v2.3.3, an existing cluster's settings can be [saved as an RKE template.](#converting-an-existing-cluster-to-use-an-rke-template) In that situation, you can also edit the cluster to update the cluster to a new revision of the template. + > **Note:** You can't change the cluster to use a different RKE template. You can only update the cluster to a new revision of the same template. + +### Converting an Existing Cluster to Use an RKE Template + +_Available as of v2.3.3_ + +This section describes how to create an RKE template from an existing cluster. + +RKE templates cannot be applied to existing clusters, except if you save an existing cluster's settings as an RKE template. This exports the cluster's settings as a new RKE template, and also binds the cluster to that template. The result is that the cluster can only be changed if the [template is updated,]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template) and the cluster is upgraded to [use a newer version of the template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) + +To convert an existing cluster to use an RKE template, + +1. From the **Global** view in Rancher, click the **Clusters** tab. +1. Go to the cluster that will be converted to use an RKE template. Click **Ellipsis (...)** > **Save as RKE Template.** +1. Enter a name for the template in the form that appears, and click **Create.** + +**Results:** + +- A new RKE template is created. +- The cluster is converted to use the new template. +- New clusters can be [created from the new template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#creating-a-cluster-from-an-rke-template) \ No newline at end of file diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md index 983aa6bc4f6..417de25704f 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md @@ -7,10 +7,24 @@ This section describes how to manage RKE templates and revisions. You an create, Template updates are handled through a revision system. When template owners want to change or update a template, they create a new revision of the template. Individual revisions cannot be edited. However, if you want to prevent a revision from being used to create a new cluster, you can disable it. -Template revisions can be used in two ways: to create a new cluster, or to upgrade a cluster that was created with an earlier version of the template. The template creator can choose a default revision, but when end users create a cluster, they can choose any template and any template revision that is available to them. After the cluster is created from a specific revision, it cannot change to another template, but the cluster can be upgraded to another available revision of the same template. +Template revisions can be used in two ways: to create a new cluster, or to upgrade a cluster that was created with an earlier version of the template. The template creator can choose a default revision, but when end users create a cluster, they can choose any template and any template revision that is available to them. After the cluster is created from a specific revision, it cannot change to another template, but the cluster can be upgraded to a newer available revision of the same template. The template owner has full control over template revisions, and can create new revisions to update the template, delete or disable revisions that should not be used to create clusters, and choose which template revision is the default. +This section covers the following topics: + +- [Prerequisites](#prerequisites) +- [Creating a template](#creating-a-template) +- [Updating a template](#updating-a-template) +- [Deleting a template](#deleting-a-template) +- [Creating a revision based on the default revision](#creating-a-revision-based-on-the-default-revision) +- [Creating a revision based on a cloned revision](#creating-a-revision-based-on-a-cloned-revision) +- [Disabling a template revision](#disabling-a-template-revision) +- [Re-enabling a disabled template revision](#re-enabling-a-disabled-template-revision) +- [Setting a template revision as default](#setting-a-template-revision-as-default) +- [Deleting a template revision](#deleting-a-template-revision) +- [Upgrading a cluster to use a new template revision](#upgrading-a-cluster-to-use-a-new-template-revision) + ### Prerequisites You can create RKE templates if you have the **Create RKE Templates** permission, which can be [given by an administrator.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creator-permissions) @@ -25,7 +39,7 @@ You can revise, share, and delete a template if you are an owner of the template 1. Optional: Share the template with other users or groups by [adding them as members.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/template-access-and-sharing/#sharing-templates-with-specific-users) You can also make the template public to share with everyone in the Rancher setup. 1. Then follow the form on screen to save the cluster configuration parameters as part of the template's revision. The revision can be marked as default for this template. -**Result:** An RKE template with one revision is configured. You can use this RKE template revision later when you [provision a Rancher-launched cluster]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters). +**Result:** An RKE template with one revision is configured. You can use this RKE template revision later when you [provision a Rancher-launched cluster]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters). ### Updating a Template @@ -33,14 +47,14 @@ When you update an RKE template, you are creating a revision of the existing tem You can't edit individual revisions. Since you can't edit individual revisions of a template, in order to prevent a revision from being used, you can [disable it.](#disabling-a-template-revision) -New template revisions can be created without affecting clusters already using a revision of the template. +When new template revisions are created, clusters using an older revision of the template are unaffected. 1. From the **Global** view, click **Tools > RKE Templates.** 1. Go to the template that you want to edit and click the **Vertical Ellipsis (...) > Edit.** 1. Edit the required information and click **Save.** 1. Optional: You can change the default revision of this template and also change who it is shared with. -**Result:** The template is updated. +**Result:** The template is updated. To apply it to a cluster using an older version of the template, refer to the section on [upgrading a cluster to use a new revision of a template.](#upgrading-a-cluster-to-use-a-new-template-revision) ### Deleting a Template @@ -112,4 +126,18 @@ To permanently delete a revision, 1. From the **Global** view, click **Tools > RKE Templates.** 1. Go to the RKE template revision that should be deleted and click the **Ellipsis (...) > Delete.** -**Result:** The RKE template revision is deleted. \ No newline at end of file +**Result:** The RKE template revision is deleted. + +### Upgrading a Cluster to Use a New Template Revision + +> This section assumes that you already have a cluster that [has an RKE template applied.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates) +> This section also assumes that you have [updated the template that the cluster is using](#updating-a-template) so that a new template revision is available. + +To upgrade a cluster to use a new template revision, + +1. From the **Global** view in Rancher, click the **Clusters** tab. +1. Go to the cluster that you want to upgrade and click **Ellipsis (...) > Edit.** +1. In the **Cluster Options** section, click the dropdown menu for the template revision, then select the new template revision. +1. Click **Save.** + +**Result:** The cluster is upgrades to use the settings defined in the new template revision. \ No newline at end of file From cf5a362de892c7e259690542ce98b944527b7305 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 19 Nov 2019 16:07:59 -0700 Subject: [PATCH 075/103] Minor edits to docs on RKE templates --- .../en/admin-settings/rke-templates/_index.md | 6 +++--- .../creating-and-revising/_index.md | 21 ++++++++++++++++++- 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md index 10dd222989f..34ac570d8e1 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/_index.md @@ -17,7 +17,7 @@ Admins control which cluster options can be changed by end users. RKE templates If a cluster was created with an RKE template, you can't change it to a different RKE template. You can only update the cluster to a new revision of the same template. -As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. +As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. The new template can also be used to launch new clusters. The core features of RKE templates allow DevOps and security teams to: @@ -50,7 +50,7 @@ RKE templates are supported for Rancher-provisioned clusters. The templates can RKE templates are for defining Kubernetes and Rancher settings. Node templates are responsible for configuring nodes. For tips on how to use RKE templates in conjunction with hardware, refer to [RKE Templates and Hardware]({{}}/rancher/v2.x/en/admin-settings/rke-templates/rke-templates-and-hardware). -RKE templates can be applied to new clusters. +RKE templates can be created from scratch to pre-define cluster configuration. They can be applied to launch new clusters, or templates can also be exported from existing running clusters. As of v2.3.3, the settings of an existing cluster can be [saved as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) This creates a new template and binds the cluster settings to the template, so that the cluster can only be upgraded if the [template is updated]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template), and the cluster is upgraded to [use a newer version of the template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#upgrading-a-cluster-to-use-a-new-template-revision) The new template can also be used to create new clusters. @@ -96,7 +96,7 @@ You can [create a cluster from a template]({{}}/rancher/v2.x/en/admin-s If the RKE template owner creates a new revision of the template, you can [upgrade your cluster to that revision.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#updating-a-cluster-created-with-an-rke-template) -RKE templates can be applied to new clusters. +RKE templates can be created from scratch to pre-define cluster configuration. They can be applied to launch new clusters, or templates can also be exported from existing running clusters. As of Rancher v2.3.3, you can [save the configuration of an existing cluster as an RKE template.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#converting-an-existing-cluster-to-use-an-rke-template) Then the cluster's settings can only be changed if the template is updated. diff --git a/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md b/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md index 417de25704f..1c0c4711596 100644 --- a/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/_index.md @@ -24,6 +24,7 @@ This section covers the following topics: - [Setting a template revision as default](#setting-a-template-revision-as-default) - [Deleting a template revision](#deleting-a-template-revision) - [Upgrading a cluster to use a new template revision](#upgrading-a-cluster-to-use-a-new-template-revision) +- [Exporting a running cluster to a new RKE template and revision](#exporting-a-running-cluster-to-a-new-rke-template-and-revision) ### Prerequisites @@ -140,4 +141,22 @@ To upgrade a cluster to use a new template revision, 1. In the **Cluster Options** section, click the dropdown menu for the template revision, then select the new template revision. 1. Click **Save.** -**Result:** The cluster is upgrades to use the settings defined in the new template revision. \ No newline at end of file +**Result:** The cluster is upgraded to use the settings defined in the new template revision. + +### Exporting a Running Cluster to a New RKE Template and Revision + +You can save an existing cluster's settings as an RKE template. + +This exports the cluster's settings as a new RKE template, and also binds the cluster to that template. The result is that the cluster can only be changed if the [template is updated,]({{}}/rancher/v2.x/en/admin-settings/rke-templates/creating-and-revising/#updating-a-template) and the cluster is upgraded to [use a newer version of the template.] + +To convert an existing cluster to use an RKE template, + +1. From the **Global** view in Rancher, click the **Clusters** tab. +1. Go to the cluster that will be converted to use an RKE template. Click **Ellipsis (...)** > **Save as RKE Template.** +1. Enter a name for the template in the form that appears, and click **Create.** + +**Results:** + +- A new RKE template is created. +- The cluster is converted to use the new template. +- New clusters can be [created from the new template and revision.]({{}}/rancher/v2.x/en/admin-settings/rke-templates/applying-templates/#creating-a-cluster-from-an-rke-template) \ No newline at end of file From afc2cab89e12b46648944ba3b089d620a1e75cf6 Mon Sep 17 00:00:00 2001 From: jodysimpson Date: Tue, 26 Nov 2019 09:32:12 -0800 Subject: [PATCH 076/103] Fixed typo on pipeline documentation --- .../en/project-admin/tools/pipelines/docs-for-v2.0.x/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/project-admin/tools/pipelines/docs-for-v2.0.x/_index.md b/content/rancher/v2.x/en/project-admin/tools/pipelines/docs-for-v2.0.x/_index.md index 7f78c61749e..0414cfa3e90 100644 --- a/content/rancher/v2.x/en/project-admin/tools/pipelines/docs-for-v2.0.x/_index.md +++ b/content/rancher/v2.x/en/project-admin/tools/pipelines/docs-for-v2.0.x/_index.md @@ -45,7 +45,7 @@ You can set up your pipeline to run a series of stages and steps to test your co - Only the branch {BRANCH NAME}: Only events triggered by changes to this branch will be built. - - Evertyhing but {BRANCH NAME}: Build any branch that triggered an event EXCEPT events from this branch. + - Everything but {BRANCH NAME}: Build any branch that triggered an event EXCEPT events from this branch. - All branches: Regardless of the branch that triggered the event always build. From 7fc3a113f44ff9cb1d346bef412b403d3cf86d8e Mon Sep 17 00:00:00 2001 From: Robert Parker Date: Tue, 26 Nov 2019 14:22:11 -0800 Subject: [PATCH 077/103] add redirects --- nginx.conf | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/nginx.conf b/nginx.conf index d9bf2807047..7431696d44b 100644 --- a/nginx.conf +++ b/nginx.conf @@ -2,6 +2,49 @@ map_hash_bucket_size 256; map $request_uri $redirect_uri { ~^/docs/rancher/v2.0(/?.*)$ /docs/rancher/v2.x$1; ~^/docs/rke/v0.1.x(/?.*)$ /docs/rke/latest$1; + ~^/docs/os/quick-start-guide/?$ /rancher-os; + ~^/docs/rancher/v1.0/zh/rancher-compose/?$ /docs/rancher/v2.x/en/k8s-in-rancher/workloads/add-a-sidecar/; + ~^/docs/rancher/v1.0/zh/rancher-compose/?$ /docs/rancher/v1.6/en/cattle/rancher-compose/; + ~^/docs/rancher/rancher-ui/applications/stacks/adding-balancers/?$ /docs/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/load-balancers/; + ~^/docs/os/running-rancheros/server/install-to-disk/?$ /os/v1.x/en/quick-start-guide/; + ~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/overview/; + ~^/docs/rancher/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/; + ~^/docs/os/running-rancheros/server/install-to-disk/?$ /docs/os/v1.x/en/quick-start-guide/; + ~^/docs/os/running-rancheros/cloud/gce/?$ /docs/os/v1.x/en/installation/running-rancheros/cloud/gce/; + ~^/docs/os/amazon-ecs/?$ /docs/os/v1.x/en/installation/running-rancheros/cloud/aws/; + ~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/overview/architecture/; + ~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/; + ~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/overview/architecture/; + ~^/docs/rancher/rancher-compose/?$ /docs/rancher/v2.x/en/; + ~^/docs/os/networking/interfaces/?$ /docs/os/v1.x/en/installation/networking/interfaces/; + ~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/; + ~^/docs/rancher/concepts/?$ /docs/rancher/v2.x/en/; + ~^/docs/rancher/rancher-services/storage-service/?$ /docs/rancher/v2.x/en/cluster-admin/volumes-and-storage/; + ~^/docs/rancher/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/installation/ha/; + ~^/docs/rancher/upgrading/?$ /docs/rancher/v2.x/en/upgrades/; + ~^/docs/rancher/configuration/access-control/?$ /docs/rancher/v2.x/en/admin-settings/rbac/; + ~^/docs/os/running-rancheros/server/install-to-disk/?$ /docs/os/v1.x/en/installation/running-rancheros/; + ~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/; + ~^/docs/os/configuration/custom-rancheros-iso/?$ /docs/os/v1.x/en/installation/configuration/; + ~^/docs/rancher/rancher-compose/?$ /docs/rancher/v2.x/en/; + ~^/docs/os/running-rancheros/server/raspberry-pi/?$ /docs/os/v1.x/en/installation/running-rancheros/server/raspberry-pi/; + ~^/docs/rancher/v1.0/zh/installing-rancher/installing-server/?$ /docs/rancher/v1.6/en/installing-rancher/installing-server/; + ~^/docs/rancher/rancher-services/metadata-service/?$ /docs/rancher/v1.6/en/rancher-services/metadata-service/; + ~^/docs/rancher/api/?$ /docs/rancher/v2.x/en/api/; + ~^/docs/os/running-rancheros/server/raspberry-pi/?$ /docs/os/v1.x/en/installation/running-rancheros/server/raspberry-pi/; + ~^/docs/os/quick-start-guide/?$ /docs/os/v1.x/en/quick-start-guide/; + ~^/docs/os/configuration/switching-consoles/?$ /docs/os/v1.x/en/about/recovery-console/; + ~^/docs/os/running-rancheros/server/install-to-disk//?$ /docs/os/v1.x/en/installation/running-rancheros/server/install-to-disk/; + ~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/; + ~^/docs/rancher/v1.2/en/installing-rancher/installing-server/multi-nodes//?$ /docs/rancher/v2.x/en/installation/ha/; + ~^/docs/os/running-rancheros/workstation/boot-from-iso/?$ /docs/os/v1.x/en/installation/running-rancheros/workstation/boot-from-iso/; + ~^/docs/os/system-services/adding-system-services/?$ /docs/os/v1.x/en/installation/system-services/adding-system-services/; + ~^/docs/rancher/installing-rancher/installing-server/?$ /docs/rancher/v2.x/en/installation/; + ~^/docs/rancher/latest/en/installing-rancher/installing-server/multi-nodes/?$ /docs/rancher/v2.x/en/installation/ha/; + ~^/docs/rancher/rancher-services/metadata-service/?$ /docs/rancher/v1.6/en/rancher-services/metadata-service/; + ~^/docs/rancher/rancher-services/health-checks/?$ /docs/rancher/v1.0/en/rancher-services/health-checks/; + ~^/docs/os/cloud-config/?$ /docs/os/v1.x/en/installation/configuration/; + ~^/docs/rancher/api/?$ /docs/rancher/v2.x/en/api/; } server { From da8e35e30697bc0d706de75de2784aae2cf02a07 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 26 Nov 2019 22:02:02 -0700 Subject: [PATCH 078/103] Add note about Helm 3 changes --- content/rancher/v2.x/en/installation/ha/helm-init/_index.md | 2 ++ content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md | 2 ++ content/rancher/v2.x/en/installation/helm-version/_index.md | 2 ++ 3 files changed, 6 insertions(+) diff --git a/content/rancher/v2.x/en/installation/ha/helm-init/_index.md b/content/rancher/v2.x/en/installation/ha/helm-init/_index.md index 4eb518162ab..e0793246f7f 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-init/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-init/_index.md @@ -9,6 +9,8 @@ For systems without direct internet access, see [Helm - Air Gap]({{< baseurl >}} Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher. +> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) + ### Install Tiller on the Cluster > **Important:** Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher. diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index 23eb02a9cf9..ecf5b847dac 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -9,6 +9,8 @@ For systems without direct internet access, see [Air Gap: High Availability Inst Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher. +> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) + ### Add the Helm Chart Repository Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories). diff --git a/content/rancher/v2.x/en/installation/helm-version/_index.md b/content/rancher/v2.x/en/installation/helm-version/_index.md index 175fe9cbb78..2c07d4c4dee 100644 --- a/content/rancher/v2.x/en/installation/helm-version/_index.md +++ b/content/rancher/v2.x/en/installation/helm-version/_index.md @@ -5,6 +5,8 @@ weight: 400 This section contains the requirements for Helm, which is the tool used to install Rancher on a high-availability Kubernetes cluster. +> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) + - Helm v2.15.1 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using. - Helm v2.15.0 should not be used, because of an issue with converting/comparing numbers. - Helm v2.12.0 should not be used, because of an issue with `cert-manager`. \ No newline at end of file From 08b6cceabc9e1602df1ae4b96c92ed7787c40a9b Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 27 Nov 2019 08:06:59 -0700 Subject: [PATCH 079/103] Revise and update vSphere docs for Rancher v2.3.3 changes (#2016) * Revise and update vSphere docs for Rancher v2.3.3 changes * Edit vSphere docs * Update _index.md * Update _index.md * Update _index.md --- .../rke-clusters/node-pools/_index.md | 2 + .../rke-clusters/node-pools/vsphere/_index.md | 167 ++-------- .../provisioning-vsphere-clusters/_index.md | 305 ++++++++++++++++++ .../creating-credentials/_index.md | 41 +++ .../enabling-uuids/_index.md | 24 ++ .../node-template-reference/_index.md | 93 ++++++ .../cloud-providers/vsphere/_index.md | 267 ++------------- .../vsphere/config-reference/_index.md | 144 +++++++++ .../vsphere/enabling-uuid/_index.md | 35 ++ .../vsphere/troubleshooting/_index.md | 27 ++ 10 files changed, 708 insertions(+), 397 deletions(-) create mode 100644 content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md create mode 100644 content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials/_index.md create mode 100644 content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md create mode 100644 content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/_index.md create mode 100644 content/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/_index.md create mode 100644 content/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid/_index.md create mode 100644 content/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting/_index.md diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/_index.md index 7c44a74833f..7fc3ae981df 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/_index.md @@ -45,6 +45,8 @@ _Available as of Rancher v2.3.0_ If a node is in a node pool, Rancher can automatically replace unreachable nodes. Rancher will use the existing node template for the given node pool to recreate the node if it becomes inactive for a specified number of minutes. +> **Important** Self-healing node pools are designed to help you replace worker nodes for stateless applications. It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications. + {{% accordion id="how-does-node-auto-replace-work" label="How does Node Auto-replace Work?" %}} Node auto-replace works on top of the Kubernetes node controller. The node controller periodically checks the status of all the nodes (configurable via the `--node-monitor-period` flag of the `kube-controller`). When a node is unreachable, the node controller will taint that node. When this occurs, Rancher will begin its deletion countdown. You can configure the amount of time Rancher waits to delete the node. If the taint is not removed before the deletion countdown ends, Rancher will proceed to delete the node object. Rancher will then provision a node in accordance with the set quantity of the node pool. {{% /accordion %}} diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md index 517c0d0f38d..7702ec28294 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md @@ -5,170 +5,39 @@ weight: 2225 aliases: - /rancher/v2.x/en/tasks/clusters/creating-a-cluster/create-cluster-vsphere/ --- -Use {{< product >}} to create a Kubernetes cluster in vSphere. -## Introduction +By using Rancher with vSphere, you can bring cloud operations on-premises. -When creating a vSphere cluster, Rancher first provisions the specified amount of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them. A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for the data, control, and worker plane respectively. +Rancher can provision nodes in vSphere and install Kubernetes on them. When creating a Kubernetes cluster in vSphere, Rancher first provisions the specified number of virtual machines by communicating with the vCenter API. Then it installs Kubernetes on top of them. ->**Note:** ->The vSphere node driver included in Rancher currently only supports the provisioning of VMs with [RancherOS]({{< baseurl >}}/os/v1.x/en/) as the guest operating system. +A vSphere cluster may consist of multiple groups of VMs with distinct properties, such as the amount of memory or the number of vCPUs. This grouping allows for fine-grained control over the sizing of nodes for each Kubernetes role. -## Prerequisites +# vSphere Enhancements -### vSphere API permissions +The vSphere node templates have been updated, allowing you to bring cloud operations on-premises with the following enhancements: -Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. If you are planning to make use of vSphere volumes for persistent storage in the cluster, there are [additional requirements]({{< baseurl >}}/rke/latest/en/config-options/cloud-providers/vsphere/) that must be met. +### Self-healing Node Pools -### Network permissions +_Available as of v2.3.0_ -You must ensure that the hosts running Rancher servers are able to establish network connections to the following network endpoints: +One of the biggest advantages of provisioning vSphere nodes with Rancher is that it allows you to take advantage of Rancher's self-healing node pools, also called the [node auto-replace feature,]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-auto-replace) in your on-premises clusters. Self-healing node pools are designed to help you replace worker nodes for stateless applications. When Rancher provisions nodes from a node template, Rancher can automatically replace unreachable nodes. -- vCenter server (usually port 443/TCP) -- Every ESXi host that is part of the datacenter to be used to provision virtual machines for your clusters (port 443/TCP). +> **Important:** It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications. +### Dynamically Populated Options for Instances and Scheduling -## Provisioning a vSphere Cluster +_Available as of v2.3.3_ -The following steps create a role with the required privileges and then assign it to a new user in the vSphere console: +Node templates for vSphere have been updated so that when you create a node template with your vSphere credentials, the template is automatically populated with the same options for provisioning VMs that you have access to in the vSphere console. -1. From the **vSphere** console, go to the **Administration** page. +For the fields to be populated, your setup needs to fulfill the [prerequisites.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#prerequisites) -2. Go to the **Roles** tab. +### More Supported Operating Systems -3. Create a new role. Give it a name and select the privileges listed in the [permissions table](#annex-vsphere-permissions). +As of Rancher v2.3.3+, you can provision VMs with any operating system that supports cloud init. - ![image]({{< baseurl >}}/img/rancher/rancherroles1.png) +In Rancher prior to v2.3.3, the vSphere node driver included in Rancher only supported the provisioning of VMs with [RancherOS]({{}}/os/v1.x/en/) as the guest operating system. -4. Go to the **Users and Groups** tab. +# Video Walkthrough of v2.3.3 Node Template Features -5. Create a new user. Fill out the form and then click **OK**. Make sure to note the username and password, as you will need it when configuring node templates in Rancher. - - ![image]({{< baseurl >}}/img/rancher/rancheruser.png) - -6. Go to the **Global Permissions** tab. - -7. Create a new Global Permission. Add the user you created earlier and assign it the role you created earlier. Click **OK**. - - ![image]({{< baseurl >}}/img/rancher/globalpermissionuser.png) - - ![image]({{< baseurl >}}/img/rancher/globalpermissionrole.png) - -## Creating vSphere Clusters - -### Create a vSphere Node Template - -To create a cluster, you need to create at least one vSphere [node template]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) that specifies how VMs are created in vSphere. - ->**Note:** ->Once you create a node template, it is saved, and you can re-use it whenever you create additional vSphere clusters. - -1. Log in with an admin account to the Rancher UI. - -2. From the user settings menu, select **Node Templates**. - -3. Click **Add Template** and then click on the **vSphere** icon. - -4. Under [Account Access](#account-access) enter the vCenter FQDN or IP address and the credentials for the vSphere user account (see [Prerequisites](#prerequisites)). - - {{< step_create-cloud-credential >}} - -5. Under [Instance Options](#instance-options), configure the number of vCPUs, memory, and disk size for the VMs created by this template. - -6. **Optional:** Enter the URL pointing to a [RancherOS]({{< baseurl >}}/os/v1.x/en/) cloud-config file in the [Cloud Init](#instance-options) field. - -7. Ensure that the [OS ISO URL](#instance-options) contains the URL of a VMware ISO release for RancherOS (`rancheros-vmware.iso`). - - ![image]({{< baseurl >}}/img/rancher/vsphere-node-template-1.png) - -8. **Optional:** Provide a set of [Configuration Parameters](#instance-options) for the VMs. - -9. Under **Scheduling**, enter the name/path of the **Data Center** to create the VMs in, the name of the **VM Network** to attach to, and the name/path of the **Datastore** to store the disks in. - - ![image]({{< baseurl >}}/img/rancher/vsphere-node-template-2.png) - -10. **Optional:** Assign labels to the VMs that can be used as a base for scheduling rules in the cluster. - -11. **Optional:** Customize the configuration of the Docker daemon on the VMs that will be created. - -10. Assign a descriptive **Name** for this template and click **Create**. - -___ - -### Create a vSphere Cluster - -After you've created a template, you can use it stand up the vSphere cluster itself. - -1. From the **Global** view, click **Add Cluster**. - -2. Choose **vSphere**. - -3. Enter a **Cluster Name**. - -4. {{< step_create-cluster_member-roles >}} - -5. {{< step_create-cluster_cluster-options >}} - -6. {{< step_create-cluster_node-pools >}} - - ![image]({{< baseurl >}}/img/rancher/vsphere-cluster-create-1.png) - -7. Review your configuration, then click **Create**. - -> **Note:** -> -> If you have a cluster with DRS enabled, setting up [VM-VM Affinity Rules](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.resmgmt.doc/GUID-7297C302-378F-4AF2-9BD6-6EDB1E0A850A.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes. - -{{< result_create-cluster >}} - -## Annex - Node Template Configuration Reference - -The tables below describe the configuration options available in the vSphere node template. - -### Account Access - -| Parameter | Required | Description | -|:------------------------:|:--------:|:------------------------------------------------------------:| -| vCenter or ESXi Server | * | IP or FQDN of the vCenter or ESXi server used for managing VMs. | -| Port | * | Port to use when connecting to the server. Defaults to `443`. | -| Username | * | vCenter/ESXi user to authenticate with the server. | -| Password | * | User's password. | - -___ - -### Instance Options - -| Parameter | Required | Description | -|:------------------------:|:--------:|:------------------------------------------------------------:| -| CPUs | * | Number of vCPUS to assign to VMs. | -| Memory | * | Amount of memory to assign to VMs. | -| Disk | * | Size of the disk (in MB) to attach to the VMs. | -| Cloud Init | | URL of a [RancherOS cloud-config]({{< baseurl >}}/os/v1.x/en/installation/configuration/) file to provision VMs with. This file allows further customization of the RancherOS operating system, such as network configuration, DNS servers, or system daemons.| -| OS ISO URL | * | URL of a RancherOS vSphere ISO file to boot the VMs from. You can find URLs for specific versions in the [Rancher OS GitHub Repo](https://github.com/rancher/os). | -| Configuration Parameters | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). | - -___ - -### Scheduling Options - -| Parameter | Required | Description | -|:------------------------:|:--------:|:------------------------------------------------------------:| -| Data Center | * | Name/path of the datacenter to create VMs in. | -| Pool | | Name/path of the resource pool to schedule the VMs in. If not specified, the default resource pool is used. | -| Host | | Name/path of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. | -| Network | * | Name of the VM network to attach VMs to. | -| Data Store | * | Datastore to store the VM disks. | -| Folder | | Name/path of folder in the datastore to create the VMs in. Must already exist. | - -___ - -## Annex - vSphere Permissions - -The following table lists the permissions required for the vSphere user account configured in the node templates: - -| Privilege Group | Operations | -|:----------------------|:-----------------------------------------------------------------------| -| Datastore | AllocateSpace
Browse
FileManagement (Low level file operations)
UpdateVirtualMachineFiles
UpdateVirtualMachineMetadata | -| Network | Assign | -| Resource | AssignVMToPool | -| Virtual Machine | Config (All)
GuestOperations (All)
Interact (All)
Inventory (All)
Provisioning (All) | +In [this YouTube video,](https://www.youtube.com/watch?v=dPIwg6x1AlU) we demonstrate how to set up a node template with the new features designed to help you bring cloud operations to on-premises clusters. diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md new file mode 100644 index 00000000000..39fa35b7528 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md @@ -0,0 +1,305 @@ +--- +title: Provisioning Kubernetes Clusters in vSphere +weight: 1 +--- + +This section explains how to configure Rancher with vSphere credentials, provision nodes in vSphere, and set up Kubernetes clusters on those nodes. + +# Prerequisites + +This section describes the requirements for setting up vSphere so that Rancher can provision VMs and clusters. + +The node templates are documented and tested with the vSphere Web Services API version 6.5. + +- [Create credentials in vSphere](#create-credentials-in-vsphere) +- [Network permissions](#network-permissions) +- [Valid ESXi License for vSphere API Access](#valid-esxi-license-for-vsphere-api-access) + +### Create Credentials in vSphere + +Before proceeding to create a cluster, you must ensure that you have a vSphere user with sufficient permissions. When you set up a node template, the template will need to use these vSphere credentials. + +Refer to this [how-to guide]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials) for instructions on how to create a user in vSphere with the required permissions. These steps result in a username and password that you will need to provide to Rancher, which allows Rancher to provision resources in vSphere. + +### Network Permissions + +There needs to be two-way communication between Rancher and the vSphere API. + +You must ensure that the hosts running Rancher servers are able to establish network connections to the following network endpoints: + +- vCenter server (usually port 443/TCP) +- Every ESXi host that is part of the datacenter to be used to provision virtual machines for your clusters (port 443/TCP). + +By default, Rancher uses port 443 to communicate with vSphere. + +The vSphere API websocket port will be 84453 by default. + +### Valid ESXi License for vSphere API Access + +The free ESXi license does not support API access. The vSphere servers must have a valid or evaluation ESXi license. + +# Creating Clusters in vSphere with Rancher + +This section describes how to set up vSphere credentials, node templates, and vSphere clusters using the Rancher UI. + +You will need to do the following: + +1. [Create a node template using vSphere credentials](#1-create-a-node-template-using-vsphere-credentials) +2. [Create a Kubernetes cluster using the node template](#2-create-a-kubernetes-cluster-using-the-node-template) +3. [Optional: Provision storage](#3-optional-provision-storage) + - [Enable the vSphere cloud provider for the cluster](#enable-the-vsphere-cloud-provider-for-the-cluster) + +### Configuration References + +For details on configuring the node template, refer to the [node template configuration reference.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/) + +Rancher uses the RKE library to provision Kubernetes clusters. For details on configuring clusters in vSphere, refer to the [cluster configuration reference in the RKE documentation.]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/) + +Note that the vSphere cloud provider must be [enabled](#enable-the-vsphere-cloud-provider-for-the-cluster) to allow dynamic provisioning of volumes. + +# 1. Create a Node Template Using vSphere Credentials + +To create a cluster, you need to create at least one vSphere [node template]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) that specifies how VMs are created in vSphere. + +After you create a node template, it is saved, and you can re-use it whenever you create additional vSphere clusters. + +To create a node template, + +1. Log in with an admin account to the Rancher UI. + +1. From the user settings menu, select **Node Templates.** + +1. Click **Add Template** and then click on the **vSphere** icon. + +Then, configure your template: + +- [A. Configure the vSphere credential](#a-configure-the-vsphere-credential) +- [B. Configure node scheduling](#b-configure-node-scheduling) +- [C. Configure instances and operating systems](#c-configure-instances-and-operating-systems) +- [D. Add networks](#d-add-networks) +- [E. If not already enabled, enable disk UUIDs](#e-if-not-already-enabled-enable-disk-uuids) +- [F. Optional: Configure node tags and custom attributes](#f-optional-configure-node-tags-and-custom-attributes) +- [G. Optional: Configure cloud-init](#g-optional-configure-cloud-init) +- [H. Saving the node template](#h-saving-the-node-template) + +### A. Configure the vSphere Credential + +The steps for configuring your vSphere credentials for the cluster are different depending on your version of Rancher. + +{{% tabs %}} +{{% tab "Rancher v2.2.0+" %}} + +Your account access information is in a [cloud credential.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) Cloud credentials are stored as Kubernetes secrets. + +You can use an existing cloud credential or create a new one. To create a new cloud credential, + +1. Click **Add New.** +1. In the **Name** field, enter a name for your vSphere credentials. +1. In the **vCenter or ESXi Server** field, enter the vCenter or ESXi hostname/IP. ESXi is the virtualization platform where you create and run virtual machines and virtual appliances. vCenter Server is the service through which you manage multiple hosts connected in a network and pool host resources. +1. Optional: In the **Port** field, configure the port of the vCenter or ESXi server. +1. In the **Username** and **Password** fields, enter your vSphere login username and password. +1. Click **Create.** + +**Result:** The node template has the credentials required to provision nodes in vSphere. + +{{% /tab %}} +{{% tab "Rancher prior to v2.2.0" %}} +In the **Account Access** section, enter the vCenter FQDN or IP address and the credentials for the vSphere user account. +{{% /tab %}} +{{% /tabs %}} + +### B. Configure Node Scheduling + +Choose what hypervisor the virtual machine will be scheduled to. The configuration options depend on your version of Rancher. + +{{% tabs %}} +{{% tab "Rancher v2.3.3+" %}} + +The fields in the **Scheduling** section should auto-populate with the data center and other scheduling options that are available to you in vSphere. + +1. In the **Data Center** field, choose the data center where the VM will be scheduled. +1. Optional: Select a **Resource Pool.** Resource pools can be used to partition available CPU and memory resources of a standalone host or cluster, and they can also be nested. +1. If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk. +1. Optional: Select a folder where the VM will be placed. The VM folders in this dropdown menu directly correspond to your VM folders in vSphere. Note: The folder name should be prefaced with `vm/` in your vSphere config file. +1. Optional: Choose a specific host to create the VM on. Leave this field blank for a standalone ESXi or for a cluster with DRS (Distributed Resource Scheduler). If specified, the host system's pool will be used and the **Resource Pool** parameter will be ignored. +{{% /tab %}} +{{% tab "Rancher prior to v2.3.3" %}} + +In the **Scheduling** section, enter: + +- The name/path of the **Data Center** to create the VMs in +- The name of the **VM Network** to attach to +- The name/path of the **Datastore** to store the disks in + + ![image]({{< baseurl >}}/img/rancher/vsphere-node-template-2.png) + +{{% /tab %}} +{{% /tabs %}} + +### C. Configure Instances and Operating Systems + +The instances are configured differently depending on your Rancher version. + +{{% tabs %}} +{{% tab "Rancher v2.3.3+" %}} + +In this section, configure the number of vCPUs, memory, and disk size for the VMs created by this template. + +In the **Creation method** field, you will configure the method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template. + +[VM templates](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html) are useful for setting up the operating system and other software, because they allow you to save time. For example, you could use a VM template to automatically install Kubernetes and Docker on each node. You can choose ISOs defined from templates in a vSphere data center or content library. + +The node can be created with any operating system that supports `cloud-init`. + +Choose the way that the VM will be created: + +- **Deploy from template: Data Center:** Choose a template that exists in the data center that you selected. +- **Deploy from template: Content Library:** In the two fields that appear when you select this option, choose the [content library](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-254B2CE8-20A8-43F0-90E8-3F6776C2C896.html). Then select the [VM template](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-F7BF0E6B-7C4F-4E46-8BBF-76229AEA7220.html) from the list of templates within the content library. This template will be used to create the new VM. +- **Clone an existing virtual machine:** In the **Virtual machine** field, choose an existing VM that the new VM will be cloned from. +- **Install from boot2docker ISO:** Ensure that the OS ISO URL contains the URL of a VMware ISO release for RancherOS (rancheros-vmware.iso). + +{{% /tab %}} +{{% tab "Rancher prior to v2.3.3" %}} + +In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template. + +Only RancherOS VMs are supported. + +Ensure that the [OS ISO URL](#instance-options) contains the URL of the VMware ISO release for RancherOS: `rancheros-vmware.iso`. + + ![image]({{< baseurl >}}/img/rancher/vsphere-node-template-1.png) + +{{% /tab %}} +{{% /tabs %}} + +### D. Add Networks + +_Available as of v2.3.3_ + +The node template now allows a VM to be provisioned with multiple networks. In the **Networks** field, you can now click **Add Network** to add any networks available to you in vSphere. + +### E. If Not Already Enabled, Enable Disk UUIDs + +In order to provision nodes with RKE, all nodes must be configured with disk UUIDs. + +As of Rancher v2.0.4, disk UUIDs are enabled in vSphere node templates by default. + +If you are using Rancher prior to v2.0.4, refer to these [instructions]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#enabling-disk-uuids-with-a-node-template) for details on how to enable a UUID with a Rancher node template. + +### F. Optional: Configure Node Tags and Custom Attributes + +The way to attach metadata to the VM is different depending on your Rancher version. + +{{% tabs %}} +{{% tab "Rancher v2.3.3+" %}} + +**Optional:** Add vSphere tags and custom attributes. Tags allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects. + +For tags, all your vSphere tags will show up as options to select from in your node template. + +In the custom attributes, Rancher will let you select all the custom attributes you have already set up in vSphere. The custom attributes are keys and you can enter values for each one. + + > **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere. These attributes allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects. + +{{% /tab %}} +{{% tab "Rancher prior to v2.3.3" %}} + +**Optional:** + + - Provide a set of configuration parameters (instance-options) for the VMs. + - Assign labels to the VMs that can be used as a base for scheduling rules in the cluster. + - Customize the configuration of the Docker daemon on the VMs that will be created. + +> **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere. These attributes allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects. + +{{% /tab %}} +{{% /tabs %}} + +### G. Optional: Configure Cloud Init + +[Cloud-init](https://cloud-init.io/) is a tool that applies user data to your nodes when they boot for the first time. + +The configuration file for `cloud-init` is named `cloud-config.yml.` In the **Cloud Init** field, it is optional to enter a file name or URL pointing to a `cloud-config.yml` file. + +You can use `cloud-init` to automate tasks that should happen when the instance boots, such as creating users, running shell commands, adding a load balancer, or preinstalling Kubernetes on the VM. + + For examples of how to write a `cloud-config` file, refer to the [cloud-init documentation.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html) + +### H. Saving the Node Template + +Assign a descriptive **Name** for this template and click **Create.** + +### Node Template Configuration Reference + +Refer to [this section]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/) for a reference on the configuration options available for vSphere node templates. + +# 2. Create a Kubernetes Cluster Using the Node Template + +After you've created a template, you can use it to stand up the vSphere cluster itself. + +To install Kubernetes on vSphere nodes, you will need to enable the vSphere cloud provider by modifying the cluster YAML file. This requirement applies to both pre-created [custom nodes]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/) and for nodes created in Rancher using the vSphere node driver. + +To create the cluster and enable the vSphere provider for cluster, follow these steps: + +- [A. Set up the cluster name and member roles](#a-set-up-the-cluster-name-and-member-roles) +- [B. Configure Kubernetes options](#b-configure-kubernetes-options) +- [C. Add node pools to the cluster](#c-add-node-pools-to-the-cluster) +- [D. Optional: Add a self-healing node pool](#d-optional-add-a-self-healing-node-pool) +- [E. Create the cluster](#e-create-the-cluster) + +### A. Set up the Cluster Name and Member Roles + +1. Log in to the Rancher UI as an admin user. +2. Navigate to **Clusters** in the **Global** view. +3. Click **Add Cluster** and select the **vSphere** infrastructure provider. +4. Assign a **Cluster Name.** +5. Assign **Member Roles** as required. {{< step_create-cluster_member-roles >}} + +> **Note:** +> +> If you have a cluster with DRS enabled, setting up [VM-VM Affinity Rules](https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.resmgmt.doc/GUID-7297C302-378F-4AF2-9BD6-6EDB1E0A850A.html) is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate ESXi hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes. + + +### B. Configure Kubernetes Options +{{}} + +### C. Add Node Pools to the Cluster +{{}} + +### D. Optional: Add a Self-Healing Node Pool + +To make a node pool self-healing, enter a number greater than zero in the **Auto Replace** column. Rancher will use the node template for the given node pool to recreate the node if it becomes inactive for that number of minutes. + +> **Note:** Self-healing node pools are designed to help you replace worker nodes for stateless applications. It is not recommended to enable node auto-replace on a node pool of master nodes or nodes with persistent volumes attached, because VMs are treated ephemerally. When a node in a node pool loses connectivity with the cluster, its persistent volumes are destroyed, resulting in data loss for stateful applications. + +### E. Create the Cluster + +Click **Create** to start provisioning the VMs and Kubernetes services. + +{{< result_create-cluster >}} + +# 3. Optional: Provision Storage + +For an example of how to provision storage in vSphere using Rancher, refer to the + [cluster administration section.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere) + + In order to provision storage in vSphere, the vSphere provider must be enabled. + +### Enable the vSphere Cloud Provider for the Cluster + +1. Set **Cloud Provider** option to `Custom`. + + ![vsphere-node-driver-cloudprovider]({{< baseurl >}}/img/rancher/vsphere-node-driver-cloudprovider.png) + +1. Click on **Edit as YAML** +1. Insert the following structure to the pre-populated cluster YAML. As of Rancher v2.3+, this structure must be placed under `rancher_kubernetes_engine_config`. In versions prior to v2.3, it has to be defined as a top-level field. Note that the `name` *must* be set to `vsphere`. + + ```yaml + rancher_kubernetes_engine_config: # Required as of Rancher v2.3+ + cloud_provider: + name: vsphere + vsphereCloudProvider: + [Insert provider configuration] + ``` + + Rancher uses RKE (the Rancher Kubernetes Engine) to provision Kubernetes clusters. Refer to the [vSphere configuration reference in the RKE documentation]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/) for details about the properties of the `vsphereCloudProvider` directive. diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials/_index.md new file mode 100644 index 00000000000..26479897974 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/creating-credentials/_index.md @@ -0,0 +1,41 @@ +--- +title: Creating Credentials in the vSphere Console +weight: 1 +--- + +This section describes how to create a vSphere username and password. You will need to provide these vSphere credentials to Rancher, which allows Rancher to provision resources in vSphere. + +The following table lists the permissions required for the vSphere user account: + +| Privilege Group | Operations | +|:----------------------|:-----------------------------------------------------------------------| +| Datastore | AllocateSpace
Browse
FileManagement (Low level file operations)
UpdateVirtualMachineFiles
UpdateVirtualMachineMetadata | +| Network | Assign | +| Resource | AssignVMToPool | +| Virtual Machine | Config (All)
GuestOperations (All)
Interact (All)
Inventory (All)
Provisioning (All) | + +The following steps create a role with the required privileges and then assign it to a new user in the vSphere console: + +1. From the **vSphere** console, go to the **Administration** page. + +2. Go to the **Roles** tab. + +3. Create a new role. Give it a name and select the privileges listed in the permissions table above. + + ![image]({{< baseurl >}}/img/rancher/rancherroles1.png) + +4. Go to the **Users and Groups** tab. + +5. Create a new user. Fill out the form and then click **OK**. Make sure to note the username and password, because you will need it when configuring node templates in Rancher. + + ![image]({{< baseurl >}}/img/rancher/rancheruser.png) + +6. Go to the **Global Permissions** tab. + +7. Create a new Global Permission. Add the user you created earlier and assign it the role you created earlier. Click **OK**. + + ![image]({{< baseurl >}}/img/rancher/globalpermissionuser.png) + + ![image]({{< baseurl >}}/img/rancher/globalpermissionrole.png) + +**Result:** You now have credentials that Rancher can use to manipulate vSphere resources. \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md new file mode 100644 index 00000000000..4cb5a130602 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md @@ -0,0 +1,24 @@ +--- +title: Enabling Disk UUIDs in Node Templates +weight: 3 +--- + +As of Rancher v2.0.4, disk UUIDs are enabled in vSphere node templates by default. + +For Rancher prior to v2.0.4, we recommend configuring a vSphere node template to automatically enable disk UUIDs because they are required for Rancher to manipulate vSphere resources. + +To enable disk UUIDs for all VMs created for a cluster, + +1. Navigate to the **Node Templates** in the Rancher UI while logged in as admin user. + +2. Add or edit an existing vSphere node template. + +3. Under **Instance Options** click on **Add Parameter**. + +4. Enter `disk.enableUUID` as key with a value of **TRUE**. + + ![vsphere-nodedriver-enable-uuid]({{< baseurl >}}/img/rke/vsphere-nodedriver-enable-uuid.png) + +5. Click **Create** or **Save**. + +**Result:** The disk UUID is enabled in the vSphere node template. \ No newline at end of file diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/_index.md new file mode 100644 index 00000000000..cdc3d70e232 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/node-template-reference/_index.md @@ -0,0 +1,93 @@ +--- +title: vSphere Node Template Configuration Reference +weight: 4 +--- + +The tables below describe the configuration options available in the vSphere node template: + +- [Account access](#account-access) +- [Instance options](#instance-options) +- [Scheduling options](#scheduling-options) + +# Account Access + +The account access parameters are different based on the Rancher version. + +{{% tabs %}} +{{% tab "Rancher v2.2.0+" %}} + +| Parameter | Required | Description | +|:----------------------|:--------:|:-----| +| Cloud Credentials | * | Your vSphere account access information, stored in a [cloud credential.]({{}}/rancher/v2.x/en/user-settings/cloud-credentials/) | + +{{% /tab %}} +{{% tab "Rancher prior to v2.2.0" %}} + +| Parameter | Required | Description | +|:------------------------|:--------:|:------------------------------------------------------------| +| vCenter or ESXi Server | * | IP or FQDN of the vCenter or ESXi server used for managing VMs. | +| Port | * | Port to use when connecting to the server. Defaults to `443`. | +| Username | * | vCenter/ESXi user to authenticate with the server. | +| Password | * | User's password. | + +{{% /tab %}} +{{% /tabs %}} + +# Instance Options + +The options for creating and configuring an instance are different depending on your Rancher version. + +{{% tabs %}} +{{% tab "Rancher v2.3.3+" %}} + +| Parameter | Required | Description | +|:----------------|:--------:|:-----------| +| CPUs | * | Number of vCPUS to assign to VMs. | +| Memory | * | Amount of memory to assign to VMs. | +| Disk | * | Size of the disk (in MB) to attach to the VMs. | +| Creation method | * | The method for setting up an operating system on the node. The operating system can be installed from an ISO or from a VM template. Depending on the creation method, you will also have to specify a VM template, content library, existing VM, or ISO. For more information on creation methods, refer to the section on [configuring instances.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/#c-configure-instances-and-operating-systems) | +| Cloud Init | | URL of a `cloud-config.yml` file or URL to provision VMs with. This file allows further customization of the operating system, such as network configuration, DNS servers, or system daemons. The operating system must support `cloud-init`. | +| Networks | | Name(s) of the network to attach the VM to. | +| Configuration Parameters used for guestinfo | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). | + +{{% /tab %}} +{{% tab "Rancher prior to v2.3.3" %}} + +| Parameter | Required | Description | +|:------------------------|:--------:|:------------------------------------------------------------| +| CPUs | * | Number of vCPUS to assign to VMs. | +| Memory | * | Amount of memory to assign to VMs. | +| Disk | * | Size of the disk (in MB) to attach to the VMs. | +| Cloud Init | | URL of a [RancherOS cloud-config]({{< baseurl >}}/os/v1.x/en/installation/configuration/) file to provision VMs with. This file allows further customization of the RancherOS operating system, such as network configuration, DNS servers, or system daemons.| +| OS ISO URL | * | URL of a RancherOS vSphere ISO file to boot the VMs from. You can find URLs for specific versions in the [Rancher OS GitHub Repo](https://github.com/rancher/os). | +| Configuration Parameters | | Additional configuration parameters for the VMs. These correspond to the [Advanced Settings](https://kb.vmware.com/s/article/1016098) in the vSphere console. Example use cases include providing RancherOS [guestinfo]({{< baseurl >}}/os/v1.x/en/installation/running-rancheros/cloud/vmware-esxi/#vmware-guestinfo) parameters or enabling disk UUIDs for the VMs (`disk.EnableUUID=TRUE`). | + +{{% /tab %}} +{{% /tabs %}} + +# Scheduling Options +The options for scheduling VMs to a hypervisor are different depending on your Rancher version. +{{% tabs %}} +{{% tab "Rancher v2.3.3+" %}} + +| Parameter | Required | Description | +|:------------------------|:--------:|:-------| +| Data Center | * | Name/path of the datacenter to create VMs in. | +| Resource Pool | | Name of the resource pool to schedule the VMs in. Leave blank for standalone ESXi. If not specified, the default resource pool is used. | +| Data Store | * | If you have a data store cluster, you can toggle the **Data Store** field. This lets you select a data store cluster where your VM will be scheduled to. If the field is not toggled, you can select an individual disk. | +| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The folder name should be prefaced with `vm/` in your vSphere config file. | +| Host | | The IP of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. | + +{{% /tab %}} +{{% tab "Rancher prior to v2.3.3" %}} + +| Parameter | Required | Description | +|:------------------------|:--------:|:------------------------------------------------------------| +| Data Center | * | Name/path of the datacenter to create VMs in. | +| Pool | | Name/path of the resource pool to schedule the VMs in. If not specified, the default resource pool is used. | +| Host | | Name/path of the host system to schedule VMs in. If specified, the host system's pool will be used and the *Pool* parameter will be ignored. | +| Network | * | Name of the VM network to attach VMs to. | +| Data Store | * | Datastore to store the VM disks. | +| Folder | | Name of a folder in the datacenter to create the VMs in. Must already exist. The folder name should be prefaced with `vm/` in your vSphere config file. | +{{% /tab %}} +{{% /tabs %}} \ No newline at end of file diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md index f517fe2d20d..d73d2169d74 100644 --- a/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/_index.md @@ -3,256 +3,27 @@ title: vSphere Cloud Provider weight: 254 --- +In order to provision Kubernetes clusters in vSphere with the RKE CLI, you must enable the vSphere cloud provider. + +The vSphere cloud provider must also be enabled in order to provision clusters with Rancher, which uses RKE as a library when provisioning [RKE clusters.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) + The [vSphere Cloud Provider](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) interacts with VMware infrastructure (vCenter or standalone ESXi server) to provision and manage storage for persistent volumes in a Kubernetes cluster. -When provisioning Kubernetes using RKE CLI or using [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) in Rancher, the vSphere Cloud Provider can be enabled by configuring the `cloud_provider` directive in the cluster YAML file. - -### Prerequisites - -1. You'll need to have credentials of a vCenter/ESXi user account with privileges allowing the cloud provider to interact with the vSphere infrastructure to provision storage. Refer to [this document](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) to create and assign a role with the required permissions in vCenter. -2. VMware Tools must be running in the Guest OS for all nodes in the cluster. -3. All nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly. See [Enabling Disk UUIDs](#enabling-disk-uuids-for-vsphere-vms). - -## Clusters provisioned with RKE CLI - -To enable the vSphere Cloud Provider in the cluster, you must add the top-level `cloud_provider` directive to the cluster configuration file, set the `name` property to `vsphere` and add the `vsphereCloudProvider` directive containing the configuration matching your infrastructure. See the [configuration reference](#configuration-reference) for the gory details. - -## Clusters provisioned with Rancher - -When provisioning clusters in Rancher using the [vSphere node driver]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/) or on pre-created [custom nodes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/custom-nodes/) the cluster YAML file must be modified in order to enable the cloud provider. - -1. Log in to the Rancher UI as admin user. -2. Navigate to **Clusters** in the **Global** view. -3. Click **Add Cluster** and select the **vSphere** infrastructure provider. -4. Assign a **Cluster Name**. -5. Assign **Member Roles** as required. -6. Expand **Cluster Options** and configure as required. -7. Set **Cloud Provider** option to `Custom`. - - ![vsphere-node-driver-cloudprovider]({{< baseurl >}}/img/rancher/vsphere-node-driver-cloudprovider.png) - -8. Click on **Edit as YAML** -9. Insert the following top-level structure to the pre-populated cluster YAML. Note that the `name` *must* be set to `vsphere`. Refer to the [configuration reference](#configuration-reference) to learn about the properties of the `vsphereCloudProvider` directive. - - ```yaml - cloud_provider: - name: vsphere - vsphereCloudProvider: - [Insert provider configuration] - ``` - -10. Configure the **Node Pools** per your requirements while ensuring to use a node template that enables disk UUIDs for the VMs (See [Annex - Enable disk UUIDs for vSphere VMs]). -11. Click on **Create** to start provisioning the VMs and Kubernetes services. - -## Configuration Reference - -> **Note:** This documentation reflects the new vSphere Cloud Provider configuration schema introduced in Kubernetes v1.9 which differs from previous versions. - -The vSphere configuration options are divided into 5 groups: - -* global -* virtual_center -* workspace -* disk -* network - -### global - -The main purpose of global options is to be able to define a common set of configuration parameters that will be inherited by all vCenters defined under the `virtual_center` directive unless explicitly defined there. - -Accordingly, the `global` directive accepts the same configuration options that are available under the `virtual_center` directive. Additionally it accepts a single parameter that can only be specified here: - -| global Options | Type | Required | Description | -|:---------------:|:-------:|:---------:|:-----------------------------------------------------------------------------:| -| insecure-flag | boolean | | Set to **true** if the vCenter/ESXi uses a self-signed certificate. | - -___ - -**Example:** - -```yaml -(...) - global: - insecure-flag: true -``` - -### virtual_center - -This configuration directive specifies the vCenters that are managing the nodes in the cluster. You must define at least one vCenter/ESXi server. If the nodes span multiple vCenters then all must be defined. - -Each vCenter is defined by adding a new entry under the `virtual_center` directive with the vCenter IP or FQDN as the name. All required parameters must be provided for each vCenter unless they are already defined under the `global` directive. - -| virtual_center Options | Type | Required | Description | -|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:| -| user | string | * | vCenter/ESXi user used to authenticate with this server. | -| password | string | * | User's password. | -| port | string | | Port to use to connect to this server. Defaults to 443. | -| datacenters | string | * | Comma-separated list of all datacenters in which cluster nodes are running in.| -| soap-roundtrip-count | uint | | Round tripper count for API requests to the vCenter (num retries = value - 1).| - - -> The following additional options (introduced in Kubernetes v1.11) are not yet supported in RKE. - -| virtual_center Options | Type | Required | Description | -|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:| -| secret-name | string | | Name of secret resource containing credential key/value pairs. Can be specified in lieu of user/password parameters.| -| secret-namespace | string | | Namespace in which the secret resource was created in. | -| ca-file | string | | Path to CA cert file used to verify the vCenter certificate. | - -___ - -**Example:** - -```yaml -(...) - virtual_center: - 172.158.111.1: {} # This vCenter inherits all it's properties from global options - 172.158.110.2: # All required options are set explicitly - user: vc-user - password: othersecret - datacenters: eu-west-2 -``` - -### workspace - -This configuration group specifies how storage for volumes is created in vSphere. -The following configuration options are available: - -| workspace Options | Type | Required | Description | -|:----------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:| -| server | string | * | IP or FQDN of the vCenter/ESXi that should be used for creating the volumes. Must match one of the vCenters defined under the `virtual_center` directive.| -| datacenter | string | * | Name of the datacenter that should be used for creating volumes. For ESXi enter *ha-datacenter*.| -| folder | string | * | Path of folder in which to create dummy VMs used for volume provisioning (relative from the root folder in vCenter), e.g. "kubernetes".| -| default-datastore | string | | Name of default datastore to place VMDKs if neither datastore or storage policy are specified in the volume options of a PVC. If datastore is located in a storage folder or is a member of a datastore cluster, specify the full path. | -| resourcepool-path | string | | Absolute or relative path to the resource pool where the dummy VMs for [Storage policy based provisioning](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html) should be created. If a relative path is specified, it is resolved with respect to the datacenter's *host* folder. Examples: `//host//Resources/`, `Resources/`. For standalone ESXi specify `Resources`.| - -___ - -**Example:** - -```yaml -(...) - workspace: - server: 172.158.111.1 # matches IP of vCenter defined in the virtual_center block - datacenter: eu-west-1 - folder: kubernetes - default-datastore: ds-1 -``` - -### disk - -The following configuration options are available under the disk directive: - -| disk Options | Type | Required | Description | -|:--------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:| -| scsicontrollertype | string | | SCSI controller type to use when attaching block storage to VMs. Must be one of: *lsilogic-sas* or *pvscsi*. Default: *pvscsi*.| - -___ - -### network - -The following configuration options are available under the network directive: - -| network Options | Type | Required | Description | -|:-------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------:| -| public-network | string | | Name of public **VM Network** to which the VMs in the cluster are connected. Used to determine public IP addresses of VMs.| - - -## Configuration Example - -Given the following: - -- VMs in the cluster are running in the same datacenter `eu-west-1` managed by the vCenter `vc.example.com`. -- The vCenter has a user `provisioner` with password `secret` with the required roles assigned, see [Prerequisites](#prerequisites). -- The vCenter has a datastore named `ds-1` which should be used to store the VMDKs for volumes. -- A `kubernetes` folder exists in vCenter. - -The corresponding configuration for the provider would then be as follows: - -```yaml -(...) -cloud_provider: - name: vsphere - vsphereCloudProvider: - virtual_center: - vc.example.com: - user: provisioner - password: secret - datacenters: eu-west-1 - workspace: - server: vc.example.com - folder: kubernetes - default-datastore: ds-1 - datacenter: eu-west-1 - -``` - -## Annex - -### Enabling disk UUIDs for vSphere VMs - -Depending on whether you are provisioning the VMs using the [vSphere node driver]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) in Rancher or using your own scripts or third-party tools, there are different methods available to enable disk UUIDs for VMs. - -#### Using the Vsphere Console - -The required property can be set while creating or modifying VMs in the vSphere Console: - -1. For each VM navigate to the tab **VM Options** and click on **Edit Configuration**. -2. Add the parameter `disk.EnableUUID` with a value of **TRUE**. - - ![vsphere-advanced-parameters]({{< baseurl >}}/img/rke/vsphere-advanced-parameters.png) - -#### Using the GOVC CLI tool - -You can also modify properties of VMs with the [govc](https://github.com/vmware/govmomi/tree/master/govc) command-line tool to enable disk UUIDs: - -```sh -$ govc vm.change -vm -e disk.enableUUID=TRUE -``` - -#### Using Rancher node template - -When creating new clusters in Rancher using vSphere node templates, you can configure the template to automatically enable disk UUIDs for all VMs created for a cluster: - -1. Navigate to the **Node Templates** in the Rancher UI while logged in as admin user. - -2. Add or edit an existing vSphere node template. - -3. Under **Instance Options** click on **Add Parameter**. - -4. Enter `disk.enableUUID` as key with a value of **TRUE**. - - ![vsphere-nodedriver-enable-uuid]({{< baseurl >}}/img/rke/vsphere-nodedriver-enable-uuid.png) - -5. Click **Create** or **Save**. - -### Troubleshooting - -If you are experiencing issues while provisioning a cluster with enabled vSphere Cloud Provider or while creating vSphere volumes for your workloads, you should inspect the logs of the following K8s services: - -- controller-manager (Manages volumes in vCenter) -- kubelet: (Mounts vSphere volumes to pods) - -If your cluster is not configured with external [Cluster Logging]({{< baseurl >}}/rancher/v2.x/en/tools/logging/), you will need to SSH into nodes to get the logs of the `kube-controller-manager` (running on one of the control plane nodes) and the `kubelet` (pertaining to the node where the stateful pod has been scheduled). - -The easiest way to create a SSH session with a node is the Rancher CLI tool. - -1. [Configure the Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/) for your cluster. -2. Run the following command to get a shell to the corresponding nodes: - - ```sh -$ rancher ssh - ``` - -3. Inspect the logs of the controller-manager and kubelet containers looking for errors related to the vSphere cloud provider: - - ```sh - $ docker logs --since 15m kube-controller-manager - $ docker logs --since 15m kubelet - ``` - +This section describes how to enable the vSphere cloud provider. You will need to use the `cloud_provider` directive in the cluster YAML file. ### Related Links -- [vSphere Storage for Kubernetes](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) -- [Kubernetes Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) +- **Configuration:** For details on vSphere configuration in RKE, refer to the [configuration reference.]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference) +- **Troubleshooting:** For guidance on troubleshooting a cluster with the vSphere cloud provider enabled, refer to the [troubleshooting section.]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting) +- **Storage:** If you are setting up storage, see the [official vSphere documentation on storage for Kubernetes,](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) or the [official Kubernetes documentation on persistent volumes.](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) If you are using Rancher, refer to the [Rancher documentation on provisioning storage in vSphere.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere) +- **For Rancher users:** Refer to the Rancher documentation on [creating vSphere Kubernetes clusters]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) and [provisioning storage.]({{}}/rancher/v2.x/en/cluster-admin/volumes-and-storage/examples/vsphere) + +# Prerequisites + +- **Credentials:** You'll need to have credentials of a vCenter/ESXi user account with privileges allowing the cloud provider to interact with the vSphere infrastructure to provision storage. Refer to [this document](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/vcp-roles.html) to create and assign a role with the required permissions in vCenter. +- **VMware Tools** must be running in the Guest OS for all nodes in the cluster. +- **Disk UUIDs:** All nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly. See the section on [enabling disk UUIDs.]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid) + +# Enabling the vSphere Provider with the RKE CLI + +To enable the vSphere Cloud Provider in the cluster, you must add the top-level `cloud_provider` directive to the cluster configuration file, set the `name` property to `vsphere` and add the `vsphereCloudProvider` directive containing the configuration matching your infrastructure. See the [configuration reference]({{}}/rke/latest/en/config-options/cloud-providers/vsphere/config-reference) for the gory details. \ No newline at end of file diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/_index.md new file mode 100644 index 00000000000..c0da0176c69 --- /dev/null +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/config-reference/_index.md @@ -0,0 +1,144 @@ +--- +title: vSphere Configuration Reference +weight: 3 +--- + +This section shows an example of how to configure the vSphere cloud provider. + +The vSphere cloud provider must be enabled to allow dynamic provisioning of volumes. + +For more details on deploying a Kubernetes cluster on vSphere, refer to the [official cloud provider documentation.](https://cloud-provider-vsphere.sigs.k8s.io/tutorials/kubernetes-on-vsphere-with-kubeadm.html) + +> **Note:** This documentation reflects the new vSphere Cloud Provider configuration schema introduced in Kubernetes v1.9 which differs from previous versions. + +# vSphere Configuration Example + +Given the following: + +- VMs in the cluster are running in the same datacenter `eu-west-1` managed by the vCenter `vc.example.com`. +- The vCenter has a user `provisioner` with password `secret` with the required roles assigned, see [Prerequisites](#prerequisites). +- The vCenter has a datastore named `ds-1` which should be used to store the VMDKs for volumes. +- A `vm/kubernetes` folder exists in vCenter. + +The corresponding configuration for the provider would then be as follows: + +```yaml +(...) +cloud_provider: + name: vsphere + vsphereCloudProvider: + virtual_center: + vc.example.com: + user: provisioner + password: secret + port: 443 + datacenters: /us-west-1 + workspace: + server: vc.example.com + folder: /us-west-1/folder/myvmfolder + default-datastore: /us-west-1/datastore/ds-1 + datacenter: /us-west-1 + resourcepool-path: /us-west-1/host/hn1/resources/myresourcepool + +``` +# Configuration Options + +The vSphere configuration options are divided into 5 groups: + +* [global](#global) +* [virtual_center](#virtual_center) +* [workspace](#workspace) +* [disk](#disk) +* [network](#network) + +### global + +The main purpose of global options is to be able to define a common set of configuration parameters that will be inherited by all vCenters defined under the `virtual_center` directive unless explicitly defined there. + +Accordingly, the `global` directive accepts the same configuration options that are available under the `virtual_center` directive. Additionally it accepts a single parameter that can only be specified here: + +| global Options | Type | Required | Description | +|:---------------:|:-------:|:---------:|:---------| +| insecure-flag | boolean | | Set to **true** if the vCenter/ESXi uses a self-signed certificate. | + +**Example:** + +```yaml +(...) + global: + insecure-flag: true +``` + +### virtual_center + +This configuration directive specifies the vCenters that are managing the nodes in the cluster. You must define at least one vCenter/ESXi server. If the nodes span multiple vCenters then all must be defined. + +Each vCenter is defined by adding a new entry under the `virtual_center` directive with the vCenter IP or FQDN as the name. All required parameters must be provided for each vCenter unless they are already defined under the `global` directive. + +| virtual_center Options | Type | Required | Description | +|:----------------------:|:--------:|:---------:|:-----------| +| user | string | * | vCenter/ESXi user used to authenticate with this server. | +| password | string | * | User's password. | +| port | string | | Port to use to connect to this server. Defaults to 443. | +| datacenters | string | * | Comma-separated list of all datacenters in which cluster nodes are running in. | +| soap-roundtrip-count | uint | | Round tripper count for API requests to the vCenter (num retries = value - 1). | + +> The following additional options (introduced in Kubernetes v1.11) are not yet supported in RKE. + +| virtual_center Options | Type | Required | Description | +|:----------------------:|:--------:|:---------:|:-------| +| secret-name | string | | Name of secret resource containing credential key/value pairs. Can be specified in lieu of user/password parameters.| +| secret-namespace | string | | Namespace in which the secret resource was created in. | +| ca-file | string | | Path to CA cert file used to verify the vCenter certificate. | + +**Example:** + +```yaml +(...) + virtual_center: + 172.158.111.1: {} # This vCenter inherits all it's properties from global options + 172.158.110.2: # All required options are set explicitly + user: vc-user + password: othersecret + datacenters: eu-west-2 +``` + +### workspace + +This configuration group specifies how storage for volumes is created in vSphere. +The following configuration options are available: + +| workspace Options | Type | Required | Description | +|:----------------------:|:--------:|:---------:|:---------| +| server | string | * | IP or FQDN of the vCenter/ESXi that should be used for creating the volumes. Must match one of the vCenters defined under the `virtual_center` directive.| +| datacenter | string | * | Name of the datacenter that should be used for creating volumes. For ESXi enter *ha-datacenter*.| +| folder | string | * | Path of folder in which to create dummy VMs used for volume provisioning (relative from the root folder in vCenter), e.g. "vm/kubernetes".| +| default-datastore | string | | Name of default datastore to place VMDKs if neither datastore or storage policy are specified in the volume options of a PVC. If datastore is located in a storage folder or is a member of a datastore cluster, specify the full path. | +| resourcepool-path | string | | Absolute or relative path to the resource pool where the dummy VMs for [Storage policy based provisioning](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html) should be created. If a relative path is specified, it is resolved with respect to the datacenter's *host* folder. Examples: `//host//Resources/`, `Resources/`. For standalone ESXi specify `Resources`. | + +**Example:** + +```yaml +(...) + workspace: + server: 172.158.111.1 # matches IP of vCenter defined in the virtual_center block + datacenter: eu-west-1 + folder: vm/kubernetes + default-datastore: ds-1 +``` + +### disk + +The following configuration options are available under the disk directive: + +| disk Options | Type | Required | Description | +|:--------------------:|:--------:|:---------:|:----------------| +| scsicontrollertype | string | | SCSI controller type to use when attaching block storage to VMs. Must be one of: *lsilogic-sas* or *pvscsi*. Default: *pvscsi*. | + +### network + +The following configuration options are available under the network directive: + +| network Options | Type | Required | Description | +|:-------------------:|:--------:|:---------:|:-----------------------------------------------------------------------------| +| public-network | string | | Name of public **VM Network** to which the VMs in the cluster are connected. Used to determine public IP addresses of VMs.| \ No newline at end of file diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid/_index.md new file mode 100644 index 00000000000..0277e725a07 --- /dev/null +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/enabling-uuid/_index.md @@ -0,0 +1,35 @@ +--- +title: Enabling Disk UUIDs for vSphere VMs +weight: 2 +--- + +In order to provision nodes with RKE, all nodes must be configured with disk UUIDs. This is required so that attached VMDKs present a consistent UUID to the VM, allowing the disk to be mounted properly. + +Depending on whether you are provisioning the VMs using the [vSphere node driver]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere) in Rancher or using your own scripts or third-party tools, there are different methods available to enable disk UUIDs for VMs: + +- [Using the vSphere console](#using-the-vsphere-console) +- [Using the GOVC CLI tool](#using-the-govc-cli-tool) +- [Using a Rancher node template](#using-a-rancher-node-template) + +### Using the vSphere Console + +The required property can be set while creating or modifying VMs in the vSphere Console: + +1. For each VM navigate to the tab **VM Options** and click on **Edit Configuration**. +2. Add the parameter `disk.EnableUUID` with a value of **TRUE**. + + ![vsphere-advanced-parameters]({{< baseurl >}}/img/rke/vsphere-advanced-parameters.png) + +### Using the GOVC CLI tool + +You can also modify properties of VMs with the [govc](https://github.com/vmware/govmomi/tree/master/govc) command-line tool to enable disk UUIDs: + +```sh +$ govc vm.change -vm -e disk.enableUUID=TRUE +``` + +### Using a Rancher Node Template + +In Rancher v2.0.4+, disk UUIDs are enabled in vSphere node templates by default. + +If you are using Rancher prior to v2.0.4, refer to the [Rancher documentation.]({{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/#enabling-disk-uuids-with-a-node-template) for details on how to enable a UUID with a Rancher node template. \ No newline at end of file diff --git a/content/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting/_index.md b/content/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting/_index.md new file mode 100644 index 00000000000..6d2cffca67f --- /dev/null +++ b/content/rke/latest/en/config-options/cloud-providers/vsphere/troubleshooting/_index.md @@ -0,0 +1,27 @@ +--- +title: Troubleshooting vSphere Clusters +weight: 4 +--- + +If you are experiencing issues while provisioning a cluster with enabled vSphere Cloud Provider or while creating vSphere volumes for your workloads, you should inspect the logs of the following K8s services: + +- controller-manager (Manages volumes in vCenter) +- kubelet: (Mounts vSphere volumes to pods) + +If your cluster is not configured with external [Cluster Logging]({{< baseurl >}}/rancher/v2.x/en/tools/logging/), you will need to SSH into nodes to get the logs of the `kube-controller-manager` (running on one of the control plane nodes) and the `kubelet` (pertaining to the node where the stateful pod has been scheduled). + +The easiest way to create a SSH session with a node is the Rancher CLI tool. + +1. [Configure the Rancher CLI]({{< baseurl >}}/rancher/v2.x/en/cli/) for your cluster. +2. Run the following command to get a shell to the corresponding nodes: + + ```sh +$ rancher ssh + ``` + +3. Inspect the logs of the controller-manager and kubelet containers looking for errors related to the vSphere cloud provider: + + ```sh + $ docker logs --since 15m kube-controller-manager + $ docker logs --since 15m kubelet + ``` From bba8d0d8bdbfdb11f5e9c05f5c5b9d68b0df8fc1 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Mon, 2 Dec 2019 12:32:42 -0700 Subject: [PATCH 080/103] Minor changes to vSphere docs --- .../rke-clusters/node-pools/vsphere/_index.md | 6 ++++-- .../vsphere/provisioning-vsphere-clusters/_index.md | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md index 7702ec28294..23b0a9bc6e1 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/_index.md @@ -34,10 +34,12 @@ For the fields to be populated, your setup needs to fulfill the [prerequisites.] ### More Supported Operating Systems -As of Rancher v2.3.3+, you can provision VMs with any operating system that supports cloud init. +In Rancher v2.3.3+, you can provision VMs with any operating system that supports `cloud-init`. Only YAML format is supported for the [cloud config.](https://cloudinit.readthedocs.io/en/latest/topics/examples.html) In Rancher prior to v2.3.3, the vSphere node driver included in Rancher only supported the provisioning of VMs with [RancherOS]({{}}/os/v1.x/en/) as the guest operating system. # Video Walkthrough of v2.3.3 Node Template Features -In [this YouTube video,](https://www.youtube.com/watch?v=dPIwg6x1AlU) we demonstrate how to set up a node template with the new features designed to help you bring cloud operations to on-premises clusters. +In this YouTube video, we demonstrate how to set up a node template with the new features designed to help you bring cloud operations to on-premises clusters. + +{{< youtube id="dPIwg6x1AlU">}} diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md index 39fa35b7528..d84c6c6ab25 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md @@ -215,11 +215,11 @@ In the custom attributes, Rancher will let you select all the custom attributes {{% /tab %}} {{% /tabs %}} -### G. Optional: Configure Cloud Init +### G. Optional: Configure cloud-init [Cloud-init](https://cloud-init.io/) is a tool that applies user data to your nodes when they boot for the first time. -The configuration file for `cloud-init` is named `cloud-config.yml.` In the **Cloud Init** field, it is optional to enter a file name or URL pointing to a `cloud-config.yml` file. +The configuration file for `cloud-init` is named `cloud-config.yml.` In the **Cloud Init** field, it is optional to enter a file name or URL pointing to a `cloud-config.yml` file. Only YAML format is supported for the cloud config. You can use `cloud-init` to automate tasks that should happen when the instance boots, such as creating users, running shell commands, adding a load balancer, or preinstalling Kubernetes on the VM. From dbd128bd88bd202ef482feee43bf0ddd09d1680f Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Mon, 2 Dec 2019 13:28:44 -0700 Subject: [PATCH 081/103] Fix link in Google OAuth doc --- .../v2.x/en/admin-settings/authentication/google/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/admin-settings/authentication/google/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/google/_index.md index b47426d3471..64de99ae633 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/google/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/google/_index.md @@ -35,7 +35,7 @@ Before you can set up Google OAuth in Rancher, you need to log in to your G Suit **Result:** Rancher has been added as an authorized domain for the Admin SDK API. ### 2. Creating OAuth2 Credentials for the Rancher Server -1. Go to the Google API console, select your project, and go to the [credentials page.]((https://console.developers.google.com/apis/credentials) ) +1. Go to the Google API console, select your project, and go to the [credentials page.](https://console.developers.google.com/apis/credentials) ![Credentials]({{}}/img/rancher/Google-Credentials-tab.png) 1. On the **Create Credentials** dropdown, select **OAuth client ID.** 1. Click **Web application.** From e524f59855f73e63b41cf14d8aaaa157158745c2 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 2 Aug 2019 16:22:07 -0700 Subject: [PATCH 082/103] Update and clarify docs on global and cluster permissions --- .../rbac/cluster-project-roles/_index.md | 54 ++++++++++---- .../rbac/global-permissions/_index.md | 72 ++++++++++++++----- 2 files changed, 94 insertions(+), 32 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md index 9872b85b113..df0130f78ef 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md @@ -31,27 +31,54 @@ Rancher lets you assign _custom cluster roles_ to a user instead of the typical #### Cluster Role Reference -The following table lists each built-in custom cluster role available in Rancher and whether it is also granted by the `Owner` or `Member` role. +The following table lists each built-in custom cluster role available and whether that level of access is included in the default cluster-level permissions, `Cluster Owner` and `Cluster Member`. | Built-in Cluster Role | Owner | Member | | ---------------------------------- | ------------- | --------------------------------- | +| Create Projects | ✓ | | +| Manage Cluster Backups             | ✓ | | +| Manage Cluster Catalogs | ✓ | | | Manage Cluster Members | ✓ | | -| Manage Cluster Catalogs | ✓ | | Manage Nodes | ✓ | | -| Manage Snapshots | ✓ || | Manage Storage | ✓ | | -| View All Projects | ✓ | | -| Create Project | ✓ | ✓ | -| View Cluster Members | ✓ | ✓ | +| View All Projects | ✓ | ✓ | | View Cluster Catalogs | ✓ | ✓ | +| View Cluster Members | ✓ | ✓ | | View Nodes | ✓ | ✓ | -| View Snapshots | ✓ | ✓ | -> **Notes:** -> ->- Each cluster role listed above, including `Owner` and `Member`, is comprised of multiple rules granting access to various resources. You can view the roles and their rules on the Global > Security > Roles page. ->- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource. ->- The `Manage Cluster Members` role allows the user to manage any members of the cluster **and** grant them any cluster scoped role regardless of their access to the cluster resources. Be cautious when assigning this role out individually. +For details on how each cluster role can access Kubernetes resources, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Clusters** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role. + +> **Note:** +>When viewing the resources associated with default roles created by Rancher, if there are multiple Kubernetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource. + +### Giving a Custom Cluster Role to a Cluster Member + +Admins can set up custom cluster roles that can be assigned to cluster owners and members. + +Cluster owners and admins can then assign those roles to cluster members. + +To create a custom cluster role, + +1. In the **Global** view under **Security > Roles,** click **Add Cluster Role.** + +1. In the **Grant Resources** section, choose any combination of operations on Kubernetes resources that will be allowed by the new role. Give the new cluster role a name and click **Create.** + +Then, from the **Cluster** view, go to the **Members** tab. From this tab, you can give the cluster role to members in two ways: + +- You can assign the role to a new member with the Rancher UI. +- You can assign the role to an existing member with the Rancher API view. + +To assign the role to a new cluster member, + +1. Click **Add Member.** Then in the **Cluster Permissions** section, you can choose your custom cluster role. + +1. When you click **Create**, the member should have the assigned role. + +To assign any custom role to an existing cluster member, + +1. Go to the member you want to give the role to. Click the **Ellipsis (...) > View in API.** + +1. In the **roleTemplateId** field, go to the drop-down menu and choose the role you want to assign to the member. Click **Show Request** and **Send Request.** After that, the member's role should be updated. ### Project Roles @@ -132,7 +159,7 @@ There are two methods for changing default cluster/project roles: >- Although you can [lock]({{< baseurl >}}/rancher/v2.x/en/admin-settings/rbac/locked-roles/) a default role, the system still assigns the role to users who create a cluster/project. >- Only users that create clusters/projects inherit their roles. Users added to the cluster/project membership afterward must be explicitly assigned their roles. -### Configuring Default Roles +### Configuring Default Roles for Cluster and Project Creators You can change the cluster or project role(s) that are automatically assigned to the creating user. @@ -162,3 +189,4 @@ When you revoke the cluster membership for a user that's explicitly assigned mem - Exercise any [individual project roles](#project-role-reference) they are assigned. If you want to completely revoke a user's access within a cluster, revoke both their cluster and project memberships. + diff --git a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md index 0300bf6d83a..00c18020f64 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md @@ -3,6 +3,8 @@ title: Global Permissions weight: 1126 --- +_Permissions_ are individual access rights that you can assign when selecting a custom permission for a user. + Global Permissions define user authorization outside the scope of any particular cluster. Out-of-the-box, there are two default global permissions: `Administrator` and `Standard User`. - **Administrator:** @@ -15,7 +17,7 @@ Global Permissions define user authorization outside the scope of any particular >**Note:** You cannot create, update, or delete Global Permissions. -### Global Permission Assignment +# Global Permission Assignment Assignment of global permissions to a user depends on their authentication source: external or local. @@ -27,53 +29,85 @@ Assignment of global permissions to a user depends on their authentication sourc When you create a new local user, you assign them a global permission as you complete the **Add User** form. -### Custom Global Permissions +# Custom Global Permissions -Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a custom set of permissions. +Using custom permissions is convenient for providing users with narrow or specialized access to Rancher. -_Permissions_ are individual access rights that you can assign when selecting a custom permission for a user. +When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, new users are assigned the user permissions. -Using custom permissions is convenient for providing users with narrow or specialized access to Rancher. See the [table below](#global-permissions-reference) for a list of individual permissions available. +However, in some organizations, these permissions may extend too much access. Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a more restrictive set of custom global permissions. -### Global Permissions Reference +The default roles, Admin and User, each come with multiple global permissions built into them. The Admin role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role. -The following table lists each custom global permission available and whether it is assigned to the default global permissions, `Administrator` and `Standard User`. +Administrators can enforce custom global permissions in three ways: + +- Changing the [default permissions for new users](#configuring-default-global-permissions) + +- Editing the [permissions of a user](#configuring-permissions-for-individual-users) + +- Choosing the **Custom** cluster permissions option when [adding a new member to a cluster]({{}}/rancher/v2.x/en/cluster-admin/cluster-members/) + +### Custom Global Permissions Reference + +The following table lists each custom global permission available and whether it is included in the default global permissions, `Administrator` and `Standard User`. | Custom Global Permission | Administrator | Standard User | | ---------------------------------- | ------------- | ------------- | +| Create Clusters | ✓ | ✓ | +| Create RKE Templates | ✓ | ✓ | | Manage Authentication | ✓ | | | Manage Catalogs | ✓ | | -| Manage Cluster Drivers | ✓ | | +| Manage Cluster Drivers | ✓ | | | Manage Node Drivers | ✓ | | | Manage PodSecurityPolicy Templates | ✓ | | | Manage Roles | ✓ | | +| Manage Settings | ✓ | | | Manage Users | ✓ | | -| Create Clusters | ✓ | ✓ | -| Create RKE Templates | ✓ | ✓ | | Use Catalog Templates | ✓ | ✓ | -| Login Access | ✓ | ✓ | +| User Base (Basic log-in access) | ✓ | ✓ | + +For details on which Kubernetes resources correspond to each global permission, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Global** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role. > **Notes:** > >- Each permission listed above is comprised of multiple individual permissions not listed in the Rancher UI. For a full list of these permissions and the rules they are comprised of, access through the API at `/v3/globalRoles`. >- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource. -When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, new users are assigned the [user](#user) permissions. However, in some organizations, these permissions may extend too much access. In this use case, you can change the default permissions to something more restrictive, such as a set of individual permissions. +### Configuring Default Global Permissions -You can assign one or more default permissions. For example, the `user` permission assigns new users a [set of individual global permissions](#global-permissions-reference). If you want to restrict the default permissions for new users, you can remove the `user` permission as default role and then assign multiple individual permissions as default instead. Conversely, you can also add administrative permissions on top of a set of other standard permissions. +If you want to restrict the default permissions for new users, you can remove the `user` permission as default role and then assign multiple individual permissions as default instead. Conversely, you can also add administrative permissions on top of a set of other standard permissions. >**Note:** Default roles are only assigned to users added from an external authentication provider. For local users, you must explicitly assign global permissions when adding a user to Rancher. You can customize these global permissions when adding the user. -### Configuring Default Global Permissions - -You can change the default global permissions that are assigned to external users upon their first log in. +To change the default global permissions that are assigned to external users upon their first log in, follow these steps: 1. From the **Global** view, select **Security > Roles** from the main menu. Make sure the **Global** tab is selected. -1. Find the permissions set that you want to use as default. Then edit the permission by selecting **Ellipsis > Edit**. +1. Find the permissions set that you want to add or remove as a default. Then edit the permission by selecting **Ellipsis > Edit**. -1. Select **Yes: Default role for new users** and then click **Save**. +1. If you want to add the permission as a default, Select **Yes: Default role for new users** and then click **Save**. 1. If you want to remove a default permission, edit the permission and select **No** from **New User Default**. -**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column. \ No newline at end of file +<<<<<<< HEAD +**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column. +======= +**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column. + +### Configuring Global Permissions for Individual Users + +To configure permission for a user, + +1. Go to the **Users** tab. + +1. On this page, go to the user whose access level you want to change and click **Ellipsis (...) > Edit.** + +1. In the **Global Permissions** section, click **Custom.** + +1. Check the boxes for each subset of permissions you want the user to have access to. + +1. Click **Save.** + +> **Result:** The user's global permissions have been updated. + +>>>>>>> Update and clarify docs on global and cluster permissions From 36124239f89983b26cbc52c7a0f4f5cb0524fb43 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 2 Aug 2019 17:25:29 -0700 Subject: [PATCH 083/103] Fix typo --- .../v2.x/en/admin-settings/rbac/global-permissions/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md index 00c18020f64..7c233473d8a 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md @@ -43,7 +43,7 @@ Administrators can enforce custom global permissions in three ways: - Changing the [default permissions for new users](#configuring-default-global-permissions) -- Editing the [permissions of a user](#configuring-permissions-for-individual-users) +- Editing the [permissions of a user](#configuring-global-permissions-for-individual-users) - Choosing the **Custom** cluster permissions option when [adding a new member to a cluster]({{}}/rancher/v2.x/en/cluster-admin/cluster-members/) From e563e3f9775500b958fb6c6f1ee436ef9de2919a Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 6 Aug 2019 10:45:04 -0700 Subject: [PATCH 084/103] Update global permissions docs --- .../rbac/global-permissions/_index.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md index 7c233473d8a..fa84b3eb7e5 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md @@ -33,19 +33,17 @@ Assignment of global permissions to a user depends on their authentication sourc Using custom permissions is convenient for providing users with narrow or specialized access to Rancher. -When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, new users are assigned the user permissions. +When a user from an [external authentication source]({{< baseurl >}}/rancher/v2.x/en/admin-settings/authentication/) signs into Rancher for the first time, they're automatically assigned a set of global permissions (hereafter, permissions). By default, after a user logs in from the first time, they are created as a user and assigned the default `user` permission. The standard `user` permission allows users to login and create clusters. However, in some organizations, these permissions may extend too much access. Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a more restrictive set of custom global permissions. The default roles, Admin and User, each come with multiple global permissions built into them. The Admin role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role. -Administrators can enforce custom global permissions in three ways: +Administrators can enforce custom global permissions in two ways: - Changing the [default permissions for new users](#configuring-default-global-permissions) -- Editing the [permissions of a user](#configuring-global-permissions-for-individual-users) - -- Choosing the **Custom** cluster permissions option when [adding a new member to a cluster]({{}}/rancher/v2.x/en/cluster-admin/cluster-members/) +- Editing the [permissions of an existing user](#configuring-global-permissions-for-individual-users) ### Custom Global Permissions Reference @@ -57,14 +55,19 @@ The following table lists each custom global permission available and whether it | Create RKE Templates | ✓ | ✓ | | Manage Authentication | ✓ | | | Manage Catalogs | ✓ | | -| Manage Cluster Drivers | ✓ | | +| Manage Cluster Drivers | ✓ | | | Manage Node Drivers | ✓ | | | Manage PodSecurityPolicy Templates | ✓ | | | Manage Roles | ✓ | | | Manage Settings | ✓ | | | Manage Users | ✓ | | | Use Catalog Templates | ✓ | ✓ | -| User Base (Basic log-in access) | ✓ | ✓ | +| User Base* (Basic log-in access) | ✓ | ✓ | + +> *This role has two names: +> +> - When you go to the Users tab and edit a user's global role, this role is called Login Access in the custom global permissions list. +> - When you go to the Security tab and edit the roles from the roles page, this role is called User Base. For details on which Kubernetes resources correspond to each global permission, you can go to the **Global** view in the Rancher UI. Then click **Security > Roles** and go to the **Global** tab. If you click an individual role, you can refer to the **Grant Resources** table to see all of the operations and resources that are permitted by the role. From e6cf88d2ef70cc79706f50ca5cd505f9eff06f6e Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Mon, 2 Dec 2019 17:46:03 -0700 Subject: [PATCH 085/103] Edit RBAC docs --- .../authentication/ad/_index.md | 2 +- .../authentication/openldap/_index.md | 4 +-- .../config-private-registry/_index.md | 2 +- .../rbac/cluster-project-roles/_index.md | 36 ++++++++----------- .../rbac/global-permissions/_index.md | 2 +- .../provisioning-vsphere-clusters/_index.md | 4 +-- .../enabling-uuids/_index.md | 2 +- .../rancher/v2.x/en/faq/technical/_index.md | 14 ++++---- .../rancher/v2.x/en/faq/telemetry/_index.md | 2 +- .../project-admin/resource-quotas/_index.md | 6 ++-- content/rancher/v2.x/en/security/_index.md | 2 +- .../v2.x/en/security/hardening-2.1/_index.md | 6 ++-- .../v2.x/en/security/hardening-2.2/_index.md | 6 ++-- .../v2.x/en/security/hardening-2.3/_index.md | 6 ++-- 14 files changed, 43 insertions(+), 51 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md index 6725f47f482..fbe444f333c 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/ad/_index.md @@ -110,7 +110,7 @@ Once you have completed the configuration, proceed by testing the connection to > **Note:** > -> The AD user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned admin privileges in Rancher. You should therefore make a conscious decision on which AD account you use to perform this step. +> The AD user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned administrator privileges in Rancher. You should therefore make a conscious decision on which AD account you use to perform this step. 1. Enter the **username** and **password** for the AD account that should be mapped to the local principal account. 2. Click **Authenticate with Active Directory** to finalise the setup. diff --git a/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md b/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md index 0ffa2607e3c..bce05911aac 100644 --- a/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md +++ b/content/rancher/v2.x/en/admin-settings/authentication/openldap/_index.md @@ -22,7 +22,7 @@ If your organization uses LDAP for user authentication, you can configure Ranche ## Prerequisites -Rancher must be configured with a LDAP bind account (aka service account) to search and retrieve LDAP entries pertaining to users and groups that should have access. It is recommended to not use an admin account or personal account for this purpose and instead create a dedicated account in OpenLDAP with read-only access to users and groups under the configured search base (see below). +Rancher must be configured with a LDAP bind account (aka service account) to search and retrieve LDAP entries pertaining to users and groups that should have access. It is recommended to not use an administrator account or personal account for this purpose and instead create a dedicated account in OpenLDAP with read-only access to users and groups under the configured search base (see below). > **Using TLS?** > @@ -109,7 +109,7 @@ Once you have completed the configuration, proceed by testing the connection to > **Note:** > -> The OpenLDAP user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned admin privileges in Rancher. You should therefore make a conscious decision on which LDAP account you use to perform this step. +> The OpenLDAP user pertaining to the credentials entered in this step will be mapped to the local principal account and assigned administrator privileges in Rancher. You should therefore make a conscious decision on which LDAP account you use to perform this step. 1. Enter the **username** and **password** for the OpenLDAP account that should be mapped to the local principal account. 2. Click **Authenticate With OpenLDAP** to test the OpenLDAP connection and finalise the setup. diff --git a/content/rancher/v2.x/en/admin-settings/config-private-registry/_index.md b/content/rancher/v2.x/en/admin-settings/config-private-registry/_index.md index 8e5441f51d1..692802124aa 100644 --- a/content/rancher/v2.x/en/admin-settings/config-private-registry/_index.md +++ b/content/rancher/v2.x/en/admin-settings/config-private-registry/_index.md @@ -19,7 +19,7 @@ If your private registry requires credentials, it cannot be used as the default # Setting a Private Registry with No Credentials as the Default Registry -1. Log into Rancher and configure the default admin password. +1. Log into Rancher and configure the default administrator password. 1. Go into the **Settings** view. diff --git a/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md index df0130f78ef..92292af2ab0 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/cluster-project-roles/_index.md @@ -11,7 +11,7 @@ The projects and clusters accessible to non-administrative users is determined b When you create a cluster or project, Rancher automatically assigns you as the `Owner` for it. Users assigned the `Owner` role can assign other users roles in the cluster or project. -> **Note:** Non-administrative users cannot access any existing projects/clusters by default. A user with appropriate permissions (typically the owner) must explicitly assign the user membership. +> **Note:** Non-administrative users cannot access any existing projects/clusters by default. A user with appropriate permissions (typically the owner) must explicitly assign the project and cluster membership. ### Cluster Roles @@ -27,7 +27,7 @@ _Cluster roles_ are roles that you can assign to users, granting them access to #### Custom Cluster Roles -Rancher lets you assign _custom cluster roles_ to a user instead of the typical `Owner` or `Member` roles. These roles can be either a built-in custom cluster role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a user within a cluster. See the table below for a list of built-in custom cluster roles. +Rancher lets you assign _custom cluster roles_ to a standard user instead of the typical `Owner` or `Member` roles. These roles can be either a built-in custom cluster role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a standard user within a cluster. See the table below for a list of built-in custom cluster roles. #### Cluster Role Reference @@ -53,32 +53,24 @@ For details on how each cluster role can access Kubernetes resources, you can go ### Giving a Custom Cluster Role to a Cluster Member -Admins can set up custom cluster roles that can be assigned to cluster owners and members. +After an administrator [sets up a custom cluster role,]({{}}/rancher/v2.x/en/admin-settings/rbac/default-custom-roles/#adding-a-custom-role) cluster owners and admins can then assign those roles to cluster members. -Cluster owners and admins can then assign those roles to cluster members. - -To create a custom cluster role, - -1. In the **Global** view under **Security > Roles,** click **Add Cluster Role.** - -1. In the **Grant Resources** section, choose any combination of operations on Kubernetes resources that will be allowed by the new role. Give the new cluster role a name and click **Create.** - -Then, from the **Cluster** view, go to the **Members** tab. From this tab, you can give the cluster role to members in two ways: - -- You can assign the role to a new member with the Rancher UI. -- You can assign the role to an existing member with the Rancher API view. +To assign a custom role to a new cluster member, you can use the Rancher UI. To modify the permissions of an existing member, you will need to use the Rancher API view. To assign the role to a new cluster member, -1. Click **Add Member.** Then in the **Cluster Permissions** section, you can choose your custom cluster role. +1. Go to the **Cluster** view, then go to the **Members** tab. +1. Click **Add Member.** Then in the **Cluster Permissions** section, choose the custom cluster role that should be assigned to the member. +1. Click **Create.** -1. When you click **Create**, the member should have the assigned role. +**Result:** The member has the assigned role. To assign any custom role to an existing cluster member, 1. Go to the member you want to give the role to. Click the **Ellipsis (...) > View in API.** +1. In the **roleTemplateId** field, go to the drop-down menu and choose the role you want to assign to the member. Click **Show Request** and **Send Request.** -1. In the **roleTemplateId** field, go to the drop-down menu and choose the role you want to assign to the member. Click **Show Request** and **Send Request.** After that, the member's role should be updated. +**Result:** The member has the assigned role. ### Project Roles @@ -103,7 +95,7 @@ _Project roles_ are roles that can be used to grant users access to a project. T #### Custom Project Roles -Rancher lets you assign _custom project roles_ to a user instead of the typical `Owner`, `Member`, or `Read Only` roles. These roles can be either a built-in custom project role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a user within a project. See the table below for a list of built-in custom project roles. +Rancher lets you assign _custom project roles_ to a standard user instead of the typical `Owner`, `Member`, or `Read Only` roles. These roles can be either a built-in custom project role or one defined by a Rancher administrator. They are convenient for defining narrow or specialized access for a standard user within a project. See the table below for a list of built-in custom project roles. #### Project Role Reference @@ -135,7 +127,7 @@ The following table lists each built-in custom project role available in Rancher > >- Each project role listed above, including `Owner`, `Member`, and `Read Only`, is comprised of multiple rules granting access to various resources. You can view the roles and their rules on the Global > Security > Roles page. >- When viewing the resources associated with default roles created by Rancher, if there are multiple Kuberenetes API resources on one line item, the resource will have `(Custom)` appended to it. These are not custom resources but just an indication that there are multiple Kubernetes API resources as one resource. ->- The `Manage Project Members` role allows the user to manage any members of the project **and** grant them any project scoped role regardless of their access to the project resources. Be cautious when assigning this role out individually. +>- The `Manage Project Members` role allows the project owner to manage any members of the project **and** grant them any project scoped role regardless of their access to the project resources. Be cautious when assigning this role out individually. ### Defining Custom Roles As previously mentioned, custom roles can be defined for use at the cluster or project level. The context field defines whether the role will appear on the cluster member page, project member page, or both. @@ -144,7 +136,7 @@ When defining a custom role, you can grant access to specific resources or speci ### Default Cluster and Project Roles -By default, when a user creates a new cluster or project, they are automatically assigned an ownership role: either [cluster owner](#cluster-roles) or [project owner](#project-roles). However, in some organizations, these roles may overextend administrative access. In this use case, you can change the default role to something more restrictive, such as a set of individual roles or a custom role. +By default, when a standard user creates a new cluster or project, they are automatically assigned an ownership role: either [cluster owner](#cluster-roles) or [project owner](#project-roles). However, in some organizations, these roles may overextend administrative access. In this use case, you can change the default role to something more restrictive, such as a set of individual roles or a custom role. There are two methods for changing default cluster/project roles: @@ -183,7 +175,7 @@ You can change the cluster or project role(s) that are automatically assigned to ### Cluster Membership Revocation Behavior -When you revoke the cluster membership for a user that's explicitly assigned membership to both the cluster _and_ a project within the cluster, that user [loses their cluster roles](#clus-roles) but [retains their project roles](#proj-roles). In other words, although you have revoked the user's permissions to access the cluster and its nodes, the user can still: +When you revoke the cluster membership for a standard user that's explicitly assigned membership to both the cluster _and_ a project within the cluster, that standard user [loses their cluster roles](#clus-roles) but [retains their project roles](#proj-roles). In other words, although you have revoked the user's permissions to access the cluster and its nodes, the standard user can still: - Access the projects they hold membership in. - Exercise any [individual project roles](#project-role-reference) they are assigned. diff --git a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md index fa84b3eb7e5..bf1834b68ee 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md @@ -37,7 +37,7 @@ When a user from an [external authentication source]({{< baseurl >}}/rancher/v2. However, in some organizations, these permissions may extend too much access. Rather than assigning users the default global permissions of `Administrator` or `Standard User`, you can assign them a more restrictive set of custom global permissions. -The default roles, Admin and User, each come with multiple global permissions built into them. The Admin role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role. +The default roles, Administrator and Standard User, each come with multiple global permissions built into them. The Administrator role includes all global permissions, while the default user role includes three global permissions: Create Clusters, Use Catalog Templates, and User Base, which is equivalent to the minimum permission to log in to Rancher. In other words, the custom global permissions are modularized so that if you want to change the default user role permissions, you can choose which subset of global permissions are included in the new default user role. Administrators can enforce custom global permissions in two ways: diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md index 39fa35b7528..9d9310ee096 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/_index.md @@ -65,7 +65,7 @@ After you create a node template, it is saved, and you can re-use it whenever yo To create a node template, -1. Log in with an admin account to the Rancher UI. +1. Log in with an administrator account to the Rancher UI. 1. From the user settings menu, select **Node Templates.** @@ -249,7 +249,7 @@ To create the cluster and enable the vSphere provider for cluster, follow these ### A. Set up the Cluster Name and Member Roles -1. Log in to the Rancher UI as an admin user. +1. Log in to the Rancher UI as an administrator. 2. Navigate to **Clusters** in the **Global** view. 3. Click **Add Cluster** and select the **vSphere** infrastructure provider. 4. Assign a **Cluster Name.** diff --git a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md index 4cb5a130602..55e8c2a2ae0 100644 --- a/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md +++ b/content/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/provisioning-vsphere-clusters/enabling-uuids/_index.md @@ -9,7 +9,7 @@ For Rancher prior to v2.0.4, we recommend configuring a vSphere node template to To enable disk UUIDs for all VMs created for a cluster, -1. Navigate to the **Node Templates** in the Rancher UI while logged in as admin user. +1. Navigate to the **Node Templates** in the Rancher UI while logged in as an administrator. 2. Add or edit an existing vSphere node template. diff --git a/content/rancher/v2.x/en/faq/technical/_index.md b/content/rancher/v2.x/en/faq/technical/_index.md index da0b2063fbc..0902e31456c 100644 --- a/content/rancher/v2.x/en/faq/technical/_index.md +++ b/content/rancher/v2.x/en/faq/technical/_index.md @@ -3,12 +3,12 @@ title: Technical weight: 8006 --- -### How can I reset the admin password? +### How can I reset the administrator password? Single node install: ``` $ docker exec -ti reset-password -New password for default admin user (user-xxxxx): +New password for default administrator (user-xxxxx): ``` @@ -16,7 +16,7 @@ High Availability install (Helm): ``` $ KUBECONFIG=./kube_config_rancher-cluster.yml $ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password -New password for default admin user (user-xxxxx): +New password for default administrator (user-xxxxx): ``` @@ -24,7 +24,7 @@ High Availability install (RKE add-on): ``` $ KUBECONFIG=./kube_config_rancher-cluster.yml $ kubectl --kubeconfig $KUBECONFIG exec -n cattle-system $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name=="cattle-server") | .metadata.name') -- reset-password -New password for default admin user (user-xxxxx): +New password for default administrator (user-xxxxx): ``` @@ -33,8 +33,8 @@ New password for default admin user (user-xxxxx): Single node install: ``` $ docker exec -ti ensure-default-admin -New default admin user (user-xxxxx) -New password for default admin user (user-xxxxx): +New default administrator (user-xxxxx) +New password for default administrator (user-xxxxx): ``` @@ -42,7 +42,7 @@ High Availability install (Helm): ``` $ KUBECONFIG=./kube_config_rancher-cluster.yml $ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- ensure-default-admin -New password for default admin user (user-xxxxx): +New password for default administrator (user-xxxxx): ``` diff --git a/content/rancher/v2.x/en/faq/telemetry/_index.md b/content/rancher/v2.x/en/faq/telemetry/_index.md index c8b04b9e05d..6ab582667e1 100644 --- a/content/rancher/v2.x/en/faq/telemetry/_index.md +++ b/content/rancher/v2.x/en/faq/telemetry/_index.md @@ -29,4 +29,4 @@ If Telemetry is not enabled, the process that collects the data is not running, ### How do I turn it on or off? -After initial setup, an admin user can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`. +After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`. diff --git a/content/rancher/v2.x/en/project-admin/resource-quotas/_index.md b/content/rancher/v2.x/en/project-admin/resource-quotas/_index.md index 150463f4a8c..b253dfb0fd4 100644 --- a/content/rancher/v2.x/en/project-admin/resource-quotas/_index.md +++ b/content/rancher/v2.x/en/project-admin/resource-quotas/_index.md @@ -15,7 +15,7 @@ Resource quotas in Rancher include the same functionality as the [native version In a standard Kubernetes deployment, resource quotas are applied to individual namespaces. However, you cannot apply the quota to your namespaces simultaneously with a single action. Instead, the resource quota must be applied multiple times. -In the following diagram, a Kubernetes admin is trying to enforce a resource quota without Rancher. The admin wants to apply a resource quota that sets the same CPU and memory limit to every namespace in his cluster (`Namespace 1-4`) . However, in the base version of Kubernetes, each namespace requires a unique resource quota. The admin has to create four different resource quotas that have the same specs configured (`Resource Quota 1-4`) and apply them individually. +In the following diagram, a Kubernetes administrator is trying to enforce a resource quota without Rancher. The administrator wants to apply a resource quota that sets the same CPU and memory limit to every namespace in his cluster (`Namespace 1-4`) . However, in the base version of Kubernetes, each namespace requires a unique resource quota. The administrator has to create four different resource quotas that have the same specs configured (`Resource Quota 1-4`) and apply them individually. Base Kubernetes: Unique Resource Quotas Being Applied to Each Namespace ![Native Kubernetes Resource Quota Implementation]({{< baseurl >}}/img/rancher/kubernetes-resource-quota.svg) @@ -33,7 +33,7 @@ The resource quota includes two limits, which you set while creating or editing This value is the default resource limit available for each namespace. When the resource quota is set on the project level, this limit is automatically propagated to each namespace in the project. Each namespace is bound to this default limit unless you [override it](#namespace-default-limit-overrides). -In the following diagram, a Rancher admin wants to apply a resource quota that sets the same CPU and memory limit for every namespace in their project (`Namespace 1-4`). However, in Rancher, the admin can set a resource quota for the project (`Project Resource Quota`) rather than individual namespaces. This quota includes resource limits for both the entire project (`Project Limit`) and individual namespaces (`Namespace Default Limit`). Rancher then propagates the `Namespace Default Limit` quotas to each namespace (`Namespace Resource Quota`). +In the following diagram, a Rancher administrator wants to apply a resource quota that sets the same CPU and memory limit for every namespace in their project (`Namespace 1-4`). However, in Rancher, the administrator can set a resource quota for the project (`Project Resource Quota`) rather than individual namespaces. This quota includes resource limits for both the entire project (`Project Limit`) and individual namespaces (`Namespace Default Limit`). Rancher then propagates the `Namespace Default Limit` quotas to each namespace (`Namespace Resource Quota`). Rancher: Resource Quotas Propagating to Each Namespace ![Rancher Resource Quota Implementation]({{< baseurl >}}/img/rancher/rancher-resource-quota.svg) @@ -80,7 +80,7 @@ When you create a resource quota, you are configuring the pool of resources avai Although the **Namespace Default Limit** propagates from the project to each namespace, in some cases, you may need to increase (or decrease) the performance for a specific namespace. In this situation, you can override the default limits by editing the namespace. -In the diagram below, the Rancher admin has a resource quota in effect for their project. However, the admin wants to override the namespace limits for `Namespace 3` so that it performs better. Therefore, the admin [raises the namespace limits]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/#editing-namespace-resource-quotas) for `Namespace 3` so that the namespace can access more resources. +In the diagram below, the Rancher administrator has a resource quota in effect for their project. However, the administrator wants to override the namespace limits for `Namespace 3` so that it performs better. Therefore, the administrator [raises the namespace limits]({{< baseurl >}}/rancher/v2.x/en/k8s-in-rancher/projects-and-namespaces/#editing-namespace-resource-quotas) for `Namespace 3` so that the namespace can access more resources. Namespace Default Limit Override ![Namespace Default Limit Override]({{< baseurl >}}/img/rancher/rancher-resource-quota-override.svg) diff --git a/content/rancher/v2.x/en/security/_index.md b/content/rancher/v2.x/en/security/_index.md index 0264aa61c58..c39ed1fca68 100644 --- a/content/rancher/v2.x/en/security/_index.md +++ b/content/rancher/v2.x/en/security/_index.md @@ -51,5 +51,5 @@ Rancher is committed to informing the community of security issues in our produc | [CVE-2019-12274](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12274) | Nodes using the built-in node drivers using a file path option allows the machine to read arbitrary files including sensitive ones from inside the Rancher server container. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) | | [CVE-2019-12303](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-12303) | Project owners can inject extra fluentd logging configurations that makes it possible to read files or execute arbitrary commands inside the fluentd container. Reported by Tyler Welton from Untamed Theory. | 5 Jun 2019 | [Rancher v2.2.4](https://github.com/rancher/rancher/releases/tag/v2.2.4), [Rancher v2.1.10](https://github.com/rancher/rancher/releases/tag/v2.1.10) and [Rancher v2.0.15](https://github.com/rancher/rancher/releases/tag/v2.0.15) | | [CVE-2019-13209](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-13209) | The vulnerability is known as a [Cross-Site Websocket Hijacking attack](https://www.christian-schneider.net/CrossSiteWebSocketHijacking.html). This attack allows an exploiter to gain access to clusters managed by Rancher with the roles/permissions of a victim. It requires that a victim to be logged into a Rancher server and then access a third-party site hosted by the exploiter. Once that is accomplished, the exploiter is able to execute commands against the Kubernetes API with the permissions and identity of the victim. Reported by Matt Belisle and Alex Stevenson from Workiva. | 15 Jul 2019 | [Rancher v2.2.5](https://github.com/rancher/rancher/releases/tag/v2.2.5), [Rancher v2.1.11](https://github.com/rancher/rancher/releases/tag/v2.1.11) and [Rancher v2.0.16](https://github.com/rancher/rancher/releases/tag/v2.0.16) | -| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them admin access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) | +| [CVE-2019-14436](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14436) | The vulnerability allows a member of a project that has access to edit role bindings to be able to assign themselves or others a cluster level role granting them administrator access to that cluster. The issue was found and reported by Michal Lipinski at Nokia. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) | | [CVE-2019-14435](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14435) | This vulnerability allows authenticated users to potentially extract otherwise private data out of IPs reachable from system service containers used by Rancher. This can include but not only limited to services such as cloud provider metadata services. Although Rancher allow users to configure whitelisted domains for system service access, this flaw can still be exploited by a carefully crafted HTTP request. The issue was found and reported by Matt Belisle and Alex Stevenson at Workiva. | 5 Aug 2019 | [Rancher v2.2.7](https://github.com/rancher/rancher/releases/tag/v2.2.7) and [Rancher v2.1.12](https://github.com/rancher/rancher/releases/tag/v2.1.12) | diff --git a/content/rancher/v2.x/en/security/hardening-2.1/_index.md b/content/rancher/v2.x/en/security/hardening-2.1/_index.md index e525794f055..565a9c2789d 100644 --- a/content/rancher/v2.x/en/security/hardening-2.1/_index.md +++ b/content/rancher/v2.x/en/security/hardening-2.1/_index.md @@ -870,7 +870,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log ## 3.2 - Rancher Management Control Plane Authentication -### 3.2.1 - Change the local admin password from the default value +### 3.2.1 - Change the local administrator password from the default value **Profile Applicability** @@ -878,11 +878,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log **Description** -The local admin password should be changed from the default. +The local administrator password should be changed from the default. **Rationale** -The default admin password is common across all Rancher installations and should be changed immediately upon startup. +The default administrator password is common across all Rancher installations and should be changed immediately upon startup. **Audit** diff --git a/content/rancher/v2.x/en/security/hardening-2.2/_index.md b/content/rancher/v2.x/en/security/hardening-2.2/_index.md index f6d24831f25..c699289667c 100644 --- a/content/rancher/v2.x/en/security/hardening-2.2/_index.md +++ b/content/rancher/v2.x/en/security/hardening-2.2/_index.md @@ -913,7 +913,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log ## 3.2 - Rancher Management Control Plane Authentication -### 3.2.1 - Change the local admin password from the default value +### 3.2.1 - Change the local administrator password from the default value **Profile Applicability** @@ -921,11 +921,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log **Description** -The local admin password should be changed from the default. +The local administrator password should be changed from the default. **Rationale** -The default admin password is common across all Rancher installations and should be changed immediately upon startup. +The default administrator password is common across all Rancher installations and should be changed immediately upon startup. **Audit** diff --git a/content/rancher/v2.x/en/security/hardening-2.3/_index.md b/content/rancher/v2.x/en/security/hardening-2.3/_index.md index ac8f578efcd..ab0ebdf9feb 100644 --- a/content/rancher/v2.x/en/security/hardening-2.3/_index.md +++ b/content/rancher/v2.x/en/security/hardening-2.3/_index.md @@ -1001,7 +1001,7 @@ Upgrade the Rancher server installation using Helm, and configure the audit log ## 3.2 - Rancher Management Control Plane Authentication -### 3.2.1 - Change the local admin password from the default value +### 3.2.1 - Change the local administrator password from the default value **Profile Applicability** @@ -1009,11 +1009,11 @@ Upgrade the Rancher server installation using Helm, and configure the audit log **Description** -The local admin password should be changed from the default. +The local administrator password should be changed from the default. **Rationale** -The default admin password is common across all Rancher installations and should be changed immediately upon startup. +The default administrator password is common across all Rancher installations and should be changed immediately upon startup. **Audit** From 8a9e52226844f1e6e3e984b4badddad08fb2bcd1 Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Mon, 2 Dec 2019 17:13:57 -0800 Subject: [PATCH 086/103] Update Helm version required for 1.16 per https://github.com/rancher/k3s/issues/887 --- content/rancher/v2.x/en/installation/helm-version/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/installation/helm-version/_index.md b/content/rancher/v2.x/en/installation/helm-version/_index.md index 2c07d4c4dee..3aa59b1fd28 100644 --- a/content/rancher/v2.x/en/installation/helm-version/_index.md +++ b/content/rancher/v2.x/en/installation/helm-version/_index.md @@ -7,6 +7,6 @@ This section contains the requirements for Helm, which is the tool used to insta > **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) -- Helm v2.15.1 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using. +- Helm v2.16.0 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using. - Helm v2.15.0 should not be used, because of an issue with converting/comparing numbers. -- Helm v2.12.0 should not be used, because of an issue with `cert-manager`. \ No newline at end of file +- Helm v2.12.0 should not be used, because of an issue with `cert-manager`. From 982bf3ac3591719c184b387060311fa4d7c7761f Mon Sep 17 00:00:00 2001 From: Guangbo Chen Date: Tue, 3 Dec 2019 10:46:31 +0800 Subject: [PATCH 087/103] Update catalog of supported questions types --- content/rancher/v2.x/en/catalog/custom/creating/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/catalog/custom/creating/_index.md b/content/rancher/v2.x/en/catalog/custom/creating/_index.md index ed6699953cb..b439be3da79 100644 --- a/content/rancher/v2.x/en/catalog/custom/creating/_index.md +++ b/content/rancher/v2.x/en/catalog/custom/creating/_index.md @@ -99,7 +99,7 @@ This reference contains variables that you can use in `questions.yml` nested und | variable | string | true | Define the variable name specified in the `values.yml` file, using `foo.bar` for nested objects. | | label | string | true | Define the UI label. | | description | string | false | Specify the description of the variable.| -| type | string | false | Default to `string` if not specified (current supported types are string, boolean, int, enum, password, storageclass and hostname).| +| type | string | false | Default to `string` if not specified (current supported types are string, multiline, boolean, int, enum, password, storageclass, hostname, pvc, and secret).| | required | bool | false | Define if the variable is required or not (true \| false)| | default | string | false | Specify the default value. | | group | string | false | Group questions by input value. | @@ -121,7 +121,7 @@ This reference contains variables that you can use in `questions.yml` nested und You can fill your custom catalogs with either Helm Charts or Rancher Charts, although we recommend Rancher Charts due to their enhanced user experience. ->**Note:** For a complete walkthrough of developing charts, see the upstream Helm chart [developer reference](https://docs.helm.sh/developing_charts/). +>**Note:** For a complete walkthrough of developing charts, see the upstream Helm chart [developer reference](https://helm.sh/docs/topics/chart_template_guide/). 1. Within the GitHub repo that you're using as your custom catalog, create a directory structure that mirrors the structure listed in [Chart Directory Structure](#chart-directory-structure). From aba2b8722cc62adf1e8eb8f8efe3f0f697f0e0c5 Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Tue, 3 Dec 2019 15:25:37 -0800 Subject: [PATCH 088/103] clarify wording for Rancher WRT RKE --- content/rancher/v2.x/en/installation/ha/_index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/_index.md b/content/rancher/v2.x/en/installation/ha/_index.md index 5c720e2373d..6e5a550a704 100644 --- a/content/rancher/v2.x/en/installation/ha/_index.md +++ b/content/rancher/v2.x/en/installation/ha/_index.md @@ -5,11 +5,11 @@ weight: 275 For production environments, we recommend installing Rancher in a high-availability configuration so that your user base can always access Rancher Server. When installed in a Kubernetes cluster, Rancher will integrate with the cluster's etcd database and take advantage of Kubernetes scheduling for high-availability. -This procedure walks you through setting up a 3-node cluster with RKE and installing the Rancher chart with the Helm package manager. +This procedure walks you through setting up a 3-node cluster with Rancher Kubernetes Engine (RKE) and installing the Rancher chart with the Helm package manager. -> **Important:** It is not supported, nor generally a good idea, to run Rancher on top of hosted Kubernetes solutions such as Amazon's EKS, or Google's GKE. These hosted Kubernetes solutions do not expose etcd to a degree that is manageable for Rancher, and their customizations can interfere with Rancher operations. It is strongly recommended to use hosted infrastructure such as EC2 or GCE instead. +> **Important:** Rancher Management Server can only be run on an RKE managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported. -> **Important:** For the best performance, we recommend this Kubernetes cluster to be dedicated only to run Rancher. After the Kubernetes cluster to run Rancher is setup, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads. +> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for Rancher Management Server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads. ## Recommended Architecture From 28c7773ac836efcc4c7941352a5eeac838c28336 Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Tue, 3 Dec 2019 15:34:36 -0800 Subject: [PATCH 089/103] Helm 3 docs update --- .../v2.x/en/installation/ha/helm-rancher/_index.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index ecf5b847dac..a81c3866f8f 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -9,7 +9,7 @@ For systems without direct internet access, see [Air Gap: High Availability Inst Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher. -> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) +> **Note:** The installation instructions assume you are using Helm 3. Migration of installs from Helm 2 to Helm 3 will be documented soon. ### Add the Helm Chart Repository @@ -21,6 +21,13 @@ Use `helm repo add` command to add the Helm chart repository that contains chart helm repo add rancher- https://releases.rancher.com/server-charts/ ``` +### Create a Namespace for Rancher +We'll need to define a namespace where the resources created by the Chart should be installed. This should always be `cattle-system`: + +``` +kubectl create namespace cattle-system +``` + ### Choose your SSL Configuration Rancher Server is designed to be secure by default and requires SSL/TLS configuration. @@ -77,7 +84,6 @@ These instructions are adapted from the [official cert-manager documentation](ht 1. Install the cert-manager Helm chart ```plain helm install \ - --name cert-manager \ --namespace cert-manager \ --version v0.9.1 \ jetstack/cert-manager @@ -108,7 +114,6 @@ The default is for Rancher to generate a CA and uses `cert-manager` to issue the ``` helm install rancher-/rancher \ - --name rancher \ --namespace cattle-system \ --set hostname=rancher.my.org ``` @@ -131,7 +136,6 @@ This option uses `cert-manager` to automatically request and renew [Let's Encryp ``` helm install rancher-/rancher \ - --name rancher \ --namespace cattle-system \ --set hostname=rancher.my.org \ --set ingress.tls.source=letsEncrypt \ @@ -158,7 +162,6 @@ Create Kubernetes secrets from your own certificates for Rancher to use. ``` helm install rancher-/rancher \ - --name rancher \ --namespace cattle-system \ --set hostname=rancher.my.org \ --set ingress.tls.source=secret From 76011a57092b64b08d528228a283803407ee7043 Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Tue, 3 Dec 2019 15:41:29 -0800 Subject: [PATCH 090/103] collapse content from helm init page into helm install --- .../en/installation/ha/helm-init/_index.md | 65 ------------------- .../ha/helm-init/troubleshooting/_index.md | 23 ------- .../en/installation/ha/helm-rancher/_index.md | 6 +- 3 files changed, 5 insertions(+), 89 deletions(-) delete mode 100644 content/rancher/v2.x/en/installation/ha/helm-init/_index.md delete mode 100644 content/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/_index.md diff --git a/content/rancher/v2.x/en/installation/ha/helm-init/_index.md b/content/rancher/v2.x/en/installation/ha/helm-init/_index.md deleted file mode 100644 index e0793246f7f..00000000000 --- a/content/rancher/v2.x/en/installation/ha/helm-init/_index.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -title: "3. Initialize Helm (Install Tiller)" -weight: 195 ---- - -Helm is the package management tool of choice for Kubernetes. Helm "charts" provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at [https://helm.sh/](https://helm.sh/). To be able to use Helm, the server-side component `tiller` needs to be installed on your cluster. - -For systems without direct internet access, see [Helm - Air Gap]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#helm) for install details. - -Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher. - -> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) - -### Install Tiller on the Cluster - -> **Important:** Due to an issue with Helm v2.12.0 and cert-manager, please use Helm v2.12.1 or higher. - -Helm installs the `tiller` service on your cluster to manage charts. Since RKE enables RBAC by default we will need to use `kubectl` to create a `serviceaccount` and `clusterrolebinding` so `tiller` has permission to deploy to the cluster. - -* Create the `ServiceAccount` in the `kube-system` namespace. -* Create the `ClusterRoleBinding` to give the `tiller` account access to the cluster. -* Finally use `helm` to install the `tiller` service - -```plain -kubectl -n kube-system create serviceaccount tiller - -kubectl create clusterrolebinding tiller \ - --clusterrole=cluster-admin \ - --serviceaccount=kube-system:tiller - -helm init --service-account tiller - -# Users in China: You will need to specify a specific tiller-image in order to initialize tiller. -# The list of tiller image tags are available here: https://dev.aliyun.com/detail.html?spm=5176.1972343.2.18.ErFNgC&repoId=62085. -# When initializing tiller, you'll need to pass in --tiller-image - -helm init --service-account tiller \ ---tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller: -``` - -> **Note:** This`tiller`install has full cluster access, which should be acceptable if the cluster is dedicated to Rancher server. Check out the [helm docs](https://docs.helm.sh/using_helm/#role-based-access-control) for restricting `tiller` access to suit your security requirements. - -### Test your Tiller installation - -Run the following command to verify the installation of `tiller` on your cluster: - -``` -kubectl -n kube-system rollout status deploy/tiller-deploy -Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available... -deployment "tiller-deploy" successfully rolled out -``` - -And run the following command to validate Helm can talk to the `tiller` service: - -``` -helm version -Client: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"} -Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"} -``` - -### Issues or errors? - -See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/) page. - -### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) diff --git a/content/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/_index.md b/content/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/_index.md deleted file mode 100644 index c73013b5cb8..00000000000 --- a/content/rancher/v2.x/en/installation/ha/helm-init/troubleshooting/_index.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Troubleshooting -weight: 276 ---- - -### Helm commands show forbidden - -When Helm is initiated in the cluster without specifying the correct `ServiceAccount`, the command `helm init` will succeed but you won't be able to execute most of the other `helm` commands. The following error will be shown: - -``` -Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system" -``` - -To resolve this, the server component (`tiller`) needs to be removed and added with the correct `ServiceAccount`. You can use `helm reset --force` to remove the `tiller` from the cluster. Please check if it is removed using `helm version --server`. - -``` -helm reset --force -Tiller (the Helm server-side component) has been uninstalled from your Kubernetes Cluster. -helm version --server -Error: could not find tiller -``` - -When you have confirmed that `tiller` has been removed, please follow the steps provided in [Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) to install `tiller` with the correct `ServiceAccount`. diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index a81c3866f8f..49bd4062e41 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -3,7 +3,7 @@ title: "4. Install Rancher" weight: 200 --- -Rancher installation is managed using the Helm package manager for Kubernetes. Use `helm` to install the prerequisite and charts to install Rancher. +Rancher installation is managed using the Helm package manager for Kubernetes. Helm “charts” provide templating syntax for Kubernetes YAML manifest documents. With Helm we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at https://helm.sh/. For systems without direct internet access, see [Air Gap: High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/). @@ -11,6 +11,10 @@ Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installat > **Note:** The installation instructions assume you are using Helm 3. Migration of installs from Helm 2 to Helm 3 will be documented soon. +### Install Helm + +Helm requires a simple CLI tool to be installed. Refer to the [instructions provided by the Helm project](https://helm.sh/docs/intro/install/) for your specific platofrm. + ### Add the Helm Chart Repository Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories). From bff5939e50e84901edbed3f1581f51af5606dc71 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Tue, 3 Dec 2019 16:47:01 -0700 Subject: [PATCH 091/103] Capitalization edits --- content/rancher/v2.x/en/installation/ha/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/_index.md b/content/rancher/v2.x/en/installation/ha/_index.md index 6e5a550a704..0784b0f76cf 100644 --- a/content/rancher/v2.x/en/installation/ha/_index.md +++ b/content/rancher/v2.x/en/installation/ha/_index.md @@ -7,9 +7,9 @@ For production environments, we recommend installing Rancher in a high-availabil This procedure walks you through setting up a 3-node cluster with Rancher Kubernetes Engine (RKE) and installing the Rancher chart with the Helm package manager. -> **Important:** Rancher Management Server can only be run on an RKE managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported. +> **Important:** The Rancher management server can only be run on an RKE-managed Kubernetes cluster. Use of Rancher on hosted Kubernetes or other providers is not supported. -> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for Rancher Management Server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads. +> **Important:** For the best performance, we recommend a dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/#cluster-creation-in-rancher) for running your workloads. ## Recommended Architecture From b2ef914ed7262f6fb8b4918ad7eb2cea26758486 Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Tue, 3 Dec 2019 15:47:28 -0800 Subject: [PATCH 092/103] mention migration path to helm3 --- content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index 49bd4062e41..ec68d1ae75e 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -9,7 +9,7 @@ For systems without direct internet access, see [Air Gap: High Availability Inst Refer to the [Helm version requirements]({{}}/rancher/v2.x/en/installation/helm-version) to choose a version of Helm to install Rancher. -> **Note:** The installation instructions assume you are using Helm 3. Migration of installs from Helm 2 to Helm 3 will be documented soon. +> **Note:** The installation instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 Migration Docs](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) ### Install Helm From f00f114a84decc7d4ae383c73cab94e5b4f15a2d Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Tue, 3 Dec 2019 15:49:36 -0800 Subject: [PATCH 093/103] forgot to remove tiller from TOC --- content/rancher/v2.x/en/installation/ha/_index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/ha/_index.md b/content/rancher/v2.x/en/installation/ha/_index.md index 5c720e2373d..70bfe70357f 100644 --- a/content/rancher/v2.x/en/installation/ha/_index.md +++ b/content/rancher/v2.x/en/installation/ha/_index.md @@ -34,7 +34,6 @@ The following CLI tools are required for this install. Please make sure these to - [Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/) - [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) -- [Initialize Helm (tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) - [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) ## Additional Install Options From 0be7beae94c68b4b752afbec8ec7e6728b86975c Mon Sep 17 00:00:00 2001 From: William Jimenez Date: Wed, 4 Dec 2019 09:55:15 -0800 Subject: [PATCH 094/103] note requirements for using PSPs in a project --- .../v2.x/en/admin-settings/pod-security-policies/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md b/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md index 51b190231ef..470f20bc51f 100644 --- a/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md +++ b/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md @@ -17,6 +17,8 @@ _Pod Security Policies_ (or PSPs) are objects that control security-sensitive as - You can override the default PSP by assigning a different PSP directly to the project. - Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked if it complies with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP. +>**NOTE:** You must enable PSPs at the cluster level before you can assign them to a project. This can be found under the "Edit cluster" menu. + Read more about Pod Security Policies in the [Kubernetes Documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/). >**Best Practice:** Set pod security at the cluster level. From af7249e411d8e005da2b533b9706a9d5a7bd3936 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Wed, 4 Dec 2019 16:04:24 -0700 Subject: [PATCH 095/103] Fix shortcode error --- content/rancher/v2.x/en/admin-settings/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/admin-settings/_index.md b/content/rancher/v2.x/en/admin-settings/_index.md index 01bbb1343ff..d23b5e3d99f 100644 --- a/content/rancher/v2.x/en/admin-settings/_index.md +++ b/content/rancher/v2.x/en/admin-settings/_index.md @@ -52,7 +52,7 @@ The information that Rancher uses to provision [RKE clusters]({{< baseurl >}}/ra Rancher Kubernetes Metadata contains Kubernetes version information which Rancher uses to provision [RKE clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/). -For more information on how metadata works and how to configure metadata config, see [Rancher Kubernetes Metadata]({{}}/rancher/v2.x/en/admin-settings/k8s-metadata). ## Enabling Experimental Features From a41f4f1fee9ef57db6f6ecdba701a864df68970a Mon Sep 17 00:00:00 2001 From: Max Gautier Date: Thu, 5 Dec 2019 14:20:20 +0100 Subject: [PATCH 096/103] Fix merge conflict marks --- .../v2.x/en/admin-settings/rbac/global-permissions/_index.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md index bf1834b68ee..829f579537e 100644 --- a/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md +++ b/content/rancher/v2.x/en/admin-settings/rbac/global-permissions/_index.md @@ -92,9 +92,6 @@ To change the default global permissions that are assigned to external users upo 1. If you want to remove a default permission, edit the permission and select **No** from **New User Default**. -<<<<<<< HEAD -**Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column. -======= **Result:** The default global permissions are configured based on your changes. Permissions assigned to new users display a check in the **New User Default** column. ### Configuring Global Permissions for Individual Users @@ -112,5 +109,3 @@ To configure permission for a user, 1. Click **Save.** > **Result:** The user's global permissions have been updated. - ->>>>>>> Update and clarify docs on global and cluster permissions From 82edc927d1abf1abaca46435d0fddbdcd630b568 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 5 Dec 2019 11:43:30 -0700 Subject: [PATCH 097/103] Fix Helm command according to Gaurav's feedback --- .../rancher/v2.x/en/installation/ha/helm-rancher/_index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index ec68d1ae75e..b35f6af9215 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -117,7 +117,7 @@ The default is for Rancher to generate a CA and uses `cert-manager` to issue the - Set the `hostname` to the DNS name you pointed at your load balancer. ``` -helm install rancher-/rancher \ +helm install rancher rancher-/rancher \ --namespace cattle-system \ --set hostname=rancher.my.org ``` @@ -139,7 +139,7 @@ This option uses `cert-manager` to automatically request and renew [Let's Encryp - Set `hostname` to the public DNS record, set `ingress.tls.source` to `letsEncrypt` and `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices) ``` -helm install rancher-/rancher \ +helm install rancher rancher-/rancher \ --namespace cattle-system \ --set hostname=rancher.my.org \ --set ingress.tls.source=letsEncrypt \ @@ -165,7 +165,7 @@ Create Kubernetes secrets from your own certificates for Rancher to use. - If you are using a Private CA signed certificate , add `--set privateCA=true` to the command shown below. ``` -helm install rancher-/rancher \ +helm install rancher rancher-/rancher \ --namespace cattle-system \ --set hostname=rancher.my.org \ --set ingress.tls.source=secret From dee287e607457ef2d48fe507087e14d428803066 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 5 Dec 2019 11:53:00 -0700 Subject: [PATCH 098/103] Update note about Helm version --- content/rancher/v2.x/en/installation/helm-version/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/helm-version/_index.md b/content/rancher/v2.x/en/installation/helm-version/_index.md index 3aa59b1fd28..73de4e8f350 100644 --- a/content/rancher/v2.x/en/installation/helm-version/_index.md +++ b/content/rancher/v2.x/en/installation/helm-version/_index.md @@ -5,7 +5,7 @@ weight: 400 This section contains the requirements for Helm, which is the tool used to install Rancher on a high-availability Kubernetes cluster. -> **Note:** The installation instructions assume you are using Helm 2. The instructions will be updated for Helm 3 soon. In the meantime, if you want to use Helm 3, refer to [these instructions.](https://github.com/ibrokethecloud/rancher-helm3) +> **Note:** The installation instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 Migration Docs](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) - Helm v2.16.0 or higher is required for Kubernetes v1.16. For the default Kubernetes version, refer to the [release notes](https://github.com/rancher/rke/releases) for the version of RKE that you are using. - Helm v2.15.0 should not be used, because of an issue with converting/comparing numbers. From 84110a6dfa93876557c17f518bcb5c2373a294d5 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 5 Dec 2019 11:57:14 -0700 Subject: [PATCH 099/103] Minor corrections to installation docs --- content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md | 2 +- .../rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md | 2 +- .../en/upgrades/upgrades/migrating-from-rke-add-on/_index.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index b35f6af9215..542a9cdddf3 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -1,5 +1,5 @@ --- -title: "4. Install Rancher" +title: "3. Install Rancher" weight: 200 --- diff --git a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md index b57911e5fb2..d1b6ef4a571 100644 --- a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md +++ b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md @@ -127,4 +127,4 @@ Save a copy of the following files in a secure location: See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/troubleshooting/) page. -### [Next: Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) +### [Next: Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) diff --git a/content/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/_index.md b/content/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/_index.md index fe49a4d2552..50cb6dc948f 100644 --- a/content/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/_index.md +++ b/content/rancher/v2.x/en/upgrades/upgrades/migrating-from-rke-add-on/_index.md @@ -105,5 +105,5 @@ addons: |- From here follow the standard install steps. -* [3 - Initialize Helm (Install tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) +* [3 - Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) * [4 - Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) From 1c4dd738e151086bd4ebf4f4d47ab53bf9ebaaa3 Mon Sep 17 00:00:00 2001 From: David Nuzik Date: Thu, 5 Dec 2019 13:17:26 -0700 Subject: [PATCH 100/103] Small tweak to K3s airgap page - Changes the Install K3s code block so it's in two separate codeblocks so that someone won't confuse it as one command for each node. --- content/k3s/latest/en/installation/airgap/_index.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/content/k3s/latest/en/installation/airgap/_index.md b/content/k3s/latest/en/installation/airgap/_index.md index f919b3768ce..e96967163c1 100644 --- a/content/k3s/latest/en/installation/airgap/_index.md +++ b/content/k3s/latest/en/installation/airgap/_index.md @@ -52,13 +52,15 @@ Also obtain the K3s install script at https://get.k3s.io Place the binary in `/usr/local/bin` on each node. Place the install script anywhere on each node, name it `install.sh`. -Install K3s on each node. The example below shows how to do this for a server or an agent (worker): +Install K3s on each server: ``` -# K3s Server INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh +``` -# K3s Agent +Install K3s on each agent: + +``` INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken ./install.sh ``` From 7583002c8f07a09dac251aa8a66d46bcfeb53e07 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Thu, 5 Dec 2019 13:42:15 -0700 Subject: [PATCH 101/103] Update cert-manager install command for Helm 3 --- .../rancher/v2.x/en/installation/ha/helm-rancher/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index 542a9cdddf3..a6dc63ea12f 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -88,9 +88,9 @@ These instructions are adapted from the [official cert-manager documentation](ht 1. Install the cert-manager Helm chart ```plain helm install \ + cert-manager jetstack/cert-manager \ --namespace cert-manager \ - --version v0.9.1 \ - jetstack/cert-manager + --version v0.9.1 ``` Once you’ve installed cert-manager, you can verify it is deployed correctly by checking the cert-manager namespace for running pods: From ccf33fc3918bd325a9d7d72ec9e14cd33699c384 Mon Sep 17 00:00:00 2001 From: panlith <55960073+panlith@users.noreply.github.com> Date: Fri, 6 Dec 2019 17:11:47 +0200 Subject: [PATCH 102/103] Update _index.md --- .../rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md index d1b6ef4a571..b9169b8b713 100644 --- a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md +++ b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md @@ -127,4 +127,4 @@ Save a copy of the following files in a secure location: See the [Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/troubleshooting/) page. -### [Next: Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) +### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) From a98fec3dd2c9ff84b0722352d150cc1419e96d51 Mon Sep 17 00:00:00 2001 From: Catherine Luse Date: Fri, 6 Dec 2019 11:51:54 -0700 Subject: [PATCH 103/103] Link to docs about editing clusters --- .../v2.x/en/admin-settings/pod-security-policies/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md b/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md index 470f20bc51f..2ff9bd75b55 100644 --- a/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md +++ b/content/rancher/v2.x/en/admin-settings/pod-security-policies/_index.md @@ -17,7 +17,7 @@ _Pod Security Policies_ (or PSPs) are objects that control security-sensitive as - You can override the default PSP by assigning a different PSP directly to the project. - Any workloads that are already running in a cluster or project before a PSP is assigned will not be checked if it complies with the PSP. Workloads would need to be cloned or upgraded to see if they pass the PSP. ->**NOTE:** You must enable PSPs at the cluster level before you can assign them to a project. This can be found under the "Edit cluster" menu. +>**Note:** You must enable PSPs at the cluster level before you can assign them to a project. This can be configured by [editing the cluster.]({{}}/rancher/v2.x/en/cluster-admin/editing-clusters/) Read more about Pod Security Policies in the [Kubernetes Documentation](https://kubernetes.io/docs/concepts/policy/pod-security-policy/).