From 5779ef5b2728408afca683c00631031b2be692e5 Mon Sep 17 00:00:00 2001 From: Craig Jellick Date: Tue, 24 Mar 2020 19:48:13 -0700 Subject: [PATCH] Add docs for automated k3s upgrades --- content/k3s/latest/en/upgrades/_index.md | 40 +----- .../latest/en/upgrades/automated/_index.md | 115 ++++++++++++++++++ .../k3s/latest/en/upgrades/basic/_index.md | 44 +++++++ 3 files changed, 162 insertions(+), 37 deletions(-) create mode 100644 content/k3s/latest/en/upgrades/automated/_index.md create mode 100644 content/k3s/latest/en/upgrades/basic/_index.md diff --git a/content/k3s/latest/en/upgrades/_index.md b/content/k3s/latest/en/upgrades/_index.md index 3ce3a0591a3..58c34361b0a 100644 --- a/content/k3s/latest/en/upgrades/_index.md +++ b/content/k3s/latest/en/upgrades/_index.md @@ -3,42 +3,8 @@ title: "Upgrades" weight: 25 --- -You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version. +This section describes how to upgrade your K3s cluster. ->**Note:** When upgrading, upgrade server nodes first one at a time, then any worker nodes. +[Upgrade basics]({{< baseurl >}}/k3s/latest/en/upgrades/basic/) describes several techniques for upgrading your cluster manually. It can also be used as a basis for upgrading through third-party Infrastructure-as-Code tools like [Terraform](https://www.terraform.io/). -### Upgrade K3s Using the Installation Script - -To upgrade K3s from an older version you can re-run the installation script using the same flags, for example: - -```sh -curl -sfL https://get.k3s.io | sh - -``` - -If you want to upgrade to specific version you can run the following command: - -```sh -curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - -``` - -### Manually Upgrade K3s Using the Binary - -Or to manually upgrade K3s: - -1. Download the desired version of K3s from [releases](https://github.com/rancher/k3s/releases/latest) -2. Install to an appropriate location (normally `/usr/local/bin/k3s`) -3. Stop the old version -4. Start the new version - -### Restarting K3s - -Restarting K3s is supported by the installation script for systemd and openrc. -To restart manually for systemd use: -```sh -sudo systemctl restart k3s -``` - -To restart manually for openrc use: -```sh -sudo service k3s restart -``` \ No newline at end of file +[Automated upgrades]({{< baseurl >}}/k3s/latest/en/upgrades/automated/) describes how to perform Kubernetes-native automated upgrades using Rancher's [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller). diff --git a/content/k3s/latest/en/upgrades/automated/_index.md b/content/k3s/latest/en/upgrades/automated/_index.md new file mode 100644 index 00000000000..c595431182d --- /dev/null +++ b/content/k3s/latest/en/upgrades/automated/_index.md @@ -0,0 +1,115 @@ +--- +title: "Automated Upgrades" +weight: 20 +--- + +>**Note:** This feature is available as of [v1.17.4+k3s1](https://github.com/rancher/k3s/releases/tag/v1.17.4%2Bk3s1) + +### Overview + +You can manage K3s cluster upgrades using Rancher's system-upgrade-controller. This is a Kubernetes-native approach to cluster upgrades. It leverages a [custom resource definition (CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#custom-resources), the `plan`, and a [controller](https://kubernetes.io/docs/concepts/architecture/controller/) that schedules upgrades based on the configured plans. + +A plan defines upgrade policies and requirements. This documentation will provide plans with defaults appropriate for upgrading a K3s cluster. For more advanced plan configuration options, please review the [CRD](https://github.com/rancher/system-upgrade-controller/blob/master/pkg/apis/upgrade.cattle.io/v1/types.go). + +The controller schedules upgrades by monitoring plans and selecting nodes to run upgrade [jobs](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) on. A plan defines which nodes should be upgraded through a [label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). When a job has run to completion successfully, the controller will label the node on which it ran accordingly. + +>**Note:** The upgrade job that is launched must be highly privileged. It is configured with the following: +> +- Host `IPC`, `NET`, and `PID` namespaces +- The `CAP_SYS_BOOT` capability +- Host root mounted at `/host` with read and write permissions + +For more details on the design and architecture of the system-upgrade-controller or its integration with K3s, see the following Git repositories: + +- [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) +- [k3s-upgrade](https://github.com/rancher/k3s-upgrade) + +To automate upgrades in this manner you must: + +1. Install the system-upgrade-controller into your cluster +1. Configure plans + + +### Install the system-upgrade-controller +The system-upgrade-controller can be installed as a deployment into your cluster. The deployment requires a service-account, clusterRoleBinding, and a configmap. To install these components, run the following command: +``` +kubectl apply -f https://raw.githubusercontent.com/rancher/system-upgrade-controller/master/manifests/system-upgrade-controller.yaml +``` +The controller can be configured and customized via the previously mentioned configmap, but the controller must be redeployed for the changes to be applied. + + +### Configure plans +It is recommended that you minimally create two plans: a plan for upgrading server (master) nodes and a plan for upgrading agent (worker) nodes. As needed, you can create additional plans to control the rollout of the upgrade across nodes. The following two example plans will upgrade your cluster to K3s v1.17.4+k3s1. Once the plans are created, the controller will pick them up and begin to upgrade your cluster. +``` +# Server plan +apiVersion: upgrade.cattle.io/v1 +kind: Plan +metadata: + name: server-plan + namespace: system-upgrade +spec: + concurrency: 1 + cordon: true + nodeSelector: + matchExpressions: + - key: node-role.kubernetes.io/master + operator: In + values: + - "true" + serviceAccountName: system-upgrade + upgrade: + image: rancher/k3s-upgrade + version: v1.17.4+k3s1 +--- +# Agent plan +apiVersion: upgrade.cattle.io/v1 +kind: Plan +metadata: + name: agent-plan + namespace: system-upgrade +spec: + concurrency: 1 + cordon: true + nodeSelector: + matchExpressions: + - key: node-role.kubernetes.io/master + operator: DoesNotExist + prepare: + args: + - prepare + - server-plan + image: rancher/k3s-upgrade:v1.17.4-k3s1 + serviceAccountName: system-upgrade + upgrade: + image: rancher/k3s-upgrade + version: v1.17.4+k3s1 +``` +There are a few important things to call out regarding these plans: + +First, the plans must be created in the same namespace where the controller was deployed. + +Second, the `concurrency` field indicates how many nodes can be upgraded at the same time. + +Third, the server-plan targets server nodes by specifying a label selector that selects nodes with the `node-role.kubernetes.io/master` label. The agent-plan targets agent nodes by specifying a label selector that select nodes without that label. + +Fourth, the `prepare` step in the agent-plan will cause upgrade jobs for that plan to wait for the server-plan to complete before they execute. + +Fifth, both plans have the `version` field set to v1.17.4+k3s1. Alternatively, you can omit the `version` field and set the `channel` field to a URL that resolves to a release of K3s. This will cause the controller to monitor that URL and upgrade the cluster any time it resolves to a new release. This is designed specifically to work with the [latest release functionality of GitHub](https://help.github.com/en/github/administering-a-repository/linking-to-releases). Thus, you can configure your plans with the following channel to ensure your cluster is always automatically upgraded to the latest release of K3s: +``` +apiVersion: upgrade.cattle.io/v1 +kind: Plan +... +spec: + ... + channel: https://github.com/rancher/k3s/releases/latest + +``` + +As stated, the upgrade will begin as soon as the controller detects that a plan was created. Updating a plan will cause the controller to re-evaluate the plan and determine if another upgrade is needed. + +You can monitor the progress of an upgrade by viewing the plan and jobs via kubectl: +``` +kubectl -n system-upgrade get plans -o yaml +kubectl -n system-upgrade get jobs -o yaml +``` + diff --git a/content/k3s/latest/en/upgrades/basic/_index.md b/content/k3s/latest/en/upgrades/basic/_index.md new file mode 100644 index 00000000000..6311b045764 --- /dev/null +++ b/content/k3s/latest/en/upgrades/basic/_index.md @@ -0,0 +1,44 @@ +--- +title: "Upgrade Basics" +weight: 10 +--- + +You can upgrade K3s by using the installation script, or by manually installing the binary of the desired version. + +>**Note:** When upgrading, upgrade server nodes first one at a time, then any worker nodes. + +### Upgrade K3s Using the Installation Script + +To upgrade K3s from an older version you can re-run the installation script using the same flags, for example: + +```sh +curl -sfL https://get.k3s.io | sh - +``` + +If you want to upgrade to specific version you can run the following command: + +```sh +curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=vX.Y.Z-rc1 sh - +``` + +### Manually Upgrade K3s Using the Binary + +Or to manually upgrade K3s: + +1. Download the desired version of the K3s binary from [releases](https://github.com/rancher/k3s/releases) +2. Copy the downloaded binary to `/usr/local/bin/k3s` (or your desired location) +3. Stop the old k3s binary +4. Launch the new k3s binary + +### Restarting K3s + +Restarting K3s is supported by the installation script for systemd and openrc. +To restart manually for systemd use: +```sh +sudo systemctl restart k3s +``` + +To restart manually for openrc use: +```sh +sudo service k3s restart +```