diff --git a/content/rancher/v2.x/en/installation/backups-and-restoration/_index.md b/content/rancher/v2.x/en/installation/backups-and-restoration/_index.md deleted file mode 100644 index bceb0fa3920..00000000000 --- a/content/rancher/v2.x/en/installation/backups-and-restoration/_index.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: Backups and Restoration -weight: 345 -aliases: - - /rancher/v2.x/en/installation/after-installation/ ---- -This section contains information about how to create backups of your Rancher data and how to restore them in a disaster scenario. - -- [Single Node Install Backup and Restoration]({{< baseurl >}}/rancher/v2.x/en/installation/backups-and-restoration/single-node-backup-and-restoration/) -- [High Availability Install Backup and Restoration]({{< baseurl >}}/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration/) diff --git a/content/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration/_index.md b/content/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration/_index.md deleted file mode 100644 index 466688d21ff..00000000000 --- a/content/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration/_index.md +++ /dev/null @@ -1,237 +0,0 @@ ---- -title: High Availability Backup and Restoration -weight: 370 -aliases: - - /rancher/v2.x/en/installation/after-installation/ha-backup-and-restoration/ ---- -This section describes how to: - -- Create backups of your high-availability Rancher install. -- Restore the backups in a disaster scenario. - -## Backup Workflow - -Backing up your high-availability Rancher cluster is process that involves completing multiple tasks. - -1. [Meet Backup Prerequisites](#1-meet-backup-prerequisites) - - Before starting, make sure you have the files needed to create backups. - -2. [Take Snapshots of the `etcd` Database](#2-take-snapshots-of-the-etcd-database) - - Take snapshots of your current `etcd` database using Rancher Kubernetes Engine (RKE). - -3. [Store Snapshot(s) Externally](#3-store-snapshots-externally) - - After taking your snapshots, export them to a safe location that won't be affected if your cluster encounters issues. - -
-### 1. Meet Backup Prerequisites - -Begin by gathering the files that you need to create backups of your Rancher install. - -#### Prerequisites - -- Rancher Kubernetes Engine v0.1.7 or later - - The commands for taking `etcd` snapshots are only available in RKE v0.1.7 and later. - -- rancher-cluster.yml - - You'll need the RKE config file you used for Rancher install, `rancher-cluster.yml`. You created this file during your chosen high-availability installation scenario: -
-
- - [High Availability Installation]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install) - - [High Availability Installation with External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install-external-lb) - -
-### 2. Take Snapshots of the `etcd` Database - -Take snapshots of your `etcd` database. You can use these snapshots later to recover from a disaster scenario. There are two ways to take snapshots: recurringly, or as a one-off. Each option is better suited to a specific use case. Read the short description below each link to know when to use each option. - -- [Option A: Recurring Snapshots](#option-a-recurring-snapshots) - - After you stand up a high-availability Rancher install, we recommend configuring RKE to automatically take recurring snapshots so that you always have a safe restoration point available. - -- [Option B: One-Time Snapshots](#option-b-one-time-snapshots) - - We advise taking one-time snapshots before events like upgrades or restoration of another snapshot. - -#### Option A: Recurring Snapshots - -For all high-availability Rancher installs, we recommend taking recurring snapshots so that you always have a safe restoration point available. - -To take recurring snapshots, enable the `etcd-snapshot` service, which is a service that's included with RKE. This service runs in a service container alongside the `etcd` container. You can enable this service by adding some code to `rancher-cluster.yml`. - -**To Enable Recuring Snapshots:** - -1. Open `rancher-cluster.yml` with your favorite text editor. - -2. Add the following code block to the bottom of the file: - - ``` - services: - etcd: - snapshot: true # enables recurring etcd snapshots - creation: 5m0s # time increment between shapshots - retention: 24h # time increment before snapshot purge - ``` - -3. Edit the code according to your requirements. - -4. Save and close `rancher-cluster.yml`. - -5. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. - -6. Run one of the following commands: - - ``` - # MacOS - ./rke_darwin-amd64 up --config rancher-cluster.yml - # Linux - ./rke_linux-amd64 up --config rancher-cluster.yml - ``` - - -**Result:** RKE is configured to take recurring snapshots of `etcd` on all nodes running the `etcd` role. Snapshots are saved to the following directory: `/opt/rke/etcd-snapshots/`. - -#### Option B: One-Time Snapshots - -When you're about to upgrade Rancher or restore it to a previous snapshot, you should snapshot your live image so that you have a backup of `etcd` in its last known state. - -**To Take a One-Time Snapshot:** - -1. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. - -2. Enter the following command. Replace `` with any name that you want to use for the snapshot (e.g. `upgrade.db`). - - ``` - # MacOS - ./rke_darwin-amd64 etcd snapshot-save --name --config rancher-cluster.yml - # Linux - ./rke_linux-amd64 etcd snapshot-save --name --config rancher-cluster.yml - ``` - -**Result:** RKE takes a snapshot of `etcd` running on each `etcd` node. The file is saved to `/opt/rke/etcd-snapshots`. - -### 3. Backup Snapshots to a Safe Location - -After taking the `etcd` snapshots, save them to a safe location so that they're unaffected if your cluster experiences a disaster scenrio. This location should be persistent. - -In this documentation, as an example, we're using Amazon S3 as our safe location, and [S3cmd](http://s3tools.org/s3cmd) as our tool to create the backups. The backup location and tool that you use are ultimately your decision. - -**Example:** - -``` -root@node:~# s3cmd mb s3://rke-etcd-snapshots -root@node:~# s3cmd /opt/rke/etcd-snapshots/snapshot.db s3://rke-etcd-snapshots/ -``` - - -## Restoration Workflow - -Following a disaster scenario, restoration of your HA Rancher installation requires you to pull your snapshot from your chosen external location and then restore it. - -1. [Create New Node and Pull Snapshot](#1-create-new-node-and-pull-snapshot) - - If one of your `etcd` nodes goes down, create a new node, and then pull the most recent `etcd` snapshot to that node. - -2. [Restore etcd Database](#2-restore-etcd-database) - - After you pull the snapshot, run the RKE command to restore the `etcd` database. - -
-### 1. Create New Node and Pull Snapshot - -If one of your `etcd` nodes go down, you need to replace it with a new node, and then pull the most recent working `etcd` snapshot to that node. - -**To Create a New Node and Pull the Latest Snapshot:** - -1. Create a new node of your choice—baremetal, on-prem virtual machine, cloud-based virtual machine, and so on. Provision it according to our [requirements]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install/# host-requirements). - -2. Log in to your new node using a remote Terminal connection. - - -3. Create a directory that mirrors your other nodes' snapshot directories: - - ``` - root@newnode:~# mkdir -p /opt/rke/etcd-snapshots - ``` - -4. Pull your most recent snapshot onto the node. Replace `` with the name of the snapshot you're restoring to. - - ``` - root@newnode:~# s3cmd get s3://rke-etcd-snapshots/ /opt/rke/etcd-snapshots/ - ``` - - >**Remember:** Our use of Amazon S3 is an example used for this documentation. The command for pulling your snapshot may vary. - - -After restoring the cluster you have to restart the kubernetes components on all nodes, otherwise there will be some conflicts with resource versions of objects stored in `etcd`, this will include restart to kubernetes components and the network components, for more information please refer to [kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#etcd-upgrade-requirements), to do that you can run the following on each node: -``` -docker restart kube-apiserver kubelet kube-controller-manager kube-scheduler kube-proxy -docker ps | grep flannel | cut -f 1 -d " " | xargs docker restart -docker ps | grep calico | cut -f 1 -d " " | xargs docker restart -``` - -### 2. Restore `etcd` Database - -To restore the most recent `etcd` snapshot on your new node, run RKE the command `rke etcd snapshot-restore`. This command reverts to any snapshot stored in `/opt/rke/etcd-snapshots` that you explicitly define. When you run `rke etcd snapshot-restore`, RKE removes the old `etcd` container if it still exists. To restore operations, RKE creates a new `etcd` cluster using the snapshot you choose. - ->**Important:** When restoring the etcd database, you must restore each `etcd` to the _same_ snapshot, this means the exact same copy, so to restore you have to copy the snapshot from one of the nodes to the others before doing the `etcd snapshot-restore`. - ->**Warning:** Restoring an `etcd` snapshot deletes your current `etcd` cluster and replaces it with a new one. Before you run the `rke etcd snapshot-restore` command, backup any important data in your current cluster. - - -1. From your workstation, open `rancher-cluster.yml` in your favorite text editor. - -2. Replace the unresponsive node (`3.3.3.3` in this example) with your new one (`4.4.4.4`). You IP addresses will be different obviously: - - nodes: - - address: 1.1.1.1 - user: root - role: [controlplane,etcd,worker] - ssh_key_path: ~/.ssh/id_rsa - - address: 2.2.2.2 - user: root - role: [controlplane,etcd,worker] - ssh_key_path: ~/.ssh/id_rsa - # - address: 3.3.3.3 # UNRESPONSIVE NODE - # user: root - # role: [controlplane,etcd,worker] - # ssh_key_path: ~/.ssh/id_rsa - - address: 4.4.4.4 # NEW NODE - user: root - role: [controlplane,etcd,worker] - ssh_key_path: ~/.ssh/id_rsa - -3. Save and close `rancher-cluster.yml`. - -4. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. - -5. Run one of the following commands to restore the `etcd` database: - - ``` - # MacOS - ./rke_darwin-amd64 etcd snapshot-restore --name --config rancher-cluster.yml - # Linux - ./rke_linux-amd64 etcd snapshot-restore --name --config rancher-cluster.yml - ``` - - -6. Run one of the following commands to bring your cluster back up: - - ``` - # MacOS - ./rke_darwin-amd64 up --config rancher-cluster.yml - # Linux - ./rke_linux-amd64 up --config rancher-cluster.yml - ``` - -7. Lastly, restart the Kubernetes components on all cluster nodes to prevent potential `etcd` conflicts. Run this command on each of your nodes. - - ``` - docker restart kube-apiserver kubelet kube-controller-manager kube-scheduler kube-proxy - docker ps | grep flannel | cut -f 1 -d " " | xargs docker restart - docker ps | grep calico | cut -f 1 -d " " | xargs docker restart - ``` diff --git a/content/rancher/v2.x/en/installation/backups-and-restoration/single-node-backup-and-restoration/_index.md b/content/rancher/v2.x/en/installation/backups-and-restoration/single-node-backup-and-restoration/_index.md deleted file mode 100644 index 2c7a1191594..00000000000 --- a/content/rancher/v2.x/en/installation/backups-and-restoration/single-node-backup-and-restoration/_index.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Single Node Backup and Restoration -weight: 365 -aliases: - - /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/ ---- - -After completing your single node installation of Rancher, or before your upgrade to a newer version of Rancher, create a backup of your current installation. Use this backup as a restoration point for your Rancher install if you encounter issues when upgrading. - -## Backing Up Your Rancher Server - ->**Prerequisite:** Open Rancher and write down the version number displayed in the lower-left of the browser (example: `v2.0.0`). You'll need this number during the backup process. - -1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. - - ``` -docker stop - ``` - - >**Tip:** You can obtain the ID for your Rancher container by entering the following command: `docker ps`. - -2. Create a backup container. This container backs up the data from your current Rancher Server, which you can use as a recovery point. - - - Replace `` with the same ID from the previous step. - - Replace `` and `` with the version of Rancher that you are currently running, as mentioned in the **Prerequisite** above. - - ``` -docker create --volumes-from \ ---name rancher-backup- rancher/rancher: - ``` - -3. Restart Rancher Server. Replace `` with the ID of your Rancher container. - - ``` -docker start - ``` - - -## Restoring Your Rancher Server - -1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. - - ``` -docker stop - ``` - -2. Launch a new Rancher Server container using the most recent `rancher-backup-` container. - - ``` -docker run -d --volumes-from rancher-backup- --restart=unless-stopped \ --p 80:80 -p 443:443 rancher/rancher: - ``` diff --git a/content/rancher/v2.x/en/upgrades/_index.md b/content/rancher/v2.x/en/upgrades/_index.md index bd5207c1578..e7b449b88bd 100644 --- a/content/rancher/v2.x/en/upgrades/_index.md +++ b/content/rancher/v2.x/en/upgrades/_index.md @@ -3,22 +3,20 @@ title: Upgrades weight: 1000 --- -This section includes instructions for upgrading Rancher from previous versions. +This section is devoted to upgrading your Rancher Server, as well as other procedures related to upgrades and system maintenance. -### Upgrading from Rancher 2.x.x +- [Upgrade Scenarios](./upgrade-scenarios/) -Each new version of Rancher 2.x.x supports upgrades from previous versions of Rancher 2.x.x. This section will be updated as soon as the first release post 2.0 is available. + When a new version of Rancher is released, follow these instructions to upgrade to the latest version. -Complete one of the upgrade procedures below based on your Rancher installation: +- [Backups](./backups/) -- [Single Node Upgrade](./single-node-upgrade/) -- [High Availability Upgrade](./ha-server-upgrade/) -- [Air Gap Upgrade](./air-gap-upgrade/) + To protect yourself from a disaster scenario, you should create Rancher backups on a regular basis. -### Migrating from Rancher 1.6.x +- [Restorations](./restorations/) -Until Rancher 2.1 is released, migrating to from Rancher 1.6.x to 2.x.x is not supported due to major code rewrites. + In a disaster scenario, you can restore your `etcd` database by restoring a backup. -For the 2.1 release, we plan to release a tool that converts Rancher Compose to Kubernetes YAML. This tool will help our Cattle users migrate from Rancher 1.6.x to 2.x.x. However, we understand that there is a learning curve switching from Cattle to Kubernetes as you deploy new workloads. Therefore, this release will include a cheatsheet for those that enjoy Cattle's simplicity but want to quickly create those workloads in Kubernetes. +- [Rollbacks](./rollbacks/) -We will continue support for Rancher 1.6.x for a minimum of one year after the 2.1 release so that 1.6.x users can plan and complete migration. + If an upgrade does not complete successfully, you can revert you Rancher Server to its previous version along with its data prior to upgrade. diff --git a/content/rancher/v2.x/en/upgrades/backups/_index.md b/content/rancher/v2.x/en/upgrades/backups/_index.md new file mode 100644 index 00000000000..1fba28d6cea --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/backups/_index.md @@ -0,0 +1,10 @@ +--- +title: Backups +weight: 50 +aliases: + - /rancher/v2.x/en/installation/after-installation/ +--- +This section contains information about how to create backups of your Rancher data and how to restore them in a disaster scenario. + +- [Single Node Install Backups](./single-node-backups/) +- [High Availability Install Backups](./ha-backups/) diff --git a/content/rancher/v2.x/en/upgrades/backups/ha-backups/_index.md b/content/rancher/v2.x/en/upgrades/backups/ha-backups/_index.md new file mode 100644 index 00000000000..cc3915dd088 --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/backups/ha-backups/_index.md @@ -0,0 +1,125 @@ +--- +title: Creating Backups—High Availability Installs +weight: 50 +aliases: + - /rancher/v2.x/en/installation/after-installation/ha-backup-and-restoration/ +--- +This section describes how to create backups of your high-availability Rancher install. + +## Backup Outline + +Backing up your high-availability Rancher cluster is process that involves completing multiple tasks. + +1. [Meet Backup Prerequisites](#1-meet-backup-prerequisites) + + Before starting, make sure you have the files needed to create backups. + +1. [Take Snapshots of the `etcd` Database](#2-take-snapshots-of-the-etcd-database) + + Take snapshots of your current `etcd` database using Rancher Kubernetes Engine (RKE). + +1. [Store Snapshot(s) Externally](#3-backup-snapshots-to-a-safe-location) + + After taking your snapshots, export them to a safe location that won't be affected if your cluster encounters issues. + +
+### 1. Meet Backup Prerequisites + +Begin by gathering the files that you need to create backups of your Rancher install. + +#### Prerequisites + +- Rancher Kubernetes Engine v0.1.7 or later + + The commands for taking `etcd` snapshots are only available in RKE v0.1.7 and later. + +- rancher-cluster.yml + + You'll need the RKE config file you used for Rancher install, `rancher-cluster.yml`. You created this file during your chosen high-availability installation scenario: +
+
+ - [High Availability Installation]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install) + - [High Availability Installation with External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install-external-lb) + +
+### 2. Take Snapshots of the `etcd` Database + +Take snapshots of your `etcd` database. You can use these snapshots later to recover from a disaster scenario. There are two ways to take snapshots: recurringly, or as a one-off. Each option is better suited to a specific use case. Read the short description below each link to know when to use each option. + +- [Option A: Recurring Snapshots](#option-a-recurring-snapshots) + + After you stand up a high-availability Rancher install, we recommend configuring RKE to automatically take recurring snapshots so that you always have a safe restoration point available. + +- [Option B: One-Time Snapshots](#option-b-one-time-snapshots) + + We advise taking one-time snapshots before events like upgrades or restoration of another snapshot. + +#### Option A: Recurring Snapshots + +For all high-availability Rancher installs, we recommend taking recurring snapshots so that you always have a safe restoration point available. + +To take recurring snapshots, enable the `etcd-snapshot` service, which is a service that's included with RKE. This service runs in a service container alongside the `etcd` container. You can enable this service by adding some code to `rancher-cluster.yml`. + +**To Enable Recurring Snapshots:** + +1. Open `rancher-cluster.yml` with your favorite text editor. + +2. Add the following code block to the bottom of the file: + + ``` + services: + etcd: + snapshot: true # enables recurring etcd snapshots + creation: 5m0s # time increment between snapshots + retention: 24h # time increment before snapshot purge + ``` + +3. Edit the code according to your requirements. + +4. Save and close `rancher-cluster.yml`. + +5. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. + +6. Run one of the following commands: + + ``` + # MacOS + ./rke_darwin-amd64 up --config rancher-cluster.yml + # Linux + ./rke_linux-amd64 up --config rancher-cluster.yml + ``` + + +**Result:** RKE is configured to take recurring snapshots of `etcd` on all nodes running the `etcd` role. Snapshots are saved to the following directory: `/opt/rke/etcd-snapshots/`. + +#### Option B: One-Time Snapshots + +When you're about to upgrade Rancher or restore it to a previous snapshot, you should snapshot your live image so that you have a backup of `etcd` in its last known state. + +**To Take a One-Time Snapshot:** + +1. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. + +2. Enter the following command. Replace `` with any name that you want to use for the snapshot (e.g. `upgrade.db`). + + ``` + # MacOS + ./rke_darwin-amd64 etcd snapshot-save --name --config rancher-cluster.yml + # Linux + ./rke_linux-amd64 etcd snapshot-save --name --config rancher-cluster.yml + ``` + +**Result:** RKE takes a snapshot of `etcd` running on each `etcd` node. The file is saved to `/opt/rke/etcd-snapshots`. + +### 3. Backup Snapshots to a Safe Location + +After taking the `etcd` snapshots, save them to a safe location so that they're unaffected if your cluster experiences a disaster scenario. This location should be persistent. + +In this documentation, as an example, we're using Amazon S3 as our safe location, and [S3cmd](http://s3tools.org/s3cmd) as our tool to create the backups. The backup location and tool that you use are ultimately your decision. + +**Example:** + +``` +root@node:~# s3cmd mb s3://rke-etcd-snapshots +root@node:~# s3cmd /opt/rke/etcd-snapshots/snapshot.db s3://rke-etcd-snapshots/ +``` diff --git a/content/rancher/v2.x/en/upgrades/backups/single-node-backups/_index.md b/content/rancher/v2.x/en/upgrades/backups/single-node-backups/_index.md new file mode 100644 index 00000000000..c07f1cfde7f --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/backups/single-node-backups/_index.md @@ -0,0 +1,36 @@ +--- +title: Creating Backups—Single Node Installs +weight: 25 +aliases: + - /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/ +--- + +After completing your single node installation of Rancher create a backup of your current installation. Use this backup as a restoration point in disaster scenarios. + +>**Prerequisite:** Open Rancher and write down the version number displayed in the lower-left of the browser (example: `v2.0.0`). You'll need this number during the backup process. + +1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. + + ``` +docker stop + ``` + + >**Tip:** You can obtain the ID for your Rancher container by entering the following command: `docker ps`. + +2. Create a backup container. This container backs up the data from your current Rancher Server, which you can use as a recovery point. + + - Replace `` with the same ID from the previous step. + - Replace `` and `` with the version of Rancher that you are currently running, as mentioned in the **Prerequisite** above. + + ``` +docker create --volumes-from \ +--name rancher-backup- rancher/rancher: + ``` + +3. Restart Rancher Server. Replace `` with the ID of your Rancher container. + + ``` +docker start + ``` + +**Result:** A backup of your Rancher Server is created. If you ever need to restore your backup, see [Restoring Backups: Single Node Installs]({{< baseurl >}}/rancher/v2.x/en/upgrades/restorations/single-node-restoration). \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/ha-server-upgrade/_index.md b/content/rancher/v2.x/en/upgrades/ha-server-upgrade/_index.md deleted file mode 100644 index 27a4e26c24a..00000000000 --- a/content/rancher/v2.x/en/upgrades/ha-server-upgrade/_index.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -title: High Availability Upgrade -weight: 1020 ---- -To upgrade Rancher 2.x running in a high availability configuration, run an upgrade command that points to your upgrade config file. - ->**Prerequisites:** -> ->- Take a [one-time etcd snapshot]({{< baseurl >}}/rancher/v2.x/en/installation/backups-and-restoration/ha-backup-and-restoration). This snapshot is a restore point in case something goes wrong during upgrade. ->- Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your workstation. ->- Confirm that the following path exists on your workstation: `~/.kube/`. If it doesn't, create it yourself. ->- Copy `kube_config_rancher-cluster.yml`, which is automatically generated after [Rancher Server installation]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install#part-11-backup-kube-config-rancher-cluster-yml), to the `~/.kube/` directory. - -1. From your workstation, open **Terminal**. - -2. Enter the following command: - - ``` -kubectl --kubeconfig=kube_config-rancher-cluster.yml set image deployment/cattle cattle-server=rancher/rancher: -n cattle-system - ``` - Replace `` with the version that you want to upgrade to. For a list of tags available, see [DockerHub](https://hub.docker.com/r/rancher/rancher/tags/). However, don't use any tag suffixed with `-rc`, as they are used for testing and are not officially supported. - - **Step Result:** The upgrade begins. Rancher Server may be unavailable for a few minutes. - -3. Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window. - -**Result:** Your Rancher Servers are upgraded. diff --git a/content/rancher/v2.x/en/upgrades/restorations/_index.md b/content/rancher/v2.x/en/upgrades/restorations/_index.md new file mode 100644 index 00000000000..88982fcc4bf --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/restorations/_index.md @@ -0,0 +1,8 @@ +--- +title: Restorations +weight: 1010 +--- +If you lose the data on your Rancher Server, you can restore it if you have backups stored in a safe location. + +- [Restoring Backups—Single Node Installs](./single-node-restoration/) +- [Restoring Backups—High Availablity Installs](./ha-restoration/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/restorations/ha-restoration/_index.md b/content/rancher/v2.x/en/upgrades/restorations/ha-restoration/_index.md new file mode 100644 index 00000000000..e5877b14a1c --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/restorations/ha-restoration/_index.md @@ -0,0 +1,115 @@ +--- +title: Restoring Backups—High Availablity Installs +weight: 370 +aliases: + - /rancher/v2.x/en/installation/after-installation/ha-backup-and-restoration/ +--- +This procedure describes how to restore your a snapshot of `etcd` if you lose your Rancher data in a disaster scenario. + +## Restoration Outline + +Following a disaster scenario, restoration of your HA Rancher installation requires you to pull your snapshot from your chosen external location and then restore it. + +1. [Create New Node and Pull Snapshot](#1-create-new-node-and-pull-snapshot) + + If one of your `etcd` nodes goes down, create a new node, and then pull the most recent `etcd` snapshot to that node. + +2. [Restore etcd Database](#2-restore-etcd-database) + + After you pull the snapshot, run the RKE command to restore the `etcd` database. + +
+### 1. Create New Node and Pull Snapshot + +If one of your `etcd` nodes go down, you need to replace it with a new node, and then pull the most recent working `etcd` snapshot to that node. + +**To Create a New Node and Pull the Latest Snapshot:** + +1. Create a new node of your choice—baremetal, on-prem virtual machine, cloud-based virtual machine, and so on. Provision it according to our [requirements]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install/#host-requirements). + +2. Log in to your new node using a remote Terminal connection. + + +3. Create a directory that mirrors your other nodes' snapshot directories: + + ``` + root@newnode:~# mkdir -p /opt/rke/etcd-snapshots + ``` + +4. Pull your most recent snapshot onto the node. Replace `` with the name of the snapshot you're restoring to. + + ``` + root@newnode:~# s3cmd get s3://rke-etcd-snapshots/ /opt/rke/etcd-snapshots/ + ``` + + >**Remember:** Our use of Amazon S3 is an example used for this documentation. The command for pulling your snapshot may vary. + + +After restoring the cluster you have to restart the kubernetes components on all nodes, otherwise there will be some conflicts with resource versions of objects stored in `etcd`, this will include restart to kubernetes components and the network components, for more information please refer to [kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#etcd-upgrade-requirements), to do that you can run the following on each node: +``` +docker restart kube-apiserver kubelet kube-controller-manager kube-scheduler kube-proxy +docker ps | grep flannel | cut -f 1 -d " " | xargs docker restart +docker ps | grep calico | cut -f 1 -d " " | xargs docker restart +``` + +### 2. Restore `etcd` Database + +To restore the most recent `etcd` snapshot on your new node, run RKE the command `rke etcd snapshot-restore`. This command reverts to any snapshot stored in `/opt/rke/etcd-snapshots` that you explicitly define. When you run `rke etcd snapshot-restore`, RKE removes the old `etcd` container if it still exists. To restore operations, RKE creates a new `etcd` cluster using the snapshot you choose. + +>**Important:** When restoring the etcd database, you must restore each `etcd` to the _same_ snapshot, this means the exact same copy, so to restore you have to copy the snapshot from one of the nodes to the others before doing the `etcd snapshot-restore`. + +>**Warning:** Restoring an `etcd` snapshot deletes your current `etcd` cluster and replaces it with a new one. Before you run the `rke etcd snapshot-restore` command, backup any important data in your current cluster. + + +1. From your workstation, open `rancher-cluster.yml` in your favorite text editor. + +2. Replace the unresponsive node (`3.3.3.3` in this example) with your new one (`4.4.4.4`). You IP addresses will be different obviously: + + nodes: + - address: 1.1.1.1 + user: root + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + - address: 2.2.2.2 + user: root + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + # - address: 3.3.3.3 # UNRESPONSIVE NODE + # user: root + # role: [controlplane,etcd,worker] + # ssh_key_path: ~/.ssh/id_rsa + - address: 4.4.4.4 # NEW NODE + user: root + role: [controlplane,etcd,worker] + ssh_key_path: ~/.ssh/id_rsa + +3. Save and close `rancher-cluster.yml`. + +4. Open **Terminal** and change directory to the location of the RKE binary. Your `rancher-cluster.yml` file must reside in the same directory. + +5. Run one of the following commands to restore the `etcd` database: + + ``` + # MacOS + ./rke_darwin-amd64 etcd snapshot-restore --name --config rancher-cluster.yml + # Linux + ./rke_linux-amd64 etcd snapshot-restore --name --config rancher-cluster.yml + ``` + + +6. Run one of the following commands to bring your cluster back up: + + ``` + # MacOS + ./rke_darwin-amd64 up --config rancher-cluster.yml + # Linux + ./rke_linux-amd64 up --config rancher-cluster.yml + ``` + +7. Lastly, restart the Kubernetes components on all cluster nodes to prevent potential `etcd` conflicts. Run this command on each of your nodes. + + ``` + docker restart kube-apiserver kubelet kube-controller-manager kube-scheduler kube-proxy + docker ps | grep flannel | cut -f 1 -d " " | xargs docker restart + docker ps | grep calico | cut -f 1 -d " " | xargs docker restart + ``` diff --git a/content/rancher/v2.x/en/upgrades/restorations/single-node-restoration/_index.md b/content/rancher/v2.x/en/upgrades/restorations/single-node-restoration/_index.md new file mode 100644 index 00000000000..6cc8c8359ca --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/restorations/single-node-restoration/_index.md @@ -0,0 +1,24 @@ +--- +title: Restoring Backups—Single Node Installs +shortTitle: Singe Node Installs +weight: 365 +aliases: + - /rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/ +--- + +Backup to a restoration point for your Rancher install if you encounter issues when upgrading. + +1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. + + ``` +docker stop + ``` + +1. Launch a new Rancher Server container using the most recent `rancher-backup-` container that you backed up. + + For more information on obtaining this container name, see [Creating Backups—Single Node Installs](/Users/markbishop/Documents/GitHub/docs/content/rancher/v2.x/en/upgrades/backups/single-node-backups/#backup). + + ``` +docker run -d --volumes-from rancher-backup- --restart=unless-stopped \ +-p 80:80 -p 443:443 rancher/rancher: + ``` diff --git a/content/rancher/v2.x/en/upgrades/rollbacks/_index.md b/content/rancher/v2.x/en/upgrades/rollbacks/_index.md new file mode 100644 index 00000000000..5c5fae9b3a8 --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/rollbacks/_index.md @@ -0,0 +1,10 @@ +--- +title: Rollbacks +shortTitle: Rollbacks +weight: 1010 +--- + +If you upgrade your Rancher Server and the upgrade is unsuccessful, you can use the backup you created during the upgrade process to restore the previous version of Rancher, along with its data. + +- [Rolling Back—Single Node Installs](./single-node-rollbacks/) +- [Rolling Back—High Availablity Installs](./ha-server-rollbacks/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/rollbacks/ha-server-rollbacks/_index.md b/content/rancher/v2.x/en/upgrades/rollbacks/ha-server-rollbacks/_index.md new file mode 100644 index 00000000000..c92a6d94109 --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/rollbacks/ha-server-rollbacks/_index.md @@ -0,0 +1,49 @@ +--- +title: Rolling Back—High Availability Installs +weight: 50 +--- + +If you upgrade Rancher, but the upgrade does not complete successfully, you may need to roll back your Rancher Server to its last healthy state before upgrade. To restore Rancher: + +- Restore the `etcd` snapshot taken before upgrade. +- Run the command to revert to your prior version of Rancher. + +>**Warning!** Rolling back to a previous version of Rancher destroys any changes made to Rancher following the upgrade. Unrecoverable data loss may occur. + +1. Open **Terminal** and change directory to the location of the RKE binary that you used during upgrade. + +1. Run one of the following commands to restore the `etcd` snapshot [that you took]({{< baseurl >}}/rancher/v2.x/en/upgrades/upgrade-scenarios/ha-server-upgrade/#snapshot) before your unsuccessful upgrade. Replace the `` placeholder with your snapshot. + + ``` + # MacOS + ./rke_darwin-amd64 etcd snapshot-restore --name --config kube_config_rancher-cluster.yml + # Linux + ./rke_linux-amd64 etcd snapshot-restore --name --config kube_config_rancher-cluster.yml + ``` + +1. Run one of the following commands to bring your cluster back up: + + ``` + # MacOS + ./rke_darwin-amd64 up --config kube_config_rancher-cluster.yml + # Linux + ./rke_linux-amd64 up --config kube_config_rancher-cluster.yml + ``` + +1. Lastly, restart the Kubernetes components on all cluster nodes to prevent potential `etcd` conflicts. Run this command on each of your nodes. + + ``` + docker restart kube-apiserver kubelet kube-controller-manager kube-scheduler kube-proxy + docker ps | grep flannel | cut -f 1 -d " " | xargs docker restart + docker ps | grep calico | cut -f 1 -d " " | xargs docker restart + ``` +1. Enter the following command: + + ``` +kubectl --kubeconfig=kube_config-rancher-cluster.yml set image deployment/cattle cattle-server=rancher/rancher: -n cattle-system + ``` + Replace `` with the version that you were running before upgrade. For a list of tags available, see [DockerHub](https://hub.docker.com/r/rancher/rancher/tags/). However, don't use any tag suffixed with `-rc`, as they are used for testing and are not officially supported. + + **Step Result:** The rollback begins. Rancher Server may be unavailable for a few minutes. + +1. Log into Rancher. Confirm that the rollback succeeded by checking the version displayed in the bottom-left corner of the browser window. \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md b/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md new file mode 100644 index 00000000000..e81262c41c8 --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/rollbacks/single-node-rollbacks/_index.md @@ -0,0 +1,27 @@ +--- +title: Rolling Back—Single Node Installs +weight: 25 +--- + +Rolling back an unsuccessful Rancher upgrade requires you to run the upgrade command again, with a couple of modifications: + +- Run the upgrade command using the server tag for your prior Rancher install. +- Run the upgrade command while pointing toward the backup container you created while completing [Single Node Upgrade]({{< baseurl >}}/rancher/v2.x/en/upgrades/upgrade-scenarios/single-node-upgrade/). + +>**Warning!** Rolling back to a previous version of Rancher destroys any changes made to Rancher following the upgrade. Unrecoverable data loss may occur. + +1. Pull the version of Rancher that you were running prior to upgrade. + + ``` +docker pull rancher/rancher: + ``` + +1. Launch a Rancher Server container that you created while completing [Step 3 of Single Node Upgrade]({{< baseurl >}}/rancher/v2.x/en/upgrades/upgrade-scenarios/single-node-upgrade/#backup). + + Replace the `` placeholders with the version number of Rancher that you're rolling back to before upgrade. + + ``` +docker run -d --volumes-from rancher-data-snapshot- --restart=unless-stopped \ +-p 80:80 -p 443:443 rancher/rancher: + ``` + >**Note:** _Do not_ stop the rollback after initiating it, even if the rollback process seems longer than expected. Stopping the rollback may result in database issues during future upgrades. \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/single-node-upgrade/_index.md b/content/rancher/v2.x/en/upgrades/single-node-upgrade/_index.md deleted file mode 100644 index 5e18b63f9df..00000000000 --- a/content/rancher/v2.x/en/upgrades/single-node-upgrade/_index.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -title: Single Node Upgrade -weight: 1010 ---- -To upgrade Rancher Server 2.x to the latest version, you need to enter only a few commands. - ->**Prerequisites:** -> -> - Create a backup of your current Rancher installation. For more information, see [Backing Up Your Rancher Server]({{< baseurl >}}/rancher/v2.x/en/installation/after-installation/single-node-backup-and-restoration/#backing-up-your-rancher-server). -> - Open Rancher and write down the version number displayed in the lower-left of the browser (example: `v2.0.0`). You'll need this number during the upgrade process. - -1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. - - ``` -docker stop - ``` - - >**Tip:** You can obtain the ID for your Rancher container by entering the following command: `docker ps`. - -2. Create a `rancher-data` container. This container backs up the data from your current Rancher Server, which you'll use to start the upgraded version of Rancher Server in step 4. - - - Replace `` with the same ID from the previous step. - - Replace `` with the version of Rancher that you are currently running, as mentioned in the **Prerequisite** above. - - ``` -docker create --volumes-from \ ---name rancher-data rancher/rancher: - ``` - -1. Create a different `rancher-data-snapshot` container. This container also backs up the data from your current Rancher Server, but you are creating this container to serve as a restoration point in case something goes wrong with your server upgrade. - - - Replace `` with the same ID from the previous step. - - Replace `` with the version of Rancher that you are currently running, as mentioned in the **Prerequisite** above. - - ``` -docker create --volumes-from \ ---name rancher-data rancher/rancher: - ``` - -3. Pull the most recent image of Rancher. - - ``` -docker pull rancher/rancher:latest - ``` - - >**Attention Air Gap Users:** - > If you are visiting this page to complete [Air Gap Upgrade]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-upgrade/), prepend your private registry URL to the image when running the `docker run` command. - > - > Example: `/rancher/rancher:latest` - > - -4. Launch a new Rancher Server container using the `rancher-data` container. - - ``` -docker run -d --volumes-from rancher-data --restart=unless-stopped \ --p 80:80 -p 443:443 rancher/rancher:latest - ``` - >**Note:** _Do not_ stop the upgrade after initiating it, even if the upgrade process seems longer than expected. Stopping the upgrade may result in database migration errors during future upgrades. - >
- >
- >**Note:** After upgrading Rancher Server, data from your upgraded server is now saved to the `rancher-data` container for use in future upgrades. - -5. Remove the previous Rancher Server container. - - If you only stop the previous Rancher Server container (and don't remove it), the container may restart after the next server reboot. - -6. Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window. - -**Result:** Rancher Server is upgraded to the latest version. diff --git a/content/rancher/v2.x/en/upgrades/upgrade-scenarios/_index.md b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/_index.md new file mode 100644 index 00000000000..e36a8a0d0dc --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/_index.md @@ -0,0 +1,22 @@ +--- +title: Upgrade Scenarios +weight: 25 +--- + +### Upgrading from Rancher 2.x.x + +Each new version of Rancher 2.x.x supports upgrades from previous versions of Rancher 2.x.x. This section will be updated as soon as the first release post 2.0 is available. + +Complete one of the upgrade procedures below based on your Rancher installation: + +- [Single Node Upgrade](./single-node-upgrade/) +- [High Availability Upgrade](./ha-server-upgrade/) +- [Air Gap Upgrade](./air-gap-upgrade/) + +### Migrating from Rancher 1.6.x + +Until Rancher 2.1 is released, migrating to from Rancher 1.6.x to 2.x.x is not supported due to major code rewrites. + +For the 2.1 release, we plan to release a tool that converts Rancher Compose to Kubernetes YAML. This tool will help our Cattle users migrate from Rancher 1.6.x to 2.x.x. However, we understand that there is a learning curve switching from Cattle to Kubernetes as you deploy new workloads. Therefore, this release will include a cheatsheet for those that enjoy Cattle's simplicity but want to quickly create those workloads in Kubernetes. + +We will continue support for Rancher 1.6.x for a minimum of one year after the 2.1 release so that 1.6.x users can plan and complete migration. \ No newline at end of file diff --git a/content/rancher/v2.x/en/upgrades/air-gap-upgrade/_index.md b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/air-gap-upgrade/_index.md similarity index 100% rename from content/rancher/v2.x/en/upgrades/air-gap-upgrade/_index.md rename to content/rancher/v2.x/en/upgrades/upgrade-scenarios/air-gap-upgrade/_index.md diff --git a/content/rancher/v2.x/en/upgrades/upgrade-scenarios/ha-server-upgrade/_index.md b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/ha-server-upgrade/_index.md new file mode 100644 index 00000000000..2af8dcbd4fc --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/ha-server-upgrade/_index.md @@ -0,0 +1,41 @@ +--- +title: High Availability Upgrade +weight: 1020 +--- +To upgrade Rancher 2.x running in a high availability configuration, run an upgrade command that points to your upgrade config file. + +>**Prerequisites:** +> +>- Install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your workstation. +>- Confirm that the following path exists on your workstation: `~/.kube/`. If it doesn't, create it yourself. +>- Copy `kube_config_rancher-cluster.yml`, which is automatically generated after [Rancher Server installation]({{< baseurl >}}/rancher/v2.x/en/installation/ha-server-install#part-11-backup-kube-config-rancher-cluster-yml), to the `~/.kube/` directory. + +1. From your workstation, open **Terminal**. + +1. Change directory to the location of the RKE binary. Your `kube_config_rancher-cluster.yml` file must reside in the same directory. + +1. Enter the following command. Replace `` with any name that you want to use for the snapshot (e.g. `upgrade.db`). + + ``` + # MacOS + ./rke_darwin-amd64 etcd snapshot-save --name --config kube_config_rancher-cluster.yml + # Linux + ./rke_linux-amd64 etcd snapshot-save --name --config kube_config_rancher-cluster.yml + ``` + + **Result:** RKE takes a snapshot of `etcd` running on each `etcd` node. The file is saved to `/opt/rke/etcd-snapshots`. + +1. Enter the following command: + + ``` +kubectl --kubeconfig=kube_config_rancher-cluster.yml set image deployment/cattle cattle-server=rancher/rancher: -n cattle-system + ``` + Replace `` with the version that you want to upgrade to. For a list of tags available, see [DockerHub](https://hub.docker.com/r/rancher/rancher/tags/). However, don't use any tag suffixed with `-rc`, as they are used for testing and are not officially supported. + + **Step Result:** The upgrade begins. Rancher Server may be unavailable for a few minutes. + +1. Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window. + +**Result:** Your Rancher Servers are upgraded. + +>**Upgrade Issues?** You can restore your Rancher Server and data that was running prior to upgrade. For more information, see [Restoring Backups—High Availablity Installs]({{< baseurl >}}/rancher/v2.x/en/upgrades/rollbacks/ha-server-rollbacks). diff --git a/content/rancher/v2.x/en/upgrades/upgrade-scenarios/single-node-upgrade/_index.md b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/single-node-upgrade/_index.md new file mode 100644 index 00000000000..853dd5d4f1e --- /dev/null +++ b/content/rancher/v2.x/en/upgrades/upgrade-scenarios/single-node-upgrade/_index.md @@ -0,0 +1,70 @@ +--- +title: Single Node Upgrade +weight: 1010 +--- +To upgrade Rancher Server 2.x after a new version is released, create a backup of your server and then run the upgrade command. + + +>**Prerequisites:** Open Rancher and write down the version number displayed in the lower-left of the browser (example: `v2.0.0`). You'll need this number during the upgrade process. + +1. Stop the container currently running Rancher Server. Replace `` with the ID of your Rancher container. + + ``` +docker stop + ``` + + >**Tip:** You can obtain the ID for your Rancher container by entering the following command: `docker ps`. + +1. Create a container of your current Rancher data for use in your upgraded Rancher Server. Name the container `rancher-data`. + + - Replace `` with the same ID from the previous step. + - Replace `` with the version of Rancher that you are currently running, as mentioned in the **Prerequisite** above. + + ``` +docker create --volumes-from \ +--name rancher-data rancher/rancher: + ``` + +1. Create another container of your current Rancher data. However, this container is a backup for restoring your Rancher Server if your upgrade is unsuccessful. Name the container `rancher-data-snapshot-`. + + - Replace `` with the same ID from the previous step. + - Replace `` with the tag for the version of Rancher currently installed. + - Replace `` with the version of Rancher that you are currently running, as mentioned in the [prerequisite](#prereq). + + ``` +docker create --volumes-from \ +--name rancher-data-snapshot- rancher/rancher: + ``` + +1. Pull the most recent image of Rancher. + + ``` +docker pull rancher/rancher:latest + ``` + + >**Attention Air Gap Users:** + > If you are visiting this page to complete [Air Gap Upgrade]({{< baseurl >}}/rancher/v2.x/en/upgrades/upgrade-scenarios/air-gap-upgrade/), prepend your private registry URL to the image when running the `docker run` command. + > + > Example: `/rancher/rancher:latest` + > + +1. Launch a new Rancher Server container using the `rancher-data` container. + + ``` +docker run -d --volumes-from rancher-data --restart=unless-stopped \ +-p 80:80 -p 443:443 rancher/rancher:latest + ``` + >**Note:** _Do not_ stop the upgrade after initiating it, even if the upgrade process seems longer than expected. Stopping the upgrade may result in database migration errors during future upgrades. + >
+ >
+ >**Note:** After upgrading Rancher Server, data from your upgraded server is now saved to the `rancher-data` container for use in future upgrades. + +1. Remove the previous Rancher Server container. + + If you only stop the previous Rancher Server container (and don't remove it), the container may restart after the next server reboot. + +1. Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window. + +**Result:** Rancher Server is upgraded to the latest version. + +>**Note:** If your upgrade does not complete successfully, you can roll Rancher Server and its data back to its last healthy state. For more information, see [Restoring Backups—Single Node Installs]({{< baseurl >}}/rancher/v2.x/en/upgrades/restorations/single-node-restoration/).