diff --git a/content/rancher/v2.x/en/backups/restorations/ha-restoration/_index.md b/content/rancher/v2.x/en/backups/restorations/ha-restoration/_index.md index 94093b8a135..f51bca614a3 100644 --- a/content/rancher/v2.x/en/backups/restorations/ha-restoration/_index.md +++ b/content/rancher/v2.x/en/backups/restorations/ha-restoration/_index.md @@ -7,19 +7,19 @@ aliases: This procedure describes how to use RKE to restore a snapshot of the Rancher Kubernetes cluster. The cluster snapshot will include Kubernetes configuration and the Rancher database and state. +Additionally, the `pki.bundle.tar.gz` file usage is no longer required as v0.2.0 has changed how the [Kubernetes cluster state is stored]({{< baseurl >}}/rke/latest/en/installation/#kubernetes-cluster-state). + ## Restore Outline - - [1. Preparation](#1-preparation) -- [2. Place Snapshot and PKI Bundle](#2-place-snapshot-and-pki-bundle) +- [2. Place Snapshot](#2-place-snapshot) - [3. Configure RKE](#3-configure-rke) - [4. Restore Database](#4-restore-database) - [5. Bring Up the Cluster](#5-bring-up-the-cluster) -
### 1. Preparation @@ -29,11 +29,22 @@ Prepare by creating 3 new nodes to be the target for the restored Rancher instan We recommend that you start with fresh nodes and a clean state. Alternatively you can clear Kubernetes and Rancher configurations from the existing nodes. This will destroy the data on these nodes. See [Node Cleanup]({{< baseurl >}}/rancher/v2.x/en/faq/cleaning-cluster-nodes/) for the procedure. -> **IMPORTANT:** Before starting the restore make sure all the kubernetes services on the old cluster nodes are stopped. We recommend powering off the nodes to be sure. +> **IMPORTANT:** Before starting the restore make sure all the Kubernetes services on the old cluster nodes are stopped. We recommend powering off the nodes to be sure. -### 2. Place Snapshot and PKI Bundle +### 2. Place Snapshot +The snapshot used to restore your etcd cluster is handled differently based on your version of RKE. -Pick a one of the clean nodes. That node will be the "target node" for the initial restore. Place the snapshot and PKI certificate bundle files in the `/opt/rke/etcd-snapshots` directory on the "target node". +#### RKE v0.2.0+ + +As of RKE v0.2.0, snapshots could be saved in an S3 compatible backend. To restore your cluster from the snapshot stored in S3 compatible backend, you can skip this step and retrieve the snapshot in [Step 4: Restore Database](#4-restore-database). Otherwise, you will need to place the snapshot directly on the nodes. + +Pick one of the clean nodes. That node will be the "target node" for the initial restore. Place your snapshot in `/opt/rke/etcd-snapshots` on the target node. + +#### RKE v0.1.x + +When you take a snapshot, RKE saves a backup of the certificates, i.e. a file named `pki.bundle.tar.gz`, in the same location. The snapshot and PKI bundle file are required for the restore process, and they are expected to be in the same location. + +Pick one of the clean nodes. That node will be the "target node" for the initial restore. Place the snapshot and PKI certificate bundle files in the `/opt/rke/etcd-snapshots` directory on the target node. * Snapshot - `.db` * PKI Bundle - `pki.bundle.tar.gz` @@ -81,15 +92,54 @@ nodes: Use RKE with the new `rancher-cluster-restore.yml` configuration and restore the database to the single "target node". +RKE will create an `etcd` container with the restored database on the target node. This container will not complete the `etcd` initialization and stay in a running state until the cluster brought up in the next step. + +#### Restoring from a Local Snapshot + +When restoring etcd from a local snapshot, the snapshot is assumed to be located on the target node in the directory `/opt/rke/etcd-snapshots`. + +> **Note:** For RKE v0.1.x, the `pki.bundle.tar.gz` file is also expected to be in the same location. + ``` rke etcd snapshot-restore --name .db --config ./rancher-cluster-restore.yml ``` -> **Note:** RKE will create an `etcd` container with the restored database on the "target node". This container will not complete the `etcd` initialization and stay in a running state until the cluster brought up in the next step. +#### Restoring from a Snapshot in S3 + +_Available as of RKE v0.2.0_ + +When restoring etcd from a snapshot located in an S3 compatible backend, the command needs the S3 information in order to connect to the S3 backend and retrieve the snapshot. + +> **Note:** Ensure your `cluster.rkestate` is present before starting the restore, as this contains your certificate data for the cluster. + +``` +$ rke etcd snapshot-restore --config cluster.yml --name snapshot-name \ +--s3 --access-key S3_ACCESS_KEY --secret-key S3_SECRET_KEY \ +--bucket-name s3-bucket-name --s3-endpoint s3.amazonaws.com +``` + +#### Options for `rke etcd snapshot-restore` + +S3 specific options are only available for RKE v0.2.0+. + +| Option | Description | S3 Specific | +| --- | --- | ---| +| `--name` value | Specify snapshot name | | +| `--config` value | Specify an alternate cluster YAML file (default: "cluster.yml") [$RKE_CONFIG] | | +| `--s3` | Enabled backup to s3 |* | +| `--s3-endpoint` value | Specify s3 endpoint url (default: "s3.amazonaws.com") | * | +| `--access-key` value | Specify s3 accessKey | *| +| `--secret-key` value | Specify s3 secretKey | *| +| `--bucket-name` value | Specify s3 bucket name | *| +| `--region` value | Specify the s3 bucket location (optional) | *| +| `--ssh-agent-auth` | [Use SSH Agent Auth defined by SSH_AUTH_SOCK]({{< baseurl >}}/rke/latest/en/config-options/#ssh-agent) | | +| `--ignore-docker-version` | [Disable Docker version check]({{< baseurl >}}/rke/latest/en/config-options/#supported-docker-versions) | ### 5. Bring Up the Cluster -Use RKE and bring up the cluster on the single "target node". +Use RKE and bring up the cluster on the single "target node." + +> **Note:** For users running RKE v0.2.0+, ensure your `cluster.rkestate` is present before starting the restore, as this contains your certificate data for the cluster. ``` rke up --config ./rancher-cluster-restore.yml @@ -179,6 +229,6 @@ rke up --config ./rancher-cluster-restore.yml #### Finishing Up -Rancher should now be running and available to manage your Kubernetes clusters. Swap your Rancher DNS or Load Balancer endpoints to target the new cluster. Once this is done the agents on your managed clusters should automatically reconnect. This may take 10-15 minutes due to reconnect back off timeouts. +Rancher should now be running and available to manage your Kubernetes clusters. Swap your Rancher DNS or Load Balancer endpoints to target the new cluster. Once this is done the agents on your managed clusters should automatically reconnect. This may take 10-15 minutes due to reconnect back off timeouts. > **IMPORTANT:** Remember to save your new RKE config (`rancher-cluster-restore.yml`) and `kubectl` credentials (`kube_config_rancher-cluster-restore.yml`) files in a safe place for future maintenance.