From e4363019fd1db007dbf56da6419b7f15111fe418 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 30 Oct 2018 12:41:36 -0700 Subject: [PATCH 01/24] first draft of air gap single node reorganization --- .../high-availability/_index.md | 4 + .../single-node/_index.md | 76 +++++++++++++++++++ 2 files changed, 80 insertions(+) create mode 100644 content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md new file mode 100644 index 00000000000..6b424422380 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md @@ -0,0 +1,4 @@ +--- +title: "Air Gap: High Availability Install" +weight: +--- \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md new file mode 100644 index 00000000000..d4c479f61ab --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md @@ -0,0 +1,76 @@ +--- +title: "Air Gap: Single Node Install" +weight: +--- + +## 1. Collect Image Sources + +All the required images for a Single Node install can be found in the `rancher-images.txt` included with the release of Rancher you are installing. + +Download the `rancher-images.txt` from the [Rancher releases](https://github.com/rancher/rancher/releases) page. + +## 2. Publish Images + +Once you have the `rancher-images.txt` file populated, publish the images from the list to your private registry. + +>**Note:** This may require up to 20GB of disk space. + +1. Browse to the [Rancher releases page](https://github.com/rancher/rancher/releases) and download the following tools for saving and publishing the images. + + | Release File | Description | + | --- | --- | + | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | + | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + +1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to a system that can reach your private registry. + + Log into your registry if required. + + ```plain + docker login + ``` + + Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + +## 3. Install Rancher + +To deploy Rancher on a single node in an air gap environment, follow the instructions in the standard [Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/single-node-install/). Parts of the install where you must complete a special action for air gap are flagged with a substitute step, which is listed in the subheading below. + +### Add Private Registry URL to Run Command + +When you get to the section [Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#2-choose-an-ssl-option-and-install-rancher), regardless of which install option you choose, prepend your Rancher image tag with your private registry URL (``), as shown in the example below. + +```plain +docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + /rancher/rancher: +``` + +## 4. Configure Rancher for the Private Registry + +Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . + +1. Log into Rancher and configure the default admin password. + +1. Go into the **Settings** view. + + ![Settings]({{< baseurl >}}/img/rancher/airgap/settings.png) + +1. Look for the setting called `system-default-registry` and choose **Edit**. + + ![Edit]({{< baseurl >}}/img/rancher/airgap/edit-system-default-registry.png) + +1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. + + ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) + +>**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. \ No newline at end of file From 24c71698b080d1e75e81a70fd016d3524fec009b Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 30 Oct 2018 14:55:23 -0700 Subject: [PATCH 02/24] finishing draft of ha air gap install --- .../high-availability/_index.md | 224 +++++++++++++++++- .../single-node/_index.md | 13 + 2 files changed, 236 insertions(+), 1 deletion(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md index 6b424422380..de3fb039deb 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md @@ -1,4 +1,226 @@ --- title: "Air Gap: High Availability Install" weight: ---- \ No newline at end of file +--- + +## Outline + + + +- [1. Collect Image Sources](#1-collect-image-sources) +- [2. Publish Images](#2-publish-images) +- [3. Install Rancher](#3-install-rancher) +- [4. Configure Rancher for the Private Registry](#4-configure-rancher-for-the-private-registry) + + + +## 1. Collect Image Sources + +Collect the list of images required for Rancher. These steps will require internet access. + +The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. + +* **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. +* **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. + + ```plain + rke config --system-images >> ./rancher-images.txt + ``` +* **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. + + Fetch the latest `cert-manager` Helm chart and parse the template for image details. + + ```plain + helm fetch stable/cert-manager + helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt + ``` + +Sort and unique the images list to remove any overlap between the sources. + +```plain +sort -u rancher-images.txt -o rancher-images.txt +``` + +## 2. Publish Images + +Once you have the `rancher-images.txt` file populated, publish the images from the list to your private registry. + +>**Note:** This may require up to 20GB of disk space. + +1. Browse to the [Rancher releases page](https://github.com/rancher/rancher/releases) and download the following tools for saving and publishing the images. + + | Release File | Description | + | --- | --- | + | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | + | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + +1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to a system that can reach your private registry. + + Log into your registry if required. + + ```plain + docker login + ``` + + Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + + +## 3. Install Rancher + +This guide will take you through the basic process of installing Rancher Server HA in a Air Gap environment. Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. + +## RKE + +On a system that has access (22/tcp and 6443/tcp) to the nodes you have built to host the Rancher server cluster, use the sample below create the `rancher-cluster.yml` file. Define your nodes and fill out the details for the private registry. + +See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. + +Replace values in the code sample according to the table below. + +| Directive Replacement | Description | +| ----------------------- | --------------------------------------------------------------------- | +| `address` | The IP address for each of your air gap nodes outside of the cluster. | +| `internal_address` | The IP address for each of your air gap nodes within the cluster. | +| `url` | The URL for your private registry. | + +```yaml +nodes: +- address: 18.222.121.187 # air gap node external IP + internal_address: 172.31.7.22 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa +- address: 18.220.193.254 # air gap node external IP + internal_address: 172.31.13.132 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa +- address: 13.59.83.89 # air gap node external IP + internal_address: 172.31.3.216 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa + +private_registries: +- url: # private registry url + user: rancher + password: "*********" + is_default: true +``` + +### Run RKE + +```plain +rke up --config ./rancher-cluster.yml +``` + +### Testing the Cluster + +Follow the rest of the [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) guide to test your cluster and verify the health of your pods before continuing. + +## Helm + +Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. + +### Initialize Helm Locally + +Skip the [Initialize Helm (Install Tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) and initialize `helm` locally on a system that has internet access. + +```plain +helm init -c +``` + +## Installing Rancher + +If you set up a default private registry with credentials in RKE, the Kubernetes `kubelet` will have the credentials for your private registry configured. + +### Render Templates + +Fetch and render the `helm` charts on a system that has internet access. + +#### Cert-Manager + +If you are installing Rancher with Rancher self-signed certificates you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. + +Fetch the latest `cert-manager` chart from the [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). + +```plain +helm fetch stable/cert-manager +``` + +Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. + +```plain +helm template ./cert-manager-.tgz --output-dir . \ +--name cert-manager --namespace kube-system \ +--set image.repository=/quay.io/jetstack/cert-manager-controller +``` + +#### Rancher + +Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). + +```plain +helm repo add rancher- https://releases.rancher.com/server-charts/ +``` + +Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). + +```plain +helm fetch rancher-/rancher +``` + +Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. + +```plain +helm template ./rancher-.tgz --output-dir . \ +--name rancher --namespace cattle-system \ +--set hostname= \ +--set rancherImage=/rancher/rancher +``` + +### Copy Manifests + +Copy the rendered manifest directories to a system that has access to the Rancher server cluster. + +### Apply the Manifests + +Use `kubectl` to create namespaces and apply the rendered manifests. + +```plain +kubectl -n kube-system apply -R -f ./cert-manager + +kubectl create namespace cattle-system +kubectl -n cattle-system apply -R -f ./rancher +``` + +Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. + +## 4. Configure Rancher for the Private Registry + +Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . + +1. Log into Rancher and configure the default admin password. + +1. Go into the **Settings** view. + + ![Settings]({{< baseurl >}}/img/rancher/airgap/settings.png) + +1. Look for the setting called `system-default-registry` and choose **Edit**. + + ![Edit]({{< baseurl >}}/img/rancher/airgap/edit-system-default-registry.png) + +1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. + + ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) + +>**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md index d4c479f61ab..14cc59d9342 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md @@ -3,8 +3,21 @@ title: "Air Gap: Single Node Install" weight: --- +## Outline + + + +- [1. Collect Image Sources](#1-collect-image-sources) +- [2. Publish Images](#2-publish-images) +- [3. Install Rancher](#3-install-rancher) +- [4. Configure Rancher for the Private Registry](#4-configure-rancher-for-the-private-registry) + + + ## 1. Collect Image Sources +Collect the list of images required for Rancher. These steps will require internet access. + All the required images for a Single Node install can be found in the `rancher-images.txt` included with the release of Rancher you are installing. Download the `rancher-images.txt` from the [Rancher releases](https://github.com/rancher/rancher/releases) page. From 70b9a3bc8291ab45741581d550135bd652b597df Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 30 Oct 2018 15:31:15 -0700 Subject: [PATCH 03/24] merged instructions into single-node air gap setup doc --- .../high-availability/_index.md | 6 +- .../single-node/_index.md | 115 ++++++++++++++++-- 2 files changed, 110 insertions(+), 11 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md index de3fb039deb..feb83631f82 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md @@ -20,13 +20,13 @@ Collect the list of images required for Rancher. These steps will require intern The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. -* **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. -* **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. +- **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. +- **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. ```plain rke config --system-images >> ./rancher-images.txt ``` -* **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. +- **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. Fetch the latest `cert-manager` Helm chart and parse the template for image details. diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md index 14cc59d9342..8705b1c3f0b 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md @@ -53,20 +53,119 @@ Once you have the `rancher-images.txt` file populated, publish the images from t ```plain ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` -## 3. Install Rancher +## 3. Provision Linux Host + +Provision a single Linux host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements) to launch your {{< product >}} Server. + +## 4. Choose an SSL Option and Install Rancher To deploy Rancher on a single node in an air gap environment, follow the instructions in the standard [Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/single-node-install/). Parts of the install where you must complete a special action for air gap are flagged with a substitute step, which is listed in the subheading below. -### Add Private Registry URL to Run Command +For development and testing environments, we recommend installing Rancher by running a single Docker container. In this installation scenario, you'll install Docker on a single Linux host, and then deploy Rancher on your host using a single Docker container. -When you get to the section [Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#2-choose-an-ssl-option-and-install-rancher), regardless of which install option you choose, prepend your Rancher image tag with your private registry URL (``), as shown in the example below. +For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster. -```plain -docker run -d --restart=unless-stopped \ - -p 80:80 -p 443:443 \ - /rancher/rancher: -``` +>**Do you want to...** +> +>- Use a proxy? See [HTTP Proxy Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/proxy/) +>- Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{< baseurl >}}/rancher/v2.x/en/admin-settings/custom-ca-root-certificate/) +>- Complete an Air Gap Installation? See [Air Gap](#air-gap) +>- Record all transactions with the Rancher API? See [API Auditing](#api-auditing) +> + +Choose from the following options. Regardless of which install option you choose, prepend your Rancher image tag with your private registry URL (``). + + +{{% accordion id="option-a" label="Option A-Default Self-Signed Certificate" %}} + +If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself. + +Log into your Linux host, and then run the minimum installation command below. + +>**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. + + docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + /rancher/rancher: + + +{{% /accordion %}} +{{% accordion id="option-b" label="Option B-Bring Your Own Certificate: Self-Signed" %}} +In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher. + +>**Prerequisites:** +>Create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice. +> +>- The certificate files must be in [PEM format](#pem). +>- In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting](#cert-order). + +After creating your certificate, run the Docker command below to install Rancher. Use the `-v` flag and provide the path to your certificates to mount them in your container. + +- Replace `` with the directory path to your certificate file. +- Replace ``,``, and `` with your certificate names. + +>**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. + + + docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + -v //:/etc/rancher/ssl/cert.pem \ + -v //:/etc/rancher/ssl/key.pem \ + -v //:/etc/rancher/ssl/cacerts.pem \ + /rancher/rancher: + + +{{% /accordion %}} +{{% accordion id="option-c" label="Option C-Bring Your Own Certificate: Signed by Recognized CA" %}} + +In production environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings. + +>**Prerequisite:** The certificate files must be in [PEM format](#pem). + +After obtaining your certificate, run the Docker command below. + +- Use the `-v` flag and provide the path to your certificates to mount them in your container. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary. + + - Replace `` with the directory path to your certificate file. + - Replace `` and `` with your certificate names. + +- Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. + +>**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. + + + docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + -v //:/etc/rancher/ssl/cert.pem \ + -v //:/etc/rancher/ssl/key.pem \ + /rancher/rancher: --no-cacerts + +{{% /accordion %}} +{{% accordion id="option-d" label="Option D-Let's Encrypt Certificate" %}} + +For production environments, you also have the options of using [Let's Encrypt](https://letsencrypt.org/) certificates. Let's Encrypt uses an http-01 challenge to verify that you have control over your domain. You can confirm that you control the domain by pointing the hostname that you want to use for Rancher access (for example, `rancher.mydomain.com`) to the IP of the machine it is running on. You can bind the hostname to the IP address by creating an A record in DNS. + +>**Prerequisites:** +> +>- Let's Encrypt is an Internet service. Therefore, this option cannot be used in an internal/air gapped network. +>- Create a record in your DNS that binds your Linux host IP address to the hostname that you want to use for Rancher access (`rancher.mydomain.com` for example). +>- Open port `TCP/80` on your Linux host. The Let's Encrypt http-01 challenge can come from any source IP address, so port `TCP/80` must be open to all IP addresses. + + +After you fulfill the prerequisites, you can install Rancher using a Let's Encrypt certificate by running the following command. Replace `` with your your domain. + +>**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. + + + docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + /rancher/rancher: + --acme-domain + +>**Remember:** Let's Encrypt provides rate limits for requesting new certificates. Therefore, limit how often you create or destroy the container. For more information, see [Let's Encrypt documentation on rate limits](https://letsencrypt.org/docs/rate-limits/). +{{% /accordion %}} ## 4. Configure Rancher for the Private Registry From 8dc35399dace6785891232a1613f3e27215f9729 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 30 Oct 2018 17:55:50 -0700 Subject: [PATCH 04/24] finishing merged draft of single node air gap install --- .../single-node/_index.md | 170 +++++++++--------- .../ha/rke-add-on/api-auditing/_index.md | 2 +- 2 files changed, 81 insertions(+), 91 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md index 8705b1c3f0b..417ae3daa0b 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md @@ -2,38 +2,55 @@ title: "Air Gap: Single Node Install" weight: --- - ## Outline -- [1. Collect Image Sources](#1-collect-image-sources) -- [2. Publish Images](#2-publish-images) -- [3. Install Rancher](#3-install-rancher) -- [4. Configure Rancher for the Private Registry](#4-configure-rancher-for-the-private-registry) +- [Prerequisites](#prerequisites) +- [Caveats](#caveats) +- [1. Provision Linux Host](#1-provision-linux-host) +- [2. Collect Image Sources](#2-collect-image-sources) +- [3. Publish Images](#3-publish-images) +- [4. Choose an SSL Option and Install Rancher](#4-choose-an-ssl-option-and-install-rancher) +- [5. Configure Rancher for the Private Registry](#5-configure-rancher-for-the-private-registry) -## 1. Collect Image Sources +## Prerequisites -Collect the list of images required for Rancher. These steps will require internet access. +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). -All the required images for a Single Node install can be found in the `rancher-images.txt` included with the release of Rancher you are installing. -Download the `rancher-images.txt` from the [Rancher releases](https://github.com/rancher/rancher/releases) page. +## Caveats -## 2. Publish Images +In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. -Once you have the `rancher-images.txt` file populated, publish the images from the list to your private registry. +As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. ->**Note:** This may require up to 20GB of disk space. -1. Browse to the [Rancher releases page](https://github.com/rancher/rancher/releases) and download the following tools for saving and publishing the images. +## 1. Provision Linux Host - | Release File | Description | - | --- | --- | - | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | - | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | +Provision a single, air gapped Linux host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements) to launch your {{< product >}} Server. + +This host should be disconnected from the internet, but should have connectivity with your private registry. + +## 2. Collect Image Sources + +Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: + + +| Release File | Description | +| --- | --- | +| `rancher-images.txt` | This file contains a list of all files needed to install Rancher. +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + + +## 3. Publish Images + +After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. + +>**Note:** Image publication may require up to 20GB of empty disk space. 1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. @@ -41,133 +58,106 @@ Once you have the `rancher-images.txt` file populated, publish the images from t ./rancher-save-images.sh --image-list ./rancher-images.txt ``` -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to a system that can reach your private registry. +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. - Log into your registry if required. + 1. Log into your registry if required. - ```plain - docker login - ``` + ```plain + docker login + ``` - Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry - ``` + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` -## 3. Provision Linux Host - -Provision a single Linux host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements) to launch your {{< product >}} Server. ## 4. Choose an SSL Option and Install Rancher -To deploy Rancher on a single node in an air gap environment, follow the instructions in the standard [Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/single-node-install/). Parts of the install where you must complete a special action for air gap are flagged with a substitute step, which is listed in the subheading below. - -For development and testing environments, we recommend installing Rancher by running a single Docker container. In this installation scenario, you'll install Docker on a single Linux host, and then deploy Rancher on your host using a single Docker container. +For development and testing in air gap environments, we recommend installing Rancher by running a single Docker container. In this installation scenario, you'll deploy Rancher to your air gap host using an image pulled from your private registry. For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster. >**Do you want to...** > ->- Use a proxy? See [HTTP Proxy Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/proxy/) ->- Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{< baseurl >}}/rancher/v2.x/en/admin-settings/custom-ca-root-certificate/) ->- Complete an Air Gap Installation? See [Air Gap](#air-gap) ->- Record all transactions with the Rancher API? See [API Auditing](#api-auditing) -> +>- Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{< baseurl >}}/rancher/v2.x/en/admin-settings/custom-ca-root-certificate/). +>- Record all transactions with the Rancher API? See [API Auditing]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#enable-api-audit-log). -Choose from the following options. Regardless of which install option you choose, prepend your Rancher image tag with your private registry URL (``). + +Choose from the following options: {{% accordion id="option-a" label="Option A-Default Self-Signed Certificate" %}} If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself. -Log into your Linux host, and then run the minimum installation command below. - ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. +Log into your Linux host, and then run the installation command below. Replace `` with your private registry URL and port. docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ /rancher/rancher: - {{% /accordion %}} {{% accordion id="option-b" label="Option B-Bring Your Own Certificate: Self-Signed" %}} In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher. >**Prerequisites:** ->Create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice. +>From a computer with an internet connection, create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice. > ->- The certificate files must be in [PEM format](#pem). ->- In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting](#cert-order). +>- The certificate files must be in [PEM format]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#pem). +>- In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [SSL FAQ / Troubleshooting]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#cert-order). After creating your certificate, run the Docker command below to install Rancher. Use the `-v` flag and provide the path to your certificates to mount them in your container. -- Replace `` with the directory path to your certificate file. -- Replace ``,``, and `` with your certificate names. +When entering the command, use the table below to replace each placeholder. ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. +Placeholder | Description +------------|------------- +`` | The path to the directory containing your certificate files. +`` | The path to your full certificate chain. +`` | The path to the private key for your certificate. +`` | The path to the certificate authority's private key. +`` | Your private registry URL and port. +``` +docker run -d --restart=unless-stopped \ +-p 80:80 -p 443:443 \ +-v //:/etc/rancher/ssl/cert.pem \ +-v //:/etc/rancher/ssl/key.pem \ +-v //:/etc/rancher/ssl/cacerts.pem \ +/rancher/rancher: +``` - docker run -d --restart=unless-stopped \ - -p 80:80 -p 443:443 \ - -v //:/etc/rancher/ssl/cert.pem \ - -v //:/etc/rancher/ssl/key.pem \ - -v //:/etc/rancher/ssl/cacerts.pem \ - /rancher/rancher: - {{% /accordion %}} {{% accordion id="option-c" label="Option C-Bring Your Own Certificate: Signed by Recognized CA" %}} In production environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings. ->**Prerequisite:** The certificate files must be in [PEM format](#pem). +>**Prerequisite:** The certificate files must be in [PEM format]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#pem). -After obtaining your certificate, run the Docker command below. +After obtaining your certificate, run the Docker command below, replacing each placeholder. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary. -- Use the `-v` flag and provide the path to your certificates to mount them in your container. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary. - - - Replace `` with the directory path to your certificate file. - - Replace `` and `` with your certificate names. - -- Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. - ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. +When entering the command, use the table below to replace each placeholder. +Placeholder | Description +------------|------------- +`` | The path to the directory containing your certificate files. +`` | The path to your full certificate chain. +`` | The path to the private key for your certificate. +`` | Your private registry URL and port. Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. + ``` docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ -v //:/etc/rancher/ssl/cert.pem \ -v //:/etc/rancher/ssl/key.pem \ /rancher/rancher: --no-cacerts + ``` {{% /accordion %}} -{{% accordion id="option-d" label="Option D-Let's Encrypt Certificate" %}} - -For production environments, you also have the options of using [Let's Encrypt](https://letsencrypt.org/) certificates. Let's Encrypt uses an http-01 challenge to verify that you have control over your domain. You can confirm that you control the domain by pointing the hostname that you want to use for Rancher access (for example, `rancher.mydomain.com`) to the IP of the machine it is running on. You can bind the hostname to the IP address by creating an A record in DNS. - ->**Prerequisites:** -> ->- Let's Encrypt is an Internet service. Therefore, this option cannot be used in an internal/air gapped network. ->- Create a record in your DNS that binds your Linux host IP address to the hostname that you want to use for Rancher access (`rancher.mydomain.com` for example). ->- Open port `TCP/80` on your Linux host. The Let's Encrypt http-01 challenge can come from any source IP address, so port `TCP/80` must be open to all IP addresses. - - -After you fulfill the prerequisites, you can install Rancher using a Let's Encrypt certificate by running the following command. Replace `` with your your domain. - ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. - - - docker run -d --restart=unless-stopped \ - -p 80:80 -p 443:443 \ - /rancher/rancher: - --acme-domain - ->**Remember:** Let's Encrypt provides rate limits for requesting new certificates. Therefore, limit how often you create or destroy the container. For more information, see [Let's Encrypt documentation on rate limits](https://letsencrypt.org/docs/rate-limits/). -{{% /accordion %}} - -## 4. Configure Rancher for the Private Registry +## 5. Configure Rancher for the Private Registry Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . diff --git a/content/rancher/v2.x/en/installation/ha/rke-add-on/api-auditing/_index.md b/content/rancher/v2.x/en/installation/ha/rke-add-on/api-auditing/_index.md index c8f6933f895..20dd6a9a417 100644 --- a/content/rancher/v2.x/en/installation/ha/rke-add-on/api-auditing/_index.md +++ b/content/rancher/v2.x/en/installation/ha/rke-add-on/api-auditing/_index.md @@ -5,7 +5,7 @@ aliases: - /rke/v0.1.x/en/config-options/add-ons/api-auditing/ --- -> #### **Important: RKE add-on install is only supported up to Rancher v2.0.8** +>**Important: RKE add-on install is only supported up to Rancher v2.0.8** > >Please use the Rancher helm chart to install HA Rancher. For details, see the [HA Install - Installation Outline]({{< baseurl >}}/rancher/v2.x/en/installation/ha/#installation-outline). > From 89e5214f37065bbf69d3da3972cb636fa6de8237 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Tue, 30 Oct 2018 18:59:30 -0700 Subject: [PATCH 05/24] adding WIP ha content --- .../high-availability/_index.md | 96 ++++++++++++------- .../rancher/v2.x/en/installation/ha/_index.md | 4 + 2 files changed, 66 insertions(+), 34 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md index feb83631f82..20a1002db00 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md @@ -14,45 +14,70 @@ weight: -## 1. Collect Image Sources +## Prerequisites -Collect the list of images required for Rancher. These steps will require internet access. +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). -The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. -- **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. -- **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. +## Caveats - ```plain - rke config --system-images >> ./rancher-images.txt - ``` -- **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. +In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - Fetch the latest `cert-manager` Helm chart and parse the template for image details. +As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - ```plain - helm fetch stable/cert-manager - helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt - ``` -Sort and unique the images list to remove any overlap between the sources. +## 1. Provision Three Linux Hosts and Load Balancer -```plain -sort -u rancher-images.txt -o rancher-images.txt -``` +Provision three air gapped Linux hosts according to our requirements below to launch Rancher in an HA configuration. -## 2. Publish Images +These hosts should be disconnected from the internet, but should have connectivity with your private registry. -Once you have the `rancher-images.txt` file populated, publish the images from the list to your private registry. +{{% tabs %}} +{{% tab "Host Requirements" %}} +View hardware and software requirements for each of your cluster nodes in [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements). +{{% /tab %}} +{{% tab "Recommended Architecture" %}} ->**Note:** This may require up to 20GB of disk space. +- DNS for Rancher should resolve to a layer 4 load balancer +- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. +- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. +- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. -1. Browse to the [Rancher releases page](https://github.com/rancher/rancher/releases) and download the following tools for saving and publishing the images. +
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
+![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) +{{% /tab %}} +{{% tab "Required Tools" %}} +The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` - | Release File | Description | - | --- | --- | - | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | - | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | +* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. +* [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. +* [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. +{{% /tab %}} + +{{% tab "Load Balancer"%}} +RKE, the installer that provisions your air gapped cluster, will configure an Ingress controller pod on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. + +Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. +{{% /tab %}} +{{% /tabs %}} + +## 2. Collect Image Sources + +Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: + + +| Release File | Description | +| --- | --- | +| `rancher-images.txt` | This file contains a list of all files needed to install Rancher. +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + + +## 3. Publish Images + +After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. + +>**Note:** Image publication may require up to 20GB of empty disk space. 1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. @@ -60,18 +85,21 @@ Once you have the `rancher-images.txt` file populated, publish the images from t ./rancher-save-images.sh --image-list ./rancher-images.txt ``` -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to a system that can reach your private registry. +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. - Log into your registry if required. + 1. Log into your registry if required. - ```plain - docker login - ``` + ```plain + docker login + ``` + + 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` - Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry ## 3. Install Rancher diff --git a/content/rancher/v2.x/en/installation/ha/_index.md b/content/rancher/v2.x/en/installation/ha/_index.md index 827560eacc6..879f9e78b44 100644 --- a/content/rancher/v2.x/en/installation/ha/_index.md +++ b/content/rancher/v2.x/en/installation/ha/_index.md @@ -16,6 +16,10 @@ This procedure walks you through setting up a 3-node cluster with RKE and instal * The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. * The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. +<<<<<<< HEAD +======= +
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
+>>>>>>> adding WIP ha content ![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) HA Rancher install with Layer 4 load balancer (TCP), depicting SSL termination at ingress controllers From 1fe0b9d7886e154897ff7aa045b6323b6eb91da9 Mon Sep 17 00:00:00 2001 From: MBishop17 Date: Wed, 31 Oct 2018 15:03:55 -0700 Subject: [PATCH 06/24] merging parts of ha content --- .../high-availability/_index.md | 73 ++++++++++++------- 1 file changed, 46 insertions(+), 27 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md index 20a1002db00..16ffa53ea0d 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md @@ -7,9 +7,21 @@ weight: -- [1. Collect Image Sources](#1-collect-image-sources) -- [2. Publish Images](#2-publish-images) -- [3. Install Rancher](#3-install-rancher) +- [Outline](#outline) +- [Prerequisites](#prerequisites) +- [Caveats](#caveats) +- [1. Provision Three Linux Hosts and Load Balancer](#1-provision-three-linux-hosts-and-load-balancer) +- [2. Collect Image Sources](#2-collect-image-sources) +- [3. Publish Images](#3-publish-images) +- [4. Install Rancher](#4-install-rancher) + - [4A. Create an RKE Config File](#4a-create-an-rke-config-file) + - [Common RKE Nodes Options](#common-rke-nodes-options) + - [4B. Run RKE](#4b-run-rke) + - [4C. Initialize Helm](#4c-initialize-helm) + - [4D. Render Templates](#4d-render-templates) + - [4E. Install Cert-Manager](#4e-install-cert-manager) + - [4F. Install Rancher](#4f-install-rancher) + - [4G. Copy and Apply Manifests](#4g-copy-and-apply-manifests) - [4. Configure Rancher for the Private Registry](#4-configure-rancher-for-the-private-registry) @@ -102,15 +114,17 @@ After downloading the release files, publish the images from `rancher-images.txt -## 3. Install Rancher +## 4. Install Rancher This guide will take you through the basic process of installing Rancher Server HA in a Air Gap environment. Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. -## RKE -On a system that has access (22/tcp and 6443/tcp) to the nodes you have built to host the Rancher server cluster, use the sample below create the `rancher-cluster.yml` file. Define your nodes and fill out the details for the private registry. +### 4A. Create an RKE Config File -See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. + +On a system that has access (22/tcp and 6443/tcp) to the nodes you have built to host the Rancher server cluster, use the sample below create the `rancher-cluster.yml` file. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. + +>**Tip:** See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. Replace values in the code sample according to the table below. @@ -120,6 +134,8 @@ Replace values in the code sample according to the table below. | `internal_address` | The IP address for each of your air gap nodes within the cluster. | | `url` | The URL for your private registry. | +> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls. + ```yaml nodes: - address: 18.222.121.187 # air gap node external IP @@ -145,39 +161,44 @@ private_registries: is_default: true ``` -### Run RKE +#### Common RKE Nodes Options -```plain +| Option | Required | Description | +| --- | --- | --- | +| `address` | yes | The public DNS or IP address | +| `user` | yes | A user that can run docker commands | +| `role` | yes | List of Kubernetes roles assigned to the node | +| `internal_address` | no | The private DNS or IP address for internal cluster traffic | +| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | + + + +### 4B. Run RKE + +After configuring `rancher-cluster.yml`, open Terminal and change directories to the RKE binary. Then enter the command below to stand up your high availability cluster. + +``` rke up --config ./rancher-cluster.yml ``` -### Testing the Cluster +### 4C. Initialize Helm -Follow the rest of the [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) guide to test your cluster and verify the health of your pods before continuing. - -## Helm Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. -### Initialize Helm Locally - Skip the [Initialize Helm (Install Tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) and initialize `helm` locally on a system that has internet access. ```plain helm init -c -``` +``` -## Installing Rancher - -If you set up a default private registry with credentials in RKE, the Kubernetes `kubelet` will have the credentials for your private registry configured. - -### Render Templates +### 4D. Render Templates Fetch and render the `helm` charts on a system that has internet access. -#### Cert-Manager +### 4E. Install Cert-Manager -If you are installing Rancher with Rancher self-signed certificates you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. +If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. Fetch the latest `cert-manager` chart from the [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). @@ -193,7 +214,7 @@ helm template ./cert-manager-.tgz --output-dir . \ --set image.repository=/quay.io/jetstack/cert-manager-controller ``` -#### Rancher +### 4F. Install Rancher Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). @@ -216,12 +237,10 @@ helm template ./rancher-.tgz --output-dir . \ --set rancherImage=/rancher/rancher ``` -### Copy Manifests +### 4G. Copy and Apply Manifests Copy the rendered manifest directories to a system that has access to the Rancher server cluster. -### Apply the Manifests - Use `kubectl` to create namespaces and apply the rendered manifests. ```plain From efc41fd600dc264fce7b7b687ee4accfc0f55593 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Fri, 2 Nov 2018 18:42:12 -0700 Subject: [PATCH 07/24] completing draft of ha air gap --- .../air-gap-high-availability}/_index.md | 130 +++++++++--------- .../air-gap-single-node}/_index.md | 0 2 files changed, 66 insertions(+), 64 deletions(-) rename content/rancher/v2.x/en/installation/{air-gap-installation/high-availability => WIP-air-gap-installation/air-gap-high-availability}/_index.md (68%) rename content/rancher/v2.x/en/installation/{air-gap-installation/single-node => WIP-air-gap-installation/air-gap-single-node}/_index.md (100%) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md b/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md similarity index 68% rename from content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md rename to content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md index 16ffa53ea0d..ab06d55cbe0 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md @@ -11,18 +11,19 @@ weight: - [Prerequisites](#prerequisites) - [Caveats](#caveats) - [1. Provision Three Linux Hosts and Load Balancer](#1-provision-three-linux-hosts-and-load-balancer) -- [2. Collect Image Sources](#2-collect-image-sources) -- [3. Publish Images](#3-publish-images) -- [4. Install Rancher](#4-install-rancher) - - [4A. Create an RKE Config File](#4a-create-an-rke-config-file) - - [Common RKE Nodes Options](#common-rke-nodes-options) - - [4B. Run RKE](#4b-run-rke) - - [4C. Initialize Helm](#4c-initialize-helm) - - [4D. Render Templates](#4d-render-templates) - - [4E. Install Cert-Manager](#4e-install-cert-manager) - - [4F. Install Rancher](#4f-install-rancher) - - [4G. Copy and Apply Manifests](#4g-copy-and-apply-manifests) -- [4. Configure Rancher for the Private Registry](#4-configure-rancher-for-the-private-registry) + - [Host Requirements](#host-requirements) + - [Recommended Architecture](#recommended-architecture) + - [Required Tools](#required-tools) + - [Load Balancer](#load-balancer) +- [2. Collect and Publish Image Sources](#2-collect-and-publish-image-sources) +- [3. Create an RKE Config File](#3-create-an-rke-config-file) + - [Common RKE Nodes Options](#common-rke-nodes-options) +- [4. Run RKE](#4-run-rke) +- [5. Initialize Helm and Render Templates](#5-initialize-helm-and-render-templates) +- [6. Optional: Install Cert-Manager](#6-optional-install-cert-manager) +- [7. Install Rancher](#7-install-rancher) +- [8. Copy and Apply Manifests](#8-copy-and-apply-manifests) +- [9. Configure Rancher for the Private Registry](#9-configure-rancher-for-the-private-registry) @@ -44,11 +45,10 @@ Provision three air gapped Linux hosts according to our requirements below to la These hosts should be disconnected from the internet, but should have connectivity with your private registry. -{{% tabs %}} -{{% tab "Host Requirements" %}} +### Host Requirements View hardware and software requirements for each of your cluster nodes in [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements). -{{% /tab %}} -{{% tab "Recommended Architecture" %}} + +### Recommended Architecture - DNS for Rancher should resolve to a layer 4 load balancer - The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. @@ -56,50 +56,61 @@ View hardware and software requirements for each of your cluster nodes in [Requi - The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment.
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
-![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) -{{% /tab %}} -{{% tab "Required Tools" %}} + +![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) + +### Required Tools + The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. * [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. -* [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. -{{% /tab %}} +* [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. + + +### Load Balancer -{{% tab "Load Balancer"%}} RKE, the installer that provisions your air gapped cluster, will configure an Ingress controller pod on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. -{{% /tab %}} -{{% /tabs %}} -## 2. Collect Image Sources -Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: +## 2. Collect and Publish Image Sources + +Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install in your air gap environment. Download the following three files: | Release File | Description | | --- | --- | -| `rancher-images.txt` | This file contains a list of all files needed to install Rancher. -| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | -| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | +| `rancher-images.txt` | Contains a list of all files needed to install Rancher. +| `rancher-save-images.sh` | Pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | Loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | -## 3. Publish Images - After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. >**Note:** Image publication may require up to 20GB of empty disk space. -1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. +1. From Terminal, change directories to the path containing the files listed above. + +1. Make `rancher-save-images.sh` an executable. + + ``` + chmod +x rancher-save-images.sh + ``` + +1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. ```plain ./rancher-save-images.sh --image-list ./rancher-images.txt ``` -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. - 1. Log into your registry if required. +1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry.files to each of the [Linux hosts](#1-provision-three-linux-hosts-and-load-balancer) that you've provisioned. + + + 1. Log into your private registry if required. ```plain docker login @@ -112,17 +123,10 @@ After downloading the release files, publish the images from `rancher-images.txt ``` +## 3. Create an RKE Config File -## 4. Install Rancher - -This guide will take you through the basic process of installing Rancher Server HA in a Air Gap environment. Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. - - -### 4A. Create an RKE Config File - - -On a system that has access (22/tcp and 6443/tcp) to the nodes you have built to host the Rancher server cluster, use the sample below create the `rancher-cluster.yml` file. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. +From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below create `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. >**Tip:** See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. @@ -161,19 +165,19 @@ private_registries: is_default: true ``` -#### Common RKE Nodes Options +### Common RKE Nodes Options -| Option | Required | Description | -| --- | --- | --- | -| `address` | yes | The public DNS or IP address | -| `user` | yes | A user that can run docker commands | -| `role` | yes | List of Kubernetes roles assigned to the node | -| `internal_address` | no | The private DNS or IP address for internal cluster traffic | -| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | +| Option | Required | Description | +| ------------------ | -------- | -------------------------------------------------------------------------------------- | +| `address` | yes | The public DNS or IP address | +| `user` | yes | A user that can run docker commands | +| `role` | yes | List of Kubernetes roles assigned to the node | +| `internal_address` | no | The private DNS or IP address for internal cluster traffic | +| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | -### 4B. Run RKE +## 4. Run RKE After configuring `rancher-cluster.yml`, open Terminal and change directories to the RKE binary. Then enter the command below to stand up your high availability cluster. @@ -181,26 +185,24 @@ After configuring `rancher-cluster.yml`, open Terminal and change directories to rke up --config ./rancher-cluster.yml ``` -### 4C. Initialize Helm +## 5. Initialize Helm and Render Templates Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. -Skip the [Initialize Helm (Install Tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) and initialize `helm` locally on a system that has internet access. +Initialize `helm` locally on a system that has internet access. ```plain helm init -c -``` +``` -### 4D. Render Templates +Then, using the same system, fetch and render the `helm` charts. -Fetch and render the `helm` charts on a system that has internet access. - -### 4E. Install Cert-Manager +## 6. Optional: Install Cert-Manager If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. -Fetch the latest `cert-manager` chart from the [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). +From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). ```plain helm fetch stable/cert-manager @@ -214,9 +216,9 @@ helm template ./cert-manager-.tgz --output-dir . \ --set image.repository=/quay.io/jetstack/cert-manager-controller ``` -### 4F. Install Rancher +## 7. Install Rancher -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). +Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. ```plain helm repo add rancher- https://releases.rancher.com/server-charts/ @@ -237,7 +239,7 @@ helm template ./rancher-.tgz --output-dir . \ --set rancherImage=/rancher/rancher ``` -### 4G. Copy and Apply Manifests +## 8. Copy and Apply Manifests Copy the rendered manifest directories to a system that has access to the Rancher server cluster. @@ -252,7 +254,7 @@ kubectl -n cattle-system apply -R -f ./rancher Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. -## 4. Configure Rancher for the Private Registry +## 9. Configure Rancher for the Private Registry Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md b/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-single-node/_index.md similarity index 100% rename from content/rancher/v2.x/en/installation/air-gap-installation/single-node/_index.md rename to content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-single-node/_index.md From 18c0df5e34477c1e757afc9abdd23d427d399b4b Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 5 Nov 2018 16:10:41 -0700 Subject: [PATCH 08/24] outlines complete --- .../air-gap-high-availability/_index.md | 275 ------------------ .../air-gap-high-availability/_index.md | 30 ++ .../config-rancher-for-private-reg/_index.md | 24 ++ .../helm-init/_index.md | 36 +++ .../install-kube/_index.md | 68 +++++ .../install-rancher/_index.md | 43 +++ .../prepare-private-registry/_index.md | 75 +++++ .../provision-hosts/_index.md | 29 ++ .../air-gap-installation/_index.md | 30 -- .../install-rancher/_index.md | 156 ---------- .../prepare-private-reg/_index.md | 81 ------ .../air-gap-single-node/_index.md | 26 ++ .../config-rancher-for-private-reg/_index.md | 5 +- .../install-rancher}/_index.md | 96 +----- .../prepare-private-registry/_index.md | 45 +++ .../provision-host/_index.md | 11 + 16 files changed, 394 insertions(+), 636 deletions(-) delete mode 100644 content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md delete mode 100644 content/rancher/v2.x/en/installation/air-gap-installation/_index.md delete mode 100644 content/rancher/v2.x/en/installation/air-gap-installation/install-rancher/_index.md delete mode 100644 content/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-single-node/_index.md rename content/rancher/v2.x/en/installation/{air-gap-installation => air-gap-single-node}/config-rancher-for-private-reg/_index.md (92%) rename content/rancher/v2.x/en/installation/{WIP-air-gap-installation/air-gap-single-node => air-gap-single-node/install-rancher}/_index.md (53%) create mode 100644 content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md diff --git a/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md deleted file mode 100644 index ab06d55cbe0..00000000000 --- a/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-high-availability/_index.md +++ /dev/null @@ -1,275 +0,0 @@ ---- -title: "Air Gap: High Availability Install" -weight: ---- - -## Outline - - - -- [Outline](#outline) -- [Prerequisites](#prerequisites) -- [Caveats](#caveats) -- [1. Provision Three Linux Hosts and Load Balancer](#1-provision-three-linux-hosts-and-load-balancer) - - [Host Requirements](#host-requirements) - - [Recommended Architecture](#recommended-architecture) - - [Required Tools](#required-tools) - - [Load Balancer](#load-balancer) -- [2. Collect and Publish Image Sources](#2-collect-and-publish-image-sources) -- [3. Create an RKE Config File](#3-create-an-rke-config-file) - - [Common RKE Nodes Options](#common-rke-nodes-options) -- [4. Run RKE](#4-run-rke) -- [5. Initialize Helm and Render Templates](#5-initialize-helm-and-render-templates) -- [6. Optional: Install Cert-Manager](#6-optional-install-cert-manager) -- [7. Install Rancher](#7-install-rancher) -- [8. Copy and Apply Manifests](#8-copy-and-apply-manifests) -- [9. Configure Rancher for the Private Registry](#9-configure-rancher-for-the-private-registry) - - - -## Prerequisites - -Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). - - -## Caveats - -In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - -As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - - -## 1. Provision Three Linux Hosts and Load Balancer - -Provision three air gapped Linux hosts according to our requirements below to launch Rancher in an HA configuration. - -These hosts should be disconnected from the internet, but should have connectivity with your private registry. - -### Host Requirements -View hardware and software requirements for each of your cluster nodes in [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements). - -### Recommended Architecture - -- DNS for Rancher should resolve to a layer 4 load balancer -- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. -- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. -- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. - -
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
- -![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) - -### Required Tools - -The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` - -* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. -* [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. -* [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. - - -### Load Balancer - -RKE, the installer that provisions your air gapped cluster, will configure an Ingress controller pod on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. - -Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. - - -## 2. Collect and Publish Image Sources - -Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install in your air gap environment. Download the following three files: - - -| Release File | Description | -| --- | --- | -| `rancher-images.txt` | Contains a list of all files needed to install Rancher. -| `rancher-save-images.sh` | Pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | -| `rancher-load-images.sh` | Loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | - - -After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. - ->**Note:** Image publication may require up to 20GB of empty disk space. - -1. From Terminal, change directories to the path containing the files listed above. - -1. Make `rancher-save-images.sh` an executable. - - ``` - chmod +x rancher-save-images.sh - ``` - -1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. - - ```plain - ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - - **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. - -1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry.files to each of the [Linux hosts](#1-provision-three-linux-hosts-and-load-balancer) that you've provisioned. - - - 1. Log into your private registry if required. - - ```plain - docker login - ``` - - 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry - ``` - - -## 3. Create an RKE Config File - - -From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below create `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. Replace the IP Addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. - ->**Tip:** See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. - -Replace values in the code sample according to the table below. - -| Directive Replacement | Description | -| ----------------------- | --------------------------------------------------------------------- | -| `address` | The IP address for each of your air gap nodes outside of the cluster. | -| `internal_address` | The IP address for each of your air gap nodes within the cluster. | -| `url` | The URL for your private registry. | - -> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls. - -```yaml -nodes: -- address: 18.222.121.187 # air gap node external IP - internal_address: 172.31.7.22 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa -- address: 18.220.193.254 # air gap node external IP - internal_address: 172.31.13.132 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa -- address: 13.59.83.89 # air gap node external IP - internal_address: 172.31.3.216 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa - -private_registries: -- url: # private registry url - user: rancher - password: "*********" - is_default: true -``` - -### Common RKE Nodes Options - -| Option | Required | Description | -| ------------------ | -------- | -------------------------------------------------------------------------------------- | -| `address` | yes | The public DNS or IP address | -| `user` | yes | A user that can run docker commands | -| `role` | yes | List of Kubernetes roles assigned to the node | -| `internal_address` | no | The private DNS or IP address for internal cluster traffic | -| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | - - - -## 4. Run RKE - -After configuring `rancher-cluster.yml`, open Terminal and change directories to the RKE binary. Then enter the command below to stand up your high availability cluster. - -``` -rke up --config ./rancher-cluster.yml -``` - -## 5. Initialize Helm and Render Templates - - -Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. - -Initialize `helm` locally on a system that has internet access. - -```plain -helm init -c -``` - -Then, using the same system, fetch and render the `helm` charts. - -## 6. Optional: Install Cert-Manager - -If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. - -From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). - -```plain -helm fetch stable/cert-manager -``` - -Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. - -```plain -helm template ./cert-manager-.tgz --output-dir . \ ---name cert-manager --namespace kube-system \ ---set image.repository=/quay.io/jetstack/cert-manager-controller -``` - -## 7. Install Rancher - -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. - -```plain -helm repo add rancher- https://releases.rancher.com/server-charts/ -``` - -Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). - -```plain -helm fetch rancher-/rancher -``` - -Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. - -```plain -helm template ./rancher-.tgz --output-dir . \ ---name rancher --namespace cattle-system \ ---set hostname= \ ---set rancherImage=/rancher/rancher -``` - -## 8. Copy and Apply Manifests - -Copy the rendered manifest directories to a system that has access to the Rancher server cluster. - -Use `kubectl` to create namespaces and apply the rendered manifests. - -```plain -kubectl -n kube-system apply -R -f ./cert-manager - -kubectl create namespace cattle-system -kubectl -n cattle-system apply -R -f ./rancher -``` - -Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. - -## 9. Configure Rancher for the Private Registry - -Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . - -1. Log into Rancher and configure the default admin password. - -1. Go into the **Settings** view. - - ![Settings]({{< baseurl >}}/img/rancher/airgap/settings.png) - -1. Look for the setting called `system-default-registry` and choose **Edit**. - - ![Edit]({{< baseurl >}}/img/rancher/airgap/edit-system-default-registry.png) - -1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. - - ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) - ->**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md new file mode 100644 index 00000000000..f45caa25ba3 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md @@ -0,0 +1,30 @@ +--- +title: "Air Gap: High Availability Install" +weight: 280 +--- + +## Prerequisites + +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). + +The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` + +* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. +* [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. +* [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. + +## Caveats + +In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. + +As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. + + +## Installation Outline + +- [1. Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/provision-hosts/) +- [2. Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/prepare-private-registry/) +- [3. Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/install-kube/) +- [4. Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/helm-init/) +- [5. Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/install-rancher/) +- [6. Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/config-rancher-for-private-reg/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md new file mode 100644 index 00000000000..e6eef0dcaed --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md @@ -0,0 +1,24 @@ +--- +title: "6. Configure Rancher for the Private Registry" +weight: +aliases: + +--- + +Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . + +1. Log into Rancher and configure the default admin password. + +1. Go into the **Settings** view. + + ![Settings]({{< baseurl >}}/img/rancher/airgap/settings.png) + +1. Look for the setting called `system-default-registry` and choose **Edit**. + + ![Edit]({{< baseurl >}}/img/rancher/airgap/edit-system-default-registry.png) + +1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. + + ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) + +>**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md new file mode 100644 index 00000000000..e924b9843df --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md @@ -0,0 +1,36 @@ +--- +title: 4. Initialize Helm +weight: 400 +aliases: +--- + +## A. Initialize Helm and Render Templates + + +Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. + +Initialize `helm` locally on a system that has internet access. + +```plain +helm init -c +``` + +Then, using the same system, fetch and render the `helm` charts. + +## B. Optional: Install Cert-Manager + +If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. + +From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). + +```plain +helm fetch stable/cert-manager +``` + +Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. + +```plain +helm template ./cert-manager-.tgz --output-dir . \ +--name cert-manager --namespace kube-system \ +--set image.repository=/quay.io/jetstack/cert-manager-controller +``` \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md new file mode 100644 index 00000000000..d0192bab02f --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md @@ -0,0 +1,68 @@ +--- +title: "3. Install Kubernetes with RKE" +weight: 300 +aliases: + +--- + +## A. Create an RKE Config File + + +From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. Replace the IP addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. + +>**Tip:** See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. + +Replace values in the code sample according to the table below. + +| Directive Replacement | Description | +| ----------------------- | --------------------------------------------------------------------- | +| `address` | The IP address for each of your air gap nodes outside of the cluster. | +| `internal_address` | The IP address for each of your air gap nodes within the cluster. | +| `url` | The URL for your private registry. | + +> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls. + +```yaml +nodes: +- address: 18.222.121.187 # air gap node external IP + internal_address: 172.31.7.22 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa +- address: 18.220.193.254 # air gap node external IP + internal_address: 172.31.13.132 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa +- address: 13.59.83.89 # air gap node external IP + internal_address: 172.31.3.216 # air gap node internal IP + user: rancher + role: [ "controlplane", "etcd", "worker" ] + ssh_key_file: /home/user/.ssh/id_rsa + +private_registries: +- url: # private registry url + user: rancher + password: "*********" + is_default: true +``` + +### Common RKE Nodes Options + +| Option | Required | Description | +| ------------------ | -------- | -------------------------------------------------------------------------------------- | +| `address` | yes | The public DNS or IP address | +| `user` | yes | A user that can run docker commands | +| `role` | yes | List of Kubernetes roles assigned to the node | +| `internal_address` | no | The private DNS or IP address for internal cluster traffic | +| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | + + + +## B. Run RKE + +After configuring `rancher-cluster.yml`, open Terminal and change directories to the RKE binary. Then enter the command below to stand up your high availability cluster. + +``` +rke up --config ./rancher-cluster.yml +``` \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md new file mode 100644 index 00000000000..68c4e941e20 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -0,0 +1,43 @@ +--- +title: "5. Install Rancher" +weight: 500 +aliases: +--- + +## A. Install Rancher + +Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. + +```plain +helm repo add rancher- https://releases.rancher.com/server-charts/ +``` + +Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). + +```plain +helm fetch rancher-/rancher +``` + +Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. + +```plain +helm template ./rancher-.tgz --output-dir . \ +--name rancher --namespace cattle-system \ +--set hostname= \ +--set rancherImage=/rancher/rancher +``` + +## B. Copy and Apply Manifests + +Copy the rendered manifest directories to a system that has access to the Rancher server cluster. + +Use `kubectl` to create namespaces and apply the rendered manifests. + +```plain +kubectl -n kube-system apply -R -f ./cert-manager + +kubectl create namespace cattle-system +kubectl -n cattle-system apply -R -f ./rancher +``` + +Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md new file mode 100644 index 00000000000..4fa593ffa4a --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -0,0 +1,75 @@ +--- +title: "2. Collect and Publish Image Sources" +weight: 200 +aliases: + +--- + +Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install in your air gap environment. Download the following three files: + + +| Release File | Description | +| --- | --- | +| `rancher-images.txt` | Contains a list of all files needed to install Rancher. +| `rancher-save-images.sh` | Pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | Loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + + +The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. + +* **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. +* **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. + + ```plain + rke config --system-images >> ./rancher-images.txt + ``` +* **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. + + Fetch the latest `cert-manager` Helm chart and parse the template for image details. + + ```plain + helm fetch stable/cert-manager + helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt + ``` + +Sort and unique the images list to remove any overlap between the sources. + +```plain +sort -u rancher-images.txt -o rancher-images.txt +``` + + +After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. + +>**Note:** Image publication may require up to 20GB of empty disk space. + +1. From Terminal, change directories to the path containing the files listed above. + +1. Make `rancher-save-images.sh` an executable. + + ``` + chmod +x rancher-save-images.sh + ``` + +1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. + +1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry.files to each of the [Linux hosts](#1-provision-three-linux-hosts-and-load-balancer) that you've provisioned. + + + 1. Log into your private registry if required. + + ```plain + docker login + ``` + + 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md new file mode 100644 index 00000000000..a344df90129 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md @@ -0,0 +1,29 @@ +--- +title: "1. Create Nodes and Load Balancer" +weight: 100 +aliases: +--- +Provision three air gapped Linux hosts according to our requirements below to launch Rancher in an HA configuration. + +These hosts should be disconnected from the internet, but should have connectivity with your private registry. + +### Host Requirements + +View hardware and software requirements for each of your cluster nodes in [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements). + +### Recommended Architecture + +- DNS for Rancher should resolve to a layer 4 load balancer +- The Load Balancer should forward port TCP/80 and TCP/443 to all 3 nodes in the Kubernetes cluster. +- The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. +- The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. + +
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
+ +![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) + +### Load Balancer + +RKE, the installer that provisions your air gapped cluster, will configure an Ingress controller pod on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. + +Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/_index.md deleted file mode 100644 index b6a943ab19c..00000000000 --- a/content/rancher/v2.x/en/installation/air-gap-installation/_index.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Air Gap Install -weight: 300 ---- -In environments where security is high priority, you can set up Rancher in an air gap configuration. Air gap installs are more secure than standard single-node or HA deployments because the network that runs Rancher is disconnected from the Internet, reducing your security surface area. - -## Prerequisites - -- Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). - - For each Rancher [release](https://github.com/rancher/rancher/releases), we provide the Docker images and scripts needed to mirror these images to your own registry. The Docker images are used when installing Rancher in an HA setup, when provisioning a cluster where Rancher is launching Kubernetes, or when you enable features like pipelines or logging. - -- **Installation Option:** Before beginning your air gap installation, choose whether you want ~~a~~ [single-node install]({{< baseurl >}}/rancher/v2.x/en/installation/single-node) or a [high availability install]({{< baseurl >}}/rancher/v2.x/en/installation/ha). View your chosen configuration's introduction notes along with Rancher's [node requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements). - -## Caveats - -Any Rancher version prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - -As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - -## Air Gap Installation Outline - -While installing Rancher in an air gap configuration, you'll complete several different tasks. - -- [1—Preparing the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/) -- [2—Installing Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/) -- [3—Configuring Rancher to default to the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/private-registry/) - - -### [Next: Prepare the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/install-rancher/_index.md deleted file mode 100644 index 2adfc00ff8c..00000000000 --- a/content/rancher/v2.x/en/installation/air-gap-installation/install-rancher/_index.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -title: 2—Installing Rancher -weight: 50 ---- - -After your private registry is set up for your Rancher installation, complete your installation. Follow one of the procedures below based on the configuration in which you want to run Rancher. - -{{% tabs %}} -{{% tab "HA Install" %}} -This guide will take you through the basic process of installing Rancher Server HA in a Air Gap environment. Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. - -## RKE - -On a system that has access (22/tcp and 6443/tcp) to the nodes you have built to host the Rancher server cluster, use the sample below create the `rancher-cluster.yml` file. Define your nodes and fill out the details for the private registry. - -See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. - -Replace values in the code sample according to the table below. - -| Directive Replacement | Description | -| ----------------------- | --------------------------------------------------------------------- | -| `address` | The IP address for each of your air gap nodes outside of the cluster. | -| `internal_address` | The IP address for each of your air gap nodes within the cluster. | -| `url` | The URL for your private registry. | - -```yaml -nodes: -- address: 18.222.121.187 # air gap node external IP - internal_address: 172.31.7.22 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa -- address: 18.220.193.254 # air gap node external IP - internal_address: 172.31.13.132 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa -- address: 13.59.83.89 # air gap node external IP - internal_address: 172.31.3.216 # air gap node internal IP - user: rancher - role: [ "controlplane", "etcd", "worker" ] - ssh_key_file: /home/user/.ssh/id_rsa - -private_registries: -- url: # private registry url - user: rancher - password: "*********" - is_default: true -``` - -### Run RKE - -```plain -rke up --config ./rancher-cluster.yml -``` - -### Testing the Cluster - -Follow the rest of the [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) guide to test your cluster and verify the health of your pods before continuing. - -## Helm - -Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. - -### Initialize Helm Locally - -Skip the [Initialize Helm (Install Tiller)]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-init/) and initialize `helm` locally on a system that has internet access. - -```plain -helm init -c -``` - -## Installing Rancher - -If you set up a default private registry with credentials in RKE, the Kubernetes `kubelet` will have the credentials for your private registry configured. - -### Render Templates - -Fetch and render the `helm` charts on a system that has internet access. - -#### Cert-Manager - -If you are installing Rancher with Rancher self-signed certificates you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. - -Fetch the latest `cert-manager` chart from the [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). - -```plain -helm fetch stable/cert-manager -``` - -Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. - -```plain -helm template ./cert-manager-.tgz --output-dir . \ ---name cert-manager --namespace kube-system \ ---set image.repository=/quay.io/jetstack/cert-manager-controller -``` - -#### Rancher - -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). - -```plain -helm repo add rancher- https://releases.rancher.com/server-charts/ -``` - -Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). - -```plain -helm fetch rancher-/rancher -``` - -Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. - -```plain -helm template ./rancher-.tgz --output-dir . \ ---name rancher --namespace cattle-system \ ---set hostname= \ ---set rancherImage=/rancher/rancher -``` - -### Copy Manifests - -Copy the rendered manifest directories to a system that has access to the Rancher server cluster. - -### Apply the Manifests - -Use `kubectl` to create namespaces and apply the rendered manifests. - -```plain -kubectl -n kube-system apply -R -f ./cert-manager - -kubectl create namespace cattle-system -kubectl -n cattle-system apply -R -f ./rancher -``` - -Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. - -{{% /tab %}} -{{% tab "Single Node" %}} -To deploy Rancher on a single node in an air gap environment, follow the instructions in the standard [Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/single-node-install/). Parts of the install where you must complete a special action for air gap are flagged with a substitute step, which is listed in the subheading below. - -### Add Private Registry URL to Run Command - -When you get to the section [Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/#2-choose-an-ssl-option-and-install-rancher), regardless of which install option you choose, prepend your Rancher image tag with your private registry URL (``), as shown in the example below. - -```plain -docker run -d --restart=unless-stopped \ - -p 80:80 -p 443:443 \ - /rancher/rancher: -``` - -{{% /tab %}} -{{% /tabs %}} - -### [Next: Configuring Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/config-rancher-for-private-reg/) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/_index.md deleted file mode 100644 index 05a6d06829e..00000000000 --- a/content/rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/_index.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: 1—Preparing the Private Registry -weight: 25 ---- - -For the first part of your air gap install, you'll prepare your private registry in order to be able to install and start using Rancher. - - - -## Image Sources - -Collect the list of images required for Rancher. These steps will require internet access. - -{{% tabs %}} -{{% tab "HA Install" %}} -The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. - -* **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. -* **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. - - ```plain - rke config --system-images >> ./rancher-images.txt - ``` -* **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. - - Fetch the latest `cert-manager` Helm chart and parse the template for image details. - - ```plain - helm fetch stable/cert-manager - helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt - ``` - -Sort and unique the images list to remove any overlap between the sources. - -```plain -sort -u rancher-images.txt -o rancher-images.txt -``` - -{{% /tab %}} -{{% tab "Single Node" %}} -All the required images for a Single Node install can be found in the `rancher-images.txt` included with the release of Rancher you are installing. - -Download the `rancher-images.txt` from the [Rancher releases](https://github.com/rancher/rancher/releases) page. - -{{% /tab %}} -{{% /tabs %}} - -## Publish Images - -Once you have the `rancher-images.txt` file populated, publish the images from the list to your private registry. - -> **NOTE** This may require up to 20GB of disk space. - -1. Browse to the [Rancher releases page](https://github.com/rancher/rancher/releases) and download the following tools for saving and publishing the images. - - | Release File | Description | - | --- | --- | - | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | - | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | - -1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. - - ```plain - ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to a system that can reach your private registry. - - Log into your registry if required. - - ```plain - docker login - ``` - - Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry - ``` - -### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/) diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md new file mode 100644 index 00000000000..c59e02d933c --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md @@ -0,0 +1,26 @@ +--- +title: "Air Gap: Single Node Install" +weight: 270 +--- + +## Prerequisites + +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). + + +## Caveats + +In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. + +As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. + +## Installation Outline + +- [1. Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/provision-host/) +- [2. Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/prepare-private-registry/) +- [3. Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/install-rancher/) +- [4. Configure Rancher for Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/config-rancher-for-private-reg/) + + + +### [Next: Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/provision-host/) diff --git a/content/rancher/v2.x/en/installation/air-gap-installation/config-rancher-for-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md similarity index 92% rename from content/rancher/v2.x/en/installation/air-gap-installation/config-rancher-for-private-reg/_index.md rename to content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md index 2f754ce09b1..896a47f3c10 100644 --- a/content/rancher/v2.x/en/installation/air-gap-installation/config-rancher-for-private-reg/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md @@ -1,6 +1,7 @@ --- -title: 3—Configuring Rancher for the Private Registry -weight: 75 +title: "4. Configure Rancher for the Private Registry" +weight: 400 +aliases: --- Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . diff --git a/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md similarity index 53% rename from content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-single-node/_index.md rename to content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md index 417ae3daa0b..b890c6eef11 100644 --- a/content/rancher/v2.x/en/installation/WIP-air-gap-installation/air-gap-single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md @@ -1,79 +1,8 @@ --- -title: "Air Gap: Single Node Install" -weight: +title: "3. Choose an SSL Option and Install Rancher" +weight: 300 +aliases: --- -## Outline - - - -- [Prerequisites](#prerequisites) -- [Caveats](#caveats) -- [1. Provision Linux Host](#1-provision-linux-host) -- [2. Collect Image Sources](#2-collect-image-sources) -- [3. Publish Images](#3-publish-images) -- [4. Choose an SSL Option and Install Rancher](#4-choose-an-ssl-option-and-install-rancher) -- [5. Configure Rancher for the Private Registry](#5-configure-rancher-for-the-private-registry) - - - -## Prerequisites - -Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). - - -## Caveats - -In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - -As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - - -## 1. Provision Linux Host - -Provision a single, air gapped Linux host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements) to launch your {{< product >}} Server. - -This host should be disconnected from the internet, but should have connectivity with your private registry. - -## 2. Collect Image Sources - -Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: - - -| Release File | Description | -| --- | --- | -| `rancher-images.txt` | This file contains a list of all files needed to install Rancher. -| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | -| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | - - -## 3. Publish Images - -After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. - ->**Note:** Image publication may require up to 20GB of empty disk space. - -1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. - - ```plain - ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. - - 1. Log into your registry if required. - - ```plain - docker login - ``` - - 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry - ``` - - -## 4. Choose an SSL Option and Install Rancher For development and testing in air gap environments, we recommend installing Rancher by running a single Docker container. In this installation scenario, you'll deploy Rancher to your air gap host using an image pulled from your private registry. @@ -157,22 +86,5 @@ Placeholder | Description ``` {{% /accordion %}} -## 5. Configure Rancher for the Private Registry -Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . - -1. Log into Rancher and configure the default admin password. - -1. Go into the **Settings** view. - - ![Settings]({{< baseurl >}}/img/rancher/airgap/settings.png) - -1. Look for the setting called `system-default-registry` and choose **Edit**. - - ![Edit]({{< baseurl >}}/img/rancher/airgap/edit-system-default-registry.png) - -1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. - - ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) - ->**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. \ No newline at end of file +### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/config-rancher-for-private-reg/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md new file mode 100644 index 00000000000..708cd336c6d --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md @@ -0,0 +1,45 @@ +--- +title: "2. Prepare Private Registry" +weight: 200 +aliases: +--- + +## A. Collect Image Sources + +Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: + + +| Release File | Description | +| --- | --- | +| `rancher-images.txt` | This file contains a list of all files needed to install Rancher. +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + + +## B. Publish Images + +After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. + +>**Note:** Image publication may require up to 20GB of empty disk space. + +1. From a system with internet access, use the `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. + + 1. Log into your registry if required. + + ```plain + docker login + ``` + + 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. + + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` + +### [Next: Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/install-rancher/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md new file mode 100644 index 00000000000..37d931f70d8 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md @@ -0,0 +1,11 @@ +--- +title: "1. Provision Linux Host" +weight: 100 +aliases: +--- + +Provision a single, air gapped Linux host according to our [Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/requirements) to launch your {{< product >}} Server. + +This host should be disconnected from the internet, but should have connectivity with your private registry. + +### [Next: Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/prepare-private-registry/) From 45a9d59042639db64d32d04dec70eb194fa171d6 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 5 Nov 2018 17:29:08 -0700 Subject: [PATCH 09/24] link fixes --- .../install-rancher/_index.md | 59 ++++++++++++++++++- .../air-gap-single-node/_index.md | 10 ++-- .../install-rancher/_index.md | 2 +- .../prepare-private-registry/_index.md | 2 +- .../provision-host/_index.md | 2 +- 5 files changed, 64 insertions(+), 11 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index 68c4e941e20..c5e14f7e6b7 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -27,6 +27,61 @@ helm template ./rancher-.tgz --output-dir . \ --set rancherImage=/rancher/rancher ``` +Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are three options for the source of the certificate: + +{{% accordion id="self-signed" label="Rancher-Generated Certificates (Default)" %}} +The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. + +- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). +- Set the `hostname` to the DNS name you pointed at your load balancer. + +```plain +helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher +``` + +{{% /accordion %}} +{{% accordion id="lets-encrypt" label="Let's Encrypt" %}} +Use [LetsEncrypt](https://letsencrypt.org/)'s free service to issue trusted SSL certs. This configuration uses http validation so the Load Balancer must have a Public DNS record and be accessible from the internet. + +- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). +- Set `hostname`, `ingress.tls.source=letsEncrypt` and LetsEncrypt options. + +``` +helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + --set ingress.tls.source=letsEncrypt \ + --set letsEncrypt.email=me@example.org +``` +{{% /accordion %}} +{{% accordion id="secret" label="Certificates for Files (Kubernetes Secret)" %}} +Create Kubernetes secrets from your own certificates for Rancher to use. + +> **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. + +- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). +- Set `hostname` and `ingress.tls.source=secret`. + +> **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` + +``` +helm install rancher-/rancher \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + --set ingress.tls.source=letsEncrypt \ +``` + +Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +{{% /accordion %}} + ## B. Copy and Apply Manifests Copy the rendered manifest directories to a system that has access to the Rancher server cluster. @@ -38,6 +93,4 @@ kubectl -n kube-system apply -R -f ./cert-manager kubectl create namespace cattle-system kubectl -n cattle-system apply -R -f ./rancher -``` - -Make sure you follow any additional instructions required by SSL install options. See [Choose your SSL Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#choose-your-ssl-configuration) for details. \ No newline at end of file +``` \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md index c59e02d933c..92c8bcd57ca 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md @@ -16,11 +16,11 @@ As of v2.1.0, registries with authentication work for installing Rancher as well ## Installation Outline -- [1. Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/provision-host/) -- [2. Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/prepare-private-registry/) -- [3. Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/install-rancher/) -- [4. Configure Rancher for Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/config-rancher-for-private-reg/) +- [1. Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/provision-host/) +- [2. Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/) +- [3. Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/) +- [4. Configure Rancher for Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/) -### [Next: Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/provision-host/) +### [Next: Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/provision-host/) diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md index b890c6eef11..b31a74d4a80 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md @@ -87,4 +87,4 @@ Placeholder | Description {{% /accordion %}} -### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/config-rancher-for-private-reg/) \ No newline at end of file +### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md index 708cd336c6d..ead48d1fc7e 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md @@ -42,4 +42,4 @@ After downloading the release files, publish the images from `rancher-images.txt ./rancher-load-images.sh --image-list ./rancher-images.txt --registry ``` -### [Next: Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/install-rancher/) \ No newline at end of file +### [Next: Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md index 37d931f70d8..567cf6d0074 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/provision-host/_index.md @@ -8,4 +8,4 @@ Provision a single, air gapped Linux host according to our [Requirements]({{< ba This host should be disconnected from the internet, but should have connectivity with your private registry. -### [Next: Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-install/air-gap-single-node/prepare-private-registry/) +### [Next: Prepare Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/) From c0298ce13002d414e6dd0d6ba0e8b2657e552d73 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 5 Nov 2018 18:45:54 -0700 Subject: [PATCH 10/24] completing draft of new air gap docs --- .../air-gap-high-availability/_index.md | 19 +++++++++------ .../helm-init/_index.md | 4 +++- .../install-kube/_index.md | 15 ++++++------ .../install-rancher/_index.md | 7 ++++-- .../prepare-private-registry/_index.md | 2 ++ .../provision-hosts/_index.md | 2 ++ .../air-gap-single-node/_index.md | 2 +- .../config-rancher-for-private-reg/_index.md | 2 +- .../install-rancher/_index.md | 24 +++++++++---------- .../prepare-private-registry/_index.md | 2 +- .../v2.x/en/installation/references/_index.md | 2 +- 11 files changed, 48 insertions(+), 33 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md index f45caa25ba3..f566c1f9c26 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md @@ -1,6 +1,8 @@ --- title: "Air Gap: High Availability Install" -weight: 280 +weight: 290 +aliases: + - /rancher/v2.x/en/installation/air-gap-installation/install-rancher/ --- ## Prerequisites @@ -22,9 +24,12 @@ As of v2.1.0, registries with authentication work for installing Rancher as well ## Installation Outline -- [1. Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/provision-hosts/) -- [2. Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/prepare-private-registry/) -- [3. Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/install-kube/) -- [4. Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/helm-init/) -- [5. Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/install-rancher/) -- [6. Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/wip-air-gap-installation/air-gap-high-availability/config-rancher-for-private-reg/) \ No newline at end of file +- [1. Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/) +- [2. Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/) +- [3. Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/) +- [4. Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/) +- [5. Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) +- [6. Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) + + +### [Next: Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md index e924b9843df..71c52220c10 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md @@ -33,4 +33,6 @@ Render the template with the option you would use to install the chart. Remember helm template ./cert-manager-.tgz --output-dir . \ --name cert-manager --namespace kube-system \ --set image.repository=/quay.io/jetstack/cert-manager-controller -``` \ No newline at end of file +``` + +### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md index d0192bab02f..dd7e05f0263 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md @@ -7,14 +7,12 @@ aliases: ## A. Create an RKE Config File +From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create a new file named `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. -From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. Replace the IP addresses in the `nodes` list with the IP address or DNS names of the 3 nodes you created. +Replace values in the code sample according to the table below with the IP address or DNS names of the 3 nodes you created. ->**Tip:** See [Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/ha/kubernetes-rke/) for more details on the options available. -Replace values in the code sample according to the table below. - -| Directive Replacement | Description | +| Placeholder | Description | | ----------------------- | --------------------------------------------------------------------- | | `address` | The IP address for each of your air gap nodes outside of the cluster. | | `internal_address` | The IP address for each of your air gap nodes within the cluster. | @@ -57,7 +55,8 @@ private_registries: | `internal_address` | no | The private DNS or IP address for internal cluster traffic | | `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | - +>**Tip:** For more details on the options available, see the RKE [Config Options](http://localhost:9001/rke/v0.1.x/en/config-options/). + ## B. Run RKE @@ -65,4 +64,6 @@ After configuring `rancher-cluster.yml`, open Terminal and change directories to ``` rke up --config ./rancher-cluster.yml -``` \ No newline at end of file +``` + +### [Next: Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index c5e14f7e6b7..86ab576d97c 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -6,7 +6,7 @@ aliases: ## A. Install Rancher -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). Please see the [High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha) guide for additional options and troubleshooting. +Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). ```plain helm repo add rancher- https://releases.rancher.com/server-charts/ @@ -26,6 +26,7 @@ helm template ./rancher-.tgz --output-dir . \ --set hostname= \ --set rancherImage=/rancher/rancher ``` +> Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are three options for the source of the certificate: @@ -93,4 +94,6 @@ kubectl -n kube-system apply -R -f ./cert-manager kubectl create namespace cattle-system kubectl -n cattle-system apply -R -f ./rancher -``` \ No newline at end of file +``` + +### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index 4fa593ffa4a..c65614336c3 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -73,3 +73,5 @@ After downloading the release files, publish the images from `rancher-images.txt ```plain ./rancher-load-images.sh --image-list ./rancher-images.txt --registry ``` + +### [Next: Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md index a344df90129..209475a4f24 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md @@ -27,3 +27,5 @@ View hardware and software requirements for each of your cluster nodes in [Requi RKE, the installer that provisions your air gapped cluster, will configure an Ingress controller pod on each of your nodes. The Ingress controller pods are bound to ports TCP/80 and TCP/443 on the host network and are the entry point for HTTPS traffic to the Rancher server. Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. + +### [Next: Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/) diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md index 92c8bcd57ca..d708feca5aa 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md @@ -1,6 +1,6 @@ --- title: "Air Gap: Single Node Install" -weight: 270 +weight: 280 --- ## Prerequisites diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md index 896a47f3c10..4d9a28bbfc4 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/_index.md @@ -4,7 +4,7 @@ weight: 400 aliases: --- -Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . +Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/). 1. Log into Rancher and configure the default admin password. diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md index b31a74d4a80..165156c337c 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md @@ -51,11 +51,11 @@ Placeholder | Description ``` docker run -d --restart=unless-stopped \ --p 80:80 -p 443:443 \ --v //:/etc/rancher/ssl/cert.pem \ --v //:/etc/rancher/ssl/key.pem \ --v //:/etc/rancher/ssl/cacerts.pem \ -/rancher/rancher: + -p 80:80 -p 443:443 \ + -v //:/etc/rancher/ssl/cert.pem \ + -v //:/etc/rancher/ssl/key.pem \ + -v //:/etc/rancher/ssl/cacerts.pem \ + /rancher/rancher: ``` @@ -77,13 +77,13 @@ Placeholder | Description `` | The path to the private key for your certificate. `` | Your private registry URL and port. Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. - ``` - docker run -d --restart=unless-stopped \ - -p 80:80 -p 443:443 \ - -v //:/etc/rancher/ssl/cert.pem \ - -v //:/etc/rancher/ssl/key.pem \ - /rancher/rancher: --no-cacerts - ``` +``` +docker run -d --restart=unless-stopped \ + -p 80:80 -p 443:443 \ + -v //:/etc/rancher/ssl/cert.pem \ + -v //:/etc/rancher/ssl/key.pem \ + /rancher/rancher: --no-cacerts +``` {{% /accordion %}} diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md index ead48d1fc7e..09ee94511ce 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md @@ -28,7 +28,7 @@ After downloading the release files, publish the images from `rancher-images.txt ./rancher-save-images.sh --image-list ./rancher-images.txt ``` -1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host](#1-provision-linux-host) that you've provisioned. +1. Copy `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` files to the [Linux host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/provision-host) that you've provisioned by completing the substeps below. 1. Log into your registry if required. diff --git a/content/rancher/v2.x/en/installation/references/_index.md b/content/rancher/v2.x/en/installation/references/_index.md index 9548d9afaed..1b90dc6694b 100644 --- a/content/rancher/v2.x/en/installation/references/_index.md +++ b/content/rancher/v2.x/en/installation/references/_index.md @@ -1,6 +1,6 @@ --- title: Port Requirements -weight: 280 +weight: 300 aliases: - /rancher/v2.x/en/hosts/amazon/#required-ports-for-rancher-to-work/ --- From cac97e22581fa93e86e2fdf21eb9ec7023436df3 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 5 Nov 2018 19:03:54 -0700 Subject: [PATCH 11/24] removing air gap notes --- .../config-rancher-for-private-reg/_index.md | 2 +- .../v2.x/en/installation/ha/helm-rancher/_index.md | 7 ++++--- .../ha/helm-rancher/chart-options/_index.md | 6 +++++- .../v2.x/en/installation/ha/kubernetes-rke/_index.md | 2 +- .../v2.x/en/installation/single-node/_index.md | 11 ++--------- .../single-node-install-external-lb/_index.md | 5 ----- 6 files changed, 13 insertions(+), 20 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md index e6eef0dcaed..08013579bc4 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md @@ -5,7 +5,7 @@ aliases: --- -Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/) . +Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/). 1. Log into Rancher and configure the default admin password. diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index babe345da90..e5d6f3979cb 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -5,7 +5,7 @@ weight: 200 Rancher installation is managed using the Helm package manager for Kubernetes. Use `helm` to install the prerequisite and charts to install Rancher. -> **Note:** For systems without direct internet access see [Installing Rancher - Air Gap]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/) for install details. +> **Note:** For systems without direct internet access see [Air Gap: High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/). ### Add the Helm Chart Repository @@ -65,8 +65,6 @@ The default is for Rancher to generate a CA and uses `cert-manager` to issue the - Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). - Set the `hostname` to the DNS name you pointed at your load balancer. ->**Using Air Gap?** [Set the `rancherImage` option]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#install-rancher-using-private-registry) in your command, pointing toward your private registry. - ``` helm install rancher-/rancher \ --name rancher \ @@ -92,9 +90,12 @@ This option uses `cert-manager` to automatically request and renew [Let's Encryp - Set `hostname` to the public DNS record, set `ingress.tls.source` to `letsEncrypt` and `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices) +<<<<<<< HEAD >**Using Air Gap?** [Set the `rancherImage` option]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#install-rancher-using-private-registry) in your command, pointing toward your private registry. +======= +>>>>>>> removing air gap notes ``` helm install rancher-/rancher \ --name rancher \ diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/_index.md index 39e9cf47214..0b09db2884b 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/_index.md @@ -78,7 +78,11 @@ kubectl -n cattle-system create secret generic tls-ca-additional --from-file=ca- ### Private Registry and Air Gap Installs -See [Installing Rancher - Air Gap]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/) for details on installing Rancher with a private registry. +For details on installing Rancher with a private registry, see: + +- [Air Gap: Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/) +- [Air Gap: High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/) + ### External TLS Termination diff --git a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md index 851052f68c3..b6bff6add8d 100644 --- a/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md +++ b/content/rancher/v2.x/en/installation/ha/kubernetes-rke/_index.md @@ -5,7 +5,7 @@ weight: 190 Use RKE to install Kubernetes with a high availability etcd configuration. -> **Note:** For systems without direct internet access see [RKE - Air Gap]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#rke) for install details. +>**Note:** For systems without direct internet access see [Air Gap: High Availability Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/) for install details. ### Create the `rancher-cluster.yml` File diff --git a/content/rancher/v2.x/en/installation/single-node/_index.md b/content/rancher/v2.x/en/installation/single-node/_index.md index dd4fe9dcc18..62ed302c57e 100644 --- a/content/rancher/v2.x/en/installation/single-node/_index.md +++ b/content/rancher/v2.x/en/installation/single-node/_index.md @@ -21,7 +21,7 @@ For security purposes, SSL (Secure Sockets Layer) is required when using Rancher > >- Use a proxy? See [HTTP Proxy Configuration]({{< baseurl >}}/rancher/v2.x/en/installation/single-node/proxy/) >- Configure custom CA root certificate to access your services? See [Custom CA root certificate]({{< baseurl >}}/rancher/v2.x/en/admin-settings/custom-ca-root-certificate/) ->- Complete an Air Gap Installation? See [Air Gap](#air-gap) +>- Complete an Air Gap Installation? See [Air Gap: Single Node Install]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/) >- Record all transactions with the Rancher API? See [API Auditing](#api-auditing) > @@ -33,7 +33,6 @@ If you are installing Rancher in a development or testing environment where iden Log into your Linux host, and then run the minimum installation command below. ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ @@ -55,8 +54,6 @@ After creating your certificate, run the Docker command below to install Rancher - Replace `` with the directory path to your certificate file. - Replace ``,``, and `` with your certificate names. ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. - ``` docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ @@ -81,8 +78,6 @@ After obtaining your certificate, run the Docker command below. - Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. - ``` docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ @@ -104,8 +99,6 @@ For production environments, you also have the options of using [Let's Encrypt]( After you fulfill the prerequisites, you can install Rancher using a Let's Encrypt certificate by running the following command. Replace `` with your your domain. ->**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image. - docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ rancher/rancher:latest \ @@ -146,7 +139,7 @@ docker run -d --restart=unless-stopped \ ### Air Gap -If you are visiting this page to complete an [Air Gap Installation]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/), you must pre-pend your private registry URL to the server tag when running the installation command in the option that you choose. Add `` with your private registry URL in front of `rancher/rancher:latest`. +If you are visiting this page to complete an air gap installation, you must pre-pend your private registry URL to the server tag when running the installation command in the option that you choose. Add `` with your private registry URL in front of `rancher/rancher:latest`. **Example:** diff --git a/content/rancher/v2.x/en/installation/single-node/single-node-install-external-lb/_index.md b/content/rancher/v2.x/en/installation/single-node/single-node-install-external-lb/_index.md index acc8ae97f56..1ee8c4fffbb 100644 --- a/content/rancher/v2.x/en/installation/single-node/single-node-install-external-lb/_index.md +++ b/content/rancher/v2.x/en/installation/single-node/single-node-install-external-lb/_index.md @@ -47,9 +47,6 @@ If you elect to use a self-signed certificate to encrypt communication, you must 1. While running the Docker command to deploy Rancher, point Docker toward your CA certificate file. - >**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image tag. - - ``` docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ @@ -71,8 +68,6 @@ If you use a certificate signed by a recognized CA, installing your certificate 1. Enter the following command. - >**Air Gap User?** [Add your private registry URL]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#add-private-registry-url-to-run-command) before the `rancher/rancher` image tag. - ``` docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ From 7be423f28e9b1e3d48230a17a2a110149f61ef76 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 5 Nov 2018 19:07:11 -0700 Subject: [PATCH 12/24] removing let's encrypt --- .../install-rancher/_index.md | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index 86ab576d97c..63a8c92f462 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -28,7 +28,7 @@ helm template ./rancher-.tgz --output-dir . \ ``` > Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). -Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are three options for the source of the certificate: +Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate: {{% accordion id="self-signed" label="Rancher-Generated Certificates (Default)" %}} The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. @@ -45,22 +45,7 @@ helm template ./rancher-.tgz --output-dir . \ ``` {{% /accordion %}} -{{% accordion id="lets-encrypt" label="Let's Encrypt" %}} -Use [LetsEncrypt](https://letsencrypt.org/)'s free service to issue trusted SSL certs. This configuration uses http validation so the Load Balancer must have a Public DNS record and be accessible from the internet. -- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). -- Set `hostname`, `ingress.tls.source=letsEncrypt` and LetsEncrypt options. - -``` -helm template ./rancher-.tgz --output-dir . \ - --name rancher \ - --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher - --set ingress.tls.source=letsEncrypt \ - --set letsEncrypt.email=me@example.org -``` -{{% /accordion %}} {{% accordion id="secret" label="Certificates for Files (Kubernetes Secret)" %}} Create Kubernetes secrets from your own certificates for Rancher to use. From b7be48b4814a7c13707f69c8cb48c364e338d828 Mon Sep 17 00:00:00 2001 From: MBishop17 Date: Tue, 6 Nov 2018 11:25:29 -0800 Subject: [PATCH 13/24] resolving Denise's comment for single node. --- .../air-gap-high-availability/_index.md | 11 ++---- .../install-rancher/_index.md | 35 ++++++++++++------- .../air-gap-single-node/_index.md | 7 ---- .../prepare-private-registry/_index.md | 4 +-- 4 files changed, 26 insertions(+), 31 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md index f566c1f9c26..8dac5518de2 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md @@ -7,21 +7,14 @@ aliases: ## Prerequisites -Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). +Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). -The following CLI tools are required for this install. Please make sure these tools are installed and available in your `$PATH` +The following CLI tools are required for this install. Make sure these tools are installed on your workstation and available in your `$PATH`. * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool. * [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. * [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. -## Caveats - -In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - -As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - - ## Installation Outline - [1. Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index 63a8c92f462..af877ab2409 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -1,5 +1,5 @@ --- -title: "5. Install Rancher" +title: "5. Choose an SSL Option and Install Rancher" weight: 500 aliases: --- @@ -26,18 +26,23 @@ helm template ./rancher-.tgz --output-dir . \ --set hostname= \ --set rancherImage=/rancher/rancher ``` -> Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). -Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate: +>Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). -{{% accordion id="self-signed" label="Rancher-Generated Certificates (Default)" %}} -The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. +Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: + +{{% accordion id="self-signed" label="Option A: Default Self-Signed Certificate" %}} +The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. Use the reference table below to replace each placeholder. + +Placeholder | Description +------------|------------- +`` | The version number of the output tarball. +`` | The DNS name you pointed at your load balancer. +`` | The DNS name for your private registry.). -- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). -- Set the `hostname` to the DNS name you pointed at your load balancer. ```plain -helm template ./rancher-.tgz --output-dir . \ +helm template ./rancher-.tgz --output-dir . \ --name rancher \ --namespace cattle-system \ --set hostname= \ @@ -46,23 +51,27 @@ helm template ./rancher-.tgz --output-dir . \ {{% /accordion %}} -{{% accordion id="secret" label="Certificates for Files (Kubernetes Secret)" %}} +{{% accordion id="secret" label="Option B: Certificates for Files (Kubernetes Secret)" %}} Create Kubernetes secrets from your own certificates for Rancher to use. > **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. -- Replace `` with the repository that you configured in [Add the Helm Chart Repository](#add-the-helm-chart-repository) (i.e. `latest` or `stable`). -- Set `hostname` and `ingress.tls.source=secret`. +Placeholder | Description +------------|------------- +`` | The version number of the output tarball. +`` | The DNS name you pointed at your load balancer. +`` | The DNS name for your private registry. + > **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` ``` -helm install rancher-/rancher \ +helm template ./rancher-.tgz --output-dir . \ --name rancher \ --namespace cattle-system \ --set hostname= \ --set rancherImage=/rancher/rancher - --set ingress.tls.source=letsEncrypt \ + --set ingress.tls.source=secret \ ``` Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md index d708feca5aa..ec84e74550e 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md @@ -8,12 +8,6 @@ weight: 280 Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). -## Caveats - -In versions of Rancher prior to v2.1.0, registries with authentication are not supported when installing Rancher in HA or provisioning clusters, but after clusters are provisioned, registries with authentication can be used in the Kubernetes clusters. - -As of v2.1.0, registries with authentication work for installing Rancher as well as provisioning clusters. - ## Installation Outline - [1. Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/provision-host/) @@ -22,5 +16,4 @@ As of v2.1.0, registries with authentication work for installing Rancher as well - [4. Configure Rancher for Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/config-rancher-for-private-reg/) - ### [Next: Provision Linux Host]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-single-node/provision-host/) diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md index 09ee94511ce..a543dfaf17f 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md @@ -12,13 +12,13 @@ Using a computer with internet access, browse to our Rancher [releases page](htt | Release File | Description | | --- | --- | | `rancher-images.txt` | This file contains a list of all files needed to install Rancher. -| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | +| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | ## B. Publish Images -After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. +After collecting the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. >**Note:** Image publication may require up to 20GB of empty disk space. From 933bc69e250fbc68f1d25e180e91b8c00c4c3a4a Mon Sep 17 00:00:00 2001 From: MBishop17 Date: Tue, 6 Nov 2018 14:14:43 -0800 Subject: [PATCH 14/24] small edits --- .../air-gap-high-availability/helm-init/_index.md | 15 ++++++++++++--- .../install-rancher/_index.md | 5 ++++- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md index 71c52220c10..f1044b45b44 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md @@ -7,7 +7,7 @@ aliases: ## A. Initialize Helm and Render Templates -Instead of installing the `tiller` agent on the cluster, render the installs on a system that has access to the internet and copy resulting manifests to a system that has access to the Rancher server cluster. +From a system that has access to the internet, render the installs et and copy resulting manifests to a system that has access to the Rancher server cluster. Initialize `helm` locally on a system that has internet access. @@ -15,7 +15,16 @@ Initialize `helm` locally on a system that has internet access. helm init -c ``` -Then, using the same system, fetch and render the `helm` charts. +Then, using the same system, fetch and render the `helm` charts. Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. + +```plain +helm template ./rancher-.tgz --output-dir . \ +--name rancher --namespace cattle-system \ +--set hostname= \ +--set rancherImage=/rancher/rancher +``` + +>Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). ## B. Optional: Install Cert-Manager @@ -35,4 +44,4 @@ helm template ./cert-manager-.tgz --output-dir . \ --set image.repository=/quay.io/jetstack/cert-manager-controller ``` -### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) \ No newline at end of file +### [Next: Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index af877ab2409..d0eb89c5e54 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -4,7 +4,7 @@ weight: 500 aliases: --- -## A. Install Rancher +## A. Render Templates and Install Rancher Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). @@ -29,6 +29,9 @@ helm template ./rancher-.tgz --output-dir . \ >Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). +## A. Choose an SSL Option and Install Rancher + + Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: {{% accordion id="self-signed" label="Option A: Default Self-Signed Certificate" %}} From 3b150f948dae31fa22ef827761caeb8abfc3d37d Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Thu, 8 Nov 2018 11:10:47 -0700 Subject: [PATCH 15/24] figcaption --- content/rancher/v2.x/en/installation/ha/_index.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/content/rancher/v2.x/en/installation/ha/_index.md b/content/rancher/v2.x/en/installation/ha/_index.md index 879f9e78b44..9bfb6cb8e56 100644 --- a/content/rancher/v2.x/en/installation/ha/_index.md +++ b/content/rancher/v2.x/en/installation/ha/_index.md @@ -16,10 +16,7 @@ This procedure walks you through setting up a 3-node cluster with RKE and instal * The Ingress controller will redirect HTTP to HTTPS and terminate SSL/TLS on port TCP/443. * The Ingress controller will forward traffic to port TCP/80 on the pod in the Rancher deployment. -<<<<<<< HEAD -=======
HA Rancher install with layer 4 load balancer, depicting SSL termination at ingress controllers
->>>>>>> adding WIP ha content ![Rancher HA]({{< baseurl >}}/img/rancher/ha/rancher2ha.svg) HA Rancher install with Layer 4 load balancer (TCP), depicting SSL termination at ingress controllers From 346a24e959bb042add3ee54dee3b3f7e44fed91b Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Thu, 8 Nov 2018 17:46:20 -0700 Subject: [PATCH 16/24] finishing single node --- .../install-rancher/_index.md | 6 +++++- .../prepare-private-registry/_index.md | 6 +++++- src/img/rancher/choose-release-version.png | Bin 0 -> 90952 bytes 3 files changed, 10 insertions(+), 2 deletions(-) create mode 100644 src/img/rancher/choose-release-version.png diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md index 165156c337c..4b550200634 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/install-rancher/_index.md @@ -21,7 +21,8 @@ Choose from the following options: If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself. -Log into your Linux host, and then run the installation command below. Replace `` with your private registry URL and port. +Log into your Linux host, and then run the installation command below. Replace `` with your private registry URL and port. Replace `` with release tag of the [Rancher version]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/) that you want to install. + docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ @@ -48,6 +49,7 @@ Placeholder | Description `` | The path to the private key for your certificate. `` | The path to the certificate authority's private key. `` | Your private registry URL and port. +`` | The release tag of the [Rancher version]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/) that you want to install. ``` docker run -d --restart=unless-stopped \ @@ -76,6 +78,8 @@ Placeholder | Description `` | The path to your full certificate chain. `` | The path to the private key for your certificate. `` | Your private registry URL and port. Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher. +`` | The release tag of the [Rancher version]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/) that you want to install. + ``` docker run -d --restart=unless-stopped \ diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md index a543dfaf17f..f7b74daa966 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/prepare-private-registry/_index.md @@ -6,7 +6,11 @@ aliases: ## A. Collect Image Sources -Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install. Download the following three files, which are required to install Rancher in an air gap environment: +Using a computer with internet access, browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher 2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. + +![Choose Release Version]({{< baseurl >}}/img/rancher/choose-release-version.png) + +From the release's **Assets** section, download the following three files, which are required to install Rancher in an air gap environment: | Release File | Description | diff --git a/src/img/rancher/choose-release-version.png b/src/img/rancher/choose-release-version.png new file mode 100644 index 0000000000000000000000000000000000000000..6a2ea34984aebddd594502b1e414a224169bcc59 GIT binary patch literal 90952 zcmZs@Wl)^W^972A1rlKK0E;f}?(XgyB)A24cXxMp5AKrS?hrhS#9{s{pG0RjTzlca=*5(ESk_xnd3@ZtR%r{78L?+-`^C2=8$ z>Iu9<2nYcPNfAL6SI83`_{_*%g14@EZWCk=4fd)GHUq@gVW3nZDJ7T6n@VB4eV{^k zB1If3CDE{=q~uw9vJm)QYf&qCtuZAn>;BDSy>&e;%fTgC=gEUFXUT1S-@EJlq+{cO zw;TF*AAkWW5`v`X|L>u|1B?yduXCIK?*&p2q}O0H2tFY0hR~Akd#te=dE~(GvTO!h zqT@H$w>yB@3H?Jj@GA^9q?j-8yZCRV>%GklNXWY)_A{wU(T%)bbYjlqgZVTyR65;w zuu~jMy3-<;BevtV+8{HQD2zs(GWt0}P1xTN{BR?C9>k)AMw4lu$aulo{@^6?TFE9F z$?lzztJ2(0xUcPsqwUKbZlXF8L#V`TF74U@G`KhJUZ=mSzR0(J>%|~NfJ{C~EL|DT zF#bEC9%d-h3>J8z{Q5XRzirS#C|x|dNFO;J%Aaekr#o5Rn5{QV@VGe$9j8rP)-lxg zhej*Y&P%z^}t{l3h<7F7Jqcg~3CSFrNYYhf{H&bGM%ehxf>HS6rxT%01mN8ndMH&xwdH zn0>9(M6*n)iDITbS`{6NRs@}lKOI%v2z zIe9%p+TSu^;yrO6Q(OhGA^uJA9mGPwGc(B8t?0LOpTV1XL9OO^6mI>C-%9Y^DfEiO z40^gxsXGrRu~6KBFPM9zcK+eot&lUHx5_kU-K>M(XnSDx=rxmm3Gm&h#Aw53o%f&h zx}gy6dlqyiolVj21y9G{C}s;@DcUxAJxPf;vT>k-42z@~7OH7GJ_tG7P_WR2BF+1F z@ljX6qGfsd`XspM4^PN?2e^6sGxZ%fNUx_mlA(O_x@IqjsDVL8Tz0yfrcOidnKyH} zL4(nnunT-HQ%mq|2Bl$QjMAO7w%Hsn6QsKHuCv}E?QKnu0XH7Q!J)49rgXQ3*k z(;T!>7o0K7WMxYEeGg(Lh^SMqMyX1n>h4RP(Mee&;0Xm6zl#rf8Xj4lrdpvK zy)Y2QEE5RQLZ&tQ$$h-&Vw%4VZ(fb{zBwWMaQ`_DYhdod7Qs}nLF$Uwk(6dOT8nl7Y85+$#Wj@8KMl`-+x7ie~?M^YQo6b7h? zcVmxs&~8cmpNr{<>{*|iQv^#s^jZpl4>g(e2zDzwZABzROqu7GfS@8;PWZkg>Z#}Z z9OZmr%RkaN87qNd(ac7zV<0G+CA$!$#s5wq1;EJZL_-Jhq!{u1#^s=Jm||2uVe7(^ z(N~k-lneCb6Y3%#x#laAVlv#rXim)JwL^$O6huOnf!CAk<-av*^TQ0`z2Gy6za^P` zys%d@q`OGvBQflxV{xS9wOw{!l3zPXU#j$MYKy&&^rZ(dEM3CVT95v9BLeW^zR@MY z3A5ez9l?Uhhu;veB=ePrkNKK{O~Vdu$PTCEz5w^JAvb_u>$bT~Rs;UIN7N|aJmCe}j9j3oN<~KWxgo_D8+0M7?6<;Li z>0bmktT^0E({>QS^w&`k(#}_(KrTjqy=3kVaqL2+=F;z(B9|%Q!s-o$*xheR7T(#> zRb5vrsXQno0sugrAxk$i;eUG@>AMJ+LqZx@Ad^+d#)Ehy%3|Ex8%SDiIF#p)>=$N$ zas~kCQ{uNGW~x)^txaI$vN&tL)o9H>A0V!e4PC7Nb}^^^2eD@@cgA`4+8EfkH1A>H7~? z7YR3k|GB3RPy-LwMBQ(E=H3YNlb$7|bzEzchb|km2Ih=Q!|ai()1db#u$tHF^dA6j zD6y0ZC6}G|g!CG*C3JeQ@#xriC=3yJm~u}O9_M2Z>EYZ-GB7X^WNQre8+~Pcggpo& zq`1t3W+LJ;B6$>T!J;B3LubEgo16?L7@%N9e23*AA$QLH%`OoJW~UI0 zt>^vu&w01d1|5iVRm@w_S@HMuT@c@Ac?DvWuXW1A*335VapXPbK9dmU@W5|N(?gcX zKe0(Dps?6*B`SUS7^X`=3x|V?56z1w3WpZS&_J7pjtDwN^y&iT5 zQc=q;7Om)dX9^|sc!PPYR^!ii7twLC;qz4eIf@vnjlN997wjb)K5Fx4CTxWXD*F8} z*og$zeEv%szQUlx>~wnQd!Diq(vGu>l&goB?8!*n9%(*}b9&qf-FJF4>n$8JR|Tyb zNrZ*WaHE^QFgMLzePFQO{^)056-8>ZfutF#sw`=)*Fsg2k}3HnZbX~gQCIJ0xhO;5 ziBIStiG2_f)K4DhiVT2&I&}PXC1;=>%0a)SQvxj{L^zFz4lTB$*v5i_S=<4Ri;098 zNruhOCk(R-7c{V}G_Xx@4U6pgBqp%lh&XV$ExVD^RavAjyn7&px{d?4oUK1J$2?($cP37kif)79->zpO9p!*^yJC;b5fcdn_ADOzc~(N+c?tF@zlE zPP8=wFmME+C1f1~*yQ^G=*%#Yxa>mS`=AV%)zMpNUJoH09_&Ju%#sk2rMwSV2Tm5s z+XGcv;$nB>vp<9(f!X6J#t*Luxa@oUkEJ1;EsuLbzl%w^v#k>hux%fhY|rolC5wc7 zMP5%d1v+3Tua;bUW-IBAgoXXuoQV4SDCk)%U?forgrsDg5#TlOlYta_0PMx803Nrx zbCLaF@;tXK_>^`RQRmX$viaKDX6LYQ9C0LP3H6>Q5pgZ<878JP+IGv77zEMv$se>f z_%PNubtJ1P)M4JLsc*FijwwcY$HF^sC31q8;>KH9x`Y)_0G-^Qa;z2tZP#)Vv|13g zwXqFa-Qz<>gEk6R0+Btk0lt^_8?qvUDFkkX5hQVvZF_!jt6RULPZS1~`3DEo4skG8 zW4by<+d)Hl@qkH!l&AqYn$3Scmr&u`P_&TAka~dU1)=R$pd{l1l%!CQL7jqx8i6Fd zv0(&ptpXmMx*R8)P@_d`Uqp2DRR(%kETQ0zi%_u>g(asq9!lU){-4Z^A3C=ryJmv! zg%eqR#j-?UFBpW3OOz0a$iL?x512X$*6MLNQ|`jtOJ3FE^CICWvXL!Nhce zd4467`5BR#@)A{C3)P)^H+$>W` zol!VJbM#-FVH<6B=X>_Vz?%=c?cOiFj*^;kkpj9f;ozmX;WjDVF1)Lw`Lh>pfbzLk zBVk+@;!sXN{+}lA`9clM`MuWJCEuTniuSp13T`2Z;FKQ|ZjSKARoVN!Qk@`J{0Tcjq$~qF7(w&;_;0}u<%LH*^=AR@bbCsWMpH9ibWEtDi zMi^74-iWROUY>~FUYaG@O9?$t_Oj)9VI?(O+~VFLfkb83hq1PY*;#AX$Kzerh3%Qm z9$rihMo%sJ@QJ6fq2fO1L_u8GGl6WGo0u3=sVinp0H4m@xV785H&_%u-OdSKBv&F9 zWUxafIiaq{nc_Y*oiL>y*XDSs8Lo(TRwmOXZtuw>K)<`Tvdwk77g$*0;UY1~vpt=I zoB(3A%`}CP!iT!@gJwV?ILoMoErT{&_duSpYJT{bT4N+Ga{+2~Kb_C(RWEE<-3WoF z&l(*Gk6y10K<9i03B7tdWJNXFd-F=F-EF9Q)$Uqjz>7h~fFei*mTWF*n^?bjIc60kG| zj>VA6F;zHl-@NABpT-4w_4}r2&5A=Ws$b71QtogAycg(Q#$7V`KGcVAvF4d9E?vijM@FFH z;M4hj#zCn0m9(?S#-)g>n9ITU#DN3SYF<(VD3m$OBeTqrJ(e*5LcBJVCq3y&nW28*1i;+06<1QP5?q{3=6kk0OnyiB^mrRMySN3Hs zt$W{T2|FhgQ$MVVqoXGg*VaKf1S-f!fGhjsmV5#o(jm9?wF!@`^_jWNvRy`eP>&Ni zw~!W5bk&YcccXZEAt#Q#V)bW6sQ6IE{$V6Yzp&uw8yH|>ppt$Rj`OV?{Sab<-7f}g zEP7|H@t}7apx}>!;_gQ-@7}F{bzXFEu%p zVs}|p99}S`cLd(Rw)A@tr}zvQ`gt1E90uiPvF^pLH!`#?w`tiP3gTA!81G0QbV-C< zEte37*UmQxoX#>(-a@RwT~ef}KFERNcK#JW=O|q)8z2(*lhZB6mjgQGtP3}tzB};d zCnDK?n04Fe4#K*`pne^{=W|P1NX1{$8cH0Su)X$k$R{mWm{|mh9L5m zBcFxSeQoSdYr(IpU>LU1iW2gY!k#~ugFVT#XTcCd-)_dFg)!_^sw_zFty+`cS&KCe zOeFs+;GrO9^*qE|#C#xH)mx&~=HQE!AbO(Cc;afClZLB+z$XJ9bs;tzXVO!<`K(cC zY^cSc6KtQ?&sQM?%YxKS#~%*o)iG#aP86H-=W>mwL^iA!gXvFz3}mkXmi6u=W-~r_ z80FMyL(z7Rj*s`<`BaUQd$Eg$5hUUTTBkU77@qxqZHpGlUI{fahIfzcHHpQMf@P;> zbfb)6D7t%bXnSoHfd0ss%t4+U)F+r!=y6ty3R0v2bICZWK(r1aMDj>>ivZw1X}0Cr z8DbScaD{KEIxnj%Y}$pPsJ{&W*N^9VFfwBt1uibGd&b# z(``lY=r`e4w5ZqoN2ev*gO<@j68%u8HveF8ayL8Rl{ZygQ{FNaVRE z(AS6>ttIMfG=g|-uYWom25iv;0%*pfhK=-t>P@YbRmz~o1L6A@e4>d0(4YOC05mG$ zW&st1oIrqTU^IhT#fOIgV zVf}8c`<@?Gz6TqG52FkHlpuR^<58WBy;^OJ9lp_P+m3#g?dg0s!+&YHA91um-Vg=4 z4_7q8up>UreDx_B-W6;e4VMROx<`%>L?@U=?94;4#ra2x{_}zcaTq-UEMqtA^r1jF zmI*L9pd)WjawlYv!;V;_&rRu!)Pt^O1QOhvOMSG@@oOHnRI6GUa1f(1Z@d+kj95kgv{&oS%2u}5M4huH> zo#>gmbQp|NNHGgAYb{5Jzk*(q_%@)JE-p8wg5=4zt*AcXzQa&EL*B^ZA!o`lmyv-W z7+!u$OKiQGZbfGu_Qj)$$Z{o?rJQ{yhg+z7ZfA4#`?)f97g%Zf!3`$XJJ(LlX$L&> zYan9CyOS&OO-yLEmHm4?`3*@|2+?^1ELGeQ5E3juB6byhlq#a2rr-2V+y9etT9)3| z@Xn0vQdQ!i4~NN#Wbm(3{Qab#h;nQMkzet!-?X3%5wgH*iV4Bo1s^{%Tv8`ek<@Vx zY04-?wDbawRFbM!HnsNXhh9w3KH9`>ggO%Q`tyc@X4_TB#=x}s(#P1r+e6_cm*Nv- zlt9%6t>_ah>bS&^8=XYPG8wQ%%5k|)$ljD3ac4F|f^H0?jWGW?0XBY=UuH*2qLI7c z>;;lpz`1}mGs2wYmoeVX1fE=E!QUD50@MqOUvcZi8pN!HkYUvBYrJD)FA6_;!yA{# z2Z2}23V$WrEzp=rAdtgTfBOH7PuAa8Z#bGJT3D^RK~VSyRna=QY{(@Xti}ZVFhi*r z|BS*SPY_&<#2X1~muD#&Bc_5gAuv&Vk8vAYqOcj(X=ZNn3nkb4BMz}wPm*}YVC<99sQUtrvX8fWqVb&P$K228vm5l2@}wmEZ2ApJi0rP5b#_*qEgvzdv)oJXcWke*4#> zZS=omwFvlg>QV<_pj1A=uq@XjP^wT{7=1PkgimCbgp)m1i|~}|_b1=5h=8qE0t}D!Hqf|Oa4 zx}Xgsj5NT#+{+VNZc$y$DQ%X!=Bc&Ko`D zq74KoxARXN0lDcQsbA6FEcNPo=~bh-Gp=&?p-?YLlH=t;@P1 z;v5bYu?J>^wF^g>QC#&RDbMFO7!ut**sG(gpm@s`Davl$uPV^QG~-(|e2m(xu{5j* zKOW=6%V*clg+!O9^C97}e=0g;L70)lS;?;fwZkko?Mlq3AT0|@OSHBTQ8H&NOTF9| zBqDpT-(H>8prb;KhXNUtBAJ{}#pA9DVuYx4v~;vxA0q2rPqJi6E{ijz;WKil&C!E1X-wZn2?KTxi6*IDIL+QKu5D zhfDgo+ti;xdYs4P#rJS(B@>zI&IV!b)|pavbugN$q~=qEW(b8gML1(tY;Y_5+|kZH zOE^*9gitn)6$U{1F3d;=r>cWxom zXUHp$=j8xM&iK;71~Q{qs3&778Pw3N3%1y2E`&j~U~RQetp4G(9$^ z?4ZU(s69e4e-6yGJg{hGB_{Dl>0OeF@Xqg^@PtsE0%9daE-r&g<2N^j+uz@&#eJdu zFkd_z>uaXwndh6m-bCf8R5bvydQ5L!ev|@%j<#S_zH0Hv&9H+|BArGwkpYYrNI$vI zXzWNXm^LD=d`W%qq0igswynbJCR`ngL$Sx!J_a_OVN>`)Ut?4cjCejtF^%%)8kz7w zuI0rTan-TG@j6r={`8Oo&kI*hqsXT4qt%-`g(Nv-8;oheB*Jfwgj@!J7atm9ulVzT zHj+&1D~U#}6!_=hAMOuMaQSFO{dr}o7s_6fUnEvC48$EDox=Y9YCyix=b;SRkLMmjXRd+ z?@*qv1)8L+c;qcD1*2t6vqxBd@pN(Nn(N?mUE%9YCAM&{74OIdSmAFyn$dQcH-IQL zZVb8HBCh16q&ZC6u;AjyQjN9_{6s1>JW#3CbNZ%7Ez07JC-8Q&pRb43ejW76sp2fA*KguIg1G!~oYMsf`^04|s4j@NCNHZ}*il1>*MJT6- z8sa6H9qFcxK!W`UmX!4I^ryy8KgeIK2BjzP->e6RmEB9F8a9#p*&7C&U(;f2m@mnjYEp5|f69{%0K40R zxM@ia)MAEDCo5l3YS=AXeIVk;wuZbhT1(LnlJLcYRsw2od@wE>!w)#t5^92izkO5B zD(qa-rKoF{Z_^^euC%y1_I?|3_8+&MnM7`IcsFRS44-^$ygx7eBCTMqz5cjejg+!| z2J5P$Qd$ledL9&iv{?gB5-t`axpZBpH^1s;O{{O(4Ch}!#Jx>MBF~z9BXyfUINzD} z1+1UHv@^_7Etsn-p5kXy4JCs$YV~;j6vAPN*Qr+iGGac-cdAOT)44Umtv(S4`Xlty z@Fo!_kz+DW1Bqqp?H-(hGtyRq{q%I%{Kt7KC6NB==WwpM`Ta!4A97q5g4B~cLSsPP zX08+S%t>bM^H^gQ&YZ-l#;$!h4yfMgyb{*rkJyHsfSs{Q9+JuY=> zNsv4kJ(AqpT)sLPsX|C(K<%r6UQgu~v32@`n;E$T(#;WGeqKhM| z8csL+O&(t<&J+$S$WdB%{;^#FZPOZUG`@@2j0DgX2f%8-ap$oi)FH}iIu7h=>g-&slsxmtO6}4hwJDN!H z`e4@B!Fj$Fd~^=2&t%W`ds(DsJlt2^-^DZoi%(;R|E)ny(rQ`bI+L;s5UOGF-is2u zSj}oB^k5P{(6Jb+o!&>d>Otu>XgdX$bds?trs+HK{Kv+rW zKiTA&On&P{WTU7KsI*NeY1BV-*U{OOuoVVqb7(5}Pbn?%yjNWRIS|Md_A2_$(yB@& znE|V2oo)^(zUqXGr)b}~mRc`zV(v2~*8qkLPN$|}m&mftm+9eyCz4Jws<0I#5-}@Q z5t02qI44gYD2L`F|HZ@d0~P$NsFv8%v_judfGA9D(Q|p;pGRiWL0=!Mdg7;aG@-)s zdl$Agzwpw+oi-=suUQ$-k9f_xz9)6n=jp1-4u6%la89T$Z)2@y>WYqIDcN+5QBLJ& zU@Ya;tK=QqCfYxQj_KcXmW|CdpxRY+ zL20XiES`)AIoC#x+WqLHOI5Gkbb3IU+5B=Ze*H}CYB;>@7g1dJsrl+awBDJHsm(Zr zykfKpb{AYC0?vC;ogc&%BwL41i-B$4`rt96~(%83$dVDJrBv7fgdV|gFPrS4E!qp~jkO%0?! zk9Kfej^#o zv=en+^q{(^tLa{Km$(*8MRbh!Qcig_%}qyiXw>rJH1g9d&xG3z{Viu6<|p1*k{D=v z=)cxj3)qP`8QQ|?meLc)Gqh_PMk)QW*p&B;%d(aOry*&1zg&C^%`z;zvaXuYyF!-h zT9kZpq}n`$lIc&$5i|ZL*tE(Nez?GeMsyQ0sQcBLM=Y+mu+trlX{V7s`cajq>zUT< z<4e!7={$ca?1qD87I51as)wD=Z_nKT&mi4Nf_$&Ch11c7=D4yagw2e*k#j(5<>P&HOyJ!ZFk;AzCDx8ZW}`ez0o8q zKh!p91C-LastOZJ-0uF7_ruZz2Z$?PuSj#pSWETMk=u0dru@TyH0%>pQ%ORGm|J-u zWgX`wFc~&1}bk%y8U<4ejjicgCvPJr_x11x)GO}Wl`ikeX$LXO|#f8aRb6Yv+8ld zPVUv0!MGC?rIQ0TabEBzd>aQZV~n<=-OJ9SUsnCejt$R$0=@bkE2)3gKQdU9C-wFN z3kkd@4~|6R2CZ-dshbM7!pxh7Po&V14Bc+b3)F?%bt8xwRx1s_|VROJRqdpCv~P)SA$PY1{~>a5pE zPl|FA7vL^;5zU$g3~EUV2Dhk(TxSFLKeHD=Q));IT9|ZNcHEs}INAtvu_C>(e!{Qv zMQ$qP;mINWdA-%`+Fpf8%rLNP=pLym>DZU}6S`rW!#ZUvWn|{yfk`rdpy*4l_OlxU zZc+$HjG35dgzs7-1UN9MazA<^0R9juHMoy{*Xw_ne_N)}zzfP*HAXP)@`_lOE0IjS zIrpjswBlZ%q4IouZ-~;?gSjGKlKjpaT6pyONZiqdyT8Okh$6KwZedAcGlV2G)m9PE z|M9TM(f+<7b1Y(fJ)MVU$gOeh`~?qjnz5PI?6(BYgzmbCAN5?@Y%-~N{>8`~6SiS1 zTX#$m+l;FFto`{jfGyMvKJfZ|I4aYl5#(rYtpl8i}Ks8UmEB$awe zZLKFa1);@8B8mKj#o5J8YI<1BW5d5y3dMy#6n{mnQp}=YIJ{@Ig3Y-9zHMP~2?v_&oY8hlHY%f; zYUgYzb^90t$xL}S|L1VN_TKIaZAZX8k%&U;(SML6j`2y2rP@rKT&HrT`N7G_vggA4 zGfm6v=4+W4fegYhdGSCl#+R>U3Y#?VulO*eM&)N1 zkb)4Z%cm%lotGlX&2o<7_swdj1c`uMUAXa8a-wTHr{3V~0g7&I$NbFEV>yvWJEdy- z?-^Z>CbioudA9E?BjU58`~{a3|8##2(H^n_7p<&j1v#lYOsQy0?ggBQRdhhvXB6dG zR|JSss8-d2f6(r(4qW?8AMsDQdw-Yc_Y<0otTS>0(l-E0Ikv^6{)UxuGi!xW9fgceS2L>`fJAi24lUqVqSr0>616DXdv&&@NX^{!3 z>~07P*^rJ|AqKAnPU9q>zaJmhqDM$#tQ^*B$Yeg?S#WDQdzZMZ$jdq(v?v*5ftqdY zNdMQzDKIVCGW^yPkbcdhldTF&D#yWCH0LC13E{)V*r z@2=~E*o)14O3vO?>{jm4J{W4!%TD3kMAkjgVFq3 zI%xR4pU1Gdfur{;G~FVJ`M1pE21hvIEpmg3KjlilyhWE5N^n5|XIr`R%VcbnfJ(Ay zLHsFrfkJp>yu}TQs>^lY`3G4EB`uy9UB=?=@5`r;dGeZs-nV=lh+#&@GV!T3d_N*{ znhY>H5yKX)di@=Z^(W=88C<6bsXRU#^65n>n?RR1aH~N!ZZUYQXSn{xOyE|Wb^UPi zY6YT%Ywo4ym5I-e#j#?j!v!|3S=ya&{lQGC(S=os2bIZzR@6BETWrtY?kP`lZhSV1 zRe#*!Z=AHeYsS*Fz@p8Xj|0i4?U`uVJrpWKd=W+bCyjBDi9U>`>_O4dVBWkQ9BfIE zqsvr=W4tmrXKHbkWuBkKUvIJ*G0-r!a@8f`8xyz;WddS194$>brBIW8gj$J%w;hS+ z$iEy}7kA*BM&tM0VMKhloyBNqYe%z)J-bUz>%DjlcMV*h2EA+>*tzdhMPgrlo=8x2 zIbBbvb(Jv%V4Q#45=D2O&Ae$5k}G%8Nf!XQbZE^EsO0$8n3q*oWr*Yk zgqi=Zr9CLZ_uOa@QNGQN_pCS3yq+*IUZ+T@r~#YxL^SB)xl2Dnxc+OyF$fXw?bF~` ziPHY-x|~&cfoVcGlGP!TF~NGO*i5XJm#s}yhZHSQp4;fvM|(<1B5AppmO_#i16}cK z?>&DdArJOaQ1Li!0FnHinDDmT*zzY95Ce|#2WR1Xe0jr(KYWgO??QF_r>QnAw76=t zzDgn0L9HXsN!^?8C{Ej+d%Ga+6L+avM|zI`w;8`59dcthT2xWB1`8;XW!)}aa%bk~ z@sUjSrH^(KC^ae!#~(pD>NCVFxBDx)a5O%=@u^Qy4U_H79>r|tb0bK#ygvHtm1TXL z|6?jgo$xD0Siq=zByysgeWSeyX>=J)4e5v-hHVr_XtfoZ3e&kr4v9n9;JJkr6qzH`^LGMgHPwA=hdhhw4jWF#oK{grJ^92083vKglQGQ%htL=54TAd_%$H zOY;&t32(;d-r~3;Ym5%Ym;Mfsp{~X!YO7rPM90m#&K;2)VJS9yWfihb1ql_MoAZUN zXV^5o#2Ny9v>!W31Azj^4m7e_a9QR>g}-V7WPX{wXw9)xQnqeL3tGJQvMo!qNjo9< zlimN5BHuF&D8L^%XMa_3q4Hmn6L$tgJj%tBf$RCdzFAX*M#nxkSk2?zJDMF=a;L9u zBt_SG57rU*r~t&N+eymNX^5CFPnYJy4k87Gm3C{T#h`C?d4sw2SMvZa@Ija0lu zGh3SN>50iUyhcd?PD~9)01@**hCP&A7@IhvYdD&IMPn{|C733(7m-fZ71k zvQEk`BY5kun`P=HWX0Y*xE~zQt%weRA|=~vRjZFH@MViovUZpii)4gt>R*n5aU2U; z%ubQSr-A{&L(eYc3pSWFRlgjRVk2lOao$j~|7?&u2ma7EpUTmSApg)6$Y?pCpo)Ix zQOOrqQIvWy%a&Elv20WD z&ln{%LCF8_PD(!(&RwFqA4{oJU~#cQG*UHu;EQ&k>*Whtr6Ar+Ge=%vSg{2CH22vVyRSF zR{P<*8Gur-6|q3%xPaG?#QnQ>U+Ln=)Tr|oC#_UD++rGen?OnbtL0g9W``4XY_bXN zd3C$zU6Il8*k3*LU-3TIjF9kKs08_ZYkqA#DNc!8P=|Y?5^3in70dm`d1cycoiVlg zX7g>v)cK}vOS|keFaEHMv>;g)i+Ml+%4XO;$6A?JUbc`CASwgp7MSaWwy&;v*pe#B z3mu?Hw~CC&IB9=0=0>VsRW6#ftP;ol4g2DLW1-P(ASPsP;Q?4s6bE?ABi|S6@(iZ& z>TkDI%_Y72d_v83B*^jiR*@(UI;0>NWUr|1;D*{1* zL2&{7nAT5#d{Q;IOD7kqGHxnK~Zj4NL5=HK3}zaYI4{^njP)0*e1 zKOtGFb>5MQ6Bf{Io3oZx8jOzR5#ho@1qnLR>As|axpycT3BvrNbH5#W0P`z`jmQc9 z*kS!Q%p~AXWiCzGHWfr|+0A|@Y^y&@@ zA~# zH^tiZJ6#y2=NQ`)i1m=ec&40`4o?Ijq999AGR>4ow%Sb#3Fx-Q`-jI_$C9#s1C*)J zjwT_Wqp~P{&krHt`W&UHFcit;9|&QeY9_^GUly$q{m>TLdB|M*)}x7lR}nqFm&1Bg zlS$UCkT~%iLEYfxQAPPSmWNMzJpSEnC2>_qZoKyvgISjqe-ytQ5ShzETltLHl&riwRHvM$XQ zgNEc3e__}70SRF`9(ML1rP@x}hD_>6-j)8aVLANBaA`jS%@Q{D8tueqsZf<3g|Y$MS6bLIgZY^6==#DZEvsh5@{c}O_Z-t^{Gxam#2X(H{gOqiw?xce}AkG#RU zF6n%TTYNP{UJd=0$VwXkmeV`J?2{#^-7$jkF^4Axmj#2ikbE02p39Sj1G9(7sns&^ zrmfk@6oYe8-uy2&KW1Ih&KljFq_R6&>0Mp8)DXYq-_|pFs)Nr!SE!=bXq3=!%~k#E z8`V9v-?ZM=8eRuwj~B^Le+0_E4)oZ`3ts@dW`Legx# z+QS1u#GHy!m9_P${14QbN;jpmwMKI=%+p~4->Lw=m7p-?GQNU@(~ifu#)$LLw33b} zi(=(V;;!!~-B5` zcer^vr1UjLgdSuTv5{%c>t2zvdZd^RYYn*J9M9;|kmf@(BMmNi`w-t z4ZSdlC)QQ71$1yxzd6(<@Gz$r9;jkHRL)xQB}(WXxSPW{*yu)H7R~IR`F#Gh)t~8bw>$G0e%wkm~H5??x;`w?|rO2Q* zw8BC5;L0rDD<@#H;b-{DrNm919md$K_{Kq6>0@BVM%em-ks1eOu&rIi!;r%VclSE|Nb2zpK zuimbh7}HIyhgj#eIb;T7;q^uz3Ez)>O;u_hlz9pq8XzLVoczEv^)th(o)HwF->* zl0p)fK2K&o-iPm$g!ILvOD67&-SB!CZu%H=R52CvMO;qAC8D9nw$D+-B=|7V4NoV5 zoWax<(Vb<6{%Ug2;Bi8Q=2xOGC9edV0NHEW7WQ~Ni`LZ=I^bn^Ej+{qhPB^AS&~Tr zwa9_noFFIRr*t-YL>P09u$6_w!P7m1z7rG@Y5Y(HCvP@ogH-XwzBu2w!vFmjK%o?o z34K}-SXsZB2<`GumLf;<1M|-?Fhrw-5OXk?B{(6K#}?sEN*qIUfu|a^uRdF;Vt+tC zR5Rb!`kGussL|EyQQWL@b=JDY@{(c(L(|_dODvU3<28FFjI$w&PI$dn*(JTu{(hbE ztRJD0fBwT7iRD((esDgK7iMB_kz8%d^9^=o1b>w7j`j@#Zj>KLAOo89r)N&lBmZiUg3Xb{ zm^vpT##&+_yzmNs451S<$dL6ouxqPJk*nDC*NnjwH$fgPysY0ush?nKR|KE9JR32c z(FX~YT@+w(GaWsTFsfLa;^|PpvyMlsmfE~nLqQ>bGpvIdNER-#_q#Khr0k*zE5p~h z#Xt75pJh(aVRzT3UrnZ7M-+3;sw}=TNSZaBNDD2Fs3zIb_`KH*5{0l-uieE}(*qOp z2ry=V+Qaz;Tz zN}HwKNhbD+Z99`>Vr#{=ZQB#uw(U&JiEZ1qlZj5=Z{zI0b*-zrx~iV)T7|nwIcKS> zDvq2}D1QX70q|zXkeH-TiJdoO#wrJ0OH-0 z#&Scri;iD4H2Q3nY{c47th1R;4v>iC733sYmIlbi{O5J)TX3M`%_o%0}M|e52YMik8hP>~`ikEL1_p zMppJ_tt>G{h4t3Q#lMCmPt6JN*3zRVDRo=TDdx%R9mo9~Pc<5pLZi)e+o=Hj#9Zhv z{dot5VQ;C?)54v=jpLSU`!l+7|90%zPp%?|1&f)j&d}&8Q(_E1G<&WK=PUD~)qxx+ zr{ZUar+hbDunl8lMK!L!{)y-6e+J1wLukrCI5xqk6`q3YG8h z6?i?#ImtWYtx8%lvV->2KQE*sfV(>!87A8^zU2j)?ItZ1=CCZPmsBsI$YaBs5e&Rj zEC4x;Iii~xD2akh9U=+Bbt?<%9<%xMLUhRZvbf)g-~wB51#il%lrgvEzHtsoVeD^J zn?q`AbcHm`aAZ)*&6WIN?14+R`(6W5pp|=M|v8Wz~p9 z8k2s_B)tXmckzWO%oWG_zDlcycG%$UgM45A^>5KwrU@fT@gF&Zd1}q{yf_s%ouCFf#-+ z4fQ#PeUCw%7a-;FGHQ=o*HtR5bW*_%gKXs#S)WCnX^ftICNHc(hAzdRU(B_jsK2+? zzsNZ9tHz%Z-vnq9Tjl8kLJJ}DF*)n)Rh7sZNX#`%T`4pg$a)Z3UGjaLa8<~V;784r z<7xlWHk1ayEL4!3@cX+*@;l)^GL>Jg`Y&iU>pz4SjnagZ@-XLAT51@rl$Y`wX5f!& zG#oEN7xkX1L#NP>2WK23Aycc}iN`(ArQI+)J2VNE=Uub+@#chTpEc?AEoP~;#YQ7y zz?3Zt$6KTg7rz%0K|#xG*wJkf+1{E?X(+w;>1^ObR zdgI?P5nm9n?;2PSWBoI8l=~rjZ5}2_wIb+Z7ww}DbJgy;c`}E&{ya#C6q^`0Yw6RN zq2+N*1T2zVOM3Ajdb=<>o0`5!JQ*irQ+@l(Lbg&WW-=*$AP97ES{*eirpGVsDjHXW zloqBar8(^S%oP+x2*T?LYsCM7*^NooEb6FS!HyLC4H;~t+Kl!Dm*apgKE zl*$YHTPg=U&y36D!Gw8br1!@8 zDWjnkc!`0Wq2EX((H+90oKen!`lXXjCv#hWG^hR9Cuf7Uw&fxKE;p}%421SZ=pug= zc6|r@C8>AvaX;l(>aq$6v*ih5obn5d?>=hMx*)tXAlSOZ0csyMEBsW?BAjSf;(W8H zX?3JyVw9f&AS(m~OE~uzDTSYT%5D&};&3V5e}#6j?q|^FVZR!6pIYcid9*lx73?wRPH^O^;P#A4XTP zFuiEe8~qaivdQek4=ojKx0k0eO&Tapuo?pB{GLqu!-TIU2t#U*Nt?Q5_IF(=ugQ&9 zKEEEhaZp4j`_2N|iHYCAx-y%fhKy!&UTy*HlSHhmv-qm|2bzWe&V6$hYQsOpt<1H5 z;ebb`=)}~sd&zH`Zw&>!>B$ql#o)iu>PZVx{b{|}Jz!oWEkf9WV|VGq!P(9WapZnt z?>Je_HOW#aNazNsnpiLuig_Q)UiP=&ErGZq_UuyF3PrTIsIeR3K}^664(rqbH=01_ z*kBYc!q3g*yO05ejgqzG#S9uOERr$jF8hB4eO+LKqjg0Dd|&nY10LesUD$q_qKuRR zHO>%5{KrnsZHew%O8}bWS&3;uXo=`k>vV^W@6&e2r%Dc@8pAD5Y3#J#mf+sWQdG{kc_-~ss8B@MXk-1*L1ViL_VP7Qs=SZrqQ6e*8%1Ti;=k=(98zyARp#pp z^lw40Gg=^#`&59hiU53H)$f0sFj^2qEYAIVyv2qNn_|`U!MT)f{9@4>cd%qrn(rwG z5Hf)iIX3#S!YhPMGinP%*LLqagSxYss|=-_&&8duq4!uy+rey36&=^O%ruiM<@wru zO}je#9kF_lFi45$sV*9C0rU6fd5CMtbl=Cb&YgYch?EYNSmfCck)0d$tGv@)k>>}( zFtFj<^G%(HS|5lgn=~ zy|1=uRU9MhKI>=XB0EaPvv9vJZ7}hDecA65N|0_+gM&t=1E;P9$PW%*b%l>QuP9X{ zY0Bu^L{wSGkTXc%V9d)6pQqWS(wac8qA?JyB#bk5qn6rIqbYD62m3)hB8eo+pxad` z7kWFqp7)AFJX*O3dRn^53fViuPgUi;z3;;;!;5IZ7AF-5u{XBxIO|t4WGy=xy{Ipn z)*@PEsfZ;_x(WxyG^=1Yk3I3P=+(mkT+-GB>HYdvQ*AQj%sc+(bHlyiIua45ZjaK; z^iDC_$Jhn_usM=Mtel3I&x{?Ci`G9Y&9}=h5-FBmjV(*HSfZ98DW~PfEUY zXt;68loWbzfD5Q(jnN5fD?0Nsfp`xu?&1||bjAFUa2W&Ci4o#8N*;g3TB2SOyTYeS zglu7i^>Mi;-cM}AyW@~jETm83u?zt04h+e*s@@}EB%4Kx&(BB~P6Zf7ova2{-F1bt zY9N@4wVui<$mwHWTIn49Mypj)8=CRx_vsiK2$_n)7#kzqOOOH|(@TV9BS_$QywlIZ zeOL!{WITK96#iJt7xpEJIfb9`_uh+`r!MwYi>{j%&-;lx3{7Pr_L!n4S zxQ*cxEZ##BxAxfHkXtHJKBWr3%Wns7q-lFTqU%>q#Sf+Lvf3E0xG(UCpsLL%#;-6U zW_1w`@i5Ew<<%7)S}_7qz|1euAskR&NQ+AdCVnpYDM#t?HRjmY`xcM#Q27!KtikRl zcoKy>VjZa)$14?qa4*-E3|_BLr6nieBxU-c891=HHc#4F+d!9p)pfqI(TVJQ8NcR! z{Chun7%A^D*r_FBtAtbb%pP+bj~+p~vkQ*MQZuGrhhXXCVSl}LnHZ_o%0gGX0Fd7i zZjr2*F5Kf`NbfLzih|IdXiPPjH(ZkfPRwOMrZANqE1p|{eg{~Hj1tyK{}Q} z;Hwkh`|%a@X_3vxl!ZXHZ$)U9yUl(M_txx{=o=s9sY!)9cNLE(Ca8VBBM$(wb_bfX zPsbKw{k6L*`%Ctp&(V{%(3nD?aM|_mF%}KMwF54n_0-rhI7%ymrfVL-QOGIVpe0aF z*$gQk#!Bu5Hh)8g>*76l6}leCUM-Oi75$Xw1+MV zeIhmhDDtE5{qbsR#OA7&e?j;N3J~hO%=6yA^F)wxsMvSLo+dKj1v?NGp{1~-EmiMW z{Y7iO8;hT*R6A-?@1lW~aS6=-5ri4b$4bx$0AcdE2e2@k#IkL8g%fF6nXbg}(Fjvi z4|N{T&D~#od>405ru_tr`D#UBd^{CKWEUIg|46$XYsUP|0WuVtA+q%jHppi_>-=qi zBVEEidUs;z+l+ME-onwieJo27qr*OXJvC*7a)3^+al3kh8HWSA$9#p~S3;pTLvgk2kz&at%0v9uMnGflH+G@xF zF=XVzHAY5yP!b=a=38MG?(h`4fQ+q!lR3W#1EQux^d%f)5!^~pQaV7f+Rm8AD8y4yqyFVC<|+6U z8#?j`_Vnz&&eN{$mO;O1#>`#oPD^t&`k6^*08OS?4{rSqEiYI3wod6X{*sINe2@hlsdoOWPHa>Bn6WB~$WKs@O3*x`3 z;C~*3_}i^4U_GE1Nb?mXc`SqE;tvDAfGNyZ)Y-*^hxh}Bl=Dl1-JHC#Gcq5k8%MT; z`WLnl4tTkL;(3(QqcPu`X8UoQ$2Tf{+)57b8>^RVuW1~{ND`^E73P zrBt>1mgCLGtRvF)8N^k2p{qveRJ+&<3yuBDLHt@&3ZZhg^cp@-^bnmud->Q0ap0u zKrv)2o+S=Wx>A;y4m_rdEFVs=p-5^rvw@~X&}hgy{(y)7GNLu;Tz9A*;>>^!{KY$k z)7ihn0*(LhSR%iq-hQ3k%w&o_)>Nl4(ByHwrq+CozEbOf2EU`b;>W}+8Xr-TlftFV z9Ox9EHHEZeTT-dTPj^xBfmwmlljr?i-Lc`P@ar`N@VZ8lI3T`|el*gB6JZTZ0;A{s z(!RSBlnJj##_qlq2zViGT`rY)?qnN3IP1 zJ~Juh6%DUzgyU;xqjq+rWBy2s#vY2H04MQ;Cpl?7ij|zD{qI2d7v!7QlRoMQb{Dz% z+xsc+P8*>Tbd2yfXre%y$&>l9@PnkbC{Ee@Ow`-#w_qRNzqpYER~qL9m-`_dpTh`v zVrIiIW58(eDnvCH68%heK)UPZO#~@XwHRCf)k?|napi{Itg%F^S5T%YI7#)Zw@y!7 zya5+iWT{S2A)7y(k^Wu2^FD|Z{nKM#<(Qe z#6W1WNn-=UZ(&UQ>8mcms!%ZF?1_&;P+om%G@>HR08_mr>4ge>5Ep*j%)pLv07>>( zx~fkz<2*gdHI_6R_CPQ_9*+KRx6L*<=&CWny2#0wfDuY7W1?0~w`?Shlm zFi=&MYm9-3@QLi+XQT3Nw^Dp3amH}oS3EpAm~=S$t4+gnvjEk(6VYAYn)NbD-)wG| zK6H=f2H9Qya0FR0Ex59%_QcYPBL&=h6+oL`kXIW0Aik5*5f#tZo|#B$gxosJ-H2ve zO;{Eq4HYj_@ZbUYZ2W-xOns}k z`ODv$Bq1?3CEPyjEfC5ZpO}NAg0DZlk7^pH{;N5%TkT^GpVEmoh?wKk8MzE~LyeLt zM%^A=+LQIBG?NyEyuP-B(`_1mMItE|i(|GcvgSV>k|1nQV6>mtRO%?TLFsB?GKAvx z7Rgj4NO<30ro>y~cm0ls!$en>g$UU00rzbmUo#A{YM}tCyBZz2%M4r4wWhF}D)G|= z*z}%&s--2qgdkRw5d9DPg>hvr{nh8qQsLS7aS|NHP_0i*r%2~{+7b)g!>)}qC{gs} zZeBP^@m94gkLb+uKnx@W3<1%3s`53t0V(dUCQGR1=dff62UR|s^SMxH3~=@3#%Dt* z-3-05plWfxajbO+_GS>-NF58hn2VoaefVwCakT2NSw@jY?qr|$3Et9Q@OMT;6boeq zi8((cUCNb>bJ!pGZ_vJcASwi?$r7@q`MH$`G>*~D*hk_iS+z)ODcfgcA#Mpy$RE@1 zjUZ?S#g>Hd3Jk=lnQ5Y8q5{V!Mkh5yH{q{_GzwfF8n(mk(1CKvqfz7PSnN;Hr=fy^Xx@{AhWbC zr`9QpB($x!gZI3V5 zz9Wy^^C6C|3|v4d-2)F5(N+OID>0?o%oG5@n<0VYg+oEug^U_Np-BpKr*D3$pqV#Z zFYVBcsiVz^r9>(?+;Ba?1Ue%KCkgKAk=FciXd=8~V+4mdBI7jR=zZq^5Pl-#G1X$S zjdRS%mxqnBXDoH#*Mv^8ZS9kBvNQ?TjeY^qN)7!NPQxzzPd#F}<#$PIM^et-N-Z@l zXy~x(EXPH)WTxtzvYIbG1Y|mmb|V(z?~UL=&jAqR76fmavJ#_>sYrl zYoIXsUg3J@x0tB>)c1CABVN+~)$;QTFdXpf@f9%xLLxnDU#P^dt)5z46FO$NSA3gl zcB(ZoEw8%iP&~aLB??pj_~Eqkn2mEpLmuo$kf=-lW}(GG?#s1Mt|0hp6+Rcl-XA~4 zYIQ^1NmX7V!o;{J#S+1f%XaSf!fI_I`B>Ti?;cUC#T=r5y?H7*D@-IO*-VaPWCwkyb)jpM? zp4Y4Ou(|n?VGY{??vEXcx8vao)uWXZK_?y0i@`I|19dEne}g!qvVskh+|m?Ewns*n z3gRw_=Hc40b3lvH>)2s6Y6W^un_}Ud5qNl~7=GqEXdfGiSt(KLZlcnh;d5zIfPV*! z{(0U{kn6>MUZJim{dIoj97@7pP;2E_>xOd2ICI0c`hU^(R~?!G)NhP0Q&=CmI8Jzo zjKGepIL|?U;=NkmRzvmPbAtDqAt0bquCi|ZV&-|xHa4g*gcQYHp)Yt^j?~6CBz2fs zmWe)gZx78JvwBqoSJ{O$u8%(q7)xQv)-*^xNbpUElSB>^@ic%b38Yat6q7 zXzDEN^h2EQXk>j;DCl<&aZWYY3LtcNWJJQq>3lg3gYbvBVj;>GrdpX+YsUjwgD^SU z`adu|-LAK-$fH`0oXWs0wu8IO2fp|wJZ;`+d z#uE4+_`+9ts9WW3g*bvPw9~`wE2G}SF zcwh<`4`hME?a(`ncK2q$Dvg6GNBLf(A#g^m@vqLVApbAKSV2El-++&t&a+=J1ugAY zZ89aB8J<@gExR^WR#sZ2e$zt0U!?O$y%-8mf!jzH$DLS_QBj@)oll$A>h&Ic;!9Va zzg;P^_7iAZ^~AGhY@I#Z6}K)gI^LVzKisRT#hx)}y~cu+`15}+?9|HkPRO#BJBL(L zt^oMkvt(LSD}TM-P5JR(9!IVeAmK~UDfJvJ%V-ZbTCf-g0_vfVk3XX067fL%!<)=N z9vr6x7Rr@72T4a{I=AD~$Uu?(B82(lvm(H*70`U&0RcaY9-5UPq}pPs%|qV`NZOxO zxSu=f%V&b-?TbdW?CbY%S_D0k>F!$OA>;J|g9KRJf1tJ}hC=Dqk-wg3v?ALKPH~tr z$zFvjFXAvz_M9`}v{8g_Q>fKG#2z$h)} z8|xBpk`(XfNq3@y8mBY}Ar~)x$}=bX8@ZHP>6NYx*Optwnh=Y1_EJn**_KAO#zIU& z-@hlVu% zm7iqkZF!B#B@Xmbivp>p>z4P-PpSCxXa1xdWRS2wQ+{8l)_jnotPMnwah`#`1HT`d zX7a9kMB=YCU+0H5%kmyY9a{tv@Cd?f+VPfaLFmg{`4zscz9@+c4v2_{Nc7*F%m&3( z57c7&ILvvkce>S?CfznPq;swRrkFQ~syOw-vZR+k!xDF~2olRq{6~;XpsIK)L_PdR zdQ-C?kfZfCZ+F#_aOT<-iYF%Be|v)5w5cTd{=7>b%@1{zSsKD~=oS!ieSB++2;bJ} z)L5xCMf^jCQcg-Qorlf6yq7iK=^2pl+)@SCDBXLg)2_0@pK^<>n8Pc#&7% zrNk6uY4sFgequXbuxjjj=Ch<|&c?s$gSoAKh6orfcHMFZgWT<7FT}-s_Qzgo+M{=H zn9ta+H)h^v_sCfpK&QjSW-FnIt*`KlFBaxTA61{Nh)Ay>LOb8#8VDFa6t#630u%wMka9LRO!%KcYV8dhonWTMS@X42<@K*@nyvw1zK z6*Yg)Ve>rmqaQIz4Chm)fSl{&IwIFoaCVP8%?@_gaW@dk=pqLkX}?69tF$H0TC-H6 z2`oOIzD%;6ivdwj3zrRpOPTlS;(LGHUD09{Pw!1xt*V_ijPtp6uOUL`ktwKjJP57-U=d!Zkjec|~h>Bh);OKhpCO`~@ zd&kMOhD8-hl862JQY<0B%nQn_vF*f}nA5)*OV6K%}Np!EMTziq^>uUvm=}{F5gl)bmD7~;c?34^| z%S|`G+&uAnG#BH8xQsAHj?-56>{OMCcTd-ROQ~tD$}|0yQ^j?}VEv?WE>q@}S^^o@ z0%Y~98MjeDU>_eX^~M1`<*)kX;#j0G5w;)}F<~b3LUEvbU`6|~ znc;W(m2U!C!hxpow8~B_go_Wm#S$_QrSyvO7NiOPCzx0$79KDq8&?prpAYz$5s@>3 z?I>H_9={D&2^N>yK|eviSnI7?d>huWBQEz;2t`B7~o97eirx=mxYv)(N?lV2u@8j33 zeqJsOUJ_U#`O@pQsn~Ych2-@M)@q23a9fE8M#qdH>Ai&o@o6e<14*a;9->;D(C*1#m!??$4{pBQE&c1sVg;FuFjTPDesGiJ%6t z)HbN>>El+x0)lT0{!;(9;y>2ck`Gl%73N#-&1Z#k1FMCi?&}<1b9Z(-%jEPRh08o= zf5?6Ed;YuI=ltf~KIhZ@atpd!$&uM73h}BVEgN=*AZ99zcudSS6jQ=9Xjs(@xws4V zMNtCYKR3@8<9@-x{2|knZ%!A|&z2>A(^|h~>M^J4W(DE2@8Fxz84JbX&0XaY@HZRt zgMYIUT;c&jZ2!vsQVHZMlcHKLVuJ^hL@68+bHns35^N(GLFkDU|6=-ImWvERoGV{; zBNbv z<8Y__o&@bBXl|5;D|uU3U4Edh?L07UdmP7&WH%sCe3C;P3D0)~f#i((k&BqPp>{3K zg4Un#KL-7eOhAxAXCd`3|BN7I%VNai+F)ERw;|=sx~UiX;;n>(r%{|KC+Sh=JmJMe zE7dK;2A)bRUoZkQ^Lq`xlE^wSNFdM|S=5{iKDJKcj6eUEgnBUi%BRmgO<@?@wcSw=4X3ClzJ7Hw#6ll`wEiaJ?;9{{5qhJF6 zaY_Hk_w?7F`F)p$y;iI+KV|WwsvT`)ydKfh1REKf7>ZTT#mLH{OhPt6V=}n-0rK;& zMHh^?3+3RZVx&D*G) zK*S09OSC4WVoW*F{l6-hmP0Z_$1MlC1q;JwY>y>KVdI%R=0GK=K$#oG8;Nax|E~!@ zrvadruxjn5QKh8Jywx>)nwO4O;o=!2GvHelzRdA%ffz&(NTHpyhHB4g)lu0K#BG&+p9#ZCi!ifLdZ2L%ze|cv;eO7+O=VLKLCPE=6tPV4 zwI(}|{`YjTP+IsnJl#KA^AjU|`{D@n7ct_0&&E$Y?SvF-RC1yv$$|V|H2}dt|KaGH+&cT*0(Q?pudH4ll~8FPJn;SDVSKyfdk2|eWR8O*UrJw6GyIM ze)^MklSizt|= zLzucx`p2y&Zuz=B3+I9pZ{_WSX3>967j>xc{)UGpd?cj-k^J5OvF3UnykHU#j#z|F zC638vLhA8!>;esqNbt+cKf<&{qbgW~8)Pa$`16lftbYf)_l9=}z4QA;6fj|^Nf0mT(u5H-piKyFp2ep-A9PGdl@Sfbx z7M#h6E!WnGu(wRFOzGylVWLwNB9`7?g@sW4!N+!6j4j#8T*6tp6Gv2-nVp#v?O#W%FPh92{UeN&A{LVo>bZb5&AKiud&xo>0o0 zzie9u{G?)nnq88sSsQ_@!)k3Mn-(4Hfg5-HI}DOJ{+p1J6eO#yQBbND zA3p51rAno8Ju~`7*Z#0vC>&O0uPIw}orxGZ3t0%sDGo*M4>@?mt-zHYLdg{#x1Npr zwJHA}b4Q|!VfBvpD6Ff4R(Xd>GAh9(Bf1X`Rs^BdAYv(BKUWT2xKT;iIwKc)pUw$C ztt;!7)Ax~ZQj47Y9$Dl+c+Jvg0@dbq9H$@2b9DX=>`RYVf}ZVvE^g2s5YPq?S~ zg~dQpF4Z-(njtql8H(r}_9!l!tl(FlS9U=m!RyU41`-w)FxSl=XawFrQ4m5_-sq(5@Ojrg30=*x(^*2*LlHy8UzfO`J=u7^Sb z-zothL~7{Bcm+(Q#4$|6q3u^4-~>LmMGlpsy(IW1g6~ChLPsSjuFI8n3fjgrya88y zE5BN760Xk!NC07==W2h!H+5MNae|-zv?PxH)P8ixTyI61>g>kvX*26Dvkrx_gKW1t z0B*FRGCx_)d@hDu4^M*?Ug<`3cQ@nOdUa#eY|P!^{*Y&?rRgXjV~T)P$TMWsSB74I z4&j(tx$wGkiN8l}z1|HssBA8;byDE&-=|PS5&qSkvmwcB|C%h3Zc00q1&7ss|*V!aqMx}W?l3Kvn3dDcpZZU+ zrR?7Rs3c*E<=nyhY>4$aZP@0OFowgFJsAbdA3J@`CYb?&VPMCb?KVT61U}V3{NOB-oxB#FE6Of6Ul0YG zXEK{(># zg2~uIx@SPS6GTU%1=+u$KsSdgq$-p%j;PYJbKK2dPis*`ZSW{V<==e-*rzTs)%)k_ zay1~KY)&%a>^Vi+D7 ztEe*r+A-p{yds=0w?bAd*(%LzB&C_7R>BK}a-c}6;{GD`b6wG&TMB~Q+MAx=d zyWI~Q*$JH?6{XHu2pszH49`Lt4vePvXa8geY3=-u+{rK&MRSXhL8F@9SuUPbeq7ntR_z=wB~QI-#d*104pN{78nZb;F2A>MVHvl9tt=zIk2--D0z15!tK%D_`F?EbDaTsl`kf$_U9%zIQ@FN({*DC1fDtv=LO~Eno1zM>@;~*gpua*)#ST~^{INhX9nZKM|ovcdDRHs>WFg!}%6E>$} z^cw+#4926rH4zm>gCdBqR~U_wV)Pd^d^}9X_4No%f8CrhyW|7Dwf;$Cv8aOs<5)-? zqQ|_^8T}wx$G5TZBA_JQU=K^%I&84e`Tf1Hat zBfP3ZRB6Y>3U9<7tc9qvNbAr5ms361V8%TPgojJ)W>w987jtMYDLnE5BfM+o4IT^( z)Qp!Fm92Vw2vP|;>6Xobx&-ynWM?{O=($J@iRY0d^|X!)GGwO}XE#o-L7z$&W_`Cm zh#a{LToc^=L20^M7PHtHNtjM92+*>}M&@=npuou+y44H|i-=YrrW9zfhEf{HO%5nU z1ZDmFd#V^QzD5d|{t-PHsxNfxtsLpf_lAtP`4pi6B@1Pf{S&(8^GfJB0WXRk3L3B4 z&wSHW&6SaW%6hcUT@oU4;}X;wi6&tzI8%}9W;2pNA9xzEwY9|eYrgo0=Y(hP05@>w zM?)Q4Ds~~H$DeEn-eDxM&}*pPuZJ)^v+;sRdF*{$mSo~%``YWTSD%cZuW`lE)e*;2 zH~GM(bO@qk>;;sl<{1MZF8zslxExss7-*?`#X>sUVyN9EXNuJEeb|0N%z20sMlw4s zpaUh!O1SOW5gBNy22;eoYgZ~X#7qr+Cn>O#+yauO&nurI8C9g(Ea!N~N2yY!0eMUf zio>jd_#e!4=-tPF=+f=epM>Y%FGdiqh6gw9B^=5RTJo~OSe;lnG#1&VpzMf|EPcBS`KEo3OAN4&2L^+c-K@UXwfK>$VIt^;D>>zY z)x@!QSeM@{Q!Z-b+=NH;pVb++oJHdFS{)N;hF`aT?4(rkr5H%nM-P5saJbI6i4l7G zr~{_W8&g?sJQ~cCDv|JaT<|Fm_78U|Y<>SEhV7MS7t^@gsll!$e&brg7ggCYw`#&> z^l>{I^|Bd67-0{LF&!87F@StKEL*EU&Ku2iUR4bH$svG>mZ(xr6qp|1!GISgmvKBu zwQxhg9kM5$nKGZb@T{N{S4nn|+DQz*!S{fK*JFqo4o>7DV?m0?@~Y=xxR{LmcDW{1 z$*W(TGCCv@)6!|leA7Slf+j5Hwv;OTz8<-bA$`ub_os$;e30O1pGeYOv*wt%b8ocH zULPIb=Xa_9F;p+vDOAzlS=FI9TFC4-PRObE)BvU5d>KSD90@VqzBxgBnfFL|JQOE# ztLcIEutoh6KqMvFyX6vfH)m_!B2i`Wv^kT(@-*|7q|0^Qd=hd)aEkB|N?e7|zSHk2 zksc>-UF$j;JJL!V@qaw)tOJhlUi~K7s))+`!4h~% zIhgJb(rIH6^_{_$f3pJb+OC^2)dKmfR>m|KeoM|+X}5|mI==8OUQGhVmH;pEVXAo` zkzPI}QElIt5vlBncftJO7+}gS3zv&@-xhpR;8sI@Qi@hWS%2Ssy2*SiL@a0f^|z{w z%}7<(Pog)$|LNGqWSgr@@M+Jq*Q$#BX#nRf626UnMaNSndUy3`5EX3jEJ zM&hDI>_=@)yGWGCTF5#%=~~|l6;U6XDYiGqw&hX;HB3jcg5^15GzvZ*3gsO5n2evY zXRb5S?pS6Q&-ur*<`54{~h24fIG|atr>QP-1wGus};RQ<;bwL>ro)QxgLTT-NNv5+Yz* zNc9<}HL!#Eo_%%amq>3_QRvp~`y16PRFYCQ7J#M!rTsbpnW3ycW+-@--fY1G8kEO>0OCRQ_CrG1BIn z2;5)t*LxlQ?rfEY%#3IZ70tSo-e`iSl10rP+8@qq9=YO1!Fy-0eijdpDBH&T?^$2Z-ama+2 z+JyEnd-u^8U_G8+{waM|vY%kk!xN-F5*G0L{(`H4jN^d1QkS&mGDAKxtNFw_bYCqh zr&v_TzR7mwGAVfdD)M|mI96gU zM{I3at~XA3@M=v-^T>!NB!wg%-h@CpP$Ljc=j_ROxvq1P{V@y=0G1U$-qGAJpC|9$ zGZ`GB6=R9~sifXRSO+I^o~OYY=oF|wBa`%V94kQfYa@~CUJ7YjqH+8w0Ol1)K}Lm7 z;4QdxQ;F+!yTDc&b?l~>n*vUPiAa#C~w*CM_uiax)9sbxBunm z8Evd#S6K?d8Zlfzib27M!!~u+L=TKutH}#VCHRV+=ZIP*Nqsi90>gAh2J`z4)a?4l zXw4qVXQ#UYm*_Ls-aGs3WS44T;&;k~A`&0TBgvspzEZ0;`+xwiI#GWhsRJ5%Uz`VN zh=-DHcEaZ8{bPH31xU9|#ejAs{W^)(Sm@lYkG;VZIxLJhukdUJoZC)t|5xh@hX8Eq zBuV~_7bMTBXJ@?5mRuxBLtMG!4_4Sg?L6#;>qp_MABiB)~bEv&@*SNXh9 z)5LR53goO?#S_e%PVkfaF60s6uu1LjW)#Y@CvsV9m5QR#0&3&0P|+<`*_R7CoY~%{ zSn^7gwe=F2Xyvu5u%o|`{Z;{`nLHL#6jKskPiunhq;czWAJVHuesOuqvYdM*kQpH83KbL)Pwj6Lh zot(B`mUip@WJi^|C_on-%Y)xiF+Cbl(0U!$m;3gMB&mSIR2C^KIWc~nix!NBm`5Sl zK?3PVw0meQlmYTkn4ywn!tRGwCn%b;yP~3h9YZNpCWX1Av5<)$TGn?F^LsRxBz;u~ zHi6>Q!6z1dhZwzMGs^RYKW;LVpAW~=WSDreaobkJ0VHwMlO?0^SbYoRaIHF4J}U$C z6gFF4rC#L^$HK(39K)axk=5zT5 zdG?6fXLiFSw}~lyJd9|YMEWw&=zw=ztUyc;b3c2tSmS~VL^3O@wJn))4cP3OQ#-f| zmB3UfNE!5&n@N1;;s+QZ3E+X@(!e)P0)<;=g+#p)v8IS$OttL_zsO6oCq{Y`u9AtT zETb6a-5q@v;4XR_(6~wXdmy}kt+#0JlB_ArQvR3m70cAvv0SR~ph?|r6(++Dxfl_@ zk^@aw_rpeA>YTL~ZxcUCPyXU>3|lz``jt%$vmo&jZ%Vuh(`9EPh3Vdhmyh>CZ(ht* zLp3|br)0e{W4g4Sf)T@*E!bG)S2WkE0?Oooanx`c+(VP^vgX(ow=f)f^i9*OoRE{h z6O)uoxiVX-T2%^7+#WD1$47bNTs<(uOEdRH6MxHP8W;IC7Oj`J#HFYoe^~2h;*?iM z7i+|#@yF&D-B?@79MiFo!dUGVy#zh`RA6&q9cefh(>Yoeoms2KXuc zfm9YiFl@uECyas|b}c7wyG)9&MSH;#O9^XXBF>SukZ%-DvZZS~0o1L4!WTSnOfAPI z4}J3#Oe$iz9E2{f9oIoi&h{f(3tBqY&?z#Q6|}TL|9GT^@4$)?rM=M0eP29#>NnK$ zFBW~eFL!>6+&cZ0G>fx)nzV6K0h6^C8LquNwXC?Jgd*~lL?`#qKd8l~0PUR=?dQ5t zHG;yh#DRuA@+EuNFMM9GRhVuI-+;P{rA5trm|Ud&eOCxBw0T>fcK3%!!d-GPGeO_3 zd2ZV?gl$N|gDR}5TAbU;3rq-Xnr4#50Mp;AqiH0(sMWb@AY%$C{N&wiJ|iRI)5?`H zKDnUR(XvYpC9I4O2gQugOY&0>E;mKqpin@aMLEqe!b*+bUnIJ?5OHbR(mLu%5|8(- zk=f398uLF`07nz}IQ}6dqp|6i{rd5R6J0+ z)HE6(XT9ORxrzcd>FlWb)sV7GZKHec>GdNSlvPdSElDuE6@UcZJoaG`SFZ}16KN`5 z63FKy+(cSNE%XBx6$;r2P*7hnXWWyYa4rbZbOOYH?i)GaBJw;@kAPZ2tA6x)C56N7 ztwl`RXs0l3q=>jY9#29LQ>&(55}R^fffSpiqD-;gu^tXrDYu>-&u5$b1X)<>nto@J ze!Y{2x?t4H@3CKw5=m*a6-(rmG%Dt^DfgRh_TxUf^YKdzkqLJKPIrW%O!R`phr-0H z2o-83Y8aoid_ffn4l}0LRYMYHt)bJfA}I^3%yQj!;PnyxAOVHgZnIE996wro7BVgC z-OC`LzYccE{io|@A(9|EHv>~;DHYf|BsyWb%}%*hd3}?7(n3HvZ9Lx6iUeA6@OOeg zgrP?i)ct*#5zV@JVz^<^m|22O#RJ=Mmd69mK2H_l+1i_UBpOWzo%ZQfNN0v=kB&vY zAe^~isQ2HO#5_Qal0ry99|UuwV|qOnsF?q$hGT^O3a;tv;cUZs5hkpp$ke~D!Lt^1 zEmw50C*DfYAIL#JFx+84kux8Ycydetgjus2U)hn63J2gog6&-el=g9n4JlPQdL=CW z*pyIHyZ9-MEfLSD=sll3zlT(jy2*eUqiZS1{h;AD9O_FWqYzfhh(i3*Po-bD>$-%d zf`kXyY1V{OE`V!WYHke{9gD>2r;F6)YwCilsjSy4^ELK)4KB4?f6-&NVZzwCKS8$I zFz)evSFr0Ex9hPn1nIfgpN?`A?qM6?154>3a-w_-2^taq;Y|Guxcl6CD~1s-5Yz_h|; zQp9~5ye*N_{Lt8mryv9M`6_koK?biUL9qbAj@!2A$q>4)e6XJ`3Ok|Tb?YY+4X`>r z$(!?V;@QfmjPtac;w7G4GDjZ<;*kq1viO6-r8A z+qP}nw(Wj<_Bs1JKjQrw6`6HKW@TnX)>;u!Mj;9XG-MO#bEVjpaMMs@SYmcsOYp>j zs(3yND(8y=o^S53pK%w!JUfZT__!Se*fDTn9lcqq%5Nucs0$+HJ0h+=2fj#WQ)<(# zzhG7^Tea7N)Y{a`#Scq*mWjY;+P6-jnyS2BC=jkia|+ic`C``p;UFsMfqFd8uDFH+ z^*1qI&02Ao4pAKwEw&P-M$+@~sS^;)gmp^}DG5s0cb!FH=Ralzn4+;KdXv1=*;Ca) zE&MEGojzg;U_d~mC03>1;&F(zkI*07E$!D@78h2y&+}sM9|4hhQkobf5VyNHLn%6> z>m0<~KW1_t0qfCG4o3_bfRiGc?BrFyZE|v(WpLAfd`3^YrtaQ*-csO{$ zrojV5@8PW}OY?@C5f@_pV)$OSy-i6+Ir)R>k*6(&18(7GLeJP&I{OZ!N2+HF>z&+0 z-{Z+2hvUP%-zk)nOV|z3ZY%+PiPhgMD1nC&Svl>0^@vR6LcjYr`jPyiMXE;)k6^S` zZ?M)XWdykN4&6hafG4GP`d|SF`cK;rSvsoz9sfHuv10$}Fxx|IE;s1nr|f!zO&KQG zSn^O1m5=y{)=)BT*GNN%ksb>ef+-rb6k)wN{uEO7GhV6Ai$0?fb0hJ2eiqQCYH?o!jj3r(ZN&U5Ay3+*Ls;wmdb6ma2H(e@WtP40pl&LlV(P#Q`l1JU;0Pdv4# z)lLf91X8uB(Up?~+-1@UvldypO8#p7F1=uD=5aSvHCG*5iorz2y~AijOWW63VS4jU ziHYxFc&jzG?M>8roCu-qb^XP?BGMIReN zr2Glud0jQt?TBb5|madIW5uca+|U~M>V8SQwvZyzKx=N0%N%+vDs@D(W1c4s|p^p4N8QjkS+1~Sinx3 zZ?$LtU^mq6#l#EhNv*jcR~C3LPtLtI6O=^Dp^WgPMd}TI74>1Kbce9v*i1+>j)*4! zCOr8auetjWu5FB;OUKCsKN~^&mi@49AmUF%9?)4|E zRywqZfn}({-|rP!VAp!ZY+7GJwzAV3cP)()r{X#9wY3+ z0@*K|v3tcjwK-emWW`&c$t&US{p%e=mzJMv6&7Z+Y+*l<3dHGN|74=^a{|MkK%S-E z4^*lBSECVafkfw7pe4DtncY5Nx8O0qeF#LA^PG87ybePo_`NG$yfj}ln}d%VVGc%g z1pdEOMV-Sv$z^x;l86Nf7Gzm#5`qqNu}Cz*tW3K%`t}mVKW?;1!@3LpVOAvl^2!AX zubBvP_KY^#EZMy~2=S^E8UIc^{j4CHUd9*=G7e3NkhYxMU*^oxXS{R!aWB;AZ0UHj$_i%X z1c#O{{5Rub(*YTErm`QP(dHwBjGK#4OVU`2+DqenTg-zY=M3Y+ZVjFwv^5KE!| z!8*IRr}1blkpNZpxI(YkEdA1T$TT?#(-$Z)81|)UtC)`q8+{Yu1?#zU5%B!&bwymz zDFWyVrIqPQuVKYg{#cUQd{m-iVStHJbB@^P{T#`m3$q7bvY|;%bGv&Kf;Kh#k)6<+ zTvVC;YJ`N#Y(GKX03+IAb+qG6T8}_$!9vU>o?U&@`B$hX6we4YMlnlQ8#Xz{VL`6V z=a0{M#|qB8W*C0_ZbETmZ$znv^Gt_^f%J(zboy9|KqRj;4W>)BJ!QI&3X%O{>izPk zm?LpE8+U_`cnci@JKy)8ozY2MmBW8%E?UH&1jTTRP_C9Syd;743P9sj-W6sD+25;J z@j2-eaOAvY;i@+lW^jLw@?GN=I!v>FXGB!Mt}n@|y>2wz_Fa&%t|Zpl$?bWS{N568 z@R7&*4U=ISmY7iLKCBLrC)vM`edSHYoiAXI@cP}_{k>L`mse9c_-d+|%Ktu$`l4^4 zIf$}};uqnJe}txrv<9(LcwdCpq`~Z~*1Ky2j%9}hVQiITBxn;H{Z4Qvf*m%C;dj~b z#~|A0^W@O8J;_zON-yX5pC6hB+fb!tCJfQiSzqtci3@ulQO#a<*lF^@M++G*;Z)f> z6-d>`@w}ZoK&T^i|7jkz+G_8+s47HKv>)z6tJZBSTb2yNjKXI=?t|~vj^_Wc+t3E_Pf;N(iL@VC*1MZM zB>2obPa6FF4F-`O4myCDt2(y`DEA>7K*!M`GDH;Pp^2M1leyB(qCI1sJhu_v{oO_u z5UZw4;{UmVbQ+E66d6ZNLOuBG7Bdbf?8cJhq^`=tV*;FQiPs`#w7>>Ihpp9wEFm$Qi8 z`;bGcs4x>RhBtlzbXKI_kn~88mMJE4kH#eC-hOfdx|u&mLd3zF0Pf~xilE3|JQFWY zcwI**r%L--=OV#MiqNbFyw3cY;6Dl#EQ|V&@jwhDjo|m{a9bGKqs&H{i&-~ky>P-J zhb)PF?9|E`_Ku^35{gnfb`lIgL~v>u_q}#P82P^NA2mNPH>`#xyAX-&5kYUcb=s)M zxNogdjN%7~=7KUjm%w4OLMe;Zvs7zP(6H4`lpcAZOzhKPO*sCJt!0Nee-7iA?$L*|DhD)TQiREBIi!}V-3lw&0Z+64~n9H%5plHh1?j5 z`7|@`Q8T~LE{AP zt>{LHj=lh8&m8t;I6_(`(ez|Qm6-9p{O0B;Ok_mYH0NFPxufOXNRs{~zUqXI9`)GD zW%so_DKYHxg}_Ba!e09>=-Fw@uH$Xae)~w*GgnpR`x~AXu|20+Ov?S3A`&1P^$tszDVKbYH2Gt0w{(ZbBWWuz14V)7MtA(-N*giRZK>r7y+SSFwo_f zO|KzaqbTHtF%#TpU^3qq;#jL||GbLUK+AmaP7{wW#74Z{$a0kH=JU@h>a$Rv_31%H zGOPx<+r7MUQ8D#<&|ijq&GIULqwqd&Sr?0?Lk;`a2obEL#Uu43PvRFcBinFWV6^xA z?H%3}+MA9gsmpuFeAlE#>Wg{E!Q(QUxBlaO|HK5mxrM0&RmeOW+- zrhke37wp^I`i+P>qHNwYKFC@yvAQQrYcj$R+-|doL2O;bk99^c`%}F!L{mALrM}UTrMa(;!MX}yE`x~nX>D;DPkL66Z`bUyW>cdWY z*Q%WTZ?HogP5=PA$knzDE!};Pg3t5keQaNrWn;o1u-SWT?4z1<(Y6N;pNjvEG8f- zZ;g=bMH?m7SatYJxUDpK`AUNjp`zz;f^%s@x6cVGJ0+Z;E$8qG_KOw$zb&;)?4Bwy z&1Bz#f_ZJw@lRB<-HwUKFA2_={I;HrZD37E<^gF<|`K5FUt;29bc%jqXhrIeqiQaUU|oQi$$am1@Q6jO zK7CkVbQ~Oxp>C)VaV~hmx{|Olj(R}MA-1t~q?-kHcs_hCV=1#>W8&d%k?dyw$X{MJ z7coX`t(uqd#_Z#~Qqd(hnA<5-!OUgcNgOVF-w5SKTa@Ms%8NNta=JTs-j^+Rq5CZV z9xbd9mA^54TGWg-&wb5}+K^TdTE9JxOvd%~*ikl8i4fcNI>BD^tlLq|-&nVOpbqtd zX+ghIPL<1)&H26m7Xj>782%5I`!MQ0L?RX;D_{ta9vjN`34%#?G3ePfM(46<-c4jN z?GaELwMJ*M0AEmJTL=#g?@?Nf`efn7mex#L7j%1AdAhls)k(4UaxBQ`Xb0Eq%=Bw9 z&AU0{+l(@4?<3bEn467LvezF#k;Jgk6@q{jsAAQgDVMRA&mbS>47UAxL%B_l>c_jf zf+H$PlAA0kj-#Tz4%|Zdd6K7?*Yb=cTigIHed?PlF3|*!!~hCsa^8Ey6%EKRDk}A# zj2HdfaDXbaec2_&S4PTuyGvOtXZi)GAxgQEfDtnMpaqLOWom zcv;g3zJi3p6i7wd7OG{l`ooI}6`=?a?x?srTb!)V5IaRLKpY-Zos|!|*0&tkzTNb$(vU78S6i8k&KDy1BYybz_KJ#5NagBL0-V4dj5ew2wLR7~9DlGr_0`7EQzC-@3#E0aArpXe?m@#G?Ol?lcekhK}!N4f+>n&mN+dz0&-y`ZMDRRYX659V%Gm} za}A}-I(Od_>7EFkVbz3?r~s2!>x?xTQos5chCZhujZCD01#e|0Qmc4HNh|p?dG2S( zT!qjuDndD(HsV>^N@DAR2F`sl7hz+wh>OUs$v8O<+TV@@q86tVr803LDAR3eo~0H` zyN?4Nn}#KT?GYB#(kWFPXAT^se`61H62e|4?Uk@RL+INCNhz%O^z`cFaE zH#Mc8%Gc_vRD{3(P6<~7at0wnFH{tT?I4UKX#)P*&FW8rW7ZEn?mXmgH&tB-SC;qY zA)Hi|!KF$wYM~8k#(LOJYpo4qNw7_5Brf8?5pDPFA@$g+4_FFo10^^4=H!x=H~6Yk zE8IWuX9{aA79tTQczQX=WSk<_^-8U0WQ#fUst|~vqU2`wTLO(E+h&>z=ra$Y{;$lfH;3qwB8gj#ID2x zw=2PH&KNIqr}fbR>h^T3C$Up)^;I%0hc#-#b;-djSFUrC9}qMhL%Wmp^EXa^WN_wO z%E)4(d2+`?^J!}4`mqkqw!f1n=$OkMeJPS?r6mDvn=s_8-c;|a69(NJE9Nl2d!fz6 z=tb*1v;(#2Uiwg|N?_5iVYzLn_kC}g`^QQRN^&=HNzlNc`4{6B)Dc$eoW)>t5Z+!_ zge(E$c=0x^e{qthb+5D(|5WHGcktZN(6Y3n`-*klOF}RWqp&mfvQy&t@YO^ zE4N5d9h~J7jUZ+ELuc4>miWp1i3oWleXh0ssACWruO))$_jD(cHX`*e?ViS;aOl;* z9Tb=(IoFnB25;jZYq=zFwXmj#*$x}zYDmmMV@J*@1p5O)WoS98^mzv(RIMsIX&Drh zBNwo;)h8&iSM)>F9u#nW43l5=4GVH_UNh@;w$TTLB~_ew8(3u!9f~ zQimc@Y<)_O$%$wManlc*8D!$#<{0Bk7h7SM+TNdoz0qF&LK6^LDAb9wN{(!_$AT+``gHfo!d}8_Fhw7Nv=|_iUt1d zFMb}VkV|GLA0JQB*>R9tJ}2>NL1dxLj@QK@55=HCDT4#2J;l+(&T>D?!l^$zBBmAr zPHkdRcPT^lluAOv+5k-W_TOntVrJS54avN^ZQ?xl=)UYr+%Thf)Pt6x>j6tVy#cZ+ z0_w-Dl*JKZ{sXRX9D|=GWzQc7O?pTSXa?BHFhMV-H4aC(_jo#ilY{ty!YPxnlN@5N zKanoI-YJ|+60-_Y{9(mGN^AD=WEWgx10jwEJB%5YjqPIq2r61<}BpUaDV;sT4 zjwXgHmCPv@BKLSKKoBHgH3h{am(9-1s~!)wm2pm1YK>qXPnkj{xhOPm(SsN9qIrr2 zLIeA1>Q%N=!I=@fLr+{1Gv(IMlenDW%o6Petx^3^0Sb0b7YjQN%BL?JR~^vE|_gb#ZaRrf;37v)S;*9CTLp(w#N zqSgG6ifdvOYV~MiO<@nzSeE(6A*q;if=(vkeuajP;0?E!v*TaUh8`M0$r^;K++D z*jVVAN{8FmEA1Ft6WW0rMlgd_?Fa~Maa1n@@UN`a+v?-^r|ROAe9JZLc4uZpqRSRM z8u4nkBE7BF1GRR2Dw*WvnesC+sji@9QLLTe%@EbCjj%PmHY9677IOARxVuBTfx>yM zQa{3(^kymSMXVcWfX>H~0mWp0;{g9qefFgl`r~Gd3@UAeST^>$sz}WqD&>pq`PBVWz*QpmJON;%;PjO(( zpFq1eWeWN18ZcW@3%QUZ9C%?j&{BfV$zfPzIL`EAAl2an6b^QjVo_pn}ei~IWky-s})8w!*~R&!|vD$ z&1(BGZ)AC|dY=i(@w5iWqHt%kty+v#5lcJwH3@394%{J$_N&xGCG7P!q?3#s#5h1M zW0C$qskB-f8x*Z1>msdo@Ky3f2u(W%r?YtkyEX3OxK5z(7zc%7o_^!<-}Iz#Ku8RJ z(|pvjU8l0;dUzlq+Zv=~jbO>qnXHNa=cJSGBV?uq8Gf%ls3YGOeA#0bVJyBzf(dwA z@#utyCA;JGp6GY_9!XNIV6l2@obY%qpWYZvSwfzUjI&bZ(h7$syYsd>1Y8`yXECBl zE8QOPTXe=mP$8tx2i3{L#$HCm2BhWAj~^NOTKTM;$SrTGJDE>i1C3T}c?JN4Qo6xW z^)tJDDck~X<@f^fkfA62&)N!d$y0y%4^5)w%&;ll7kDaI6us_h`IKckSh7X0m@nG( z2yTX(7c!kP7AD;xv@*Cn<*S|)2W`BBSnAH(LxYUM{Qkr)EWB}@u#Y5SEuTK(NE<-k zBOY(h`Y&~IfRfD_8=m3h>)bJe1-DK5EY=}X`pso~2RY^514kXim!}8oV^Y8V{<1y# zU$tnKt@fZ5;%MXk%Kc@F_Ya=NJ+t7O_)1Z-C0Ng!3rx&|2#*HN2l_nam`$;H^W3O2 z-_2j{xx)Fs-#Js3B4PM0_|p*##lbo8kG2ikZ!c_wF^|Y6x>Ls$B|Pj*cK4Qe`&!4% zq*A?^NFTt5HV+)HH!8W8%b2KF+sogtQ9N4@(RbgX2P)!&1u{F3zCMn~mQytc&$jt5 zQ}>n`d~i{5==Dx3;%B%@up>I%VI*eyQS z7||p$GLn5g-__L@HjBL8?=D)nT=Uw-cC&a`H;vIZ<6{SBJTH#Z2Ke$tr3XnVsEffP z4IEHwvV+x0z0rMl9>hv%Is5@BM)`}R2>>tph}T{7-hK2d(cEIlBF|6SI>Uh*(GN82 z`%EI~b%GJ_=cM>Rve~zgaI%mzprH?74YR#zvWqKl7b__85+sgL%I0rxFl+JmOx|%j z{5qro~m8!fQV&NLW}6-8i87~ANGa!98g#w1+yS0y2b`#LCDed#BF!FtxL zKAM;|;YazptbRhuua|pD103^#KlD!6QCl0V&^Snu?2Y?O?OjG_Dp!)CRgDzWii~n% zyFz66wp~ud7zxAsxI@2J)f3V=cU9qJ`g~#S$$7pJGhU5|qT`$Gop9Qk>!01y=<^gm zTd=zNG#%Lph`It#h1@gcaS}xp3>Be6xj_T~fRaN4v_Im-&_Z2dO?!uyZ2}&i5TADw zm>KgA165a80hi*e<7wZu;qfK|Q%fNj$W;H_sj!!0dKGSjcz`zk23HnRPv;%k{tT7du`MoSwV}-PCHpE)sPKY)0}N$!tTrdt96Le*b7LkR_C&-s*~i@bU6NX)J^h zc8EY;=Y4SLZweQM$8veH3_1tc>R;trNu3}D)e^EfU5FaW#Z-!gGw0gAWr@)gVUBP2 zCy|U5cUnY=e$hir5dGPLy$YIEHPxGW_iFBRWR9lRfedM7%6b{gG((c0I5_Sxd)b6~ zwzKqoQ^N7*?HeNS7;vCIS;bq5u^kfjT3|0hjIaCI^s&PMZAm)sN))^623ch*4k)dm zTk5zYdWlp{lq}Gxx*?h_bEFn{brDUz9Fh}>JCW~rmHTP6!gc*257xT%>cP#N@&x|i z0B~m+GW2;t=ddh!*p{?#pvmz2@Y{m zw@vQ&Hoe!4bs>~X&+_#NS;X3B@1o&sf~~U-R%5U zen~7)EQxXV{ORa|$VU|1u3qj#u*%ouLpNpiXK*t|9uAxLq&KR@m(+HzCPv1B5WUAx#$ zb6_uGg-Rm*hIUu$1AFR|6|0A6DGb-F+S~N#NJx~BuF*j((B0MoQh2Kqn}+1g)~FR~ z=4J*4?c*5nTh)Pj&G7OU(PyYd!1ly#7k%+a?`?6Q0_cMcYEveNTU@L(Yruz~vqq-V zRQf+L-7+vd^JVrMl5bL=L|tk7*vQBsj8LL)nO8rrG1=#<&jgB3#|91$Bx9v-O=!o4 zAXei-!+wh?=cS7hx(yMw-@lUHpU*;cxIYSnpvHz%#IKwehK)c_r*XlFV6v2u!-!9~ z-fq@Y61Nj4YoBcX66eLD*>uKvs}C|6!dS#5S4Un$ViSaVj`2(42n~lHe9K~tWnfUc z0lf~N3&}%L&PqRO4?hWr$>0-s zx-BN;m^{^={@kxiFQv*CfW7P7BjZ6mzdeK=H;=;u8nj)h3oTrJy!k*hb5zH-w^io- zokNUgCRS*T4+EfSsAs^`-vn{T=c)WJO_s8@4aahDq_10dxX^|BKv7DXmw1M2SV$pT ziFvhVGI3TAiekW;KU0Y7Zl>#DH#SDVVnuxH@W`}=QhKiAg33BG5J;5&M*UAadn6vE zFgG%0qlOlRqqM{C?hg5wQux@ZX?GG`i@;B{q)RKqil zd7`52omd{>-U)-JKq*IO(U%RSwJ%V#pz-B6!TjL^hvXQG=ExV!_7G1tjD&!qN^yRc zN);LrP((B39EY&_A>yE{_NU4PtL}z2CteE>mrqc0*J)gYP znyjfBaaV(WrK{T6&0Rlvyps3B?u3=IZSE0XEE1Y4AL@WtHt>B*=C6 zmnwrmPvts(de9jz>?Jnw#6Z>V=$5gSabmi@zsTROw*E?Qyy~xv8mn>dp}v1PP<&q4 zJ`BGvv?y>d{*$fLO#*%UBABb~D9i}z5f<$b7~Fr9CPZQCbTeW$P~HZcZ>W*}s7y>{ zQ(M^gHkS*F6~Z1z;-lqh&1L3k<2+ZbgRbZiLV9`D9p==rz|!2NOm9H>oeb*j_lIwG zBM2?x3STRTTFtl#1 z&TKf-PDjUCddxI?y6S#Fp?z~PB?a{`ej+Iw|r3n{(oU?egK1Q5qOcJ!6z z7@%AW@~Idcc0mL9oXnvXwLiiVUzaQ)*=C@mVt7Ji4dvQ|(08H5He7!c444oGSOVk_8n)Wja7?n=K4i6$9Q$c0-Cg@vrO zgds*d!s*;2z;u)dh5AQ(y&CF8umhKnLr{$u?9_wa5My9?(-a>|6VEHgX1W_1T^)wf z7atR-1m06KDgP14ujCMLSi+FywkOi)2kp~I3O6icRX8M3z zywd4!JA<8NE#@ll0*#w9yT@P$u0S%k6@dfzXE-6785HZ}nW7fhH(tE`<*lL75Q*Pc z8?$p;(%GEb&3cf&lMvkWA^d??C-1H%BY20e4I=zBrF7TH0yiX+h~M(Yzs_X_yx!>K z`C6oEPLvev0S2zD48oL?YXfi3VEEQUT@NQK5;_^h=q_WO??lU;??SZc+w`*_C14eF zuMV%XFIoo<;K6+*8RwK-ft?zXymv;VQJ=Dop^xoK4pUZj z+DNN$ulBQvYCg!S)mg#9UWWTOa>H>b1v%%DDHB70vtXmVe4O1}h!QvP8=Sc;H#>2e z_{4|t&=|<2c!L&kK)~fNoaZ-Zg>xQfgbB}&1$6Eqj8w#C{Pm3t7`wJv{7oFh`8P=> zPRzD(Bohe%N8v88B+5GCcNhfuFmL#u?5-#d8yKP+i(>lI`5>Y7&0qIqdM~W|3TuSoW8NZa*o(?5XSg^&Zy86+s#)F+t@%zi}YM#GVJm9*U zD;Y1oQa4SNOy-T%9q1gwJk~RpYce;n#C?Iqd`ZrGvyIi&o}XR#k~V^j(F7n_f{5W2 z>=2jwJDzODHsYw&t@Bf7?jL7|0o#8mrEzMV1;u$GkYF!~?+jZe7O4h$O+cDH$-;AHP zmROoi2kFsyjNOb+XoJlz2f8+t!x`>O@jqOtZsL<+o_p7KLeIB+yDn`+h_ezjP4Am=_CwJ_wCOGC3Pp?*eU5WY_Nftlk`2LLfV1I zGJ`9~>UE|Qd~gnr%KO`@_@ESzWg>?o^&T`Mw%ORPuLD1k(Xl`8WX;)F9I_a!L!we1 zt}ooajv}th@IHW=?B@FA)m50WHzG|@FM^2-(L0%4^-6E&J#2b4zv__rJ{v$~X|RK~ z8Ei6|chPH6r7R#c?1k6iD{(a+4n5E_QX+!B$F_xqp3(+5)3d8%+Z%o(m%6ef`})+i_caZRYnT5nsdQCTNsT)>aoddd z?!j&O-?Jqd43%)sGkqE`9~ds;G#M_2dRIGVZmq1QVIbkP!VP*`))bV7OO{=r^YFrf z@o7bTgbPMd3il~Hm`6j0;~CctC1HpTaASo8VDY)IFPdz=@Uc|4V%ss6^cJ7z_OY!^ zF5GBom-4(G4pR~V3MT(JVxC#nn?&QWMW4Bi{XFCMd>eD!cJ1=QquTUFvKLO-haF1h zb%lbQl`57U6EP`F+7pWL9rb_F^?|NQ*6mGW8LQ@*Y%R5-_Pp1L+?#b2eDrF*dA>?L zZp8*G9R;a0aa)N_+YF!L;l^__21)9CFcs-`qwOPz;bF#6zi&UmuDf3FFmSuH zO}~D`@>r=1*Ga^>0p~$CH+ghj2IBS#ft#Y{_3dh}kGXP`^nHo1wunl9kw$LqPVmiTOJ$mqq?>Bl0*_+Bn&dv*fiG6E{7ri2lr7{EV`)dIDp3>uWSWj$b zw3P9Pds_)*%3P|?!Lf#P3S{9PUzhO!(^O}evyTUorQ*>{aIWL^1l?Bn&k6$iIp06e zM|Ax+TU9pO2N(kTAR(=mP;49aIIuKa;cQ^6M(ya^H*T9mRZ;er7l?8^{|ql{@qD7SIi&o- zVR5!Wzjrrkquo(&QTtUfw0FZE?=VI4CjQ5kaHvJe6M=`T0-{yB@%L18rSDje?Lpw5 z%PJ$7je8XvnWYMWr%(qX1>-%IC`L=q-Z;HWapqBUxTmY>+o_#Kt*Mx~3WeAqAhdb_8s~xpYNJjL7;n@vNS^GKAW!R zdrpT;l~|BWN`ng?I0M}SQlYh#wI%s};uA-t*Ga3R9_tS=hSR)t*-KKVoZL&x11(bP z282!-<1LicWp^BnzrBT|&^Feu+9kbS(dx;HiJvT5MvU2bW#nzFMpo;q;`5%7lYlWG zVG2Jx=?MdRtH*8`F3xLVRxgcfog+uK&^lz2$+EbXPY1Mnh{F3-8=p%K0PTH(e=4w5 zH&x{xMrWg91gg%H(P@C)aef;A`W@ei{&JGtWdtsEl2RKkImF|_Yo9wTZzK0?9+9@& ze@y`#?%*Z7PagBlbdSBOfn+mLT3Xy4&NWnP^Jd{8*BYIifJyXBndRu)9<6>f%LUvo zJTqLrm!8LYF^DGLZWfG^RQ2KbMrqkQ&-6}-x(W>mPklApPG`jsl{+DGpzYn-(`~41 z^{=se=7$^@uC6RT{^2uk`(O;OzgxXDq@%roxG>~Nyae~dmGxa=AI;*>P>pG-w?#J_ zI3L0jn#|%^u6|+{Xg3y6y6O$RF=P;i@M8cb69xzO>-_z_M{c2uo(R*C*l`FZR$LzI zNbFLkR>h;q-=Bt}c@-}7lS2r=|By6Dn-@&8c|^o2Jg!#60x3F1Zo65#(I;Va=! z1THUor2^UiN&@-0zC{=0`(g0#|Eu=hRp|0b1c1^D@&6jGev8aS31q)}{U0^I&VQ$b zF1GSX{Qqkx^DW|n{6h%*U$uK-AVQbd-C~LC|4P99T)suC^212~jQ>w>Idq8dHFId9 z%CE%#*RUSxTNEvj^&iXsqvjRs8=Be4UOG$m{~Bg~i(HUzh5pat;|O4(^ZNn$L|%xW z|5dyA7S#j(lKnr6b-zcl=>K;^)%cP6?@P#uffpmhg&~5E6dd3)RkDDIBqd@l<(=(_ znTaxDOfB7bRQ!{x(BNzK_QH@BU-~_2_X^)L=6=#0wj!`?031g^NBLMjgoTIeH)!SC zryR-AG$mN0>{z2%*?U85FhfolOCCo_a}SE+ls0x`rqVA>jVsEG`I{S6lrWY*eOq2A zTBOWRrYJl(PT#g$FHz?2JL z_N{py-Os?Cj{=uDU-OsNe9T!<@TZ6;-EAeP95cu!v=G>V&qG!SA3o7i5Fp0{EQb>a z%owt!=FZ#`6D+S7{GkC3>@FhtUP5z-33oyKvV?!30iEx}i|q@9PRMhomTA?eeLQYP z2C(DWoD6Yd(@=3@#Ib>p7Vc`uwnIVkRhNrp83lOp|91Z z7ntv#FKl*$yeRs}2K)GAPUUktQP#w95RCwERNNwI%vG0MZHWjP5)vl?0{2Kl3RxLu zWd6`B9M%O5H&)%_NK*8N)r2K7c<^o`#@~`y1^?DF8^o}!>lU`^i837_ylj82zK--! z+%12<0d&(bXP5KL(oGIn5#^)4u>wfN;Ew=t5T84BwKxKPj!Gpk}j z=WpnKx}X?+zLBdDAsruDDl8Q=XMwlt8HEW60)!hyb*a3vc8G`%xCH=YvYdPW$cSh{ zMjinw_we2vyQI{Q%vpk{)FMXDRvq-!m?JjPT!+2Zw~+iQ*NE-0w*U(8Z0Dbo?ikK7lVTa*!`w$NPuq4Aud|Q&^L1XsT3}5XEKD+?vo-0qiG{ML>c7>pU5`gZ zRKJ*g{Fz9S(F~>IdqEk;&<2(+ZqwmG;QRRL4-+ z)@b@{4Jl=9E8*W3b?^Rfv3w4i_bu^)GeTEbSkr}*#dVb_`v?t-XL|{yO{y?<&MUCwEXtk<0 z)8pzywCN^{z&WH}Tb(rlSrNX8wyNA^OXK~?MU*#KwR>ffl;6%`t!Gff8Y8oY00SzW zIv_fBCNv+hQ1uQ`CZoYXOoqhT$58{Amz!Lgiy$qlUws~t{XS8_oiKW?xHjUm!2Pl$ z`6s3m6Op_F+7P}T?!J0TSu7EtLDzyNsjAv#ssce!BMLM#Ls{5%_eV09`{7lB=YlEb zLIC)5VDTnu+>pyHa+qCDV5^<25`Hn5LRn>mVNe- zU5~I=S0Coo^gEtmblauTHh)vSnXC`2T8(ykPv!61nOIi1`2Bf51D(t<&B5+=a>vz3 zGWBoovOKobEWL5xW#@xD zelj%YGg#&}kY5)4D5ZkazblNo-Z#y4d%SqwDc3~Mr;Qs+T{+D)Xjsex&x{DmRJ(>& zsp5q8X-9_&MkVZX*iqdon6zyRen8$(V`H@g zYxYPD*P9h0$#KH0Dn8rcRZf*Y0f9NR$lMG{=Nn;t;|mz{0AvsuF51TdGJGbSFIAXA2>=5c#DmhF0?s(q$p z$%Yw})!=L-WX3iWCcuQQ-)r4T_G1$hyYjAYPrsH~Rk-7B)tlFMM)9g!QP%^qu2u1B z1yFvT2q-T<)iU^`v|4dU8(Vubv=XPC zl?q=v?j`iN&>9ws{`R^B{`s63;wPe2q|>BsTI#eyy|FgIm8-4Tf_v)0)r+*WH;O z@*~DO9&FhC!3b^i)p`PKr#QVoXr6lIorKE4**4#SJ8uU@s*4SvrR=*h^w1}G;ehOM zW?S0V3lD>R%F}Ky$dBqHGTGmC`0gv^n~%l-cB0i7rY*M9i_&W~hDWg2T#8YxUS~Ci zbM#VJY}f%BwH1o^^T=G3EIeE{3A>Uwzw*|l-NB+AE?#^w0CXn{=3HG5LN5g0@c|y; zx3>xhQ|MISiX;a-1yg70buL5P2d$q;*|!>?rDJWU|NNlaAMpi_uCp_ox0=7J4 z=GYX6M-IBEBm=U+{Hy$04g-s9oX!ap3uP8cc8V-gn8}u6RnZ&H0vnv%bt$Rv?*PLq z`rFm#Wy_z4`oZU@*ljVM&IOsa`b+JxN5hIsA*8;X1AVAqTb!27RULZRKw(P!$;Rq+8wGv- z-b@+;va1dBt$Gd_Fsy0H@R|w(OGY2X3n?HZV*zpN$|%D!23;Ys&LupbinAnr91&R_tV8n z?(CT@DXi^xJcUZt9FFu>r_d>joe^oPCg&Uly*zV+fXjRN_O=>GkHrs+wP(3-3DZI| zN)vJmP?Jq=FpdTYVBWmVdv(FsZ>+>R7Fu3zA;#E~vf)#(Yjprz!Lc+eGBRAxdRvjf zWIG}{IX$E(s$T!IXXSQj-wn^2U3)KZK+O-`6c!Y=1*M#u59#8480jz5w^t=BJ-x}; z*bx0Cy6)wYy4baW4m#wSywD6W#<*3j>W#j+Re>YCZ?6y}t~M8-(axE9Gi*sPZ3_M! zWHJd-;IxoJkUvG|4bSYy4E5(hLu0x!fg0;PTUM}`=HxrUqRJGDko!QCMY?(Xgqg1fsza0Z9q?he6SgF|q4C%9YCkZ*-4)bUcKWaVZr;TsJP(b!u zd*7x%7{D8JCA0Ao^{}ccY5-S=knM8!Gach76E?{W;}z{FZF-4|ZV3T)p1@R(lA%h_!*=8iq-VT0R}z8oi6_@pwXDY}ixYJdg;SCjhjmlz z4%H^Y9d=Rw0G+mRBU8M3si?ZnN2g1B6-{%xIG}^{DodQyQi+K^(x{uj33?*ml-<9B$1)WuB;#%o;afT z`l=t>3$z=h!I*6CLH?(^7kthJ?M7ux)?e6V02#fCOe1U{&7POhig7nGjJC_LtzRr_7y)ryXQi47AW~=?034i@c+m zep}9dV@{EprH6xy$n(GdeBe|PWF1TaT6E*u0N#pVty&teL*23j&^U~%?? z(L~8HrS{VsmOZXOXw09X%I1OU>m=?SIQC8We80|4$&ACuT8CiKGza28Wr-(YU*Isy z;Pmk_IoIEV@WK^ULN}7$Yppjcc|lFp;lKLIU^LiI4R;3O0z3BVWgqKaH*{>JkFLAm zk-pk=?ONZe{Zdu@0G-POVJ^8$4s972tpURJ?zk82zj4(y)psFVmAap8!gnl0M!Ukl zSHen1(Q3>3NXW?TIgSb>kiJwA_E8GlOw1y7R^yi3qQPTLary&xtp5qE3$FI78g^= zs*2l{e}~WzgQ8#P4@T3(HlTH>WThM4wJbv|sON!VRvMFADx}qe@0x`j)SV{TZ^9}= zRm-~4w$I?a?+!eK|DvMKHTcz(FN0h81Jp~EnE$GihzcIaI|Oyh_4sRw1z0atSkDc3 zk%r_hmawEmjsf#r7q?PEX??Q0y*54fqLyVM%==Q420)f{Ov$fB`b8(2$U1Hf;JtSQ zF3=q(MT;wI5J-8EMj)8D1habOa0P-;C(XBFWII0+OX%65(2OvN+gtIs(4OiNI5n$i zkqpyy*SPXcxGO4^i5N5=O?@N^I3b z`?xiU!VC(BKp&L#hO`Uuu=8T=l%YV0Eyq8UJ>w@se>wmh~GhaQOuiZfuCz!gbmJn*)A7M zq#E!|Xmx)j4cD?E$m7}f3T-7soJKK8!-+0;O~q_|yEAg*2z@WWKyl;hnP9iBUQtF= zOzG6Q9(7pRrR_rr?$7$+KxopBe!$C8EgWW8NJ7BLwf&tltac+aBvRDq2E^9w6O4?_ zov%e`+ta|nUsm+R94XsP5LM743R@^EK4n&wY02`13(RIF$UpTwchPM4)`OAKy%ErY zpFk_4WT#a-`}%9cywjUAuvI3MS)NtY4HPs$%9fLn8=An5GaARStt8j! zSrrt-swg7bj6UwV6Nu>kGi{gKyDadmOMBGjk+vrbpwluj(qp1F%jacSz}T8Mqub1R|ga%z44!Me&&-hKO`)A zA0a*m5QlIn!C2T*M_*b5r0>+$i|I4oOxuV+-Bv|NivMQmtlfyJ(jXh)fucF$Sk}WH zt_X)4437l8-BOI_aCSohL^XLyp?)0>yORdqdzyB>M@OID=^+#HYuXInQ-?x~ zjKTNL(Oi7@!odqbXV2%>>&MUtQq1=}4X#d9A3+MQ7j(n#Y}9+Ip-&rVW@BM=It5jY zMB^!PY;Ii)|9LQiRZxV%MVluB$ZUUK;@tHVSV$_xWIjhW$&`)b8&1FWD7+76AG+Rn zAaKn48149l!moWGPHW>1BRTEr>5ckqjbI4*4(nWlUxb}%*S_yh_xYgQ0w7E*=JCMI zoZD*a&yx+)VipSd*~50*{)A^Wc?7|b_SK#hE@1l0uLDk!#LB|+b)BPf!+HW1YBiX_ zx#Ed=v2445$?cwRJd*Z8?_R$T+S>}uh6{C)^l5t6U1T2!gD2iu1mb=I#Nl_*C)Z|+ z)J#6<7s~nB{!}2FK12++D4jyDbFF2@82RD86Cp(+YHvR&6vmvb*iBKNReoC*-nT|@_qmSStaP9hi zhSQNeM%YQ<+9CH<@cTPamZK)mM0jRmxq|fez+1lxc|2odg1dQBsyc5OBEL??5l2<% z?h=qDM@Vg?my`jX{UEkU6@t1~D-u&=)q)6N8E0N&7b=^dG#~ZPOxjUVZodM={l=!^ zvai`z!ZNYd_Z)frY6T+Zg6wB6LL13ZK42RsTQ8jMR{8+UJ=@0_1;x8Ku7Wo%1z@Rle5voBY#M z4DN1k&kIIB*L`1QeF)jM6t)R&*{+YI`Nn0-7@1&%2#I@_nkB9aweV}l;2E=0!0&@? zQS5lH9t_m;Po(ZTRVd%KquvA8HI8Z~ET8CLt3{x*{nx5RU}w$9rf(4&y*%4Z%KIyZ zLIQPT74i!K65)>p_-BGcrbiqK;VRpSx~&S&0Wr=W5S^{gL^ zM&8ETN`}LnTz+mzW9NAEl_=$DsbUr}%+V5ZGVBHlo&zNm#cJ^~ z(b;=9wByXJ(>zYGEN%A>DwW73$+$dfalAB{-X~}1GZQhM;#e)HW*mPq9!2{_gt{BI zNm}pOm5q{Xm6VmM9i7iiaY0+qdJ}qemT7VQ(JoW8X6bPj1;Cnnq8eK2L5n-yxf1;e z{#e3TR3mNCcMpz+5GA4*sZBdM|Hu){j{_~bO$hNx4z)+v$Q(5gXc;1s_=%`|7qVwH z{}{U^A1f^hp0mibjYbJ|uLl@}v8QoNN3S6zaAPBFKfXt8M2tn=6wW#yOpC+XS7kUM zMvC*K-)WmvDiv$T61S$4q#nmoRcYo|V8v?u2zIi6`B=m4oK!IK7TTQPZfr zZ?&sM#I|EO+r$^?DgZ9O&yK$VgXeur5Lnbp?#k06md2s%UBH3yMM-IdD@t)*!S^G^ z`cr+TS^2$`-`dt;RV{i2ZUf0AbZmKmCicpNW4}VK)(;vU_qdRpPL;9Q*KO!V326no zgic~ROd05#Hf#@I=)5HO%?~(mth^HN*0Iw9sx%B|^apWkr)0H~=aN!YA+eYx8L6?9 zAfo6tJ=XMTn`%S{+2x8q!Z8{=BT6Z4CVYmcPk3`dIDy%%h(Bm#K_1F8^EbCsdk_KA z>RN^FE{y{pCR$-bCI(*NXdhwfnG#y&yyo}so8*KC zCY$(v&E66V_w0`kEXPw^^`3BmPSPw-#<%+;7>R2!5;H6FJXFn{_U z9Ie_A)o5Zm^583)zaM_4ksp-Wt<7LznF^a2Cws~j(jv`HFciQxcS%Of^nYWR44DsR zUhRZBAAS@s<}{X;L1_T90vfi09dd69#IpxjDiZk*woE%vrr`=lIfV~D6xXQ-zYbdu zpoUTt^7bs~p5o7Vjw2Je=>6h+M?teTQ)7TdZ9d=JIAcEeG~wn~e4=fTes)a6qjc8D zIX9a+X?LOm`YIYU*2!h-Ab#`h1HJvvmdwq>vf{y-Bz@aKy4z;nN!nmrp~TT1DtNY`A_WaPR4E!N7KP~HXcX{ zxLN>`U90Y5NhN+Dg?7(Gy*->5`i7^b?A8oNqnNx$NwVf~Vcy3mMy*TVbktS+YGV-m z_8W=CeI5ZRSk%Nc;3T%AtHo)f^GhjqzD(krSvd+VNowI)yCW_bK@uymr&V)hn<>5B1xJlfe`UawlsY%p*ee# z?BHS^zbQ>%Awa5>`zRZfm$3O2Q!I%y>AC8_;4Taww^~#6#DcCR(vU>*7n`wn2LADQ zs!F8bM8K&mEw^@!+2j&=ZjeoVVX>lpIjWKt)pg%Bw<5^G@XRis9y1fmF)SLXguDh zQ?3IvMZD)@v*!)v0};2nlXG|~iCZ@D`}hc_^k~x5I4ER%&Rd@j-u?2&7(zP`<#3eH zrN2obC;J0QndkWyPs{mjZi$H&hhZpRI5U~diNO^H;~7!M&d;|Q89vgjCC}_+kKL-{ z-I{|d=qoFq*aFtG7)>qO(hSz94F!+g(x!Vm3Chx#=T?yCHmDQj+hcBe;Q)d8$CRz@ z3n*gFPE_$=#C}-fK98q#B-Ae?UT zm)OBNCXcu22#)gcCoPRS?f<@=J$PzgN^0Jk*EgvJ(!)OULhX+Yn>QGz*-e{WRIujz zjZ$j>XvcS+)F(hvQvFhKbh_0WX8BqNv;sz_Xz#CYs^3Z0>G1TM>3H{C5dXN^@M>#7dw9G-;vzE$rmNmQdh};HAeCf8!X6gbp9Pt`>Q%C-$|PMSFV)( zS3@N)t`Fg>r2qFMzeN5`VQ^&4;{A6-G>;8pLv{oNhyUj@qIpV~5JOABYP=Nx_gnuj zCaWly{`7<%Xas1yI-bRA(K$rJ< zvWcR39k8!~??z(@gNILpH7i9LgaZ1dVhgLAad~)Fj#6{stMrv>Vv&BBV)aJKX7TDF zW-CU$Nrck=e9`(&xl@I_t?Y#WSBlsFoNm~<%gYF*WQP&P&^Ll-C?%4Ct2J2KeLxMUQc5;O`mVw z74vVCPj#>UyP~SgcTy&Qu|vDvqk6ttP9k2qTzwL>@9W|>?KH|d8s%Ri>fc_P+fOYQri=NKFhPe)kM{$SOmFs6yzBf!5h z7{-2V!Qx}3b2n8ze&k6$6bKFQ{pd3)9Zs}F8E6QCjhl+%VJi`>RM|rbjg~Ez#CLJ6 z*HdJFd0qB_kU`>)Vw6}CGT;4xtfuc!c!?CVcsIn-AVMBWm19LxfUIg-Q-(xfNytvM zdITI+xs5lujCYr3GJ{jOICO_6kp>$^$#kc>%oy9wj1|5%lEPB>w2I~Q+H%@J{t#@? z&qQ+9(Ba>zCZv~$fl-i^=)%J4tc&0)3ClV=2g)=q&F_uT@%CPiQ7Es~wEH5kenAbc zch1+asde6f^Ze~6kMB=3xwyEqLB%lH0wNw>Vbix2Ob{-S3(eyS+C7B4oPXE_8V@rEN_JdY(H zTS|0UGc_bm3a+g&t@puY4zF5;oYNYo;0=U!;kI|Q!aaF)?>0z1PH!YFB}J7G-KRHU zLlkA4n>t6HNi??={^^?u*Sm`fQuiDbvdODB!Hj_&a~v=li-~4@oefc9JGJUgC2lsh z)Z8Z_O_Ioh@m%pVY6aR^qZ3kwvmsn9wULeJ4XM860mRi0@UpD)X@WsUAijP>TguI| z;n*)mwt0?hKnQzw9d^F1)7|%+0eM{bnU-X>Qb!&Hdbz;H+?VNI3Q$)r4?_wIw924<-F# zm7o_N9nT2}$2(%!i28)G;55=sYXq)<*8W1C^Q=3-W8VM|QrEJ5Pn2*DgW>p6`|OCt z_v$1rkI&E98|Zh}>b7F~K`UNNbXyinS+L@7SkW2@J8$h~-IQFvTT#@Cfas0wJ2A1` z=0nnaxf&MAwGq(S*`Bn_0=yeB`>2#S>=QWn11N@6N^rgRKtAGFfq6$YeFier?#93H zX7+3E%f(`aQ_M4}NUSfn@^rQ4tzU;?T$vBpzo!Qf^lBJp%?AnFcVlj~@nCzzG*jqb z4!>j)TxC{~sn5*q$W<6NTVD65^ATs++BH!{R7w3VV|>^%#Ti69MLBVhe5E#1l`QV63oD z{OYbpp}&C_8o6al6D)%=IkUHe=fv-r4O4B3!CYN4y!Ki2N|iR0P7o_ANo!%~y6Y1j z&=76s3)u9n7913v6UiTiA6%bCs;Huz;SY6zjONMk+tW(y-5d%W;P9q%L&4Kt3&A@Q zwUR5=O+h9$Dt1>f_(Tmet0a#XSH)<~is5Z9hcVi9u+FuZfY}~-m$pm8y?uoKhcVOX z+iBM)+Iq>l7cnls?jLa;wBTnJ`<91B;*ufS=CZpq{&}P@`e-wf*6R)hD8gbxj@p<2 z{os2Xr`NG6JV2KSx7~W*;(c8n~@c@H}U|Xwu$#2-m4tWIMC^KWK5}m1OKq`e~}J=Tt}RSUi}KDPXokd>0MK6aA1LJwR6< z@^GIf_d6T;F)5na8lm=1A)6g47(Usx+Hdppofv`-PvHJ7^Y0gGHjj`*8>shUUtD0D zbzz8s782!D;KL=a+B3^{iEx=arpPjik@kw-6vyYExJ801#GpDq;r3B0Ho8cO9~WQk z-lK}Jm*8CuTH!7yVs?J#3*d6+Pz$$WX<9&X&<(8{#9ap=aeMc~J;cJ**(--bns!b2 zG3W3A#fA(Sx>@GKJ;mNJpa0rx$`(W!o$C>Chz;AR#rHNcM`JT_uST4EIgXDW&b#|w zRmO3L>zb2@En>TJG)U4SrOZ75h7ZG^-oD=3m@w+R62S(4$Y7Zhed$L`LTNCu_V+eu z-91OIT$!p8B#!}KfZD~e z8o$R9mb0kwXd;5|PH0;RQFuOPo%9^>lK9ERjo!W_JQ8*h8e@;M(gzCF*!JbNcwH4gDFP@vA9I_aI`^Q6v*H7JK3Nm;E`%5m8wIt- zyU)n&F*pn`FTzg$oZqt{v_V5nv9s@sI>mQ96-Tz0zujG_GWv#Ge)W2;h%!%oEhwP8 z5C|%0{N0Bo^;G>%pO1Jr>sr${LceSk$$P4T!t!Q#d8(N%WDj@2cohZyIV||`^n_G@ z$KvcWKp8EPD4y>_LNY}zQ(&{wiG_~_B-exiO%fHrz=rLy)s1nC;`K$nZu&Dz;d~Iu z`dLUeSw1DrVRRaflD3JZIhw-PCmOi7gJEtShFkZ~lE>4~6gfXBc{{$t`79t33d*3% z)1oQ4yemEF=f+VtI$$U0=H`R33OCdZWIbj<>8nN65Fh=!#04=38jP-oTAtc*py4<*GrlSa5{MwvM@aboDJytPadrim0j$UQ_=K+Q*7B7El3-K}fG_Azv{+c6`dIQwQAC^} zRKtEE!7U}cB18i~>mSjLy8uv?N|pQDXYl-C z+<$CqoaSCf$?a@f4;vhR@bU#?vmNhPEiZ@Y9p(;cjc#r00TfTX70cj|B1!w@=LKua zkC=@7CZQrr`~bwGx|IN;+qgSD(>CiTh5jS#o*!R;U8El9CeF&NU`N^@+A1b@>*GNg zj(997yVc==FFprK8b<1LG`pI0i8*HCXrW>na`!3zJU^Usz+fYK^nR(xYSw|Y4`Rg_ zhL#i3jTXN+cH%OFKtDfUW4MHaKZ|nLQ3k+ylK|3q7xSOE^+<{_3XSth3+u?@v8?P2 zlJLV&b91Yu5}lzIW0a4IX8Pl8W&&o4tC2HRr|H)aiV$wIs+D~z`5y`|-+qZR zfcf?SOvKgPa@p_MLi6ie&##U_tyP$zSY0L->BC}VvNXyS3aEATZq#!PZZEV*AZkXN zU=*7f6EiJ-7DUvc7|tFUu*Uez>eQF)=4$lar7(O)g8m%YNyYV33&j*TJWd!d%KTE} z-|L=i_KZfN-V|MW$qJ7fXx?YEu{1v!%GYGc{HxCD=K4h6&Ldz~{la_-?2HpJj&5NY z*Qr*Fnsx-Lzl}=oneL5-B$8E*N4xCMORTfIBrAV8wXT|>OEz#Bbt3q-w&vQ^gLng=?mgGZc|5vqP6K^(Z3$JS}d)CLB zXX>(bKOp}&T}5<$Y)+DIHBz5dwR^#|qbd##kNTA(MJGrjh#V~`K_k!8X~l|B1=Xu-(mnt**VP%lU|gra#T0-MXBGfn6ttQtn$8j3%-}Hi zlh%+N9zppYdrdqiEjp5MCF^DAZ@Zy9Sgyz{nAs6S1jqaB4vL&Z2W5^yeUD0akvO;n zj;A1qZqd)bVEXpfWr57CeDK|q7Q&tK-#GiIC;ZbA92C_Di^Xv|Zl`)ej(fNAH3>>A zr=#OgL^#p%&(e&ax)XLvQx>%FC&#TfDHw#9KfuUEP6Zc@uZ+IXEleL+=TA`y^bhQh z>DZcU@A-dNB1^??qaXN@osTid*Q;?}9%3M2e`QudON>@b=XT=ljY25uAg=)YzKl6A zl>+y5V$aCiu|&R><;)~#kwivewyLy+vya3M^+xjpTiD8#xEX~Qhu|wiqHc`t2cnWM zToa~#s2*e}b9{v%UhSXNiB2mO(zpj>!+`c)_Wsxvigj^<;7M9%;;fw5_%mDHe)|UW zu-ZpQHor;TJaYH-0mRK^Zv~$~81$G3CkZW#?**pb6hfN3WHcR71g2P6^A5CiKc_B7p*Qysl;|oVok4LOJ>BI_kpX0?#S??A7#PG|N!9KItoR%_t z!@jyNKa$QA`WmJB06&kB#3KAOr!5P3J8rAgR!zMyv|-zypu|%6vlOLn&&XWz10iZ5 zV^4a96BAX7<$)n`7eJ>q>P-10(>eKBmLQdWe8Q?q=qq-8Y=hkaDSY<;ryiA2*3Qi% zQuO2;MVR|kbOlR5OQy0DBXMSmcn(E)FfkHK zWc~_FQ}~i&GH}bIWLbBeT7pqPa5zE7-eCvnTfq#TCo5aztgXN~g+}a-$($R=MJO|+ zdH#rtmq^KcSk*0B{_br^H*#7{g`MpKZI}sLMEd&DOz1+cwOn?z*hJA!uVA+{!J)nZ zBPKAUYpq4;?()PNZ7d$p)|TQ&vZ;}+30y1ICO(T!)aUeC!DwRknAKibXD}9WA2)vV zTxQ`=&yLQzM|QFor)g5-_Q8sbHq3a`C?|-qyf?t=__28+TEk7n+lBk)w`-bi(J=dk zf$1zY$WfKLTO-M`j)ls*5eVYJ{-d5He(|11_`@O1B|;#BjPR2st63fuwJ#MjrJcVo zG-Uai*$N7_Fnp2Z#>$5uLWQ~flzcwgN}k%m>LH-j2TH5|8tWP^(w@cNjUXyM!YD~}4K=()$mEPh2pNAf@w_wPL@3UU82@6g$Z^3ECw0NgVkAmmKJOmf$FThV*=J?|ehH@F zzcs!VL;9DDyDP`pd(@f_PSBFzFE_VocHW0Z@glVKXhqgFNFJ zj`Y0d+Q#Dkr4eNGPA$5C>`tc%2={fF@P~!{i;%m(0gK`M7qwYh>o$zIVxhuu zq|-B(D-^3#Aq><`>p!SaY z?=64*Yq-e&0CLa%UvJoXHmUi4M+hXAf&k<=0xXt)3U~h@zpYsif~>2UG5e3^@*hGy zz4w=5JBg9Q{aebKng~J2V=`*1|C-_bH(f4ZFFio|KWn$0A+>ZaJXW%w|7V7990i19 z52Ho^|1GiARfJgkkw(Ev?q87Y^cQ5el3@n_Ywef}B(1q$L}&M}83KRB_{3b4#$W%( z-2b0X+WEe%{9pUG=dh)z6A{AtEgiYY0BB@POsH1>km^zgU7deOwQlg=NeK@AsCVH7 zE1Hi{F}b)sQ0|Bm;em4hs6x00%i zkfY?${H-g*mXfe-H%DOS^r|(Q(`5JV8gb{%pR>k{8FXx~$Kq&e0xGLUcb+jKWkDag z;?FN}8Q-s>hKo}G*_ZK)BhWSG2m&7SG5L_}|!%#umRgFR=R@3;s`>1BnVM`tKyVAP0OecosVoEB7+o+?+ zl>>eB@_C_%d8zvVah|KS&UZn_w1~%$6CdQMj0lI;iHQlW!|zfktFaL`bK`J7UqdhK z4Ts1v`#;-z2hh(f|2!u%ra@GaKp?Tbr%j%Ihg{_4t;VQ1#ocKW8Re4S%6hK zI+eGw747%_DNW$q8-R;zeEAkkvhrr3Mi${ocol*LqI{Pesb#i_`DZ7J`0hhD6Xc25 zf))qM0Zgn3dus@EiKwJhZet7h@GJCcKfx8y@h-cs`U5W2)HibIBA1cG(bixR1f*P> zQcBDSN)>F=nzi<9QW&?jD4Q^GrStOe>>OTJ<0THU9%B}hvog#p_+W%bn5)|zJNuy@ zbNk$WH|*89+{7y=W?gRwTeOtwX&`%)L8Za>#xy^s7wh^_A5Vv9k)qhII0xz|@(1T` z*+3&0{l6v@JYD<=U!>h_bI{P0oD9|D-{&{zcQ%$IY9OLkv%Mow9ITDPUP7f9_e^p_ zI*VisFDzdTd*FJmJ#vM<=VvXKj+?svO=gQcN;wg>wGpi`)~D^p4(`{%vBxv%FI-xQ zx7tnhBaa?6A?|GC2}e(cyQ60WPT1(Cf*FyOKOOY$5<$8)(5HF*cC*JZ*Ube>Nqz1} z<1{(HXS-!Df*)?1JM5MLzgJ7=Kunb320d^wHcNu%uNH_;E zT?JkL(|UBRL@*;|Z5e$aL8+;sh&?b$5JlJVwRy)ImY(0!$qs?Qv+7-kAHYmd7*ToyH zM@Bk&>T6$s_nb_LcNGoh{DQs3gZ(bEER0L@f^XIb8Rx5>a8o3*1@R`Hf5edmg{ZKf)nQ(J)fcyuAH+N} zuL|bHAL3Z{^3xS#W02z8@n6rT$V^e^$=ERZk@sa@f7^XZQ?s>h^BazB7Eoe0dC8D` zb##CxlQ5_=*vh`_$$~<&46m>^pz3ZF43rA9#^lEvtR%+qd-5cMJISd1DZuRK28Tf7 zImypRZG|=o(3h86&AC^uWgR)IL*8=Tw-d zs%iS^`25(}Op`pYVu;kQi5dD95d=tqXelu770Bj^JoY|UlE0o}15=g zNOMOn%k#IJk?kf)PgtXh+}g=KFCe8}HI-^b-=AV(?VVx@p8%rmHzcDKeGcCVL~dxr zBq!rP-88Z5!>Fx-G(md9g^!7Rv2=-8w5;M+_Ffnr1POG^N*pizmLaI|chzyJm)ASr zxggqhQxvf|4eo83T}uUESSRA^-kJQ*&<;VR^m%65AHc~Rl-*h`G@&k|FeNa?UaQc6 zAD3O#pcY+ov*r*BnN(;DhfG!l`rf;AKmX=L6IZ8R>@sBIxgFYOl0%$hvogvt2MN*T z=N7ov+Z+(*%!{K1e(j?v))s|>SUZ#pvPPK@?!cOvQ|HvwgmA1K$NObY){Vi#OTn)( zgfZ}cep^%MQ-2Qz2)d}tL{i`g7V#aTE8!Rm+rV?3q3#?(k$%qZf#Y&42+WlTBYudh zrTF1^_PgfEaod%!&}9XoKjM!JLGx3B(B6& zB}n_$nW+0S?$D&Ec){&;5&V^}>*l|r6_UOwjb z>%BxwLej$H;~tMUe8~jp%GsWI^yA7P3HozX7N|`r2 zU7?-yHUJua_IyqYp_sJ}+|ZaIH~rt9}Tbk_&Vz!1MoE#QyM%!Y>e8 zP|%L}-IQQ#6grJvIt=>FjJSYxWSqT-U)JO$T_s5-4uj?tK|)wgMWk>}wBMoH)aJKS zu{%G@W~`+dOm_os$hjYmpSK+60e8RWOAYaBH+i%(5*C$|)L{Bte>3-C!xhgTvv$n4 zW84oVAMjY%r-b7H-B;Y9vlZWAYv06CJl4NPNXaruKk|D9B2JBQKGx>VLzUi>pO+hl z&z|Rj6mh`6T)o-M*HdB~hOMvXC{)({U2qT3IQq*Uc04L!mgVzr=d(YxT6{&^Y=?V2 za}#>&91Vu2HaGkhZv@d->-q${vL2PImLrsTB-@MF&Fq9h(lt{BgTPdzWPrkh6UUYZ zyoGVzr%)eG&7I1-cRB9E7-OX$!V9D^OIvD53<_uzFRcdR4A2c%b}$6kWh_U#JVSFZ zU7De4*xG#3cw$jsQNBDV1AVMTY1x}xt2gB^@1+IJXV#iM^r&)Hv0nbH@L@l;R@DFb zW4=W*W@@lXZP(818Y-GUJzJ05SvIm88O|3TS@YcO|K-nf#nlJEdz=+@Qxyzlccfxr zWY8@5+4vStaiyzEuAr?c(*zOQI6a_HAO;<6Zf= zr=15O5AS<{$|2S?p59m+^TYmghv7eOg5jkPA=4uS;Vg|Avd>o#um9<4S#CmfI}P!t zOZijZ(`{q%(Pb*#aQQG#DYw(LIE|-W=?m@qTT2>q>fq(JCL#hpwPi_(q!NT#>S24a z-T@?r$=uz(GgX1J7mBi4@oiL^)-!>~17WO*EYpu=2b2PfP<*zu_uSVcx&dmtGi3GM zYBr0HvV4<;Hbq0(LCGn(OD*o7%65xb$E*I^3!oQ(2AgP!Xf|C-5pMBYqMWj+f z${q_f%eBKk)-6qQzsIbBb~MX%@cew>(%@(y@!neWSk^+`9kQ`DUx&~!D7W&vd7Mk% z2@<)#8wwi|6Sv1VqB*2-O9r%^AJnov_C|S*mMju0Z4tSp(REG<9>(r%?~x?3602mt z{F1U9EHI-QS#Um7bTpD|LZvqQqC+$|8so%1Yk^&ez{5zgyL=a?`D*Qe;wGRsy_Yg3 z=j7gmx3^(BAuA{MElt@xl;>oavwdV&6Z>Ovc&4sxP&Z@5ml}y{+|WKK2b8@+5#2vE zpo|}^nUc5bpAY9um$C(&bUuE}`YD`qAUM|3cPVD+f`GcVc15oD`Tj*Me9kgOwm6yd zM3Fa3hys01a?MaCgk|RkNWU<-DAX7CGwE%mV7$vy@~(K@C+E|VqDjz}WV(mb7B zBXMld)n9WTA4F~38)MVoNL?R`^71`*i-+~`@cCV|PcXoej*&{tfizIkPe{_W#rOiy zvG^d`&PL~d{fi{yI#C&ECI@`tw3I4Ykg!;N4luS*tlLRIMEm*K74X2R`;4^V!FOC= zGX0A{CF^_LXQRJx5vZmN`_C~>6uoBJWw*8i9X3bPoAF`J^oWCA%4?`x!V;|6P zlihDADk(kTYx|s;3o__?!Y-SFM?sizSW3VA)Uw4?!cw9FBq01j{qY_#aF6t>n*ouW zCbEfHBprY~zds7nWR&4+&h^e@*JyE4?T4by)Xr`?nBiGRN;~h$RA^&H zqKWZBAyZ`Ba;NLDPr|Cj5_8CQ(6S*XW}zz)GjAHG7}BgOh@`6i|6jGDw9)3qX?#^<(FF;y*!%R z9ko>K@oucc!cq6Lyjiy5P}{$X`^55-F?(m3Eh}1-Dj1q;FqM;?x0U%+>x-Q(F~YSS|8toEemQOok^MQQvZZ`D!B zB^5dR4%zKr~aa_#Cfsor+VZa=!cEoyUuZ(Q+)`czWMJzlToWo1tsyXhJ1L-^7Zxw+%rx(&t_Wa3=hp4)ai99E zU!!((gtCh>WD~;8^Lu-gT;3$l{%8=lQ?XKhYhGaOZlipV`KPn{R8-Nng)WN2K>YC4 z`hBIZb~8ihj|QT=Fljm2kh>h$1_?;>qw(ZhfHs_bc6yYO(xH4>>|A?)z*!L#4lx_A z6{2dnN6i_Py{=O}YSa;&GPKwyINL15TfBF5Qhb^I?U@^M-sd0N57mBsqv-y%?{?KF z_)@Ma%k$&2)s8iic~7WCEfx8)Fhe?XzKM^#z;1z(;7973bgf7dd96iMDs9@d<^BZG zk8DdS#jZ3O#7(fmy`N;<42#mr4*7Qa_ebMk21IBz|mI8Yj7=4|}d5f*0RX>m$i~5VMeRKA?P7_~< zQnXv2Vt%{7226JsZ>UnPk{npoM&3HS!OZo1#+{8VIQCc&0$0tCvRD~u=WO=yEN_C0 z%z+br$L%o`8Qdq6lB8+gDpY6gpb4U!F|&1$4oi)6*k=B*}v!_N?nm3HLQN^3@=H zIAg@ImfNaesv;*J*QnhofAjfF@p_yKUi_-rrdy%~Yst zzP$Qv5&(;M!`9A}Q5m$iF4`(IPg=YJfbR)qP0Eu>A^6xM; zEN=(d@0^Vh<@101#Qd0UFq1KK$jqvY;3h|?VEIdnUW;(Dc;Tp=+wZqA!&#}$HX4=V za0<6csH^8IU?}=NHKuP!&k&|*ai_d5komMw=j#g%$$A`f>gm-qE}Zn7v&+PatCGqq z`~A<8!NTzf2OWymn2+sKn;mhQltfri$#K8W(=m?)eWFD+6s&Yxg~Kkh%=_DE_^yHj z(B_6quLtYA&jhFM73!@onsXw5PGC??vo(q}ewk$?yNajI%PDm^raCNaYC>hQ1CUo$ zEq8kH^#RztH~ok$ZTmL-eZQt;VIT<*e-vRJ$oe$*BuMhK?oGq7kv3mc*!6>$g*5k~ z1o=Z+t+N%hM;GP*ZF|gQ5yM!`I_A21Ei==FuUx8<2j_mDs>e+RUle>LaLh$437yxM zDM)IOYTIxv(&b*){k#uv0WBoh>8L?4I9S7G--aZHbDWRk0~h?-HM~x`{T@B8Hyfya zI^+}xub${262*hipKoS5#(S1*Pt*SOIcJMwXZ3jUWU9TWt<@9K%AveS+lpsCw)&gY zW(}LkYLM?MXd(8GdDf)EmeKDlb*X8)+f!s@P~`wL@}A~1gAzmImJmvhzyi3zX77=q zu*hFRYs;+E8+4ce{i?eJzO3F08Q*_Eq;IU_Fn>)8?4q!&J6f3Um*tqXEBJ;BsjwDM zIvgKXu#mW-*23BtG_)po`SK)AwCITEuC+v$XGy2_G_?GdR=ziZ=Uxh-}lE?Y*3x{xc4TOUZ;U~1#_6eLJ>ob;q`CCmq|iZD%FvbZo0*vtx8@+jhs=E6=<4|9Q_i7-yVuKJKq; ztW~qBX3bhPf7g9q_tg^^Wl2&=X)ZFC9R5(1A`}-<9$qx-MCflG0r;Fg!4&fp%_M8pRlXq|`@VXAJY&xkz3c)l$A{g+D5f z^0NP-No-nBT|R2ofWDG@-eqWSx)L3(zwtK4XJbq<+0rP@Fj;9aJ0y)hO35ZJezXMC zWpYmGsNq+5zJsbNNzOJ)U3Wh=0pF=?ek>Xwa^LFkAw382l*Sfu`IawK4gKn)WXe&8 zowb&K@Kt4+&aA+CU}$TPAoGhvvX)}E=>VP2y(|I6@wKVh9`n7aFPgEg#S`}UZ4#cS z3~kBg+nMX1(L<<4i7BbB@9Aa%`MhYqqAMNBibf>H+&xac@2{36UvR&dn*X&8LVGW3 z*wBb^!D#wc(@GYEOhoj<5Fm{c>?U5HT~NWj7zJ?5Tx9Ar{W_q|w5wkCOd>uyz)hbJ zr=-|pap0`)veY-+_A6x5F#GT@>Q|)>D1Ri)%-XBn5wlByVse9N^dF>zjXOFPMaWhN3;ckPIv8BY(&QwPKd=?=81`E18B>N%v=UUo_3McAp^ z^W~I_+$MXGe^1L0_`;^Cxw<>f4QdHRdi3UI`4da5(9 z=U%I~>g6Hd_F>UWc{)izO{rUvELTmxRp)Ja0BUIlBpOa0G&8rYBxu(f zUR?^I1!qY8c59DY^PK3fS7Hl3XHcNRjWi`fonr1Eu7j#$d<_><`g|SGt_Jp~FTtD6 zE|9NSIZYnZ@Noz4CwX)Fc|?U(*+F~b;h(P_;e4RVUH>IKWr~7|C(2)KeK++nLta-I z;Y``f0d4#0-1=86Z1DwxApQ1~`?mgGN`M$z98;6N#6c6n*PNK^cq zT$1FVZTM_{WtXfH?LWPM1bXyNd;yVm`?sqWKtYtHVMs!Y z;@@8U0FDR*H1X`;u1*1?dCe{2;u_@t?FBJ#L}1{Q%l{uH%+Z2F0eHbMqCi>lU<1aG z=Rxt1Aj8B&^t_f*Wb%Sno$$?Oq0bDWZUf{W<%b4}iOiqVC0d|_Z(${FyUu^Msvf7o z8_1+eW>~LzebE+k_5A9vVvEgIhe{@{(wmS1_gfFX9;~1Kvv=6!pYxRvm~4kA2f86o zq}K5pRtqs<6Kn$G|gE0->!ur-|8IK0&v+6jqH=N4q zxQ2YB0ZZrV*U`!ZK19hek$DzojD1=&SULVUtgR#q#48xzoqyP~zy~sMPs>FZzpUDN z80tf&=qLtJH-q#&(m3QB$~+ALU7{x(1run64Qikp8r`?{7dHg`>=1$FUs*z6FZ2~8;1Oq9ElJ2i##njERscE^jQ&PaHq4b{b4{2)sN-H4zWQ9&C_^bcTETIf zg=OuEQeB=q>{*xpbw3MQPZc87wZ#DUG&Me&6iTscJ^XMn47vH(!2HeL?Z{!_xb<<2 zmEt*z84YEw^RQ1LWvC`4;S9PRF(vDuKM01)K@(o`@tD-a`*Jg>7bIEzcwzU%vz61h zfv~aqKupl%T2VYKN%(XDX+1+*>16$8=gxA}$S3b_!U( z6u(0DWG{AkylHK1arp^n7L0g3f!F~|X7gW55f4bu<5rJjp#5v0Rs=+B%#m(J!_j`P zcJ`2d2fvT;@nAFZJ(!Jp#}Bdc`{j*#5P3g@k^OmpOpm1M+Vx^#VHI#!uMYGxtI;BC z-RoLD{_M60M^w+2IAPJAct~ z{HTZF#n|HdYrf%ViBgQN*+1jVskrQn3f$PZB5R7MNv4QEz8Q>})i>?$?@dwWU;IQ{ zIX$RcWinntms>0CfD8ta*Ych_HKdYL58?=FXCE-EhR`*{*~8SFUceDMy+HJNY2@6= zEF|9po2&#l1dVTHphFl3!<;SQ4Hmov>d+X(fn>$QU)aU=8L{n*ny1GvVfFjUtU62{ zd|f+$H^L#fu#YhEr;Vv*a3CvbXJW94G<>SU{s|&qGYs#BFWDnT>Up2)%l&*IR4mj{=8SgJ#xMs_sh<1HX8cWs2I7viXJ3*Gd+0 zJb;YWo$pzG0(Z=iPJv4@iw<9JfNccckAJ?8BVy7s^3p~hjFJyKp`^)=iN-uoWq?P#> zf1kvBBXv+W3dW!KIh&JB3H$n;T9HyuC>@U}h_5CR_@{k1V~$_~&*tdI&_g*@Qw(&u z%(LE(45RYmq=wl>0l}A$3!8e}6u>_XTgNKWTShXQPmXdo`$6?k! zOexIJaU75KYW!89Ivc+N5dEqTl8>)9CRqJ8@I!lS2TYBTw66SKWxtRhW>m7oWTAH~ zHa2gyX3!|wC|ODVM1FLD#((yGchDkf1X_Yki4fEHWyaDg74!p>*WSx$=*7m5er4Z2 zd!v=W<)u{>!f;`}?)(%6{9(#p8So{5^ht9+Hy+$GN+bzf)wdY@MLw8vmE0M2n|Q;$ zoF%T~Dvn5B!EQ9f!~c45PI>QAh%75!V_ibTU@vR%>VehCE`T+F6IHRuQp81R72KcKuH&77aTz4VOjV&u4^&32$Yue88&sGpk)D=-fo6n6P z#YVwWS+1$%Bx1}~h)>K=%y5AYWv$uONquFDtu{WCu$L^6eTZ=-qJj;~?un&c;!vMQ z2U(<797@3r3qSy7yOKaE)6afM5fZd~gS26UcB&4&FeVW7` zN!aESCG-fF&~cny@obVvfBqFf&P|IGT9Fpmqp2A_48l@#CCp+Mz68{e zeq(8jTAjdkW}Ty(&j&sd?6_selMmXvvo&%#lBl9twDD>zc++!$zF!CWVIrR6cD9?1 zwG>Gls91@^!!)@g>p8&C*c*3|cB1dDq$U3E408id81pmz9pCn!2CbOd-SpWTCGpW| z1|sMkDcEB$^vO_k9;k0h$>|-ife9kOf9~OJ+D7O=5R^ZM%D17nhn6zao7K7aC4`5W zJN*M_S-IO`7X|2J1zD+S@J5FVZZ>DKhIklC>7j+OJQKL5UkC*L(P2n(`wK)&d`lK8u9Ms{Ik=$MF3>9)UOPG) zM};wvW|Gil^FtFn`(lZ+?9J?s$&(&;}EUI)MAs236boK?;L)oSuz`4iAddo@8-Y4h(T&5+fStEwl?p zyH&J4KP`1?+$UV=UFEj*!2{EpcCJv1#D#owXOiKMytj<x0TSM2(XwAE zB^#kEC5mHb5k^B{Zf6BaHs2*69o33f2y^dDkp?!)(9T;t>IleMZY(>7Vr4cZx3({6 z+U)&d#~Z3xFCuqveDQQ#xPFU<2_3AiV~Q9xz(B~uN<1H4__QL}Cu7$*e05+~wSu7I zV-+O$Df?%*L}S~1zYG}4Uihc=msU6qd^e%GRmo~}R22_hc6=O!F@DYQOiL+oi2rm9 zu;?O{_dkn48n_sw2I!K2iU-+0iiaKCi0*I+o$19gfrr1}1FeDMpI z4$~~pG95aXS&%xWE{g$bkV$JcE!w6gGt=^h z5d-lGSO^FT>39<;a?J02fN2)zqcEA>JarKdQ)_%0D_2S%j!*?4h@U|rH$ew-CiQDuh*A)KIup7y_$%vu2x71#}MLu|xSjzMr z!U67T4}M6N!2RoPx{TEZG}wW2`3^QS#Y6`+QgikF?f@GTZx@OOu#`3gK>PVg6*5yV z&nURTs_o;5(a4U;&)jb4r>fFx8a>dqYVS8mBtxdT!7GYysMbs#^CBqOsxS+d6ZU9< zYwb&E2Xr7(wBUone%~*>6QKE!ptlio@XsuG75?PJd`0LRpZ$vy2G>dV^>F1+nIZ{+ zDJ06~u_s9^)oiQ(-3U(aaBqXdRW9H6;Tnx%M@N3lGamgFzyaf6G%138s{T|1=;=x@ z=TThAk{Wjm*M`8pHQMi=@E~r1PvvI8%E^{@P58f8DX=B_$t{v8@lsq4(NqEP?HEOB zYd6O_KH0KO)kocxc=<~!s6!zZ(B;;IK zTO0#fk~$+}Y*#oI^q7DY`eH0J%c-lvWRafZVw$MK%<-)Yg&B-P75c9^NOvyU2Wnb4 zc-bL2y*Bq>Ipp~U!tk)y-ZyA$AGf_xZ0uzW+|n>25s6lmArl={)24rjXGYoK-^y2A zZzZ_ovCX>PHx@Z|w9y19$SR|Z=tkl;Phy)rjFuxmG6Ii_PmISh>wVdRggMm(gybF% zzmal@IgrRCRE^&&tbF0knx4uj}_e(#bWdr@hQeO3-|q)wB@fBZ=ow0cz1w%zlFZszqh2vB0UstnJs zUKS)g(ZU+}4X+nlN_|tu9M|0$MwM)SN}-C~K-3QziGP5zA^`D&DQl;gGL*b+?`MMn z4kf_lSbpv80g+e2dbOEfN=7pYL|gCg&Ifyul3r53`5-Q`o%D=I`gF({ii^9t@m@7D zHt6Rod!Wz{a(I2v6Ho3GJtuxG?bWFnagv#psPT+uGFhgALwKj$S{V0_YauNsVamTA zE#SYAoz3J9UwrnQPqPxmM75JE{?DSo-U%t+KcWDy*cyBX>`^NY=Ww|$H5FEJx&xa# zCXyOc@?jVyk;|hNcwB`{cTW?BGuK~LXC2Ohl|TnZ#J8S|mkVJPAzgdJ217zR`2VN^ zqIUU=IGPr{t^lxKc^(!_P*f^ylm1;;Ss1&cHmN5j-fG}i99~K-Gx+9{2Kwc;lm!;8 zjyj+M8aRFSo3Ko=LlK&$=*Z}7e371103+(*Z zf>!n8W|PFMA>=UCv*U+~R*SRk?h@xuV@`sNO6aQqeu06>DV=c{N?`o~4qDOA^+Bz_ zoUV5r9eu*-^FMSk1=&_ROQplJc-rP;2Vxj}=wmt-e|uyvluIgxJDcM$1qB`9sIqbZ zPfiyf8n|6A64*$cINipqt>^jHr`EUY*h9ol3DYqH2|*ekr0SQy2Qjqw7B>)eoQuKW zMr~^x*wph$A`#AA1gxY7LmcaAQ=hAIF$;e`^YP^a47Iwy{ngJ%fWunP5wCilk)QWk ze3NKB`wA0CA}ILAY9S`DmdFjh!hV*KyN1rpE^=v&96dDs$?a33vHpwOudwdYBT~ciAL93i*Tx>x>Va_zY4@)^QDL;>~iB#j0y z*VlKOT-E-ur9$eqaU1biyh;upY*9Y9kR+KG9LY7@p#wyI>VQqSl-VF=`iJU$XKV79 znjYKSjOa-iZe-~^s}3k?%u>b0Fg$;(!P$nL-!H%8u`{eiqjnnPVOkaOebA9)b&~n3 zLN+x^q^$i(@%#As^$Zy3Y)0@(V&kaT^Aa&R`HP_M8d2s35K{jfBJWQllogbh2L`UX z!fV9+D2w+S9?{W?eDnZI;18DPKQyNaE~AJs!FzgBzE(TGTf>?*(xy(TDqB+l z78Z&aKU_F2+A30o7Bmx9Uakxi;>P06<^+kj2ls@i_Z(|tg)Umr-BDrSv1-Y*UcUO3 z6;hKpRDjQ&(4@%FDdFz9tR?t05oA7b9uskYn8p##(vX=$tpW5f2wZ0f9K@tWeI*l~ z<5#Ncckv-5Bo%R<{1Xwd}0-1=|GW%bW&G-r32Nj(D8+6YEB5D$T!5Q9vNw&e~ zUj`s&?_W!&FwRft?h{bx`EM_}f$?-W*aE}q6r*gK!iUxWNS_;1i1m_!3SN`7trx2yf1(0vE&>-c|z?&81^bw5hk{oB=uPY~M= zC7J%;pgRCKqQL8fwST+%|8~NBfEVnG6(mmxoV*;`_>FKC4q8e{BjTwEGe$mJ zV@yKu&#b_I)>I1?QW`-91^bE|DO%Iq39{;bHdwX7CrPf%X*vSll1X2o*-kjwIRzTt z7wBmGi_y!4Eee)*>U4mSNAT=Zg$Ohh7L3dS#9F5QRd1sL4?_=Zx<~A&`-{-v?v`(5 z@{PB()nasbU}C+!_U||bItLrJE+6WwxoqIDLHC9)f`{d>a!4;qCbu`FSvK=s>PrIr zV2z_3njUxVP-9AB?u=_kay~fsD%g@gagd}kl0qI@(E#F--1-hpmSD&^oWzXpsu^>) zU2~5azeijWeCX5?wxF>&tXT;}k}18%T5{*p-ajP>fhx?L>d@Hu2wk`~O0%>bS(`rG zQ~IbJfq+No4wMu{Xm@^sadRq_MgPxWak!mT~0R^&J;}?`d_@t~yP= zM#hNXFirt~0GxT7!)&h);8tU*l_7v9rD0d0F?=qMWeceId*~s;k1e)VgSmIk#6mTf z2#H|QsghvdygsgwBH3jo73ikMR>uct&~$WRd}yJFZqNB-YV8KG(454o$<=^OGTf5= z*aeo!ZKZS>*}t`D*-J&{oCdG!5_DBn&S-P{*7~hwpskrwf4+A2H-d?p6mVtzl^~W= z`u-y)uDq%R{RJ1dcBhif!pc(2F&@hJmw1nr>H`jri^tm))Co%AQp;J;Kq{vr{)Sd- zaTjS^i^2ftH!5nQ*CcCtM!yBWl{RD7aMP(48UO`Hn34?5xcWqcVH;m)R|ncaAI&S} zNbMF+G_B-!70+NQl~#ln3jj(wh?K78801(5vxP9l@!fW3^Y0dCUsPx7ipO4fuyppJ ztL`vh9dxYjGgP>VDn;LqtAuK*AgHDRrY_Rpva|48jdb*=-Ry@QNenS2Tca?eLk;F( zF5tK_DM6*rOhOSEg43rWZ`pe3`sH+O6QgA`{p)%o?7smYSw;=;icggVFj)F4UUDKW zP?9$G-i?9XZaIkV^1NFt8Y@4&s^aDSmC^=XfTdWWAfvo#^Q54MQLSq>s6BEl%GdY%{@D2|Rb!u?}@4v2aGjqQE)$#EX;+Ou*l*+w2-M|43tV^mC1tJo4cV)%w z=|f+|FF?e_t!Ff+8kRm{qQai_71z38sgo}@Z5O}gNf_xsX{m59oYB~d;sgycF-;{5 z8}aO@ds81__wo*h^%mA;VyUW0wv4gBj%)yR)$=~Ih_iYG)_NuU@M9s8BADCf+;||> z;_vQ77Zmb1n}G>;sKC6bV#e5(M9Q8ad+%dU9VdGogE@d>_XP{x*e9?k1G#)b8GkjU zdFw!(Ok-LaYSzBpLp zWDgs7!1!L`$H0GAOPa~D0nsc)3R&l}uL_+h3#tiHaM7N)q$Er+Ha*_?hYb8+M1Wbf8Giwa)u*03DE359U{uf^N)-(TH8$-{9r^$I0Sd@1bw zr{lUHnK=jKVtQ$M_0_-TnrCflCdU1mg>^V&g_O{a`O|ouheY8(OgQAnoHl;2`WKzq z7TvlM$3y6JTpx64HZ!b!DPnYTWN7W0HZcXai=js@d?X58SbOyvoo7bPI-%)v`2dkd9Cgv+k3lft@Uo$!|<9;z#n|S16NulZ(Oi zYz;RH)(qtgVNNxr$^r4s*1OA#g0kV4KocQ2q_~5eFE3AfCtsNmd$^AtR#rP?jsYfqxNY=-~KRQg95a@7g$rR|pZA`O#4%wamL)FpklJSf*%6Fa{!{{5c^ z3K>-THFdmUz$H3k=BJU*6WpSai|RZ|wA=vBp-5gl^NAxD9m6Ic3lzUS1-sJK9v0E- zlhzpbnuj9@zi$+TD-F-psJe*7)!;V5HzgIbeu|0O?GsWCtC6guwe?u-6p!f>1yae6 z6sk=CsH>PF70w`!{o@t*fbu>#-b9u`xJZstIvCXkKV7*@?s8smIH|FxHEP^q^Y+1Q zV1_(fF8hyu!l*<7JR^@lIhqn=Niz!+%bRVqj`_AHoxwYo{^J>$yta|Z$ys66;cU@! zej;F&{f)=|jZ%$?MqIon8%&SWn9F$4a#%xCFlp%c-`Rb)&Bh`P6`a2B53XDHd{R2V zU2}2ViL9v|XR!-=FdafG&X+P!*H+48q@u+~ntl-IN$((p#{aH?9EOVL=Oqs!U)IoT$i)eXEARxgee9MZ zn_8_a*l|gGp>9{u^aNc?yi#iB@wNAC5%~!`dQgN5>e(S4Or#+c96+9Q$@7T;*oOXi zq=8>}4n&&*e4qM7(GC<`4QC{=l=<22z(RQs$8dS5`PenANwI6`0A|l=Gq1)at$*oNhZ0Wk0#~CQ<`s9cy|6&fZv2@c7-k6)`>CsWH3A_tg_W} z#zvuNSd)24gFm}rAxS?@VmR5)kn-$0VtUEJ!tQf3J-a#?k+75~pWKupI!^jTtb|^+sqP2aJ?Mmu1E-_xC9O;=y%hRr}fJn8m{MGU` z9L=$mN({pjh<3 zzXBQw{CL%J1mWiAtFf>QtXh+*xPd`}B8^95{X<~r2kBd=d9x5I@34VLkpZaeBVQ`k zG)C%XUD?jOszVgm)g*L95L-hW;=Te8Z}PB#6`_X~Nyj)Z&Ie=4v@A_KW&0?+%>uDG zQ~6FWgWwaH8NF{T1X^x*>S+6`?H9W6@bU+|8%fM46<&RDJbH+`au*qzxgrRa%rK^# z=UFo-6@HZj&A+jG+JXDDG`|@1t{N+ff$)oxYlovzzzZLigKcbW$C4 zG_lUbMZlVp^dm9GGHMHpu24hjZrXen!LfqpLCMUD{nsbSVh#ER|`ZDE18jv|2Q&g#b5g~L{?(53x3)Ni$FZt!Pn|C>?ha8_ZJzL`U_i&F%t>f|vvfR~7E7Z-!qzv)D zK?u!-J&qB!O$yec!4)CH-P{-(XhK5rKoxWx71mk^q0o2y30ruUe>Jm&Qe$Tl|C7-# zS_y|PuG5^YqFy-x?&d7ThoG(7Y`^08emp;PR@G_tEBm((T5vT}M8I%R5HC-~FS_+% zBAFT7Bs$u#I|hg=A7@y22H}YOEBSkGBO#_r2`H?p!9L5vdE3H&W=aKg`32dvOi}9X zXT6cc+em%a-?Jj6EV9pO79v&L8kX?1WO{#S((mhgFdkAVC~k|ITrE2*BN--6ZCYRl zl1c`*g$1L@4X=0Q*;E#5uSu>GIoi~I>ro!ZU2X&6G$amrHlx*vulLvLI)-^P#f!_! zRPBrO*cPEUV(v0x)Dr;~MA6JOEthx1*)WCi`KyQ1rnXCU zbp%-r@3QH8y-BxO*dpaI6li-aE-O=0M?*LZ@Y$Oq-}VjpswVe?!CFPvxdwwg*~}^m zAx?(2!8mhz=2p{{J=*9)oyk{fN(Vy@#)V$3B8jYZNPLWb!W2*f5Lt^1X$snbX_Dpq zj>r9RgqFhV>6jITqqbmi;VpZ=4@c(PgMV+(@XZkgdlX>l@4!|i#c;3P4LH#f+6KqF zmWHctJ~&bQL{Rd%{%NFg!^JJIf!oIoX(#Y#33Z|mzk{S5D{FN@}%J3+iPJlUs8 z)P}GWee6Gy(f>LtkKSJk)<8KqR{Rc`=Bp83y{p@K=;o)y-+e(GF);+8?nWVhZKR)a2_c-D<8oAb;(>ZhdwuQqM;m%bX2dhOfy`2Hd) zw862l9y!1Z3|-9Dw0nU^y3Ays3YbWLG~U69>wfo z!<8_yXiK{>-OL8J&;eElEJ?zm+)l zp2op(Hl7m&`B1<)Z;7J5SAti(#7|c7iPeg&B4o_m8ez;~0JY~F49A{;~R_2Z1J5<9iwf&;yjXH3{Gp1nF>7Ri59Yn|@#PuEc!7lk!UMu>KeRmkb=O<>RMFs!+JH+7sKhDqY)mhVDEogWG(3cZDhmRaWF<)cjt{CPt z*;H;zUVi8Kp=}rL7d{C$VE7F&@IV{?8NtPj$3^;irc$JY4;~KKe7`Bt@&%S#7!=6y zhG|~=t9tJ29gbDqaZmpK`UmatY@XCZek51&8}Qco6hcSfost2q)6@LTKg%{)Ww3ja z(s%8FuDMPIuN+3zDQCK0EHkDdq;J7M21{I;QAa8|<1JKG4LR9mxVaN2a(g9jyUaD$ z!!*A;ViEc;!s*lczUnH7ZCO8%#Cs66)FSfUWUN2#v(S}3P;54Lrf!DE8h{^DQR)gW0@|9=8+p@P90$iG%$JLo9n(B=Ehue@C{z=>J&88p1HgGBPSFddTbB%MG%b8) zQz`WA&}q#L5y2n8WKX6PJ8Q>$l$;lzxF)KHn0w; z&i)Wp3}2?7Ph=%5{sU^dibQFZZ{6{D!eVF>addS}x<#jtDv)1eaC<%9O30a>?zo-+ z($Sf7PLB7GO!=Omdy<>`ge{{)wJ{ZRbmknom|zO z)LJKh7yb-LL655ch!!;VnKu_N<37c__fX+=q z(COhv(&=wUC@ZJ;dI@*lVlD%YCO+UGGAz^+&qivhpJXbj%Q|O{alu?Z7YPN|ml& zg=sbd)xtv(vyPgvh=BwdznG)*DjrFus&U|*?ZRs2;Qp=7<3VaBEF?0mh!!;L66g$p&x!NA~jrlNr8yLgWUi3j_(Yt8_^nh`d*xAvFtEDy@Q!cV- zN2PCpS43J`4wqg;^a31MhjH$`wzt-`9CMahBUfyLj5+###i&F%M&Vf!haEa`)s@hO zi@h!>-@F3BnAR~Nz-7ly8TFTPsH_7KQ`!7EN|^;q*=%|0_V%>bSX#wm-e zWjJPnoJZw6tertg^kJ`ej4NVG0beS`5`LYb3jw3>$mhnWdWhty z#0PVOCsrd3LNGD)ajSpN1vMpAUO~<*?Tr)biAJNHUSt`?qfaX2g`H<`ObAaGOB8bx z33Ty8NpYvR5b$?&k_ULAou_`c5~hzkl&#r;b;o{jpH>z7p47||tRWokOn#8pHU(SK z6Es@_`9tAUT0N>DORb!1cD`@_#P=NWt&Ua8?g<+f}RnVQ)T5TX-}ZPhX6n zsLwFvn_*@T0o{pdIBUq!#*1NQYftWNrl5{CR*}TRo_Y$CfF9AF+?7hBK+alR3n$4L zlfEZ48F50@)oD}_69c6L-UO{cxi|!PR=1<}hvPCxn+Do4^kT8EMQcX9dVABxUS+Xi z$Y7Y)_~oM!$DY2vY*fi5K@qB+VHY=vr8P4_MM*>XbB<)=^w0EQa4xi?{p_dARQ(!) znFkc6fpyLTA56yvdES*nbfV5O0ae9cN4c71<7SVv>gy}Y=FdEJ!prRDg(3DF5V_k1 z(A$Hl0uG6;wx_|kUnjAm8G{2yf5+WtmP8rz#&3Akba~wiwUW$}(3N#M(|Ylne?}@H zgARQPv5tlbvcg;a$Poj(9^QBD9`1c+VI_r3ZB}zk>gfA*x8`~)aYS~rySMP{@9$|( zj2Yvlw9ZBiT!2Q};W3z%iGbb82Ahu6AKjtGRG8do8pG$hZ6=u0+&fdw#ilm{%2zc0 zaPz|L0)gyL`1ZK?H z2Qto18gAgWv`cvGxrMp*HULy9wUPfst2}dt*>$p!p8G2=YGin?Hn`G1r|dMiMCrb?fW>{qQZ@Cwz?P(; zVmk|(xZUwaC>jWw#jMud>vv z`Ze0J{+3Z*g}1b07wV>l4+}B@!*93o6UBRcG2b z(3QVJiLP-A-BgQFSV+ zXAY}yAxml=1@}%XK%@Tw z$vwR!i{xI*}|olb8o6M;`;Ee_-HXZtzUI9%GIn0;|c_T8nzGfhEqJSs1C@CoMXBReWIW7QDd3UW% z6dmv%Y-FdiM6IE81<aw2Ue~*S}2-h_@Y3p_hPUFv(Uh3Z6vE4wWU1G znVaHdRNi?fj3x6Ahql8Cm*=BE8@ivokjs!ZYdm8X+hEY;#Awf&a-&bV`pmM!3TU>% z^>uxb5^uc-A3?Y+f%yQ(IzEf{=GPwK`<s_4DwM9@zmUyU zw2m-&SeKa$>rfWHq0F(oQ%ax8)+8_qGx1MGs1+xz8u9HMwdb(zJ|A0p99^8$YAadH z1K78jbwRscCGfq2a1v9}uK!{TVD}u71sXZu{RGd~`K7;KNTcMc^{F0!krE3OfNb+@ zr#(p~Z8V2abnusLVM9}@e|NTqqIE&Rz?yMq?rw1R=X-zL(r>jYz4Mi+)1u>aU2u6X z$<5*1X{CbP2Pb2C#4bqK`$dwm4w*6vilUy&J8U&~}k z7$?3H(T@taWa{hv03_B$T(bbu3@a4E_6Jp7W1{kg2j5+qCEp%k@*Ds{UFcZkEN5;9 zhk$hqw}KtFmYvM==-M;*IBi>U$S?J_zGD>27p^%(s)s{3N3!bZ>)$cwvCe%4v9Hs( zv^)XROsPu>;%Md1b=#~|kb9+mg>z1n_w$2ob04UNW*}xKB(5kbZwR-Ul&6v<2sUV^ zhFlfJn~C?$JJ$Y3A>KiF`iCt3BREHLnJYK%8#ooT-Lf27nZlAo&$5G*YHO!Y^B_k!4yj z;>Py{Y~q*b-e$Y#k3DDFZuMK2e5ot=DET;n zN$mY6Mu&(FG@VT{bjc^zMDV>7bZ=UCaesk_w?4ayVNg^Ch=2@`VIuIXkqxDlK1omD z16gi1##hXYhk~y|q1{`kKmIiL>{Au}8_6T`8GS>-PCSGE$tV6PFK6KJ8Oo!Rt^8S^ ze^5vql%Yd}4Y6e*U++@{=1*2TojKm0 z)?`}ti>s}2R(U!+_8U2_4<)j@&8sncoRuNPsDU?Ti%d*#O@rvO{N5`MTFt7B;%dV% zn&ZX$j_*<->8mwvu6)VPs}1d7d2`eC@FfME zS#m9RaqU-+?Rt+!$Gewa)S&w9NRj=>A$WHM1#A&6+|I9W37Np_i30m8VZm5LLxk{u zfI$l+S@PjCJES^38%A02I?%M_Ju95=7Ui-PkJD31s7FdcM;?bDSS*vJ?Fwde^HcP zh=T@+Tz-mxFLc>05Fxf+ZaubMd^k=-m||i4B^~t=w(#P9_SUADFE@J0UBanQMv@&}STXKVTjh+vJ^pG`o{eDXCf#ek|&h zNmIWFGk%2xL!i#Uu8mm{6&gUN9QtiJoT;rp=u(jNExe&@UC zEwq_!w^m;Vm(_|ceTZpPF&eR+vDR{U$G?}U_{WXw%I3V6to#MV1@{Nqmt=QrhaRrb z#{m@<_TT4&>()UUViGegyl8$JhrJy*Pn1qtrOdBOt)XvSCj+yUyza>$0-W=qq2`zd z4Wn(FK4|t!enW~c=HV-o83+9go$3*eVs6CpNTVliud^p#_}F-Rt@`tCotV0szJ$j1 zR5$-#U3EBQ@%5-Bi%l5RWLiY&5w!3R+emify>TJ(Yo-WOi%OCyG2I^IGR4-mD5B-A z5uLuWL$*u6Nf?#=Lv++H;(spQo!_kCZdW!*WZA+iZ&ElHsU$-9^cj7mg5m)Mtk98+989;JHxk8yR~+902}_mPPCKa z{E66Xsn1wOa@>36c*{#Au_Rk<71x~3hTqm%^r(`11sX+;smR#_e}(;R0VxHZ+nc5L zmhj07im2eg1hke-*2xv|kCSC7aw`9Yl1~bGcZfEc;7<=iK(=P-^ZH_;swUrMEQz>0`XUkac|c$ zuG>Qg-X_dJA0NS+%p)vF+{S`(5wN@sObHjqcJIRMmuTotC=;8)8;qIsoWf=`-u7I! z(BAFQWw_gpi~yO1^Sls5N9tM$)|l&e#xiW+vpn{sr8HP@F=kr*AMIV|Hym8Ik01!q zqmN!ji3uWxQKCm1C5V<_L`jq(qW2PQ#%RGvV$=j7VxoKW8lC7-!i=b+i;Vty?sMO} z-uoxqweI2P4Sge}1Him8GZ+kE;%rU`P3sf>0{ z;AFLz^_;kYy6tf_@azw-37K>M(s;Zu=hWmikCitKqAn|aq84g*-0P6AUxULAXxSg( zB~M!Qc20IxZ6Vyvh0JNXboBP4%8k|HNW3lq*t=zdjAhpa`bvq3y4BPgt@+=oJDGz?C_s2J?$cfTPM1`yn@FNEQUvSpaxYd>Z$*f@3tG#bw@EvvKsp4I~H*Im9ME5h{V;buwBgI zI`CT59%$=0Yv`b6MLf*uHK!{_$aHH+L63=$pq?%La4m7K>Sr{$V$bKweFpfR;E$G{ z92)9T6g>OrL1WC0_OH5PVd`^;l^1*0piEIi;o%3uM0nTYM~9fJumHDMCiIrN7MR^)Y0;$s}??WgP6No;sd$} z@S&V~Y8oda4yX`P_zGkEQ|^c|G1Ay1d28VGMUx82d|k{}t50`RF_xSpXE^CeVo*ha zJuA1(5pZSTllI{B#us^CnS`D(Y2f9A@SqS8#dE3P<#+*8wIrtDY^%tDGJxIZ zCE`Z+Ebs?1?lG&~p2`Zpnt;dJc9%8J;n=~_gDoKSew~V6B!Km=V3!wm#R=tNhK5|K zfyz$M0c{^FU`Cj;8B)SSKBzASeM8@{R(;d>bnee5&e=(yyiV>OSAf(Ad$flw86Ryw z=ER_P3;a7b(Hj>gK?CBz9fozSD;2Sx=E@DqAHw&+&Qt=WAGBppbRnAm61o4F{hdYwzV5y6U1F|N#*Vw z3hLvR@C`Qk}Ihxw!7MmJf4c4;LV#ow2tr zer@@+L`*YOa%$G}N0wAOc;a-tBq(D>D(V4$RC+7)!zg)thzxnzMS$hX5En|v?o?`x zg6H{psF7A-xUe*Wfyd3uNRp8qd#mf|GA-fmDT@P~>tx`eY_jHXDl<#BTuKT`$${$xm<`_x<_)71!w77a^Xw}-i%Q5!$26YGQJOa#5SS7p^aJpNyVrQH7N&X<2B+W*; zJ-UibVG+R^B{;=ps-uB**Q74D-t5ux<~k6UKo2Bm4(`9srT$v#R^GN)*SzNEsW;SH zzsGe7Z5xaK>6+{Wle?Tkh-4n(s9M;E2gM8Obk21RZy|(cH%l4WvdBy>9g+Ii6R||IUfb2LWr%de+Y=I|k3Vv|iC# zy{A1gaBcL0(?hIC(aFxLB22G#EJtq{TGEC-bzKhqgtQ@r!juPPtLq7~baId*B6`)N zD(tR&71MrE7xueH^&W;@-GHRa(3-wKi(>q( z`^Tc5n|ie5H@dn%DxITy8_{KiYsxb$sys?Y@$JcPm*=N}o*Pccb`^au-lUGT`PoG1 zP(3n0-je@?33bY}zkd+~X%{D^qbPc&f6o{!d>#5D2pbZ_Cg1bS{+L_{oE4zrWJtm7 zPB7S7zqX{d{hcmtB<=c3A9WB^^vj`l^YwQ$C}3QoVZE0t>jB2&Q|B^2 zV(lA9w-YfHl+~i{CBkofl^=Tl&JW8IQ}xbV5qq)@dm}CLm2ev^fyB8f3G;-gIcXV= zfY_lOTh9UqS2}2+Gwyu2-9MVDIZZoD3S}?ia4$U2mTo(!o-`+PV0Jo0s&U}2WGG}I zJ`3-5$W64zXDh+oIKqE!^$7mx{ey;AMsx?`9wUV=*{Gp*As9cc^NkYo#wXunwgASB z3pZ@c?6r*RNq+~Blx-a?VbeW2If9aZGIh!1XeI~cGs($B9KX_0;}(liQ^G@K65sWKZl^6$m(+XT?oDiZXC-y20}AyGke38xCJ8y5)kSPhkBujtoPj+oS-V$hNJ5?0 za<}hn)#0M<3PZ64;E}{95Yei&Eb-4kRb_H?8*BFg^QWl{$mJJ1JteG%cm%_Pzh7q- z&$4m5e7raf`vbG0VmN)DU%T z;g$75mZ*-`{Tl@p#0JmHAbSb-B7BV0TZGdyhqmW4z=|Jt9h{UsWHhkoNS9s(EU}fn zXJ+@H!REa}f@It{>{~lBu%?Ju=WTYa-|yR)7kF~dcg{0V5(Rq2{}kRmzs<|yo?y2w z{DKm?`gW|+O91ZpWf=eiW6AV_th^@ru04Gb*S>rYf3(BC2cAXn=oCI{BbuB|{p|EQ zJ|HCyo;+xWn4~**!!MiJip-`NYkiAo5vo%9;5M35*U)>9g30+zi>)!fTvyLgc# zzcw64l=rYVY+Im-D9h9ZUx_bYmU0e2vS}Koq%4)Hbr+e0SPDCx(~#A(84rD5?0jF^ zmty#V80({Y22p7^4i;>8xDPBtE^_Iaz6IN%9K9)T_O2FnL?)DIr~Q!Gkzy3CbZ^#A<2_ zqrD@A`34_*#=un1eG-fSi;S1PBHd|9=ORcP+YGh2d_ZRt>@<{gJdks_%nbkX)G;Uoe5=InpO8W znY906tcTo>;*>k@i^8nSTfFpOzv(}B&61mVo3F?E);w_@;Gt-i{<`O)oU0(f56vR9 z>qxeI{6yjlc0b-SIp-drEXd?jW9G2qTZ?_Y74VpfBy2Ylk^spGY0$DZYhI_P1Hn zElvo1u4|T0m|@x~2E{KW`n408lyokSE5+VxGpH;1j4-;nwKFRl?aHZq(hB+Tx_vX( zz^8{&Bp2o!ZF`MfI^;Hl{VRP*2lzWoq#8cYtxU#<;kp>QZK(U*!BE|ifGL8}=(_F$ zs^CXg^TTq9BHzB7OF&tn4&A<6eoUiZG}uN`857R>m}D&beIDE#e_S(_t-A=k5@{7^ zYnP3=0(YceQ_mPs!sRS}&nJ7+S}VAF+xRDZrjomy>`nMThXW&McynPQq%5kjx&O0S zm=vCneTbdFr2jwF|8Y&ohdgl|)((F#*ZPv|WK{CsW=C3FeP*PC)<1J$vSl-?qpKr* z21Z<^Y!56PTgnCaT#WyAJYYlzKmC$%FUK!mx8m+&F`sEx+0?9-=w5_@q;fr5M_F4Q z$6_g=w&KM<A#>St|T%|PAOi_Um z$GDU9zxX6qfzB1J2#9HRtlWP6bWY(N+{&)R3qhH%c@4|{E@4(oQgxD9G>UnTFl}M5 zul3)taU~)z?|m&r1^!oxWiaQ?g}C{Y5Lj%Ck1;lFra#kkEWccHnx-N_L|ai|Um0Ph zW)b)p@E_OXg=wih#*H%UK<<0g!U zY5xG%OJa;{>;Ua`eCi!xz*721;r5kJ=Y2 zC<}jZ982Bd23&FS>s*Ngq~7nSDNY$U`@ke6VAd#yn@OoiZxmKi_!`rIiq6%f0+I;j zlF=d_Nb9E`?#8w1ETp6l%&<=`0Ay2ReV&|6(CG%kQ`<$3#YjhG&q_hD7^A--p5qJ; z13fgx3nvSwoGlD3ZvEP|Y+knNb$w~AmD8XU>{K9N_K$(}`IDI;(jN5SmoN)($R!!l&jfA{tF0WZ<+uA literal 0 HcmV?d00001 From f910d6b9adba0efc323812ed1aeea9de21d9ae2b Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Thu, 8 Nov 2018 18:19:33 -0700 Subject: [PATCH 17/24] cert manager edits --- .../prepare-private-registry/_index.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index c65614336c3..91683272363 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -23,7 +23,8 @@ The Rancher HA install uses images from 3 sources. Combine the 3 sources into a ```plain rke config --system-images >> ./rancher-images.txt ``` -* **Cert-Manager** - (Optional) If you choose to install with Rancher Self-Signed TLS certificates, you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. +* **Cert-Manager** - (Conditional) Rancher requires a TLS certificate. During installation, if you elect to use the default Rancher self-signed TLS certificates +[Option A: Default Self-Signed Certificates]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/#a-choose-an-ssl-option-and-install-rancher) in [Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/), you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. Fetch the latest `cert-manager` Helm chart and parse the template for image details. @@ -39,12 +40,10 @@ sort -u rancher-images.txt -o rancher-images.txt ``` -After downloading the release files, publish the images from `rancher-images.txt` to your private registry using the image scripts. +Using a computer with access to the internet, move the images from `rancher-images.txt` to your private registry using the image scripts. >**Note:** Image publication may require up to 20GB of empty disk space. -1. From Terminal, change directories to the path containing the files listed above. - 1. Make `rancher-save-images.sh` an executable. ``` From 82c548b8f39084206c6045d392e35d6a3f8db911 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Thu, 8 Nov 2018 19:42:12 -0700 Subject: [PATCH 18/24] completing draft of ha:prepar private reg --- .../prepare-private-registry/_index.md | 85 ++++++++++--------- 1 file changed, 47 insertions(+), 38 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index 91683272363..5d5090b9b4b 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -1,64 +1,73 @@ --- -title: "2. Collect and Publish Image Sources" +title: "2. Prepare Private Registry" weight: 200 aliases: --- -Using a computer with internet access, browse to our Rancher [releases page](https://github.com/rancher/rancher/releases) and find the version that you want to install in your air gap environment. Download the following three files: +## A. Collect Images + +Start by collecting all the images needed to install Rancher in an air gap environment. You'll collect images from your chosen Rancher release, RKE, and (if you're using a self-signed TLS certificate) Cert-Manager. + +1. Using a computer with internet access, browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.1.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. + + ![Choose Release Version]({{< baseurl >}}/img/rancher/choose-release-version.png) + +2. From the release's **Assets** section (in the picture above), download the following three files, which are required to install Rancher in an air gap environment: -| Release File | Description | -| --- | --- | -| `rancher-images.txt` | Contains a list of all files needed to install Rancher. -| `rancher-save-images.sh` | Pulls all the images in the `rancher-images.txt` from various public registries and saves all of the images as `rancher-images.tar.gz`. | -| `rancher-load-images.sh` | Loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | + | Release File | Description | + | --- | --- | + | `rancher-images.txt` | This file contains a list of all files needed to install Rancher. + | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | + | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | +3. Combine the 3 three files above into a file named `rancher-images.txt`. -The Rancher HA install uses images from 3 sources. Combine the 3 sources into a file named `rancher-images.txt`. - -* **Rancher** - Images required by Rancher. Download the `rancher-images.txt` file from [Rancher releases](https://github.com/rancher/rancher/releases) page for the version of Rancher you are installing. -* **RKE** - Images required by `rke` to install Kubernetes. Run `rke` and add the images to the end of `rancher-images.txt`. + 1. Make `rancher-save-images.sh` an executable. - ```plain + ``` + chmod +x rancher-save-images.sh + ``` + + 1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. + +1. From the directory that contains the RKE binary, use RKE to `rancher-images.txt` as well. + + ``` rke config --system-images >> ./rancher-images.txt ``` -* **Cert-Manager** - (Conditional) Rancher requires a TLS certificate. During installation, if you elect to use the default Rancher self-signed TLS certificates -[Option A: Default Self-Signed Certificates]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/#a-choose-an-ssl-option-and-install-rancher) in [Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/), you will need the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image. You may skip this image if you are using you using your own certificates. +1. **Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip this image if you are using you using your own certificates. - Fetch the latest `cert-manager` Helm chart and parse the template for image details. + 1. Fetch the latest `cert-manager` Helm chart and parse the template for image details. - ```plain - helm fetch stable/cert-manager - helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt - ``` + ```plain + helm fetch stable/cert-manager + helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt + ``` + + 2. Sort and unique the images list to remove any overlap between the sources. + + ```plain + sort -u rancher-images.txt -o rancher-images.txt + ``` -Sort and unique the images list to remove any overlap between the sources. - -```plain -sort -u rancher-images.txt -o rancher-images.txt -``` +## B. Publish Images -Using a computer with access to the internet, move the images from `rancher-images.txt` to your private registry using the image scripts. +Using a computer with access to the internet and your private registry, move the images from `rancher-images.txt` to your private registry using the image scripts. >**Note:** Image publication may require up to 20GB of empty disk space. -1. Make `rancher-save-images.sh` an executable. - ``` - chmod +x rancher-save-images.sh - ``` -1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. - - ```plain - ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - - **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. - -1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry.files to each of the [Linux hosts](#1-provision-three-linux-hosts-and-load-balancer) that you've provisioned. +1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry. 1. Log into your private registry if required. From fba4696a568e35321bdcdfacba968d22b2c199bc Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Fri, 9 Nov 2018 13:50:30 -0700 Subject: [PATCH 19/24] adding content --- .../helm-init/_index.md | 47 --------- .../install-kube/_index.md | 45 ++++----- .../install-rancher-2/_index.md | 96 +++++++++++++++++++ .../install-rancher/_index.md | 67 +++++++++---- .../prepare-private-registry/_index.md | 11 +-- 5 files changed, 169 insertions(+), 97 deletions(-) delete mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md create mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md deleted file mode 100644 index f1044b45b44..00000000000 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/_index.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: 4. Initialize Helm -weight: 400 -aliases: ---- - -## A. Initialize Helm and Render Templates - - -From a system that has access to the internet, render the installs et and copy resulting manifests to a system that has access to the Rancher server cluster. - -Initialize `helm` locally on a system that has internet access. - -```plain -helm init -c -``` - -Then, using the same system, fetch and render the `helm` charts. Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. - -```plain -helm template ./rancher-.tgz --output-dir . \ ---name rancher --namespace cattle-system \ ---set hostname= \ ---set rancherImage=/rancher/rancher -``` - ->Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). - -## B. Optional: Install Cert-Manager - -If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. - -From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). - -```plain -helm fetch stable/cert-manager -``` - -Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. - -```plain -helm template ./cert-manager-.tgz --output-dir . \ ---name cert-manager --namespace kube-system \ ---set image.repository=/quay.io/jetstack/cert-manager-controller -``` - -### [Next: Choose an SSL Option and Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md index dd7e05f0263..a9c7e2ee3f3 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md @@ -9,31 +9,37 @@ aliases: From a system that can access ports 22/tcp and 6443/tcp on your host nodes, use the sample below to create a new file named `rancher-cluster.yml`. This file is a Rancher Kubernetes Engine configuration file (RKE config file), which is a configuration for the cluster you're deploying Rancher to. -Replace values in the code sample according to the table below with the IP address or DNS names of the 3 nodes you created. +Replace values in the code sample below with help of the _RKE Options_ table. Use the IP address or DNS names of the [3 nodes]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts) you created. + +>**Tip:** For more details on the options available, see the RKE [Config Options](http://localhost:9001/rke/v0.1.x/en/config-options/). + +
RKE Options
+ +| Option | Required | Description | +| ------------------ | -------- | -------------------------------------------------------------------------------------- | +| `address` | ✓ | The DNS or IP address for the node within the air gap network (don't use a public address). | +| `user` | ✓ | A user that can run docker commands. | +| `role` | ✓ | List of Kubernetes roles assigned to the node. | +| `internal_address` | | The private DNS or IP address used for internal cluster traffic. | +| `ssh_key_path` | | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`). | -| Placeholder | Description | -| ----------------------- | --------------------------------------------------------------------- | -| `address` | The IP address for each of your air gap nodes outside of the cluster. | -| `internal_address` | The IP address for each of your air gap nodes within the cluster. | -| `url` | The URL for your private registry. | - -> **Note:** If your node has public and internal addresses, it is recommended to set the `internal_address:` so Kubernetes will use it for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address:` if you want to use self-referencing security groups or firewalls. +> **Note:** Although this is an air gap configuration, we still recommend setting values for both `address` and `external address`. In this context, `address` refers to the IP address of the node within the air gap network, not a public address. `internal_address` is the IP address that your nodes use for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls. ```yaml nodes: -- address: 18.222.121.187 # air gap node external IP - internal_address: 172.31.7.22 # air gap node internal IP +- address: 10.10.3.187 # node air gap network IP + internal_address: 172.31.7.22 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] ssh_key_file: /home/user/.ssh/id_rsa -- address: 18.220.193.254 # air gap node external IP - internal_address: 172.31.13.132 # air gap node internal IP +- address: 10.10.3.254 # node air gap network IP + internal_address: 172.31.13.132 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] ssh_key_file: /home/user/.ssh/id_rsa -- address: 13.59.83.89 # air gap node external IP - internal_address: 172.31.3.216 # air gap node internal IP +- address: 10.10.3.89 # node air gap network IP + internal_address: 172.31.3.216 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] ssh_key_file: /home/user/.ssh/id_rsa @@ -45,17 +51,6 @@ private_registries: is_default: true ``` -### Common RKE Nodes Options - -| Option | Required | Description | -| ------------------ | -------- | -------------------------------------------------------------------------------------- | -| `address` | yes | The public DNS or IP address | -| `user` | yes | A user that can run docker commands | -| `role` | yes | List of Kubernetes roles assigned to the node | -| `internal_address` | no | The private DNS or IP address for internal cluster traffic | -| `ssh_key_path` | no | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`) | - ->**Tip:** For more details on the options available, see the RKE [Config Options](http://localhost:9001/rke/v0.1.x/en/config-options/). ## B. Run RKE diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md new file mode 100644 index 00000000000..d0eb89c5e54 --- /dev/null +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md @@ -0,0 +1,96 @@ +--- +title: "5. Choose an SSL Option and Install Rancher" +weight: 500 +aliases: +--- + +## A. Render Templates and Install Rancher + +Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). + +```plain +helm repo add rancher- https://releases.rancher.com/server-charts/ +``` + +Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). + +```plain +helm fetch rancher-/rancher +``` + +Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. + +```plain +helm template ./rancher-.tgz --output-dir . \ +--name rancher --namespace cattle-system \ +--set hostname= \ +--set rancherImage=/rancher/rancher +``` + +>Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). + +## A. Choose an SSL Option and Install Rancher + + +Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: + +{{% accordion id="self-signed" label="Option A: Default Self-Signed Certificate" %}} +The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. Use the reference table below to replace each placeholder. + +Placeholder | Description +------------|------------- +`` | The version number of the output tarball. +`` | The DNS name you pointed at your load balancer. +`` | The DNS name for your private registry.). + + +```plain +helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher +``` + +{{% /accordion %}} + +{{% accordion id="secret" label="Option B: Certificates for Files (Kubernetes Secret)" %}} +Create Kubernetes secrets from your own certificates for Rancher to use. + +> **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. + +Placeholder | Description +------------|------------- +`` | The version number of the output tarball. +`` | The DNS name you pointed at your load balancer. +`` | The DNS name for your private registry. + + +> **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` + +``` +helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + --set ingress.tls.source=secret \ +``` + +Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +{{% /accordion %}} + +## B. Copy and Apply Manifests + +Copy the rendered manifest directories to a system that has access to the Rancher server cluster. + +Use `kubectl` to create namespaces and apply the rendered manifests. + +```plain +kubectl -n kube-system apply -R -f ./cert-manager + +kubectl create namespace cattle-system +kubectl -n cattle-system apply -R -f ./rancher +``` + +### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index d0eb89c5e54..d518ad67f4f 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -1,35 +1,64 @@ --- -title: "5. Choose an SSL Option and Install Rancher" -weight: 500 +title: 4. Install Rancher +weight: 400 aliases: --- -## A. Render Templates and Install Rancher +## A. Add the Helm Chart Repository and Render Templates -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). -```plain -helm repo add rancher- https://releases.rancher.com/server-charts/ -``` +From a system that has access to the internet, render the installs and copy the resulting manifests to a system that has access to the Rancher server cluster. -Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). +1. Initialize `helm` locally on a system that has internet access. -```plain -helm fetch rancher-/rancher -``` + ```plain + helm init -c + ``` -Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. +2. Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories). -```plain -helm template ./rancher-.tgz --output-dir . \ ---name rancher --namespace cattle-system \ ---set hostname= \ ---set rancherImage=/rancher/rancher -``` + Replace both occurences of `` with the Helm chart repository that you want to use (i.e. `latest` or `stable`). + + ``` + helm repo add rancher- https://releases.rancher.com/server-charts/ + ``` +3. Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). + + ```plain + helm fetch rancher-/rancher + ``` + +4. Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. + + ```plain + helm template ./rancher-.tgz --output-dir . \ + --name rancher --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + ``` >Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). -## A. Choose an SSL Option and Install Rancher +## B. Optional: Install Cert-Manager + +If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. + +From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). + +```plain +helm fetch stable/cert-manager +``` + +Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. + +```plain +helm template ./cert-manager-.tgz --output-dir . \ +--name cert-manager --namespace kube-system \ +--set image.repository=/quay.io/jetstack/cert-manager-controller +``` + + +## D. Choose an SSL Option and Install Rancher Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index 5d5090b9b4b..ab03a7cfa78 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -13,7 +13,7 @@ Start by collecting all the images needed to install Rancher in an air gap envir ![Choose Release Version]({{< baseurl >}}/img/rancher/choose-release-version.png) -2. From the release's **Assets** section (in the picture above), download the following three files, which are required to install Rancher in an air gap environment: +2. From the release's **Assets** section (pictured above), download the following three files, which are required to install Rancher in an air gap environment: | Release File | Description | @@ -22,7 +22,7 @@ Start by collecting all the images needed to install Rancher in an air gap envir | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | -3. Combine the 3 three files above into a file named `rancher-images.txt`. +3. Use the three files above to populate `rancher-images.txt` with the images needed for installation. 1. Make `rancher-save-images.sh` an executable. @@ -36,14 +36,14 @@ Start by collecting all the images needed to install Rancher in an air gap envir ./rancher-save-images.sh --image-list ./rancher-images.txt ``` - **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. -1. From the directory that contains the RKE binary, use RKE to `rancher-images.txt` as well. +1. From the directory that contains the RKE binary, add RKE's images to `rancher-images.txt`. ``` rke config --system-images >> ./rancher-images.txt ``` -1. **Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip this image if you are using you using your own certificates. +1. **Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip to [B. Publish Images](#b-publish-images ) if you are using you using your own certificates. 1. Fetch the latest `cert-manager` Helm chart and parse the template for image details. @@ -66,7 +66,6 @@ Using a computer with access to the internet and your private registry, move the >**Note:** Image publication may require up to 20GB of empty disk space. - 1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry. From a553d04cea5488c4c28e255e8b54779a8698f929 Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Fri, 9 Nov 2018 14:31:19 -0700 Subject: [PATCH 20/24] complete draft --- .../air-gap-high-availability/_index.md | 7 +- .../config-rancher-for-private-reg/_index.md | 8 +- .../install-kube/_index.md | 2 +- .../install-rancher-2/_index.md | 96 --------------- .../install-rancher/_index.md | 109 ++++++++++-------- .../air-gap-single-node/_index.md | 1 + 6 files changed, 72 insertions(+), 151 deletions(-) delete mode 100644 content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md index 8dac5518de2..acf44ed8fb1 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md @@ -15,14 +15,15 @@ The following CLI tools are required for this install. Make sure these tools are * [rke]({{< baseurl >}}/rke/v0.1.x/en/installation/) - Rancher Kubernetes Engine, cli for building Kubernetes clusters. * [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. +>**Note:** If you install Rancher in an HA configuration in an air gap environment, you cannot transition to a single-node setup during future upgrades. + ## Installation Outline - [1. Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/) - [2. Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/) - [3. Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/) -- [4. Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/) -- [5. Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) -- [6. Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) +- [4. Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/) +- [5. Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) ### [Next: Create Nodes and Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md index 08013579bc4..b5d0a5cc75f 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/_index.md @@ -1,5 +1,5 @@ --- -title: "6. Configure Rancher for the Private Registry" +title: "5. Configure Rancher for the Private Registry" weight: aliases: @@ -7,6 +7,8 @@ aliases: Rancher needs to be configured to use the private registry in order to provision any [Rancher launched Kubernetes clusters]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/) or [Rancher tools]({{< baseurl >}}/rancher/v2.x/en/tools/). +>**Note:** If you want to configure Rancher for your private registry when when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. + 1. Log into Rancher and configure the default admin password. 1. Go into the **Settings** view. @@ -19,6 +21,4 @@ Rancher needs to be configured to use the private registry in order to provision 1. Change the value to your registry (e.g. `registry.yourdomain.com:port`). Do not prefix the registry with `http://` or `https://`. - ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) - ->**Note:** If you want to configure the setting when starting the rancher/rancher container, you can use the environment variable `CATTLE_SYSTEM_DEFAULT_REGISTRY`. + ![Save]({{< baseurl >}}/img/rancher/airgap/enter-system-default-registry.png) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md index a9c7e2ee3f3..8a54cfc80e3 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md @@ -61,4 +61,4 @@ After configuring `rancher-cluster.yml`, open Terminal and change directories to rke up --config ./rancher-cluster.yml ``` -### [Next: Initialize Helm]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/helm-init/) +### [Next: Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md deleted file mode 100644 index d0eb89c5e54..00000000000 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher-2/_index.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: "5. Choose an SSL Option and Install Rancher" -weight: 500 -aliases: ---- - -## A. Render Templates and Install Rancher - -Add the Helm chart repository that contains charts to install Rancher. Replace `` with the [repository that you're using]({{< baseurl >}}/rancher/v2.x/en/installation/server-tags/#helm-chart-repositories) (i.e. `latest` or `stable`). - -```plain -helm repo add rancher- https://releases.rancher.com/server-charts/ -``` - -Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file. Replace `` with the repo you're using (`latest` or `stable`). - -```plain -helm fetch rancher-/rancher -``` - -Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. - -```plain -helm template ./rancher-.tgz --output-dir . \ ---name rancher --namespace cattle-system \ ---set hostname= \ ---set rancherImage=/rancher/rancher -``` - ->Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). - -## A. Choose an SSL Option and Install Rancher - - -Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: - -{{% accordion id="self-signed" label="Option A: Default Self-Signed Certificate" %}} -The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. Use the reference table below to replace each placeholder. - -Placeholder | Description -------------|------------- -`` | The version number of the output tarball. -`` | The DNS name you pointed at your load balancer. -`` | The DNS name for your private registry.). - - -```plain -helm template ./rancher-.tgz --output-dir . \ - --name rancher \ - --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher -``` - -{{% /accordion %}} - -{{% accordion id="secret" label="Option B: Certificates for Files (Kubernetes Secret)" %}} -Create Kubernetes secrets from your own certificates for Rancher to use. - -> **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. - -Placeholder | Description -------------|------------- -`` | The version number of the output tarball. -`` | The DNS name you pointed at your load balancer. -`` | The DNS name for your private registry. - - -> **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` - -``` -helm template ./rancher-.tgz --output-dir . \ - --name rancher \ - --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher - --set ingress.tls.source=secret \ -``` - -Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. -{{% /accordion %}} - -## B. Copy and Apply Manifests - -Copy the rendered manifest directories to a system that has access to the Rancher server cluster. - -Use `kubectl` to create namespaces and apply the rendered manifests. - -```plain -kubectl -n kube-system apply -R -f ./cert-manager - -kubectl create namespace cattle-system -kubectl -n cattle-system apply -R -f ./rancher -``` - -### [Next: Configure Rancher for the Private Registry]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/config-rancher-for-private-reg/) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index d518ad67f4f..c5ec3257902 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -39,77 +39,92 @@ From a system that has access to the internet, render the installs and copy the >Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). -## B. Optional: Install Cert-Manager -If you are installing Rancher with its self-signed certificates, you will need to install 'cert-manager' on your cluster. If you are installing your own certificates you may skip this section. +## B. Choose your SSL Configuration -From a system connected to the internet, fetch the latest `cert-manager` chart available from thea [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). +Rancher Server is designed to be secure by default and requires SSL/TLS configuration. -```plain -helm fetch stable/cert-manager -``` +For HA air gap configurations, there are two recommended options for the source of the certificate. -Render the template with the option you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. +> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/chart-options/#external-tls-termination). -```plain -helm template ./cert-manager-.tgz --output-dir . \ ---name cert-manager --namespace kube-system \ ---set image.repository=/quay.io/jetstack/cert-manager-controller -``` +| Configuration | Chart option | Description | Requires cert-manager | +|-----|-----|-----|-----| +| [Rancher Generated Self-Signed Certificates](#self-signed) | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)
This is the **default** | yes | +| [Certificates from Files](#secret) | `ingress.tls.source=secret` | Use your own certificate files by creating Kubernetes Secret(s) | no | +## C. Install Rancher -## D. Choose an SSL Option and Install Rancher - - -Rancher server is designed to be secure by default and requires SSL/TLS configuration. There are two options for the source of the certificate in an HA air gap setup: +Based on the choice your made in [B. Choose your SSL Coniguration](#b-optional-install-cert-manager), complete one of the procedures below. {{% accordion id="self-signed" label="Option A: Default Self-Signed Certificate" %}} -The default is for Rancher to generate a CA and use the `cert-manager` to issue the certificate for access to the Rancher server interface. Use the reference table below to replace each placeholder. +By default, Rancher generates a CA and uses cert manger to issue the certificate for access to the Rancher server interface. -Placeholder | Description -------------|------------- -`` | The version number of the output tarball. -`` | The DNS name you pointed at your load balancer. -`` | The DNS name for your private registry.). +1. From a system connected to the internet, fetch the latest cert-manager chart available from the [official Helm chart repository](https://github.com/helm/charts/tree/master/stable). + ```plain + helm fetch stable/cert-manager + ``` -```plain -helm template ./rancher-.tgz --output-dir . \ - --name rancher \ - --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher -``` + ``` + +1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. + + ```plain + helm template ./cert-manager-.tgz --output-dir . \ + --name cert-manager --namespace kube-system \ + --set image.repository=/quay.io/jetstack/cert-manager-controller + ``` + +1. Render the Rancher template with the options you would like to use to install the chart. Use the reference table below to replace each placeholder. + + Placeholder | Description + ------------|------------- + `` | The version number of the output tarball. + `` | The DNS name you pointed at your load balancer. + `` | The DNS name for your private registry.). + + + ```plain + helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + ``` {{% /accordion %}} {{% accordion id="secret" label="Option B: Certificates for Files (Kubernetes Secret)" %}} -Create Kubernetes secrets from your own certificates for Rancher to use. -> **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. +1. Create Kubernetes secrets from your own certificates for Rancher to use. -Placeholder | Description -------------|------------- -`` | The version number of the output tarball. -`` | The DNS name you pointed at your load balancer. -`` | The DNS name for your private registry. + > **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. +1. Render the Rancher template with the options you would like to use to install the chart. Use the reference table below to replace each placeholder. -> **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` - -``` -helm template ./rancher-.tgz --output-dir . \ - --name rancher \ - --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher - --set ingress.tls.source=secret \ + Placeholder | Description + ------------|------------- + `` | The version number of the output tarball. + `` | The DNS name you pointed at your load balancer. + `` | The DNS name for your private registry. + + + > **Note:** If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret` + + ``` + helm template ./rancher-.tgz --output-dir . \ + --name rancher \ + --namespace cattle-system \ + --set hostname= \ + --set rancherImage=/rancher/rancher + --set ingress.tls.source=secret \ ``` -Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +1. Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. {{% /accordion %}} -## B. Copy and Apply Manifests +## D. Copy and Apply Manifests Copy the rendered manifest directories to a system that has access to the Rancher server cluster. diff --git a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md index ec84e74550e..3f7fd9e6302 100644 --- a/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-single-node/_index.md @@ -7,6 +7,7 @@ weight: 280 Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machine. If you need help with creating a private registry, please refer to the [Docker documentation](https://docs.docker.com/registry/). +>**Note:** If you install Rancher on a single node in an air gap environment, you cannot transition to a HA configuration during future upgrades. ## Installation Outline From f72eb04099d7d9a1acc3f972cead448f4232195d Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Fri, 9 Nov 2018 16:09:18 -0700 Subject: [PATCH 21/24] making updates per Jason --- .../install-kube/_index.md | 12 ++--- .../install-rancher/_index.md | 25 +++------- .../prepare-private-registry/_index.md | 48 +++++++++---------- .../provision-hosts/_index.md | 5 ++ .../en/installation/ha/helm-rancher/_index.md | 6 --- 5 files changed, 40 insertions(+), 56 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md index 8a54cfc80e3..30695200b6e 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/_index.md @@ -17,28 +17,28 @@ Replace values in the code sample below with help of the _RKE Options_ table. Us | Option | Required | Description | | ------------------ | -------- | -------------------------------------------------------------------------------------- | -| `address` | ✓ | The DNS or IP address for the node within the air gap network (don't use a public address). | +| `address` | ✓ | The DNS or IP address for the node within the air gap network. | | `user` | ✓ | A user that can run docker commands. | | `role` | ✓ | List of Kubernetes roles assigned to the node. | -| `internal_address` | | The private DNS or IP address used for internal cluster traffic. | +| `internal_address` | optional1 | The DNS or IP address used for internal cluster traffic. | | `ssh_key_path` | | Path to SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`). | -> **Note:** Although this is an air gap configuration, we still recommend setting values for both `address` and `external address`. In this context, `address` refers to the IP address of the node within the air gap network, not a public address. `internal_address` is the IP address that your nodes use for intra-cluster communication. Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls. +> 1 Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls. ```yaml nodes: -- address: 10.10.3.187 # node air gap network IP +- address: 10.10.3.187 # node air gap network IP internal_address: 172.31.7.22 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] ssh_key_file: /home/user/.ssh/id_rsa -- address: 10.10.3.254 # node air gap network IP +- address: 10.10.3.254 # node air gap network IP internal_address: 172.31.13.132 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] ssh_key_file: /home/user/.ssh/id_rsa -- address: 10.10.3.89 # node air gap network IP +- address: 10.10.3.89 # node air gap network IP internal_address: 172.31.3.216 # node intra-cluster IP user: rancher role: [ "controlplane", "etcd", "worker" ] diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index c5ec3257902..758758c3e73 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -4,12 +4,12 @@ weight: 400 aliases: --- -## A. Add the Helm Chart Repository and Render Templates +## A. Add the Helm Chart Repository From a system that has access to the internet, render the installs and copy the resulting manifests to a system that has access to the Rancher server cluster. -1. Initialize `helm` locally on a system that has internet access. +1. If you haven't already, initialize `helm` locally on a system that has internet access. ```plain helm init -c @@ -28,15 +28,6 @@ From a system that has access to the internet, render the installs and copy the helm fetch rancher-/rancher ``` -4. Render the template with the options you would use to install the chart. See [Install Rancher]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/) for details on the various options. Remember to set the `rancherImage` option to pull the image from your private registry. This will create a `rancher` directory with the Kubernetes manifest files. - - ```plain - helm template ./rancher-.tgz --output-dir . \ - --name rancher --namespace cattle-system \ - --set hostname= \ - --set rancherImage=/rancher/rancher - ``` - >Want additional options? Need help troubleshooting? See [High Availability Install: Advanced Options]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/#advanced-configurations). @@ -66,8 +57,6 @@ By default, Rancher generates a CA and uses cert manger to issue the certificate helm fetch stable/cert-manager ``` - ``` - 1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files. ```plain @@ -76,7 +65,7 @@ By default, Rancher generates a CA and uses cert manger to issue the certificate --set image.repository=/quay.io/jetstack/cert-manager-controller ``` -1. Render the Rancher template with the options you would like to use to install the chart. Use the reference table below to replace each placeholder. +1. Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Placeholder | Description ------------|------------- @@ -101,7 +90,7 @@ By default, Rancher generates a CA and uses cert manger to issue the certificate > **Note:** The common name for the cert will need to match the `hostname` option or the ingress controller will fail to provision the site for Rancher. -1. Render the Rancher template with the options you would like to use to install the chart. Use the reference table below to replace each placeholder. +1. Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Placeholder | Description ------------|------------- @@ -121,12 +110,12 @@ By default, Rancher generates a CA and uses cert manger to issue the certificate --set ingress.tls.source=secret \ ``` -1. Now that Rancher is running, see [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. +1. See [Adding TLS Secrets]({{< baseurl >}}/rancher/v2.x/en/installation/ha/helm-rancher/tls-secrets/) to publish the certificate files so Rancher and the ingress controller can use them. {{% /accordion %}} -## D. Copy and Apply Manifests +## D. Install Rancher -Copy the rendered manifest directories to a system that has access to the Rancher server cluster. +Copy the rendered manifest directories to a system that has access to the Rancher server cluster to complete installation. Use `kubectl` to create namespaces and apply the rendered manifests. diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index ab03a7cfa78..0dccd3b3818 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -22,28 +22,20 @@ Start by collecting all the images needed to install Rancher in an air gap envir | `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. | | `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. | -3. Use the three files above to populate `rancher-images.txt` with the images needed for installation. - 1. Make `rancher-save-images.sh` an executable. +1. Make `rancher-save-images.sh` an executable. - ``` - chmod +x rancher-save-images.sh - ``` + ``` + chmod +x rancher-save-images.sh + ``` - 1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. - - ```plain - ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - - **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. -1. From the directory that contains the RKE binary, add RKE's images to `rancher-images.txt`. +1. From the directory that contains the RKE binary, add RKE's images to `rancher-images.txt`, which is a list of all the files needed to install Rancher. ``` rke config --system-images >> ./rancher-images.txt ``` -1. **Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip to [B. Publish Images](#b-publish-images ) if you are using you using your own certificates. +1. **Default Rancher Generated Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip to [B. Publish Images](#b-publish-images ) if you are using you using your own certificates. 1. Fetch the latest `cert-manager` Helm chart and parse the template for image details. @@ -58,6 +50,14 @@ Start by collecting all the images needed to install Rancher in an air gap envir sort -u rancher-images.txt -o rancher-images.txt ``` +1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images. + + ```plain + ./rancher-save-images.sh --image-list ./rancher-images.txt + ``` + + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory. + ## B. Publish Images @@ -65,20 +65,16 @@ Using a computer with access to the internet and your private registry, move the >**Note:** Image publication may require up to 20GB of empty disk space. +1. Log into your private registry if required. -1. Push `rancher-load-images.sh`, `rancher-images.txt` and `rancher-images.tar.gz` to your private registry. + ```plain + docker login + ``` +1. Use `rancher-load-images.sh` to extract, tag and push `rancher-images.txt` and `rancher-images.tar.gz` to your private registry. - 1. Log into your private registry if required. - - ```plain - docker login - ``` - - 1. Use `rancher-load-images.sh` to extract, tag and push the images to your private registry. - - ```plain - ./rancher-load-images.sh --image-list ./rancher-images.txt --registry - ``` + ```plain + ./rancher-load-images.sh --image-list ./rancher-images.txt --registry + ``` ### [Next: Install Kubernetes with RKE]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/install-kube/) \ No newline at end of file diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md index 209475a4f24..aa911439019 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/provision-hosts/_index.md @@ -28,4 +28,9 @@ RKE, the installer that provisions your air gapped cluster, will configure an In Configure a load balancer as a basic Layer 4 TCP forwarder. The exact configuration will vary depending on your environment. +**Load Balancer Configuration Samples:** + +- [NGINX]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/nginx) +- [Amazon NLB]({{< baseurl >}}/rancher/v2.x/en/installation/ha/create-nodes-lb/nlb) + ### [Next: Collect and Publish Image Sources]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/) diff --git a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md index e5d6f3979cb..6e7646069f4 100644 --- a/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/ha/helm-rancher/_index.md @@ -90,12 +90,6 @@ This option uses `cert-manager` to automatically request and renew [Let's Encryp - Set `hostname` to the public DNS record, set `ingress.tls.source` to `letsEncrypt` and `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices) -<<<<<<< HEAD - ->**Using Air Gap?** [Set the `rancherImage` option]({{< baseurl >}}/rancher/v2.x/en/installation/air-gap-installation/install-rancher/#install-rancher-using-private-registry) in your command, pointing toward your private registry. - -======= ->>>>>>> removing air gap notes ``` helm install rancher-/rancher \ --name rancher \ From 975bf31a3c6696805f912f68a0cd5b2745e31e8d Mon Sep 17 00:00:00 2001 From: Sebastiaan van Steenis Date: Fri, 9 Nov 2018 19:04:33 +0100 Subject: [PATCH 22/24] Add production ready cluster setup --- .../cluster-provisioning/production/_index.md | 137 ++++++++++++++++++ src/diagrams/clusterdiagram.xml | 1 + src/img/rancher/clusterdiagram.svg | 2 + 3 files changed, 140 insertions(+) create mode 100644 content/rancher/v2.x/en/cluster-provisioning/production/_index.md create mode 100644 src/diagrams/clusterdiagram.xml create mode 100644 src/img/rancher/clusterdiagram.svg diff --git a/content/rancher/v2.x/en/cluster-provisioning/production/_index.md b/content/rancher/v2.x/en/cluster-provisioning/production/_index.md new file mode 100644 index 00000000000..4c05895ede2 --- /dev/null +++ b/content/rancher/v2.x/en/cluster-provisioning/production/_index.md @@ -0,0 +1,137 @@ +--- +title: Production Ready Cluster +weight: 2510 +--- + +While Rancher makes it easy to create Kubernetes clusters, a production ready cluster takes more consideration and planning. There are three roles that can be assigned to nodes: `etcd`, `controlplane` and `worker`. In the next sections each of the roles will be described in more detail. + +When designing your cluster(s), you have two options: + +* Use dedicated nodes for each role. This ensures resource availability for the components needed for the specified role. It also strictly isolates network traffic between each of the roles according to the [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/). +* Assign the `etcd` and `controlplane` roles to the same nodes. These nodes must meet the hardware requirements for both roles. + +>**Note:** Do not add the `worker` role to any node configured with either the `etcd` or `controlplane` role. This will make the nodes schedulable for regular workloads, which could interfere with critical cluster components running on the nodes with the `etcd` or `controlplane` role. + +## etcd + +Nodes with the `etcd` role run etcd, which is a consistent and highly available key value store used as Kubernetes’ backing store for all cluster data. etcd replicates the data to each node. + +>**Note:** Nodes with the `etcd` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default. + +### Hardware Requirements + +Please see [Kubernetes: Building Large Clusters](https://kubernetes.io/docs/setup/cluster-large/) and [etcd: Hardware Recommendations](https://coreos.com/etcd/docs/latest/op-guide/hardware.html) for the hardware requirements. + +### Count of etcd Nodes + +The number of nodes that you can lose at once while maintaining cluster availability is determined by the number of nodes assigned the `etcd` role. For a cluster with n members, the minimum is (n/2)+1. Therefore, we recommend creating an `etcd` node in 3 different availability zones to survive the loss of one availability zone within a region. If you use only two zones, you can only survive the loss of the zone where you don't lose the majority of nodes. + +| Nodes with `etcd` role | Majority | Failure Tolerance | +|--------------|------------|-------------------| +| 1 | 1 | 0 | +| 2 | 2 | 0 | +| 3 | 2 | **1** | +| 4 | 3 | 1 | +| 5 | 3 | **2** | +| 6 | 4 | 2 | +| 7 | 4 | **3** | +| 8 | 5 | 3 | +| 9 | 5 | **4** | + +References: + +* [etcd cluster size](https://coreos.com/etcd/docs/latest/v2/admin_guide.html#optimal-cluster-size) +* [Operating etcd clusters for Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/) + +### Network Latency + +Rancher recommends minimizing latency between the etcd nodes. The default setting for `heartbeat-interval` is `500`, and the default setting for `election-timeout` is `5000`. These settings allow etcd to run in most networks (except really high latency networks). + +References: + +* [etcd Tuning](https://coreos.com/etcd/docs/latest/tuning.html) + +### Backups + +etcd is the location where the state of your cluster is stored. Losing etcd data means losing your cluster. Make sure you configure [etcd Recurring Snapshots]({{< baseurl >}}/rancher/v2.x/en/backups/backups/ha-backups/#option-a-recurring-snapshots) for your cluster(s), and make sure the snapshots are stored externally (off the node) as well. + +## controlplane + +Nodes with the `controlplane` role run the Kubernetes master components (excluding `etcd`, as it's a separate role). See [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components) for a detailed list of components. + +>**Note:** Nodes with the `controlplane` role are shown as `Unschedulable` in the UI, meaning no pods will be scheduled to these nodes by default. + +References: + +* [Kubernetes: Master Components](https://kubernetes.io/docs/concepts/overview/components/#master-components) + +### Hardware Requirements + +Please see [Kubernetes: Building Large Clusters](https://kubernetes.io/docs/setup/cluster-large/) for the hardware requirements. + +### Count of controlplane Nodes + +Adding more than one node with the `controlplane` role makes every master component highly available. See below for a breakdown of how high availability is achieved per component. + +#### kube-apiserver + +The Kubernetes API server (`kube-apiserver`) scales horizontally. Each node with the role `controlplane` will be added to the NGINX proxy on the nodes with components that need to access the Kubernetes API server. This means that if a node becomes unreachable, the local NGINX proxy on the node will forward the request to another Kubernetes API server in the list. + +#### kube-controller-manager + +The Kubernetes controller manager uses leader election using an endpoint in Kubernetes. One instance of the `kube-controller-manager` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive). + +#### kube-scheduler + +The Kubernetes scheduler uses leader election using an endpoint in Kubernetes. One instance of the `kube-scheduler` will create an entry in the Kubernetes endpoints and updates that entry in a configured interval. Other instances will see an active leader and wait for that entry to expire (for example, when a node is unresponsive). + +## worker + +Nodes with the `worker` role run the Kubernetes node components. See [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components) for a detailed list of components. + +References: + +* [Kubernetes: Node Components](https://kubernetes.io/docs/concepts/overview/components/#node-components) + +### Hardware Requirements + +The hardware requirements for nodes with the `worker` role mostly depend on your workloads. The minimum to run the Kubernetes node components is 1 CPU (core) and 1GB of memory. + +### Count of worker Nodes + +Adding more than one node with the `worker` role will make sure your workloads can be rescheduled if a node fails. + +## Networking + +Cluster nodes should be located within a single region. Most cloud providers provide multiple availability zones within a region, which can be used to create higher availability for your cluster. Using multiple availability zones is fine for nodes with any role. If you are using [Kubernetes Cloud Provider]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/options/cloud-providers/) resources, consult the documentation for any restrictions (i.e. zone storage restrictions). + +## Cluster Diagram + +This diagram is applicable to Kubernetes clusters built using RKE or [Rancher Launched Kubernetes]({{< baseurl >}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/). + +![Cluster diagram]({{< baseurl >}}/img/rancher/clusterdiagram.svg)
+Lines show the traffic flow between components. Colors are used purely for visual aid + +## Production checklist + +* Nodes should have one of the following role configurations: + * `etcd` + * `controlplane` + * `etcd` and `controlplane` + * `worker` (the `worker` role should not be used or added on nodes with the `etcd` or `controlplane` role) +* Network traffic is only strictly allowed according to [Port Requirements]({{< baseurl >}}/rancher/v2.x/en/installation/references/). +* Have at least three nodes with the role `etcd` to survive losing one node. Increase this count for higher node fault toleration, and spread them across (availability) zones to provide even better fault tolerance. +* Assign two or more nodes the `controlplane` role for master component high availability. +* Assign two or more nodes the `worker` role for workload rescheduling upon node failure. +* Enable etcd snapshots. Verify that snapshots are being created, and run a disaster recovery scenario to verify the snapshots are valid. +* Perform load tests on your cluster to verify that its hardware can support your workloads. +* Configure alerts/notifiers for Kubernetes components (System Service). +* Configure logging for cluster analysis and post-mortems. + +## RKE cluster running Rancher HA + +You may have noticed that our [High Availability (HA) Install]({{< baseurl >}}/rancher/v2.x/en/installation/ha/) instructions do not meet our definition of a production-ready cluster, as there are no dedicated nodes for the `worker` role. However, for your Rancher installation, this three node cluster is valid, as: + +* It allows one `etcd` node failure. +* It maintains multiple instances of the master components by having multiple `controlplane` nodes. +* No other workloads than Rancher itself should be created on this cluster. diff --git a/src/diagrams/clusterdiagram.xml b/src/diagrams/clusterdiagram.xml new file mode 100644 index 00000000000..9606ecb5514 --- /dev/null +++ b/src/diagrams/clusterdiagram.xml @@ -0,0 +1 @@ +7Z1bb6u4FoB/TR5T4QsYHtvu2WdGoy1V6sPMfqSJm6JNQw6ht/Prj0mAwLIJlxhDZmilNjUGA+vzYt1wF+T+9fM/sb97+RGtebjA1vpzQb4tMEaUWOJX2vKVtbgeObZs4mCdtZ0aHoP/8awx23HzFqz5vtIxiaIwCXbVxlW03fJVUmnz4zj6qHZ7jsLqqDt/w6WGx5Ufyq1/Bevk5djqYnZq/50Hm5d8ZOR4xy1P/urXJo7ettl4C0yeD1/Hza9+fqzsQvcv/jr6KDWR3xbkPo6i5Pjp9fOeh+nNzW/bcb/vNVuL8475Nmmzg0OPe7z74RvPT9kJxb536+A9PcHkK7spzn/f0rO6S/hnsvTDYLNdkNtUBmIoHosPYtvhXm+T5f4g0XQrcnefp33Fp032+zBG2vkgxjCKK4OI+0bW6be8719R/Csd73gEcWnHg1QPLJoP55+34sql4IOEeHoPkNj88RIk/HHnr9KtHwJp0faSvIbZ5kJG6R+r6DVYic/p1e6TOPrF74uTJ8y99+6/F1tydHB6pUEYlnpaFrNvXdGe3chv2V0kd+88TgJB4m22IYl22Y367r8GYTqjdtFuF2wF4neh/8TDu4K5fIBttBWXcheJSwqSdA9iFXchPT7/rMUFFRCK2c2jV57EX6JLtgPBxz2yee24GcYfp0nCHPfY9lKaIOJ6s8mZTcxNcegTnOJDxmcN3LQeVhUDT7GCimrHqWHBV8/WM4LyjqOnKImy1lJvQqln/6aCSIeokXtjV6TNqJe3lOSNc9GW5V0o8kvkja9ONd2LfeIovYaH0BdTsJuO6ndtdeevpH/WiQPoRDhPsEor2opZQhxy+Swhs040phOFZVuRNEFIkjRymULSyL1c0qyTpPvorz/fnni85YkwvSVebh/+EId75PF7o/F1hZB5nk98tzVkKP1WQbaJ/XXAT50zraODPYaBniFE1jOIKvQM1fAwdlvBVwbIknHIno/h4Zn8w98KNyqeoTELjWcpjLjBsPE6YwNpeFy98PVbOINiGBSqtPYHAyV3GC6w9kP+nJiz9XmyWs9BiEkY3EvktLS4qZY4RL1jWm+IHdt+j/bJP894mpCFLjQnhMOT2VDHLLSose52EhR7ui3kyRXzcI2PO9tyzT3siOzKbTfB9nO5i6PPL5Xkpybi4tHRSsSFgjAoYkKhJlCEZQaTsO3MeuAK9IAMyRIb1AN2u5BO2dScMTCDgVFlcbmX/FD34JgZGYwRbNA9duwZkckjorAqHdtkEMXpliK4Qgim5GpaXkXWjLZ1NHUkxxmaDcwrUAlEClYxh5lUCaxdsGp+cEyLEhdTg4zINQRzOGJ4ETOjCRZXFZEAcuTb9W1aPZvektDf71PRlEXHP4Pk79Lnn6nkxEVI8lj73H1eqWTqrFz+9JzuvxUXUBws/eN0tOOJ8bVUpNt4t0t3UhX/z9tiHvpJ8F49vOruZiM8RMEhQ5QJk7r2jWszRPHhJ0EVwS6RdZNvEj8d1wVlPvvoLV7x7JgnCXYcxvOqG3F1kMSPNzyRBhHy9b9K3XZph339pdqWAy/u7DnD/siq9BcfjmdwYraQXjuMVQH2GeN+GDP3htTyxVxXRV9niM8O4npnB9EEMfUkK8z1zp+1tIfrVfa4GGRPZcB3BLkHrhn6loR+AbJ1fSATis8wRoXBzU4K2fWcfiSfHwVZyIQ+Jlgq1HDPn7W0A7KwXpJVJajDk3yG13rK25IsTv5AxaLiRB1lWDaaxyIeEbeqnTAwEdtCjdzqgRgDB9LFLQInnL3MUYst6O9ptiOQ3S5TP1dxjBBasykbt4pDkZGZPWLNHrHNoJBN5tyKkrE5gDrp0JhMidEMPXLaxdnnFL15DsyqC3yxupjj6OYhMZmjR44GR2iOTR17MOSZCLGeH8ZMiLVIHLcMscL+ukOsSLkoxBya6sWxzRwDoanzoxgKTdm20y00Je+gOzSFWqa/Zy9/BC//lP8cx8mf094GSuIsKGSjVjtrt2LM7OSPXDgpUWLWyWftqmtnJ988B2bVxeVv7cxOvnlIjDr5TFVjPTv5vZwjbKsdE91efsM4Ztz8Iovc0s2H/bW7+UxDKdXs5ufCAVVOVdnp8vMbhjHk6LuwNqrBz5f6a3fzexe37sUtSc4SXjxkLenxqZgAlmVbvIJ5UZDys0z231Xqfy5qHtzF8dpOgGrhSmHZlitXspzWaEofy+t3wCdza8XO5GN54FiaoEdyuALBxXXBLo5cxgB20UB+X3NkOPJPfDMJ8NO0wHBe6GY/f7m6gv5xPoyF/tJBNdGQruAvETwSGQZ7ophhVoPhgmjDLpdjn0/yCWFfkM3OaHw8uMpXYj+uxpdWEu6r7h0Fi4NQT2ln6h0E62wHoH5+v9dceoJ61QpU5WKvnmLWaFnsFXl96lPn5V7/KVEuG4242Cvyuq9IMi/3OgVo4Bolhpd7LbK184KvV4eK4QVfsTWvYHGdoBhdKhFnocI5J3dlkJhd+QhbGmpvu3n+3w9fiz5vIuILXkVcTC6ohXLTNLc5+rr3RbI/z+sOFMp11Odb69mD8lxHr1ePLQ3ltn3RVSaicSUbd0FeGSQjcpd2QqGp4nXXwsnvyW7xAm7OrjMMu9IJN8AL++unV1U7Nb7ibY9omcZj8HQsGIv/kgd9/s6K1AYHgsve60oOWEPDpWHRokvhUui/U6zfIl5FdVpFWuuBx4G41NTw+mZp1pnjPvARfEz3TWJhYDgMlcOykaccp/a8PPV5te2vfxpoqDkbYBpco461obHaV8dSOAug1asLXjI0XKpY7/gP8JOO9SoJ1aV+JTu9jClxNKlYisyoWLgYLm7IltrgHcau/bXPAuX/nxp0FmT/F7wuRsDOlUQeNioqB9Kv/tOAyPNgVF29hK63tDhRa4MYg3nA4HpJw4QWvIYaXxiJ0E61hkXm+lJdYziwsnJHMvImqM6s6rG4doHbZPfF2gV+nj0Q1TCI0IS1FKTQzjU2zXVR2V5fv95vrbp+VDuKut5xdbXTgGL7ysbqgehAC9IhOHeyGvLa0nsYU7F1M208gzH62xog1qEIdYzLNGLaoMYjUY3Pv5oxPNXGkxsTo9qbGtRA4hQm2RqZLupFzCCcT5XxEDae4ZgYwtPTyyAaYMP8Rmu9DCK31DbEdOaYjce08YiyZEDLLz+fizXXz4R+TBd1RRPyCx1g+tqdVXN+IGiMD6SqHRADb7KhYX/9WGt4pfnKsVZkXEbGmkBK+tZYUJAKoQPVWEhYNxjRw2NtPEUzOaytyWG9hDT2ToOD4wyUooGUNtnVg1OdX/e/luriNZsJUU3VkHSGGhrWQ+VbINQNhvXwUBuP4nFbuEPVd+vL+XJUhtq6sWxaATv/A2bSJdaLYXqyPrnsInQiMfT9Wi/KAsuFWlbb9cLLeDitG16ezcbBy56eKnUgYL1fe6fgTeSWyrQXYMaDXQrA8BnCmhdQGAKlcWvWYDVQb5BAgGGotXKcPJcFxqm1gOnZ/hqeysbriBVUs1GpnmAdhAMfnH3Ta9KK+ANl1zB4KR81uFC2e7a/Bq6Nh3E7aes2PtSFXBNF8ea4ZiUo7ulduglzAMOt8FQt3UQNbxQRONk69tfvchkP+nZT7uNMgrGDvtdWwUxpt2lAvW7TAPa/cBqIP+MoXc3n1D32dy8/ojVPe/wf \ No newline at end of file diff --git a/src/img/rancher/clusterdiagram.svg b/src/img/rancher/clusterdiagram.svg new file mode 100644 index 00000000000..0a12d697a61 --- /dev/null +++ b/src/img/rancher/clusterdiagram.svg @@ -0,0 +1,2 @@ + +
Worker
[Not supported by viewer]

<font><br></font>
Control Plane

[Not supported by viewer]

<font><br></font>
Kubernetes
API Server
[Not supported by viewer]
Kubernetes
Controller Manager
[Not supported by viewer]
Kubernetes
Scheduler
[Not supported by viewer]
etcd
[Not supported by viewer]
Host
[Not supported by viewer]
Kubernetes
Kubelet
[Not supported by viewer]
nginx-proxy
nginx-proxy<br>
Kubernetes
Kubelet
[Not supported by viewer]
etcd
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer]

<font><br></font>
Kubernetes
Kubelet
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer]
nginx-proxy
nginx-proxy<br>
Host
[Not supported by viewer]
nginx-proxy
nginx-proxy<br>
Kubernetes
Kubelet
[Not supported by viewer]
etcd
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer]
Host
[Not supported by viewer]
nginx-proxy
nginx-proxy<br>
Kubernetes
Kubelet
[Not supported by viewer]
etcd
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer]

<font><br></font>
Kubernetes
API Server
[Not supported by viewer]
Kubernetes
Controller Manager
[Not supported by viewer]
Kubernetes
Scheduler
[Not supported by viewer]
Kubernetes
Kubelet
[Not supported by viewer]
Kubernetes
Proxy
[Not supported by viewer] \ No newline at end of file From 98978c4241406e8e7397d05c158ef06a9182814f Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 26 Nov 2018 12:33:02 -0700 Subject: [PATCH 23/24] adding aliases for air gap changes --- .../v2.x/en/installation/air-gap-high-availability/_index.md | 2 +- .../air-gap-high-availability/install-rancher/_index.md | 1 + .../prepare-private-registry/_index.md | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md index acf44ed8fb1..5ccf9b0fa9d 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/_index.md @@ -2,7 +2,7 @@ title: "Air Gap: High Availability Install" weight: 290 aliases: - - /rancher/v2.x/en/installation/air-gap-installation/install-rancher/ + - /rancher/v2.x/en/installation/air-gap-installation/ --- ## Prerequisites diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md index 758758c3e73..59fbb8f4540 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/install-rancher/_index.md @@ -2,6 +2,7 @@ title: 4. Install Rancher weight: 400 aliases: + - /rancher/v2.x/en/installation/air-gap-installation/install-rancher/ --- ## A. Add the Helm Chart Repository diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index 0dccd3b3818..1ff4b401f8f 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -2,7 +2,7 @@ title: "2. Prepare Private Registry" weight: 200 aliases: - + - /rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/ --- ## A. Collect Images From c82bf82a3d708186fa123fa3c1f24fb86e91d57c Mon Sep 17 00:00:00 2001 From: Mark Bishop Date: Mon, 26 Nov 2018 12:46:40 -0700 Subject: [PATCH 24/24] fixing path --- .../prepare-private-registry/_index.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md index 1ff4b401f8f..bc631103e63 100644 --- a/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md +++ b/content/rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/_index.md @@ -2,12 +2,12 @@ title: "2. Prepare Private Registry" weight: 200 aliases: - - /rancher/v2.x/en/installation/air-gap-high-availability/prepare-private-registry/ + - /rancher/v2.x/en/installation/air-gap-installation/prepare-private-reg/ --- ## A. Collect Images -Start by collecting all the images needed to install Rancher in an air gap environment. You'll collect images from your chosen Rancher release, RKE, and (if you're using a self-signed TLS certificate) Cert-Manager. +Start by collecting all the images needed to install Rancher in an air gap environment. You'll collect images from your chosen Rancher release, RKE, and (if you're using a self-signed TLS certificate) Cert-Manager. 1. Using a computer with internet access, browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.1.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. @@ -24,28 +24,28 @@ Start by collecting all the images needed to install Rancher in an air gap envir 1. Make `rancher-save-images.sh` an executable. - + ``` chmod +x rancher-save-images.sh ``` - -1. From the directory that contains the RKE binary, add RKE's images to `rancher-images.txt`, which is a list of all the files needed to install Rancher. - + +1. From the directory that contains the RKE binary, add RKE's images to `rancher-images.txt`, which is a list of all the files needed to install Rancher. + ``` rke config --system-images >> ./rancher-images.txt ``` 1. **Default Rancher Generated Self-Signed Certificate Users Only:** If you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://github.com/helm/charts/tree/master/stable/cert-manager) image to `rancher-images.txt` as well. You may skip to [B. Publish Images](#b-publish-images ) if you are using you using your own certificates. - + 1. Fetch the latest `cert-manager` Helm chart and parse the template for image details. - + ```plain helm fetch stable/cert-manager helm template ./cert-manager-.tgz | grep -oP '(?<=image: ").*(?=")' >> ./rancher-images.txt ``` - + 2. Sort and unique the images list to remove any overlap between the sources. - + ```plain sort -u rancher-images.txt -o rancher-images.txt ``` @@ -54,8 +54,8 @@ Start by collecting all the images needed to install Rancher in an air gap envir ```plain ./rancher-save-images.sh --image-list ./rancher-images.txt - ``` - + ``` + **Step Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory. ## B. Publish Images