Merge pull request #3568 from btat/merge-master-to-staging

Merge master to staging
This commit is contained in:
Billy Tat
2021-09-29 20:33:02 -07:00
committed by GitHub
63 changed files with 316 additions and 418 deletions
+75 -1
View File
@@ -11,6 +11,7 @@ This section contains advanced information describing the different ways you can
- [Certificate rotation](#certificate-rotation)
- [Auto-deploying manifests](#auto-deploying-manifests)
- [Using Docker as the container runtime](#using-docker-as-the-container-runtime)
- [Using etcdctl](#using-etcdctl)
- [Configuring containerd](#configuring-containerd)
- [Secrets Encryption Config (Experimental)](#secrets-encryption-config-experimental)
- [Running K3s with Rootless mode (Experimental)](#running-k3s-with-rootless-mode-experimental)
@@ -22,6 +23,7 @@ This section contains advanced information describing the different ways you can
- [Enabling cgroups for Raspbian Buster](#enabling-cgroups-for-raspbian-buster)
- [SELinux Support](#selinux-support)
- [Additional preparation for (Red Hat/CentOS) Enterprise Linux](#additional-preparation-for-red-hat-centos-enterprise-linux)
- [Enabling Lazy Pulling of eStargz (Experimental)](#enabling-lazy-pulling-of-estargz-experimental)
# Certificate Rotation
@@ -31,7 +33,7 @@ If the certificates are expired or have fewer than 90 days remaining before they
# Auto-Deploying Manifests
Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`.
Any file found in `/var/lib/rancher/k3s/server/manifests` will automatically be deployed to Kubernetes in a manner similar to `kubectl apply`, both on startup and when the file is changed on disk. Deleting files out of this directory will not delete the corresponding resources from the cluster.
For information about deploying Helm charts, refer to the section about [Helm.](../helm)
@@ -116,6 +118,24 @@ rancher/metrics-server v0.3.6 9dd718864ce61 39.9MB
rancher/pause 3.1 da86e6ba6ca19 742kB
```
# Using etcdctl
etcdctl provides a CLI for etcd.
If you would like to use etcdctl after installing K3s with embedded etcd, install etcdctl using the [official documentation.](https://etcd.io/docs/latest/install/)
```
$ VERSION="v3.5.0"
$ curl -L https://github.com/etcd-io/etcd/releases/download/${VERSION}/etcd-${VERSION}-linux-amd64.tar.gz --output etcdctl-${VERSION}-linux-amd64.tar.gz
$ sudo tar -zxvf etcdctl-${VERSION}-linux-amd64.tar.gz -C /usr/local/bin
```
Then start using etcdctl commands with the appropriate K3s flags:
```
$ sudo etcdctl --cacert=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --cert=/var/lib/rancher/k3s/server/tls/etcd/client.crt --key=/var/lib/rancher/k3s/server/tls/etcd/client.key version
```
# Configuring containerd
K3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`.
@@ -394,3 +414,57 @@ If enabled, it is required to disable nm-cloud-setup and reboot the node:
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
reboot
```
# Enabling Lazy Pulling of eStargz (Experimental)
## What's lazy pulling and eStargz?
Pulling images is known as one of the time-consuming steps in the container lifecycle.
According to [Harter, et al.](https://www.usenix.org/conference/fast16/technical-sessions/presentation/harter),
> pulling packages accounts for 76% of container start time, but only 6.4% of that data is read
To address this issue, k3s experimentally supports *lazy pulling* of image contents.
This allows k3s to start a container before the entire image has been pulled.
Instead, the necessary chunks of contents (e.g. individual files) are fetched on-demand.
Especially for large images, this technique can shorten the container startup latency.
To enable lazy pulling, the target image needs to be formatted as [*eStargz*](https://github.com/containerd/stargz-snapshotter/blob/main/docs/stargz-estargz.md).
This is an OCI-alternative but 100% OCI-compatible image format for lazy pulling.
Because of the compatibility, eStargz can be pushed to standard container registries (e.g. ghcr.io) as well as this is *still runnable* even on eStargz-agnostic runtimes.
eStargz is developed based on the [stargz format proposed by Google CRFS project](https://github.com/google/crfs) but comes with practical features including content verification and performance optimization.
For more details about lazy pulling and eStargz, please refer to [Stargz Snapshotter project repository](https://github.com/containerd/stargz-snapshotter).
## Configure k3s for lazy pulling of eStargz
As shown in the following, `--snapshotter=stargz` option is needed for k3s server and agent.
```
k3s server --snapshotter=stargz
```
With this configuration, you can perform lazy pulling for eStargz-formatted images.
The following Pod manifest uses eStargz-formatted `node:13.13.0` image (`ghcr.io/stargz-containers/node:13.13.0-esgz`).
k3s performs lazy pulling for this image.
```
apiVersion: v1
kind: Pod
metadata:
name: nodejs
spec:
containers:
- name: nodejs-estargz
image: ghcr.io/stargz-containers/node:13.13.0-esgz
command: ["node"]
args:
- -e
- var http = require('http');
http.createServer(function(req, res) {
res.writeHead(200);
res.end('Hello World!\n');
}).listen(80);
ports:
- containerPort: 80
```
@@ -33,7 +33,7 @@ As of Rancher v2.4.0,
# Tested Kubernetes Versions
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For example, Rancher v2.3.0 is was tested with Kubernetes v1.15.4, v1.14.7, and v1.13.11. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.4.17/)
# How Upgrades Work
@@ -16,6 +16,8 @@ The pod can be scheduled to any of the hosts you used for your cluster, but that
To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
> **Note:** This container [does not support ARM nodes](https://github.com/leodotcloud/swiss-army-knife/issues/18), such as a Raspberry Pi. This will be seen in the pod logs as `exec user process caused: exec format error`.
1. Save the following file as `overlaytest.yml`
```
+1 -1
View File
@@ -8,7 +8,7 @@ aliases:
In this section, you'll learn how to create backups of Rancher, how to restore Rancher from backup, and how to migrate Rancher to a new Kubernetes cluster.
As of Rancher v2.5, the `rancher-backup` operator is used to backup and restore Rancher. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/main/charts/rancher-backup)
As of Rancher v2.5, the `rancher-backup` operator is used to backup and restore Rancher. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/release-v2.5/charts/rancher-backup)
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
@@ -15,7 +15,7 @@ This guide outlines a reference architecture for provisioning downstream Rancher
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/rancher/solution_overview.drawio.svg)
![Solution Overview]({{<baseurl>}}/img/rancher/solution_overview.drawio.svg)
# 1. VM Considerations
@@ -108,7 +108,7 @@ Monitoring the availability and performance of all your internal workloads is vi
In addition to monitoring workloads to detect performance, availability or scalability problems, the cluster and the workloads running into it should also be monitored for potential security problems. A good starting point is to frequently run and alert on [CIS Scans]({{<baseurl>}}/rancher/v2.5/en/cis-scans/v2.5/) which check if the cluster is configured according to security best practices.
For the workloads, you can have a look at Kubernetes and Container security solutions like [Falko](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).
For the workloads, you can have a look at Kubernetes and Container security solutions like [Falco](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).
# Setting up Alerts
@@ -120,4 +120,4 @@ When setting up alerts, configure them for all the workloads that are critical t
If an alert starts firing, but there is nothing you can do about it at the moment, it's also fine to silence the alert for a certain amount of time, so that you can look at it later.
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting).
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting).
@@ -22,8 +22,8 @@ Rancher provides an intuitive user interface for interacting with your clusters.
You can use the Kubernetes command-line tool, [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/), to manage your clusters. You have two options for using kubectl:
- **Rancher kubectl shell:** Interact with your clusters by launching a kubectl shell available in the Rancher UI. This option requires no configuration actions on your part. For more information, see [Accessing Clusters with kubectl Shell]({{<baseurl>}}/rancher/v2.5/en/k8s-in-rancher/kubectl/).
- **Terminal remote connection:** You can also interact with your clusters by installing [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your local desktop and then copying the cluster's kubeconfig file to your local `~/.kube/config` directory. For more information, see [Accessing Clusters with kubectl and a kubeconfig File](./kubectl/).
- **Rancher kubectl shell:** Interact with your clusters by launching a kubectl shell available in the Rancher UI. This option requires no configuration actions on your part. For more information, see [Accessing Clusters with kubectl Shell]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/).
- **Terminal remote connection:** You can also interact with your clusters by installing [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your local desktop and then copying the cluster's kubeconfig file to your local `~/.kube/config` directory. For more information, see [Accessing Clusters with kubectl and a kubeconfig File]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/).
### Rancher CLI
@@ -11,7 +11,7 @@ aliases:
If you want to provide a user with access and permissions to _all_ projects, nodes, and resources within a cluster, assign the user a cluster membership.
>**Tip:** Want to provide a user with access to a _specific_ project within a cluster? See [Adding Project Members]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/projects-and-namespaces/project-members/) instead.
>**Tip:** Want to provide a user with access to a _specific_ project within a cluster? See [Adding Project Members]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/cluster-project-roles/) instead.
There are two contexts where you can add cluster members:
@@ -108,13 +108,13 @@ The address range assigned to the services in the cluster. Must be a valid CIDR
_Mutable: no_
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/private-clusters/).
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/editing-clusters/gke-config-reference/private-clusters/).
Assign nodes only internal IP addresses. Private cluster nodes cannot access the public internet unless additional networking steps are taken in GCP.
### Enable Private Endpoint
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{< baseurl >}}/rancher/v2.5/en/cluster-provisioning/hosted-kubernetes-clusters/gke/#private-clusters).
> Warning: private clusters require additional planning and configuration outside of Rancher. Refer to the [private cluster guide]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/editing-clusters/gke-config-reference/private-clusters/).
_Mutable: no_
@@ -26,7 +26,7 @@ This section covers the following topics:
# Tested Kubernetes Versions
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For example, Rancher v2.3.0 is was tested with Kubernetes v1.15.4, v1.14.7, and v1.13.11. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.5.9/)
# How Upgrades Work
@@ -115,9 +115,7 @@ To give role-based access to your service principal,
Use Rancher to set up and configure your Kubernetes cluster.
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Create**.
1. From the **Clusters** page, click **Add Cluster**.
1. Choose **Azure AKS**.
@@ -125,7 +123,8 @@ Use Rancher to set up and configure your Kubernetes cluster.
1. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
1. Use your subscription ID, client ID, and client secret to give your cluster access to AKS. If you don't have all of that information, you can retrieve it using these instructions:
1. Use your subscription ID, tenant ID, client ID, and client secret to give your cluster access to AKS. If you don't have all of that information, you can retrieve it using these instructions:
- **Tenant ID:** To get the Tenant ID, you can go to the Azure Portal, then click **Azure Active Directory**, then click **Properties** and find the **Tenant ID** field.
- **Client ID:** To get the Client ID, you can go to the Azure Portal, then click **Azure Active Directory**, then click **Enterprise applications.** Click **All applications.** Select your application, click **Properties,** and copy the application ID.
- **Client secret:** If you didn't copy the client secret when creating the service principal, you can get a new one if you go to the app registration detail page, then click **Certificates & secrets**, then click **New client secret.**
- **Subscription ID:** You can get the subscription ID is available in the portal from **All services > Subscriptions.**
@@ -50,9 +50,7 @@ For more detailed information on IAM policies for EKS, refer to the official [do
Use Rancher to set up and configure your Kubernetes cluster.
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Create**.
1. From the **Clusters** page, click **Add Cluster**.
1. Choose **Amazon EKS**.
@@ -64,9 +64,8 @@ To get the project ID of an existing project, refer to the Google cloud document
### 2. Create the GKE Cluster
Use Rancher to set up and configure your Kubernetes cluster.
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Create**.
1. Click **Google GKE**.
1. From the **Clusters** page, click **Add Cluster**.
1. Under **With a hosted Kubernetes provider,** click **Google GKE**.
1. Enter a **Cluster Name**.
1. Optional: Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
1. Optional: Add Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) or [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to the cluster.
@@ -38,9 +38,8 @@ If you are registering a K3s cluster, make sure the `cluster.yml` is readable. I
# Registering a Cluster
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Import Existing**.
2. Click the type of Kubernetes cluster you want to import.
1. From the **Clusters** page, click **Add Cluster**.
2. Under **Register an existing Kubernetes cluster**, click the type of Kubernetes cluster you want to register.
3. Enter a **Cluster Name**.
4. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
5. For Rancher v2.5.6+, use **Agent Environment Variables** under **Cluster Options** to set environment variables for [rancher cluster agent]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/rancher-agents/). The environment variables can be set using key value pairs. If rancher agent requires use of proxy to communicate with Rancher server, `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` environment variables can be set using agent environment variables.
@@ -101,7 +100,7 @@ After registering a cluster, the cluster owner can:
- [Manage cluster access]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/cluster-project-roles/) through role-based access control
- Enable [monitoring, alerts and notifiers]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/)
- Enable [logging]({{<baseurl>}}/rancher/v2.5/en/logging/v2.5/)
- Enable [Istio]({{<baseurl>}}/rancher/v2.5/en/istio/v2.5/)
- Enable [Istio]({{<baseurl>}}/rancher/v2.5/en/istio/)
- Use [pipelines]({{<baseurl>}}/rancher/v2.5/en/project-admin/pipelines/)
- Manage projects and workloads
@@ -142,7 +141,7 @@ After registering a cluster, the cluster owner can:
- [Manage cluster access]({{<baseurl>}}/rancher/v2.5/en/admin-settings/rbac/cluster-project-roles/) through role-based access control
- Enable [monitoring, alerts and notifiers]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/)
- Enable [logging]({{<baseurl>}}/rancher/v2.5/en/logging/v2.5/)
- Enable [Istio]({{<baseurl>}}/rancher/v2.5/en/istio/v2.5/)
- Enable [Istio]({{<baseurl>}}/rancher/v2.5/en/istio/)
- Use [pipelines]({{<baseurl>}}/rancher/v2.5/en/project-admin/pipelines/)
- Manage projects and workloads
@@ -25,10 +25,8 @@ The Cloud Provider Interface (CPI) should be installed first before installing t
### 1. Create a vSphere cluster
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Create.**
1. Click **VMWare vSphere.**
1. Under **Cluster Options** in the **Cloud Provider** section, select **External (Out-of-tree)**. This sets the cloud provider option on the Kubernetes cluster to `external` which sets your Kubernetes cluster up to be configured with an out-of-tree cloud provider.
1. On the Clusters page, click on **Add Cluster** and select the **vSphere** option or **Existing Nodes** option.
1. Under **Cluster Options** in the **Cloud Provider** section, select **External (Out-of-tree)**. This sets the cloud provider option on the Kubernetes cluster to `external` which sets your Kubernetes cluster up to be configured with an out-of-tree cloud provider.
1. Finish creating your cluster.
### 2. Install the CPI plugin
@@ -45,40 +45,38 @@ Provision the host according to the [installation requirements]({{<baseurl>}}/ra
Clusters won't begin provisioning until all three node roles (worker, etcd and controlplane) are present.
1. In the upper left corner, click **≡ > Cluster Management.**
1. From the **Clusters** page, click **Add Cluster**.
1. From the **Clusters** page, click **Create.**
2. Choose **Custom**.
1. Click **Custom.**
3. Enter a **Cluster Name**.
1. Enter a **Cluster Name**.
4. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
1. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
1. Use **Cluster Options** to choose the version of Kubernetes, what network provider will be used and if you want to enable project network isolation. To see more cluster options, click on **Show advanced options.**
5. Use **Cluster Options** to choose the version of Kubernetes, what network provider will be used and if you want to enable project network isolation. To see more cluster options, click on **Show advanced options.**
>**Using Windows nodes as Kubernetes workers?**
>
>- See [Enable the Windows Support Option]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/windows-clusters/).
>- The only Network Provider available for clusters with Windows support is Flannel.
1. <a id="step-6"></a>Click **Next**.
6. <a id="step-6"></a>Click **Next**.
1. From **Node Role**, choose the roles that you want filled by a cluster node. You must provision at least one node for each role: `etcd`, `worker`, and `control plane`. All three roles are required for a custom cluster to finish provisioning. For more information on roles, see [this section.]({{<baseurl>}}/rancher/v2.5/en/overview/concepts/#roles-for-nodes-in-kubernetes-clusters)
7. From **Node Role**, choose the roles that you want filled by a cluster node. You must provision at least one node for each role: `etcd`, `worker`, and `control plane`. All three roles are required for a custom cluster to finish provisioning. For more information on roles, see [this section.]({{<baseurl>}}/rancher/v2.5/en/overview/concepts/#roles-for-nodes-in-kubernetes-clusters)
>**Notes:**
>
>- Using Windows nodes as Kubernetes workers? See [this section]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/windows-clusters/).
>- Bare-Metal Server Reminder: If you plan on dedicating bare-metal servers to each role, you must provision a bare-metal server for each role (i.e. provision multiple bare-metal servers).
1. <a id="step-8"></a>**Optional**: Click **[Show advanced options]({{<baseurl>}}/rancher/v2.5/en/admin-settings/agent-options/)** to specify IP address(es) to use when registering the node, override the hostname of the node, or to add [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) or [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to the node.
8. <a id="step-8"></a>**Optional**: Click **[Show advanced options]({{<baseurl>}}/rancher/v2.5/en/admin-settings/agent-options/)** to specify IP address(es) to use when registering the node, override the hostname of the node, or to add [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) or [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to the node.
1. Copy the command displayed on screen to your clipboard.
9. Copy the command displayed on screen to your clipboard.
1. Log in to your Linux host using your preferred shell, such as PuTTy or a remote Terminal connection. Run the command copied to your clipboard.
10. Log in to your Linux host using your preferred shell, such as PuTTy or a remote Terminal connection. Run the command copied to your clipboard.
>**Note:** Repeat steps 7-10 if you want to dedicate specific hosts to specific node roles. Repeat the steps as many times as needed.
1. When you finish running the command(s) on your Linux host(s), click **Done**.
11. When you finish running the command(s) on your Linux host(s), click **Done**.
**Result:**
@@ -101,8 +101,8 @@ When you create the node pool, you can specify the amount of time in minutes tha
You can also enable node auto-replace after the cluster is created with the following steps:
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the list of clusters, go to the cluster where you want to enable node auto-replace. Click the vertical &#8942; **(…)**, and click **Edit Config.**
1. From the Global view, click the Clusters tab.
1. Go to the cluster where you want to enable node auto-replace, click the vertical &#8942; **(…)**, and click **Edit.**
1. In the **Node Pools** section, go to the node pool where you want to enable node auto-replace. In the **Recreate Unreachable After** field, enter the number of minutes that Rancher should wait for a node to respond before replacing the node.
1. Click **Save.**
@@ -112,8 +112,8 @@ You can also enable node auto-replace after the cluster is created with the foll
You can disable node auto-replace from the Rancher UI with the following steps:
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the list of clusters, go to the cluster where you want to enable node auto-replace. Click the vertical &#8942; **(…)**, and click **Edit Config.**
1. From the Global view, click the Clusters tab.
1. Go to the cluster where you want to enable node auto-replace, click the vertical &#8942; **(…)**, and click **Edit.**
1. In the **Node Pools** section, go to the node pool where you want to enable node auto-replace. In the **Recreate Unreachable After** field, enter 0.
1. Click **Save.**
@@ -46,11 +46,10 @@ The creation of this service principal returns three pieces of identification in
### 1. Create your cloud credentials
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Cloud Credentials.**
1. Click **Create.**
1. Click **Azure.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Cloud Credentials.**
1. Click **Add Cloud Credential.**
1. Enter a name for the cloud credential.
1. In the **Cloud Credential Type** field, select **Azure**.
1. Enter your Azure credentials.
1. Click **Create.**
@@ -60,8 +59,7 @@ The creation of this service principal returns three pieces of identification in
Creating a [node template]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) for Azure will allow Rancher to provision new nodes in Azure. Node templates can be reused for other clusters.
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Node Templates.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Node Templates.**
1. Click **Add Template.**
1. Fill out a node template for Azure. For help filling out the form, refer to [Azure Node Template Configuration.](./azure-node-template-config)
@@ -71,8 +69,7 @@ Use Rancher to create a Kubernetes cluster in Azure.
Clusters won't begin provisioning until all three node roles (worker, etcd and controlplane) are present.
1. In the upper left corner, click **≡ > Cluster Management**.
1. Click **Create**.
1. From the **Clusters** page, click **Add Cluster**.
1. Choose **Azure**.
1. Enter a **Cluster Name**.
1. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
@@ -19,11 +19,10 @@ Then you will create a DigitalOcean cluster in Rancher, and when configuring the
### 1. Create your cloud credentials
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Cloud Credentials.**
1. Click **Create.**
1. Click **Digital Ocean.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Cloud Credentials.**
1. Click **Add Cloud Credential.**
1. Enter a name for the cloud credential.
1. In the **Cloud Credential Type** field, select **DigitalOcean**.
1. Enter your Digital Ocean credentials.
1. Click **Create.**
@@ -33,16 +32,15 @@ Then you will create a DigitalOcean cluster in Rancher, and when configuring the
Creating a [node template]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) for DigitalOcean will allow Rancher to provision new nodes in DigitalOcean. Node templates can be reused for other clusters.
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Node Templates.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Node Templates.**
1. Click **Add Template.**
1. Fill out a node template for DigitalOcean. For help filling out the form, refer to [DigitalOcean Node Template Configuration.](./do-node-template-config)
### 3. Create a cluster with node pools using the node template
Clusters won't begin provisioning until all three node roles (worker, etcd and controlplane) are present.
1. In the upper left corner, click **≡ > Cluster Management**.
1. Click **Create**.
1. From the **Clusters** page, click **Add Cluster**.
1. Choose **DigitalOcean**.
1. Enter a **Cluster Name**.
1. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
@@ -31,13 +31,12 @@ The steps to create a cluster differ based on your Rancher version.
### 1. Create your cloud credentials
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Cloud Credentials.**
1. Click **Create.**
1. Click **Amazon.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Cloud Credentials.**
1. Click **Add Cloud Credential.**
1. Enter a name for the cloud credential.
1. In the **Cloud Credential Type** field, select **Amazon.**
1. In the **Region** field, select the AWS region where your cluster nodes will be located.
1. Enter your AWS EC2 **Access Key** and **Secret Key.**
1. In the **Default Region** field, select the AWS region where your cluster nodes will be located.
1. Click **Create.**
**Result:** You have created the cloud credentials that will be used to provision nodes in your cluster. You can reuse these credentials for other node templates, or in other clusters.
@@ -46,8 +45,8 @@ The steps to create a cluster differ based on your Rancher version.
Creating a [node template]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) for EC2 will allow Rancher to provision new nodes in EC2. Node templates can be reused for other clusters.
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Node Templates.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Node Templates.**
1. Click **Add Template.**
1. Fill out a node template for EC2. For help filling out the form, refer to [EC2 Node Template Configuration.](./ec2-node-template-config)
### 3. Create a cluster with node pools using the node template
@@ -56,8 +55,7 @@ Add one or more node pools to your cluster. For more information about node pool
Clusters won't begin provisioning until all three node roles (worker, etcd and controlplane) are present.
1. In the upper left corner, click **≡ > Cluster Management**.
1. Click **Create**.
1. From the **Clusters** page, click **Add Cluster**.
1. Choose **Amazon EC2**.
1. Enter a **Cluster Name**.
1. Create a node pool for each Kubernetes role. For each node pool, choose a node template that you created. For more information about node pools, including best practices for assigning Kubernetes roles to them, see [this section.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools)
@@ -75,7 +73,6 @@ You can access your cluster after its state is updated to **Active.**
- `Default`, containing the `default` namespace
- `System`, containing the `cattle-system`, `ingress-nginx`, `kube-public`, and `kube-system` namespaces
### Optional Next Steps
After creating your cluster, you can access it through the Rancher UI. As a best practice, we recommend setting up these alternate ways of accessing your cluster:
@@ -58,10 +58,10 @@ The a vSphere cluster is created in Rancher depends on the Rancher version.
### 1. Create your cloud credentials
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Cloud Credentials.**
1. Click **Create.**
1. Click **VMware vSphere.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Cloud Credentials.**
1. Click **Add Cloud Credential.**
1. Enter a name for the cloud credential.
1. In the **Cloud Credential Type** field, select **VMware vSphere**.
1. Enter your vSphere credentials. For help, refer to **Account Access** in the [node template configuration reference.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/vsphere-node-template-config/)
1. Click **Create.**
@@ -71,8 +71,8 @@ The a vSphere cluster is created in Rancher depends on the Rancher version.
Creating a [node template]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/#node-templates) for vSphere will allow Rancher to provision new nodes in vSphere. Node templates can be reused for other clusters.
1. In the upper left corner, click **≡ > Cluster Management.**
1. In the left navigation menu, click **Node Templates.**
1. In the Rancher UI, click the user profile button in the upper right corner, and click **Node Templates.**
1. Click **Add Template.**
1. Fill out a node template for vSphere. For help filling out the form, refer to the vSphere node template [configuration reference.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/node-pools/vsphere/vsphere-node-template-config/).
### 3. Create a cluster with node pools using the node template
@@ -81,9 +81,8 @@ Use Rancher to create a Kubernetes cluster in vSphere.
Clusters won't begin provisioning until all three node roles (worker, etcd and controlplane) are present.
1. In the upper left corner, click **≡ > Cluster Management**.
1. Click **Create**.
1. Select the **VMware vSphere** infrastructure provider.
1. Navigate to **Clusters** in the **Global** view.
1. Click **Add Cluster** and select the **vSphere** infrastructure provider.
1. Enter a **Cluster Name.**
1. Use **Member Roles** to configure user authorization for the cluster. Click **Add Member** to add users that can access the cluster. Use the **Role** drop-down to set permissions for each user.
1. Use **Cluster Options** to choose the version of Kubernetes that will be installed, what network provider will be used and if you want to enable project network isolation. To see more cluster options, click on **Show advanced options.** For help configuring the cluster, refer to the [RKE cluster configuration reference.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/options)
@@ -109,4 +108,4 @@ After creating your cluster, you can access it through the Rancher UI. As a best
- **Access your cluster with the kubectl CLI:** Follow [these steps]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#accessing-clusters-with-kubectl-on-your-workstation) to access clusters with kubectl on your workstation. In this case, you will be authenticated through the Rancher servers authentication proxy, then Rancher will connect you to the downstream cluster. This method lets you manage the cluster without the Rancher UI.
- **Access your cluster with the kubectl CLI, using the authorized cluster endpoint:** Follow [these steps]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/cluster-access/kubectl/#authenticating-directly-with-a-downstream-cluster) to access your cluster with kubectl directly, without authenticating through Rancher. We recommend setting up this alternative method to access your cluster so that in case you cant connect to Rancher, you can still access the cluster.
- **Provision Storage:** For an example of how to provision storage in vSphere using Rancher, refer to [this section.]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/examples/vsphere) In order to dynamically provision storage in vSphere, the vSphere provider must be [enabled.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere)
- **Provision Storage:** For an example of how to provision storage in vSphere using Rancher, refer to [this section.]({{<baseurl>}}/rancher/v2.5/en/cluster-admin/volumes-and-storage/examples/vsphere) In order to dynamically provision storage in vSphere, the vSphere provider must be [enabled.]({{<baseurl>}}/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere)
@@ -34,7 +34,7 @@ You can check the default firewall rules with this command:
sudo iptables --list
```
This section describes how to use `firewalld` to apply the [firewall port rules]({{<baseurl>}}/rancher/v2.5/en/installation/references) for nodes in a high-availability Rancher server cluster.
This section describes how to use `firewalld` to apply the [firewall port rules]({{<baseurl>}}/rancher/v2.5/en/installation/requirements/ports) for nodes in a high-availability Rancher server cluster.
# Prerequisite
@@ -12,8 +12,6 @@ This section describes how to install a Kubernetes cluster. This cluster should
> As of Rancher v2.5, Rancher can run on any Kubernetes cluster, included hosted Kubernetes solutions such as Amazon EKS. The below instructions represent only one possible way to install Kubernetes.
The Rancher management server can only be run on Kubernetes cluster in an infrastructure provider where Kubernetes is installed using RKE or K3s. Use of Rancher on hosted Kubernetes providers, such as EKS, is not supported.
For systems without direct internet access, refer to [Air Gap: Kubernetes install.]({{<baseurl>}}/rancher/v2.5/en/installation/air-gap-high-availability/)
> **Single-node Installation Tip:**
@@ -6,6 +6,7 @@ aliases:
- /rancher/v2.5/en/installation/air-gap-high-availability/config-rancher-system-charts/_index.md
- /rancher/v2.5/en/installation/options/local-system-charts
- /rancher/v2.x/en/installation/resources/local-system-charts/
- /rancher/v2.x/en/installation/options/local-system-charts/
---
The [System Charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS.
@@ -122,13 +122,13 @@ Let's say you wanted to send all logs in your cluster to an `elasticsearch` clus
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: "example-es"
namespace: "cattle-logging-system"
name: "example-es"
namespace: "cattle-logging-system"
spec:
elasticsearch:
host: elasticsearch.example.com
port: 9200
scheme: http
elasticsearch:
host: elasticsearch.example.com
port: 9200
scheme: http
```
We have created this `ClusterOutput`, without elasticsearch configuration, in the same namespace as our operator: `cattle-logging-system.`. Any time we create a `ClusterFlow` or `ClusterOutput`, we have to put it in the `cattle-logging-system` namespace.
@@ -139,8 +139,8 @@ Now that we have configured where we want the logs to go, let's configure all lo
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: "cattle-logging-system"
name: "all-logs"
namespace: "cattle-logging-system"
spec:
globalOutputRefs:
- "example-es"
@@ -189,13 +189,13 @@ With `coolapp` running, we will follow a similar path as when we created a `Clus
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: "devteam-splunk"
namespace: "devteam"
name: "devteam-splunk"
namespace: "devteam"
spec:
SplunkHec:
host: splunk.example.com
port: 8088
protocol: http
splunkHec:
hec_host: splunk.example.com
hec_port: 8088
protocol: http
```
Once again, let's feed our `Output` some logs:
@@ -204,8 +204,8 @@ Once again, let's feed our `Output` some logs:
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: "devteam-logs"
namespace: "devteam"
name: "devteam-logs"
namespace: "devteam"
spec:
localOutputRefs:
- "devteam-splunk"
@@ -218,37 +218,37 @@ Let's say you wanted to send all logs in your cluster to an `syslog` server. Fir
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: "example-syslog"
namespace: "cattle-logging-system"
spec:
syslog:
buffer:
timekey: 30s
timekey_use_utc: true
timekey_wait: 10s
flush_interval: 5s
format:
type: json
app_name_field: test
host: syslog.example.com
insecure: true
port: 514
transport: tcp
kind: ClusterOutput
metadata:
name: "example-syslog"
namespace: "cattle-logging-system"
spec:
syslog:
buffer:
timekey: 30s
timekey_use_utc: true
timekey_wait: 10s
flush_interval: 5s
format:
type: json
app_name_field: test
host: syslog.example.com
insecure: true
port: 514
transport: tcp
```
Now that we have configured where we want the logs to go, let's configure all logs to go to that `Output`.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: cattle-logging-system
spec:
globalOutputRefs:
- "example-syslog"
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: cattle-logging-system
spec:
globalOutputRefs:
- "example-syslog"
```
### Unsupported Outputs
@@ -65,7 +65,7 @@ Rancher v2.5.4 introduced the capability to configure PrometheusRules by filling
|-------|----------------|
| Alert Name | The name of the alert. Must be a valid label value. |
| Wait To Fire For | Duration in seconds. Alerts are considered firing once they have been returned for this long. Alerts which have not yet fired for long enough are considered pending. |
| PromQL Expression | The PromQL expression to evaluate. Prometheus will evaluate the current value of this PromQL expression on every evaluation cycle and all resultant time series will become pending/firing alerts. For more information, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) or our [example PromQL expressions.](../expression) |
| PromQL Expression | The PromQL expression to evaluate. Prometheus will evaluate the current value of this PromQL expression on every evaluation cycle and all resultant time series will become pending/firing alerts. For more information, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) or our [example PromQL expressions.](../../../expression) |
| Labels | Labels to add or overwrite for each alert. |
| Severity | When enabled, labels are attached to the alert or record that identify it by the severity level. |
| Severity Label Value | Critical, warning, or none |
@@ -85,4 +85,4 @@ Rancher v2.5.4 introduced the capability to configure PrometheusRules by filling
{{% tab "Rancher v2.5.0-v2.5.3" %}}
For Rancher v2.5.0-v2.5.3, PrometheusRules must be configured in YAML. For examples, refer to the Prometheus documentation on [recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) and [alerting rules.](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
{{% /tab %}}
{{% /tabs %}}
{{% /tabs %}}
@@ -49,7 +49,7 @@ If the scrape configuration you want cannot be specified via a ServiceMonitor or
A [scrape_config section](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) specifies a set of targets and parameters describing how to scrape them. In the general case, one scrape configuration specifies a single job.
An example of where this might be used is with Istio. For more information, see [this section.](https://rancher.com/docs/rancher/v2.5/en/istio/v2.5/configuration-reference/selectors-and-scrape)
An example of where this might be used is with Istio. For more information, see [this section.](https://rancher.com/docs/rancher/v2.5/en/istio/configuration-reference/selectors-and-scrape)
# Configuring Applications Packaged within Monitoring v2
@@ -64,7 +64,7 @@ But in the top level chart you can add values that override values that exist in
### Increase the Replicas of Alertmanager
As part of the chart deployment options, you can opt to increase the number of replicas of the Alertmanager deployed onto your cluster. The replicas can all be managed using the same underlying Alertmanager Config Secret. For more information on the Alertmanager Config Secret, refer to [this section.](../configuration/advanced/alertmanager/#multiple-alertmanager-replicas)
As part of the chart deployment options, you can opt to increase the number of replicas of the Alertmanager deployed onto your cluster. The replicas can all be managed using the same underlying Alertmanager Config Secret. For more information on the Alertmanager Config Secret, refer to [this section]({{<baseurl>}}/monitoring-alerting/configuration/advanced/alertmanager/#multiple-alertmanager-replicas)
### Configuring the Namespace for a Persistent Grafana Dashboard
@@ -69,6 +69,6 @@ match_re:
{{% /tab %}}
{{% tab "Rancher v2.5.0-2.5.3" %}}
The Alertmanager must be configured in YAML, as shown in this [example.](./examples/#alertmanager-config)
The Alertmanager must be configured in YAML, as shown in this [example.](../examples/#alertmanager-config)
{{% /tab %}}
{{% /tabs %}}
@@ -15,15 +15,15 @@ To see the default dashboards for time series data visualization, go to the Graf
### Customizing Grafana
To view and customize the PromQL queries powering the Grafana dashboard, see [this page.](./customize-grafana)
To view and customize the PromQL queries powering the Grafana dashboard, see [this page.]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/guides/customize-grafana)
### Persistent Grafana Dashboards
To create a persistent Grafana dashboard, see [this page.](./persist-grafana)
To create a persistent Grafana dashboard, see [this page.]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/guides/persist-grafana)
### Access to Grafana
For information about role-based access control for Grafana, see [this section.](./rbac/#role-based-access-control-for-grafana)
For information about role-based access control for Grafana, see [this section.]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/rbac/#role-based-access-control-for-grafana)
# Alertmanager UI
@@ -43,15 +43,13 @@ To see the Alertmanager UI, go to the **Cluster Explorer.** In the top left corn
**Result:** The Alertmanager UI opens in a new tab. For help with configuration, refer to the [official Alertmanager documentation.](https://prometheus.io/docs/alerting/latest/alertmanager/)
For more information on configuring Alertmanager in Rancher, see [this page.](./configuration/alertmanager)
<figcaption>The Alertmanager UI</figcaption>
![Alertmanager UI]({{<baseurl>}}/img/rancher/alertmanager-ui.png)
### Viewing Default Alerts
To see alerts that are fired by default, go to the [Alertmanager UI](./alertmanager-ui) and click **Expand all groups.**
To see alerts that are fired by default, go to the Alertmanager UI and click **Expand all groups.**
# Prometheus UI
@@ -81,6 +79,4 @@ To see the PrometheusRules, install `rancher-monitoring`. Then go to the **Clust
You can also see the rules in the Prometheus UI:
<figcaption>Rules in the Prometheus UI</figcaption>
![PrometheusRules UI]({{<baseurl>}}/img/rancher/prometheus-rules-ui.png)
For more information on configuring PrometheusRules in Rancher, see [this page.](./configuration/prometheusrules)
![PrometheusRules UI]({{<baseurl>}}/img/rancher/prometheus-rules-ui.png)
@@ -41,7 +41,7 @@ For more information on how to configure Monitoring & Alerting V2, see [this pag
Project owners and members no longer get access to Grafana or Prometheus by default. If view-only users had access to Grafana, they would be able to see data from any namespace. For Kiali, any user can edit things they dont own in any namespace.
For more information about role-based access control in `rancher-monitoring`, refer to [this page.](../rbac)
For more information about role-based access control in `rancher-monitoring`, refer to [this page.]({{<baseurl>}}/rancher/v2.5/en/monitoring-alerting/rbac)
# Migrating from Monitoring V1 to Monitoring V2
@@ -83,7 +83,7 @@ The following are the options for the stats command:
# Remove
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.5/en/backups/rancher-backups) before executing the command.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.5/en/backups/back-up-rancher) before executing the command.
When you install Rancher on a Kubernetes cluster, it will create Kubernetes resources to run and to store configuration data. If you want to remove Rancher from your cluster, you can use the `remove` subcommand to remove the Kubernetes resources. When you use the `remove` subcommand, the following resources will be removed:
@@ -103,7 +103,7 @@ When you install Rancher on a Kubernetes cluster, it will create Kubernetes reso
When you run the command below, all the resources listed [above](#remove) will be removed from the cluster.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.5/en/backups/rancher-backups) before executing the command.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.5/en/backups/back-up-rancher) before executing the command.
```
./system-tools remove --kubeconfig <KUBECONFIG> --namespace <NAMESPACE>
@@ -18,6 +18,8 @@ The pod can be scheduled to any of the hosts you used for your cluster, but that
To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
> **Note:** This container [does not support ARM nodes](https://github.com/leodotcloud/swiss-army-knife/issues/18), such as a Raspberry Pi. This will be seen in the pod logs as `exec user process caused: exec format error`.
1. Save the following file as `overlaytest.yml`
```
+3 -1
View File
@@ -7,6 +7,8 @@ metaDescription: "Rancher 2 adds significant value on top of Kubernetes: managin
insertOneSix: false
weight: 1
ctaBanner: 0
aliases:
- /rancher/v2.x/en/
---
Rancher was originally built to work with multiple orchestrators, and it included its own orchestrator called Cattle. With the rise of Kubernetes in the marketplace, Rancher 2 exclusively deploys and manages Kubernetes clusters running anywhere, on any provider.
@@ -18,4 +20,4 @@ Rancher adds significant value on top of Kubernetes, first by centralizing authe
It then enables detailed monitoring and alerting for clusters and their resources, ships logs to external providers, and integrates directly with Helm via the Application Catalog. If you have an external CI/CD system, you can plug it into Rancher, but if you don't, Rancher even includes Fleet to help you automatically deploy and upgrade workloads.
Rancher is a _complete_ container management platform for Kubernetes, giving you the tools to successfully run Kubernetes anywhere.
Rancher is a _complete_ container management platform for Kubernetes, giving you the tools to successfully run Kubernetes anywhere.
+1 -1
View File
@@ -5,7 +5,7 @@ weight: 5
In this section, you'll learn how to create backups of Rancher, how to restore Rancher from backup, and how to migrate Rancher to a new Kubernetes cluster.
The `rancher-backup` operator is used to backup and restore Rancher on any Kubernetes cluster. This application is a Helm chart, and it can be deployed through the Rancher **Apps & Marketplace** page, or by using the Helm CLI. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/main/charts/rancher-backup)
The `rancher-backup` operator is used to backup and restore Rancher on any Kubernetes cluster. This application is a Helm chart, and it can be deployed through the Rancher **Apps & Marketplace** page, or by using the Helm CLI. The `rancher-backup` Helm chart is [here.](https://github.com/rancher/charts/tree/release-v2.6/charts/rancher-backup)
The backup-restore operator needs to be installed in the local cluster, and only backs up the Rancher app. The backup and restore operations are performed only in the local Kubernetes cluster.
@@ -12,7 +12,7 @@ This guide outlines a reference architecture for provisioning downstream Rancher
<figcaption>Solution Overview</figcaption>
![Solution Overview](/img/rancher/solution_overview.drawio.svg)
![Solution Overview]({{<baseurl>}}/img/rancher/solution_overview.drawio.svg)
# 1. VM Considerations
@@ -105,7 +105,7 @@ Monitoring the availability and performance of all your internal workloads is vi
In addition to monitoring workloads to detect performance, availability or scalability problems, the cluster and the workloads running into it should also be monitored for potential security problems. A good starting point is to frequently run and alert on [CIS Scans]({{<baseurl>}}/rancher/v2.6/en/cis-scans/) which check if the cluster is configured according to security best practices.
For the workloads, you can have a look at Kubernetes and Container security solutions like [Falko](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).
For the workloads, you can have a look at Kubernetes and Container security solutions like [Falco](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).
# Setting up Alerts
@@ -117,4 +117,4 @@ When setting up alerts, configure them for all the workloads that are critical t
If an alert starts firing, but there is nothing you can do about it at the moment, it's also fine to silence the alert for a certain amount of time, so that you can look at it later.
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting).
You can find more information on how to set up alerts and notification channels in the [Rancher Documentation]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting).
@@ -110,7 +110,7 @@ Project network isolation is available if you are using any RKE network plugin t
### Kubernetes Cloud Providers
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
>**Note:** If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file) to configure the cloud provider. Please reference the [RKE cloud provider documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
@@ -41,7 +41,7 @@ For more details on the different networking providers and how to configure them
#### Cloud Provider
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
You can configure a [Kubernetes cloud provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers). If you want to use dynamically provisioned [volumes and storage]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/) in Kubernetes, typically you must select the specific cloud provider in order to use it. For example, if you want to use Amazon EBS, you would need to select the `aws` cloud provider.
>**Note:** If the cloud provider you want to use is not listed as an option, you will need to use the [config file option](#cluster-config-file) to configure the cloud provider. Please reference [this documentation]({{<baseurl>}}/rke/latest/en/config-options/cloud-providers/) on how to configure the cloud provider.
@@ -37,7 +37,7 @@ You can assign the following resources directly to namespaces:
- [Workloads]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/workloads/)
- [Load Balancers/Ingress]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/load-balancers-and-ingress/)
- [Service Discovery Records]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/service-discovery/)
- [Persistent Volume Claims]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/volumes-and-storage/persistent-volume-claims/)
- [Persistent Volume Claims]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/)
- [Certificates]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/certificates/)
- [ConfigMaps]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/configmaps/)
- [Registries]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/registries/)
@@ -70,7 +70,7 @@ In the base version of Kubernetes, features like role-based access rights or clu
You can use projects to perform actions such as:
- Assign users to a group of namespaces (i.e., [project membership]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/project-members)).
- Assign users to a group of namespaces (i.e., [project membership]({{<baseurl>}}/rancher/v2.6/en/admin-settings/rbac/cluster-project-roles)).
- Assign users specific roles in a project. A role can be owner, member, read-only, or [custom]({{<baseurl>}}/rancher/v2.6/en/admin-settings/rbac/default-custom-roles/).
- Assign resources to the project.
- Assign Pod Security Policies.
@@ -112,7 +112,7 @@ Standard users are only authorized for project access in two situations:
# Pod Security Policies
Rancher extends Kubernetes to allow the application of [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) at the [project level]({{<baseurl>}}/rancher/v2.6/en/project-admin/pod-security-policies) in addition to the [cluster level.](../pod-security-policy) However, as a best practice, we recommend applying Pod Security Policies at the cluster level.
Rancher extends Kubernetes to allow the application of [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) at the project level in addition to the cluster level. However, as a best practice, we recommend applying Pod Security Policies at the cluster level.
# Creating Projects
@@ -160,12 +160,12 @@ To add members:
### 4. Optional: Add Resource Quotas
Resource quotas limit the resources that a project (and its namespaces) can consume. For more information, see [Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas).
Resource quotas limit the resources that a project (and its namespaces) can consume. For more information, see [Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/).
To add a resource quota,
1. In the **Resource Quotas** tab, click **Add Resource**.
1. Select a **Resource Type**. For more information, see [Resource Quotas.]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas/).
1. Select a **Resource Type**. For more information, see [Resource Quotas.]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/).
1. Enter values for the **Project Limit** and the **Namespace Default Limit**.
1. **Optional:** Specify **Container Default Resource Limit**, which will be applied to every container started in the project. The parameter is recommended if you have CPU or Memory limits set by the Resource Quota. It can be overridden on per an individual namespace or a container level. For more information, see [Container Default Resource Limit]({{<baseurl>}}/rancher/v2.6/en/project-admin/resource-quotas/)
1. Click **Create**.
@@ -24,7 +24,7 @@ This section covers the following topics:
# Tested Kubernetes Versions
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For example, Rancher v2.3.0 is was tested with Kubernetes v1.15.4, v1.14.7, and v1.13.11. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.3.0/)
Before a new version of Rancher is released, it's tested with the latest minor versions of Kubernetes to ensure compatibility. For details on which versions of Kubernetes were tested on each Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.6.0/)
# How Upgrades Work
@@ -3,7 +3,7 @@ title: vSphere Storage
weight: 3055
---
To provide stateful workloads with vSphere storage, we recommend creating a vSphereVolume StorageClass. This practice dynamically provisions vSphere storage when workloads request volumes through a [persistent volume claim]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/volumes-and-storage/persistent-volume-claims/).
To provide stateful workloads with vSphere storage, we recommend creating a vSphereVolume StorageClass. This practice dynamically provisions vSphere storage when workloads request volumes through a PersistentVolumeClaim.
In order to dynamically provision storage in vSphere, the vSphere provider must be [enabled.]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere)
@@ -67,4 +67,4 @@ If you provision hosts using Rancher Machine Azure driver, you will need to edit
You should already assign custom hosts to this Network Security Group during provisioning.
Only hosts expected to be load balancer back ends need to be in this group.
Only hosts expected to be load balancer back ends need to be in this group.
@@ -15,7 +15,7 @@ Then you will create an EC2 cluster in Rancher, and when configuring the new clu
- **AWS EC2 Access Key and Secret Key** that will be used to create the instances. See [Amazon Documentation: Creating Access Keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) how to create an Access Key and Secret Key.
- **IAM Policy created** to add to the user of the Access Key And Secret Key. See [Amazon Documentation: Creating IAM Policies (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start) how to create an IAM policy. See our three example JSON policies below:
- [Example IAM Policy](#example-iam-policy)
- [Example IAM Policy with PassRole](#example-iam-policy-with-passrole) (needed if you want to use [Kubernetes Cloud Provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/options/cloud-providers) or want to pass an IAM Profile to an instance)
- [Example IAM Policy with PassRole](#example-iam-policy-with-passrole) (needed if you want to use [Kubernetes Cloud Provider]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers) or want to pass an IAM Profile to an instance)
- [Example IAM Policy to allow encrypted EBS volumes](#example-iam-policy-to-allow-encrypted-ebs-volumes)
- **IAM Policy added as Permission** to the user. See [Amazon Documentation: Adding Permissions to a User (Console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) how to attach it to an user.
@@ -59,6 +59,8 @@ For the support lifecycle dates for Windows Server, see the [Microsoft Documenta
Kubernetes v1.15+ is required.
If you are using Kubernetes v1.21 with Windows Server 20H2 Standard Core, the patch "2019-08 Servicing Stack Update for Windows Server" must be installed on the node.
### Node Requirements
The hosts in the cluster need to have at least:
@@ -152,7 +154,7 @@ You will provision three nodes:
| Node 2 | Linux (Ubuntu Server 18.04 recommended) |
| Node 3 | Windows (Windows Server core version 1809 or above required) |
If your nodes are hosted by a **Cloud Provider** and you want automation support such as loadbalancers or persistent storage devices, your nodes have additional configuration requirements. For details, see [Selecting Cloud Providers.]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/options/cloud-providers)
If your nodes are hosted by a **Cloud Provider** and you want automation support such as loadbalancers or persistent storage devices, your nodes have additional configuration requirements. For details, see [Selecting Cloud Providers.]({{<baseurl>}}/rancher/v2.6/en/cluster-provisioning/rke-clusters/cloud-providers)
# 2. Create the Cluster on Existing Nodes
@@ -24,6 +24,7 @@ For information on enabling experimental features, refer to [this page.]({{<base
| Option | Default Value | Description |
| ------------------------- | ------------- | ---------------------------------------------------------------------------------- |
| `bootstrapPassword` | " " | `string` - Set the [bootstrap password](#bootstrap-password) for the first admin user. After logging in, the admin will need to reset their password. A randomly generated bootstrap password is used if this value is not set.
| `hostname` | " " | `string` - the Fully Qualified Domain Name for your Rancher Server |
| `ingress.tls.source` | "rancher" | `string` - Where to get the cert for the ingress. - "rancher, letsEncrypt, secret" |
| `letsEncrypt.email` | " " | `string` - Your email address |
@@ -68,6 +69,22 @@ For information on enabling experimental features, refer to [this page.]({{<base
### Bootstrap Password
When Rancher starts for the first time, a password is randomly generated for the first admin user. When the admin first logs in to Rancher, the UI shows commands that can be used to retrieve the bootstrap password. The admin needs to run those commands and log in with the bootstrap password. Then Rancher gives the admin an opportunity to reset the password.
If you want to use a specific bootstrap password instead of a randomly generated one, provide the password.
```plain
--set bootstrapPassword="rancher"
```
The password, whether provided or generated, will be stored in a Kubernetes secret. After Rancher is installed, the UI will show instructions for how to retrieve the password using kubectl:
```
kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{ .data.bootstrapPassword|base64decode}}{{ "\n" }}'
```
### API Audit Log
Enabling the [API Audit Log]({{<baseurl>}}/rancher/v2.6/en/installation/resources/advanced/api-audit-log)
@@ -27,27 +27,27 @@ The following is a list of the feature flags available in Rancher:
- `harvester`: This feature flag is available starting in v2.6.1. It is used to manage access to the Virtualization Management page where users can navigate directly to Harvester clusters and access the Harvester UI.
- `rke2`: We have introduced the ability to provision RKE2 clusters as tech preview. By default, this feature flag is enabled, which allows users to attempt to provision these type of clusters.
- `token-hashing`: This feature enables one-way [hashing of tokens]({{<baseurl>}}/rancher/v2.6/en/api/api-tokens) and cannot be disabled once enabled.
- `fleet`: Rancher comes with [Fleet]({{<baseurl>}}/rancher/v2.6/en/deploy-across-clusters/fleet) preinstalled in v2.5+. Rancher v2.6's new provisioning system leverages Fleet's bundle deployment capabilities in order to manage clusters at scale. Therefore, in Rancher v2.6, Fleet can no longer be disabled. If Fleet was disabled in Rancher v2.5.x, it will become enabled if Rancher is upgraded to v2.6.x.
- `continuous-delivery`: In Rancher v2.5.x, Fleet came with a GitOps feature that could not be disabled separately from Fleet. In Rancher v2.6, the `continuous-delivery` feature flag was introduced to allow the GitOps feature of Fleet to be disabled. For more information, see [this page.](./continuous-delivery)
- `istio-virtual-service-ui`: This feature enables a [UI to create, read, update, and delete Istio virtual services and destination rules]({{<baseurl>}}/rancher/v2.6/en/installation/resources/feature-flags/istio-virtual-service-ui), which are traffic management features of Istio.
- `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{<baseurl>}}/rancher/v2.6/en/installation/resources/feature-flags/enable-not-default-storage-drivers) In other words, it enables types for storage providers and provisioners that are not enabled by default.
- `fleet`: The previous `fleet` feature flag is now required to be enabled as the Fleet capabilities are leveraged within the new provisioning framework. If you had this feature flag disabled in earlier versions, upon upgrading to Rancher v2.6, the flag will automatically be enabled. See this [page]({{<baseurl>}}/rancher/v2.6/en/deploy-across-clusters/fleet) for more information.
- `continuous-delivery`: In Rancher v2.5.x, Fleet came with a GitOps feature that could not be disabled separately from Fleet. In Rancher v2.6, the `continuous-delivery` feature flag was introduced to allow the GitOps feature of Fleet to be disabled. For more information, see [this page.](./continuous-delivery).
- `legacy`: There are a set of features from previous versions that are slowly being phased out of Rancher for newer iterations of the feature. This is a mix of deprecated features as well as features that will eventually be moved to newer variations in Rancher. By default, this feature flag is disabled for new installations. If you are upgrading from a previous version, this feature flag would be enabled.
- `token-hashing`: Used to enable new token-hashing feature. Once enabled, existing tokens will be hashed and all new tokens will be hashed automatically using the SHA256 algorithm. Once a token is hashed it cannot be undone. Once this feature flag is enabled, it cannot be disabled. See [hashing of tokens]({{<baseurl>}}/rancher/v2.6/en/api/api-tokens) for more information.
- `unsupported-storage-drivers`: This feature [allows unsupported storage drivers.]({{<baseurl>}}/rancher/v2.6/en/installation/resources/feature-flags/enable-not-default-storage-drivers). In other words, it enables types for storage providers and provisioners that are not enabled by default.
- `istio-virtual-service-ui`: This feature enables a [UI to create, read, update, and delete Istio virtual services and destination rules,]({{<baseurl>}}/rancher/v2.6/en/installation/resources/feature-flags/istio-virtual-service-ui) which are traffic management features of Istio.
- `multi-cluster-management`: Used for multi-cluster provisioning and management of Kubernetes clusters. This feature flag can only be set at install time and not changed afterwards.
The below table shows the availability and default value for feature flags in Rancher:
| Feature Flag Name | Default Value | Status | Available as of | Rancher Restart Required? |
| ----------------------------- | ------------- | ------------ | --------------- |---|
| `dashboard` | `true` | Experimental | v2.4.0 | x |
| `dashboard` | `true` | GA* and no longer a feature flag | v2.5.0 | x |
| `istio-virtual-service-ui` | `false` | Experimental | v2.3.0 | |
| `istio-virtual-service-ui` | `true` | GA* | v2.3.2 | |
| `proxy` | `false` | Experimental | v2.4.0 | |
| `proxy` | N/A | Discontinued | v2.5.0 | |
| `unsupported-storage-drivers` | `false` | Experimental | v2.3.0 | |
| `fleet` | `true` | GA* | v2.5.0 | |
| `fleet` | `true` | Can no longer be disabled | v2.6.0 | N/A |
| `continuous-delivery` | `true` | GA* | v2.6.0 | |
| `token-hashing` | `false` | GA* | v2.6.0 | |
| `token-hashing` | `false` for new installs, `true` for upgrades | GA* | v2.6.0 | |
| `legacy` | `false` for new installs, `true` for upgrades | GA* | v2.6.0 | |
| `multi-cluster-management` | `false` | GA* | v2.5.0 | |
| `harvester` | `true` | Experimental | v2.6.1 | |
| `rke2` | `true` | Experimental | v2.6.0 | |
@@ -64,8 +64,7 @@ When you install Rancher, enable the feature you want with a feature flag. The c
When installing Rancher with a Helm chart, use the `--features` option. In the below example, two features are enabled by passing the feature flag names names in a comma separated list:
```
helm install rancher-latest/rancher \
--name rancher \
helm install rancher rancher-latest/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set 'extraEnv[0].name=CATTLE_FEATURES'
@@ -80,7 +79,7 @@ For an air gap installation of Rancher, you need to add a Helm chart repository
Here is an example of a command for passing in the feature flag names when rendering the Helm template. In the below example, two features are enabled by passing the feature flag names in a comma separated list.
The Helm 3 command is as follows:
The Helm command is as follows:
```
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
@@ -95,20 +94,6 @@ helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--set 'extraEnv[0].value=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true'
```
The Helm 2 command is as follows:
```
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
--set 'extraEnv[0].name=CATTLE_FEATURES'
--set 'extraEnv[0].value=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true'
```
### Enabling Features for Docker Installs
When installing Rancher with Docker, use the `--features` option. In the below example, two features are enabled by passing the feature flag names in a comma separated list:
@@ -9,8 +9,6 @@ This section describes how to install a Kubernetes cluster. This cluster should
> Rancher can run on any Kubernetes cluster, included hosted Kubernetes solutions such as Amazon EKS. The below instructions represent only one possible way to install Kubernetes.
The Rancher management server can only be run on Kubernetes cluster in an infrastructure provider where Kubernetes is installed using RKE or K3s. Use of Rancher on hosted Kubernetes providers, such as EKS, is not supported.
For systems without direct internet access, refer to [Air Gap: Kubernetes install.]({{<baseurl>}}/rancher/v2.6/en/installation/other-installation-methods/air-gap/)
> **Single-node Installation Tip:**
@@ -27,8 +27,6 @@ To address these changes, this guide will do two things:
The namespace used in these instructions depends on the namespace cert-manager is currently installed in. If it is in kube-system use that in the instructions below. You can verify by running `kubectl get pods --all-namespaces` and checking which namespace the cert-manager-\* pods are listed in. Do not change the namespace cert-manager is running in or this can cause issues.
> These instructions have been updated for Helm 3. If you are still using Helm 2, refer to [these instructions.]({{<baseurl>}}/rancher/v2.6/en/installation/resoruces/upgrading-cert-manager/helm-2-instructions)
In order to upgrade cert-manager, follow these instructions:
### Option A: Upgrade cert-manager with Internet Access
@@ -1,170 +0,0 @@
---
title: Upgrading Cert-Manager with Helm 2
weight: 2040
---
Rancher uses cert-manager to automatically generate and renew TLS certificates for HA deployments of Rancher. As of Fall 2019, three important changes to cert-manager are set to occur that you need to take action on if you have an HA deployment of Rancher:
1. [Let's Encrypt will be blocking cert-manager instances older than 0.8.0 starting November 1st 2019.](https://community.letsencrypt.org/t/blocking-old-cert-manager-versions/98753)
1. [Cert-manager is deprecating and replacing the certificate.spec.acme.solvers field](https://docs.cert-manager.io/en/latest/tasks/upgrading/upgrading-0.7-0.8.html#upgrading-from-v0-7-to-v0-8). This change has no exact deadline.
1. [Cert-manager is deprecating `v1alpha1` API and replacing its API group](https://cert-manager.io/docs/installation/upgrading/upgrading-0.10-0.11/)
To address these changes, this guide will do two things:
1. Document the procedure for upgrading cert-manager
1. Explain the cert-manager API changes and link to cert-manager's offficial documentation for migrating your data
> **Important:**
> If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, you need to reinstall both of them:
> 1. Take a one-time snapshot of your Kubernetes cluster running Rancher server
> 2. Uninstall Rancher, cert-manager, and the CustomResourceDefinition for cert-manager
> 3. Install the newer version of Rancher and cert-manager
> The reason is that when Helm upgrades Rancher, it will reject the upgrade and show error messages if the running Rancher app does not match the chart template used to install it. Because cert-manager changed its API group and we cannot modify released charts for Rancher, there will always be a mismatch on the cert-manager's API version, therefore the upgrade will be rejected.
## Upgrade Cert-Manager Only
> **Note:**
> These instructions are applied if you have no plan to upgrade Rancher.
The namespace used in these instructions depends on the namespace cert-manager is currently installed in. If it is in kube-system use that in the instructions below. You can verify by running `kubectl get pods --all-namespaces` and checking which namespace the cert-manager-\* pods are listed in. Do not change the namespace cert-manager is running in or this can cause issues.
In order to upgrade cert-manager, follow these instructions:
{{% accordion id="normal" label="Upgrading cert-manager with Internet access" %}}
1. Back up existing resources as a precaution
```plain
kubectl get -o yaml --all-namespaces issuer,clusterissuer,certificates > cert-manager-backup.yaml
```
1. Delete the existing deployment
```plain
helm delete --purge cert-manager
```
1. Install the CustomResourceDefinition resources separately
```plain
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml
```
1. Add the Jetstack Helm repository
```plain
helm repo add jetstack https://charts.jetstack.io
```
1. Update your local Helm chart repository cache
```plain
helm repo update
```
1. Install the new version of cert-manager
```plain
helm install --version 0.12.0 --name cert-manager --namespace kube-system jetstack/cert-manager
```
{{% /accordion %}}
{{% accordion id="airgap" label="Upgrading cert-manager in an airgapped environment" %}}
### Prerequisites
Before you can perform the upgrade, you must prepare your air gapped environment by adding the necessary container images to your private registry and downloading or rendering the required Kubernetes manifest files.
1. Follow the guide to [Prepare your Private Registry]({{<baseurl>}}/rancher/v2.6/en/installation/other-installation-methods/air-gap/populate-private-registry/) with the images needed for the upgrade.
1. From a system connected to the internet, add the cert-manager repo to Helm
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
```
1. Fetch the latest cert-manager chart available from the [Helm chart repository](https://hub.helm.sh/charts/jetstack/cert-manager).
```plain
helm fetch jetstack/cert-manager --version v0.12.0
```
1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files.
```plain
helm template ./cert-manager-v0.12.0.tgz --output-dir . \
--name cert-manager --namespace kube-system \
--set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller
--set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook
--set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector
```
1. Download the required CRD file for cert-manager
```plain
curl -L -o cert-manager/cert-manager-crd.yaml https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml
```
### Install cert-manager
1. Back up existing resources as a precaution
```plain
kubectl get -o yaml --all-namespaces issuer,clusterissuer,certificates > cert-manager-backup.yaml
```
1. Delete the existing cert-manager installation
```plain
kubectl -n kube-system delete deployment,sa,clusterrole,clusterrolebinding -l 'app=cert-manager' -l 'chart=cert-manager-v0.5.2'
```
1. Install the CustomResourceDefinition resources separately
```plain
kubectl apply -f cert-manager/cert-manager-crd.yaml
```
1. Install cert-manager
```plain
kubectl -n kube-system apply -R -f ./cert-manager
```
{{% /accordion %}}
Once youve installed cert-manager, you can verify it is deployed correctly by checking the kube-system namespace for running pods:
```
kubectl get pods --namespace kube-system
NAME READY STATUS RESTARTS AGE
cert-manager-7cbdc48784-rpgnt 1/1 Running 0 3m
cert-manager-webhook-5b5dd6999-kst4x 1/1 Running 0 3m
cert-manager-cainjector-3ba5cd2bcd-de332x 1/1 Running 0 3m
```
If the webhook pod (2nd line) is in a ContainerCreating state, it may still be waiting for the Secret to be mounted into the pod. Wait a couple of minutes for this to happen but if you experience problems, please check cert-manager's [troubleshooting](https://docs.cert-manager.io/en/latest/getting-started/troubleshooting.html) guide.
> **Note:** The above instructions ask you to add the disable-validation label to the kube-system namespace. Here are additional resources that explain why this is necessary:
>
> - [Information on the disable-validation label](https://docs.cert-manager.io/en/latest/tasks/upgrading/upgrading-0.4-0.5.html?highlight=certmanager.k8s.io%2Fdisable-validation#disabling-resource-validation-on-the-cert-manager-namespace)
> - [Information on webhook validation for certificates](https://docs.cert-manager.io/en/latest/getting-started/webhook.html)
## Cert-Manager API change and data migration
Cert-manager has deprecated the use of the `certificate.spec.acme.solvers` field and will drop support for it completely in an upcoming release.
Per the cert-manager documentation, a new format for configuring ACME certificate resources was introduced in v0.8. Specifically, the challenge solver configuration field was moved. Both the old format and new are supported as of v0.9, but support for the old format will be dropped in an upcoming release of cert-manager. The cert-manager documentation strongly recommends that after upgrading you update your ACME Issuer and Certificate resources to the new format.
Details about the change and migration instructions can be found in the [cert-manager v0.7 to v0.8 upgrade instructions](https://cert-manager.io/docs/installation/upgrading/upgrading-0.7-0.8/).
The v0.11 release marks the removal of the v1alpha1 API that was used in previous versions of cert-manager, as well as our API group changing to be `cert-manager.io` instead of `certmanager.k8s.io.`
We have also removed support for the old configuration format that was deprecated in the v0.8 release. This means you must transition to using the new solvers style configuration format for your ACME issuers before upgrading to v0.11. For more information, see the [upgrading to v0.8 guide](https://cert-manager.io/docs/installation/upgrading/upgrading-0.7-0.8/).
Details about the change and migration instructions can be found in the [cert-manager v0.10 to v0.11 upgrade instructions](https://cert-manager.io/docs/installation/upgrading/upgrading-0.10-0.11/).
For information on upgrading from all other versions of cert-manager, refer to the [official documentation](https://cert-manager.io/docs/installation/upgrading/).
@@ -42,4 +42,4 @@ To add the annotation to a workload,
> **NOTE:** If you are having issues with a Job you deployed not completing, you will need to add this annotation to your pod using the provided steps. Since Istio Sidecars run indefinitely, a Job cannot be considered complete even after its task has completed.
### [Next: Select the Nodes ]({{<baseurl>}}/rancher/v2.6/en/istio/setup/node-selectors)
### [Next: Add Deployments with the Istio Sidecar ]({{<baseurl>}}/rancher/v2.6/en/istio/setup/deploy-workloads)
@@ -145,13 +145,13 @@ With `coolapp` running, we will follow a similar path as when we created a `Clus
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: "devteam-splunk"
namespace: "devteam"
name: "devteam-splunk"
namespace: "devteam"
spec:
SplunkHec:
host: splunk.example.com
port: 8088
protocol: http
splunkHec:
hec_host: splunk.example.com
hec_port: 8088
protocol: http
```
Once again, let's feed our `Output` some logs:
@@ -160,8 +160,8 @@ Once again, let's feed our `Output` some logs:
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: "devteam-logs"
namespace: "devteam"
name: "devteam-logs"
namespace: "devteam"
spec:
localOutputRefs:
- "devteam-splunk"
@@ -174,37 +174,37 @@ Let's say you wanted to send all logs in your cluster to an `syslog` server. Fir
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: "example-syslog"
namespace: "cattle-logging-system"
spec:
syslog:
buffer:
timekey: 30s
timekey_use_utc: true
timekey_wait: 10s
flush_interval: 5s
format:
type: json
app_name_field: test
host: syslog.example.com
insecure: true
port: 514
transport: tcp
kind: ClusterOutput
metadata:
name: "example-syslog"
namespace: "cattle-logging-system"
spec:
syslog:
buffer:
timekey: 30s
timekey_use_utc: true
timekey_wait: 10s
flush_interval: 5s
format:
type: json
app_name_field: test
host: syslog.example.com
insecure: true
port: 514
transport: tcp
```
Now that we have configured where we want the logs to go, let's configure all logs to go to that `Output`.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: cattle-logging-system
spec:
globalOutputRefs:
- "example-syslog"
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: cattle-logging-system
spec:
globalOutputRefs:
- "example-syslog"
```
### Unsupported Outputs
@@ -57,7 +57,7 @@ For examples, refer to the Prometheus documentation on [recording rules](https:/
|-------|----------------|
| Alert Name | The name of the alert. Must be a valid label value. |
| Wait To Fire For | Duration in seconds. Alerts are considered firing once they have been returned for this long. Alerts which have not yet fired for long enough are considered pending. |
| PromQL Expression | The PromQL expression to evaluate. Prometheus will evaluate the current value of this PromQL expression on every evaluation cycle and all resultant time series will become pending/firing alerts. For more information, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) or our [example PromQL expressions.](../expression) |
| PromQL Expression | The PromQL expression to evaluate. Prometheus will evaluate the current value of this PromQL expression on every evaluation cycle and all resultant time series will become pending/firing alerts. For more information, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) or our [example PromQL expressions.](../../../expression) |
| Labels | Labels to add or overwrite for each alert. |
| Severity | When enabled, labels are attached to the alert or record that identify it by the severity level. |
| Severity Label Value | Critical, warning, or none |
@@ -71,4 +71,4 @@ For examples, refer to the Prometheus documentation on [recording rules](https:/
|-------|----------------|
| Time Series Name | The name of the time series to output to. Must be a valid metric name. |
| PromQL Expression | The PromQL expression to evaluate. Prometheus will evaluate the current value of this PromQL expression on every evaluation cycle and the result will be recorded as a new set of time series with the metric name as given by 'record'. For more information about expressions, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) or our [example PromQL expressions.](../expression) |
| Labels | Labels to add or overwrite before storing the result. |
| Labels | Labels to add or overwrite before storing the result. |
@@ -38,7 +38,7 @@ For more information on how to configure Monitoring & Alerting V2, see [this pag
Project owners and members no longer get access to Grafana or Prometheus by default. If view-only users had access to Grafana, they would be able to see data from any namespace. For Kiali, any user can edit things they dont own in any namespace.
For more information about role-based access control in `rancher-monitoring`, refer to [this page.](../rbac)
For more information about role-based access control in `rancher-monitoring`, refer to [this page.]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting/rbac)
# Migrating from Monitoring V1 to Monitoring V2
@@ -126,7 +126,7 @@ or add the Prometheus Rule through the Cluster Explorer
{{< img "/img/rancher/monitoring/migration/alert_2.4_to_2.5_target.png" "">}}
For more details on how to configure PrometheusRules in Monitoring V2 see [Monitoring Configuration]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting/configuration/prometheusrules).
For more details on how to configure PrometheusRules in Monitoring V2 see [Monitoring Configuration]({{<baseurl>}}/rancher/v2.6/en/monitoring-alerting/configuration/).
### Migrating Notifiers
@@ -108,4 +108,4 @@ For more best practices for downstream clusters, refer to the [production checkl
If you are using an [authorized cluster endpoint,]({{<baseurl>}}/rancher/v2.6/en/overview/architecture/#4-authorized-cluster-endpoint) we recommend creating an FQDN pointing to a load balancer which balances traffic across your nodes with the `controlplane` role.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/kubeconfig/) and [API keys]({{<baseurl>}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information.
If you are using private CA signed certificates on the load balancer, you have to supply the CA certificate, which will be included in the generated kubeconfig file to validate the certificate chain. See the documentation on [kubeconfig files]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/cluster-access/kubectl/) and [API keys]({{<baseurl>}}/rancher/v2.6/en/user-settings/api-keys/#creating-an-api-key) for more information.
@@ -12,7 +12,7 @@ Resources that you can assign directly to namespaces include:
- [Workloads]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/workloads/)
- [Load Balancers/Ingress]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/load-balancers-and-ingress/)
- [Service Discovery Records]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/service-discovery/)
- [Persistent Volume Claims]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/volumes-and-storage/persistent-volume-claims/)
- [Persistent Volume Claims]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/volumes-and-storage/)
- [Certificates]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/certificates/)
- [ConfigMaps]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/configmaps/)
- [Registries]({{<baseurl>}}/rancher/v2.6/en/k8s-in-rancher/registries/)
@@ -34,7 +34,7 @@ Create a new namespace to isolate apps and resources in a project.
1. Click **Cluster > Projects/Namespaces**.
1. Go to the project where you want to add a namespace and click **Create Namespace**. Alternately, go to **Not in a Project** to create a namespace not associated with a project.
1. **Optional:** If your project has [Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas) in effect, you can override the default resource **Limits** (which places a cap on the resources that the namespace can consume).
1. **Optional:** If your project has [Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/project-admin/resource-quotas) in effect, you can override the default resource **Limits** (which places a cap on the resources that the namespace can consume).
1. Enter a **Name** and then click **Create**.
@@ -54,7 +54,7 @@ Cluster admins and members may occasionally need to move a namespace to another
>**Notes:**
>
>- Don't move the namespaces in the `System` project. Moving these namespaces can adversely affect cluster networking.
>- You cannot move a namespace into a project that already has a [resource quota]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas/) configured.
>- You cannot move a namespace into a project that already has a [resource quota]({{<baseurl>}}/rancher/v2.6/en/project-admin/resource-quotas)configured.
>- If you move a namespace from a project that has a quota set to a project with no quota set, the quota is removed from the namespace.
1. Choose a new project for the new namespace and then click **Move**. Alternatively, you can remove the namespace from all projects by selecting **None**.
@@ -13,7 +13,7 @@ Resource quotas in Rancher include the same functionality as the [native version
### Applying Resource Quotas to Existing Projects
Edit [resource quotas]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas) when:
Edit resource quotas when:
- You want to limit the resources that a project and its namespaces can use.
- You want to scale the resources available to a project up or down when a research quota is already in effect.
@@ -9,7 +9,7 @@ To avoid setting these limits on each and every container during workload creati
### Editing the Container Default Resource Limit
Edit [container default resource limit]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas/) when:
Edit the container default resource limit when:
- You have a CPU or Memory resource quota set on a project, and want to supply the corresponding default values for a container.
- You want to edit the default container resource limit.
@@ -14,7 +14,7 @@ How to: [Editing Namespace Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/cluste
### Editing Namespace Resource Quotas
If there is a [resource quota]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas) configured for a project, you can override the namespace default limit to provide a specific namespace with access to more (or less) project resources.
If there is a resource quota configured for a project, you can override the namespace default limit to provide a specific namespace with access to more (or less) project resources.
1. In the upper left corner, click **☰ > Cluster Management**.
1. On the **Clusters** page, go to the cluster where you want to edit a namespace resource quota and click **Explore**.
@@ -22,7 +22,7 @@ If there is a [resource quota]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/proje
1. Find the namespace for which you want to edit the resource quota. Click **⋮ > Edit Config**.
1. Edit the resource limits. These limits determine the resources available to the namespace. The limits must be set within the configured project limits.
For more information about each **Resource Type**, see [Resource Quotas]({{<baseurl>}}/rancher/v2.6/en/cluster-admin/projects-and-namespaces/resource-quotas/).
For more information about each **Resource Type**, see [the type reference]({{<baseurl>}}/rancher/v2.6/en/project-admin/resource-quotas/quota-type-reference/).
>**Note:**
>
@@ -81,7 +81,7 @@ The following are the options for the stats command:
# Remove
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.6/en/backups/rancher-backups) before executing the command.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.6/en/backups/back-up-rancher) before executing the command.
When you install Rancher on a Kubernetes cluster, it will create Kubernetes resources to run and to store configuration data. If you want to remove Rancher from your cluster, you can use the `remove` subcommand to remove the Kubernetes resources. When you use the `remove` subcommand, the following resources will be removed:
@@ -101,7 +101,7 @@ When you install Rancher on a Kubernetes cluster, it will create Kubernetes reso
When you run the command below, all the resources listed [above](#remove) will be removed from the cluster.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.6/en/backups/rancher-backups) before executing the command.
>**Warning:** This command will remove data from your etcd nodes. Make sure you have created a [backup of etcd]({{<baseurl>}}/rancher/v2.6/en/backups/back-up-rancher) before executing the command.
```
./system-tools remove --kubeconfig <KUBECONFIG> --namespace <NAMESPACE>
@@ -16,6 +16,8 @@ The pod can be scheduled to any of the hosts you used for your cluster, but that
To test the overlay network, you can launch the following `DaemonSet` definition. This will run a `swiss-army-knife` container on every host (image was developed by Rancher engineers and can be found here: https://github.com/rancherlabs/swiss-army-knife), which we will use to run a `ping` test between containers on all hosts.
> **Note:** This container [does not support ARM nodes](https://github.com/leodotcloud/swiss-army-knife/issues/18), such as a Raspberry Pi. This will be seen in the pod logs as `exec user process caused: exec format error`.
1. Save the following file as `overlaytest.yml`
```
@@ -116,10 +116,10 @@ ingress:
### Configuring network options
{{% tabs %}}
{{% tab "v1.30" %}}
{{% tab "v1.3.x" %}}
For Kubernetes v1.21 and up, the NGINX ingress controller no longer runs in `hostNetwork: true` but uses hostPorts for port `80` and port `443`. This was done so the admission webhook can be configured to be accessed using ClusterIP so it can only be reached inside the cluster. If you want to change the mode and/or the ports, see the options below.
{{% /tab %}}
{{% tab "v1.1.11 - v1.2.8" %}}
{{% tab "v1.1.11 and up & v1.2.x" %}}
By default, the nginx ingress controller is configured using `hostNetwork: true` on the default ports `80` and `443`. If you want to change the mode and/or the ports, see the options below.
{{% /tab %}}
{{% /tabs %}}
@@ -137,6 +137,14 @@ ingress:
https-port: 8443
```
Configure the nginx ingress controller using `hostNetwork`:
```yaml
ingress:
provider: nginx
network_mode: hostNetwork
```
Configure the nginx ingress controller with no network mode which will make it run on the overlay network (for example, if you want to expose the nginx ingress controller using a `LoadBalancer`) and override the default ports:
```yaml
+3 -2
View File
@@ -1,3 +1,4 @@
When contributing to docs, please update the versioned docs, for example, the docs in the v2.6 folder.
### For Rancher (product) docs only
When contributing to docs, please update the versioned docs. For example, the docs in the v2.6 folder of the `rancher` folder.
Doc versions older than the latest minor version should only be updated to fix inaccuracies or make minor updates as necessary. The majority of new content should be added to the folder for the latest minor version.
Doc versions older than the latest minor version should only be updated to fix inaccuracies or make minor updates as necessary. The majority of new content should be added to the folder for the latest minor version.
+3 -3
View File
@@ -2035,9 +2035,9 @@ lodash.sortby@^4.7.0:
integrity sha1-7dFMgk4sycHgsKG0K7UhBRakJDg=
lodash@^4.13.1, lodash@^4.17.10, lodash@^4.17.5:
version "4.17.19"
resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.19.tgz#e48ddedbe30b3321783c5b4301fbd353bc1e4a4b"
integrity sha512-JNvd8XER9GQX0v2qJgsaN/mzFCNA5BRe/j8JN9d+tWyGLSodKQHKFicdwNYzWwI3wjRnaKPsGj1XkBjx/F96DQ==
version "4.17.21"
resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.21.tgz#679591c564c3bffaae8454cf0b3df370c3d6911c"
integrity sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==
loose-envify@^1.0.0:
version "1.4.0"