resolving merging conflict

This commit is contained in:
martyav
2024-06-11 10:24:10 -04:00
1153 changed files with 152469 additions and 579 deletions
@@ -107,7 +107,7 @@ Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` a
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### CNI Features by Provider
@@ -32,12 +32,12 @@ The usage below defines rules about what the audit log should record and what da
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Request Metadata | Request Body | Response Metadata | Response Body |
| --------------------- | ---------------- | ------------ | ----------------- | ------------- |
| `0` | | | | |
| `1` | ✓ | | | |
| `2` | ✓ | ✓ | | |
| `3` | ✓ | ✓ | ✓ | ✓ |
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
@@ -2,7 +2,13 @@
title: Opening Ports with firewalld
---
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
:::danger
Enabling firewalld can cause serious network communication problems.
CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
@@ -0,0 +1,17 @@
---
title: Glossary
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/glossary"/>
</head>
This page covers Rancher-specific terminology and symbols which might be unfamiliar, or which differ between Rancher versions.
```mdx-code-block
import Glossary, {toc as GlossaryTOC} from "/shared-files/_glossary.md"
<Glossary />
export const toc = GlossaryTOC;
```
@@ -37,4 +37,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
### Developing your own node driver
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
Node drivers are implemented with [Rancher Machine](https://github.com/rancher/machine), a fork of [Docker Machine](https://github.com/docker/machine). Docker Machine is no longer under active development.
Refer to the original [Docker Machine documentation](https://github.com/docker/docs/blob/vnext-engine/machine/overview.md) for details on how to develop your own node drivers.
@@ -302,7 +302,7 @@ cloud-provider|-|Cloud provider type|
|max-node-provision-time|"15m"|Maximum time CA waits for node to be provisioned|
|nodes|-|sets min,max size and other configuration data for a node group in a format accepted by cloud provider. Can be used multiple times. Format: `<min>:<max>:<other...>`|
|node-group-auto-discovery|-|One or more definition(s) of node group auto-discovery. A definition is expressed `<name of discoverer>:[<key>[=<value>]]`|
|estimator|-|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|estimator|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|expander|"random"|Type of node group expander to be used in scale up. Available values: `["random","most-pods","least-waste","price","priority"]`|
|ignore-daemonsets-utilization|false|Should CA ignore DaemonSet pods when calculating resource utilization for scaling down|
|ignore-mirror-pods-utilization|false|Should CA ignore Mirror pods when calculating resource utilization for scaling down|
@@ -19,7 +19,7 @@ Before creating the service principal, you need to obtain the following informat
- Your subscription ID
- Your tenant ID
- An app ID (also called a client ID)
- Client ID (also known as app ID)
- Client secret
- A resource group
@@ -27,45 +27,35 @@ The below sections describe how to set up these prerequisites using either the A
### Setting Up the Service Principal with the Azure Command Line Tool
You can create the service principal by running this command:
You must assign roles to the service principal so that it has communication privileges with the AKS API. It also needs access to create and list virtual networks.
In the following example, the command creates the service principal and gives it the Contributor role. The Contributor role can manage anything on AKS but cannot give access to others. Note that you must provide `scopes` a full path to at least one Azure resource:
```
az ad sp create-for-rbac --skip-assignment
az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>
```
The result should show information about the new service principal:
```
{
"appId": "xxxx--xxx",
"displayName": "<SERVICE-PRINCIPAL-NAME>",
"name": "http://<SERVICE-PRINCIPAL-NAME>",
"password": "<SECRET>",
"tenant": "<TENANT NAME>"
"displayName": "<service-principal-name>",
"name": "http://<service-principal-name>",
"password": "<secret>",
"tenant": "<tenant-name>"
}
```
You also need to add roles to the service principal so that it has privileges for communication with the AKS API. It also needs access to create and list virtual networks.
Below is an example command for assigning the Contributor role to a service principal. Contributors can manage anything on AKS but cannot give access to others:
The following creates a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-cli) to contain your Azure resources:
```
az role assignment create \
--assignee $appId \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
You can also create the service principal and give it Contributor privileges by combining the two commands into one. In this command, the scope needs to provide a full path to an Azure resource:
```
az ad sp create-for-rbac \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
az group create --location <azure-location-name> --resource-group <resource-group-name>
```
### Setting Up the Service Principal from the Azure Portal
You can also follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
Follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
1. Go to the Microsoft Azure Portal [home page](https://portal.azure.com).
@@ -75,7 +65,7 @@ You can also follow these instructions to set up a service principal and give it
1. Click **New registration.**
1. Enter a name. This will be the name of your service principal.
1. Enter a name for your service principal.
1. Optional: Choose which accounts can use the service principal.
@@ -17,7 +17,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -105,7 +105,7 @@ Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` a
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### CNI Features by Provider
@@ -32,12 +32,12 @@ The usage below defines rules about what the audit log should record and what da
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Request Metadata | Request Body | Response Metadata | Response Body |
| --------------------- | ---------------- | ------------ | ----------------- | ------------- |
| `0` | | | | |
| `1` | ✓ | | | |
| `2` | ✓ | ✓ | | |
| `3` | ✓ | ✓ | ✓ | ✓ |
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
@@ -2,7 +2,17 @@
title: Opening Ports with firewalld
---
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
:::danger
Enabling firewalld can cause serious network communication problems.
For proper network function, firewalld must be disabled on systems running RKE2. [Firewalld conflicts with Canal](https://docs.rke2.io/known_issues#firewalld-conflicts-with-default-networking), RKE2's default networking stack.
Firewalld must also be disabled on systems running Kubernetes 1.19 and later.
If you enable firewalld on systems running Kubernetes 1.18 or earlier, understand that this may cause networking issues. CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
+17
View File
@@ -0,0 +1,17 @@
---
title: Glossary
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/glossary"/>
</head>
This page covers Rancher-specific terminology and symbols which might be unfamiliar, or which differ between Rancher versions.
```mdx-code-block
import Glossary, {toc as GlossaryTOC} from "/shared-files/_glossary.md"
<Glossary />
export const toc = GlossaryTOC;
```
@@ -37,4 +37,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
### Developing your own node driver
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
Node drivers are implemented with [Rancher Machine](https://github.com/rancher/machine), a fork of [Docker Machine](https://github.com/docker/machine). Docker Machine is no longer under active development.
Refer to the original [Docker Machine documentation](https://github.com/docker/docs/blob/vnext-engine/machine/overview.md) for details on how to develop your own node drivers.
@@ -304,7 +304,7 @@ cloud-provider|-|Cloud provider type|
|max-node-provision-time|"15m"|Maximum time CA waits for node to be provisioned|
|nodes|-|sets min,max size and other configuration data for a node group in a format accepted by cloud provider. Can be used multiple times. Format: `<min>:<max>:<other...>`|
|node-group-auto-discovery|-|One or more definition(s) of node group auto-discovery. A definition is expressed `<name of discoverer>:[<key>[=<value>]]`|
|estimator|-|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|estimator|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|expander|"random"|Type of node group expander to be used in scale up. Available values: `["random","most-pods","least-waste","price","priority"]`|
|ignore-daemonsets-utilization|false|Should CA ignore DaemonSet pods when calculating resource utilization for scaling down|
|ignore-mirror-pods-utilization|false|Should CA ignore Mirror pods when calculating resource utilization for scaling down|
@@ -19,7 +19,7 @@ Before creating the service principal, you need to obtain the following informat
- Your subscription ID
- Your tenant ID
- An app ID (also called a client ID)
- Client ID (also known as app ID)
- Client secret
- A resource group
@@ -27,45 +27,35 @@ The below sections describe how to set up these prerequisites using either the A
### Setting Up the Service Principal with the Azure Command Line Tool
You can create the service principal by running this command:
You must assign roles to the service principal so that it has communication privileges with the AKS API. It also needs access to create and list virtual networks.
In the following example, the command creates the service principal and gives it the Contributor role. The Contributor role can manage anything on AKS but cannot give access to others. Note that you must provide `scopes` a full path to at least one Azure resource:
```
az ad sp create-for-rbac --skip-assignment
az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>
```
The result should show information about the new service principal:
```
{
"appId": "xxxx--xxx",
"displayName": "<SERVICE-PRINCIPAL-NAME>",
"name": "http://<SERVICE-PRINCIPAL-NAME>",
"password": "<SECRET>",
"tenant": "<TENANT NAME>"
"displayName": "<service-principal-name>",
"name": "http://<service-principal-name>",
"password": "<secret>",
"tenant": "<tenant-name>"
}
```
You also need to add roles to the service principal so that it has privileges for communication with the AKS API. It also needs access to create and list virtual networks.
Below is an example command for assigning the Contributor role to a service principal. Contributors can manage anything on AKS but cannot give access to others:
The following creates a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-cli) to contain your Azure resources:
```
az role assignment create \
--assignee $appId \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
You can also create the service principal and give it Contributor privileges by combining the two commands into one. In this command, the scope needs to provide a full path to an Azure resource:
```
az ad sp create-for-rbac \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
az group create --location <azure-location-name> --resource-group <resource-group-name>
```
### Setting Up the Service Principal from the Azure Portal
You can also follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
Follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
1. Go to the Microsoft Azure Portal [home page](https://portal.azure.com).
@@ -75,7 +65,7 @@ You can also follow these instructions to set up a service principal and give it
1. Click **New registration.**
1. Enter a name. This will be the name of your service principal.
1. Enter a name for your service principal.
1. Optional: Choose which accounts can use the service principal.
@@ -16,7 +16,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -24,7 +24,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -18,7 +18,7 @@ Rancher is committed to informing the community of security issues in our produc
| [CVE-2022-31247](https://github.com/rancher/rancher/security/advisories/GHSA-6x34-89p7-95wg) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where a flaw with authorization logic allows privilege escalation in downstream clusters through cluster role template binding (CRTB) and project role template binding (PRTB). The vulnerability can be exploited by any user who has permissions to create/edit CRTB or PRTB (such as `cluster-owner`, `manage cluster members`, `project-owner`, and `manage project members`) to gain owner permission in another project in the same cluster or in another project on a different downstream cluster. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36783](https://github.com/rancher/rancher/security/advisories/GHSA-8w87-58w6-hfv8) | It was discovered that in Rancher versions up to and including 2.5.12 and 2.6.3, there is a failure to properly sanitize credentials in cluster template answers. This failure can lead to plaintext storage and exposure of credentials, passwords, and API tokens. The exposed credentials are visible in Rancher to authenticated `Cluster Owners`, `Cluster Members`, `Project Owners`, and `Project Members` on the endpoints `/v1/management.cattle.io.clusters`, `/v3/clusters`, and `/k8s/clusters/local/apis/management.cattle.io/v3/clusters`. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36782](https://github.com/rancher/rancher/security/advisories/GHSA-g7j7-h4q8-8w2f) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where sensitive fields like passwords, API keys, and Rancher's service account token (used to provision clusters) were stored in plaintext directly on Kubernetes objects like `Clusters` (e.g., `cluster.management.cattle.io`). Anyone with read access to those objects in the Kubernetes API could retrieve the plaintext version of those sensitive data. The issue was partially found and reported by Florian Struck (from [Continum AG](https://www.continum.net/)) and [Marco Stuurman](https://github.com/fe-ax) (from [Shock Media B.V.](https://www.shockmedia.nl/)). | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://github.com/weaveworks/weave/blob/master/site/tasks/manage/security-untrusted-networks.md) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rbac/) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Fleet](https://rancher.com/docs/rancher/v2.6/en/deploy-across-clusters/fleet/) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
@@ -103,19 +103,3 @@ When the MTU is incorrectly configured (either on hosts running Rancher, nodes i
See [Google Cloud VPN: MTU Considerations](https://cloud.google.com/vpn/docs/concepts/mtu-considerations#gateway_mtu_vs_system_mtu) for an example how to configure MTU correctly when using Google Cloud VPN between Rancher and cluster nodes.
### Resolved issues
#### Overlay network broken when using Canal/Flannel due to missing node annotations
| | |
|------------|------------|
| GitHub issue | [#13644](https://github.com/rancher/rancher/issues/13644) |
| Resolved in | v2.1.2 |
To check if your cluster is affected, the following command will list nodes that are broken (this command requires `jq` to be installed):
```
kubectl get nodes -o json | jq '.items[].metadata | select(.annotations["flannel.alpha.coreos.com/public-ip"] == null or .annotations["flannel.alpha.coreos.com/kube-subnet-manager"] == null or .annotations["flannel.alpha.coreos.com/backend-type"] == null or .annotations["flannel.alpha.coreos.com/backend-data"] == null) | .name'
```
If there is no output, the cluster is not affected.
@@ -94,7 +94,7 @@ Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` a
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### RKE2 Kubernetes clusters
@@ -10,10 +10,6 @@ This FAQ is a work in progress designed to answer the questions most frequently
See the [Technical FAQ](technical-items.md) for frequently asked technical questions.
## Does Rancher v2.x support Docker Swarm and Mesos as environment types?
Swarm and Mesos are no longer selectable options when you create a new environment in Rancher v2.x. However, both Swarm and Mesos will continue to be available as Catalog applications you can deploy. It was a tough decision to make but, in the end, it came down to adoption. For example, out of more than 15,000 clusters, only about 200 were running Swarm.
## Is it possible to manage Azure Kubernetes Services with Rancher v2.x?
Yes. See our [Cluster Administration](../how-to-guides/new-user-guides/manage-clusters/manage-clusters.md) guide for what Rancher features are available on AKS, as well as our [documentation on AKS](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md).
+17
View File
@@ -0,0 +1,17 @@
---
title: Glossary
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/glossary"/>
</head>
This page covers Rancher-specific terminology and symbols which might be unfamiliar, or which differ between Rancher versions.
```mdx-code-block
import Glossary, {toc as GlossaryTOC} from "/shared-files/_glossary.md"
<Glossary />
export const toc = GlossaryTOC;
```
@@ -36,12 +36,12 @@ The usage below defines rules about what the audit log should record and what da
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Request Metadata | Request Body | Response Metadata | Response Body |
| --------------------- | ---------------- | ------------ | ----------------- | ------------- |
| `0` | | | | |
| `1` | ✓ | | | |
| `2` | ✓ | ✓ | | |
| `3` | ✓ | ✓ | ✓ | ✓ |
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
@@ -6,7 +6,17 @@ title: Opening Ports with firewalld
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/open-ports-with-firewalld"/>
</head>
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
:::danger
Enabling firewalld can cause serious network communication problems.
For proper network function, firewalld must be disabled on systems running RKE2. [Firewalld conflicts with Canal](https://docs.rke2.io/known_issues#firewalld-conflicts-with-default-networking), RKE2's default networking stack.
Firewalld must also be disabled on systems running Kubernetes 1.19 and later.
If you enable firewalld on systems running Kubernetes 1.18 or earlier, understand that this may cause networking issues. CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
@@ -42,4 +42,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
### Developing your own node driver
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
Node drivers are implemented with [Rancher Machine](https://github.com/rancher/machine), a fork of [Docker Machine](https://github.com/docker/machine). Docker Machine is no longer under active development.
Refer to the original [Docker Machine documentation](https://github.com/docker/docs/blob/vnext-engine/machine/overview.md) for details on how to develop your own node drivers.
@@ -49,7 +49,6 @@ Setting | Value
:::
1. After you complete the **Configure Okta Account** form, click **Enable**.
Rancher redirects you to the IdP login page. Enter credentials that authenticate with Okta IdP to validate your Rancher Okta configuration.
@@ -22,59 +22,48 @@ To interact with Azure APIs, an AKS cluster requires an Azure Active Directory (
Before creating the service principal, you need to obtain the following information from the [Microsoft Azure Portal](https://portal.azure.com):
- Subscription ID
- Client ID
- Client ID (also known as app ID)
- Client secret
The below sections describe how to set up these prerequisites using either the Azure command line tool or the Azure portal.
### Setting Up the Service Principal with the Azure Command Line Tool
You can create the service principal by running this command:
You must assign roles to the service principal so that it has communication privileges with the AKS API. It also needs access to create and list virtual networks.
In the following example, the command creates the service principal and gives it the Contributor role. The Contributor role can manage anything on AKS but cannot give access to others. Note that you must provide `scopes` a full path to at least one Azure resource:
```
az ad sp create-for-rbac --skip-assignment
az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>
```
The result should show information about the new service principal:
```
{
"appId": "xxxx--xxx",
"displayName": "<SERVICE-PRINCIPAL-NAME>",
"name": "http://<SERVICE-PRINCIPAL-NAME>",
"password": "<SECRET>",
"tenant": "<TENANT NAME>"
"displayName": "<service-principal-name>",
"name": "http://<service-principal-name>",
"password": "<secret>",
"tenant": "<tenant-name>"
}
```
You also need to add roles to the service principal so that it has privileges for communication with the AKS API. It also needs access to create and list virtual networks.
Below is an example command for assigning the Contributor role to a service principal. Contributors can manage anything on AKS but cannot give access to others:
The following creates a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-cli) to contain your Azure resources:
```
az role assignment create \
--assignee $appId \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
You can also create the service principal and give it Contributor privileges by combining the two commands into one. In this command, the scope needs to provide a full path to an Azure resource:
```
az ad sp create-for-rbac \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
az group create --location <azure-location-name> --resource-group <resource-group-name>
```
### Setting Up the Service Principal from the Azure Portal
You can also follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
Follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
1. Go to the Microsoft Azure Portal [home page](https://portal.azure.com).
1. Click **Azure Active Directory**.
1. Click **App registrations**.
1. Click **New registration**.
1. Enter a name. This will be the name of your service principal.
1. Enter a name for your service principal.
1. Optional: Choose which accounts can use the service principal.
1. Click **Register**.
1. You should now see the name of your service principal under **Azure Active Directory > App registrations**.
@@ -96,7 +85,7 @@ To give role-based access to your service principal,
**Result:** Your service principal now has access to AKS.
## 1. Create the AKS Cloud Credentials
## Create the AKS Cloud Credentials
1. In the Rancher UI, click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
@@ -105,7 +94,7 @@ To give role-based access to your service principal,
1. Fill out the form. For help with filling out the form, see the [configuration reference.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/aks-cluster-configuration.md#cloud-credentials)
1. Click **Create**.
## 2. Create the AKS Cluster
## Create the AKS Cluster
Use Rancher to set up and configure your Kubernetes cluster.
@@ -119,7 +108,8 @@ Use Rancher to set up and configure your Kubernetes cluster.
You can access your cluster after its state is updated to **Active**.
## Role-based Access Control
## Configure Role-based Access Control
When provisioning an AKS cluster in the Rancher UI, RBAC is not configurable because it is required to be enabled.
RBAC is required for AKS clusters that are registered or imported into Rancher.
@@ -70,7 +70,7 @@ StatefulSets manage the deployment and scaling of Pods while maintaining a stick
1. Click **StatefulSet**.
1. In the **Volume Claim Templates** tab, click **Add Claim Template**.
1. Enter a name for the persistent volume.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Launch**.
@@ -84,7 +84,7 @@ To attach the PVC to an existing workload,
1. Go to the workload that will use storage provisioned with the StorageClass that you cared at click **⋮ > Edit Config**.
1. In the **Volume Claim Templates** section, click **Add Claim Template**.
1. Enter a persistent volume name.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Save**.
@@ -304,7 +304,7 @@ cloud-provider|-|Cloud provider type|
|max-node-provision-time|"15m"|Maximum time CA waits for node to be provisioned|
|nodes|-|sets min,max size and other configuration data for a node group in a format accepted by cloud provider. Can be used multiple times. Format: `<min>:<max>:<other...>`|
|node-group-auto-discovery|-|One or more definition(s) of node group auto-discovery. A definition is expressed `<name of discoverer>:[<key>[=<value>]]`|
|estimator|-|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|estimator|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|expander|"random"|Type of node group expander to be used in scale up. Available values: `["random","most-pods","least-waste","price","priority"]`|
|ignore-daemonsets-utilization|false|Should CA ignore DaemonSet pods when calculating resource utilization for scaling down|
|ignore-mirror-pods-utilization|false|Should CA ignore Mirror pods when calculating resource utilization for scaling down|
@@ -49,7 +49,6 @@ This is typical in Rancher, as many operations create new `RoleBinding` objects
You can reduce the number of `RoleBindings` in the upstream cluster in the following ways:
* Limit the use of the [Restricted Admin](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin) role. Apply other roles wherever possible.
* If you use [external authentication](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md), use groups to assign roles.
* Only add users to clusters and projects when necessary.
* Remove clusters and projects when they are no longer needed.
* Only use custom roles if necessary.
@@ -59,6 +58,12 @@ You can reduce the number of `RoleBindings` in the upstream cluster in the follo
* Kubernetes permissions are always "additive" (allow-list) rather than "subtractive" (deny-list). Try to minimize configurations that gives access to all but one aspect of a cluster, project, or namespace, as that will result in the creation of a high number of `RoleBinding` objects.
* Experiment to see if creating new projects or clusters manifests in fewer `RoleBindings` for your specific use case.
### Using External Authentication
If you have fifty or more users, you should configure an [external authentication provider](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md). This is necessary for better performance.
After you configure external authentication, make sure to assign permissions to groups instead of to individual users. This helps reduce the `RoleBinding` object count.
### RoleBinding Count Estimation
Predicting how many `RoleBinding` objects a given configuration will create is complicated. However, the following considerations can offer a rough estimate:
@@ -105,6 +110,14 @@ Although managed Kubernetes services make it easier to deploy and run Kubernetes
Use RKE2 for large scale use cases.
### Keep all Upstream Cluster Nodes co-located
To provide high availability, Kubernetes is designed to run nodes and control components in different zones. However, if nodes and control plane components are located in different zones, network traffic may be slower.
Traffic between Rancher components and the Kubernetes API is especially sensitive to network latency, as is etcd traffic between nodes.
To improve performance, run all upstream node clusters in the same location. In particular, make sure that latency between etcd nodes and Rancher is as low as possible.
### Keeping Kubernetes Versions Up to Date
You should keep the local Kubernetes cluster up to date. This will ensure that your cluster has all available performance enhancements and bug fixes.
@@ -118,3 +131,13 @@ The two main bottlenecks to [etcd performance](https://etcd.io/docs/v3.4/op-guid
It's best to run etcd on exactly three nodes, as adding more nodes will reduce operation speed. This may be counter-intuitive to common scaling approaches, but it's due to etcd's [replication mechanisms](https://etcd.io/docs/v3.5/faq/#what-is-maximum-cluster-size).
Etcd performance will also be negatively affected by network latency between nodes as that will slow down network communication. Etcd nodes should be located together with Rancher nodes.
### Browser Requirements
At high scale, Rancher transfers more data from the upstream cluster to UI components running in the browser, and those components also need to perform more processing.
For best performance, ensure that the host running the hardware meets these requirements:
- 2020 i5 10th generation Intel (4 cores) or equivalent
- 8 GB RAM
- Total network bandwith to the upstream cluster: 72 Mb/s (equivalent to a single 802.11n Wi-Fi 4 link stream, ~8 MB/s http download throughput)
- Round-trip time (ping time) from browser to upstream cluster: 150 ms or less
@@ -144,13 +144,13 @@ Project network isolation is available if you are using any RKE2 network plugin
#### CoreDNS
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#coredns) for additional CoreDNS configurations.
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#coredns) for additional CoreDNS configurations.
#### NGINX Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
#### Metrics Server
@@ -17,7 +17,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -25,7 +25,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -27,7 +27,7 @@ Rancher is committed to informing the community of security issues in our produc
| [CVE-2022-31247](https://github.com/rancher/rancher/security/advisories/GHSA-6x34-89p7-95wg) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where a flaw with authorization logic allows privilege escalation in downstream clusters through cluster role template binding (CRTB) and project role template binding (PRTB). The vulnerability can be exploited by any user who has permissions to create/edit CRTB or PRTB (such as `cluster-owner`, `manage cluster members`, `project-owner`, and `manage project members`) to gain owner permission in another project in the same cluster or in another project on a different downstream cluster. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36783](https://github.com/rancher/rancher/security/advisories/GHSA-8w87-58w6-hfv8) | It was discovered that in Rancher versions up to and including 2.5.12 and 2.6.3, there is a failure to properly sanitize credentials in cluster template answers. This failure can lead to plaintext storage and exposure of credentials, passwords, and API tokens. The exposed credentials are visible in Rancher to authenticated `Cluster Owners`, `Cluster Members`, `Project Owners`, and `Project Members` on the endpoints `/v1/management.cattle.io.clusters`, `/v3/clusters`, and `/k8s/clusters/local/apis/management.cattle.io/v3/clusters`. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36782](https://github.com/rancher/rancher/security/advisories/GHSA-g7j7-h4q8-8w2f) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where sensitive fields like passwords, API keys, and Rancher's service account token (used to provision clusters) were stored in plaintext directly on Kubernetes objects like `Clusters` (e.g., `cluster.management.cattle.io`). Anyone with read access to those objects in the Kubernetes API could retrieve the plaintext version of those sensitive data. The issue was partially found and reported by Florian Struck (from [Continum AG](https://www.continum.net/)) and [Marco Stuurman](https://github.com/fe-ax) (from [Shock Media B.V.](https://www.shockmedia.nl/)). | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](https://rancher.com/docs/rancher/v2.6/en/faq/networking/cni-providers/#weave) Container Network Interface (CNI) when configured through [RKE templates](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rke-templates/). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://github.com/weaveworks/weave/blob/master/site/tasks/manage/security-untrusted-networks.md) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](https://rancher.com/docs/rancher/v2.6/en/admin-settings/rbac/) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Fleet](https://rancher.com/docs/rancher/v2.6/en/deploy-across-clusters/fleet/) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
@@ -106,20 +106,3 @@ When the MTU is incorrectly configured (either on hosts running Rancher, nodes i
* `read tcp: i/o timeout`
See [Google Cloud VPN: MTU Considerations](https://cloud.google.com/vpn/docs/concepts/mtu-considerations#gateway_mtu_vs_system_mtu) for an example how to configure MTU correctly when using Google Cloud VPN between Rancher and cluster nodes.
### Resolved issues
#### Overlay network broken when using Canal/Flannel due to missing node annotations
| | |
|------------|------------|
| GitHub issue | [#13644](https://github.com/rancher/rancher/issues/13644) |
| Resolved in | v2.1.2 |
To check if your cluster is affected, the following command will list nodes that are broken (this command requires `jq` to be installed):
```
kubectl get nodes -o json | jq '.items[].metadata | select(.annotations["flannel.alpha.coreos.com/public-ip"] == null or .annotations["flannel.alpha.coreos.com/kube-subnet-manager"] == null or .annotations["flannel.alpha.coreos.com/backend-type"] == null or .annotations["flannel.alpha.coreos.com/backend-data"] == null) | .name'
```
If there is no output, the cluster is not affected.
@@ -96,7 +96,7 @@ Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` a
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### RKE2 Kubernetes clusters
@@ -16,6 +16,7 @@ Rancher will publish deprecated features as part of the [release notes](https://
| Patch Version | Release Date |
|---------------|---------------|
| [2.7.13](https://github.com/rancher/rancher/releases/tag/v2.7.13) | May 16, 2024 |
| [2.7.12](https://github.com/rancher/rancher/releases/tag/v2.7.12) | Mar 28, 2024 |
| [2.7.11](https://github.com/rancher/rancher/releases/tag/v2.7.11) | Mar 1, 2024 |
| [2.7.10](https://github.com/rancher/rancher/releases/tag/v2.7.10) | Feb 8, 2024 |
@@ -10,10 +10,6 @@ This FAQ is a work in progress designed to answer the questions most frequently
See the [Technical FAQ](technical-items.md) for frequently asked technical questions.
## Does Rancher v2.x support Docker Swarm and Mesos as environment types?
Swarm and Mesos are no longer selectable options when you create a new environment in Rancher v2.x. However, both Swarm and Mesos will continue to be available as Catalog applications you can deploy. It was a tough decision to make but, in the end, it came down to adoption. For example, out of more than 15,000 clusters, only about 200 were running Swarm.
## Is it possible to manage Azure Kubernetes Services with Rancher v2.x?
Yes. See our [Cluster Administration](../how-to-guides/new-user-guides/manage-clusters/manage-clusters.md) guide for what Rancher features are available on AKS, as well as our [documentation on AKS](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md).
+17
View File
@@ -0,0 +1,17 @@
---
title: Glossary
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/glossary"/>
</head>
This page covers Rancher-specific terminology and symbols which might be unfamiliar, or which differ between Rancher versions.
```mdx-code-block
import Glossary, {toc as GlossaryTOC} from "/shared-files/_glossary.md"
<Glossary />
export const toc = GlossaryTOC;
```
@@ -36,12 +36,12 @@ The usage below defines rules about what the audit log should record and what da
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Request Metadata | Request Body | Response Metadata | Response Body |
| --------------------- | ---------------- | ------------ | ----------------- | ------------- |
| `0` | | | | |
| `1` | ✓ | | | |
| `2` | ✓ | ✓ | | |
| `3` | ✓ | ✓ | ✓ | ✓ |
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
@@ -77,3 +77,80 @@ key.pfx=`base64-content`
```
Then **Cert File Path** would be set to `/etc/alertmanager/secrets/cert.pem`.
## Rancher Performance Dashboard
When monitoring is installed on the upstream (local) cluster, you are given basic health metrics about the Rancher pods, such as CPU and memory data. To get advanced metrics for your local Rancher server, you must additionally enable the Rancher Performance Dashboard for Grafana.
This dashboard provides access to the following advanced metrics:
- Handler Average Execution Times Over Last 5 Minutes
- Rancher API Average Request Times Over Last 5 Minutes
- Subscribe Average Request Times Over Last 5 Minutes
- Lasso Controller Work Queue Depth (Top 20)
- Number of Rancher Requests (Top 20)
- Number of Failed Rancher API Requests (Top 20)
- K8s Proxy Store Average Request Times Over Last 5 Minutes (Top 20)
- K8s Proxy Client Average Request Times Over Last 5 Minutes (Top 20)
- Cached Objects by GroupVersionKind (Top 20)
- Lasso Handler Executions (Top 20)
- Handler Executions Over Last 2 Minutes (Top 20)
- Total Handler Executions with Error (Top 20)
- Data Transmitted by Remote Dialer Sessions (Top 20)
- Errors for Remote Dialer Sessions (Top 20)
- Remote Dialer Connections Removed (Top 20)
- Remote Dialer Connections Added by Client (Top 20)
:::note
Profiling data (such as advanced memory or CPU analysis) is not present as it is a very context-dependent technique that's meant for debugging and not intended for normal observation.
:::
### Enabling the Rancher Performance Dashboard
To enable the Rancher Performance Dashboard:
<Tabs groupid="UIorCLI">
<TabItem value="Helm">
Use the following options with the Helm CLI:
```bash
--set extraEnv\[0\].name="CATTLE_PROMETHEUS_METRICS" --set-string extraEnv\[0\].value=true
```
You can also include the following snippet in your Rancher Helm chart's values.yaml file:
```yaml
extraEnv:
- name: "CATTLE_PROMETHEUS_METRICS"
value: "true"
```
</TabItem>
<TabItem value="UI">
1. Click **☰ > Cluster Management**.
1. Go to the row of the `local` cluster and click **Explore**.
1. Click **Workloads > Deployments**.
1. Use the dropdown menu at the top to filter for **All Namespaces**.
1. Under the `cattle-system` namespace, go to the `rancher` row and click **⋮ > Edit Config**
1. Under **Environment Variables**, click **Add Variable**.
1. For **Type**, select `Key/Value Pair`.
1. For **Variable Name**, enter `CATTLE_PROMETHEUS_METRICS`.
1. For **Value**, enter `true`.
1. Click **Save** to apply the change.
</TabItem>
</Tabs>
### Accessing the Rancher Performance Dashboard
1. Click **☰ > Cluster Management**.
1. Go to the row of the `local` cluster and click **Explore**.
1. Click **Monitoring**
1. Select the **Grafana** dashboard.
1. From the sidebar, click **Search dashboards**.
1. Enter `Rancher Performance Debugging` and select it.
@@ -6,7 +6,17 @@ title: Opening Ports with firewalld
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/open-ports-with-firewalld"/>
</head>
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
:::danger
Enabling firewalld can cause serious network communication problems.
For proper network function, firewalld must be disabled on systems running RKE2. [Firewalld conflicts with Canal](https://docs.rke2.io/known_issues#firewalld-conflicts-with-default-networking), RKE2's default networking stack.
Firewalld must also be disabled on systems running Kubernetes 1.19 and later.
If you enable firewalld on systems running Kubernetes 1.18 or earlier, understand that this may cause networking issues. CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
@@ -56,4 +56,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
### Developing Your Own Node Drivers
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
Node drivers are implemented with [Rancher Machine](https://github.com/rancher/machine), a fork of [Docker Machine](https://github.com/docker/machine). Docker Machine is no longer under active development.
Refer to the original [Docker Machine documentation](https://github.com/docker/docs/blob/vnext-engine/machine/overview.md) for details on how to develop your own node drivers.
@@ -51,7 +51,6 @@ You can integrate Okta with Rancher, so that authenticated users can access Ranc
:::
1. After you complete the **Configure Okta Account** form, click **Enable**.
Rancher redirects you to the IdP login page. Enter credentials that authenticate with Okta IdP to validate your Rancher Okta configuration.
@@ -79,15 +79,20 @@ If you are using [local snapshots](./back-up-rancher-launched-kubernetes-cluster
1. In the **Clusters** page, go to the cluster where you want to remove nodes.
1. In the **Machines** tab, click **⋮ > Delete** on each node you want to delete. Initially, you will see the nodes hang in a `deleting` state, but once all etcd nodes are deleting, they will be removed together. This is due to the fact that Rancher sees all etcd nodes deleting and proceeds to "short circuit" the etcd safe-removal logic.
1. After all etcd nodes are removed, add a new etcd node that you are planning to restore from.
1. After all etcd nodes are removed, add the new etcd node that you are planning to restore from. Assign the new node the role of `all` (etcd, controlplane, and worker).
- For custom clusters, go to the **Registration** tab then copy and run the registration command on your node. If the node has previously been used in a cluster, [clean the node](../manage-clusters/clean-cluster-nodes.md#cleaning-up-nodes) first.
- If the node was previously in a cluster, [clean the node](../manage-clusters/clean-cluster-nodes.md#cleaning-up-nodes) first.
- For custom clusters, go to the **Registration** tab and check the box for `etcd, controlplane, and worker`. Then copy and run the registration command on your node.
- For node driver clusters, a new node is provisioned automatically.
At this point, Rancher will indicate that restoration from etcd snapshot is required.
1. Restore from an etcd snapshot.
:::note
As the etcd node is a clean node, you may need to manually create the `/var/lib/rancher/<k3s/rke2>/server/db/snapshots/` path.
:::
- For S3 snapshots, restore using the UI.
1. Click the **Snapshots** tab to view the list of saved snapshots.
1. Go to the snapshot you want to restore and click **⋮ > Restore**.
@@ -95,7 +100,15 @@ If you are using [local snapshots](./back-up-rancher-launched-kubernetes-cluster
1. Click **Restore**.
- For local snapshots, restore using the UI is **not** available.
1. In the upper right corner, click **⋮ > Edit YAML**.
1. Define `spec.cluster.rkeConfig.etcdSnapshotRestore.name` as the filename of the snapshot on disk in `/var/lib/rancher/<k3s/rke2>/server/db/snapshots/`.
1. The example YAML below can be added under your `rkeConfig` to configure the etcd restore:
```yaml
...
rkeConfig:
etcdSnapshotRestore:
name: <string> # This field is required. Refers to the filename of the associated etcdsnapshot object.
...
```
1. After restoration is successful, you can scale your etcd nodes back up to the desired redundancy.
@@ -167,6 +167,23 @@ spec:
Only Helm 3 compatible charts are supported.
### Refresh Chart Repositories
The **Refresh** button can be used to sync changes from selected Helm chart repositories on the **Repositories** page.
To refresh a chart repository:
1. Click **☰ > Cluster Management**.
1. Find the name of the cluster whose repositories you want to access. Click **Explore** at the end of the cluster's row.
1. In the left navigation menu on the **Cluster Dashboard**, click **Apps > Repositories**.
1. Use the toggle next to the **State** field to select all repositories, or toggle specified chart repositories to sync changes.
1. Click **Refresh**.
1. The **⋮** at the end of each chart repository row also includes a **Refresh** option, which can be clicked to refresh the respective repository.
Non-Airgap Rancher installations upon refresh will reflect any chart repository changes immediately and you will see the **State** field for updated repositories move from `In Progress` to `Active` once the action is completed.
Airgap installations where Rancher is configured to use the packaged copy of Helm system charts ([`useBundledSystemChart=true`](../../../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md#helm-chart-options-for-air-gap-installations)) will only refer to the [system-chart](https://github.com/rancher/system-charts) repository that comes bundled and will not be able to be refreshed or synced.
## Deploy and Upgrade Charts
To install and deploy a chart:
@@ -21,65 +21,48 @@ To interact with Azure APIs, an AKS cluster requires an Azure Active Directory (
Before creating the service principal, you need to obtain the following information from the [Microsoft Azure Portal](https://portal.azure.com):
- Subscription ID
- Client ID
- Client ID (also known as app ID)
- Client secret
The below sections describe how to set up these prerequisites using either the Azure command line tool or the Azure portal.
### Setting Up the Service Principal with the Azure Command Line Tool
You can create the service principal by running this command:
You must assign roles to the service principal so that it has communication privileges with the AKS API. It also needs access to create and list virtual networks.
In the following example, the command creates the service principal and gives it the Contributor role. The Contributor role can manage anything on AKS but cannot give access to others. Note that you must provide `scopes` a full path to at least one Azure resource:
```
az ad sp create-for-rbac --skip-assignment
az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>
```
The result should show information about the new service principal:
```
{
"appId": "xxxx--xxx",
"displayName": "<SERVICE-PRINCIPAL-NAME>",
"name": "http://<SERVICE-PRINCIPAL-NAME>",
"password": "<SECRET>",
"tenant": "<TENANT NAME>"
"displayName": "<service-principal-name>",
"name": "http://<service-principal-name>",
"password": "<secret>",
"tenant": "<tenant-name>"
}
```
You also need to add roles to the service principal so that it has privileges for communication with the AKS API. It also needs access to create and list virtual networks.
Below is an example command for assigning the Contributor role to a service principal. Contributors can manage anything on AKS but cannot give access to others:
The following creates a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-cli) to contain your Azure resources:
```
az role assignment create \
--assignee $appId \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
You can also create the service principal and give it Contributor privileges by combining the two commands into one. In this command, the scope needs to provide a full path to an Azure resource:
```
az ad sp create-for-rbac \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
Create the Resource Group by running this command:
```
az group create --location AZURE_LOCATION_NAME --resource-group AZURE_RESOURCE_GROUP_NAME
az group create --location <azure-location-name> --resource-group <resource-group-name>
```
### Setting Up the Service Principal from the Azure Portal
You can also follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
Follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
1. Go to the Microsoft Azure Portal [home page](https://portal.azure.com).
1. Click **Azure Active Directory**.
1. Click **App registrations**.
1. Click **New registration**.
1. Enter a name. This will be the name of your service principal.
1. Enter a name for your service principal.
1. Optional: Choose which accounts can use the service principal.
1. Click **Register**.
1. You should now see the name of your service principal under **Azure Active Directory > App registrations**.
@@ -101,7 +84,7 @@ To give role-based access to your service principal,
**Result:** Your service principal now has access to AKS.
## 1. Create the AKS Cloud Credentials
## Create the AKS Cloud Credentials
1. In the Rancher UI, click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
@@ -110,7 +93,7 @@ To give role-based access to your service principal,
1. Fill out the form. For help with filling out the form, see the [configuration reference.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/aks-cluster-configuration.md#cloud-credentials)
1. Click **Create**.
## 2. Create the AKS Cluster
## Create the AKS Cluster
Use Rancher to set up and configure your Kubernetes cluster.
@@ -124,7 +107,8 @@ Use Rancher to set up and configure your Kubernetes cluster.
You can access your cluster after its state is updated to **Active**.
## Role-based Access Control
## Configure Role-based Access Control
When provisioning an AKS cluster in the Rancher UI, RBAC is not configurable because it is required to be enabled.
RBAC is required for AKS clusters that are registered or imported into Rancher.
@@ -135,8 +119,8 @@ Assign the Rancher AKSv2 role to the service principal with the Azure Command Li
```
az role assignment create \
--assignee CLIENT_ID \
--scope "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP_NAME" \
--assignee <client-id> \
--scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>" \
--role "Rancher AKSv2"
```
@@ -70,7 +70,7 @@ StatefulSets manage the deployment and scaling of Pods while maintaining a stick
1. Click **StatefulSet**.
1. In the **Volume Claim Templates** tab, click **Add Claim Template**.
1. Enter a name for the persistent volume.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Launch**.
@@ -84,7 +84,7 @@ To attach the PVC to an existing workload,
1. Go to the workload that will use storage provisioned with the StorageClass that you cared at click **⋮ > Edit Config**.
1. In the **Volume Claim Templates** section, click **Add Claim Template**.
1. Enter a persistent volume name.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Save**.
@@ -304,7 +304,7 @@ cloud-provider|-|Cloud provider type|
|max-node-provision-time|"15m"|Maximum time CA waits for node to be provisioned|
|nodes|-|sets min,max size and other configuration data for a node group in a format accepted by cloud provider. Can be used multiple times. Format: `<min>:<max>:<other...>`|
|node-group-auto-discovery|-|One or more definition(s) of node group auto-discovery. A definition is expressed `<name of discoverer>:[<key>[=<value>]]`|
|estimator|-|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|estimator|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|expander|"random"|Type of node group expander to be used in scale up. Available values: `["random","most-pods","least-waste","price","priority"]`|
|ignore-daemonsets-utilization|false|Should CA ignore DaemonSet pods when calculating resource utilization for scaling down|
|ignore-mirror-pods-utilization|false|Should CA ignore Mirror pods when calculating resource utilization for scaling down|
@@ -19,19 +19,20 @@ In order to deploy and run the adapter successfully, you need to ensure its vers
| Rancher Version | Adapter Version |
|-----------------|:---------------:|
| v2.7.0 | v2.0.0 |
| v2.7.1 | v2.0.0 |
| v2.7.2 | v2.0.1 |
| v2.7.3 | v2.0.1 |
| v2.7.4 | v2.0.1 |
| v2.7.5 | v2.0.2 |
| v2.7.6 | v2.0.2 |
| v2.7.7 | v2.0.2 |
| v2.7.8 | v2.0.2 |
| v2.7.9 | v2.0.2 |
| v2.7.10 | v2.0.2 |
| v2.7.11 | v2.0.4 |
| v2.7.13 | v2.0.4 |
| v2.7.12 | v2.0.4 |
| v2.7.11 | v2.0.4 |
| v2.7.10 | v2.0.2 |
| v2.7.9 | v2.0.2 |
| v2.7.8 | v2.0.2 |
| v2.7.7 | v2.0.2 |
| v2.7.6 | v2.0.2 |
| v2.7.5 | v2.0.2 |
| v2.7.4 | v2.0.1 |
| v2.7.3 | v2.0.1 |
| v2.7.2 | v2.0.1 |
| v2.7.1 | v2.0.0 |
| v2.7.0 | v2.0.0 |
### 1. Gain Access to the Local Cluster
@@ -185,7 +185,7 @@ For detailed information on the values supported by the chart and their usage, r
:::note
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/tasks/capi-operator/intro) in the Rancher Turtles documentation for assistance.
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/contributing/install_capi_operator) in the Rancher Turtles documentation for assistance.
:::
@@ -49,7 +49,6 @@ This is typical in Rancher, as many operations create new `RoleBinding` objects
You can reduce the number of `RoleBindings` in the upstream cluster in the following ways:
* Limit the use of the [Restricted Admin](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin) role. Apply other roles wherever possible.
* If you use [external authentication](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md), use groups to assign roles.
* Only add users to clusters and projects when necessary.
* Remove clusters and projects when they are no longer needed.
* Only use custom roles if necessary.
@@ -59,6 +58,12 @@ You can reduce the number of `RoleBindings` in the upstream cluster in the follo
* Kubernetes permissions are always "additive" (allow-list) rather than "subtractive" (deny-list). Try to minimize configurations that gives access to all but one aspect of a cluster, project, or namespace, as that will result in the creation of a high number of `RoleBinding` objects.
* Experiment to see if creating new projects or clusters manifests in fewer `RoleBindings` for your specific use case.
### Using External Authentication
If you have fifty or more users, you should configure an [external authentication provider](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md). This is necessary for better performance.
After you configure external authentication, make sure to assign permissions to groups instead of to individual users. This helps reduce the `RoleBinding` object count.
### RoleBinding Count Estimation
Predicting how many `RoleBinding` objects a given configuration will create is complicated. However, the following considerations can offer a rough estimate:
@@ -105,6 +110,14 @@ Although managed Kubernetes services make it easier to deploy and run Kubernetes
Use RKE2 for large scale use cases.
### Keep all Upstream Cluster Nodes co-located
To provide high availability, Kubernetes is designed to run nodes and control components in different zones. However, if nodes and control plane components are located in different zones, network traffic may be slower.
Traffic between Rancher components and the Kubernetes API is especially sensitive to network latency, as is etcd traffic between nodes.
To improve performance, run all upstream node clusters in the same location. In particular, make sure that latency between etcd nodes and Rancher is as low as possible.
### Keeping Kubernetes Versions Up to Date
You should keep the local Kubernetes cluster up to date. This will ensure that your cluster has all available performance enhancements and bug fixes.
@@ -118,3 +131,13 @@ The two main bottlenecks to [etcd performance](https://etcd.io/docs/v3.4/op-guid
It's best to run etcd on exactly three nodes, as adding more nodes will reduce operation speed. This may be counter-intuitive to common scaling approaches, but it's due to etcd's [replication mechanisms](https://etcd.io/docs/v3.5/faq/#what-is-maximum-cluster-size).
Etcd performance will also be negatively affected by network latency between nodes as that will slow down network communication. Etcd nodes should be located together with Rancher nodes.
### Browser Requirements
At high scale, Rancher transfers more data from the upstream cluster to UI components running in the browser, and those components also need to perform more processing.
For best performance, ensure that the host running the hardware meets these requirements:
- 2020 i5 10th generation Intel (4 cores) or equivalent
- 8 GB RAM
- Total network bandwith to the upstream cluster: 72 Mb/s (equivalent to a single 802.11n Wi-Fi 4 link stream, ~8 MB/s http download throughput)
- Round-trip time (ping time) from browser to upstream cluster: 150 ms or less
@@ -149,13 +149,13 @@ Project network isolation is available if you are using any RKE2 network plugin
##### CoreDNS
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#coredns) for additional CoreDNS configurations.
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#coredns) for additional CoreDNS configurations.
##### NGINX Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
##### Metrics Server
@@ -17,7 +17,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -25,7 +25,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -102,7 +102,7 @@ The `rancher-restricted` template is provided by Rancher to enforce the highly-r
</TabItem>
<TabItem value="v1.24 and Older">
K3s v1.24 and older support [Pod Security Policy (PSP)](https://v1-24.docs.kubernetes.io/docs/concepts/security/pod-security-policy/) for controlling pod security.
K3s v1.24 and older support [Pod Security Policy (PSP)](https://github.com/kubernetes/website/blob/release-1.24/content/en/docs/concepts/security/pod-security-policy.md) for controlling pod security.
You can enable PSPs by passing the following flags in the cluster configuration in Rancher:
@@ -27,7 +27,7 @@ Rancher is committed to informing the community of security issues in our produc
| [CVE-2022-31247](https://github.com/rancher/rancher/security/advisories/GHSA-6x34-89p7-95wg) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where a flaw with authorization logic allows privilege escalation in downstream clusters through cluster role template binding (CRTB) and project role template binding (PRTB). The vulnerability can be exploited by any user who has permissions to create/edit CRTB or PRTB (such as `cluster-owner`, `manage cluster members`, `project-owner`, and `manage project members`) to gain owner permission in another project in the same cluster or in another project on a different downstream cluster. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36783](https://github.com/rancher/rancher/security/advisories/GHSA-8w87-58w6-hfv8) | It was discovered that in Rancher versions up to and including 2.5.12 and 2.6.3, there is a failure to properly sanitize credentials in cluster template answers. This failure can lead to plaintext storage and exposure of credentials, passwords, and API tokens. The exposed credentials are visible in Rancher to authenticated `Cluster Owners`, `Cluster Members`, `Project Owners`, and `Project Members` on the endpoints `/v1/management.cattle.io.clusters`, `/v3/clusters`, and `/k8s/clusters/local/apis/management.cattle.io/v3/clusters`. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36782](https://github.com/rancher/rancher/security/advisories/GHSA-g7j7-h4q8-8w2f) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where sensitive fields like passwords, API keys, and Rancher's service account token (used to provision clusters) were stored in plaintext directly on Kubernetes objects like `Clusters` (e.g., `cluster.management.cattle.io`). Anyone with read access to those objects in the Kubernetes API could retrieve the plaintext version of those sensitive data. The issue was partially found and reported by Florian Struck (from [Continum AG](https://www.continum.net/)) and [Marco Stuurman](https://github.com/fe-ax) (from [Shock Media B.V.](https://www.shockmedia.nl/)). | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](../../faq/container-network-interface-providers.md#weave) Container Network Interface (CNI) when configured through [RKE templates](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-rke1-templates/about-rke1-templates.md). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](../../faq/container-network-interface-providers.md#weave) Container Network Interface (CNI) when configured through [RKE templates](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-rke1-templates/about-rke1-templates.md). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://github.com/weaveworks/weave/blob/master/site/tasks/manage/security-untrusted-networks.md) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Continuous Delivery with Fleet](../../integrations-in-rancher/fleet-gitops-at-scale/fleet-gitops-at-scale.md) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
@@ -20,6 +20,7 @@ Each Rancher version is designed to be compatible with a single version of the w
| Rancher Version | Webhook Version | Availability in Prime | Availability in Community |
|-----------------|-----------------|-----------------------|---------------------------|
| v2.7.13 | v0.3.8 | &check; | N/A |
| v2.7.12 | v0.3.7 | &check; | N/A |
| v2.7.11 | v0.3.7 | &check; | N/A |
| v2.7.10 | v0.3.6 | &check; | &check; |
@@ -106,20 +106,3 @@ When the MTU is incorrectly configured (either on hosts running Rancher, nodes i
* `read tcp: i/o timeout`
See [Google Cloud VPN: MTU Considerations](https://cloud.google.com/vpn/docs/concepts/mtu-considerations#gateway_mtu_vs_system_mtu) for an example how to configure MTU correctly when using Google Cloud VPN between Rancher and cluster nodes.
### Resolved issues
#### Overlay network broken when using Canal/Flannel due to missing node annotations
| | |
|------------|------------|
| GitHub issue | [#13644](https://github.com/rancher/rancher/issues/13644) |
| Resolved in | v2.1.2 |
To check if your cluster is affected, the following command will list nodes that are broken (this command requires `jq` to be installed):
```
kubectl get nodes -o json | jq '.items[].metadata | select(.annotations["flannel.alpha.coreos.com/public-ip"] == null or .annotations["flannel.alpha.coreos.com/kube-subnet-manager"] == null or .annotations["flannel.alpha.coreos.com/backend-type"] == null or .annotations["flannel.alpha.coreos.com/backend-data"] == null) | .name'
```
If there is no output, the cluster is not affected.
@@ -96,7 +96,7 @@ Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` a
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### RKE2 Kubernetes clusters
@@ -16,6 +16,7 @@ Rancher will publish deprecated features as part of the [release notes](https://
| Patch Version | Release Date |
|---------------|---------------|
| [2.8.4](https://github.com/rancher/rancher/releases/tag/v2.8.4) | May 16, 2024 |
| [2.8.3](https://github.com/rancher/rancher/releases/tag/v2.8.3) | Mar 28, 2024 |
| [2.8.2](https://github.com/rancher/rancher/releases/tag/v2.8.2) | Feb 8, 2024 |
| [2.8.1](https://github.com/rancher/rancher/releases/tag/v2.8.1) | Jan 22, 2024 |
@@ -10,10 +10,6 @@ This FAQ is a work in progress designed to answer the questions most frequently
See the [Technical FAQ](technical-items.md) for frequently asked technical questions.
## Does Rancher v2.x support Docker Swarm and Mesos as environment types?
Swarm and Mesos are no longer selectable options when you create a new environment in Rancher v2.x. However, both Swarm and Mesos will continue to be available as Catalog applications you can deploy. It was a tough decision to make but, in the end, it came down to adoption. For example, out of more than 15,000 clusters, only about 200 were running Swarm.
## Is it possible to manage Azure Kubernetes Services with Rancher v2.x?
Yes. See our [Cluster Administration](../how-to-guides/new-user-guides/manage-clusters/manage-clusters.md) guide for what Rancher features are available on AKS, as well as our [documentation on AKS](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md).
+17
View File
@@ -0,0 +1,17 @@
---
title: Glossary
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/glossary"/>
</head>
This page covers Rancher-specific terminology and symbols which might be unfamiliar, or which differ between Rancher versions.
```mdx-code-block
import Glossary, {toc as GlossaryTOC} from "/shared-files/_glossary.md"
<Glossary />
export const toc = GlossaryTOC;
```
@@ -14,20 +14,46 @@ For configuration details, refer to the [official Kubernetes documentation](http
<Tabs groupId="k8s-distro">
<TabItem value="RKE2/K3s" default>
<TabItem value="RKE2" default>
### Method 1 (Recommended): Set `audit-policy-file` in `machineGlobalConfig`
You can set `audit-policy-file` in the configuration file. Rancher delivers the file to the path `/var/lib/rancher/rke2/etc/config-files/audit-policy-file` in control plane nodes, and sets the proper options in the RKE2 server.
Example:
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
rkeConfig:
machineGlobalConfig:
audit-policy-file: |
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: RequestResponse
resources:
- group: ""
resources:
- pods
```
### Method 2: Use the Directives, `machineSelectorFiles` and `machineGlobalConfig`
:::note
This feature is available in Rancher v2.7.2 and above.
This feature is available in Rancher v2.7.2 and later.
:::
As a prerequisite, you need to create a secret or configmap which will be the source of the audit policy.
You can use `machineSelectorFiles` to deliver the audit policy file to the control plane nodes, and `machineGlobalConfig` to set the options on kube-apiserver.
The secret or configmap must meet the following two requirements:
As a prerequisite, you must create a [secret](../new-user-guides/kubernetes-resources-setup/secrets.md) or [configmap](../new-user-guides/kubernetes-resources-setup/configmaps.md) to be the source of the audit policy.
The secret or configmap must meet the following requirements:
1. It must be in the `fleet-default` namespace where the Cluster object exists.
2. It must have the annotation `rke.cattle.io/object-authorized-for-clusters: cluster-name1,cluster-name2` which permits the target clusters to use it.
2. It must have the annotation `rke.cattle.io/object-authorized-for-clusters: <cluster-name1>,<cluster-name2>` which permits the target clusters to use it.
:::tip
@@ -46,11 +72,11 @@ kind: Secret
metadata:
annotations:
rke.cattle.io/object-authorized-for-clusters: cluster1
name: name1
name: <name1>
namespace: fleet-default
```
The audit log can be enabled and configured by editing the cluster in YAML and utilizing the `machineSelectorFiles` and `machineGlobalConfig` directives.
Enable and configure the audit log by editing the cluster in YAML, and utilizing the `machineSelectorFiles` and `machineGlobalConfig` directives.
Example:
@@ -77,7 +103,88 @@ spec:
rke.cattle.io/control-plane-role: 'true'
```
For more information about cluster configuration, refer to the REK2 or K3s cluster configuration reference pages.
:::tip
You can also use the directive `machineSelectorConfig` with proper machineLabelSelectors to achieve the same effect.
:::
For more information about cluster configuration, refer to the [RKE2 cluster configuration reference](../../reference-guides/cluster-configuration/rancher-server-configuration/rke2-cluster-configuration.md) pages.
</TabItem>
<TabItem value="K3s">
:::note
This feature is available in Rancher v2.7.2 and later.
:::
You can use `machineSelectorFiles` to deliver the audit policy file to the control plane nodes, and `machineGlobalConfig` to set the options on kube-apiserver.
As a prerequisite, you must create a [secret](../new-user-guides/kubernetes-resources-setup/secrets.md) or [configmap](../new-user-guides/kubernetes-resources-setup/configmaps.md) to be the source of the audit policy.
The secret or configmap must meet the following requirements:
1. It must be in the `fleet-default` namespace where the Cluster object exists.
2. It must have the annotation `rke.cattle.io/object-authorized-for-clusters: <cluster-name1>,<cluster-name2>` which permits the target clusters to use it.
:::tip
Rancher Dashboard provides an easy-to-use form for creating the [secret](../new-user-guides/kubernetes-resources-setup/secrets.md) or [configmap](../new-user-guides/kubernetes-resources-setup/configmaps.md).
:::
Example:
```yaml
apiVersion: v1
data:
audit-policy: >-
IyBMb2cgYWxsIHJlcXVlc3RzIGF0IHRoZSBNZXRhZGF0YSBsZXZlbC4KYXBpVmVyc2lvbjogYXVkaXQuazhzLmlvL3YxCmtpbmQ6IFBvbGljeQpydWxlczoKLSBsZXZlbDogTWV0YWRhdGE=
kind: Secret
metadata:
annotations:
rke.cattle.io/object-authorized-for-clusters: cluster1
name: <name1>
namespace: fleet-default
```
Enable and configure the audit log by editing the cluster in YAML, and utilizing the `machineSelectorFiles` and `machineGlobalConfig` directives.
Example:
```yaml
apiVersion: provisioning.cattle.io/v1
kind: Cluster
spec:
rkeConfig:
machineGlobalConfig:
kube-apiserver-arg:
- audit-policy-file=<customized-path>/dev-audit-policy.yaml
- audit-log-path=<customized-path>/dev-audit.logs
machineSelectorFiles:
- fileSources:
- configMap:
name: ''
secret:
items:
- key: audit-policy
path: <customized-path>/dev-audit-policy.yaml
name: dev-audit-policy
machineLabelSelector:
matchLabels:
rke.cattle.io/control-plane-role: 'true'
```
:::tip
You can also use the directive `machineSelectorConfig` with proper machineLabelSelectors to achieve the same effect.
:::
For more information about cluster configuration, refer to the [K3s cluster configuration reference](../../reference-guides/cluster-configuration/rancher-server-configuration/k3s-cluster-configuration.md) pages.
</TabItem>
@@ -139,4 +246,3 @@ For configuration details, refer to the official [RKE1 documentation](https://rk
</TabItem>
</Tabs>
@@ -36,12 +36,12 @@ The usage below defines rules about what the audit log should record and what da
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Request Metadata | Request Body | Response Metadata | Response Body |
| --------------------- | ---------------- | ------------ | ----------------- | ------------- |
| `0` | | | | |
| `1` | ✓ | | | |
| `2` | ✓ | ✓ | | |
| `3` | ✓ | ✓ | ✓ | ✓ |
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
@@ -77,3 +77,79 @@ key.pfx=`base64-content`
```
Then **Cert File Path** would be set to `/etc/alertmanager/secrets/cert.pem`.
## Rancher Performance Dashboard
When monitoring is installed on the upstream (local) cluster, you are given basic health metrics about the Rancher pods, such as CPU and memory data. To get advanced metrics for your local Rancher server, you must additionally enable the Rancher Performance Dashboard for Grafana.
This dashboard provides access to the following advanced metrics:
- Handler Average Execution Times Over Last 5 Minutes
- Rancher API Average Request Times Over Last 5 Minutes
- Subscribe Average Request Times Over Last 5 Minutes
- Lasso Controller Work Queue Depth (Top 20)
- Number of Rancher Requests (Top 20)
- Number of Failed Rancher API Requests (Top 20)
- K8s Proxy Store Average Request Times Over Last 5 Minutes (Top 20)
- K8s Proxy Client Average Request Times Over Last 5 Minutes (Top 20)
- Cached Objects by GroupVersionKind (Top 20)
- Lasso Handler Executions (Top 20)
- Handler Executions Over Last 2 Minutes (Top 20)
- Total Handler Executions with Error (Top 20)
- Data Transmitted by Remote Dialer Sessions (Top 20)
- Errors for Remote Dialer Sessions (Top 20)
- Remote Dialer Connections Removed (Top 20)
- Remote Dialer Connections Added by Client (Top 20)
:::note
Profiling data (such as advanced memory or CPU analysis) is not present as it is a very context-dependent technique that's meant for debugging and not intended for normal observation.
:::
### Enabling the Rancher Performance Dashboard
To enable the Rancher Performance Dashboard:
<Tabs groupid="UIorCLI">
<TabItem value="Helm">
Use the following options with the Helm CLI:
```bash
--set extraEnv\[0\].name="CATTLE_PROMETHEUS_METRICS" --set-string extraEnv\[0\].value=true
```
You can also include the following snippet in your Rancher Helm chart's values.yaml file:
```yaml
extraEnv:
- name: "CATTLE_PROMETHEUS_METRICS"
value: "true"
```
</TabItem>
<TabItem value="UI">
1. Click **☰ > Cluster Management**.
1. Go to the row of the `local` cluster and click **Explore**.
1. Click **Workloads > Deployments**.
1. Use the dropdown menu at the top to filter for **All Namespaces**.
1. Under the `cattle-system` namespace, go to the `rancher` row and click **⋮ > Edit Config**
1. Under **Environment Variables**, click **Add Variable**.
1. For **Type**, select `Key/Value Pair`.
1. For **Variable Name**, enter `CATTLE_PROMETHEUS_METRICS`.
1. For **Value**, enter `true`.
1. Click **Save** to apply the change.
</TabItem>
</Tabs>
### Accessing the Rancher Performance Dashboard
1. Click **☰ > Cluster Management**.
1. Go to the row of the `local` cluster and click **Explore**.
1. Click **Monitoring**
1. Select the **Grafana** dashboard.
1. From the sidebar, click **Search dashboards**.
1. Enter `Rancher Performance Debugging` and select it.
@@ -6,7 +6,17 @@ title: Opening Ports with firewalld
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/open-ports-with-firewalld"/>
</head>
> We recommend disabling firewalld. For Kubernetes 1.19.x and higher, firewalld must be turned off.
:::danger
Enabling firewalld can cause serious network communication problems.
For proper network function, firewalld must be disabled on systems running RKE2. [Firewalld conflicts with Canal](https://docs.rke2.io/known_issues#firewalld-conflicts-with-default-networking), RKE2's default networking stack.
Firewalld must also be disabled on systems running Kubernetes 1.19 and later.
If you enable firewalld on systems running Kubernetes 1.18 or earlier, understand that this may cause networking issues. CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
@@ -56,4 +56,6 @@ If you want to use a node driver that Rancher doesn't support out-of-the-box, yo
### Developing Your Own Node Drivers
Node drivers are implemented with [Docker Machine](https://docs.docker.com/machine/).
Node drivers are implemented with [Rancher Machine](https://github.com/rancher/machine), a fork of [Docker Machine](https://github.com/docker/machine). Docker Machine is no longer under active development.
Refer to the original [Docker Machine documentation](https://github.com/docker/docs/blob/vnext-engine/machine/overview.md) for details on how to develop your own node drivers.
@@ -51,7 +51,6 @@ You can integrate Okta with Rancher, so that authenticated users can access Ranc
:::
1. After you complete the **Configure Okta Account** form, click **Enable**.
Rancher redirects you to the IdP login page. Enter credentials that authenticate with Okta IdP to validate your Rancher Okta configuration.
@@ -79,15 +79,20 @@ If you are using [local snapshots](./back-up-rancher-launched-kubernetes-cluster
1. In the **Clusters** page, go to the cluster where you want to remove nodes.
1. In the **Machines** tab, click **⋮ > Delete** on each node you want to delete. Initially, you will see the nodes hang in a `deleting` state, but once all etcd nodes are deleting, they will be removed together. This is due to the fact that Rancher sees all etcd nodes deleting and proceeds to "short circuit" the etcd safe-removal logic.
1. After all etcd nodes are removed, add a new etcd node that you are planning to restore from.
1. After all etcd nodes are removed, add the new etcd node that you are planning to restore from. Assign the new node the role of `all` (etcd, controlplane, and worker).
- For custom clusters, go to the **Registration** tab then copy and run the registration command on your node. If the node has previously been used in a cluster, [clean the node](../manage-clusters/clean-cluster-nodes.md#cleaning-up-nodes) first.
- If the node was previously in a cluster, [clean the node](../manage-clusters/clean-cluster-nodes.md#cleaning-up-nodes) first.
- For custom clusters, go to the **Registration** tab and check the box for `etcd, controlplane, and worker`. Then copy and run the registration command on your node.
- For node driver clusters, a new node is provisioned automatically.
At this point, Rancher will indicate that restoration from etcd snapshot is required.
1. Restore from an etcd snapshot.
:::note
As the etcd node is a clean node, you may need to manually create the `/var/lib/rancher/<k3s/rke2>/server/db/snapshots/` path.
:::
- For S3 snapshots, restore using the UI.
1. Click the **Snapshots** tab to view the list of saved snapshots.
1. Go to the snapshot you want to restore and click **⋮ > Restore**.
@@ -95,7 +100,15 @@ If you are using [local snapshots](./back-up-rancher-launched-kubernetes-cluster
1. Click **Restore**.
- For local snapshots, restore using the UI is **not** available.
1. In the upper right corner, click **⋮ > Edit YAML**.
1. Define `spec.cluster.rkeConfig.etcdSnapshotRestore.name` as the filename of the snapshot on disk in `/var/lib/rancher/<k3s/rke2>/server/db/snapshots/`.
1. The example YAML below can be added under your `rkeConfig` to configure the etcd restore:
```yaml
...
rkeConfig:
etcdSnapshotRestore:
name: <string> # This field is required. Refers to the filename of the associated etcdsnapshot object.
...
```
1. After restoration is successful, you can scale your etcd nodes back up to the desired redundancy.
@@ -166,6 +166,23 @@ spec:
Only Helm 3 compatible charts are supported.
### Refresh Chart Repositories
The **Refresh** button can be used to sync changes from selected Helm chart repositories on the **Repositories** page.
To refresh a chart repository:
1. Click **☰ > Cluster Management**.
1. Find the name of the cluster whose repositories you want to access. Click **Explore** at the end of the cluster's row.
1. In the left navigation menu on the **Cluster Dashboard**, click **Apps > Repositories**.
1. Use the toggle next to the **State** field to select all repositories, or toggle specified chart repositories to sync changes.
1. Click **Refresh**.
1. The **⋮** at the end of each chart repository row also includes a **Refresh** option, which can be clicked to refresh the respective repository.
Non-Airgap Rancher installations upon refresh will reflect any chart repository changes immediately and you will see the **State** field for updated repositories move from `In Progress` to `Active` once the action is completed.
Airgap installations where Rancher is configured to use the packaged copy of Helm system charts ([`useBundledSystemChart=true`](../../../getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md#helm-chart-options-for-air-gap-installations)) will only refer to the [system-chart](https://github.com/rancher/system-charts) repository that comes bundled and will not be able to be refreshed or synced.
## Deploy and Upgrade Charts
To install and deploy a chart:
@@ -21,65 +21,48 @@ To interact with Azure APIs, an AKS cluster requires an Azure Active Directory (
Before creating the service principal, you need to obtain the following information from the [Microsoft Azure Portal](https://portal.azure.com):
- Subscription ID
- Client ID
- Client ID (also known as app ID)
- Client secret
The below sections describe how to set up these prerequisites using either the Azure command line tool or the Azure portal.
### Setting Up the Service Principal with the Azure Command Line Tool
You can create the service principal by running this command:
You must assign roles to the service principal so that it has communication privileges with the AKS API. It also needs access to create and list virtual networks.
In the following example, the command creates the service principal and gives it the Contributor role. The Contributor role can manage anything on AKS but cannot give access to others. Note that you must provide `scopes` a full path to at least one Azure resource:
```
az ad sp create-for-rbac --skip-assignment
az ad sp create-for-rbac --role Contributor --scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>
```
The result should show information about the new service principal:
```
{
"appId": "xxxx--xxx",
"displayName": "<SERVICE-PRINCIPAL-NAME>",
"name": "http://<SERVICE-PRINCIPAL-NAME>",
"password": "<SECRET>",
"tenant": "<TENANT NAME>"
"displayName": "<service-principal-name>",
"name": "http://<service-principal-name>",
"password": "<secret>",
"tenant": "<tenant-name>"
}
```
You also need to add roles to the service principal so that it has privileges for communication with the AKS API. It also needs access to create and list virtual networks.
Below is an example command for assigning the Contributor role to a service principal. Contributors can manage anything on AKS but cannot give access to others:
The following creates a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-cli) to contain your Azure resources:
```
az role assignment create \
--assignee $appId \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
You can also create the service principal and give it Contributor privileges by combining the two commands into one. In this command, the scope needs to provide a full path to an Azure resource:
```
az ad sp create-for-rbac \
--scope /subscriptions/$<SUBSCRIPTION-ID>/resourceGroups/$<GROUP> \
--role Contributor
```
Create the Resource Group by running this command:
```
az group create --location AZURE_LOCATION_NAME --resource-group AZURE_RESOURCE_GROUP_NAME
az group create --location <azure-location-name> --resource-group <resource-group-name>
```
### Setting Up the Service Principal from the Azure Portal
You can also follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
Follow these instructions to set up a service principal and give it role-based access from the Azure Portal.
1. Go to the Microsoft Azure Portal [home page](https://portal.azure.com).
1. Click **Azure Active Directory**.
1. Click **App registrations**.
1. Click **New registration**.
1. Enter a name. This will be the name of your service principal.
1. Enter a name for your service principal.
1. Optional: Choose which accounts can use the service principal.
1. Click **Register**.
1. You should now see the name of your service principal under **Azure Active Directory > App registrations**.
@@ -101,7 +84,7 @@ To give role-based access to your service principal,
**Result:** Your service principal now has access to AKS.
## 1. Create the AKS Cloud Credentials
## Create the AKS Cloud Credentials
1. In the Rancher UI, click **☰ > Cluster Management**.
1. Click **Cloud Credentials**.
@@ -110,7 +93,7 @@ To give role-based access to your service principal,
1. Fill out the form. For help with filling out the form, see the [configuration reference.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/aks-cluster-configuration.md#cloud-credentials)
1. Click **Create**.
## 2. Create the AKS Cluster
## Create the AKS Cluster
Use Rancher to set up and configure your Kubernetes cluster.
@@ -124,7 +107,8 @@ Use Rancher to set up and configure your Kubernetes cluster.
You can access your cluster after its state is updated to **Active**.
## Role-based Access Control
## Configure Role-based Access Control
When provisioning an AKS cluster in the Rancher UI, RBAC is not configurable because it is required to be enabled.
RBAC is required for AKS clusters that are registered or imported into Rancher.
@@ -135,8 +119,8 @@ Assign the Rancher AKSv2 role to the service principal with the Azure Command Li
```
az role assignment create \
--assignee CLIENT_ID \
--scope "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP_NAME" \
--assignee <client-id> \
--scope "/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>" \
--role "Rancher AKSv2"
```
@@ -70,7 +70,7 @@ StatefulSets manage the deployment and scaling of Pods while maintaining a stick
1. Click **StatefulSet**.
1. In the **Volume Claim Templates** tab, click **Add Claim Template**.
1. Enter a name for the persistent volume.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Launch**.
@@ -84,7 +84,7 @@ To attach the PVC to an existing workload,
1. Go to the workload that will use storage provisioned with the StorageClass that you cared at click **⋮ > Edit Config**.
1. In the **Volume Claim Templates** section, click **Add Claim Template**.
1. Enter a persistent volume name.
1. In the **StorageClass* field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **StorageClass** field, select the StorageClass that will dynamically provision storage for pods managed by this StatefulSet.
1. In the **Mount Point** field, enter the path that the workload will use to access the volume.
1. Click **Save**.
@@ -304,7 +304,7 @@ cloud-provider|-|Cloud provider type|
|max-node-provision-time|"15m"|Maximum time CA waits for node to be provisioned|
|nodes|-|sets min,max size and other configuration data for a node group in a format accepted by cloud provider. Can be used multiple times. Format: `<min>:<max>:<other...>`|
|node-group-auto-discovery|-|One or more definition(s) of node group auto-discovery. A definition is expressed `<name of discoverer>:[<key>[=<value>]]`|
|estimator|-|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|estimator|"binpacking"|Type of resource estimator to be used in scale up. Available values: ["binpacking"]|
|expander|"random"|Type of node group expander to be used in scale up. Available values: `["random","most-pods","least-waste","price","priority"]`|
|ignore-daemonsets-utilization|false|Should CA ignore DaemonSet pods when calculating resource utilization for scaling down|
|ignore-mirror-pods-utilization|false|Should CA ignore Mirror pods when calculating resource utilization for scaling down|
@@ -19,10 +19,11 @@ In order to deploy and run the adapter successfully, you need to ensure its vers
| Rancher Version | Adapter Version |
|-----------------|:----------------:|
| v2.8.0 | v103.0.0+up3.0.0 |
| v2.8.1 | v103.0.0+up3.0.0 |
| v2.8.2 | v103.0.0+up3.0.0 |
| v2.8.4 | v103.0.1+up3.0.1 |
| v2.8.3 | v103.0.1+up3.0.1 |
| v2.8.2 | v103.0.0+up3.0.0 |
| v2.8.1 | v103.0.0+up3.0.0 |
| v2.8.0 | v103.0.0+up3.0.0 |
### 1. Gain Access to the Local Cluster
@@ -185,7 +185,7 @@ For detailed information on the values supported by the chart and their usage, r
:::note
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/tasks/capi-operator/intro) in the Rancher Turtles documentation for assistance.
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/contributing/install_capi_operator) in the Rancher Turtles documentation for assistance.
:::
@@ -49,7 +49,6 @@ This is typical in Rancher, as many operations create new `RoleBinding` objects
You can reduce the number of `RoleBindings` in the upstream cluster in the following ways:
* Limit the use of the [Restricted Admin](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin) role. Apply other roles wherever possible.
* If you use [external authentication](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md), use groups to assign roles.
* Only add users to clusters and projects when necessary.
* Remove clusters and projects when they are no longer needed.
* Only use custom roles if necessary.
@@ -59,6 +58,12 @@ You can reduce the number of `RoleBindings` in the upstream cluster in the follo
* Kubernetes permissions are always "additive" (allow-list) rather than "subtractive" (deny-list). Try to minimize configurations that gives access to all but one aspect of a cluster, project, or namespace, as that will result in the creation of a high number of `RoleBinding` objects.
* Experiment to see if creating new projects or clusters manifests in fewer `RoleBindings` for your specific use case.
### Using External Authentication
If you have fifty or more users, you should configure an [external authentication provider](../../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/authentication-config.md). This is necessary for better performance.
After you configure external authentication, make sure to assign permissions to groups instead of to individual users. This helps reduce the `RoleBinding` object count.
### RoleBinding Count Estimation
Predicting how many `RoleBinding` objects a given configuration will create is complicated. However, the following considerations can offer a rough estimate:
@@ -105,6 +110,14 @@ Although managed Kubernetes services make it easier to deploy and run Kubernetes
Use RKE2 for large scale use cases.
### Keep all Upstream Cluster Nodes co-located
To provide high availability, Kubernetes is designed to run nodes and control components in different zones. However, if nodes and control plane components are located in different zones, network traffic may be slower.
Traffic between Rancher components and the Kubernetes API is especially sensitive to network latency, as is etcd traffic between nodes.
To improve performance, run all upstream node clusters in the same location. In particular, make sure that latency between etcd nodes and Rancher is as low as possible.
### Keeping Kubernetes Versions Up to Date
You should keep the local Kubernetes cluster up to date. This will ensure that your cluster has all available performance enhancements and bug fixes.
@@ -118,3 +131,13 @@ The two main bottlenecks to [etcd performance](https://etcd.io/docs/v3.4/op-guid
It's best to run etcd on exactly three nodes, as adding more nodes will reduce operation speed. This may be counter-intuitive to common scaling approaches, but it's due to etcd's [replication mechanisms](https://etcd.io/docs/v3.5/faq/#what-is-maximum-cluster-size).
Etcd performance will also be negatively affected by network latency between nodes as that will slow down network communication. Etcd nodes should be located together with Rancher nodes.
### Browser Requirements
At high scale, Rancher transfers more data from the upstream cluster to UI components running in the browser, and those components also need to perform more processing.
For best performance, ensure that the host running the hardware meets these requirements:
- 2020 i5 10th generation Intel (4 cores) or equivalent
- 8 GB RAM
- Total network bandwith to the upstream cluster: 72 Mb/s (equivalent to a single 802.11n Wi-Fi 4 link stream, ~8 MB/s http download throughput)
- Round-trip time (ping time) from browser to upstream cluster: 150 ms or less
@@ -149,13 +149,13 @@ Project network isolation is available if you are using any RKE2 network plugin
##### CoreDNS
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#coredns) for additional CoreDNS configurations.
By default, [CoreDNS](https://coredns.io/) is installed as the default DNS provider. If CoreDNS is not installed, an alternate DNS provider must be installed yourself. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#coredns) for additional CoreDNS configurations.
##### NGINX Ingress
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
If you want to publish your applications in a high-availability configuration, and you're hosting your nodes with a cloud-provider that doesn't have a native load-balancing feature, enable this option to use NGINX Ingress within the cluster. Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking#nginx-ingress-controller) for additional configuration options.
Refer to the [RKE2 documentation](https://docs.rke2.io/networking/networking_services#nginx-ingress-controller) for additional configuration options.
##### Metrics Server
@@ -17,7 +17,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -25,7 +25,7 @@ Logging is helpful because it allows you to:
- Look for trends in your environment
- Save your logs to a safe location outside of your cluster
- Stay informed of events like a container crashing, a pod eviction, or a node dying
- More easily debugg and troubleshoot problems
- More easily debug and troubleshoot problems
Rancher can integrate with Elasticsearch, splunk, kafka, syslog, and fluentd.
@@ -102,7 +102,7 @@ The `rancher-restricted` template is provided by Rancher to enforce the highly-r
</TabItem>
<TabItem value="v1.24 and Older">
K3s v1.24 and older support [Pod Security Policy (PSP)](https://v1-24.docs.kubernetes.io/docs/concepts/security/pod-security-policy/) for controlling pod security.
K3s v1.24 and older support [Pod Security Policy (PSP)](https://github.com/kubernetes/website/blob/release-1.24/content/en/docs/concepts/security/pod-security-policy.md) for controlling pod security.
You can enable PSPs by passing the following flags in the cluster configuration in Rancher:
@@ -27,7 +27,7 @@ Rancher is committed to informing the community of security issues in our produc
| [CVE-2022-31247](https://github.com/rancher/rancher/security/advisories/GHSA-6x34-89p7-95wg) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where a flaw with authorization logic allows privilege escalation in downstream clusters through cluster role template binding (CRTB) and project role template binding (PRTB). The vulnerability can be exploited by any user who has permissions to create/edit CRTB or PRTB (such as `cluster-owner`, `manage cluster members`, `project-owner`, and `manage project members`) to gain owner permission in another project in the same cluster or in another project on a different downstream cluster. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36783](https://github.com/rancher/rancher/security/advisories/GHSA-8w87-58w6-hfv8) | It was discovered that in Rancher versions up to and including 2.5.12 and 2.6.3, there is a failure to properly sanitize credentials in cluster template answers. This failure can lead to plaintext storage and exposure of credentials, passwords, and API tokens. The exposed credentials are visible in Rancher to authenticated `Cluster Owners`, `Cluster Members`, `Project Owners`, and `Project Members` on the endpoints `/v1/management.cattle.io.clusters`, `/v3/clusters`, and `/k8s/clusters/local/apis/management.cattle.io/v3/clusters`. | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2021-36782](https://github.com/rancher/rancher/security/advisories/GHSA-g7j7-h4q8-8w2f) | An issue was discovered in Rancher versions up to and including 2.5.15 and 2.6.6 where sensitive fields like passwords, API keys, and Rancher's service account token (used to provision clusters) were stored in plaintext directly on Kubernetes objects like `Clusters` (e.g., `cluster.management.cattle.io`). Anyone with read access to those objects in the Kubernetes API could retrieve the plaintext version of those sensitive data. The issue was partially found and reported by Florian Struck (from [Continum AG](https://www.continum.net/)) and [Marco Stuurman](https://github.com/fe-ax) (from [Shock Media B.V.](https://www.shockmedia.nl/)). | 18 August 2022 | [Rancher v2.6.7](https://github.com/rancher/rancher/releases/tag/v2.6.7) and [Rancher v2.5.16](https://github.com/rancher/rancher/releases/tag/v2.5.16) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](../../faq/container-network-interface-providers.md#weave) Container Network Interface (CNI) when configured through [RKE templates](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-rke1-templates/about-rke1-templates.md). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://www.weave.works/docs/net/latest/tasks/manage/security-untrusted-networks/) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2022-21951](https://github.com/rancher/rancher/security/advisories/GHSA-vrph-m5jj-c46c) | This vulnerability only affects customers using [Weave](../../faq/container-network-interface-providers.md#weave) Container Network Interface (CNI) when configured through [RKE templates](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-rke1-templates/about-rke1-templates.md). A vulnerability was discovered in Rancher versions 2.5.0 up to and including 2.5.13, and 2.6.0 up to and including 2.6.4, where a user interface (UI) issue with RKE templates does not include a value for the Weave password when Weave is chosen as the CNI. If a cluster is created based on the mentioned template, and Weave is configured as the CNI, no password will be created for [network encryption](https://github.com/weaveworks/weave/blob/master/site/tasks/manage/security-untrusted-networks.md) in Weave; therefore, network traffic in the cluster will be sent unencrypted. | 24 May 2022 | [Rancher v2.6.5](https://github.com/rancher/rancher/releases/tag/v2.6.5) and [Rancher v2.5.14](https://github.com/rancher/rancher/releases/tag/v2.5.14) |
| [CVE-2021-36784](https://github.com/rancher/rancher/security/advisories/GHSA-jwvr-vv7p-gpwq) | A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 which allows users who have create or update permissions on [Global Roles](../../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/manage-role-based-access-control-rbac.md) to escalate their permissions, or those of another user, to admin-level permissions. Global Roles grant users Rancher-wide permissions, such as the ability to create clusters. In the identified versions of Rancher, when users are given permission to edit or create Global Roles, they are not restricted to only granting permissions which they already posses. This vulnerability affects customers who utilize non-admin users that are able to create or edit Global Roles. The most common use case for this scenario is the `restricted-admin` role. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [CVE-2021-4200](https://github.com/rancher/rancher/security/advisories/GHSA-hx8w-ghh8-r4xf) | This vulnerability only affects customers using the `restricted-admin` role in Rancher. A vulnerability was discovered in Rancher versions from 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3 where the `global-data` role in `cattle-global-data` namespace grants write access to the Catalogs. Since each user with any level of catalog access was bound to the `global-data` role, this grants write access to templates (`CatalogTemplates`) and template versions (`CatalogTemplateVersions`) for any user with any level of catalog access. New users created in Rancher are by default assigned to the `user` role (standard user), which is not designed to grant write catalog access. This vulnerability effectively elevates the privilege of any user to write access for the catalog template and catalog template version resources. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
| [GHSA-wm2r-rp98-8pmh](https://github.com/rancher/rancher/security/advisories/GHSA-wm2r-rp98-8pmh) | This vulnerability only affects customers using [Continuous Delivery with Fleet](../../integrations-in-rancher/fleet/fleet.md) for continuous delivery with authenticated Git and/or Helm repositories. An issue was discovered in `go-getter` library in versions prior to [`v1.5.11`](https://github.com/hashicorp/go-getter/releases/tag/v1.5.11) that exposes SSH private keys in base64 format due to a failure in redacting such information from error messages. The vulnerable version of this library is used in Rancher through Fleet in versions of Fleet prior to [`v0.3.9`](https://github.com/rancher/fleet/releases/tag/v0.3.9). This issue affects Rancher versions 2.5.0 up to and including 2.5.12 and from 2.6.0 up to and including 2.6.3. The issue was found and reported by Dagan Henderson from Raft Engineering. | 14 Apr 2022 | [Rancher v2.6.4](https://github.com/rancher/rancher/releases/tag/v2.6.4) and [Rancher v2.5.13](https://github.com/rancher/rancher/releases/tag/v2.5.13) |
@@ -20,6 +20,7 @@ Each Rancher version is designed to be compatible with a single version of the w
| Rancher Version | Webhook Version | Availability in Prime | Availability in Community |
|-----------------|-----------------|-----------------------|---------------------------|
| v2.8.4 | v0.4.5 | &check; | &check; |
| v2.8.3 | v0.4.3 | &check; | &check; |
| v2.8.2 | v0.4.2 | &check; | &check; |
| v2.8.1 | v0.4.2 | &check; | &check; |
@@ -106,20 +106,3 @@ When the MTU is incorrectly configured (either on hosts running Rancher, nodes i
* `read tcp: i/o timeout`
See [Google Cloud VPN: MTU Considerations](https://cloud.google.com/vpn/docs/concepts/mtu-considerations#gateway_mtu_vs_system_mtu) for an example how to configure MTU correctly when using Google Cloud VPN between Rancher and cluster nodes.
### Resolved issues
#### Overlay network broken when using Canal/Flannel due to missing node annotations
| | |
|------------|------------|
| GitHub issue | [#13644](https://github.com/rancher/rancher/issues/13644) |
| Resolved in | v2.1.2 |
To check if your cluster is affected, the following command will list nodes that are broken (this command requires `jq` to be installed):
```
kubectl get nodes -o json | jq '.items[].metadata | select(.annotations["flannel.alpha.coreos.com/public-ip"] == null or .annotations["flannel.alpha.coreos.com/kube-subnet-manager"] == null or .annotations["flannel.alpha.coreos.com/backend-type"] == null or .annotations["flannel.alpha.coreos.com/backend-data"] == null) | .name'
```
If there is no output, the cluster is not affected.
@@ -0,0 +1,17 @@
---
title: API Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/api-reference"/>
</head>
:::note
At this time, not all Rancher resources are available through the Rancher Kubernetes API.
:::
import ApiDocMdx from '@theme/ApiDocMdx';
<ApiDocMdx id="rancher-api" />
@@ -0,0 +1,85 @@
---
title: Using API Tokens
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/api-tokens"/>
</head>
Rancher v2.8.0 introduced the [Rancher Kubernetes API](./api-reference.mdx) which can be used to manage Rancher resources through `kubectl`. This page covers information on API tokens used with the [Rancher CLI](../reference-guides/cli-with-rancher), [kubeconfig files](../how-to-guides/new-user-guides/manage-clusters/access-clusters/authorized-cluster-endpoint.md#about-the-kubeconfig-file), Terraform and the [v3 API browser](./v3-rancher-api-guide.md#enable-view-in-api).
By default, some cluster-level API tokens are generated with infinite time-to-live (`ttl=0`). In other words, API tokens with `ttl=0` never expire unless you invalidate them. Tokens are not invalidated by changing a password.
You can deactivate API tokens by deleting them or by deactivating the user account.
## Deleting Tokens
To delete a token:
1. Go to the list of all tokens in the Rancher API view at `https://<Rancher-Server-IP>/v3/tokens`.
1. Access the token you want to delete by its ID. For example, `https://<Rancher-Server-IP>/v3/tokens/kubectl-shell-user-vqkqt`
1. Click **Delete**.
The following is a complete list of tokens generated with `ttl=0`:
| Token | Description |
| ----------------- | -------------------------------------------------------------------------------------- |
| `kubectl-shell-*` | Access to `kubectl` shell in the browser |
| `agent-*` | Token for agent deployment |
| `compose-token-*` | Token for compose |
| `helm-token-*` | Token for Helm chart deployment |
| `telemetry-*` | Telemetry token |
| `drain-node-*` | Token for drain (Rancher uses `kubectl` for drain because there is no native Kubernetes API). |
## Setting TTL on Kubeconfig Tokens
Admins can set a global time-to-live (TTL) on Kubeconfig tokens. Changing the default kubeconfig TTL can be done by navigating to global settings and setting [`kubeconfig-default-token-ttl-minutes`](#kubeconfig-default-token-ttl-minutes) to the desired duration in minutes. As of Rancher v2.8, the default value of [`kubeconfig-default-token-ttl-minutes`](#kubeconfig-default-token-ttl-minutes) is `43200`, which means that tokens expire in 30 days.
:::note
This setting is used by all kubeconfig tokens except those created by the CLI to [generate kubeconfig tokens](#disable-tokens-in-generated-kubeconfigs).
:::
## Disable Tokens in Generated Kubeconfigs
Set the `kubeconfig-generate-token` setting to `false`. This setting instructs Rancher to no longer automatically generate a token when a user clicks on download a kubeconfig file. When this setting is deactivated, a generated kubeconfig references the [Rancher CLI](../reference-guides/cli-with-rancher/kubectl-utility.md#authentication-with-kubectl-and-kubeconfig-tokens-with-ttl) to retrieve a short-lived token for the cluster. When this kubeconfig is used in a client, such as `kubectl`, the Rancher CLI needs to be installed to complete the log in request.
## Token Hashing
You can [enable token hashing](../how-to-guides/advanced-user-guides/enable-experimental-features/enable-experimental-features.md), where tokens undergo a one-way hash using the SHA256 algorithm. This is a non-reversible process: once enabled, this feature cannot be disabled. You should first evaluate this setting in a test environment, and/or take backups before enabling.
This feature affects all tokens which include, but are not limited to, the following:
- Kubeconfig tokens
- Bearer tokens API keys/calls
- Tokens used by internal operations
## Token Settings
These global settings affect Rancher token behavior.
| Setting | Description |
| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [`auth-user-session-ttl-minutes`](#auth-user-session-ttl-minutes) | TTL in minutes on a user auth session token. |
| [`kubeconfig-default-token-ttl-minutes`](#kubeconfig-default-token-ttl-minutes) | Default TTL applied to all kubeconfig tokens except for tokens [generated by Rancher CLI](#disable-tokens-in-generated-kubeconfigs). |
| [`auth-token-max-ttl-minutes`](#auth-token-max-ttl-minutes) | Max TTL for all tokens except those controlled by [`auth-user-session-ttl-minutes`](#auth-user-session-ttl-minutes). |
| [`kubeconfig-generate-token`](#kubeconfig-generate-token) | If true, automatically generate tokens when a user downloads a kubeconfig. |
### auth-user-session-ttl-minutes
Time to live (TTL) duration in minutes, used to determine when a user auth session token expires. When expired, the user must log in and obtain a new token. This setting is not affected by [`auth-token-max-ttl-minutes`](#auth-token-max-ttl-minutes). Session tokens are created when a user logs into Rancher.
### kubeconfig-default-token-ttl-minutes
Time to live (TTL) duration in minutes, used to determine when a kubeconfig token expires. When the token is expired, the API rejects the token. This setting can't be larger than [`auth-token-max-ttl-minutes`](#auth-token-max-ttl-minutes). This setting applies to tokens generated in a requested kubeconfig file, except for tokens [generated by Rancher CLI](#disable-tokens-in-generated-kubeconfigs). As of Rancher v2.8, the default duration is `43200`, which means that tokens expire in 30 days.
### auth-token-max-ttl-minutes
Maximum Time to Live (TTL) in minutes allowed for auth tokens. If a user attempts to create a token with a TTL greater than `auth-token-max-ttl-minutes`, Rancher sets the token TTL to the value of `auth-token-max-ttl-minutes`. Applies to all kubeconfig tokens and API tokens. As of Rancher v2.8, the default duration is `129600`, which means that tokens expire in 90 days.
### kubeconfig-generate-token
When true, kubeconfigs requested through the UI contain a valid token. When false, kubeconfigs contain a command that uses the Rancher CLI to prompt the user to log in. [The CLI then retrieves and caches a token for the user](../reference-guides/cli-with-rancher/kubectl-utility.md#authentication-with-kubectl-and-kubeconfig-tokens-with-ttl).
@@ -0,0 +1,152 @@
---
title: RK-API Quick Start Guide
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/quickstart"/>
</head>
You can access Rancher's resources through the Kubernetes API. This guide helps you get started on using this API as a Rancher user.
1. In the upper left corner, click **☰ > Global Settings**.
2. Find and copy the address in the `server-url` field.
3. [Create](../reference-guides/user-settings/api-keys.md#creating-an-api-key) a Rancher API key with no scope.
:::danger
A Rancher API key with no scope grants unrestricted access to all resources that the user can access. To prevent unauthorized use, this key should be stored securely and rotated frequently.
:::
4. Create a `kubeconfig.yaml` file. Replace `$SERVER_URL` with the server url and `$API_KEY` with your Rancher API key:
```yaml
apiVersion: v1
kind: Config
clusters:
- name: "rancher"
cluster:
server: "$SERVER_URL"
users:
- name: "rancher"
user:
token: "$API_KEY"
contexts:
- name: "rancher"
context:
user: "rancher"
cluster: "rancher"
current-context: "rancher"
```
You can use this file with any compatible tool, such as kubectl or [client-go](https://github.com/kubernetes/client-go). For a quick demo, see the [kubectl example](#api-kubectl-example).
For more information on handling more complex certificate setups, see [Specifying CA Certs](#specifying-ca-certs).
For more information on available kubeconfig options, see the [upstream documentation](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/).
## API kubectl Example
In this example, we'll show how to use kubectl to create a project, followed by deleting it. For a list of other Rancher resources available, refer to the [API Reference](./api-reference.mdx) page.
:::note
At this time, not all Rancher resources are available through the Rancher Kubernetes API.
:::
1. Set your KUBECONFIG environment variable to the kubeconfig file you just created:
```bash
export KUBECONFIG=$(pwd)/kubeconfig.yaml
```
2. Use `kubectl explain` to view the available fields for projects, or complex sub-fields of resources:
```bash
kubectl explain projects
kubectl explain projects.spec
```
Not all resources may have detailed output.
3. Add the following content to a file named `project.yaml`:
```yaml
apiVersion: management.cattle.io/v3
kind: Project
metadata:
# name should be unique across all projects in every cluster
name: p-abc123
# generateName can be used instead of `name` to randomly generate a name.
# generateName: p-
# namespace should match spec.ClusterName.
namespace: local
spec:
# clusterName should match `metadata.Name` of the target cluster.
clusterName: local
description: Example Project
# displayName is the human-readable name and is visible from the UI.
displayName: Example
```
4. Create the project:
```bash
kubectl create -f project.yaml
```
5. Delete the project:
How you delete the project depends on how you created the project name.
**A. If you used `name` when creating the project**:
```bash
kubectl delete -f project.yaml
```
**B. If you used `generateName`**:
Replace `$PROJECT_NAME` with the randomly generated name of the project displayed by Kubectl after you created the project.
```bash
kubectl delete project $PROJECT_NAME -n local
```
## Specifying CA Certs
To ensure that your tools can recognize Rancher's CA certificates, most setups require additional modifications to the above template.
1. In the upper left corner, click **☰ > Global Settings**.
2. Find and copy the value in the `ca-certs` field.
3. Save the value in a file named `rancher.crt`.
:::note
If your Rancher instance is proxied by another service, you must extract the certificate that the service is using, and add it to the kubeconfig file, as demonstrated in step 5.
:::
4. The following commands convert `rancher.crt` to base64 output, trim all new-lines, and update the cluster in the kubeconfig with the certificate, then finish by removing the `rancher.crt` file:
```bash
export KUBECONFIG=$PATH_TO_RANCHER_KUBECONFIG
kubectl config set clusters.rancher.certificate-authority-data $(cat rancher.crt | base64 -i - | tr -d '\n')
rm rancher.crt
```
5. (Optional) If you use self-signed certificatess that aren't trusted by your system, you can set the insecure option in your kubeconfig with kubectl:
:::danger
This option shouldn't be used in production as it is a security risk.
:::
```bash
export KUBECONFIG=$PATH_TO_RANCHER_KUBECONFIG
kubectl config set clusters.rancher.insecure-skip-tls-verify true
```
If your Rancher instance is proxied by another service, you must extract the certificate that the service is using, and add it to the kubeconfig file, as demonstrated above.
@@ -0,0 +1,94 @@
---
title: Previous v3 Rancher API Guide
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/v3-rancher-api-guide"/>
</head>
Rancher v2.8.0 introduced the Rancher Kubernetes API (RK-API). The previous v3 Rancher API is still available. This page describes the v3 API. For more information on RK-API, see the [RK-API quickstart](./quickstart.md) and [reference guide](./api-reference.mdx).
## How to Use the API
The previous v3 API has its own user interface accessible from a [web browser](./v3-rancher-api-guide.md#enable-view-in-api). This is an easy way to see resources, perform actions, and see the equivalent `curl` or HTTP request & response. To access it:
<Tabs>
<TabItem value="Rancher v2.6.4+">
1. Click your user avatar in the upper right corner.
1. Click **Account & API Keys**.
1. Under the **API Keys** section, find the **API Endpoint** field and click the link. The link looks something like `https://<RANCHER_FQDN>/v3`, where `<RANCHER_FQDN>` is the fully qualified domain name of your Rancher deployment.
</TabItem>
<TabItem value="Rancher before v2.6.4">
Go to the URL endpoint at `https://<RANCHER_FQDN>/v3`, where `<RANCHER_FQDN>` is the fully qualified domain name of your Rancher deployment.
</TabItem>
</Tabs>
## Authentication
API requests must include authentication information. Authentication is done with HTTP basic authentication using [API keys](../reference-guides/user-settings/api-keys.md). API keys can create new clusters and have access to multiple clusters via `/v3/clusters/`. [Cluster and project roles](../how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/cluster-and-project-roles.md) apply to these keys and restrict what clusters and projects the account can see and what actions they can take.
By default, certain cluster-level API tokens are generated with infinite time-to-live (`ttl=0`). In other words, API tokens with `ttl=0` never expire unless you invalidate them. For details on how to invalidate them, refer to the [API tokens page](api-tokens.md).
## Making Requests
The API is generally RESTful but has several features to make the definition of everything discoverable by a client so that generic clients can be written instead of having to write specific code for every type of resource. For detailed info about the generic API spec, [see further documentation](https://github.com/rancher/api-spec/blob/master/specification.md).
- Every type has a Schema which describes:
- The URL to get to the collection of this type of resource.
- Every field the resource can have, along with their type, basic validation rules, whether they are required or optional, etc.
- Every action that is possible on this type of resource, with their inputs and outputs (also as schemas).
- Every field that allows filtering.
- What HTTP verb methods are available for the collection itself, or for individual resources in the collection.
The design allows you to load just the list of schemas and access everything about the API. The UI for the API contains no code specific to Rancher itself. The URL to get Schemas is sent in every HTTP response as a `X-Api-Schemas` header. From there you can follow the `collection` link on each schema to know where to list resources, and follow other `links` inside of the returned resources to get any other information.
In practice, you may just want to construct URL strings. We highly suggest limiting this to the top-level to list a collection (`/v3/<type>`) or get a specific resource (`/v3/<type>/<id>`). Anything deeper than that is subject to change in future releases.
Resources have relationships between each other called links. Each resource includes a map of `links` with the name of the link and the URL where you can retrieve that information. Again, you should `GET` the resource and then follow the URL in the `links` map, not construct these strings yourself.
Most resources have actions, which do something or change the state of the resource. To use them, send a HTTP `POST` to the URL in the `actions` map of the action you want. Certain actions require input or produce output. See the individual documentation for each type or the schemas for specific information.
To edit a resource, send a HTTP `PUT` to the `links.update` link on the resource with the fields that you want to change. If the link is missing then you don't have permission to update the resource. Unknown fields and ones that are not editable are ignored.
To delete a resource, send a HTTP `DELETE` to the `links.remove` link on the resource. If the link is missing then you don't have permission to update the resource.
To create a new resource, HTTP `POST` to the collection URL in the schema (which is `/v3/<type>`).
## Filtering
Most collections can be filtered on the server-side by common fields using HTTP query parameters. The `filters` map shows you what fields can be filtered on and what the filtered values were for the request you made. The API UI has controls to setup filtering and show you the appropriate request. For simple "equals" matches it's just `field=value`. Modifiers can be added to the field name, for example, `field_gt=42` for "field is greater than 42." See the [API spec](https://github.com/rancher/api-spec/blob/master/specification.md#filtering) for full details.
## Sorting
Most collections can be sorted on the server-side by common fields using HTTP query parameters. The `sortLinks` map shows you what sorts are available, along with the URL to get the collection sorted by that. It also includes info about what the current response was sorted by, if specified.
## Pagination
API responses are paginated with a limit of 100 resources per page by default. This can be changed with the `limit` query parameter, up to a maximum of 1000, for example, `/v3/pods?limit=1000`. The `pagination` map in collection responses tells you whether or not you have the full result set and has a link to the next page if you do not.
## Capturing v3 API Calls
You can use browser developer tools to capture how the v3 API is called. For example, you could follow these steps to use the Chrome developer tools to get the API call for provisioning an RKE cluster:
1. In the Rancher UI, go to **Cluster Management** and click **Create.**
1. Click one of the cluster types. This example uses Digital Ocean.
1. Fill out the form with a cluster name and node template, but don't click **Create**.
1. You need to open the developer tools before the cluster creation to see the API call being recorded. To open the tools, right-click the Rancher UI and click **Inspect.**
1. In the developer tools, click the **Network** tab.
1. On the **Network** tab, make sure **Fetch/XHR** is selected.
1. In the Rancher UI, click **Create**. In the developer tools, you should see a new network request with the name `cluster?_replace=true`.
1. Right-click `cluster?_replace=true` and click **Copy > Copy as cURL.**
1. Paste the result into any text editor. You can see the POST request, including the URL it was sent to, all headers, and the full body of the request. This command can be used to create a cluster from the command line. Note: the request should be stored in a safe place because it contains credentials.
### Enable View in API
You can also view captured v3 API calls for your respective clusters and resources. This feature is not enabled by default. To enable it:
1. Click your **User Tile** in the top right corner of the UI and select **Preferences** from the drop-down menu.
2. Under the **Advanced Features** section, click **Enable "View in API"**
Once checked, the **View in API** link is displayed under the **⋮** sub-menu on resource pages in the UI.
@@ -0,0 +1,135 @@
---
title: Projects
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/api/workflows/projects"/>
</head>
## Creating a Project
Project resources may only be created on the management cluster. See below for [creating namespaces under projects in a managed cluster](#creating-a-namespace-in-a-project).
### Creating a Basic Project
```bash
kubectl create -f - <<EOF
apiVersion: management.cattle.io/v3
kind: Project
metadata:
generateName: p-
namespace: c-m-abcde
spec:
clusterName: c-m-abcde
displayName: myproject
EOF
```
Use `metadata.generateName` to ensure a unique project ID, but note that `kubectl apply` does not work with `metadata.generateName`, so `kubectl create` must be used instead.
Set `metadata.namespace` and `spec.clusterName` to the ID for the cluster the project belongs to.
If you create a project through a cluster member account, you must include the annotation, `field.cattle.io/creatorId`, and set it to the cluster member account's user ID.
```bash
kubectl create -f - <<EOF
apiVersion: management.cattle.io/v3
kind: Project
metadata:
annotations:
field.cattle.io/creatorId:
user-id
generateName: p-
namespace: c-m-abcde
spec:
clusterName: c-m-abcde
displayName: myproject
EOF
```
Setting the `field.cattle.io/creatorId` field allows the cluster member account to see project resources with the `get` command and view the project in the Rancher UI. Cluster owner and admin accounts don't need to set this annotation to perform these tasks.
### Creating a Project With a Resource Quota
Refer to [Kubernetes Resource Quota](https://kubernetes.io/docs/concepts/policy/resource-quotas/).
```bash
kubectl create -f - <<EOF
apiVersion: management.cattle.io/v3
kind: Project
metadata:
generateName: p-
namespace: c-m-abcde
spec:
clusterName: c-m-abcde
displayName: myproject
resourceQuota:
limit:
limitsCpu: 1000m
namespaceDefaultResourceQuota:
limit:
limitsCpu: 50m
EOF
```
### Creating a Project With Container Limit Ranges
Refer to [Kubernetes Limit Ranges](https://kubernetes.io/docs/concepts/policy/limit-range/).
```bash
kubectl create -f - <<EOF
apiVersion: management.cattle.io/v3
kind: Project
metadata:
generateName: p-
namespace: c-m-abcde
spec:
clusterName: c-m-abcde
displayName: myproject
containerDefaultResourceLimit:
limitsCpu: 100m
limitsMemory: 100Mi
requestsCpu: 50m
requestsMemory: 50Mi
```
## Creating a Namespace in a Project
The Project resource resides in the management cluster, even if the Project is for a managed cluster. The namespaces under the project reside in the managed cluster.
On the management cluster, look up the project ID for the cluster you are administrating since it generated using `metadata.generateName`:
```bash
kubectl --namespace c-m-abcde get projects
```
On the managed cluster, create a namespace with a project annotation:
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: mynamespace
annotations:
field.cattle.io/projectId: c-m-abcde:p-vwxyz
EOF
```
Note the format, `<cluster ID>:<project ID>`.
## Deleting a Project
Look up the project to delete in the cluster namespace:
```bash
kubectl --namespace c-m-abcde get projects
```
Delete the project under the cluster namespace:
```bash
kubectl --namespace c-m-abcde delete project p-vwxyz
```
Note that this command doesn't delete the namespaces and resources that formerly belonged to the project.
@@ -0,0 +1,9 @@
---
title: RKE Cluster Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration"/>
</head>
This page has moved [here.](../../../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md)
@@ -0,0 +1,140 @@
---
title: Contributing to Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/contribute-to-rancher"/>
</head>
Learn about the repositories used for Rancher and Rancher docs, how to build Rancher repositories, and what information to include when you file an issue.
For more detailed information on how to contribute to the development of Rancher projects, refer to the [Rancher Developer Wiki](https://github.com/rancher/rancher/wiki). The wiki has resources on many topics, including the following:
- How to set up the Rancher development environment and run tests
- The typical flow of an issue through the development lifecycle
- Coding guidelines and development best practices
- Debugging and troubleshooting
- Developing the Rancher API
On the Rancher Users Slack, the channel for developers is **#developer**.
## Rancher Docs
If you have suggestions for the documentation on this website, [open](https://github.com/rancher/rancher-docs/issues/new/choose) an issue in the main [Rancher docs](https://github.com/rancher/rancher-docs) repository. This repo contains documentation for Rancher v2.0 and later.
See the [Rancher docs README](https://github.com/rancher/rancher-docs#readme) for more details on contributing to and building the Rancher v2.x docs repo.
For documentation describing Rancher v1.6 and earlier, see the [Rancher 1.x docs](https://github.com/rancher/rancher.github.io) repo, which contains source files for https://rancher.com/docs/rancher/v1.6/en/.
## Rancher Repositories
All of repositories are located within our main GitHub organization. There are many repositories used for Rancher, but we'll provide descriptions of some of the main ones used in Rancher.
Repository | URL | Description
-----------|-----|-------------
Rancher | https://github.com/rancher/rancher | This repository is the main source code for Rancher 2.x.
Types | https://github.com/rancher/types | This repository is the repository that has all the API types for Rancher 2.x.
API Framework | https://github.com/rancher/norman | This repository is an API framework for building Rancher style APIs backed by Kubernetes Custom Resources.
User Interface | https://github.com/rancher/dashboard/ | This repository is the source of the Dashboard UI.
(Rancher) Docker Machine | https://github.com/rancher/machine | This repository is the source of the Docker Machine binary used when using Node Drivers. This is a fork of the `docker/machine` repository.
machine-package | https://github.com/rancher/machine-package | This repository is used to build the Rancher Docker Machine binary.
kontainer-engine | https://github.com/rancher/kontainer-engine | This repository is the source of kontainer-engine, the tool to provision hosted Kubernetes clusters.
RKE repository | https://github.com/rancher/rke | This repository is the source of Rancher Kubernetes Engine, the tool to provision Kubernetes clusters on any machine.
CLI | https://github.com/rancher/cli | This repository is the source code for the Rancher CLI used in Rancher 2.x.
(Rancher) Helm repository | https://github.com/rancher/helm | This repository is the source of the packaged Helm binary. This is a fork of the `helm/helm` repository.
Telemetry repository | https://github.com/rancher/telemetry | This repository is the source for the Telemetry binary.
loglevel repository | https://github.com/rancher/loglevel | This repository is the source of the loglevel binary, used to dynamically change log levels.
To see all libraries/projects used in Rancher, see the [`go.mod` file](https://github.com/rancher/rancher/blob/master/go.mod) in the `rancher/rancher` repository.
![Rancher diagram](/img/ranchercomponentsdiagram-2.6.svg)<br/>
<sup>Rancher components used for provisioning/managing Kubernetes clusters.</sup>
### Building Rancher Repositories
Every repository should have a Makefile and can be built using the `make` command. The `make` targets are based on the scripts in the `/scripts` directory in the repository, and each target will use [Dapper](https://github.com/rancher/dapper) to run the target in an isolated environment. The `Dockerfile.dapper` will be used for this process, and includes all the necessary build tooling needed.
The default target is `ci`, and will run `./scripts/validate`, `./scripts/build`, `./scripts/test` and `./scripts/package`. The resulting binaries of the build will be in `./build/bin` and are usually also packaged in a Docker image.
### Rancher Bugs, Issues or Questions
If you find any bugs or are having any trouble, please search the [reported issue](https://github.com/rancher/rancher/issues) as someone may have experienced the same issue or we are actively working on a solution.
If you can't find anything related to your issue, contact us by [filing an issue](https://github.com/rancher/rancher/issues/new). Though we have many repositories related to Rancher, we want the bugs filed in the Rancher repository so we won't miss them! If you want to ask a question or ask fellow users about an use case, we suggest creating a post on the [Rancher Forums](https://forums.rancher.com).
#### Checklist for Filing Issues
Please follow this checklist when filing an issue which will helps us investigate and fix the issue. More info means more data we can use to determine what is causing the issue or what might be related to the issue.
:::note
For large amounts of data, please use [GitHub Gist](https://gist.github.com/) or similar and link the created resource in the issue.
:::
:::note Important:
Please remove any sensitive data as it will be publicly viewable.
:::
- **Resources:** Provide as much as detail as possible on the used resources. As the source of the issue can be many things, including as much of detail as possible helps to determine the root cause. See some examples below:
- **Hosts:** What specifications does the host have, like CPU/memory/disk, what cloud does it happen on, what Amazon Machine Image are you using, what DigitalOcean droplet are you using, what image are you provisioning that we can rebuild or use when we try to reproduce
- **Operating System:** What operating system are you using? Providing specifics helps here like the output of `cat /etc/os-release` for exact OS release and `uname -r` for exact kernel used
- **Docker:** What Docker version are you using, how did you install it? Most of the details of Docker can be found by supplying output of `docker version` and `docker info`
- **Environment:** Are you in a proxy environment, are you using recognized CA/self signed certificates, are you using an external loadbalancer
- **Rancher:** What version of Rancher are you using, this can be found on the bottom left of the UI or be retrieved from the image tag you are running on the host
- **Clusters:** What kind of cluster did you create, how did you create it, what did you specify when you were creating it
- **Steps to reproduce the issue:** Provide as much detail on how you got into the reported situation. This helps the person to reproduce the situation you are in.
- Provide manual steps or automation scripts used to get from a newly created setup to the situation you reported.
- **Logs:** Provide data/logs from the used resources.
- Rancher
- Docker install
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher:|rancher/rancher " | awk '{ print $1 }')
```
- Kubernetes install using `kubectl`
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if Rancher is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
-l app=rancher \
--timestamps=true
```
- Docker install using `docker` on each of the nodes in the RKE cluster
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher@|rancher_rancher" | awk '{ print $1 }')
```
- Kubernetes Install with RKE Add-On
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if the Rancher server is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
--timestamps=true \
-f $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name="cattle-server") | .metadata.name')
```
- System logging (these might not all exist, depending on operating system)
- `/var/log/messages`
- `/var/log/syslog`
- `/var/log/kern.log`
- Docker daemon logging (these might not all exist, depending on operating system)
- `/var/log/docker.log`
- **Metrics:** If you are experiencing performance issues, please provide as much of data (files or screenshots) of metrics which can help determining what is going on. If you have an issue related to a machine, it helps to supply output of `top`, `free -m`, `df` which shows processes/memory/disk usage.
@@ -0,0 +1,199 @@
---
title: Container Network Interface (CNI) Providers
description: Learn about Container Network Interface (CNI), the CNI providers Rancher provides, the features they offer, and how to choose a provider for you
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/container-network-interface-providers"/>
</head>
## What is CNI?
CNI (Container Network Interface), a [Cloud Native Computing Foundation project](https://cncf.io/), consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of plugins. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.
Kubernetes uses CNI as an interface between network providers and Kubernetes pod networking.
![CNI Logo](/img/cni-logo.png)
For more information visit [CNI GitHub project](https://github.com/containernetworking/cni).
## What Network Models are Used in CNI?
CNI network providers implement their network fabric using either an encapsulated network model such as Virtual Extensible Lan ([VXLAN](https://github.com/flannel-io/flannel/blob/master/Documentation/backends.md#vxlan)) or an unencapsulated network model such as Border Gateway Protocol ([BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)).
### What is an Encapsulated Network?
This network model provides a logical Layer 2 (L2) network encapsulated over the existing Layer 3 (L3) network topology that spans the Kubernetes cluster nodes. With this model you have an isolated L2 network for containers without needing routing distribution, all at the cost of minimal overhead in terms of processing and increased IP package size, which comes from an IP header generated by overlay encapsulation. Encapsulation information is distributed by UDP ports between Kubernetes workers, interchanging network control plane information about how MAC addresses can be reached. Common encapsulation used in this kind of network model is VXLAN, Internet Protocol Security (IPSec), and IP-in-IP.
In simple terms, this network model generates a kind of network bridge extended between Kubernetes workers, where pods are connected.
This network model is used when an extended L2 bridge is preferred. This network model is sensitive to L3 network latencies of the Kubernetes workers. If datacenters are in distinct geolocations, be sure to have low latencies between them to avoid eventual network segmentation.
CNI network providers using this network model include Flannel, Canal, Weave, and Cilium. By default, Calico is not using this model, but it can be configured to do so.
![Encapsulated Network](/img/encapsulated-network.png)
### What is an Unencapsulated Network?
This network model provides an L3 network to route packets between containers. This model doesn't generate an isolated l2 network, nor generates overhead. These benefits come at the cost of Kubernetes workers having to manage any route distribution that's needed. Instead of using IP headers for encapsulation, this network model uses a network protocol between Kubernetes workers to distribute routing information to reach pods, such as [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol).
In simple terms, this network model generates a kind of network router extended between Kubernetes workers, which provides information about how to reach pods.
This network model is used when a routed L3 network is preferred. This mode dynamically updates routes at the OS level for Kubernetes workers. It's less sensitive to latency.
CNI network providers using this network model include Calico and Cilium. Cilium may be configured with this model although it is not the default mode.
![Unencapsulated Network](/img/unencapsulated-network.png)
## What CNI Providers are Provided by Rancher?
### RKE Kubernetes clusters
Out-of-the-box, Rancher provides the following CNI network providers for RKE Kubernetes clusters: Canal, Flannel, Calico, and Weave.
You can choose your CNI network provider when you create new Kubernetes clusters from Rancher.
#### Canal
![Canal Logo](/img/canal-logo.png)
Canal is a CNI network provider that gives you the best of Flannel and Calico. It allows users to easily deploy Calico and Flannel networking together as a unified networking solution, combining Calico’s network policy enforcement with the rich superset of Calico (unencapsulated) and/or Flannel (encapsulated) network connectivity options.
In Rancher, Canal is the default CNI network provider combined with Flannel and VXLAN encapsulation.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (health checks). If using Wireguard, you should open UDP ports `51820` and `51821`. For more details, refer to [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md).
![](/img/canal-diagram.png)
For more information, see the [Canal GitHub Page.](https://github.com/projectcalico/canal)
#### Flannel
![Flannel Logo](/img/flannel-logo.png)
Flannel is a simple and easy way to configure L3 network fabric designed for Kubernetes. Flannel runs a single binary agent named flanneld on each host, which is responsible for allocating a subnet lease to each host out of a larger, preconfigured address space. Flannel uses either the Kubernetes API or etcd directly to store the network configuration, the allocated subnets, and any auxiliary data (such as the host's public IP). Packets are forwarded using one of several backend mechanisms, with the default encapsulation being [VXLAN](https://github.com/flannel-io/flannel/blob/master/Documentation/backends.md#vxlan).
Encapsulated traffic is unencrypted by default. Flannel provides two solutions for encryption:
* [IPSec](https://github.com/flannel-io/flannel/blob/master/Documentation/backends.md#ipsec), which makes use of [strongSwan](https://www.strongswan.org/) to establish encrypted IPSec tunnels between Kubernetes workers. It is an experimental backend for encryption.
* [WireGuard](https://github.com/flannel-io/flannel/blob/master/Documentation/backends.md#wireguard), which is a more faster-performing alternative to strongSwan.
Kubernetes workers should open UDP port `8472` (VXLAN). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
![Flannel Diagram](/img/flannel-diagram.png)
For more information, see the [Flannel GitHub Page](https://github.com/flannel-io/flannel).
#### Weave
<DeprecationWeave />
![Weave Logo](/img/weave-logo.png)
Weave enables networking and network policy in Kubernetes clusters across the cloud. Additionally, it support encrypting traffic between the peers.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See the [port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
For more information, see the following pages:
- [Weave Net Official Site](https://www.weave.works/)
### RKE2 Kubernetes clusters
Out-of-the-box, Rancher provides the following CNI network providers for RKE2 Kubernetes clusters: [Canal](#canal) (see above section), Calico, and Cilium.
You can choose your CNI network provider when you create new Kubernetes clusters from Rancher.
#### Calico
![Calico Logo](/img/calico-logo.png)
Calico enables networking and network policy in Kubernetes clusters across the cloud. By default, Calico uses a pure, unencapsulated IP network fabric and policy engine to provide networking for your Kubernetes workloads. Workloads are able to communicate over both cloud infrastructure and on-prem using BGP.
Calico also provides a stateless IP-in-IP or VXLAN encapsulation mode that can be used, if necessary. Calico also offers policy isolation, allowing you to secure and govern your Kubernetes workloads using advanced ingress and egress policies.
Kubernetes workers should open TCP port `179` if using BGP or UDP port `4789` if using VXLAN encapsulation. In addition, TCP port `5473` is needed when using Typha. See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
:::note Important:
In Rancher v2.6.3, Calico probes fail on Windows nodes upon RKE2 installation. <b>Note that this issue is resolved in v2.6.4.</b>
- To work around this issue, first navigate to `https://<rancherserverurl>/v3/settings/windows-rke2-install-script`.
- There, change the current setting: `https://raw.githubusercontent.com/rancher/wins/v0.1.3/install.ps1` to this new setting: `https://raw.githubusercontent.com/rancher/rke2/master/windows/rke2-install.ps1`.
:::
![Calico Diagram](/img/calico-diagram.svg)
For more information, see the following pages:
- [Project Calico Official Site](https://www.projectcalico.org/)
- [Project Calico GitHub Page](https://github.com/projectcalico/calico)
#### Cilium
![Cilium Logo](/img/cilium-logo.png)
Cilium enables networking and network policies (L3, L4, and L7) in Kubernetes. By default, Cilium uses eBPF technologies to route packets inside the node and VXLAN to send packets to other nodes. Unencapsulated techniques can also be configured.
Cilium recommends kernel versions greater than 5.2 to be able to leverage the full potential of eBPF. Kubernetes workers should open TCP port `8472` for VXLAN and TCP port `4240` for health checks. In addition, ICMP 8/0 must be enabled for health checks. For more information, check [Cilium System Requirements](https://docs.cilium.io/en/latest/operations/system_requirements/#firewall-requirements).
##### Ingress Routing Across Nodes in Cilium
<br/>
By default, Cilium does not allow pods to contact pods on other nodes. To work around this, enable the ingress controller to route requests across nodes with a `CiliumNetworkPolicy`.
After selecting the Cilium CNI and enabling Project Network Isolation for your new cluster, configure as follows:
```
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: hn-nodes
namespace: default
spec:
endpointSelector: {}
ingress:
- fromEntities:
- remote-node
```
## CNI Features by Provider
The following table summarizes the different features available for each CNI network provider provided by Rancher.
| Provider | Network Model | Route Distribution | Network Policies | Mesh | External Datastore | Encryption | Ingress/Egress Policies |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| Canal | Encapsulated (VXLAN) | No | Yes | No | K8s API | Yes | Yes |
| Flannel | Encapsulated (VXLAN) | No | No | No | K8s API | Yes | No |
| Calico | Encapsulated (VXLAN,IPIP) OR Unencapsulated | Yes | Yes | Yes | Etcd and K8s API | Yes | Yes |
| Weave | Encapsulated | Yes | Yes | Yes | No | Yes | Yes |
| Cilium | Encapsulated (VXLAN) | Yes | Yes | Yes | Etcd and K8s API | Yes | Yes |
- Network Model: Encapsulated or unencapsulated. For more information, see [What Network Models are Used in CNI?](#what-network-models-are-used-in-cni)
- Route Distribution: An exterior gateway protocol designed to exchange routing and reachability information on the Internet. BGP can assist with pod-to-pod networking between clusters. This feature is a must on unencapsulated CNI network providers, and it is typically done by BGP. If you plan to build clusters split across network segments, route distribution is a feature that's nice-to-have.
- Network Policies: Kubernetes offers functionality to enforce rules about which services can communicate with each other using network policies. This feature is stable as of Kubernetes v1.7 and is ready to use with certain networking plugins.
- Mesh: This feature allows service-to-service networking communication between distinct Kubernetes clusters.
- External Datastore: CNI network providers with this feature need an external datastore for its data.
- Encryption: This feature allows cyphered and secure network control and data planes.
- Ingress/Egress Policies: This feature allows you to manage routing control for both Kubernetes and non-Kubernetes communications.
## CNI Community Popularity
<CNIPopularityTable />
## Which CNI Provider Should I Use?
It depends on your project needs. There are many different providers, which each have various features and options. There isn't one provider that meets everyone's needs.
Canal is the default CNI network provider. We recommend it for most use cases. It provides encapsulated networking for containers with Flannel, while adding Calico network policies that can provide project/namespace isolation in terms of networking.
## How can I configure a CNI network provider?
Please see [Cluster Options](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md) on how to configure a network provider for your cluster. For more advanced configuration options, please see how to configure your cluster using a [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#rke-cluster-config-file-reference) and the options for [Network Plug-ins](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/).
@@ -0,0 +1,27 @@
---
title: Deprecated Features in Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/deprecated-features"/>
</head>
### What is Rancher's deprecation policy?
We have published our official deprecation policy in the support [terms of service](https://rancher.com/support-maintenance-terms).
### Where can I find out which features have been deprecated in Rancher?
Rancher will publish deprecated features as part of the [release notes](https://github.com/rancher/rancher/releases) for Rancher found on GitHub. Please consult the following patch releases for deprecated features:
| Patch Version | Release Date |
|---------------|---------------|
| [2.8.4](https://github.com/rancher/rancher/releases/tag/v2.8.4) | May 16, 2024 |
| [2.8.3](https://github.com/rancher/rancher/releases/tag/v2.8.3) | Mar 28, 2024 |
| [2.8.2](https://github.com/rancher/rancher/releases/tag/v2.8.2) | Feb 8, 2024 |
| [2.8.1](https://github.com/rancher/rancher/releases/tag/v2.8.1) | Jan 22, 2024 |
| [2.8.0](https://github.com/rancher/rancher/releases/tag/v2.8.0) | Dec 6, 2023 |
### What can I expect when a feature is marked for deprecation?
In the release where functionality is marked as "Deprecated", it will still be available and supported allowing upgrades to follow the usual procedure. Once upgraded, users/admins should start planning to move away from the deprecated functionality before upgrading to the release it marked as removed. The recommendation for new deployments is to not use the deprecated feature.
@@ -0,0 +1,49 @@
---
title: Dockershim FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/dockershim"/>
</head>
The Dockershim is the CRI compliant layer between the Kubelet and the Docker daemon. As part of the Kubernetes 1.20 release, the [deprecation of the in-tree Dockershim was announced](https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/). Removal is currently scheduled for Kubernetes 1.24. For more information on the deprecation and its timelines, see the [Kubernetes Dockershim Deprecation FAQ](https://kubernetes.io/blog/2020/12/02/dockershim-faq/#when-will-dockershim-be-removed).
RKE clusters, starting with Kubernetes 1.21, now support the external Dockershim to continue leveraging Docker as the CRI runtime. We now implement the upstream open source community Dockershim announced by [Mirantis and Docker](https://www.mirantis.com/blog/mirantis-to-take-over-support-of-kubernetes-dockershim-2/) to ensure RKE clusters can continue to leverage Docker.
To enable the external Dockershim, configure the following option.
```
enable_cri_dockerd: true
```
For users looking to use another container runtime, Rancher has the edge-focused K3s and datacenter-focused RKE2 Kubernetes distributions that use containerd as the default runtime. Imported RKE2 and K3s Kubernetes clusters can then be upgraded and managed through Rancher even after the removal of in-tree Dockershim in Kubernetes 1.24.
### FAQ
<br/>
Q. Do I have to upgrade Rancher to get Rancher’s support of the upstream Dockershim?
The upstream support of Dockershim begins for RKE in Kubernetes 1.21. You will need to be on Rancher 2.6 or above to have support for RKE with Kubernetes 1.21. See our [support matrix](https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.6.0/) for details.
<br/>
Q. I am currently on RKE with Kubernetes 1.20. Do I need to upgrade to RKE with Kubernetes 1.21 sooner to avoid being out of support for Dockershim?
A. The version of Dockershim in RKE with Kubernetes 1.20 will continue to work and is not scheduled for removal upstream until Kubernetes 1.24. It will only emit a warning of its future deprecation, which Rancher has mitigated in RKE with Kubernetes 1.21. You can plan your upgrade to Kubernetes 1.21 as you would normally, but should consider enabling the external Dockershim by Kubernetes 1.22. The external Dockershim will need to be enabled before upgrading to Kubernetes 1.24, at which point the existing implementation will be removed.
For more information on the deprecation and its timeline, see the [Kubernetes Dockershim Deprecation FAQ](https://kubernetes.io/blog/2020/12/02/dockershim-faq/#when-will-dockershim-be-removed).
<br/>
Q: What are my other options if I don’t want to depend on the Dockershim?
A: You can use a runtime like containerd with Kubernetes that does not require Dockershim support. RKE2 or K3s are two options for doing this.
<br/>
Q: If I am already using RKE1 and want to switch to RKE2, what are my migration options?
A: Rancher is exploring the possibility of an in-place upgrade path. Alternatively you can always migrate workloads from one cluster to another using kubectl.
<br/>
@@ -0,0 +1,47 @@
---
title: General FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/general-faq"/>
</head>
This FAQ is a work in progress designed to answer the questions most frequently asked about Rancher v2.x.
See the [Technical FAQ](technical-items.md) for frequently asked technical questions.
## Is it possible to manage Azure Kubernetes Services with Rancher v2.x?
Yes. See our [Cluster Administration](../how-to-guides/new-user-guides/manage-clusters/manage-clusters.md) guide for what Rancher features are available on AKS, as well as our [documentation on AKS](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md).
## Does Rancher support Windows?
Yes. Rancher supports Windows Server 1809 containers. For details on how to set up a cluster with Windows worker nodes, refer to the section on [configuring custom clusters for Windows.](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/use-windows-clusters/use-windows-clusters.md)
## Does Rancher support Istio?
Yes. Rancher supports [Istio](../integrations-in-rancher/istio/istio.md).
## Will Rancher v2.x support Hashicorp's Vault for storing secrets?
Secrets management is on our roadmap but we haven't assigned it to a specific release yet.
## Does Rancher v2.x support RKT containers as well?
At this time, we only support Docker.
## Does Rancher v2.x support Calico, Contiv, Contrail, Flannel, Weave net, etc., for embedded and registered Kubernetes?
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico and Weave. Always refer to the [Rancher Support Matrix](https://rancher.com/support-maintenance-terms/) for details about what is officially supported.
## Are you planning on supporting Traefik for existing setups?
We don't currently plan on providing embedded Traefik support, but we're still exploring load-balancing approaches.
## Can I import OpenShift Kubernetes clusters into v2.x?
Our goal is to run any Kubernetes clusters. Therefore, Rancher v2.x should work with OpenShift, but we haven't tested it yet.
## Is Longhorn integrated with Rancher?
Yes. Longhorn is integrated with Rancher v2.5 and later.
@@ -0,0 +1,33 @@
---
title: Installing and Configuring kubectl
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/install-and-configure-kubectl"/>
</head>
`kubectl` is a CLI utility for running commands against Kubernetes clusters. It's required for many maintenance and administrative tasks in Rancher 2.x.
### Installation
See [kubectl Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation on your operating system.
### Configuration
When you create a Kubernetes cluster with RKE, RKE creates a `kube_config_cluster.yml` in the local directory that contains credentials to connect to your new cluster with tools like `kubectl` or `helm`.
You can copy this file as `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_cluster.yml`.
```
export KUBECONFIG=$(pwd)/kube_config_cluster.yml
```
Test your connectivity with `kubectl` and see if you can get the list of nodes back.
```
kubectl get nodes
NAME STATUS ROLES AGE VERSION
165.227.114.63 Ready controlplane,etcd,worker 11m v1.10.1
165.227.116.167 Ready controlplane,etcd,worker 11m v1.10.1
165.227.127.226 Ready controlplane,etcd,worker 11m v1.10.1
```
@@ -0,0 +1,65 @@
---
title: Rancher is No Longer Needed
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/rancher-is-no-longer-needed"/>
</head>
This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted.
### If the Rancher server is deleted, what happens to the workloads in my downstream clusters?
If Rancher is ever deleted or unrecoverable, all workloads in the downstream Kubernetes clusters managed by Rancher will continue to function as normal.
### If the Rancher server is deleted, how do I access my downstream clusters?
The capability to access a downstream cluster without Rancher depends on the type of cluster and the way that the cluster was created. To summarize:
- **Registered clusters:** The cluster will be unaffected and you can access the cluster using the same methods that you did before the cluster was registered into Rancher.
- **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
- **RKE clusters:** To access an [RKE cluster,](../how-to-guides/new-user-guides/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) the cluster must have the [authorized cluster endpoint](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.](../how-to-guides/new-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
### What if I don't want Rancher anymore?
:::note
The previously recommended [System Tools](../reference-guides/system-tools.md) has been deprecated since June 2022.
:::
If you [installed Rancher on a Kubernetes cluster,](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) remove Rancher by using the [Rancher Cleanup](https://github.com/rancher/rancher-cleanup) tool.
Uninstalling Rancher in high-availability (HA) mode will also remove all `helm-operation-*` pods and the following apps:
- fleet
- fleet-agent
- rancher-operator
- rancher-webhook
Custom resources (CRDs) and custom namespaces will still need to be manually removed.
If you installed Rancher with Docker, you can uninstall Rancher by removing the single Docker container that it runs in.
Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
### What if I don't want my registered cluster managed by Rancher?
If a registered cluster is deleted from the Rancher UI, the cluster is detached from Rancher, leaving it intact and accessible by the same methods that were used to access it before it was registered in Rancher.
To detach the cluster,
1. In the upper left corner, click **☰ > Cluster Management**.
2. Go to the registered cluster that should be detached from Rancher and click **⋮ > Delete**.
3. Click **Delete**.
**Result:** The registered cluster is detached from Rancher and functions normally outside of Rancher.
### What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?
At this time, there is no functionality to detach these clusters from Rancher. In this context, "detach" is defined as the ability to remove Rancher components from the cluster and manage access to the cluster independently of Rancher.
The capability to manage these clusters without Rancher is being tracked in this [issue.](https://github.com/rancher/rancher/issues/25234)
For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
@@ -0,0 +1,22 @@
---
title: Security FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/security"/>
</head>
### Is there a Hardening Guide?
The Hardening Guide is located in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
### Have hardened Rancher Kubernetes clusters been evaluated by the CIS Kubernetes Benchmark? Where can I find the results?
We have run the CIS Kubernetes benchmark against a hardened Rancher Kubernetes cluster. The results of that assessment can be found in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
### How does Rancher verify communication with downstream clusters, and what are some associated security concerns?
Communication between the Rancher server and downstream clusters is performed through agents. Rancher uses either a registered certificate authority (CA) bundle or the local trust store to verify communication between Rancher agents and the Rancher server. Using a CA bundle for verification is more strict, as only the certificates based on that bundle are trusted. If TLS verification for a explicit CA bundle fails, Rancher may fall back to using the local trust store for verifying future communication. Any CA within the local trust store can then be used to generate a valid certificate.
As described in [Rancher Security Update CVE-2024-22030](https://www.suse.com/c/rancher-security-update/), under a narrow set of circumstances, malicious actors can take over Rancher nodes by exploiting the behavior of Rancher CAs. For the attack to succeed, the malicious actor must generate a valid certificate from either a valid CA in the targeted Rancher server, or from a valid registered CA. The attacker also needs to either hijack or spoof the Rancher server-url as a preliminary step. Rancher is currently evaluating Rancher CA behavior to mitigate against this and any similar avenues of attack.
@@ -0,0 +1,180 @@
---
title: Technical FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/technical-items"/>
</head>
### How can I reset the administrator password?
Docker Install:
```
$ docker exec -ti <container_id> reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher --no-headers | head -1 | awk '{ print $1 }') -c rancher -- reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
### I deleted/deactivated the last admin, how can I fix it?
Docker Install:
```
$ docker exec -ti <container_id> ensure-default-admin
New default administrator (user-xxxxx)
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- ensure-default-admin
New password for default administrator (user-xxxxx):
<new_password>
```
### How can I enable debug logging?
See [Troubleshooting: Logging](../troubleshooting/other-troubleshooting-tips/logging.md)
### My ClusterIP does not respond to ping
ClusterIP is a virtual IP, which will not respond to ping. Best way to test if the ClusterIP is configured correctly, is by using `curl` to access the IP and port to see if it responds.
### Where can I manage Node Templates?
Node Templates can be accessed by opening your account menu (top right) and selecting `Node Templates`.
### Why is my Layer-4 Load Balancer in `Pending` state?
The Layer-4 Load Balancer is created as `type: LoadBalancer`. In Kubernetes, this needs a cloud provider or controller that can satisfy these requests, otherwise these will be in `Pending` state forever. More information can be found on [Cloud Providers](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-cloud-providers/set-up-cloud-providers.md) or [Create External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)
### Where is the state of Rancher stored?
- Docker Install: in the embedded etcd of the `rancher/rancher` container, located at `/var/lib/rancher`.
- Kubernetes install: in the etcd of the RKE cluster created to run Rancher.
### How are the supported Docker versions determined?
We follow the validated Docker versions for upstream Kubernetes releases. The validated versions can be found under [External Dependencies](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#external-dependencies) in the Kubernetes release CHANGELOG.md.
### How can I access nodes created by Rancher?
SSH keys to access the nodes created by Rancher can be downloaded via the **Nodes** view. Choose the node which you want to access and click on the vertical ⋮ button at the end of the row, and choose **Download Keys** as shown in the picture below.
![Download Keys](/img/downloadsshkeys.png)
Unzip the downloaded zip file, and use the file `id_rsa` to connect to you host. Be sure to use the correct username (`rancher` or `docker` for RancherOS, `ubuntu` for Ubuntu, `ec2-user` for Amazon Linux)
```
$ ssh -i id_rsa user@ip_of_node
```
### How can I automate task X in Rancher?
The UI consists of static files, and works based on responses of the API. That means every action/task that you can execute in the UI, can be automated via the API. There are 2 ways to do this:
* Visit `https://your_rancher_ip/v3` and browse the API options.
* Capture the API calls when using the UI (Most commonly used for this is [Chrome Developer Tools](https://developers.google.com/web/tools/chrome-devtools/#network) but you can use anything you like)
### The IP address of a node changed, how can I recover?
A node is required to have a static IP configured (or a reserved IP via DHCP). If the IP of a node has changed, you will have to remove it from the cluster and readd it. After it is removed, Rancher will update the cluster to the correct state. If the cluster is no longer in `Provisioning` state, the node is removed from the cluster.
When the IP address of the node changed, Rancher lost connection to the node, so it will be unable to clean the node properly. See [Cleaning cluster nodes](../how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md) to clean the node.
When the node is removed from the cluster, and the node is cleaned, you can readd the node to the cluster.
### How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?
You can add more arguments/binds/environment variables via the [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#rke-cluster-config-file-reference) option in Cluster Options. For more information, see the [Extra Args, Extra Binds, and Extra Environment Variables](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/) in the RKE documentation or browse the [Example Cluster.ymls](https://rancher.com/docs/rke/latest/en/example-yamls/).
### How do I check if my certificate chain is valid?
Use the `openssl verify` command to validate your certificate chain:
:::tip
Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS-installed certificates are not used when verifying manually.
:::
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you receive the error `unable to get local issuer certificate`, the chain is incomplete. This usually means that there is an intermediate CA certificate that issued your server certificate. If you already have this certificate, you can use it in the verification of the certificate like shown below:
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem -untrusted intermediate.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you have successfully verified your certificate chain, you should include needed intermediate CA certificates in the server certificate to complete the certificate chain for any connection made to Rancher (for example, by the Rancher agent). The order of the certificates in the server certificate file should be first the server certificate itself (contents of `rancher.yourdomain.com.pem`), followed by intermediate CA certificate(s) (contents of `intermediate.pem`).
```
-----BEGIN CERTIFICATE-----
%YOUR_CERTIFICATE%
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
%YOUR_INTERMEDIATE_CERTIFICATE%
-----END CERTIFICATE-----
```
If you still get errors during verification, you can retrieve the subject and the issuer of the server certificate using the following command:
```
openssl x509 -noout -subject -issuer -in rancher.yourdomain.com.pem
subject= /C=GB/ST=England/O=Alice Ltd/CN=rancher.yourdomain.com
issuer= /C=GB/ST=England/O=Alice Ltd/CN=Alice Intermediate CA
```
### How do I check `Common Name` and `Subject Alternative Names` in my server certificate?
Although technically an entry in `Subject Alternative Names` is required, having the hostname in both `Common Name` and as entry in `Subject Alternative Names` gives you maximum compatibility with older browser/applications.
Check `Common Name`:
```
openssl x509 -noout -subject -in cert.pem
subject= /CN=rancher.my.org
```
Check `Subject Alternative Names`:
```
openssl x509 -noout -in cert.pem -text | grep DNS
DNS:rancher.my.org
```
### Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?
This is due to a combination of the following default Kubernetes settings:
* kubelet
* `node-status-update-frequency`: Specifies how often kubelet posts node status to master (default 10s)
* kube-controller-manager
* `node-monitor-period`: The period for syncing NodeStatus in NodeController (default 5s)
* `node-monitor-grace-period`: Amount of time which we allow running Node to be unresponsive before marking it unhealthy (default 40s)
* `pod-eviction-timeout`: The grace period for deleting pods on failed nodes (default 5m0s)
See [Kubernetes: kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) and [Kubernetes: kube-controller-manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) for more information on these settings.
In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. See [Kubernetes: Taint based Evictions](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) for more information.
* kube-apiserver (Kubernetes v1.13 and up)
* `default-not-ready-toleration-seconds`: Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.
* `default-unreachable-toleration-seconds`: Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.
### Can I use keyboard shortcuts in the UI?
Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
@@ -0,0 +1,35 @@
---
title: Telemetry FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/telemetry"/>
</head>
### What is Telemetry?
Telemetry collects aggregate information about the size of Rancher installations, versions of components used, and which features are used. This information is used by Rancher Labs to help make the product better and is not shared with third-parties.
### What information is collected?
No specific identifying information like usernames, passwords, or the names or addresses of user resources will ever be collected.
The primary things collected include:
- Aggregate counts (smallest, average, largest, total) of nodes per-cluster and their size (e.g. CPU cores & RAM).
- Aggregate counts of logical resources like Clusters, Projects, Namespaces, and Pods.
- Counts of what driver was used to deploy clusters and nodes (e.g. GKE vs EC2 vs Imported vs Custom).
- Versions of Kubernetes components, Operating Systems and Docker that are deployed on nodes.
- Whether some optional components are enabled or not (e.g. which auth providers are used).
- The image name & version of Rancher that is running.
- A unique randomly-generated identifier for this installation.
### Can I see the information that is being sent?
If Telemetry is enabled, you can go to `https://<your rancher server>/v1-telemetry` in your installation to see the current data.
If Telemetry is not enabled, the process that collects the data is not running, so there is nothing being collected to look at.
### How do I turn it on or off?
After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.

Some files were not shown because too many files have changed in this diff Show More