public/rancher-docs
1
0
Fork 0
mirror of https://github.com/rancher/rancher-docs.git synced 2026-09-25 12:38:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'rancher:main' into add-v2.11

Browse Source
This commit is contained in:
Sunil Singh
2025-02-18 14:29:48 -08:00
parent 9755f89c14 8c76072fd2
commit 10b84c443b
461 changed files with 1319 additions and 2503 deletions
Download Patch File Download Diff File Expand all files Collapse all files
versioned_docs/version-2.10/getting-started/installation-and-upgrade/installation-references/feature-flags.md
+1 -1
View File
@@ -16,7 +16,7 @@ Some feature flags require a restart of the Rancher container. Features that req
:::
The following is a list of feature flags available in Rancher. If you've upgraded from a previous Rancher version, you may see additional flags in the Rancher UI, such as `proxy` or `dashboard` (both [discontinued](/versioned_docs/version-2.5/reference-guides/installation-references/feature-flags.md)):
The following is a list of feature flags available in Rancher. If you've upgraded from a previous Rancher version, you may see additional flags in the Rancher UI, such as `proxy` or `dashboard` (both [discontinued](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.5/reference-guides/installation-references/feature-flags.md)):
- `clean-stale-secrets`: Removes stale secrets from the `cattle-impersonation-system` namespace. This slowly cleans up old secrets which are no longer being used by the impersonation system.
- `continuous-delivery`: Allows Fleet GitOps to be disabled separately from Fleet. See [Continuous Delivery.](../../../how-to-guides/advanced-user-guides/enable-experimental-features/continuous-delivery.md) for more information.
versioned_docs/version-2.10/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/configure-keycloak-oidc.md
+99 -68
View File
@@ -3,7 +3,7 @@ title: Configure Keycloak (OIDC)
description: Create a Keycloak OpenID Connect (OIDC) client and configure Rancher to work with Keycloak. By the end your users will be able to sign into Rancher using their Keycloak logins
---
<head>
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/configure-keycloak-oidc"/>
</head>
@@ -17,46 +17,88 @@ If you have an existing configuration using the SAML protocol and want to switch
- On Rancher, Keycloak (SAML) is disabled.
- You must have a [Keycloak IdP Server](https://www.keycloak.org/guides#getting-started) configured.
- In Keycloak, create a [new OIDC client](https://www.keycloak.org/docs/latest/server_admin/#oidc-clients), with the settings below. See the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/#oidc-clients) for help.
- Follow the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/#proc-creating-oidc-client_server_administration_guide) to create a new OIDC client with the settings below.
Setting | Value
------------|------------
`Client ID` | &lt;CLIENT_ID> (e.g. `rancher`)
`Name` | &lt;CLIENT_NAME> (e.g. `rancher`)
`Client Protocol` | `openid-connect`
`Access Type` | `confidential`
`Valid Redirect URI` | `https://yourRancherHostURL/verify-auth`
| Setting | Value |
| ------------|------------|
| `Client ID` | &lt;client-id> (e.g. `rancher`) |
| `Name` | &lt;client-name> (e.g. `rancher`) |
| `Client type` | `OpenID Connect` |
| `Client authentication` | `ON` |
| `Valid Redirect URI` | `https://yourRancherHostURL/verify-auth` |
- In the new OIDC client, create [Mappers](https://www.keycloak.org/docs/latest/server_admin/#_protocol-mappers) to expose the users fields.
- Create a new "Groups Mapper" with the settings below.
1. In the navigation menu, click **Clients**.
1. Click the **Clients list** tab.
1. Find and click the client you created.
1. Click the **Client scopes** tab.
1. Find and click the link labeled `<client-name>-dedicated`. For example, if you named your client `rancher`, look for the link named `rancher-dedicated`.
1. Click the **Mappers** tab.
1. Click **Configure a new mapper**. If you already have existing mappers configured, click the arrow next to **Add mapper** and select **By configuration**. Repeat this process and create these mappers:
- From the mappings table, select **Group Membership** and configure a new "Groups Mapper" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Groups Mapper`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `groups`
`Add to ID token` | `OFF`
`Add to access token` | `OFF`
`Add to user info` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Groups Mapper` |
| `Mapper Type` | `Group Membership` |
| `Token Claim Name` | `groups` |
| `Full group path` | `OFF` |
| `Add to ID token` | `OFF` |
| `Add to access token` | `OFF` |
| `Add to user info` | `ON` |
- Create a new "Client Audience" with the settings below.
- From the mappings table, select **Audience** and configure a new "Client Audience" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Client Audience`
`Mapper Type` | `Audience`
`Included Client Audience` | &lt;CLIENT_NAME>
`Add to access token` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Client Audience` |
| `Mapper Type` | `Audience` |
| `Included Client Audience` | &lt;client-name> |
| `Add to ID token` | `OFF` |
| `Add to access token` | `ON` |
- Create a new "Groups Path" with the settings below.
- From the mappings table, select **Group Membership** and configure a new "Groups Path" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Group Path`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `full_group_path`
`Full group path` | `ON`
`Add to user info` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Group Path` |
| `Mapper Type` | `Group Membership` |
| `Token Claim Name` | `full_group_path` |
| `Full group path` | `ON` |
| `Add to ID token` | `ON` |
| `Add to access token` | `ON` |
| `Add to user info` | `ON` |
- Add the following role mappings to all users or groups that need to query the Keycloak users.
<Tabs>
<TabItem value="Users">
1. In the navigation menu, click **Users**.
1. Click the user you want to add role mappings to.
1. Click the **Role mapping** tab.
1. Click **Assign role**.
1. Select the following roles:
- query-users
- query-groups
- view-users
1. Click **Assign**.
</TabItem>
<TabItem value="Groups">
1. In the navigation menu, click **Groups**.
1. Click the group you want to add role mappings to.
1. Click the **Role mapping** tab.
1. Click **Assign role**.
1. Select the following roles:
- query-users
- query-groups
- view-users
1. Click **Assign**.
</TabItem>
</Tabs>
## Configuring Keycloak in Rancher
@@ -64,6 +106,13 @@ If you have an existing configuration using the SAML protocol and want to switch
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (OIDC)**.
1. Complete the **Configure a Keycloak OIDC account** form. For help with filling the form, see the [configuration reference](#configuration-reference).
:::note
When configuring the **Endpoints** section using the **Generate** option, Rancher includes `/auth` as part of the context path in the **Issuer** and **Auth Endpoint** fields, which is only valid for Keycloak 16 or older. You must configure endpoints using the **Specify** option for [Keycloak 17](https://www.keycloak.org/docs/latest/release_notes/index.html#keycloak-17-0-0) and newer, which have [migrated to Quarkus](https://www.keycloak.org/migration/migrating-to-quarkus).
:::
1. After you complete the **Configure a Keycloak OIDC account** form, click **Enable**.
Rancher redirects you to the IdP login page. Enter credentials that authenticate with Keycloak IdP to validate your Rancher Keycloak configuration.
@@ -74,7 +123,7 @@ If you have an existing configuration using the SAML protocol and want to switch
:::
**Result:** Rancher is configured to work with Keycloak using the OIDC protocol. Your users can now sign into Rancher using their Keycloak logins.
**Result:** Rancher is configured to work with Keycloak using the OIDC protocol. Your users can now sign in to Rancher using their Keycloak logins.
## Configuration Reference
@@ -94,37 +143,19 @@ If you have an existing configuration using the SAML protocol and want to switch
This section describes the process to transition from using Rancher with Keycloak (SAML) to Keycloak (OIDC).
### Reconfigure Keycloak
1. Reconfigure Keycloak.
1. Configure a new `OpenID Connect` client according to the [Prerequisites](#prerequisites). Ensure the same `Valid Redirect URIs` are set.
1. Configure mappers for the new client according to the [Prerequisites](#prerequisites).
1. Before configuring Rancher to use Keycloak (OIDC), Keycloak (SAML) must be first disabled.
1. In the Rancher UI, click **☰ > Users & Authentication**.
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (SAML)**.
1. Click **Disable**.
1. Follow the steps in [Configuring Keycloak in Rancher](#configuring-keycloak-in-rancher).
1. Change the existing client to use the OIDC protocol. In the Keycloak console, select **Clients**, select the SAML client to migrate, select the **Settings** tab, change `Client Protocol` from `saml` to `openid-connect`, and click **Save**
:::caution
1. Verify the `Valid Redirect URIs` are still valid.
1. Select the **Mappers** tab and create a new Mapper with the settings below.
Setting | Value
------------|------------
`Name` | `Groups Mapper`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `groups`
`Add to ID token` | `ON`
`Add to access token` | `ON`
`Add to user info` | `ON`
### Reconfigure Rancher
Before configuring Rancher to use Keycloak (OIDC), Keycloak (SAML) must be first disabled.
1. In the Rancher UI, click **☰ > Users & Authentication**.
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (SAML)**.
1. Click **Disable**.
Configure Rancher to use Keycloak (OIDC) by following the steps in [this section](#configuring-keycloak-in-rancher).
:::note
After configuration is completed, Rancher user permissions will need to be reapplied as they are not automatically migrated.
After configuration is completed, Rancher user permissions need to be reapplied as they are not automatically migrated.
:::
@@ -132,21 +163,21 @@ After configuration is completed, Rancher user permissions will need to be reapp
If you are experiencing issues while testing the connection to the Keycloak server, first double-check the configuration options of your OIDC client. You may also inspect the Rancher logs to help pinpoint what's causing issues. Debug logs may contain more detailed information about the error. Please refer to [How can I enable debug logging](../../../../faq/technical-items.md#how-can-i-enable-debug-logging) in this documentation.
All Keycloak related log entries will be prepended with either `[generic oidc]` or `[keycloak oidc]`.
All Keycloak related log entries are prepended with either `[generic oidc]` or `[keycloak oidc]`.
### You are not redirected to Keycloak
When you fill the **Configure a Keycloak OIDC account** form and click on **Enable**, you are not redirected to your IdP.
When you fill the **Configure a Keycloak OIDC account** form and click **Enable**, you are not redirected to your IdP.
* Verify your Keycloak client configuration.
Verify your Keycloak client configuration.
### The generated `Issuer` and `Auth Endpoint` are incorrect
* On the **Configure a Keycloak OIDC account** form, change **Endpoints** to `Specify (advanced)` and override the `Issuer` and `Auth Endpoint` values. To find the values, go to the Keycloak console and select **Realm Settings**, select the **General** tab, and click **OpenID Endpoint Configuration**. The JSON output will display values for `issuer` and `authorization_endpoint`.
On the **Configure a Keycloak OIDC account** form, change **Endpoints** to `Specify (advanced)` and override the `Issuer` and `Auth Endpoint` values. To find the values, go to the Keycloak console and select **Realm Settings**, select the **General** tab, and click **OpenID Endpoint Configuration**. The JSON output displays values for `issuer` and `authorization_endpoint`.
### Keycloak Error: "Invalid grant_type"
* In some cases, this error message may be misleading and is actually caused by setting the `Valid Redirect URI` incorrectly.
In some cases, this error message may be misleading and is caused by setting the `Valid Redirect URI` incorrectly.
### Unable to See Groups When Assigning Global Roles
versioned_docs/version-2.10/integrations-in-rancher/cluster-api/cluster-api.md
+1 -1
View File
@@ -9,6 +9,6 @@ title: Cluster API (CAPI) with Rancher Turtles
[Rancher Turtles](https://turtles.docs.rancher.com/) is a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/#operators-in-kubernetes) that manages the lifecycle of provisioned Kubernetes clusters, by providing integration between your Cluster API (CAPI) and Rancher. With Rancher Turtles, you can:
- Import CAPI clusters into Rancher, by installing the Rancher Cluster Agent in CAPI provisioned clusters.
- Configure the [CAPI Operator](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values#cluster-api-operator-values).
- Configure the [CAPI Operator](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html#cluster-api-operator-values).
The [Overview](./overview.md) section outlines installation options, Rancher Turtles architecture, and a brief demo. For more details, see the [Rancher Turtles documentation](https://turtles.docs.rancher.com/).
versioned_docs/version-2.10/integrations-in-rancher/cluster-api/overview.md
+5 -5
View File
@@ -16,7 +16,7 @@ Below is a visual representation of the key components of Rancher Turtles and th
As defined by [Supply-chain Levels for Software Artifacts (SLSA)](https://slsa.dev/spec/v1.0/about), SLSA is a set of incrementally adoptable guidelines for supply chain security, established by industry consensus. The specification set by SLSA is useful for both software producers and consumers: producers can follow SLSA’s guidelines to make their software supply chain more secure, and consumers can use SLSA to make decisions about whether to trust a software package.
Rancher Turtles meets [SLSA Level 3](https://slsa.dev/spec/v1.0/levels#build-l3) requirements as an appropriate hardened build platform, with consistent build processes, and provenance distribution. For more information, visit the [Rancher Turtles Security](https://turtles.docs.rancher.com/security/slsa) document.
Rancher Turtles meets [SLSA Level 3](https://slsa.dev/spec/v1.0/levels#build-l3) requirements as an appropriate hardened build platform, with consistent build processes, and provenance distribution. For more information, visit the [Rancher Turtles Security](https://turtles.docs.rancher.com/turtles/next/en/security/slsa.html) document.
## Prerequisites
@@ -92,7 +92,7 @@ By adding the Turtles repository via the Rancher UI, Rancher can process the ins
1. Click **Rancher Turtles - the Cluster API Extension**.
1. Click **Install > Next > Install**.
This process uses the default values for the Helm chart, which are good for most installations. If your configuration requires overriding some of these defaults, you can either specify the values during installation from the Rancher UI or you can [manually install the chart via Helm](#installing-via-helm). For details about available values, see the Rancher Turtles [Helm chart reference guide](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values).
This process uses the default values for the Helm chart, which are good for most installations. If your configuration requires overriding some of these defaults, you can either specify the values during installation from the Rancher UI or you can [manually install the chart via Helm](#installing-via-helm). For details about available values, see the Rancher Turtles [Helm chart reference guide](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html).
The installation may take a few minutes and after completing you can see the following new deployments in the cluster:
@@ -115,7 +115,7 @@ There are two ways to install Rancher Turtles with Helm, depending on whether yo
The CAPI Operator is required for installing Rancher Turtles. You can choose whether you want to take care of this dependency yourself or let the Rancher Turtles Helm chart manage it for you. [Installing Turtles as a dependency](#installing-rancher-turtles-with-cluster-api-capi-operator-as-a-helm-dependency) is simpler, but your best option depends on your specific configuration.
The CAPI Operator allows for handling the lifecycle of [CAPI providers](https://turtles.docs.rancher.com/tasks/capi-operator/installing_core_provider) using a declarative approach, extending the capabilities of `clusterctl`. If you want to learn more about it, you can refer to [Cluster API Operator book](https://cluster-api-operator.sigs.k8s.io/).
The CAPI Operator allows for handling the lifecycle of [CAPI providers](https://turtles.docs.rancher.com/turtles/next/en/tasks/capi-operator/installing_core_provider) using a declarative approach, extending the capabilities of `clusterctl`. If you want to learn more about it, you can refer to [Cluster API Operator book](https://cluster-api-operator.sigs.k8s.io/).
#### Installing Rancher Turtles with `Cluster API (CAPI) Operator` as a Helm dependency
@@ -177,7 +177,7 @@ stringData:
:::info
For detailed information on the values supported by the chart and their usage, refer to [Helm chart options](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values)
For detailed information on the values supported by the chart and their usage, refer to [Helm chart options](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html)
:::
@@ -185,7 +185,7 @@ For detailed information on the values supported by the chart and their usage, r
:::note
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/contributing/install_capi_operator) in the Rancher Turtles documentation for assistance.
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/turtles/next/en/contributing/install_capi_operator.html) in the Rancher Turtles documentation for assistance.
:::
versioned_docs/version-2.10/integrations-in-rancher/integrations-in-rancher.md
+1 -1
View File
@@ -15,4 +15,4 @@ To learn more and get started with Rancher Prime, please visit [this page](https
import DocCardList from '@theme/DocCardList';
import { useCurrentSidebarCategory } from '@docusaurus/theme-common/internal';
<DocCardList items={useCurrentSidebarCategory().items.slice(0,9)} />
<DocCardList items={useCurrentSidebarCategory().items.slice(0,10)} />
versioned_docs/version-2.11/getting-started/installation-and-upgrade/installation-references/feature-flags.md
+1 -1
View File
@@ -16,7 +16,7 @@ Some feature flags require a restart of the Rancher container. Features that req
:::
The following is a list of feature flags available in Rancher. If you've upgraded from a previous Rancher version, you may see additional flags in the Rancher UI, such as `proxy` or `dashboard` (both [discontinued](/versioned_docs/version-2.5/reference-guides/installation-references/feature-flags.md)):
The following is a list of feature flags available in Rancher. If you've upgraded from a previous Rancher version, you may see additional flags in the Rancher UI, such as `proxy` or `dashboard` (both [discontinued](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.5/reference-guides/installation-references/feature-flags.md)):
- `clean-stale-secrets`: Removes stale secrets from the `cattle-impersonation-system` namespace. This slowly cleans up old secrets which are no longer being used by the impersonation system.
- `continuous-delivery`: Allows Fleet GitOps to be disabled separately from Fleet. See [Continuous Delivery.](../../../how-to-guides/advanced-user-guides/enable-experimental-features/continuous-delivery.md) for more information.
versioned_docs/version-2.11/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/configure-keycloak-oidc.md
+98 -77
View File
@@ -3,7 +3,7 @@ title: Configure Keycloak (OIDC)
description: Create a Keycloak OpenID Connect (OIDC) client and configure Rancher to work with Keycloak. By the end your users will be able to sign into Rancher using their Keycloak logins
---
<head>
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config/configure-keycloak-oidc"/>
</head>
@@ -17,55 +17,88 @@ If you have an existing configuration using the SAML protocol and want to switch
- On Rancher, Keycloak (SAML) is disabled.
- You must have a [Keycloak IdP Server](https://www.keycloak.org/guides#getting-started) configured.
- In Keycloak, create a [new OIDC client](https://www.keycloak.org/docs/latest/server_admin/#oidc-clients), with the settings below. See the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/#oidc-clients) for help.
- Follow the [Keycloak documentation](https://www.keycloak.org/docs/latest/server_admin/#proc-creating-oidc-client_server_administration_guide) to create a new OIDC client with the settings below.
Setting | Value
------------|------------
`Client ID` | &lt;CLIENT_ID> (e.g. `rancher`)
`Name` | &lt;CLIENT_NAME> (e.g. `rancher`)
`Client Protocol` | `openid-connect`
`Access Type` | `confidential`
`Valid Redirect URI` | `https://yourRancherHostURL/verify-auth`
| Setting | Value |
| ------------|------------|
| `Client ID` | &lt;client-id> (e.g. `rancher`) |
| `Name` | &lt;client-name> (e.g. `rancher`) |
| `Client type` | `OpenID Connect` |
| `Client authentication` | `ON` |
| `Valid Redirect URI` | `https://yourRancherHostURL/verify-auth` |
- In the new OIDC client, create [Mappers](https://www.keycloak.org/docs/latest/server_admin/#_protocol-mappers) to expose the users fields.
- Create a new "Groups Mapper" with the settings below.
1. In the navigation menu, click **Clients**.
1. Click the **Clients list** tab.
1. Find and click the client you created.
1. Click the **Client scopes** tab.
1. Find and click the link labeled `<client-name>-dedicated`. For example, if you named your client `rancher`, look for the link named `rancher-dedicated`.
1. Click the **Mappers** tab.
1. Click **Configure a new mapper**. If you already have existing mappers configured, click the arrow next to **Add mapper** and select **By configuration**. Repeat this process and create these mappers:
- From the mappings table, select **Group Membership** and configure a new "Groups Mapper" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Groups Mapper`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `groups`
`Full group path` | `OFF`
`Add to ID token` | `OFF`
`Add to access token` | `OFF`
`Add to user info` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Groups Mapper` |
| `Mapper Type` | `Group Membership` |
| `Token Claim Name` | `groups` |
| `Full group path` | `OFF` |
| `Add to ID token` | `OFF` |
| `Add to access token` | `OFF` |
| `Add to user info` | `ON` |
- Create a new "Client Audience" with the settings below.
- From the mappings table, select **Audience** and configure a new "Client Audience" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Client Audience`
`Mapper Type` | `Audience`
`Included Client Audience` | &lt;CLIENT_NAME>
`Add to ID token` | `OFF`
`Add to access token` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Client Audience` |
| `Mapper Type` | `Audience` |
| `Included Client Audience` | &lt;client-name> |
| `Add to ID token` | `OFF` |
| `Add to access token` | `ON` |
- Create a new "Groups Path" with the settings below.
- From the mappings table, select **Group Membership** and configure a new "Groups Path" with the settings below. For settings that are not mentioned, use the default value.
Setting | Value
------------|------------
`Name` | `Group Path`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `full_group_path`
`Full group path` | `ON`
`Add to ID token` | `ON`
`Add to access token` | `ON`
`Add to user info` | `ON`
| Setting | Value |
| ------------|------------|
| `Name` | `Group Path` |
| `Mapper Type` | `Group Membership` |
| `Token Claim Name` | `full_group_path` |
| `Full group path` | `ON` |
| `Add to ID token` | `ON` |
| `Add to access token` | `ON` |
| `Add to user info` | `ON` |
- Go to **Role Mappings > Client Roles > realm-management** and add the following Role Mappings to all users or groups that need to query the Keycloak users.
- query-users
- query-groups
- view-users
- Add the following role mappings to all users or groups that need to query the Keycloak users.
<Tabs>
<TabItem value="Users">
1. In the navigation menu, click **Users**.
1. Click the user you want to add role mappings to.
1. Click the **Role mapping** tab.
1. Click **Assign role**.
1. Select the following roles:
- query-users
- query-groups
- view-users
1. Click **Assign**.
</TabItem>
<TabItem value="Groups">
1. In the navigation menu, click **Groups**.
1. Click the group you want to add role mappings to.
1. Click the **Role mapping** tab.
1. Click **Assign role**.
1. Select the following roles:
- query-users
- query-groups
- view-users
1. Click **Assign**.
</TabItem>
</Tabs>
## Configuring Keycloak in Rancher
@@ -73,6 +106,13 @@ If you have an existing configuration using the SAML protocol and want to switch
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (OIDC)**.
1. Complete the **Configure a Keycloak OIDC account** form. For help with filling the form, see the [configuration reference](#configuration-reference).
:::note
When configuring the **Endpoints** section using the **Generate** option, Rancher includes `/auth` as part of the context path in the **Issuer** and **Auth Endpoint** fields, which is only valid for Keycloak 16 or older. You must configure endpoints using the **Specify** option for [Keycloak 17](https://www.keycloak.org/docs/latest/release_notes/index.html#keycloak-17-0-0) and newer, which have [migrated to Quarkus](https://www.keycloak.org/migration/migrating-to-quarkus).
:::
1. After you complete the **Configure a Keycloak OIDC account** form, click **Enable**.
Rancher redirects you to the IdP login page. Enter credentials that authenticate with Keycloak IdP to validate your Rancher Keycloak configuration.
@@ -83,7 +123,7 @@ If you have an existing configuration using the SAML protocol and want to switch
:::
**Result:** Rancher is configured to work with Keycloak using the OIDC protocol. Your users can now sign into Rancher using their Keycloak logins.
**Result:** Rancher is configured to work with Keycloak using the OIDC protocol. Your users can now sign in to Rancher using their Keycloak logins.
## Configuration Reference
@@ -103,37 +143,19 @@ If you have an existing configuration using the SAML protocol and want to switch
This section describes the process to transition from using Rancher with Keycloak (SAML) to Keycloak (OIDC).
### Reconfigure Keycloak
1. Reconfigure Keycloak.
1. Configure a new `OpenID Connect` client according to the [Prerequisites](#prerequisites). Ensure the same `Valid Redirect URIs` are set.
1. Configure mappers for the new client according to the [Prerequisites](#prerequisites).
1. Before configuring Rancher to use Keycloak (OIDC), Keycloak (SAML) must be first disabled.
1. In the Rancher UI, click **☰ > Users & Authentication**.
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (SAML)**.
1. Click **Disable**.
1. Follow the steps in [Configuring Keycloak in Rancher](#configuring-keycloak-in-rancher).
1. Change the existing client to use the OIDC protocol. In the Keycloak console, select **Clients**, select the SAML client to migrate, select the **Settings** tab, change `Client Protocol` from `saml` to `openid-connect`, and click **Save**
:::caution
1. Verify the `Valid Redirect URIs` are still valid.
1. Select the **Mappers** tab and create a new Mapper with the settings below.
Setting | Value
------------|------------
`Name` | `Groups Mapper`
`Mapper Type` | `Group Membership`
`Token Claim Name` | `groups`
`Add to ID token` | `ON`
`Add to access token` | `ON`
`Add to user info` | `ON`
### Reconfigure Rancher
Before configuring Rancher to use Keycloak (OIDC), Keycloak (SAML) must be first disabled.
1. In the Rancher UI, click **☰ > Users & Authentication**.
1. In the left navigation bar, click **Auth Provider**.
1. Select **Keycloak (SAML)**.
1. Click **Disable**.
Configure Rancher to use Keycloak (OIDC) by following the steps in [this section](#configuring-keycloak-in-rancher).
:::note
After configuration is completed, Rancher user permissions will need to be reapplied as they are not automatically migrated.
After configuration is completed, Rancher user permissions need to be reapplied as they are not automatically migrated.
:::
@@ -141,21 +163,21 @@ After configuration is completed, Rancher user permissions will need to be reapp
If you are experiencing issues while testing the connection to the Keycloak server, first double-check the configuration options of your OIDC client. You may also inspect the Rancher logs to help pinpoint what's causing issues. Debug logs may contain more detailed information about the error. Please refer to [How can I enable debug logging](../../../../faq/technical-items.md#how-can-i-enable-debug-logging) in this documentation.
All Keycloak related log entries will be prepended with either `[generic oidc]` or `[keycloak oidc]`.
All Keycloak related log entries are prepended with either `[generic oidc]` or `[keycloak oidc]`.
### You are not redirected to Keycloak
When you fill the **Configure a Keycloak OIDC account** form and click on **Enable**, you are not redirected to your IdP.
When you fill the **Configure a Keycloak OIDC account** form and click **Enable**, you are not redirected to your IdP.
* Verify your Keycloak client configuration.
Verify your Keycloak client configuration.
### The generated `Issuer` and `Auth Endpoint` are incorrect
* On the **Configure a Keycloak OIDC account** form, change **Endpoints** to `Specify (advanced)` and override the `Issuer` and `Auth Endpoint` values. To find the values, go to the Keycloak console and select **Realm Settings**, select the **General** tab, and click **OpenID Endpoint Configuration**. The JSON output will display values for `issuer` and `authorization_endpoint`.
On the **Configure a Keycloak OIDC account** form, change **Endpoints** to `Specify (advanced)` and override the `Issuer` and `Auth Endpoint` values. To find the values, go to the Keycloak console and select **Realm Settings**, select the **General** tab, and click **OpenID Endpoint Configuration**. The JSON output displays values for `issuer` and `authorization_endpoint`.
### Keycloak Error: "Invalid grant_type"
* In some cases, this error message may be misleading and is actually caused by setting the `Valid Redirect URI` incorrectly.
In some cases, this error message may be misleading and is caused by setting the `Valid Redirect URI` incorrectly.
### Unable to See Groups When Assigning Global Roles
@@ -172,4 +194,3 @@ To resolve this, you can either:
3. Save your changes.
2. Reconfigure your Keycloak OIDC setup using a user that is assigned to at least one group in Keycloak.
versioned_docs/version-2.11/integrations-in-rancher/cluster-api/cluster-api.md
+1 -1
View File
@@ -9,6 +9,6 @@ title: Cluster API (CAPI) with Rancher Turtles
[Rancher Turtles](https://turtles.docs.rancher.com/) is a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/#operators-in-kubernetes) that manages the lifecycle of provisioned Kubernetes clusters, by providing integration between your Cluster API (CAPI) and Rancher. With Rancher Turtles, you can:
- Import CAPI clusters into Rancher, by installing the Rancher Cluster Agent in CAPI provisioned clusters.
- Configure the [CAPI Operator](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values#cluster-api-operator-values).
- Configure the [CAPI Operator](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html#cluster-api-operator-values).
The [Overview](./overview.md) section outlines installation options, Rancher Turtles architecture, and a brief demo. For more details, see the [Rancher Turtles documentation](https://turtles.docs.rancher.com/).
versioned_docs/version-2.11/integrations-in-rancher/cluster-api/overview.md
+5 -5
View File
@@ -16,7 +16,7 @@ Below is a visual representation of the key components of Rancher Turtles and th
As defined by [Supply-chain Levels for Software Artifacts (SLSA)](https://slsa.dev/spec/v1.0/about), SLSA is a set of incrementally adoptable guidelines for supply chain security, established by industry consensus. The specification set by SLSA is useful for both software producers and consumers: producers can follow SLSA’s guidelines to make their software supply chain more secure, and consumers can use SLSA to make decisions about whether to trust a software package.
Rancher Turtles meets [SLSA Level 3](https://slsa.dev/spec/v1.0/levels#build-l3) requirements as an appropriate hardened build platform, with consistent build processes, and provenance distribution. For more information, visit the [Rancher Turtles Security](https://turtles.docs.rancher.com/security/slsa) document.
Rancher Turtles meets [SLSA Level 3](https://slsa.dev/spec/v1.0/levels#build-l3) requirements as an appropriate hardened build platform, with consistent build processes, and provenance distribution. For more information, visit the [Rancher Turtles Security](https://turtles.docs.rancher.com/turtles/next/en/security/slsa.html) document.
## Prerequisites
@@ -92,7 +92,7 @@ By adding the Turtles repository via the Rancher UI, Rancher can process the ins
1. Click **Rancher Turtles - the Cluster API Extension**.
1. Click **Install > Next > Install**.
This process uses the default values for the Helm chart, which are good for most installations. If your configuration requires overriding some of these defaults, you can either specify the values during installation from the Rancher UI or you can [manually install the chart via Helm](#installing-via-helm). For details about available values, see the Rancher Turtles [Helm chart reference guide](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values).
This process uses the default values for the Helm chart, which are good for most installations. If your configuration requires overriding some of these defaults, you can either specify the values during installation from the Rancher UI or you can [manually install the chart via Helm](#installing-via-helm). For details about available values, see the Rancher Turtles [Helm chart reference guide](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html).
The installation may take a few minutes and after completing you can see the following new deployments in the cluster:
@@ -115,7 +115,7 @@ There are two ways to install Rancher Turtles with Helm, depending on whether yo
The CAPI Operator is required for installing Rancher Turtles. You can choose whether you want to take care of this dependency yourself or let the Rancher Turtles Helm chart manage it for you. [Installing Turtles as a dependency](#installing-rancher-turtles-with-cluster-api-capi-operator-as-a-helm-dependency) is simpler, but your best option depends on your specific configuration.
The CAPI Operator allows for handling the lifecycle of [CAPI providers](https://turtles.docs.rancher.com/tasks/capi-operator/installing_core_provider) using a declarative approach, extending the capabilities of `clusterctl`. If you want to learn more about it, you can refer to [Cluster API Operator book](https://cluster-api-operator.sigs.k8s.io/).
The CAPI Operator allows for handling the lifecycle of [CAPI providers](https://turtles.docs.rancher.com/turtles/next/en/tasks/capi-operator/installing_core_provider.html) using a declarative approach, extending the capabilities of `clusterctl`. If you want to learn more about it, you can refer to [Cluster API Operator book](https://cluster-api-operator.sigs.k8s.io/).
#### Installing Rancher Turtles with `Cluster API (CAPI) Operator` as a Helm dependency
@@ -177,7 +177,7 @@ stringData:
:::info
For detailed information on the values supported by the chart and their usage, refer to [Helm chart options](https://turtles.docs.rancher.com/reference-guides/rancher-turtles-chart/values)
For detailed information on the values supported by the chart and their usage, refer to [Helm chart options](https://turtles.docs.rancher.com/turtles/next/en/reference-guides/rancher-turtles-chart/values.html)
:::
@@ -185,7 +185,7 @@ For detailed information on the values supported by the chart and their usage, r
:::note
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/contributing/install_capi_operator) in the Rancher Turtles documentation for assistance.
Remember that if you opt for this installation option, you must manage the CAPI Operator installation yourself. You can follow the [CAPI Operator guide](https://turtles.docs.rancher.com/turtles/next/en/contributing/install_capi_operator.html) in the Rancher Turtles documentation for assistance.
:::
versioned_docs/version-2.11/integrations-in-rancher/integrations-in-rancher.md
+1 -1
View File
@@ -15,4 +15,4 @@ To learn more and get started with Rancher Prime, please visit [this page](https
import DocCardList from '@theme/DocCardList';
import { useCurrentSidebarCategory } from '@docusaurus/theme-common/internal';
<DocCardList items={useCurrentSidebarCategory().items.slice(0,9)} />
<DocCardList items={useCurrentSidebarCategory().items.slice(0,10)} />
versioned_docs/version-2.5/contribute-to-rancher.md
-140
View File
@@ -1,140 +0,0 @@
---
title: Contributing to Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/contribute-to-rancher"/>
</head>
Learn about the repositories used for Rancher and Rancher docs, how to build Rancher repositories, and what information to include when you file an issue.
For more detailed information on how to contribute to the development of Rancher projects, refer to the [Rancher Developer Wiki](https://github.com/rancher/rancher/wiki). The wiki has resources on many topics, including the following:
- How to set up the Rancher development environment and run tests
- The typical flow of an issue through the development lifecycle
- Coding guidelines and development best practices
- Debugging and troubleshooting
- Developing the Rancher API
On the Rancher Users Slack, the channel for developers is **#developer**.
## Rancher Docs
If you have suggestions for the documentation on this website, [open](https://github.com/rancher/rancher-docs/issues/new/choose) an issue in the main [Rancher docs](https://github.com/rancher/rancher-docs) repository. This repo contains documentation for Rancher v2.0 and later.
See the [Rancher docs README](https://github.com/rancher/rancher-docs#readme) for more details on contributing to and building the Rancher v2.x docs repo.
For documentation describing Rancher v1.6 and earlier, see the [Rancher 1.x docs](https://github.com/rancher/rancher.github.io) repo, which contains source files for https://rancher.com/docs/rancher/v1.6/en/.
## Rancher Repositories
All of repositories are located within our main GitHub organization. There are many repositories used for Rancher, but we'll provide descriptions of some of the main ones used in Rancher.
Repository | URL | Description
-----------|-----|-------------
Rancher | https://github.com/rancher/rancher | This repository is the main source code for Rancher 2.x.
Types | https://github.com/rancher/types | This repository is the repository that has all the API types for Rancher 2.x.
API Framework | https://github.com/rancher/norman | This repository is an API framework for building Rancher style APIs backed by Kubernetes Custom Resources.
User Interface | https://github.com/rancher/dashboard/ | This repository is the source of the Dashboard UI.
(Rancher) Docker Machine | https://github.com/rancher/machine | This repository is the source of the Docker Machine binary used when using Node Drivers. This is a fork of the `docker/machine` repository.
machine-package | https://github.com/rancher/machine-package | This repository is used to build the Rancher Docker Machine binary.
kontainer-engine | https://github.com/rancher/kontainer-engine | This repository is the source of kontainer-engine, the tool to provision hosted Kubernetes clusters.
RKE repository | https://github.com/rancher/rke | This repository is the source of Rancher Kubernetes Engine, the tool to provision Kubernetes clusters on any machine.
CLI | https://github.com/rancher/cli | This repository is the source code for the Rancher CLI used in Rancher 2.x.
(Rancher) Helm repository | https://github.com/rancher/helm | This repository is the source of the packaged Helm binary. This is a fork of the `helm/helm` repository.
Telemetry repository | https://github.com/rancher/telemetry | This repository is the source for the Telemetry binary.
loglevel repository | https://github.com/rancher/loglevel | This repository is the source of the loglevel binary, used to dynamically change log levels.
To see all libraries/projects used in Rancher, see the [`go.mod` file](https://github.com/rancher/rancher/blob/master/go.mod) in the `rancher/rancher` repository.
![Rancher diagram](/img/ranchercomponentsdiagram-2.6.svg)<br/>
<sup>Rancher components used for provisioning/managing Kubernetes clusters.</sup>
### Building Rancher Repositories
Every repository should have a Makefile and can be built using the `make` command. The `make` targets are based on the scripts in the `/scripts` directory in the repository, and each target will use [Dapper](https://github.com/rancher/dapper) to run the target in an isolated environment. The `Dockerfile.dapper` will be used for this process, and includes all the necessary build tooling needed.
The default target is `ci`, and will run `./scripts/validate`, `./scripts/build`, `./scripts/test` and `./scripts/package`. The resulting binaries of the build will be in `./build/bin` and are usually also packaged in a Docker image.
### Rancher Bugs, Issues or Questions
If you find any bugs or are having any trouble, please search the [reported issue](https://github.com/rancher/rancher/issues) as someone may have experienced the same issue or we are actively working on a solution.
If you can't find anything related to your issue, contact us by [filing an issue](https://github.com/rancher/rancher/issues/new). Though we have many repositories related to Rancher, we want the bugs filed in the Rancher repository so we won't miss them! If you want to ask a question or ask fellow users about an use case, we suggest creating a post on the [Rancher Forums](https://forums.rancher.com).
#### Checklist for Filing Issues
Please follow this checklist when filing an issue which will helps us investigate and fix the issue. More info means more data we can use to determine what is causing the issue or what might be related to the issue.
:::note
For large amounts of data, please use [GitHub Gist](https://gist.github.com/) or similar and link the created resource in the issue.
:::
:::note Important:
Please remove any sensitive data as it will be publicly viewable.
:::
- **Resources:** Provide as much as detail as possible on the used resources. As the source of the issue can be many things, including as much of detail as possible helps to determine the root cause. See some examples below:
- **Hosts:** What specifications does the host have, like CPU/memory/disk, what cloud does it happen on, what Amazon Machine Image are you using, what DigitalOcean droplet are you using, what image are you provisioning that we can rebuild or use when we try to reproduce
- **Operating System:** What operating system are you using? Providing specifics helps here like the output of `cat /etc/os-release` for exact OS release and `uname -r` for exact kernel used
- **Docker:** What Docker version are you using, how did you install it? Most of the details of Docker can be found by supplying output of `docker version` and `docker info`
- **Environment:** Are you in a proxy environment, are you using recognized CA/self signed certificates, are you using an external loadbalancer
- **Rancher:** What version of Rancher are you using, this can be found on the bottom left of the UI or be retrieved from the image tag you are running on the host
- **Clusters:** What kind of cluster did you create, how did you create it, what did you specify when you were creating it
- **Steps to reproduce the issue:** Provide as much detail on how you got into the reported situation. This helps the person to reproduce the situation you are in.
- Provide manual steps or automation scripts used to get from a newly created setup to the situation you reported.
- **Logs:** Provide data/logs from the used resources.
- Rancher
- Docker install
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher:|rancher/rancher " | awk '{ print $1 }')
```
- Kubernetes install using `kubectl`
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if Rancher is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
-l app=rancher \
--timestamps=true
```
- Docker install using `docker` on each of the nodes in the RKE cluster
```
docker logs \
--timestamps \
$(docker ps | grep -E "rancher/rancher@|rancher_rancher" | awk '{ print $1 }')
```
- Kubernetes Install with RKE Add-On
:::note
Make sure you configured the correct kubeconfig (for example, `export KUBECONFIG=$PWD/kube_config_cluster.yml` if the Rancher server is installed on a Kubernetes cluster) or are using the embedded kubectl via the UI.
:::
```
kubectl -n cattle-system \
logs \
--timestamps=true \
-f $(kubectl --kubeconfig $KUBECONFIG get pods -n cattle-system -o json | jq -r '.items[] | select(.spec.containers[].name="cattle-server") | .metadata.name')
```
- System logging (these might not all exist, depending on operating system)
- `/var/log/messages`
- `/var/log/syslog`
- `/var/log/kern.log`
- Docker daemon logging (these might not all exist, depending on operating system)
- `/var/log/docker.log`
- **Metrics:** If you are experiencing performance issues, please provide as much of data (files or screenshots) of metrics which can help determining what is going on. If you have an issue related to a machine, it helps to supply output of `top`, `free -m`, `df` which shows processes/memory/disk usage.
versioned_docs/version-2.5/explanations.md
-5
View File
@@ -1,5 +0,0 @@
---
title: Explanations
---
**Explanatory docs** are concerned primarily with providing theoretical knowledge for the "why" behind a task or a topic. Explanations are "understanding-oriented" in nature and will clarify a topic in order to broaden the user's knowledge. In this section, users can find additional context and background, alternatives or even opinions on topics, and often historical reasons, constraints, and insights into why a process works the way that it does.
versioned_docs/version-2.5/explanations/integrations-in-rancher/cis-scans/cis-scans.md
-180
View File
@@ -1,180 +0,0 @@
---
title: CIS Scans
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans"/>
</head>
Rancher can run a security scan to check whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark.
The `rancher-cis-benchmark` app leverages <a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench,</a> an open-source tool from Aqua Security, to check clusters for CIS Kubernetes Benchmark compliance. Also, to generate a cluster-wide report, the application utilizes <a href="https://github.com/vmware-tanzu/sonobuoy" target="_blank">Sonobuoy</a> for report aggregation.
## Changes in Rancher v2.5
We now support running CIS scans on any Kubernetes cluster, including hosted Kubernetes providers such as EKS, AKS, and GKE. Previously it was only supported to run CIS scans on RKE Kubernetes clusters.
In Rancher v2.4, the CIS scan tool was available from the **cluster manager** in the Rancher UI. Now it is available in the **Cluster Explorer** and it can be enabled and deployed using a Helm chart. It can be installed from the Rancher UI, but it can also be installed independently of Rancher. It deploys a CIS scan operator for the cluster, and deploys Kubernetes custom resources for cluster scans. The custom resources can be managed directly from the **Cluster Explorer.**
In v1 of the CIS scan tool, which was available in Rancher v2.4 through the cluster manager, recurring scans could be scheduled. The ability to schedule recurring scans is now also available for CIS v2 from Rancher v2.5.4.
Support for alerting for the cluster scan results is now also available from Rancher v2.5.4.
In Rancher v2.4, permissive and hardened profiles were included. In Rancher v2.5.0 and in v2.5.4, more profiles were included.
<Tabs>
<TabItem value="Profiles in v2.5.4">
- Generic CIS 1.5
- Generic CIS 1.6
- RKE permissive 1.5
- RKE hardened 1.5
- RKE permissive 1.6
- RKE hardened 1.6
- EKS
- GKE
- RKE2 permissive 1.5
- RKE2 permissive 1.5
</TabItem>
<TabItem value="Profiles in v2.5.0-v2.5.3">
- Generic CIS 1.5
- RKE permissive
- RKE hardened
- EKS
- GKE
</TabItem>
</Tabs>
<br/>
The default profile and the supported CIS benchmark version depends on the type of cluster that will be scanned and the Rancher version:
<Tabs>
<TabItem value="v2.5.4">
The `rancher-cis-benchmark` supports the CIS 1.6 Benchmark version.
- For RKE Kubernetes clusters, the RKE Permissive 1.6 profile is the default.
- EKS and GKE have their own CIS Benchmarks published by `kube-bench`. The corresponding test profiles are used by default for those clusters.
- For RKE2 Kubernetes clusters, the RKE2 Permissive 1.5 profile is the default.
- For cluster types other than RKE, RKE2, EKS and GKE, the Generic CIS 1.5 profile will be used by default.
</TabItem>
<TabItem value="v2.5.0-v2.5.3">
The `rancher-cis-benchmark` supports the CIS 1.5 Benchmark version.
- For RKE Kubernetes clusters, the RKE permissive profile is the default.
- EKS and GKE have their own CIS Benchmarks published by `kube-bench`. The corresponding test profiles are used by default for those clusters.
- For cluster types other than RKE, EKS and GKE, the Generic CIS 1.5 profile will be used by default.
</TabItem>
</Tabs>
> **Note:** CIS v1 cannot run on a cluster when CIS v2 is deployed. In other words, after `rancher-cis-benchmark` is installed, you can't run scans by going to the Cluster Manager view in the Rancher UI and clicking <b>Tools > CIS Scans.</b>
## About the CIS Benchmark
The Center for Internet Security is a 501(c\)(3) non-profit organization, formed in October 2000, with a mission to "identify, develop, validate, promote, and sustain best practice solutions for cyber defense and build and lead communities to enable an environment of trust in cyberspace". The organization is headquartered in East Greenbush, New York, with members including large corporations, government agencies, and academic institutions.
CIS Benchmarks are best practices for the secure configuration of a target system. CIS Benchmarks are developed through the generous volunteer efforts of subject matter experts, technology vendors, public and private community members, and the CIS Benchmark Development team.
The official Benchmark documents are available through the CIS website. The sign-up form to access the documents is
<a href="https://learn.cisecurity.org/benchmarks" target="_blank">here.</a>
## About the Generated Report
Each scan generates a report can be viewed in the Rancher UI and can be downloaded in CSV format.
From Rancher v2.5.4, the scan uses the CIS Benchmark v1.6 by default. In Rancher v2.5.0-2.5.3, the CIS Benchmark v1.5. is used.
The Benchmark version is included in the generated report.
The Benchmark provides recommendations of two types: Automated and Manual. Recommendations marked as Manual in the Benchmark are not included in the generated report.
Some tests are designated as "Not Applicable." These tests will not be run on any CIS scan because of the way that Rancher provisions RKE clusters. For information on how test results can be audited, and why some tests are designated to be not applicable, refer to Rancher's [self-assessment guide](../../../reference-guides/rancher-security/rancher-security.md#the-cis-benchmark-and-self-assessment) for the corresponding Kubernetes version.
The report contains the following information:
| Column in Report | Description |
|------------------|-------------|
| `id` | The ID number of the CIS Benchmark. |
| `description` | The description of the CIS Benchmark test. |
| `remediation` | What needs to be fixed in order to pass the test. |
| `state` | Indicates if the test passed, failed, was skipped, or was not applicable. |
| `node_type` | The node role, which affects which tests are run on the node. Master tests are run on controlplane nodes, etcd tests are run on etcd nodes, and node tests are run on the worker nodes. |
| `audit` | This is the audit check that `kube-bench` runs for this test. |
| `audit_config` | Any configuration applicable to the audit script. |
| `test_info` | Test-related info as reported by `kube-bench`, if any. |
| `commands` | Test-related commands as reported by `kube-bench`, if any. |
| `config_commands` | Test-related configuration data as reported by `kube-bench`, if any. |
| `actual_value` | The test's actual value, present if reported by `kube-bench`. |
| `expected_result` | The test's expected result, present if reported by `kube-bench`. |
Refer to the [table in the cluster hardening guide](../../../reference-guides/rancher-security/rancher-security.md) for information on which versions of Kubernetes, the Benchmark, Rancher, and our cluster hardening guide correspond to each other. Also refer to the hardening guide for configuration files of CIS-compliant clusters and information on remediating failed tests.
## Test Profiles
The following profiles are available:
<Tabs>
<TabItem value="Profiles in v2.5.4">
- Generic CIS 1.5
- Generic CIS 1.6
- RKE permissive 1.5
- RKE hardened 1.5
- RKE permissive 1.6
- RKE hardened 1.6
- EKS
- GKE
- RKE2 permissive 1.5
- RKE2 permissive 1.5
</TabItem>
<TabItem value="Profiles in v2.5.0-v2.5.3">
- Generic CIS 1.5
- RKE permissive
- RKE hardened
- EKS
- GKE
</TabItem>
</Tabs>
You also have the ability to customize a profile by saving a set of tests to skip.
All profiles will have a set of not applicable tests that will be skipped during the CIS scan. These tests are not applicable based on how a RKE cluster manages Kubernetes.
There are two types of RKE cluster scan profiles:
- **Permissive:** This profile has a set of tests that have been will be skipped as these tests will fail on a default RKE Kubernetes cluster. Besides the list of skipped tests, the profile will also not run the not applicable tests.
- **Hardened:** This profile will not skip any tests, except for the non-applicable tests.
The EKS and GKE cluster scan profiles are based on CIS Benchmark versions that are specific to those types of clusters.
In order to pass the "Hardened" profile, you will need to follow the steps on the [hardening guide](../../../reference-guides/rancher-security/rancher-security.md#rancher-hardening-guide) and use the `cluster.yml` defined in the hardening guide to provision a hardened cluster.
## About Skipped and Not Applicable Tests
For a list of skipped and not applicable tests, refer to [this page](skipped-and-not-applicable-tests.md).
For now, only user-defined skipped tests are marked as skipped in the generated report.
Any skipped tests that are defined as being skipped by one of the default profiles are marked as not applicable.
## Roles-based Access Control
For information about permissions, refer to [this page](rbac-for-cis-scans.md).
## Configuration
For more information about configuring the custom resources for the scans, profiles, and benchmark versions, refer to [this page](configuration-reference.md).
## How-to Guides
Please refer [here](../../../how-to-guides/advanced-user-guides/cis-scan-guides/cis-scan-guides.md) for how-to guides on CIS scans.
versioned_docs/version-2.5/explanations/integrations-in-rancher/cis-scans/configuration-reference.md
-97
View File
@@ -1,97 +0,0 @@
---
title: Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans/configuration-reference"/>
</head>
This configuration reference is intended to help you manage the custom resources created by the `rancher-cis-benchmark` application. These resources are used for performing CIS scans on a cluster, skipping tests, setting the test profile that will be used during a scan, and other customization.
To configure the custom resources, go to the **Cluster Explorer** in the Rancher UI. In dropdown menu in the top left corner, click **Cluster Explorer > CIS Benchmark.**
## Scans
A scan is created to trigger a CIS scan on the cluster based on the defined profile. A report is created after the scan is completed.
When configuring a scan, you need to define the name of the scan profile that will be used with the `scanProfileName` directive.
An example ClusterScan custom resource is below:
```yaml
apiVersion: cis.cattle.io/v1
kind: ClusterScan
metadata:
name: rke-cis
spec:
scanProfileName: rke-profile-hardened
```
## Profiles
A profile contains the configuration for the CIS scan, which includes the benchmark version to use and any specific tests to skip in that benchmark.
> By default, a few ClusterScanProfiles are installed as part of the `rancher-cis-benchmark` chart. If a user edits these default benchmarks or profiles, the next chart update will reset them back. So it is advisable for users to not edit the default ClusterScanProfiles.
Users can clone the ClusterScanProfiles to create custom profiles.
Skipped tests are listed under the `skipTests` directive.
When you create a new profile, you will also need to give it a name.
An example `ClusterScanProfile` is below:
```yaml
apiVersion: cis.cattle.io/v1
kind: ClusterScanProfile
metadata:
annotations:
meta.helm.sh/release-name: clusterscan-operator
meta.helm.sh/release-namespace: cis-operator-system
labels:
app.kubernetes.io/managed-by: Helm
name: "<example-profile>"
spec:
benchmarkVersion: cis-1.5
skipTests:
- "1.1.20"
- "1.1.21"
```
## Benchmark Versions
A benchmark version is the name of benchmark to run using `kube-bench`, as well as the valid configuration parameters for that benchmark.
A `ClusterScanBenchmark` defines the CIS `BenchmarkVersion` name and test configurations. The `BenchmarkVersion` name is a parameter provided to the `kube-bench` tool.
By default, a few `BenchmarkVersion` names and test configurations are packaged as part of the CIS scan application. When this feature is enabled, these default BenchmarkVersions will be automatically installed and available for users to create a ClusterScanProfile.
> If the default BenchmarkVersions are edited, the next chart update will reset them back. Therefore we don't recommend editing the default ClusterScanBenchmarks.
A ClusterScanBenchmark consists of the fields:
- `ClusterProvider`: This is the cluster provider name for which this benchmark is applicable. For example: RKE, EKS, GKE, etc. Leave it empty if this benchmark can be run on any cluster type.
- `MinKubernetesVersion`: Specifies the cluster's minimum kubernetes version necessary to run this benchmark. Leave it empty if there is no dependency on a particular Kubernetes version.
- `MaxKubernetesVersion`: Specifies the cluster's maximum Kubernetes version necessary to run this benchmark. Leave it empty if there is no dependency on a particular k8s version.
An example `ClusterScanBenchmark` is below:
```yaml
apiVersion: cis.cattle.io/v1
kind: ClusterScanBenchmark
metadata:
annotations:
meta.helm.sh/release-name: clusterscan-operator
meta.helm.sh/release-namespace: cis-operator-system
creationTimestamp: "2020-08-28T18:18:07Z"
generation: 1
labels:
app.kubernetes.io/managed-by: Helm
name: cis-1.5
resourceVersion: "203878"
selfLink: /apis/cis.cattle.io/v1/clusterscanbenchmarks/cis-1.5
uid: 309e543e-9102-4091-be91-08d7af7fb7a7
spec:
clusterProvider: ""
minKubernetesVersion: 1.15.0
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/cis-scans/custom-benchmark.md
-80
View File
@@ -1,80 +0,0 @@
---
title: Creating a Custom Benchmark Version for Running a Cluster Scan
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans/custom-benchmark"/>
</head>
_Available as of v2.5.4_
Each Benchmark Version defines a set of test configuration files that define the CIS tests to be run by the <a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench</a> tool.
The `rancher-cis-benchmark` application installs a few default Benchmark Versions which are listed under CIS Benchmark application menu.
But there could be some Kubernetes cluster setups that require custom configurations of the Benchmark tests. For example, the path to the Kubernetes config files or certs might be different than the standard location where the upstream CIS Benchmarks look for them.
It is now possible to create a custom Benchmark Version for running a cluster scan using the `rancher-cis-benchmark` application.
When a cluster scan is run, you need to select a Profile which points to a specific Benchmark Version.
Follow all the steps below to add a custom Benchmark Version and run a scan using it.
## 1. Prepare the Custom Benchmark Version ConfigMap
To create a custom benchmark version, first you need to create a ConfigMap containing the benchmark version's config files and upload it to your Kubernetes cluster where you want to run the scan.
To prepare a custom benchmark version ConfigMap, suppose we want to add a custom Benchmark Version named `foo`.
1. Create a directory named `foo` and inside this directory, place all the config YAML files that the <a href="https://github.com/aquasecurity/kube-bench" target="_blank">kube-bench</a> tool looks for. For example, here are the config YAML files for a Generic CIS 1.5 Benchmark Version https://github.com/aquasecurity/kube-bench/tree/master/cfg/cis-1.5
1. Place the complete `config.yaml` file, which includes all the components that should be tested.
1. Add the Benchmark version name to the `target_mapping` section of the `config.yaml`:
```yaml
target_mapping:
"foo":
- "master"
- "node"
- "controlplane"
- "etcd"
- "policies"
```
1. Upload this directory to your Kubernetes Cluster by creating a ConfigMap:
```yaml
kubectl create configmap -n <namespace> foo --from-file=<path to directory foo>
```
## 2. Add a Custom Benchmark Version to a Cluster
1. Once the ConfigMap has been created in your cluster, navigate to the **Cluster Explorer** in the Rancher UI.
1. In the top left dropdown menu, click **Cluster Explorer > CIS Benchmark.**
1. In the **Benchmark Versions** section, click **Create.**
1. Enter the **Name** and a description for your custom benchmark version.
1. Choose the cluster provider that your benchmark version applies to.
1. Choose the ConfigMap you have uploaded from the dropdown.
1. Add the minimum and maximum Kubernetes version limits applicable, if any.
1. Click **Create.**
## 3. Create a New Profile for the Custom Benchmark Version
To run a scan using your custom benchmark version, you need to add a new Profile pointing to this benchmark version.
1. Once the custom benchmark version has been created in your cluster, navigate to the **Cluster Explorer** in the Rancher UI.
1. In the top left dropdown menu, click **Cluster Explorer > CIS Benchmark.**
1. In the **Profiles** section, click **Create.**
1. Provide a **Name** and description. In this example, we name it `foo-profile`.
1. Choose the Benchmark Version `foo` from the dropdown.
1. Click **Create.**
## 4. Run a Scan Using the Custom Benchmark Version
Once the Profile pointing to your custom benchmark version `foo` has been created, you can create a new Scan to run the custom test configs in the Benchmark Version.
To run a scan,
1. Go to the **Cluster Explorer** in the Rancher UI. In the top left dropdown menu, click **Cluster Explorer > CIS Benchmark.**
1. In the **Scans** section, click **Create.**
1. Choose the new cluster scan profile `foo-profile`.
1. Click **Create.**
**Result:** A report is generated with the scan results. To see the results, click the name of the scan that appears.
versioned_docs/version-2.5/explanations/integrations-in-rancher/cis-scans/rbac-for-cis-scans.md
-52
View File
@@ -1,52 +0,0 @@
---
title: Roles-based Access Control
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans/rbac-for-cis-scans"/>
</head>
This section describes the permissions required to use the rancher-cis-benchmark App.
The rancher-cis-benchmark is a cluster-admin only feature by default.
However, the `rancher-cis-benchmark` chart installs these two default `ClusterRoles`:
- cis-admin
- cis-view
In Rancher, only cluster owners and global administrators have `cis-admin` access by default.
Note: If you were using the `cis-edit` role added in Rancher v2.5 setup, it has now been removed since
Rancher v2.5.2 because it essentially is same as `cis-admin`. If you happen to create any clusterrolebindings
for `cis-edit`, please update them to use `cis-admin` ClusterRole instead.
## Cluster-Admin Access
Rancher CIS Scans is a cluster-admin only feature by default.
This means only the Rancher global admins, and the cluster’s cluster-owner can:
- Install/Uninstall the rancher-cis-benchmark App
- See the navigation links for CIS Benchmark CRDs - ClusterScanBenchmarks, ClusterScanProfiles, ClusterScans
- List the default ClusterScanBenchmarks and ClusterScanProfiles
- Create/Edit/Delete new ClusterScanProfiles
- Create/Edit/Delete a new ClusterScan to run the CIS scan on the cluster
- View and Download the ClusterScanReport created after the ClusterScan is complete
## Summary of Default Permissions for Kubernetes Default Roles
The rancher-cis-benchmark creates three `ClusterRoles` and adds the CIS Benchmark CRD access to the following default K8s `ClusterRoles`:
| ClusterRole created by chart | Default K8s ClusterRole | Permissions given with Role
| ------------------------------| ---------------------------| ---------------------------|
| `cis-admin` | `admin`| Ability to CRUD clusterscanbenchmarks, clusterscanprofiles, clusterscans, clusterscanreports CR
| `cis-view` | `view `| Ability to List(R) clusterscanbenchmarks, clusterscanprofiles, clusterscans, clusterscanreports CR
By default only cluster-owner role will have ability to manage and use `rancher-cis-benchmark` feature.
The other Rancher roles (cluster-member, project-owner, project-member) do not have any default permissions to manage and use rancher-cis-benchmark resources.
But if a cluster-owner wants to delegate access to other users, they can do so by creating ClusterRoleBindings between these users and the above CIS ClusterRoles manually.
There is no automatic role aggregation supported for the `rancher-cis-benchmark` ClusterRoles.
versioned_docs/version-2.5/explanations/integrations-in-rancher/cis-scans/skipped-and-not-applicable-tests.md
-57
View File
@@ -1,57 +0,0 @@
---
title: Skipped and Not Applicable Tests
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/cis-scans/skipped-and-not-applicable-tests"/>
</head>
This section lists the tests that are skipped in the permissive test profile for RKE.
> All the tests that are skipped and not applicable on this page will be counted as Not Applicable in the v2.5 generated report. The skipped test count will only mention the user-defined skipped tests. This allows user-skipped tests to be distinguished from the tests that are skipped by default in the RKE permissive test profile.
## CIS Benchmark v1.5
### CIS Benchmark v1.5 Skipped Tests
| Number | Description | Reason for Skipping |
| ---------- | ------------- | --------- |
| 1.1.12 | Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) | A system service account is required for etcd data directory ownership. Refer to Rancher's hardening guide for more details on how to configure this ownership. |
| 1.2.6 | Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers. |
| 1.2.16 | Ensure that the admission control plugin PodSecurityPolicy is set (Automated) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 1.2.33 | Ensure that the --encryption-provider-config argument is set as appropriate (Manual) | Enabling encryption changes how data can be recovered as data is encrypted. |
| 1.2.34 | Ensure that encryption providers are appropriately configured (Manual) | Enabling encryption changes how data can be recovered as data is encrypted. |
| 4.2.6 | Ensure that the --protect-kernel-defaults argument is set to true (Automated) | System level configurations are required before provisioning the cluster in order for this argument to be set to true. |
| 4.2.10 | Ensure that the--tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) | When generating serving certificates, functionality could break in conjunction with hostname overrides which are required for certain cloud providers. |
| 5.1.5 | Ensure that default service accounts are not actively used. (Automated) | Kubernetes provides default service accounts to be used. |
| 5.2.2 | Minimize the admission of containers wishing to share the host process ID namespace (Automated) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.3 | Minimize the admission of containers wishing to share the host IPC namespace (Automated) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.4 | Minimize the admission of containers wishing to share the host network namespace (Automated) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.2.5 | Minimize the admission of containers with allowPrivilegeEscalation (Automated) | Enabling Pod Security Policy can cause applications to unexpectedly fail. |
| 5.3.2 | Ensure that all Namespaces have Network Policies defined (Automated) | Enabling Network Policies can prevent certain applications from communicating with each other. |
| 5.6.4 | The default namespace should not be used (Automated) | Kubernetes provides a default namespace. |
### CIS Benchmark v1.5 Not Applicable Tests
| Number | Description | Reason for being not applicable |
| ---------- | ------------- | --------- |
| 1.1.1 | Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver. All configuration is passed in as arguments at container run time. |
| 1.1.2 | Ensure that the API server pod specification file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for kube-apiserver. All configuration is passed in as arguments at container run time. |
| 1.1.3 | Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.4 | Ensure that the controller manager pod specification file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.5 | Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.6 | Ensure that the scheduler pod specification file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.7 | Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for etcd. All configuration is passed in as arguments at container run time. |
| 1.1.8 | Ensure that the etcd pod specification file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for etcd. All configuration is passed in as arguments at container run time. |
| 1.1.13 | Ensure that the admin.conf file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes. |
| 1.1.14 | Ensure that the admin.conf file ownership is set to root:root (Automated) | Clusters provisioned by RKE does not store the kubernetes default kubeconfig credentials file on the nodes. |
| 1.1.15 | Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.16 | Ensure that the scheduler.conf file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for scheduler. All configuration is passed in as arguments at container run time. |
| 1.1.17 | Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.1.18 | Ensure that the controller-manager.conf file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn't require or maintain a configuration file for controller-manager. All configuration is passed in as arguments at container run time. |
| 1.3.6 | Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) | Clusters provisioned by RKE handles certificate rotation directly through RKE. |
| 4.1.1 | Ensure that the kubelet service file permissions are set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service. All configuration is passed in as arguments at container run time. |
| 4.1.2 | Ensure that the kubelet service file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet service. All configuration is passed in as arguments at container run time. |
| 4.1.9 | Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Automated) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet. All configuration is passed in as arguments at container run time. |
| 4.1.10 | Ensure that the kubelet configuration file ownership is set to root:root (Automated) | Clusters provisioned by RKE doesn’t require or maintain a configuration file for the kubelet. All configuration is passed in as arguments at container run time. |
| 4.2.12 | Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) | Clusters provisioned by RKE handles certificate rotation directly through RKE. |
versioned_docs/version-2.5/explanations/integrations-in-rancher/fleet-gitops-at-scale/architecture.md
-12
View File
@@ -1,12 +0,0 @@
---
title: Fleet Architecture
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/fleet-gitops-at-scale/architecture"/>
</head>
Fleet can manage deployments from git of raw Kubernetes YAML, Helm charts, or Kustomize or any combination of the three. Regardless of the source, all resources are dynamically turned into Helm charts, and Helm is used as the engine to deploy everything in the cluster. This gives you a high degree of control, consistency, and auditability. Fleet focuses not only on the ability to scale, but to give one a high degree of control and visibility to exactly what is installed on the cluster.
![Architecture](/img/fleet-architecture.svg)
versioned_docs/version-2.5/explanations/integrations-in-rancher/fleet-gitops-at-scale/fleet-gitops-at-scale.md
-92
View File
@@ -1,92 +0,0 @@
---
title: Continuous Delivery with Fleet
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/deploy-apps-across-clusters/fleet"/>
</head>
_Available as of Rancher v2.5_
## What is Continuous Delivery with Fleet?
Continuous Delivery is Rancher's GitOps functionality, which is provided via integration with Fleet.
- *Cluster engine*: Fleet is a container management and deployment engine designed to offer users more control on the local cluster and constant monitoring through GitOps. Fleet focuses not only on the ability to scale, but it also gives users a high degree of control and visibility to monitor exactly what is installed on the cluster.
- *Deployment management*: Fleet can manage deployments from git of raw Kubernetes YAML, Helm charts, Kustomize, or any combination of the three. Regardless of the source, all resources are dynamically turned into Helm charts, and Helm is used as the engine to deploy all resources in the cluster. As a result, users can enjoy a high degree of control, consistency, and auditability of their clusters.
## Architecture
For information about how Fleet works, see [this page](architecture.md).
## Accessing Fleet in the Rancher UI
Fleet comes preinstalled in Rancher v2.5. Users can leverage continuous delivery to deploy their applications to the Kubernetes clusters in the git repository without any manual operation by following **gitops** practice. For additional information on Continuous Delivery and other Fleet troubleshooting tips, refer [here](https://fleet.rancher.io/troubleshooting).
Follow the steps below to access Continuous Delivery in the Rancher UI:
1. Click **Cluster Explorer** in the Rancher UI.
1. In the top left dropdown menu, click **Cluster Explorer > Continuous Delivery**.
1. Select your namespace at the top of the menu, noting the following:
- By default, **fleet-default** is selected which includes all downstream clusters that are registered through Rancher.
- You may switch to **fleet-local**, which only contains the **local** cluster, or you may create your own workspace to which you may assign and move clusters.
- You can then manage clusters by clicking on **Clusters** on the left navigation bar.
1. Click on **Gitrepos** on the left navigation bar to deploy the gitrepo into your clusters in the current workspace.
1. Select your [git repository](https://fleet.rancher.io/gitrepo-add) and [target clusters/cluster group](https://fleet.rancher.io/gitrepo-targets). You can also create the cluster group in the UI by clicking on **Cluster Groups** from the left navigation bar.
1. Once the gitrepo is deployed, you can monitor the application through the Rancher UI.
## Windows Support
_Available as of v2.5.6_
For details on support for clusters with Windows nodes, see [this page](windows-support.md).
## GitHub Repository
The Fleet Helm charts are available [here](https://github.com/rancher/fleet/releases/tag/v0.3.10).
## Using Fleet Behind a Proxy
_Available as of v2.5.8_
For details on using Fleet behind a proxy, see [this page](use-fleet-behind-a-proxy.md).
## Helm Chart Dependencies
In order for Helm charts with dependencies to deploy successfully, you must run a manual command (as listed below), as it is up to the user to fulfill the dependency list. If you do not do this and proceed to clone your repository and run `helm install`, your installation will fail because the dependencies will be missing.
The Helm chart in the git repository must include its dependencies in the charts subdirectory. You must either manually run `helm dependencies update $chart` or run `helm dependencies build $chart` locally, then commit the complete charts directory to your git repository. Note that you will update your commands with the applicable parameters
## Troubleshooting
- **Known Issue**: Fleet becomes inoperable after a restore using the [backup-restore-operator](../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md#1-install-the-rancher-backup-operator). We will update the community once a permanent solution is in place.
- **Temporary Workaround**:
1. Find the two service account tokens listed in the fleet-controller and the fleet-controller-bootstrap service accounts. These are under the fleet-system namespace of the local cluster.
1. Remove the non-existent token secret. Doing so allows for only one entry to be present for the service account token secret that actually exists.
1. Delete the fleet-controller Pod in the fleet-system namespace to reschedule.
1. After the service account token issue is resolved, you can force redeployment of the fleet-agents. In the Rancher UI, go to **☰ > Cluster Management**, click on **Clusters** page, then click **Force Update**.
1. If the fleet-agent bundles remain in a `Modified` state after Step 4, update the field `spec.forceSyncGeneration` for the fleet-agent bundle to force re-creation.
---
- **Known Issue**: clientSecretName and helmSecretName secrets for Fleet gitrepos are not included in the backup nor restore created by the [backup-restore-operator](../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md#1-install-the-rancher-backup-operator). We will update the community once a permanent solution is in place.
- **Temporary Workaround**: By default, user-defined secrets are not backed up in Fleet. It is necessary to recreate secrets if performing a disaster recovery restore or migration of Rancher into a fresh cluster. To modify resourceSet to include extra resources you want to backup, refer to docs [here](https://github.com/rancher/backup-restore-operator#user-flow).
## Documentation
The Fleet documentation is at https://fleet.rancher.io/.
versioned_docs/version-2.5/explanations/integrations-in-rancher/fleet-gitops-at-scale/use-fleet-behind-a-proxy.md
-60
View File
@@ -1,60 +0,0 @@
---
title: Using Fleet Behind a Proxy
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/fleet-gitops-at-scale/use-fleet-behind-a-proxy"/>
</head>
_Available as of v2.5.8_
In this section, you'll learn how to enable Fleet in a setup that has a Rancher server with a public IP a Kubernetes cluster that has no public IP, but is configured to use a proxy.
Rancher does not establish connections with registered downstream clusters. The Rancher agent deployed on the downstream cluster must be able to establish the connection with Rancher.
To set up Fleet to work behind a proxy, you will need to set the **Agent Environment Variables** for the downstream cluster. These are cluster-level configuration options.
Through the Rancher UI, you can configure these environment variables for any cluster type, including registered and custom clusters. The variables can be added while editing an existing cluster or while provisioning a new cluster.
For public downstream clusters, it is sufficient to [set the required environment variables in the Rancher UI.](#setting-environment-variables-in-the-rancher-ui)
For private nodes or private clusters, the environment variables need to be set on the nodes themselves. Then the environment variables are configured from the Rancher UI, typically when provisioning a custom cluster or when registering the private cluster. For an example of how to set the environment variables on Ubuntu node in a K3s Kubernetes cluster, see [this section.](#setting-environment-variables-on-private-nodes)
## Required Environment Variables
When adding Fleet agent environment variables for the proxy, replace <PROXY_IP> with your private proxy IP.
| Variable Name | Value |
|------------------|--------|
| `HTTP_PROXY` | http://<PROXY_IP>:8888 |
| `HTTPS_PROXY` | http://<PROXY_IP>:8888
| `NO_PROXY` | 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local |
## Setting Environment Variables in the Rancher UI
To add the environment variable to an existing cluster,
1. In the Rancher UI, go to the cluster view for Kubernetes cluster that needs to use a proxy.
1. Click **&#8942; > Edit**.
1. Click **Advanced Options.**
1. Click **Add Environment Variable.**
1. Enter the [required environment variables](#required-environment-variables)
1. Click **Save.**
**Result:** The Fleet agent works behind a proxy.
## Setting Environment Variables on Private Nodes
For private nodes and private clusters, the proxy environment variables need to be set on the nodes themselves, as well as configured from the Rancher UI.
This example shows how the environment variables would be set up on an Ubuntu node in a K3s Kubernetes cluster:
```
ssh -o ForwardAgent=yes ubuntu@<public_proxy_ip>
ssh <k3s_ip>
export proxy_private_ip=<private_proxy_ip>
export HTTP_PROXY=http://${proxy_private_ip}:8888
export HTTPS_PROXY=http://${proxy_private_ip}:8888
export NO_PROXY=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/fleet-gitops-at-scale/windows-support.md
-25
View File
@@ -1,25 +0,0 @@
---
title: Windows Support
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/fleet-gitops-at-scale/windows-support"/>
</head>
Prior to Rancher v2.5.6, the `agent` did not have native Windows manifests on downstream clusters with Windows nodes. This would result in a failing `agent` pod for the cluster.
If you are upgrading from an older version of Rancher to v2.5.6+, you can deploy a working `agent` with the following workflow *in the downstream cluster*:
1. Cordon all Windows nodes.
1. Apply the below toleration to the `agent` workload.
1. Uncordon all Windows nodes.
1. Delete all `agent` pods. New pods should be created with the new toleration.
1. Once the `agent` pods are running, and auto-update is enabled for Fleet, they should be updated to a Windows-compatible `agent` version.
```yaml
tolerations:
- effect: NoSchedule
key: cattle.io/os
operator: Equal
value: linux
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/integrations-in-rancher.md
-11
View File
@@ -1,11 +0,0 @@
---
title: Integrations in Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/cloud-marketplace"/>
</head>
Over time, Rancher has accrued several products and projects that have been integrated into the Rancher UI.
Examples of some of these integrations are [Continuous Delivery with Fleet](fleet-gitops-at-scale/fleet-gitops-at-scale.md) and [Monitoring and Alerting](monitoring-and-alerting/monitoring-and-alerting.md).
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/configuration-options/configuration-options.md
-43
View File
@@ -1,43 +0,0 @@
---
title: Configuration Options
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options"/>
</head>
### Egress Support
By default the Egress gateway is disabled, but can be enabled on install or upgrade through the values.yaml or via the [overlay file](#overlay-file).
### Enabling Automatic Sidecar Injection
Automatic sidecar injection is disabled by default. To enable this, set the `sidecarInjectorWebhook.enableNamespacesByDefault=true` in the values.yaml on install or upgrade. This automatically enables Istio sidecar injection into all new namespaces that are deployed.
### Overlay File
An Overlay File is designed to support extensive configuration of your Istio installation. It allows you to make changes to any values available in the [IstioOperator API](https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/). This will ensure you can customize the default installation to fit any scenario.
The Overlay File will add configuration on top of the default installation that is provided from the Istio chart installation. This means you do not need to redefine the components that already defined for installation.
For more information on Overlay Files, refer to the [Istio documentation.](https://istio.io/latest/docs/setup/install/istioctl/#configure-component-settings)
### Selectors and Scrape Configs
The Monitoring app sets `prometheus.prometheusSpec.ignoreNamespaceSelectors=false` which enables monitoring across all namespaces by default. This ensures you can view traffic, metrics and graphs for resources deployed in a namespace with `istio-injection=enabled` label.
If you would like to limit Prometheus to specific namespaces, set `prometheus.prometheusSpec.ignoreNamespaceSelectors=true`. Once you do this, you must perform some additional configuration to continue to monitor your resources.
For details, refer to [this section.](selectors-and-scrape-configurations.md)
### Enable Istio with Pod Security Policies
Refer to [this section.](pod-security-policies.md)
### Additional Steps for Installing Istio on an RKE2 Cluster
Refer to [this section.](install-istio-on-rke2-cluster.md)
### Additional Steps for Project Network Isolation
Refer to [this section.](project-network-isolation.md)
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/configuration-options/install-istio-on-rke2-cluster.md
-38
View File
@@ -1,38 +0,0 @@
---
title: Additional Steps for Installing Istio on an RKE2 Cluster
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/install-istio-on-rke2-cluster"/>
</head>
Through the **Cluster Explorer,** when installing or upgrading Istio through **Apps & Marketplace,**
1. Click **Components.**
1. Check the box next to **Enabled CNI.**
1. Add a custom overlay file specifying `cniBinDir` and `cniConfDir`. For more information on these options, refer to the [Istio documentation.](https://istio.io/latest/docs/setup/additional-setup/cni/#helm-chart-parameters) An example is below:
```yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
cni:
enabled: true
values:
cni:
image: rancher/istio-install-cni:1.7.3
excludeNamespaces:
- istio-system
- kube-system
logLevel: info
cniBinDir: /opt/cni/bin
cniConfDir: /etc/cni/net.d
```
1. After installing Istio, you'll notice the cni-node pods in the istio-system namespace in a CrashLoopBackoff error. Manually edit the `istio-cni-node` daemonset to include the following on the `install-cni` container:
```yaml
securityContext:
privileged: true
```
**Result:** Now you should be able to utilize Istio as desired, including sidecar injection and monitoring via Kiali.
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/configuration-options/pod-security-policies.md
-108
View File
@@ -1,108 +0,0 @@
---
title: Enable Istio with Pod Security Policies
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/pod-security-policies"/>
</head>
If you have restrictive Pod Security Policies enabled, then Istio may not be able to function correctly, because it needs certain permissions in order to install itself and manage pod infrastructure. In this section, we will configure a cluster with PSPs enabled for an Istio install, and also set up the Istio CNI plugin.
The Istio CNI plugin removes the need for each application pod to have a privileged `NET_ADMIN` container. For further information, see the [Istio CNI Plugin docs](https://istio.io/docs/setup/additional-setup/cni). Please note that the [Istio CNI Plugin is in alpha](https://istio.io/about/feature-stages/).
The steps differ based on the Rancher version.
<Tabs>
<TabItem value="v2.5.4+">
> **Prerequisites:**
>
> - The cluster must be an RKE Kubernetes cluster.
> - The cluster must have been created with a default PodSecurityPolicy.
>
> To enable pod security policy support when creating a Kubernetes cluster in the Rancher UI, go to <b>Advanced Options.</b> In the <b>Pod Security Policy Support</b> section, click <b>Enabled.</b> Then select a default pod security policy.
1. [Set the PodSecurityPolicy to unrestricted](#1-set-the-podsecuritypolicy-to-unrestricted)
2. [Enable the CNI](#2-enable-the-cni)
3. [Verify that the CNI is working.](#3-verify-that-the-cni-is-working)
### 1. Set the PodSecurityPolicy to unrestricted
An unrestricted PSP allows Istio to be installed.
Set the PSP to `unrestricted` in the project where is Istio is installed, or the project where you plan to install Istio.
1. From the cluster view of the **Cluster Manager,** select **Projects/Namespaces.**
1. Find the **Project: System** and select the **&#8942; > Edit**.
1. Change the Pod Security Policy option to be unrestricted, then click **Save.**
### 2. Enable the CNI
When installing or upgrading Istio through **Apps & Marketplace,**
1. Click **Components.**
2. Check the box next to **Enabled CNI.**
3. Finish installing or upgrading Istio.
The CNI can also be enabled by editing the `values.yaml`:
```
istio_cni.enabled: true
```
Istio should install successfully with the CNI enabled in the cluster.
### 3. Verify that the CNI is working
Verify that the CNI is working by deploying a [sample application](https://istio.io/latest/docs/examples/bookinfo/) or deploying one of your own applications.
</TabItem>
<TabItem value="v2.5.0-v2.5.3">
> **Prerequisites:**
>
> - The cluster must be an RKE Kubernetes cluster.
> - The cluster must have been created with a default PodSecurityPolicy.
>
> To enable pod security policy support when creating a Kubernetes cluster in the Rancher UI, go to <b>Advanced Options.</b> In the <b>Pod Security Policy Support</b> section, click <b>Enabled.</b> Then select a default pod security policy.
1. [Configure the System Project Policy to allow Istio install.](#1-configure-the-system-project-policy-to-allow-istio-install)
2. [Install the CNI plugin in the System project.](#2-install-the-cni-plugin-in-the-system-project)
3. [Install Istio.](#3-install-istio)
### 1. Configure the System Project Policy to allow Istio install
1. From the cluster view of the **Cluster Manager,** select **Projects/Namespaces.**
1. Find the **Project: System** and select the **&#8942; > Edit**.
1. Change the Pod Security Policy option to be unrestricted, then click Save.
### 2. Install the CNI Plugin in the System Project
1. From the main menu of the **Dashboard**, select **Projects/Namespaces**.
1. Select the **Project: System** project.
1. Choose **Tools > Catalogs** in the navigation bar.
1. Add a catalog with the following:
1. Name: istio-cni
1. Catalog URL: https://github.com/istio/cni
1. Branch: The branch that matches your current release, for example: `release-1.4`.
1. From the main menu select **Apps**
1. Click Launch and select istio-cni
1. Update the namespace to be "kube-system"
1. In the answers section, click "Edit as YAML" and paste in the following, then click launch:
```
---
logLevel: "info"
excludeNamespaces:
- "istio-system"
- "kube-system"
```
### 3. Install Istio
Follow the [primary instructions](../../../../how-to-guides/advanced-user-guides/istio-setup-guide/enable-istio-in-cluster.md), adding a custom answer: `istio_cni.enabled: true`.
After Istio has finished installing, the Apps page in System Projects should show both istio and `istio-cni` applications deployed successfully. Sidecar injection will now be functional.
</TabItem>
</Tabs>
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/configuration-options/project-network-isolation.md
-25
View File
@@ -1,25 +0,0 @@
---
title: Additional Steps for Project Network Isolation
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/project-network-isolation"/>
</head>
In clusters where:
- You are using the Canal network plugin with Rancher before v2.5.8, or you are using Rancher v2.5.8+ with an any RKE network plug-in that supports the enforcement of Kubernetes network policies, such as Canal or the Cisco ACI plugin
- The Project Network Isolation option is enabled
- You install the Istio Ingress module
The Istio Ingress Gateway pod won't be able to redirect ingress traffic to the workloads by default. This is because all the namespaces will be inaccessible from the namespace where Istio is installed. You have two options.
The first option is to add a new Network Policy in each of the namespaces where you intend to have ingress controlled by Istio. Your policy should include the following lines:
```
- podSelector:
matchLabels:
app: istio-ingressgateway
```
The second option is to move the `istio-system` namespace to the `system` project, which by default is excluded from the network isolation.
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/configuration-options/selectors-and-scrape-configurations.md
-124
View File
@@ -1,124 +0,0 @@
---
title: Selectors and Scrape Configs
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/configuration-options/selectors-and-scrape-configurations"/>
</head>
The Monitoring app sets `prometheus.prometheusSpec.ignoreNamespaceSelectors=false`, which enables monitoring across all namespaces by default.
This ensures you can view traffic, metrics and graphs for resources deployed in a namespace with `istio-injection=enabled` label.
If you would like to limit Prometheus to specific namespaces, set `prometheus.prometheusSpec.ignoreNamespaceSelectors=true`. Once you do this, you must perform some additional configuration to continue to monitor your resources.
### Limiting Monitoring to Specific Namespaces by Setting ignoreNamespaceSelectors to True
This limits monitoring to specific namespaces.
1. From the **Cluster Explorer**, navigate to **Installed Apps** if Monitoring is already installed, or **Charts** in **Apps & Marketplace**
1. If starting a new install, **Click** the **rancher-monitoring** chart, then in **Chart Options** click **Edit as Yaml**.
1. If updating an existing installation, click on **Upgrade**, then in **Chart Options** click **Edit as Yaml**.
1. Set`prometheus.prometheusSpec.ignoreNamespaceSelectors=true`
1. Complete install or upgrade
**Result:** Prometheus will be limited to specific namespaces which means one of the following configurations will need to be set up to continue to view data in various dashboards
### Enabling Prometheus to Detect Resources in Other Namespaces
There are two different ways to enable Prometheus to detect resources in other namespaces when `prometheus.prometheusSpec.ignoreNamespaceSelectors=true`:
- **Monitoring specific namespaces:** Add a Service Monitor or Pod Monitor in the namespace with the targets you want to scrape.
- **Monitoring across namespaces:** Add an `additionalScrapeConfig` to your rancher-monitoring instance to scrape all targets in all namespaces.
### Monitoring Specific Namespaces: Create a Service Monitor or Pod Monitor
This option allows you to define which specific services or pods you would like monitored in a specific namespace.
The usability tradeoff is that you have to create the service monitor or pod monitor per namespace since you cannot monitor across namespaces.
> **Prerequisite:** Define a ServiceMonitor or PodMonitor for `<your namespace>`. An example ServiceMonitor is provided below.
1. From the **Cluster Explorer**, open the kubectl shell
1. Run `kubectl create -f <name of service/pod monitor file>.yaml` if the file is stored locally in your cluster.
1. Or run `cat<< EOF | kubectl apply -f -`, paste the file contents into the terminal, then run `EOF` to complete the command.
1. If starting a new install, **Click** the **rancher-monitoring** chart and scroll down to **Preview Yaml**.
1. Run `kubectl label namespace <your namespace> istio-injection=enabled` to enable the envoy sidecar injection
**Result:** `<your namespace>` can be scraped by prometheus.
<figcaption>Example Service Monitor for Istio Proxies</figcaption>
```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: envoy-stats-monitor
namespace: istio-system
labels:
monitoring: istio-proxies
spec:
selector:
matchExpressions:
- {key: istio-prometheus-ignore, operator: DoesNotExist}
namespaceSelector:
any: true
jobLabel: envoy-stats
endpoints:
- path: /stats/prometheus
targetPort: 15090
interval: 15s
relabelings:
- sourceLabels: [__meta_kubernetes_pod_container_port_name]
action: keep
regex: '.*-envoy-prom'
- action: labeldrop
regex: "__meta_kubernetes_pod_label_(.+)"
- sourceLabels: [__meta_kubernetes_namespace]
action: replace
targetLabel: namespace
- sourceLabels: [__meta_kubernetes_pod_name]
action: replace
targetLabel: pod_name
```
### Monitoring across namespaces: Set ignoreNamespaceSelectors to False
This enables monitoring across namespaces by giving Prometheus additional scrape configurations.
The usability tradeoff is that all of Prometheus' `additionalScrapeConfigs` are maintained in a single Secret. This could make upgrading difficult if monitoring is already deployed with additionalScrapeConfigs before installing Istio.
1. If starting a new install, **Click** the **rancher-monitoring** chart, then in **Chart Options** click **Edit as Yaml**.
1. If updating an existing installation, click on **Upgrade**, then in **Chart Options** click **Edit as Yaml**.
1. If updating an existing installation, click on **Upgrade** and then **Preview Yaml**.
1. Set`prometheus.prometheusSpec.additionalScrapeConfigs` array to the **Additional Scrape Config** provided below.
1. Complete install or upgrade
**Result:** All namespaces with the `istio-injection=enabled` label will be scraped by prometheus.
<figcaption>Additional Scrape Config</figcaption>
``` yaml
- job_name: 'istio/envoy-stats'
scrape_interval: 15s
metrics_path: /stats/prometheus
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_container_port_name]
action: keep
regex: '.*-envoy-prom'
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:15090
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: pod_name
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/cpu-and-memory-allocations.md
-76
View File
@@ -1,76 +0,0 @@
---
title: CPU and Memory Allocations
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/cpu-and-memory-allocations"/>
</head>
This section describes the minimum recommended computing resources for the Istio components in a cluster.
The CPU and memory allocations for each component are [configurable.](#configuring-resource-allocations)
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough CPU and memory to run all of the components of Istio.
> **Tip:** In larger deployments, it is strongly advised that the infrastructure be placed on dedicated nodes in the cluster by adding a node selector for each Istio component.
The table below shows a summary of the minimum recommended resource requests and limits for the CPU and memory of each core Istio component.
In Kubernetes, the resource request indicates that the workload will not be deployed on a node unless the node has at least the specified amount of memory and CPU available. If the workload surpasses the limit for CPU or memory, it can be terminated or evicted from the node. For more information on managing resource limits for containers, refer to the [Kubernetes documentation.](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)
<Tabs>
<TabItem value="v2.5.6+">
| Workload | CPU - Request | Memory - Request | CPU - Limit | Memory - Limit |
|----------------------|---------------|------------|-----------------|-------------------|
| ingress gateway | 100m | 128mi | 2000m | 1024mi |
| egress gateway | 100m | 128mi | 2000m | 1024mi |
| istiod | 500m | 2048mi | No limit | No limit |
| proxy | 10m | 10mi | 2000m | 1024mi |
| **Totals:** | **710m** | **2314Mi** | **6000m** | **3072Mi** |
</TabItem>
<TabItem value="v2.5.0-v2.5.5">
Workload | CPU - Request | Memory - Request | CPU - Limit | Mem - Limit | Configurable
---------:|---------------:|---------------:|-------------:|-------------:|-------------:
Istiod | 500m | 2048Mi | No limit | No limit | Y |
Istio-Mixer | 1000m | 1000Mi | 4800m | 4000Mi | Y |
Istio-ingressgateway | 100m | 128Mi | 2000m | 1024Mi | Y |
Others | 10m | - | - | - | Y |
Totals: | 1710m | 3304Mi | >8800m | >6048Mi | -
</TabItem>
</Tabs>
## Configuring Resource Allocations
You can individually configure the resource allocation for each type of Istio component. This section includes the default resource allocations for each component.
To make it easier to schedule the workloads to a node, a cluster-admin can reduce the CPU and memory resource requests for the component. However, the default CPU and memory allocations are the minimum that we recommend.
You can find more information about Istio configuration in the [official Istio documentation](https://istio.io/).
To configure the resources allocated to an Istio component,
1. In the Rancher **Cluster Explorer**, navigate to your Istio installation in **Apps & Marketplace**
1. Click **Upgrade** to edit the base components via changes to the values.yaml or add an [overlay file](configuration-options/configuration-options.md#overlay-file). For more information about editing the overlay file, see [this section.](cpu-and-memory-allocations.md#editing-the-overlay-file)
1. Change the CPU or memory allocations, the nodes where each component will be scheduled to, or the node tolerations.
1. Click **Upgrade.** to rollout changes
**Result:** The resource allocations for the Istio components are updated.
### Editing the Overlay File
The overlay file can contain any of the values in the [Istio Operator spec.](https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioOperatorSpec) The overlay file included with the Istio application is just one example of a potential configuration of the overlay file.
As long as the file contains `kind: IstioOperator` and the YAML options are valid, the file can be used as an overlay.
In the example overlay file provided with the Istio application, the following section allows you to change Kubernetes resources:
```
# k8s:
# resources:
# requests:
# cpu: 200m
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/disable-istio.md
-40
View File
@@ -1,40 +0,0 @@
---
title: Disabling Istio
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/disable-istio"/>
</head>
This section describes how to uninstall Istio in a cluster or disable a namespace, or workload.
## Uninstall Istio in a Cluster
To uninstall Istio,
1. From the **Cluster Explorer,** navigate to **Installed Apps** in **Apps & Marketplace** and locate the `rancher-istio` installation.
1. Select `rancher-istio` in the `istio-system namespace and click **Delete**
1. After `rancher-istio` is deleted, you can then select all the remaining apps in the `istio-system` namespace and click **Delete**
**Result:** The `rancher-istio` app in the cluster gets removed. The Istio sidecar cannot be deployed on any workloads in the cluster.
**Note:** You can no longer disable and re-enable your Istio installation. If you would like to save your settings for a future install, view and save individual YAMLs to refer back to / reuse for future installations.
**Troubleshooting Uninstall:** If you didn't follow the uninstall steps, you may encounter a warning during uninstall:
`Error: uninstallation completed with 1 error(s): unable to build kubernetes objects for delete: unable to recognize "": no matches for kind "MonitoringDashboard" in version "monitoring.kiali.io/v1alpha1"`
This could mean a few things. You either selected all the apps in the `istio-system` namespace and deleted them at the same time, or you deleted `rancher-istio` chart dependencies prior to deleting the `rancher-istio` chart. Since the uninstall did not complete properly, you will have resources remaining in the `istio-system` namespace that you will need to manually clean up. Another option to avoid manual clean up is to install `rancher-istio` again, then uninstall it in the correct order.
## Disable Istio in a Namespace
1. From the **Cluster Explorer** view, use the side-nav to select **Namespaces** page
1. On the **Namespace** page, you will see a list of namespaces. Go to the namespace where you want to disable and click the select **Edit as Form** or **Edit as Yaml**
1. Remove the `istio-injection=enabled` label from the namespace
1. Click **Save**
**Result:** When workloads are deployed in this namespace, they will not have the Istio sidecar.
## Remove the Istio Sidecar from a Workload
Disable Istio in the namespace, then redeploy the workloads with in it. They will be deployed without the Istio sidecar.
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/istio.md
-118
View File
@@ -1,118 +0,0 @@
---
title: Istio
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio"/>
</head>
[Istio](https://istio.io/) is an open-source tool that makes it easier for DevOps teams to observe, secure, control, and troubleshoot the traffic within a complex network of microservices.
As a network of microservices changes and grows, the interactions between them can become increasingly difficult to manage and understand. In such a situation, it is useful to have a service mesh as a separate infrastructure layer. Istio's service mesh lets you manipulate traffic between microservices without changing the microservices directly.
Our integration of Istio is designed so that a Rancher operator, such as an administrator or cluster owner, can deliver Istio to a team of developers. Then developers can use Istio to enforce security policies, troubleshoot problems, or manage traffic for green/blue deployments, canary deployments, or A/B testing.
This core service mesh provides features that include but are not limited to the following:
- **Traffic Management** such as ingress and egress routing, circuit breaking, mirroring.
- **Security** with resources to authenticate and authorize traffic and users, mTLS included.
- **Observability** of logs, metrics, and distributed traffic flows.
After [setting up istio](../../../how-to-guides/advanced-user-guides/istio-setup-guide/istio-setup-guide.md) you can leverage Istio's control plane functionality through the Cluster Explorer, `kubectl`, or `istioctl`.
Istio needs to be set up by a `cluster-admin` before it can be used in a project.
## What's New in Rancher v2.5
The overall architecture of Istio has been simplified. A single component, Istiod, has been created by combining Pilot, Citadel, Galley and the sidecar injector. Node Agent functionality has also been merged into istio-agent.
Addons that were previously installed by Istio (cert-manager, Grafana, Jaeger, Kiali, Prometheus, Zipkin) will now need to be installed separately. Istio will support installation of integrations that are from the Istio Project and will maintain compatibility with those that are not.
A Prometheus integration will still be available through an installation of [Rancher Monitoring](../monitoring-and-alerting/monitoring-and-alerting.md), or by installing your own Prometheus operator. Rancher's Istio chart will also install Kiali by default to ensure you can get a full picture of your microservices out of the box.
Istio has migrated away from Helm as a way to install Istio and now provides installation through the istioctl binary or Istio Operator. To ensure the easiest interaction with Istio, Rancher's Istio will maintain a Helm chart that utilizes the istioctl binary to manage your Istio installation.
This Helm chart will be available via the Apps and Marketplace in the UI. A user that has access to the Rancher Chart's catalog will need to set up Istio before it can be used in the project.
## Tools Bundled with Istio
Our [Istio](https://istio.io/) installer wraps the istioctl binary commands in a handy Helm chart, including an overlay file option to allow complex customization.
It also includes the following:
### Kiali
Kiali is a comprehensive visualization aid used for graphing traffic flow throughout the service mesh. It allows you to see how they are connected, including the traffic rates and latencies between them.
You can check the health of the service mesh, or drill down to see the incoming and outgoing requests to a single component.
### Jaeger
_Bundled as of v2.5.4_
Our Istio installer includes a quick-start, all-in-one installation of [Jaeger,](https://www.jaegertracing.io/) a tool used for tracing distributed systems.
Note that this is not a production-qualified deployment of Jaeger. This deployment uses an in-memory storage component, while a persistent storage component is recommended for production. For more information on which deployment strategy you may need, refer to the [Jaeger documentation.](https://www.jaegertracing.io/docs/latest/operator/#production-strategy)
## Prerequisites
Before enabling Istio, we recommend that you confirm that your Rancher worker nodes have enough [CPU and memory](cpu-and-memory-allocations.md) to run all of the components of Istio.
If you are installing Istio on RKE2 cluster, some additional steps are required. For details, see [this section.](#additional-steps-for-installing-istio-on-an-rke2-cluster)
## Setup Guide
Refer to the [setup guide](../../../how-to-guides/advanced-user-guides/istio-setup-guide/istio-setup-guide.md) for instructions on how to set up Istio and use it in a project.
## Remove Istio
To remove Istio components from a cluster, namespace, or workload, refer to the section on [uninstalling Istio.](disable-istio.md)
## Migrate From Previous Istio Version
There is no upgrade path for Istio versions less than 1.7.x. To successfully install Istio in the **Cluster Explorer**, you will need to disable your existing Istio in the **Cluster Manager**.
If you have a significant amount of additional Istio CRDs you might consider manually migrating CRDs that are supported in both versions of Istio. You can do this by running `kubectl get <resource> -n istio-system -o yaml`, save the output yaml and re-apply in the new version.
Another option is to manually uninstall istio resources one at a time, but leave the resources that are supported in both versions of Istio and that will not be installed by the newest version. This method is more likely to result in issues installing the new version, but could be a good option depending on your situation.
## Accessing Visualizations
> By default, only cluster-admins have access to Kiali. For instructions on how to allow admin, edit or views roles to access them, see [this section.](rbac-for-istio.md)
After Istio is set up in a cluster, Grafana, Prometheus,and Kiali are available in the Rancher UI.
To access the Grafana and Prometheus visualizations, from the **Cluster Explorer** navigate to the **Monitoring** app overview page, and click on **Grafana** or **Prometheus**
To access the Kiali visualization, from the **Cluster Explorer** navigate to the **Istio** app overview page, and click on **Kiali**. From here you can access the **Traffic Graph** tab or the **Traffic Metrics** tab to see network visualizations and metrics.
By default, all namespace will picked up by prometheus and make data available for Kiali graphs. Refer to [selector/scrape config setup](configuration-options/selectors-and-scrape-configurations.md) if you would like to use a different configuration for prometheus data scraping.
Your access to the visualizations depend on your role. Grafana and Prometheus are only available for `cluster-admin` roles. The Kiali UI is available only to `cluster-admin` by default, but `cluster-admin` can allow other roles to access them by editing the Istio values.yaml.
## Architecture
Istio installs a service mesh that uses [Envoy](https://www.envoyproxy.io) sidecar proxies to intercept traffic to each workload. These sidecars intercept and manage service-to-service communication, allowing fine-grained observation and control over traffic within the cluster.
Only workloads that have the Istio sidecar injected can be tracked and controlled by Istio.
When a namespace has Istio enabled, new workloads deployed in the namespace will automatically have the Istio sidecar. You need to manually enable Istio in preexisting workloads.
For more information on the Istio sidecar, refer to the [Istio sidecare-injection docs](https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/) and for more information on Istio's architecture, refer to the [Istio Architecture docs](https://istio.io/latest/docs/ops/deployment/architecture/)
### Multiple Ingresses
By default, each Rancher-provisioned cluster has one NGINX ingress controller allowing traffic into the cluster. Istio also installs an ingress gateway by default into the `istio-system` namespace. The result is that your cluster will have two ingresses in your cluster.
![In an Istio-enabled cluster, you can have two ingresses: the default Nginx ingress, and the default Istio controller.](/img/istio-ingress.svg)
Additional Istio Ingress gateways can be enabled via the [overlay file](configuration-options/configuration-options.md#overlay-file).
### Egress Support
By default the Egress gateway is disabled, but can be enabled on install or upgrade through the values.yaml or via the [overlay file](configuration-options/configuration-options.md#overlay-file).
## Additional Steps for Installing Istio on an RKE2 Cluster
To install Istio on an RKE2 cluster, follow the steps in [this section.](configuration-options/install-istio-on-rke2-cluster.md)
versioned_docs/version-2.5/explanations/integrations-in-rancher/istio/rbac-for-istio.md
-47
View File
@@ -1,47 +0,0 @@
---
title: Role-based Access Control
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/istio/rbac-for-istio"/>
</head>
This section describes the permissions required to access Istio features.
The rancher istio chart installs three `ClusterRoles`
## Cluster-Admin Access
By default, only those with the `cluster-admin` `ClusterRole` can:
- Install istio app in a cluster
- Configure resource allocations for Istio
## Admin and Edit access
By default, only Admin and Edit roles can:
- Enable and disable Istio sidecar auto-injection for namespaces
- Add the Istio sidecar to workloads
- View the traffic metrics and traffic graph for the cluster
- Configure Istio's resources (such as the gateway, destination rules, or virtual services)
## Summary of Default Permissions for Kubernetes Default roles
Istio creates three `ClusterRoles` and adds Istio CRD access to the following default K8s `ClusterRole`:
ClusterRole create by chart | Default K8s ClusterRole | Rancher Role |
------------------------------:| ---------------------------:|---------:|
`istio-admin` | admin| Project Owner |
`istio-edit`| edit | Project Member |
`istio-view` | view | Read-only |
Rancher will continue to use cluster-owner, cluster-member, project-owner, project-member, etc as role names, but will utilize default roles to determine access. For each default K8s `ClusterRole` there are different Istio CRD permissions and K8s actions (Create ( C ), Get ( G ), List ( L ), Watch ( W ), Update ( U ), Patch ( P ), Delete( D ), All ( * )) that can be performed.
|CRDs | Admin | Edit | View
|----------------------------| ------| -----| -----
| <ul><li>`config.istio.io`</li><ul><li>`adapters`</li><li>`attributemanifests`</li><li>`handlers`</li><li>`httpapispecbindings`</li><li>`httpapispecs`</li><li>`instances`</li><li>`quotaspecbindings`</li><li>`quotaspecs`</li><li>`rules`</li><li>`templates`</li></ul></ul>| GLW | GLW | GLW
|<ul><li>`networking.istio.io`</li><ul><li>`destinationrules`</li><li>`envoyfilters`</li><li>`gateways`</li><li>`serviceentries`</li><li>`sidecars`</li><li>`virtualservices`</li><li>`workloadentries`</li></ul></ul>| * | * | GLW
|<ul><li>`security.istio.io`</li><ul><li>`authorizationpolicies`</li><li>`peerauthentications`</li><li>`requestauthentications`</li></ul></ul>| * | * | GLW
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/custom-resource-configuration/custom-resource-configuration.md
-12
View File
@@ -1,12 +0,0 @@
---
title: Custom Resource Configuration
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/custom-resource-configuration"/>
</head>
The following Custom Resource Definitions are used to configure logging:
- [Flow and ClusterFlow](flows-and-clusterflows.md)
- [Output and ClusterOutput](outputs-and-clusteroutputs.md)
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/custom-resource-configuration/flows-and-clusterflows.md
-151
View File
@@ -1,151 +0,0 @@
---
title: Flows and ClusterFlows
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/custom-resource-configuration/flows-and-clusterflows"/>
</head>
See the [Logging operator documentation](https://kube-logging.github.io/docs/configuration/flow/) for the full details on how to configure `Flows` and `ClusterFlows`.
See [Rancher Integration with Logging Services: Troubleshooting](../logging.md#The-Logging-Buffer-Overloads-Pods) for how to resolve memory problems with the logging buffer.
## Configuration
## Changes in v2.5.8
The `Flows` and `ClusterFlows` can now be configured by filling out forms in the Rancher UI.
## Flows
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
A `Flow` defines which logs to collect and filter and which `Output` to send the logs to. The `Flow` is a namespaced resource, which means logs will only be collected from the namespace that the `Flow` is deployed in.
For more details about the `Flow` custom resource, see [FlowSpec.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/flow_types/)
</TabItem>
<TabItem value="Rancher before v2.5.8">
A `Flow` defines which logs to collect and filter and which `Output` to send the logs to. The `Flow` is a namespaced resource, which means logs will only be collected from the namespace that the `Flow` is deployed in.
`Flows` need to be defined in YAML.
For more details about the `Flow` custom resource, see [FlowSpec.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/flow_types/)
</TabItem>
</Tabs>
### Matches
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
Match statements are used to select which containers to pull logs from.
You can specify match statements to select or exclude logs according to Kubernetes labels, container and host names. Match statements are evaluated in the order they are defined and processed only until the first matching select or exclude rule applies.
Matches can be configured by filling out the `Flow` or `ClusterFlow` forms in the Rancher UI.
For detailed examples on using the match statement, see the [official documentation on log routing.](https://kube-logging.github.io/docs/configuration/log-routing/)
</TabItem>
<TabItem value="Rancher before v2.5.8">
Match statements are used to select which containers to pull logs from.
You can specify match statements to select or exclude logs according to Kubernetes labels, container and host names. Match statements are evaluated in the order they are defined and processed only until the first matching select or exclude rule applies.
For detailed examples on using the match statement, see the [official documentation on log routing.](https://kube-logging.github.io/docs/configuration/log-routing/)
</TabItem>
</Tabs>
### Filters
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
You can define one or more filters within a `Flow`. Filters can perform various actions on the logs, such as adding data, transforming the logs, or parsing values from the records. The filters in the `Flow` are applied in the same order they appear in the definition.
For a list of filters supported by the Logging operator, see [the official documentation on Fluentd filters](https://kube-logging.github.io/docs/configuration/plugins/filters/).
Filters need to be configured in YAML.
</TabItem>
<TabItem value="Rancher before v2.5.8">
### Filters
You can define one or more filters within a `Flow`. Filters can perform various actions on the logs, such as adding data, transforming the logs, or parsing values from the records. The filters in the `Flow` are applied in the same order they appear in the definition.
For a list of filters supported by the Logging operator, see [the official documentation on Fluentd filters](https://kube-logging.github.io/docs/configuration/plugins/filters/).
</TabItem>
</Tabs>
### Outputs
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
This `Output` will receive logs from the `Flow`. Because the `Flow` is a namespaced resource, the `Output` must reside in same namespace as the `Flow`.
`Outputs` can be referenced when filling out the `Flow` or `ClusterFlow` forms in the Rancher UI.
</TabItem>
<TabItem value="Rancher before v2.5.8">
This `Output` will receive logs from the `Flow`. Because the `Flow` is a namespaced resource, the `Output` must reside in same namespace as the `Flow`.
</TabItem>
</Tabs>
## ClusterFlows
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
Matches, filters and `Outputs` are configured for `ClusterFlows` in the same way that they are configured for `Flows`. The key difference is that the `ClusterFlow` is scoped at the cluster level and can configure log collection across all namespaces.
After `ClusterFlow` selects logs from all namespaces in the cluster, logs from the cluster will be collected and logged to the selected `ClusterOutput`.
</TabItem>
<TabItem value="Rancher before v2.5.8">
Matches, filters and `Outputs` are also configured for `ClusterFlows`. The only difference is that the `ClusterFlow` is scoped at the cluster level and can configure log collection across all namespaces.
`ClusterFlow` selects logs from all namespaces in the cluster. Logs from the cluster will be collected and logged to the selected `ClusterOutput`.
`ClusterFlows` need to be defined in YAML.
</TabItem>
</Tabs>
## YAML Example
The following example `Flow` transforms the log messages from the default namespace and sends them to an S3 `Output`:
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: flow-sample
namespace: default
spec:
filters:
- parser:
remove_key_name_field: true
parse:
type: nginx
- tag_normaliser:
format: ${namespace_name}.${pod_name}.${container_name}
localOutputRefs:
- s3-output
match:
- select:
labels:
app: nginx
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/custom-resource-configuration/outputs-and-clusteroutputs.md
-323
View File
@@ -1,323 +0,0 @@
---
title: Outputs and ClusterOutputs
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/custom-resource-configuration/outputs-and-clusteroutputs"/>
</head>
See the [Logging operator documentation](https://kube-logging.github.io/docs/configuration/flow/) for the full details on how to configure `Flows` and `ClusterFlows`.
See [Rancher Integration with Logging Services: Troubleshooting](../logging.md#The-Logging-Buffer-Overloads-Pods) for how to resolve memory problems with the logging buffer.
## Outputs
As of Rancher v2.5.8, the `Outputs` and `ClusterOutputs` can now be configured by filling out forms in the Rancher UI.
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
The `Output` resource defines where your `Flows` can send the log messages. `Outputs` are the final stage for a logging `Flow`.
The `Output` is a namespaced resource, which means only a `Flow` within the same namespace can access it.
You can use secrets in these definitions, but they must also be in the same namespace.
For the details of `Output` custom resource, see [OutputSpec.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/output_types/).
The Rancher UI provides forms for configuring the following `Output` types:
- Amazon ElasticSearch
- Azure Storage
- Cloudwatch
- Datadog
- Elasticsearch
- File
- Fluentd
- GCS
- Kafka
- Kinesis Stream
- LogDNA
- LogZ
- Loki
- New Relic
- Splunk
- SumoLogic
- Syslog
The Rancher UI provides forms for configuring the `Output` type, target, and access credentials if applicable.
For example configuration for each logging plugin supported by the logging operator, see the [Logging operator documentation](https://kube-logging.github.io/docs/configuration/plugins/outputs/).
</TabItem>
<TabItem value="Rancher before v2.5.8">
The `Output` resource defines where your `Flows` can send the log messages. `Outputs` are the final stage for a logging `Flow`.
The `Output` is a namespaced resource, which means only a `Flow` within the same namespace can access it.
You can use secrets in these definitions, but they must also be in the same namespace.
`Outputs` are configured in YAML. For the details of `Output` custom resource, see [OutputSpec.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/output_types/)
For examples of configuration for each logging plugin supported by the logging operator, see the [logging operator documentation.](https://kube-logging.github.io/docs/configuration/plugins/outputs/)
</TabItem>
</Tabs>
## ClusterOutputs
<Tabs groupId="rancher-version">
<TabItem value="Rancher v2.5.8+">
`ClusterOutput` defines an `Output` without namespace restrictions. It is only effective when deployed in the same namespace as the logging operator.
For the details of the `ClusterOutput` custom resource, see [ClusterOutput.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/clusteroutput_types/)
</TabItem>
<TabItem value="Rancher before v2.5.8">
`ClusterOutput` defines an `Output` without namespace restrictions. It is only effective when deployed in the same namespace as the logging operator.
The Rancher UI provides forms for configuring the `ClusterOutput` type, target, and access credentials if applicable.
`ClusterOutputs` are configured in YAML. For the details of `ClusterOutput` custom resource, see [ClusterOutput.](https://kube-logging.github.io/docs/configuration/crds/v1beta1/clusteroutput_types/)
For example configuration for each logging plugin supported by the logging operator, see the [logging operator documentation.](https://kube-logging.github.io/docs/configuration/plugins/outputs/)
</TabItem>
</Tabs>
## YAML Examples
Once logging is installed, you can use these examples to help craft your own logging pipeline.
### Cluster Output to ElasticSearch
Let's say you wanted to send all logs in your cluster to an `elasticsearch` cluster. First, we create a cluster `Output`.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: "example-es"
namespace: "cattle-logging-system"
spec:
elasticsearch:
host: elasticsearch.example.com
port: 9200
scheme: http
```
We have created this `ClusterOutput`, without elasticsearch configuration, in the same namespace as our operator: `cattle-logging-system.`. Any time we create a `ClusterFlow` or `ClusterOutput`, we have to put it in the `cattle-logging-system` namespace.
Now that we have configured where we want the logs to go, let's configure all logs to go to that `ClusterOutput`.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: "cattle-logging-system"
spec:
globalOutputRefs:
- "example-es"
```
We should now see our configured index with logs in it.
### Output to Splunk
What if we have an application team who only wants logs from a specific namespaces sent to a `splunk` server? For this case, we can use namespaced `Outputs` and `Flows`.
Before we start, let's set up that team's application: `coolapp`.
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: devteam
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: coolapp
namespace: devteam
labels:
app: coolapp
spec:
replicas: 2
selector:
matchLabels:
app: coolapp
template:
metadata:
labels:
app: coolapp
spec:
containers:
- name: generator
image: paynejacob/loggenerator:latest
```
With `coolapp` running, we will follow a similar path as when we created a `ClusterOutput`. However, unlike `ClusterOutputs`, we create our `Output` in our application's namespace.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: "devteam-splunk"
namespace: "devteam"
spec:
splunkHec:
hec_host: splunk.example.com
hec_port: 8088
protocol: http
```
Once again, let's feed our `Output` some logs:
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: "devteam-logs"
namespace: "devteam"
spec:
localOutputRefs:
- "devteam-splunk"
```
### Output to Syslog
Let's say you wanted to send all logs in your cluster to an `syslog` server. First, we create a `ClusterOutput`:
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: "example-syslog"
namespace: "cattle-logging-system"
spec:
syslog:
buffer:
timekey: 30s
timekey_use_utc: true
timekey_wait: 10s
flush_interval: 5s
format:
type: json
app_name_field: test
host: syslog.example.com
insecure: true
port: 514
transport: tcp
```
Now that we have configured where we want the logs to go, let's configure all logs to go to that `Output`.
```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: "all-logs"
namespace: cattle-logging-system
spec:
globalOutputRefs:
- "example-syslog"
```
### Unsupported Outputs
For the final example, we create an `Output` to write logs to a destination that is not supported out of the box:
> **Note on syslog** As of Rancher v2.5.4, `syslog` is a supported `Output`. However, this example still provides an overview on using unsupported plugins.
```yaml
apiVersion: v1
kind: Secret
metadata:
name: syslog-config
namespace: cattle-logging-system
type: Opaque
stringData:
fluent-bit.conf: |
[INPUT]
Name forward
Port 24224
[OUTPUT]
Name syslog
InstanceName syslog-output
Match *
Addr syslog.example.com
Port 514
Cluster ranchers
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: fluentbit-syslog-forwarder
namespace: cattle-logging-system
labels:
output: syslog
spec:
selector:
matchLabels:
output: syslog
template:
metadata:
labels:
output: syslog
spec:
containers:
- name: fluentbit
image: paynejacob/fluent-bit-out-syslog:latest
ports:
- containerPort: 24224
volumeMounts:
- mountPath: "/fluent-bit/etc/"
name: configuration
volumes:
- name: configuration
secret:
secretName: syslog-config
---
apiVersion: v1
kind: Service
metadata:
name: syslog-forwarder
namespace: cattle-logging-system
spec:
selector:
output: syslog
ports:
- protocol: TCP
port: 24224
targetPort: 24224
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: all-logs
namespace: cattle-logging-system
spec:
globalOutputRefs:
- syslog
---
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: syslog
namespace: cattle-logging-system
spec:
forward:
servers:
- host: "syslog-forwarder.cattle-logging-system"
require_ack_response: false
ignore_network_errors_at_startup: false
```
Let's break down what is happening here. First, we create a deployment of a container that has the additional `syslog` plugin and accepts logs forwarded from another `fluentd`. Next we create an `Output` configured as a forwarder to our deployment. The deployment `fluentd` will then forward all logs to the configured `syslog` destination.
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/logging-architecture.md
-43
View File
@@ -1,43 +0,0 @@
---
title: Logging Architecture
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/logging-architecture"/>
</head>
This section summarizes the architecture of the Rancher logging application.
For more details about how the Logging operator works, see the [official documentation.](https://kube-logging.github.io/docs/#architecture)
### Changes in Rancher v2.5
The following changes were introduced to logging in Rancher v2.5:
- The [Logging operator](https://kube-logging.github.io/docs/) now powers Rancher's logging solution in place of the former, in-house solution.
- [Fluent Bit](https://fluentbit.io/) is now used to aggregate the logs, and [Fluentd](https://www.fluentd.org/) is used for filtering the messages and routing them to the `Outputs`. Previously, only Fluentd was used.
- Logging can be configured with a Kubernetes manifest, because logging now uses a Kubernetes operator with Custom Resource Definitions.
- We now support filtering logs.
- We now support writing logs to multiple `Outputs`.
- We now always collect Control Plane and etcd logs.
## How the Logging Operator Works
The Logging operator automates the deployment and configuration of a Kubernetes logging pipeline. It deploys and configures a Fluent Bit DaemonSet on every node to collect container and application logs from the node file system.
Fluent Bit queries the Kubernetes API and enriches the logs with metadata about the pods, and transfers both the logs and the metadata to Fluentd. Fluentd receives, filters, and transfers logs to multiple `Outputs`.
The following custom resources are used to define how logs are filtered and sent to their `Outputs`:
- A `Flow` is a namespaced custom resource that uses filters and selectors to route log messages to the appropriate `Outputs`.
- A `ClusterFlow` is used to route cluster-level log messages.
- An `Output` is a namespaced resource that defines where the log messages are sent.
- A `ClusterOutput` defines an `Output` that is available from all `Flows` and `ClusterFlows`.
Each `Flow` must reference an `Output`, and each `ClusterFlow` must reference a `ClusterOutput`.
The following figure from the [Logging operator documentation](https://kube-logging.github.io/docs/#architecture) shows the new logging architecture:
<figcaption>How the Logging Operator Works with Fluentd and Fluent Bit</figcaption>
![How the Logging Operator Works with Fluentd](/img/banzai-cloud-logging-operator.png)
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/logging-helm-chart-options.md
-71
View File
@@ -1,71 +0,0 @@
---
title: rancher-logging Helm Chart Options
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/logging-helm-chart-options"/>
</head>
## Enable/Disable Windows Node Logging
_Available as of v2.5.8_
You can enable or disable Windows node logging by setting `global.cattle.windows.enabled` to either `true` or `false` in the `values.yaml`.
By default, Windows node logging will be enabled if the Cluster Explorer UI is used to install the logging application on a Windows cluster.
In this scenario, setting `global.cattle.windows.enabled` to `false` will disable Windows node logging on the cluster.
When disabled, logs will still be collected from Linux nodes within the Windows cluster.
> Note: Currently an [issue](https://github.com/rancher/rancher/issues/32325) exists where Windows nodeAgents are not deleted when performing a `helm upgrade` after disabling Windows logging in a Windows cluster. In this scenario, users may need to manually remove the Windows nodeAgents if they are already installed.
## Working with a Custom Docker Root Directory
_Applies to v2.5.6+_
If using a custom Docker root directory, you can set `global.dockerRootDirectory` in `values.yaml`.
This will ensure that the Logging CRs created will use your specified path rather than the default Docker `data-root` location.
Note that this only affects Linux nodes.
If there are any Windows nodes in the cluster, the change will not be applicable to those nodes.
## Adding NodeSelector Settings and Tolerations for Custom Taints
You can add your own `nodeSelector` settings and add `tolerations` for additional taints by editing the logging Helm chart values. For details, see [this page.](taints-and-tolerations.md)
## Enabling the Logging Application to Work with SELinux
_Available as of v2.5.8_
> **Requirements:** Logging v2 was tested with SELinux on RHEL/CentOS 7 and 8.
[Security-Enhanced Linux (SELinux)](https://en.wikipedia.org/wiki/Security-Enhanced_Linux) is a security enhancement to Linux. After being historically used by government agencies, SELinux is now industry standard and is enabled by default on CentOS 7 and 8.
To use Logging v2 with SELinux, we recommend installing the `rancher-selinux` RPM according to these [instructions.](../../../reference-guides/rancher-security/selinux-rpm/selinux-rpm.md)
Then, when installing the logging application, configure the chart to be SELinux aware by changing `global.seLinux.enabled` to `true` in the `values.yaml`.
## Additional Logging Sources
By default, Rancher collects logs for [control plane components](https://kubernetes.io/docs/concepts/overview/components/#control-plane-components) and [node components](https://kubernetes.io/docs/concepts/overview/components/#node-components) for all cluster types.
In some cases, Rancher may be able to collect additional logs.
The following table summarizes the sources where additional logs may be collected for each node types:
| Logging Source | Linux Nodes (including in Windows cluster) | Windows Nodes |
| --- | --- | ---|
| RKE | ✓ | ✓ |
| RKE2 | ✓ | |
| K3s | ✓ | |
| AKS | ✓ | |
| EKS | ✓ | |
| GKE | ✓ | |
To enable hosted Kubernetes providers as additional logging sources, go to **Cluster Explorer > Logging > Chart Options** and select the **Enable enhanced cloud provider logging** option.
When enabled, Rancher collects all additional node and control plane logs the provider has made available, which may vary between providers
If you're already using a cloud provider's own logging solution such as AWS CloudWatch or Google Cloud operations suite (formerly Stackdriver), it is not necessary to enable this option as the native solution will have unrestricted access to all logs.
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/logging.md
-139
View File
@@ -1,139 +0,0 @@
---
title: Rancher Integration with Logging Services
description: Rancher integrates with popular logging services. Learn the requirements and benefits of integrating with logging services, and enable logging on your cluster.
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging"/>
</head>
The [Logging operator](https://kube-logging.github.io/docs/) now powers Rancher's logging solution in place of the former, in-house solution.
For an overview of the changes in v2.5, see [this section.](logging-architecture.md#changes-in-rancher-v25) For information about migrating from Logging V1, see [this page.](migrate-to-rancher-v2.5+-logging.md)
## Enabling Logging
You can enable the logging for a Rancher managed cluster by going to the Apps page and installing the logging app.
1. In the Rancher UI, go to the cluster where you want to install logging and click **Cluster Explorer**.
1. Click **Apps**.
1. Click the `rancher-logging` app.
1. Scroll to the bottom of the Helm chart README and click **Install**.
**Result:** The logging app is deployed in the `cattle-logging-system` namespace.
## Uninstall Logging
1. From the **Cluster Explorer**, click **Apps & Marketplace**.
1. Click **Installed Apps**.
1. Go to the `cattle-logging-system` namespace and check the boxes for `rancher-logging` and `rancher-logging-crd`.
1. Click **Delete**.
1. Confirm **Delete**.
**Result** `rancher-logging` is uninstalled.
## Architecture
For more information about how the logging application works, see [this section.](logging-architecture.md)
## Role-based Access Control
Rancher logging has two roles, `logging-admin` and `logging-view`. For more information on how and when to use these roles, see [this page.](rbac-for-logging.md)
## Configuring Logging Custom Resources
To manage `Flows,` `ClusterFlows`, `Outputs`, and `ClusterOutputs`, go to the **Cluster Explorer** in the Rancher UI. In the upper left corner, click **Cluster Explorer > Logging**.
### Flows and ClusterFlows
For help with configuring `Flows` and `ClusterFlows`, see [this page.](custom-resource-configuration/flows-and-clusterflows.md)
### Outputs and ClusterOutputs
For help with configuring `Outputs` and `ClusterOutputs`, see [this page.](custom-resource-configuration/outputs-and-clusteroutputs.md)
## Configuring the Logging Helm Chart
For a list of options that can be configured when the logging application is installed or upgraded, see [this page.](logging-helm-chart-options.md)
### Windows Support
<Tabs>
<TabItem value="Rancher v2.5.8+">
As of Rancher v2.5.8, logging support for Windows clusters has been added and logs can be collected from Windows nodes.
For details on how to enable or disable Windows node logging, see [this section.](logging-helm-chart-options.md#enabledisable-windows-node-logging)
</TabItem>
<TabItem value="Rancher before v2.5.8">
Clusters with Windows workers support exporting logs from Linux nodes, but Windows node logs are currently unable to be exported.
Only Linux node logs are able to be exported.
To allow the logging pods to be scheduled on Linux nodes, tolerations must be added to the pods. Refer to the [Working with Taints and Tolerations](taints-and-tolerations.md) section for details and an example.
</TabItem>
</Tabs>
### Working with a Custom Docker Root Directory
For details on using a custom Docker root directory, see [this section.](logging-helm-chart-options.md#working-with-a-custom-docker-root-directory)
### Working with Taints and Tolerations
For information on how to use taints and tolerations with the logging application, see [this page.](taints-and-tolerations.md)
### Logging V2 with SELinux
_Available as of v2.5.8_
For information on enabling the logging application for SELinux-enabled nodes, see [this section.](logging-helm-chart-options.md#enabling-the-logging-application-to-work-with-selinux)
### Additional Logging Sources
By default, Rancher collects logs for control plane components and node components for all cluster types. In some cases additional logs can be collected. For details, see [this section.](logging-helm-chart-options.md#additional-logging-sources)
## Troubleshooting
### The Logging Buffer Overloads Pods
Depending on your configuration, the default buffer size may be too large and cause pod failures. One way to reduce the load is to lower the logger's flush interval. This prevents logs from overfilling the buffer. You can also add more flush threads to handle moments when many logs are attempting to fill the buffer at once.
For a more complete description of how to configure the logging buffer to suit your organization's needs, see the official Logging operator documentation on [buffers](https://kube-logging.github.io/docs/configuration/plugins/outputs/buffer/) and on [Fluentd configuration](https://kube-logging.github.io/docs/logging-infrastructure/fluentd/).
### The `cattle-logging` Namespace Being Recreated
If your cluster previously deployed logging from the Cluster Manager UI, you may encounter an issue where its `cattle-logging` namespace is continually being recreated.
The solution is to delete all `clusterloggings.management.cattle.io` and `projectloggings.management.cattle.io` custom resources from the cluster specific namespace in the management cluster.
The existence of these custom resources causes Rancher to create the `cattle-logging` namespace in the downstream cluster if it does not exist.
The cluster namespace matches the cluster ID, so we need to find the cluster ID for each cluster.
1. In your web browser, navigate to your cluster(s) in either the Cluster Manager UI or the Cluster Explorer UI.
2. Copy the `<cluster-id>` portion from one of the URLs below. The `<cluster-id>` portion is the cluster namespace name.
```bash
# Cluster Management UI
https://<your-url>/c/<cluster-id>/
# Cluster Explorer UI (Dashboard)
https://<your-url>/dashboard/c/<cluster-id>/
```
Now that we have the `<cluster-id>` namespace, we can delete the CRs that cause `cattle-logging` to be continually recreated.
*Warning:* ensure that logging, the version installed from the Cluster Manager UI, is not currently in use.
```bash
kubectl delete clusterloggings.management.cattle.io -n <cluster-id>
kubectl delete projectloggings.management.cattle.io -n <cluster-id>
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/migrate-to-rancher-v2.5+-logging.md
-180
View File
@@ -1,180 +0,0 @@
---
title: Migrating to Rancher v2.5 Logging
---
Starting in v2.5, the logging feature available within Rancher has been completely overhauled. The [Logging operator](https://github.com/kube-logging/logging-operator) has been adopted; Rancher configures this tooling for use when deploying logging.
Among the many features and changes in the new logging functionality is the removal of project-specific logging configurations. Instead, one now configures logging at the namespace level. Cluster-level logging remains available, but configuration options differ.
> Note: The pre-v2.5 user interface is now referred to as the _Cluster Manager_. The v2.5+ dashboard is referred to as the _Cluster Explorer_.
## Installation
To install logging in Rancher v2.5+, refer to the [installation instructions](logging.md#enabling-logging).
### Terminology
In v2.5, logging configuration is centralized under a _Logging_ menu option available in the _Cluster Explorer_. It is from this menu option that logging for both cluster and namespace is configured.
> Note: Logging is installed on a per-cluster basis. You will need to navigate between clusters to configure logging for each cluster.
There are four key concepts to understand for v2.5+ logging:
1. Outputs
`Outputs` are a configuration resource that determine a destination for collected logs. This is where settings for aggregators such as ElasticSearch, Kafka, etc. are stored. `Outputs` are namespaced resources.
2. Flows
`Flows` are a configuration resource that determine collection, filtering, and destination rules for logs. It is within a flow that one will configure what logs to collect, how to mutate or filter them, and which `Outputs` to send the logs to. `Flows` are namespaced resources, and can connect either to an `Output` in the same namespace, or a `ClusterOutput`.
3. ClusterOutputs
`ClusterOutputs` serve the same functionality as `Outputs`, except they are a cluster-scoped resource. `ClusterOutputs` are necessary when collecting logs cluster-wide, or if you wish to provide an `Output` to all namespaces in your cluster.
4. ClusterFlows
`ClusterFlows` serve the same function as `Flows`, but at the cluster level. They are used to configure log collection for an entire cluster, instead of on a per-namespace level. `ClusterFlows` are also where mutations and filters are defined, same as `Flows` (in functionality).
## Cluster Logging
To configure cluster-wide logging for v2.5+ logging, one needs to set up a `ClusterFlow`. This object defines the source of logs, any transformations or filters to be applied, and finally the `Output` (or `Outputs`) for the logs.
> Important: `ClusterFlows` must be defined within the `cattle-logging-system` namespace. `ClusterFlows` will not work if defined in any other namespace.
In legacy logging, in order to collect logs from across the entire cluster, one only needed to enable cluster-level logging and define the desired `Output`. This basic approach remains in v2.5+ logging. To replicate legacy cluster-level logging, follow these steps:
1. Define a `ClusterOutput` according to the instructions found under [Output Configuration](#output-configuration)
2. Create a `ClusterFlow`, ensuring that it is set to be created in the `cattle-logging-system` namespace
1. Remove all _Include_ and _Exclude_ rules from the `Flow` definition. This ensures that all logs are gathered.
2. You do not need to configure any filters if you do not wish - default behavior does not require their creation
3. Define your cluster `Output` or `Outputs`
This will result in logs from all sources in the cluster (all pods, and all system components) being collected and sent to the `Output` or `Outputs` you defined in the `ClusterFlow`.
## Project Logging
Logging in v2.5+ is not project-aware. This means that in order to collect logs from pods running in project namespaces, you will need to define `Flows` for those namespaces.
To collect logs from a specific namespace, follow these steps:
1. Define an `Output` or `ClusterOutput` according to the instructions found under [Output Configuration](#output-configuration)
2. Create a `Flow`, ensuring that it is set to be created in the namespace in which you want to gather logs.
1. If you wish to define _Include_ or _Exclude_ rules, you may do so. Otherwise, removal of all rules will result in all pods in the target namespace having their logs collected.
2. You do not need to configure any filters if you do not wish - default behavior does not require their creation
3. Define your outputs - these can be either `ClusterOutput` or `Output` objects.
This will result in logs from all sources in the namespace (pods) being collected and sent to the `Output` (or `Outputs`) you defined in your `Flow`.
> To collect logs from a project, repeat the above steps for every namespace within the project. Alternatively, you can label your project workloads with a common label (e.g. `project=my-project`) and use a `ClusterFlow` to collect logs from all pods matching this label.
## Output Configuration
In legacy logging, there are five logging destinations to choose from: Elasticsearch, Splunk, Kafka, Fluentd, and Syslog. With the exception of Syslog, all of these destinations are available in logging v2.5+.
### Elasticsearch
| Legacy Logging | v2.5+ Logging | Notes |
|-----------------------------------------------|-----------------------------------|-----------------------------------------------------------|
| Endpoint | Target -> Host | Make sure to specify Scheme (https/http), as well as Port |
| X-Pack Security -> Username | Access -> User | |
| X-Pack Security -> Password | Access -> Password | Password must now be stored in a secret |
| SSL Configuration -> Client Private Key | SSL -> Client Key | Key must now be stored in a secret |
| SSL Configuration -> Client Certificate | SSL -> Client Cert | Certificate must now be stored in a secret |
| SSL Configuration -> Client Key Password | SSL -> Client Key Pass | Password must now be stored in a secret |
| SSL Configuration -> Enabled SSL Verification | SSL -> Certificate Authority File | Certificate must now be stored in a secret |
In legacy logging, indices were automatically created according to the format in the "Index Patterns" section. In v2.5 logging, default behavior has been changed to logging to a single index. You can still configure index pattern functionality on the `Output` object by editing as YAML and inputting the following values:
```yaml
...
spec:
elasticsearch:
...
logstash_format: true
logstash_prefix: <desired prefix>
logstash_dateformat: "%Y-%m-%d"
```
Replace `<desired prefix>` with the prefix for the indices that will be created. In legacy logging, this defaulted to the name of the cluster.
### Splunk
| Legacy Logging | v2.5+ Logging | Notes |
|------------------------------------------|----------------------------------------|----------------------------------------------------------------------------------------|
| HEC Configuration -> Endpoint | Target -> Host | Protocol (https/http) and port must be defined separately from the host |
| HEC Configuration -> Token | Access -> Token | Token must now be stored as a secret |
| HEC Configuration -> Index | Edit as YAML -> `index` | `index` field must be added as YAML key under `spec.splunkHec` |
| HEC Configuration -> Source | Edit as YAML -> `source` | `source` field must be added as YAML key under `spec.splunkHec` |
| SSL Configuration -> Client Private Key | Edit as YAML -> `client_key` | `client_key` field must be added as YAML key under `spec.splunkHec`. See (1) |
| SSL Configuration -> Client Certificate | Edit as YAML -> `client_cert` | `client_cert` field must be added as YAML key under `spec.splunkHec`. See (1) |
| SSL Configuration -> Client Key Password | _Not Supported_ | Specifying a password for the client private key is not currently supported. |
| SSL Configuration -> SSL Verify | Edit as YAML -> `ca_file` or `ca_path` | `ca_file` or `ca_path` field must be added as YAML key under `spec.splunkHec`. See (2) |
_(1) `client_key` and `client_cert` values must be paths to the key and cert files, respectively. These files must be mounted into the `rancher-logging-fluentd` pod in order to be used._
_(2) Users can configure either `ca_file` (a path to a PEM-encoded CA certificate) or `ca_path` (a path to a directory containing CA certificates in PEM format). These files must be mounted into the `rancher-logging-fluentd` pod in order to be used._
### Kafka
| Legacy Logging | v2.5+ Logging | Notes |
|-----------------------------------------|----------------------------|------------------------------------------------------|
| Kafka Configuration -> Endpoint Type | - | Zookeeper is no longer supported as an endpoint type |
| Kafka Configuration -> Endpoint | Target -> Brokers | Comma-separated list of brokers (host:port) |
| Kafka Configuration -> Topic | Target -> Default Topic | |
| SSL Configuration -> Client Private Key | SSL -> SSL Client Cert | Certificate must be stored as a secret |
| SSL Configuration -> Client Certificate | SSL -> SSL Client Cert Key | Key must be stored as a secret |
| SSL Configuration -> CA Certificate PEM | SSL -> SSL CA Cert | Certificate must be stored as a secret |
| SASL Configuration -> Username | Access -> Username | Username must be stored in a secret |
| SASL Configuration -> Password | Access -> Password | Password must be stored in a secret |
| SASL Configuration -> Scram Mechanism | Access -> Scram Mechanism | Input mechanism as string, e.g. "sha256" or "sha512" |
### Fluentd
As of v2.5.2, it is only possible to add a single Fluentd server using the "Edit as Form" option. To add multiple servers, edit the `Output` as YAML and input multiple servers.
| Legacy Logging | v2.5+ Logging | Notes |
|------------------------------------------|-----------------------------------------------------|----------------------------------------------------------------------|
| Fluentd Configuration -> Endpoint | Target -> Host, Port | Input the host and port separately |
| Fluentd Configuration -> Shared Key | Access -> Shared Key | Shared key must be stored as a secret |
| Fluentd Configuration -> Username | Access -> Username | Username must be stored as a secret |
| Fluentd Configuration -> Password | Access -> Password | Password must be stored as a secret |
| Fluentd Configuration -> Hostname | Edit as YAML -> `host` | `host` field set as YAML key under `spec.forward.servers[n]` |
| Fluentd Configuration -> Weight | Edit as YAML -> `weight` | `weight` field set as YAML key under `spec.forward.servers[n]` |
| SSL Configuration -> Use TLS | - | Do not need to explicitly enable. Define client cert fields instead. |
| SSL Configuration -> Client Private Key | Edit as YAML -> `tls_private_key_path` | Field set as YAML key under `spec.forward`. See (1) |
| SSL Configuration -> Client Certificate | Edit as YAML -> `tls_client_cert_path` | Field set as YAML key under `spec.forward`. See (1) |
| SSL Configuration -> Client Key Password | Edit as YAML -> `tls_client_private_key_passphrase` | Field set as YAML key under `spec.forward`. See (1) |
| SSL Configuration -> SSL Verify | Edit as YAML -> `tls_insecure_mode` | Field set as YAML key under `spec.forward`. Default: `false` |
| SSL Configuration -> CA Certificate PEM | Edit as YAML -> `tls_cert_path` | Field set as YAML key under `spec.forward`. See (1) |
| Enable Gzip Compression | - | No longer supported in v2.5+ logging |
_(1) These values are to be specified as paths to files. Those files must be mounted into the `rancher-logging-fluentd` pod in order to be used._
### Syslog
As of v2.5.2, syslog is not currently supported for `Outputs` using v2.5+ logging.
## Custom Log Fields
In order to add custom log fields, you will need to add the following YAML to your `Flow` configuration:
```
...
spec:
filters:
- record_modifier:
records:
- foo: "bar"
```
(replace `foo: "bar"` with custom log fields you wish to add)
## System Logging
In legacy logging, collecting logs from system components was accomplished by checking a box labeled "Include System Log" when setting up cluster logging. In v2.5+ logging, system logs are gathered in one of two ways:
1. Gather all cluster logs, not specifying any match or exclusion rules. This results in all container logs from the cluster being collected, which includes system logs.
2. Specifically target system logs by adding match rules for system components. Specific match rules depend on the component being collected.
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/rbac-for-logging.md
-23
View File
@@ -1,23 +0,0 @@
---
title: Role-based Access Control for Logging
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/rbac-for-logging"/>
</head>
Rancher logging has two roles, `logging-admin` and `logging-view`.
- `logging-admin` gives users full access to namespaced `Flows` and `Outputs`
- `logging-view` allows users to *view* namespaced `Flows` and `Outputs`, and `ClusterFlows` and `ClusterOutputs`
> **Why choose one role over the other?** Edit access to `ClusterFlow` and `ClusterOutput` resources is powerful. Any user with it has edit access for all logs in the cluster.
In Rancher, the cluster administrator role is the only role with full access to all `rancher-logging` resources. Cluster members are not able to edit or read any logging resources. Project owners and members have the following privileges:
Project Owners | Project Members
--- | ---
able to create namespaced `Flows` and `Outputs` in their projects' namespaces | only able to view the `Flows` and `Outputs` in projects' namespaces
can collect logs from anything in their projects' namespaces | cannot collect any logs in their projects' namespaces
Both project owners and project members require at least *one* namespace in their project to use logging. If they do not, then they may not see the logging button in the top nav dropdown.
versioned_docs/version-2.5/explanations/integrations-in-rancher/logging/taints-and-tolerations.md
-83
View File
@@ -1,83 +0,0 @@
---
title: Working with Taints and Tolerations
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/taints-and-tolerations"/>
</head>
"Tainting" a Kubernetes node causes pods to repel running on that node.
Unless the pods have a `toleration` for that node's taint, they will run on other nodes in the cluster.
[Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) can work in conjunction with the `nodeSelector` [field](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) within the `PodSpec`, which enables the *opposite* effect of a taint.
Using `nodeSelector` gives pods an affinity towards certain nodes.
Both provide choice for the what node(s) the pod will run on.
- [Default Implementation in Rancher's Logging Stack](#default-implementation-in-ranchers-logging-stack)
- [Adding NodeSelector Settings and Tolerations for Custom Taints](#adding-nodeselector-settings-and-tolerations-for-custom-taints)
## Default Implementation in Rancher's Logging Stack
<Tabs>
<TabItem value="Rancher v2.5.8+">
By default, Rancher taints all Linux nodes with `cattle.io/os=linux`, and does not taint Windows nodes.
The logging stack pods have `tolerations` for this taint, which enables them to run on Linux nodes.
Moreover, most logging stack pods run on Linux only and have a `nodeSelector` added to ensure they run on Linux nodes.
</TabItem>
<TabItem value="Rancher before v2.5.8">
By default, Rancher taints all Linux nodes with `cattle.io/os=linux`, and does not taint Windows nodes.
The logging stack pods have `tolerations` for this taint, which enables them to run on Linux nodes.
Moreover, we can populate the `nodeSelector` to ensure that our pods *only* run on Linux nodes.
</TabItem>
</Tabs>
This example Pod YAML file shows a nodeSelector being used with a toleration:
```yaml
apiVersion: v1
kind: Pod
# metadata...
spec:
# containers...
tolerations:
- key: cattle.io/os
operator: "Equal"
value: "linux"
effect: NoSchedule
nodeSelector:
kubernetes.io/os: linux
```
In the above example, we ensure that our pod only runs on Linux nodes, and we add a `toleration` for the taint we have on all of our Linux nodes.
You can do the same with Rancher's existing taints, or with your own custom ones.
## Adding NodeSelector Settings and Tolerations for Custom Taints
If you would like to add your own `nodeSelector` settings, or if you would like to add `tolerations` for additional taints, you can pass the following to the chart's values.
```yaml
tolerations:
# insert tolerations...
nodeSelector:
# insert nodeSelector...
```
These values will add both settings to the `fluentd`, `fluentbit`, and `logging-operator` containers.
Essentially, these are global settings for all pods in the logging stack.
However, if you would like to add tolerations for *only* the `fluentbit` container, you can add the following to the chart's values.
```yaml
fluentbit_tolerations:
# insert tolerations list for fluentbit containers only...
```
versioned_docs/version-2.5/explanations/integrations-in-rancher/longhorn.md
-81
View File
@@ -1,81 +0,0 @@
---
title: Cloud Native Storage with Longhorn
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/longhorn"/>
</head>
[Longhorn](https://longhorn.io/) is a lightweight, reliable and easy-to-use distributed block storage system for Kubernetes.
Longhorn is free, open source software. Originally developed by Rancher Labs, it is now being developed as a sandbox project of the Cloud Native Computing Foundation. It can be installed on any Kubernetes cluster with Helm, with kubectl, or with the Rancher UI. You can learn more about its architecture [here.](https://longhorn.io/docs/1.0.2/concepts/)
With Longhorn, you can:
- Use Longhorn volumes as persistent storage for the distributed stateful applications in your Kubernetes cluster
- Partition your block storage into Longhorn volumes so that you can use Kubernetes volumes with or without a cloud provider
- Replicate block storage across multiple nodes and data centers to increase availability
- Store backup data in external storage such as NFS or AWS S3
- Create cross-cluster disaster recovery volumes so that data from a primary Kubernetes cluster can be quickly recovered from backup in a second Kubernetes cluster
- Schedule recurring snapshots of a volume, and schedule recurring backups to NFS or S3-compatible secondary storage
- Restore volumes from backup
- Upgrade Longhorn without disrupting persistent volumes
<figcaption>Longhorn Dashboard</figcaption>
![Longhorn Dashboard](/img/longhorn-screenshot.png)
### New in Rancher v2.5
Before Rancher v2.5, Longhorn could be installed as a Rancher catalog app. In Rancher v2.5, the catalog system was replaced by the **Apps & Marketplace,** and it became possible to install Longhorn as an app from that page.
The **Cluster Explorer** now allows you to manipulate Longhorn's Kubernetes resources from the Rancher UI. So now you can control the Longhorn functionality with the Longhorn UI, or with kubectl, or by manipulating Longhorn's Kubernetes custom resources in the Rancher UI.
These instructions assume you are using Rancher v2.5, but Longhorn can be installed with earlier Rancher versions. For documentation about installing Longhorn as a catalog app using the legacy Rancher UI, refer to the [Longhorn documentation.](https://longhorn.io/docs/1.0.2/deploy/install/install-with-rancher/)
### Installing Longhorn with Rancher
1. Fulfill all [Installation Requirements.](https://github.com/longhorn/website/blob/master/content/docs/archives/1.1.0/deploy/install/_index.md#installation-requirements)
1. Go to the **Cluster Explorer** in the Rancher UI.
1. Click **Apps.**
1. Click `longhorn`.
1. Optional: To customize the initial settings, click **Longhorn Default Settings** and edit the configuration. For help customizing the settings, refer to the [Longhorn documentation.](https://github.com/longhorn/website/blob/master/content/docs/archives/1.0.2/references/settings.md)
1. Click **Install.**
**Result:** Longhorn is deployed in the Kubernetes cluster.
### Accessing Longhorn from the Rancher UI
1. From the **Cluster Explorer," go to the top left dropdown menu and click **Cluster Explorer > Longhorn.**
1. On this page, you can edit Kubernetes resources managed by Longhorn. To view the Longhorn UI, click the **Longhorn** button in the **Overview** section.
**Result:** You will be taken to the Longhorn UI, where you can manage your Longhorn volumes and their replicas in the Kubernetes cluster, as well as secondary backups of your Longhorn storage that may exist in another Kubernetes cluster or in S3.
### Uninstalling Longhorn from the Rancher UI
1. Click **Cluster Explorer > Apps & Marketplace.**
1. Click **Installed Apps.**
1. Go to the `longhorn-system` namespace and check the boxes next to the `longhorn` and `longhorn-crd` apps.
1. Click **Delete,** and confirm **Delete.**
**Result:** Longhorn is uninstalled.
### GitHub Repository
The Longhorn project is available [here.](https://github.com/longhorn/longhorn)
### Documentation
The Longhorn documentation is [here.](https://longhorn.io/docs/)
### Architecture
Longhorn creates a dedicated storage controller for each volume and synchronously replicates the volume across multiple replicas stored on multiple nodes.
The storage controller and replicas are themselves orchestrated using Kubernetes.
You can learn more about its architecture [here.](https://longhorn.io/docs/1.0.2/concepts/)
<figcaption>Longhorn Architecture</figcaption>
![Longhorn Architecture](/img/longhorn-architecture.svg)
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/built-in-dashboards.md
-85
View File
@@ -1,85 +0,0 @@
---
title: Built-in Dashboards
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting/built-in-dashboards"/>
</head>
## Grafana UI
[Grafana](https://grafana.com/grafana/) allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data driven culture.
To see the default dashboards for time series data visualization, go to the Grafana UI.
### Customizing Grafana
To view and customize the PromQL queries powering the Grafana dashboard, see [this page.](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/customize-grafana-dashboard.md)
### Persistent Grafana Dashboards
To create a persistent Grafana dashboard, see [this page.](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/create-persistent-grafana-dashboard.md)
### Access to Grafana
For information about role-based access control for Grafana, see [this section.](rbac-for-monitoring.md#role-based-access-control-for-grafana)
## Alertmanager UI
When `rancher-monitoring` is installed, the Prometheus Alertmanager UI is deployed, allowing you to view your alerts and the current Alertmanager configuration.
> This section assumes familiarity with how monitoring components work together. For more information about Alertmanager, see [How Alertmanager Works.](how-monitoring-works.md#3-how-alertmanager-works)
### Accessing the Alertmanager UI
The Alertmanager UI lets you see the most recently fired alerts.
> **Prerequisite:** The `rancher-monitoring` application must be installed.
To see the Alertmanager UI, go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Alertmanager.**
**Result:** The Alertmanager UI opens in a new tab. For help with configuration, refer to the [official Alertmanager documentation.](https://prometheus.io/docs/alerting/latest/alertmanager/)
<figcaption>The Alertmanager UI</figcaption>
![Alertmanager UI](/img/alertmanager-ui.png)
### Viewing Default Alerts
To see alerts that are fired by default, go to the Alertmanager UI and click **Expand all groups.**
## Prometheus UI
By default, the [kube-state-metrics service](https://github.com/kubernetes/kube-state-metrics) provides a wealth of information about CPU and memory utilization to the monitoring application. These metrics cover Kubernetes resources across namespaces. This means that in order to see resource metrics for a service, you don't need to create a new ServiceMonitor for it. Because the data is already in the time series database, you can go to the Prometheus UI and run a PromQL query to get the information. The same query can be used to configure a Grafana dashboard to show a graph of those metrics over time.
To see the Prometheus UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Graph.**
<figcaption>Prometheus Graph UI</figcaption>
![Prometheus Graph UI](/img/prometheus-graph-ui.png)
### Viewing the Prometheus Targets
To see what services you are monitoring, you will need to see your targets. Targets are set up by ServiceMonitors and PodMonitors as sources to scrape metrics from. You won't need to directly edit targets, but the Prometheus UI can be useful for giving you an overview of all of the sources of metrics that are being scraped.
To see the Prometheus Targets, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Targets.**
<figcaption>Targets in the Prometheus UI</figcaption>
![Prometheus Targets UI](/img/prometheus-targets-ui.png)
### Viewing the PrometheusRules
When you define a Rule (which is declared within a RuleGroup in a PrometheusRule resource), the [spec of the Rule itself](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#rule) contains labels that are used by Alertmanager to figure out which Route should receive a certain Alert.
To see the PrometheusRules, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Prometheus Rules.**
You can also see the rules in the Prometheus UI:
<figcaption>Rules in the Prometheus UI</figcaption>
![PrometheusRules UI](/img/prometheus-rules-ui.png)
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md
-250
View File
@@ -1,250 +0,0 @@
---
title: How Monitoring Works
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works"/>
</head>
## 1. Architecture Overview
_**The following sections describe how data flows through the Monitoring V2 application:**_
### Prometheus Operator
Prometheus Operator observes ServiceMonitors, PodMonitors, and PrometheusRules being created. When the Prometheus configuration resources are created, Prometheus Operator calls the Prometheus API to sync the new configuration. As the diagram at the end of this section shows, the Prometheus Operator acts as the intermediary between Prometheus and Kubernetes, calling the Prometheus API to synchronize Prometheus with the monitoring-related resources in Kubernetes.
### ServiceMonitors and PodMonitors
ServiceMonitors and PodMonitors declaratively specify targets, such as Services and Pods, that need to be monitored.
- Targets are scraped on a recurring schedule based on the configured Prometheus scrape interval, and the metrics that are scraped are stored into the Prometheus Time Series Database (TSDB).
- In order to perform the scrape, ServiceMonitors and PodMonitors are defined with label selectors that determine which Services or Pods should be scraped and endpoints that determine how the scrape should happen on the given target, e.g., scrape/metrics in TCP 10252, proxying through IP addr x.x.x.x.
- Out of the box, Monitoring V2 comes with certain pre-configured exporters that are deployed based on the type of Kubernetes cluster that it is deployed on. For more information, see [Scraping and Exposing Metrics](#5-scraping-and-exposing-metrics).
### How PushProx Works
- Certain internal Kubernetes components are scraped via a proxy deployed as part of Monitoring V2 called **PushProx**. The Kubernetes components that expose metrics to Prometheus through PushProx are the following:
`kube-controller-manager`, `kube-scheduler`, `etcd`, and `kube-proxy`.
- For each PushProx exporter, we deploy one PushProx client onto all target nodes. For example, a PushProx client is deployed onto all controlplane nodes for kube-controller-manager, all etcd nodes for kube-etcd, and all nodes for kubelet.
- We deploy exactly one PushProx proxy per exporter. The process for exporting metrics is as follows:
1. The PushProx Client establishes an outbound connection with the PushProx Proxy.
1. The client then polls the proxy for scrape requests that have come into the proxy.
1. When the proxy receives a scrape request from Prometheus, the client sees it as a result of the poll.
1. The client scrapes the internal component.
1. The internal component responds by pushing metrics back to the proxy.
<figcaption><br/>Process for Exporting Metrics with PushProx:<br/></figcaption>
![Process for Exporting Metrics with PushProx](/img/pushprox-process.svg)
### PrometheusRules
PrometheusRules allow users to define rules for what metrics or time series database queries should result in alerts being fired. Rules are evaluated on an interval.
- **Recording rules** create a new time series based on existing series that have been collected. They are frequently used to precompute complex queries.
- **Alerting rules** run a particular query and fire an alert from Prometheus if the query evaluates to a non-zero value.
### Alert Routing
Once Prometheus determines that an alert needs to be fired, alerts are forwarded to **Alertmanager**.
- Alerts contain labels that come from the PromQL query itself and additional labels and annotations that can be provided as part of specifying the initial PrometheusRule.
- Before receiving any alerts, Alertmanager will use the **routes** and **receivers** specified in its configuration to form a routing tree on which all incoming alerts are evaluated. Each node of the routing tree can specify additional grouping, labeling, and filtering that needs to happen based on the labels attached to the Prometheus alert. A node on the routing tree (usually a leaf node) can also specify that an alert that reaches it needs to be sent out to a configured Receiver, e.g., Slack, PagerDuty, SMS, etc. Note that Alertmanager will send an alert first to **alertingDriver**, then alertingDriver will send or forward alert to the proper destination.
- Routes and receivers are also stored in the Kubernetes API via the Alertmanager Secret. When the Secret is updated, Alertmanager is also updated automatically. Note that routing occurs via labels only (not via annotations, etc.).
## 2. How Prometheus Works
### Storing Time Series Data
After collecting metrics from exporters, Prometheus stores the time series in a local on-disk time series database. Prometheus optionally integrates with remote systems, but `rancher-monitoring` uses local storage for the time series database.
Once stored, users can query this TSDB using PromQL, the query language for Prometheus.
PromQL queries can be visualized in one of two ways:
1. By supplying the query in Prometheus's Graph UI, which will show a simple graphical view of the data.
1. By creating a Grafana Dashboard that contains the PromQL query and additional formatting directives that label axes, add units, change colors, use alternative visualizations, etc.
### Defining Rules for Prometheus
Rules define queries that Prometheus needs to execute on a regular `evaluationInterval` to perform certain actions, such as firing an alert (alerting rules) or precomputing a query based on others existing in its TSDB (recording rules). These rules are encoded in PrometheusRules custom resources. When PrometheusRule custom resources are created or updated, the Prometheus Operator observes the change and calls the Prometheus API to synchronize the set of rules that Prometheus is currently evaluating on a regular interval.
A PrometheusRule allows you to define one or more RuleGroups. Each RuleGroup consists of a set of Rule objects that can each represent either an alerting or a recording rule with the following fields:
- The name of the new alert or record
- A PromQL expression for the new alert or record
- Labels that should be attached to the alert or record that identify it (e.g. cluster name or severity)
- Annotations that encode any additional important pieces of information that need to be displayed on the notification for an alert (e.g. summary, description, message, runbook URL, etc.). This field is not required for recording rules.
Upon evaluating a [rule](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#rule), Prometheus runs the provided PromQL query, adds the provided labels, and runs the appropriate action for the rule. If the rule triggers an alert, Prometheus also adds the provided annotations. For example, an Alerting Rule that adds `team: front-end` as a label to the provided PromQL query will append that label to the fired alert, which will allow Alertmanager to forward the alert to the correct Receiver.
### Alerting and Recording Rules
Prometheus doesn't maintain the state of whether alerts are active. It fires alerts repetitively at every evaluation interval, relying on Alertmanager to group and filter the alerts into meaningful notifications.
The `evaluation_interval` constant defines how often Prometheus evaluates its alerting rules against the time series database. Similar to the `scrape_interval`, the `evaluation_interval` also defaults to one minute.
The rules are contained in a set of rule files. Rule files include both alerting rules and recording rules, but only alerting rules result in alerts being fired after their evaluation.
For recording rules, Prometheus runs a query, then stores it as a time series. This synthetic time series is useful for storing the results of an expensive or time-consuming query so that it can be queried more quickly in the future.
Alerting rules are more commonly used. Whenever an alerting rule evaluates to a positive number, Prometheus fires an alert.
The Rule file adds labels and annotations to alerts before firing them, depending on the use case:
- Labels indicate information that identifies the alert and could affect the routing of the alert. For example, if when sending an alert about a certain container, the container ID could be used as a label.
- Annotations denote information that doesn't affect where an alert is routed, for example, a runbook or an error message.
## 3. How Alertmanager Works
The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of the following tasks:
- Deduplicating, grouping, and routing alerts to the correct receiver integration such as email, PagerDuty, or OpsGenie
- Silencing and inhibition of alerts
- Tracking alerts that fire over time
- Sending out the status of whether an alert is currently firing, or if it is resolved
### Alerts Forwarded by alertingDrivers
When alertingDrivers are installed, this creates a `Service` that can be used as the receiver's URL for Teams or SMS, based on the alertingDriver's configuration. The URL in the Receiver points to the alertingDrivers; so the Alertmanager sends alert first to alertingDriver, then alertingDriver forwards or sends alert to the proper destination.
### Routing Alerts to Receivers
Alertmanager coordinates where alerts are sent. It allows you to group alerts based on labels and fire them based on whether certain labels are matched. One top-level route accepts all alerts. From there, Alertmanager continues routing alerts to receivers based on whether they match the conditions of the next route.
While the Rancher UI forms only allow editing a routing tree that is two levels deep, you can configure more deeply nested routing structures by editing the Alertmanager Secret.
### Configuring Multiple Receivers
By editing the forms in the Rancher UI, you can set up a Receiver resource with all the information Alertmanager needs to send alerts to your notification system.
By editing custom YAML in the Alertmanager or Receiver configuration, you can also send alerts to multiple notification systems. For more information, see the section on configuring [Receivers.](../../../reference-guides/monitoring-v2-configuration/receivers.md#configuring-multiple-receivers)
## 4. Monitoring V2 Specific Components
Prometheus Operator introduces a set of [Custom Resource Definitions](https://github.com/prometheus-operator/prometheus-operator#customresourcedefinitions) that allow users to deploy and manage Prometheus and Alertmanager instances by creating and modifying those custom resources on a cluster.
Prometheus Operator will automatically update your Prometheus configuration based on the live state of the resources and configuration options that are edited in the Rancher UI.
### Resources Deployed by Default
By default, a set of resources curated by the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project are deployed onto your cluster as part of installing the Rancher Monitoring Application to set up a basic Monitoring/Alerting stack.
The resources that get deployed onto your cluster to support this solution can be found in the [`rancher-monitoring`](https://github.com/rancher/charts/tree/main/charts/rancher-monitoring) Helm chart, which closely tracks the upstream [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) Helm chart maintained by the Prometheus community with certain changes tracked in the [CHANGELOG.md](https://github.com/rancher/charts/blob/main/charts/rancher-monitoring/CHANGELOG.md).
### Default Exporters
Monitoring V2 deploys three default exporters that provide additional metrics for Prometheus to store:
1. `node-exporter`: exposes hardware and OS metrics for Linux hosts. For more information on `node-exporter`, refer to the [upstream documentation](https://prometheus.io/docs/guides/node-exporter/).
1. `windows-exporter`: exposes hardware and OS metrics for Windows hosts (only deployed on Windows clusters). For more information on `windows-exporter`, refer to the [upstream documentation](https://github.com/prometheus-community/windows_exporter).
1. `kube-state-metrics`: expose additional metrics that track the state of resources contained in the Kubernetes API (e.g., pods, workloads, etc.). For more information on `kube-state-metrics`, refer to the [upstream documentation](https://github.com/kubernetes/kube-state-metrics/tree/master/docs).
ServiceMonitors and PodMonitors will scrape these exporters, as defined [here](#defining-what-metrics-are-scraped). Prometheus stores these metrics, and you can query the results via either Prometheus's UI or Grafana.
See the [architecture](#1-architecture-overview) section for more information on recording rules, alerting rules, and Alertmanager.
### Components Exposed in the Rancher UI
When the monitoring application is installed, you will be able to edit the following components in the Rancher UI:
| Component | Type of Component | Purpose and Common Use Cases for Editing |
|--------------|------------------------|---------------------------|
| ServiceMonitor | Custom resource | Sets up Kubernetes Services to scrape custom metrics from. Automatically updates the scrape configuration in the Prometheus custom resource. |
| PodMonitor | Custom resource | Sets up Kubernetes Pods to scrape custom metrics from. Automatically updates the scrape configuration in the Prometheus custom resource. |
| Receiver | Configuration block (part of Alertmanager) | Modifies information on where to send an alert (e.g., Slack, PagerDuty, etc.) and any necessary information to send the alert (e.g., TLS certs, proxy URLs, etc.). Automatically updates the Alertmanager custom resource. |
| Route | Configuration block (part of Alertmanager) | Modifies the routing tree that is used to filter, label, and group alerts based on labels and send them to the appropriate Receiver. Automatically updates the Alertmanager custom resource. |
| PrometheusRule | Custom resource | Defines additional queries that need to trigger alerts or define materialized views of existing series that are within Prometheus's TSDB. Automatically updates the Prometheus custom resource. |
### PushProx
PushProx allows Prometheus to scrape metrics across a network boundary, which prevents users from having to expose metrics ports for internal Kubernetes components on each node in a Kubernetes cluster.
Since the metrics for Kubernetes components are generally exposed on the host network of nodes in the cluster, PushProx deploys a DaemonSet of clients that sit on the hostNetwork of each node and make an outbound connection to a single proxy that is sitting on the Kubernetes API. Prometheus can then be configured to proxy scrape requests through the proxy to each client, which allows it to scrape metrics from the internal Kubernetes components without requiring any inbound node ports to be open.
Refer to [Scraping Metrics with PushProx](#scraping-metrics-with-pushprox) for more.
## 5. Scraping and Exposing Metrics
### Defining what Metrics are Scraped
ServiceMonitors and PodMonitors define targets that are intended for Prometheus to scrape. The [Prometheus custom resource](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/design.md#prometheus) tells Prometheus which ServiceMonitors or PodMonitors it should use to find out where to scrape metrics from.
The Prometheus Operator observes the ServiceMonitors and PodMonitors. When it observes that they are created or updated, it calls the Prometheus API to update the scrape configuration in the Prometheus custom resource and keep it in sync with the scrape configuration in the ServiceMonitors or PodMonitors. This scrape configuration tells Prometheus which endpoints to scrape metrics from and how it will label the metrics from those endpoints.
Prometheus scrapes all of the metrics defined in its scrape configuration at every `scrape_interval`, which is one minute by default.
The scrape configuration can be viewed as part of the Prometheus custom resource that is exposed in the Rancher UI.
### How the Prometheus Operator Sets up Metrics Scraping
The Prometheus Deployment or StatefulSet scrapes metrics, and the configuration of Prometheus is controlled by the Prometheus custom resources. The Prometheus Operator watches for Prometheus and Alertmanager resources, and when they are created, the Prometheus Operator creates a Deployment or StatefulSet for Prometheus or Alertmanager with the user-defined configuration.
When the Prometheus Operator observes ServiceMonitors, PodMonitors, and PrometheusRules being created, it knows that the scrape configuration needs to be updated in Prometheus. It updates Prometheus by first updating the configuration and rules files in the volumes of Prometheus's Deployment or StatefulSet. Then it calls the Prometheus API to sync the new configuration, resulting in the Prometheus Deployment or StatefulSet to be modified in place.
### How Kubernetes Component Metrics are Exposed
Prometheus scrapes metrics from deployments known as [exporters,](https://prometheus.io/docs/instrumenting/exporters/) which export the time series data in a format that Prometheus can ingest. In Prometheus, time series consist of streams of timestamped values belonging to the same metric and the same set of labeled dimensions.
### Scraping Metrics with PushProx
Certain internal Kubernetes components are scraped via a proxy deployed as part of Monitoring V2 called PushProx. For detailed information on PushProx, refer [here](#how-pushprox-works) and to the above [architecture](#1-architecture-overview) section.
### Scraping Metrics
The following Kubernetes components are directly scraped by Prometheus:
- kubelet*
- ingress-nginx**
- coreDns/kubeDns
- kube-api-server
\* You can optionally use `hardenedKubelet.enabled` to use a PushProx, but that is not the default.
** For RKE and RKE2 clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component.
### Scraping Metrics Based on Kubernetes Distribution
Metrics are scraped differently based on the Kubernetes distribution. For help with terminology, refer [here](#terminology). For details, see the table below:
<figcaption>How Metrics are Exposed to Prometheus</figcaption>
| Kubernetes Component | RKE | RKE2 | KubeADM | K3s |
|-----|-----|-----|-----|-----|
| kube-controller-manager | rkeControllerManager.enabled |rke2ControllerManager.enabled | kubeAdmControllerManager.enabled | k3sServer.enabled |
| kube-scheduler | rkeScheduler.enabled | rke2Scheduler.enabled |kubeAdmScheduler.enabled | k3sServer.enabled |
| etcd | rkeEtcd.enabled | rke2Etcd.enabled | kubeAdmEtcd.enabled | Not available |
| kube-proxy | rkeProxy.enabled | rke2Proxy.enabled | kubeAdmProxy.enabled | k3sServer.enabled |
| kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet |
| ingress-nginx* | Collects metrics directly exposed by kubelet, exposed by rkeIngressNginx.enabled | Collects metrics directly exposed by kubelet, Exposed by rke2IngressNginx.enabled | Not available | Not available |
| coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns |
| kube-api-server | Collects metrics directly exposed by kube-api-server |Collects metrics directly exposed by kube-api-server | Collects metrics directly exposed by kube-appi-server | Collects metrics directly exposed by kube-api-server |
\* For RKE and RKE2 clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component.
### Terminology
- **kube-scheduler:** The internal Kubernetes component that uses information in the pod spec to decide on which node to run a pod.
- **kube-controller-manager:** The internal Kubernetes component that is responsible for node management (detecting if a node fails), pod replication and endpoint creation.
- **etcd:** The internal Kubernetes component that is the distributed key/value store which Kubernetes uses for persistent storage of all cluster information.
- **kube-proxy:** The internal Kubernetes component that watches the API server for pods/services changes in order to maintain the network up to date.
- **kubelet:** The internal Kubernetes component that watches the API server for pods on a node and makes sure they are running.
- **ingress-nginx:** An Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.
- **coreDns/kubeDns:** The internal Kubernetes component responsible for DNS.
- **kube-api-server:** The main internal Kubernetes component that is responsible for exposing APIs for the other master components.
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/monitoring-and-alerting.md
-99
View File
@@ -1,99 +0,0 @@
---
title: Monitoring and Alerting
description: Prometheus lets you view metrics from your different Rancher and Kubernetes objects. Learn about the scope of monitoring and how to enable cluster monitoring
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting"/>
</head>
Using the `rancher-monitoring` application, you can quickly deploy leading open-source monitoring and alerting solutions onto your cluster.
## Features
Prometheus lets you view metrics from your Rancher and Kubernetes objects. Using timestamps, Prometheus lets you query and view these metrics in easy-to-read graphs and visuals, either through the Rancher UI or Grafana, which is an analytics viewing platform deployed along with Prometheus.
By viewing data that Prometheus scrapes from your cluster control plane, nodes, and deployments, you can stay on top of everything happening in your cluster. You can then use these analytics to better run your organization: stop system emergencies before they start, develop maintenance strategies, or restore crashed servers.
The `rancher-monitoring` operator, introduced in Rancher v2.5, is powered by [Prometheus](https://prometheus.io/), [Grafana](https://grafana.com/grafana/), [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), the [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator), and the [Prometheus adapter.](https://github.com/DirectXMan12/k8s-prometheus-adapter)
The monitoring application allows you to:
- Monitor the state and processes of your cluster nodes, Kubernetes components, and software deployments
- Define alerts based on metrics collected via Prometheus
- Create custom Grafana dashboards
- Configure alert-based notifications via Email, Slack, PagerDuty, etc. using Prometheus Alertmanager
- Defines precomputed, frequently needed or computationally expensive expressions as new time series based on metrics collected via Prometheus
- Expose collected metrics from Prometheus to the Kubernetes Custom Metrics API via Prometheus Adapter for use in HPA
## How Monitoring Works
For an explanation of how the monitoring components work together, see [this page.](how-monitoring-works.md)
## Default Components and Deployments
### Built-in Dashboards
By default, the monitoring application deploys Grafana dashboards (curated by the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) project) onto a cluster.
It also deploys an Alertmanager UI and a Prometheus UI. For more information about these tools, see [Built-in Dashboards.](built-in-dashboards.md)
### Default Metrics Exporters
By default, Rancher Monitoring deploys exporters (such as [node-exporter](https://github.com/prometheus/node_exporter) and [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)).
These default exporters automatically scrape metrics for CPU and memory from all components of your Kubernetes cluster, including your workloads.
### Default Alerts
The monitoring application deploys some alerts by default. To see the default alerts, go to the [Alertmanager UI](built-in-dashboards.md#alertmanager-ui) and click **Expand all groups.**
### Components Exposed in the Rancher UI
For a list of monitoring components exposed in the Rancher UI, along with common use cases for editing them, see [this section.](how-monitoring-works.md#components-exposed-in-the-rancher-ui)
## Role-based Access Control
For information on configuring access to monitoring, see [this page.](rbac-for-monitoring.md)
## Guides
- [Enable monitoring](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/enable-monitoring.md)
- [Uninstall monitoring](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/uninstall-monitoring.md)
- [Monitoring workloads](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/set-up-monitoring-for-workloads.md)
- [Customizing Grafana dashboards](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/customize-grafana-dashboard.md)
- [Persistent Grafana dashboards](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/create-persistent-grafana-dashboard.md)
- [Debugging high memory usage](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/debug-high-memory-usage.md)
- [Migrating from Monitoring V1 to V2](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/migrate-to-rancher-v2.5+-monitoring.md)
## Configuration
### Configuring Monitoring Resources in Rancher
> The configuration reference assumes familiarity with how monitoring components work together. For more information, see [How Monitoring Works.](how-monitoring-works.md)
- [ServiceMonitor and PodMonitor](../../../reference-guides/monitoring-v2-configuration/servicemonitors-and-podmonitors.md)
- [Receiver](../../../reference-guides/monitoring-v2-configuration/receivers.md)
- [Route](../../../reference-guides/monitoring-v2-configuration/routes.md)
- [PrometheusRule](../../../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/prometheusrules.md)
- [Prometheus](../../../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/prometheus.md)
- [Alertmanager](../../../how-to-guides/advanced-user-guides/monitoring-v2-configuration-guides/advanced-configuration/alertmanager.md)
### Configuring Helm Chart Options
For more information on `rancher-monitoring` chart options, including options to set resource limits and requests, see [this page.](../../../reference-guides/monitoring-v2-configuration/helm-chart-options.md)
## Windows Cluster Support
_Available as of v2.5.8_
When deployed onto an RKE1 Windows cluster, Monitoring V2 will now automatically deploy a [windows-exporter](https://github.com/prometheus-community/windows_exporter) DaemonSet and set up a ServiceMonitor to collect metrics from each of the deployed Pods. This will populate Prometheus with `windows_` metrics that are akin to the `node_` metrics exported by [node_exporter](https://github.com/prometheus/node_exporter) for Linux hosts.
To be able to fully deploy Monitoring V2 for Windows, all of your Windows hosts must have a minimum [wins](https://github.com/rancher/wins) version of v0.1.0.
For more details on how to upgrade wins on existing Windows hosts, refer to the section on [Windows cluster support for Monitoring V2.](windows-support.md)
## Known Issues
There is a [known issue](https://github.com/rancher/rancher/issues/28787#issuecomment-693611821) that K3s clusters require more default memory. If you are enabling monitoring on a K3s cluster, we recommend setting `prometheus.prometheusSpec.resources.memory.limit` to 2500 Mi and `prometheus.prometheusSpec.resources.memory.request` to 1750 Mi.
For tips on debugging high memory usage, see [this page.](../../../how-to-guides/advanced-user-guides/monitoring-alerting-guides/debug-high-memory-usage.md)
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/promql-expressions.md
-369
View File
@@ -1,369 +0,0 @@
---
title: PromQL Expression Reference
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting/promql-expressions"/>
</head>
The PromQL expressions in this doc can be used to configure alerts.
For more information about querying the Prometheus time series database, refer to the official [Prometheus documentation.](https://prometheus.io/docs/prometheus/latest/querying/basics/)
## Cluster Metrics
### Cluster CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - (avg(irate(node_cpu_seconds_total{mode="idle"}[5m])) by (instance))` |
| Summary | `1 - (avg(irate(node_cpu_seconds_total{mode="idle"}[5m])))` |
### Cluster Load Average
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>load1</td><td>`sum(node_load1) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr><tr><td>load5</td><td>`sum(node_load5) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr><tr><td>load15</td><td>`sum(node_load15) by (instance) / count(node_cpu_seconds_total{mode="system"}) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>load1</td><td>`sum(node_load1) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr><tr><td>load5</td><td>`sum(node_load5) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr><tr><td>load15</td><td>`sum(node_load15) by (instance) / count(node_cpu_seconds_total{mode="system"})`</td></tr></table> |
### Cluster Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - sum(node_memory_MemAvailable_bytes) by (instance) / sum(node_memory_MemTotal_bytes) by (instance)` |
| Summary | `1 - sum(node_memory_MemAvailable_bytes) / sum(node_memory_MemTotal_bytes)` |
### Cluster Disk Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `(sum(node_filesystem_size_bytes{device!="rootfs"}) by (instance) - sum(node_filesystem_free_bytes{device!="rootfs"}) by (instance)) / sum(node_filesystem_size_bytes{device!="rootfs"}) by (instance)` |
| Summary | `(sum(node_filesystem_size_bytes{device!="rootfs"}) - sum(node_filesystem_free_bytes{device!="rootfs"})) / sum(node_filesystem_size_bytes{device!="rootfs"})` |
### Cluster Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total[5m])) by (instance)`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total[5m]))`</td></tr></table> |
### Cluster Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr></table> |
| Summary | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr></table> |
### Cluster Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m])) by (instance)</code></td></tr></table> |
| Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;"}[5m]))</code></td></tr></table> |
## Node Metrics
### Node CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `avg(irate(node_cpu_seconds_total{mode!="idle", instance=~"$instance"}[5m])) by (mode)` |
| Summary | `1 - (avg(irate(node_cpu_seconds_total{mode="idle", instance=~"$instance"}[5m])))` |
### Node Load Average
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>load1</td><td>`sum(node_load1{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load5</td><td>`sum(node_load5{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load15</td><td>`sum(node_load15{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr></table> |
| Summary | <table><tr><td>load1</td><td>`sum(node_load1{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load5</td><td>`sum(node_load5{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr><tr><td>load15</td><td>`sum(node_load15{instance=~"$instance"}) / count(node_cpu_seconds_total{mode="system",instance=~"$instance"})`</td></tr></table> |
### Node Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `1 - sum(node_memory_MemAvailable_bytes{instance=~"$instance"}) / sum(node_memory_MemTotal_bytes{instance=~"$instance"})` |
| Summary | `1 - sum(node_memory_MemAvailable_bytes{instance=~"$instance"}) / sum(node_memory_MemTotal_bytes{instance=~"$instance"}) ` |
### Node Disk Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `(sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) by (device) - sum(node_filesystem_free_bytes{device!="rootfs",instance=~"$instance"}) by (device)) / sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) by (device)` |
| Summary | `(sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"}) - sum(node_filesystem_free_bytes{device!="rootfs",instance=~"$instance"})) / sum(node_filesystem_size_bytes{device!="rootfs",instance=~"$instance"})` |
### Node Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total{instance=~"$instance"}[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total{instance=~"$instance"}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(node_disk_read_bytes_total{instance=~"$instance"}[5m]))`</td></tr><tr><td>written</td><td>`sum(rate(node_disk_written_bytes_total{instance=~"$instance"}[5m]))`</td></tr></table> |
### Node Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr></table> |
| Summary | <table><tr><td>receive-dropped</td><td><code>sum(rate(node_network_receive_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>receive-errs</td><td><code>sum(rate(node_network_receive_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>receive-packets</td><td><code>sum(rate(node_network_receive_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-dropped</td><td><code>sum(rate(node_network_transmit_drop_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-errs</td><td><code>sum(rate(node_network_transmit_errs_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit-packets</td><td><code>sum(rate(node_network_transmit_packets_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr></table> |
### Node Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m])) by (device)</code></td></tr></table> |
| Summary | <table><tr><td>receive</td><td><code>sum(rate(node_network_receive_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr><tr><td>transmit</td><td><code>sum(rate(node_network_transmit_bytes_total{device!~"lo &#124; veth.&ast; &#124; docker.&ast; &#124; flannel.&ast; &#124; cali.&ast; &#124; cbr.&ast;",instance=~"$instance"}[5m]))</code></td></tr></table> |
## Etcd Metrics
### Etcd Has a Leader
`max(etcd_server_has_leader)`
### Number of Times the Leader Changes
`max(etcd_server_leader_changes_seen_total)`
### Number of Failed Proposals
`sum(etcd_server_proposals_failed_total)`
### GRPC Client Traffic
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>in</td><td>`sum(rate(etcd_network_client_grpc_received_bytes_total[5m])) by (instance)`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_client_grpc_sent_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>in</td><td>`sum(rate(etcd_network_client_grpc_received_bytes_total[5m]))`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_client_grpc_sent_bytes_total[5m]))`</td></tr></table> |
### Peer Traffic
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>in</td><td>`sum(rate(etcd_network_peer_received_bytes_total[5m])) by (instance)`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_peer_sent_bytes_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>in</td><td>`sum(rate(etcd_network_peer_received_bytes_total[5m]))`</td></tr><tr><td>out</td><td>`sum(rate(etcd_network_peer_sent_bytes_total[5m]))`</td></tr></table> |
### DB Size
| Catalog | Expression |
| --- | --- |
| Detail | `sum(etcd_debugging_mvcc_db_total_size_in_bytes) by (instance)` |
| Summary | `sum(etcd_debugging_mvcc_db_total_size_in_bytes)` |
### Active Streams
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>lease-watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) by (instance) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) by (instance)`</td></tr><tr><td>watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) by (instance) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>lease-watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"})`</td></tr><tr><td>watch</td><td>`sum(grpc_server_started_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}) - sum(grpc_server_handled_total{grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"})`</td></tr></table> |
### Raft Proposals
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>applied</td><td>`sum(increase(etcd_server_proposals_applied_total[5m])) by (instance)`</td></tr><tr><td>committed</td><td>`sum(increase(etcd_server_proposals_committed_total[5m])) by (instance)`</td></tr><tr><td>pending</td><td>`sum(increase(etcd_server_proposals_pending[5m])) by (instance)`</td></tr><tr><td>failed</td><td>`sum(increase(etcd_server_proposals_failed_total[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>applied</td><td>`sum(increase(etcd_server_proposals_applied_total[5m]))`</td></tr><tr><td>committed</td><td>`sum(increase(etcd_server_proposals_committed_total[5m]))`</td></tr><tr><td>pending</td><td>`sum(increase(etcd_server_proposals_pending[5m]))`</td></tr><tr><td>failed</td><td>`sum(increase(etcd_server_proposals_failed_total[5m]))`</td></tr></table> |
### RPC Rate
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>total</td><td>`sum(rate(grpc_server_started_total{grpc_type="unary"}[5m])) by (instance)`</td></tr><tr><td>fail</td><td>`sum(rate(grpc_server_handled_total{grpc_type="unary",grpc_code!="OK"}[5m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>total</td><td>`sum(rate(grpc_server_started_total{grpc_type="unary"}[5m]))`</td></tr><tr><td>fail</td><td>`sum(rate(grpc_server_handled_total{grpc_type="unary",grpc_code!="OK"}[5m]))`</td></tr></table> |
### Disk Operations
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>commit-called-by-backend</td><td>`sum(rate(etcd_disk_backend_commit_duration_seconds_sum[1m])) by (instance)`</td></tr><tr><td>fsync-called-by-wal</td><td>`sum(rate(etcd_disk_wal_fsync_duration_seconds_sum[1m])) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>commit-called-by-backend</td><td>`sum(rate(etcd_disk_backend_commit_duration_seconds_sum[1m]))`</td></tr><tr><td>fsync-called-by-wal</td><td>`sum(rate(etcd_disk_wal_fsync_duration_seconds_sum[1m]))`</td></tr></table> |
### Disk Sync Duration
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>wal</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le))`</td></tr><tr><td>db</td><td>`histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le))`</td></tr></table> |
| Summary | <table><tr><td>wal</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr><tr><td>db</td><td>`sum(histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket[5m])) by (instance, le)))`</td></tr></table> |
## Kubernetes Components Metrics
### API Server Request Latency
| Catalog | Expression |
| --- | --- |
| Detail | `avg(apiserver_request_latencies_sum / apiserver_request_latencies_count) by (instance, verb) /1e+06` |
| Summary | `avg(apiserver_request_latencies_sum / apiserver_request_latencies_count) by (instance) /1e+06` |
### API Server Request Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(apiserver_request_count[5m])) by (instance, code)` |
| Summary | `sum(rate(apiserver_request_count[5m])) by (instance)` |
### Scheduling Failed Pods
| Catalog | Expression |
| --- | --- |
| Detail | `sum(kube_pod_status_scheduled{condition="false"})` |
| Summary | `sum(kube_pod_status_scheduled{condition="false"})` |
### Controller Manager Queue Depth
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>volumes</td><td>`sum(volumes_depth) by instance`</td></tr><tr><td>deployment</td><td>`sum(deployment_depth) by instance`</td></tr><tr><td>replicaset</td><td>`sum(replicaset_depth) by instance`</td></tr><tr><td>service</td><td>`sum(service_depth) by instance`</td></tr><tr><td>serviceaccount</td><td>`sum(serviceaccount_depth) by instance`</td></tr><tr><td>endpoint</td><td>`sum(endpoint_depth) by instance`</td></tr><tr><td>daemonset</td><td>`sum(daemonset_depth) by instance`</td></tr><tr><td>statefulset</td><td>`sum(statefulset_depth) by instance`</td></tr><tr><td>replicationmanager</td><td>`sum(replicationmanager_depth) by instance`</td></tr></table> |
| Summary | <table><tr><td>volumes</td><td>`sum(volumes_depth)`</td></tr><tr><td>deployment</td><td>`sum(deployment_depth)`</td></tr><tr><td>replicaset</td><td>`sum(replicaset_depth)`</td></tr><tr><td>service</td><td>`sum(service_depth)`</td></tr><tr><td>serviceaccount</td><td>`sum(serviceaccount_depth)`</td></tr><tr><td>endpoint</td><td>`sum(endpoint_depth)`</td></tr><tr><td>daemonset</td><td>`sum(daemonset_depth)`</td></tr><tr><td>statefulset</td><td>`sum(statefulset_depth)`</td></tr><tr><td>replicationmanager</td><td>`sum(replicationmanager_depth)`</td></tr></table> |
### Scheduler E2E Scheduling Latency
| Catalog | Expression |
| --- | --- |
| Detail | `histogram_quantile(0.99, sum(scheduler_e2e_scheduling_latency_microseconds_bucket) by (le, instance)) / 1e+06` |
| Summary | `sum(histogram_quantile(0.99, sum(scheduler_e2e_scheduling_latency_microseconds_bucket) by (le, instance)) / 1e+06)` |
### Scheduler Preemption Attempts
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(scheduler_total_preemption_attempts[5m])) by (instance)` |
| Summary | `sum(rate(scheduler_total_preemption_attempts[5m]))` |
### Ingress Controller Connections
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"}) by (instance)`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"}) by (instance)`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"}) by (instance)`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m]))) by (instance)`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m]))) by (instance)`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m]))) by (instance)`</td></tr></table> |
| Summary | <table><tr><td>reading</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="reading"})`</td></tr><tr><td>waiting</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="waiting"})`</td></tr><tr><td>writing</td><td>`sum(nginx_ingress_controller_nginx_process_connections{state="writing"})`</td></tr><tr><td>accepted</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="accepted"}[5m])))`</td></tr><tr><td>active</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="active"}[5m])))`</td></tr><tr><td>handled</td><td>`sum(ceil(increase(nginx_ingress_controller_nginx_process_connections_total{state="handled"}[5m])))`</td></tr></table> |
### Ingress Controller Request Process Time
| Catalog | Expression |
| --- | --- |
| Detail | `topk(10, histogram_quantile(0.95,sum by (le, host, path)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |
| Summary | `topk(10, histogram_quantile(0.95,sum by (le, host)(rate(nginx_ingress_controller_request_duration_seconds_bucket{host!="_"}[5m]))))` |
## Rancher Logging Metrics
### Fluentd Buffer Queue Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_buffer_queue_length[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_output_status_buffer_queue_length[5m]))` |
### Fluentd Input Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_input_status_num_records_total[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_input_status_num_records_total[5m]))` |
### Fluentd Output Errors Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_num_errors[5m])) by (type)` |
| Summary | `sum(rate(fluentd_output_status_num_errors[5m]))` |
### Fluentd Output Rate
| Catalog | Expression |
| --- | --- |
| Detail | `sum(rate(fluentd_output_status_num_records_total[5m])) by (instance)` |
| Summary | `sum(rate(fluentd_output_status_num_records_total[5m]))` |
## Workload Metrics
### Workload CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `sum(container_memory_working_set_bytes{namespace="$namespace",pod_name=~"$podName", container_name!=""}) by (pod_name)` |
| Summary | `sum(container_memory_working_set_bytes{namespace="$namespace",pod_name=~"$podName", container_name!=""})` |
### Workload Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
### Workload Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m])) by (pod_name)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name=~"$podName",container_name!=""}[5m]))`</td></tr></table> |
## Pod Metrics
### Pod CPU Utilization
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m])) by (container_name)`</td></tr></table> |
| Summary | <table><tr><td>cfs throttled seconds</td><td>`sum(rate(container_cpu_cfs_throttled_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>usage seconds</td><td>`sum(rate(container_cpu_usage_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>system seconds</td><td>`sum(rate(container_cpu_system_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr><tr><td>user seconds</td><td>`sum(rate(container_cpu_user_seconds_total{container_name!="POD",namespace="$namespace",pod_name="$podName", container_name!=""}[5m]))`</td></tr></table> |
### Pod Memory Utilization
| Catalog | Expression |
| --- | --- |
| Detail | `sum(container_memory_working_set_bytes{container_name!="POD",namespace="$namespace",pod_name="$podName",container_name!=""}) by (container_name)` |
| Summary | `sum(container_memory_working_set_bytes{container_name!="POD",namespace="$namespace",pod_name="$podName",container_name!=""})` |
### Pod Network Packets
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>receive-packets</td><td>`sum(rate(container_network_receive_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-dropped</td><td>`sum(rate(container_network_receive_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>receive-errors</td><td>`sum(rate(container_network_receive_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-packets</td><td>`sum(rate(container_network_transmit_packets_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-dropped</td><td>`sum(rate(container_network_transmit_packets_dropped_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit-errors</td><td>`sum(rate(container_network_transmit_errors_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
### Pod Network I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
| Summary | <table><tr><td>receive</td><td>`sum(rate(container_network_receive_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>transmit</td><td>`sum(rate(container_network_transmit_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
### Pod Disk I/O
| Catalog | Expression |
| --- | --- |
| Detail | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m])) by (container_name)`</td></tr></table> |
| Summary | <table><tr><td>read</td><td>`sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr><tr><td>write</td><td>`sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name!=""}[5m]))`</td></tr></table> |
## Container Metrics
### Container CPU Utilization
| Catalog | Expression |
| --- | --- |
| cfs throttled seconds | `sum(rate(container_cpu_cfs_throttled_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| usage seconds | `sum(rate(container_cpu_usage_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| system seconds | `sum(rate(container_cpu_system_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| user seconds | `sum(rate(container_cpu_user_seconds_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
### Container Memory Utilization
`sum(container_memory_working_set_bytes{namespace="$namespace",pod_name="$podName",container_name="$containerName"})`
### Container Disk I/O
| Catalog | Expression |
| --- | --- |
| read | `sum(rate(container_fs_reads_bytes_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
| write | `sum(rate(container_fs_writes_bytes_total{namespace="$namespace",pod_name="$podName",container_name="$containerName"}[5m]))` |
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/rbac-for-monitoring.md
-169
View File
@@ -1,169 +0,0 @@
---
title: Role-based Access Control
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting/rbac-for-monitoring"/>
</head>
This section describes the expectations for RBAC for Rancher Monitoring.
## Cluster Admins
By default, only those with the cluster-admin `ClusterRole` should be able to:
- Install the `rancher-monitoring` App onto a cluster and all other relevant configuration performed on the chart deploy
- e.g. whether default dashboards are created, what exporters are deployed onto the cluster to collect metrics, etc.
- Create / modify / delete Prometheus deployments in the cluster via Prometheus CRs
- Create / modify / delete Alertmanager deployments in the cluster via Alertmanager CRs
- Persist new Grafana dashboards or datasources via creating ConfigMaps in the appropriate namespace
- Expose certain Prometheus metrics to the k8s Custom Metrics API for HPA via a Secret in the `cattle-monitoring-system` namespace
## Users with Kubernetes ClusterRole-based Permissions
The `rancher-monitoring` chart installs the following three `ClusterRoles`. By default, they aggregate into the corresponding k8s `ClusterRoles`:
| ClusterRole | Aggregates To Default K8s ClusterRole |
| ------------------------------| ---------------------------|
| `monitoring-admin` | `admin`|
| `monitoring-edit` | `edit` |
| `monitoring-view` | `view ` |
These `ClusterRoles` provide different levels of access to the Monitoring CRDs based on the actions that can be performed:
| CRDs (monitoring.coreos.com) | Admin | Edit | View |
| ------------------------------| ---------------------------| ---------------------------| ---------------------------|
| <ul><li>`prometheuses`</li><li>`alertmanagers`</li></ul>| Get, List, Watch | Get, List, Watch | Get, List, Watch |
| <ul><li>`servicemonitors`</li><li>`podmonitors`</li><li>`prometheusrules`</li></ul>| * | * | Get, List, Watch |
On a high level, the following permissions are assigned by default as a result.
### Users with Kubernetes Admin/Edit Permissions
Only those with the the cluster-admin, admin or edit `ClusterRole` should be able to:
- Modify the scrape configuration of Prometheus deployments via ServiceMonitor and PodMonitor CRs
- Modify the alerting / recording rules of a Prometheus deployment via PrometheusRules CRs
### Users with Kubernetes View Permissions
Only those with who have some Kubernetes `ClusterRole` should be able to:
- View the configuration of Prometheuses that are deployed within the cluster
- View the configuration of Alertmanagers that are deployed within the cluster
- View the scrape configuration of Prometheus deployments via ServiceMonitor and PodMonitor CRs
- View the alerting/recording rules of a Prometheus deployment via PrometheusRules CRs
### Additional Monitoring Roles
Monitoring also creates additional `Roles` that are not assigned to users by default but are created within the cluster. They can be bound to a namespace by deploying a `RoleBinding` that references it. To define a `RoleBinding` with `kubectl` instead of through Rancher, click [here](#assigning-roles-and-clusterroles-with-kubectl).
Admins should use these roles to provide more fine-grained access to users:
| Role | Purpose |
| ------------------------------| ---------------------------|
| monitoring-config-admin | Allow admins to assign roles to users to be able to view / modify Secrets and ConfigMaps within the cattle-monitoring-system namespace. Modifying Secrets / ConfigMaps in this namespace could allow users to alter the cluster's Alertmanager configuration, Prometheus Adapter configuration, additional Grafana datasources, TLS secrets, etc. |
| monitoring-config-edit | Allow admins to assign roles to users to be able to view / modify Secrets and ConfigMaps within the cattle-monitoring-system namespace. Modifying Secrets / ConfigMaps in this namespace could allow users to alter the cluster's Alertmanager configuration, Prometheus Adapter configuration, additional Grafana datasources, TLS secrets, etc. |
| monitoring-config-view | Allow admins to assign roles to users to be able to view Secrets and ConfigMaps within the cattle-monitoring-system namespace. Viewing Secrets / ConfigMaps in this namespace could allow users to observe the cluster's Alertmanager configuration, Prometheus Adapter configuration, additional Grafana datasources, TLS secrets, etc. |
| monitoring-dashboard-admin | Allow admins to assign roles to users to be able to edit / view ConfigMaps within the cattle-dashboards namespace. ConfigMaps in this namespace will correspond to Grafana Dashboards that are persisted onto the cluster. |
| monitoring-dashboard-edit | Allow admins to assign roles to users to be able to edit / view ConfigMaps within the cattle-dashboards namespace. ConfigMaps in this namespace will correspond to Grafana Dashboards that are persisted onto the cluster. |
| monitoring-dashboard-view | Allow admins to assign roles to users to be able to view ConfigMaps within the cattle-dashboards namespace. ConfigMaps in this namespace will correspond to Grafana Dashboards that are persisted onto the cluster. |
### Additional Monitoring ClusterRoles
Monitoring also creates additional `ClusterRoles` that are not assigned to users by default but are created within the cluster. They are not aggregated by default but can be bound to a namespace by deploying a `RoleBinding` or `ClusterRoleBinding` that references it. To define a `RoleBinding` with `kubectl` instead of through Rancher, click [here](#assigning-roles-and-clusterroles-with-kubectl).
| Role | Purpose |
| ------------------------------| ---------------------------|
| monitoring-ui-view | _Available as of Monitoring v2 14.5.100+_ Provides read-only access to external Monitoring UIs by giving a user permission to list the Prometheus, Alertmanager, and Grafana endpoints and make GET requests to Prometheus, Grafana, and Alertmanager UIs through the Rancher proxy. |
### Assigning Roles and ClusterRoles with kubectl
An alternative method to using Rancher to attach a `Role` or `ClusterRole` to a user or group is by defining bindings in YAML files that you create. You must first configure the `RoleBinding` with the YAML file, then you apply the config changes by running the `kubectl apply` command.
* **Roles**: Below is an example of a YAML file to help you configure `RoleBindings` in Kubernetes to attach to a user. You will need to fill in the name below, and name is case-sensitive.
```yaml
# monitoring-config-view-role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: monitoring-config-view
namespace: cattle-monitoring-system
roleRef:
kind: Role
name: monitoring-config-view
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: User
name: u-b4qkhsnliz # this can be found via `kubectl get users -A`
apiGroup: rbac.authorization.k8s.io
```
* **kubectl**: Below is an example of a `kubectl` command used to apply the binding you've created in the YAML file. As noted, you will need to fill in your YAML filename accordingly.
* **`kubectl apply -f monitoring-config-view-role-binding.yaml`
## Users with Rancher Cluster Manager Based Permissions
The relationship between the default roles deployed by Rancher Cluster Manager (i.e. cluster-owner, cluster-member, project-owner, project-member), the default k8s roles, and the roles deployed by the rancher-monitoring chart are detailed in the table below:
<figcaption>Default Rancher Permissions and Corresponding Kubernetes ClusterRoles</figcaption>
| Cluster Manager Role | k8s Role | Monitoring ClusterRole / Role | ClusterRoleBinding or RoleBinding? |
| --------- | --------- | --------- | --------- |
| cluster-owner | cluster-admin | N/A | ClusterRoleBinding |
| cluster-member | admin | monitoring-admin | ClusterRoleBinding |
| project-owner | admin | monitoring-admin | RoleBinding within Project namespace |
| project-member | edit | monitoring-edit | RoleBinding within Project namespace |
In addition to these default Roles, the following additional Rancher project roles can be applied to members of your Cluster to provide additional access to Monitoring. These Rancher Roles will be tied to ClusterRoles deployed by the Monitoring chart:
<figcaption>Non-default Rancher Permissions and Corresponding Kubernetes ClusterRoles</figcaption>
| Cluster Manager Role | Kubernetes ClusterRole | Available In Rancher From | Available in Monitoring v2 From |
|--------------------------|-------------------------------|-------|------|
| View Monitoring* | [monitoring-ui-view](#additional-monitoring-clusterroles) | 2.4.8+ | 9.4.204+ |
\* A User bound to the **View Monitoring** Rancher Role only has permissions to access external Monitoring UIs if provided links to those UIs. In order to access the Monitoring Pane on Cluster Explorer to get those links, the User must be a Project Member of at least one Project.
### Differences in 2.5.x
Users with the project-member or project-owners roles assigned will not be given access to either Prometheus or Grafana in Rancher 2.5.x since we only create Grafana or Prometheus on a cluster-level.
In addition, while project owners will still be only able to add ServiceMonitors / PodMonitors that scrape resources within their project's namespace by default, PrometheusRules are not scoped to a single namespace / project. Therefore, any alert rules or recording rules created by project-owners within their project namespace will be applied across the entire cluster, although they will be unable to view / edit / delete any rules that were created outside the project's namespace.
### Assigning Additional Access
If cluster-admins would like to provide additional admin/edit access to users outside of the roles offered by the rancher-monitoring chart, the following table identifies the potential impact:
|CRDs (monitoring.coreos.com) | Can it cause impact outside of a namespace / project? | Impact |
|----------------------------| ------| ----------------------------|
| `prometheuses`| Yes, this resource can scrape metrics from any targets across the entire cluster (unless the Operator itself is otherwise configured). | User will be able to define the configuration of new cluster-level Prometheus deployments that should be created in the cluster. |
| `alertmanagers`| No | User will be able to define the configuration of new cluster-level Alertmanager deployments that should be created in the cluster. Note: if you just want to allow users to configure settings like Routes and Receivers, you should just provide access to the Alertmanager Config Secret instead. |
| <ul><li>`servicemonitors`</li><li>`podmonitors`</li></ul>| No, not by default; this is configurable via `ignoreNamespaceSelectors` on the Prometheus CR. | User will be able to set up scrapes by Prometheus on endpoints exposed by Services / Pods within the namespace they are given this permission in. |
| `prometheusrules`| Yes, PrometheusRules are cluster-scoped. | User will be able to define alert or recording rules on Prometheus based on any series collected across the entire cluster. |
| k8s Resources | Namespace | Can it cause impact outside of a namespace / project? | Impact |
|----------------------------| ------| ------| ----------------------------|
| <ul><li>`secrets`</li><li>`configmaps`</li></ul>| `cattle-monitoring-system` | Yes, Configs and Secrets in this namespace can impact the entire monitoring / alerting pipeline. | User will be able to create or edit Secrets / ConfigMaps such as the Alertmanager Config, Prometheus Adapter Config, TLS secrets, additional Grafana datasources, etc. This can have broad impact on all cluster monitoring / alerting. |
| <ul><li>`secrets`</li><li>`configmaps`</li></ul>| `cattle-dashboards` | Yes, Configs and Secrets in this namespace can create dashboards that make queries on all metrics collected at a cluster-level. | User will be able to create Secrets / ConfigMaps that persist new Grafana Dashboards only. |
## Role-based Access Control for Grafana
Rancher allows any users who are authenticated by Kubernetes and have access the Grafana service deployed by the Rancher Monitoring chart to access Grafana via the Rancher Dashboard UI. By default, all users who are able to access Grafana are given the [Viewer](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#viewer-role) role, which allows them to view any of the default dashboards deployed by Rancher.
However, users can choose to log in to Grafana as an [Admin](https://grafana.com/docs/grafana/latest/permissions/organization_roles/#admin-role) if necessary. The default Admin username and password for the Grafana instance will be `admin`/`prom-operator`, but alternative credentials can also be supplied on deploying or upgrading the chart.
To see the Grafana UI, install `rancher-monitoring`. Then go to the **Cluster Explorer.** In the top left corner, click **Cluster Explorer > Monitoring.** Then click **Grafana.
<figcaption>Cluster Compute Resources Dashboard in Grafana</figcaption>
![Cluster Compute Resources Dashboard in Grafana](/img/cluster-compute-resources-dashboard.png)
<figcaption>Default Dashboards in Grafana</figcaption>
![Default Dashboards in Grafana](/img/grafana-default-dashboard.png)
versioned_docs/version-2.5/explanations/integrations-in-rancher/monitoring-and-alerting/windows-support.md
-55
View File
@@ -1,55 +0,0 @@
---
title: Windows Cluster Support for Monitoring V2
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/monitoring-and-alerting/windows-support"/>
</head>
_Available as of v2.5.8_
Starting at Monitoring V2 14.5.100 (used by default in Rancher 2.5.8), Monitoring V2 can now be deployed on a Windows cluster and will scrape metrics from Windows nodes using [prometheus-community/windows_exporter](https://github.com/prometheus-community/windows_exporter) (previously named `wmi_exporter`).
## Comparison to Monitoring V1
Unlike Monitoring V1 for Windows, metrics collected by `windows_exporter` will be labeled as `windows_` instead of `wmi_` in accordance to a naming change from upstream from `wmi_exporter` to `windows_exporter`.
In addition, Monitoring V2 for Windows will no longer require users to keep port 9796 open on Windows hosts since the host metrics will published directly onto a port exposed on the windows-exporter Pod. This feature was powered by recent changes made by `wins` v0.1.0 to support publishing ports exposed on the hostNetwork on Pods that use wins to run a privileged Windows binary as a host process.
## Cluster Requirements
Monitoring V2 for Windows can only scrape metrics from Windows hosts that have a minimum `wins` version of v0.1.0. To be able to fully deploy Monitoring V2 for Windows, all of your hosts must meet this requirement.
If you provision a fresh RKE1 cluster in Rancher 2.5.8, your cluster should already meet this requirement.
### Upgrading Existing Clusters to wins v0.1.0
If the cluster was provisioned before Rancher 2.5.8 (even if the current Rancher version is 2.5.8), you will not be able to successfully deploy Monitoring V2 for Windows until you upgrade the wins version on each host to at least v0.1.0.
To facilitate this upgrade, Rancher 2.5.8 has released a brand new Helm chart called `rancher-wins-upgrader`.
> **Prerequisite:** Make sure Monitoring V1 for Windows is uninstalled.
1. Deploy `rancher-wins-upgrader` with the following override:
```yaml
# Masquerading bootstraps the wins-upgrader installation via
# a previously whitelisted process path since the normal install path,
# c:\etc\rancher\wins\wins-upgrade.exe is not normally whitelisted.
# In this case, we are using the previously whitelisted process
# path used by Monitoring V1.
masquerade:
enabled: true
as: c:\\etc\wmi-exporter\wmi-exporter.exe
```
> **Note for Non-Default Windows Prefix Path:** If you set up the RKE cluster with a `cluster.yml` that has a non-default `win_prefix_path`, you will need to update the `masquerade.as` field with your prefix path in place of `c:\\`.
>
> For example, if you have `win_prefix_path: 'c:\host\opt\'`, then you will need to set `as: c:\host\opt\etc\wmi-exporter\wmi-exporter.exe`.
2. Once all your hosts have been successfully upgraded, please ensure that you deploy the Helm chart once again with default values to avoid conflicts with the following settings:
```yaml
masquerade:
enabled: false
```
**Result:** The hosts are ready for Monitoring V2 to be installed. You may choose to uninstall the `rancher-wins-upgrader` chart or keep it in your cluster to facilitate future upgrades.
For more information on how it can be used, please see the [README.md](https://github.com/rancher/wins/blob/master/charts/rancher-wins-upgrader/README.md) of the chart.
versioned_docs/version-2.5/explanations/integrations-in-rancher/opa-gatekeeper.md
-111
View File
@@ -1,111 +0,0 @@
---
title: OPA Gatekeeper
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/integrations-in-rancher/opa-gatekeeper"/>
</head>
To ensure consistency and compliance, every organization needs the ability to define and enforce policies in its environment in an automated way. [OPA (Open Policy Agent)](https://www.openpolicyagent.org/) is a policy engine that facilitates policy-based control for cloud native environments. Rancher provides the ability to enable OPA Gatekeeper in Kubernetes clusters, and also installs a couple of built-in policy definitions, which are also called constraint templates.
OPA provides a high-level declarative language that lets you specify policy as code and ability to extend simple APIs to offload policy decision-making.
[OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is a project that provides integration between OPA and Kubernetes. OPA Gatekeeper provides:
- An extensible, parameterized policy library.
- Native Kubernetes CRDs for instantiating the policy library, also called “constraints."
- Native Kubernetes CRDs for extending the policy library, also called "constraint templates."
- Audit functionality.
To read more about OPA, please refer to the [official documentation.](https://www.openpolicyagent.org/docs/latest/)
## How the OPA Gatekeeper Integration Works
Kubernetes provides the ability to extend API server functionality via admission controller webhooks, which are invoked whenever a resource is created, updated or deleted. Gatekeeper is installed as a validating webhook and enforces policies defined by Kubernetes custom resource definitions. In addition to the admission control usage, Gatekeeper provides the capability to audit existing resources in Kubernetes clusters and mark current violations of enabled policies.
OPA Gatekeeper is made available via Rancher's Helm system chart, and it is installed in a namespace named `gatekeeper-system.`
## Enabling OPA Gatekeeper in a Cluster
> In Rancher v2.5, the OPA Gatekeeper application was improved. The Rancher v2.4 feature can't be upgraded to the new version in Rancher v2.5. If you installed OPA Gatekeeper in Rancher v2.4, you will need to uninstall OPA Gatekeeper and its CRDs from the old UI, then reinstall it in Rancher v2.5. To uninstall the CRDs run the following command in the kubectl console `kubectl delete crd configs.config.gatekeeper.sh constrainttemplates.templates.gatekeeper.sh`.
> **Prerequisite:** Only administrators and cluster owners can enable OPA Gatekeeper.
OPA Gatekeeper can be installed from the new **Cluster Explorer** view in Rancher v2.5, or from the cluster manager view.
### Enabling OPA Gatekeeper from Cluster Explorer
1. Go to the cluster view in the Rancher UI. Click **Cluster Explorer.**
1. Click **Apps** in the top navigation bar.
1. Click **rancher-gatekeeper.**
1. Click **Install.**
**Result:** OPA Gatekeeper is deployed in your Kubernetes cluster.
### Enabling OPA Gatekeeper from the Cluster Manager View
1. Go to the cluster view in the Rancher UI.
1. Click **Tools > OPA Gatekeeper.**
1. Click **Install.**
**Result:** OPA Gatekeeper is deployed in your Kubernetes cluster.
## Constraint Templates
[Constraint templates](https://github.com/open-policy-agent/gatekeeper#constraint-templates) are Kubernetes custom resources that define the schema and Rego logic of the OPA policy to be applied by Gatekeeper. For more information on the Rego policy language, refer to the [official documentation.](https://www.openpolicyagent.org/docs/latest/policy-language/)
When OPA Gatekeeper is enabled, Rancher installs some templates by default.
To list the constraint templates installed in the cluster, go to the left side menu under OPA Gatekeeper and click on **Templates.**
Rancher also provides the ability to create your own constraint templates by importing YAML definitions.
## Creating and Configuring Constraints
[Constraints](https://github.com/open-policy-agent/gatekeeper#constraints) are Kubernetes custom resources that define the scope of objects to which a specific constraint template applies to. The complete policy is defined by constraint templates and constraints together.
> **Prerequisites:** OPA Gatekeeper must be enabled in the cluster.
To list the constraints installed, go to the left side menu under OPA Gatekeeper, and click on **Constraints.**
New constraints can be created from a constraint template.
Rancher provides the ability to create a constraint by using a convenient form that lets you input the various constraint fields.
The **Edit as yaml** option is also available to configure the the constraint's yaml definition.
### Exempting Rancher's System Namespaces from Constraints
When a constraint is created, ensure that it does not apply to any Rancher or Kubernetes system namespaces. If the system namespaces are not excluded, then it is possible to see many resources under them marked as violations of the constraint.
To limit the scope of the constraint only to user namespaces, always specify these namespaces under the **Match** field of the constraint.
Also, the constraint may interfere with other Rancher functionality and deny system workloads from being deployed. To avoid this, exclude all Rancher-specific namespaces from your constraints.
## Enforcing Constraints in your Cluster
When the **Enforcement Action** is **Deny,** the constraint is immediately enabled and will deny any requests that violate the policy defined. By default, the enforcement value is **Deny.**
When the **Enforcement Action** is **Dryrun,** then any resources that violate the policy are only recorded under the constraint's status field.
To enforce constraints, create a constraint using the form. In the **Enforcement Action** field, choose **Deny.**
## Audit and Violations in your Cluster
OPA Gatekeeper runs a periodic audit to check if any existing resource violates any enforced constraint. The audit-interval (default 300s) can be configured while installing Gatekeeper.
On the Gatekeeper page, any violations of the defined constraints are listed.
Also under **Constraints,** the number of violations of the constraint can be found.
The detail view of each constraint lists information about the resource that violated the constraint.
## Disabling Gatekeeper
1. Navigate to the cluster's Dashboard view
1. On the left side menu, expand the cluster menu and click on **OPA Gatekeeper.**
1. Click the **&#8942; > Disable**.
**Result:** Upon disabling OPA Gatekeeper, all constraint templates and constraints will also be deleted.
versioned_docs/version-2.5/faq.md
-73
View File
@@ -1,73 +0,0 @@
---
title: FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/general-faq"/>
</head>
This FAQ is a work in progress designed to answer the questions our users most frequently ask about Rancher v2.x.
See [Technical FAQ](faq/technical-items.md), for frequently asked technical questions.
<br/>
**Does Rancher v2.x support Docker Swarm and Mesos as environment types?**
When creating an environment in Rancher v2.x, Swarm and Mesos will no longer be standard options you can select. However, both Swarm and Mesos will continue to be available as Catalog applications you can deploy. It was a tough decision to make but, in the end, it came down to adoption. For example, out of more than 15,000 clusters, only about 200 or so are running Swarm.
<br/>
**Is it possible to manage Azure Kubernetes Services with Rancher v2.x?**
Yes.
<br/>
**Does Rancher support Windows?**
As of Rancher 2.3.0, we support Windows Server 1809 containers. For details on how to set up a cluster with Windows worker nodes, refer to the section on [configuring custom clusters for Windows.](how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-windows-clusters/use-windows-clusters.md)
<br/>
**Does Rancher support Istio?**
As of Rancher 2.3.0, we support [Istio.](explanations/integrations-in-rancher/istio/istio.md)
Furthermore, Istio is implemented in our micro-PaaS "Rio", which works on Rancher 2.x along with any CNCF compliant Kubernetes cluster. You can read more about it [here](https://rio.io/)
<br/>
**Will Rancher v2.x support Hashicorp's Vault for storing secrets?**
There is no built-in integration of Rancher and Hashicorp's Vault. Rancher manages Kubernetes and integrates with secrets via the Kubernetes API. Thus in any downstream (managed) cluster, you can use a secret vault of your choice provided it integrates with Kubernetes, including [Vault](https://www.vaultproject.io/docs/platform/k8s).
<br/>
**Does Rancher v2.x support RKT containers as well?**
At this time, we only support Docker.
<br/>
**Does Rancher v2.x support Calico, Contiv, Contrail, Flannel, Weave net, etc., for embedded and registered Kubernetes?**
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico and Weave. Always refer to the [Rancher Support Matrix](https://rancher.com/support-maintenance-terms/) for details about what is officially supported.
<br/>
**Are you planning on supporting Traefik for existing setups?**
We don't currently plan on providing embedded Traefik support, but we're still exploring load-balancing approaches.
<br/>
**Can I import OpenShift Kubernetes clusters into v2.x?**
Our goal is to run any upstream Kubernetes clusters. Therefore, Rancher v2.x should work with OpenShift, but we haven't tested it yet.
<br/>
**Are you going to integrate Longhorn?**
Yes. Longhorn was integrated into Rancher v2.5+.
versioned_docs/version-2.5/faq/container-network-interface-providers.md
-147
View File
@@ -1,147 +0,0 @@
---
title: Container Network Interface (CNI) Providers
description: Learn about Container Network Interface (CNI), the CNI providers Rancher provides, the features they offer, and how to choose a provider for you
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/container-network-interface-providers"/>
</head>
## What is CNI?
CNI (Container Network Interface), a [Cloud Native Computing Foundation project](https://cncf.io/), consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of plugins. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.
Kubernetes uses CNI as an interface between network providers and Kubernetes pod networking.
![CNI Logo](/img/cni-logo.png)
For more information visit [CNI GitHub project](https://github.com/containernetworking/cni).
### What Network Models are Used in CNI?
CNI network providers implement their network fabric using either an encapsulated network model such as Virtual Extensible Lan ([VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan)) or an unencapsulated network model such as Border Gateway Protocol ([BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)).
#### What is an Encapsulated Network?
This network model provides a logical Layer 2 (L2) network encapsulated over the existing Layer 3 (L3) network topology that spans the Kubernetes cluster nodes. With this model you have an isolated L2 network for containers without needing routing distribution, all at the cost of minimal overhead in terms of processing and increased IP package size, which comes from an IP header generated by overlay encapsulation. Encapsulation information is distributed by UDP ports between Kubernetes workers, interchanging network control plane information about how MAC addresses can be reached. Common encapsulation used in this kind of network model is VXLAN, Internet Protocol Security (IPSec), and IP-in-IP.
In simple terms, this network model generates a kind of network bridge extended between Kubernetes workers, where pods are connected.
This network model is used when an extended L2 bridge is preferred. This network model is sensitive to L3 network latencies of the Kubernetes workers. If datacenters are in distinct geolocations, be sure to have low latencies between them to avoid eventual network segmentation.
CNI network providers using this network model include Flannel, Canal, and Weave.
![Encapsulated Network](/img/encapsulated-network.png)
#### What is an Unencapsulated Network?
This network model provides an L3 network to route packets between containers. This model doesn't generate an isolated l2 network, nor generates overhead. These benefits come at the cost of Kubernetes workers having to manage any route distribution that's needed. Instead of using IP headers for encapsulation, this network model uses a network protocol between Kubernetes workers to distribute routing information to reach pods, such as [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol).
In simple terms, this network model generates a kind of network router extended between Kubernetes workers, which provides information about how to reach pods.
This network model is used when a routed L3 network is preferred. This mode dynamically updates routes at the OS level for Kubernetes workers. It's less sensitive to latency.
CNI network providers using this network model include Calico and Romana.
![Unencapsulated Network](/img/unencapsulated-network.png)
### What CNI Providers are Provided by Rancher?
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico, and Weave. You can choose your CNI network provider when you create new Kubernetes clusters from Rancher.
#### Canal
![Canal Logo](/img/canal-logo.png)
Canal is a CNI network provider that gives you the best of Flannel and Calico. It allows users to easily deploy Calico and Flannel networking together as a unified networking solution, combining Calico’s network policy enforcement with the rich superset of Calico (unencapsulated) and/or Flannel (encapsulated) network connectivity options.
In Rancher, Canal is the default CNI network provider combined with Flannel and VXLAN encapsulation.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). For details, refer to [the port requirements for user clusters.](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md)
![](/img/canal-diagram.png)
For more information, see the [Canal GitHub Page.](https://github.com/projectcalico/canal)
#### Flannel
![Flannel Logo](/img/flannel-logo.png)
Flannel is a simple and easy way to configure L3 network fabric designed for Kubernetes. Flannel runs a single binary agent named flanneld on each host, which is responsible for allocating a subnet lease to each host out of a larger, preconfigured address space. Flannel uses either the Kubernetes API or etcd directly to store the network configuration, the allocated subnets, and any auxiliary data (such as the host's public IP). Packets are forwarded using one of several backend mechanisms, with the default encapsulation being [VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan).
Encapsulated traffic is unencrypted by default. Therefore, flannel provides an experimental backend for encryption, [IPSec](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#ipsec), which makes use of [strongSwan](https://www.strongswan.org/) to establish encrypted IPSec tunnels between Kubernetes workers.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
![Flannel Diagram](/img/flannel-diagram.png)
For more information, see the [Flannel GitHub Page](https://github.com/coreos/flannel).
#### Calico
![Calico Logo](/img/calico-logo.png)
Calico enables networking and network policy in Kubernetes clusters across the cloud. Calico uses a pure, unencapsulated IP network fabric and policy engine to provide networking for your Kubernetes workloads. Workloads are able to communicate over both cloud infrastructure and on-prem using BGP.
Calico also provides a stateless IP-in-IP encapsulation mode that can be used, if necessary. Calico also offers policy isolation, allowing you to secure and govern your Kubernetes workloads using advanced ingress and egress policies.
Kubernetes workers should open TCP port `179` (BGP). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
![Calico Diagram](/img/calico-diagram.svg)
For more information, see the following pages:
- [Project Calico Official Site](https://www.projectcalico.org/)
- [Project Calico GitHub Page](https://github.com/projectcalico/calico)
#### Weave
![Weave Logo](/img/weave-logo.png)
Weave enables networking and network policy in Kubernetes clusters across the cloud. Additionally, it support encrypting traffic between the peers.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See the [port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
For more information, see the following pages:
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### CNI Features by Provider
The following table summarizes the different features available for each CNI network provider provided by Rancher.
| Provider | Network Model | Route Distribution | Network Policies | Mesh | External Datastore | Encryption | Ingress/Egress Policies |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| Canal | Encapsulated (VXLAN) | No | Yes | No | K8S API | No | Yes |
| Flannel | Encapsulated (VXLAN) | No | No | No | K8S API | No | No |
| Calico | Encapsulated (VXLAN,IPIP) OR Unencapsulated | Yes | Yes | Yes | Etcd and K8S API | No | Yes |
| Weave | Encapsulated | Yes | Yes | Yes | No | Yes | Yes |
- Network Model: Encapsulated or unencapsulated. For more information, see [What Network Models are Used in CNI?](#what-network-models-are-used-in-cni)
- Route Distribution: An exterior gateway protocol designed to exchange routing and reachability information on the Internet. BGP can assist with pod-to-pod networking between clusters. This feature is a must on unencapsulated CNI network providers, and it is typically done by BGP. If you plan to build clusters split across network segments, route distribution is a feature that's nice-to-have.
- Network Policies: Kubernetes offers functionality to enforce rules about which services can communicate with each other using network policies. This feature is stable as of Kubernetes v1.7 and is ready to use with certain networking plugins.
- Mesh: This feature allows service-to-service networking communication between distinct Kubernetes clusters.
- External Datastore: CNI network providers with this feature need an external datastore for its data.
- Encryption: This feature allows cyphered and secure network control and data planes.
- Ingress/Egress Policies: This feature allows you to manage routing control for both Kubernetes and non-Kubernetes communications.
### CNI Community Popularity
<CNIPopularityTable />
### Which CNI Provider Should I Use?
It depends on your project needs. There are many different providers, which each have various features and options. There isn't one provider that meets everyone's needs.
Canal is the default CNI network provider. We recommend it for most use cases. It provides encapsulated networking for containers with Flannel, while adding Calico network policies that can provide project/namespace isolation in terms of networking.
### How can I configure a CNI network provider?
Please see [Cluster Options](cluster-provisioning/rke-clusters/options/) on how to configure a network provider for your cluster. For more advanced configuration options, please see how to configure your cluster using a [Config File](cluster-provisioning/rke-clusters/options/#cluster-config-file) and the options for [Network Plug-ins](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/).
versioned_docs/version-2.5/faq/deprecated-features.md
-19
View File
@@ -1,19 +0,0 @@
---
title: Deprecated Features in Rancher v2.5
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/deprecated-features"/>
</head>
## What is Rancher's deprecation policy?
Starting in Rancher 2.5 we have published our official deprecation policy in the support [terms of service](https://rancher.com/support-maintenance-terms).
## Where can I find out which features have been deprecated in Rancher 2.5?
Rancher will publish deprecated features as part of the [release notes](https://github.com/rancher/rancher/releases/tag/v2.5.0) for Rancher found on GitHub.
## What can I expect when a feature is marked for deprecation?
In the release where functionality is marked as Deprecated it will still be available and supported allowing upgrades to follow the usual procedure. Once upgraded, users/admins should start planning to move away from the deprecated functionality before upgrading to the release it marked as removed. The recommendation for new deployments is to not use the deprecated feature.
versioned_docs/version-2.5/faq/install-and-configure-kubectl.md
-33
View File
@@ -1,33 +0,0 @@
---
title: Installing and Configuring kubectl
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/install-and-configure-kubectl"/>
</head>
`kubectl` is a CLI utility for running commands against Kubernetes clusters. It's required for many maintenance and administrative tasks in Rancher 2.x.
## Installation
See [kubectl Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation on your operating system.
## Configuration
When you create a Kubernetes cluster with RKE, RKE creates a `kube_config_cluster.yml` in the local directory that contains credentials to connect to your new cluster with tools like `kubectl` or `helm`.
You can copy this file to `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_cluster.yml`.
```
export KUBECONFIG=$(pwd)/kube_config_cluster.yml
```
Test your connectivity with `kubectl` and see if you can get the list of nodes back.
```
kubectl get nodes
NAME STATUS ROLES AGE VERSION
165.227.114.63 Ready controlplane,etcd,worker 11m v1.10.1
165.227.116.167 Ready controlplane,etcd,worker 11m v1.10.1
165.227.127.226 Ready controlplane,etcd,worker 11m v1.10.1
```
versioned_docs/version-2.5/faq/networking/networking.md
-8
View File
@@ -1,8 +0,0 @@
---
title: Networking
---
Networking FAQ's
- [CNI Providers](../container-network-interface-providers.md)
versioned_docs/version-2.5/faq/rancher-is-no-longer-needed.md
-59
View File
@@ -1,59 +0,0 @@
---
title: Rancher is No Longer Needed
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/rancher-is-no-longer-needed"/>
</head>
This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted.
## If the Rancher server is deleted, what happens to the workloads in my downstream clusters?
If Rancher is ever deleted or unrecoverable, all workloads in the downstream Kubernetes clusters managed by Rancher will continue to function as normal.
## If the Rancher server is deleted, how do I access my downstream clusters?
The capability to access a downstream cluster without Rancher depends on the type of cluster and the way that the cluster was created. To summarize:
- **Registered clusters:** The cluster will be unaffected and you can access the cluster using the same methods that you did before the cluster was registered into Rancher.
- **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
- **RKE clusters:** Please note that you will no longer be able to manage the individual Kubernetes components or perform any upgrades on them after the deletion of the Rancher server. However, you can still access the cluster to manage your workloads. To access an [RKE cluster,](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) the cluster must have the [authorized cluster endpoint](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.](../how-to-guides/advanced-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
## What if I don't want Rancher anymore?
If you [installed Rancher on a Kubernetes cluster,](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) remove Rancher by using the [System Tools](../reference-guides/system-tools.md) with the `remove` subcommand.
As of Rancher v2.5.8, uninstalling Rancher in high-availability (HA) mode will also remove all `helm-operation-*` pods and the following apps:
- fleet
- fleet-agent
- rancher-operator
- rancher-webhook
Custom resources (CRDs) and custom namespaces will still need to be manually removed.
If you installed Rancher with Docker, you can uninstall Rancher by removing the single Docker container that it runs in.
Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
## What if I don't want my registered cluster managed by Rancher?
If a registered cluster is deleted from the Rancher UI, the cluster is detached from Rancher, leaving it intact and accessible by the same methods that were used to access it before it was registered in Rancher.
To detach the cluster,
1. From the **Global** view in Rancher, go to the **Clusters** tab.
2. Go to the registered cluster that should be detached from Rancher and click **&#8942; > Delete.**
3. Click **Delete.**
**Result:** The registered cluster is detached from Rancher and functions normally outside of Rancher.
## What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?
At this time, there is no functionality to detach these clusters from Rancher. In this context, "detach" is defined as the ability to remove Rancher components from the cluster and manage access to the cluster independently of Rancher.
The capability to manage these clusters without Rancher is being tracked in this [issue.](https://github.com/rancher/rancher/issues/25234)
For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
versioned_docs/version-2.5/faq/security.md
-15
View File
@@ -1,15 +0,0 @@
---
title: Security FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/security"/>
</head>
## Is there a Hardening Guide?
The Hardening Guide is now located in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
## What are the results of Rancher's Kubernetes cluster when it is CIS benchmarked?
We have run the CIS Kubernetes benchmark against a hardened Rancher Kubernetes cluster. The results of that assessment can be found in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
versioned_docs/version-2.5/faq/technical-items.md
-186
View File
@@ -1,186 +0,0 @@
---
title: Technical FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/technical-items"/>
</head>
## How can I reset the administrator password?
Docker install:
```
$ docker exec -ti <container_id> reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password
New password for default administrator (user-xxxxx):
<new_password>
```
## I deleted/deactivated the last admin, how can I fix it?
Docker install:
```
$ docker exec -ti <container_id> ensure-default-admin
New default administrator (user-xxxxx)
New password for default administrator (user-xxxxx):
<new_password>
```
Kubernetes install (Helm):
```
$ KUBECONFIG=./kube_config_cluster.yml
$ kubectl --kubeconfig $KUBECONFIG -n cattle-system exec $(kubectl --kubeconfig $KUBECONFIG -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- ensure-default-admin
New password for default administrator (user-xxxxx):
<new_password>
```
## How can I enable debug logging?
See [Troubleshooting: Logging](../troubleshooting/other-troubleshooting-tips/logging.md)
## My ClusterIP does not respond to ping
ClusterIP is a virtual IP, which will not respond to ping. Best way to test if the ClusterIP is configured correctly, is by using `curl` to access the IP and port to see if it responds.
## Where can I manage Node Templates?
Node Templates can be accessed by opening your account menu (top right) and selecting `Node Templates`.
## Why is my Layer-4 Load Balancer in `Pending` state?
The Layer-4 Load Balancer is created as `type: LoadBalancer`. In Kubernetes, this needs a cloud provider or controller that can satisfy these requests, otherwise these will be in `Pending` state forever. More information can be found on [Cloud Providers](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/set-up-cloud-providers/set-up-cloud-providers.md) or [Create External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)
## Where is the state of Rancher stored?
- Docker Install: in the embedded etcd of the `rancher/rancher` container, located at `/var/lib/rancher`.
- Kubernetes install: in the etcd of the RKE cluster created to run Rancher.
## How are the supported Docker versions determined?
We follow the validated Docker versions for upstream Kubernetes releases. The validated versions can be found under [External Dependencies](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#external-dependencies) in the Kubernetes release CHANGELOG.md.
## How can I access nodes created by Rancher?
SSH keys to access the nodes created by Rancher can be downloaded via the **Nodes** view. Choose the node which you want to access and click on the vertical &#8942; button at the end of the row, and choose **Download Keys** as shown in the picture below.
![Download Keys](/img/downloadsshkeys.png)
Unzip the downloaded zip file, and use the file `id_rsa` to connect to you host. Be sure to use the correct username (`rancher` or `docker` for RancherOS, `ubuntu` for Ubuntu, `ec2-user` for Amazon Linux)
```
$ ssh -i id_rsa user@ip_of_node
```
## How can I automate task X in Rancher?
The UI consists of static files, and works based on responses of the API. That means every action/task that you can execute in the UI, can be automated via the API. There are 2 ways to do this:
* Visit `https://your_rancher_ip/v3` and browse the API options.
* Capture the API calls when using the UI (Most commonly used for this is [Chrome Developer Tools](https://developers.google.com/web/tools/chrome-devtools/#network) but you can use anything you like)
## The IP address of a node changed, how can I recover?
A node is required to have a static IP configured (or a reserved IP via DHCP). If the IP of a node has changed, you will have to remove it from the cluster and readd it. After it is removed, Rancher will update the cluster to the correct state. If the cluster is no longer in `Provisioning` state, the node is removed from the cluster.
When the IP address of the node changed, Rancher lost connection to the node, so it will be unable to clean the node properly. See [Cleaning cluster nodes](../how-to-guides/advanced-user-guides/manage-clusters/clean-cluster-nodes.md) to clean the node.
When the node is removed from the cluster, and the node is cleaned, you can readd the node to the cluster.
## How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?
You can add more arguments/binds/environment variables via the [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#editing-clusters-with-yaml) option in Cluster Options. For more information, see the [Extra Args, Extra Binds, and Extra Environment Variables](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/) in the RKE documentation or browse the [Example Cluster.ymls](https://rancher.com/docs/rke/latest/en/example-yamls/).
## How do I check if my certificate chain is valid?
Use the `openssl verify` command to validate your certificate chain:
:::tip
Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS installed certificates are not used when verifying manually.
:::
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you receive the error `unable to get local issuer certificate`, the chain is incomplete. This usually means that there is an intermediate CA certificate that issued your server certificate. If you already have this certificate, you can use it in the verification of the certificate like shown below:
```
SSL_CERT_DIR=/dummy SSL_CERT_FILE=/dummy openssl verify -CAfile ca.pem -untrusted intermediate.pem rancher.yourdomain.com.pem
rancher.yourdomain.com.pem: OK
```
If you have successfully verified your certificate chain, you should include needed intermediate CA certificates in the server certificate to complete the certificate chain for any connection made to Rancher (for example, by the Rancher agent). The order of the certificates in the server certificate file should be first the server certificate itself (contents of `rancher.yourdomain.com.pem`), followed by intermediate CA certificate(s) (contents of `intermediate.pem`).
```
-----BEGIN CERTIFICATE-----
%YOUR_CERTIFICATE%
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
%YOUR_INTERMEDIATE_CERTIFICATE%
-----END CERTIFICATE-----
```
If you still get errors during verification, you can retrieve the subject and the issuer of the server certificate using the following command:
```
openssl x509 -noout -subject -issuer -in rancher.yourdomain.com.pem
subject= /C=GB/ST=England/O=Alice Ltd/CN=rancher.yourdomain.com
issuer= /C=GB/ST=England/O=Alice Ltd/CN=Alice Intermediate CA
```
## How do I check `Common Name` and `Subject Alternative Names` in my server certificate?
Although technically an entry in `Subject Alternative Names` is required, having the hostname in both `Common Name` and as entry in `Subject Alternative Names` gives you maximum compatibility with older browser/applications.
Check `Common Name`:
```
openssl x509 -noout -subject -in cert.pem
subject= /CN=rancher.my.org
```
Check `Subject Alternative Names`:
```
openssl x509 -noout -in cert.pem -text | grep DNS
DNS:rancher.my.org
```
## Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?
This is due to a combination of the following default Kubernetes settings:
* kubelet
* `node-status-update-frequency`: Specifies how often kubelet posts node status to master (default 10s)
* kube-controller-manager
* `node-monitor-period`: The period for syncing NodeStatus in NodeController (default 5s)
* `node-monitor-grace-period`: Amount of time which we allow running Node to be unresponsive before marking it unhealthy (default 40s)
* `pod-eviction-timeout`: The grace period for deleting pods on failed nodes (default 5m0s)
See [Kubernetes: kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) and [Kubernetes: kube-controller-manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) for more information on these settings.
In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. See [Kubernetes: Taint based Evictions](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) for more information.
* kube-apiserver (Kubernetes v1.13 and up)
* `default-not-ready-toleration-seconds`: Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.
* `default-unreachable-toleration-seconds`: Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.
## Can I use keyboard shortcuts in the UI?
Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
## What does `Unknown schema for type:` errors followed by something like `catalog.cattle.io.operation` mean when trying to modify an App?
This error occurs when Kubernetes can not find the CRD mentioned. The vast majority of the time these are a result of missing RBAC permissions. Try with an admin user and if this works, add permissions for the resource mentioned by the error (ie. `Get`, `List`, `Patch` as needed).
versioned_docs/version-2.5/faq/telemetry.md
-35
View File
@@ -1,35 +0,0 @@
---
title: Telemetry FAQ
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/faq/telemetry"/>
</head>
## What is Telemetry?
Telemetry collects aggregate information about the size of Rancher installations, versions of components used, and which features are used. This information is used by Rancher Labs to help make the product better and is not shared with third-parties.
## What information is collected?
No specific identifying information like usernames, passwords, or the names or addresses of user resources will ever be collected.
The primary things collected include:
- Aggregate counts (smallest, average, largest, total) of nodes per-cluster and their size (e.g. CPU cores & RAM).
- Aggregate counts of logical resources like Clusters, Projects, Namespaces, and Pods.
- Counts of what driver was used to deploy clusters and nodes (e.g. GKE vs EC2 vs Imported vs Custom).
- Versions of Kubernetes components, Operating Systems and Docker that are deployed on nodes.
- Whether some optional components are enabled or not (e.g. which auth providers are used).
- The image name & version of Rancher that is running.
- A unique randomly-generated identifier for this installation.
## Can I see the information that is being sent?
If Telemetry is enabled, you can go to `https://<your rancher server>/v1-telemetry` in your installation to see the current data.
If Telemetry is not enabled, the process that collects the data is not running, so there is nothing being collected to look at.
## How do I turn it on or off?
After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
versioned_docs/version-2.5/getting-started.md
-9
View File
@@ -1,9 +0,0 @@
---
title: Getting Started
---
To get up and running with Rancher quickly, we have included a **Getting Started** section.
The goal of this section is to be able to assist users in deploying Rancher and workloads and to install or upgrade Rancher quickly and effectively.
Please see the [introduction](getting-started/introduction/introduction.md), [quick start guides](getting-started/quick-start-guides/quick-start-guides.md), and the [installation and upgrade](getting-started/installation-and-upgrade/installation-and-upgrade.md) sections for more.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/advanced-options.md
-9
View File
@@ -1,9 +0,0 @@
---
title: Advanced
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/advanced-user-guides"/>
</head>
The documents in this section contain resources for less common use cases.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/advanced-use-cases/configure-layer-7-nginx-load-balancer.md
-246
View File
@@ -1,246 +0,0 @@
---
title: Docker Install with TLS Termination at Layer-7 NGINX Load Balancer
---
For development and testing environments that have a special requirement to terminate TLS/SSL at a load balancer instead of your Rancher Server container, deploy Rancher and configure a load balancer to work with it conjunction.
A layer-7 load balancer can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with.
This install procedure walks you through deployment of Rancher using a single container, and then provides a sample configuration for a layer-7 NGINX load balancer.
> **Want to skip the external load balancer?**
> See [Docker Installation](../../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) instead.
## Requirements for OS, Docker, Hardware, and Networking
Make sure that your node fulfills the general [installation requirements.](../../installation-requirements/installation-requirements.md)
## 1. Provision Linux Host
Provision a single Linux host according to our [Requirements](../../installation-requirements/installation-requirements.md) to launch your Rancher Server.
## 2. Choose an SSL Option and Install Rancher
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
> **Do you want to...**
>
> - Complete an Air Gap Installation?
> - Record all transactions with the Rancher API?
>
> See [Advanced Options](#advanced-options) below before continuing.
Choose from the following options:
<details id="option-a">
<summary>Option A-Bring Your Own Certificate: Self-Signed</summary>
If you elect to use a self-signed certificate to encrypt communication, you must install the certificate on your load balancer (which you'll do later) and your Rancher container. Run the Docker command to deploy Rancher, pointing it toward your certificate.
> **Prerequisites:**
> Create a self-signed certificate.
>
> - The certificate files must be in PEM format.
**To Install Rancher Using a Self-Signed Cert:**
1. While running the Docker command to deploy Rancher, point Docker toward your CA certificate file.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /etc/your_certificate_directory/cacerts.pem:/etc/rancher/ssl/cacerts.pem \
rancher/rancher:latest
```
</details>
<details id="option-b">
<summary>Option B-Bring Your Own Certificate: Signed by Recognized CA</summary>
If your cluster is public facing, it's best to use a certificate signed by a recognized CA.
> **Prerequisites:**
>
> - The certificate files must be in PEM format.
**To Install Rancher Using a Cert Signed by a Recognized CA:**
If you use a certificate signed by a recognized CA, installing your certificate in the Rancher container isn't necessary. We do have to make sure there is no default CA certificate generated and stored, you can do this by passing the `--no-cacerts` parameter to the container.
1. Enter the following command.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
rancher/rancher:latest --no-cacerts
```
</details>
## 3. Configure Load Balancer
When using a load balancer in front of your Rancher container, there's no need for the container to redirect port communication from port 80 or port 443. By passing the header `X-Forwarded-Proto: https` header, this redirect is disabled.
The load balancer or proxy has to be configured to support the following:
- **WebSocket** connections
- **SPDY** / **HTTP/2** protocols
- Passing / setting the following headers:
| Header | Value | Description |
|--------|-------|-------------|
| `Host` | Hostname used to reach Rancher. | To identify the server requested by the client.
| `X-Forwarded-Proto` | `https` | To identify the protocol that a client used to connect to the load balancer or proxy.<br /><br/>**Note:** If this header is present, `rancher/rancher` does not redirect HTTP to HTTPS.
| `X-Forwarded-Port` | Port used to reach Rancher. | To identify the protocol that client used to connect to the load balancer or proxy.
| `X-Forwarded-For` | IP of the client connection. | To identify the originating IP address of a client.
### Example NGINX configuration
This NGINX configuration is tested on NGINX 1.14.
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/).
- Replace `rancher-server` with the IP address or hostname of the node running the Rancher container.
- Replace both occurrences of `FQDN` to the DNS name for Rancher.
- Replace `/certs/fullchain.pem` and `/certs/privkey.pem` to the location of the server certificate and the server certificate key respectively.
```
worker_processes 4;
worker_rlimit_nofile 40000;
events {
worker_connections 8192;
}
http {
upstream rancher {
server rancher-server:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name FQDN;
ssl_certificate /certs/fullchain.pem;
ssl_certificate_key /certs/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name FQDN;
return 301 https://$server_name$request_uri;
}
}
```
<br/>
## What's Next?
- **Recommended:** Review Single Node [Backup](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-docker-installed-rancher.md) and [Restore](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-docker-installed-rancher.md). Although you don't have any data you need to back up right now, we recommend creating backups after regular Rancher use.
- Create a Kubernetes cluster: [Provisioning Kubernetes Clusters](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md).
<br/>
## FAQ and Troubleshooting
For help troubleshooting certificates, see [this section.](../../other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting.md)
## Advanced Options
### API Auditing
If you want to record all transactions with the Rancher API, enable the [API Auditing](./enable-api-audit-log.md) feature by adding the flags below into your install command.
-e AUDIT_LEVEL=1 \
-e AUDIT_LOG_PATH=/var/log/auditlog/rancher-api-audit.log \
-e AUDIT_LOG_MAXAGE=20 \
-e AUDIT_LOG_MAXBACKUP=20 \
-e AUDIT_LOG_MAXSIZE=100 \
### Air Gap
If you are visiting this page to complete an [Air Gap Installation](../../other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md), you must pre-pend your private registry URL to the server tag when running the installation command in the option that you choose. Add `<REGISTRY.DOMAIN.COM:PORT>` with your private registry URL in front of `rancher/rancher:latest`.
**Example:**
<REGISTRY.DOMAIN.COM:PORT>/rancher/rancher:latest
### Persistent Data
Rancher uses etcd as a datastore. When Rancher is installed with Docker, the embedded etcd is being used. The persistent data is at the following path in the container: `/var/lib/rancher`.
You can bind mount a host volume to this location to preserve data on the host it is running on:
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /opt/rancher:/var/lib/rancher \
--privileged \
rancher/rancher:latest
```
As of Rancher v2.5, privileged access is [required.](../../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
This layer 7 NGINX configuration is tested on NGINX version 1.13 (mainline) and 1.14 (stable).
> **Note:** This NGINX configuration is only an example and may not suit your environment. For complete documentation, see [NGINX Load Balancing - TCP and UDP Load Balancer](https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/).
```
upstream rancher {
server rancher-server:80;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2;
server_name rancher.yourdomain.com;
ssl_certificate /etc/your_certificate_directory/fullchain.pem;
ssl_certificate_key /etc/your_certificate_directory/privkey.pem;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes. Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name rancher.yourdomain.com;
return 301 https://$server_name$request_uri;
}
```
<br/>
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/advanced-use-cases/enable-api-audit-log.md
-565
View File
@@ -1,565 +0,0 @@
---
title: Enabling the API Audit Log to Record System Events
---
You can enable the API audit log to record the sequence of system events initiated by individual users. You can know what happened, when it happened, who initiated it, and what cluster it affected. When you enable this feature, all requests to the Rancher API and all responses from it are written to a log.
You can enable API Auditing during Rancher installation or upgrade.
## Enabling API Audit Log
The Audit Log is enabled and configured by passing environment variables to the Rancher server container. See the following to enable on your installation.
- [Docker Install](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log)
- [Kubernetes Install](../../../../reference-guides/installation-references/helm-chart-options.md#api-audit-log)
## API Audit Log Options
The usage below defines rules about what the audit log should record and what data it should include:
| Parameter | Description |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `AUDIT_LEVEL` | `0` - Disable audit log (default setting).<br/>`1` - Log event metadata.<br/>`2` - Log event metadata and request body.<br/>`3` - Log event metadata, request body, and response body. Each log transaction for a request/response pair uses the same `auditID` value.<br/><br/>See [Audit Level Logging](#audit-log-levels) for a table that displays what each setting logs. |
| `AUDIT_LOG_PATH` | Log path for Rancher Server API. Default path is `/var/log/auditlog/rancher-api-audit.log`. You can mount the log directory to host. <br/><br/>Usage Example: `AUDIT_LOG_PATH=/my/custom/path/`<br/> |
| `AUDIT_LOG_MAXAGE` | Defined the maximum number of days to retain old audit log files. Default is 10 days. |
| `AUDIT_LOG_MAXBACKUP` | Defines the maximum number of audit log files to retain. Default is 10. |
| `AUDIT_LOG_MAXSIZE` | Defines the maximum size in megabytes of the audit log file before it gets rotated. Default size is 100M. |
<br/>
### Audit Log Levels
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
| --------------------- | -------- | ------------ | ------------- |
| `0` | | | |
| `1` | ✓ | | |
| `2` | ✓ | ✓ | |
| `3` | ✓ | ✓ | ✓ |
## Viewing API Audit Logs
### Docker Install
Share the `AUDIT_LOG_PATH` directory (Default: `/var/log/auditlog`) with the host system. The log can be parsed by standard CLI tools or forwarded on to a log collection tool like Fluentd, Filebeat, Logstash, etc.
### Kubernetes Install
Enabling the API Audit Log with the Helm chart install will create a `rancher-audit-log` sidecar container in the Rancher pod. This container will stream the log to standard output (stdout). You can view the log as you would any container log.
The `rancher-audit-log` container is part of the `rancher` pod in the `cattle-system` namespace.
#### CLI
```bash
kubectl -n cattle-system logs -f rancher-84d886bdbb-s4s69 rancher-audit-log
```
#### Rancher Web GUI
1. From the context menu, select **Cluster: local > System**.
1. From the main navigation bar, choose **Resources > Workloads.** Find the `cattle-system` namespace. Open the `rancher` workload by clicking its link.
1. Pick one of the `rancher` pods and select **&#8942; > View Logs**.
1. From the **Logs** drop-down, select `rancher-audit-log`.
#### Shipping the Audit Log
You can enable Rancher's built in log collection and shipping for the cluster to ship the audit and other services logs to a supported collection endpoint. See [Logging](../../../../explanations/integrations-in-rancher/logging/logging.md) for details.
## Audit Log Samples
After you enable auditing, each API request or response is logged by Rancher in the form of JSON. Each of the following code samples provide examples of how to identify each API transaction.
### Metadata Level
If you set your `AUDIT_LEVEL` to `1`, Rancher logs the metadata header for every API request, but not the body. The header provides basic information about the API transaction, such as the transaction's ID, who initiated the transaction, the time it occurred, etc.
```json
{
"auditID": "30022177-9e2e-43d1-b0d0-06ef9d3db183",
"requestURI": "/v3/schemas",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "GET",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:22:43 +0800"
}
```
### Metadata and Request Body Level
If you set your `AUDIT_LEVEL` to `2`, Rancher logs the metadata header and body for every API request.
The code sample below depicts an API request, with both its metadata header and body.
```json
{
"auditID": "ef1d249e-bfac-4fd0-a61f-cbdcad53b9bb",
"requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "PUT",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:28:08 +0800",
"requestBody": {
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"paused": false,
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements",
"requests": {},
"limits": {}
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container",
"environmentFrom": [],
"capAdd": [],
"capDrop": [],
"livenessProbe": null,
"volumeMounts": []
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"name": "nginx",
"namespaceId": "default",
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport",
"type": "publicEndpoint"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"scheduling": {
"node": {}
},
"description": "my description",
"volumes": []
}
}
```
### Metadata, Request Body, and Response Body Level
If you set your `AUDIT_LEVEL` to `3`, Rancher logs:
- The metadata header and body for every API request.
- The metadata header and body for every API response.
#### Request
The code sample below depicts an API request, with both its metadata header and body.
```json
{
"auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af",
"requestURI": "/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"sourceIPs": ["::1"],
"user": {
"name": "user-f4tt2",
"group": ["system:authenticated"]
},
"verb": "PUT",
"stage": "RequestReceived",
"stageTimestamp": "2018-07-20 10:33:06 +0800",
"requestBody": {
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"paused": false,
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements",
"requests": {},
"limits": {}
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container",
"environmentFrom": [],
"capAdd": [],
"capDrop": [],
"livenessProbe": null,
"volumeMounts": []
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"name": "nginx",
"namespaceId": "default",
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport",
"type": "publicEndpoint"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"scheduling": {
"node": {}
},
"description": "my decript",
"volumes": []
}
}
```
#### Response
The code sample below depicts an API response, with both its metadata header and body.
```json
{
"auditID": "a886fd9f-5d6b-4ae3-9a10-5bff8f3d68af",
"responseStatus": "200",
"stage": "ResponseComplete",
"stageTimestamp": "2018-07-20 10:33:06 +0800",
"responseBody": {
"actionLinks": {
"pause": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=pause",
"resume": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=resume",
"rollback": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx?action=rollback"
},
"annotations": {},
"baseType": "workload",
"containers": [
{
"allowPrivilegeEscalation": false,
"image": "nginx",
"imagePullPolicy": "Always",
"initContainer": false,
"name": "nginx",
"ports": [
{
"containerPort": 80,
"dnsName": "nginx-nodeport",
"kind": "NodePort",
"name": "80tcp01",
"protocol": "TCP",
"sourcePort": 0,
"type": "/v3/project/schemas/containerPort"
}
],
"privileged": false,
"readOnly": false,
"resources": {
"type": "/v3/project/schemas/resourceRequirements"
},
"restartCount": 0,
"runAsNonRoot": false,
"stdin": true,
"stdinOnce": false,
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"tty": true,
"type": "/v3/project/schemas/container"
}
],
"created": "2018-07-18T07:34:16Z",
"createdTS": 1531899256000,
"creatorId": null,
"deploymentConfig": {
"maxSurge": 1,
"maxUnavailable": 0,
"minReadySeconds": 0,
"progressDeadlineSeconds": 600,
"revisionHistoryLimit": 10,
"strategy": "RollingUpdate"
},
"deploymentStatus": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2018-07-18T07:34:38Z",
"lastTransitionTimeTS": 1531899278000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2018-07-18T07:34:16Z",
"lastTransitionTimeTS": 1531899256000,
"lastUpdateTime": "2018-07-18T07:34:38Z",
"lastUpdateTimeTS": 1531899278000,
"message": "ReplicaSet \"nginx-64d85666f9\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 2,
"readyReplicas": 1,
"replicas": 1,
"type": "/v3/project/schemas/deploymentStatus",
"unavailableReplicas": 0,
"updatedReplicas": 1
},
"dnsPolicy": "ClusterFirst",
"hostIPC": false,
"hostNetwork": false,
"hostPID": false,
"id": "deployment:default:nginx",
"labels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"links": {
"remove": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"revisions": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/revisions",
"self": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"update": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx",
"yaml": "https://localhost:8443/v3/project/c-bcz5t:p-fdr4s/workloads/deployment:default:nginx/yaml"
},
"name": "nginx",
"namespaceId": "default",
"paused": false,
"projectId": "c-bcz5t:p-fdr4s",
"publicEndpoints": [
{
"addresses": ["10.64.3.58"],
"allNodes": true,
"ingressId": null,
"nodeId": null,
"podId": null,
"port": 30917,
"protocol": "TCP",
"serviceId": "default:nginx-nodeport"
}
],
"restartPolicy": "Always",
"scale": 1,
"schedulerName": "default-scheduler",
"selector": {
"matchLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
},
"type": "/v3/project/schemas/labelSelector"
},
"state": "active",
"terminationGracePeriodSeconds": 30,
"transitioning": "no",
"transitioningMessage": "",
"type": "deployment",
"uuid": "f998037d-8a5c-11e8-a4cf-0245a7ebb0fd",
"workloadAnnotations": {
"deployment.kubernetes.io/revision": "1",
"field.cattle.io/creatorId": "user-f4tt2"
},
"workloadLabels": {
"workload.user.cattle.io/workloadselector": "deployment-default-nginx"
}
}
}
```
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/advanced-use-cases/open-ports-with-firewalld.md
-117
View File
@@ -1,117 +0,0 @@
---
title: Opening Ports with firewalld
---
:::danger
Enabling firewalld can cause serious network communication problems.
For proper network function, firewalld must be disabled on systems running RKE2. [Firewalld conflicts with Canal](https://docs.rke2.io/known_issues#firewalld-conflicts-with-default-networking), RKE2's default networking stack.
Firewalld must also be disabled on systems running Kubernetes 1.19 and later.
If you enable firewalld on systems running Kubernetes 1.18 or earlier, understand that this may cause networking issues. CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
For example, one Oracle Linux image in AWS has REJECT rules that stop Helm from communicating with Tiller:
```
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
```
You can check the default firewall rules with this command:
```
sudo iptables --list
```
This section describes how to use `firewalld` to apply the [firewall port rules](../../installation-requirements/port-requirements.md) for nodes in a high-availability Rancher server cluster.
## Prerequisite
Install v7.x or later ofv`firewalld`:
```
yum install firewalld
systemctl start firewalld
systemctl enable firewalld
```
## Applying Firewall Port Rules
In the Rancher high-availability installation instructions, the Rancher server is set up on three nodes that have all three Kubernetes roles: etcd, controlplane, and worker. If your Rancher server nodes have all three roles, run the following commands on each node:
```
firewall-cmd --permanent --add-port=22/tcp
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=443/tcp
firewall-cmd --permanent --add-port=2376/tcp
firewall-cmd --permanent --add-port=2379/tcp
firewall-cmd --permanent --add-port=2380/tcp
firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=8472/udp
firewall-cmd --permanent --add-port=9099/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10254/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --permanent --add-port=30000-32767/udp
```
If your Rancher server nodes have separate roles, use the following commands based on the role of the node:
```
# For etcd nodes, run the following commands:
firewall-cmd --permanent --add-port=2376/tcp
firewall-cmd --permanent --add-port=2379/tcp
firewall-cmd --permanent --add-port=2380/tcp
firewall-cmd --permanent --add-port=8472/udp
firewall-cmd --permanent --add-port=9099/tcp
firewall-cmd --permanent --add-port=10250/tcp
# For control plane nodes, run the following commands:
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=443/tcp
firewall-cmd --permanent --add-port=2376/tcp
firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=8472/udp
firewall-cmd --permanent --add-port=9099/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10254/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --permanent --add-port=30000-32767/udp
# For worker nodes, run the following commands:
firewall-cmd --permanent --add-port=22/tcp
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=443/tcp
firewall-cmd --permanent --add-port=2376/tcp
firewall-cmd --permanent --add-port=8472/udp
firewall-cmd --permanent --add-port=9099/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10254/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --permanent --add-port=30000-32767/udp
```
After the `firewall-cmd` commands have been run on a node, use the following command to enable the firewall rules:
```
firewall-cmd --reload
```
**Result:** The firewall is updated so that Helm can communicate with the Rancher server nodes.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/advanced-use-cases/tune-etcd-for-large-installs.md
-39
View File
@@ -1,39 +0,0 @@
---
title: Tuning etcd for Large Installations
---
When running larger Rancher installations with 15 or more clusters it is recommended to increase the default keyspace for etcd from the default 2GB. The maximum setting is 8GB and the host should have enough RAM to keep the entire dataset in memory. When increasing this value you should also increase the size of the host. The keyspace size can also be adjusted in smaller installations if you anticipate a high rate of change of pods during the garbage collection interval.
The etcd data set is automatically cleaned up on a five minute interval by Kubernetes. There are situations, e.g. deployment thrashing, where enough events could be written to etcd and deleted before garbage collection occurs and cleans things up causing the keyspace to fill up. If you see `mvcc: database space exceeded` errors, in the etcd logs or Kubernetes API server logs, you should consider increasing the keyspace size. This can be accomplished by setting the [quota-backend-bytes](https://etcd.io/docs/v3.3/op-guide/maintenance/#space-quota) setting on the etcd servers.
## Example: This Snippet of the RKE Cluster.yml file Increases the Keyspace Size to 5GB
```yaml
# RKE cluster.yml
---
services:
etcd:
extra_args:
quota-backend-bytes: 5368709120
```
## Scaling etcd Disk Performance
You can follow the recommendations from [the etcd docs](https://etcd.io/docs/v3.3/tuning/#disk) on how to tune the disk priority on the host.
Additionally, to reduce IO contention on the disks for etcd, you can use a dedicated device for the data and wal directory. Based on etcd best practices, mirroring RAID configurations are unnecessary because etcd replicates data between the nodes in the cluster. You can use striping RAID configurations to increase available IOPS.
To implement this solution in an RKE cluster, the `/var/lib/etcd/data` and `/var/lib/etcd/wal` directories will need to have disks mounted and formatted on the underlying host. In the `extra_args` directive of the `etcd` service, you must include the `wal_dir` directory. Without specifying the `wal_dir`, etcd process will try to manipulate the underlying `wal` mount with insufficient permissions.
```yaml
# RKE cluster.yml
---
services:
etcd:
extra_args:
data-dir: '/var/lib/rancher/etcd/data/'
wal-dir: '/var/lib/rancher/etcd/wal/wal_dir'
extra_binds:
- '/var/lib/etcd/data:/var/lib/rancher/etcd/data'
- '/var/lib/etcd/wal:/var/lib/rancher/etcd/wal'
```
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/enable-experimental-features/enable-experimental-features.md
-126
View File
@@ -1,126 +0,0 @@
---
title: Enabling Experimental Features
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/pages-for-subheaders/enable-experimental-features"/>
</head>
Rancher includes some features that are experimental and disabled by default. You might want to enable these features, for example, if you decide that the benefits of using an [unsupported storage type](unsupported-storage-drivers.md) outweighs the risk of using an untested feature. Feature flags were introduced to allow you to try these features that are not enabled by default.
The features can be enabled in three ways:
- [Enable features when starting Rancher.](#enabling-features-when-starting-rancher) When installing Rancher with a CLI, you can use a feature flag to enable a feature by default.
- [Enable features from the Rancher UI](#enabling-features-with-the-rancher-ui) by going to the **Settings** page.
- [Enable features with the Rancher API](#enabling-features-with-the-rancher-api) after installing Rancher.
Each feature has two values:
- A default value, which can be configured with a flag or environment variable from the command line
- A set value, which can be configured with the Rancher API or UI
If no value has been set, Rancher uses the default value.
Because the API sets the actual value and the command line sets the default value, that means that if you enable or disable a feature with the API or UI, it will override any value set with the command line.
For example, if you install Rancher, then set a feature flag to true with the Rancher API, then upgrade Rancher with a command that sets the feature flag to false, the default value will still be false, but the feature will still be enabled because it was set with the Rancher API. If you then deleted the set value (true) with the Rancher API, setting it to NULL, the default value (false) would take effect. See the [feature flags page](../../../../reference-guides/installation-references/feature-flags.md) for more information.
## Enabling Features when Starting Rancher
When you install Rancher, enable the feature you want with a feature flag. The command is different depending on whether you are installing Rancher on a single node or if you are doing a Kubernetes Installation of Rancher.
### Enabling Features for Kubernetes Installs
:::note
Values set from the Rancher API will override the value passed in through the command line.
:::
When installing Rancher with a Helm chart, use the `--set` option. In the below example, two features are enabled by passing the feature flag names in a comma separated list:
```
helm install rancher rancher-latest/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set 'extraEnv[0].name=CATTLE_FEATURES'
--set 'extraEnv[0].value=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true'
```
:::note
If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
:::
### Rendering the Helm Chart for Air Gap Installations
For an air gap installation of Rancher, you need to add a Helm chart repository and render a Helm template before installing Rancher with Helm. For details, refer to the [air gap installation documentation.](../../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md)
Here is an example of a command for passing in the feature flag names when rendering the Helm template. In the below example, two features are enabled by passing the feature flag names in a comma separated list.
The Helm command is as follows:
```
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--no-hooks \ # prevent files for Helm hooks from being generated
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
--set 'extraEnv[0].name=CATTLE_FEATURES'
--set 'extraEnv[0].value=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true'
```
### Enabling Features for Docker Installs
When installing Rancher with Docker, use the `--features` option. In the below example, two features are enabled by passing the feature flag names in a comma separated list:
```
docker run -d -p 80:80 -p 443:443 \
--restart=unless-stopped \
rancher/rancher:rancher-latest \
--features=<FEATURE-FLAG-NAME-1>=true,<FEATURE-FLAG-NAME-2>=true
```
## Enabling Features with the Rancher UI
1. In the upper left corner, click **☰ > Global Settings**.
1. Click **Feature Flags**.
1. To enable a feature, go to the disabled feature you want to enable and click **⋮ > Activate**.
**Result:** The feature is enabled.
### Disabling Features with the Rancher UI
1. In the upper left corner, click **☰ > Global Settings**.
1. Click **Feature Flags**. You will see a list of experimental features.
1. To disable a feature, go to the enabled feature you want to disable and click **⋮ > Deactivate**.
**Result:** The feature is disabled.
## Enabling Features with the Rancher API
1. Go to `<RANCHER-SERVER-URL>/v3/features`.
1. In the `data` section, you will see an array containing all of the features that can be turned on with feature flags. The name of the feature is in the `id` field. Click the name of the feature you want to enable.
1. In the upper left corner of the screen, under **Operations,** click **Edit**.
1. In the **Value** drop-down menu, click **True**.
1. Click **Show Request**.
1. Click **Send Request**.
1. Click **Close**.
**Result:** The feature is enabled.
### Disabling Features with the Rancher API
1. Go to `<RANCHER-SERVER-URL>/v3/features`.
1. In the `data` section, you will see an array containing all of the features that can be turned on with feature flags. The name of the feature is in the `id` field. Click the name of the feature you want to enable.
1. In the upper left corner of the screen, under **Operations,** click **Edit**.
1. In the **Value** drop-down menu, click **False**.
1. Click **Show Request**.
1. Click **Send Request**.
1. Click **Close**.
**Result:** The feature is disabled.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/enable-experimental-features/istio-traffic-management-features.md
-30
View File
@@ -1,30 +0,0 @@
---
title: UI for Istio Virtual Services and Destination Rules
---
This feature enables a UI that lets you create, read, update and delete virtual services and destination rules, which are traffic management features of Istio.
> **Prerequisite:** Turning on this feature does not enable Istio. A cluster administrator needs to [enable Istio for the cluster](../../../../how-to-guides/advanced-user-guides/istio-setup-guide/istio-setup-guide.md) in order to use the feature.
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.](enable-experimental-features.md)
Environment Variable Key | Default Value | Status | Available as of
---|---|---|---
`istio-virtual-service-ui` |`false` | Experimental | v2.3.0
`istio-virtual-service-ui` | `true` | GA | v2.3.2
## About this Feature
A central advantage of Istio's traffic management features is that they allow dynamic request routing, which is useful for canary deployments, blue/green deployments, or A/B testing.
When enabled, this feature turns on a page that lets you configure some traffic management features of Istio using the Rancher UI. Without this feature, you need to use `kubectl` to manage traffic with Istio.
The feature enables two UI tabs: one tab for **Virtual Services** and another for **Destination Rules.**
- **Virtual services** intercept and direct traffic to your Kubernetes services, allowing you to direct percentages of traffic from a request to different services. You can use them to define a set of routing rules to apply when a host is addressed. For details, refer to the [Istio documentation.](https://istio.io/docs/reference/config/networking/v1alpha3/virtual-service/)
- **Destination rules** serve as the single source of truth about which service versions are available to receive traffic from virtual services. You can use these resources to define policies that apply to traffic that is intended for a service after routing has occurred. For details, refer to the [Istio documentation.](https://istio.io/docs/reference/config/networking/v1alpha3/destination-rule)
To see these tabs,
1. Go to the project view in Rancher and click **Resources > Istio.**
1. You will see tabs for **Traffic Graph,** which has the Kiali network visualization integrated into the UI, and **Traffic Metrics,** which shows metrics for the success rate and request volume of traffic to your services, among other metrics. Next to these tabs, you should see the tabs for **Virtual Services** and **Destination Rules.**
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/enable-experimental-features/rancher-on-arm64.md
-43
View File
@@ -1,43 +0,0 @@
---
title: "Running on ARM64 (Experimental)"
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/enable-experimental-features/rancher-on-arm64"/>
</head>
> **Important:**
>
> Running on an ARM64 platform is currently an experimental feature and is not yet officially supported in Rancher. Therefore, we do not recommend using ARM64 based nodes in a production environment.
The following options are available when using an ARM64 platform:
- Running Rancher on ARM64 based node(s)
- Only for Docker Install. Please note that the following installation command replaces the examples found in the [Docker Install](../../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) link:
```
# In the last line `rancher/rancher:vX.Y.Z`, be certain to replace "X.Y.Z" with a released version in which ARM64 builds exist. For example, if your matching version is v2.5.8, you would fill in this line with `rancher/rancher:v2.5.8`.
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:vX.Y.Z
```
> **Note:** To check if your specific released version is compatible with the ARM64 architecture, you may navigate to your
> version's release notes in the following two ways:
>
> - Manually find your version using https://github.com/rancher/rancher/releases.
> - Go directly to your version using the tag and the specific version number. If you plan to use v2.5.8, for example, you may
> navigate to https://github.com/rancher/rancher/releases/tag/v2.5.8.
- Create custom cluster and adding ARM64 based node(s)
- Kubernetes cluster version must be 1.12 or higher
- CNI Network Provider must be [Flannel](../../../../faq/container-network-interface-providers.md#flannel)
- Importing clusters that contain ARM64 based nodes
- Kubernetes cluster version must be 1.12 or higher
Please see [Cluster Options](cluster-provisioning/rke-clusters/options/) how to configure the cluster options.
The following features are not tested:
- Monitoring, alerts, notifiers, pipelines and logging
- Launching apps from the catalog
versioned_docs/version-2.5/getting-started/installation-and-upgrade/advanced-options/enable-experimental-features/unsupported-storage-drivers.md
-39
View File
@@ -1,39 +0,0 @@
---
title: Allowing Unsupported Storage Drivers
---
This feature allows you to use types for storage providers and provisioners that are not enabled by default.
To enable or disable this feature, refer to the instructions on [the main page about enabling experimental features.](../../../../reference-guides/installation-references/feature-flags.md)
Environment Variable Key | Default Value | Description
---|---|---
`unsupported-storage-drivers` | `false` | This feature enables types for storage providers and provisioners that are not enabled by default.
### Types for Persistent Volume Plugins that are Enabled by Default
Below is a list of storage types for persistent volume plugins that are enabled by default. When enabling this feature flag, any persistent volume plugins that are not on this list are considered experimental and unsupported:
Name | Plugin
--------|----------
Amazon EBS Disk | `aws-ebs`
AzureFile | `azure-file`
AzureDisk | `azure-disk`
Google Persistent Disk | `gce-pd`
Longhorn | `flex-volume-longhorn`
VMware vSphere Volume | `vsphere-volume`
Local | `local`
Network File System | `nfs`
hostPath | `host-path`
### Types for StorageClass that are Enabled by Default
Below is a list of storage types for a StorageClass that are enabled by default. When enabling this feature flag, any persistent volume plugins that are not on this list are considered experimental and unsupported:
Name | Plugin
--------|--------
Amazon EBS Disk | `aws-ebs`
AzureFile | `azure-file`
AzureDisk | `azure-disk`
Google Persistent Disk | `gce-pd`
Longhorn | `flex-volume-longhorn`
VMware vSphere Volume | `vsphere-volume`
Local | `local`
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/air-gapped-upgrades.md
-135
View File
@@ -1,135 +0,0 @@
---
title: Rendering the Helm Template in an Air Gapped Environment
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/air-gapped-upgrades"/>
</head>
> These instructions assume you have already followed the instructions for a Kubernetes upgrade on [this page,](upgrades.md) including the prerequisites, up until step 3. Upgrade Rancher.
## Rancher Helm Template Options
Render the Rancher template using the same chosen options that were used when installing Rancher. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
Based on the choice you made during installation, complete one of the procedures below.
Placeholder | Description
------------|-------------
`<VERSION>` | The version number of the output tarball.
`<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.
`<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.
### Option A: Default Self-signed Certificate
<Tabs>
<TabItem value="Rancher v2.5.8+">
```
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--no-hooks \ # prevent files for Helm hooks from being generated
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set certmanager.version=<CERTMANAGER_VERSION> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
</TabItem>
<TabItem value="Rancher before v2.5.8">
```plain
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set certmanager.version=<CERTMANAGER_VERSION> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
</TabItem>
</Tabs>
### Option B: Certificates from Files using Kubernetes Secrets
<Tabs>
<TabItem value="Rancher v2.5.8+">
```plain
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--no-hooks \ # prevent files for Helm hooks from being generated
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret`:
```plain
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--no-hooks \ # prevent files for Helm hooks from being generated
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set privateCA=true \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
</TabItem>
<TabItem value="Rancher before v2.5.8">
```plain
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret`:
```plain
helm template rancher ./rancher-<VERSION>.tgz --output-dir . \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set privateCA=true \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
</TabItem>
</Tabs>
## Apply the Rendered Templates
Copy the rendered manifest directories to a system with access to the Rancher server cluster and apply the rendered templates.
Use `kubectl` to apply the rendered manifests.
```plain
kubectl -n cattle-system apply -R -f ./rancher
```
## Verify the Upgrade
Log into Rancher to confirm that the upgrade succeeded.
>**Having network issues following upgrade?**
>
> See [Restoring Cluster Networking](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/namespace-migration.md).
## Known Upgrade Issues
A list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md
-310
View File
@@ -1,310 +0,0 @@
---
title: Install/Upgrade Rancher on a Kubernetes Cluster
description: Learn how to install Rancher in development and production environments. Read about single node and high availability installation
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster"/>
</head>
In this section, you'll learn how to deploy Rancher on a Kubernetes cluster using the Helm CLI.
## Prerequisites
- [Kubernetes Cluster](#kubernetes-cluster)
- [CLI Tools](#cli-tools)
- [Ingress Controller (Only for Hosted Kubernetes)](#ingress-controller-for-hosted-kubernetes)
### Kubernetes Cluster
Set up the Rancher server's local Kubernetes cluster.
Rancher can be installed on any Kubernetes cluster. This cluster can use upstream Kubernetes, or it can use one of Rancher's Kubernetes distributions, or it can be a managed Kubernetes cluster from a provider such as Amazon EKS.
For help setting up a Kubernetes cluster, we provide these tutorials:
- **RKE:** For the tutorial to install an RKE Kubernetes cluster, refer to [this page.](../../../how-to-guides/new-user-guides/kubernetes-cluster-setup/rke1-for-rancher.md) For help setting up the infrastructure for a high-availability RKE cluster, refer to [this page.](../../../how-to-guides/new-user-guides/infrastructure-setup/ha-rke1-kubernetes-cluster.md)
- **K3s:** For the tutorial to install a K3s Kubernetes cluster, refer to [this page.](../../../how-to-guides/new-user-guides/kubernetes-cluster-setup/k3s-for-rancher.md) For help setting up the infrastructure for a high-availability K3s cluster, refer to [this page.](../../../how-to-guides/new-user-guides/infrastructure-setup/ha-k3s-kubernetes-cluster.md)
- **RKE2:** For the tutorial to install an RKE2 Kubernetes cluster, refer to [this page.](../../../how-to-guides/new-user-guides/kubernetes-cluster-setup/rke2-for-rancher.md) For help setting up the infrastructure for a high-availability RKE2 cluster, refer to [this page.](../../../how-to-guides/new-user-guides/infrastructure-setup/ha-rke2-kubernetes-cluster.md)
- **Amazon EKS:** For details on how to install Rancher on Amazon EKS, including how to install an ingress so that the Rancher server can be accessed, refer to [this page.](rancher-on-amazon-eks.md)
- **AKS:** For details on how to install Rancher with Azure Kubernetes Service, including how to install an ingress so that the Rancher server can be accessed, refer to [this page.](rancher-on-aks.md)
- **GKE:** For details on how to install Rancher with Google Kubernetes Engine, including how to install an ingress so that the Rancher server can be accessed, refer to [this page.](rancher-on-gke.md)
### CLI Tools
The following CLI tools are required for setting up the Kubernetes cluster. Please make sure these tools are installed and available in your `$PATH`.
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) - Kubernetes command-line tool.
- [helm](https://docs.helm.sh/using_helm/#installing-helm) - Package management for Kubernetes. Refer to the [Helm version requirements](../resources/helm-version-requirements.md) to choose a version of Helm to install Rancher. Refer to the [instructions provided by the Helm project](https://helm.sh/docs/intro/install/) for your specific platform.
### Ingress Controller (For Hosted Kubernetes)
To deploy Rancher v2.5 on a hosted Kubernetes cluster such as EKS, GKE, or AKS, you should deploy a compatible Ingress controller first to configure [SSL termination on Rancher.](#3-choose-your-ssl-configuration)
For an example of how to deploy an ingress on EKS, refer to [this section.](rancher-on-amazon-eks.md#5-install-an-ingress)
## Install the Rancher Helm Chart
Rancher is installed using the Helm package manager for Kubernetes. Helm charts provide templating syntax for Kubernetes YAML manifest documents.
With Helm, we can create configurable deployments instead of just using static files. For more information about creating your own catalog of deployments, check out the docs at https://helm.sh/.
For systems without direct internet access, see [Air Gap: Kubernetes install](../other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md).
To choose a Rancher version to install, refer to [Choosing a Rancher Version.](../resources/choose-a-rancher-version.md)
To choose a version of Helm to install Rancher with, refer to the [Helm version requirements](../resources/helm-version-requirements.md)
> **Note:** The installation instructions assume you are using Helm 3. For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 migration docs.](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) This [section](../resources/helm-version-requirements.md) provides a copy of the older installation instructions for Rancher installed on an RKE Kubernetes cluster with Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible.
To set up Rancher,
1. [Add the Helm chart repository](#1-add-the-helm-chart-repository)
2. [Create a namespace for Rancher](#2-create-a-namespace-for-rancher)
3. [Choose your SSL configuration](#3-choose-your-ssl-configuration)
4. [Install cert-manager](#4-install-cert-manager) (unless you are bringing your own certificates, or TLS will be terminated on a load balancer)
5. [Install Rancher with Helm and your chosen certificate option](#5-install-rancher-with-helm-and-your-chosen-certificate-option)
6. [Verify that the Rancher server is successfully deployed](#6-verify-that-the-rancher-server-is-successfully-deployed)
7. [Save your options](#7-save-your-options)
### 1. Add the Helm Chart Repository
Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Rancher Version](../resources/choose-a-rancher-version.md).
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
Note: Upgrades are not supported to, from, or between Alphas.
### 2. Create a Namespace for Rancher
We'll need to define a Kubernetes namespace where the resources created by the Chart should be installed. This should always be `cattle-system`:
```
kubectl create namespace cattle-system
```
### 3. Choose your SSL Configuration
The Rancher management server is designed to be secure by default and requires SSL/TLS configuration.
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer](../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination).
There are three recommended options for the source of the certificate used for TLS termination at the Rancher server:
- **Rancher-generated TLS certificate:** In this case, you will need to install `cert-manager` into the cluster. Rancher utilizes `cert-manager` to issue and maintain its certificates. Rancher will generate a CA certificate of its own, and sign a cert using that CA. `cert-manager` is then responsible for managing that certificate.
- **Let's Encrypt:** The Let's Encrypt option also uses `cert-manager`. However, in this case, cert-manager is combined with a special Issuer for Let's Encrypt that performs all actions (including request and validation) necessary for getting a Let's Encrypt issued cert. This configuration uses HTTP validation (`HTTP-01`), so the load balancer must have a public DNS record and be accessible from the internet.
- **Bring your own certificate:** This option allows you to bring your own public- or private-CA signed certificate. Rancher will use that certificate to secure websocket and HTTPS traffic. In this case, you must upload this certificate (and associated key) as PEM-encoded files with the name `tls.crt` and `tls.key`. If you are using a private CA, you must also upload that certificate. This is due to the fact that this private CA may not be trusted by your nodes. Rancher will take that CA certificate, and generate a checksum from it, which the various Rancher components will use to validate their connection to Rancher.
| Configuration | Helm Chart Option | Requires cert-manager |
| ------------------------------ | ----------------------- | ------------------------------------- |
| Rancher Generated Certificates (Default) | `ingress.tls.source=rancher` | [yes](#4-install-cert-manager) |
| Let’s Encrypt | `ingress.tls.source=letsEncrypt` | [yes](#4-install-cert-manager) |
| Certificates from Files | `ingress.tls.source=secret` | no |
### 4. Install cert-manager
> You should skip this step if you are bringing your own certificate files (option `ingress.tls.source=secret`), or if you use [TLS termination on an external load balancer](../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination).
This step is only required to use certificates issued by Rancher's generated CA (`ingress.tls.source=rancher`) or to request Let's Encrypt issued certificates (`ingress.tls.source=letsEncrypt`).
<details id="cert-manager">
<summary>Click to Expand</summary>
> **Important:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.11.0, please see our [upgrade documentation](../resources/upgrade-cert-manager.md/).
These instructions are adapted from the [official cert-manager documentation](https://cert-manager.io/docs/installation/kubernetes/#installing-with-helm).
```
# If you have installed the CRDs manually instead of with the `--set installCRDs=true` option added to your Helm install command, you should upgrade your CRD resources before upgrading the Helm chart:
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.1/cert-manager.crds.yaml
# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io
# Update your local Helm chart repository cache
helm repo update
# Install the cert-manager Helm chart
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--set installCRDs=true \
--version v1.5.1
```
Once you’ve installed cert-manager, you can verify it is deployed correctly by checking the cert-manager namespace for running pods:
```
kubectl get pods --namespace cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-5c6866597-zw7kh 1/1 Running 0 2m
cert-manager-cainjector-577f6d9fd7-tr77l 1/1 Running 0 2m
cert-manager-webhook-787858fcdb-nlzsq 1/1 Running 0 2m
```
</details>
### 5. Install Rancher with Helm and Your Chosen Certificate Option
The exact command to install Rancher differs depending on the certificate configuration.
However, irrespective of the certificate configuration, the name of the Rancher installation in the `cattle-system` namespace should always be `rancher`.
<Tabs>
<TabItem value="Rancher-generated Certificates">
The default is for Rancher to generate a self-signed CA, and uses `cert-manager` to issue the certificate for access to the Rancher server interface.
Because `rancher` is the default option for `ingress.tls.source`, we are not specifying `ingress.tls.source` when running the `helm install` command.
- Set `hostname` to the DNS record that resolves to your load balancer.
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set replicas=3
```
Wait for Rancher to be rolled out:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
</TabItem>
<TabItem value="Let's Encrypt">
This option uses `cert-manager` to automatically request and renew [Let's Encrypt](https://letsencrypt.org/) certificates. This is a free service that provides you with a valid certificate as Let's Encrypt is a trusted CA.
>**Note:** You need to have port 80 open as the HTTP-01 challenge can only be done on port 80.
In the following command,
- Set `hostname` to the public DNS record that resolves to your load balancer.
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
- Set `ingress.tls.source` to `letsEncrypt`.
- Set `letsEncrypt.email` to the email address used for communication about your certificate (for example, expiry notices).
- Set `letsEncrypt.ingress.class` to whatever your ingress controller is, e.g., `traefik`, `nginx`, `haproxy`, etc.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set replicas=3 \
--set ingress.tls.source=letsEncrypt \
--set letsEncrypt.email=me@example.org \
--set letsEncrypt.ingress.class=nginx
```
Wait for Rancher to be rolled out:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
</TabItem>
<TabItem value="Certificates from Files">
In this option, Kubernetes secrets are created from your own certificates for Rancher to use.
When you run this command, the `hostname` option must match the `Common Name` or a `Subject Alternative Names` entry in the server certificate, or the Ingress controller will fail to configure correctly.
Although an entry in the `Subject Alternative Names` is technically required, having a matching `Common Name` maximizes compatibility with older browsers and applications.
> If you want to check if your certificates are correct, see [How do I check Common Name and Subject Alternative Names in my server certificate?](../../../faq/technical-items.md#how-do-i-check-common-name-and-subject-alternative-names-in-my-server-certificate)
- Set `hostname` as appropriate for your certificate, as described above.
- Set `replicas` to the number of replicas to use for the Rancher Deployment. This defaults to 3; if you have less than 3 nodes in your cluster you should reduce it accordingly.
- Set `ingress.tls.source` to `secret`.
- To install a specific Rancher version, use the `--version` flag, example: `--version 2.3.6`.
- If you are installing an alpha version, Helm requires adding the `--devel` option to the command.
```
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set replicas=3 \
--set ingress.tls.source=secret
```
If you are using a Private CA signed certificate , add `--set privateCA=true` to the command:
```
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--set ingress.tls.source=secret \
--set privateCA=true
```
Now that Rancher is deployed, see [Adding TLS Secrets](../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the Ingress controller can use them.
</TabItem>
</Tabs>
The Rancher chart configuration has many options for customizing the installation to suit your specific environment. Here are some common advanced scenarios.
- [HTTP Proxy](../../../reference-guides/installation-references/helm-chart-options.md#http-proxy)
- [Private Docker Image Registry](../../../reference-guides/installation-references/helm-chart-options.md#private-registry-and-air-gap-installs)
- [TLS Termination on an External Load Balancer](../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination)
See the [Chart Options](../../../reference-guides/installation-references/helm-chart-options.md) for the full list of options.
### 6. Verify that the Rancher Server is Successfully Deployed
After adding the secrets, check if Rancher was rolled out successfully:
```
kubectl -n cattle-system rollout status deploy/rancher
Waiting for deployment "rancher" rollout to finish: 0 of 3 updated replicas are available...
deployment "rancher" successfully rolled out
```
If you see the following error: `error: deployment "rancher" exceeded its progress deadline`, you can check the status of the deployment by running the following command:
```
kubectl -n cattle-system get deploy rancher
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
rancher 3 3 3 3 3m
```
It should show the same count for `DESIRED` and `AVAILABLE`.
### 7. Save Your Options
Make sure you save the `--set` options you used. You will need to use the same options when you upgrade Rancher to new versions with Helm.
### Finishing Up
That's it. You should have a functional Rancher server.
In a web browser, go to the DNS name that forwards traffic to your load balancer. Then you should be greeted by the colorful login page.
Doesn't work? Take a look at the [Troubleshooting](troubleshooting.md) Page
### Optional Next Steps
Enable the Enterprise Cluster Manager.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks.md
-131
View File
@@ -1,131 +0,0 @@
---
title: Installing Rancher on Azure Kubernetes Service
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-aks"/>
</head>
This page covers how to install Rancher on Microsoft's Azure Kubernetes Service (AKS).
The guide uses command line tools to provision an AKS cluster with an ingress. If you prefer to provision your cluster using the Azure portal, refer to the [official documentation](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal).
If you already have an AKS Kubernetes cluster, skip to the step about [installing an ingress.](#5-install-an-ingress) Then install the Rancher Helm chart following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart)
## Prerequisites
>**Note**
>Deploying to Microsoft Azure will incur charges.
- [Microsoft Azure Account](https://azure.microsoft.com/en-us/free/): A Microsoft Azure Account is required to create resources for deploying Rancher and Kubernetes.
- [Microsoft Azure Subscription](https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/create-subscription#create-a-subscription-in-the-azure-portal): Use this link to follow a tutorial to create a Microsoft Azure subscription if you don't have one yet.
- [Micsoroft Azure Tenant](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-create-new-tenant): Use this link and follow instructions to create a Microsoft Azure tenant.
- Your subscription has sufficient quota for at least 2 vCPUs. For details on Rancher server resource requirements, refer to [this section](../installation-requirements/installation-requirements.md#rke-and-hosted-kubernetes)
- When installing Rancher with Helm in Azure, use the L7 load balancer to avoid networking issues. For more information, refer to the documentation on [Azure load balancer limitations](https://docs.microsoft.com/en-us/azure/load-balancer/components#limitations).
## 1. Prepare your Workstation
Install the following command line tools on your workstation:
- The Azure CLI, **az:** For help, refer to these [installation steps.](https://docs.microsoft.com/en-us/cli/azure/)
- **kubectl:** For help, refer to these [installation steps.](https://kubernetes.io/docs/tasks/tools/#kubectl)
- **helm:** For help, refer to these [installation steps.](https://helm.sh/docs/intro/install/)
## 2. Create a Resource Group
After installing the CLI, you will need to log in with your Azure account.
```
az login
```
Create a [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) to hold all relevant resources for your cluster. Use a location that applies to your use case.
```
az group create --name rancher-rg --location eastus
```
## 3. Create the AKS Cluster
To create an AKS cluster, run the following command. Use a VM size that applies to your use case. Refer to [this article](https://docs.microsoft.com/en-us/azure/virtual-machines/sizes) for available sizes and options. When choosing a Kubernetes version, be sure to first consult the [support matrix](https://rancher.com/support-matrix/) to find the highest version of Kubernetes that has been validated for your Rancher version.
```
az aks create \
--resource-group rancher-rg \
--name rancher-server \
--kubernetes-version 1.20.5 \
--node-count 3 \
--node-vm-size Standard_D2_v3
```
The cluster will take some time to be deployed.
## 4. Get Access Credentials
After the cluster is deployed, get the access credentials.
```
az aks get-credentials --resource-group rancher-rg --name rancher-server
```
This command merges your cluster's credentials into the existing kubeconfig and allows `kubectl` to interact with the cluster.
## 5. Install an Ingress
The cluster needs an Ingress so that Rancher can be accessed from outside the cluster. Installing an Ingress requires allocating a public IP address. Ensure you have sufficient quota, otherwise it will fail to assign the IP address. Limits for public IP addresses are applicable at a regional level per subscription.
To make sure that you choose the correct Ingress-NGINX Helm chart, first find an `Ingress-NGINX version` that's compatible with your Kubernetes version in the [Kubernetes/ingress-nginx support table](https://github.com/kubernetes/ingress-nginx#supported-versions-table).
Then, list the Helm charts available to you by running the following command:
```
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm search repo ingress-nginx -l
```
The `helm search` command's output contains an `APP VERSION` column. The versions under this column are equivalent to the `Ingress-NGINX version` you chose earlier. Using the app version, select a chart version that bundles an app compatible with your Kubernetes install. For example, if you have Kubernetes v1.20, you can select the v4.2.5 Helm chart, since Ingress-NGINX v1.3.1 comes bundled with that chart, and v1.3.1 is compatible with Kubernetes v1.20. When in doubt, select the most recent compatible version.
Now that you know which Helm chart `version` you need, run the following command. It installs an `nginx-ingress-controller` with a Kubernetes load balancer service:
```
helm upgrade --install \
ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.service.type=LoadBalancer \
--version 4.2.5 \
--create-namespace
```
## 6. Get Load Balancer IP
To get the address of the load balancer, run:
```
kubectl get service ingress-nginx-controller --namespace=ingress-nginx
```
The result should look similar to the following:
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
AGE
ingress-nginx-controller LoadBalancer 10.0.116.18 40.31.180.83 80:31229/TCP,443:31050/TCP
67s
```
Save the `EXTERNAL-IP`.
## 7. Set up DNS
External traffic to the Rancher server will need to be directed at the load balancer you created.
Set up a DNS to point at the `EXTERNAL-IP` that you saved. This DNS will be used as the Rancher server URL.
There are many valid ways to set up the DNS. For help, refer to the [Azure DNS documentation](https://docs.microsoft.com/en-us/azure/dns/)
## 8. Install the Rancher Helm Chart
Next, install the Rancher Helm chart by following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart) The Helm instructions are the same for installing Rancher on any Kubernetes distribution.
Use that DNS name from the previous step as the Rancher server URL when you install Rancher. It can be passed in as a Helm option. For example, if the DNS name is `rancher.my.org`, you could run the Helm installation command with the option `--set hostname=rancher.my.org`.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-amazon-eks.md
-143
View File
@@ -1,143 +0,0 @@
---
title: Installing Rancher on Amazon EKS
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-amazon-eks"/>
</head>
This page covers installing Rancher on an Amazon EKS cluster.
If you already have an EKS Kubernetes cluster, skip to the step about [installing an ingress.](#5-install-an-ingress) Then install the Rancher Helm chart following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart)
## Creating an EKS Cluster for the Rancher Server
In this section, you'll install an EKS cluster with an ingress by using command line tools. This guide may be useful if you want to use fewer resources while trying out Rancher on EKS.
> **Prerequisites:**
>
> - You should already have an AWS account.
> - It is recommended to use an IAM user instead of the root AWS account. You will need the IAM user's access key and secret key to configure the AWS command line interface.
> - The IAM user needs the minimum IAM policies described in the official [eksctl documentation.](https://eksctl.io/usage/minimum-iam-policies/)
### 1. Prepare your Workstation
Install the following command line tools on your workstation:
- **The AWS CLI v2:** For help, refer to these [installation steps.](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
- **eksctl:** For help, refer to these [installation steps.](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)
- **kubectl:** For help, refer to these [installation steps.](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html)
- **helm:** For help, refer to these [installation steps.](https://helm.sh/docs/intro/install/)
### 2. Configure the AWS CLI
To configure the AWS CLI, run the following command:
```
aws configure
```
Then enter the following values:
| Value | Description |
|-------|-------------|
| AWS Access Key ID | The access key credential for the IAM user with EKS permissions. |
| AWS Secret Access Key | The secret key credential for the IAM user with EKS permissions. |
| Default region name | An [AWS region](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html#Concepts.RegionsAndAvailabilityZones.Regions) where the cluster nodes will be located. |
| Default output format | Enter `json`. |
### 3. Create the EKS Cluster
To create an EKS cluster, run the following command. Use the AWS region that applies to your use case. When choosing a Kubernetes version, be sure to first consult the [support matrix](https://rancher.com/support-matrix/) to find the highest version of Kubernetes that has been validated for your Rancher version.
```
eksctl create cluster \
--name rancher-server \
--version 1.20 \
--region us-west-2 \
--nodegroup-name ranchernodes \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4 \
--managed
```
The cluster will take some time to be deployed with CloudFormation.
### 4. Test the Cluster
To test the cluster, run:
```
eksctl get cluster
```
The result should look like the following:
```
eksctl get cluster
2021-03-18 15:09:35 [ℹ] eksctl version 0.40.0
2021-03-18 15:09:35 [ℹ] using region us-west-2
NAME REGION EKSCTL CREATED
rancher-server-cluster us-west-2 True
```
### 5. Install an Ingress
The cluster needs an Ingress so that Rancher can be accessed from outside the cluster.
To make sure that you choose the correct Ingress-NGINX Helm chart, first find an `Ingress-NGINX version` that's compatible with your Kubernetes version in the [Kubernetes/ingress-nginx support table](https://github.com/kubernetes/ingress-nginx#supported-versions-table).
Then, list the Helm charts available to you by running the following command:
```
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm search repo ingress-nginx -l
```
The `helm search` command's output contains an `APP VERSION` column. The versions under this column are equivalent to the `Ingress-NGINX version` you chose earlier. Using the app version, select a chart version that bundles an app compatible with your Kubernetes install. For example, if you have Kubernetes v1.20, you can select the v4.2.5 Helm chart, since Ingress-NGINX v1.3.1 comes bundled with that chart, and v1.3.1 is compatible with Kubernetes v1.20. When in doubt, select the most recent compatible version.
Now that you know which Helm chart `version` you need, run the following command. It installs an `nginx-ingress-controller` with a Kubernetes load balancer service:
```
helm upgrade --install \
ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.service.type=LoadBalancer \
--version 4.2.5 \
--create-namespace
```
### 6. Get Load Balancer IP
To get the address of the load balancer, run:
```
kubectl get service ingress-nginx-controller --namespace=ingress-nginx
```
The result should look similar to the following:
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
AGE
ingress-nginx-controller LoadBalancer 10.100.90.18 a904a952c73bf4f668a17c46ac7c56ab-962521486.us-west-2.elb.amazonaws.com 80:31229/TCP,443:31050/TCP
27m
```
Save the `EXTERNAL-IP`.
### 7. Set up DNS
External traffic to the Rancher server will need to be directed at the load balancer you created.
Set up a DNS to point at the external IP that you saved. This DNS will be used as the Rancher server URL.
There are many valid ways to set up the DNS. For help, refer to the AWS documentation on [routing traffic to an ELB load balancer.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer.html)
### 8. Install the Rancher Helm Chart
Next, install the Rancher Helm chart by following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart) The Helm instructions are the same for installing Rancher on any Kubernetes distribution.
Use that DNS name from the previous step as the Rancher server URL when you install Rancher. It can be passed in as a Helm option. For example, if the DNS name is `rancher.my.org`, you could run the Helm installation command with the option `--set hostname=rancher.my.org`.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-gke.md
-185
View File
@@ -1,185 +0,0 @@
---
title: Installing Rancher on a Google Kubernetes Engine Cluster
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rancher-on-gke"/>
</head>
In this section, you'll learn how to install Rancher using Google Kubernetes Engine.
If you already have a GKE Kubernetes cluster, skip to the step about [installing an ingress.](#7-install-an-ingress) Then install the Rancher Helm chart following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart)
## Prerequisites
- You will need a Google account.
- You will need a Google Cloud billing account. You can manage your Cloud Billing accounts using the Google Cloud Console. For more information about the Cloud Console, visit [General guide to the console.](https://support.google.com/cloud/answer/3465889?hl=en&ref_topic=3340599)
- You will need a cloud quota for at least one in-use IP address and at least 2 CPUs. For more details about hardware requirements for the Rancher server, refer to [this section.](../installation-requirements/installation-requirements.md#rke-and-hosted-kubernetes)
## 1. Enable the Kubernetes Engine API
Take the following steps to enable the Kubernetes Engine API:
1. Visit the [Kubernetes Engine page](https://console.cloud.google.com/projectselector/kubernetes?_ga=2.169595943.767329331.1617810440-856599067.1617343886) in the Google Cloud Console.
1. Create or select a project.
1. Open the project and enable the Kubernetes Engine API for the project. Wait for the API and related services to be enabled. This can take several minutes.
1. Make sure that billing is enabled for your Cloud project. For information on how to enable billing for your project, refer to the [Google Cloud documentation.](https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project)
## 2. Open the Cloud Shell
Cloud Shell is a shell environment for managing resources hosted on Google Cloud. Cloud Shell comes preinstalled with the `gcloud` command-line tool and kubectl command-line tool. The `gcloud` tool provides the primary command-line interface for Google Cloud, and `kubectl` provides the primary command-line interface for running commands against Kubernetes clusters.
The following sections describe how to launch the cloud shell from the Google Cloud Console or from your local workstation.
### Cloud Shell
To launch the shell from the [Google Cloud Console,](https://console.cloud.google.com) go to the upper-right corner of the console and click the terminal button. When hovering over the button, it is labeled **Activate Cloud Shell.**
### Local Shell
To install `gcloud` and `kubectl`, perform the following steps:
1. Install the Cloud SDK by following [these steps.](https://cloud.google.com/sdk/docs/install) The Cloud SDK includes the `gcloud` command-line tool. The steps vary based on your OS.
1. After installing Cloud SDK, install the `kubectl` command-line tool by running the following command:
```
gcloud components install kubectl
```
In a later step, `kubectl` will be configured to use the new GKE cluster.
1. [Install Helm 3](https://helm.sh/docs/intro/install/) if it is not already installed.
1. Enable Helm experimental [support for OCI images](https://github.com/helm/community/blob/master/hips/hip-0006.md) with the `HELM_EXPERIMENTAL_OCI` variable. Add the following line to `~/.bashrc` (or `~/.bash_profile` in macOS, or wherever your shell stores environment variables):
```
export HELM_EXPERIMENTAL_OCI=1
```
1. Run the following command to load your updated `.bashrc` file:
```
source ~/.bashrc
```
If you are running macOS, use this command:
```
source ~/.bash_profile
```
## 3. Configure the gcloud CLI
Set up default gcloud settings using one of the following methods:
- Using gcloud init, if you want to be walked through setting defaults.
- Using gcloud config, to individually set your project ID, zone, and region.
<Tabs>
<TabItem value="Using gloud init">
1. Run gcloud init and follow the directions:
```
gcloud init
```
If you are using SSH on a remote server, use the --console-only flag to prevent the command from launching a browser:
```
gcloud init --console-only
```
2. Follow the instructions to authorize gcloud to use your Google Cloud account and select the new project that you created.
</TabItem>
<TabItem value="Using gcloud config">
</TabItem>
</Tabs>
## 4. Confirm that gcloud is configured correctly
Run:
```
gcloud config list
```
The output should resemble the following:
```
[compute]
region = us-west1 # Your chosen region
zone = us-west1-b # Your chosen zone
[core]
account = <Your email>
disable_usage_reporting = True
project = <Your project ID>
Your active configuration is: [default]
```
## 5. Create a GKE Cluster
The following command creates a three-node cluster.
Replace `cluster-name` with the name of your new cluster.
When choosing a Kubernetes version, be sure to first consult the [support matrix](https://rancher.com/support-matrix/) to find the highest version of Kubernetes that has been validated for your Rancher version.
```
gcloud container clusters create cluster-name --num-nodes=3 --cluster-version=1.20.10-gke.301
```
## 6. Get Authentication Credentials
After creating your cluster, you need to get authentication credentials to interact with the cluster:
```
gcloud container clusters get-credentials cluster-name
```
This command configures `kubectl` to use the cluster you created.
## 7. Install an Ingress
The cluster needs an Ingress so that Rancher can be accessed from outside the cluster.
The following command installs an `nginx-ingress-controller` with a LoadBalancer service:
```
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm upgrade --install \
ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--set controller.service.type=LoadBalancer \
--version 3.12.0 \
--create-namespace
```
## 8. Get the Load Balancer IP
To get the address of the load balancer, run:
```
kubectl get service ingress-nginx-controller --namespace=ingress-nginx
```
The result should look similar to the following:
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.3.244.156 35.233.206.34 80:31876/TCP,443:32497/TCP 81s
```
Save the `EXTERNAL-IP`.
## 9. Set up DNS
External traffic to the Rancher server will need to be directed at the load balancer you created.
Set up a DNS to point at the external IP that you saved. This DNS will be used as the Rancher server URL.
There are many valid ways to set up the DNS. For help, refer to the Google Cloud documentation about [managing DNS records.](https://cloud.google.com/dns/docs/records)
## 10. Install the Rancher Helm chart
Next, install the Rancher Helm chart by following the instructions on [this page.](install-upgrade-on-a-kubernetes-cluster.md#install-the-rancher-helm-chart) The Helm instructions are the same for installing Rancher on any Kubernetes distribution.
Use the DNS name from the previous step as the Rancher server URL when you install Rancher. It can be passed in as a Helm option. For example, if the DNS name is `rancher.my.org`, you could run the Helm installation command with the option `--set hostname=rancher.my.org`.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rollbacks.md
-94
View File
@@ -1,94 +0,0 @@
---
title: Rollbacks
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/rollbacks"/>
</head>
## Rolling Back to Rancher v2.5.0+
To roll back to Rancher v2.5.0+, use the **Rancher Backups** application and restore Rancher from backup.
Rancher has to be started with the lower/previous version after a rollback.
A restore is performed by creating a Restore custom resource.
> **Important**
>
> * Follow the instructions from this page for restoring rancher on the same cluster where it was backed up from. In order to migrate rancher to a new cluster, follow the steps to [migrate rancher.](../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
> * While restoring Rancher on the same setup, the Rancher deployment is manually scaled down before the restore starts, then the operator will scale it back up once the restore completes. As a result, Rancher and its UI will be unavailable until the restore is complete. While the UI is unavailable, use the original cluster kubeconfig with the restore YAML file: `kubectl create -f restore.yaml`.
### Create the Restore Custom Resource
1. In the **Cluster Explorer,** go to the dropdown menu in the upper left corner and click **Rancher Backups.**
* **Note:** If the Rancher Backups app is not visible in the dropdown, you will need to install it from the Charts page in **Apps & Marketplace**. Refer [here](../../../how-to-guides/new-user-guides/helm-charts-in-rancher.md#charts) for more information.
1. Click **Restore.**
1. Create the Restore with the form or with YAML. For help creating the Restore resource using the online form, refer to the [configuration reference](../../../reference-guides/backup-restore-configuration/restore-configuration.md) and to the [examples.](../../../reference-guides/backup-restore-configuration/examples.md)
1. To use the YAML editor, you can click **Create > Create from YAML.** Enter the Restore YAML. The following is an example Restore custom resource:
```yaml
apiVersion: resources.cattle.io/v1
kind: Restore
metadata:
name: restore-migration
spec:
backupFilename: backup-b0450532-cee1-4aa1-a881-f5f48a007b1c-2020-09-15T07-27-09Z.tar.gz
encryptionConfigSecretName: encryptionconfig
storageLocation:
s3:
credentialSecretName: s3-creds
credentialSecretNamespace: default
bucketName: rancher-backups
folder: rancher
region: us-west-2
endpoint: s3.us-west-2.amazonaws.com
```
For help configuring the Restore, refer to the [configuration reference](../../../reference-guides/backup-restore-configuration/restore-configuration.md) and to the [examples.](../../../reference-guides/backup-restore-configuration/examples.md)
1. Click **Create.**
**Result:** The backup file is created and updated to the target storage location. The resources are restored in this order:
1. Custom Resource Definitions (CRDs)
2. Cluster-scoped resources
3. Namespaced resources
To check how the restore is progressing, you can check the logs of the operator. Follow these steps to get the logs:
```yaml
kubectl get pods -n cattle-resources-system
kubectl logs -n cattle-resources-system -f
```
### Roll back to a previous Rancher version
Rancher can be rolled back using the Helm CLI. To roll back to the previous version:
```yaml
helm rollback rancher -n cattle-system
```
If the previous revision is not the intended target, you can specify a revision to roll back to. To see the deployment history:
```yaml
helm history rancher -n cattle-system
```
When the target revision is determined, perform the rollback. This example will roll back to revision `3`:
```yaml
helm rollback rancher 3 -n cattle-system
```
## Rolling Back to Rancher v2.2-v2.4+
To roll back to Rancher before v2.5, follow the procedure detailed here: [Restoring Backups — Kubernetes installs](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-rancher-launched-kubernetes-clusters-from-backup.md) Restoring a snapshot of the Rancher server cluster will revert Rancher to the version and state at the time of the snapshot.
For information on how to roll back Rancher installed with Docker, refer to [this page.](../other-installation-methods/rancher-on-a-single-node-with-docker/roll-back-docker-installed-rancher.md)
> Managed clusters are authoritative for their state. This means restoring the rancher server will not revert workload deployments or changes made on managed clusters after the snapshot was taken.
## Rolling Back to Rancher v2.0-v2.1
Rolling back to Rancher v2.0-v2.1 is no longer supported. The instructions for rolling back to these versions are preserved [here](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-rancher-launched-kubernetes-clusters-from-backup.md) and are intended to be used only in cases where upgrading to Rancher v2.2+ is not feasible.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/troubleshooting.md
-188
View File
@@ -1,188 +0,0 @@
---
title: Troubleshooting the Rancher Server Kubernetes Cluster
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/troubleshooting"/>
</head>
This section describes how to troubleshoot an installation of Rancher on a Kubernetes cluster.
### Relevant Namespaces
Most of the troubleshooting will be done on objects in these 3 namespaces.
- `cattle-system` - `rancher` deployment and pods.
- `ingress-nginx` - Ingress controller pods and services.
- `cert-manager` - `cert-manager` pods.
### "default backend - 404"
A number of things can cause the ingress-controller not to forward traffic to your rancher instance. Most of the time its due to a bad ssl configuration.
Things to check
- [Is Rancher Running](#check-if-rancher-is-running)
- [Cert CN is "Kubernetes Ingress Controller Fake Certificate"](#cert-cn-is-kubernetes-ingress-controller-fake-certificate)
### Check if Rancher is Running
Use `kubectl` to check the `cattle-system` system namespace and see if the Rancher pods are in a Running state.
```
kubectl -n cattle-system get pods
NAME READY STATUS RESTARTS AGE
pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m
```
If the state is not `Running`, run a `describe` on the pod and check the Events.
```
kubectl -n cattle-system describe pod
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11m default-scheduler Successfully assigned rancher-784d94f59b-vgqzh to localhost
Normal SuccessfulMountVolume 11m kubelet, localhost MountVolume.SetUp succeeded for volume "rancher-token-dj4mt"
Normal Pulling 11m kubelet, localhost pulling image "rancher/rancher:v2.0.4"
Normal Pulled 11m kubelet, localhost Successfully pulled image "rancher/rancher:v2.0.4"
Normal Created 11m kubelet, localhost Created container
Normal Started 11m kubelet, localhost Started container
```
### Check the Rancher Logs
Use `kubectl` to list the pods.
```
kubectl -n cattle-system get pods
NAME READY STATUS RESTARTS AGE
pod/rancher-784d94f59b-vgqzh 1/1 Running 0 10m
```
Use `kubectl` and the pod name to list the logs from the pod.
```
kubectl -n cattle-system logs -f rancher-784d94f59b-vgqzh
```
### Cert CN is "Kubernetes Ingress Controller Fake Certificate"
Use your browser to check the certificate details. If it says the Common Name is "Kubernetes Ingress Controller Fake Certificate", something may have gone wrong with reading or issuing your SSL cert.
> **Note:** if you are using LetsEncrypt to issue certs it can sometimes take a few minutes to issue the cert.
### Checking for issues with cert-manager issued certs (Rancher Generated or LetsEncrypt)
`cert-manager` has 3 parts.
- `cert-manager` pod in the `cert-manager` namespace.
- `Issuer` object in the `cattle-system` namespace.
- `Certificate` object in the `cattle-system` namespace.
Work backwards and do a `kubectl describe` on each object and check the events. You can track down what might be missing.
For example there is a problem with the Issuer:
```
kubectl -n cattle-system describe certificate
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning IssuerNotReady 18s (x23 over 19m) cert-manager Issuer rancher not ready
```
```
kubectl -n cattle-system describe issuer
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ErrInitIssuer 19m (x12 over 19m) cert-manager Error initializing issuer: secret "tls-rancher" not found
Warning ErrGetKeyPair 9m (x16 over 19m) cert-manager Error getting keypair for CA issuer: secret "tls-rancher" not found
```
### Checking for Issues with Your Own SSL Certs
Your certs get applied directly to the Ingress object in the `cattle-system` namespace.
Check the status of the Ingress object and see if its ready.
```
kubectl -n cattle-system describe ingress
```
If its ready and the SSL is still not working you may have a malformed cert or secret.
Check the nginx-ingress-controller logs. Because the nginx-ingress-controller has multiple containers in its pod you will need to specify the name of the container.
```
kubectl -n ingress-nginx logs -f nginx-ingress-controller-rfjrq nginx-ingress-controller
...
W0705 23:04:58.240571 7 backend_ssl.go:49] error obtaining PEM from secret cattle-system/tls-rancher-ingress: error retrieving secret cattle-system/tls-rancher-ingress: secret cattle-system/tls-rancher-ingress was not found
```
### No matches for kind "Issuer"
The SSL configuration option you have chosen requires cert-manager to be installed before installing Rancher or else the following error is shown:
```
Error: validation failed: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1"
```
Install cert-manager and try installing Rancher again.
### Canal Pods show READY 2/3
The most common cause of this issue is port 8472/UDP is not open between the nodes. Check your local firewall, network routing or security groups.
Once the network issue is resolved, the `canal` pods should timeout and restart to establish their connections.
### nginx-ingress-controller Pods show RESTARTS
The most common cause of this issue is the `canal` pods have failed to establish the overlay network. See [canal Pods show READY `2/3`](#canal-pods-show-ready-23) for troubleshooting.
### Failed to dial to /var/run/docker.sock: ssh: rejected: administratively prohibited (open failed)
Some causes of this error include:
* User specified to connect with does not have permission to access the Docker socket. This can be checked by logging into the host and running the command `docker ps`:
```
$ ssh user@server
user@server$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
```
See [Manage Docker as a non-root user](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user) how to set this up properly.
* When using RedHat/CentOS as operating system, you cannot use the user `root` to connect to the nodes because of [Bugzilla #1527565](https://bugzilla.redhat.com/show_bug.cgi?id=1527565). You will need to add a separate user and configure it to access the Docker socket. See [Manage Docker as a non-root user](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user) how to set this up properly.
* SSH server version is not version 6.7 or higher. This is needed for socket forwarding to work, which is used to connect to the Docker socket over SSH. This can be checked using `sshd -V` on the host you are connecting to, or using netcat:
```
$ nc xxx.xxx.xxx.xxx 22
SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.10
```
### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: Error configuring SSH: ssh: no key found
The key file specified as `ssh_key_path` cannot be accessed. Make sure that you specified the private key file (not the public key, `.pub`), and that the user that is running the `rke` command can access the private key file.
### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
The key file specified as `ssh_key_path` is not correct for accessing the node. Double-check if you specified the correct `ssh_key_path` for the node and if you specified the correct user to connect with.
### Failed to dial ssh using address [xxx.xxx.xxx.xxx:xx]: Error configuring SSH: ssh: cannot decode encrypted private keys
If you want to use encrypted private keys, you should use `ssh-agent` to load your keys with your passphrase. If the `SSH_AUTH_SOCK` environment variable is found in the environment where the `rke` command is run, it will be used automatically to connect to the node.
### Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
The node is not reachable on the configured `address` and `port`.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades.md
-192
View File
@@ -1,192 +0,0 @@
---
title: Upgrades
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades"/>
</head>
The following instructions will guide you through upgrading a Rancher server that was installed on a Kubernetes cluster with Helm. These steps also apply to air gap installs with Helm.
For the instructions to upgrade Rancher installed on Kubernetes with RancherD, refer to [this page.](../other-installation-methods/install-rancher-on-linux/upgrade-rancherd.md)
For the instructions to upgrade Rancher installed with Docker, refer to [this page.](../other-installation-methods/rancher-on-a-single-node-with-docker/upgrade-docker-installed-rancher.md)
To upgrade the components in your Kubernetes cluster, or the definition of the [Kubernetes services](https://rancher.com/docs/rke/latest/en/config-options/services/) or [add-ons](https://rancher.com/docs/rke/latest/en/config-options/add-ons/), refer to the [upgrade documentation for RKE](https://rancher.com/docs/rke/latest/en/upgrades/), the Rancher Kubernetes Engine.
## Prerequisites
### Access to kubeconfig
Helm should be run from the same location as your kubeconfig file, or the same location where you run your kubectl commands from.
If you installed Kubernetes with RKE, the config will have been created in the directory you ran `rke up` in.
The kubeconfig can also be manually targeted for the intended cluster with the `--kubeconfig` tag (see: https://helm.sh/docs/helm/helm/)
### Review Known Issues
Review the list of known issues for each Rancher version, which can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
Note that upgrades _to_ or _from_ any chart in the [rancher-alpha repository](../resources/choose-a-rancher-version.md#helm-chart-repositories) aren't supported.
### Helm Version
The upgrade instructions assume you are using Helm 3.
For migration of installs started with Helm 2, refer to the official [Helm 2 to 3 migration docs.](https://helm.sh/blog/migrate-from-helm-v2-to-helm-v3/) The [Helm 2 upgrade page here](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/helm2.md) provides a copy of the older upgrade instructions that used Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible.
### For air gap installs: Populate private registry
For [air gap installs only,](../other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) collect and populate images for the new Rancher server version. Follow the guide to [populate your private registry](../other-installation-methods/air-gapped-helm-cli-install/publish-images.md) with the images for the Rancher version that you want to upgrade to.
### For upgrades from a Rancher server with a hidden local cluster
If you are upgrading to Rancher v2.5 from a Rancher server that was started with the Helm chart option `--add-local=false`, you will need to drop that flag when upgrading. Otherwise, the Rancher server will not start. The `restricted-admin` role can be used to continue restricting access to the local cluster. For more information, see [this section.](../../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#upgrading-from-rancher-with-a-hidden-local-cluster)
### For upgrades with cert-manager older than 0.8.0
[Let's Encrypt will be blocking cert-manager instances older than 0.8.0 starting November 1st 2019.](https://community.letsencrypt.org/t/blocking-old-cert-manager-versions/98753) Upgrade cert-manager to the latest version by following [these instructions.](../resources/upgrade-cert-manager.md)
## Upgrade Outline
Follow the steps to upgrade Rancher server:
- [1. Back up your Kubernetes cluster that is running Rancher server](#1-back-up-your-kubernetes-cluster-that-is-running-rancher-server)
- [2. Update the Helm chart repository](#2-update-the-helm-chart-repository)
- [3. Upgrade Rancher](#3-upgrade-rancher)
- [4. Verify the Upgrade](#4-verify-the-upgrade)
## 1. Back up Your Kubernetes Cluster that is Running Rancher Server
Use the [backup application](../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md) to back up Rancher.
You'll use the backup as a restoration point if something goes wrong during upgrade.
## 2. Update the Helm chart repository
1. Update your local Helm repo cache:
```
helm repo update
```
1. Get the repository name that you used to install Rancher.
For information about the repos and their differences, see [Helm Chart Repositories](../resources/choose-a-rancher-version.md#helm-chart-repositories).
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
Note: Upgrades are not supported to, from, or between Alphas.
```
helm repo list
NAME URL
stable https://charts.helm.sh/stable
rancher-<CHART_REPO> https://releases.rancher.com/server-charts/<CHART_REPO>
```
> **Note:** If you want to switch to a different Helm chart repository, please follow the [steps on how to switch repositories](../resources/choose-a-rancher-version.md#switching-to-a-different-helm-chart-repository). If you switch repositories, make sure to list the repositories again before continuing onto Step 3 to ensure you have the correct one added.
1. Fetch the latest chart to install Rancher from the Helm chart repository.
This command will pull down the latest charts and save it in the current directory as a `.tgz` file.
```plain
helm fetch rancher-<CHART_REPO>/rancher
```
You can fetch the chart for the specific version you are upgrading to by adding in the `--version=` tag. For example:
```plain
helm fetch rancher-<CHART_REPO>/rancher --version=2.5.16
```
## 3. Upgrade Rancher
This section describes how to upgrade normal (Internet-connected) or air gap installations of Rancher with Helm.
> **Air Gap Instructions:** If you are installing Rancher in an air gapped environment, skip the rest of this page and render the Helm template by following the instructions on [this page.](air-gapped-upgrades.md)
Get the values, which were passed with `--set`, from the current Rancher Helm chart that is installed.
```
helm get values rancher -n cattle-system
hostname: rancher.my.org
```
> **Note:** There will be more values that are listed with this command. This is just an example of one of the values.
If you are also upgrading cert-manager to the latest version from a version older than 0.11.0, follow [Option B: Reinstalling Rancher and cert-manager.](#option-b-reinstalling-rancher-and-cert-manager)
Otherwise, follow [Option A: Upgrading Rancher.](#option-a-upgrading-rancher)
### Option A: Upgrading Rancher
Upgrade Rancher to the latest version with all your settings.
Take all the values from the previous step and append them to the command using `--set key=value`:
```
helm upgrade rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org
```
> **Note:** The above is an example, there may be more values from the previous step that need to be appended.
Alternatively, it's possible to export the current values to a file and reference that file during upgrade. For example, to only change the Rancher version:
```
helm get values rancher -n cattle-system -o yaml > values.yaml
helm upgrade rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
-f values.yaml \
--version=2.5.16
```
### Option B: Reinstalling Rancher and cert-manager
If you are currently running the cert-manager whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manager due to the API change in cert-manager v0.11.
1. Uninstall Rancher
```
helm delete rancher -n cattle-system
```
2. Uninstall and reinstall `cert-manager` according to the instructions on the [Upgrading Cert-Manager](../resources/upgrade-cert-manager.md) page.
3. Reinstall Rancher to the latest version with all your settings. Take all the values from the step 1 and append them to the command using `--set key=value`. Note: There will be many more options from the step 1 that need to be appended.
```
helm install rancher rancher-<CHART_REPO>/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org
```
## 4. Verify the Upgrade
Log into Rancher to confirm that the upgrade succeeded.
>**Having network issues following upgrade?**
>
> See [Restoring Cluster Networking](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/namespace-migration.md).
## Known Upgrade Issues
A list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/installation-and-upgrade.md
-111
View File
@@ -1,111 +0,0 @@
---
title: Installing/Upgrading Rancher
description: Learn how to install Rancher in development and production environments. Read about single node and high availability installation
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade"/>
</head>
This section provides an overview of the architecture options of installing Rancher, describing advantages of each option.
## Terminology
In this section,
- **The Rancher server** manages and provisions Kubernetes clusters. You can interact with downstream Kubernetes clusters through the Rancher server's user interface.
- **RKE (Rancher Kubernetes Engine)** is a certified Kubernetes distribution and CLI/library which creates and manages a Kubernetes cluster.
- **K3s (Lightweight Kubernetes)** is also a fully compliant Kubernetes distribution. It is newer than RKE, easier to use, and more lightweight, with a binary size of less than 100 MB.
- **RKE2** is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.
- **RancherD** was an experimental tool for installing Rancher; a single binary that first launched an RKE2 Kubernetes cluster, then installed the Rancher server Helm chart on the cluster. It was available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
## Changes to Installation in Rancher v2.5
In Rancher v2.5, the Rancher management server can be installed on any Kubernetes cluster, including hosted clusters, such as Amazon EKS clusters.
For Docker installations, a local Kubernetes cluster is installed in the single Docker container, and Rancher is installed on the local cluster.
The `restrictedAdmin` Helm chart option was added. When this option is set to true, the initial Rancher user has restricted access to the local Kubernetes cluster to prevent privilege escalation. For more information, see the section about the [restricted-admin role.](../../how-to-guides/advanced-user-guides/authentication-permissions-and-global-configuration/manage-role-based-access-control-rbac/global-permissions.md#restricted-admin)
## Overview of Installation Options
Rancher can be installed on these main architectures:
### High-availability Kubernetes Install with the Helm CLI
We recommend using Helm, a Kubernetes package manager, to install Rancher on multiple nodes on a dedicated Kubernetes cluster. For RKE clusters, three nodes are required to achieve a high-availability cluster. For K3s clusters, only two nodes are required.
### High-availability Kubernetes Install with RancherD
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
RancherD is a single binary that first launches an RKE2 Kubernetes cluster, then installs the Rancher server Helm chart on the cluster.
In both the RancherD install and the Helm CLI install, Rancher is installed as a Helm chart on a Kubernetes cluster.
Configuration and upgrading are also simplified with RancherD. When you upgrade the RancherD binary, both the Kubernetes cluster and the Rancher Helm chart are upgraded.
### Single-node Kubernetes Install
Rancher can be installed on a single-node Kubernetes cluster. In this case, the Rancher server doesn't have high availability, which is important for running Rancher in production.
However, this option is useful if you want to save resources by using a single node in the short term, while preserving a high-availability migration path. In the future, you can add nodes to the cluster to get a high-availability Rancher server.
### Docker Install
For test and demonstration purposes, Rancher can be installed with Docker on a single node.
The Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster.](../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
### Other Options
There are also separate instructions for installing Rancher in an air gap environment or behind an HTTP proxy:
| Level of Internet Access | Kubernetes Installation - Strongly Recommended | Docker Installation |
| ---------------------------------- | ------------------------------ | ---------- |
| With direct access to the Internet | [Docs](install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) | [Docs](other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) |
| Behind an HTTP proxy | [Docs](other-installation-methods/rancher-behind-an-http-proxy/rancher-behind-an-http-proxy.md) | These [docs,](other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) plus this [configuration](../../reference-guides/single-node-rancher-in-docker/http-proxy-configuration.md) |
| In an air gap environment | [Docs](other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) | [Docs](other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) |
We recommend installing Rancher on a Kubernetes cluster, because in a multi-node cluster, the Rancher management server becomes highly available. This high-availability configuration helps maintain consistent access to the downstream Kubernetes clusters that Rancher will manage.
For that reason, we recommend that for a production-grade architecture, you should set up a high-availability Kubernetes cluster, then install Rancher on it. After Rancher is installed, you can use Rancher to deploy and manage Kubernetes clusters.
For testing or demonstration purposes, you can install Rancher in single Docker container. In this Docker install, you can use Rancher to set up Kubernetes clusters out-of-the-box. The Docker install allows you to explore the Rancher server functionality, but it is intended to be used for development and testing purposes only.
Our [instructions for installing Rancher on Kubernetes](install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) describe how to first use K3s or RKE to create and manage a Kubernetes cluster, then install Rancher onto that cluster.
When the nodes in your Kubernetes cluster are running and fulfill the [node requirements,](installation-requirements/installation-requirements.md) you will use Helm to deploy Rancher onto Kubernetes. Helm uses Rancher's Helm chart to install a replica of Rancher on each node in the Kubernetes cluster. We recommend using a load balancer to direct traffic to each replica of Rancher in the cluster.
For a longer discussion of Rancher architecture, refer to the [architecture overview,](../../reference-guides/rancher-manager-architecture/rancher-manager-architecture.md) [recommendations for production-grade architecture,](../../reference-guides/rancher-manager-architecture/architecture-recommendations.md) or our [best practices guide.](../../reference-guides/best-practices/rancher-server/tips-for-running-rancher.md)
## Prerequisites
Before installing Rancher, make sure that your nodes fulfill all of the [installation requirements.](installation-requirements/installation-requirements.md)
## Architecture Tip
For the best performance and greater security, we recommend a separate, dedicated Kubernetes cluster for the Rancher management server. Running user workloads on this cluster is not advised. After deploying Rancher, you can [create or import clusters](../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) for running your workloads.
For more architecture recommendations, refer to [this page.](../../reference-guides/rancher-manager-architecture/architecture-recommendations.md)
### More Options for Installations on a Kubernetes Cluster
Refer to the [Helm chart options](../../reference-guides/installation-references/helm-chart-options.md) for details on installing Rancher on a Kubernetes cluster with other configurations, including:
- With [API auditing to record all transactions](../../reference-guides/installation-references/helm-chart-options.md#api-audit-log)
- With [TLS termination on a load balancer](../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination)
- With a [custom Ingress](../../reference-guides/installation-references/helm-chart-options.md#customizing-your-ingress)
In the Rancher installation instructions, we recommend using K3s or RKE to set up a Kubernetes cluster before installing Rancher on the cluster. Both K3s and RKE have many configuration options for customizing the Kubernetes cluster to suit your specific environment. For the full list of their capabilities, refer to their documentation:
- [RKE configuration options](https://rancher.com/docs/rke/latest/en/config-options/)
- [K3s configuration options](https://rancher.com/docs/k3s/latest/en/installation/install-options/)
### More Options for Installations with Docker
Refer to the [docs about options for Docker installs](other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) for details about other configurations including:
- With [API auditing to record all transactions](../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log)
- With an [external load balancer](advanced-options/advanced-use-cases/configure-layer-7-nginx-load-balancer.md/)
- With a [persistent data store](../../reference-guides/single-node-rancher-in-docker/advanced-options.md#persistent-data)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/installation-requirements/install-docker.md
-21
View File
@@ -1,21 +0,0 @@
---
title: Installing Docker
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/installation-requirements/install-docker"/>
</head>
Docker is required to be installed on nodes where the Rancher server will be installed with Helm or Docker.
There are a couple of options for installing Docker. One option is to refer to the [official Docker documentation](https://docs.docker.com/install/) about how to install Docker on Linux. The steps will vary based on the Linux distribution.
Another option is to use one of Rancher's Docker installation scripts, which are available for most recent versions of Docker. Rancher has installation scripts for every version of upstream Docker that Kubernetes supports.
For example, this command could be used to install on one of the main Linux distributions, such as SUSE Linux Enterprise or Ubuntu:
```bash
curl https://releases.rancher.com/install-docker/<version-number>.sh | sh
```
Consult the [Rancher support matrix](https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-5-16/) to match a validated Docker version with your operating system and version of Rancher. Although the support matrix lists validated Docker versions down to the patch version, only the major and minor version of the release are relevant for the Docker installation scripts.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/installation-requirements/installation-requirements.md
-195
View File
@@ -1,195 +0,0 @@
---
title: Installation Requirements
description: Learn the node requirements for each node running Rancher server when you’re configuring Rancher to run either in a Docker or Kubernetes setup
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/installation-requirements"/>
</head>
This page describes the software, hardware, and networking requirements for the nodes where the Rancher server will be installed. The Rancher server can be installed on a single node or a high-availability Kubernetes cluster.
:::note Important:
If you install Rancher on a Kubernetes cluster, requirements are different from the [node requirements for downstream user clusters,](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md) which will run your apps and services.
:::
The Rancher UI works best in Firefox or Chromium based browsers (Chrome, Edge, Opera, Brave, etc).
See our page on [best practices](../../../reference-guides/best-practices/rancher-server/tips-for-running-rancher.md) for a list of recommendations for running a Rancher server in production.
## Kubernetes Compatibility with Rancher
Rancher needs to be installed on a supported Kubernetes version. Consult the [Rancher support matrix](https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-5-16/) to ensure that your intended version of Kubernetes is supported.
## Operating Systems and Container Runtime Requirements
All supported operating systems are 64-bit x86. Rancher should work with any modern Linux distribution.
The [Rancher support matrix](https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-5-16/) lists which OS and Docker versions were tested for each Rancher version.
Docker is required for nodes that will run RKE clusters. It is not required for RKE2 or K3s clusters.
The `ntp` (Network Time Protocol) package should be installed. This prevents errors with certificate validation that can occur when the time is not synchronized between the client and server.
Some distributions of Linux may have default firewall rules that block communication with Helm. We recommend disabling firewalld. For Kubernetes 1.19 and 1.20, firewalld must be turned off.
> If you don't feel comfortable doing so you might check suggestions in the [respective issue](https://github.com/rancher/rancher/issues/28840). Some users were successful [creating a separate firewalld zone with a policy of ACCEPT for the Pod CIDR](https://github.com/rancher/rancher/issues/28840#issuecomment-787404822).
If you plan to run Rancher on ARM64, see [Running on ARM64 (Experimental).](../advanced-options/enable-experimental-features/rancher-on-arm64.md)
### RKE Specific Requirements
For the container runtime, RKE should work with any modern Docker version.
Note that the following sysctl setting must be applied:
```
net.bridge.bridge-nf-call-iptables=1
```
### K3s Specific Requirements
For the container runtime, K3s should work with any modern version of Docker or containerd.
Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [Rancher support matrix](https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-5-16/). To specify the K3s version, use the INSTALL_K3S_VERSION environment variable when running the K3s installation script.
If you are installing Rancher on a K3s cluster with **Raspbian Buster**, follow [these steps](https://rancher.com/docs/k3s/latest/en/advanced/#enabling-legacy-iptables-on-raspbian-buster) to switch to legacy iptables.
If you are installing Rancher on a K3s cluster with Alpine Linux, follow [these steps](https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup) for additional setup.
### RancherD Specific Requirements
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
At this time, only Linux OSes that leverage systemd are supported.
To install RancherD on SELinux Enforcing CentOS 8 or RHEL 8 nodes, some [additional steps](#rancherd-on-selinux-enforcing-centos-8-or-rhel-8-nodes) are required.
Docker is not required for RancherD installs.
### RKE2 Specific Requirements
_The RKE2 install is available as of v2.5.6._
For details on which OS versions were tested with RKE2, refer to the [Rancher support matrix](https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-5-16/).
Docker is not required for RKE2 installs.
The Ingress should be deployed as DaemonSet to ensure your load balancer can successfully route traffic to all nodes. Currently, RKE2 deploys nginx-ingress as a deployment by default, so you will need to deploy it as a DaemonSet by following [these steps.](../../../how-to-guides/new-user-guides/kubernetes-cluster-setup/rke2-for-rancher.md#5-configure-nginx-to-be-a-daemonset)
### Installing Docker
Docker is required for Helm chart installs, and it can be installed by following the steps in the official [Docker documentation.](https://docs.docker.com/) Rancher also provides [scripts](install-docker.md) to install Docker with one command.
Docker is not required for RancherD installs.
## Hardware Requirements
The following sections describe the CPU, memory, and disk requirements for the nodes where the Rancher server is installed.
## CPU and Memory
Hardware requirements scale based on the size of your Rancher deployment. Provision each individual node according to the requirements. The requirements are different depending on if you are installing Rancher in a single container with Docker, or if you are installing Rancher on a Kubernetes cluster.
### RKE and Hosted Kubernetes
These CPU and memory requirements apply to each host in the Kubernetes cluster where the Rancher server is installed.
These requirements apply to RKE Kubernetes clusters, as well as to hosted Kubernetes clusters such as EKS.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | ---------- | ------------ | -------| ------- |
| Small | Up to 150 | Up to 1500 | 2 | 8 GB |
| Medium | Up to 300 | Up to 3000 | 4 | 16 GB |
| Large | Up to 500 | Up to 5000 | 8 | 32 GB |
| X-Large | Up to 1000 | Up to 10,000 | 16 | 64 GB |
| XX-Large | Up to 2000 | Up to 20,000 | 32 | 128 GB |
Every use case and environment is different. Please [contact Rancher](https://rancher.com/contact/) to review yours.
### K3s Kubernetes
These CPU and memory requirements apply to each host in a [K3s Kubernetes cluster where the Rancher server is installed.](../install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md)
| Deployment Size | Clusters | Nodes | vCPUs | RAM | Database Size |
| --------------- | ---------- | ------------ | -------| ---------| ------------------------- |
| Small | Up to 150 | Up to 1500 | 2 | 8 GB | 2 cores, 4 GB + 1000 IOPS |
| Medium | Up to 300 | Up to 3000 | 4 | 16 GB | 2 cores, 4 GB + 1000 IOPS |
| Large | Up to 500 | Up to 5000 | 8 | 32 GB | 2 cores, 4 GB + 1000 IOPS |
| X-Large | Up to 1000 | Up to 10,000 | 16 | 64 GB | 2 cores, 4 GB + 1000 IOPS |
| XX-Large | Up to 2000 | Up to 20,000 | 32 | 128 GB | 2 cores, 4 GB + 1000 IOPS |
Every use case and environment is different. Please [contact Rancher](https://rancher.com/contact/) to review yours.
### RancherD
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
These CPU and memory requirements apply to each instance with RancherD installed. Minimum recommendations are outlined here.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | -------- | --------- | ----- | ---- |
| Small | Up to 5 | Up to 50 | 2 | 5 GB |
| Medium | Up to 15 | Up to 200 | 3 | 9 GB |
### RKE2 Kubernetes
These CPU and memory requirements apply to each instance with RKE2 installed. Minimum recommendations are outlined here.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | -------- | --------- | ----- | ---- |
| Small | Up to 5 | Up to 50 | 2 | 5 GB |
| Medium | Up to 15 | Up to 200 | 3 | 9 GB |
### Docker
These CPU and memory requirements apply to a host with a [single-node](../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) installation of Rancher.
| Deployment Size | Clusters | Nodes | vCPUs | RAM |
| --------------- | -------- | --------- | ----- | ---- |
| Small | Up to 5 | Up to 50 | 1 | 4 GB |
| Medium | Up to 15 | Up to 200 | 2 | 8 GB |
## Ingress
Each node in the Kubernetes cluster that Rancher is installed on should run an Ingress.
The Ingress should be deployed as DaemonSet to ensure your load balancer can successfully route traffic to all nodes.
For RKE, K3s and RancherD installations, you don't have to install the Ingress manually because it is installed by default.
For hosted Kubernetes clusters (EKS, GKE, AKS) and RKE2 installations, you will need to set up the ingress.
### Ingress for RKE2
Currently, RKE2 deploys nginx-ingress as a deployment by default, so you will need to deploy it as a DaemonSet by following [these steps.](../../../how-to-guides/new-user-guides/kubernetes-cluster-setup/rke2-for-rancher.md#5-configure-nginx-to-be-a-daemonset)
### Ingress for EKS
For an example of how to deploy an nginx-ingress-controller with a LoadBalancer service, refer to [this section.](../install-upgrade-on-a-kubernetes-cluster/rancher-on-amazon-eks.md#5-install-an-ingress)
## Disks
Rancher performance depends on etcd in the cluster performance. To ensure optimal speed, we recommend always using SSD disks to back your Rancher management Kubernetes cluster. On cloud providers, you will also want to use the minimum size that allows the maximum IOPS. In larger clusters, consider using dedicated storage devices for etcd data and wal directories.
## Networking Requirements
This section describes the networking requirements for the node(s) where the Rancher server is installed.
### Node IP Addresses
Each node used should have a static IP configured, regardless of whether you are installing Rancher on a single node or on an HA cluster. In case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.
### Port Requirements
To operate properly, Rancher requires a number of ports to be open on Rancher nodes and on downstream Kubernetes cluster nodes. [Port Requirements](port-requirements.md) lists all the necessary ports for Rancher and Downstream Clusters for the different cluster types.
## RancherD on SELinux Enforcing CentOS 8 or RHEL 8 Nodes
Before installing Rancher on SELinux Enforcing CentOS 8 nodes or RHEL 8 nodes, you must install `container-selinux` and `iptables`:
```
sudo yum install iptables
sudo yum install container-selinux
```
versioned_docs/version-2.5/getting-started/installation-and-upgrade/installation-requirements/port-requirements.md
-321
View File
@@ -1,321 +0,0 @@
---
title: Port Requirements
description: Read about port requirements needed in order for Rancher to operate properly, both for Rancher nodes and downstream Kubernetes cluster nodes
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/installation-requirements/port-requirements"/>
</head>
import PortsIaasNodes from '@site/src/components/PortsIaasNodes'
import PortsCustomNodes from '@site/src/components/PortsCustomNodes'
import PortsImportedHosted from '@site/src/components/PortsImportedHosted'
To operate properly, Rancher requires a number of ports to be open on Rancher nodes and on downstream Kubernetes cluster nodes.
## Rancher Nodes
The following table lists the ports that need to be open to and from nodes that are running the Rancher server.
The port requirements differ based on the Rancher server architecture.
As of Rancher v2.5, Rancher can be installed on any Kubernetes cluster. For Rancher installs on a K3s, RKE, or RKE2 Kubernetes cluster, refer to the tabs below. For other Kubernetes distributions, refer to the distribution's documentation for the port requirements for cluster nodes.
> **Notes:**
>
> - Rancher nodes may also require additional outbound access for any external authentication provider which is configured (LDAP for example).
> - Kubernetes recommends TCP 30000-32767 for node port services.
> - For firewalls, traffic may need to be enabled within the cluster and pod CIDR.
### Ports for Rancher Server Nodes on K3s
<details>
<summary>Click to expand</summary>
The K3s server needs port 6443 to be accessible by the nodes.
The nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used. The node should not listen on any other port. K3s uses reverse tunneling such that the nodes make outbound connections to the server and all kubelet traffic runs through that tunnel. However, if you do not use Flannel and provide your own custom CNI, then port 8472 is not needed by K3s.
If you wish to utilize the metrics server, you will need to open port 10250 on each node.
> **Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472.
The following tables break down the port requirements for inbound and outbound traffic:
<figcaption>Inbound Rules for Rancher Server Nodes</figcaption>
| Protocol | Port | Source | Description
|-----|-----|----------------|---|
| TCP | 80 | Load balancer/proxy that does external SSL termination | Rancher UI/API when external SSL termination is used |
| TCP | 443 | <ul><li>server nodes</li><li>agent nodes</li><li>hosted/registered Kubernetes</li><li>any source that needs to be able to use the Rancher UI or API</li></ul> | Rancher agent, Rancher UI/API, kubectl |
| TCP | 6443 | K3s server nodes | Kubernetes API
| UDP | 8472 | K3s server and agent nodes | Required only for Flannel VXLAN.
| TCP | 10250 | K3s server and agent nodes | kubelet
<figcaption>Outbound Rules for Rancher Nodes</figcaption>
| Protocol | Port | Destination | Description |
| -------- | ---- | -------------------------------------------------------- | --------------------------------------------- |
| TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
| TCP | 443 | git.rancher.io | Rancher catalog |
| TCP | 2376 | Any node IP from a node created using Node driver | Docker daemon TLS port used by Docker Machine |
| TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |
</details>
### Ports for Rancher Server Nodes on RKE
<details>
<summary>Click to expand</summary>
Typically Rancher is installed on three RKE nodes that all have the etcd, control plane and worker roles.
The following tables break down the port requirements for traffic between the Rancher nodes:
<figcaption>Rules for traffic between Rancher nodes</figcaption>
| Protocol | Port | Description |
|-----|-----|----------------|
| TCP | 443 | Rancher agents |
| TCP | 2379 | etcd client requests |
| TCP | 2380 | etcd peer communication |
| TCP | 6443 | Kubernetes apiserver |
| UDP | 8472 | Canal/Flannel VXLAN overlay networking |
| TCP | 9099 | Canal/Flannel livenessProbe/readinessProbe |
| TCP | 10250 | Metrics server communication with all nodes |
| TCP | 10254 | Ingress controller livenessProbe/readinessProbe |
The following tables break down the port requirements for inbound and outbound traffic:
<figcaption>Inbound Rules for Rancher Nodes</figcaption>
| Protocol | Port | Source | Description |
|-----|-----|----------------|---|
| TCP | 22 | RKE CLI | SSH provisioning of node by RKE |
| TCP | 80 | Load Balancer/Reverse Proxy | HTTP traffic to Rancher UI/API |
| TCP | 443 | <ul><li>Load Balancer/Reverse Proxy</li><li>IPs of all cluster nodes and other API/UI clients</li></ul> | HTTPS traffic to Rancher UI/API |
| TCP | 6443 | Kubernetes API clients | HTTPS traffic to Kubernetes API |
<figcaption>Outbound Rules for Rancher Nodes</figcaption>
| Protocol | Port | Destination | Description |
|-----|-----|----------------|---|
| TCP | 443 | `35.160.43.145`,`35.167.242.46`,`52.33.59.17` | Rancher catalog (git.rancher.io) |
| TCP | 22 | Any node created using a node driver | SSH provisioning of node by node driver |
| TCP | 2376 | Any node created using a node driver | Docker daemon TLS port used by node driver |
| TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |
| TCP | Provider dependent | Port of the Kubernetes API endpoint in hosted cluster | Kubernetes API |
</details>
### Ports for Rancher Server Nodes on RancherD or RKE2
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
<details>
<summary>Click to expand</summary>
The RancherD (or RKE2) server needs port 6443 and 9345 to be accessible by other nodes in the cluster.
All nodes need to be able to reach other nodes over UDP port 8472 when Flannel VXLAN is used.
If you wish to utilize the metrics server, you will need to open port 10250 on each node.
**Important:** The VXLAN port on nodes should not be exposed to the world as it opens up your cluster network to be accessed by anyone. Run your nodes behind a firewall/security group that disables access to port 8472.
<figcaption>Inbound Rules for RancherD or RKE2 Server Nodes</figcaption>
| Protocol | Port | Source | Description
|-----|-----|----------------|---|
| TCP | 9345 | RancherD/RKE2 agent nodes | Kubernetes API
| TCP | 6443 | RancherD/RKE2 agent nodes | Kubernetes API
| UDP | 8472 | RancherD/RKE2 server and agent nodes | Required only for Flannel VXLAN
| TCP | 10250 | RancherD/RKE2 server and agent nodes | kubelet
| TCP | 2379 | RancherD/RKE2 server nodes | etcd client port
| TCP | 2380 | RancherD/RKE2 server nodes | etcd peer port
| TCP | 30000-32767 | RancherD/RKE2 server and agent nodes | NodePort port range
| HTTP | 8080 | Load balancer/proxy that does external SSL termination | Rancher UI/API when external SSL termination is used |
| HTTPS | 8443 | <ul><li>hosted/registered Kubernetes</li><li>any source that needs to be able to use the Rancher UI or API</li></ul> | Rancher agent, Rancher UI/API, kubectl. Not needed if you have LB doing TLS termination. |
Typically all outbound traffic is allowed.
</details>
### Ports for Rancher Server in Docker
<details>
<summary>Click to expand</summary>
The following tables break down the port requirements for Rancher nodes, for inbound and outbound traffic:
<figcaption>Inbound Rules for Rancher Node</figcaption>
| Protocol | Port | Source | Description
|-----|-----|----------------|---|
| TCP | 80 | Load balancer/proxy that does external SSL termination | Rancher UI/API when external SSL termination is used
| TCP | 443 | <ul><li>hosted/registered Kubernetes</li><li>any source that needs to be able to use the Rancher UI or API</li></ul> | Rancher agent, Rancher UI/API, kubectl
<figcaption>Outbound Rules for Rancher Node</figcaption>
| Protocol | Port | Source | Description |
|-----|-----|----------------|---|
| TCP | 22 | Any node IP from a node created using Node Driver | SSH provisioning of nodes using Node Driver |
| TCP | 443 | git.rancher.io | Rancher catalog |
| TCP | 2376 | Any node IP from a node created using a node driver | Docker daemon TLS port used by Docker Machine |
| TCP | 6443 | Hosted/Imported Kubernetes API | Kubernetes API server |
</details>
## Downstream Kubernetes Cluster Nodes
Downstream Kubernetes clusters run your apps and services. This section describes what ports need to be opened on the nodes in downstream clusters so that Rancher can communicate with them.
The port requirements differ depending on how the downstream cluster was launched. Each of the tabs below list the ports that need to be opened for different [cluster types](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md).
The following diagram depicts the ports that are opened for each [cluster type](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md).
<figcaption>Port Requirements for the Rancher Management Plane</figcaption>
![Basic Port Requirements](/img/port-communications.svg)
>**Tip:**
>
>If security isn't a large concern and you're okay with opening a few additional ports, you can use the table in [Commonly Used Ports](#commonly-used-ports) as your port reference instead of the comprehensive tables below.
### Ports for Rancher Launched Kubernetes Clusters using Node Pools
<details>
<summary>Click to expand</summary>
The following table depicts the port requirements for [Rancher Launched Kubernetes](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) with nodes created in an [Infrastructure Provider](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/use-new-nodes-in-an-infra-provider.md).
>**Note:**
>The required ports are automatically opened by Rancher during creation of clusters in cloud providers like Amazon EC2 or DigitalOcean.
<PortsIaasNodes/>
</details>
### Ports for Rancher Launched Kubernetes Clusters using Custom Nodes
<details>
<summary>Click to expand</summary>
The following table depicts the port requirements for [Rancher Launched Kubernetes](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) with [Custom Nodes](../../../reference-guides/cluster-configuration/rancher-server-configuration/use-existing-nodes/use-existing-nodes.md).
<PortsCustomNodes/>
</details>
### Ports for Hosted Kubernetes Clusters
<details>
<summary>Click to expand</summary>
The following table depicts the port requirements for [hosted clusters](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/set-up-clusters-from-hosted-kubernetes-providers/set-up-clusters-from-hosted-kubernetes-providers.md).
<PortsImportedHosted/>
</details>
### Ports for Registered Clusters
Note: Registered clusters were called imported clusters before Rancher v2.5.
<details>
<summary>Click to expand</summary>
The following table depicts the port requirements for [registered clusters](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/register-existing-clusters.md).
<PortsImportedHosted/>
</details>
## Other Port Considerations
### Commonly Used Ports
These ports are typically opened on your Kubernetes nodes, regardless of what type of cluster it is.
import CommonPortsTable from '../../../shared-files/_common-ports-table.md';
<CommonPortsTable />
----
### Local Node Traffic
Ports marked as `local traffic` (i.e., `9099 TCP`) in the above requirements are used for Kubernetes healthchecks (`livenessProbe` and`readinessProbe`).
These healthchecks are executed on the node itself. In most cloud environments, this local traffic is allowed by default.
However, this traffic may be blocked when:
- You have applied strict host firewall policies on the node.
- You are using nodes that have multiple interfaces (multihomed).
In these cases, you have to explicitly allow this traffic in your host firewall, or in case of public/private cloud hosted machines (i.e. AWS or OpenStack), in your security group configuration. Keep in mind that when using a security group as source or destination in your security group, explicitly opening ports only applies to the private interface of the nodes / instances.
### Rancher AWS EC2 Security Group
When using the [AWS EC2 node driver](../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster.md) to provision cluster nodes in Rancher, you can choose to let Rancher create a security group called `rancher-nodes`. The following rules are automatically added to this security group.
| Type | Protocol | Port Range | Source/Destination | Rule Type |
|-----------------|:--------:|:-----------:|------------------------|:---------:|
| SSH | TCP | 22 | 0.0.0.0/0 | Inbound |
| HTTP | TCP | 80 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 443 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 2376 | 0.0.0.0/0 | Inbound |
| Custom TCP Rule | TCP | 2379-2380 | sg-xxx (rancher-nodes) | Inbound |
| Custom UDP Rule | UDP | 4789 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 6443 | 0.0.0.0/0 | Inbound |
| Custom UDP Rule | UDP | 8472 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 10250-10252 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 10256 | sg-xxx (rancher-nodes) | Inbound |
| Custom TCP Rule | TCP | 30000-32767 | 0.0.0.0/0 | Inbound |
| Custom UDP Rule | UDP | 30000-32767 | 0.0.0.0/0 | Inbound |
| All traffic | All | All | 0.0.0.0/0 | Outbound |
### Opening SUSE Linux Ports
SUSE Linux may have a firewall that blocks all ports by default. To open the ports needed for adding the host to a custom cluster,
<Tabs>
<TabItem value="SLES 15 / openSUSE Leap 15">
1. SSH into the instance.
1. Start YaST in text mode:
```
sudo yast2
```
1. Navigate to **Security and Users** > **Firewall** > **Zones:public** > **Ports**. To navigate within the interface, follow these [instructions](https://doc.opensuse.org/documentation/leap/reference/html/book-reference/cha-yast-text.html#sec-yast-cli-navigate).
1. To open the required ports, enter them into the **TCP Ports** and **UDP Ports** fields. In this example, ports 9796 and 10250 are also opened for monitoring. The resulting fields should look similar to the following:
```yaml
TCP Ports
22, 80, 443, 2376, 2379, 2380, 6443, 9099, 9796, 10250, 10254, 30000-32767
UDP Ports
8472, 30000-32767
```
1. When all required ports are enter, select **Accept**.
</TabItem>
<TabItem value="SLES 12 / openSUSE Leap 42">
1. SSH into the instance.
1. Edit /`etc/sysconfig/SuSEfirewall2` and open the required ports. In this example, ports 9796 and 10250 are also opened for monitoring:
```
FW_SERVICES_EXT_TCP="22 80 443 2376 2379 2380 6443 9099 9796 10250 10254 30000:32767"
FW_SERVICES_EXT_UDP="8472 30000:32767"
FW_ROUTE=yes
```
1. Restart the firewall with the new ports:
```
SuSEfirewall2
```
</TabItem>
</Tabs>
**Result:** The node has the open ports required to be added to a custom cluster.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/air-gapped-helm-cli-install.md
-30
View File
@@ -1,30 +0,0 @@
---
title: Air Gapped Helm CLI Install
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install"/>
</head>
This section is about using the Helm CLI to install the Rancher server in an air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy.
The installation steps differ depending on whether Rancher is installed on an RKE Kubernetes cluster, a K3s Kubernetes cluster, or a single Docker container.
For more information on each installation option, refer to [this page.](../../installation-and-upgrade.md)
Throughout the installation instructions, there will be _tabs_ for each installation option.
> **Important:** If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker Installation to a Kubernetes Installation.
## Installation Outline
1. [Set up infrastructure and private registry](infrastructure-private-registry.md)
2. [Collect and publish images to your private registry](publish-images.md)
3. [Set up a Kubernetes cluster (Skip this step for Docker installations)](install-kubernetes.md)
4. [Install Rancher](install-rancher-ha.md)
## Upgrades
To upgrade Rancher with Helm CLI in an air gap environment, follow [this procedure.](../../install-upgrade-on-a-kubernetes-cluster/upgrades.md)
### [Next: Prepare your Node(s)](infrastructure-private-registry.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/docker-install-commands.md
-137
View File
@@ -1,137 +0,0 @@
---
title: Docker Install Commands
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/docker-install-commands"/>
</head>
The Docker installation is for Rancher users who want to test out Rancher.
Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server.
For Rancher v2.5+, the backup application can be used to migrate the Rancher server from a Docker install to a Kubernetes install using [these steps.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
| Environment Variable Key | Environment Variable Value | Description |
| -------------------------------- | -------------------------------- | ---- |
| `CATTLE_SYSTEM_DEFAULT_REGISTRY` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `CATTLE_SYSTEM_CATALOG` | `bundled` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. |
> **Do you want to...**
>
> - Configure custom CA root certificate to access your services? See [Custom CA root certificate](../../resources/custom-ca-root-certificates.md/).
> - Record all transactions with the Rancher API? See [API Auditing](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log).
Choose from the following options:
## Option A: Default Self-Signed Certificate
<details id="option-a">
<summary>Click to expand</summary>
If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself.
Log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to install. |
As of Rancher v2.5, privileged access is [required.](../rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged \
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
## Option B: Bring Your Own Certificate: Self-Signed
<details id="option-b">
<summary>Click to expand</summary>
In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher.
> **Prerequisites:**
> From a computer with an internet connection, create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice.
>
> - The certificate files must be in PEM format.
> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](../rancher-on-a-single-node-with-docker/certificate-troubleshooting.md)
After creating your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Use the `-v` flag and provide the path to your certificates to mount them in your container.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
| `<CA_CERTS.pem>` | The path to the certificate authority's certificate. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to install. |
As of Rancher v2.5, privileged access is [required.](../rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged \
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
## Option C: Bring Your Own Certificate: Signed by Recognized CA
<details id="option-c">
<summary>Click to expand</summary>
In development or testing environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings.
> **Prerequisite:** The certificate files must be in PEM format.
After obtaining your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port. |
| `<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to install. |
> **Note:** Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher.
As of Rancher v2.5, privileged access is [required.](../rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
--no-cacerts \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
</details>
:::note
If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login.
:::
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry.md
-179
View File
@@ -1,179 +0,0 @@
---
title: '1. Set up Infrastructure and Private Registry'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/infrastructure-private-registry"/>
</head>
In this section, you will provision the underlying infrastructure for your Rancher management server in an air gapped environment. You will also set up the private Docker registry that must be available to your Rancher node(s).
An air gapped environment is an environment where the Rancher server is installed offline or behind a firewall.
The infrastructure depends on whether you are installing Rancher on a K3s Kubernetes cluster, an RKE Kubernetes cluster, or a single Docker container. For more information on each installation option, refer to [this page.](../../installation-and-upgrade.md)
As of Rancher v2.5, Rancher can be installed on any Kubernetes cluster. The RKE and K3s Kubernetes infrastructure tutorials below are still included for convenience.
<Tabs>
<TabItem value="K3s">
We recommend setting up the following infrastructure for a high-availability installation:
- **Two Linux nodes,** typically virtual machines, in the infrastructure provider of your choice.
- **An external database** to store the cluster data. PostgreSQL, MySQL, and etcd are supported.
- **A load balancer** to direct traffic to the two nodes.
- **A DNS record** to map a URL to the load balancer. This will become the Rancher server URL, and downstream Kubernetes clusters will need to reach it.
- **A private Docker registry** to distribute Docker images to your machines.
## 1. Set up Linux Nodes
These hosts will be disconnected from the internet, but require being able to connect with your private registry.
Make sure that your nodes fulfill the general installation requirements for [OS, container runtime, hardware, and networking.](../../installation-requirements/installation-requirements.md)
For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.
## 2. Set up External Datastore
The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to Kubernetes operators. The available options allow you to select a datastore that best fits your use case.
For a high-availability K3s installation, you will need to set up one of the following external databases:
* [PostgreSQL](https://www.postgresql.org/) (certified against versions 10.7 and 11.5)
* [MySQL](https://www.mysql.com/) (certified against version 5.7)
* [etcd](https://etcd.io/) (certified against version 3.3.15)
When you install Kubernetes, you will pass in details for K3s to connect to the database.
For an example of one way to set up the database, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/mysql-database-in-amazon-rds.md) for setting up a MySQL database on Amazon's RDS service.
For the complete list of options that are available for configuring a K3s cluster datastore, refer to the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/datastore/)
## 3. Set up the Load Balancer
You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server.
When Kubernetes gets set up in a later step, the K3s tool will deploy a Traefik Ingress controller. This controller will listen on ports 80 and 443 of the worker nodes, answering traffic destined for specific hostnames.
When Rancher is installed (also in a later step), the Rancher system creates an Ingress resource. That Ingress tells the Traefik Ingress controller to listen for traffic destined for the Rancher hostname. The Traefik Ingress controller, when receiving traffic destined for the Rancher hostname, will forward that traffic to the running Rancher pods in the cluster.
For your implementation, consider if you want or need to use a Layer-4 or Layer-7 load balancer:
- **A layer-4 load balancer** is the simpler of the two choices, in which you are forwarding TCP traffic to your nodes. We recommend configuring your load balancer as a Layer 4 balancer, forwarding traffic to ports TCP/80 and TCP/443 to the Rancher management cluster nodes. The Ingress controller on the cluster will redirect HTTP traffic to HTTPS and terminate SSL/TLS on port TCP/443. The Ingress controller will forward traffic to port TCP/80 to the Ingress pod in the Rancher deployment.
- **A layer-7 load balancer** is a bit more complicated but can offer features that you may want. For instance, a layer-7 load balancer is capable of handling TLS termination at the load balancer, as opposed to Rancher doing TLS termination itself. This can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with. If you decide to terminate the SSL/TLS traffic on a layer-7 load balancer, you will need to use the `--set tls=external` option when installing Rancher in a later step. For more information, refer to the [Rancher Helm chart options.](../../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination)
For an example showing how to set up an NGINX load balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/nginx-load-balancer.md)
For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/amazon-elb-load-balancer.md)
> **Important:**
> Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.
## 4. Set up the DNS Record
Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer.
Depending on your environment, this may be an A record pointing to the load balancer IP, or it may be a CNAME pointing to the load balancer hostname. In either case, make sure this record is the hostname that you intend Rancher to respond on.
You will need to specify this hostname in a later step when you install Rancher, and it is not possible to change it later. Make sure that your decision is a final one.
For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)
## 5. Set up a Private Docker Registry
Rancher supports air gap installs using a private registry. You must have your own private registry or other means of distributing Docker images to your machines.
In a later step, when you set up your K3s Kubernetes cluster, you will create a [private registries configuration file](https://rancher.com/docs/k3s/latest/en/installation/private-registry/) with details from this registry.
If you need help with creating a private registry, please refer to the [official Docker documentation.](https://docs.docker.com/registry/deploying/#run-an-externally-accessible-registry)
</TabItem>
<TabItem value="RKE">
To install the Rancher management server on a high-availability RKE cluster, we recommend setting up the following infrastructure:
- **Three Linux nodes,** typically virtual machines, in an infrastructure provider such as Amazon's EC2, Google Compute Engine, or vSphere.
- **A load balancer** to direct front-end traffic to the three nodes.
- **A DNS record** to map a URL to the load balancer. This will become the Rancher server URL, and downstream Kubernetes clusters will need to reach it.
- **A private Docker registry** to distribute Docker images to your machines.
These nodes must be in the same region/data center. You may place these servers in separate availability zones.
## Why Three Nodes?
In an RKE cluster, Rancher server data is stored on etcd. This etcd database runs on all three nodes.
The etcd database requires an odd number of nodes so that it can always elect a leader with a majority of the etcd cluster. If the etcd database cannot elect a leader, etcd can suffer from [split brain](https://www.quora.com/What-is-split-brain-in-distributed-systems), requiring the cluster to be restored from backup. If one of the three etcd nodes fails, the two remaining nodes can elect a leader because they have the majority of the total number of etcd nodes.
## 1. Set up Linux Nodes
These hosts will be disconnected from the internet, but require being able to connect with your private registry.
Make sure that your nodes fulfill the general installation requirements for [OS, container runtime, hardware, and networking.](../../installation-requirements/installation-requirements.md)
For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.
## 2. Set up the Load Balancer
You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server.
When Kubernetes gets set up in a later step, the RKE tool will deploy an NGINX Ingress controller. This controller will listen on ports 80 and 443 of the worker nodes, answering traffic destined for specific hostnames.
When Rancher is installed (also in a later step), the Rancher system creates an Ingress resource. That Ingress tells the NGINX Ingress controller to listen for traffic destined for the Rancher hostname. The NGINX Ingress controller, when receiving traffic destined for the Rancher hostname, will forward that traffic to the running Rancher pods in the cluster.
For your implementation, consider if you want or need to use a Layer-4 or Layer-7 load balancer:
- **A layer-4 load balancer** is the simpler of the two choices, in which you are forwarding TCP traffic to your nodes. We recommend configuring your load balancer as a Layer 4 balancer, forwarding traffic to ports TCP/80 and TCP/443 to the Rancher management cluster nodes. The Ingress controller on the cluster will redirect HTTP traffic to HTTPS and terminate SSL/TLS on port TCP/443. The Ingress controller will forward traffic to port TCP/80 to the Ingress pod in the Rancher deployment.
- **A layer-7 load balancer** is a bit more complicated but can offer features that you may want. For instance, a layer-7 load balancer is capable of handling TLS termination at the load balancer, as opposed to Rancher doing TLS termination itself. This can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with. If you decide to terminate the SSL/TLS traffic on a layer-7 load balancer, you will need to use the `--set tls=external` option when installing Rancher in a later step. For more information, refer to the [Rancher Helm chart options.](../../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination)
For an example showing how to set up an NGINX load balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/nginx-load-balancer.md)
For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/amazon-elb-load-balancer.md)
> **Important:**
> Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.
## 3. Set up the DNS Record
Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer.
Depending on your environment, this may be an A record pointing to the LB IP, or it may be a CNAME pointing to the load balancer hostname. In either case, make sure this record is the hostname that you intend Rancher to respond on.
You will need to specify this hostname in a later step when you install Rancher, and it is not possible to change it later. Make sure that your decision is a final one.
For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)
## 4. Set up a Private Docker Registry
Rancher supports air gap installs using a secure Docker private registry. You must have your own private registry or other means of distributing Docker images to your machines.
In a later step, when you set up your RKE Kubernetes cluster, you will create a [private registries configuration file](https://rancher.com/docs/rke/latest/en/config-options/private-registries/) with details from this registry.
If you need help with creating a private registry, please refer to the [official Docker documentation.](https://docs.docker.com/registry/deploying/#run-an-externally-accessible-registry)
</TabItem>
<TabItem value="Docker">
> The Docker installation is for Rancher users that are wanting to test out Rancher. Since there is only one node and a single Docker container, if the node goes down, you will lose all the data of your Rancher server.
>
> As of Rancher v2.5, the Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
## 1. Set up a Linux Node
This host will be disconnected from the Internet, but needs to be able to connect to your private registry.
Make sure that your node fulfills the general installation requirements for [OS, Docker, hardware, and networking.](../../installation-requirements/installation-requirements.md)
For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.
## 2. Set up a Private Docker Registry
Rancher supports air gap installs using a Docker private registry on your bastion server. You must have your own private registry or other means of distributing Docker images to your machines.
If you need help with creating a private registry, please refer to the [official Docker documentation.](https://docs.docker.com/registry/)
</TabItem>
</Tabs>
## [Next: Collect and Publish Images to your Private Registry](publish-images.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-kubernetes.md
-228
View File
@@ -1,228 +0,0 @@
---
title: '3. Install Kubernetes (Skip for Docker Installs)'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-kubernetes"/>
</head>
> Skip this section if you are installing Rancher on a single node with Docker.
This section describes how to install a Kubernetes cluster according to our [best practices for the Rancher server environment.](../../../../reference-guides/rancher-manager-architecture/architecture-recommendations.md#environment-for-kubernetes-installations) This cluster should be dedicated to run only the Rancher server.
As of Rancher v2.5, Rancher can be installed on any Kubernetes cluster, including hosted Kubernetes providers.
The steps to set up an air-gapped Kubernetes cluster on RKE or K3s are shown below.
<Tabs>
<TabItem value="K3s">
In this guide, we are assuming you have created your nodes in your air gapped environment and have a secure Docker private registry on your bastion server.
## Installation Outline
1. [Prepare Images Directory](#1-prepare-images-directory)
2. [Create Registry YAML](#2-create-registry-yaml)
3. [Install K3s](#3-install-k3s)
4. [Save and Start Using the kubeconfig File](#4-save-and-start-using-the-kubeconfig-file)
## 1. Prepare Images Directory
Obtain the images tar file for your architecture from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be running.
Place the tar file in the `images` directory before starting K3s on each node, for example:
```sh
sudo mkdir -p /var/lib/rancher/k3s/agent/images/
sudo cp ./k3s-airgap-images-$ARCH.tar /var/lib/rancher/k3s/agent/images/
```
## 2. Create Registry YAML
Create the registries.yaml file at `/etc/rancher/k3s/registries.yaml`. This will tell K3s the necessary details to connect to your private registry.
The registries.yaml file should look like this before plugging in the necessary information:
```
---
mirrors:
customreg:
endpoint:
- "https://ip-to-server:5000"
configs:
customreg:
auth:
username: xxxxxx # this is the registry username
password: xxxxxx # this is the registry password
tls:
cert_file: <path to the cert file used in the registry>
key_file: <path to the key file used in the registry>
ca_file: <path to the ca file used in the registry>
```
Note, at this time only secure registries are supported with K3s (SSL with custom CA).
For more information on private registries configuration file for K3s, refer to the [K3s documentation.](https://rancher.com/docs/k3s/latest/en/installation/private-registry/)
## 3. Install K3s
Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the [support maintenance terms.](https://rancher.com/support-maintenance-terms/)
To specify the K3s version, use the INSTALL_K3S_VERSION environment variable when running the K3s installation script.
Obtain the K3s binary from the [releases](https://github.com/rancher/k3s/releases) page, matching the same version used to get the airgap images tar.
Also obtain the K3s install script at https://get.k3s.io
Place the binary in `/usr/local/bin` on each node.
Place the install script anywhere on each node, and name it `install.sh`.
Install K3s on each server:
```
INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
```
Install K3s on each agent:
```
INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken ./install.sh
```
Note, take care to ensure you replace `myserver` with the IP or valid DNS of the server and replace `mynodetoken` with the node-token from the server.
The node-token is on the server at `/var/lib/rancher/k3s/server/node-token`
>**Note:** K3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks.
## 4. Save and Start Using the kubeconfig File
When you installed K3s on each Rancher server node, a `kubeconfig` file was created on the node at `/etc/rancher/k3s/k3s.yaml`. This file contains credentials for full access to the cluster, and you should save this file in a secure location.
To use this `kubeconfig` file,
1. Install [kubectl,](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) a Kubernetes command-line tool.
2. Copy the file at `/etc/rancher/k3s/k3s.yaml` and save it to the directory `~/.kube/config` on your local machine.
3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your load balancer, referring to port 6443. (The Kubernetes API server will be reached at port 6443, while the Rancher server will be reached at ports 80 and 443.) Here is an example `k3s.yaml`:
```
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: [CERTIFICATE-DATA]
server: [LOAD-BALANCER-DNS]:6443 # Edit this line
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
password: [PASSWORD]
username: admin
```
**Result:** You can now use `kubectl` to manage your K3s cluster. If you have more than one kubeconfig file, you can specify which one you want to use by passing in the path to the file when using `kubectl`:
```
kubectl --kubeconfig ~/.kube/config/k3s.yaml get pods --all-namespaces
```
For more information about the `kubeconfig` file, refer to the [K3s documentation](https://rancher.com/docs/k3s/latest/en/cluster-access/) or the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) about organizing cluster access using `kubeconfig` files.
## Note on Upgrading
Upgrading an air-gap environment can be accomplished in the following manner:
1. Download the new air-gap images (tar file) from the [releases](https://github.com/rancher/k3s/releases) page for the version of K3s you will be upgrading to. Place the tar in the `/var/lib/rancher/k3s/agent/images/` directory on each node. Delete the old tar file.
2. Copy and replace the old K3s binary in `/usr/local/bin` on each node. Copy over the install script at https://get.k3s.io (as it is possible it has changed since the last release). Run the script again just as you had done in the past with the same environment variables.
3. Restart the K3s service (if not restarted automatically by installer).
</TabItem>
<TabItem value="RKE">
We will create a Kubernetes cluster using Rancher Kubernetes Engine (RKE). Before being able to start your Kubernetes cluster, you’ll need to install RKE and create a RKE config file.
## 1. Install RKE
Install RKE by following the instructions in the [RKE documentation.](https://rancher.com/docs/rke/latest/en/installation/)
## 2. Create an RKE Config File
From a system that can access ports 22/TCP and 6443/TCP on the Linux host node(s) that you set up in a previous step, use the sample below to create a new file named `rancher-cluster.yml`.
This file is an RKE configuration file, which is a configuration for the cluster you're deploying Rancher to.
Replace values in the code sample below with help of the _RKE Options_ table. Use the IP address or DNS names of the three nodes you created.
> **Tip:** For more details on the options available, see the RKE [Config Options](https://rancher.com/docs/rke/latest/en/config-options/).
<figcaption>RKE Options</figcaption>
| Option | Required | Description |
| ------------------ | -------------------- | --------------------------------------------------------------------------------------- |
| `address` | ✓ | The DNS or IP address for the node within the air gapped network. |
| `user` | ✓ | A user that can run Docker commands. |
| `role` | ✓ | List of Kubernetes roles assigned to the node. |
| `internal_address` | optional<sup>1</sup> | The DNS or IP address used for internal cluster traffic. |
| `ssh_key_path` | | Path to the SSH private key used to authenticate to the node (defaults to `~/.ssh/id_rsa`). |
> <sup>1</sup> Some services like AWS EC2 require setting the `internal_address` if you want to use self-referencing security groups or firewalls.
```yaml
nodes:
- address: 10.10.3.187 # node air gap network IP
internal_address: 172.31.7.22 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
- address: 10.10.3.254 # node air gap network IP
internal_address: 172.31.13.132 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
- address: 10.10.3.89 # node air gap network IP
internal_address: 172.31.3.216 # node intra-cluster IP
user: rancher
role: ['controlplane', 'etcd', 'worker']
ssh_key_path: /home/user/.ssh/id_rsa
private_registries:
- url: <REGISTRY.YOURDOMAIN.COM:PORT> # private registry url
user: rancher
password: '*********'
is_default: true
```
## 3. Run RKE
After configuring `rancher-cluster.yml`, bring up your Kubernetes cluster:
```
rke up --config ./rancher-cluster.yml
```
## 4. Save Your Files
> **Important**
> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster.
Save a copy of the following files in a secure location:
- `rancher-cluster.yml`: The RKE cluster configuration file.
- `kube_config_cluster.yml`: The [Kubeconfig file](https://rancher.com/docs/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster.
- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file](https://rancher.com/docs/rke/latest/en/installation/#kubernetes-cluster-state), this file contains the current state of the cluster including the RKE configuration and the certificates.<br/><br/>_The Kubernetes Cluster State file is only created when using RKE v0.2.0 or higher._
</TabItem>
</Tabs>
> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file.
## Issues or Errors?
See the [Troubleshooting](../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md) page.
## [Next: Install Rancher](install-rancher-ha.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha.md
-252
View File
@@ -1,252 +0,0 @@
---
title: 4. Install Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/install-rancher-ha"/>
</head>
This section is about how to deploy Rancher for your air gapped environment in a high-availability Kubernetes installation. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy.
## Privileged Access for Rancher
When the Rancher server is deployed in the Docker container, a local Kubernetes cluster is installed within the container for Rancher to use. Because many features of Rancher run as deployments, and privileged mode is required to run containers within containers, you will need to install Rancher with the `--privileged` option.
## Docker Instructions
If you want to continue the air gapped installation using Docker commands, skip the rest of this page and follow the instructions on [this page.](docker-install-commands.md)
## Kubernetes Instructions
Rancher recommends installing Rancher on a Kubernetes cluster. A highly available Kubernetes install is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails.
### 1. Add the Helm Chart Repository
From a system that has access to the internet, fetch the latest Helm chart and copy the resulting manifests to a system that has access to the Rancher server cluster.
1. If you haven't already, install `helm` locally on a workstation that has internet access. Note: Refer to the [Helm version requirements](../../resources/helm-version-requirements.md) to choose a version of Helm to install Rancher.
2. Use `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Rancher Version](../../resources/choose-a-rancher-version.md).
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-alpha https://releases.rancher.com/server-charts/alpha
```
Note: Upgrades are not supported to, from, or between Alphas.
3. Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file.
```plain
helm fetch rancher-<CHART_REPO>/rancher
```
If you require a specific version of Rancher, you can fetch this with the Helm `--version` parameter like in the following example:
```plain
helm fetch rancher-stable/rancher --version=v2.4.8
```
### 2. Choose your SSL Configuration
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
When Rancher is installed on an air gapped Kubernetes cluster, there are two recommended options for the source of the certificate.
:::note
If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer](../../installation-references/helm-chart-options.md#external-tls-termination).
:::
| Configuration | Chart option | Description | Requires cert-manager |
| ------------------------------------------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
| Rancher Generated Self-Signed Certificates | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)<br/> This is the **default** and does not need to be added when rendering the Helm template. | yes |
| Certificates from Files | `ingress.tls.source=secret` | Use your own certificate files by creating Kubernetes Secret(s). <br/> This option must be passed when rendering the Rancher Helm template. | no |
### Helm Chart Options for Air Gap Installations
When setting up the Rancher Helm template, there are several options in the Helm chart that are designed specifically for air gap installations.
| Chart Option | Chart Value | Description |
| ----------------------- | -------------------------------- | ---- |
| `certmanager.version` | `<version>` | Configure proper Rancher TLS issuer depending of running cert-manager version. |
| `systemDefaultRegistry` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `useBundledSystemChart` | `true` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. |
### 3. Fetch the Cert-Manager Chart
Based on the choice your made in [2. Choose your SSL Configuration](#2-choose-your-ssl-configuration), complete one of the procedures below.
#### Option A: Default Self-Signed Certificate
By default, Rancher generates a CA and uses cert-manager to issue the certificate for access to the Rancher server interface.
:::note
Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.11.0, please see our [upgrade cert-manager documentation](../../resources/upgrade-cert-manager.md).
:::
##### 1. Add the cert-manager Repo
From a system connected to the internet, add the cert-manager repo to Helm:
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
```
##### 2. Fetch the cert-manager Chart
Fetch the latest cert-manager chart available from the [Helm chart repository](https://artifacthub.io/packages/helm/cert-manager/cert-manager).
```plain
helm fetch jetstack/cert-manager --version v1.11.0
```
##### 3. Retrieve the cert-manager CRDs
Download the required CRD file for cert-manager:
```plain
curl -L -o cert-manager-crd.yaml https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.crds.yaml
```
### 4. Install Rancher
Copy the fetched charts to a system that has access to the Rancher server cluster to complete installation.
#### 1. Install cert-manager
Install cert-manager with the same options you would use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry.
:::note
To see options on how to customize the cert-manager install (including for cases where your cluster uses PodSecurityPolicies), see the [cert-manager docs](https://artifacthub.io/packages/helm/cert-manager/cert-manager#configuration).
:::
<details id="install-cert-manager">
<summary>Click to expand</summary>
If you are using self-signed certificates, install cert-manager:
1. Create the namespace for cert-manager.
```plain
kubectl create namespace cert-manager
```
2. Create the cert-manager CustomResourceDefinitions (CRDs).
```plain
kubectl apply -f cert-manager-crd.yaml
```
3. Install cert-manager.
```plain
helm install cert-manager ./cert-manager-v1.11.0.tgz \
--namespace cert-manager \
--set image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-controller \
--set webhook.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-webhook \
--set cainjector.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-cainjector \
--set startupapicheck.image.repository=<REGISTRY.YOURDOMAIN.COM:PORT>/quay.io/jetstack/cert-manager-ctl
```
</details>
#### 2. Install Rancher
First, refer to [Adding TLS Secrets](../../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
Then, create the namespace for Rancher using kubectl:
```plain
kubectl create namespace cattle-system
```
Next, install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
Placeholder | Description
------------|-------------
`<VERSION>` | The version number of the output tarball.
`<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.
`<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.
```plain
helm install rancher ./rancher-<VERSION>.tgz \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set certmanager.version=<CERTMANAGER_VERSION> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
**Optional**: To install a specific Rancher version, set the `rancherImageTag` value, example: `--set rancherImageTag=v2.5.8`
#### Option B: Certificates From Files Using Kubernetes Secrets
##### 1. Create Secrets
Create Kubernetes secrets from your own certificates for Rancher to use. The common name for the cert will need to match the `hostname` option in the command below, or the ingress controller will fail to provision the site for Rancher.
##### 2. Install Rancher
Install Rancher, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
For Kubernetes v1.25 or later, set `global.cattle.psp.enabled` to `false` when using Rancher v2.7.2-v2.7.4. This is not necessary for Rancher v2.7.5 and above, but you can still manually set the option if you choose.
| Placeholder | Description |
| -------------------------------- | ----------------------------------------------- |
| `<VERSION>` | The version number of the output tarball. |
| `<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer. |
| `<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry. |
```plain
helm install rancher ./rancher-<VERSION>.tgz \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
If you are using a Private CA signed cert, add `--set privateCA=true` following `--set ingress.tls.source=secret`:
```plain
helm install rancher ./rancher-<VERSION>.tgz \
--namespace cattle-system \
--set hostname=<RANCHER.YOURDOMAIN.COM> \
--set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
--set ingress.tls.source=secret \
--set privateCA=true \
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Use the packaged Rancher system charts
```
The installation is complete.
:::caution
If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login. Leaving this active in an air-gapped environment can cause issues if the sockets cannot be opened successfully.
:::
## Additional Resources
These resources could be helpful when installing Rancher:
- [Importing and installing extensions in an air-gapped environment](../../../../integrations-in-rancher/rancher-extensions.md#importing-and-installing-extensions-in-an-air-gapped-environment)
- [Rancher Helm chart options](../../installation-references/helm-chart-options.md)
- [Adding TLS secrets](../../resources/add-tls-secrets.md)
- [Troubleshooting Rancher Kubernetes Installations](../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/publish-images.md
-292
View File
@@ -1,292 +0,0 @@
---
title: '2. Collect and Publish Images to your Private Registry'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install/publish-images"/>
</head>
This section describes how to set up your private registry so that when you install Rancher, Rancher will pull all the required images from this registry.
By default, all images used to [provision Kubernetes clusters](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md) or launch any tools in Rancher, e.g. monitoring and logging, are pulled from Docker Hub. In an air gapped installation of Rancher, you will need a private registry that is located somewhere accessible by your Rancher server. Then, you will load the registry with all the images.
Populating the private registry with images is the same process for installing Rancher with Docker and for installing Rancher on a Kubernetes cluster.
The steps in this section differ depending on whether or not you are planning to use Rancher to provision a downstream cluster with Windows nodes or not. By default, we provide the steps of how to populate your private registry assuming that Rancher will provision downstream Kubernetes clusters with only Linux nodes. But if you plan on provisioning any [downstream Kubernetes clusters using Windows nodes](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-windows-clusters/use-windows-clusters.md), there are separate instructions to support the images needed.
> **Prerequisites:**
>
> You must have a [private registry](https://docs.docker.com/registry/deploying/#run-an-externally-accessible-registry) available to use.
>
> If the registry has certs, follow [this K3s documentation](https://rancher.com/docs/k3s/latest/en/installation/private-registry/) about adding a private registry. The certs and registry configuration files need to be mounted into the Rancher container.
<Tabs>
<TabItem value="Linux Only Clusters">
For Rancher servers that will only provision Linux clusters, these are the steps to populate your private registry.
1. [Find the required assets for your Rancher version](#1-find-the-required-assets-for-your-rancher-version)
2. [Collect the cert-manager image](#2-collect-the-cert-manager-image) (unless you are bringing your own certificates or terminating TLS on a load balancer)
3. [Save the images to your workstation](#3-save-the-images-to-your-workstation)
4. [Populate the private registry](#4-populate-the-private-registry)
### Prerequisites
These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space.
If you will use ARM64 hosts, the registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests.
### 1. Find the required assets for your Rancher version
1. Go to our [releases page,](https://github.com/rancher/rancher/releases) find the Rancher v2.x.x release that you want to install, and click **Assets.** Note: Don't use releases marked `rc` or `Pre-release`, as they are not stable for production environments.
2. From the release's **Assets** section, download the following files, which are required to install Rancher in an air gap environment:
| Release File | Description |
| ---------------- | -------------- |
| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. |
| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. |
| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. |
### 2. Collect the cert-manager image
> Skip this step if you are using your own certificates, or if you are terminating TLS on an external load balancer.
In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well.
1. Fetch the latest `cert-manager` Helm chart and parse the template for image details:
> **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation](../../resources/upgrade-cert-manager.md).
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm fetch jetstack/cert-manager --version v1.5.1
helm template ./cert-manager-<version>.tgz | awk '$1 ~ /image:/ {print $2}' | sed s/\"//g >> ./rancher-images.txt
```
2. Sort and unique the images list to remove any overlap between the sources:
```plain
sort -u rancher-images.txt -o rancher-images.txt
```
### 3. Save the images to your workstation
1. Make `rancher-save-images.sh` an executable:
```
chmod +x rancher-save-images.sh
```
1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images:
```plain
./rancher-save-images.sh --image-list ./rancher-images.txt
```
**Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory.
### 4. Populate the private registry
Next, you will move the images in the `rancher-images.tar.gz` to your private registry using the scripts to load the images.
Move the images in the `rancher-images.tar.gz` to your private registry using the scripts to load the images.
The `rancher-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script. The `rancher-images.tar.gz` should also be in the same directory.
1. Log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Make `rancher-load-images.sh` an executable:
```
chmod +x rancher-load-images.sh
```
1. Use `rancher-load-images.sh` to extract, tag and push `rancher-images.txt` and `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.sh --image-list ./rancher-images.txt --registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
</TabItem>
<TabItem value="Linux and Windows Clusters">
For Rancher servers that will provision Linux and Windows clusters, there are distinctive steps to populate your private registry for the Windows images and the Linux images. Since a Windows cluster is a mix of Linux and Windows nodes, the Linux images pushed into the private registry are manifests.
## Windows Steps
The Windows images need to be collected and pushed from a Windows server workstation.
1. <a href="#windows-1">Find the required assets for your Rancher version</a>
2. <a href="#windows-2">Save the images to your Windows Server workstation</a>
3. <a href="#windows-3">Prepare the Docker daemon</a>
4. <a href="#windows-4">Populate the private registry</a>
### Prerequisites
These steps expect you to use a Windows Server 1809 workstation that has internet access, access to your private registry, and at least 50 GB of disk space.
The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters.
Your registry must support manifests. As of April 2020, Amazon Elastic Container Registry does not support manifests.
<a name="windows-1"></a>
### 1. Find the required assets for your Rancher version
1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments.
2. From the release's "Assets" section, download the following files:
| Release File | Description |
|----------------------------|------------------|
| `rancher-windows-images.txt` | This file contains a list of Windows images needed to provision Windows clusters. |
| `rancher-save-images.ps1` | This script pulls all the images in the `rancher-windows-images.txt` from Docker Hub and saves all of the images as `rancher-windows-images.tar.gz`. |
| `rancher-load-images.ps1` | This script loads the images from the `rancher-windows-images.tar.gz` file and pushes them to your private registry. |
<a name="windows-2"></a>
### 2. Save the images to your Windows Server workstation
1. Using `powershell`, go to the directory that has the files that were downloaded in the previous step.
1. Run `rancher-save-images.ps1` to create a tarball of all the required images:
```plain
./rancher-save-images.ps1
```
**Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-windows-images.tar.gz`. Check that the output is in the directory.
<a name="windows-3"></a>
### 3. Prepare the Docker daemon
Append your private registry address to the `allow-nondistributable-artifacts` config field in the Docker daemon (`C:\ProgramData\Docker\config\daemon.json`). Since the base image of Windows images are maintained by the `mcr.microsoft.com` registry, this step is required as the layers in the Microsoft registry are missing from Docker Hub and need to be pulled into the private registry.
```
{
...
"allow-nondistributable-artifacts": [
...
"<REGISTRY.YOURDOMAIN.COM:PORT>"
]
...
}
```
<a name="windows-4"></a>
### 4. Populate the private registry
Move the images in the `rancher-windows-images.tar.gz` to your private registry using the scripts to load the images.
The `rancher-windows-images.txt` is expected to be on the workstation in the same directory that you are running the `rancher-load-images.ps1` script. The `rancher-windows-images.tar.gz` should also be in the same directory.
1. Using `powershell`, log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Using `powershell`, use `rancher-load-images.ps1` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.ps1 --registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
## Linux Steps
The Linux images need to be collected and pushed from a Linux host, but _must be done after_ populating the Windows images into the private registry. These step are different from the Linux only steps as the Linux images that are pushed will actually manifests that support Windows and Linux images.
1. <a href="#linux-1">Find the required assets for your Rancher version</a>
2. <a href="#linux-2">Collect all the required images</a>
3. <a href="#linux-3">Save the images to your Linux workstation</a>
4. <a href="#linux-4">Populate the private registry</a>
### Prerequisites
You must populate the private registry with the Windows images before populating the private registry with Linux images. If you have already populated the registry with Linux images, you will need to follow these instructions again as they will publish manifests that support Windows and Linux images.
These steps expect you to use a Linux workstation that has internet access, access to your private registry, and at least 20 GB of disk space.
The workstation must have Docker 18.02+ in order to support manifests, which are required when provisioning Windows clusters.
<a name="linux-1"></a>
### 1. Find the required assets for your Rancher version
1. Browse to our [releases page](https://github.com/rancher/rancher/releases) and find the Rancher v2.x.x release that you want to install. Don't download releases marked `rc` or `Pre-release`, as they are not stable for production environments. Click **Assets.**
2. From the release's **Assets** section, download the following files:
| Release File | Description |
|----------------------------| -------------------------- |
| `rancher-images.txt` | This file contains a list of images needed to install Rancher, provision clusters and user Rancher tools. |
| `rancher-windows-images.txt` | This file contains a list of images needed to provision Windows clusters. |
| `rancher-save-images.sh` | This script pulls all the images in the `rancher-images.txt` from Docker Hub and saves all of the images as `rancher-images.tar.gz`. |
| `rancher-load-images.sh` | This script loads images from the `rancher-images.tar.gz` file and pushes them to your private registry. |
<a name="linux-2"></a>
### 2. Collect all the required images
**For Kubernetes Installs using Rancher Generated Self-Signed Certificate:** In a Kubernetes Install, if you elect to use the Rancher default self-signed TLS certificates, you must add the [`cert-manager`](https://hub.helm.sh/charts/jetstack/cert-manager) image to `rancher-images.txt` as well. You skip this step if you are using you using your own certificates.
1. Fetch the latest `cert-manager` Helm chart and parse the template for image details:
> **Note:** Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.12.0, please see our [upgrade documentation](../../resources/upgrade-cert-manager.md).
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm fetch jetstack/cert-manager --version v0.12.0
helm template ./cert-manager-<version>.tgz | | awk '$1 ~ /image:/ {print $2}' | sed s/\"//g >> ./rancher-images.txt
```
2. Sort and unique the images list to remove any overlap between the sources:
```plain
sort -u rancher-images.txt -o rancher-images.txt
```
<a name="linux-3"></a>
### 3. Save the images to your workstation
1. Make `rancher-save-images.sh` an executable:
```
chmod +x rancher-save-images.sh
```
1. Run `rancher-save-images.sh` with the `rancher-images.txt` image list to create a tarball of all the required images:
```plain
./rancher-save-images.sh --image-list ./rancher-images.txt
```
**Result:** Docker begins pulling the images used for an air gap install. Be patient. This process takes a few minutes. When the process completes, your current directory will output a tarball named `rancher-images.tar.gz`. Check that the output is in the directory.
<a name="linux-4"></a>
### 4. Populate the private registry
Move the images in the `rancher-images.tar.gz` to your private registry using the `rancher-load-images.sh script` to load the images.
The image list, `rancher-images.txt` or `rancher-windows-images.txt`, is expected to be on the workstation in the same directory that you are running the `rancher-load-images.sh` script. The `rancher-images.tar.gz` should also be in the same directory.
1. Log into your private registry if required:
```plain
docker login <REGISTRY.YOURDOMAIN.COM:PORT>
```
1. Make `rancher-load-images.sh` an executable:
```
chmod +x rancher-load-images.sh
```
1. Use `rancher-load-images.sh` to extract, tag and push the images from `rancher-images.tar.gz` to your private registry:
```plain
./rancher-load-images.sh --image-list ./rancher-images.txt \
--windows-image-list ./rancher-windows-images.txt \
--registry <REGISTRY.YOURDOMAIN.COM:PORT>
```
</TabItem>
</Tabs>
### [Next step for Kubernetes Installs - Launch a Kubernetes Cluster](install-kubernetes.md)
### [Next step for Docker Installs - Install Rancher](install-rancher-ha.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/install-rancher-on-linux/install-rancher-on-linux.md
-238
View File
@@ -1,238 +0,0 @@
---
title: Install/Upgrade Rancher with RancherD
---
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
We are excited to introduce a new, simpler way to install Rancher called RancherD.
RancherD is a single binary that first launches an RKE2 Kubernetes cluster, then installs the Rancher server Helm chart on the cluster.
- [About RancherD Installs](#about-rancherd-installs)
- [Prerequisites](#prerequisites)
- [Part I: Installing Rancher](#part-i-installing-rancher)
- [Part II: High Availability](#part-ii-high-availability)
- [Upgrades](#upgrades)
- [Configuration](#configuration)
- [Uninstall](#uninstall)
- [RKE2 Documentation](#rke2-documentation)
## About RancherD Installs
When RancherD is launched on a host, it first installs an RKE2 Kubernetes cluster, then deploys Rancher on the cluster as a Kubernetes daemonset.
In both the RancherD install and the Helm CLI install, Rancher is installed as a Helm chart on a Kubernetes cluster.
Configuration and upgrading are also simplified with RancherD. When you upgrade the RancherD binary, both the Kubernetes cluster and the Rancher Helm chart are upgraded.
In Part I of these instructions, you'll learn how to launch RancherD on a single node. The result of following the steps in Part I is a single-node [RKE2](https://docs.rke2.io/) Kubernetes cluster with the Rancher server installed. This cluster can easily become high availability later. If Rancher only needs to manage the local Kubernetes cluster, the installation is complete.
Part II explains how to convert the single-node Rancher installation into a high-availability installation. If the Rancher server will manage downstream Kubernetes clusters, it is important to follow these steps. A discussion of recommended architecture for highly available Rancher deployments can be found in our [Best Practices Guide.](../../../../reference-guides/best-practices/rancher-server/rancher-server.md)
## Prerequisites
### Node Requirements
RancherD must be launched on a Linux OS. At this time, only OSes that leverage systemd are supported.
The Linux node needs to fulfill the [installation requirements](../../installation-requirements/installation-requirements.md) for hardware and networking. Docker is not required for RancherD installs.
To install RancherD on SELinux Enforcing CentOS 8 nodes or RHEL 8 nodes, some [additional steps](../../installation-requirements/installation-requirements.md#rancherd-on-selinux-enforcing-centos-8-or-rhel-8-nodes) are required.
### Root Access
Before running the installation commands, you will need to log in as root:
```
sudo -s
```
### Fixed Registration Address
A fixed registration address is recommended for single-node installs and required for high-availability installs with RancherD.
The fixed registration address is an endpoint that is used for two purposes:
- To access the Kubernetes API. So you can, for example, modify your [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file to point to it instead of a specific node.
- To add new nodes to the Kubernetes cluster. To add nodes to the cluster later, you will run a command on the node that will specify the fixed registration address of the cluster.
If you are installing Rancher on a single node, the fixed registration address makes it possible to add more nodes to the cluster so that you can convert the single-node install to a high-availability install without causing downtime to the cluster. If you don't set up this address when installing the single-node Kubernetes cluster, you would need to re-run the installation script with a fixed registration address in order to add new nodes to the cluster.
The fixed registration can be the IP or hostname of any of the server nodes, but in many cases those may change over time as nodes are created and destroyed. Therefore, you should have a stable endpoint in front of the server nodes.
This endpoint can be set up using any number of approaches, such as:
* A layer 4 (TCP) load balancer
* Round-robin DNS
* Virtual or elastic IP addresses
The following should be taken into consideration when configuring the load balancer or other endpoint:
- The RancherD server process listens on port 9345 for new nodes to register.
- The Kubernetes API is served on port 6443, as normal.
- In RancherD installs, the Rancher UI is served on port 8443 by default. (This is different from Helm chart installs, where port 443 is used by default.)
## Part I: Installing Rancher
### 1. Set up Configurations
To avoid certificate errors with the fixed registration address, you should launch the server with the `tls-san` parameter set. This parameter should refer to your fixed registration address.
This option adds an additional hostname or IP as a Subject Alternative Name in the server's TLS cert, and it can be specified as a list if you would like to access the Kubernetes cluster via both the IP and the hostname.
Create the RancherD config file at `/etc/rancher/rke2/config.yaml`:
```yaml
token: my-shared-secret
tls-san:
- my-fixed-registration-address.com
- another-kubernetes-domain.com
```
The first server node establishes the secret token that other nodes would register with if they are added to the cluster.
If you do not specify a pre-shared secret, RancherD will generate one and place it at `/var/lib/rancher/rke2/server/node-token`.
To specify your own pre-shared secret as the token, set the `token` argument on startup.
Installing Rancher this way will use Rancher-generated certificates. To use your own self-signed or trusted certificates, refer to the [configuration guide.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/rancherd-configuration-reference.md#certificates-for-the-rancher-server)
For information on customizing the RancherD Helm chart values.yaml, refer to [this section.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/rancherd-configuration-reference.md#customizing-the-rancherd-helm-chart)
### 2. Launch the first server node
Run the RancherD installer:
```
curl -sfL https://get.rancher.io | sh -
```
The RancherD version can be specified using the `INSTALL_RANCHERD_VERSION` environment variable:
```
curl -sfL https://get.rancher.io | INSTALL_RANCHERD_VERSION=v2.5.4-rc6 sh -
```
Once installed, the `rancherd` binary will be on your PATH. You can check out its help text like this:
```
rancherd --help
NAME:
rancherd - Rancher Kubernetes Engine 2
...
```
Next, launch RancherD:
```
systemctl enable rancherd-server.service
systemctl start rancherd-server.service
```
When RancherD launches, it installs an RKE2 Kubernetes cluster. Use the following command to see the logs of the Kubernetes cluster as it comes up:
```
journalctl -eu rancherd-server -f
```
### 3. Set up the kubeconfig file with kubectl
Once the Kubernetes cluster is up, set up RancherD’s kubeconfig file and `kubectl`:
```
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml PATH=$PATH:/var/lib/rancher/rke2/bin
```
### 4. Verify that Rancher is installed on the Kubernetes cluster
Now, you can start issuing `kubectl` commands. Use the following commands to verify that Rancher is deployed as a daemonset on the cluster:
```
kubectl get daemonset rancher -n cattle-system
kubectl get pod -n cattle-system
```
If you watch the pods, you will see the following pods installed:
- `helm-operation` pods in the `cattle-system` namespace
- a `rancher` pod and `rancher-webhook` pod in the `cattle-system` namespace
- a `fleet-agent`, `fleet-controller`, and `gitjob` pod in the `fleet-system` namespace
- a `rancher-operator` pod in the `rancher-operator-system` namespace
### 5. Set the initial Rancher password
Once the `rancher` pod is up and running, run the following:
```
rancherd reset-admin
```
This will give you the URL, username and password needed to log into Rancher. Follow that URL, plug in the credentials, and you’re up and running with Rancher!
If Rancher will only manage the local Kubernetes cluster, the installation is complete.
## Part II: High Availability
If you plan to use the Rancher server to manage downstream Kubernetes clusters, Rancher needs to be highly available. In these steps, you will add more nodes to achieve a high-availability cluster. Since Rancher is running as a daemonset, it will automatically launch on the nodes you add.
An odd number of nodes is required because the etcd cluster, which contains the cluster data, needs a majority of live nodes to avoid losing quorum. A loss of quorum could require the cluster to be restored from backup. Therefore, we recommend using three nodes.
When following these steps, you should still be logged in as root.
### 1. Configure the fixed registration address on a new node
Additional server nodes are launched much like the first, except that you must specify the `server` and `token` parameters so that they can successfully connect to the initial server node.
Here is an example of what the RancherD config file would look like for additional server nodes. By default, this config file is expected to be located at `/etc/rancher/rke2/config.yaml`.
```yaml
server: https://my-fixed-registration-address.com:9345
token: my-shared-secret
tls-san:
- my-fixed-registration-address.com
- another-kubernetes-domain.com
```
### 2. Launch an additional server node
Run the installer on the new node:
```
curl -sfL https://get.rancher.io | sh -
```
This will download RancherD and install it as a systemd unit on your host.
Next, launch RancherD:
```
systemctl enable rancherd-server.service
systemctl start rancherd-server.service
```
### 3. Repeat
Repeat steps one and two for another Linux node, bringing the number of nodes in the cluster to three.
**Result:** Rancher is highly available and the installation is complete.
## Upgrades
For information on upgrades and rollbacks, refer to [this page.](upgrade-rancherd.md)
## Configuration
For information on how to configure certificates, node taints, Rancher Helm chart options, or RancherD CLI options, refer to the [configuration reference.](../../../../reference-guides/cluster-configuration/rancher-server-configuration/rancherd-configuration-reference.md)
## Uninstall
To uninstall RancherD from your system, run the command below. This will shut down the process, remove the RancherD binary, and clean up files used by RancherD.
```
rancherd-uninstall.sh
```
## RKE2 Documentation
For more information on RKE2, the Kubernetes distribution used to provision the underlying cluster, refer to the documentation [here.](https://docs.rke2.io/)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/install-rancher-on-linux/roll-back-rancherd.md
-7
View File
@@ -1,7 +0,0 @@
---
title: Rollbacks
---
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
To roll back Rancher to a previous version, re-run the installation script with the previous version specified in the `INSTALL_RANCHERD_VERSION` environment variable.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/install-rancher-on-linux/upgrade-rancherd.md
-67
View File
@@ -1,67 +0,0 @@
---
title: Upgrades
---
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
When RancherD is upgraded, the Rancher Helm controller and the Fleet pods are upgraded.
During a RancherD upgrade, there is very little downtime, but it is possible that RKE2 may be down for a minute, during which you could lose access to Rancher.
When Rancher is installed with RancherD, the underlying Kubernetes cluster can't be upgraded from the Rancher UI. It needs to be upgraded using the RancherD CLI.
## Upgrading the Rancher Helm Chart without Upgrading the Underlying Cluster
To upgrade Rancher without upgrading the underlying Kubernetes cluster, follow these steps.
> Before upgrading, we recommend that you should:
>
> - Create a backup of the Rancher server using the [backup application.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md)
> - Review the known issues for the Rancher version you are upgrading to. The known issues are listed in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
1. Uninstall the chart with Helm:
```
helm uninstall rancher
```
2. Reinstall the Rancher chart with Helm. To install a specific Rancher version, use the `--version` flag. For example:
```
helm install rancher rancher-latest/rancher \
--namespace cattle-system \
--set hostname=rancher.my.org \
--version 2.5.1
```
**Result:** Rancher is upgraded to the new version.
If necessary, restore Rancher from backup by following [these steps.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-rancher.md)
## Upgrading Both Rancher and the Underlying Cluster
Upgrade both RancherD and the underlying Kubernetes cluster by re-running the RancherD installation script.
> Before upgrading, we recommend that you should:
>
> - Create a backup of the Rancher server using the [backup application.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher.md)
> - Review the known issues for the Rancher version you are upgrading to. The known issues are listed in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12)
```
sudo curl -sfL https://get.rancher.io | sudo sh -
```
To specify a specific version to upgrade to, use `INSTALL_RANCHERD_VERSION` environment variable:
```
curl -sfL https://get.rancher.io | INSTALL_RANCHERD_VERSION=v2.5.1 sh -
```
Then launch the server:
```
systemctl enable rancherd-server
systemctl start rancherd-server
```
The upgrade can also be performed by manually installing the binary of the desired version.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/other-installation-methods.md
-23
View File
@@ -1,23 +0,0 @@
---
title: Other Installation Methods
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods"/>
</head>
### Air Gapped Installations
Follow [these steps](air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) to install the Rancher server in an air gapped environment.
An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy.
### Docker Installations
The [single-node Docker installation](rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md) is for Rancher users that are wanting to test out Rancher. Instead of running on a Kubernetes cluster using Helm, you install the Rancher server component on a single node using a `docker run` command.
The Docker installation is for development and testing environments only.
Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server.
For Rancher v2.5+, the Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster.](../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/install-kubernetes.md
-154
View File
@@ -1,154 +0,0 @@
---
title: '2. Install Kubernetes'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/install-kubernetes"/>
</head>
Once the infrastructure is ready, you can continue with setting up an RKE cluster to install Rancher in.
### Installing Docker
First, you have to install Docker and setup the HTTP proxy on all three Linux nodes. For this perform the following steps on all three nodes.
For convenience export the IP address and port of your proxy into an environment variable and set up the HTTP_PROXY variables for your current shell:
```
export proxy_host="10.0.0.5:8888"
export HTTP_PROXY=http://${proxy_host}
export HTTPS_PROXY=http://${proxy_host}
export NO_PROXY=127.0.0.0/8,10.0.0.0/8,cattle-system.svc,172.16.0.0/12,192.168.0.0/16
```
Next configure apt to use this proxy when installing packages. If you are not using Ubuntu, you have to adapt this step accordingly:
```
cat <<EOF | sudo tee /etc/apt/apt.conf.d/proxy.conf > /dev/null
Acquire::http::Proxy "http://${proxy_host}/";
Acquire::https::Proxy "http://${proxy_host}/";
EOF
```
Now you can install Docker:
```
curl -sL https://releases.rancher.com/install-docker/19.03.sh | sh
```
Then ensure that your current user is able to access the Docker daemon without sudo:
```
sudo usermod -aG docker YOUR_USERNAME
```
And configure the Docker daemon to use the proxy to pull images:
```
sudo mkdir -p /etc/systemd/system/docker.service.d
cat <<EOF | sudo tee /etc/systemd/system/docker.service.d/http-proxy.conf > /dev/null
[Service]
Environment="HTTP_PROXY=http://${proxy_host}"
Environment="HTTPS_PROXY=http://${proxy_host}"
Environment="NO_PROXY=127.0.0.0/8,10.0.0.0/8,cattle-system.svc,172.16.0.0/12,192.168.0.0/16"
EOF
```
To apply the configuration, restart the Docker daemon:
```
sudo systemctl daemon-reload
sudo systemctl restart docker
```
### Creating the RKE Cluster
You need several command line tools on the host where you have SSH access to the Linux nodes to create and interact with the cluster:
* [RKE CLI binary](https://rancher.com/docs/rke/latest/en/installation/#download-the-rke-binary)
```
sudo curl -fsSL -o /usr/local/bin/rke https://github.com/rancher/rke/releases/download/v1.1.4/rke_linux-amd64
sudo chmod +x /usr/local/bin/rke
```
* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
```
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
```
* [helm](https://helm.sh/docs/intro/install/)
```
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod +x get_helm.sh
sudo ./get_helm.sh
```
Next, create a YAML file that describes the RKE cluster. Ensure that the IP addresses of the nodes and the SSH username are correct. For more information on the cluster YAML, have a look at the [RKE documentation](https://rancher.com/docs/rke/latest/en/example-yamls/).
```
nodes:
- address: 10.0.1.200
user: ubuntu
role: [controlplane,worker,etcd]
- address: 10.0.1.201
user: ubuntu
role: [controlplane,worker,etcd]
- address: 10.0.1.202
user: ubuntu
role: [controlplane,worker,etcd]
services:
etcd:
backup_config:
interval_hours: 12
retention: 6
```
After that, you can create the Kubernetes cluster by running:
```
rke up --config rancher-cluster.yaml
```
RKE creates a state file called `rancher-cluster.rkestate`, this is needed if you want to perform updates, modify your cluster configuration or restore it from a backup. It also creates a `kube_config_cluster.yaml` file, that you can use to connect to the remote Kubernetes cluster locally with tools like kubectl or Helm. Make sure to save all of these files in a secure location, for example by putting them into a version control system.
To have a look at your cluster run:
```
export KUBECONFIG=kube_config_cluster.yaml
kubectl cluster-info
kubectl get pods --all-namespaces
```
You can also verify that your external load balancer works, and the DNS entry is set up correctly. If you send a request to either, you should receive HTTP 404 response from the ingress controller:
```
$ curl 10.0.1.100
default backend - 404
$ curl rancher.example.com
default backend - 404
```
### Save Your Files
> **Important**
> The files mentioned below are needed to maintain, troubleshoot and upgrade your cluster.
Save a copy of the following files in a secure location:
- `rancher-cluster.yml`: The RKE cluster configuration file.
- `kube_config_cluster.yml`: The [Kubeconfig file](https://rancher.com/docs/rke/latest/en/kubeconfig/) for the cluster, this file contains credentials for full access to the cluster.
- `rancher-cluster.rkestate`: The [Kubernetes Cluster State file](https://rancher.com/docs/rke/latest/en/installation/#kubernetes-cluster-state), this file contains the current state of the cluster including the RKE configuration and the certificates.
> **Note:** The "rancher-cluster" parts of the two latter file names are dependent on how you name the RKE cluster configuration file.
### Issues or errors?
See the [Troubleshooting](../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md) page.
### [Next: Install Rancher](install-rancher.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/install-rancher.md
-92
View File
@@ -1,92 +0,0 @@
---
title: 3. Install Rancher
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/install-rancher"/>
</head>
Now that you have a running RKE cluster, you can install Rancher in it. For security reasons all traffic to Rancher must be encrypted with TLS. For this tutorial you are going to automatically issue a self-signed certificate through [cert-manager](https://cert-manager.io/). In a real-world use-case you will likely use Let's Encrypt or provide your own certificate.
> **Note:** These installation instructions assume you are using Helm 3.
### Install cert-manager
Add the cert-manager Helm repository:
```
helm repo add jetstack https://charts.jetstack.io
```
Create a namespace for cert-manager:
```
kubectl create namespace cert-manager
```
Install the CustomResourceDefinitions of cert-manager:
```
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.5.1/cert-manager.crds.yaml
```
And install it with Helm. Note that cert-manager also needs your proxy configured in case it needs to communicate with Let's Encrypt or other external certificate issuers:
```
helm upgrade --install cert-manager jetstack/cert-manager \
--namespace cert-manager --version v1.5.1 \
--set http_proxy=http://${proxy_host} \
--set https_proxy=http://${proxy_host} \
--set noProxy=127.0.0.0/8\\,10.0.0.0/8\\,cattle-system.svc\\,172.16.0.0/12\\,192.168.0.0/16\\,.svc\\,.cluster.local
```
Now you should wait until cert-manager is finished starting up:
```
kubectl rollout status deployment -n cert-manager cert-manager
kubectl rollout status deployment -n cert-manager cert-manager-webhook
```
### Install Rancher
Next you can install Rancher itself. First, add the Helm repository:
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
Create a namespace:
```
kubectl create namespace cattle-system
```
And install Rancher with Helm. Rancher also needs a proxy configuration so that it can communicate with external application catalogs or retrieve Kubernetes version update metadata.
Note that `rancher.cattle-system` must be added to the noProxy list (as shown below) so that Fleet can communicate directly to Rancher with Kubernetes service DNS using service discovery.
```
helm upgrade --install rancher rancher-latest/rancher \
--namespace cattle-system \
--set hostname=rancher.example.com \
--set proxy=http://${proxy_host}
--set noProxy=127.0.0.0/8\\,10.0.0.0/8\\,cattle-system.svc\\,172.16.0.0/12\\,192.168.0.0/16\\,.svc\\,.cluster.local,rancher.cattle-system
```
After waiting for the deployment to finish:
```
kubectl rollout status deployment -n cattle-system rancher
```
You can now navigate to `https://rancher.example.com` and start using Rancher.
> **Note:** If you don't intend to send telemetry data, opt out [telemetry](../../../../faq/telemetry.md) during the initial login. Leaving this active in an air-gapped environment can cause issues if the sockets cannot be opened successfully.
### Additional Resources
These resources could be helpful when installing Rancher:
- [Rancher Helm chart options](../../../../reference-guides/installation-references/helm-chart-options.md)
- [Adding TLS secrets](../../resources/add-tls-secrets.md)
- [Troubleshooting Rancher Kubernetes Installations](../../install-upgrade-on-a-kubernetes-cluster/troubleshooting.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/rancher-behind-an-http-proxy.md
-17
View File
@@ -1,17 +0,0 @@
---
title: Installing Rancher behind an HTTP Proxy
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy"/>
</head>
In a lot of enterprise environments, servers or VMs running on premise do not have direct Internet access, but must connect to external services through a HTTP(S) proxy for security reasons. This tutorial shows step by step how to set up a highly available Rancher installation in such an environment.
Alternatively, it is also possible to set up Rancher completely air-gapped without any Internet access. This process is described in detail in the [Rancher docs](../air-gapped-helm-cli-install/air-gapped-helm-cli-install.md).
## Installation Outline
1. [Set up infrastructure](set-up-infrastructure.md)
2. [Set up a Kubernetes cluster](install-kubernetes.md)
3. [Install Rancher](install-rancher.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/set-up-infrastructure.md
-64
View File
@@ -1,64 +0,0 @@
---
title: '1. Set up Infrastructure'
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-behind-an-http-proxy/set-up-infrastructure"/>
</head>
In this section, you will provision the underlying infrastructure for your Rancher management server with internet access through a HTTP proxy.
To install the Rancher management server on a high-availability RKE cluster, we recommend setting up the following infrastructure:
- **Three Linux nodes,** typically virtual machines, in an infrastructure provider such as Amazon's EC2, Google Compute Engine, or vSphere.
- **A load balancer** to direct front-end traffic to the three nodes.
- **A DNS record** to map a URL to the load balancer. This will become the Rancher server URL, and downstream Kubernetes clusters will need to reach it.
These nodes must be in the same region/data center. You may place these servers in separate availability zones.
### Why three nodes?
In an RKE cluster, Rancher server data is stored on etcd. This etcd database runs on all three nodes.
The etcd database requires an odd number of nodes so that it can always elect a leader with a majority of the etcd cluster. If the etcd database cannot elect a leader, etcd can suffer from [split brain](https://www.quora.com/What-is-split-brain-in-distributed-systems), requiring the cluster to be restored from backup. If one of the three etcd nodes fails, the two remaining nodes can elect a leader because they have the majority of the total number of etcd nodes.
### 1. Set up Linux Nodes
These hosts will connect to the internet through an HTTP proxy.
Make sure that your nodes fulfill the general installation requirements for [OS, container runtime, hardware, and networking.](../../installation-requirements/installation-requirements.md)
For an example of one way to set up Linux nodes, refer to this [tutorial](../../../../how-to-guides/new-user-guides/infrastructure-setup/nodes-in-amazon-ec2.md) for setting up nodes as instances in Amazon EC2.
### 2. Set up the Load Balancer
You will also need to set up a load balancer to direct traffic to the Rancher replica on both nodes. That will prevent an outage of any single node from taking down communications to the Rancher management server.
When Kubernetes gets set up in a later step, the RKE tool will deploy an NGINX Ingress controller. This controller will listen on ports 80 and 443 of the worker nodes, answering traffic destined for specific hostnames.
When Rancher is installed (also in a later step), the Rancher system creates an Ingress resource. That Ingress tells the NGINX Ingress controller to listen for traffic destined for the Rancher hostname. The NGINX Ingress controller, when receiving traffic destined for the Rancher hostname, will forward that traffic to the running Rancher pods in the cluster.
For your implementation, consider if you want or need to use a Layer-4 or Layer-7 load balancer:
- **A layer-4 load balancer** is the simpler of the two choices, in which you are forwarding TCP traffic to your nodes. We recommend configuring your load balancer as a Layer 4 balancer, forwarding traffic to ports TCP/80 and TCP/443 to the Rancher management cluster nodes. The Ingress controller on the cluster will redirect HTTP traffic to HTTPS and terminate SSL/TLS on port TCP/443. The Ingress controller will forward traffic to port TCP/80 to the Ingress pod in the Rancher deployment.
- **A layer-7 load balancer** is a bit more complicated but can offer features that you may want. For instance, a layer-7 load balancer is capable of handling TLS termination at the load balancer, as opposed to Rancher doing TLS termination itself. This can be beneficial if you want to centralize your TLS termination in your infrastructure. Layer-7 load balancing also offers the capability for your load balancer to make decisions based on HTTP attributes such as cookies, etc. that a layer-4 load balancer is not able to concern itself with. If you decide to terminate the SSL/TLS traffic on a layer-7 load balancer, you will need to use the `--set tls=external` option when installing Rancher in a later step. For more information, refer to the [Rancher Helm chart options.](../../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination)
For an example showing how to set up an NGINX load balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/nginx-load-balancer.md)
For a how-to guide for setting up an Amazon ELB Network Load Balancer, refer to [this page.](../../../../how-to-guides/new-user-guides/infrastructure-setup/amazon-elb-load-balancer.md)
> **Important:**
> Do not use this load balancer (i.e, the `local` cluster Ingress) to load balance applications other than Rancher following installation. Sharing this Ingress with other applications may result in websocket errors to Rancher following Ingress configuration reloads for other apps. We recommend dedicating the `local` cluster to Rancher and no other applications.
### 3. Set up the DNS Record
Once you have set up your load balancer, you will need to create a DNS record to send traffic to this load balancer.
Depending on your environment, this may be an A record pointing to the LB IP, or it may be a CNAME pointing to the load balancer hostname. In either case, make sure this record is the hostname that you intend Rancher to respond on.
You will need to specify this hostname in a later step when you install Rancher, and it is not possible to change it later. Make sure that your decision is a final one.
For a how-to guide for setting up a DNS record to route domain traffic to an Amazon ELB load balancer, refer to the [official AWS documentation.](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer)
### [Next: Set up a Kubernetes cluster](install-kubernetes.md)
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting.md
-94
View File
@@ -1,94 +0,0 @@
---
title: Troubleshooting Certificates
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting"/>
</head>
<DockerSupportWarning />
## How Do I Know if My Certificates are in PEM Format?
You can recognize the PEM format by the following traits:
- The file begins with the following header:
```
-----BEGIN CERTIFICATE-----
```
- The header is followed by a long string of characters.
- The file ends with a footer:
-----END CERTIFICATE-----
PEM Certificate Example:
```
----BEGIN CERTIFICATE-----
MIIGVDCCBDygAwIBAgIJAMiIrEm29kRLMA0GCSqGSIb3DQEBCwUAMHkxCzAJBgNV
... more lines
VWQqljhfacYPgp8KJUJENQ9h5hZ2nSCrI+W00Jcw4QcEdCI8HL5wmg==
-----END CERTIFICATE-----
```
PEM Certificate Key Example:
```
-----BEGIN RSA PRIVATE KEY-----
MIIGVDCCBDygAwIBAgIJAMiIrEm29kRLMA0GCSqGSIb3DQEBCwUAMHkxCzAJBgNV
... more lines
VWQqljhfacYPgp8KJUJENQ9h5hZ2nSCrI+W00Jcw4QcEdCI8HL5wmg==
-----END RSA PRIVATE KEY-----
```
If your key looks like the example below, see [Converting a Certificate Key From PKCS8 to PKCS1.](#converting-a-certificate-key-from-pkcs8-to-pkcs1)
```
-----BEGIN PRIVATE KEY-----
MIIGVDCCBDygAwIBAgIJAMiIrEm29kRLMA0GCSqGSIb3DQEBCwUAMHkxCzAJBgNV
... more lines
VWQqljhfacYPgp8KJUJENQ9h5hZ2nSCrI+W00Jcw4QcEdCI8HL5wmg==
-----END PRIVATE KEY-----
```
## Converting a Certificate Key From PKCS8 to PKCS1
If you are using a PKCS8 certificate key file, Rancher will log the following line:
```
ListenConfigController cli-config [listener] failed with : failed to read private key: asn1: structure error: tags don't match (2 vs {class:0 tag:16 length:13 isCompound:true})
```
To make this work, you will need to convert the key from PKCS8 to PKCS1 using the command below:
```
openssl rsa -in key.pem -out convertedkey.pem
```
You can now use `convertedkey.pem` as certificate key file for Rancher.
## What is the Order of Certificates if I Want to Add My Intermediate(s)?
The order of adding certificates is as follows:
```
-----BEGIN CERTIFICATE-----
%YOUR_CERTIFICATE%
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
%YOUR_INTERMEDIATE_CERTIFICATE%
-----END CERTIFICATE-----
```
## How Do I Validate My Certificate Chain?
You can validate the certificate chain by using the `openssl` binary. If the output of the command (see the command example below) ends with `Verify return code: 0 (ok)`, your certificate chain is valid. The `ca.pem` file must be the same as you added to the `rancher/rancher` container.
When using a certificate signed by a recognized Certificate Authority, you can omit the `-CAfile` parameter.
Command:
```
openssl s_client -CAfile ca.pem -connect rancher.yourdomain.com:443
...
Verify return code: 0 (ok)
```
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md
-178
View File
@@ -1,178 +0,0 @@
---
title: Installing Rancher on a Single Node Using Docker
description: For development and testing environments only, use a Docker install. Install Docker on a single Linux host, and deploy Rancher with a single Docker container.
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker"/>
</head>
<DockerSupportWarning />
Rancher can be installed by running a single Docker container.
In this installation scenario, you'll install Docker on a single Linux host, and then deploy Rancher on your host using a single Docker container.
> **Want to use an external load balancer?**
> See [Docker Install with an External Load Balancer](../../advanced-options/advanced-use-cases/configure-layer-7-nginx-load-balancer.md) instead.
A Docker installation of Rancher is recommended only for development and testing purposes. The ability to migrate Rancher to a high-availability cluster depends on the Rancher version:
The Rancher backup operator can be used to migrate Rancher from the single Docker container install to an installation on a high-availability Kubernetes cluster. For details, refer to the documentation on [migrating Rancher to a new cluster.](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/migrate-rancher-to-new-cluster.md)
### Privileged Access for Rancher v2.5+
When the Rancher server is deployed in the Docker container, a local Kubernetes cluster is installed within the container for Rancher to use. Because many features of Rancher run as deployments, and privileged mode is required to run containers within containers, you will need to install Rancher with the `--privileged` option.
## Requirements for OS, Docker, Hardware, and Networking
Make sure that your node fulfills the general [installation requirements.](../../installation-requirements/installation-requirements.md)
## 1. Provision Linux Host
Provision a single Linux host according to our [Requirements](../../installation-requirements/installation-requirements.md) to launch your Rancher server.
## 2. Choose an SSL Option and Install Rancher
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
> **Do you want to...**
>
> - Use a proxy? See [HTTP Proxy Configuration](../../../../reference-guides/single-node-rancher-in-docker/http-proxy-configuration.md)
> - Configure custom CA root certificate to access your services? See [Custom CA root certificate](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#custom-ca-certificate)
> - Complete an Air Gap Installation? See [Air Gap Install](../air-gapped-helm-cli-install/air-gapped-helm-cli-install.md)
> - Record all transactions with the Rancher API? See [API Auditing](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log)
Choose from the following options:
- [Option A: Default Rancher-generated Self-signed Certificate](#option-a-default-rancher-generated-self-signed-certificate)
- [Option B: Bring Your Own Certificate, Self-signed](#option-b-bring-your-own-certificate-self-signed)
- [Option C: Bring Your Own Certificate, Signed by a Recognized CA](#option-c-bring-your-own-certificate-signed-by-a-recognized-ca)
- [Option D: Let's Encrypt Certificate](#option-d-lets-encrypt-certificate)
### Option A: Default Rancher-generated Self-signed Certificate
If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself.
Log into your Linux host, and then run the minimum installation command below.
As of Rancher v2.5, privileged access is [required.](#privileged-access-for-rancher-v25)
```bash
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:latest
```
### Option B: Bring Your Own Certificate, Self-signed
In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher.
> **Prerequisites:**
> Create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice.
>
> - The certificate files must be in PEM format.
> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](certificate-troubleshooting.md)
After creating your certificate, run the Docker command below to install Rancher. Use the `-v` flag and provide the path to your certificates to mount them in your container.
| Placeholder | Description |
| ------------------- | --------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
| `<CA_CERTS.pem>` | The path to the certificate authority's certificate. |
As of Rancher v2.5, privileged access is [required.](#privileged-access-for-rancher-v25)
```bash
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
--privileged \
rancher/rancher:latest
```
### Option C: Bring Your Own Certificate, Signed by a Recognized CA
In production environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings.
> **Prerequisites:**
>
> - The certificate files must be in PEM format.
> - In your certificate file, include all intermediate certificates provided by the recognized CA. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](certificate-troubleshooting.md)
After obtaining your certificate, run the Docker command below.
- Use the `-v` flag and provide the path to your certificates to mount them in your container. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary.
- Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher.
| Placeholder | Description |
| ------------------- | ----------------------------- |
| `<CERT_DIRECTORY>` | The path to the directory containing your certificate files. |
| `<FULL_CHAIN.pem>` | The path to your full certificate chain. |
| `<PRIVATE_KEY.pem>` | The path to the private key for your certificate. |
As of Rancher v2.5, privileged access is [required.](#privileged-access-for-rancher-v25)
```bash
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
--privileged \
rancher/rancher:latest \
--no-cacerts
```
### Option D: Let's Encrypt Certificate
> **Remember:** Let's Encrypt provides rate limits for requesting new certificates. Therefore, limit how often you create or destroy the container. For more information, see [Let's Encrypt documentation on rate limits](https://letsencrypt.org/docs/rate-limits/).
For production environments, you also have the option of using [Let's Encrypt](https://letsencrypt.org/) certificates. Let's Encrypt uses an http-01 challenge to verify that you have control over your domain. You can confirm that you control the domain by pointing the hostname that you want to use for Rancher access (for example, `rancher.mydomain.com`) to the IP of the machine it is running on. You can bind the hostname to the IP address by creating an A record in DNS.
> **Prerequisites:**
>
> - Let's Encrypt is an Internet service. Therefore, this option cannot be used in an internal/air gapped network.
> - Create a record in your DNS that binds your Linux host IP address to the hostname that you want to use for Rancher access (`rancher.mydomain.com` for example).
> - Open port `TCP/80` on your Linux host. The Let's Encrypt http-01 challenge can come from any source IP address, so port `TCP/80` must be open to all IP addresses.
After you fulfill the prerequisites, you can install Rancher using a Let's Encrypt certificate by running the following command.
| Placeholder | Description |
| ----------------- | ------------------- |
| `<YOUR.DNS.NAME>` | Your domain address |
As of Rancher v2.5, privileged access is [required.](#privileged-access-for-rancher-v25)
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:latest \
--acme-domain <YOUR.DNS.NAME>
```
## Advanced Options
When installing Rancher on a single node with Docker, there are several advanced options that can be enabled:
- Custom CA Certificate
- API Audit Log
- TLS Settings
- Air Gap
- Persistent Data
- Running `rancher/rancher` and `rancher/rancher-agent` on the Same Node
Refer to [this page](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md) for details.
## Troubleshooting
Refer to [this page](certificate-troubleshooting.md) for frequently asked questions and troubleshooting tips.
## What's Next?
- **Recommended:** Review Single Node [Backup](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-docker-installed-rancher.md) and [Restore](../../../../how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/restore-docker-installed-rancher.md). Although you don't have any data you need to back up right now, we recommend creating backups after regular Rancher use.
- Create a Kubernetes cluster: [Provisioning Kubernetes Clusters](../../../../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/kubernetes-clusters-in-rancher-setup.md).
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/roll-back-docker-installed-rancher.md
-89
View File
@@ -1,89 +0,0 @@
---
title: Rolling Back Rancher Installed with Docker
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/roll-back-docker-installed-rancher"/>
</head>
<DockerSupportWarning />
If a Rancher upgrade does not complete successfully, you'll have to roll back to your Rancher setup that you were using before [Docker Upgrade](upgrade-docker-installed-rancher.md). Rolling back restores:
- Your previous version of Rancher.
- Your data backup created before upgrade.
## Before You Start
During rollback to a prior version of Rancher, you'll enter a series of commands, filling placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (`<EXAMPLE>`). Here's an example of a command with a placeholder:
```
docker pull rancher/rancher:<PRIOR_RANCHER_VERSION>
```
In this command, `<PRIOR_RANCHER_VERSION>` is the version of Rancher you were running before your unsuccessful upgrade. `v2.0.5` for example.
Cross reference the image and reference table below to learn how to obtain this placeholder data. Write down or copy this information before starting the procedure below.
<sup>Terminal <code>docker ps</code> Command, Displaying Where to Find <code>&lt;PRIOR_RANCHER_VERSION&gt;</code> and <code>&lt;RANCHER_CONTAINER_NAME&gt;</code></sup>![Placeholder Reference](/img/placeholder-ref-2.png)
| Placeholder | Example | Description |
| -------------------------- | -------------------------- | ------------------------------------------------------- |
| `<PRIOR_RANCHER_VERSION>` | `v2.0.5` | The rancher/rancher image you used before upgrade. |
| `<RANCHER_CONTAINER_NAME>` | `festive_mestorf` | The name of your Rancher container. |
| `<RANCHER_VERSION>` | `v2.0.5` | The version of Rancher that the backup is for. |
| `<DATE>` | `9-27-18` | The date that the data container or backup was created. |
<br/>
You can obtain `<PRIOR_RANCHER_VERSION>` and `<RANCHER_CONTAINER_NAME>` by logging into your Rancher Server by remote connection and entering the command to view the containers that are running: `docker ps`. You can also view containers that are stopped using a different command: `docker ps -a`. Use these commands for help anytime during while creating backups.
## Rolling Back Rancher
If you have issues upgrading Rancher, roll it back to its latest known healthy state by pulling the last version you used and then restoring the backup you made before upgrade.
>**Warning!** Rolling back to a previous version of Rancher destroys any changes made to Rancher following the upgrade. Unrecoverable data loss may occur.
1. Using a remote Terminal connection, log into the node running your Rancher Server.
1. Pull the version of Rancher that you were running before upgrade. Replace the `<PRIOR_RANCHER_VERSION>` with that version.
For example, if you were running Rancher v2.0.5 before upgrade, pull v2.0.5.
```
docker pull rancher/rancher:<PRIOR_RANCHER_VERSION>
```
1. Stop the container currently running Rancher Server. Replace `<RANCHER_CONTAINER_NAME>` with the name of your Rancher container.
```
docker stop <RANCHER_CONTAINER_NAME>
```
You can obtain the name for your Rancher container by entering `docker ps`.
1. Move the backup tarball that you created during completion of [Docker Upgrade](upgrade-docker-installed-rancher.md) onto your Rancher Server. Change to the directory that you moved it to. Enter `dir` to confirm that it's there.
If you followed the naming convention we suggested in [Docker Upgrade](upgrade-docker-installed-rancher.md), it will have a name similar to (`rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz`).
1. Run the following command to replace the data in the `rancher-data` container with the data in the backup tarball, replacing the placeholder. Don't forget to close the quotes.
```
docker run --volumes-from rancher-data \
-v $PWD:/backup busybox sh -c "rm /var/lib/rancher/* -rf \
&& tar zxvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz"
```
1. Start a new Rancher Server container with the `<PRIOR_RANCHER_VERSION>` tag placeholder pointing to the data container.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:<PRIOR_RANCHER_VERSION>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
>**Note:** _Do not_ stop the rollback after initiating it, even if the rollback process seems longer than expected. Stopping the rollback may result in database issues during future upgrades.
1. Wait a few moments and then open Rancher in a web browser. Confirm that the rollback succeeded and that your data is restored.
**Result:** Rancher is rolled back to its version and data state before upgrade.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/upgrade-docker-installed-rancher.md
-372
View File
@@ -1,372 +0,0 @@
---
title: Upgrading Rancher Installed with Docker
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/rancher-on-a-single-node-with-docker/upgrade-docker-installed-rancher"/>
</head>
The following instructions will guide you through upgrading a Rancher server that was installed with Docker.
<DockerSupportWarning />
## Prerequisites
- **Review the [known upgrade issues](../../install-upgrade-on-a-kubernetes-cluster/upgrades.md#known-upgrade-issues) in the Rancher documentation for the most noteworthy issues to consider when upgrading Rancher. A more complete list of known issues for each Rancher version can be found in the release notes on [GitHub](https://github.com/rancher/rancher/releases) and on the [Rancher forums.](https://forums.rancher.com/c/announcements/12) Note that upgrades to or from any chart in the [rancher-alpha repository](../../resources/choose-a-rancher-version.md#helm-chart-repositories) aren’t supported.
- **For [air gap installs only,](../air-gapped-helm-cli-install/air-gapped-helm-cli-install.md) collect and populate images for the new Rancher server version.** Follow the guide to [populate your private registry](../air-gapped-helm-cli-install/publish-images.md) with the images for the Rancher version that you want to upgrade to.
## Placeholder Review
During upgrade, you'll enter a series of commands, filling placeholders with data from your environment. These placeholders are denoted with angled brackets and all capital letters (`<EXAMPLE>`).
Here's an **example** of a command with a placeholder:
```
docker stop <RANCHER_CONTAINER_NAME>
```
In this command, `<RANCHER_CONTAINER_NAME>` is the name of your Rancher container.
## Get Data for Upgrade Commands
To obtain the data to replace the placeholders, run:
```
docker ps
```
Write down or copy this information before starting the upgrade.
<sup>Terminal <code>docker ps</code> Command, Displaying Where to Find <code>&lt;RANCHER_CONTAINER_TAG&gt;</code> and <code>&lt;RANCHER_CONTAINER_NAME&gt;</code></sup>
![Placeholder Reference](/img/placeholder-ref.png)
| Placeholder | Example | Description |
| -------------------------- | -------------------------- | --------------------------------------------------------- |
| `<RANCHER_CONTAINER_TAG>` | `v2.1.3` | The rancher/rancher image you pulled for initial install. |
| `<RANCHER_CONTAINER_NAME>` | `festive_mestorf` | The name of your Rancher container. |
| `<RANCHER_VERSION>` | `v2.1.3` | The version of Rancher that you're creating a backup for. |
| `<DATE>` | `2018-12-19` | The date that the data container or backup was created. |
<br/>
You can obtain `<RANCHER_CONTAINER_TAG>` and `<RANCHER_CONTAINER_NAME>` by logging into your Rancher server by remote connection and entering the command to view the containers that are running: `docker ps`. You can also view containers that are stopped using a different command: `docker ps -a`. Use these commands for help anytime during while creating backups.
## Upgrade Outline
During upgrade, you create a copy of the data from your current Rancher container and a backup in case something goes wrong. Then you deploy the new version of Rancher in a new container using your existing data. Follow the steps to upgrade Rancher server:
- [1. Create a copy of the data from your Rancher server container](#1-create-a-copy-of-the-data-from-your-rancher-server-container)
- [2. Create a backup tarball](#2-create-a-backup-tarball)
- [3. Pull the new Docker image](#3-pull-the-new-docker-image)
- [4. Start the new Rancher server container](#4-start-the-new-rancher-server-container)
- [5. Verify the Upgrade](#5-verify-the-upgrade)
- [6. Clean up your old Rancher server container](#6-clean-up-your-old-rancher-server-container)
## 1. Create a copy of the data from your Rancher server container
1. Using a remote Terminal connection, log into the node running your Rancher server.
1. Stop the container currently running Rancher server. Replace `<RANCHER_CONTAINER_NAME>` with the name of your Rancher container.
```
docker stop <RANCHER_CONTAINER_NAME>
```
1. <a id="backup"></a>Use the command below, replacing each placeholder, to create a data container from the Rancher container that you just stopped.
```
docker create --volumes-from <RANCHER_CONTAINER_NAME> --name rancher-data rancher/rancher:<RANCHER_CONTAINER_TAG>
```
## 2. Create a backup tarball
1. <a id="tarball"></a>From the data container that you just created (<code>rancher-data</code>), create a backup tarball (<code>rancher-data-backup-&lt;RANCHER_VERSION&gt;-&lt;DATE&gt;.tar.gz</code>).
This tarball will serve as a rollback point if something goes wrong during upgrade. Use the following command, replacing each placeholder.
```
docker run --volumes-from rancher-data -v "$PWD:/backup" --rm busybox tar zcvf /backup/rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz /var/lib/rancher
```
**Step Result:** When you enter this command, a series of commands should run.
1. Enter the `ls` command to confirm that the backup tarball was created. It will have a name similar to `rancher-data-backup-<RANCHER_VERSION>-<DATE>.tar.gz`.
```
[rancher@ip-10-0-0-50 ~]$ ls
rancher-data-backup-v2.1.3-20181219.tar.gz
```
1. Move your backup tarball to a safe location external from your Rancher server.
## 3. Pull the New Docker Image
Pull the image of the Rancher version that you want to upgrade to.
Placeholder | Description
------------|-------------
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
```
docker pull rancher/rancher:<RANCHER_VERSION_TAG>
```
## 4. Start the New Rancher Server Container
Start a new Rancher server container using the data from the `rancher-data` container. Remember to pass in all the environment variables that you had used when you started the original container.
>**Important:** _Do not_ stop the upgrade after initiating it, even if the upgrade process seems longer than expected. Stopping the upgrade may result in database migration errors during future upgrades.
If you used a proxy, see [HTTP Proxy Configuration.](../../../../reference-guides/single-node-rancher-in-docker/http-proxy-configuration.md)
If you configured a custom CA root certificate to access your services, see [Custom CA root certificate.](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#custom-ca-certificate)
If you are recording all transactions with the Rancher API, see [API Auditing](../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log)
To see the command to use when starting the new Rancher server container, choose from the following options:
- Docker Upgrade
- Docker Upgrade for Air Gap Installs
<Tabs>
<TabItem value="Docker Upgrade">
Select which option you had installed Rancher server
### Option A: Default Self-Signed Certificate
<details id="option-a">
<summary>Click to expand</summary>
If you have selected to use the Rancher generated self-signed certificate, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container.
Placeholder | Description
------------|-------------
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:<RANCHER_VERSION_TAG>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
### Option B: Bring Your Own Certificate: Self-Signed
<details id="option-b">
<summary>Click to expand</summary>
If you have selected to bring your own self-signed certificate, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container and need to have access to the same certificate that you had originally installed with.
>**Reminder of the Cert Prerequisite:** The certificate files must be in PEM format. In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates.
Placeholder | Description
------------|-------------
`<CERT_DIRECTORY>` | The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>` | The path to your full certificate chain.
`<PRIVATE_KEY.pem>` | The path to the private key for your certificate.
`<CA_CERTS.pem>` | The path to the certificate authority's certificate.
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
--privileged \
rancher/rancher:<RANCHER_VERSION_TAG>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
### Option C: Bring Your Own Certificate: Signed by Recognized CA
<details id="option-c">
<summary>Click to expand</summary>
If you have selected to use a certificate signed by a recognized CA, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container and need to have access to the same certificates that you had originally installed with. Remember to include `--no-cacerts` as an argument to the container to disable the default CA certificate generated by Rancher.
>**Reminder of the Cert Prerequisite:** The certificate files must be in PEM format. In your certificate file, include all intermediate certificates provided by the recognized CA. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](certificate-troubleshooting.md)
Placeholder | Description
------------|-------------
`<CERT_DIRECTORY>` | The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>` | The path to your full certificate chain.
`<PRIVATE_KEY.pem>` | The path to the private key for your certificate.
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
--privileged \
rancher/rancher:<RANCHER_VERSION_TAG> \
--no-cacerts
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
### Option D: Let's Encrypt Certificate
<details id="option-d">
<summary>Click to expand</summary>
>**Remember:** Let's Encrypt provides rate limits for requesting new certificates. Therefore, limit how often you create or destroy the container. For more information, see [Let's Encrypt documentation on rate limits](https://letsencrypt.org/docs/rate-limits/).
If you have selected to use [Let's Encrypt](https://letsencrypt.org/) certificates, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container and need to provide the domain that you had used when you originally installed Rancher.
>**Reminder of the Cert Prerequisites:**
>
>- Create a record in your DNS that binds your Linux host IP address to the hostname that you want to use for Rancher access (`rancher.mydomain.com` for example).
>- Open port `TCP/80` on your Linux host. The Let's Encrypt http-01 challenge can come from any source IP address, so port `TCP/80` must be open to all IP addresses.
Placeholder | Description
------------|-------------
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
`<YOUR.DNS.NAME>` | The domain address that you had originally started with
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
--privileged \
rancher/rancher:<RANCHER_VERSION_TAG> \
--acme-domain <YOUR.DNS.NAME>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
</TabItem>
<TabItem value="Docker Air Gap Upgrade">
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
When starting the new Rancher server container, choose from the following options:
### Option A: Default Self-Signed Certificate
<details id="option-a">
<summary>Click to expand</summary>
If you have selected to use the Rancher generated self-signed certificate, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container.
Placeholder | Description
------------|-------------
`<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port.
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to to upgrade to.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged \
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
### Option B: Bring Your Own Certificate: Self-Signed
<details id="option-b">
<summary>Click to expand</summary>
If you have selected to bring your own self-signed certificate, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container and need to have access to the same certificate that you had originally installed with.
>**Reminder of the Prerequisite:** The certificate files must be in PEM format. In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](certificate-troubleshooting.md)
Placeholder | Description
------------|-------------
`<CERT_DIRECTORY>` | The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>` | The path to your full certificate chain.
`<PRIVATE_KEY.pem>` | The path to the private key for your certificate.
`<CA_CERTS.pem>` | The path to the certificate authority's certificate.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port.
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
```
docker run -d --restart=unless-stopped \
-p 80:80 -p 443:443 \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-v /<CERT_DIRECTORY>/<CA_CERTS.pem>:/etc/rancher/ssl/cacerts.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged \
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
### Option C: Bring Your Own Certificate: Signed by Recognized CA
<details id="option-c">
<summary>Click to expand</summary>
If you have selected to use a certificate signed by a recognized CA, you add the `--volumes-from rancher-data` to the command that you had started your original Rancher server container and need to have access to the same certificates that you had originally installed with.
>**Reminder of the Prerequisite:** The certificate files must be in PEM format. In your certificate file, include all intermediate certificates provided by the recognized CA. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](certificate-troubleshooting.md)
Placeholder | Description
------------|-------------
`<CERT_DIRECTORY>` | The path to the directory containing your certificate files.
`<FULL_CHAIN.pem>` | The path to your full certificate chain.
`<PRIVATE_KEY.pem>` | The path to the private key for your certificate.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | Your private registry URL and port.
`<RANCHER_VERSION_TAG>` | The release tag of the [Rancher version](../../../../reference-guides/installation-references/helm-chart-options.md) that you want to upgrade to.
> **Note:** Use the `--no-cacerts` as argument to the container to disable the default CA certificate generated by Rancher.
```
docker run -d --volumes-from rancher-data \
--restart=unless-stopped \
-p 80:80 -p 443:443 \
--no-cacerts \
-v /<CERT_DIRECTORY>/<FULL_CHAIN.pem>:/etc/rancher/ssl/cert.pem \
-v /<CERT_DIRECTORY>/<PRIVATE_KEY.pem>:/etc/rancher/ssl/key.pem \
-e CATTLE_SYSTEM_DEFAULT_REGISTRY=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Set a default private registry to be used in Rancher
-e CATTLE_SYSTEM_CATALOG=bundled \ # Use the packaged Rancher system charts
--privileged
<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher:<RANCHER_VERSION_TAG>
```
As of Rancher v2.5, privileged access is [required.](rancher-on-a-single-node-with-docker.md#privileged-access-for-rancher-v25)
</details>
</TabItem>
</Tabs>
**Result:** You have upgraded Rancher. Data from your upgraded server is now saved to the `rancher-data` container for use in future upgrades.
## 5. Verify the Upgrade
Log into Rancher. Confirm that the upgrade succeeded by checking the version displayed in the bottom-left corner of the browser window.
>**Having network issues in your user clusters following upgrade?**
>
> See [Restoring Cluster Networking](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.0-2.4/getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/namespace-migration.md).
## 6. Clean up Your Old Rancher Server Container
Remove the previous Rancher server container. If you only stop the previous Rancher server container (and don't remove it), the container may restart after the next server reboot.
## Rolling Back
If your upgrade does not complete successfully, you can roll back Rancher server and its data back to its last healthy state. For more information, see [Docker Rollback](roll-back-docker-installed-rancher.md).
versioned_docs/version-2.5/getting-started/installation-and-upgrade/resources/add-tls-secrets.md
-41
View File
@@ -1,41 +0,0 @@
---
title: Adding TLS Secrets
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/resources/add-tls-secrets"/>
</head>
Kubernetes will create all the objects and services for Rancher, but it will not become available until we populate the `tls-rancher-ingress` secret in the `cattle-system` namespace with the certificate and key.
Combine the server certificate followed by any intermediate certificate(s) needed into a file named `tls.crt`. Copy your certificate key into a file named `tls.key`.
For example, [acme.sh](https://acme.sh) provides server certificate and CA chains in `fullchain.cer` file.
This `fullchain.cer` should be renamed to `tls.crt` & certificate key file as `tls.key`.
Use `kubectl` with the `tls` secret type to create the secrets.
```
kubectl -n cattle-system create secret tls tls-rancher-ingress \
--cert=tls.crt \
--key=tls.key
```
> **Note:** If you want to replace the certificate, you can delete the `tls-rancher-ingress` secret using `kubectl -n cattle-system delete secret tls-rancher-ingress` and add a new one using the command shown above. If you are using a private CA signed certificate, replacing the certificate is only possible if the new certificate is signed by the same CA as the certificate currently in use.
## Using a Private CA Signed Certificate
If you are using a private CA, Rancher requires a copy of the private CA's root certificate or certificate chain, which the Rancher Agent uses to validate the connection to the server.
Create a file named `cacerts.pem` that only contains the root CA certificate or certificate chain from your private CA, and use `kubectl` to create the `tls-ca` secret in the `cattle-system` namespace.
```
kubectl -n cattle-system create secret generic tls-ca \
--from-file=cacerts.pem
```
> **Note:** The configured `tls-ca` secret is retrieved when Rancher starts. On a running Rancher installation the updated CA will take effect after new Rancher pods are started.
## Updating a Private CA Certificate
Follow the steps on [this page](update-rancher-certificate.md) to update the SSL certificate of the ingress in a Rancher [high availability Kubernetes installation](../install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) or to switch from the default self-signed certificate to a custom certificate.
versioned_docs/version-2.5/getting-started/installation-and-upgrade/resources/choose-a-rancher-version.md
-116
View File
@@ -1,116 +0,0 @@
---
title: Choosing a Rancher Version
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/resources/choose-a-rancher-version"/>
</head>
This section describes how to choose a Rancher version.
For a high-availability installation of Rancher, which is recommended for production, the Rancher server is installed using a **Helm chart** on a Kubernetes cluster. Refer to the [Helm version requirements](./helm-version-requirements.md) to choose a version of Helm to install Rancher.
For Docker installations of Rancher, which is used for development and testing, you will install Rancher as a **Docker image.**
The Helm chart version also applies to RancherD installs because RancherD installs the Rancher Helm chart on a Kubernetes cluster.
> **Note:** RancherD was an experimental feature available as part of Rancher v2.5.4 through v2.5.10 but is now deprecated and not available for recent releases.
<Tabs>
<TabItem value="Helm Charts">
When installing, upgrading, or rolling back Rancher Server when it is [installed on a Kubernetes cluster](../install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md), Rancher server is installed using a Helm chart on a Kubernetes cluster. Therefore, as you prepare to install or upgrade a high availability Rancher configuration, you must add a Helm chart repository that contains the charts for installing Rancher.
Refer to the [Helm version requirements](./helm-version-requirements.md) to choose a version of Helm to install Rancher.
### Helm Chart Repositories
Rancher provides several different Helm chart repositories to choose from. We align our latest and stable Helm chart repositories with the Docker tags that are used for a Docker installation. Therefore, the `rancher-latest` repository will contain charts for all the Rancher versions that have been tagged as `rancher/rancher:latest`. When a Rancher version has been promoted to the `rancher/rancher:stable`, it will get added to the `rancher-stable` repository.
| Type | Command to Add the Repo | Description of the Repo |
| -------------- | ------------ | ----------------- |
| rancher-latest | `helm repo add rancher-latest https://releases.rancher.com/server-charts/latest` | Adds a repository of Helm charts for the latest versions of Rancher. We recommend using this repo for testing out new Rancher builds. |
| rancher-stable | `helm repo add rancher-stable https://releases.rancher.com/server-charts/stable` | Adds a repository of Helm charts for older, stable versions of Rancher. We recommend using this repo for production environments. |
| rancher-alpha | `helm repo add rancher-alpha https://releases.rancher.com/server-charts/alpha` | Adds a repository of Helm charts for alpha versions of Rancher for previewing upcoming releases. These releases are discouraged in production environments. Upgrades _to_ or _from_ charts in the rancher-alpha repository to any other chart, regardless or repository, aren't supported. |
Instructions on when to select these repos are available below in [Switching to a Different Helm Chart Repository](#switching-to-a-different-helm-chart-repository).
> **Note:** All charts in the `rancher-stable` repository will correspond with any Rancher version tagged as `stable`.
### Helm Chart Versions
Rancher Helm chart versions match the Rancher version (i.e `appVersion`). Once you've added the repo you can search it to show available versions with the following command:<br/>
&nbsp;&nbsp;&nbsp;&nbsp;`helm search repo --versions`
If you have several repos you can specify the repo name, ie. `helm search repo rancher-stable/rancher --versions` <br/>
For more information, see https://helm.sh/docs/helm/helm_search_repo/
To fetch a specific version of your chosen repo, define the `--version` parameter like in the following example:<br/>
&nbsp;&nbsp;&nbsp;&nbsp;`helm fetch rancher-stable/rancher --version=2.4.8`
### Switching to a Different Helm Chart Repository
After installing Rancher, if you want to change which Helm chart repository to install Rancher from, you will need to follow these steps.
> **Note:** Because the rancher-alpha repository contains only alpha charts, switching between the rancher-alpha repository and the rancher-stable or rancher-latest repository for upgrades is not supported.
- Latest: Recommended for trying out the newest features
```
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
```
- Stable: Recommended for production environments
```
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
- Alpha: Experimental preview of upcoming releases.
```
helm repo add rancher-alpha https://releases.rancher.com/server-charts/alpha
```
Note: Upgrades are not supported to, from, or between Alphas.
1. List the current Helm chart repositories.
```plain
helm repo list
NAME URL
stable https://charts.helm.sh/stable
rancher-<CHART_REPO> https://releases.rancher.com/server-charts/<CHART_REPO>
```
2. Remove the existing Helm Chart repository that contains your charts to install Rancher, which will either be `rancher-stable` or `rancher-latest` depending on what you had initially added.
```plain
helm repo remove rancher-<CHART_REPO>
```
3. Add the Helm chart repository that you want to start installing Rancher from.
```plain
helm repo add rancher-<CHART_REPO> https://releases.rancher.com/server-charts/<CHART_REPO>
```
4. Continue to follow the steps to [upgrade Rancher](../install-upgrade-on-a-kubernetes-cluster/upgrades.md) from the new Helm chart repository.
</TabItem>
<TabItem value="Docker Images">
When performing [Docker installs](../other-installation-methods/rancher-on-a-single-node-with-docker/rancher-on-a-single-node-with-docker.md), upgrades, or rollbacks, you can use _tags_ to install a specific version of Rancher.
### Server Tags
Rancher Server is distributed as a Docker image, which have tags attached to them. You can specify this tag when entering the command to deploy Rancher. Remember that if you use a tag without an explicit version (like `latest` or `stable`), you must explicitly pull a new version of that image tag. Otherwise, any image cached on the host will be used.
| Tag | Description |
| -------------------------- | ------ |
| `rancher/rancher:latest` | Our latest development release. These builds are validated through our CI automation framework. These releases are not recommended for production environments. |
| `rancher/rancher:stable` | Our newest stable release. This tag is recommended for production. |
| `rancher/rancher:<v2.X.X>` | You can install specific versions of Rancher by using the tag from a previous release. See what's available at Docker Hub. |
> **Notes:**
>
> - The `master` tag or any tag with `-rc` or another suffix is meant for the Rancher testing team to validate. You should not use these tags, as these builds are not officially supported.
> - Want to install an alpha review for preview? Install using one of the alpha tags listed on our [announcements page](https://forums.rancher.com/c/announcements) (e.g., `v2.2.0-alpha1`). Caveat: Alpha releases cannot be upgraded to or from any other release.
</TabItem>
</Tabs>
versioned_docs/version-2.5/getting-started/installation-and-upgrade/resources/custom-ca-root-certificates.md
-28
View File
@@ -1,28 +0,0 @@
---
title: About Custom CA Root Certificates
---
<head>
<link rel="canonical" href="https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/resources/custom-ca-root-certificates"/>
</head>
If you're using Rancher in an internal production environment where you aren't exposing apps publicly, use a certificate from a private certificate authority (CA).
Services that Rancher needs to access are sometimes configured with a certificate from a custom/internal CA root, also known as self signed certificate. If the presented certificate from the service cannot be validated by Rancher, the following error displays: `x509: certificate signed by unknown authority`.
To validate the certificate, the CA root certificates need to be added to Rancher. As Rancher is written in Go, we can use the environment variable `SSL_CERT_DIR` to point to the directory where the CA root certificates are located in the container. The CA root certificates directory can be mounted using the Docker volume option (`-v host-source-directory:container-destination-directory`) when starting the Rancher container.
Examples of services that Rancher can access:
- Catalogs
- Authentication providers
- Accessing hosting/cloud API when using Node Drivers
## Installing with the custom CA Certificate
For details on starting a Rancher container with your private CA certificates mounted, refer to the installation docs:
- [Docker install Custom CA certificate options](../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#custom-ca-certificate)
- [Kubernetes install options for Additional Trusted CAs](../../../reference-guides/installation-references/helm-chart-options.md#additional-trusted-cas)

Some files were not shown because too many files have changed in this diff Show More