Clarify the configuration for external TLS termination with NGINX
v0.22, including:
- Instructions to enable the `use-forwarded-headers` option for
ingress in both RKE and RKE2 installations.
- A new section for RKE2 detailing how to create a custom
configuration file for this setting.
- Updates to the required headers section to include
`X-Forwarded-Proto` and `X-Forwarded-Port`.
The documentation has been revised to provide clearer guidance on
configuring Ingress for external TLS with NGINX v0.22. Key changes
include:
- Removed references to NGINX v0.25.
- Added instructions for enabling the `use-forwarded-headers`
option in the `cluster.yml` for RKE installations.
- Included steps for creating a custom
`rke2-ingress-nginx-config.yaml` for RKE2 installations.
- Provided a YAML snippet for HelmChartConfig to demonstrate
how to set the `use-forwarded-headers` option in Helm chart
values.
Updating external TLS termination required settings. Updating the documentation to be clear on RKE configuration vs RKE2 configuration.
Based on closed issue: https://github.com/rancher/rancher/issues/35088
When cleaning nodes for re-use which were deployed manually with RKE2, the process differs slightly. This is more important in reverse so I'm editing that doc as well. I think there's like a note callout or something that might be better?
This adds a few points which should shed light on how to search users
using a drop-down list in Rancher, whether to add users to clusters or
to projects.
Migrating rancher across different Kubernetes distributions causes the local
cluster object not to specify the correct distribution and driver.
This commit adds a workaround to be applied after restoration for a migration
and before bringing Rancher up, so that the local cluster will have the
correct data for the new distribution it runs on.
* [2.9.4] versions table entry
* [2.9.4] webhook table entry
* [2.9.4] csp adapter table entry
* [2.9.4] deprecated features table entry
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* [2.9.4] Update link to prime quick start guide for v2.9
---------
Co-authored-by: Billy Tat <btat@suse.com>
* Use crds.enabled to install cert-manager
Signed-off-by: Dharmit Shah <dharmit.shah@suse.com>
* versioned
* reverted changes to earliest versions and slightly modified comment
* spacing issue
---------
Signed-off-by: Dharmit Shah <dharmit.shah@suse.com>
Co-authored-by: Dharmit Shah <dharmit.shah@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
* Use crds.enabled to install cert-manager
Signed-off-by: Dharmit Shah <dharmit.shah@suse.com>
* versioned
* reverted changes to earliest versions and slightly modified comment
* spacing issue
---------
Signed-off-by: Dharmit Shah <dharmit.shah@suse.com>
Co-authored-by: Dharmit Shah <dharmit.shah@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
* [2.9.0] #1403 update feature flags - uiextension
* bit about noAuth
* pronouns
* reorder sentences
* updated according to suggestions from diogoasouza
* generally available
* versioning
* Add back external-rules
Co-authored-by: Billy Tat <btat@suse.com>
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
---------
Co-authored-by: Lucas Saintarbor <lucas.saintarbor@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
* Adding external-rules v2.9 section
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Revising the Feature Flag page after review and adding a behavior change section regarding external objects to the Cluster and Project Roles user guide page.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Applying suggestion after review.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Moving the external-rules into below table to denote removed status and note information about default behavior.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Correct merge conflict
Co-authored-by: Billy Tat <btat@suse.com>
---------
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
Co-authored-by: Lucas Saintarbor <lucas.saintarbor@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
* Update CSP Adapter version table for v2.9.0
* Update CSP Adapter version table for v2.9.0 (docs folder)
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
---------
Co-authored-by: Billy Tat <btat@suse.com>
* Update webook table for v2.9 / latest
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
---------
Co-authored-by: Billy Tat <btat@suse.com>
* Add version table for v2.9.0
* Update src/pages/versions.md
Co-authored-by: Billy Tat <btat@suse.com>
---------
Co-authored-by: Billy Tat <btat@suse.com>
* 1370-optional-filter-on-azure-ad-auth-group-memberships
* assorted revisions, including adding a section and moving the step for initial setup
* reduce repetition by linking to section
* added image
* slight reword
* apply to 2.9
* Debug mode for fleet in rancher
Note how to enable debug mode for fleet in rancher in Troubleshotting section added.
* suggestions from ci incorporated
* Apply suggestions from code review
Co-authored-by: Marty Hernandez Avedon <marty.avedon@suse.com>
* versioning
---------
Co-authored-by: Marty Hernandez Avedon <marty.avedon@suse.com>
* 1151 Add documentation for OCI feature in Apps & Marketplace Section
* rename file, edit intro, add canonical link
* edited refresh instructions
* edited update instructions
* edited delete instructions
* initial attempt at add an oci registry
* completed edit of first section, made misc revisions to other text
* escape angle brackets
* moved file
* added edited text of helm-charts-in-rancher from #1320
* instructions updated to sync with #1320
* updated with suggestions from review
* more edits
* more suggestions from reviews
* most changes addressed except line 101
* line 101 rate limiting addressed
* Apply suggestions from code review
Co-authored-by: Diogo Souza <diogo.souza@suse.com>
Co-authored-by: Sakala Venkata Krishna Rohit <rohitsakala@gmail.com>
* Apply suggestions from code review
Co-authored-by: Diogo Souza <diogo.souza@suse.com>
* Apply suggestions from code review
Co-authored-by: Sakala Venkata Krishna Rohit <rohitsakala@gmail.com>
* rename
* clarified what triggers the exp backoff
* edit suggestions on clarifications
* Apply suggestions from code review
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* Apply suggestions from code review
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* sidebar for v2.9
* experimental feature caution
* ported to docs/
* removed slashes as they were displaying
* using literals to get around annoying confusion around how to treat slashes
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* revised caution box
* drop the yet
* rm'd 2 instances of 'rate-limiting'
* added link to limitations section
* rm'd bit that begs the question about other options
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* Apply suggestions from code review
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
---------
Co-authored-by: Diogo Souza <diogo.souza@suse.com>
Co-authored-by: Sakala Venkata Krishna Rohit <rohitsakala@gmail.com>
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
Update versioned_docs/version-2.9/how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md
Update versioned_docs/version-2.7/how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md
Update versioned_docs/version-2.8/how-to-guides/new-user-guides/manage-clusters/clean-cluster-nodes.md
Co-Authored-By: Billy Tat <btat@suse.com>
* Adding note on generating session token and the service account name expected.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Updating after review and researching Kiali docs. Added link to Kiali docs and Kiali token authentication strategy page.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Updating link with better example
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Updating the Istio - Generate and View Traffic from Istio page to include text guiding users on token generation for Kiali login. This commit has strictly structural changes to the page.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Removing repetitive intro line.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Updating note across versions to include information on Istio auth strategy and adding text into Generate and View Traffic - Prereq section regarding auth strategy and the service account name to specify during token generation.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Applying review suggestions.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* Fixing header to h2 and rephrasing optional text after review.
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
---------
Signed-off-by: Sunil Singh <sunil.singh@suse.com>
* 1358 Link to 'Settings for etcd tuning' no longer valid
updates dead link to new URL
* updated etcd version links
2.0-2.4 was tricky, I opted for etcd v3.3 as it's identical to the text on v3.4 except for a single clarifying heading
* update link in v2.5 to etcd v3.3 to cover downstream
* other 3.4 files in /docs
* other 3.4 files in /v2.9
* other 3.4 files in /v2.8
* other 3.4 files in /v2.7
* other 3.4 files in /v2.6
* other 3.4 files in /v2.5
* other 3.4 files in /v2.0-2.4
* rm'ing '.0' from URLs
* update v2.6 links to account for differing etcd versions
* chinese links
* Warn not to allow nonadmin on Rancher local cluster
* versioning
* Apply suggestions from code review
Co-authored-by: Lucas Saintarbor <lucas.saintarbor@suse.com>
* versioning applied to suggestion
---------
Co-authored-by: Lucas Saintarbor <lucas.saintarbor@suse.com>
* SURE-8285 Document user retention feature
* fixing changes for global-configuration.md
* versioning plus the enable user authentication page
* Apply suggestions from code review
Co-authored-by: Andy Pitcher <andy.pitcher@suse.com>
Co-authored-by: Peter Matseykanets <peterm@mail.ru>
Co-authored-by: pdellamore <pietro.dellamore@suse.com>
* versioning, updates to descriptions of settings
* correcting versioning & rm v2.9 file
* one more correction
* if you can't remove the file, just make sure it's synced...
* Update docs/how-to-guides/advanced-user-guides/enable-user-retention.md
Co-authored-by: Peter Matseykanets <peterm@mail.ru>
* user-last-login-default
* more settings, more details about deletion behavior
* versioning
* more explanation for last-login
* revising blurb about disabling
* describing deletion behavior less direly
* corrected commands
* fix overwrite
* corrected commands
* updated description of zero value
* sidebars
* canonical link fixed
* rm'd parenthetical versioning remark
* heading capitalization, log in vs login
* more log in
* revise command, important box, enabling user retention section
* clarifying that some optional settings are also global
* rm patch mention from 2.9
* explain how to view settings for individual users
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* duplicate word rm'd
---------
Co-authored-by: Andy Pitcher <andy.pitcher@suse.com>
Co-authored-by: Peter Matseykanets <peterm@mail.ru>
Co-authored-by: pdellamore <pietro.dellamore@suse.com>
Co-authored-by: Billy Tat <btat@suse.com>
We currently recommend that users push to release branches, but we often don't have such branches, and instead use milestones and labels to track changes intended for releases
* 1283 update Rancher security best practices to address public IP exposure
* link and bullet points
* Update docs/reference-guides/rancher-security/rancher-security-best-practices.md
* versioning
* typo
* 358 Glossary project
initial draft + styling for definition tags
* added sidebar, revised styling
* redundant styling specification
* typo
* switching to shared-file/import structure so we have a single source file to update regardless of version
to achieve this, we needed to use an madx-code-block to give us a proper side navigation TOC
* updated styling + synonyms
* moved version back to before def as grouping it with the synonyms/related terms was visually confusing
added some definitions (catalogs, downstream cluster)
* styling
* filling out definitions icons through M
* build failed due to comment tag?
* rem'd comments as they were causing the build to fail due to unexpected token error
* revised some definitions
* revised wording and some formatting fixes
* syncing with list in PR
* versioning
* rm'd placeholders
* added Sunil's definitions
* tag
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* rm'd placeholder
* rancher server definition + synonyms
* rm'd note from files to put in issue
* rm rancher enterprise, rke government
* + extension catalogs & registered cluster, syncing related terms, syncing language in definitions
* extra tag
* versioning info for rke2
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* updated RKE def
* copy-paste error correction
---------
Co-authored-by: Billy Tat <btat@suse.com>
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* 358 Glossary project
initial draft + styling for definition tags
* added sidebar, revised styling
* redundant styling specification
* typo
* switching to shared-file/import structure so we have a single source file to update regardless of version
to achieve this, we needed to use an madx-code-block to give us a proper side navigation TOC
* updated styling + synonyms
* moved version back to before def as grouping it with the synonyms/related terms was visually confusing
added some definitions (catalogs, downstream cluster)
* styling
* filling out definitions icons through M
* build failed due to comment tag?
* rem'd comments as they were causing the build to fail due to unexpected token error
* revised some definitions
* revised wording and some formatting fixes
* syncing with list in PR
* versioning
* rm'd placeholders
* added Sunil's definitions
* tag
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* rm'd placeholder
* rancher server definition + synonyms
* rm'd note from files to put in issue
* rm rancher enterprise, rke government
* + extension catalogs & registered cluster, syncing related terms, syncing language in definitions
* extra tag
* versioning info for rke2
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* Apply suggestions from code review
Co-authored-by: Billy Tat <btat@suse.com>
* updated RKE def
---------
Co-authored-by: Billy Tat <btat@suse.com>
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* Update general-faq.md
removing notice about mesos & swarm. it's been 6+ years since 2.0 was released, time to move on.
* versioning
---------
Co-authored-by: martyav <marty.avedon@suse.com>
* 999 Clarify support and stipulations for use of firewall in documentation
added scarier warning about firewalld usage
* revised language slightly
* Update docs/how-to-guides/advanced-user-guides/open-ports-with-firewalld.md
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
* versioning, updated link, & abbreviated warning for v2.0-2.4
---------
Co-authored-by: Sunil Singh <sunil.singh@suse.com>
- Verify if changes pertain to other versions of Rancher. If they do, finalize the edits on one version of the page, then apply the edits to the other versions.
- If the pull request is dependent on an upcoming release, make sure to target the release branch instead of `main`.
- If the pull request is dependent on an upcoming release, remember to add a "MERGE ON RELEASE" label and set the proper milestone.
## Description
@@ -24,4 +24,4 @@ Fixes #[issue_number]
<!--
Any additional notes a reviewer should know before we review.
# This action gets all changed markdown files in /docs and /versioned_docs using tj-actions/changed-files@v42
# It compares new commits in the PR with the base commit (github.event.pull_request.base.sha)
# It checks if no markdown files are changed
# It shows a count of markdown files and lists all changed markdown files
# It uses Vale (https://vale.sh/docs/vale-cli/installation/) to provide feedback base off the SUSE Style Guide / OpenSUSE style rules (https://github.com/openSUSE/suse-vale-styleguide)
name:Style check
on:[pull_request]
jobs:
vale-lint:
name:runner / vale
runs-on:ubuntu-latest
steps:
- uses:actions/checkout@v4
continue-on-error:true
with:
fetch-depth:0# OR "2" -> To retrieve the preceding commit.
submodules:true
- name:Get all changed markdown files
continue-on-error:true
id:changed-markdown-files
uses:tj-actions/changed-files@v42
with:
# Avoid using single or double quotes for multiline patterns
@@ -15,9 +15,9 @@ To get started, [fork](https://github.com/rancher/rancher-docs/fork) and clone t
Our repository doesn't allow you to make changes directly to the `main` branch. Create a working branch and make pull requests from your fork to [rancher/rancher-docs](https://github.com/rancher/rancher-docs).
For most updates, you'll need to edit a file in the `/docs` directory, which represents the ["Latest"](https://ranchermanager.docs.rancher.com/) version of our published documentation. The "Latest" version is a mirror of the most recently released version of Rancher. As of December 2023, the most recently released version of Rancher is 2.8.
For most updates, you'll need to edit a file in the `/docs` directory, which represents the ["Latest"](https://ranchermanager.docs.rancher.com/) version of our published documentation. The "Latest" version is a mirror of the most recently released version of Rancher. As of August 2024, the most recently released version of Rancher is 2.9.
Whenever an update is made to `/docs`, you should apply the same change to the corresponding file in `/versioned_docs/version-2.8`. If a change only affects older versions, you don't need to mirror it to the `/docs` directory.
Whenever an update is made to `/docs`, you should apply the same change to the corresponding file in `/versioned_docs/version-2.9`. If a change only affects older versions, you don't need to mirror it to the `/docs` directory.
If a file is moved or renamed, you'll also need to edit the `sidebars.js` files for each affected version, as well as the list of redirects in `docusaurus.config.js`. See [Moving or Renaming Docs](./moving-or-renaming-docs.md).
@@ -93,7 +93,7 @@ Subsequent executions will check for updated dependencies, if there are none, it
> Monitoring V1 requires a Kubernetes verison less than or equal to v1.20.x. To install monitoring on Kubernetes v1.21+, you will need to [migrate to Monitoring V2.](/versioned_docs/version-2.5/how-to-guides/advanced-user-guides/monitoring-alerting-guides/migrate-to-rancher-v2.5%2B-monitoring.md)
> Monitoring V1 requires a Kubernetes verison less than or equal to v1.20.x. To install monitoring on Kubernetes v1.21+, you will need to [migrate to Monitoring V2.](https://github.com/rancher/rancher-docs/tree/main/archived_docs/en/version-2.5/how-to-guides/advanced-user-guides/monitoring-alerting-guides/migrate-to-rancher-v2.5%2B-monitoring.md)
1. From the **Global** view, navigate to the cluster that you want to configure cluster monitoring.
While configuring monitoring at either the [cluster level](cluster-monitoring.md) or [project level](./project-monitoring.md), there are multiple options that can be configured.
- [Basic Configuration](#basic-configuration)
- [Advanced Options](#advanced-options)
- [Node Exporter](#node-exporter)
- [Persistent Storage](#persistent-storage)
- [Remote Storage](#remote-storage)
## Basic Configuration
Option | Description
-------|-------------
Data Retention | How long your Prometheus instance retains monitoring data scraped from Rancher objects before it's purged.
[Enable Node Exporter](#node-exporter) | Whether or not to deploy the node exporter.
Node Exporter Host Port | The host port on which data is exposed, i.e. data that Prometheus collects from your node hardware. Required if you have enabled the node exporter.
[Enable Persistent Storage](#persistent-storage) for Prometheus | Whether or not to configure storage for Prometheus so that metrics can be retained even if the Prometheus pod fails.
[Enable Persistent Storage](#persistent-storage) for Grafana | Whether or not to configure storage for Grafana so that the Grafana dashboards and configuration can be retained even if the Grafana pod fails.
Prometheus [CPU Limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu) | CPU resource limit for the Prometheus pod.
Prometheus [CPU Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu) | CPU reservation for the Prometheus pod.
Prometheus [Memory Limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource limit for the Prometheus pod.
Prometheus [Memory Reservation](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory) | Memory resource requests for the Prometheus pod.
Selector | Ability to select the nodes in which Prometheus and Grafana pods are deployed to. To use this option, the nodes must have labels.
## Advanced Options
Since monitoring is an [application](https://github.com/rancher/system-charts/tree/dev/charts/rancher-monitoring) from the [Rancher catalog](../../../how-to-guides/new-user-guides/helm-charts-in-rancher/helm-charts-in-rancher.md), it can be configured like any other catalog application, by passing in values to Helm.
> **Warning:** Any modification to the application without understanding the entire application can lead to catastrophic errors.
### Prometheus RemoteRead and RemoteWrite
_Available as of v2.4.0_
Prometheus RemoteRead and RemoteWrite can be configured as custom answers in the **Advanced Options** section.
For more information on remote endpoints and storage, refer to the [Prometheus documentation.](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage)
The Prometheus operator documentation contains the full [RemoteReadSpec](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#remotereadspec) and [RemoteWriteSpec.](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api-reference/api.md#remotewritespec)
The [node exporter](https://github.com/prometheus/node_exporter/blob/master/README.md) is a popular open source exporter, which exposes the metrics for hardware and \*NIX kernels OS. It is designed to monitor the host system. However, there are still issues with namespaces when running it in a container, mostly around filesystem mount spaces. In order to monitor actual network metrics for the container network, the node exporter must be deployed with the `hostNetwork` mode.
When configuring Prometheus and enabling the node exporter, enter a host port in the **Node Exporter Host Port** that will not produce port conflicts with existing applications. The host port chosen must be open to allow internal traffic between Prometheus and the Node Exporter.
>**Warning:** In order for Prometheus to collect the metrics of the node exporter, after enabling cluster monitoring, you must open the <b>Node Exporter Host Port</b> in the host firewall rules to allow intranet access. By default, `9796` is used as that host port.
## Persistent Storage
>**Prerequisite:** Configure one or more StorageClasses to use as [persistent storage](../../../how-to-guides/advanced-user-guides/manage-clusters/create-kubernetes-persistent-storage/create-kubernetes-persistent-storage.md) for your Prometheus or Grafana pod.
By default, when you enable Prometheus for either a cluster or project, all monitoring data that Prometheus collects is stored on its own pod. With local storage, if the Prometheus or Grafana pods fail, all the data is lost. Rancher recommends configuring an external persistent storage to the cluster. With the external persistent storage, if the Prometheus or Grafana pods fail, the new pods can recover using data from the persistent storage.
When enabling persistent storage for Prometheus or Grafana, specify the size of the persistent volume and select the StorageClass.
## Remote Storage
>**Prerequisite:** Need a remote storage endpoint to be available. The possible list of integrations is available [here](https://prometheus.io/docs/operating/integrations/)
Using advanced options, remote storage integration for the Prometheus installation can be configured as follows:
Additional fields can be set up based on the [ReadSpec](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotereadspec) and [RemoteWriteSpec](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#remotewritespec)
description:Learn about Container Network Interface (CNI), the CNI providers Rancher provides, the features they offer, and how to choose a provider for you
CNI (Container Network Interface), a [Cloud Native Computing Foundation project](https://cncf.io/), consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of plugins. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.
Kubernetes uses CNI as an interface between network providers and Kubernetes pod networking.

For more information visit [CNI GitHub project](https://github.com/containernetworking/cni).
### What Network Models are Used in CNI?
CNI network providers implement their network fabric using either an encapsulated network model such as Virtual Extensible Lan ([VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan)) or an unencapsulated network model such as Border Gateway Protocol ([BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol)).
#### What is an Encapsulated Network?
This network model provides a logical Layer 2 (L2) network encapsulated over the existing Layer 3 (L3) network topology that spans the Kubernetes cluster nodes. With this model you have an isolated L2 network for containers without needing routing distribution, all at the cost of minimal overhead in terms of processing and increased IP package size, which comes from an IP header generated by overlay encapsulation. Encapsulation information is distributed by UDP ports between Kubernetes workers, interchanging network control plane information about how MAC addresses can be reached. Common encapsulation used in this kind of network model is VXLAN, Internet Protocol Security (IPSec), and IP-in-IP.
In simple terms, this network model generates a kind of network bridge extended between Kubernetes workers, where pods are connected.
This network model is used when an extended L2 bridge is preferred. This network model is sensitive to L3 network latencies of the Kubernetes workers. If datacenters are in distinct geolocations, be sure to have low latencies between them to avoid eventual network segmentation.
CNI network providers using this network model include Flannel, Canal, and Weave.
This network model provides an L3 network to route packets between containers. This model doesn't generate an isolated l2 network, nor generates overhead. These benefits come at the cost of Kubernetes workers having to manage any route distribution that's needed. Instead of using IP headers for encapsulation, this network model uses a network protocol between Kubernetes workers to distribute routing information to reach pods, such as [BGP](https://en.wikipedia.org/wiki/Border_Gateway_Protocol).
In simple terms, this network model generates a kind of network router extended between Kubernetes workers, which provides information about how to reach pods.
This network model is used when a routed L3 network is preferred. This mode dynamically updates routes at the OS level for Kubernetes workers. It's less sensitive to latency.
CNI network providers using this network model include Calico and Romana.
Out-of-the-box, Rancher provides the following CNI network providers for Kubernetes clusters: Canal, Flannel, Calico and Weave (Weave is available as of v2.2.0). You can choose your CNI network provider when you create new Kubernetes clusters from Rancher.
#### Canal

Canal is a CNI network provider that gives you the best of Flannel and Calico. It allows users to easily deploy Calico and Flannel networking together as a unified networking solution, combining Calico’s network policy enforcement with the rich superset of Calico (unencapsulated) and/or Flannel (encapsulated) network connectivity options.
In Rancher, Canal is the default CNI network provider combined with Flannel and VXLAN encapsulation.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). For details, refer to [the port requirements for user clusters.](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md)

For more information, see the [Canal GitHub Page.](https://github.com/projectcalico/canal)
#### Flannel

Flannel is a simple and easy way to configure L3 network fabric designed for Kubernetes. Flannel runs a single binary agent named flanneld on each host, which is responsible for allocating a subnet lease to each host out of a larger, preconfigured address space. Flannel uses either the Kubernetes API or etcd directly to store the network configuration, the allocated subnets, and any auxiliary data (such as the host's public IP). Packets are forwarded using one of several backend mechanisms, with the default encapsulation being [VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan).
Encapsulated traffic is unencrypted by default. Therefore, flannel provides an experimental backend for encryption, [IPSec](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#ipsec), which makes use of [strongSwan](https://www.strongswan.org/) to establish encrypted IPSec tunnels between Kubernetes workers.
Kubernetes workers should open UDP port `8472` (VXLAN) and TCP port `9099` (healthcheck). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.

For more information, see the [Flannel GitHub Page](https://github.com/coreos/flannel).
#### Calico

Calico enables networking and network policy in Kubernetes clusters across the cloud. Calico uses a pure, unencapsulated IP network fabric and policy engine to provide networking for your Kubernetes workloads. Workloads are able to communicate over both cloud infrastructure and on-prem using BGP.
Calico also provides a stateless IP-in-IP encapsulation mode that can be used, if necessary. Calico also offers policy isolation, allowing you to secure and govern your Kubernetes workloads using advanced ingress and egress policies.
Kubernetes workers should open TCP port `179` (BGP). See [the port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.

For more information, see the following pages:
- [Project Calico Official Site](https://www.projectcalico.org/)
Weave enables networking and network policy in Kubernetes clusters across the cloud. Additionally, it support encrypting traffic between the peers.
Kubernetes workers should open TCP port `6783` (control port), UDP port `6783` and UDP port `6784` (data ports). See the [port requirements for user clusters](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/node-requirements-for-rancher-managed-clusters.md#networking-requirements) for more details.
For more information, see the following pages:
- [Weave Net Official Site](https://github.com/weaveworks/weave/blob/master/site/overview.md)
### CNI Features by Provider
The following table summarizes the different features available for each CNI network provider provided by Rancher.
- Network Model: Encapsulated or unencapsulated. For more information, see [What Network Models are Used in CNI?](#what-network-models-are-used-in-cni)
- Route Distribution: An exterior gateway protocol designed to exchange routing and reachability information on the Internet. BGP can assist with pod-to-pod networking between clusters. This feature is a must on unencapsulated CNI network providers, and it is typically done by BGP. If you plan to build clusters split across network segments, route distribution is a feature that's nice-to-have.
- Network Policies: Kubernetes offers functionality to enforce rules about which services can communicate with each other using network policies. This feature is stable as of Kubernetes v1.7 and is ready to use with certain networking plugins.
- Mesh: This feature allows service-to-service networking communication between distinct Kubernetes clusters.
- External Datastore: CNI network providers with this feature need an external datastore for its data.
- Encryption: This feature allows cyphered and secure network control and data planes.
- Ingress/Egress Policies: This feature allows you to manage routing control for both Kubernetes and non-Kubernetes communications.
### CNI Community Popularity
<CNIPopularityTable />
### Which CNI Provider Should I Use?
It depends on your project needs. There are many different providers, which each have various features and options. There isn't one provider that meets everyone's needs.
As of Rancher v2.0.7, Canal is the default CNI network provider. We recommend it for most use cases. It provides encapsulated networking for containers with Flannel, while adding Calico network policies that can provide project/namespace isolation in terms of networking.
### How can I configure a CNI network provider?
Please see [Cluster Options](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md) on how to configure a network provider for your cluster. For more advanced configuration options, please see how to configure your cluster using a [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#cluster-config-file) and the options for [Network Plug-ins](https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/).
`kubectl` is a CLI utility for running commands against Kubernetes clusters. It's required for many maintenance and administrative tasks in Rancher 2.x.
## Installation
See [kubectl Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation on your operating system.
## Configuration
When you create a Kubernetes cluster with RKE, RKE creates a `kube_config_rancher-cluster.yml` in the local directory that contains credentials to connect to your new cluster with tools like `kubectl` or `helm`.
You can copy this file to `$HOME/.kube/config` or if you are working with multiple Kubernetes clusters, set the `KUBECONFIG` environmental variable to the path of `kube_config_rancher-cluster.yml`.
This page is intended to answer questions about what happens if you don't want Rancher anymore, if you don't want a cluster to be managed by Rancher anymore, or if the Rancher server is deleted.
## If the Rancher server is deleted, what happens to the workloads in my downstream clusters?
If Rancher is ever deleted or unrecoverable, all workloads in the downstream Kubernetes clusters managed by Rancher will continue to function as normal.
## If the Rancher server is deleted, how do I access my downstream clusters?
The capability to access a downstream cluster without Rancher depends on the type of cluster and the way that the cluster was created. To summarize:
- **Imported clusters:** The cluster will be unaffected and you can access the cluster using the same methods that you did before the cluster was imported into Rancher.
- **Hosted Kubernetes clusters:** If you created the cluster in a cloud-hosted Kubernetes provider such as EKS, GKE, or AKS, you can continue to manage the cluster using your provider's cloud credentials.
- **RKE clusters:** To access an [RKE cluster,](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/launch-kubernetes-with-rancher.md) the cluster must have the [authorized cluster endpoint](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#4-authorized-cluster-endpoint) enabled, and you must have already downloaded the cluster's kubeconfig file from the Rancher UI. (The authorized cluster endpoint is enabled by default for RKE clusters.) With this endpoint, you can access your cluster with kubectl directly instead of communicating through the Rancher server's [authentication proxy.](../reference-guides/rancher-manager-architecture/communicating-with-downstream-user-clusters.md#1-the-authentication-proxy) For instructions on how to configure kubectl to use the authorized cluster endpoint, refer to the section about directly accessing clusters with [kubectl and the kubeconfig file.](../how-to-guides/advanced-user-guides/manage-clusters/access-clusters/use-kubectl-and-kubeconfig.md#authenticating-directly-with-a-downstream-cluster) These clusters will use a snapshot of the authentication as it was configured when Rancher was removed.
## What if I don't want Rancher anymore?
If you [installed Rancher on a Kubernetes cluster,](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/install-upgrade-on-a-kubernetes-cluster.md) remove Rancher by using the [System Tools](../reference-guides/system-tools.md) with the `remove` subcommand.
If you installed Rancher with Docker, you can uninstall Rancher by removing the single Docker container that it runs in.
Imported clusters will not be affected by Rancher being removed. For other types of clusters, refer to the section on [accessing downstream clusters when Rancher is removed.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
## What if I don't want my imported cluster managed by Rancher?
If an imported cluster is deleted from the Rancher UI, the cluster is detached from Rancher, leaving it intact and accessible by the same methods that were used to access it before it was imported into Rancher.
To detach the cluster,
1. From the **Global** view in Rancher, go to the **Clusters** tab.
2. Go to the imported cluster that should be detached from Rancher and click **⋮ > Delete.**
3. Click **Delete.**
**Result:** The imported cluster is detached from Rancher and functions normally outside of Rancher.
## What if I don't want my RKE cluster or hosted Kubernetes cluster managed by Rancher?
At this time, there is no functionality to detach these clusters from Rancher. In this context, "detach" is defined as the ability to remove Rancher components from the cluster and manage access to the cluster independently of Rancher.
The capability to manage these clusters without Rancher is being tracked in this [issue.](https://github.com/rancher/rancher/issues/25234)
For information about how to access clusters if the Rancher server is deleted, refer to [this section.](#if-the-rancher-server-is-deleted-how-do-i-access-my-downstream-clusters)
The Hardening Guide is now located in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
## What are the results of Rancher's Kubernetes cluster when it is CIS benchmarked?
We have run the CIS Kubernetes benchmark against a hardened Rancher Kubernetes cluster. The results of that assessment can be found in the main [Security](../reference-guides/rancher-security/rancher-security.md) section.
New password for default administrator (user-xxxxx):
<new_password>
```
### RKE add-on install (only supported up to Rancher v2.0.8)
:::tip
> If you are currently using the RKE add-on install method, see [Migrating from a Kubernetes Install with an RKE Add-on](../getting-started/installation-and-upgrade/install-upgrade-on-a-kubernetes-cluster/upgrades/migrating-from-rke-add-on.md) for details on how to start using the Helm chart.
New password for default administrator (user-xxxxx):
<new_password>
```
## How can I enable debug logging?
See [Troubleshooting: Logging](../troubleshooting/other-troubleshooting-tips/logging.md)
## My ClusterIP does not respond to ping
ClusterIP is a virtual IP, which will not respond to ping. Best way to test if the ClusterIP is configured correctly, is by using `curl` to access the IP and port to see if it responds.
## Where can I manage Node Templates?
Node Templates can be accessed by opening your account menu (top right) and selecting `Node Templates`.
## Why is my Layer-4 Load Balancer in `Pending` state?
The Layer-4 Load Balancer is created as `type: LoadBalancer`. In Kubernetes, this needs a cloud provider or controller that can satisfy these requests, otherwise these will be in `Pending` state forever. More information can be found on [Cloud Providers](../how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/set-up-cloud-providers/set-up-cloud-providers.md) or [Create External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)
## Where is the state of Rancher stored?
- Docker Install: in the embedded etcd of the `rancher/rancher` container, located at `/var/lib/rancher`.
- Kubernetes install: in the etcd of the RKE cluster created to run Rancher.
## How are the supported Docker versions determined?
We follow the validated Docker versions for upstream Kubernetes releases. The validated versions can be found under [External Dependencies](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#external-dependencies) in the Kubernetes release CHANGELOG.md.
## How can I access nodes created by Rancher?
SSH keys to access the nodes created by Rancher can be downloaded via the **Nodes** view. Choose the node which you want to access and click on the vertical ⋮ button at the end of the row, and choose **Download Keys** as shown in the picture below.

Unzip the downloaded zip file, and use the file `id_rsa` to connect to you host. Be sure to use the correct username (`rancher` or `docker` for RancherOS, `ubuntu` for Ubuntu, `ec2-user` for Amazon Linux)
```
$ ssh -i id_rsa user@ip_of_node
```
## How can I automate task X in Rancher?
The UI consists of static files, and works based on responses of the API. That means every action/task that you can execute in the UI, can be automated via the API. There are 2 ways to do this:
* Visit `https://your_rancher_ip/v3` and browse the API options.
* Capture the API calls when using the UI (Most commonly used for this is [Chrome Developer Tools](https://developers.google.com/web/tools/chrome-devtools/#network) but you can use anything you like)
## The IP address of a node changed, how can I recover?
A node is required to have a static IP configured (or a reserved IP via DHCP). If the IP of a node has changed, you will have to remove it from the cluster and readd it. After it is removed, Rancher will update the cluster to the correct state. If the cluster is no longer in `Provisioning` state, the node is removed from the cluster.
When the IP address of the node changed, Rancher lost connection to the node, so it will be unable to clean the node properly. See [Cleaning cluster nodes](../how-to-guides/advanced-user-guides/manage-clusters/clean-cluster-nodes.md) to clean the node.
When the node is removed from the cluster, and the node is cleaned, you can readd the node to the cluster.
## How can I add more arguments/binds/environment variables to Kubernetes components in a Rancher Launched Kubernetes cluster?
You can add more arguments/binds/environment variables via the [Config File](../reference-guides/cluster-configuration/rancher-server-configuration/rke1-cluster-configuration.md#cluster-config-file) option in Cluster Options. For more information, see the [Extra Args, Extra Binds, and Extra Environment Variables](https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/) in the RKE documentation or browse the [Example Cluster.ymls](https://rancher.com/docs/rke/latest/en/example-yamls/).
## How do I check if my certificate chain is valid?
Use the `openssl verify` command to validate your certificate chain:
:::tip
Configure `SSL_CERT_DIR` and `SSL_CERT_FILE` to a dummy location to make sure the OS installed certificates are not used when verifying manually.
If you receive the error `unable to get local issuer certificate`, the chain is incomplete. This usually means that there is an intermediate CA certificate that issued your server certificate. If you already have this certificate, you can use it in the verification of the certificate like shown below:
If you have successfully verified your certificate chain, you should include needed intermediate CA certificates in the server certificate to complete the certificate chain for any connection made to Rancher (for example, by the Rancher agent). The order of the certificates in the server certificate file should be first the server certificate itself (contents of `rancher.yourdomain.com.pem`), followed by intermediate CA certificate(s) (contents of `intermediate.pem`).
```
-----BEGIN CERTIFICATE-----
%YOUR_CERTIFICATE%
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
%YOUR_INTERMEDIATE_CERTIFICATE%
-----END CERTIFICATE-----
```
If you still get errors during verification, you can retrieve the subject and the issuer of the server certificate using the following command:
issuer= /C=GB/ST=England/O=Alice Ltd/CN=Alice Intermediate CA
```
## How do I check `Common Name` and `Subject Alternative Names` in my server certificate?
Although technically an entry in `Subject Alternative Names` is required, having the hostname in both `Common Name` and as entry in `Subject Alternative Names` gives you maximum compatibility with older browser/applications.
Check `Common Name`:
```
openssl x509 -noout -subject -in cert.pem
subject= /CN=rancher.my.org
```
Check `Subject Alternative Names`:
```
openssl x509 -noout -in cert.pem -text | grep DNS
DNS:rancher.my.org
```
## Why does it take 5+ minutes for a pod to be rescheduled when a node has failed?
This is due to a combination of the following default Kubernetes settings:
* kubelet
*`node-status-update-frequency`: Specifies how often kubelet posts node status to master (default 10s)
* kube-controller-manager
*`node-monitor-period`: The period for syncing NodeStatus in NodeController (default 5s)
*`node-monitor-grace-period`: Amount of time which we allow running Node to be unresponsive before marking it unhealthy (default 40s)
*`pod-eviction-timeout`: The grace period for deleting pods on failed nodes (default 5m0s)
See [Kubernetes: kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) and [Kubernetes: kube-controller-manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) for more information on these settings.
In Kubernetes v1.13, the `TaintBasedEvictions` feature is enabled by default. See [Kubernetes: Taint based Evictions](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions) for more information.
* kube-apiserver (Kubernetes v1.13 and up)
*`default-not-ready-toleration-seconds`: Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.
*`default-unreachable-toleration-seconds`: Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.
## Can I use keyboard shortcuts in the UI?
Yes, most parts of the UI can be reached using keyboard shortcuts. For an overview of the available shortcuts, press `?` anywhere in the UI.
Telemetry collects aggregate information about the size of Rancher installations, versions of components used, and which features are used. This information is used by Rancher Labs to help make the product better and is not shared with third-parties.
## What information is collected?
No specific identifying information like usernames, passwords, or the names or addresses of user resources will ever be collected.
The primary things collected include:
- Aggregate counts (smallest, average, largest, total) of nodes per-cluster and their size (e.g. CPU cores & RAM).
- Aggregate counts of logical resources like Clusters, Projects, Namespaces, and Pods.
- Counts of what driver was used to deploy clusters and nodes (e.g. GKE vs EC2 vs Imported vs Custom).
- Versions of Kubernetes components, Operating Systems and Docker that are deployed on nodes.
- Whether some optional components are enabled or not (e.g. which auth providers are used).
- The image name & version of Rancher that is running.
- A unique randomly-generated identifier for this installation.
## Can I see the information that is being sent?
If Telemetry is enabled, you can go to `https://<your rancher server>/v1-telemetry` in your installation to see the current data.
If Telemetry is not enabled, the process that collects the data is not running, so there is nothing being collected to look at.
## How do I turn it on or off?
After initial setup, an administrator can go to the `Settings` page in the `Global` section of the UI and click Edit to change the `telemetry-opt` setting to either `in` or `out`.
This section is about how to deploy Rancher for your air gapped environment. An air gapped environment could be where Rancher server will be installed offline, behind a firewall, or behind a proxy. There are _tabs_ for either a high availability (recommended) or a Docker installation.
Rancher recommends installing Rancher on a Kubernetes cluster. A highly available Kubernetes Installation is comprised of three nodes running the Rancher server components on a Kubernetes cluster. The persistence layer (etcd) is also replicated on these three nodes, providing redundancy and data duplication in case one of the nodes fails.
### A. Add the Helm Chart Repository
From a system that has access to the internet, fetch the latest Helm chart and copy the resulting manifests to a system that has access to the Rancher server cluster.
1. If you haven't already, initialize Helm locally on a workstation that has internet access.
:::note
Refer to the [Helm version requirements](../../../resources/choose-a-rancher-version.md) to choose a version of Helm to install Rancher.
:::
```plain
helm init -c
```
2. Use the `helm repo add` command to add the Helm chart repository that contains charts to install Rancher. For more information about the repository choices and which is best for your use case, see [Choosing a Version of Rancher](../../../resources/choose-a-rancher-version.md).
- Latest: Recommended for trying out the newest features
Note: Upgrades are not supported to, from, or between Alphas.
3. Fetch the latest Rancher chart. This will pull down the chart and save it in the current directory as a `.tgz` file.
```plain
helm fetch rancher-<CHART_REPO>/rancher
```
> Want additional options? See the Rancher [Helm chart options](../../../../../reference-guides/installation-references/helm-chart-options.md).
### B. Choose your SSL Configuration
Rancher Server is designed to be secure by default and requires SSL/TLS configuration.
When Rancher is installed on an air gapped Kubernetes cluster, there are two recommended options for the source of the certificate.
> **Note:** If you want terminate SSL/TLS externally, see [TLS termination on an External Load Balancer](../../../../../reference-guides/installation-references/helm-chart-options.md#external-tls-termination).
| Rancher Generated Self-Signed Certificates | `ingress.tls.source=rancher` | Use certificates issued by Rancher's generated CA (self signed)<br/> This is the **default** and does not need to be added when rendering the Helm template. | yes |
| Certificates from Files | `ingress.tls.source=secret` | Use your own certificate files by creating Kubernetes Secret(s). <br/> This option must be passed when rendering the Rancher Helm template. | no |
### C. Render the Rancher Helm Template
When setting up the Rancher Helm template, there are several options in the Helm chart that are designed specifically for air gap installations.
| `systemDefaultRegistry` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `useBundledSystemChart` | `true` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. _Available as of v2.3.0_ |
Based on the choice your made in [B. Choose your SSL Configuration](#b-choose-your-ssl-configuration), complete one of the procedures below.
By default, Rancher generates a CA and uses cert-manager to issue the certificate for access to the Rancher server interface.
> **Note:**
> Recent changes to cert-manager require an upgrade. If you are upgrading Rancher and using a version of cert-manager older than v0.11.0, please see our [upgrade cert-manager documentation](../../../resources/upgrade-cert-manager.md).
1. From a system connected to the internet, add the cert-manager repo to Helm.
```plain
helm repo add jetstack https://charts.jetstack.io
helm repo update
```
1. Fetch the latest cert-manager chart available from the [Helm chart repository](https://hub.helm.sh/charts/jetstack/cert-manager).
1. Render the cert manager template with the options you would like to use to install the chart. Remember to set the `image.repository` option to pull the image from your private registry. This will create a `cert-manager` directory with the Kubernetes manifest files.
1. Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
Placeholder | Description
------------|-------------
`<VERSION>` | The version number of the output tarball.
`<RANCHER.YOURDOMAIN.COM>` | The DNS name you pointed at your load balancer.
`<REGISTRY.YOURDOMAIN.COM:PORT>` | The DNS name for your private registry.
`<CERTMANAGER_VERSION>` | Cert-manager version running on k8s cluster.
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts
```
</details>
<details id="secret">
<summary>Option B: Certificates From Files using Kubernetes Secrets</summary>
Create Kubernetes secrets from your own certificates for Rancher to use. The common name for the cert will need to match the `hostname` option in the command below, or the ingress controller will fail to provision the site for Rancher.
Render the Rancher template, declaring your chosen options. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.
--set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
--set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts
```
Then refer to [Adding TLS Secrets](../../../resources/add-tls-secrets.md) to publish the certificate files so Rancher and the ingress controller can use them.
</details>
### D. Install Rancher
Copy the rendered manifest directories to a system that has access to the Rancher server cluster to complete installation.
Use `kubectl` to create namespaces and apply the rendered manifests.
If you choose to use self-signed certificates in [B. Choose your SSL Configuration](#b-choose-your-ssl-configuration), install cert-manager.
> If you are running Kubernetes v1.15 or below, you will need to add the `--validate=false flag to your kubectl apply command above else you will receive a validation error relating to the x-kubernetes-preserve-unknown-fields field in cert-manager’s CustomResourceDefinition resources. This is a benign error and occurs due to the way kubectl performs resource validation.
1. Launch cert-manager.
```plain
kubectl apply -R -f ./cert-manager
```
</details>
Install Rancher:
```plain
kubectl create namespace cattle-system
kubectl -n cattle-system apply -R -f ./rancher
```
**Step Result:** If you are installing Rancher v2.3.0+, the installation is complete.
### E. For Rancher versions before v2.3.0, Configure System Charts
If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in GitHub, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../../resources/local-system-charts.md).
### Additional Resources
These resources could be helpful when installing Rancher:
The Docker installation is for Rancher users that are wanting to **test** out Rancher. Instead of running on a Kubernetes cluster, you install the Rancher server component on a single node using a `docker run` command. Since there is only one node and a single Docker container, if the node goes down, there is no copy of the etcd data available on other nodes and you will lose all the data of your Rancher server. **Important: If you install Rancher following the Docker installation guide, there is no upgrade path to transition your Docker installation to a Kubernetes Installation.** Instead of running the single node installation, you have the option to follow the Kubernetes Install guide, but only use one node to install Rancher. Afterwards, you can scale up the etcd nodes in your Kubernetes cluster to make it a Kubernetes Installation.
For security purposes, SSL (Secure Sockets Layer) is required when using Rancher. SSL secures all Rancher network communication, like when you login or interact with a cluster.
| `CATTLE_SYSTEM_DEFAULT_REGISTRY` | `<REGISTRY.YOURDOMAIN.COM:PORT>` | Configure Rancher server to always pull from your private registry when provisioning clusters. |
| `CATTLE_SYSTEM_CATALOG` | `bundled` | Configure Rancher server to use the packaged copy of Helm system charts. The [system charts](https://github.com/rancher/system-charts) repository contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. These [Helm charts](https://github.com/rancher/system-charts) are located in GitHub, but since you are in an air gapped environment, using the charts that are bundled within Rancher is much easier than setting up a Git mirror. _Available as of v2.3.0_ |
> **Do you want to...**
>
> - Configure custom CA root certificate to access your services? See [Custom CA root certificate](../../../../../reference-guides/installation-references/helm-chart-options.md#additional-trusted-cas).
> - Record all transactions with the Rancher API? See [API Auditing](../../../../../reference-guides/single-node-rancher-in-docker/advanced-options.md#api-audit-log).
- For Rancher before v2.3.0, you will need to mirror the `system-charts` repository to a location in your network that Rancher can reach. Then, after Rancher is installed, you will need to configure Rancher to use that repository. For details, refer to the documentation on [setting up the system charts for Rancher before v2.3.0.](../../../resources/local-system-charts.md)
If you are installing Rancher in a development or testing environment where identity verification isn't a concern, install Rancher using the self-signed certificate that it generates. This installation option omits the hassle of generating a certificate yourself.
Log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder.
<summary>Option B-Bring Your Own Certificate: Self-Signed</summary>
In development or testing environments where your team will access your Rancher server, create a self-signed certificate for use with your install so that your team can verify they're connecting to your instance of Rancher.
> **Prerequisites:**
> From a computer with an internet connection, create a self-signed certificate using [OpenSSL](https://www.openssl.org/) or another method of your choice.
>
> - The certificate files must be in PEM format.
> - In your certificate file, include all intermediate certificates in the chain. Order your certificates with your certificate first, followed by the intermediates. For an example, see [Certificate Troubleshooting.](../../../other-installation-methods/rancher-on-a-single-node-with-docker/certificate-troubleshooting.md)
After creating your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Use the `-v` flag and provide the path to your certificates to mount them in your container.
<summary>Option C-Bring Your Own Certificate: Signed by Recognized CA</summary>
In development or testing environments where you're exposing an app publicly, use a certificate signed by a recognized CA so that your user base doesn't encounter security warnings.
> **Prerequisite:** The certificate files must be in PEM format.
After obtaining your certificate, log into your Linux host, and then run the installation command below. When entering the command, use the table below to replace each placeholder. Because your certificate is signed by a recognized CA, mounting an additional CA certificate file is unnecessary.
If you are installing Rancher v2.3.0+, the installation is complete.
If you are installing Rancher versions before v2.3.0, you will not be able to use the packaged system charts. Since the Rancher system charts are hosted in GitHub, an air gapped installation will not be able to access these charts. Therefore, you must [configure the Rancher system charts](../../../resources/local-system-charts.md).
title:Enabling the API Audit Log to Record System Events
---
You can enable the API audit log to record the sequence of system events initiated by individual users. You can know what happened, when it happened, who initiated it, and what cluster it affected. When you enable this feature, all requests to the Rancher API and all responses from it are written to a log.
You can enable API Auditing during Rancher installation or upgrade.
## Enabling API Audit Log
The Audit Log is enabled and configured by passing environment variables to the Rancher server container. See the following to enable on your installation.
| `AUDIT_LEVEL` | `0` - Disable audit log (default setting).<br/>`1` - Log event metadata.<br/>`2` - Log event metadata and request body.<br/>`3` - Log event metadata, request body, and response body. Each log transaction for a request/response pair uses the same `auditID` value.<br/><br/>See [Audit Level Logging](#audit-log-levels) for a table that displays what each setting logs. |
| `AUDIT_LOG_PATH` | Log path for Rancher Server API. Default path is `/var/log/auditlog/rancher-api-audit.log`. You can mount the log directory to host. <br/><br/>Usage Example: `AUDIT_LOG_PATH=/my/custom/path/`<br/> |
| `AUDIT_LOG_MAXAGE` | Defined the maximum number of days to retain old audit log files. Default is 10 days. |
| `AUDIT_LOG_MAXBACKUP` | Defines the maximum number of audit log files to retain. Default is 10. |
| `AUDIT_LOG_MAXSIZE` | Defines the maximum size in megabytes of the audit log file before it gets rotated. Default size is 100M. |
<br/>
### Audit Log Levels
The following table displays what parts of API transactions are logged for each [`AUDIT_LEVEL`](#api-audit-log-options) setting.
| `AUDIT_LEVEL` Setting | Metadata | Request Body | Response Body |
Share the `AUDIT_LOG_PATH` directory (Default: `/var/log/auditlog`) with the host system. The log can be parsed by standard CLI tools or forwarded on to a log collection tool like Fluentd, Filebeat, Logstash, etc.
### Kubernetes Install
Enabling the API Audit Log with the Helm chart install will create a `rancher-audit-log` sidecar container in the Rancher pod. This container will stream the log to standard output (stdout). You can view the log as you would any container log.
The `rancher-audit-log` container is part of the `rancher` pod in the `cattle-system` namespace.
1. From the context menu, select **Cluster: local > System**.
1. From the main navigation bar, choose **Resources > Workloads.** (In versions before v2.3.0, choose **Workloads** on the main navigation bar.) Find the `cattle-system` namespace. Open the `rancher` workload by clicking its link.
1. Pick one of the `rancher` pods and select **⋮ > View Logs**.
1. From the **Logs** drop-down, select `rancher-audit-log`.
#### Shipping the Audit Log
You can enable Rancher's built in log collection and shipping for the cluster to ship the audit and other services logs to a supported collection endpoint. See [Rancher Tools - Logging](../../../../explanations/integrations-in-rancher/cluster-logging/cluster-logging.md) for details.
## Audit Log Samples
After you enable auditing, each API request or response is logged by Rancher in the form of JSON. Each of the following code samples provide examples of how to identify each API transaction.
### Metadata Level
If you set your `AUDIT_LEVEL` to `1`, Rancher logs the metadata header for every API request, but not the body. The header provides basic information about the API transaction, such as the transaction's ID, who initiated the transaction, the time it occurred, etc.
```json
{
"auditID":"30022177-9e2e-43d1-b0d0-06ef9d3db183",
"requestURI":"/v3/schemas",
"sourceIPs":["::1"],
"user":{
"name":"user-f4tt2",
"group":["system:authenticated"]
},
"verb":"GET",
"stage":"RequestReceived",
"stageTimestamp":"2018-07-20 10:22:43 +0800"
}
```
### Metadata and Request Body Level
If you set your `AUDIT_LEVEL` to `2`, Rancher logs the metadata header and body for every API request.
The code sample below depicts an API request, with both its metadata header and body.
@@ -155,15 +155,32 @@ You may terminate the SSL/TLS on a L7 load balancer external to the Rancher clus
Your load balancer must support long lived websocket connections and will need to insert proxy headers so Rancher can route links correctly.
#### Configuring Ingress for External TLS when Using NGINX v0.25
### Configuring Ingress for External TLS when Using NGINX v0.22
In NGINX v0.25, the behavior of NGINX has [changed](https://github.com/kubernetes/ingress-nginx/blob/main/Changelog.md#0220) regarding forwarding headers and external TLS termination. Therefore, in the scenario that you are using external TLS termination configuration with NGINX v0.25, you must edit the `cluster.yml` to enable the `use-forwarded-headers` option for ingress:
In NGINX v0.22, the behavior of NGINX has [changed](https://github.com/kubernetes/ingress-nginx/blob/06efac9f0b6f8f84b553f58ccecf79dc42c75cc6/Changelog.md) regarding forwarding headers and external TLS termination. Therefore, in the scenario that you are using external TLS termination configuration with NGINX v0.22, you must enable the `use-forwarded-headers` option for ingress:
For RKE installations, edit the `cluster.yml` to add the following settings.
```yaml
ingress:
provider: nginx
options:
use-forwarded-headers: "true"
use-forwarded-headers: 'true'
```
For RKE2 installations, you can create a custom `rke2-ingress-nginx-config.yaml` file at `/var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml` containing this required setting to enable using forwarded headers with external TLS termination. Without this required setting applied, the external LB will continuously respond with redirect loops it receives from the ingress controller. (This can be created before or after rancher is installed, rke2 server agent will notice this addition and automatically apply it.)
Enabling firewalld can cause serious network communication problems.
CNIs in Kubernetes dynamically update iptables and networking rules independently of any external firewalls, such as firewalld. This can cause unexpected behavior when the CNI and the external firewall conflict.
:::
Some distributions of Linux [derived from RHEL,](https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Rebuilds) including Oracle Linux, may have default firewall rules that block communication with Helm.
For example, one Oracle Linux image in AWS has REJECT rules that stop Helm from communicating with Tiller:
```
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT icmp -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
```
You can check the default firewall rules with this command:
```
sudo iptables --list
```
This section describes how to use `firewalld` to apply the [firewall port rules](../../installation-requirements/port-requirements.md) for nodes in a high-availability Rancher server cluster.
## Prerequisite
Install v7.x or later ofv`firewalld`:
```
yum install firewalld
systemctl start firewalld
systemctl enable firewalld
```
## Applying Firewall Port Rules
In the Rancher high-availability installation instructions, the Rancher server is set up on three nodes that have all three Kubernetes roles: etcd, controlplane, and worker. If your Rancher server nodes have all three roles, run the following commands on each node:
When running larger Rancher installations with 15 or more clusters it is recommended to increase the default keyspace for etcd from the default 2GB. The maximum setting is 8GB and the host should have enough RAM to keep the entire dataset in memory. When increasing this value you should also increase the size of the host. The keyspace size can also be adjusted in smaller installations if you anticipate a high rate of change of pods during the garbage collection interval.
The etcd data set is automatically cleaned up on a five minute interval by Kubernetes. There are situations, e.g. deployment thrashing, where enough events could be written to etcd and deleted before garbage collection occurs and cleans things up causing the keyspace to fill up. If you see `mvcc: database space exceeded` errors, in the etcd logs or Kubernetes API server logs, you should consider increasing the keyspace size. This can be accomplished by setting the [quota-backend-bytes](https://etcd.io/docs/v3.3/op-guide/maintenance/#space-quota) setting on the etcd servers.
## Example: This Snippet of the RKE Cluster.yml file Increases the Keyspace Size to 5GB
```yaml
# RKE cluster.yml
---
services:
etcd:
extra_args:
quota-backend-bytes:5368709120
```
## Scaling etcd Disk Performance
You can follow the recommendations from [the etcd docs](https://etcd.io/docs/v3.3/tuning/#disk) on how to tune the disk priority on the host.
Additionally, to reduce IO contention on the disks for etcd, you can use a dedicated device for the data and wal directory. Based on etcd best practices, mirroring RAID configurations are unnecessary because etcd replicates data between the nodes in the cluster. You can use striping RAID configurations to increase available IOPS.
To implement this solution in an RKE cluster, the `/var/lib/etcd/data` and `/var/lib/etcd/wal` directories will need to have disks mounted and formatted on the underlying host. In the `extra_args` directive of the `etcd` service, you must include the `wal_dir` directory. Without specifying the `wal_dir`, etcd process will try to manipulate the underlying `wal` mount with insufficient permissions.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.