Fix tables of contents, headers and formatting

2026-09-25 12:38:05 +00:00 · 2022-09-10 00:26:51 -07:00
parent 11ebb7d341
commit 5fdfa52386
336 changed files with 1249 additions and 3710 deletions
@@ -11,18 +11,8 @@ The Backup Create page lets you configure a schedule, enable encryption and spec

 ![](/img/backup_restore/backup/backup.png)

- [Schedule](#schedule)
- [Encryption](#encryption)
- [Storage Location](#storage-location)
-  - [S3](#s3)
-  - [Example S3 Storage Configuration](#example-s3-storage-configuration)
-  - [Example MinIO Configuration](#example-minio-configuration)
-  - [Example credentialSecret](#example-credentialsecret)
-  - [IAM Permissions for EC2 Nodes to Access S3](#iam-permissions-for-ec2-nodes-to-access-s3)
- [Examples](#examples)

-
-# Schedule
+## Schedule

 Select the first option to perform a one-time backup, or select the second option to schedule recurring backups. Selecting **Recurring Backups** lets you configure following two fields:

@@ -38,7 +28,7 @@ Select the first option to perform a one-time backup, or select the second optio
 | `schedule` |  Provide the cron string for scheduling recurring backups.  |
 | `retentionCount` |  Provide the number of backup files to be retained.  |

-# Encryption
+## Encryption

 The rancher-backup gathers resources by making calls to the kube-apiserver. Objects returned by apiserver are decrypted, so even if [encryption At rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) is enabled, even the encrypted objects gathered by the backup will be in plaintext.

@@ -74,7 +64,7 @@ In the example command above, the name `encryptionconfig` can be changed to anyt
 | ---------------- | ---------------- |
 | `encryptionConfigSecretName` |  Provide the name of the Secret from `cattle-resources-system` namespace, that contains the encryption config file.  |

-# Storage Location
+## Storage Location

 ![](/img/backup_restore/backup/storageLocation.png)

@@ -181,6 +171,6 @@ To allow a node to access S3, follow the instructions in the [AWS documentation]

 After the role is created, and you have attached the corresponding instance profile to your EC2 instance(s), the `credentialSecretName` directive can be left empty in the Backup custom resource.

-# Examples
+## Examples

 For example Backup custom resources, refer to [this page.](examples.md#backup)
@@ -12,25 +12,7 @@ The default backup storage location is configured when the `rancher-backup` oper

 Encrypted backups can only be restored if the Restore custom resource uses the same encryption configuration secret that was used to create the backup.

- [Backup](#backup)
-  - [Backup in the default location with encryption](#backup-in-the-default-location-with-encryption)
-  - [Recurring backup in the default location](#recurring-backup-in-the-default-location)
-  - [Encrypted recurring backup in the default location](#encrypted-recurring-backup-in-the-default-location)
-  - [Encrypted backup in Minio](#encrypted-backup-in-minio)
-  - [Backup in S3 using AWS credential secret](#backup-in-s3-using-aws-credential-secret)
-  - [Recurring backup in S3 using AWS credential secret](#recurring-backup-in-s3-using-aws-credential-secret)
-  - [Backup from EC2 nodes with IAM permission to access S3](#backup-from-ec2-nodes-with-iam-permission-to-access-s3)
- [Restore](#restore)
-  - [Restore using the default backup file location](#restore-using-the-default-backup-file-location)
-  - [Restore for Rancher migration](#restore-for-rancher-migration)
-  - [Restore from encrypted backup](#restore-from-encrypted-backup)
-  - [Restore an encrypted backup from Minio](#restore-an-encrypted-backup-from-minio)
-  - [Restore from backup using an AWS credential secret to access S3](#restore-from-backup-using-an-aws-credential-secret-to-access-s3)
-  - [Restore from EC2 nodes with IAM permissions to access S3](#restore-from-ec2-nodes-with-iam-permissions-to-access-s3)
- [Example Credential Secret for Storing Backups in S3](#example-credential-secret-for-storing-backups-in-s3)
- [Example EncryptionConfiguration](#example-encryptionconfiguration)
-
-# Backup
+## Backup

 This section contains example Backup custom resources.

@@ -154,7 +136,7 @@ spec:
  encryptionConfigSecretName: encryptionconfig
 ```

-# Restore
+## Restore

 This section contains example Restore custom resources.

@@ -11,15 +11,8 @@ The Restore Create page lets you provide details of the backup to restore from

 ![](/img/backup_restore/restore/restore.png)

- [Backup Source](#backup-source)
-	- [An Existing Backup Config](#an-existing-backup-config)
-	- [The default storage target](#the-default-storage-target)
-	- [An S3-compatible object store](#an-s3-compatible-object-store)
- [Encryption](#encryption)
- [Prune during restore](#prune-during-restore)
- [Getting the Backup Filename from S3](#getting-the-backup-filename-from-s3)

-# Backup Source
+## Backup Source
 Provide details of the backup file and its storage location, which the operator will then use to perform the restore. Select from the following options to provide these details


@@ -46,7 +39,7 @@ Select this option if no default storage location is configured at the operator-

 ![](/img/backup_restore/restore/s3store.png) 

-# Encryption
+## Encryption

 If the backup was created with encryption enabled, its file will have `.enc` suffix. Choosing such a Backup, or providing a backup filename with `.enc` suffix will display another dropdown named **Encryption Config Secret**.

@@ -63,7 +56,7 @@ The `Encryption Config Secret` dropdown will filter out and list only those Secr
 > **Important**
 This field should only be set if the backup was created with encryption enabled. Providing the incorrect encryption config will cause the restore to fail.

-# Prune During Restore
+## Prune During Restore

 * **Prune**:  In order to fully restore Rancher from a backup, and to go back to the exact state it was at when the backup was performed, we need to delete any additional resources that were created by Rancher after the backup was taken. The operator does so if the **Prune** flag is enabled. Prune is enabled by default and it is recommended to keep it enabled.
 * **Delete Timeout**: This is the amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again.
@@ -73,7 +66,7 @@ This field should only be set if the backup was created with encryption enabled.
 | `prune` |  Delete the resources managed by Rancher that are not present in the backup (Recommended).  |
 | `deleteTimeoutSeconds` |  Amount of time the operator will wait while deleting a resource before editing the resource to remove finalizers and attempt deletion again.  |

-# Getting the Backup Filename from S3
+## Getting the Backup Filename from S3

 This is the name of the backup file that the `rancher-backup` operator will use to perform the restore.

@@ -11,15 +11,8 @@ Configure a storage location where all backups are saved by default. You will ha

 Only one storage location can be configured at the operator level.

- [Storage Location Configuration](#storage-location-configuration)
-  - [No Default Storage Location](#no-default-storage-location)
-  - [S3-compatible Object Store](#s3-compatible-object-store)
-  - [Use an existing StorageClass](#existing-storageclass)
-  - [Use an existing PersistentVolume](#existing-persistent-volume)
- [Encryption](#encryption)
- [Example values.yaml for the rancher-backup Helm Chart](#example-values-yaml-for-the-rancher-backup-helm-chart)

-# Storage Location Configuration
+## Storage Location Configuration

 ### No Default Storage Location

@@ -55,7 +48,7 @@ Select an existing Persistent Volume (PV) that will be used to store your backup
 It is highly recommended to use a Persistent Volume with a reclaim policy of "Retain". Otherwise if the PVC created by the `rancher-backup` chart gets deleted (either during app upgrade, or accidentally), the PV will get deleted too, which means all backups saved in it will get deleted.


-# Example values.yaml for the rancher-backup Helm Chart
+## Example values.yaml for the rancher-backup Helm Chart

 The documented `values.yaml` file that can be used to configure `rancher-backup` operator when the Helm CLI is used can be found in the [backup-restore-operator repository.](https://github.com/rancher/backup-restore-operator/blob/release/v1.0/charts/rancher-backup/values.yaml)

@@ -12,7 +12,7 @@ In this guide, we recommend best practices for cluster-level logging and applica
 - [Application Logging](#application-logging)
 - [General Best Practices](#general-best-practices)

-# Changes in Logging in Rancher v2.5
+## Changes in Logging in Rancher v2.5

 Before Rancher v2.5, logging in Rancher has historically been a pretty static integration. There were a fixed list of aggregators to choose from (ElasticSearch, Splunk, Kafka, Fluentd and Syslog), and only two configuration points to choose (Cluster-level and Project-level).

@@ -20,7 +20,7 @@ Logging in 2.5 has been completely overhauled to provide a more flexible experie

 "Under the hood", Rancher logging uses the Banzai Cloud logging operator. We provide manageability of this operator (and its resources), and tie that experience in with managing your Rancher clusters. 

-# Cluster-level Logging
+## Cluster-level Logging

 ### Cluster-wide Scraping

@@ -38,7 +38,7 @@ Currently (as of v2.5.1) the logs from RKE containers are collected, but are not

 A future release of Rancher will include the source container name which will enable filtering of these component logs. Once that change is made, you will be able to customize a _ClusterFlow_ to retrieve **only** the Kubernetes component logs, and direct them to an appropriate output.

-# Application Logging
+## Application Logging

 Best practice not only in Kubernetes but in all container-based applications is to direct application logs to `stdout`/`stderr`. The container runtime will then trap these logs and do **something** with them - typically writing them to a file. Depending on the container runtime (and its configuration), these logs can end up in any number of locations.

@@ -54,7 +54,7 @@ The goal of setting up a streaming sidecar is to take log files that are written

 To set this up, edit your workload resource (e.g. Deployment) and add the following sidecar definition:

-```
+```yaml
 ...
 containers:
 - args:
@@ -74,7 +74,7 @@ This will add a container to your workload definition that will now stream the c

 This log stream is then automatically collected according to any _Flows_ or _ClusterFlows_ you have setup. You may also wish to consider creating a _Flow_ specifically for this log file by targeting the name of the container. See example:

-```
+```yaml
 ...
 spec:
  match:
@@ -85,7 +85,7 @@ spec:
 ```


-# General Best Practices
+## General Best Practices

 - Where possible, output structured log entries (e.g. `syslog`, JSON). This makes handling of the log entry easier as there are already parsers written for these formats. 
 - Try to provide the name of the application that is creating the log entry, in the entry itself. This can make troubleshooting easier as Kubernetes objects do not always carry the name of the application as the object name. For instance, a pod ID may be something like `myapp-098kjhsdf098sdf98` which does not provide much information about the application running inside the container. 
@@ -10,19 +10,12 @@ Configuring sensible monitoring and alerting rules is vital for running any prod

 The [Rancher monitoring documentation](../../../pages-for-subheaders/monitoring-and-alerting.md) describes how you can set up a complete Prometheus and Grafana stack. Out of the box this will scrape monitoring data from all system and Kubernetes components in your cluster and provide sensible dashboards and alerts for them to get started. But for a reliable setup, you also need to monitor your own workloads and adapt Prometheus and Grafana to your own specific use cases and cluster sizes. This document aims to give you best practices for this.

- [What to Monitor](#what-to-monitor)
- [Configuring Prometheus Resource Usage](#configuring-prometheus-resource-usage)
- [Scraping Custom Workloads](#scraping-custom-workloads)
- [Monitoring in a (Micro)Service Architecture](#monitoring-in-a-micro-service-architecture)
- [Real User Monitoring](#real-user-monitoring)
- [Security Monitoring](#security-monitoring)
- [Setting up Alerts](#setting-up-alerts)

-# What to Monitor
+## What to Monitor

 Kubernetes itself, as well as applications running inside of it, form a distributed system where different components interact with each other. For the whole system and each individual component, you have to ensure performance, availability, reliability and scalability. A good resource with more details and information is Google's free [Site Reliability Engineering Book](https://landing.google.com/sre/sre-book/), especially the chapter about [Monitoring distributed systems](https://landing.google.com/sre/sre-book/chapters/monitoring-distributed-systems/).

-# Configuring Prometheus Resource Usage
+## Configuring Prometheus Resource Usage

 When installing the integrated monitoring stack, Rancher allows to configure several settings that are dependent on the size of your cluster and the workloads running in it. This chapter covers these in more detail.

@@ -66,7 +59,7 @@ Prometheus is not meant to store metrics for a long amount of time, but should o

 In order to store some, or all metrics for a long time, you can leverage Prometheus' [remote read/write](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations) capabilities to connect it to storage systems like [Thanos](https://thanos.io/), [InfluxDB](https://www.influxdata.com/), [M3DB](https://www.m3db.io/), or others. You can find an example setup in this [blog post](https://rancher.com/blog/2020/prometheus-metric-federation).

-# Scraping Custom Workloads
+## Scraping Custom Workloads

 While the integrated Rancher Monitoring already scrapes system metrics from a cluster's nodes and system components, the custom workloads that you deploy on Kubernetes should also be scraped for data. For that you can configure Prometheus to do an HTTP request to an endpoint of your applications in a certain interval. These endpoints should then return their metrics in a Prometheus format.

@@ -94,23 +87,23 @@ To still get metrics for these use cases, you can set up [prometheus-pushgateway

 Sometimes it is useful to monitor workloads from the outside. For this, you can use the [Prometheus blackbox-exporter](https://github.com/prometheus/blackbox_exporter) which allows probing any kind of endpoint over HTTP, HTTPS, DNS, TCP and ICMP.

-# Monitoring in a (Micro)Service Architecture
+## Monitoring in a (Micro)Service Architecture

 If you have a (micro)service architecture where multiple individual workloads within your cluster are communicating with each other, it is really important to have detailed metrics and traces about this traffic to understand how all these workloads are communicating with each other and where a problem or bottleneck may be.

 Of course you can monitor all this internal traffic in all your workloads and expose these metrics to Prometheus. But this can quickly become quite work intensive. Service Meshes like Istio, which can be installed with [a click](../../../pages-for-subheaders/istio.md) in Rancher, can do this automatically and provide rich telemetry about the traffic between all services.

-# Real User Monitoring
+## Real User Monitoring

 Monitoring the availability and performance of all your internal workloads is vitally important to run stable, reliable and fast applications. But these metrics only show you parts of the picture. To get a complete view it is also necessary to know how your end users are actually perceiving it. For this you can look into various [Real user monitoring solutions](https://en.wikipedia.org/wiki/Real_user_monitoring).

-# Security Monitoring
+## Security Monitoring

 In addition to monitoring workloads to detect performance, availability or scalability problems, the cluster and the workloads running into it should also be monitored for potential security problems. A good starting point is to frequently run and alert on [CIS Scans](../../../pages-for-subheaders/cis-scans.md) which check if the cluster is configured according to security best practices.

 For the workloads, you can have a look at Kubernetes and Container security solutions like [Falco](https://falco.org/), [Aqua Kubernetes Security](https://www.aquasec.com/solutions/kubernetes-container-security/), [SysDig](https://sysdig.com/).

-# Setting up Alerts
+## Setting up Alerts

 Getting all the metrics into a monitoring systems and visualizing them in dashboards is great, but you also want to be pro-actively alerted if something goes wrong.

@@ -9,17 +9,12 @@ aliases:

 This guide outlines a reference architecture for installing Rancher on an RKE Kubernetes cluster in a vSphere environment, in addition to standard vSphere best practices as documented by VMware.

- [1. Load Balancer Considerations](#1-load-balancer-considerations)
- [2. VM Considerations](#2-vm-considerations)
- [3. Network Considerations](#3-network-considerations)
- [4. Storage Considerations](#4-storage-considerations)
- [5. Backups and Disaster Recovery](#5-backups-and-disaster-recovery)

 <figcaption>Solution Overview</figcaption>

 ![Solution Overview](/img/rancher-on-prem-vsphere.svg)

-# 1. Load Balancer Considerations
+## 1. Load Balancer Considerations

 A load balancer is required to direct traffic to the Rancher workloads residing on the RKE nodes.

@@ -45,7 +40,7 @@ Avoid implementing a software load balancer within the management cluster.

 Configure appropriate Firewall / ACL rules to only expose access to Rancher

-# 2. VM Considerations
+## 2. VM Considerations

 ### Size the VM's According to Rancher Documentation

@@ -67,7 +62,7 @@ Doing so will ensure node VM's are spread across multiple datastores - preventin

 It’s important to follow K8s and etcd best practices when deploying your nodes, including disabling swap, double-checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node.

-# 3. Network Considerations 
+## 3. Network Considerations 

 ### Leverage Low Latency, High Bandwidth Connectivity Between ETCD Nodes

@@ -77,13 +72,13 @@ Deploy etcd members within a single data center where possible to avoid latency

 Each node used should have a static IP configured. In the case of DHCP, each node should have a DHCP reservation to make sure the node gets the same IP allocated.

-# 4. Storage Considerations
+## 4. Storage Considerations

 ### Leverage SSD Drives for ETCD Nodes

 ETCD is very sensitive to write latency. Therefore, leverage SSD disks where possible. 

-# 5. Backups and Disaster Recovery
+## 5. Backups and Disaster Recovery

 ### Perform Regular Management Cluster Backups

@@ -11,7 +11,7 @@ There are two recommended deployment strategies for a Rancher server that manage
 * [Hub and Spoke](#hub-and-spoke-strategy)
 * [Regional](#regional-strategy)

-# Hub & Spoke Strategy
+## Hub & Spoke Strategy
 ---

 In this deployment scenario, there is a single Rancher control plane managing Kubernetes clusters across the globe. The control plane would be run on a high-availability Kubernetes cluster, and there would be impact due to latencies.
@@ -29,7 +29,7 @@ In this deployment scenario, there is a single Rancher control plane managing Ku
 * Subject to network latencies.
 * If the control plane goes out, global provisioning of new services is unavailable until it is restored. However, each Kubernetes cluster can continue to be managed individually.

-# Regional Strategy
+## Regional Strategy
 ---
 In the regional deployment model a control plane is deployed in close proximity to the compute nodes.

@@ -2,10 +2,6 @@
 title: kubectl Utility
 ---

- [kubectl](#kubectl)
-  - [kubectl Utility](#kubectl-utility)
-  - [Authentication with kubectl and kubeconfig Tokens with TTL](#authentication-with-kubectl-and-kubeconfig-tokens-with-ttl) 
-
 # kubectl

 Interact with Rancher using kubectl.
@@ -3,15 +3,6 @@ title: Rancher CLI
 description: Interact with Rancher using command line interface (CLI) tools from your workstation.
 ---

- [Rancher CLI](#rancher-cli)
-  - [Download Rancher CLI](#download-rancher-cli)
-  - [Requirements](#requirements)
-  - [CLI Authentication](#cli-authentication)
-  - [Project Selection](#project-selection)
-  - [Commands](#commands)
-  - [Rancher CLI Help](#rancher-cli-help)
-  - [Limitations](#limitations)
-
 The Rancher CLI (Command Line Interface) is a unified tool that you can use to interact with Rancher. With this tool, you can operate Rancher using a command line rather than the GUI.

 ### Download Rancher CLI
@@ -9,14 +9,8 @@ aliases:

 The following node template configuration reference applies to Rancher v2.3.3+.

- [Account Access](#account-access)
- [Scheduling](#scheduling)
- [Instance Options](#instance-options)
- [Networks](#networks)
- [Node tags and custom attributes](#node-tags-and-custom-attributes)
- [cloud-init](#cloud-init)

-# Account Access
+## Account Access

 | Parameter                | Required | Description |
 |:----------------------|:--------:|:-----|
@@ -44,7 +38,7 @@ The fields in the **Scheduling** section should auto-populate with the data cent
 | Folder |    |  Name of a folder in the datacenter to create the VMs in. Must already exist. The VM folders in this dropdown menu directly correspond to your VM folders in vSphere. The folder name should be prefaced with `vm/` in your vSphere config file. |
 | Host |  | The IP of the host system to schedule VMs in. Leave this field blank for a standalone ESXi or for a cluster with DRS (Distributed Resource Scheduler). If specified, the host system's pool will be used and the **Resource Pool** parameter will be ignored.   |

-# Instance Options
+## Instance Options

 In the **Instance Options** section, configure the number of vCPUs, memory, and disk size for the VMs created by this template.

@@ -72,11 +66,11 @@ Choose the way that the VM will be created:
 - **Clone an existing virtual machine:** In the **Virtual machine** field, choose an existing VM that the new VM will be cloned from.
 - **Install from boot2docker ISO:** Ensure that the **OS ISO URL** field contains the URL of a VMware ISO release for RancherOS (`rancheros-vmware.iso`). Note that this URL must be accessible from the nodes running your Rancher server installation.

-# Networks
+## Networks

 The node template now allows a VM to be provisioned with multiple networks. In the **Networks** field, you can now click **Add Network** to add any networks available to you in vSphere.

-# Node Tags and Custom Attributes
+## Node Tags and Custom Attributes

 Tags allow you to attach metadata to objects in the vSphere inventory to make it easier to sort and search for these objects.

@@ -86,7 +80,7 @@ In the custom attributes, Rancher will let you select all the custom attributes

 > **Note:** Custom attributes are a legacy feature that will eventually be removed from vSphere.

-# cloud-init
+## cloud-init

 [Cloud-init](https://cloudinit.readthedocs.io/en/latest/) allows you to initialize your nodes by applying configuration on the first boot. This may involve things such as creating users, authorizing SSH keys or setting up the network.

@@ -15,18 +15,8 @@ For help choosing a Helm chart version, refer to [this page.](../../getting-star

 For information on enabling experimental features, refer to [this page.](../../pages-for-subheaders/enable-experimental-features.md)

- [Common Options](#common-options)
- [Advanced Options](#advanced-options)
- [API Audit Log](#api-audit-log)
- [Setting Extra Environment Variables](#setting-extra-environment-variables)
- [TLS Settings](#tls-settings)
- [Customizing your Ingress](#customizing-your-ingress)
- [HTTP Proxy](#http-proxy)
- [Additional Trusted CAs](#additional-trusted-cas)
- [Private Registry and Air Gap Installs](#private-registry-and-air-gap-installs)
- [External TLS Termination](#external-tls-termination)

-### Common Options
+## Common Options

 | Option                    | Default Value | Description                                                                        |
 | ------------------------- | ------------- | ---------------------------------------------------------------------------------- |
@@ -38,7 +28,7 @@ For information on enabling experimental features, refer to [this page.](../../p

 <br/>

-### Advanced Options
+## Advanced Options

 | Option                         | Default Value                                         | Description                                                                                                                                       |
 | ------------------------------ | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -7,34 +7,24 @@ aliases:

 This page explains concepts related to Kubernetes that are important for understanding how Rancher works. The descriptions below provide a simplified interview of Kubernetes components. For more details, refer to the [official documentation on Kubernetes components.](https://kubernetes.io/docs/concepts/overview/components/)

-This section covers the following topics:

- [About Docker](#about-docker)
- [About Kubernetes](#about-kubernetes)
- [What is a Kubernetes Cluster?](#what-is-a-kubernetes-cluster)
- [Roles for Nodes in Kubernetes Clusters](#roles-for-nodes-in-kubernetes-clusters)
-  - [etcd Nodes](#etcd-nodes)
-  - [Controlplane Nodes](#controlplane-nodes)
-  - [Worker Nodes](#worker-nodes)
- [About Helm](#about-helm)
-
-# About Docker
+## About Docker

 Docker is the container packaging and runtime standard. Developers build container images from Dockerfiles and distribute container images from Docker registries. [Docker Hub](https://hub.docker.com) is the most popular public registry. Many organizations also set up private Docker registries. Docker is primarily used to manage containers on individual nodes.

 >**Note:** Although Rancher 1.6 supported Docker Swarm clustering technology, it is no longer supported in Rancher 2.x due to the success of Kubernetes.

-# About Kubernetes
+## About Kubernetes

 Kubernetes is the container cluster management standard. YAML files specify containers and other resources that form an application. Kubernetes performs functions such as scheduling, scaling, service discovery, health check, secret management, and configuration management.

-# What is a Kubernetes Cluster?
+## What is a Kubernetes Cluster?

 A cluster is a group of computers that work together as a single system.

 A _Kubernetes Cluster_ is a cluster that uses the [Kubernetes container-orchestration system](https://kubernetes.io/) to deploy, maintain, and scale Docker containers, allowing your organization to automate application operations.

-# Roles for Nodes in Kubernetes Clusters
+## Roles for Nodes in Kubernetes Clusters

 Each computing resource in a Kubernetes cluster is called a _node_. Nodes can be either bare-metal servers or virtual machines. Kubernetes classifies nodes into three types: _etcd_ nodes, _control plane_ nodes, and _worker_ nodes.

@@ -65,7 +55,7 @@ Each [worker node](https://kubernetes.io/docs/concepts/architecture/nodes/) runs

 Worker nodes also run storage and networking drivers, and ingress controllers when required. You create as many worker nodes as necessary to run your  [workloads](../pages-for-subheaders/workloads-and-pods.md).

-# About Helm
+## About Helm

 For high-availability installations of Rancher, Helm is the tool used to install Rancher on a Kubernetes cluster.

@@ -3,15 +3,8 @@ title: Helm Chart Options
 weight: 8
 ---

- [Configuring Resource Limits and Requests](#configuring-resource-limits-and-requests)
- [Trusted CA for Notifiers](#trusted-ca-for-notifiers)
- [Additional Scrape Configurations](#additional-scrape-configurations)
- [Configuring Applications Packaged within Monitoring V2](#configuring-applications-packaged-within-monitoring-v2)
- [Increase the Replicas of Alertmanager](#increase-the-replicas-of-alertmanager)
- [Configuring the Namespace for a Persistent Grafana Dashboard](#configuring-the-namespace-for-a-persistent-grafana-dashboard)

-
-# Configuring Resource Limits and Requests
+## Configuring Resource Limits and Requests

 The resource requests and limits can be configured when installing `rancher-monitoring`.

@@ -32,7 +25,7 @@ The default values in the table below are the minimum required resource limits a
 At least 50Gi storage is recommended.


-# Trusted CA for Notifiers
+## Trusted CA for Notifiers

 If you need to add a trusted CA to your notifier, follow these steps:

@@ -43,7 +36,7 @@ If you need to add a trusted CA to your notifier, follow these steps:
 **Result:** The default Alertmanager custom resource will have access to your trusted CA.


-# Additional Scrape Configurations
+## Additional Scrape Configurations

 If the scrape configuration you want cannot be specified via a ServiceMonitor or PodMonitor at the moment, you can provide an `additionalScrapeConfigSecret` on deploying or upgrading `rancher-monitoring`.

@@ -52,7 +45,7 @@ A [scrape_config section](https://prometheus.io/docs/prometheus/latest/configura
 An example of where this might be used is with Istio. For more information, see [this section.](https://rancher.com/docs/rancher/v2.5/en/istio/configuration-reference/selectors-and-scrape)


-# Configuring Applications Packaged within Monitoring v2
+## Configuring Applications Packaged within Monitoring v2

 We deploy kube-state-metrics and node-exporter with monitoring v2. Node exporter are deployed as DaemonSets. In the monitoring v2 helm chart, in the values.yaml, each of the things are deployed as sub charts.

@@ -17,26 +17,8 @@ The [Alertmanager Config](https://prometheus.io/docs/alerting/latest/configurati

 > This section assumes familiarity with how monitoring components work together. For more information about Alertmanager, see [this section.](../../explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md#3-how-alertmanager-works)

- [Creating Receivers in the Rancher UI](#creating-receivers-in-the-rancher-ui)
- [Receiver Configuration](#receiver-configuration)
-  - [Slack](#slack)
-  - [Email](#email)
-  - [PagerDuty](#pagerduty)
-  - [Opsgenie](#opsgenie)
-  - [Webhook](#webhook)
-  - [Custom](#custom)
-  - [Teams](#teams)
-  - [SMS](#sms)
- [Route Configuration](#route-configuration)
-  - [Receiver](#receiver)
-  - [Grouping](#grouping)
-  - [Matching](#matching)
- [Configuring Multiple Receivers](#configuring-multiple-receivers)
- [Example Alertmanager Config](examples.md#example-alertmanager-config)
- [Example Route Config for CIS Scan Alerts](#example-route-config-for-cis-scan-alerts)
- [Trusted CA for Notifiers](#trusted-ca-for-notifiers)

-# Creating Receivers in the Rancher UI
+## Creating Receivers in the Rancher UI
 _Available as of v2.5.4_

 > **Prerequisites:**
@@ -53,7 +35,7 @@ To create notification receivers in the Rancher UI,

 **Result:** Alerts can be configured to send notifications to the receiver(s).

-# Receiver Configuration
+## Receiver Configuration

 The notification integrations are configured with the `receiver`, which is explained in the [Prometheus documentation.](https://prometheus.io/docs/alerting/latest/configuration/#receiver)

@@ -91,7 +73,7 @@ The following types of receivers can be configured in the Rancher UI:

 The custom receiver option can be used to configure any receiver in YAML that cannot be configured by filling out the other forms in the Rancher UI.

-# Slack
+## Slack

 | Field | Type | Description |
 |------|--------------|------|
@@ -100,7 +82,7 @@ The custom receiver option can be used to configure any receiver in YAML that ca
 | Proxy URL   |    String    |  Proxy for the webhook notifications.  |
 | Enable Send Resolved Alerts |   Bool    |  Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |

-# Email
+## Email

 | Field | Type | Description |
 |------|--------------|------|
@@ -117,7 +99,7 @@ SMTP options:
 | Username |   String   | Enter a username to authenticate with the SMTP server. |
 | Password |   String    | Enter a password to authenticate with the SMTP server. |

-# PagerDuty
+## PagerDuty

 | Field | Type | Description |
 |------|------|-------|
@@ -126,7 +108,7 @@ SMTP options:
 | Proxy URL | String |  Proxy for the PagerDuty notifications.  |
 | Enable Send Resolved Alerts |  Bool    |   Whether to send a follow-up notification if an alert has been resolved (e.g. [Resolved] High CPU Usage). |

-# Opsgenie
+## Opsgenie

 | Field | Description |
 |------|-------------|
@@ -141,7 +123,7 @@ Opsgenie Responders:
 | Type | String | Schedule, Team, User, or Escalation. For more information on alert responders, refer to the [Opsgenie documentation.](https://docs.opsgenie.com/docs/alert-recipients-and-teams) |
 | Send To | String | Id, Name, or Username of the Opsgenie recipient. |

-# Webhook
+## Webhook

 | Field |    Description |
 |-------|--------------|
@@ -151,11 +133,11 @@ Opsgenie Responders:

 <!-- TODO add info on webhook for teams and sms and link to them -->

-# Custom
+## Custom

 The YAML provided here will be directly appended to your receiver within the Alertmanager Config Secret.

-# Teams
+## Teams

 ### Enabling the Teams Receiver for Rancher Managed Clusters

@@ -189,7 +171,7 @@ url: http://rancher-alerting-drivers-prom2teams.ns-1.svc:8089/v2/teams-instance-

 <!-- https://github.com/idealista/prom2teams -->

-# SMS
+## SMS

 ### Enabling the SMS Receiver for Rancher Managed Clusters

@@ -315,7 +297,7 @@ The Alertmanager must be configured in YAML, as shown in these [examples.](#exam
 </TabItem>
 </Tabs>

-# Configuring Multiple Receivers
+## Configuring Multiple Receivers

 By editing the forms in the Rancher UI, you can set up a Receiver resource with all the information Alertmanager needs to send alerts to your notification system.

@@ -324,7 +306,7 @@ It is also possible to send alerts to multiple notification systems. One way is
 You can also set up multiple receivers by using the `continue` option for a route, so that the alerts sent to a receiver continue being evaluated in the next level of the routing tree, which could contain another receiver.


-# Example Alertmanager Configs
+## Example Alertmanager Configs

 ### Slack
 To set up notifications via Slack, the following Alertmanager Config YAML can be placed into the `alertmanager.yaml` key of the Alertmanager Config Secret, where the `api_url` should be updated to use your Webhook URL from Slack:
@@ -371,7 +353,7 @@ receivers:
  - service_key: 'database-integration-key'
 ```

-# Example Route Config for CIS Scan Alerts
+## Example Route Config for CIS Scan Alerts

 While configuring the routes for `rancher-cis-benchmark` alerts, you can specify the matching using the key-value pair `job: rancher-cis-scan`.

@@ -396,6 +378,6 @@ spec:
 For more information on enabling alerting for `rancher-cis-benchmark`, see [this section.](../../pages-for-subheaders/cis-scan-guides.md#enabling-alerting-for-rancher-cis-benchmark)


-# Trusted CA for Notifiers
+## Trusted CA for Notifiers

 If you need to add a trusted CA to your notifier, follow the steps in [this section.](helm-chart-options.md#trusted-ca-for-notifiers)
@@ -15,13 +15,9 @@ For more information about configuring routes, refer to the [official Alertmanag

 > This section assumes familiarity with how monitoring components work together. For more information, see [this section.](../../explanations/integrations-in-rancher/monitoring-and-alerting/how-monitoring-works.md)

- [Route Restrictions](#route-restrictions)
- [Route Configuration](#route-configuration)
-  - [Receiver](#receiver)
-  - [Grouping](#grouping)
-  - [Matching](#matching)

-# Route Restrictions
+
+## Route Restrictions

 Alertmanager proxies alerts for Prometheus based on its receivers and a routing tree that filters alerts to certain receivers based on labels.

@@ -31,7 +27,7 @@ In the Rancher UI for configuring routes and receivers, you can configure routin

 Each receiver is for one or more notification providers. So if you know that every alert for Slack should also go to PagerDuty, you can configure both in the same receiver.

-# Route Configuration
+## Route Configuration

 ### Note on Labels and Annotations

@@ -29,30 +29,31 @@ This section assumes that you understand how persistent storage works in Kuberne
    - **Add Volume > Use an existing persistent volume (claim)**

 1.  Complete the form that displays to choose a persistent volume for the internal Docker registry.
+
    <Tabs>
-      <TabItem value="Add a new persistent volume">
+    <TabItem value="Add a new persistent volume">

-        1. Enter a **Name** for the volume claim.
-        1. Select a volume claim **Source**:
-            - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
-            - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
-        1. From the **Customize** section, choose the read/write access for the volume.
-        1. Click **Define**.
+    1. Enter a **Name** for the volume claim.
+    1. Select a volume claim **Source**:
+        - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
+        - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
+    1. From the **Customize** section, choose the read/write access for the volume.
+    1. Click **Define**.

-      </TabItem>
-      <TabItem value="Use an existing persistent volume">
+    </TabItem>
+    <TabItem value="Use an existing persistent volume">

-        1. Enter a **Name** for the volume claim.
-        1. Choose a **Persistent Volume Claim** from the drop-down.
-        1. From the **Customize** section, choose the read/write access for the volume.
-        1. Click **Define**.
+      1. Enter a **Name** for the volume claim.
+      1. Choose a **Persistent Volume Claim** from the drop-down.
+      1. From the **Customize** section, choose the read/write access for the volume.
+      1. Click **Define**.

-      </TabItem>
+    </TabItem>
    </Tabs>

-1. From the **Mount Point** field, enter `/var/lib/registry`, which is the data storage path inside the Docker registry container.
+4. From the **Mount Point** field, enter `/var/lib/registry`, which is the data storage path inside the Docker registry container.

-1. Click **Upgrade**.
+5. Click **Upgrade**.

 ### B. Configuring Persistent Data for Minio

@@ -64,25 +65,26 @@ This section assumes that you understand how persistent storage works in Kuberne
    - **Add Volume > Use an existing persistent volume (claim)**

 1.  Complete the form that displays to choose a persistent volume for the internal Docker registry.
+
    <Tabs>
-      <TabItem value="Add a new persistent volume">
+    <TabItem value="Add a new persistent volume">

-        1. Enter a **Name** for the volume claim.
-        1. Select a volume claim **Source**:
-            - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
-            - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
-        1. From the **Customize** section, choose the read/write access for the volume.
-        1. Click **Define**.
+    1. Enter a **Name** for the volume claim.
+    1. Select a volume claim **Source**:
+        - If you select **Use a Storage Class to provision a new persistent volume**, select a storage class and enter a **Capacity**.
+        - If you select **Use an existing persistent volume**, choose a **Persistent Volume** from the drop-down.
+    1. From the **Customize** section, choose the read/write access for the volume.
+    1. Click **Define**.

-      </TabItem>
-      <TabItem value="Use an existing persistent volume">
+    </TabItem>
+    <TabItem value="Use an existing persistent volume">

-        1. Enter a **Name** for the volume claim.
-        1. Choose a **Persistent Volume Claim** from the drop-down.
-        1. From the **Customize** section, choose the read/write access for the volume.
-        1. Click **Define**.
+    1. Enter a **Name** for the volume claim.
+    1. Choose a **Persistent Volume Claim** from the drop-down.
+    1. From the **Customize** section, choose the read/write access for the volume.
+    1. Click **Define**.

-      </TabItem>
+    </TabItem>
    </Tabs>

 1. From the **Mount Point** field, enter `/data`, which is the data storage path inside the Minio container.
@@ -8,26 +8,8 @@ aliases:

 In this section, you'll learn how to configure pipelines.

- [Step Types](#step-types)
- [Step Type: Run Script](#step-type-run-script)
- [Step Type: Build and Publish Images](#step-type-build-and-publish-images)
- [Step Type: Publish Catalog Template](#step-type-publish-catalog-template)
- [Step Type: Deploy YAML](#step-type-deploy-yaml)
- [Step Type: Deploy Catalog App](#step-type-deploy-catalog-app)
- [Notifications](#notifications)
- [Timeouts](#timeouts)
- [Triggers and Trigger Rules](#triggers-and-trigger-rules)
- [Environment Variables](#environment-variables)
- [Secrets](#secrets)
- [Pipeline Variable Substitution Reference](#pipeline-variable-substitution-reference)
- [Global Pipeline Execution Settings](#global-pipeline-execution-settings)
-  - [Executor Quota](#executor-quota)
-  - [Resource Quota for Executors](#resource-quota-for-executors)
-  - [Custom CA](#custom-ca)
- [Persistent Data for Pipeline Components](#persistent-data-for-pipeline-components)
- [Example rancher-pipeline.yml](#example-rancher-pipeline-yml)

-# Step Types
+## Step Types

 Within each stage, you can add as many steps as you'd like. When there are multiple steps in one stage, they run concurrently.

@@ -83,7 +65,7 @@ stages:
        pushRemote: true
        registry: reg.example.com
 ```
-# Step Type: Run Script
+## Step Type: Run Script

 The **Run Script** step executes arbitrary commands in the workspace inside a specified container. You can use it to build, test and do more, given whatever utilities the base image provides. For your convenience, you can use variables to refer to metadata of a pipeline execution. Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables.

@@ -103,7 +85,7 @@ stages:
      image: golang
      shellScript: go build
 ```
-# Step Type: Build and Publish Images
+## Step Type: Build and Publish Images

 The **Build and Publish Image** step builds and publishes a Docker image. This process requires a Dockerfile in your source code's repository to complete successfully.

@@ -153,7 +135,7 @@ stages:
      PLUGIN_INSECURE: "true"
 ```

-# Step Type: Publish Catalog Template
+## Step Type: Publish Catalog Template

 The **Publish Catalog Template** step publishes a version of a catalog app template (i.e. Helm chart) to a git hosted chart repository. It generates a git commit and pushes it to your chart repository. This process requires a chart folder in your source code's repository and a pre-configured secret in the dedicated pipeline namespace to complete successfully. Any variables in the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) is supported for any file in the chart folder.

@@ -209,7 +191,7 @@ stages:
      sourceKey: DEPLOY_KEY
 ```

-# Step Type: Deploy YAML
+## Step Type: Deploy YAML

 This step deploys arbitrary Kubernetes resources to the project. This deployment requires a Kubernetes manifest file to be present in the source code repository. Pipeline variable substitution is supported in the manifest file. You can view an example file at [GitHub](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml). Please refer to the [pipeline variable substitution reference](#pipeline-variable-substitution-reference) for the list of available variables.

@@ -232,7 +214,7 @@ stages:
      path: ./deployment.yaml
 ```

-# Step Type :Deploy Catalog App
+## Step Type :Deploy Catalog App

 The **Deploy Catalog App** step deploys a catalog app in the project. It will install a new app if it is not present, or upgrade an existing one.

@@ -278,7 +260,7 @@ stages:
      targetNamespace: test
 ```

-# Timeouts
+## Timeouts

 By default, each pipeline execution has a timeout of 60 minutes. If the pipeline execution cannot complete within its timeout period, the pipeline is aborted.

@@ -302,7 +284,7 @@ stages:
 timeout: 30
 ```

-# Notifications
+## Notifications

 You can enable notifications to any notifiers based on the build status of a pipeline. Before enabling notifications, Rancher recommends [setting up notifiers](../monitoring-v2-configuration/receivers.md) so it will be easy to add recipients immediately.

@@ -350,7 +332,7 @@ notification:
  message: "my-message"
 ```

-# Triggers and Trigger Rules
+## Triggers and Trigger Rules

 After you configure a pipeline, you can trigger it using different methods:

@@ -374,12 +356,6 @@ If all conditions evaluate to `true`, then the pipeline/stage/step is executed.

 Wildcard character (`*`) expansion is supported in `branch` conditions.

-This section covers the following topics:
-
- [Configuring pipeline triggers](#configuring-pipeline-triggers)
- [Configuring stage triggers](#configuring-stage-triggers)
- [Configuring step triggers](#configuring-step-triggers)
- [Configuring triggers by YAML](#configuring-triggers-by-yaml)

 ### Configuring Pipeline Triggers

@@ -475,7 +451,7 @@ branch:
  exclude: [ dev ]
 ```

-# Environment Variables
+## Environment Variables

 When configuring a pipeline, certain [step types](#step-types) allow you to use environment variables to configure the step's script.

@@ -512,7 +488,7 @@ stages:
        SECOND_KEY: VALUE2
 ```

-# Secrets
+## Secrets

 If you need to use security-sensitive information in your pipeline scripts (like a password), you can pass them in using Kubernetes [secrets](../../how-to-guides/new-user-guides/kubernetes-resources-setup/secrets.md).

@@ -555,7 +531,7 @@ stages:
        targetKey: ALIAS_ENV
 ```

-# Pipeline Variable Substitution Reference
+## Pipeline Variable Substitution Reference

 For your convenience, the following variables are available for your pipeline configuration scripts. During pipeline executions, these variables are replaced by metadata. You can reference them in the form of `${VAR_NAME}`.

@@ -574,7 +550,7 @@ Variable Name           | Description
 `CICD_REGISTRY`           | Address for the Docker registry for the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step.
 `CICD_IMAGE`              | Name of the image built from the previous publish image step, available in the Kubernetes manifest file of a `Deploy YAML` step. It does not contain the image tag.<br/><br/> [Example](https://github.com/rancher/pipeline-example-go/blob/master/deployment.yaml)

-# Global Pipeline Execution Settings
+## Global Pipeline Execution Settings

 After configuring a version control provider, there are several options that can be configured globally on how pipelines are executed in Rancher. These settings can be edited by selecting **Tools > Pipelines** in the navigation bar.

@@ -637,12 +613,12 @@ If you want to use a version control provider with a certificate from a custom/i

 **Result:** Pipelines can be used and new pods will be able to work with the self-signed-certificate.

-# Persistent Data for Pipeline Components
+## Persistent Data for Pipeline Components

 The internal Docker registry and the Minio workloads use ephemeral volumes by default. This default storage works out-of-the-box and makes testing easy, but you lose the build images and build logs if the node running the Docker Registry or Minio fails. In most cases this is fine. If you want build images and logs to survive node failures, you can configure the Docker Registry and Minio to use persistent volumes.

 For details on setting up persistent storage for pipelines, refer to [this page.](configure-persistent-data.md)

-# Example rancher-pipeline.yml
+## Example rancher-pipeline.yml

 An example pipeline configuration file is on [this page.](example-yaml.md)
@@ -6,17 +6,7 @@ aliases:
  - /rancher/v2.x/en/cluster-admin/tools/
 ---

-Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently. Tools are divided into following categories:
-
-<!-- TOC -->
-
- [Logging](#logging)
- [Monitoring and Alerts](#monitoring-and-alerts)
- [Istio](#istio)
- [OPA Gatekeeper](#opa-gatekeeper)
- [CIS Scans](#cis-scans)
-
-<!-- /TOC -->
+Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently.


 # Logging
@@ -7,15 +7,6 @@ aliases:

 Kubernetes cluster. If you are installing Rancher on a single node, the main architecture recommendation that applies to your installation is that the node running Rancher should be [separate from downstream clusters.](#separation-of-rancher-and-user-clusters)

-This section covers the following topics:
-
- [Separation of Rancher and User Clusters](#separation-of-rancher-and-user-clusters)
- [Why HA is Better for Rancher in Production](#why-ha-is-better-for-rancher-in-production)
- [Recommended Load Balancer Configuration for Kubernetes Installations](#recommended-load-balancer-configuration-for-kubernetes-installations)
- [Environment for Kubernetes Installations](#environment-for-kubernetes-installations)
- [Recommended Node Roles for Kubernetes Installations](#recommended-node-roles-for-kubernetes-installations)
- [Architecture for an Authorized Cluster Endpoint](#architecture-for-an-authorized-cluster-endpoint)
-
 # Separation of Rancher and User Clusters

 A user cluster is a downstream Kubernetes cluster that runs your apps and services.
@@ -5,14 +5,7 @@ aliases:
  - /rancher/v2.x/en/project-admin/tools/
 ---

-Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently. Tools are divided into following categories:
-<!-- TOC -->
-
- [Notifiers and Alerts](#notifiers-and-alerts)
- [Logging](#logging)
- [Monitoring](#monitoring)
-
-<!-- /TOC -->
+Rancher contains a variety of tools that aren't included in Kubernetes to assist in your DevOps operations. Rancher can integrate with external services to help your clusters run more efficiently.

 ## Notifiers and Alerts

@@ -8,7 +8,7 @@ The `rancher-selinux` RPM only contains policies for the [rancher-logging applic

 The `rancher-selinux` GitHub repository is [here.](https://github.com/rancher/rancher-selinux)

-# Installing the rancher-selinux RPM
+## Installing the rancher-selinux RPM

 :::note Requirement:

@@ -53,7 +53,7 @@ Install the RPM:
 yum -y install rancher-selinux
 ```

-# Configuring the Logging Application to Work with SELinux
+## Configuring the Logging Application to Work with SELinux

 :::note Requirement:

@@ -1,5 +1,5 @@
 ---
-title: Example YAML
+title: RKE1 Example YAML
 weight: 60
 aliases:
  - /rancher/v2.x/en/admin-settings/rke-templates/example-yaml/