Merge pull request #2238 from sunilarjun/update-ingress

Update ingress-nginx pt 1
This commit is contained in:
Sunil Singh
2026-03-19 16:06:15 -07:00
committed by GitHub
152 changed files with 549 additions and 3701 deletions
@@ -8,14 +8,12 @@ title: Setting up Amazon ELB Network Load Balancer
This how-to guide describes how to set up a Network Load Balancer (NLB) in Amazon's EC2 service that will direct traffic to multiple instances on EC2.
These examples show the load balancer being configured to direct traffic to three Rancher server nodes. If Rancher is installed on an RKE Kubernetes cluster, three nodes are required. If Rancher is installed on a K3s Kubernetes cluster, only two nodes are required.
These examples show the load balancer being configured to direct traffic to three Rancher server nodes. If Rancher is installed on a K3s Kubernetes cluster, only two nodes are required.
This tutorial is about one possible way to set up your load balancer, not the only way. Other types of load balancers, such as a Classic Load Balancer or Application Load Balancer, could also direct traffic to the Rancher server nodes.
Rancher only supports using the Amazon NLB when terminating traffic in `tcp` mode for port 443 rather than `tls` mode. This is due to the fact that the NLB does not inject the correct headers into requests when terminated at the NLB. This means that if you want to use certificates managed by the Amazon Certificate Manager (ACM), you should use an ALB.
## Requirements
These instructions assume you have already created Linux instances in EC2. The load balancer will direct traffic to these nodes.
@@ -26,7 +24,7 @@ Begin by creating two target groups for the **TCP** protocol, one with TCP port
Your first NLB configuration step is to create two target groups. Technically, only port 443 is needed to access Rancher, but it's convenient to add a listener for port 80, because traffic to port 80 will be automatically redirected to port 443.
Regardless of whether an NGINX Ingress or Traefik Ingress controller is used, the Ingress should redirect traffic from port 80 to port 443.
The Traefik Ingress should redirect traffic from port 80 to port 443.
1. Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started. Make sure to select the **Region** where your EC2 instances (Linux nodes) are created.
1. Select **Services** and choose **EC2**, find the section **Load Balancing** and open **Target Groups**.
@@ -34,7 +32,7 @@ Regardless of whether an NGINX Ingress or Traefik Ingress controller is used, th
:::note
Health checks are handled differently based on the Ingress. For details, refer to [this section.](#health-check-paths-for-nginx-ingress-and-traefik-ingresses)
For details on Traefik Ingress health checks, refer to [this section.](#health-check-paths-for-traefik-ingresses)
:::
@@ -167,13 +165,10 @@ After AWS creates the NLB, click **Close**.
6. Click **Save** in the top right of the screen.
## Health Check Paths for NGINX Ingress and Traefik Ingresses
## Health Check Paths for Traefik Ingresses
K3s and RKE Kubernetes clusters handle health checks differently because they use different Ingresses by default.
K3s Kubernetes clusters use Traefik as the default Ingress.
For RKE Kubernetes clusters, NGINX Ingress is used by default, whereas for K3s Kubernetes clusters, Traefik is the default Ingress.
The health check path is `/ping`. By default `/ping` is always matched (regardless of Host), and a response from [Traefik itself](https://docs.traefik.io/operations/ping/) is always served.
- **Traefik:** The health check path is `/ping`. By default `/ping` is always matched (regardless of Host), and a response from [Traefik itself](https://docs.traefik.io/operations/ping/) is always served.
- **NGINX Ingress:** The default backend of the NGINX Ingress controller has a `/healthz` endpoint. By default `/healthz` is always matched (regardless of Host), and a response from [`ingress-nginx` itself](https://github.com/kubernetes/ingress-nginx/blob/0cbe783f43a9313c9c26136e888324b1ee91a72f/charts/ingress-nginx/values.yaml#L212) is always served.
To simulate an accurate health check, it is a best practice to use the Host header (Rancher hostname) combined with `/ping` or `/healthz` (for K3s or for RKE clusters, respectively) wherever possible, to get a response from the Rancher Pods, not the Ingress.
To simulate an accurate health check, it is a best practice to use the Host header (Rancher hostname) combined with `/ping` or `/healthz` wherever possible, to get a response from the Rancher Pods, not the Ingress.
@@ -91,7 +91,7 @@ To use this `kubeconfig` file,
1. Install [kubectl,](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) a Kubernetes command-line tool.
2. Copy the file at `/etc/rancher/rke2/rke2.yaml` and save it to the directory `~/.kube/config` on your local machine.
3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your control-plane load balancer, on port 6443. (The RKE2 Kubernetes API Server uses port 6443, while the Rancher server will be served via the NGINX Ingress on ports 80 and 443.) Here is an example `rke2.yaml`:
3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your control-plane load balancer, on port 6443. (The RKE2 Kubernetes API Server uses port 6443, while the Rancher server will be served via the Traefik Ingress on ports 80 and 443.) Here is an example `rke2.yaml`:
```yml
apiVersion: v1
@@ -140,7 +140,8 @@ kube-system etcd-rke2-server-2 1/1 Ru
kube-system etcd-rke2-server-3 1/1 Running 0 56s
kube-system helm-install-rke2-canal-hs6sx 0/1 Completed 0 2m17s
kube-system helm-install-rke2-coredns-xmzm8 0/1 Completed 0 2m17s
kube-system helm-install-rke2-ingress-nginx-flwnl 0/1 Completed 0 2m17s
kube-system helm-install-traefik-crd-z8vsz 0/1 Completed 0 2m17s
kube-system helm-install-traefik-flwnl 0/1 Completed 0 2m17s
kube-system helm-install-rke2-metrics-server-7sggn 0/1 Completed 0 2m17s
kube-system kube-apiserver-rke2-server-1 1/1 Running 0 116s
kube-system kube-apiserver-rke2-server-2 1/1 Running 0 66s
@@ -160,9 +161,8 @@ kube-system rke2-canal-swfmq 2/2 Ru
kube-system rke2-coredns-rke2-coredns-547d5499cb-6tvwb 1/1 Running 0 92s
kube-system rke2-coredns-rke2-coredns-547d5499cb-rdttj 1/1 Running 0 2m8s
kube-system rke2-coredns-rke2-coredns-autoscaler-65c9bb465d-85sq5 1/1 Running 0 2m8s
kube-system rke2-ingress-nginx-controller-69qxc 1/1 Running 0 52s
kube-system rke2-ingress-nginx-controller-7hprp 1/1 Running 0 52s
kube-system rke2-ingress-nginx-controller-x658h 1/1 Running 0 52s
kube-system traefik-7c844b766f-m9p2w 1/1 Running 0 52s
kube-system traefik-7c844b766f-s4l9k 1/1 Running 0 52s
kube-system rke2-metrics-server-6564db4569-vdfkn 1/1 Running 0 66s
```
@@ -40,6 +40,4 @@ You must have an SSL certificate that Ingress can use to encrypt and decrypt com
## Labels and Annotations
Add [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) and/or [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to provide metadata for your Ingress controller.
For a list of annotations available for use, see the [Nginx Ingress Controller Documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/).
Please refer to the Traefik documentation for the full list of Ingress NGINX annotations that are [supported](https://doc.traefik.io/traefik/reference/routing-configuration/kubernetes/ingress-nginx/#annotations-support) and [unsupported](https://doc.traefik.io/traefik/reference/routing-configuration/kubernetes/ingress-nginx/#unsupported-annotations) by Traefik's kubernetesIngressNginx provider.
@@ -15,6 +15,14 @@ Layer-4 load balancer (or the external load balancer) forwards traffic to Nodepo
Often, the Layer-4 load balancer is supported by the underlying cloud provider, so when you deploy RKE clusters on bare-metal servers and vSphere clusters, Layer-4 load balancer is not supported. However, a single [globally managed config-map](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) can be used to expose services on NGINX or third-party ingress.
:::warning
**Ingress-NGINX EOL:** The community `ingress-nginx` controller reaches End-of-Life (EOL) in March 2026. Traefik is the recommended migration path for Rancher environments.
Traefik includes a native Ingress NGINX provider. This allows you to migrate from NGINX without rewriting your existing Ingress objects, as Traefik will automatically interpret `nginx.ingress.kubernetes.io` annotations. If you are upgrading a cluster that is already using `ingress-nginx`, follow this [guide](https://doc.traefik.io/traefik/migrate/nginx-to-traefik/) for more information.
:::
:::note
It is possible to deploy a cluster with a non-cloud load balancer, such as [MetalLB.](https://metallb.universe.tf/) However, that use case is more advanced than the Layer-4 load balancer supported by a cloud provider, and it is not configurable in Rancher or RKE.
@@ -36,13 +36,21 @@ Load Balancers have a couple of limitations you should be aware of:
## Ingress
:::warning
**Ingress-NGINX EOL:** The community `ingress-nginx` controller reaches End-of-Life (EOL) in March 2026. Traefik is the recommended migration path for Rancher environments.
Traefik includes a native Ingress NGINX provider. This allows you to migrate from NGINX without rewriting your existing Ingress objects, as Traefik will automatically interpret `nginx.ingress.kubernetes.io` annotations. If you are upgrading a cluster that is already using `ingress-nginx`, follow this [guide](https://doc.traefik.io/traefik/migrate/nginx-to-traefik/) for more information.
:::
As mentioned in the limitations above, the disadvantages of using a load balancer are:
- Load Balancers can only handle one IP address per service.
- If you run multiple services in your cluster, you must have a load balancer for each service.
- It can be expensive to have a load balancer for every service.
In contrast, when an ingress is used as the entrypoint into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service.
In contrast, when an ingress is used as the entry point into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service.
Therefore, it is useful to have an ingress if you want multiple services to be exposed with the same IP address, the same Layer 7 protocol, or the same privileged node-ports: 80 and 443.