mirror of
https://github.com/rancher/rancher-docs.git
synced 2026-05-30 00:25:34 +00:00
Merge pull request #2238 from sunilarjun/update-ingress
Update ingress-nginx pt 1
This commit is contained in:
Sunil Singh
GitHub
+1
-4
@@ -10,10 +10,7 @@ Changing the default TLS settings depends on the chosen installation method.
|
||||
|
||||
## Running Rancher in a highly available Kubernetes cluster
|
||||
|
||||
When you install Rancher inside of a Kubernetes cluster, TLS is offloaded at the cluster's ingress controller. The possible TLS settings depend on the used ingress controller:
|
||||
|
||||
* nginx-ingress-controller (default for RKE1 and RKE2): [Default TLS Version and Ciphers](https://kubernetes.github.io/ingress-nginx/user-guide/tls/#default-tls-version-and-ciphers).
|
||||
* traefik (default for K3s): [TLS Options](https://doc.traefik.io/traefik/https/tls/#tls-options).
|
||||
When you install a Rancher managed Kubernetes cluster, TLS is offloaded at the cluster's ingress controller. Traefik is the default ingress for K3s and can be used with RKE2, refer to [TLS Options](https://doc.traefik.io/traefik/https/tls/#tls-options) for further information.
|
||||
|
||||
## Running Rancher in a single Docker container
|
||||
|
||||
|
||||
+7
-12
@@ -8,14 +8,12 @@ title: Setting up Amazon ELB Network Load Balancer
|
||||
|
||||
This how-to guide describes how to set up a Network Load Balancer (NLB) in Amazon's EC2 service that will direct traffic to multiple instances on EC2.
|
||||
|
||||
These examples show the load balancer being configured to direct traffic to three Rancher server nodes. If Rancher is installed on an RKE Kubernetes cluster, three nodes are required. If Rancher is installed on a K3s Kubernetes cluster, only two nodes are required.
|
||||
These examples show the load balancer being configured to direct traffic to three Rancher server nodes. If Rancher is installed on a K3s Kubernetes cluster, only two nodes are required.
|
||||
|
||||
This tutorial is about one possible way to set up your load balancer, not the only way. Other types of load balancers, such as a Classic Load Balancer or Application Load Balancer, could also direct traffic to the Rancher server nodes.
|
||||
|
||||
Rancher only supports using the Amazon NLB when terminating traffic in `tcp` mode for port 443 rather than `tls` mode. This is due to the fact that the NLB does not inject the correct headers into requests when terminated at the NLB. This means that if you want to use certificates managed by the Amazon Certificate Manager (ACM), you should use an ALB.
|
||||
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
These instructions assume you have already created Linux instances in EC2. The load balancer will direct traffic to these nodes.
|
||||
@@ -26,7 +24,7 @@ Begin by creating two target groups for the **TCP** protocol, one with TCP port
|
||||
|
||||
Your first NLB configuration step is to create two target groups. Technically, only port 443 is needed to access Rancher, but it's convenient to add a listener for port 80, because traffic to port 80 will be automatically redirected to port 443.
|
||||
|
||||
Regardless of whether an NGINX Ingress or Traefik Ingress controller is used, the Ingress should redirect traffic from port 80 to port 443.
|
||||
The Traefik Ingress should redirect traffic from port 80 to port 443.
|
||||
|
||||
1. Log into the [Amazon AWS Console](https://console.aws.amazon.com/ec2/) to get started. Make sure to select the **Region** where your EC2 instances (Linux nodes) are created.
|
||||
1. Select **Services** and choose **EC2**, find the section **Load Balancing** and open **Target Groups**.
|
||||
@@ -34,7 +32,7 @@ Regardless of whether an NGINX Ingress or Traefik Ingress controller is used, th
|
||||
|
||||
:::note
|
||||
|
||||
Health checks are handled differently based on the Ingress. For details, refer to [this section.](#health-check-paths-for-nginx-ingress-and-traefik-ingresses)
|
||||
For details on Traefik Ingress health checks, refer to [this section.](#health-check-paths-for-traefik-ingresses)
|
||||
|
||||
:::
|
||||
|
||||
@@ -167,13 +165,10 @@ After AWS creates the NLB, click **Close**.
|
||||
|
||||
6. Click **Save** in the top right of the screen.
|
||||
|
||||
## Health Check Paths for NGINX Ingress and Traefik Ingresses
|
||||
## Health Check Paths for Traefik Ingresses
|
||||
|
||||
K3s and RKE Kubernetes clusters handle health checks differently because they use different Ingresses by default.
|
||||
K3s Kubernetes clusters use Traefik as the default Ingress.
|
||||
|
||||
For RKE Kubernetes clusters, NGINX Ingress is used by default, whereas for K3s Kubernetes clusters, Traefik is the default Ingress.
|
||||
The health check path is `/ping`. By default `/ping` is always matched (regardless of Host), and a response from [Traefik itself](https://docs.traefik.io/operations/ping/) is always served.
|
||||
|
||||
- **Traefik:** The health check path is `/ping`. By default `/ping` is always matched (regardless of Host), and a response from [Traefik itself](https://docs.traefik.io/operations/ping/) is always served.
|
||||
- **NGINX Ingress:** The default backend of the NGINX Ingress controller has a `/healthz` endpoint. By default `/healthz` is always matched (regardless of Host), and a response from [`ingress-nginx` itself](https://github.com/kubernetes/ingress-nginx/blob/0cbe783f43a9313c9c26136e888324b1ee91a72f/charts/ingress-nginx/values.yaml#L212) is always served.
|
||||
|
||||
To simulate an accurate health check, it is a best practice to use the Host header (Rancher hostname) combined with `/ping` or `/healthz` (for K3s or for RKE clusters, respectively) wherever possible, to get a response from the Rancher Pods, not the Ingress.
|
||||
To simulate an accurate health check, it is a best practice to use the Host header (Rancher hostname) combined with `/ping` or `/healthz` wherever possible, to get a response from the Rancher Pods, not the Ingress.
|
||||
|
||||
+5
-5
@@ -91,7 +91,7 @@ To use this `kubeconfig` file,
|
||||
|
||||
1. Install [kubectl,](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl) a Kubernetes command-line tool.
|
||||
2. Copy the file at `/etc/rancher/rke2/rke2.yaml` and save it to the directory `~/.kube/config` on your local machine.
|
||||
3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your control-plane load balancer, on port 6443. (The RKE2 Kubernetes API Server uses port 6443, while the Rancher server will be served via the NGINX Ingress on ports 80 and 443.) Here is an example `rke2.yaml`:
|
||||
3. In the kubeconfig file, the `server` directive is defined as localhost. Configure the server as the DNS of your control-plane load balancer, on port 6443. (The RKE2 Kubernetes API Server uses port 6443, while the Rancher server will be served via the Traefik Ingress on ports 80 and 443.) Here is an example `rke2.yaml`:
|
||||
|
||||
```yml
|
||||
apiVersion: v1
|
||||
@@ -140,7 +140,8 @@ kube-system etcd-rke2-server-2 1/1 Ru
|
||||
kube-system etcd-rke2-server-3 1/1 Running 0 56s
|
||||
kube-system helm-install-rke2-canal-hs6sx 0/1 Completed 0 2m17s
|
||||
kube-system helm-install-rke2-coredns-xmzm8 0/1 Completed 0 2m17s
|
||||
kube-system helm-install-rke2-ingress-nginx-flwnl 0/1 Completed 0 2m17s
|
||||
kube-system helm-install-traefik-crd-z8vsz 0/1 Completed 0 2m17s
|
||||
kube-system helm-install-traefik-flwnl 0/1 Completed 0 2m17s
|
||||
kube-system helm-install-rke2-metrics-server-7sggn 0/1 Completed 0 2m17s
|
||||
kube-system kube-apiserver-rke2-server-1 1/1 Running 0 116s
|
||||
kube-system kube-apiserver-rke2-server-2 1/1 Running 0 66s
|
||||
@@ -160,9 +161,8 @@ kube-system rke2-canal-swfmq 2/2 Ru
|
||||
kube-system rke2-coredns-rke2-coredns-547d5499cb-6tvwb 1/1 Running 0 92s
|
||||
kube-system rke2-coredns-rke2-coredns-547d5499cb-rdttj 1/1 Running 0 2m8s
|
||||
kube-system rke2-coredns-rke2-coredns-autoscaler-65c9bb465d-85sq5 1/1 Running 0 2m8s
|
||||
kube-system rke2-ingress-nginx-controller-69qxc 1/1 Running 0 52s
|
||||
kube-system rke2-ingress-nginx-controller-7hprp 1/1 Running 0 52s
|
||||
kube-system rke2-ingress-nginx-controller-x658h 1/1 Running 0 52s
|
||||
kube-system traefik-7c844b766f-m9p2w 1/1 Running 0 52s
|
||||
kube-system traefik-7c844b766f-s4l9k 1/1 Running 0 52s
|
||||
kube-system rke2-metrics-server-6564db4569-vdfkn 1/1 Running 0 66s
|
||||
```
|
||||
|
||||
|
||||
+1
-3
@@ -40,6 +40,4 @@ You must have an SSL certificate that Ingress can use to encrypt and decrypt com
|
||||
|
||||
## Labels and Annotations
|
||||
|
||||
Add [Labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) and/or [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to provide metadata for your Ingress controller.
|
||||
|
||||
For a list of annotations available for use, see the [Nginx Ingress Controller Documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/).
|
||||
Please refer to the Traefik documentation for the full list of Ingress NGINX annotations that are [supported](https://doc.traefik.io/traefik/reference/routing-configuration/kubernetes/ingress-nginx/#annotations-support) and [unsupported](https://doc.traefik.io/traefik/reference/routing-configuration/kubernetes/ingress-nginx/#unsupported-annotations) by Traefik's kubernetesIngressNginx provider.
|
||||
|
||||
+8
@@ -15,6 +15,14 @@ Layer-4 load balancer (or the external load balancer) forwards traffic to Nodepo
|
||||
|
||||
Often, the Layer-4 load balancer is supported by the underlying cloud provider, so when you deploy RKE clusters on bare-metal servers and vSphere clusters, Layer-4 load balancer is not supported. However, a single [globally managed config-map](https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/) can be used to expose services on NGINX or third-party ingress.
|
||||
|
||||
:::warning
|
||||
|
||||
**Ingress-NGINX EOL:** The community `ingress-nginx` controller reaches End-of-Life (EOL) in March 2026. Traefik is the recommended migration path for Rancher environments.
|
||||
|
||||
Traefik includes a native Ingress NGINX provider. This allows you to migrate from NGINX without rewriting your existing Ingress objects, as Traefik will automatically interpret `nginx.ingress.kubernetes.io` annotations. If you are upgrading a cluster that is already using `ingress-nginx`, follow this [guide](https://doc.traefik.io/traefik/migrate/nginx-to-traefik/) for more information.
|
||||
|
||||
:::
|
||||
|
||||
:::note
|
||||
|
||||
It is possible to deploy a cluster with a non-cloud load balancer, such as [MetalLB.](https://metallb.universe.tf/) However, that use case is more advanced than the Layer-4 load balancer supported by a cloud provider, and it is not configurable in Rancher or RKE.
|
||||
|
||||
+9
-1
@@ -36,13 +36,21 @@ Load Balancers have a couple of limitations you should be aware of:
|
||||
|
||||
## Ingress
|
||||
|
||||
:::warning
|
||||
|
||||
**Ingress-NGINX EOL:** The community `ingress-nginx` controller reaches End-of-Life (EOL) in March 2026. Traefik is the recommended migration path for Rancher environments.
|
||||
|
||||
Traefik includes a native Ingress NGINX provider. This allows you to migrate from NGINX without rewriting your existing Ingress objects, as Traefik will automatically interpret `nginx.ingress.kubernetes.io` annotations. If you are upgrading a cluster that is already using `ingress-nginx`, follow this [guide](https://doc.traefik.io/traefik/migrate/nginx-to-traefik/) for more information.
|
||||
|
||||
:::
|
||||
|
||||
As mentioned in the limitations above, the disadvantages of using a load balancer are:
|
||||
|
||||
- Load Balancers can only handle one IP address per service.
|
||||
- If you run multiple services in your cluster, you must have a load balancer for each service.
|
||||
- It can be expensive to have a load balancer for every service.
|
||||
|
||||
In contrast, when an ingress is used as the entrypoint into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service.
|
||||
In contrast, when an ingress is used as the entry point into a cluster, the ingress can route traffic to multiple services with greater flexibility. It can map multiple HTTP requests to services without individual IP addresses for each service.
|
||||
|
||||
Therefore, it is useful to have an ingress if you want multiple services to be exposed with the same IP address, the same Layer 7 protocol, or the same privileged node-ports: 80 and 443.
|
||||
|
||||
|
||||
+5
-2
@@ -211,12 +211,13 @@ The following Kubernetes components are directly scraped by Prometheus:
|
||||
|
||||
- kubelet*
|
||||
- ingress-nginx**
|
||||
- Traefik**
|
||||
- coreDns/kubeDns
|
||||
- kube-api-server
|
||||
|
||||
\* You can optionally use `hardenedKubelet.enabled` to use a PushProx, but that is not the default.
|
||||
|
||||
** For RKE and RKE2 clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component.
|
||||
** For RKE clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component. For RKE2 clusters, Traefik is deployed by default and treated as an internal Kubernetes component.
|
||||
|
||||
|
||||
### Scraping Metrics Based on Kubernetes Distribution
|
||||
@@ -233,10 +234,11 @@ Metrics are scraped differently based on the Kubernetes distribution. For help w
|
||||
| kube-proxy | rkeProxy.enabled | rke2Proxy.enabled | kubeAdmProxy.enabled | k3sServer.enabled |
|
||||
| kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet |
|
||||
| ingress-nginx* | Collects metrics directly exposed by kubelet, exposed by rkeIngressNginx.enabled | Collects metrics directly exposed by kubelet, Exposed by rke2IngressNginx.enabled | Not available | Not available |
|
||||
| Traefik* | Collects metrics directly exposed by kubelet | Collects metrics directly exposed by kubelet, Exposed by rke2IngressNginx.enabled | Not available | Not available |
|
||||
| coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns | Collects metrics directly exposed by coreDns/kubeDns |
|
||||
| kube-api-server | Collects metrics directly exposed by kube-api-server |Collects metrics directly exposed by kube-api-server | Collects metrics directly exposed by kube-appi-server | Collects metrics directly exposed by kube-api-server |
|
||||
|
||||
\* For RKE and RKE2 clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component.
|
||||
\* For RKE clusters, ingress-nginx is deployed by default and treated as an internal Kubernetes component. For RKE2 clusters, Traefik is deployed by default and treated as an internal Kubernetes component.
|
||||
|
||||
### Terminology
|
||||
|
||||
@@ -246,5 +248,6 @@ Metrics are scraped differently based on the Kubernetes distribution. For help w
|
||||
- **kube-proxy:** The internal Kubernetes component that watches the API server for pods/services changes in order to maintain the network up to date.
|
||||
- **kubelet:** The internal Kubernetes component that watches the API server for pods on a node and makes sure they are running.
|
||||
- **ingress-nginx:** An Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.
|
||||
- **Traefik:** An Ingress controller for Kubernetes that can be used as a reverse proxy and load balancer.
|
||||
- **coreDns/kubeDns:** The internal Kubernetes component responsible for DNS.
|
||||
- **kube-api-server:** The main internal Kubernetes component that is responsible for exposing APIs for the other master components.
|
||||
|
||||
+12
-12
@@ -78,22 +78,22 @@ kubectl -n kube-system get endpoints kube-scheduler -o jsonpath='{.metadata.anno
|
||||
|
||||
## Ingress Controller
|
||||
|
||||
The default Ingress Controller is NGINX and is deployed as a DaemonSet in the `ingress-nginx` namespace. The pods are only scheduled to nodes with the `worker` role.
|
||||
The default Ingress Controller is Traefik and is deployed as a DaemonSet in the `traefik` namespace. The pods are only scheduled to nodes with the `worker` role.
|
||||
|
||||
Check if the pods are running on all nodes:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx get pods -o wide
|
||||
kubectl -n traefik get pods -o wide
|
||||
```
|
||||
|
||||
Example output:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx get pods -o wide
|
||||
kubectl -n traefik get pods -o wide
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
default-http-backend-797c5bc547-kwwlq 1/1 Running 0 17m x.x.x.x worker-1
|
||||
nginx-ingress-controller-4qd64 1/1 Running 0 14m x.x.x.x worker-1
|
||||
nginx-ingress-controller-8wxhm 1/1 Running 0 13m x.x.x.x worker-0
|
||||
traefik-4qd64 1/1 Running 0 14m x.x.x.x worker-1
|
||||
traefik-8wxhm 1/1 Running 0 13m x.x.x.x worker-0
|
||||
```
|
||||
|
||||
If a pod is unable to run (Status is not **Running**, Ready status is not showing `1/1` or you see a high count of Restarts), check the pod details, logs and namespace events.
|
||||
@@ -101,27 +101,27 @@ If a pod is unable to run (Status is not **Running**, Ready status is not showin
|
||||
### Pod details
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx describe pods -l app=ingress-nginx
|
||||
kubectl -n traefik describe pods -l app=traefik
|
||||
```
|
||||
|
||||
### Pod container logs
|
||||
|
||||
The below command can show the logs of all the pods labeled "app=ingress-nginx", but it will display only 10 lines of log because of the restrictions of the `kubectl logs` command. Refer to `--tail` of `kubectl logs -h` for more information.
|
||||
The below command can show the logs of all the pods labeled "app=traefik", but it will display only 10 lines of log because of the restrictions of the `kubectl logs` command. Refer to `--tail` of `kubectl logs -h` for more information.
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx logs -l app=ingress-nginx
|
||||
kubectl -n traefik logs -l app=traefik
|
||||
```
|
||||
|
||||
If the full log is needed, specify the pod name in the trailing command:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx logs <pod name>
|
||||
kubectl -n traefik logs <pod name>
|
||||
```
|
||||
|
||||
### Namespace events
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx get events
|
||||
kubectl -n traefik get events
|
||||
```
|
||||
|
||||
### Debug logging
|
||||
@@ -129,7 +129,7 @@ kubectl -n ingress-nginx get events
|
||||
To enable debug logging:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx patch ds nginx-ingress-controller --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--v=5"}]'
|
||||
kubectl -n traefik patch ds traefik --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--v=5"}]'
|
||||
```
|
||||
|
||||
### Check configuration
|
||||
@@ -137,7 +137,7 @@ kubectl -n ingress-nginx patch ds nginx-ingress-controller --type='json' -p='[{"
|
||||
Retrieve generated configuration in each pod:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx get pods -l app=ingress-nginx --no-headers -o custom-columns=.NAME:.metadata.name | while read pod; do kubectl -n ingress-nginx exec $pod -- cat /etc/nginx/nginx.conf; done
|
||||
kubectl -n traefik get pods -l app=traefik --no-headers -o custom-columns=.NAME:.metadata.name | while read pod; do kubectl -n traefik exec $pod -- cat /etc/nginx/nginx.conf; done
|
||||
```
|
||||
|
||||
## Rancher agents
|
||||
|
||||
+1
-1
@@ -69,7 +69,7 @@ rancher rancher.yourdomain.com x.x.x.x,x.x.x.x,x.x.x.x 80, 443 2m
|
||||
When accessing your configured Rancher FQDN does not show you the UI, check the ingress controller logging to see what happens when you try to access Rancher:
|
||||
|
||||
```
|
||||
kubectl -n ingress-nginx logs -l app=ingress-nginx
|
||||
kubectl -n traefik logs -l app=traefik
|
||||
```
|
||||
|
||||
## Leader Election
|
||||
|
||||
Reference in New Issue
Block a user