Update the Alerting HA docs (#60292)
* Update the Alerting HA docs * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update HA introduction paragraph * Apply prettier Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
This commit is contained in:
co-authored by
brendamuir
parent
1b860a4074
commit
9941dfc2b6
@@ -15,9 +15,9 @@ weight: 300
|
||||
|
||||
# Enable alerting high availability
|
||||
|
||||
You can enable alerting high availability support by updating the Grafana configuration file. On Kubernetes, you can enable alerting high availability by updating the Kubernetes container definition.
|
||||
You can enable alerting high availability support by updating the Grafana configuration file. If you run Grafana in a Kubernetes cluster, additional steps are required. Both options are described below.
|
||||
|
||||
## Update Grafana configuration file
|
||||
## Enable alerting high availability in Grafana
|
||||
|
||||
### Before you begin
|
||||
|
||||
@@ -31,9 +31,9 @@ Since gossiping of notifications and silences uses both TCP and UDP port `9094`,
|
||||
3. Set `[ha_listen_address]` to the instance IP address using a format of `host:port` (or the [Pod's](https://kubernetes.io/docs/concepts/workloads/pods/) IP in the case of using Kubernetes).
|
||||
By default, it is set to listen to all interfaces (`0.0.0.0`).
|
||||
|
||||
## Update Kubernetes container definition
|
||||
## Enable alerting high availability using Kubernetes
|
||||
|
||||
If you are using Kubernetes, you can expose the pod IP [through an environment variable](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) via the container definition such as:
|
||||
If you are using Kubernetes, you can expose the pod IP [through an environment variable](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) via the container definition.
|
||||
|
||||
```bash
|
||||
env:
|
||||
@@ -42,3 +42,57 @@ env:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
```
|
||||
|
||||
1. Add the port 9094 to the Grafana deployment:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- containerPort: 3000
|
||||
name: http-grafana
|
||||
protocol: TCP
|
||||
- containerPort: 9094
|
||||
name: grafana-alert
|
||||
protocol: TCP
|
||||
```
|
||||
|
||||
2. Add the environment variables to the Grafana deployment:
|
||||
|
||||
```yaml
|
||||
env:
|
||||
- name: POD_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
```
|
||||
|
||||
3. Create a headless service that returns the pod IP instead of the service IP, which is what the `ha_peers` need:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: grafana-alerting
|
||||
namespace: grafana
|
||||
labels:
|
||||
app.kubernetes.io/name: grafana-alerting
|
||||
app.kubernetes.io/part-of: grafana
|
||||
spec:
|
||||
type: ClusterIP
|
||||
clusterIP: 'None'
|
||||
ports:
|
||||
- port: 9094
|
||||
selector:
|
||||
app: grafana
|
||||
```
|
||||
|
||||
4. Make sure your grafana deployment has the label matching the selector, e.g. `app:grafana`.
|
||||
|
||||
5. Add in the grafana.ini:
|
||||
|
||||
```bash
|
||||
[unified_alerting]
|
||||
enabled = true
|
||||
ha_listen_address = "${POD_IP}:9094"
|
||||
ha_peers = "grafana-alerting.grafana:9094"
|
||||
ha_advertise_address = "${POD_IP}:9094"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user