Alerting: Send alerts to external Alertmanager(s) (#37298)

* Alerting: Send alerts to external Alertmanager(s)

Within this PR we're adding support for registering or unregistering
sending to a set of external alertmanagers. A few of the things that are
going are:

- Introduce a new table to hold "admin" (either org or global)
  configuration we can change at runtime.
- A new periodic check that polls for this configuration and adjusts the
  "senders" accordingly.
- Introduces a new concept of "senders" that are responsible for
  shipping the alerts to the external Alertmanager(s). In a nutshell,
this is the Prometheus notifier (the one in charge of sending the alert)
mapped to a multi-tenant map.

There are a few code movements here and there but those are minor, I
tried to keep things intact as much as possible so that we could have an
easier diff.

pkg/services/sqlstore/migrations/ualert/tables.go

@@ -19,6 +19,9 @@ func AddTablesMigrations(mg *migrator.Migrator) {
 
 	// Create Alertmanager configurations
 	AddAlertmanagerConfigMigrations(mg)
+
+	// Create Admin Configuration
+	AddAlertAdminConfigMigrations(mg)
 }
 
 // AddAlertDefinitionMigrations should not be modified.
@@ -265,3 +268,23 @@ func AddAlertmanagerConfigMigrations(mg *migrator.Migrator) {
 	mg.AddMigration("alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql", migrator.NewRawSQLMigration("").
 		Mysql("ALTER TABLE alert_configuration MODIFY alertmanager_configuration MEDIUMTEXT;"))
 }
+
+func AddAlertAdminConfigMigrations(mg *migrator.Migrator) {
+	adminConfiguration := migrator.Table{
+		Name: "ngalert_configuration",
+		Columns: []*migrator.Column{
+			{Name: "id", Type: migrator.DB_BigInt, IsPrimaryKey: true, IsAutoIncrement: true},
+			{Name: "org_id", Type: migrator.DB_BigInt, Nullable: false},
+			{Name: "alertmanagers", Type: migrator.DB_Text, Nullable: true},
+
+			{Name: "created_at", Type: migrator.DB_Int, Nullable: false},
+			{Name: "updated_at", Type: migrator.DB_Int, Nullable: false},
+		},
+		Indices: []*migrator.Index{
+			{Cols: []string{"org_id"}, Type: migrator.UniqueIndex},
+		},
+	}
+
+	mg.AddMigration("create_ngalert_configuration_table", migrator.NewAddTableMigration(adminConfiguration))
+	mg.AddMigration("add index in ngalert_configuration on org_id column", migrator.NewAddIndexMigration(adminConfiguration, adminConfiguration.Indices[0]))
+}