Secrets: add better error handling for secret plugin failures when updating datasources (#50542)

* Add protobuf config and generated code, and client wrapper

* wire up loading of secretsmanager plugin, using renderer plugin as a model

* update kvstore provider to check if we should use the grpc plugin. return false always in OSS

* add OSS remote plugin check

* refactor wire gen file

* log which secrets manager is being used

* Fix argument types for remote checker

* Turns out if err != nil, then the result is always nil. Return empty values if there is an error.

* remove duplicate import

* ensure atomicity by adding secret management as a step to sql operations and rolling back if necessary

* Update pkg/services/secrets/kvstore/kvstore.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/services/secrets/kvstore/kvstore.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* refactor RemotePluginCheck interface to just return the Plugin client directly

* rename struct to something less silly

* add special error handling for remote secrets management

* switch to errors.as instead of type inference

* remove unnecessary rollback call

* just declare error once

* refactor .proto file according to prior PR suggestions

* re-generate protobuf files and fix compilation errors

* only wrap (ergo display in the front end) errors that are user friendly from the plugin

* rename error type to suggest user friendly only

* rename plugin functions to be more descriptive

* change delete message name

* Revert "change delete message name"

This reverts commit 8ca978301e.

* Revert "rename plugin functions to be more descriptive"

This reverts commit 4355c9b9ff.

* fix pointer to pointer problem

* change plugin user error to just hold a string

* fix sequencing problem with datasource updates

* clean up some return statements

* need to wrap multiple transactions with the InTransaction() func in order to keep the lock

* make linter happy

* revert input var name

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

pkg/models/datasource.go

@@ -81,6 +81,15 @@ func (ds DataSource) AllowedCookies() []string {
 	return []string{}
 }
 
+// Specific error type for grpc secrets management so that we can show more detailed plugin errors to users
+type ErrDatasourceSecretsPluginUserFriendly struct {
+	Err string
+}
+
+func (e ErrDatasourceSecretsPluginUserFriendly) Error() string {
+	return e.Err
+}
+
 // ----------------------
 // COMMANDS
 
@@ -104,6 +113,7 @@ type AddDataSourceCommand struct {
 	UserId                  int64             `json:"-"`
 	ReadOnly                bool              `json:"-"`
 	EncryptedSecureJsonData map[string][]byte `json:"-"`
+	UpdateSecretFn          UpdateSecretFn    `json:"-"`
 
 	Result *DataSource `json:"-"`
 }
@@ -129,6 +139,7 @@ type UpdateDataSourceCommand struct {
 	Id                      int64             `json:"-"`
 	ReadOnly                bool              `json:"-"`
 	EncryptedSecureJsonData map[string][]byte `json:"-"`
+	UpdateSecretFn          UpdateSecretFn    `json:"-"`
 
 	Result *DataSource `json:"-"`
 }
@@ -143,8 +154,13 @@ type DeleteDataSourceCommand struct {
 	OrgID int64
 
 	DeletedDatasourcesCount int64
+
+	UpdateSecretFn UpdateSecretFn
 }
 
+// Function for updating secrets along with datasources, to ensure atomicity
+type UpdateSecretFn func() error
+
 // ---------------------
 // QUERIES