Storage Api: Add metrics (#85316)

* Storage server runs own instrumentation server if its the sole target. Starts adding some sample metrics for now.

* adds metric for failed optimistic locks

* refactors metrics registration to own method on service for testability. Adds tests.

* Register sql storage server metrics from within the service

* fixes test

* troubleshooting drone test failures. Maybe timing when starting instrumentation server?

* Waits until instrumentation server has started. Updates tests.

* defer wont get called unless theres an error. removing.

* wait for instrumentation server to be running

* linter - close res body

* use port 3000 for metrics and removes test metric inc() call

* fixes test - updates port

* refactors module server to provide an instrumentation server module when there is no ALL or CORE target provided and running as single target

* make instrumentation server a dependency of all modules that do not run their own http server

* adds module server test

* adds tests for instrumentation service and removes old tests that aren't needed

* ignore error in test

* uses helper to start and run service

* when running wait on ctx done or http server err

* wait for http server

* removes println

* updates module server test to be integration test

* require no error in goroutine

* skips integration test when GRAFANA_TEST_DB not defined

* move http server start into start, verify returned content

* make test error when run fails

* try waiting longer and see if drone tests pass

* update integration test mysql creds to match drone

* go back to only waiting half second

* debug log drone mysql connection string

* use same db connection config as drone

* try using same hostname as drone

* cant use localhost as mysql hostname in drone tests. Need to parse it from the cfg db connection string

---------

Co-authored-by: Dan Cech <dcech@grafana.com>
This commit is contained in:
owensmallwood
2024-04-08 08:35:01 -06:00
committed by GitHub
co-authored by Dan Cech
parent 3d9a0c8398
commit 37d39de36d
8 changed files with 277 additions and 10 deletions
+9 -2
View File
@@ -10,7 +10,6 @@ import (
"sync"
"github.com/grafana/dskit/services"
"github.com/grafana/grafana/pkg/api"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/modules"
@@ -117,6 +116,14 @@ func (s *ModuleServer) Run() error {
m := modules.New(s.cfg.Target)
// only run the instrumentation server module if were not running a module that already contains an http server
m.RegisterInvisibleModule(modules.InstrumentationServer, func() (services.Service, error) {
if m.IsModuleEnabled(modules.All) || m.IsModuleEnabled(modules.Core) {
return services.NewBasicService(nil, nil, nil).WithName(modules.InstrumentationServer), nil
}
return NewInstrumentationService(s.log)
})
m.RegisterModule(modules.Core, func() (services.Service, error) {
return NewService(s.cfg, s.opts, s.apiOpts)
})
@@ -131,7 +138,7 @@ func (s *ModuleServer) Run() error {
//}
m.RegisterModule(modules.StorageServer, func() (services.Service, error) {
return storageServer.ProvideService(s.cfg, s.features)
return storageServer.ProvideService(s.cfg, s.features, s.log)
})
m.RegisterModule(modules.All, nil)