* first pass of adding quotas service resource server
* passes prom reg as param
init quota service as part of server params
* init quota service as part of server params
* adds config and only creates quota service when overrides file path is defined
* when quota service enabled, check quota on create and log result
* update log message
* adds tests for quota service
* adds tests for config reloading when the file changes
* fix linter errors
* fix comment
* use startAndAwaitRunning
* Simplifies quotas service. Call manager.GetConfig() when getting quota instead of watching for changes.
* adds tracing to quotas service
* adds nsr attributes to traces when getting quotas and resource stats
* update comment
* update comment
remove check for nil overrides since it will (should) never happen
* fix linter error
* refactors naming to overrides service
checks quotas in separate function
* fix quotas naming
* fixes more quotas -> overrides naming
* use logger from ctx
* linter - remove trailing whitespace
* log FromContext() when checking quotas
* adds events to spans instead of create new spans
updates tenant -> namespace naming
few other minor fixes
* Modify index eviction mechanism such that unowned indexes are also evicted.
* Propagate OwnsIndex function to bleve backend
Fix tests.
Stop eviction goroutine when stopping backend.
Make linter happy.
Make sure we stop backend created by tests.
Review suggestion.
Removed newline.
* Remove support for initMinSize.
Remove support for searchAfterWrite option, now it defaults to true.
* Remove reference to deprecated feature toggle.
* Remove feature toggle completely.
* Remove code related to indexing on watch events.
* Fix compilation error.
* Remove unused field.
* setup distributor module
* move lifecycler into resource server provider
* remove ring/client pool setup from distributor module and use the same ring/client pool between storage server module and distributor module
* implement resourcestore server methods
* make healthcheck fail if ring is not running
* Updates the instrumentation_server service to use mux instead of the builtin router, and have it store the router in the module server: this is so we can register the /ring endpoint to check the status of the ring
* Create a new Ring service that depends on the instrumentation server and declares it as a dependency for the storage server
* Create standalone MemberlistKV service for Ring service to use
* Update the storage server Search and GetStats handler to distribute requests if applicable
* Use authlib repo. Use otel
* Use interceptors on the provider level
* Create a new wire set with otel
* Lint
* Fix test
* make update-workflow
* make update-workspace
* make update-workspace. Try to add authlib as enterprise imports
* make update-workspace
* refactor grafana_index_server_index_size to calculate in a goroutine instead of at scrape time and remove grafana_index_server_indexed_docs metric
* use wire to inject bleve index metrics
* remove sprinkles metrics from bleve index metrics
* log error when trying to calculate file index size and bump interval to 1m instead of 5s
* move prometheus.register for unified storage metrics into metrics.go and do most of the plumbing to get it to work
* convert StorageApiMetrics to pointer and check for nil before using it
* rename type and variables to something more sensible
---------
Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
* add feature flag
* puts search permission filtering behind a feature flag
* fixes issue with doc match id. When the match is for an in-memory index, the internal id is a string (this is what we expected). However, when its a file-based index, the internal id is a binary encoded int64 that point to something internally. So to get the id, we need to use ExternalID() instead of relying on the indexInternalID to be the correct format.
* adds debug log
* update comment
* formatting
* Remove "wrapper" interface and only check feature toggle for grpc and cloud mode
* Only set name for update checks
* Set dashboard permissions for admin user
* Adds back indexer metrics. Uses config values instead of hardcoded ones.
* cast to int64
* remove unused func
* Index metrics impl doesn't depend on Bleve. Adds a TotalDocs func to SearchBackend interface.
* adds config setting for index_min_count
* rename arg
* rename metric label to namespace instead of slug
* adds default "do nothing" case to satisfy linter
* moves bleve index metrics to search package
* make bleve backend private, dont need to pass in prom reg
* imports
* adds bleve metrics to resource package to avoid circular deps
* WIP - build tenant indexes concurrently
* adds local dev seeders and readme for indexer
* update logging and adds locking in getShard()
* update logs
* Adds Namespaces func. Initializes index after ResourceServer is created.
* fixes Count() and updates test lint issues
* Initialize index separately. Don't do it when creating the resource server. Makes testing really awkward.
* fix lint error
* handles error when getting namespaces
* updates test and index helper funcs
* Add parents field to folder DTO
* Allow subfolder creation when folder flag is enabled
* Update UnstructuredToLegacyFolder
* Include parents field when creating folder
* Reuse MySQL and Postgres Grafana config instead of the object
- Only reuse the Grafana DB object for SQLite. Support for SQLite will be added in a different PR
- Fail when reusing the Grafana DB object if it is using DB instrumentation
- In the case that we have to reuse a Grafana DB with its instrumentation, fail with an error that describes a workaround
- Add regression tests to reproduce incident 2144
* remove temp file
* fix linter
* fix linter x2
* fix linter x3
* adds Filter gRPC and make protobuf
* adds route for querying the filter gRPC
* wires up Filter gRPC call
* [WIP] index from start
* renames gRPC endpoint to "Search"
* adds /apis/search route into k8s routes. Hacky for now.
* updates readme - wrong casing
* adds feature toggle for unified storage search
* hides US search behind feature flag. Clean up print statements.
* removes indexer - will be added in another PR
* Search: Add API Builder
* adds required method
* implementing UpdateAPIGroupInfo (WIP)
* adds groupversion
* commenting out for now
* remove unneeded code from experimenting and update register.go to match interface required
* namespaces search route
---------
Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
