* Unified Storage: allow rebuilding indexes for resource from a new grpc endpoint
* remove log line
* fix trace def
* lint
* fix after rebase
* addressing code review changes
* update with one channel per rebuild request
* other review suggestions
* update with review suggestions
* run mockery generate for MockResourceClient
* update tests
* update tests and lint
* fix test
* passes nsr to GetResourceStats instead of just namespace
* removes ns check
* fixes failing tests
* make update-workspace
* pass group and resource from rebuild request when getting resource stats
* Use timestamps reported via GetResourceLastImportTimes to trigger index rebuilds.
* Add test for old last import time.
* Don't reindex after bulk-import. It is now done indirectly via LastImportTime on all instances that own the index.
* Modify index eviction mechanism such that unowned indexes are also evicted.
* Propagate OwnsIndex function to bleve backend
Fix tests.
Stop eviction goroutine when stopping backend.
Make linter happy.
Make sure we stop backend created by tests.
Review suggestion.
Removed newline.
* Remove support for initMinSize.
Remove support for searchAfterWrite option, now it defaults to true.
* Remove reference to deprecated feature toggle.
* Remove feature toggle completely.
* Remove code related to indexing on watch events.
* Fix compilation error.
* Remove unused field.
* Add details to tracing spans when creating index.
* Log reason for building index.
* Log reason for building index.
* Remove initialization of labels to avoid unnecessary metrics.
* Track succcessful, failed and skiped index builds. Track index build time for individual index, not all indexes.
* Revert removal of labels initialization.
* Get ResourceStats before indexing
* Replaced localcache.CacheService to handle expiration faster (localcache.CacheService / gocache.Cache only expires values at specific interval, but we need to close index faster)
* singleflight getOrBuildIndex for the same key
* expire only in-memory indexes
* file-based indexes have new name on each rebuild
* Sanitize file path segments, verify that generated path is within the root dir.
* Add comment and test for cleanOldIndexes.
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold
* Refactor mockResourceIndex
* Test setting of indexing threshold configs
* Tweak comments, log
* Fix logging in buildEmptyIndex
* Export and reuse TestDocumentBuilderSupplier
* Reuse MockResourceIndex
* refactor grafana_index_server_index_size to calculate in a goroutine instead of at scrape time and remove grafana_index_server_indexed_docs metric
* use wire to inject bleve index metrics
* remove sprinkles metrics from bleve index metrics
* log error when trying to calculate file index size and bump interval to 1m instead of 5s
* add support for deleted query param
* support tag query param in modes 2 and below
* handle dashboardIds
* hhandle dashboardUIDs
* handle folderUIDs query param
* handle page query param when hitting legacy storage
* handle sort query param
* handle type query param
* re-enable search fallback
* remove folder search workaround and fix /api/search to return both folders and dashboards when no title or type is provided
---------
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
* adds extra debug logs and a new metric for poller query latency to help us better understand watch and index latency for write events
* adds trace span to the index for handling index write events
Rafael Bortolon Paulovic
João Calisto
Peter Štibraný
owensmallwood
Jean-Philippe Quéméner
Will Assis
Arati R.
Mustafa Sencer Özcan
Stephanie Hingtgen
Georges Chaudy
Ryan McKinley
Igor Suleymanov