9d2eadcfd2
PoC: replace using testing.Short in storage/unified package with integration test skip method ( #107887 )
...
* PoC: check if testing.Short is called from integration tests only.
* Rename helper function.
* Fix logic.
* Remove skipping of integration tests from non-integration tests.
* Remove skipping of integration tests from non-integration tests.
* Fix import.
2025-07-09 15:32:10 +00:00
8283d35e56
Secrets: make operations sync ( #107732 )
...
* Secrets: make operations sync
* k8s gen / update query to list secure values to include the version
* always store new version of a secret
* make update-workspace
* go mod tidy
* update queries
* update queries
* improve and use testutils in decrypt_store_test
* fix broken test
* make update-workspace
* ./hack/update-codegen.sh secret
* update Test_SecureValueMetadataStorage_CreateAndRead
* undo dependency changes
* linter: fix remaining errors
---------
Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
2025-07-09 10:43:34 -03:00
3f502f305d
Chore: Update mocks with recent mockery ( #107816 )
2025-07-09 09:15:34 +02:00
7787ff6b2e
unistore: add more kvstore tests ( #107692 )
...
add more kvstore tests
2025-07-07 15:00:40 +02:00
650abfd2ff
Unified Storage: Remove distributor todo comment ( #107623 )
...
Remove todo
2025-07-04 12:59:22 +00:00
6bb74ff56a
unistore: add key validation ( #107622 )
2025-07-04 12:52:24 +00:00
46c38fdbb7
SecretsManager: Introduce worker and secret async service ( #107614 )
...
SecretsManager: Introduce worker and secret aysnc service
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
2025-07-04 13:13:48 +01:00
dcb853d309
Unified Search: Remove resource-server-specific methods from distributor ( #107607 )
...
* Remove resource-server-specific methods from distributor
* Remove BlobStoreServer interface implementation from distributor
2025-07-04 12:48:27 +02:00
15e1aa8855
SecretsManager: Introduce decrypt store ( #107586 )
...
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com >
2025-07-04 09:22:10 +01:00
a59ec345c2
SecretsManager: Introduce metrics and logs ( #107582 )
...
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
2025-07-03 17:32:18 +01:00
cfd3b9f582
SecretsManager: outbox use message id ( #107472 )
...
* SecretsManager: outbox use message id
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com >
* Remove query timestamp
* Add missing query
---------
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com >
2025-07-03 15:21:47 +01:00
8d8b824f73
unistore: skipping badger test failing atm ( #107572 )
...
skipping badger test failing atm
2025-07-03 12:31:07 +00:00
4d8678c7f2
SecretsManager: Add base encryption manager ( #107562 )
...
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
2025-07-03 11:29:14 +01:00
b7153d4d20
fix: grpc resource delete error when qos enabled ( #107560 )
2025-07-03 12:02:05 +02:00
041c343a86
Unified storage: Respect GF_DATABASE_URL override ( #105331 )
...
* Database for unified storage resources now reuses DB code that respects URL override.
Access instrument_queries via section getter.
2025-07-03 09:57:40 +00:00
e076c74869
sqltemplate, dbimpl: Remove single-method function types ( #107525 )
...
* Remove dbProviderFunc function.
This removes one extra indirection that made the code bit more difficult to navigate.
* Remove indirection function types implementing single-method interfaces.
This streamlines the code and makes it bit easier to navigate.
* Update pkg/storage/unified/sql/sqltemplate/dialect_mysql.go
Co-authored-by: Mustafa Sencer Özcan <32759850+mustafasencer@users.noreply.github.com >
---------
Co-authored-by: Mustafa Sencer Özcan <32759850+mustafasencer@users.noreply.github.com >
2025-07-03 10:38:12 +02:00
f32d944b23
Secrets: Add initial tracing instrumentation ( #107513 )
2025-07-02 14:43:36 +02:00
d55541735a
unistore: use different ns for each test ( #107509 )
...
use different ns for each test
2025-07-02 13:50:30 +02:00
696657bdd1
unistore: add kv based storage backend ( #107305 )
...
* Add datastore
* too many slashes
* lint
* add metadata store
* simplify meta
* Add eventstore
* golint
* lint
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
* readcloser
* remove get prefix
* use dedicated keys
* parsekey
* sameresource
* unrelated
* name
* renmae tests
* add key validation
* fix tests
* refactor a bit
* lint
* allow empty ns
* get keys instead of list
* rename the functions
* refactor yield candidate
* update test
* unistore: add LastResourceVersion to datastore
* lint
* use map string
* missing err check
* fix
* Add storage backend
* remove hasmore
* fix tests
* small refactor
* pre-alloc
* extract the folder
* lint
* refactor
* handle context canceled in ListHistory to pass the tests
* fix the resource test
* unistore: provide generic tests for the kv interface (#107443 )
unistore: move the kv tests to the testing package
* Update pkg/storage/unified/resource/storage_backend_test.go
Co-authored-by: Peter Štibraný <pstibrany@gmail.com >
* address comments
* comments
* comments
* comments
* normalise the names and add helper method
* events comments
* rename function
---------
Co-authored-by: Peter Štibraný <pstibrany@gmail.com >
2025-07-02 10:57:37 +00:00
3674d952a6
fix: use database section for resource server db engine configuration ( #107495 )
2025-07-02 09:00:51 +00:00
01c844b69f
SecretsManager: Revert adding data key tracer ( #107499 )
...
Remove data key tracer
2025-07-02 09:09:12 +01:00
fe7ca61a51
unified-storage: Missing rename of index_server to search_server ( #107469 )
...
* rename
2025-07-01 14:35:46 -04:00
f09e85c048
unified-storage: Distributor rename to better reflect that it'll be used for search ( #107409 )
...
* rename distributor/ring references to "storage-api" to "search-server"
2025-07-01 11:15:10 -04:00
9a92900ef3
fix(unified-storage): remove 's' in config parameter ( #107457 )
2025-07-01 14:06:21 +02:00
d76e55371a
fix(unified-storage): use the provided connection config parameters ( #107455 )
...
* fix(unified-storage): use the provided connection config parameters
* extend tests
* make update-workspace
2025-07-01 11:25:16 +00:00
974a2c47f9
feat(unified-storage): add qos support for the resource server ( #105939 )
2025-07-01 09:22:55 +00:00
0fccc01ebe
SecretsManager: add data key store ( #107396 )
...
* SecretsManager: Add data key store
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com >
* SecretsManager: Add wiring of data key store
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com >
---------
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com >
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com >
2025-06-30 17:17:07 +01:00
46094a16c7
unistore: add LastResourceVersion to the datastore ( #107303 )
...
* Add datastore
* too many slashes
* lint
* add metadata store
* simplify meta
* Add eventstore
* golint
* lint
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
* readcloser
* remove get prefix
* use dedicated keys
* parsekey
* sameresource
* unrelated
* name
* renmae tests
* add key validation
* fix tests
* refactor a bit
* lint
* allow empty ns
* get keys instead of list
* rename the functions
* refactor yield candidate
* update test
* unistore: add LastResourceVersion to datastore
* lint
* use map string
* missing err check
* fix
2025-06-30 15:49:26 +02:00
7aad041f70
unified-storage: Add integration tests for distributor ( #105771 )
...
* Add integration tests for distributor
2025-06-30 09:31:58 -04:00
7940da4803
Fix organization deletion error messages ( #107380 )
...
* Fix organization deletion error messages
- Improve error message clarity when attempting to delete active organization
- Fix incorrect 'Failed to update organization' message to 'Failed to delete organization'
- Update comment to be more precise about the check being performed
Fixes #92792
* Improve error handling in organization deletion service
- Add contextual error message when dashboard deletion fails during org deletion
- Include organization ID in error message for better debugging
- Import fmt package for error formatting
This helps administrators understand which specific organization failed
during the deletion process when dashboard cleanup encounters issues.
* Fix typo in unified resource access error message
Change 'Namespace missmatch' to 'Namespace mismatch' for correct spelling.
2025-06-30 13:58:37 +02:00
7ebea688ef
fix(unified-storage): return empty list instead of nil ( #107384 )
2025-06-30 13:53:58 +02:00
4a272fb61b
unistore: add eventstore and notifier ( #107182 )
...
* Add datastore
* too many slashes
* lint
* add metadata store
* simplify meta
* Add eventstore
* golint
* lint
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
* readcloser
* remove get prefix
* use dedicated keys
* parsekey
* sameresource
* unrelated
* name
* renmae tests
* add key validation
* fix tests
* refactor a bit
* lint
* allow empty ns
* get keys instead of list
* rename the functions
* refactor yield candidate
* update test
* lint
* missing err check
* address comments
* increase the timeout
2025-06-30 11:20:57 +02:00
f66a693438
Chore: Rename integration tests to follow the common convention ( #105987 )
...
* automatically rename integration tests to follow the common convention
* name tests differently
* alter column type to bigint
* update another column to bigint
* add another alter
* fix subquery for mysql
2025-06-29 16:56:24 +02:00
930c9b3beb
fix: double registration panic in auth fallback ( #107330 )
...
fix double registration panic in auth fallback
2025-06-27 19:22:16 +02:00
6f3b619f59
Remove use of prometheus.DefaultRegisterer from ProvideUnifiedStorageGrpcService. ( #107315 )
2025-06-27 15:58:00 +02:00
f4ee58db50
unified-storage: split resource index client ( #106297 )
...
* split resource server and index server grpc connection if defined in config
2025-06-27 08:15:52 -04:00
0982cfd9a0
Unified Storage/Search: Add max count config for indexing ( #107255 )
...
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold
* Refactor mockResourceIndex
* Test setting of indexing threshold configs
* Tweak comments, log
* Fix logging in buildEmptyIndex
* Export and reuse TestDocumentBuilderSupplier
* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
b2aaa9b3ce
kvstore: ensure we return notFound on delete ( #107288 )
...
* kvstore: ensure we return notFound on delete
* fix tests
2025-06-27 12:00:03 +02:00
2e9559a376
Include resource name in the event. ( #107185 )
2025-06-27 09:29:49 +00:00
53ec379af8
unistore: add metadata store ( #107151 )
...
* Add datastore
* too many slashes
* lint
* add metadata store
* simplify meta
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
* readcloser
* remove get prefix
* use dedicated keys
* parsekey
* sameresource
* unrelated
* name
* renmae tests
* fix tests
* lint
* allow empty ns
* get keys instead of list
* rename the functions
* refactor yield candidate
2025-06-27 08:25:27 +00:00
dc79e6f3dc
unified-storage: fix auth handling in distributor ( #106062 )
...
* disable auth handling in the distributor
2025-06-26 12:32:59 -04:00
2297417374
Unistore: Add datastore ( #107137 )
...
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
2025-06-26 10:36:31 +02:00
79fe8a9902
Library Panels: Modify connection api endpoint to be compatible with unified storage ( #107088 )
2025-06-25 22:21:56 +00:00
eeb01126b3
grpc: wrapContext should not replace existing metadata, but append to it. ( #107126 )
...
* grpc: wrapContext should not replace existing metadata, but append to it.
* Add test for wrapContext if there's no previous metadata.
* Rename encodeIdentityInMetadata to encodeIdentityInMetadataPairs
2025-06-25 08:06:45 +00:00
580b410e8c
DualWriter: Minor cleanup ( #107028 )
2025-06-25 07:41:51 +00:00
5150ebfba8
unistore: index search references ( #106954 )
2025-06-24 14:54:07 -05:00
53cb80e58c
Unistore: save should use a Reader not ReadCloser ( #107128 )
...
* save should use a Reader not ReadCloser
* save should use a Reader not ReadCloser
2025-06-24 15:37:22 +00:00
27a5bab1c1
unistore: Add kvstore interface ( #107094 )
...
* Add kvstore interface
* add owner
* go lint
* remove comment
* update comment
* remove GetOptions
* add sortorder unspecified
* nit
* nit
* nit
* move txn
* use io.reader
* use io.reader
* change again the default order + comments
* change again the default order + comments
* use readcloser for Save
2025-06-24 15:50:57 +02:00
6e5056e1be
Add tracing to bulk process. ( #107047 )
...
Add logging to bulk process.
2025-06-24 09:45:32 +01:00
9062d88ea0
unistore: filter trash requests ( #106767 )
...
* deleted by user
* use the correct checker
* add tests
* refactor
2025-06-24 10:17:34 +02:00
Peter Štibraný
Bruno
Ryan McKinley
Georges Chaudy
Arati R.
Dana Axinte
Mustafa Sencer Özcan
Matheus Macabu
Will Assis
Jean-Philippe Quéméner
Jo
Serge Zaitsev
Stephanie Hingtgen