Commit Graph

533 Commits

Author SHA1 Message Date
Dana Axinte 0fccc01ebe SecretsManager: add data key store (#107396)
* SecretsManager: Add data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

* SecretsManager: Add wiring of data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-30 17:17:07 +01:00
Georges Chaudy 46094a16c7 unistore: add LastResourceVersion to the datastore (#107303)
* Add datastore

* too many slashes

* lint

* add metadata store

* simplify meta

* Add eventstore

* golint

* lint

* Add datastore

* too many slashes

* lint

* pr comments

* extract ParseKey

* readcloser

* remove get prefix

* use dedicated keys

* parsekey

* sameresource

* unrelated

* name

* renmae tests

* add key validation

* fix tests

* refactor a bit

* lint

* allow empty ns

* get keys instead of list

* rename the functions

* refactor yield candidate

* update test

* unistore: add LastResourceVersion to datastore

* lint

* use map string

* missing err check

* fix
2025-06-30 15:49:26 +02:00
Will Assis 7aad041f70 unified-storage: Add integration tests for distributor (#105771)
* Add integration tests for distributor
2025-06-30 09:31:58 -04:00
Jo 7940da4803 Fix organization deletion error messages (#107380)
* Fix organization deletion error messages

- Improve error message clarity when attempting to delete active organization
- Fix incorrect 'Failed to update organization' message to 'Failed to delete organization'
- Update comment to be more precise about the check being performed

Fixes #92792

* Improve error handling in organization deletion service

- Add contextual error message when dashboard deletion fails during org deletion
- Include organization ID in error message for better debugging
- Import fmt package for error formatting

This helps administrators understand which specific organization failed
during the deletion process when dashboard cleanup encounters issues.

* Fix typo in unified resource access error message

Change 'Namespace missmatch' to 'Namespace mismatch' for correct spelling.
2025-06-30 13:58:37 +02:00
Jean-Philippe Quéméner 7ebea688ef fix(unified-storage): return empty list instead of nil (#107384) 2025-06-30 13:53:58 +02:00
Georges Chaudy 4a272fb61b unistore: add eventstore and notifier (#107182)
* Add datastore

* too many slashes

* lint

* add metadata store

* simplify meta

* Add eventstore

* golint

* lint

* Add datastore

* too many slashes

* lint

* pr comments

* extract ParseKey

* readcloser

* remove get prefix

* use dedicated keys

* parsekey

* sameresource

* unrelated

* name

* renmae tests

* add key validation

* fix tests

* refactor a bit

* lint

* allow empty ns

* get keys instead of list

* rename the functions

* refactor yield candidate

* update test

* lint

* missing err check

* address comments

* increase the timeout
2025-06-30 11:20:57 +02:00
Serge Zaitsev f66a693438 Chore: Rename integration tests to follow the common convention (#105987)
* automatically rename integration tests to follow the common convention

* name tests differently

* alter column type to bigint

* update another column to bigint

* add another alter

* fix subquery for mysql
2025-06-29 16:56:24 +02:00
Will Assis 930c9b3beb fix: double registration panic in auth fallback (#107330)
fix double registration panic in auth fallback
2025-06-27 19:22:16 +02:00
Peter Štibraný 6f3b619f59 Remove use of prometheus.DefaultRegisterer from ProvideUnifiedStorageGrpcService. (#107315) 2025-06-27 15:58:00 +02:00
Will Assis f4ee58db50 unified-storage: split resource index client (#106297)
* split resource server and index server grpc connection if defined in config
2025-06-27 08:15:52 -04:00
Arati R. 0982cfd9a0 Unified Storage/Search: Add max count config for indexing (#107255)
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold

* Refactor mockResourceIndex

* Test setting of indexing threshold configs

* Tweak comments, log

* Fix logging in buildEmptyIndex

* Export and reuse TestDocumentBuilderSupplier

* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
Georges Chaudy b2aaa9b3ce kvstore: ensure we return notFound on delete (#107288)
* kvstore: ensure we return notFound on delete

* fix tests
2025-06-27 12:00:03 +02:00
Peter Štibraný 2e9559a376 Include resource name in the event. (#107185) 2025-06-27 09:29:49 +00:00
Georges Chaudy 53ec379af8 unistore: add metadata store (#107151)
* Add datastore

* too many slashes

* lint

* add metadata store

* simplify meta

* Add datastore

* too many slashes

* lint

* pr comments

* extract ParseKey

* readcloser

* remove get prefix

* use dedicated keys

* parsekey

* sameresource

* unrelated

* name

* renmae tests

* fix tests

* lint

* allow empty ns

* get keys instead of list

* rename the functions

* refactor yield candidate
2025-06-27 08:25:27 +00:00
Will Assis dc79e6f3dc unified-storage: fix auth handling in distributor (#106062)
* disable auth handling in the distributor
2025-06-26 12:32:59 -04:00
Georges Chaudy 2297417374 Unistore: Add datastore (#107137)
* Add datastore

* too many slashes

* lint

* pr comments

* extract ParseKey
2025-06-26 10:36:31 +02:00
Stephanie Hingtgen 79fe8a9902 Library Panels: Modify connection api endpoint to be compatible with unified storage (#107088) 2025-06-25 22:21:56 +00:00
Peter Štibraný eeb01126b3 grpc: wrapContext should not replace existing metadata, but append to it. (#107126)
* grpc: wrapContext should not replace existing metadata, but append to it.

* Add test for wrapContext if there's no previous metadata.

* Rename encodeIdentityInMetadata to encodeIdentityInMetadataPairs
2025-06-25 08:06:45 +00:00
Ryan McKinley 580b410e8c DualWriter: Minor cleanup (#107028) 2025-06-25 07:41:51 +00:00
Georges Chaudy 5150ebfba8 unistore: index search references (#106954) 2025-06-24 14:54:07 -05:00
Georges Chaudy 53cb80e58c Unistore: save should use a Reader not ReadCloser (#107128)
* save should use a Reader not ReadCloser

* save should use a Reader not ReadCloser
2025-06-24 15:37:22 +00:00
Georges Chaudy 27a5bab1c1 unistore: Add kvstore interface (#107094)
* Add kvstore interface

* add owner

* go lint

* remove comment

* update comment

* remove GetOptions

* add sortorder unspecified

* nit

* nit

* nit

* move txn

* use io.reader

* use io.reader

* change again the default order + comments

* change again the default order + comments

* use readcloser for Save
2025-06-24 15:50:57 +02:00
Peter Štibraný 6e5056e1be Add tracing to bulk process. (#107047)
Add logging to bulk process.
2025-06-24 09:45:32 +01:00
Georges Chaudy 9062d88ea0 unistore: filter trash requests (#106767)
* deleted by user

* use the correct checker

* add tests

* refactor
2025-06-24 10:17:34 +02:00
Stephanie Hingtgen 63bc65fa4c Dashboards: Fix history list for dashboard uids that end in - (#107073) 2025-06-22 06:56:45 -06:00
Ryan McKinley 945bc53b4c DualWrite: Manage values from KV store (not file) (#106772) 2025-06-18 10:37:44 +03:00
Mustafa Sencer Özcan b48337a7c8 feat: introduce ttl cache for bleve indices (#106842) 2025-06-17 16:28:51 +02:00
Dana Axinte dbe815ee68 SecretsManager: keepers with secure values credentials (#106761)
* SecretsManager: keepers with secure values

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

* Keepers: Refactor extract secure values remove extra helper functions

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-16 14:37:36 +01:00
Dana Axinte 6097841e67 SecretsManager: add secure value store (#106708)
* SecretsManager: add secure value model and sql templates

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: secure value rest layer to use store

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: temporary add actor prefix to decrypters

* Remove list securevalue by namefor now

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-06-16 10:19:44 +01:00
Jean-Philippe Quéméner 5f21f320f7 fix(unified-storage): use continue token containing both formats for dualwriter (#106525) 2025-06-13 15:59:46 +02:00
Stephanie Hingtgen 5135d5c87d Unified storage: Reconstruct index in the background every 24h (#106422) 2025-06-12 14:34:48 -05:00
Dana Axinte de28231f2f SecretsManager: Add outbox store (#106613)
SecretsManager: add outbox store

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-12 13:31:48 +01:00
Dana Axinte c22b4845bb SecretsManager: Add encrypted value store (#106607)
* SecretsManager: add encrypted value store

Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: wiring of encrypted value store

---------

Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-06-12 11:52:01 +01:00
Stephanie Hingtgen 7864c1660f K8s: Get trash fixes (#106411) 2025-06-11 13:54:38 -05:00
Jean-Philippe Quéméner 1196d66c5e feat(unified-storage): log continue token on error (#106475) 2025-06-10 16:41:39 +02:00
Jean-Philippe Quéméner 1a216a8fff feat(unified-storage): use logger from context for dualwriter (#106473) 2025-06-10 15:33:14 +02:00
maicon 6fb0ed9886 Bump tempo to 20250529124718-87c2dc380cec (#106220)
* Bump tempo to 20250529124718-87c2dc380cec

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* Add owners to pkg/build

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* update workspace

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* update pkg/build/go.sum

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* update pkg/build/go.sum

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
2025-05-29 23:43:13 -05:00
Stephanie Hingtgen ab99211b40 Dashboard Provisioning: Reduce db load (#106114) 2025-05-29 09:39:07 +03:00
Jean-Philippe Quéméner 4519ef8d21 chore(unified-storage): increase background request timeout (#106186) 2025-05-28 22:59:28 +02:00
Ryan McKinley d4c446f42c Chore: Update k8s.io to v0.33.1 (#105307) 2025-05-28 00:13:18 +03:00
Jean-Philippe Quéméner 9ba031364b fix(unified-storage): use batching for index creation (#106034) 2025-05-27 09:48:39 +02:00
Ryan McKinley 326081cdb7 Chore: Remove unified storage go modules (#106035) 2025-05-27 09:49:21 +03:00
Jean-Philippe Quéméner 814707152d chore(unified-storage): contextualize the index build logger (#106025) 2025-05-26 21:33:51 +02:00
Dana Axinte 5401175562 SecretsManager: Conditionally lock DB before migrations using config setting (#106003)
Secrets: Conditionally lock DB before migrations using config setting (#105949)

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-26 18:28:53 +01:00
Georges Chaudy 04d39cbbc6 unistore: split ListIterator and ListHistory in StorageBackend (#105654)
split listIterator from ListHistory
2025-05-23 15:00:18 +02:00
Jean-Philippe Quéméner 392c1a71c9 Storage: Take migration_locking setting into account (#105938) 2025-05-23 14:11:21 +02:00
Peter Štibraný c4d3eb1cd0 Remove support for Google Spanner database. (#105846)
* Remove support for Google Spanner database.
2025-05-23 11:35:59 +02:00
Jean-Philippe Quéméner eb8f3bc071 fix(unified-storage): wait for result to be available (#105922) 2025-05-23 11:03:01 +02:00
Jean-Philippe Quéméner e57be36936 fix(unified-storage): race in resource server watch (#105786) 2025-05-23 09:19:20 +02:00
Jean-Philippe Quéméner 95eabcf03d refactor(unified-storage): rename cache to avoid import conflicts (#105831) 2025-05-22 17:12:31 +02:00