Commit Graph

647 Commits

Author SHA1 Message Date
Peter Štibraný ad571b50e9 Create context with deadline inside goroutine. (#110297) 2025-08-28 15:33:34 +00:00
Peter Štibraný 7a8010be0c search-after-write: improve observability (#110288)
* Improve tracing and observability around updating of index during search.
2025-08-28 16:02:47 +02:00
maicon 12382d7f20 unistore/apistore: improve package docs (#110242)
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
2025-08-27 19:00:05 -03:00
owensmallwood 699bffc9e4 Unified Storage: Dont need to test casing with ordering (#110245)
Dont need to test casing with ordering.
2025-08-27 14:06:43 -06:00
Will Assis 7921c7da23 unified-storage: reuse index if RV is set even if it is outdated (#110184)
* unified-storage: reuse index if RV is set even if it is outdated and searchAfterWrite FF is enabled
2025-08-27 11:10:54 -04:00
Matheus Macabu 85c567609d Secrets: Add result label for decryption errors (#110213) 2025-08-27 14:09:43 +02:00
owensmallwood b4ed217656 Unified Storage: Removes check for name ordering and adds test (#110183)
removes check for name ordering. Adds test to assert expected name and rv ordering.
2025-08-26 12:39:30 -06:00
Mustafa Sencer Özcan 3056924d10 fix: enforce timeout for requests run in qos (#110162) 2025-08-26 18:40:22 +02:00
Matheus Macabu 78ac555c4d CI: Run gofmt on Linting action (#110159)
* CI: Run gofmt on Linting action

* Chore: Run gofmt on all packages
2025-08-26 16:14:03 +02:00
Peter Štibraný 067b6e20a2 Fix name of the summary, remove _total suffix (#110143) 2025-08-26 08:48:05 +00:00
owensmallwood bacb5c576c Unified Storage: Fix yield statement (#110134)
fix yield
2025-08-25 20:06:24 -06:00
Ryan McKinley 27c92a6644 Dashboards: Make conversion data optional (#109965) 2025-08-25 13:47:42 +03:00
Stephanie Hingtgen 1091054c25 Unistore: Wire up inline secure values (#110072)
* Unistore: Wire up inline secure values

* add validation and test

* linter
2025-08-25 02:48:49 -06:00
Peter Štibraný 4c996a8a7f search: Guarantee search-after-write consistency (#109972) 2025-08-25 10:13:07 +02:00
lean.dev 21fc58f632 SecretsManager: fix span name for SetVersion operations (#110035)
SecretsManager: fix span name for SetVersion operations
2025-08-22 12:01:41 +00:00
owensmallwood cace999671 Unified Storage: Add ListSinceModified to StorageBackend (#109697)
* WIP added ListSinceModified to StorageBackend interface

* fix compile time check

* Fix method name

* Fix naming

* fix the rest of the ListSinceModified names

* Uses resource key without name field

* get latest rv from resource_version. Update test.

* adds moar tests

* adds method stub for ListModifiedSince to other StorageBackend implementations

* adds dummy impl to noop storage backend for ListModifiedSince

* skip tests for badger kv backend for now

* fixes tests and adds badgerkv impl for ListModifiedSince

* add badger kv impl

* adds test for new query

* adds test data for new query

* adds ListModifiedSince stub to mockStorageBackend

* uncomment tests

* refactors ListModifiedSince to return an iter.seq2 and handles deduplication. Updates tests. Updates query result sorting.

* remove comments

* remove folder from query (dont need it, yet?)

* regen test queries

* updates test

* updates function comment

* use resourcepb.ResourceKey instead of ModifiedResourceKey

* wrap seq in single transaction. Rollback transaction after 30s if iterator never used. Only track last seen event. Formatting.

* skip TestListModifiedSince for kv backend

* use WatchEvent_Type for action type

* remove redundant fields from order by clause and regen test data for query

* remove redundant fields from order by clause and regen test data for query
2025-08-20 11:54:31 -06:00
Will Assis 34496e137c unified-storage: Search after write save rv to index (#109641)
* save rv to index after index is built
2025-08-19 09:41:35 -04:00
Matheus Macabu ea833ddf1f Unified/APIStore: Fix flaky tests on secure value support inline (#109806) 2025-08-19 09:49:44 +02:00
Peter Štibraný c508b01cc5 chore: Close bleve indexes created in temp dir. (#109799)
* Close indexes created in temp dir.
2025-08-18 16:27:57 +02:00
Matheus Macabu 10508d0614 Secret/Keepers: Return already exists error from DB when creating keeper (#109782) 2025-08-18 14:42:55 +02:00
maicon 5270d8e53f fix(unified-storage): remove permissions after executing on both stores (#109722)
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
2025-08-15 15:57:42 +00:00
Ryan McKinley 5007f362f3 Chore: fix flakey secure value test (#109724) 2025-08-15 07:58:50 +00:00
Will Assis 43cda1f1a4 unified-storage: disable watcher when search-after-write feature flag is enabled (#109634)
* disable indexer watcher when search-after-write feature flag is enabled
2025-08-14 10:18:09 -04:00
Ryan McKinley d3df5b8ddd Secrets: Manage secure values inside any resource (#107803) 2025-08-14 12:31:24 +00:00
Matheus Macabu dfae5e5b4d Secrets: Add namespace matches checks to authorizer and secure value client (#109651)
* Decrypt: Add namespace matches to authorizer

* SecureValueClient: Add namespace matches when auth checking
2025-08-14 11:50:56 +02:00
lean.dev 3106abf1a6 SecretsManager: clear unused metrics and standarize labels(#109515) 2025-08-13 14:01:50 +01:00
Mustafa Sencer Özcan 7b4fcd486a Revert "fix: unified resource server list queries order column" (#109529) 2025-08-12 17:29:50 +02:00
Peter Štibraný 298b492f1a search queue index mutex (#109474)
Use custom mutex for index, and don't hold it during BulkIndex.
2025-08-11 16:20:47 +00:00
Ryan McKinley e0404f924c K8s/SecureValues: Wire InlineSecureValueSupport to apistore (#109449)
* inline wire

* extra fields

* add variable

* wire
2025-08-11 15:22:56 +03:00
William Wernert 6851d8ae0c Storage: Skip slow test TestIntegrationBenchmarkResourceServer (#109424) 2025-08-08 20:51:12 +00:00
owensmallwood c07c0f27d2 Unified Storage: Makes writing and reading index thread-safe in the queue processor (#109420)
makes writing and reading index thread-safe
2025-08-08 12:53:01 -06:00
Peter Štibraný 9b0dc4614c Cleanup old indexes in the same goroutine as build of the index. (#109382)
* Cleanup old indexes in the same goroutine as build of the index.

* Add test.
2025-08-08 14:59:59 +02:00
maicon a5abc6727f Unistore/dualwriter: delegate SetDefaultPermissions to Unified on Mode3 (#109308)
* Unistore/dualwriter: delegate SetDefaultPermissions to Unified on Mode3

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
2025-08-07 18:41:41 -03:00
owensmallwood e667dfed29 Unified Storage: Update index when indexqueueprocessor exists (#109351)
* Update readme

* When queue processor already exists, update the index on it in case it has been closed.

* Adds regression test.
2025-08-07 11:38:24 -06:00
Ryan McKinley ace670c6d0 Secrets: Support variadic args in decrypt service (#109280) 2025-08-07 09:54:24 +00:00
Stephanie Hingtgen 68059344f8 Unified storage: Remove unifiedStorageSearchPermissionFiltering feature flag (#109216)
* Unified storage: Remove unifiedStorageSearchPermissionFiltering ff

* unit test
2025-08-06 02:04:32 -06:00
Bruno Abrantes f3b8a891af chore: use native histogram for search shadow traffic match percentage (#109189)
Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-08-06 08:56:59 +02:00
Bruno Abrantes cb921dc47a feat: compare legacy and unified search results via histogram (#109022)
* feat: compare legacy and unified search results via histogram

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: handle cases where request type is not set

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: use struct instead of bool because it's more memory efficient

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: calculate recall percentage rather than union between legacy and unified

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

---------

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-08-05 11:17:32 +02:00
Matheus Macabu 81f544e186 Secrets: Add service name as explicit parameter for Decrypt (#109054)
* Secrets: Add service name as explicit parameter for Decrypt

* Apply suggestions from code review

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-08-04 08:51:15 +02:00
Stephanie Hingtgen 172a69da75 Feature toggle: Cleanup old ones (#109072) 2025-08-01 19:29:42 +00:00
Matheus Macabu 7374df7945 Secrets: Add inline secure value create method (#108987) 2025-08-01 13:57:51 +02:00
lean.dev e022b6593b SecretsManager: fix log (#108970) 2025-07-31 11:51:04 +00:00
Peter Štibraný 372f115db9 Finish building index even if original context is canceled. (#108969) 2025-07-31 13:45:55 +02:00
Serge Zaitsev 6b1143565a Chore: Make cgo optional (for sqlite) (#108756)
* make cgo optional for sqlite

* update go.mod; check error code differently

* reduce api surface even more

* move test errors into sqlite package

* add a comment
2025-07-31 09:25:19 +00:00
Matheus Macabu 80d7892d6a Secrets: Save owner reference fields in secure value db table (#108905)
* Secrets: Save owner reference fields in secure value db table

* Save api group and version separately
2025-07-31 10:42:19 +02:00
Bruno Abrantes 91e5e8019d chore: add documentation on unified search (#108847)
* chore: add documentation on unified search

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: add additional needed feature flags for unified search

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: add docs about index_min_count and index_max_count

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: add documentation about sortable fields and the discrepancy with search/sortable

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: kubernetesClientDashboardsFolders feature flag is no more, remove it from the docs

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: simplify request flow diagrams

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

---------

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-07-31 07:18:29 +00:00
lean.dev 6bf542889a SecretsManager: Refactor and clean metrics (#108908) 2025-07-30 20:00:30 +01:00
Peter Štibraný c28b2215e0 Improve indexing observability (#108901)
* Add details to tracing spans when creating index.

* Log reason for building index.

* Log reason for building index.

* Remove initialization of labels to avoid unnecessary metrics.

* Track succcessful, failed and skiped index builds. Track index build time for individual index, not all indexes.

* Revert removal of labels initialization.
2025-07-30 16:34:15 +02:00
Stephanie Hingtgen 1f025fe1a3 K8s: Remove kubernetesClientDashboardsFolders feature flag (#108626) 2025-07-29 16:52:57 -05:00
Peter Štibraný 41319f90bb search: Handle index build errors gracefully (#108862)
* Close new index if we fail to build it.

* Respect context cancellation in getOrCreateIndex.
2025-07-29 17:40:16 +02:00