Commit Graph
50 Commits
Author SHA1 Message Date
Bruno 7698970f22 Secrets: changes to allow a 3rd party keeper / secret references (#115156)
* Secrets: changes to allow a 3rd party keeper / secret references

* fix test

* make gofmt

* lint

* fix tests

* assign aws secrets manager to @grafana/grafana-operator-experience-squad

* rename Keeper.Reference to Keeper.RetrieveReference

* rename ModelSecretsManager to ModelAWSSecretsManager

* validator: ensure that only one of keeper.Spec.Aws.AccessKey or keeper.Spec.Aws.AssumeRole are set

* move secrets manager dep / go mod tidy

* move secrets manager dep

* keeper validator: move 3rd party secret stores validation to their own functions

* add github.com/aws/aws-sdk-go-v2/service/secretsmanager pkg/extensions/enterprise_imports

* make update-workspace

* undo go.mod changes in /apps

* make update-workspace

* fix test

* add github.com/aws/aws-sdk-go-v2/service/secretsmanager to enterprise_imports

* make update-workspace

* gcworker: handle refs

* make update-workspace

* create toggle: FeatureStageExperimental

* allow features.IsEnabled for now

* format
2026-01-06 11:30:04 -03:00
Matheus Macabu 21c1d9aedd Secrets: Remove unused methods and dependencies from secure value service (#114467) 2025-11-26 12:58:00 +01:00
BrunoandMatheus Macabu 0d67442f1a Secrets manager: create secure value using the active keeper (#114039)
* Secrets manager: create secure value using the active keeper

* SecureValueService.Update: fetch secure value from db to get the keeper

* fix keeper_store_test.go

* SecureValueService: fix bug in update where the current version keeper wasn't being passed to the createNewVersion method

* make gofmt

* remove outdated test

* update TestModel

* undo enterprise_imports changes

* use xkube.Namespace

* migrator: set secret_secure_value.keeper to 'system' when the column is null

* indent cue

* fix tests

* fix enterprise imports

* properly fix enterprise imports

* make update-workspace

* go mod tidy

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
2025-11-21 11:20:16 -03:00
Michael Mandrus cf242def3a Secrets: Refactor data_key_id out of the encoded secure value payload (#112077)
* Secrets: Refactor data_key_id out of the encoded secure value payload (#111852)

* everything compiles

* tests pass

* remove file included by accident

* add entry to gitignore

* some scaffolding for the migration executor

* remove file

* implement and test the migration

* use xkube.Namespace in our interfaces

* add todo

* update wire deps

* add some logs

* fix wire dependency ordering

* create tests to validate error conditions during migrations

* only run the migration as an MT api server

* formatting issues

* change detection of secrets running as MT server

* add todo

* use more specific initializer flags

* make secrets playwright tests work

* set new properties to true by default

* remove developer mode flag

* fix unit tests
2025-10-30 23:04:32 -04:00
Stephanie Hingtgen ce50f2cf61 Revert "Secrets: Refactor data_key_id out of the encoded secure value payload" (#112034)
Revert "Secrets: Refactor data_key_id out of the encoded secure value payload…"

This reverts commit acad92864e.
2025-10-05 09:53:43 +03:00
Michael Mandrus acad92864e Secrets: Refactor data_key_id out of the encoded secure value payload (#111852)
* everything compiles

* tests pass

* remove file included by accident

* add entry to gitignore

* some scaffolding for the migration executor

* remove file

* implement and test the migration

* use xkube.Namespace in our interfaces

* add todo

* update wire deps

* add some logs

* fix wire dependency ordering

* create tests to validate error conditions during migrations
2025-10-03 15:25:46 -04:00
Matheus Macabu ffa9444256 Secrets: Add more details about decryption when there is an error (#111741) 2025-09-29 18:29:52 +02:00
e69cc03ef9 Secret: add ability to configure extra owner decrypters (#111301)
---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-09-19 07:41:56 -05:00
f8cd7049e8 Secrets: garbage collection (#110247)
* clean up older secret versions

* start gargbage collection worker as background service

* make gen-go

* fix typo

* make update-workspace

* undo go mod changes

* undo go work sum changes

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* default gc_worker_batch_size to 1 minute

* fix typo

* fix typo

* add test to ensure cleaning up secure values is idempotent

* make gen-go

* make update-workspace

* undo go.mod and .sum changes

* undo enterprise imports

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-09-02 11:11:01 -03:00
Matheus Macabu 1e926a29c0 Secrets: Extract external facing decrypt types to apps (#110432) 2025-09-02 10:30:29 +02:00
Matheus Macabu 537ac8ec68 Secrets: Validate name/namespace with standard K8s validator (#109868)
* Secrets: Validate name/namespace with standard K8s validator

* Secrets: Simplify error message for mismatched owner inline secure values
2025-08-19 16:55:52 +02:00
Matheus Macabu 10508d0614 Secret/Keepers: Return already exists error from DB when creating keeper (#109782) 2025-08-18 14:42:55 +02:00
Matheus Macabu dfae5e5b4d Secrets: Add namespace matches checks to authorizer and secure value client (#109651)
* Decrypt: Add namespace matches to authorizer

* SecureValueClient: Add namespace matches when auth checking
2025-08-14 11:50:56 +02:00
Matheus Macabu 01c1a6ce5b SecureValues: Run mutate and validation hooks in service layer (#109379)
* SecureValues: Run mutate and validation hooks in service layer

* add some unit tests
2025-08-08 13:15:23 +02:00
Ryan McKinley ace670c6d0 Secrets: Support variadic args in decrypt service (#109280) 2025-08-07 09:54:24 +00:00
Matheus Macabu b0b1340c19 Secrets: Remove Close from decrypt client interface (#109250) 2025-08-06 14:02:13 +02:00
Matheus Macabu c9daccfd28 Secrets: Update inline delete when owned to take multiple names (#109227)
* Secrets: Update proto inline secure values to take variadic names on delete

* Secrets: Update inline delete when owned to take multiple names

* trigger CI
2025-08-06 11:45:03 +02:00
Matheus MacabuandStephanie Hingtgen 81f544e186 Secrets: Add service name as explicit parameter for Decrypt (#109054)
* Secrets: Add service name as explicit parameter for Decrypt

* Apply suggestions from code review

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-08-04 08:51:15 +02:00
Matheus Macabu 988439e0b8 Secrets: Simplify CanReference interface to only pass secure value names (#109030) 2025-08-01 14:00:01 +02:00
Matheus Macabu cfb5f5601f Secrets: Add inline secure value can reference method (#108965) 2025-07-31 16:42:40 +02:00
Dana Axinte a3cdfce25a SecretsManager: Consolidation service and ability to run via cli (#108774)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* initial secrets consolidation

* Revert defaults

* More verbose description of the operation

* Add consolidation tests and tracing

* Fix lint

* Revert debug log
2025-07-31 14:45:59 +01:00
lean.dev 6bf542889a SecretsManager: Refactor and clean metrics (#108908) 2025-07-30 20:00:30 +01:00
Stephanie Hingtgen ef9f9c2d8e Secrets service: Allow decrypt through gRPC connection (#108365) 2025-07-29 07:51:37 -05:00
Dana Axinte 2ea77a7c05 SecretsManager: Add ability to list all encrypted values (#108512)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* fix lint
2025-07-28 10:50:24 +01:00
Dana Axinte b1b9cc43a8 SecretsManager: Adding ability to disable all DEKs (#108444)
* Adding dek deactivation and rename list dek

* disable data keys from manager

* separate interface and don't use in encryption manager
2025-07-25 17:11:17 +01:00
Matheus Macabu 0e7b041b27 Secrets: Move decrypt types to contracts and export public at root pkg (#108376)
* Secrets: Move decrypt types to contracts and export public at root pkg

* Provisioning: Replace decrypt pkg imports

* Merge wire changes
2025-07-23 12:14:28 +02:00
Matheus Macabu 9e4f2cad47 Secrets: Re-export and map domain errors into K8s API errors in ST client (#108226) 2025-07-21 09:32:20 +00:00
Matheus Macabu 6c69ae244e Secrets: Add single tenant SecureValueClient (#108099)
* Secrets: Add single tenant SecureValueClient

* SecureValueClient: Rename file

* SecureValueClient: Move original type to contracts package and export it by aliasing
2025-07-17 10:56:49 +02:00
Bruno 01692bc876 Secrets: remove unused SecureValueMetadataStorage.ReadForDecrypt method (#108181)
* Secrets: remove unused SecureValueMetadataStorage.ReadForDecrypt method

* remove unused struct: secureValueForDecrypt
2025-07-16 11:52:26 -03:00
Matheus Macabu 3cda233468 Secrets: Remove temporary allowlist (#108118) 2025-07-16 09:39:33 +02:00
Matheus Macabu 4954822cbb Secrets: Add separate package for resource validation (#108097)
* Secrets: Add SecureValueService interface

* Secrets: Move resource validators to their own package/structs for reusing
2025-07-15 12:33:14 +02:00
Bruno baa89f3eac Secrets: encryption encryption storage uses versioning (#108036)
* Secrets: delete unused FakeKeeper

* Secrets: encrypted value storage stores versions

* add version to span

* trigger build

* remove ineffectual assignment

* lint

* drop secret_encrypted_value.uid / add name and version columns
2025-07-14 09:28:07 -03:00
Matheus Macabu 9c1b2fb792 Secrets: Bump API version to v1beta1 (#108026) 2025-07-11 19:14:05 +02:00
8283d35e56 Secrets: make operations sync (#107732)
* Secrets: make operations sync

* k8s gen / update query to list secure values to include the version

* always store new version of a secret

* make update-workspace

* go mod tidy

* update queries

* update queries

* improve and use testutils in decrypt_store_test

* fix broken test

* make update-workspace

* ./hack/update-codegen.sh secret

* update Test_SecureValueMetadataStorage_CreateAndRead

* undo dependency changes

* linter: fix remaining errors

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-09 10:43:34 -03:00
cfd3b9f582 SecretsManager: outbox use message id (#107472)
* SecretsManager: outbox use message id

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* Remove query timestamp

* Add missing query

---------

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
2025-07-03 15:21:47 +01:00
4d8678c7f2 SecretsManager: Add base encryption manager (#107562)
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-03 11:29:14 +01:00
Dana AxinteandMatheus Macabu d28594d2f9 SecretsManager: Limit of 24kiB for raw input for secure value (#107403)
* SecureValues: Add limit of 24kiB for raw input

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Fix lint

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-30 22:28:41 +01:00
0fccc01ebe SecretsManager: add data key store (#107396)
* SecretsManager: Add data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

* SecretsManager: Add wiring of data key store

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-30 17:17:07 +01:00
6097841e67 SecretsManager: add secure value store (#106708)
* SecretsManager: add secure value model and sql templates

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: secure value rest layer to use store

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: temporary add actor prefix to decrypters

* Remove list securevalue by namefor now

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-06-16 10:19:44 +01:00
de28231f2f SecretsManager: Add outbox store (#106613)
SecretsManager: add outbox store

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-06-12 13:31:48 +01:00
b4cd51810b SecretsManager: Various utils for usage insights, outbox and secretkeeper (#106010)
* SecretsManager: utils for usage insights on ST mode

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* SecretsManager: add assert

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>

* SecretsManager: Remove encryption scope option

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* SecretsManager: add fake keeper

Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
2025-05-28 12:46:54 +01:00
Dana AxinteandMatheus Macabu 5401175562 SecretsManager: Conditionally lock DB before migrations using config setting (#106003)
Secrets: Conditionally lock DB before migrations using config setting (#105949)

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-26 18:28:53 +01:00
6e5e133f7d SecretsManager: Introduce db migrator with keeper table (#105538)
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-21 15:02:51 +01:00
Matheus Macabu a1e71fc85f SecretsManager: Update decrypt authorization with service identity (#105668)
(cherry picked from commit 9aea342be1)
2025-05-20 16:24:51 +02:00
a7922912fe SecretsManager: Introduce secrets database wrapper (#105472)
SecretsManager: Introduce secret database wrapper

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-20 11:48:47 +01:00
Dana AxinteandMatheus Macabu 581d153604 SecretsManager: Introduce DecryptAuthorizer (#105459)
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-15 17:52:14 +01:00
8c64078965 SecretsManager: Keeper and secure value contracts, secretkeeper changes (#105379)
Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-05-14 15:24:25 +01:00
6922315d7c SecretsManager: Add Keeper service and SQL Keeper (#102554)
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-03-21 11:38:43 +01:00
4c59219adb SecretsManager: Add reststorage logic with validations (#102464)
* SecretsManager: rename keeper + securevalues rest file

* SecretsManager: add rest of xkube helper methods

* SecretsManager: add domain errors to contracts

* SecretsManager: copy over Keeper reststorage from feature branch

* SecretsManager: copy over SecureValue reststorage from feature branch

---------

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-03-19 16:31:10 +01:00
3aba5cb2b7 SecretsManager: Bootstrap API service (#102444)
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-03-19 13:41:29 +01:00