Commit Graph

466 Commits

Author SHA1 Message Date
Bruno f8cd7049e8 Secrets: garbage collection (#110247)
* clean up older secret versions

* start gargbage collection worker as background service

* make gen-go

* fix typo

* make update-workspace

* undo go mod changes

* undo go work sum changes

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/registry/apis/secret/garbagecollectionworker/worker.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* default gc_worker_batch_size to 1 minute

* fix typo

* fix typo

* add test to ensure cleaning up secure values is idempotent

* make gen-go

* make update-workspace

* undo go.mod and .sum changes

* undo enterprise imports

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-09-02 11:11:01 -03:00
Matheus Macabu 1e926a29c0 Secrets: Extract external facing decrypt types to apps (#110432) 2025-09-02 10:30:29 +02:00
Roberto Jiménez Sánchez 4eadc823a9 Provisioning: Move repository package to provisioning app (#110228)
* Move repository package to apps

* Move operators to grafana/grafana

* Go mod tidy

* Own package by git sync team for now

* Merged

* Do not use settings in local extra

* Remove dependency on webhook extra

* Hack to work around issue with secure contracts

* Sync Go modules

* Revert "Move operators to grafana/grafana"

This reverts commit 9f19b30a2e.
2025-09-02 09:45:44 +02:00
Stephanie Hingtgen 232d68fb8c Controllers: Make available as a target (#110357)
* Controllers: Add to build process
* Allow setting through env variables
2025-08-30 12:27:50 +02:00
Nathan Vērzemnieks 72eeefabd7 Revert: DataSource: Support config CRUD from apiservers (#106996) (#110342)
Revert "DataSource: Support config CRUD from apiservers (#106996)"

This reverts commit eda94a6434.
2025-08-29 14:49:57 +02:00
Ezequiel Victorero 4b43877324 ShortURL: Use the k8s API in the cleanup process (#109938) 2025-08-28 17:40:45 -03:00
Ryan McKinley eda94a6434 DataSource: Support config CRUD from apiservers (#106996) 2025-08-28 22:28:26 +03:00
Ryan McKinley 43648d20c3 Preferences: Add read-only APIServer for preferences and dashboard stars (#106109) 2025-08-28 19:51:32 +03:00
Roberto Jiménez Sánchez e196f13b2b Provisioning: Introduce Repository Factory with extras (#110018) 2025-08-26 10:37:47 +02:00
Ryan McKinley ce65391067 Provisioning: Use inline secrets for gitsync (#109908)
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
2025-08-22 18:38:28 +02:00
Alexander Zobnin 2b254ed623 Zanzana: Add server side metrics (#109923)
* Zanzana: Add metrics to server side

* Zanzana: Collect check duration

* add metrics for other methods
2025-08-21 15:20:07 +02:00
Ryan McKinley fa81fae1e3 Provisioning: Add inline secure values to repository schema (#109594) 2025-08-20 09:05:41 +00:00
Gábor Farkas 2e5b55a855 datasources: querier: renamed the "mt" builder to "qs" builder (#109779) 2025-08-19 12:37:56 +02:00
Mariell Hoversholm 0fc29cbaae Rendering: Remove SVG sanitization (#109797) 2025-08-19 11:34:12 +02:00
maicon 5270d8e53f fix(unified-storage): remove permissions after executing on both stores (#109722)
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Jean-Philippe Quéméner <jeanphilippe.quemener@grafana.com>
2025-08-15 15:57:42 +00:00
Sofia Papagiannaki 402572c580 Add ConfigProvider and modify quota.Service to use it (#109395)
* Add config provider and integrate with wire setup

* Refactor quota service to use config provider for configuration management

* Enhance OSSConfigProvider to include logging and update ProvideService to return an error. Refactor server initialization to handle potential errors from config provider. Remove unnecessary wire binding for OSSConfigProvider.

* Update CODEOWNERS to include the configprovider package under the grafana-backend-services-squad.

* Refactor quota service initialization to include context in multiple service providers. Update tests and service implementations to ensure proper context handling during service creation.
2025-08-12 09:42:10 +03:00
Ryan McKinley e0404f924c K8s/SecureValues: Wire InlineSecureValueSupport to apistore (#109449)
* inline wire

* extra fields

* add variable

* wire
2025-08-11 15:22:56 +03:00
Matheus Macabu 01c1a6ce5b SecureValues: Run mutate and validation hooks in service layer (#109379)
* SecureValues: Run mutate and validation hooks in service layer

* add some unit tests
2025-08-08 13:15:23 +02:00
Todd Treece ce2697bb07 Plugins: Add app (#108422) 2025-08-06 13:09:10 -04:00
Haris Rozajac 5ad751ea28 Dashboard Migrations: V28 singlestat panel and deprecated variable properties (#108416)
Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
2025-08-06 17:51:55 +02:00
Stephanie Hingtgen 68059344f8 Unified storage: Remove unifiedStorageSearchPermissionFiltering feature flag (#109216)
* Unified storage: Remove unifiedStorageSearchPermissionFiltering ff

* unit test
2025-08-06 02:04:32 -06:00
Matheus Macabu fc117afebc Secrets: Provide grpc and local inline secure value client (#109179)
* TestUtils: Add helper to create certs

* InlineSecureValue: Move to its own package rather than service

* Settings: Rename decrypt server configs to grpc prefixed as it is more generic

* InlineSecureValue: Add gRPC client

* InlineSecureValue: Add service provider grpc/local depending on setting.Cfg

* Wire: Add ProvideInlineSecureValueService to basic wireset

* InlineSecureValue: Re-export in root secret pkg and generate mocks for interface

* Settings: Unify decrypt+inline setting into generic grpc client
2025-08-06 09:04:55 +02:00
Ezequiel Victorero e88b54e9d3 ShortURL: Create k8s API (#108821) 2025-08-04 09:12:12 -03:00
Dana Axinte a3cdfce25a SecretsManager: Consolidation service and ability to run via cli (#108774)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* initial secrets consolidation

* Revert defaults

* More verbose description of the operation

* Add consolidation tests and tracing

* Fix lint

* Revert debug log
2025-07-31 14:45:59 +01:00
lean.dev 6bf542889a SecretsManager: Refactor and clean metrics (#108908) 2025-07-30 20:00:30 +01:00
Mustafa Sencer Özcan 98e37f2ca9 fix: special handling of unmarshalling for invalid json dashboards (#108519)
* implement special dashboard fallback logic when dashboard data has invalid json behind feature flag

---------

Co-authored-by: Will Assis <william@williamassis.com>
2025-07-30 10:18:38 -04:00
Zoltán Bedi 3ff4c6a2d2 Postgres: Switch the datasource plugin from lib/pq to pgx (#108443)
* Postgres: Switch the datasource plugin from lib/pq to pgx

* Fix lint
2025-07-30 14:47:25 +02:00
Stephanie Hingtgen ef9f9c2d8e Secrets service: Allow decrypt through gRPC connection (#108365) 2025-07-29 07:51:37 -05:00
Tania 27c395694d OpenFeature: Initialize early (#108594)
* Move OpenFeatureInit

* Remove unused import

* Remove todo
2025-07-28 16:05:57 +02:00
Gabriel MABILLE 1a7a7f1d99 grafana-iam: Wire the roles api (#108577) 2025-07-28 13:36:27 +02:00
Gábor Farkas 8d48dbce66 datasources: querier: handle single-tenant instance config (#108469) 2025-07-28 11:40:26 +02:00
Matheus Macabu 0e7b041b27 Secrets: Move decrypt types to contracts and export public at root pkg (#108376)
* Secrets: Move decrypt types to contracts and export public at root pkg

* Provisioning: Replace decrypt pkg imports

* Merge wire changes
2025-07-23 12:14:28 +02:00
Josh Hunt 1bd9541507 FrontendService: Add tracing and logging middleware (#107956)
* FrontendService: Add tracing and logging middleware

* tests!

* middleware tests

* context middleware test

* revert http_server back to previous version

* fix lint

* fix test

* use http.NotFound instead of custom http handler

* use existing tracer for package

* use otel/trace.Tracer in request_tracing middleware

* tidy up tracing in contextMiddleware

* fix 404 test

* remove spans from contextMiddleware

* comment
2025-07-22 18:55:44 +01:00
Will Assis 34f1be796d unified-storage: expose ring replication factor config (#106345)
* config ring replication factor

* change default

* rename

* fix test

* fix
2025-07-22 09:32:22 +02:00
Todd Treece e5d2f92384 K8s: Add App SDK installer (#107773) 2025-07-21 13:32:15 -04:00
Matheus Macabu f7e55f2c5d Secrets: Add authz checks for the single-tenant SecureValue client (#108216) 2025-07-21 11:09:07 +02:00
Sarah Zinger 3fad863fd1 Query Service: Combine SSE handling in single tenant and multi tenant paths (#108041)
* parse via sse

I need to figure out how to handle the pipeline.execute with our own
client. I think this is important for MT reasons, just like using our
own cache (via legacy) is important.

parsing is done though!

* WIP nonsense

* horrible code but i think it works

* Add support for sql expressions config settings

* Cleanup:
- remove spew from nodes.go
- uncomment out plugin context and use in single tenant flow
- make code more readable and add comments

* Cleanup:
- create separate file for mt ds client builder
- ensure error handling is the same for both expressions and regular queries
- other cleanup

* not working but good thoughts

* WIP, vector not working for non sse

* super hacky but i think vectors work now

* delete delete delete

* Comments for future ref

* break out query handling and start test

* add prom debugger

* clean up: remove comments and commented out bits

* fix query_test

* add prom debugger

* create table-driven tests with testsdata files

* Fix test

* Add test

* go mod??

* idk

* Remove comment

* go enterprise issue maybe

* Fix codeowners

* Delete

* Remove test data

* Clean up

* logger

* Remove go changes hopefully

* idk go man

* sad

* idk i ran go mod tidy and this is what it wants

* Fix readme, with much help from adam

* some linting and testing errors

* lint

* fix lint

* fix lint register.go

* another lint

* address lint in test

* fix dead code and linters for query_test

* Go mod?

* Struggling with go mod

* Fix test

* Fix another test

* Revert headers change

* Its difficult to test this in OSS as it depends on functionality defined in enterprise, let's bring these tests back in some form in enterprise

* Fix codeowners

---------

Co-authored-by: Adam Simpson <adam@adamsimpson.net>
2025-07-17 17:22:55 -04:00
Will Browne f6ed9e6ff0 Plugins: Introduce plugin asset provider (#108063)
* introduce plugin asset provider

* simply with PR feedback

* fix linter
2025-07-17 16:20:35 +01:00
Roberto Jiménez Sánchez 810868c156 Provisioning: use new secure value client (#108227)
* Use provider in Webhooks registration

* First stab at using the new client

* Simplify mock

* Able to generate graph in OSS

* Remove line already ensured by provider

* Handle the K8s not found error as well

* Commit regenerated wire file

* Add the hack also for deletion

* Fix secrets test util

* Format file
2025-07-17 15:17:14 +02:00
Roberto Jiménez Sánchez b232ba5396 Provisioning: Move provisioning extras to OSS extensions (#108167)
* Move provisioning extensions to OSS

* Fix the wire stuff

* Move it to extras packages

* Add hack to assert test differently for enterprise
2025-07-16 13:24:24 +02:00
Matheus Macabu 2c4bbf8b1d Secrets: Reduce amount of feature toggle checks to only app entrypoint (#108110)
* Secrets: Reduce amount of feature toggle checks to only app entrypoint

* Wire: Fix merge conflict

* Wire: Fix merge conflict
2025-07-16 10:56:59 +02:00
Matheus Macabu 3cda233468 Secrets: Remove temporary allowlist (#108118) 2025-07-16 09:39:33 +02:00
Michael Mandrus ab51794bdb SecretsManager: Unify KMS across OSS and Enterprise (#108085)
* everything is compiling

* tests passing

* remove used object

* write a test for secret key upgrades

* misc cleanup

* clean up some wording

* lint issues

* fix a typo

* import hashicorp dependency explicitly

* simplify oss kmsprovider package structure

* consolidate current provider and available providers

* add a new manager configuration test

* fix hashivault import

* fix import issue

* fix unit tests

* Update go.mod

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-15 14:39:20 -04:00
Stephanie Hingtgen d39a47a89b Provisioning: Begin using secrets store (#108044)
- Provisioning: Begin using secrets store
- Refactor integration with secrets store
- Add back the legacy service
- Separate concerns for encrypt and decrypt
- Handle update within Encrypt function
- Add interface for secure value service
- Add feature flag for using secrets service
- Add the dual service for temporary solution.

* Add first integration tests for encrypted tokens
* Add integration test for app platform secrets
* Validate it has the name or not
* Create wire provider
* Always save to the secret if provided secret

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
2025-07-15 15:43:17 +02:00
Matheus Macabu 4954822cbb Secrets: Add separate package for resource validation (#108097)
* Secrets: Add SecureValueService interface

* Secrets: Move resource validators to their own package/structs for reusing
2025-07-15 12:33:14 +02:00
Tania e079cb3738 Simplify open feature setup (#107632)
* Simplify open feature setup

* Fix linter issues

* Apply review feedback

* Fix integration tests setup
2025-07-14 16:22:04 +02:00
Roberto Jimenez Sanchez 1912c4ccc3 Regenerate wire 2025-07-11 23:02:19 +02:00
Misi 1f3dc0533c Auth: Add tracing to auth clients and AuthToken service (#107878)
* Add tracing to auth clients + authtoken svc

* Fix span names

* Fix ext_jwt.go

* Fix idimpl/service

* Update wire_gen.go

* Add tracing to JWT client

* Lint
2025-07-10 15:41:00 +02:00
Josh Hunt 37bfea8685 FrontendService: Add tracing to frontend service (#107236)
* FrontendService: Add tracing to frontend service

* wire gen

* fix test
2025-07-09 19:21:31 +01:00
Bruno 8283d35e56 Secrets: make operations sync (#107732)
* Secrets: make operations sync

* k8s gen / update query to list secure values to include the version

* always store new version of a secret

* make update-workspace

* go mod tidy

* update queries

* update queries

* improve and use testutils in decrypt_store_test

* fix broken test

* make update-workspace

* ./hack/update-codegen.sh secret

* update Test_SecureValueMetadataStorage_CreateAndRead

* undo dependency changes

* linter: fix remaining errors

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-09 10:43:34 -03:00