Commit Graph
1244 Commits
Author SHA1 Message Date
Adam SimpsonSarah Zinger sarah.zinger@grafana.comGábor Farkas gabor.farkas@gmail.com
2d1271e981 query service: improve error handling and logging (#108948)
Co-authored-by: Sarah Zinger sarah.zinger@grafana.com
Co-authored-by: Gábor Farkas gabor.farkas@gmail.com
2025-07-31 14:10:21 -04:00
Matheus Macabu cfb5f5601f Secrets: Add inline secure value can reference method (#108965) 2025-07-31 16:42:40 +02:00
Dana Axinte a3cdfce25a SecretsManager: Consolidation service and ability to run via cli (#108774)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* initial secrets consolidation

* Revert defaults

* More verbose description of the operation

* Add consolidation tests and tracing

* Fix lint

* Revert debug log
2025-07-31 14:45:59 +01:00
lean.dev 6bf542889a SecretsManager: Refactor and clean metrics (#108908) 2025-07-30 20:00:30 +01:00
Sarah Zinger 60ce4ad1fa Ds-querier: add slug to logs (#108822) 2025-07-30 17:46:38 +00:00
Mustafa Sencer ÖzcanandWill Assis 98e37f2ca9 fix: special handling of unmarshalling for invalid json dashboards (#108519)
* implement special dashboard fallback logic when dashboard data has invalid json behind feature flag

---------

Co-authored-by: Will Assis <william@williamassis.com>
2025-07-30 10:18:38 -04:00
+4 20b08ccaab Add v2beta1 api version: Consolidate schema breaking changes (#108172)
* Revert "Revert: Future-proofing query and data source model in Dashboard Sche… (#107985)"

This reverts commit 13a89d4ae3.

* Revert "Revert "Schema V2: Simplify annotations v1<->v2 conversions" (#107984)"

This reverts commit 2b8c5bea1a.

* make gen apps

* e2e update

* Use v2alpha2 by default (#108177)

* Use v2alpha2 by default

* Apply only DS changes to alpha2

* Use v2alpha2 by default except to query

* Create a v2 index in @grafana/schema

* Update path and apply lint

* Update tests

* Update imports to v2 status

* Fix failing openapi test

* Schemav2 breaking changes: conversion implementation (#108224)

* provision v2alpha1 dashboard

* Run conversions for DS refactor

* Run snapshot testing on conversions

* Normalize output name

* Update snapshots to include all panel and variable cases

* fix lint

* fix lint

* fix test and go lint

* more go lint

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variable (#108237)

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variables

* add conversion

* App Installer: Authorizer support (#108419)

* Chore: use `satisfies` and remove a load of `any`s (#108397)

use satisfies and remove a load of anys

* improve logging and fail unified-storage migration with more than 0 errors (#108471)

improve logging and fail unified-storage migration with more than 0 errors

* fix conversion test

* Secrets: Create more granular fixed roles for SecureValues (#108382)

* Provisioning: Fix bug in job progress recording (#108440)

Fix bug in job progress recording

* Provisioning: Fix ImportAllPanelsFromLocalRepository test (#108441)

* Provisioning: Skip flaky test

* Fix flaky provisioning test

* Fix lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* BulkDeleteProvisionedResource: Move progress bar into a second step (#108417)

* Move progress bar into a second step

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* [Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body (#108476)

[Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body to make sure new versions have the same settings.

---------

Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>

* Dashboard Schema V2: Refactor VizConfigKind to follow DataQueryKind convention (#108148)

* Dashboards API: Register v2alpha2 API

* Prepare conversion functions

* Fix test

* Refactor VizConfigKind to follow DataQueryKind convention

* fix tests

* use new dataquerykind convention alpha 2

* add conversion

* fix tests

* fix tests

* fix another test

* Fix merge

---------

Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>

* fix k8s codegen

* Update e2e-playwright/dashboards/TestV2Dashboard.json

* Update e2e/dashboards/TestV2Dashboard.json

* revert app generation for non-related apps

* try again

* another try

* also revert folder and secret app generation

* v2alpha1 provisioned dashboard

* Fix kind

* Fix conversion snapshots

* Update API discovery registry

* Rename to v2beta1

* Rename migrations

* Update apps/dashboard/pkg/apis/dashboard/v2beta1/doc.go

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

* Ensure conditional rendering and other non changed properties

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Haris Rozajac <58232930+harisrozajac@users.noreply.github.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-07-30 15:01:27 +02:00
Bruno Abrantes a81c28b153 chore: include stackId when marking a dashboard as rejected during migration (#108887)
Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-07-30 08:37:10 +00:00
Roberto Jiménez Sánchez f4e45bf3bc Provisioning: bulk move by name (#108869)
* Add ResourceRef to move job spec
* Implement move by name
* Use map to struct
2025-07-30 11:30:24 +03:00
Roberto Jiménez Sánchez 1a246739ed Provisioning: Add bulk move job (#108762)
* Add move job to spec

* Add move worker

* Single commit mode

* Add initial integration tests

* Format code

* Improve test setup

* Add fixes for integration tests

* Implement move operation

* Format file

* Fix API documentation
2025-07-29 17:24:17 +02:00
Tania 2c7349f863 OpenFeature: Use types from Go Feature Flag pkg (#108802)
* Use go feature flag types

* Add go-feature-flag module

* Update go.mod

* Update test

* Fix
2025-07-29 15:05:59 +02:00
Stephanie Hingtgen ef9f9c2d8e Secrets service: Allow decrypt through gRPC connection (#108365) 2025-07-29 07:51:37 -05:00
Roberto Jiménez Sánchez c7c0268594 Provisioning: bulk delete resources by name (#108833)
* Add resource reference to spec

* Add FindResourcePath

* Fix formatting

* Use ForKind client

* Add unit test for new method

* Format code

* Add integration tests

* Fix unit tests

* Fix formatting

* Find out preferred version based on group and kind

* Handle trailing slash for folders

* Format code

* Fix linting

* Add integration test for folder bulk deletion

* Format code

* Format discovery file

* Deduplicate paths for deletion
2025-07-29 14:29:35 +02:00
Matheus Macabu 724c775bab Secret: Fix create/delete basic role configuration for secure values (#108840)
They should be for Admins not Editors
2025-07-29 11:19:39 +02:00
Roberto Jiménez Sánchez 6af1bfe21b Provisioning: Use commit only once mode in bulk delete (#108759)
Use commit only once mode in bulk delete
2025-07-29 10:37:27 +02:00
Serge ZaitsevandMatheus Macabu a95fb3a37c Chore: Omit integration tests if short test flag is passed (#108777)
* omit integration tests if short test flag is passed

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/tests/api/alerting/api_ruler_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* Update pkg/services/ngalert/models/receivers_test.go

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>

* fix the rest

* false positive

---------

Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
2025-07-28 13:38:54 +02:00
Gabriel MABILLE 1a7a7f1d99 grafana-iam: Wire the roles api (#108577) 2025-07-28 13:36:27 +02:00
Dana Axinte 2ea77a7c05 SecretsManager: Add ability to list all encrypted values (#108512)
* list all encrypted values and count

* separate interfaces

* add time filter to global queries

* fix lint
2025-07-28 10:50:24 +01:00
Gábor Farkas 8d48dbce66 datasources: querier: handle single-tenant instance config (#108469) 2025-07-28 11:40:26 +02:00
Roberto Jiménez SánchezandStephanie Hingtgen 20cea80795 Provisioning: Add bulk delete job (#108580)
* Add delete job type

* Regenerate spec

* Add first implementation of worker

* Move interface and mock to repository package

* Add unit tests

* Add integration tests

* Fix linting and spec

* Regenerate client

* Format file

* go fmt

* fix

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-07-25 16:38:10 +00:00
Dana Axinte b1b9cc43a8 SecretsManager: Adding ability to disable all DEKs (#108444)
* Adding dek deactivation and rename list dek

* disable data keys from manager

* separate interface and don't use in encryption manager
2025-07-25 17:11:17 +01:00
Victor Cinaglia 5f6fc38430 iam/authn: Introduce feature flag for authz resource mutations (#108698)
* iam/authz: introduce feature flag for authz resource mutations

* lint: fix typo
2025-07-25 12:05:32 -03:00
Roberto Jiménez Sánchez 16339b0788 Provisioning: Move repository file API (#108514)
* Bump nanogit version
* Embed git repository in the Github one
2025-07-24 23:28:59 -05:00
Tania d3b6d7c45e OpenFeature: Add config setting to disable API (#108620)
* Add config option to disable ofrep api

* Apply review feedback
2025-07-24 19:45:22 +00:00
Roberto Jiménez SánchezandStephanie Hingtgen e9b9618fb0 Provisioning: Fix stage options for export (#108585)
* Add ref option to stage options

* Fix the issue with ref in export worker

* Add unit tests for export stage options

* Do not fail if ref is equal to the stage branch

* Format code again

* fix test

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-07-24 16:53:33 +00:00
Roberto Jiménez Sánchez 915f47befd Provisioning: commit only once staged changes (#108590) 2025-07-24 10:57:02 -05:00
Victor CinagliaandCharandas Batra 4c17c1cbb6 IAM: Instantiate DualWriter only when in single-tenant mode (#108558)
* Instantiate dual writer only in ST case

* minor cleanup

---------

Co-authored-by: Charandas Batra <charandas.batra@grafana.com>
2025-07-24 09:33:53 -03:00
Tania 60e0d2d136 OpenFeature: Update authenticated request check (#108539) 2025-07-23 20:27:21 +02:00
Matheus Macabu 0e7b041b27 Secrets: Move decrypt types to contracts and export public at root pkg (#108376)
* Secrets: Move decrypt types to contracts and export public at root pkg

* Provisioning: Replace decrypt pkg imports

* Merge wire changes
2025-07-23 12:14:28 +02:00
Roberto Jiménez Sánchez 946358900c Provisioning: Add warning job status and different strictness (#108446)
* Add warning job status

* Different strictness for export than for pulling

* Fix some unit tests

* Fix unit tests

* Update frontend to display the warning status

* Fix status badge

* Add missing return statement

* Improve text in alert
2025-07-23 09:33:25 +00:00
Roberto Jiménez Sánchez 482872c3bf Provisioning: Fix bug in job progress recording (#108440)
Fix bug in job progress recording
2025-07-22 14:21:22 +00:00
Matheus Macabu 9402071d33 Secrets: Create more granular fixed roles for SecureValues (#108382) 2025-07-22 16:20:54 +02:00
Will Assis 71cb623cb7 improve logging and fail unified-storage migration with more than 0 errors (#108471)
improve logging and fail unified-storage migration with more than 0 errors
2025-07-22 13:44:04 +00:00
d72a70d246 Dashboards API: v2alpha2 missing pieces (#108293)
* Dashboards API: v2alpha2 missing pieces

* Fix issue with dashboard client scope for alpha versions

As we now have 2 different alpha versions for v2 we need to store the
clients separately.

* Improve debuggability of provisioning export test

- Add a helper function to print the tree structure.
- Be explicit about the expected file names expected in each case.

* Update pkg/registry/apis/dashboard/mutate.go

* Update pkg/services/authz/zanzana/server/server.go

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>

* Review

* go lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
2025-07-22 11:44:05 +02:00
Todd Treece e5d2f92384 K8s: Add App SDK installer (#107773) 2025-07-21 13:32:15 -04:00
Bruno Abrantes be79eb0ace fix: handle dashboards without version entries in unified storage migration (#108139)
* fix: handle dashboards without version entries in migration

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: adds AllowFallback flag to enable COALESCE logic explicitly during migrations

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: update dependencies

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* fix: maintain original behavior when not using AllowFallback option

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

---------

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-07-21 15:32:24 +02:00
Matheus Macabu 9e4f2cad47 Secrets: Re-export and map domain errors into K8s API errors in ST client (#108226) 2025-07-21 09:32:20 +00:00
Matheus Macabu f7e55f2c5d Secrets: Add authz checks for the single-tenant SecureValue client (#108216) 2025-07-21 11:09:07 +02:00
Matheus Macabu 47abf5f811 Secrets: Clean up unused code (#108366)
* Secrets: Remove unused tracectx pkg

* Secrets: Remove unused assert pkg

* Secrets: Remove unused encryption interface

* Secrets: Remove unused encryption file/ref
2025-07-21 11:08:49 +02:00
2dba473015 Feature/unified storage search dual reader (#108291)
* Add UnifiedStorageSearchDualReaderEnabled feature flag

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* Refactor UniSearch Dual Reader

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* Run make gen-feature-toggles

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* fix: unit tests search_client

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

* feat: cancels shadow search requests after 500ms

Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Signed-off-by: Bruno Abrantes <bruno@brunoabrantes.com>
Co-authored-by: Will Assis <william@williamassis.com>
Co-authored-by: Bruno Abrantes <bruno@brunoabrantes.com>
2025-07-18 14:43:56 +02:00
Roberto Jiménez Sánchez 4847882ee7 Provisioning: handle .git extension more gracefully (#108213)
* Move .git to repository packages

* Bump nanogit 2025-07-17

This version handles `.git` extension internally so that the client
doesn't have to worry about it

* Put back mutation for Github

* Mutate also git URL for clarity
2025-07-18 13:25:21 +02:00
Roberto Jiménez Sánchez 446054a61d Provisioning: fix race condition in usage metrics collection (#108289)
* Register usage stats after lister is created

For enterprise where metrics are collected more frequently and in
different ways the collection was happening before the listing was
initialized.

* Move usage to separate package
2025-07-18 13:24:58 +02:00
Roberto Jiménez Sánchez e3422c415f Provisioning: fix hack-introduced bug on repository updates (#108298)
* Add log line to see which secret service we are using
* Add hack to make test and create endpoint work until we have InLineSecureValues
2025-07-18 10:18:07 +00:00
Misi d5f47d2a6b IAM: Comment out DualWriter setup for Users (#108241)
* Try 1

* Comment out DW setup for users
2025-07-18 09:29:09 +00:00
Roberto Jiménez Sánchez 5cf48294f7 Provisioning: fix authorized github client initialization (#108290)
* Add logging

* Fix the condition for the github token

* Revert "Add logging"

This reverts commit 71cbc675f6.

* Fix condition
2025-07-18 10:23:55 +02:00
Sarah ZingerandAdam Simpson 3fad863fd1 Query Service: Combine SSE handling in single tenant and multi tenant paths (#108041)
* parse via sse

I need to figure out how to handle the pipeline.execute with our own
client. I think this is important for MT reasons, just like using our
own cache (via legacy) is important.

parsing is done though!

* WIP nonsense

* horrible code but i think it works

* Add support for sql expressions config settings

* Cleanup:
- remove spew from nodes.go
- uncomment out plugin context and use in single tenant flow
- make code more readable and add comments

* Cleanup:
- create separate file for mt ds client builder
- ensure error handling is the same for both expressions and regular queries
- other cleanup

* not working but good thoughts

* WIP, vector not working for non sse

* super hacky but i think vectors work now

* delete delete delete

* Comments for future ref

* break out query handling and start test

* add prom debugger

* clean up: remove comments and commented out bits

* fix query_test

* add prom debugger

* create table-driven tests with testsdata files

* Fix test

* Add test

* go mod??

* idk

* Remove comment

* go enterprise issue maybe

* Fix codeowners

* Delete

* Remove test data

* Clean up

* logger

* Remove go changes hopefully

* idk go man

* sad

* idk i ran go mod tidy and this is what it wants

* Fix readme, with much help from adam

* some linting and testing errors

* lint

* fix lint

* fix lint register.go

* another lint

* address lint in test

* fix dead code and linters for query_test

* Go mod?

* Struggling with go mod

* Fix test

* Fix another test

* Revert headers change

* Its difficult to test this in OSS as it depends on functionality defined in enterprise, let's bring these tests back in some form in enterprise

* Fix codeowners

---------

Co-authored-by: Adam Simpson <adam@adamsimpson.net>
2025-07-17 17:22:55 -04:00
Roberto Jiménez Sánchez 810868c156 Provisioning: use new secure value client (#108227)
* Use provider in Webhooks registration

* First stab at using the new client

* Simplify mock

* Able to generate graph in OSS

* Remove line already ensured by provider

* Handle the K8s not found error as well

* Commit regenerated wire file

* Add the hack also for deletion

* Fix secrets test util

* Format file
2025-07-17 15:17:14 +02:00
MisiandGabriel Mabille c6a6b9fdd2 IAM: Create and delete user from the legacy store (#107694)
* Add Create for User + DualWriter setup

* Add delete User

* Fix delete + access check

* Add tests for delete user

* Add tests for create user

* Fixes

* Use sqlx session to fix database locked issues

* wip authz checks

* legacyAccessClient

* Update legacyAccessClient, add tests for create user

* Close rows before running other queries

* Use ExecWithReturningId

* Verify deletion in the tests

* Add Validate and Mutate

* Other changes

* Address feedback

* Update tests

---------

Co-authored-by: Gabriel Mabille <gabriel.mabille@grafana.com>
2025-07-17 11:50:40 +02:00
Andres Martinez Gotor 8e663b50e9 Advisor: Allow to define enterprise checks (#108114) 2025-07-17 11:37:32 +02:00
Matheus Macabu 6c69ae244e Secrets: Add single tenant SecureValueClient (#108099)
* Secrets: Add single tenant SecureValueClient

* SecureValueClient: Rename file

* SecureValueClient: Move original type to contracts package and export it by aliasing
2025-07-17 10:56:49 +02:00