Commit Graph
1244 Commits
Author SHA1 Message Date
beejeebus 6d64c373ce Allow FlagQueryServiceWithConnections to enable datasource config CRUD
The FlagGrafanaAPIServerWithExperimentalAPIs is only available when
`app_mode=development`. We have a more specific flag that is usable in
production, so use that.

Also, there was some old code constraining these APIs to a static list
of datasources. We don't need that anymore, so this PR removes it.

The FlagQueryServiceWithConnections is left as is, because there are
multiple existing tests that rely on this development-only, experimental
flag. I don't want to understand why that is.
2025-11-12 08:48:46 -05:00
e463781077 Schema: convert dashboards from v1beta1 to v2beta1 (#109037)
- Implement full conversion pipeline from v1beta1 → v2beta1
- Ensure frontend–backend parity for all dashboard serialization paths
- Add automatic data loss detection for conversions (panels, queries, annotations, links, variables)
- Extract atomic conversion functions for v0 → v1beta1 → v2alpha1 → v2beta1
- Introduce conversion metrics and detailed logging for loss tracking
- Normalize datasource resolution, defaults, and annotation processing
- Improve panel layout serialization and y-coordinate normalization
- Fix inconsistencies in nested panels and collapsed row behavior
- Refine variable handling:
  - Filter refId from variable query specs
  - Default variable refresh to 'never' (matches frontend)
  - Fix constant and interval variable handling for missing queries
- Unify schema defaults (enable, hide, iconColor, editable, liveNow)
- Fix pluginId usage (UID vs type) and datasource references
- Fix metrics.go bug swallowing errors (return nil → return err)
- Add tests for version-specific conversion error handling
- Add data loss detection tests using source/target version comparison
- Clean up lint issues, legacy code, and redundant files
- Update OpenAPI snapshots and migrated dashboards
- Improve backend migrator to reuse datasource provider and match frontend logic

Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Oscar Kilhed <oscar.kilhed@grafana.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-11-12 11:43:46 +01:00
Alexander Zobnin 48438f49e7 grafana-iam: refactor resource permissions write to zanzana (#113700)
* grafana-iam: refactor resource permissions write to zanzana

* Fix linter

* fix linter
2025-11-12 10:23:15 +01:00
Gábor Farkas 34e85113d2 datasources: apiserver: removed unnecessary code (#113384)
* datasources: apiserver: removed unnecessary code

* workspace fixes
2025-11-11 09:25:22 +01:00
Moustafa Baiou dd0a2d4cff Alerting: Add validation to check updates on rule groups (#113669)
This moves some of the validation logic for rule groups from the legacy storage layer to the validator.
2025-11-10 14:40:35 -05:00
Gábor Farkas 2dc48c0b98 datasources: querier: add mode-info (#113592)
* datasources: querier: add mode-info

* fixed unit test
2025-11-10 15:48:59 +01:00
Roberto Jiménez Sánchez 746efb4c56 Provisioning: Include ref field in DELETE endpoint response for branch operations (#113615)
Fix: Include ref field in DELETE endpoint response for branch operations

When deleting a dashboard file via DELETE endpoint with a ref query parameter
(for branch operations), the response was missing the ref field. This caused
the frontend branch workflow success handler to fail silently.

The issue was an inverted boolean condition in the Delete method. The code
was setting file.Ref = opts.Ref when shouldUpdateGrafanaDB returned true
(main branch operations), but it should have been setting it when false
(branch operations), since we read the file with an empty ref.

Fixed by inverting the condition from:
  if r.shouldUpdateGrafanaDB(opts, nil)
to:
  if !r.shouldUpdateGrafanaDB(opts, nil)

This ensures the ref field is properly included in the ResourceWrapper
response for branch operations.
2025-11-10 13:34:46 +01:00
Ezequiel Victorero 32db7e176d ShortURL: Fix wrong creation timestamp conversion (#113646) 2025-11-10 09:14:51 -03:00
Ezequiel Victorero cd91f858e6 ShortURL: Graduate api v1alpha1 to v1beta1 (#113597) 2025-11-10 07:29:33 -03:00
Ezequiel Victorero 24e4e0946d Dashboard: Improve search error response (#113617)
* Dashboard: Improve search error response

* improve errors
2025-11-08 12:58:51 +01:00
beejeebus 0e9fe9dc40 Register external datasource plugins on startup
Current code only registers core datasource k8s api groups.

Add external plugins.

Companion grafana-enterprise PR:

https://github.com/grafana/grafana-enterprise/pull/10125
2025-11-07 14:42:41 -05:00
Moustafa Baiou 1e1adafeec Alerting: Add admission hooks for rules app (#113429)
This adds validating admission hooks to enforce the requirements on AlertRules and RecordingRules that are currently enforced through the provisioning service and storage mechanisms in preparation of a consistent validation in both legacy storage and unified storage. It also adds a mutating admission hook to the app to ensure that folder annotations and folder labels are kept in sync so we can perform label-selector lists.
2025-11-07 12:01:16 -05:00
Roberto Jiménez Sánchez 02464c19b8 Provisioning: Add validation for Job specifications (#113590)
* Validate Job Specs

* Add comprehensive unit test coverage for job validator

- Added 8 new test cases to improve coverage from 88.9% to ~100%
- Tests for migrate action without options
- Tests for delete/move actions with resources (missing kind)
- Tests for move action with valid resources
- Tests for move/delete with both paths and resources
- Tests for move action with invalid source paths
- Tests for push action with valid paths

Now covers all validation paths including resource validation and
edge cases for all job action types.

* Add integration tests for job validation

Added comprehensive integration tests that verify the job validator properly
rejects invalid job specifications via the API:

- Test job without action (required field)
- Test job with invalid action
- Test pull job without pull options
- Test push job without push options
- Test push job with invalid branch name (consecutive dots)
- Test push job with path traversal attempt
- Test delete job without paths or resources
- Test delete job with invalid path (path traversal)
- Test move job without target path
- Test move job without paths or resources
- Test move job with invalid target path (path traversal)
- Test migrate job without migrate options
- Test valid pull job to ensure validation doesn't block legitimate requests

These tests verify that the admission controller properly validates job specs
before they are persisted, ensuring security (path traversal prevention) and
data integrity (required fields/options).

* Remove valid job test case from integration tests

Removed the positive test case as it's not necessary for validation testing.
The integration tests now focus solely on verifying that invalid job specs
are properly rejected by the admission controller.

* Fix movejob_test to expect validation error at creation time

Updated the 'move without target path' test to expect the job creation
to fail with a validation error, rather than expecting the job to be
created and then fail during execution.

This aligns with the new job validation logic which rejects invalid
job specs at the API admission control level (422 Unprocessable Entity)
before they can be persisted.

This is better behavior as it prevents invalid jobs from being created
in the first place, rather than allowing them to be created and then
failing during execution.

* Simplify action validation using slices.Contains

Replaced manual loop with slices.Contains for cleaner, more idiomatic Go code.
This reduces code complexity while maintaining the same validation logic.

- Added import for 'slices' package
- Replaced 8-line loop with 1-line slices.Contains call
- All unit tests pass

* Refactor job action validation in ValidateJob function

Removed the hardcoded valid actions array and simplified the validation logic. The function now directly appends an error for invalid actions, improving code clarity and maintainability. This change aligns with the recent updates to job validation, ensuring that invalid job specifications are properly handled.
2025-11-07 16:31:50 +00:00
Jean-Philippe Quéméner 1ca95cda4a fix(folders): prevent circular dependencies (#113595) 2025-11-07 14:19:55 +00:00
Jo 176b0f8b48 IAM: Refactor user org hooks to use MutateRequest API (#113392)
* update with mutation hooks

* add missing delete mutation
2025-11-07 14:36:53 +01:00
Gabriel MABILLEandjguer e90759e5af grafana-iam: enable dual writing for resource permissions (#112793)
* `grafana-iam`: enable dual writing for resource permissions

Co-authored-by: jguer <joao.guerreiro@grafana.com>

* copy paste mistake

* Reduce complexity

* nits to make the code easy to review

* Forgot to check the error

---------

Co-authored-by: jguer <joao.guerreiro@grafana.com>
2025-11-07 13:50:40 +01:00
Alex KhomenkoandRoberto Jimenez Sanchez 8cb5f5646a Provisioning: Fix miscellaneous issues with setting and displaying sync status (#113529)
* Provisioning: Preserve in progress job data

* Refactor code and cover more situations

* Fix linting

* Fix issue with remove path operation for started time

* Cleanup

* prettier

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
2025-11-07 12:27:25 +01:00
Serge ZaitsevandTania B. 95ea758475 Chore: Start annotations app (#113018)
* annotation legacy store with api server, read only

* Add a feature flag for annotations app

* implement list filters

* annotations are not addressable by ID for read operations

* fix registry apps test

* add ownership for an app

* disable linter

* typo, of course

* fix go workspace

* update workspace

* copy annotation app in dockerfile

* update workspace

---------

Co-authored-by: Tania B. <10127682+undef1nd@users.noreply.github.com>
2025-11-06 19:22:20 +01:00
Gábor Farkas acb0320796 datasources: apiserver: do not enable extra methods by default (#113395) 2025-11-06 15:34:32 +01:00
Ryan McKinley 95ffd1a55a LibraryPanel: Cleanup service calls (#113277)
* cleanup

* library panel via search

* test cleanup

* merge main

* add FindDashboards mock

* no matching dashbaords should return empty

* do not alllow name and libraryPanel query
2025-11-06 15:31:02 +01:00
Jean-Philippe QuéménerandMustafa Sencer Özcan fd14d4a5ed feat(unified-storage): add tracing to dual writer and legacy storage (#113504)
Co-authored-by: Mustafa Sencer Özcan <32759850+mustafasencer@users.noreply.github.com>
2025-11-06 11:42:46 +00:00
Gabriel MABILLEandjguer ff53276870 grafana-iam: Instantiate ExternalGroupMappingStorage as a NoopStorage (#113499)
Co-authored-by: jguer <joao.guerreiro@grafana.com>
2025-11-06 11:00:37 +01:00
Misi 06373ae47b IAM: Add ExternalGroupMapping kind for TeamSync (#113052)
* wip

* wip

* Add authorizer -> VERIFY it's working correctly

* Update openapi definitions

* Authorizer wip

* regen apis

* Increase timeout of pg int tests to 20m

* Revert "Increase timeout of pg int tests to 20m"

This reverts commit 8c20568217.

* Fix NewTestStore when Truncate is enabled
2025-11-05 18:02:34 +01:00
Mustafa Sencer Özcan b97fb638ad fix: only validate allowed descendants for folder deletion (#113440) 2025-11-05 14:04:43 +01:00
Roberto Jiménez SánchezandCopilot 571e5c2e3c Provisioning: Fix data race in job progress and leasing (#113157)
* Fix data race in provisioning job execution

* Fix TODO

* Update pkg/registry/apis/provisioning/jobs/driver.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update pkg/registry/apis/provisioning/jobs/driver.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix unlocking issue on panic

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-05 10:07:21 +00:00
Jean-Philippe Quéméner 98ec655f33 fix: add resource type to not empty log (#113432) 2025-11-05 10:12:44 +01:00
Stephanie HingtgenandMatheus Macabu 4cecab3185 Dashboards: add isPublic to dto and remove public endpoint call (#113334)
---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
2025-11-04 16:57:05 -06:00
Jo 769787ea40 IAM: Improve team binding hooks error handling and validation (#113393)
small team hook fixes
2025-11-04 11:54:38 +01:00
Gábor Farkas ac162e203b datasources: querier: do not forward these headers (#113383) 2025-11-04 09:01:16 +01:00
Jo eeddc8cd18 Zanzana: Add team binding hooks (#113274)
add team binding hooks
2025-11-03 15:39:20 +01:00
Jo 99e4583cd1 Zanzana: Add user org role hooks (#113276)
* add user org role hooks

* update with feedback
2025-11-03 15:39:12 +01:00
CharandasandIevaVasiljeva 6c728f8dec Provisioning: allow access check to proceed even when non access policy (#112946)
* Provisioning: allow access check to proceed even when non access policy

* Provisioning: access checker needs this for MT

* add permissions registration

* remove scopes

* use in MT for now

* no need to document an internal flag here

* revert vscode change

* refactor the authZ permission evaluation and mapper code to allow evaluating unscoped actions beyond creation

* update wire

* gofmt

* add boolean to struct

---------

Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
2025-11-02 13:14:08 -08:00
Stephanie Hingtgen 445e88cb93 Dashboard Provisioning: Add duplicate cleanup for modes 0-2 (#113336) 2025-11-02 20:39:15 +00:00
Owen Diehl b0ecc36fd6 [App-Platform,Git-Sync]: Add sync job & phase metrics (#113253)
* [git-sync] add sync job & phase metrics

* refactoring sync jobs for clarity and ease of instrumentation

* make gofmt
2025-10-31 10:24:17 -07:00
Jo a339a2dad3 AuthZ: Instantiate needed clients for IAM hooks (#113268)
build API with zanzana hooks support
2025-10-31 15:37:43 +01:00
Tania 4813256e0a OpenFeature: Log key name and namespace in handler (#113288) 2025-10-31 12:31:48 +00:00
Misi e09deb6889 IAM: Use the new authorizer for Teams (#113275)
Use the new authorizer for Teams
2025-10-31 10:48:01 +00:00
Ryan McKinley 53aed34652 LibraryPanels: Support CRUD via apiserver (#113035) 2025-10-31 08:14:38 +03:00
Michael Mandrus cf242def3a Secrets: Refactor data_key_id out of the encoded secure value payload (#112077)
* Secrets: Refactor data_key_id out of the encoded secure value payload (#111852)

* everything compiles

* tests pass

* remove file included by accident

* add entry to gitignore

* some scaffolding for the migration executor

* remove file

* implement and test the migration

* use xkube.Namespace in our interfaces

* add todo

* update wire deps

* add some logs

* fix wire dependency ordering

* create tests to validate error conditions during migrations

* only run the migration as an MT api server

* formatting issues

* change detection of secrets running as MT server

* add todo

* use more specific initializer flags

* make secrets playwright tests work

* set new properties to true by default

* remove developer mode flag

* fix unit tests
2025-10-30 23:04:32 -04:00
Misi 44beedd09a IAM: Handle NULL external_uid, is_provisioned correctly for Teams (#113219)
* Handle NULL external_uid correctly with MySQL

* Add NULL handling to is_provisioned column
2025-10-30 15:50:40 +00:00
Kyle BrandtandTodd Treece c3d7dbc258 SQL Expressions: Add endpoint to get Schemas (#108864)
Return the SQL schema for all DS queries in request (to provide information to AI / Autocomplete for SQL expressions).

All DS queries are treated as if they were inputs to SQL expressions in terms of conversion, regardless if they are selected in a query or not.

Requires feature toggle queryService = true

Endpoint is apis/query.grafana.app/v0alpha1/namespaces/default/sqlschemas

---------

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
2025-10-30 10:05:12 -04:00
Ryan McKinley d303746ff9 ShortURL: Use UpdateStatus client (#111170) 2025-10-30 15:57:03 +03:00
Costa Alexoglou bbfb8268d1 Provisioning: concurrent deletes in finalizers and 404 handling (#113155)
* fix: concurrent deletes in finalizers and 404 handling

* chore: feedback review

* fix: broken tests
2025-10-30 11:55:36 +01:00
Todd Treece de88abafdd Plugins API: Merge meta and installs (#112962) 2025-10-29 13:32:31 -04:00
Costa Alexoglou 25dd7e927f feat: add granular context timeout (#112952)
* feat: add granular context timeout

* test: forced thread timeout

* fix: test assumption for blocks

* test: new change

* trigger build

* remove loggers to test

* test with fmt.print

* fix: deadlocks
2025-10-29 08:41:53 +01:00
Ryan McKinley d72e048bfe Chore: Use Kind().GroupVersionResource() (#113133) 2025-10-28 14:54:10 -07:00
Bruno 437dcc875c QueryCaching: Use CachingServiceClient for query caching (#112128)
* Integrate mt querier with query caching

* typo

* let the caller set cache status response header

* fix TestQueryAPI

* make gen-go

* handle CachingServiceClient being nil and make gen-go

* include namespace in cache key

* set signed in user namespace in query_test.go

* fix test

* remove commented out code

* undo services/query/query.go changes

* make gen-go

* remove namespace requirement

* fix tests

* fix test

* remove namespace from SignedInUser in tests

* make gen-go
2025-10-28 11:41:46 -03:00
Jo d216d75fbb Access: Add CoreRole/Role Delete/Update hooks for OpenFGA (#112839)
* Add delete and update hooks for roles/core roles

no need to capture non reference types

small cleanup on vars

* fix ticket priming in hooks

* fix ticket priming in hooks

* Revert "fix ticket priming in hooks"

This reverts commit f8e953ca09.

* use old testing blocks

* protect runtime obj in go func

* update test for correctness

* separate files for test correctness. fix leaking goroutines in go tests

* go workspace fixes

* attribute owner

* clean up go mod
2025-10-27 17:20:59 +00:00
Austin PondandOwen Diehl bf65c43783 Apps: Add Example App to ./apps (#112069)
* [API Server] Add Example App for reference use.

* Remove Printlns.

* Upgrade app-sdk to v0.46.0, update apps to handle breaking changes.

* Only start the reconciler for the example app if the v1alpha1 API version is enabled.

* Some comment doc updates.

* Run make update-workspace

* Set codeowner for /apps/example

* Run make gofmt and make update-workspace

* Run prettier on apps/example/README.md

* Add COPY apps/example to Dockerfile

* Add an authorizer to the example app.

* Fix import ordering.

* Update apps/example/kinds/manifest.cue

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

* Run make update-workspace

* Re-run make gen-go for enterprise import updates

* Run make update-workspace

---------

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
2025-10-27 12:01:10 -04:00
Mihai Turdean f019d58a99 [grafana-iam] Add resourcePermissions hooks to sync write to Zanzana on UPDATE and DELETE (#112767) 2025-10-24 11:45:24 -06:00