Commit Graph

54 Commits

Author SHA1 Message Date
Roberto Jiménez Sánchez e2913815d3 Provisioning: Build resource clients for operators (#110699)
---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-09-05 17:52:57 +00:00
Roberto Jiménez Sánchez 1b9e479b68 Provisioning: Abstract away how we build discovery and dynamic clients (#110662)
Abstract away how we get discovery and dynamic clients
2025-09-05 09:22:30 +00:00
Roberto Jiménez Sánchez 7d630ec3b1 Provisioning: Refactor tweaks to support MT controllers (#110581)
* Refactor common code to support MT controllers

* Delete original status files
2025-09-04 10:06:50 +00:00
Roberto Jiménez Sánchez 4eadc823a9 Provisioning: Move repository package to provisioning app (#110228)
* Move repository package to apps

* Move operators to grafana/grafana

* Go mod tidy

* Own package by git sync team for now

* Merged

* Do not use settings in local extra

* Remove dependency on webhook extra

* Hack to work around issue with secure contracts

* Sync Go modules

* Revert "Move operators to grafana/grafana"

This reverts commit 9f19b30a2e.
2025-09-02 09:45:44 +02:00
Misi a5c05ba9c1 IAM: Moving code to the /pkg/apps/iam folder (#109985)
* wip

* Gen GetTeams with app sdk

* Revert some changes, cleanup

* Format iam_manifest.go

* Remove generated file

* Regenerate openapi defs

* Cleanup

* Remove TODO
2025-08-28 12:32:15 +02:00
Roberto Jiménez Sánchez 93a35fc7be Provisioning: Move apifmt, loki and safepath to provisioning app (#110226)
* Move apifmt

* Move safepath

* Move Loki package

* Regenerate Loki mock

* Missing file for Loki
2025-08-27 13:26:48 -05:00
Costa Alexoglou 9785e573aa Provisioning: Fix Dashboard Creation For First-Level Repository Folders (#109962) 2025-08-27 12:20:57 +02:00
Roberto Jiménez Sánchez 1ff39510d3 Provisioning: Check For Resource Ownership Before Operations (#109582) 2025-08-15 10:05:53 +03:00
Roberto Jiménez Sánchez f2303a8ad9 Provisioning: Include Ref URLs in Job Status (#109464)
* Add URLs to Job spec

* Rename them as RefURLs

* Implement RefURLs for Github

* Add Ref URLs to Jobs

* Worker Test

* Create the branch in the staged writer

* Regenerate Git mock

* Format code

* Consolidate ResourceURLs and RefURLs into one

* Fix broken tests
2025-08-12 11:57:37 +02:00
Roberto Jiménez Sánchez 8a690e977e Provisioning: Return unmanaged count in stats (#109462)
* Unmanaged counts

* Add integration test

* Generate open api spec
2025-08-12 09:17:25 +02:00
Stephanie Hingtgen 9f4773c9a5 Provisioning: Extract to apps submodule (#109074) 2025-08-01 14:35:18 -05:00
Roberto Jiménez Sánchez 9ca0750134 Provisioning: skip export of already managed resources and parent folder export (#108893)
* Skip export on already managed resources

* Add integration test

* Add integration test

* Handle nothing to commit error

* Fix leaky abstraction issue

* Handle the no commit error on commit and not on push

* Fix linting

* Some fixes for integration test

* Improve tree to work with a root

* Some fixes with hacks

* Add additional checks

* Fix comment

* Fix path problems in test

* Fix more stuff

* Revert to use empty tree

* Remove changes in tree

* Finally fix the tests work

* Remove stale comment

* Fix linting

* Revert changes in test

* Fix error message for folder not found in resource tree

Co-authored-by: roberto.jimenez <roberto.jimenez@grafana.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-01 11:17:15 +00:00
Dominik Prokop 20b08ccaab Add v2beta1 api version: Consolidate schema breaking changes (#108172)
* Revert "Revert: Future-proofing query and data source model in Dashboard Sche… (#107985)"

This reverts commit 13a89d4ae3.

* Revert "Revert "Schema V2: Simplify annotations v1<->v2 conversions" (#107984)"

This reverts commit 2b8c5bea1a.

* make gen apps

* e2e update

* Use v2alpha2 by default (#108177)

* Use v2alpha2 by default

* Apply only DS changes to alpha2

* Use v2alpha2 by default except to query

* Create a v2 index in @grafana/schema

* Update path and apply lint

* Update tests

* Update imports to v2 status

* Fix failing openapi test

* Schemav2 breaking changes: conversion implementation (#108224)

* provision v2alpha1 dashboard

* Run conversions for DS refactor

* Run snapshot testing on conversions

* Normalize output name

* Update snapshots to include all panel and variable cases

* fix lint

* fix lint

* fix test and go lint

* more go lint

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variable (#108237)

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variables

* add conversion

* App Installer: Authorizer support (#108419)

* Chore: use `satisfies` and remove a load of `any`s (#108397)

use satisfies and remove a load of anys

* improve logging and fail unified-storage migration with more than 0 errors (#108471)

improve logging and fail unified-storage migration with more than 0 errors

* fix conversion test

* Secrets: Create more granular fixed roles for SecureValues (#108382)

* Provisioning: Fix bug in job progress recording (#108440)

Fix bug in job progress recording

* Provisioning: Fix ImportAllPanelsFromLocalRepository test (#108441)

* Provisioning: Skip flaky test

* Fix flaky provisioning test

* Fix lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* BulkDeleteProvisionedResource: Move progress bar into a second step (#108417)

* Move progress bar into a second step

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* [Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body (#108476)

[Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body to make sure new versions have the same settings.

---------

Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>

* Dashboard Schema V2: Refactor VizConfigKind to follow DataQueryKind convention (#108148)

* Dashboards API: Register v2alpha2 API

* Prepare conversion functions

* Fix test

* Refactor VizConfigKind to follow DataQueryKind convention

* fix tests

* use new dataquerykind convention alpha 2

* add conversion

* fix tests

* fix tests

* fix another test

* Fix merge

---------

Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>

* fix k8s codegen

* Update e2e-playwright/dashboards/TestV2Dashboard.json

* Update e2e/dashboards/TestV2Dashboard.json

* revert app generation for non-related apps

* try again

* another try

* also revert folder and secret app generation

* v2alpha1 provisioned dashboard

* Fix kind

* Fix conversion snapshots

* Update API discovery registry

* Rename to v2beta1

* Rename migrations

* Update apps/dashboard/pkg/apis/dashboard/v2beta1/doc.go

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

* Ensure conditional rendering and other non changed properties

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Haris Rozajac <58232930+harisrozajac@users.noreply.github.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-07-30 15:01:27 +02:00
Roberto Jiménez Sánchez c7c0268594 Provisioning: bulk delete resources by name (#108833)
* Add resource reference to spec

* Add FindResourcePath

* Fix formatting

* Use ForKind client

* Add unit test for new method

* Format code

* Add integration tests

* Fix unit tests

* Fix formatting

* Find out preferred version based on group and kind

* Handle trailing slash for folders

* Format code

* Fix linting

* Add integration test for folder bulk deletion

* Format code

* Format discovery file

* Deduplicate paths for deletion
2025-07-29 14:29:35 +02:00
Roberto Jiménez Sánchez 16339b0788 Provisioning: Move repository file API (#108514)
* Bump nanogit version
* Embed git repository in the Github one
2025-07-24 23:28:59 -05:00
Roberto Jiménez Sánchez 946358900c Provisioning: Add warning job status and different strictness (#108446)
* Add warning job status

* Different strictness for export than for pulling

* Fix some unit tests

* Fix unit tests

* Update frontend to display the warning status

* Fix status badge

* Add missing return statement

* Improve text in alert
2025-07-23 09:33:25 +00:00
Dominik Prokop d72a70d246 Dashboards API: v2alpha2 missing pieces (#108293)
* Dashboards API: v2alpha2 missing pieces

* Fix issue with dashboard client scope for alpha versions

As we now have 2 different alpha versions for v2 we need to store the
clients separately.

* Improve debuggability of provisioning export test

- Add a helper function to print the tree structure.
- Be explicit about the expected file names expected in each case.

* Update pkg/registry/apis/dashboard/mutate.go

* Update pkg/services/authz/zanzana/server/server.go

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>

* Review

* go lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
2025-07-22 11:44:05 +02:00
Roberto Jiménez Sánchez 7e0848294e Provisioning: Use Nanogit for basic git operations in Github repository type (#107889) 2025-07-10 09:46:38 -07:00
Roberto Jimenez Sanchez 106206ae93 Ignore delete error if not found 2025-07-10 12:45:35 +02:00
Stephanie Hingtgen 2484402f7a address PR comments 2025-07-10 12:45:35 +02:00
Stephanie Hingtgen 4386085aa9 fix linter 2025-07-10 12:45:35 +02:00
Stephanie Hingtgen bd81243bbb Git sync: Implement folder deletion 2025-07-10 12:45:35 +02:00
Ryan McKinley 3f502f305d Chore: Update mocks with recent mockery (#107816) 2025-07-09 09:15:34 +02:00
Yunwen Zheng fe4abf2221 Git Sync UI: Delete Provisioned Dashboard Flow (#106593)
* DeleteProvisionedDashboardDrawer: delete provisioned dashboard flow set up with drawer

* clean up

* add tests

* more test and clean up

* revert endpoint change

* adjust tests

* remove unuse codes

* fix type, fix test, add read only message

* small changes

* fix test, i18n fix

* comments

* Fix bug for file deletion using a branch

* PR comments update

* Use the provided ref for parser so that URLs work

* call useDeleteRepositoryFilesWithPathMutation in delete drawer component directly

* remove console log

* Update public/app/features/dashboard-scene/components/Provisioned/DashboardEditFormSharedFields.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* PR comments

* use string for fields

* extract handle request logic from save form and delete form and put into one hook

* Add test for useProvisionedRequestHandler

* Update public/app/features/dashboard-scene/components/Provisioned/DashboardEditFormSharedFields.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/dashboard-scene/components/Provisioned/DashboardEditFormSharedFields.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/dashboard-scene/components/Provisioned/DashboardEditFormSharedFields.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/dashboard-scene/components/Provisioned/DashboardEditFormSharedFields.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.test.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* use drawer close in both save form and delete form

* Add back panelEditor onDiscard

* add panelEditor onDiscard to delete flow

* Update public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
2025-06-18 15:14:03 -04:00
Jean-Philippe Quéméner 002f46736a refactor(unified-storage): move generated protos to own pkg (#105356) 2025-05-15 21:36:52 +02:00
Ryan McKinley 2e51096eb4 Provisioning: Export dashboards with conversion errors (#104369) 2025-04-24 14:16:25 +01:00
Ryan McKinley b09d79b21c K8s/Dashboard: Promote from alpha1 to beta1 (#104009) 2025-04-23 20:54:35 +03:00
Ryan McKinley d6dbc0a421 Provisioning: Migrate use history only with github (#104219) 2025-04-21 21:10:37 +03:00
Roberto Jiménez Sánchez 3dda7ccc30 Provisioning: folder not found legacy migration (#104225) 2025-04-21 16:20:39 +03:00
Ryan McKinley c8f65a0348 Provisionig: Add skipDryRun parameter to the /files/ resource (#104152) 2025-04-21 11:49:06 +03:00
Roberto Jiménez Sánchez 55a2b77386 Provisioning: refactor commit signature logic (#104055)
* WIP: Separate signature package

* Add some unit tests

* Test factory

* Finish unit test coverage signature package

* Fix register

* Add FIXME

* Add more coverage

* Add more coverage

* Fix migrate tests

* Fix unit tests
2025-04-16 10:04:19 +01:00
Roberto Jiménez Sánchez 08316103b5 Provisioning: refactor and test migrate job (#103976)
* Rename and make folder migrator public

* Move skipRepo out of tree

* Move folder logic out of worker

* Use wrapper function for clone

* Introduce storage swapper

* Use RepositoryResources interface

* Add TODO

* Signature with option

* Use plural for folders

* Split more logic out of worker

* Refactor further

* Remove todo

* Refactor further

* Test worker

* Add test for unified storage

* Add mock for bulk store

* Clean line

* Test more about storage

* Happy path storage test

* Finish storage tests

* Add more interfaces

* Add some tests legacy

* Complete test coverage legacy

* Complete coverage legacy

* Fix compilation

* Add initial folders tests

* Finish coverage for folders

* Test namespace cleaner

* Fix typo

* Add some tests for resources

* Finish coverage migrate package

* Update tests

* Fix jobs tests

* Unit test users

* Convert user tests to table tests

* Fix linting in tests

* Fix typo

* Use PreloadAllUserInfo

* Add FIXME
2025-04-15 15:02:19 +01:00
Ryan McKinley 0283c98e30 K8s/Folders: Use v1beta1 and app-sdk based spec (#103975) 2025-04-14 23:20:10 +03:00
Marco de Abreu c47ab101d1 Dashboards: Add Dashboard Schema validation (2) (#103844)
* Activate schema validation and align underlying systems

* update to save as v0 if not the right schema version

* Resolve merge conflicts

* Move RequireApiErrorStatus to tests package

* Add mutation tests

* Fix lint

* Only do min version check if dashboard is v1

* Fix lint and disable provisioning test

* Revert provisioning changes

* Revert more tests and add schema test

* Reran gen

* SQL Dashboard save

* Adjust APIVERSION

* Fixed mutation test

* Add logging on downgrade

---------

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-04-11 23:05:41 +02:00
Ryan McKinley ac7edd3032 Provisioning: Skip validation when writing v1 dashbaords (#103893) 2025-04-11 16:04:00 +00:00
Ryan McKinley 664e5255fe Provisioning: Use role based access when the target does not yet exist (#103862)
* role based fallback

* disable permissions cache with provisioning

* fallback to role based

* test with editor (not admin)

* test with editor (not admin)

* fix imports

* lint

* editor can create folders
2025-04-11 17:47:26 +03:00
Stephanie Hingtgen f5ad1ef69b K8s: Folders: Add v1 api (#103842) 2025-04-11 13:09:52 +01:00
Ryan McKinley 0d20680695 Provisioning: Improve PR feedback (#103537)
* fix ref

* dryRun before save

* reuse code for create vs update

* update comments

* can update PRs

* return useful errors

* update preview links

* lint fix

* update some wording

* update mocks

* fixed

* Update pkg/registry/apis/provisioning/resources/dualwriter.go

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* more updates

* fix link

* error string

* dry run

* refactor galore

* with template test

* very basic tests

* more test cases

* remove generator

* more tests

* fix lint

* multiple files now

* merge main

* merge main

* fix snapshots

* fix snapshots

* slugify title

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
2025-04-11 11:06:16 +01:00
Roberto Jiménez Sánchez 47664a7d51 Provisioning: unit test sync job (#103636)
* Split in multiple files

* Refactor sync even further

* Move more things between RepositoryResources

* Add status patcher

* Interface for sync functions

* Interface for compare function

* Add syncer back

* Move interfaces

* Move execute complete

* Return currentRef in syncer

* Add repository status test

* Add initial sync tests

* Fix a couple of spots

* Make initial sync tests work

* Fix register.go

* Add initial tests for sync worker

* Finish tests for sync

* Add incremental tests

* Add TODO

* Finish incremental tests

* Move folder creation to full sync

* Move interfaces

* Add initial full sync tests

* Update tests

* Reshape things

* Add changes test

* Fix register

* Add some tests

* Add more tests

* Add test

* WIP

* WIP: delete test

* Add more full test

* More tests

* Add tests for folder creation

* Add folder tests full sync

* Full coverage full sync

* Clean up tests

* Add more tests for changes function

* Enhance tests for Changes function to cover error scenarios and empty paths

- Added test cases for handling empty file paths and folder resources.
- Updated error message formatting in the Compare function for clarity.

* Add unit tests

* Failed initial patch

* Add tests failed repository resources

* Add test failed getting client

* Test for successful and unsuccessful syncs

* Add final tests for worker

* Fix existing tests

* Add missing test

* Fix spelling mistake

* Fix flake in changes test
2025-04-10 13:55:49 +01:00
Mariell Hoversholm 757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
Ryan McKinley 3f9e01b371 Provisioning: Use AccessChecker to verify if request has access to the parsed object (#103646) 2025-04-09 15:48:58 +01:00
Ryan McKinley d3e6e308a0 K8s/ManagedBy: Enforce who can CRUD provisioning resources (#103322) 2025-04-08 14:17:33 +03:00
Roberto Jiménez Sánchez 4f05b42275 Provisioning: initial unit tests push job (#103577)
* Add repository resources interface for export worker

* Add mocks for repository resources

* Add unit tests for ExportWorker's IsSupported method

* Add unit tests for ExportWorker's Process method, covering scenarios for missing export settings, write permissions, branch restrictions, and client creation failures.

* Fix unit tests

* Single function

* Add more unit tests

* Add test for failed folder

* Fail export folder errors

* Add another test

* Positive folder export

* Too many folder export errors

* Too many errors on folder export

* Partial folder errors

* Add test for nested folder

* Add test dashboard export

* More cases

* Ignore existing dashboards

* Fix folder tests

* Fix clonable test

* Add clone failure test

* Add test clean up without push

* Working tests

* Use mock clonable

* Add unit tests for IsWriteAllowed

* Add behaviour to cover ref equal to configured branch

* Fix worker test

* Fix linting

* Split clone and push

* Wrapper for clone and push
2025-04-08 12:44:11 +02:00
Ryan McKinley 1faa855ff1 Provisioning: Improve sample provisioning (#103508) 2025-04-07 12:37:18 +03:00
Roberto Jiménez Sánchez 364247e26b Provisioning: add most mocks to unit test provisioning jobs (#103446)
* Use interface for factories

* Add most mocks to unit test jobs

* Generate more mocks for main jobs package

* Interface for parser

* Use clients directly without parser

* Regenerate mock

* Eliminate resource clients from Parser

* Interface for folder tree
2025-04-04 15:42:20 +03:00
Roberto Jiménez Sánchez ea02e2e081 Provisioning: refactor dry-run and run logic to be stricter and more concise (#103357)
* Separate DryRun into separate method

* Fix linting

* Remove errors

* Remove checks in dualwriter

* Fix unit tests

* Add TODOs

* Dry Run as non-critical error

* Add TODOs

* Address TODO

* Fix tests

* Fix linting

* Deprecate dashboard name from path completely

* Use MissingName error also in parser

* Return 206 for non-critical errors

* Remove TODOs for previous dry-run
2025-04-04 13:31:48 +03:00
Ryan McKinley c5d76a8bba Provisioning: Require a name in the saved resource (#103309) 2025-04-03 18:58:05 +03:00
Roberto Jiménez Sánchez 800aa7827d Provisioning: Ensure path to file exists in Files endpoints (#103343)
* Create ensure folder path exists

* Fix issue with folder names generated in 2 different ways
2025-04-03 13:50:41 +02:00
Ryan McKinley 34192e3ad0 Provisioning: Support migrate to local (#103247) 2025-04-02 11:25:06 +03:00
Ryan McKinley f087d5f61b Provisioning: validate files we read (#103197) 2025-04-01 17:49:08 +03:00