Commit Graph

166 Commits

Author SHA1 Message Date
Roberto Jiménez Sánchez e2913815d3 Provisioning: Build resource clients for operators (#110699)
---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-09-05 17:52:57 +00:00
Roberto Jiménez Sánchez 1b9e479b68 Provisioning: Abstract away how we build discovery and dynamic clients (#110662)
Abstract away how we get discovery and dynamic clients
2025-09-05 09:22:30 +00:00
Stephanie Hingtgen 29ef525923 Provisioning: Allow disabling controllers (#110641) 2025-09-05 07:48:23 +02:00
Stephanie Hingtgen 9ddc70423b Provisioning: Cleanup tester interface (#110640)
* Provisioning: Cleanup tester interface

* undo accidental change

* cleanup

* cleanup test
2025-09-05 07:47:27 +02:00
Stephanie Hingtgen f302a3d538 Provisioning: Cleanup unused variables in controller (#110637) 2025-09-05 00:51:26 +00:00
Roberto Jiménez Sánchez 7d630ec3b1 Provisioning: Refactor tweaks to support MT controllers (#110581)
* Refactor common code to support MT controllers

* Delete original status files
2025-09-04 10:06:50 +00:00
Daniele Stefano Ferru 451d6abe15 Provisioning: Fix patching released resources when Repository is deleted (#110295)
* Provisioning: Use merge patch instead of json path to release orphan resources

* rolling back to json Patch

* adding TODO for testing

* adding integration test

* using struct

* addressing comments on tests
2025-09-02 21:13:43 +02:00
Matheus Macabu 1e926a29c0 Secrets: Extract external facing decrypt types to apps (#110432) 2025-09-02 10:30:29 +02:00
Roberto Jiménez Sánchez 4eadc823a9 Provisioning: Move repository package to provisioning app (#110228)
* Move repository package to apps

* Move operators to grafana/grafana

* Go mod tidy

* Own package by git sync team for now

* Merged

* Do not use settings in local extra

* Remove dependency on webhook extra

* Hack to work around issue with secure contracts

* Sync Go modules

* Revert "Move operators to grafana/grafana"

This reverts commit 9f19b30a2e.
2025-09-02 09:45:44 +02:00
Misi a5c05ba9c1 IAM: Moving code to the /pkg/apps/iam folder (#109985)
* wip

* Gen GetTeams with app sdk

* Revert some changes, cleanup

* Format iam_manifest.go

* Remove generated file

* Regenerate openapi defs

* Cleanup

* Remove TODO
2025-08-28 12:32:15 +02:00
Roberto Jiménez Sánchez 93a35fc7be Provisioning: Move apifmt, loki and safepath to provisioning app (#110226)
* Move apifmt

* Move safepath

* Move Loki package

* Regenerate Loki mock

* Missing file for Loki
2025-08-27 13:26:48 -05:00
Costa Alexoglou 9785e573aa Provisioning: Fix Dashboard Creation For First-Level Repository Folders (#109962) 2025-08-27 12:20:57 +02:00
Roberto Jiménez Sánchez e196f13b2b Provisioning: Introduce Repository Factory with extras (#110018) 2025-08-26 10:37:47 +02:00
Ryan McKinley 99f640fbab Provisioning: Always fetch token (#110116) 2025-08-25 18:57:27 +02:00
Roberto Jiménez Sánchez e7ccefcf92 Provisioning: Add Standalone Job Controller Without Job Processing (#109610)
* Add standalone job controller
* Add makefile
* Add limit on the current implementation
* Move job controllers to app package
* Add TLS flags
2025-08-25 08:48:40 +00:00
Roberto Jiménez Sánchez 9a668ce06f Provisioning: Use API Client For Job Processing (#110044) 2025-08-25 09:46:20 +02:00
Ryan McKinley ce65391067 Provisioning: Use inline secrets for gitsync (#109908)
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
2025-08-22 18:38:28 +02:00
Roberto Jiménez Sánchez ed13225b71 Provisioning: Refactor History Writer to Use API Client (#110031)
* Separate interfaces for writing and reading in job history

* Register the 2 writers

* Commit unsaved file

* Do not validate or mutate history jobs

* Change the TODOs
2025-08-22 13:41:00 +02:00
Ryan McKinley 018e810d43 Provisioning: Remove clonedir (not used by nanogit) (#109954)
remove clone dir
2025-08-21 10:04:30 +00:00
Roberto Jiménez Sánchez 61d137992b Provisioning: Mark repository as unhealthy if hooks fail (#109788) 2025-08-21 08:32:23 +00:00
Roberto Jiménez Sánchez c37a03263f Provisioning: Fix Bug Blocking Changing Pull Target During Onboarding (#109892)
* Fix bug changing target for unsynced repository

* Fix linting
2025-08-20 09:02:19 +00:00
Stephanie Hingtgen 77c96cbd06 Secrets: Refactor setting additional decrypter (#109877) 2025-08-19 16:10:43 -05:00
Mariell Hoversholm 0fc29cbaae Rendering: Remove SVG sanitization (#109797) 2025-08-19 11:34:12 +02:00
Roberto Jiménez Sánchez 1ff39510d3 Provisioning: Check For Resource Ownership Before Operations (#109582) 2025-08-15 10:05:53 +03:00
Roberto Jiménez Sánchez ffc7508a46 Provisioning: Enforce instance repository isolation (#109512)
* Fix validation on repository creation

* Fix linting

* Do not count the provided one

* Fix test

* Fix tests
2025-08-14 10:19:40 +00:00
Stephanie Hingtgen 85166512cb Provisioning: Update for standalone (#109524) 2025-08-12 17:37:16 -05:00
Ryan McKinley edcb6e6895 Provisioning: Fix OpenAPI generation (#109513)
* post-process spec

* update client

* fix the openapi

* revert playlist change

* Fix unmanaged resource counts

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
2025-08-12 17:25:19 +00:00
Roberto Jiménez Sánchez 5c5729a25d Provisioning: Fix migration behavior for folder-type repositories (#109518)
Fix migration issues for folder sync
2025-08-12 16:48:13 +02:00
Roberto Jiménez Sánchez f2303a8ad9 Provisioning: Include Ref URLs in Job Status (#109464)
* Add URLs to Job spec

* Rename them as RefURLs

* Implement RefURLs for Github

* Add Ref URLs to Jobs

* Worker Test

* Create the branch in the staged writer

* Regenerate Git mock

* Format code

* Consolidate ResourceURLs and RefURLs into one

* Fix broken tests
2025-08-12 11:57:37 +02:00
Roberto Jiménez Sánchez 8a690e977e Provisioning: Return unmanaged count in stats (#109462)
* Unmanaged counts

* Add integration test

* Generate open api spec
2025-08-12 09:17:25 +02:00
Ryan McKinley e0404f924c K8s/SecureValues: Wire InlineSecureValueSupport to apistore (#109449)
* inline wire

* extra fields

* add variable

* wire
2025-08-11 15:22:56 +03:00
Ryan McKinley 439d4d383c Provisioning: Hide HistoricJobs from OpenAPI spec (#109454)
histtoric jobs
2025-08-11 10:47:33 +00:00
Roberto Jiménez Sánchez f80627f228 Provisioning: Store Recent Jobs Temporarilly in Unified Storage (#109392)
* Add HistoryJob back to Spec

* Generate client

* Put back History Jobs

* Add a controller to remove historic jobs after some minutes

* Start only if Loki is not used

* Format code

* Update OpenAPI spec

* Change log level

* Fix condition

* Fix staticcheck

* Use provisioning identity

* Fix registration APIs

* Fix readonly issue

* Update OpenAPI
2025-08-11 13:10:15 +03:00
Roberto Jiménez Sánchez 2a0944a92a Provisioning: Miscellanous Fixes For Flaky Integration Tests (#109330)
* Enhance error messages individual files errors

* Split integration tests into multiple files

* Fix missing import

* Rename file to repository test

q

* Wrap worker errors with more context

* Fixed tests

* Add nolint

* Wait for specific job in multiple spots
2025-08-07 14:26:22 +02:00
Roberto Jiménez Sánchez 2b15fa0492 Provisioning: Fix missing ref in staging options (#109311)
Fix missing ref in staging options
2025-08-07 12:00:31 +02:00
Ryan McKinley ace670c6d0 Secrets: Support variadic args in decrypt service (#109280) 2025-08-07 09:54:24 +00:00
Roberto Jiménez Sánchez c82d2af867 Provisioning: introduce jobs controller (#109252)
* Add basic job controller
* Replace the existing in-memory channel
2025-08-07 12:51:29 +03:00
Roberto Jiménez Sánchez 0b5fc9a736 Provisioning: Use loki for recent jobs (#109249) 2025-08-06 18:09:28 +00:00
Roberto Jiménez Sánchez fec9cd550a Provisioning: Reclaim executing jobs (#109228)
* Reclaim Executing Jobs

* Remove FIXME

* Format code

* Fetch job before update

* Add claims for all namespaces

* Remove unused import

* Update pkg/registry/apis/provisioning/jobs/concurrent_driver.go

* Update pkg/registry/apis/provisioning/jobs/concurrent_driver.go

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-08-06 17:35:31 +00:00
Matheus Macabu b0b1340c19 Secrets: Remove Close from decrypt client interface (#109250) 2025-08-06 14:02:13 +02:00
Matheus Macabu fc13a1ae0c Provisioning: Change decrypter service identity to api group name (#109190) 2025-08-06 09:05:03 +02:00
Matheus Macabu fc117afebc Secrets: Provide grpc and local inline secure value client (#109179)
* TestUtils: Add helper to create certs

* InlineSecureValue: Move to its own package rather than service

* Settings: Rename decrypt server configs to grpc prefixed as it is more generic

* InlineSecureValue: Add gRPC client

* InlineSecureValue: Add service provider grpc/local depending on setting.Cfg

* Wire: Add ProvideInlineSecureValueService to basic wireset

* InlineSecureValue: Re-export in root secret pkg and generate mocks for interface

* Settings: Unify decrypt+inline setting into generic grpc client
2025-08-06 09:04:55 +02:00
Roberto Jiménez Sánchez 377cf1fef9 Provisioning: concurrent job queue processing (#109181)
* Add concurrent job processing

* Use 5 drivers instead

* Revert "Use 5 drivers instead"

This reverts commit 9c1fc3c70b.

* Format file
2025-08-05 13:38:37 -05:00
Roberto Jiménez Sánchez d874bc08b7 Provisioning: Skip incremental rync ref didn't change (#109174)
Skip incremental sync if ref didn't change

Avoids unnecessary load on the job queue by not queuing if the ref
didn't change from the last sync.
2025-08-05 13:26:47 +02:00
Matheus Macabu 81f544e186 Secrets: Add service name as explicit parameter for Decrypt (#109054)
* Secrets: Add service name as explicit parameter for Decrypt

* Apply suggestions from code review

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-08-04 08:51:15 +02:00
Stephanie Hingtgen 9f4773c9a5 Provisioning: Extract to apps submodule (#109074) 2025-08-01 14:35:18 -05:00
Roberto Jiménez Sánchez 9ca0750134 Provisioning: skip export of already managed resources and parent folder export (#108893)
* Skip export on already managed resources

* Add integration test

* Add integration test

* Handle nothing to commit error

* Fix leaky abstraction issue

* Handle the no commit error on commit and not on push

* Fix linting

* Some fixes for integration test

* Improve tree to work with a root

* Some fixes with hacks

* Add additional checks

* Fix comment

* Fix path problems in test

* Fix more stuff

* Revert to use empty tree

* Remove changes in tree

* Finally fix the tests work

* Remove stale comment

* Fix linting

* Revert changes in test

* Fix error message for folder not found in resource tree

Co-authored-by: roberto.jimenez <roberto.jimenez@grafana.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-01 11:17:15 +00:00
Dominik Prokop 20b08ccaab Add v2beta1 api version: Consolidate schema breaking changes (#108172)
* Revert "Revert: Future-proofing query and data source model in Dashboard Sche… (#107985)"

This reverts commit 13a89d4ae3.

* Revert "Revert "Schema V2: Simplify annotations v1<->v2 conversions" (#107984)"

This reverts commit 2b8c5bea1a.

* make gen apps

* e2e update

* Use v2alpha2 by default (#108177)

* Use v2alpha2 by default

* Apply only DS changes to alpha2

* Use v2alpha2 by default except to query

* Create a v2 index in @grafana/schema

* Update path and apply lint

* Update tests

* Update imports to v2 status

* Fix failing openapi test

* Schemav2 breaking changes: conversion implementation (#108224)

* provision v2alpha1 dashboard

* Run conversions for DS refactor

* Run snapshot testing on conversions

* Normalize output name

* Update snapshots to include all panel and variable cases

* fix lint

* fix lint

* fix test and go lint

* more go lint

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variable (#108237)

* Schema v2: Introduce group/datasource convention to GroupBy and AdHoc variables

* add conversion

* App Installer: Authorizer support (#108419)

* Chore: use `satisfies` and remove a load of `any`s (#108397)

use satisfies and remove a load of anys

* improve logging and fail unified-storage migration with more than 0 errors (#108471)

improve logging and fail unified-storage migration with more than 0 errors

* fix conversion test

* Secrets: Create more granular fixed roles for SecureValues (#108382)

* Provisioning: Fix bug in job progress recording (#108440)

Fix bug in job progress recording

* Provisioning: Fix ImportAllPanelsFromLocalRepository test (#108441)

* Provisioning: Skip flaky test

* Fix flaky provisioning test

* Fix lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* BulkDeleteProvisionedResource: Move progress bar into a second step (#108417)

* Move progress bar into a second step

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* [Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body (#108476)

[Dashboard Schema Codegen] Move dashboard CUE codegen block back up into kind body to make sure new versions have the same settings.

---------

Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>

* Dashboard Schema V2: Refactor VizConfigKind to follow DataQueryKind convention (#108148)

* Dashboards API: Register v2alpha2 API

* Prepare conversion functions

* Fix test

* Refactor VizConfigKind to follow DataQueryKind convention

* fix tests

* use new dataquerykind convention alpha 2

* add conversion

* fix tests

* fix tests

* fix another test

* Fix merge

---------

Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>

* fix k8s codegen

* Update e2e-playwright/dashboards/TestV2Dashboard.json

* Update e2e/dashboards/TestV2Dashboard.json

* revert app generation for non-related apps

* try again

* another try

* also revert folder and secret app generation

* v2alpha1 provisioned dashboard

* Fix kind

* Fix conversion snapshots

* Update API discovery registry

* Rename to v2beta1

* Rename migrations

* Update apps/dashboard/pkg/apis/dashboard/v2beta1/doc.go

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

* Ensure conditional rendering and other non changed properties

---------

Co-authored-by: Ivan Ortega <ivanortegaalba@gmail.com>
Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
Co-authored-by: Will Assis <35489495+gassiss@users.noreply.github.com>
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Yunwen Zheng <yunwen.zheng@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Austin Pond <IfSentient@users.noreply.github.com>
Co-authored-by: Haris Rozajac <58232930+harisrozajac@users.noreply.github.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
2025-07-30 15:01:27 +02:00
Roberto Jiménez Sánchez f4e45bf3bc Provisioning: bulk move by name (#108869)
* Add ResourceRef to move job spec
* Implement move by name
* Use map to struct
2025-07-30 11:30:24 +03:00
Roberto Jiménez Sánchez 1a246739ed Provisioning: Add bulk move job (#108762)
* Add move job to spec

* Add move worker

* Single commit mode

* Add initial integration tests

* Format code

* Improve test setup

* Add fixes for integration tests

* Implement move operation

* Format file

* Fix API documentation
2025-07-29 17:24:17 +02:00