* Deprecate Legacy Storage Migration in Backend
* Change the messaging around legacy storage
* Disable cards to connect
* Commit import changes
* Block repository creation if resources are in legacy storage
* Update error message
* Prettify
* chore: uncomment unified migration
* chore: adapt and fix tests
* Remove legacy storage migration from frontend
* Refactor provisioning job options by removing legacy storage and history fields
- Removed the `History` field from `MigrateJobOptions` and related references in the codebase.
- Eliminated the `LegacyStorage` field from `RepositoryViewList` and its associated comments.
- Updated tests and generated OpenAPI schema to reflect these changes.
- Simplified the `MigrationWorker` by removing dependencies on legacy storage checks.
* Refactor OpenAPI schema and tests to remove deprecated fields
- Removed the `history` field from `MigrateJobOptions` and updated the OpenAPI schema accordingly.
- Eliminated the `legacyStorage` field from `RepositoryViewList` and its associated comments in the schema.
- Updated integration tests to reflect the removal of these fields.
* Fix typescript errors
* Refactor provisioning code to remove legacy storage dependencies
- Eliminated references to `dualwrite.Service` and related legacy storage checks across multiple files.
- Updated `APIBuilder`, `RepositoryController`, and `SyncWorker` to streamline resource handling without legacy storage considerations.
- Adjusted tests to reflect the removal of legacy storage mocks and dependencies, ensuring cleaner and more maintainable code.
* Fix unit tests
* Remove more references to legacy
* Enhance provisioning wizard with migration options
- Added a checkbox for migrating existing resources in the BootstrapStep component.
- Updated the form context to track the new migration option.
- Adjusted the SynchronizeStep and useCreateSyncJob hook to incorporate the migration logic.
- Enhanced localization with new descriptions and labels for migration features.
* Remove unused variable and dualwrite reference in provisioning code
- Eliminated an unused variable declaration in `provisioning_manifest.go`.
- Removed the `nil` reference for dualwrite in `repo_operator.go`, aligning with the standalone operator's assumption of unified storage.
* Update go.mod and go.sum to include new dependencies
- Added `github.com/grafana/grafana-app-sdk` version `0.48.5` and several indirect dependencies including `github.com/getkin/kin-openapi`, `github.com/hashicorp/errwrap`, and others.
- Updated `go.sum` to reflect the new dependencies and their respective versions.
* Refactor provisioning components for improved readability
- Simplified the import statement in HomePage.tsx by removing unnecessary line breaks.
- Consolidated props in the SynchronizeStep component for cleaner code.
- Enhanced the layout of the ProvisioningWizard component by streamlining the rendering of the SynchronizeStep.
* Deprecate MigrationWorker and clean up related comments
- Removed the deprecated MigrationWorker implementation and its associated comments from the provisioning code.
- This change reflects the ongoing effort to eliminate legacy components and improve code maintainability.
* Fix linting issues
* Add explicit comment
* Update useResourceStats hook in BootstrapStep component to accept selected target
- Modified the BootstrapStep component to pass the selected target to the useResourceStats hook.
- Updated related tests to reflect the change in expected arguments for the useResourceStats hook.
* fix(provisioning): Update migrate tests to match export-then-sync behavior for all repository types
Updates test expectations for folder-type repositories to match the
implementation changes where both folder and instance repository types
now run export followed by sync. Only the namespace cleaner is skipped
for folder-type repositories.
Changes:
- Update "should run export and sync for folder-type repositories" test to include export mocks
- Update "should fail when sync job fails for folder-type repositories" test to include export mocks
- Rename test to clarify that both export and sync run for folder types
- Add proper mock expectations for SetMessage, StrictMaxErrors, Process, and ResetResults
All migrate package tests now pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Update provisioning wizard text and improve resource counting display
- Enhanced descriptions for migrating existing resources to clarify that unmanaged resources will also be included.
- Refactored BootstrapStepResourceCounting component to simplify the rendering logic and ensure both external storage and unmanaged resources are displayed correctly.
- Updated alert messages in SynchronizeStep to reflect accurate information regarding resource management during migration.
- Adjusted localization strings for consistency with the new descriptions.
* Update provisioning wizard alert messages for clarity and accuracy
- Revised alert points to indicate that resources can still be modified during migration, with a note on potential export issues.
- Clarified that resources will be marked as managed post-provisioning and that dashboards remain accessible throughout the process.
* Fix issue with trigger wrong type of job
* Fix export failure when folder already exists in repository
When exporting resources to a repository, if a folder already exists,
the Read() method would fail with "path component is empty" error.
This occurred because:
1. Folders are identified by trailing slash (e.g., "Legacy Folder/")
2. The Read() method passes this path directly to GetTreeByPath()
3. GetTreeByPath() splits the path by "/" creating empty components
4. This causes the "path component is empty" error
The fix strips the trailing slash before calling GetTreeByPath() to
avoid empty path components, while still using the trailing slash
convention to identify directories.
The Create() method already handles this correctly by appending
".keep" to directory paths, which is why the first export succeeded
but subsequent exports failed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix folder tree not updated when folder already exists in repository
When exporting resources and a folder already exists in the repository,
the folder was not being added to the FolderManager's tree. This caused
subsequent dashboard exports to fail with "folder NOT found in tree".
The fix adds the folder to fm.tree even when it already exists in the
repository, ensuring all folders are available for resource lookups.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Revert "Merge remote-tracking branch 'origin/uncomment-unified-migration-code' into cleanup/deprecate-legacy-storage-migration-in-provisioning"
This reverts commit 6440fae342, reversing
changes made to ec39fb04f2.
* fix: handle empty folder titles in path construction
- Skip folders with empty titles in dirPath to avoid empty path components
- Skip folders with empty paths before checking if they exist in repository
- Fix unit tests to properly check useResourceStats hook calls with type annotations
* Update workspace
* Fix BootstrapStep tests after reverting unified migration merge
Updated test expectations to match the current component behavior where
resource counts are displayed for both instance and folder sync options.
- Changed 'Empty' count expectation from 3 to 4 (2 cards × 2 counts each)
- Changed '7 resources' test to use findAllByText instead of findByText
since the count appears in multiple cards
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove bubbletee deps
* Fix workspace
* provisioning: update error message to reference enableMigration config
Update the error message when provisioning cannot be used due to
incompatible data format to instruct users to enable data migration
for folders and dashboards using the enableMigration configuration
introduced in PR #114857.
Also update the test helper to include EnableMigration: true for both
dashboards and folders to match the new configuration pattern.
* provisioning: add comment explaining Mode5 and EnableMigration requirement
Add a comment in the integration test helper explaining that Provisioning
requires Mode5 (unified storage) and EnableMigration (data migration) as
it expects resources to be fully migrated to unified storage.
* Remove migrate resources checkbox from folder type provisioning wizard
- Remove checkbox UI for migrating existing resources in folder type
- Remove migrateExistingResources from migration logic
- Simplify migration to only use requiresMigration flag
- Remove unused translation keys
- Update i18n strings
* Fix linting
* Remove unnecessary React Fragment wrapper in BootstrapStep
* Address comments
---------
Co-authored-by: Rafael Paulovic <rafael.paulovic@grafana.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix race condition causing unhealthy repository message to be lost
This commit fixes a race condition in the provisioning repository controller
where the "Repository is unhealthy" message in the sync status could be lost
due to status updates being based on stale repository objects.
## Problem
The issue occurred in the `process` function when:
1. Repository object was fetched from cache with old status
2. `RefreshHealth` immediately patched the health status to "unhealthy"
3. `determineSyncStatusOps` used the stale object to check if unhealthy
message was already set
4. A second patch operation based on stale data would overwrite the
health status update
## Solution
Introduced `RefreshHealthWithPatchOps` method that returns patch operations
instead of immediately applying them. This allows batching all status updates
(health + sync) into a single atomic patch operation, eliminating the race
condition.
## Changes
- Added `HealthCheckerInterface` for better testability
- Added `RefreshHealthWithPatchOps` method to return patch ops without applying
- Updated `process` function to batch health and sync status updates
- Added comprehensive unit tests for the fix
Fixes the issue where unhealthy repositories don't show the "Repository is
unhealthy" message in their sync status.
* Fix staticcheck lint error: remove unnecessary nil check for slice
* provisioning: detect stale sync status and trigger resync
When sync jobs expire and are cleaned up by the expired job cleanup
controller, the Repository sync status remains stuck in Pending or
Working state. This prevents new sync jobs from being queued because
shouldResync() blocks on these states.
This change adds detection logic in shouldResync() to check if a sync
job referenced in the sync status still exists. If the job doesn't exist
(NotFound), we trigger a resync to reconcile the stale state.
Fixesgrafana/git-ui-sync-project#626
* test: remove unused mocks and fix test case
- Remove unused mockRepositoryLister and mockRepositoryNamespaceLister types
- Remove unused imports (labels, listers)
- Remove test case for sync disabled scenario as we don't care about sync enabled state when detecting stale status
* Provisioning: Improve logging and tracing in job processing
- Add comprehensive tracing with OpenTelemetry spans across all job operations
- Enhance logging with consistent style: lowercase, concise messages, appropriate log levels
- Use past tense for completed lifecycle events (e.g., 'stopped' vs 'stop')
- Add structured logging with contextual attributes for better searchability
- Handle graceful shutdowns without throwing errors on context cancellation
- Refactor Cleanup method into listExpiredJobs and cleanUpExpiredJob for better code quality
- Avoid double logging by only logging errors when handled locally
- Add tracing and logging to historyjob controller cleanup operations
Files modified:
- pkg/registry/apis/provisioning/jobs/driver.go: Add tracing spans and improve error handling for graceful shutdown
- pkg/registry/apis/provisioning/jobs/concurrent_driver.go: Add tracing and consistent logging
- pkg/registry/apis/provisioning/jobs/persistentstore.go: Add comprehensive tracing and logging to all public methods, refactor cleanup
- apps/provisioning/pkg/controller/historyjob.go: Add tracing and improve logging consistency
* Update pkg/registry/apis/provisioning/jobs/persistentstore.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor logging in persistentstore.go
- Remove debug log statements at the start of job operations for cleaner output
- Maintain structured logging with contextual attributes for improved traceability
Files modified:
- pkg/registry/apis/provisioning/jobs/persistentstore.go: Clean up logging for job operations
* Enhance logging and tracing in provisioning job operations
- Introduce OpenTelemetry spans for better observability in job processing and webhook handling
- Improve structured logging with contextual attributes for key operations
- Remove unnecessary tracing spans in long-running functions to streamline performance
- Update error handling to record errors in spans for better traceability
Files modified:
- pkg/registry/apis/provisioning/controller/repository.go: Add tracing and structured logging to sync job operations
- pkg/registry/apis/provisioning/jobs/concurrent_driver.go: Remove tracing span from long-running function
- pkg/registry/apis/provisioning/jobs/driver.go: Enhance logging and tracing in job processing
- pkg/registry/apis/provisioning/webhooks/webhook.go: Implement tracing and structured logging for webhook connections
* Update pkg/registry/apis/provisioning/jobs/driver.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Improve error handling in ConcurrentJobDriver to differentiate between graceful shutdown and unexpected stops
* Remove unused import in driver.go to clean up code
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Provisioning: Preserve in progress job data
* Refactor code and cover more situations
* Fix linting
* Fix issue with remove path operation for started time
* Cleanup
* prettier
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
* adding some logs to better understand what might be happening
* only focus this PR on improve logging in finalizer handling
* debug log before calling finalizers
* working on finalizers
* removing last todos, adding unit tests
* better use SupportedFinalizers name
* addressing comments
* wip: fix tests and add delete error in status
* chore: codegen
* chore: codegen openapi
* Merge remote-tracking branch 'origin/main' into provisioning-finalisers-fix-2
* update frontend client
* fix: errors in testing
* fix: breaking test
---------
Co-authored-by: Daniele Ferru <daniele.ferru@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Provisioning: Use merge patch instead of json path to release orphan resources
* rolling back to json Patch
* adding TODO for testing
* adding integration test
* using struct
* addressing comments on tests
* Move repository package to apps
* Move operators to grafana/grafana
* Go mod tidy
* Own package by git sync team for now
* Merged
* Do not use settings in local extra
* Remove dependency on webhook extra
* Hack to work around issue with secure contracts
* Sync Go modules
* Revert "Move operators to grafana/grafana"
This reverts commit 9f19b30a2e.
* Add HistoryJob back to Spec
* Generate client
* Put back History Jobs
* Add a controller to remove historic jobs after some minutes
* Start only if Loki is not used
* Format code
* Update OpenAPI spec
* Change log level
* Fix condition
* Fix staticcheck
* Use provisioning identity
* Fix registration APIs
* Fix readonly issue
* Update OpenAPI
- Add hooks to git, github and github webhooks to remove the.
- Implement deletion in secrets package.
- Add `Mutator` interface and hooks so that we can register any mutator.
- Add unit test coverage to those mutators.
- Move provider specific mutation from the massive `register.go` to the respective packages (e.g. `git` , `github`, etc).
- Add integration test for removal.
- Change the decryption fallback to simply check for the repository prefix.
- Provisioning: Begin using secrets store
- Refactor integration with secrets store
- Add back the legacy service
- Separate concerns for encrypt and decrypt
- Handle update within Encrypt function
- Add interface for secure value service
- Add feature flag for using secrets service
- Add the dual service for temporary solution.
* Add first integration tests for encrypted tokens
* Add integration test for app platform secrets
* Validate it has the name or not
* Create wire provider
* Always save to the secret if provided secret
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Roberto Jiménez Sánchez <jszroberto@gmail.com>
* Spike: Extras
* Attempt to wire it up
* Hack
* Fix issue with jobs
* Wire more things up
* Fix more wiring stuff
* Remove webhook secret key from main registration
* Move secret encryption also outside register
* Add TODOs in code
* Add more explanations
* Move connectors to different package
* Move pull request job into webhooks
* Separate registration
* Remove duplicate files
* Fix missing function
* Extract webhook repository logic out of the core github repository
* Use status patcher in webhook connector
* Fix change in go mod
* Change hooks signature
* Remove TODOs
* Remove Webhook methos from go-git
* Remove leftover
* Fix mistake in OpenAPI spec
* Fix some tests
* Fix some issues
* Fix linting
* Split in multiple files
* Refactor sync even further
* Move more things between RepositoryResources
* Add status patcher
* Interface for sync functions
* Interface for compare function
* Add syncer back
* Move interfaces
* Move execute complete
* Return currentRef in syncer
* Add repository status test
* Add initial sync tests
* Fix a couple of spots
* Make initial sync tests work
* Fix register.go
* Add initial tests for sync worker
* Finish tests for sync
* Add incremental tests
* Add TODO
* Finish incremental tests
* Move folder creation to full sync
* Move interfaces
* Add initial full sync tests
* Update tests
* Reshape things
* Add changes test
* Fix register
* Add some tests
* Add more tests
* Add test
* WIP
* WIP: delete test
* Add more full test
* More tests
* Add tests for folder creation
* Add folder tests full sync
* Full coverage full sync
* Clean up tests
* Add more tests for changes function
* Enhance tests for Changes function to cover error scenarios and empty paths
- Added test cases for handling empty file paths and folder resources.
- Updated error message formatting in the Compare function for clarity.
* Add unit tests
* Failed initial patch
* Add tests failed repository resources
* Add test failed getting client
* Test for successful and unsuccessful syncs
* Add final tests for worker
* Fix existing tests
* Add missing test
* Fix spelling mistake
* Fix flake in changes test
* Use interface for factories
* Add most mocks to unit test jobs
* Generate more mocks for main jobs package
* Interface for parser
* Use clients directly without parser
* Regenerate mock
* Eliminate resource clients from Parser
* Interface for folder tree
* Provisioning: Gracefully handle AlreadyExists errors
When this occurs, the job is already in the store. The vast majority of use-cases (which currently is all of them!) will
be fine with just accepting that as an OK.
* chore: make update-workspace
* fix: return error, but expect it in tests
* [Provisioning] Pay back some technical debt (#100720)
* Handle pagination in github client
* Add some unit test coverage
* Remove unknown repository
* Remove unknown leftover
* Revert "Add some unit test coverage"
This reverts commit 420c9674d2.
* Revert "Revert "Add some unit test coverage""
This reverts commit f7eca41957.
* Revert unit tests in github package
* Remove S3 case as it's now deprecated
* [Provisioning] Consolidate job status report in JobProgressRecorder (#100718)
* Log also successful operation
* Consolidate stop logic under TooManyErrors
* Use error for TooManyErrors
* Pass the progress recorder
* Define JobProgressRecorder interface
* Do not expect workers to return status
* Remove scenarios due to pointers
* Use recorder to manage the entire state
* Provisioning: Support rotating secrets (#100705)
* Provisioning: Refactor webhook to another interface (#100733)
* POC/Provisioning: Remove S3 references (#100734)
* Remove unused script
* Remove s3 references
* Provisioning: Keep the existing k8s name if it is specified in metadata (#100672)
* keep name
* keep name
* Revert "keep name"
This reverts commit 29f87bcaeb.
* Commit stale go.mod
* Keep name also for sync deletions
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
* Update preview banner copy
* Query Library: Move backend to enterprise (#100371)
* Fix wire
* Fix jobs table re-renders
* Provisioning: Refactor history to its own interface (#100735)
* Provisioning: Refactor history to its own interface
* refactor: use VersionedRepository
* Update API
* Provisioning: Test the GitHub client directly (#100808)
* Provisioning: Test the GitHub client directly
Instead of mocking the abstracted client, test it as well by mocking the underlying GitHub client. This also lets us
remove the mock for the abstracted client.
* refactor: move out helpers
* chore: set dependency owner
* Provisioning: Better clone/push error support (#100854)
* Provisioning: Replace searcher with one that knows about modes (#100857)
* Provisioning: Start in "mode5" when nothing exists in legacy (#100862)
* [Provisioning] Fix duplicate sync jobs triggered in controller (#100870)
* Improve logging on reasons why the controller triggered
* Fix messaging for sync job
* fix lint
* Provisioning: Move legacy export/import into a single migrate job (#100865)
* [Provisioning] Miscellanenous bug fixes and improvements (#100976)
* Error if found duplicate ID
* Fix issue with manual test button
* Fix issue with health errors not going away
* Display status in sync overview
* Use patch operations instead
* Trigger sync job after status update
* Convert Export Tab into modal
* Remove unused FieldSet import
* Only last 8 jobs
* Remove Links card
* Use button for Github Source Code
* Add actions to resources page
* Add resource column to Repository Resources
* Display Job Spec in RecentJobs
* Display dates in history page
* Display Avatar if available
* Improve styling of the avatar
* Update betterer
* Remove duplicate history header in history
* Commit betterer
* Address code styling issues
* update flags
* github v69
* v69
* POC/Provisioning: Add wizard (#100596)
* Chore: make update-workspace
* Chore: Fix lints (#101039)
* Provisioning: Workflows as write access (#101031)
* workflow as write access
* workflow as write access
* workflow as write access
* Update pkg/registry/apis/provisioning/repository/test.go
Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>
* POC/Provisioning: Add wizard (#100596)
* update refs
* update refs
* lint fix
* lint fix
* lint fix
* default everythign to read only
* reuse form components
* remove main
---------
Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* merge main
* Fix workflow types
* Betterer
* [Provisioning] Fix webhook and finalizer issues (#101052)
Fix webhook and finalizer issue
The maximum number of webhooks per repository is 20
* [Provisioning] Fix issue with last ref (#101056)
* Fix issue with last ref
* Update frontend code
* Fix the local tmp test
* Use lastRef
* POC/Provisioning: Simplify connect step (#101064)
* Fix sending workflows
* Use write for local
* Move connect action to the next step
* Remove wizard props
* Typo
* Redirect to wizard
* Show repo link after successful export
* Provisioning: Avoid starting sync jobs when using legacy storage (#101114)
* avoid starting sync jobs on legacy
* newlines
* Provisioning: Onboarding landing page (#101112)
* add landing page before wizard
* Update onboarding page
* Update URL
* Remove unused
* Add deleteAll button
* Improved text
* betterer
---------
Co-authored-by: Clarity-89 <homes89@ukr.net>
* Provisioning: use the sync job to finish the migrate job (#101107)
* Provisioning: Show progress more often (#101128)
* show progress bar earlier
* show progress bar earlier
* update wording to be less specific
* POC/Provisioning: Enable sync (#101131)
* update preview banner
* actualy remove and don't crash without provisioning flag
* Update db banner
* Provisioning: Export oldest items first (#101189)
* Provisioning: better branch handling (#101188)
* add missing file
* Provisioning: Fix tests (#101197)
* Provisioning: Refactor tests to be multiple functions
* Provisioning: Fix tests
* fix: make github-example sync
* fix misspell
* Provisioning: avoid migration wizard if things are already in unified storage (#101204)
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* more lint
* POC/Provisioning: Handle connect step errors (#101192)
* Provisioning: Unify status handling in repository and migrate steps
* Refactor: Move WorkflowsField outside RepositoryStep and use proper type
* Refactor: Improve repository verification error handling and UI
* Refactor: Simplify repository verification error handling
* Refactor: Simplify RepositoryStep component structure
* Refactor: Improve error handling in RepositoryStep
* Refactor: Remove redundant repository creation logic from ProvisioningWizard
* Refactor: Simplify RequestErrorAlert component
* show github error
* now will verify
* test .git
* recover from bad config
* Update error handling
* Remove unused prop
* merge upstream
* Show migration summary
* Update text
* Improve text
* Betterer
* [Provisioning] Review controller changes (#101216)
* Review health check conditions
* Move down the logic to set up the sync status
* Skip if it's only a health check rerun
* Fix health check conditions
* Preserve last ref
* Format code
* Rename to shouldSkipSync
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
* fix promotion step
* In the promotion pipeline, publish should depend on build
* fix promotion pipeline
* [Provisioning] Use smaller methods to process repository events (#101240)
* update codegen
* merge main
* Provisioning: Avoid localhost error loop (#101253)
* Provisioning: Update the recent jobs formatting (#101250)
format history
* [Provisioning] Refactor Pull Request & Lint worker (#101273)
* Refactor the code
* Refactor into separate files
* Consolidate linter flag in one spot
* Use global feature flags
* Commit betterer
* Remove from JSON the intermidiate flag
* Use again spec
* Clean up
* Revert changes in test
* POC/Provisioning: Remove sync confirm modal (#101281)
* [Provisioning] Remove linting from MVP (#101286)
* Remove Linting backend
* Re-generate client
* POC/Provisioning: Unify tags (#101218)
* Unify tags
* add both tags
* add tag types
* Check for the redirect only once
* Add fetch settings with delay hook
* Refetch settings
* Split hooks into separate files
* Cleanup
* Prettier
* Prettier
* Remove lint code
* Betterer
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Provisioning: Update token instructions (#101280)
* Provisioning: Use blob storage rather than local file system to save images (#101298)
* [Provisioning] Add more explicit setup warnings if webhook integration and image rendering are disabled (#101304)
* Setup warnings if renders or webhooks are not possible
* Improve display
* Use a single Alert
* Make design more compact
* Only display local config is missing critical feature toggles
* Improve styling of required flags
* Add file name to the custom ini itself
* Add copy button
* Add FeatureSection Component
* Commit betterer
* Use an interactive table
* Use a modal for instructions
* Use the same modal for example config
* Improve setup steps
* Improve stepper
* Copy code ctrl + c
* Make it more compact
* Select feature to enable
* Improve the height of alerts
* Separate components
* Better warnings
* Improve the page
* Improve the cards
* Improve cards even more
* Improve cards
* Improve cards
* Optional copy in code block
* Add side bar with steps
* Improve styling
* Style modal
* Clean up Code
* Remove index file
* Simplify Step Component
* Commit betterer
* Simplify components
* Use CodeEditor and Clipboard components
* Do not show scrollback on minimum size
* Fix positioning of footer
* Separate Component for Feature
* Use different styling
* Commit betterer
* Use more Grafana components in the FeatureCard
* Separate sidebar into own components
* Simplify sidebar code
* Commit betterer
* Remove connector
* Simplify styling further
* Use cards
* Improve code
* Use more grafana component in InstructionsModal
* Further simplify
* Simplify the code
* Simplify style
* Clean up
* Simplify the Wizard
* Use little icons
* Improve feature cards
* Improve cards
* Commit betterer
* Add description to feature setup
* Improve instructions for snapshot preview
* Move all files into Setup folder
* Commit betterer
* Clean up the warnings code
* Improve coding
* Move sidebar item to separate fiel
* Rename components
* Fix issues
* Use stack instead
* Improve style
* Don't show setup button if configured already
* Simplify again CSS
* Use secondary actions
* Style a bit more
* Improve wording
* Update warning
* Refer to docs in Image Renderer
* More clean up
* Revert changes in generated client
* Fix typos and imports
* Fix lint errors
* Provisioning: better error support (#101490)
* update openapi snapshot
* fix build
* Provisioning: Only show setup page when feature toggles are missing (#101502)
* form fixing
* form fixing
* always send UID
* Same onboarding page regarless of migration (#101557)
* backend building... frontend still broken
* rename sync with main
* Provisioning: Update dashboard badge (#101599)
* Rename to push / pull everything user-facing (#101577)
* Rename to push / pull all everything user-facing
* Use automatic pulling wording
* Provisioning: Migrate when using unified storage (#101572)
* migrate when not unified
* Update pkg/registry/apis/provisioning/register.go
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
* variables
* merge main
---------
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
* Add tabs and features tab to listing page (#101570)
* List features in onboarding page (#101558)
* merge main
* POC/Provisioning: Check if the instance is provisioned (#101601)
* Check if the instance is provisioned
* Fix lints
* Fix getting config for new dashboard
* Fixes after merge
* More fixes
* Show success message
* Fix default value
* Add test
* Fix lints
* Provisioning: Include URLs in ResourceWrapper response (#101511)
* Convert Migrate wizard into a Connection Wizard (#101575)
* Convert Migrate wizard into a Connection Wizard
* Remove duplicate empty state
* Allow users to select target in the first step
* Remove file created by merge
* Select target based on existing connections
* Default option for targets and explainatory alert
* Do not display connect button if single connection
* Display target as tag in repository card
* Add Pull Step
* Fix linting
* User decides if migrate or connect
* Improve style based based on review
* Provisioning: Return upsert resource when writing (#101574)
* [Provisioning] Getting Started Page and Tab (#101701)
* merge main
* fix go.mod
* Provisioning: Redirect to the new URL after save (#101757)
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* POC/Provisioning: create folder (#101619)
* Add NewProvisionedFolderForm
* Cleanup
* Add folder api
* Register API
* Do not show provisioned badge for instances
* Enable saving
* Show saved alert
* Fixes
* Fix deps
* Cleanup
* Add test
* Add test
* Updates
* Fix test
* Fix import
* [Provisioning] Display instance connection directly in home (#101720)
* Show tabs directly in home page for instance sync
* Display actions also in the home page
* Add delete button to actions
* Fix issue with files tab
* Display tabs also in instant sync
* Generate connection name for instance sync
* Fix issues when no repositories
* Set active tab
* Remove leftover
* Rename component for listing folder repository
* Fix linting issues
* Fix merge error
* Rename to HomePage
* Split folder list into separate component
* Create own component for repository card
* Improve RepositoryCard readability
* Improve RepositoryCard styling
* Make view primary button
* Fix syntax
* Fix generateName
* merge main
* Provisioning: Move folder management into its own helper (#101864)
* folder
* update folder links
* Fix test
* update
* cleanup
---------
Co-authored-by: Clarity-89 <homes89@ukr.net>
* fixed folder issue
* POC/Provisioning: Create folder from root (#101921)
* Enable creating folder at the root
* Fix test
* Add interceptor
* Provisioning: Expose stats (#101927)
* [Provisioning] Unified onboarding wizard (#101952)
* Spike the solution
* More work
* Add more situations
* Attempt to display count of dashboards and folders
* Attempt with file count
* Do not display options if not possible
* Improve styling resources
* Use another API
* Fix issue with selection
* Style a bit
* Fix more issues
* Make the sync step work
* Improve links
* Use LinkButton
* Start pull automatically
* Start migration automatically
* Fix issue with options
* Fix issues
* Fix loading error
* Improve more things
* Improve styling
* Improve messaging
* Set the autofocus
* Fix some issues
* Fix issue with disabled options
* Only resources
* Finish settings depending on configuration
* Move title to wizard
* Fix title
* Improve styling
* Badge
* Explain on hover
* Improve styling
* Disabled at the bottom
* History & identifiers
* Improve wording
* Add padding left and right disable options
* Delete repository
* Improve buttons
* Give index time to catch up
* Improve buttons
* Handle steps with only forms
* Fix issue with initial migrate or pull
* Commit betterer
* Error messages
* Use memo
* Revampt that a bit
* Attempt to simplify the state and components
* Improve the component for Migrate
* Commit betterer
* Fix issue in next button
* Clean up more
* Start for boostrap step
* Fix issue with running status
* Fix issue with loading bootstrapping
* Improve loading
* Improve more the loading
* Fix issue with loading
* Empty tree
* Handle error
* Fix issue with looping
* Remove commented out lines
* Add comment
* Remove accidental file
* Fix imports
* Improve MigrateStep and PullStep
* Use hook for step status
* JobStep component
* Refactor data fetching
* Validate with Github
* Fix issue with failed error
* Fix next on success
* Address small comments
* Separate file for WizardContent
* Fix linting
* Use step approach also for bootstrap
* Make the logic for moving between steps clearer
* Fix navigation issue
* Clean up some logic
* Use useAsync for JobStep steps
* Revert "Use useAsync for JobStep steps"
This reverts commit 242a275cc9.
* Provisioning: use service to get counts (#101972)
counts
* must migrate when using legacy storage
* Revert "Revert "Use useAsync for JobStep steps""
This reverts commit a420d0ac36.
* Fix async conditions
* Organize imports
* Separate component for BootstrapOptionCard
* BootstrapOptionsList
* Remove duplicate definitions
---------
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Merge
* Fix utils
* POC/Provisioning: Fix folder path for dashboard (#101997)
* Provisioning: Fix folder path for dashboards
* Fix isNew
* Update test
* Fix any error
* Betterer
* [Provisioning] Improve progress recording and updates (#102035)
* do not validate on delete
* Provisioning: Implement authorizer for remaining resources (#101945)
* feat: implement authorizer for remaining resources
* fix: don't allow viewers to write files
* security: harden blob id fetching
* add integration test for admin vs viewer
* feat: only Get is a valid verb for reads in our subresources
Co-Authored-By: Ryan McKinley <ryantxu@gmail.com>
* feat: allow render for all requests
* refactor: use guards
Not changing code that goes `if a { } else if b { } else { }` as the semantic meaning of the different branches is
easier to parse.
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* POC/Provisioning: Improve wizard setup (#102066)
* wizard actions
* workign better
* remove more memo
* show polling interval
* cleanup
* finalizers
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/WizardContent.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/WizardContent.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* updates from alex
* updates from alex
* Simplify actions
* Extract props
* history supported form legacy only (for now)
* More refactor
* change order
* Fix cleanup finalizer
* show kinds
* fix lint
---------
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
* Fix resource condition (#102086)
* Fix job summary stats issue (#102084)
* merge main
* Repository link should point to configured branch (#102092)
* Add getRepoHref
* Fix random string generation
* Redirect to Home on repository deletion (#102096)
* Fix extra commas in pull request comment (#102108)
* Fix image rendering endpoint (#102107)
* POC/Provisioning: Support migrate... when starting with unified storage (#102097)
* use same clone
* now using upsert
* Fix lint
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
* Clean up unprovisioned resources after unified storage migration (#102126)
* Clean up unprovisioned resources after migrate
* Clean up unprovisioned resources after migrate
* Update pkg/registry/apis/provisioning/jobs/migrate/resources.go
* Reset summary between export and pull (#102101)
* Reset summary between export and pull
* Add reset results to unified storage migration
* Provisioning: always dirty (#102151)
* fix test version
* log the watch line
* POC/Provisioning: Disable repository list watch (#102169)
* Disable watch for repo list endpoint
* Add comment
* Remove another watch
* Provisioning: Avoid calling test on every update (#102161)
test less often
* Provisioning: Support prefixes in GitHub repositories (#101969)
* feat: add a Prefix property to GitHub repo spec
* feat: make nested folders work properly
* feat: use subdir for go-git export
* fix: placeholder for prefix should be grafana/
* feat: rename prefix to path
* fix: json name should be path, too
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* chore: regen apis
* fix: copy 'path'
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Merge
* Fix duplicate type
* Provisioning: Test export functionality (#101336)
* Provisioning: Test export functionality
* fix: use context.Background
* test: add more cases for local repo path resolving
* test: rework test inputs
* test: try to make github test work
* fix: clear global state
* Update api client imports
* Fix import
* Fix test
* Update codegen
* Provisioning: Make it green (#102271)
* chore: remove unused functions
* chore: update betterer results
* chore: update openapi spec
* chore: yarn generate-apis
* fix: specify default false if undefined
* Use AnnoKeyManagerIdentity
* Add manager kind
* POC/Provisioning: Update component structure (#102297)
* Update project structure
* Update imports
* Remove unused components
* Copy fixes
* Typo
* More copy fixes
* Betterer
* Update test
* merge main
* Provisioning: Replace hardcoded clients with discovery client (#101918)
* disco client
* discovery client
* merge main
* merge main
* keep factory
* keep factory
* find preffered version for delete factory
* use same folders request
* merge main
* with integration test
* POC/Provisioning: Compare spec in test rather than raw JSON (#102352)
* compare spec not json
* compare spec not json
* [Provisioning] Add in-code TODOs in API Server area (#102360)
* Add TODOs for files endpoint
* Add TODO history endpoint
* Add TODO to move files logic to resource package
* Add TODO to not use private fields directly
* Remove unnecessary checks in list connector
* Add pagination TODO in lister
* Add TODO to rename resources
* Add todo about cloning too early
* Add TODO to propose to merge sync and migrate endpoints
* Add TODOs in register
* Add more TODOs in connectors & routes
* Add TODOs about prefix
* Change it to remove
* Update pkg/registry/apis/provisioning/test.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* POC/Provisioning: Switch folders to use managedBy (#102362)
* Folders: Switch to managedBy
* Fix create folder
* Fix tests
* Do not allow changing folder from dashboard settings
* Update imports
* Update provisioned meta
* Do not show provisioned badge for child folders
* Fix folder title
* Update folder actions
* Update new provisioned folder form
* Remove unused code
* Fix condition
* Reset default values on change
* Remove duplicate nav item
* Add managedBy to DashboardQueryResult
* Provisioning: support watch over live (in feature branch) (#102408)
* Provisioning: watch cleanup (#102424)
* fix lint
* Provisioning: Add basic usage stats (#102405)
* [Provisioning] Add limitations to Github Repository (#102451)
* Put limits to Github
* File is too large
* Move constants
* Embed ListOptions again
* Remove TODO
* Provisioning: Pick a better default title (#102516)
better title
* Provisioning: sanitize pull request urls (#102517)
* [Provisioning] Clean up clone after export and migrate (#102467)
* Remove clone directory on clone failure
* Defer remove clones
* Log error if removal fails
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* [Provisioning] Limit path length and depth in APIs (#102472)
* Limit filepath length in files API calls
* Add common utility to deal with paths
* Use the existing function
* Fix import
* Update pkg/registry/apis/provisioning/safepath/limit.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Fix issue after website commit
* Fix linting issue in test
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Add timeout, JSON check and max body size to endpoints (#102443)
* Add timeout, JSON check and max body size to endpoints
* Use http.MaxBytesReader instead
* Use MaxBytesReader also for reading the entire body
* Add empty line
* Add unit tests
* Fix integration tests
* Update pkg/registry/apis/provisioning/render.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Do not use utils :)
* Fix comment on unmarshalJSON
* 25MB for webhook events
* Remove content type check for files write
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* fix imports
* Provisioning: Remove export option from the UI (#102511)
* [Provisioning] Remove unused checkout method in go-git (#102460)
* [Provisioning] Limit max number of repositories to 10 (#102542)
* Limit to maximum 10 repositories in backend
* Change messaging
* Do not display connect button if more than 10
* Only fetch settings once
* watch repos
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* [Provisioning] Limit size and time to git clone and push in go-git (#102458)
* Limit git clone and push time and size
* Fix linting
* Use transport instead to limit
* Remove not supported
* Add TODO to make timeout configurable
* chore: make update-workspace
* Provisioning: Implement a new job queue (#102446)
* feat: implement a new job queue
Outstanding problems:
* Status isn't saved.
* Progress updates don't work (due to status not being saved probably?).
* feat: properly save status
* chore: document label
* chore: assumptions do hold
* fix: support multi-tenant job drivers
* fix: use namespace=*
* fix: set resource back to pointer when updating job progress
If we don't do this, we start rejecting job progress updates as the version falls out of sync.
* feat: make job APIs read-only
* fix: complete job when worker returns
* fix: set namespace on requests from controller
* test: check historic jobs
* chore: regen apis
* feat: start augmenting frontend
* feat: add jobs to authorizer
* feat: use watch from input
* fix: make frontend subscribe to historic jobs
* fix: lint
* chore: yarn prettier:write
* fix: frontend lints
* test: allow for empty state in historicjobs
* test: set content type for export request
* fix: always set job name on insert
* fix: import
* fix: use dashes not colons
* fix: job status should expect a historic job transition
* fix: allow PR jobs from multiple PRs
* feat: same name for sync and migrate jobs
* feat: generate a job name in the store
* refactor: rename to persistentStore
* feat: remove status subresources on jobs
* feat: join jobs into one card
* chore: regen openapi snapshot
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
* Provisioning: Use a complete storage for jobs (#102605)
* feat: add a complete strategy to apiserver
* feat: use the complete storage strategy for jobs
* test: behaviour changed in main
* [Provisioning] Consolidate file path handling (#102617)
* Add more cases for validation
* Call the method dir
* Clean files endpoint
* Simplify further
* Fix issues with folder
* Add Dir function
* Use walk function in folders
* Move things from ID
* Fix some tests
* Add tree
* Sync worker and changes
* Add more TODOs
* Add normal join
* Remove things in local
* Consolidate single Join
* Call it safe
* Add new IsPathSupported action
* Move the depth to resources
* Add more cases
* Improve trie implementation
* Add tests trie
* Fix trie tests
* Improve trie tests
* Add tests for walk
* Fix linting
* Add unit tests filepath
* Remove TODO
* Remove another TODO
* Unsupported file extension error
* Add documentation for IsPathSupported
* Filepath unit tests
* Use safepath to validate github path
* Remove TODO in wrapper
* Use trailing slash in folder internal object
* Fix changes test
* Include dot
* Add TODO to explore own type for path
* Fix frontend lint
* Fix unit tests
* Fix provisioning integration tests
---------
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
Roberto Jiménez Sánchez
Daniele Stefano Ferru
Alex Khomenko
Costa Alexoglou
Stephanie Hingtgen
Costa Alexoglou
Ryan McKinley
Roberto Jiménez Sánchez
Matheus Macabu
Mariell Hoversholm