Commit Graph
11 Commits
Author SHA1 Message Date
f0c95a0a10 Provisioning: Add new error framework to handle folder creation failures gracefully. (#114824)
* Implement hierarchical error handling for folder creation failures

This commit implements hierarchical error handling to improve sync robustness
when folder creation fails. Instead of failing the entire sync, the system now:

1. Tracks failed folder creations and automatically skips nested resources
2. Records skipped resources with FileActionIgnored (doesn't count toward error limits)
3. Allows other folder hierarchies to continue processing
4. Prevents folder deletion when child resource deletions fail

Key Changes:

- Add PathCreationError type to track which folder path failed
- Modify progress recorder to automatically detect and track failures via Record()
- Add IsNestedUnderFailedCreation() and HasFailedDeletionsUnder() checks
- Update full and incremental sync to skip nested resources after folder failures
- Deletions proceed even if parent folder creation failed (resource may exist from previous sync)
- FileActionIgnored results don't count toward error limits

Example behavior improvement:
Before: /monitoring folder creation fails → all nested resources fail → other folders never processed
After: /monitoring folder creation fails → nested resources ignored → /applications folder succeeds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* provisioning: refactor hierarchical errors in folder management.

* Move test to the corresponding package

* Refactor timeout handling in applyChanges functions

- Introduced wrapWithTimeout function to streamline timeout context management for applyChange calls.
- Updated applyFoldersSerially and applyIncrementalChanges to utilize the new timeout wrapper.
- Removed redundant logging and error handling code related to timeout in favor of centralized handling in wrapWithTimeout.
- Adjusted test expectations to reflect changes in error reporting for context deadlines.

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-12 09:07:04 +00:00
Owen Diehl b0ecc36fd6 [App-Platform,Git-Sync]: Add sync job & phase metrics (#113253)
* [git-sync] add sync job & phase metrics

* refactoring sync jobs for clarity and ease of instrumentation

* make gofmt
2025-10-31 10:24:17 -07:00
Costa Alexoglou 25dd7e927f feat: add granular context timeout (#112952)
* feat: add granular context timeout

* test: forced thread timeout

* fix: test assumption for blocks

* test: new change

* trigger build

* remove loggers to test

* test with fmt.print

* fix: deadlocks
2025-10-29 08:41:53 +01:00
Costa Alexoglou 163a88056e feat: parallel processChange (#112198)
* feat: parallel processChange

* all except move work

* fix: tests and order of operations

* fix: tests

* chore: review feedback

* chore: review feedback
2025-10-16 19:17:04 +02:00
Ryan McKinley 2486dba881 Provisioning: use kind consistently for provisioning stats (#111977) 2025-10-04 09:02:02 -05:00
Stephanie Hingtgen e53b5902fa Provisioning: Wire up tracing and add trace for job sync (#111455) 2025-09-22 22:23:29 -06:00
Roberto Jiménez Sánchez 1b9e479b68 Provisioning: Abstract away how we build discovery and dynamic clients (#110662)
Abstract away how we get discovery and dynamic clients
2025-09-05 09:22:30 +00:00
Roberto Jiménez Sánchez 4eadc823a9 Provisioning: Move repository package to provisioning app (#110228)
* Move repository package to apps

* Move operators to grafana/grafana

* Go mod tidy

* Own package by git sync team for now

* Merged

* Do not use settings in local extra

* Remove dependency on webhook extra

* Hack to work around issue with secure contracts

* Sync Go modules

* Revert "Move operators to grafana/grafana"

This reverts commit 9f19b30a2e.
2025-09-02 09:45:44 +02:00
Roberto Jiménez Sánchez 2a0944a92a Provisioning: Miscellanous Fixes For Flaky Integration Tests (#109330)
* Enhance error messages individual files errors

* Split integration tests into multiple files

* Fix missing import

* Rename file to repository test

q

* Wrap worker errors with more context

* Fixed tests

* Add nolint

* Wait for specific job in multiple spots
2025-08-07 14:26:22 +02:00
Stephanie Hingtgen 9f4773c9a5 Provisioning: Extract to apps submodule (#109074) 2025-08-01 14:35:18 -05:00
Roberto Jiménez Sánchez 47664a7d51 Provisioning: unit test sync job (#103636)
* Split in multiple files

* Refactor sync even further

* Move more things between RepositoryResources

* Add status patcher

* Interface for sync functions

* Interface for compare function

* Add syncer back

* Move interfaces

* Move execute complete

* Return currentRef in syncer

* Add repository status test

* Add initial sync tests

* Fix a couple of spots

* Make initial sync tests work

* Fix register.go

* Add initial tests for sync worker

* Finish tests for sync

* Add incremental tests

* Add TODO

* Finish incremental tests

* Move folder creation to full sync

* Move interfaces

* Add initial full sync tests

* Update tests

* Reshape things

* Add changes test

* Fix register

* Add some tests

* Add more tests

* Add test

* WIP

* WIP: delete test

* Add more full test

* More tests

* Add tests for folder creation

* Add folder tests full sync

* Full coverage full sync

* Clean up tests

* Add more tests for changes function

* Enhance tests for Changes function to cover error scenarios and empty paths

- Added test cases for handling empty file paths and folder resources.
- Updated error message formatting in the Compare function for clarity.

* Add unit tests

* Failed initial patch

* Add tests failed repository resources

* Add test failed getting client

* Test for successful and unsuccessful syncs

* Add final tests for worker

* Fix existing tests

* Add missing test

* Fix spelling mistake

* Fix flake in changes test
2025-04-10 13:55:49 +01:00