* Provisioning: Improve logging and tracing in job processing
- Add comprehensive tracing with OpenTelemetry spans across all job operations
- Enhance logging with consistent style: lowercase, concise messages, appropriate log levels
- Use past tense for completed lifecycle events (e.g., 'stopped' vs 'stop')
- Add structured logging with contextual attributes for better searchability
- Handle graceful shutdowns without throwing errors on context cancellation
- Refactor Cleanup method into listExpiredJobs and cleanUpExpiredJob for better code quality
- Avoid double logging by only logging errors when handled locally
- Add tracing and logging to historyjob controller cleanup operations
Files modified:
- pkg/registry/apis/provisioning/jobs/driver.go: Add tracing spans and improve error handling for graceful shutdown
- pkg/registry/apis/provisioning/jobs/concurrent_driver.go: Add tracing and consistent logging
- pkg/registry/apis/provisioning/jobs/persistentstore.go: Add comprehensive tracing and logging to all public methods, refactor cleanup
- apps/provisioning/pkg/controller/historyjob.go: Add tracing and improve logging consistency
* Update pkg/registry/apis/provisioning/jobs/persistentstore.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor logging in persistentstore.go
- Remove debug log statements at the start of job operations for cleaner output
- Maintain structured logging with contextual attributes for improved traceability
Files modified:
- pkg/registry/apis/provisioning/jobs/persistentstore.go: Clean up logging for job operations
* Enhance logging and tracing in provisioning job operations
- Introduce OpenTelemetry spans for better observability in job processing and webhook handling
- Improve structured logging with contextual attributes for key operations
- Remove unnecessary tracing spans in long-running functions to streamline performance
- Update error handling to record errors in spans for better traceability
Files modified:
- pkg/registry/apis/provisioning/controller/repository.go: Add tracing and structured logging to sync job operations
- pkg/registry/apis/provisioning/jobs/concurrent_driver.go: Remove tracing span from long-running function
- pkg/registry/apis/provisioning/jobs/driver.go: Enhance logging and tracing in job processing
- pkg/registry/apis/provisioning/webhooks/webhook.go: Implement tracing and structured logging for webhook connections
* Update pkg/registry/apis/provisioning/jobs/driver.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Improve error handling in ConcurrentJobDriver to differentiate between graceful shutdown and unexpected stops
* Remove unused import in driver.go to clean up code
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The FlagGrafanaAPIServerWithExperimentalAPIs is only available when
`app_mode=development`. We have a more specific flag that is usable in
production, so use that.
Also, there was some old code constraining these APIs to a static list
of datasources. We don't need that anymore, so this PR removes it.
The FlagQueryServiceWithConnections is left as is, because there are
multiple existing tests that rely on this development-only, experimental
flag. I don't want to understand why that is.
Remove extractFieldsNameDeduplication feature toggle
The extractFieldsNameDeduplication feature has reached GA (General Availability),
so the feature toggle is no longer needed. The name deduplication behavior is
now permanently enabled.
Changes:
- Removed feature toggle definition from registry.go
- Removed feature toggle check in extractFields.ts (always deduplicate now)
- Updated tests to remove feature toggle manipulation
- Regenerated feature toggle files
If an alert rule with an invalid receiver is created it breaks the entire alertmanager configuration rather than preventing the save.
This fixes the issue by erroring on save and apply, and logging invalid receivers only when applying the config after an update.
Introduced in #111838
* Extend interpolate endpoint to support community dashboard json interpolation
Added unit tests
* Implement Frontend Side
- Show tabs
- Fetch Community dashboads
- Use DashboardCard component
- Search grafana dashboard in the community tab
- Make Tabs and pagination sticky
- Adjust titles to be scoped by datasource name/type
- Add skeleton loading for community tabs and pagination
- Add dashboard details tooltip
- Bring old datasource-provisioned box back and rely on new feature toggle for community dashboards
- update i18n
- update swagger
---------
Co-authored-by: Juan Cabanas <juan.cabanas@grafana.com>
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
* Remove dashboardDsAdHocFiltering feature toggle
The dashboardDsAdHocFiltering feature toggle has been enabled by default
and is now in General Availability stage. This commit removes the feature
toggle and makes the AdHoc filtering functionality for the dashboard
datasource permanently available.
Changes:
- Remove feature toggle from registry.go
- Regenerate feature toggle files
- Remove conditional checks in frontend code
- Update tests to reflect permanent enablement
- Always show AdHoc Filters toggle in dashboard query editor
- Always enable dashboard datasource in DataSourcePicker for variables
* Remove unused imports
* Fix Prettier formatting issues
* Alerting: Fix support for converted Prometheus rules in app-platform apis
Retrieving converted Prometheus retrieval rules was not supported in the app-platform apis and was throwing a 500 error due to the provenance not being handled properly.
Also adds a test to cover converted Prometheus rules when getting rules.
Closes https://github.com/grafana/alerting-squad/issues/1200
* add test to confirm provenance compatibility
Fix: Include ref field in DELETE endpoint response for branch operations
When deleting a dashboard file via DELETE endpoint with a ref query parameter
(for branch operations), the response was missing the ref field. This caused
the frontend branch workflow success handler to fail silently.
The issue was an inverted boolean condition in the Delete method. The code
was setting file.Ref = opts.Ref when shouldUpdateGrafanaDB returned true
(main branch operations), but it should have been setting it when false
(branch operations), since we read the file with an empty ref.
Fixed by inverting the condition from:
if r.shouldUpdateGrafanaDB(opts, nil)
to:
if !r.shouldUpdateGrafanaDB(opts, nil)
This ensures the ref field is properly included in the ResourceWrapper
response for branch operations.
This adds validating admission hooks to enforce the requirements on AlertRules and RecordingRules that are currently enforced through the provisioning service and storage mechanisms in preparation of a consistent validation in both legacy storage and unified storage. It also adds a mutating admission hook to the app to ensure that folder annotations and folder labels are kept in sync so we can perform label-selector lists.
* Validate Job Specs
* Add comprehensive unit test coverage for job validator
- Added 8 new test cases to improve coverage from 88.9% to ~100%
- Tests for migrate action without options
- Tests for delete/move actions with resources (missing kind)
- Tests for move action with valid resources
- Tests for move/delete with both paths and resources
- Tests for move action with invalid source paths
- Tests for push action with valid paths
Now covers all validation paths including resource validation and
edge cases for all job action types.
* Add integration tests for job validation
Added comprehensive integration tests that verify the job validator properly
rejects invalid job specifications via the API:
- Test job without action (required field)
- Test job with invalid action
- Test pull job without pull options
- Test push job without push options
- Test push job with invalid branch name (consecutive dots)
- Test push job with path traversal attempt
- Test delete job without paths or resources
- Test delete job with invalid path (path traversal)
- Test move job without target path
- Test move job without paths or resources
- Test move job with invalid target path (path traversal)
- Test migrate job without migrate options
- Test valid pull job to ensure validation doesn't block legitimate requests
These tests verify that the admission controller properly validates job specs
before they are persisted, ensuring security (path traversal prevention) and
data integrity (required fields/options).
* Remove valid job test case from integration tests
Removed the positive test case as it's not necessary for validation testing.
The integration tests now focus solely on verifying that invalid job specs
are properly rejected by the admission controller.
* Fix movejob_test to expect validation error at creation time
Updated the 'move without target path' test to expect the job creation
to fail with a validation error, rather than expecting the job to be
created and then fail during execution.
This aligns with the new job validation logic which rejects invalid
job specs at the API admission control level (422 Unprocessable Entity)
before they can be persisted.
This is better behavior as it prevents invalid jobs from being created
in the first place, rather than allowing them to be created and then
failing during execution.
* Simplify action validation using slices.Contains
Replaced manual loop with slices.Contains for cleaner, more idiomatic Go code.
This reduces code complexity while maintaining the same validation logic.
- Added import for 'slices' package
- Replaced 8-line loop with 1-line slices.Contains call
- All unit tests pass
* Refactor job action validation in ValidateJob function
Removed the hardcoded valid actions array and simplified the validation logic. The function now directly appends an error for invalid actions, improving code clarity and maintainability. This change aligns with the recent updates to job validation, ensuring that invalid job specifications are properly handled.
* Provisioning: Preserve in progress job data
* Refactor code and cover more situations
* Fix linting
* Fix issue with remove path operation for started time
* Cleanup
* prettier
---------
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>
What is this feature?
This PR implements compressed periodic save for alert state storage, providing a more efficient alternative to regular periodic saves by grouping alert instances by rule UID and storing them using protobuf and snappy compression. When enabled via the state_compressed_periodic_save_enabled configuration option, the system groups alert instances by their alert rule, compresses each group using protobuf serialization and snappy compression, and processes all rules within a single database transaction at specified intervals instead of syncing after every alert evaluation cycle.
Why do we need this feature?
During discussions in PR #111357, we identified the need for a compressed approach to periodic alert state storage that could further reduce database load beyond the jitter mechanism. While the jitter feature distributes database operations over time, this compressed periodic save approach reduces the frequency of database operations by batching alert state updates at explicitly declared intervals rather than syncing after every alert evaluation cycle.
This approach provides several key benefits:
- Reduced Database Frequency: Instead of frequent sync operations tied to alert evaluation cycles, updates occur only at configured intervals
- Storage Efficiency: Rule-based grouping with protobuf and snappy compression significantly reduces storage requirements
The compressed periodic save complements the existing jitter mechanism by providing an alternative strategy focused on reducing overall database interaction frequency while maintaining data integrity through compression and batching.
Who is this feature for?
- Platform/Infrastructure teams managing large-scale Grafana deployments with high alert cardinality
- Organizations looking to optimize storage costs and database performance for alerting workloads
- Production environments with 1000+ alert rules where database write frequency is a concern
Andres Martinez Gotor
MikeandMatheus Macabu
Mustafa Sencer Özcan
Roberto Jiménez SánchezandCopilot
beejeebus
Jean-Philippe Quéméner
Gareth
Sam Jewell
Alexander Zobnin
Moustafa Baiou
Josh Hunt
Ashley Harrison
Rafael Bortolon Paulovic
Gábor Farkas
Adam Simpson
Brendan O'HandleyandTania B.
ismail simsek
Paul Marbach
Bruno
Tom Ratcliffe
Ezequiel Victorero
Misi
Jo
Gabriel MABILLEandjguer
Alex KhomenkoandRoberto Jimenez Sanchez
Seunghun Shin