What is this feature?
This PR implements compressed periodic save for alert state storage, providing a more efficient alternative to regular periodic saves by grouping alert instances by rule UID and storing them using protobuf and snappy compression. When enabled via the state_compressed_periodic_save_enabled configuration option, the system groups alert instances by their alert rule, compresses each group using protobuf serialization and snappy compression, and processes all rules within a single database transaction at specified intervals instead of syncing after every alert evaluation cycle.
Why do we need this feature?
During discussions in PR #111357, we identified the need for a compressed approach to periodic alert state storage that could further reduce database load beyond the jitter mechanism. While the jitter feature distributes database operations over time, this compressed periodic save approach reduces the frequency of database operations by batching alert state updates at explicitly declared intervals rather than syncing after every alert evaluation cycle.
This approach provides several key benefits:
- Reduced Database Frequency: Instead of frequent sync operations tied to alert evaluation cycles, updates occur only at configured intervals
- Storage Efficiency: Rule-based grouping with protobuf and snappy compression significantly reduces storage requirements
The compressed periodic save complements the existing jitter mechanism by providing an alternative strategy focused on reducing overall database interaction frequency while maintaining data integrity through compression and batching.
Who is this feature for?
- Platform/Infrastructure teams managing large-scale Grafana deployments with high alert cardinality
- Organizations looking to optimize storage costs and database performance for alerting workloads
- Production environments with 1000+ alert rules where database write frequency is a concern
* annotation legacy store with api server, read only
* Add a feature flag for annotations app
* implement list filters
* annotations are not addressable by ID for read operations
* fix registry apps test
* add ownership for an app
* disable linter
* typo, of course
* fix go workspace
* update workspace
* copy annotation app in dockerfile
* update workspace
---------
Co-authored-by: Tania B. <10127682+undef1nd@users.noreply.github.com>
* cleanup
* library panel via search
* test cleanup
* merge main
* add FindDashboards mock
* no matching dashbaords should return empty
* do not alllow name and libraryPanel query
* fix: delete folders using postorder
* chore: use helper function and do not add method to Folder store
- addresses other review comments fixing log messages and cleans up the unit tests
* chore: run library element tests on modes 2,3,5 only
* chore: adjust to folder.SortByPostorder(folders []*Folder)
* chore: run library panels tests in mode 2,3,5 only
* chore: run tests in all modes and increase timeout
- adjusting the modes and tweaking configs will be done separately
* fix: delete subfolder dangling panels and error if used
* chore: add observation about library panel DeleteInFolders
- logs folders UIDs on DeleteInFolders error
* chore: add integration test for blocking library panel deletion and handling dangling library panels
* chore: fix integration test on mode 4 and 5
Run the app runner in a goroutine in the post-start hook, as the '/readyz' endpoint for the API server waits for a non-nil error response from the post-start hook to mark it as ready.
The query which fetches alert rules in a paginated manner ordered by `rule_group` can result in strange and inconsistent ordering when the database uses a case-insensitive collation for the `rule_group` column. This can lead to scenarios where rules from different groups are interleaved in the results, making pagination unreliable and the returned number of rule_groups incorrect.
Related to #88990
* Provisioning: allow access check to proceed even when non access policy
* Provisioning: access checker needs this for MT
* add permissions registration
* remove scopes
* use in MT for now
* no need to document an internal flag here
* revert vscode change
* refactor the authZ permission evaluation and mapper code to allow evaluating unscoped actions beyond creation
* update wire
* gofmt
* add boolean to struct
---------
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
* Jaeger: Migrate Services and Operations to the gRPC Jaeger endpoint (#112384)
* add grpc feature toggle
* move types into types.go
* creates grpc client functions for services and operations
* Call grpc services function when feature flag is enabled for health check
* remove unnecessary double encoding
* check for successful status code before decoding response and return nil in case of successful response
* remove duplicate code
* use variable
* fix error type in testsz
* Jaeger: Migrate search and Trace Search calls to use gRPC endpoint (#112610)
* move all types into types package except for JagerClient
* move all helper functions into utils package
* change return type of search function to be frames and add grpc search functionality
* fix tests
* fix types and the way we check error response from grpc
* change trace name and duration unit conversion
* fix types and add tests
* support queryAttributes
* quick limit implementation in post processing
* add todo for attributes / tags
* make trace functionality ready to support grpc flow
* add functions to process search response for a specific trace and create the Trace frame
* tests for helper funtions
* remove grpc querying for now!
* change logic to be able to process and support multiple resource spans
* remove logic for gRPC from grpc_client.go
* add equivalent fields for logs and references
* add tests for grpcTraceResponse function
* fix types after merge with main
* fix status code checks and return nil for error on successful responses
* enable reading through config flag for trace search
* create sigle key value type since they are similar for OTLP and non OTLP based formats
* reference right type
* convert events and links into references and logs
* add status code, status message and kind to data frame
* fix tests to accomodate new format
* remove unused function and add more tests
* remove edit flag for jsonc golden test files
* add clarifying comment
* fix tests and linting
* fix golden files for testing
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add clarifying comment
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove unnecessary logging statement
* fix downstream errors
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* use downstreamerrorf where applicable and add missing downstream eror sources.
* tests
---------
Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Return the SQL schema for all DS queries in request (to provide information to AI / Autocomplete for SQL expressions).
All DS queries are treated as if they were inputs to SQL expressions in terms of conversion, regardless if they are selected in a query or not.
Requires feature toggle queryService = true
Endpoint is apis/query.grafana.app/v0alpha1/namespaces/default/sqlschemas
---------
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
* Something is working
* Progress
* Update
* Update
* Update
* Some new unit tests
* Fix
* time shift fix
* Update
* Always show hidden toggle
* Update
Ezequiel Victorero
Paul Marbach
Misi
Jean-Philippe Quéméner
Alexa Vargas
Juan Cabanas
Seunghun Shin
Serge Zaitsev
Mihai Turdean
Ryan McKinley
linoman
Rafael Bortolon Paulovic
Esteban Beltran
colin-stuart
Alexander Zobnin
Jesse David Peterson
Austin Pond
Moustafa Baiou
Bruno
Matias Chomicki
Charandas
Stephanie Hingtgen
Jocelyn Collado-Kuri
Janos Gub
Alex Khomenko
Nathan Marrs
Kyle Brandt
maicon
Yuri Tseretyan
Todd Treece
Matheus Macabu
Ashley Harrison
Javier Ruiz
Torkel Ödegaard