What is this feature?
This PR implements compressed periodic save for alert state storage, providing a more efficient alternative to regular periodic saves by grouping alert instances by rule UID and storing them using protobuf and snappy compression. When enabled via the state_compressed_periodic_save_enabled configuration option, the system groups alert instances by their alert rule, compresses each group using protobuf serialization and snappy compression, and processes all rules within a single database transaction at specified intervals instead of syncing after every alert evaluation cycle.
Why do we need this feature?
During discussions in PR #111357, we identified the need for a compressed approach to periodic alert state storage that could further reduce database load beyond the jitter mechanism. While the jitter feature distributes database operations over time, this compressed periodic save approach reduces the frequency of database operations by batching alert state updates at explicitly declared intervals rather than syncing after every alert evaluation cycle.
This approach provides several key benefits:
- Reduced Database Frequency: Instead of frequent sync operations tied to alert evaluation cycles, updates occur only at configured intervals
- Storage Efficiency: Rule-based grouping with protobuf and snappy compression significantly reduces storage requirements
The compressed periodic save complements the existing jitter mechanism by providing an alternative strategy focused on reducing overall database interaction frequency while maintaining data integrity through compression and batching.
Who is this feature for?
- Platform/Infrastructure teams managing large-scale Grafana deployments with high alert cardinality
- Organizations looking to optimize storage costs and database performance for alerting workloads
- Production environments with 1000+ alert rules where database write frequency is a concern
* annotation legacy store with api server, read only
* Add a feature flag for annotations app
* implement list filters
* annotations are not addressable by ID for read operations
* fix registry apps test
* add ownership for an app
* disable linter
* typo, of course
* fix go workspace
* update workspace
* copy annotation app in dockerfile
* update workspace
---------
Co-authored-by: Tania B. <10127682+undef1nd@users.noreply.github.com>
* cleanup
* library panel via search
* test cleanup
* merge main
* add FindDashboards mock
* no matching dashbaords should return empty
* do not alllow name and libraryPanel query
* fix: delete folders using postorder
* chore: use helper function and do not add method to Folder store
- addresses other review comments fixing log messages and cleans up the unit tests
* chore: run library element tests on modes 2,3,5 only
* chore: adjust to folder.SortByPostorder(folders []*Folder)
* chore: run library panels tests in mode 2,3,5 only
* chore: run tests in all modes and increase timeout
- adjusting the modes and tweaking configs will be done separately
* fix: delete subfolder dangling panels and error if used
* chore: add observation about library panel DeleteInFolders
- logs folders UIDs on DeleteInFolders error
* chore: add integration test for blocking library panel deletion and handling dangling library panels
* chore: fix integration test on mode 4 and 5
Run the app runner in a goroutine in the post-start hook, as the '/readyz' endpoint for the API server waits for a non-nil error response from the post-start hook to mark it as ready.
The query which fetches alert rules in a paginated manner ordered by `rule_group` can result in strange and inconsistent ordering when the database uses a case-insensitive collation for the `rule_group` column. This can lead to scenarios where rules from different groups are interleaved in the results, making pagination unreliable and the returned number of rule_groups incorrect.
Related to #88990
* Provisioning: allow access check to proceed even when non access policy
* Provisioning: access checker needs this for MT
* add permissions registration
* remove scopes
* use in MT for now
* no need to document an internal flag here
* revert vscode change
* refactor the authZ permission evaluation and mapper code to allow evaluating unscoped actions beyond creation
* update wire
* gofmt
* add boolean to struct
---------
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
* Jaeger: Migrate Services and Operations to the gRPC Jaeger endpoint (#112384)
* add grpc feature toggle
* move types into types.go
* creates grpc client functions for services and operations
* Call grpc services function when feature flag is enabled for health check
* remove unnecessary double encoding
* check for successful status code before decoding response and return nil in case of successful response
* remove duplicate code
* use variable
* fix error type in testsz
* Jaeger: Migrate search and Trace Search calls to use gRPC endpoint (#112610)
* move all types into types package except for JagerClient
* move all helper functions into utils package
* change return type of search function to be frames and add grpc search functionality
* fix tests
* fix types and the way we check error response from grpc
* change trace name and duration unit conversion
* fix types and add tests
* support queryAttributes
* quick limit implementation in post processing
* add todo for attributes / tags
* make trace functionality ready to support grpc flow
* add functions to process search response for a specific trace and create the Trace frame
* tests for helper funtions
* remove grpc querying for now!
* change logic to be able to process and support multiple resource spans
* remove logic for gRPC from grpc_client.go
* add equivalent fields for logs and references
* add tests for grpcTraceResponse function
* fix types after merge with main
* fix status code checks and return nil for error on successful responses
* enable reading through config flag for trace search
* create sigle key value type since they are similar for OTLP and non OTLP based formats
* reference right type
* convert events and links into references and logs
* add status code, status message and kind to data frame
* fix tests to accomodate new format
* remove unused function and add more tests
* remove edit flag for jsonc golden test files
* add clarifying comment
* fix tests and linting
* fix golden files for testing
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix typo
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add clarifying comment
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove unnecessary logging statement
* fix downstream errors
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* use downstreamerrorf where applicable and add missing downstream eror sources.
* tests
---------
Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Return the SQL schema for all DS queries in request (to provide information to AI / Autocomplete for SQL expressions).
All DS queries are treated as if they were inputs to SQL expressions in terms of conversion, regardless if they are selected in a query or not.
Requires feature toggle queryService = true
Endpoint is apis/query.grafana.app/v0alpha1/namespaces/default/sqlschemas
---------
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
* Something is working
* Progress
* Update
* Update
* Update
* Some new unit tests
* Fix
* time shift fix
* Update
* Always show hidden toggle
* Update
* Grafana: Add random walk configuration options to Grafana datasource
Add UI controls to configure random walk parameters (min, max, start value,
spread, noise, drop percent) to match TestData datasource functionality.
- Add RandomWalkEditor component with inline number inputs for all parameters
- Update GrafanaQuery type to include random walk configuration fields
- Update backend to parse and apply query parameters to RandomWalk function
- Configuration options match TestData datasource UX for consistency
* Grafana: Add series count support to random walk
Add ability to generate multiple random walk series in a single query
for complete parity with TestData datasource.
- Add seriesCount field to RandomWalkEditor
- Update backend to loop and generate multiple frames based on series count
- Default to 1 series if not specified for backward compatibility
* Grafana: Improve random walk editor UI with better organization and tooltips
Enhance the random walk configuration UI for better usability:
- Organize fields into two logical rows (core config vs fine-tuning)
- Add helpful tooltips to all fields explaining their purpose
- Increase label width to prevent text wrapping
- Group related fields visually for better comprehension
Row 1: Series count, Start value, Min, Max (basic shape and range)
Row 2: Spread, Noise, Drop % (randomness and variation controls)
This provides a cleaner, more intuitive experience compared to TestData's
single-row layout, making it easier for users to configure random walks.
* Grafana: Format RandomWalkEditor code
Apply consistent formatting to RandomWalkEditor component.
* Grafana: Add E2E tests for random walk configuration
Add comprehensive Playwright E2E tests to verify random walk functionality:
- Test that all configuration fields render correctly
- Test min/max value constraints
- Test multiple series generation
- Test spread and noise parameters
- Test drop percentage for simulating missing data
- Test that tooltips are present and functional
These tests ensure the random walk configuration works end-to-end from
UI input to data rendering in panels.
* Grafana: Fix E2E tests for random walk configuration
Fix Playwright test selectors and assertions to work reliably:
- Use specific element IDs to avoid selector conflicts
- Remove flaky dropPercent check from rendering test (covered separately)
- Simplify test assertions to focus on functional verification
- All 7 tests now passing consistently
Tests verify: field rendering, min/max constraints, series count,
spread/noise configuration, drop percentage, and tooltips.
* Grafana: Add advanced E2E tests for random walk
Add two additional tests for better coverage:
- Test configuration value persistence across interactions
- Test that series count actually generates the expected number of series
These tests verify deeper functionality beyond basic UI rendering,
ensuring the random walk feature works correctly end-to-end.
All 9 tests passing consistently (20.1s runtime).
* Grafana: Remove redundant Min/Max tooltips
Remove tooltips from Min and Max fields that just repeated the label text.
These fields are self-explanatory and don't need tooltip icons.
Keeps the UI cleaner while maintaining helpful tooltips on fields that
actually benefit from explanation (Series count, Start value, Spread,
Noise, Drop %).
* Grafana: Add CODEOWNERS entry for random walk E2E tests
Add codeowner assignment for the new grafana-datasource-random-walk.spec.ts
test file to @grafana/grafana-frontend-platform, matching the ownership of
the Grafana datasource code.
* Add dashboardTemplates feature toggle and put new changes behind this toggle to limit impact
* Grafana: Add unit tests for dashboardTemplates feature toggle
Add unit tests to verify RandomWalkEditor renders correctly based on
the dashboardTemplates feature toggle:
- Test that random walk editor renders when FF is enabled
- Test that random walk editor is hidden when FF is disabled
These tests ensure the feature toggle works as expected and prevents
the random walk configuration UI from appearing when the feature is disabled.
* revert previous codeowners change as not necessary
* Grafana: Remove redundant E2E test for feature flag disabled
Remove E2E test for dashboardTemplates feature flag disabled scenario
since it's already covered by unit tests and E2E environment can't
reliably control server-side feature flags.
Feature flag behavior is properly tested in QueryEditor.test.tsx unit tests.
E2E tests focus on functional validation when the feature is enabled.
* fix lint
Paul Marbach
Misi
Jean-Philippe Quéméner
Alexa Vargas
Juan Cabanas
Seunghun Shin
Serge Zaitsev
Mihai Turdean
Ryan McKinley
linoman
Rafael Bortolon Paulovic
Esteban Beltran
colin-stuart
Alexander Zobnin
Jesse David Peterson
Austin Pond
Moustafa Baiou
Bruno
Matias Chomicki
Charandas
Stephanie Hingtgen
Jocelyn Collado-Kuri
Janos Gub
Alex Khomenko
Nathan Marrs
Kyle Brandt
maicon
Yuri Tseretyan
Todd Treece
Matheus Macabu
Ashley Harrison
Javier Ruiz
Torkel Ödegaard