* Secrets: Refactor data_key_id out of the encoded secure value payload (#111852)
* everything compiles
* tests pass
* remove file included by accident
* add entry to gitignore
* some scaffolding for the migration executor
* remove file
* implement and test the migration
* use xkube.Namespace in our interfaces
* add todo
* update wire deps
* add some logs
* fix wire dependency ordering
* create tests to validate error conditions during migrations
* only run the migration as an MT api server
* formatting issues
* change detection of secrets running as MT server
* add todo
* use more specific initializer flags
* make secrets playwright tests work
* set new properties to true by default
* remove developer mode flag
* fix unit tests
* Update targeting key to namespace
* Pass eval ctx attributes to frontend
* Add context config to the frontend
* Fix OF namespace not defined in e2e tests
* Set default namespace and targeting key
* remove e2e server config now that namespace is default
---------
Co-authored-by: joshhunt <josh.hunt@grafana.com>
* Don't update index more often than specified index_min_update_interval.
* Add artificial sleep at the end of write operations.
* Improve test: check for number of update calls, make diff check less flaky.
* Make test less flaky by allowing for higher diff variance.
* Make test less flaky by allowing for expected update calls variance.
What is this feature?
This PR implements a jitter mechanism for periodic alert state storage to distribute database load over time instead of processing all alert instances simultaneously. When enabled via the state_periodic_save_jitter_enabled configuration option, the system spreads batch write operations across 85% of the save interval window, preventing database load spikes in high-cardinality alerting environments.
Why do we need this feature?
In production environments with high alert cardinality, the current periodic batch storage can cause database performance issues by processing all alert instances simultaneously at fixed intervals. Even when using periodic batch storage to improve performance, concentrating all database operations at a single point in time can overwhelm database resources, especially in resource-constrained environments.
Rather than performing all INSERT operations at once during the periodic save, distributing these operations across the time window until the next save cycle can maintain more stable service operation within limited database resources. This approach prevents resource saturation by spreading the database load over the available time interval, allowing the system to operate more gracefully within existing resource constraints.
For example, with 200,000 alert instances using a 5-minute interval and 4,000 batch size, instead of executing 50 batch operations simultaneously, the jitter mechanism distributes these operations across approximately 4.25 minutes (85% of 5 minutes), with each batch executed roughly every 5.2 seconds.
This PR provides system-level protection against such load spikes by distributing operations across time, reducing peak resource usage while maintaining the benefits of periodic batch storage. The jitter mechanism is particularly valuable in resource-constrained environments where maintaining consistent database performance is more critical than precise timing of state updates.
* Remove support for initMinSize.
Remove support for searchAfterWrite option, now it defaults to true.
* Remove reference to deprecated feature toggle.
* Remove feature toggle completely.
* Remove code related to indexing on watch events.
* Fix compilation error.
* Remove unused field.
* Configurable repository types in monolith and operator
* Default to Github in operators
* Regenerate wire
* Fix and implement unit tests
* Same types for enterprise tests
* Remove unnecessary conversion
* Remove the issue with import cycles
* feat(plugins): add a way to expose core apis only to certain plugins
* review: update naming
* review: update the owners of the feature toggle
* feat: share the restricted apis with extensions
* fix: linters
* feat: remove the `addPanel` api
* chore: fix linting and betterer issue
* tests: use `@ts-expect-error` for more clarity
sql_expression_query_length_limit
Set the maximum length of a SQL query that can be used in a SQL expression. Default is 10000 characters. A setting of 0 means no limit.
* Add setting to disable username based brute force login protection
* Use new DisableUsernameLoginProtection setting in tests where appropriate
* Update documentation for other brute force directives
* Avoid unecessary database calls
* Add test cases for username and IP protection settings
* TestUtils: Add helper to create certs
* InlineSecureValue: Move to its own package rather than service
* Settings: Rename decrypt server configs to grpc prefixed as it is more generic
* InlineSecureValue: Add gRPC client
* InlineSecureValue: Add service provider grpc/local depending on setting.Cfg
* Wire: Add ProvideInlineSecureValueService to basic wireset
* InlineSecureValue: Re-export in root secret pkg and generate mocks for interface
* Settings: Unify decrypt+inline setting into generic grpc client
* Cloud migrations: store snapshots in the database
* update github.com/grafana/grafana-cloud-migration-snapshot to v1.9.0
* make update-workspace
* use new field name in test
* return error after call to fmt.Errorf
* create methods for readability / fix session deletiong not deleting snapshots
* remove debugging changes
* update sample.ini
* update tests to include OrgID in ListSnapshotsQuery
* lint
* lint
* Update pkg/services/cloudmigration/cloudmigrationimpl/snapshot_mgmt.go
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
* remove TODO
* Update pkg/services/cloudmigration/cloudmigrationimpl/snapshot_mgmt.go
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
* remove one of the debug logs
---------
Co-authored-by: Matheus Macabu <macabu@users.noreply.github.com>
