Commit Graph

26 Commits

Author SHA1 Message Date
Peter Štibraný 8a160a8ca1 Convert unique keys in 3 tables to primary keys (#115421)
* Added method for adding migrations for convering unique to primary key.

Based on existing migration for `file` table (in `db_file_storage.go`) migrations.

* Added better default migration names. Added ability to override migration name.

* Use ConvertUniqueKeyToPrimaryKey for cloud_migration_snapshot_partition table.

* Convert resource_version UQE to PK.

* Convert secret_encrypted_value UQE to PK.

* Removed extra test.

* Removed testdata.

* Remove support for renaming migrations for now. We can bring it in later, when we want to convert existing migrations for file, file_meta and setting tables.

* Revert removal of ColumnName to ease backporting, since this field is referenced from enterprise code.

* Use quoted identifiers in Postgres statement.
2025-12-17 15:37:49 +01:00
Will Assis 1b59c82b74 Revert "Unified-storage: sql backend key path backfill (#115033)" (#115257)
This reverts commit b2dd095bd8.
2025-12-12 17:00:08 +00:00
Will Assis b2dd095bd8 Unified-storage: sql backend key path backfill (#115033)
* unified-storage: add migration to backfill key_path in resource_history
2025-12-12 08:09:51 -05:00
Will Assis 30045c02c0 unified-storage: add index on resource_history key_path column (#115113) 2025-12-10 16:03:58 -05:00
Rafael Bortolon Paulovic ff97bfc772 fix(unified): key_path column default (#114859)
fix: key_path column default
2025-12-04 12:40:21 -05:00
Will Assis 994e1dd58f unified-storage: sqlkv migrations (#114790)
* unified-storage: create resource_events table and add key_path column to resource_history

* Update resource_history_insert template

* update test snapshots

* use latin encoding for key_path and bump size to 2048
2025-12-04 09:21:17 -05:00
Peter Štibraný a4aa3529c8 Cleanup old entries from resource_last_import_time table. (#112438)
* Cleanup old entries from resource_last_import_time table.

* Add index for last_import_time column.

* Address review feedback.
2025-10-23 11:17:08 +00:00
Peter Štibraný d801b87db9 LastImportTime for resource. (#112153)
* LastImportTime for resource.

* Make StorageBackendImpl implement GetResourceLastImportTimes

* More missing implementations of GetResourceLastImportTimes

* Fix import.

* Skip TestGetResourceLastImportTime in TestBadgerKVStorageBackend.

* Implement GetResourceLastImportTimes by mockStorageBackend

* Bump test tolerance.

* Fix postgres query and timezone.

* Fix postgres query and timezone.

* Make linter happy.
2025-10-09 11:27:11 +02:00
Jean-Philippe Quéméner 392c1a71c9 Storage: Take migration_locking setting into account (#105938) 2025-05-23 14:11:21 +02:00
Serge Zaitsev 694b9dfe50 Chore: Replace xorm.io/xorm imports (#104458)
* replace xorm.io/xorm imports

* replace xorm from other go.mod files

* clean up workspace

* nolint does not make sense anymore as it is not a module

* try if nolint directive helps

* use nolint:all for xorm

* add more nolints

* try to skip xorm in linter config

* exclude xorm differently

* retrigger ci
2025-05-02 17:13:01 +02:00
Ryan McKinley 9c49c601f1 Unistore: Add generation column (#102222) 2025-04-01 20:38:23 +03:00
Mustafa Sencer Özcan a4e8bd16de fix: add index for resource history get with trash on (#102894) 2025-03-26 15:52:13 +01:00
Ryan McKinley 9d30911107 SQL/Storage: Use xorm code migration for DeletionMarker (#99353) 2025-01-22 14:07:08 +03:00
Ryan McKinley 429da7fd68 SQL/Blob: Add support for blob storage to SQL backend (#98192) 2025-01-08 22:08:10 +02:00
Georges Chaudy f4e91da6b1 Unistore: fix fodler migration (#96299)
configure default migration for resource folder
2024-11-12 16:06:42 +02:00
Georges Chaudy c3571752b6 Unitstore: Fix the Folder migration (#96078)
* UniStore: Add folder to the SQL backend

This reverts commit aaa4f4a23e.

* fix folder migration
2024-11-12 12:52:04 +01:00
Leonor Oliveira aaa4f4a23e Revert "UniStore: Add folder to the SQL backend" (#95966)
Revert "UniStore: Add folder to the SQL backend (#95671)"

This reverts commit f075662696.
2024-11-06 17:04:57 +02:00
Georges Chaudy f075662696 UniStore: Add folder to the SQL backend (#95671)
* Add folder to the SQL backend

* remove unused key

* fix tests

* Return folders from watch
2024-11-05 10:37:23 +01:00
Scott Lepper 337f2a0a8b [search] add index; increase page size (#95515)
[search] add index; increase page size
2024-10-28 15:48:49 -04:00
owensmallwood 2ffb88b0ee Unified Storage: Add index for poller query (#95293)
add index for poller query
2024-10-23 15:40:56 -06:00
Prem Saraswat 5c03c14b25 resource-api: Loosen name validation to match K8s requirements (#93404)
* resource-api: Loosen name validation to match K8s requirements

This patch modifies some of the requirements for name validation of
objects in Resource API to match Kubernetes.

The limit we have on characters in name is 64, but some resources allow
upto 253 characters. Similarly we also include `:` in the regex, as many
objects in default K8s setup use it in the name (the group
`system:masters` for example)

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>

* Update the name column length in migrator and update e2e test to verify

---------

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
2024-10-10 11:03:18 -04:00
Georges Chaudy 03f55e5288 Unistore : Ensure Watch works in HA mode (#94120)
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097)"

This reverts commit 7c3fc2f261.

* make previous_resource_version nullable

* handle nil case
2024-10-07 10:01:53 +02:00
Georges Chaudy 7c3fc2f261 Revert "Unistore : Ensure Watch works in HA mode." (#94097)
Revert "Unistore : Ensure Watch works in HA mode. (#93428)"

This reverts commit 0a26c9e9ae.
2024-10-01 18:45:47 +00:00
Georges Chaudy 0a26c9e9ae Unistore : Ensure Watch works in HA mode. (#93428)
* Replace Watch with WatchNext

* remove watchset

* fix previous page and closing the channel

* Remove the broadcaster cache to prevent dupplicated events

* add watch bookmark

* add watch bookmark

* cleanup comments

* disable the tests for bookmarks for now

* Ensure we send previosu events

* lint

* re-introduce the cache

* load from cache

* disabling legacy test

* disabling legacy test

* Update pkg/storage/unified/resource/server.go

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>

* Could not read previous events

* add proper migration

* Add previous_resource_version to both history and resource

* First event should have an RV of 2 and not 1

* Test both storage backends

* fix the inital RV for the sql backend

* ensure graceful stop of the stream decoder

* gocyclo

---------

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-09-30 13:14:07 +02:00
Diego Augusto Molina 399d77a0fd Resource server improvements and fixes (#90715)
* cleanup dependencies and improve list method
* Improve Resource Server API, remove unnecessary dependencies
* Reduce the API footprint of ResourceDBInterface and its implementation
* Improve LifecycleHooks to use context
* Improve testing
* reduce API size and improve code
* sqltemplate: add DialectForDriver func and improve naming
* improve lifecycle API
* many small fixes after adding more tests
2024-07-22 20:08:30 +03:00
Georges Chaudy 08c611c68b ResourceServer: Resource store sql backend (#90170) 2024-07-18 08:03:18 -07:00