Commit Graph
35 Commits
Author SHA1 Message Date
Jean-Philippe Quéméner e92a976de2 fix(unified-storage): process list items concurrently (#113801) 2025-11-13 12:08:14 +01:00
Jean-Philippe Quéméner 76ab09a6a2 feat(unified-storage): add tracing to apistore (#113714) 2025-11-12 09:48:56 +01:00
Stephanie Hingtgen 7f805d8e1a Unistore: Add comment on how RV checks are working (#111898) 2025-10-01 16:02:45 -05:00
Ryan McKinley c530cacb1c Storage: Propagate RV to server on update+delete (#111866) 2025-10-01 18:00:02 +03:00
Ryan McKinley cfbf64c3fd Validation: Move validation into apimachinery package (#111736) 2025-09-30 12:59:33 +00:00
Ryan McKinley 0a79b3bdc5 Chore: Upgrade k8s.io/api v0.34.1 and grafana-app-sdk v0.43.1 (#111009) 2025-09-16 13:35:20 +03:00
Ryan McKinley d3df5b8ddd Secrets: Manage secure values inside any resource (#107803) 2025-08-14 12:31:24 +00:00
Ryan McKinley e0404f924c K8s/SecureValues: Wire InlineSecureValueSupport to apistore (#109449)
* inline wire

* extra fields

* add variable

* wire
2025-08-11 15:22:56 +03:00
Ryan McKinley d4c446f42c Chore: Update k8s.io to v0.33.1 (#105307) 2025-05-28 00:13:18 +03:00
Ryan McKinley 1dd59ca599 DualWriter: remove RV+UID for secondary update (#105543) 2025-05-21 18:25:40 +00:00
Jean-Philippe Quéméner 002f46736a refactor(unified-storage): move generated protos to own pkg (#105356) 2025-05-15 21:36:52 +02:00
Matheus Macabu 4b426238bd Dependencies: Bump github.com/openfga/openfga from v1.8.6 to v1.8.12 (#105193)
* Dependencies: Bump github.com/openfga/openfga from v1.8.6 to v1.8.12

* Linter: Replace x/exp/rand with math/rand/v2

* NGAlert: Fix test after linter fixes
2025-05-14 11:09:00 +03:00
Ryan McKinley efd9334295 Provisioning: route provisioning storage to the provisioning client (#104082) 2025-04-18 21:47:37 +03:00
Ryan McKinleyandGeorges Chaudy af8a70bbab K8s/Permissions: Enable a grant-permissions annotation action to set default permissions (#102527)
* create permissions

* add key

* lint

* structure as a delayed callback

* legacy API hook

* merge main

* wired up

* and folders

* watch repos

* missing return statement

* Set the correct permissions

* add TestAfterCreatePermissionCreator

* do not add perms on folder create

* fix tests

* add annotation on create

* lint

* lint

* ensure we set permissions when the FT is disabled

* remove custom folder_storage

* fix lint

* change default

* lint

* lint

* fix: annotation

* ensure permissions are added on folder legacy

* remove folderstorage again

* fix tests

* add FT

* undo change to folder

* dashboard on create

* remove annotation for folder

* fix tests

* fix prepare after rebase

* fix tests

* fix tests

* fix tests

* lint

* address comments

* add test for prepareObjectForStorage

* add again skipIfMode as per comment

---------

Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2025-04-09 13:05:37 +02:00
Ryan McKinley d3e6e308a0 K8s/ManagedBy: Enforce who can CRUD provisioning resources (#103322) 2025-04-08 14:17:33 +03:00
Ryan McKinley 2e2b5942c8 K8s/Unified: Consolidate generation logic in apistore client (#102260) 2025-03-21 10:45:25 +02:00
Ryan McKinley 0845c781ae K8s/Storage: Refactor Upsert (create from update) (#102528) 2025-03-20 19:57:05 +03:00
Todd Treece 3992ac2ac1 Dashboard: Multi-version builder (#100305) 2025-02-21 06:50:29 -05:00
Georges Chaudy 2e32a3d19f unistore: fix dashboard conversion events (#101053)
fix dashboard conversion events
2025-02-20 14:28:03 +01:00
maiconandGeorges Chaudy fd377cfe66 Unistore: map grpc status to http status (#100942)
* Unistore: map grpc status to http status

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2025-02-19 15:29:57 -03:00
Ryan McKinley 6683cdc082 K8s: Update from v0.31.3 to v0.32.0 (#98507) 2025-01-06 18:20:35 +03:00
Ryan McKinley bc535181cf K8s/Dashboard: Always set an internl id (even in mode 4+) (#98320) 2024-12-20 22:17:59 +02:00
Stephanie Hingtgen 6571451a57 K8s: dashboards: fix conversion (#96648) 2024-11-19 05:32:05 -07:00
Ryan McKinley c0de407fee K8s/Dashboards: Delegate large objects to blob store (#94943) 2024-11-09 08:09:46 +03:00
Ryan McKinley 4e1f0dadbd UnifiedStorage: Default to running unified-grpc in integration tests (#93492) 2024-10-31 16:29:32 +03:00
a03652494c Dual Writer simplification (#93852)
* All objects should have an UID

* Now with a different error message

* Simplify create on DW 2: use the same object to write to both storages

* Run only one test

* Add check for status code

* Add name if it's not present in mode2

* Populate UID in legacy

* Remove logs and commented code

* Change dualwriter1

* Remove commented code

* Fix list test

* remove get on update from dualwriter 2

* Get object before updating. Better var renaming

* Finish rebasing

* Comment test

* Uncomment tests

* Update legacy first. Add preconditions

* Remove preconditions

* Fix update test

* copy RV from unified to legacy objects

* revert changes to playlist xorm store

* Improve logging. Add go routines for mode3

* Add tests for async funcs in mode3

* Lint

* Lint

* Lint. Start to fix tests

* Fix watcher tests

* Fix store tests

* Fiinish fixing watcher tests

* Fix server tests

* add name check

* Update pkg/apiserver/rest/dualwriter_mode1.go

Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>

* All objects should have an UID

* Now with a different error message

* Simplify create on DW 2: use the same object to write to both storages

* Run only one test

* Add check for status code

* Add name if it's not present in mode2

* Populate UID in legacy

* Remove logs and commented code

* Change dualwriter1

* Remove commented code

* Fix list test

* remove get on update from dualwriter 2

* Get object before updating. Better var renaming

* Finish rebasing

* Comment test

* Uncomment tests

* Fix update test

* revert changes to playlist xorm store

* Improve logging. Add go routines for mode3

* Lint

* Fix watcher tests

* Fiinish fixing watcher tests

* Add mode 5 with etcd test case. Add early check to fail on populated RV in payload

* we can't set RV to the found object when updating

* Lint

* Don't fail on update playlists

* Name should not be different when updating and it should be not empty on creating

* Fix tests

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* Lint

* Fix mode 5 tests

* Lint

* Add generateName condition on every mode. Fix tests

* Lint

* Add condition on where name or generate name have to be set

* Fix test

* Lint

* Fix folders test

* We dont need to send name for mode1

* Fail if UID is not present

* Remove change from not running test

* Remove unused line

* Lint

* Update pkg/storage/unified/apistore/store.go

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* Improve error message

* Fix broken watcher test

* Fail on name mismatch on update

* Remove log

* Make sure UIDs match on create in both stores

* Lint

* Write first to unified storage

* Remove uid setting

* Remove RV only in mode2

* Fix test. Remove log line

* test

* No need to asser on RV in mode3

* Remove RV check due to race condition

* Update dualwriter.go

Co-authored-by: Georges Chaudy <chaudyg@gmail.com>

* Update pkg/storage/unified/client.go

* remove unused parameter

* log an error for object is missing UID instead of returning an error

* remove obj.SetResourceVersion("")

* log an error for object is missing UID instead of returning an error

* FInalise merge

* Move RV check to where it was

* Remove name check

* Remove server check for backwards compatibility

* Remove unused fn

* Move test checks for another PR

* Dont commit go work sum changes

* Only log error if RV is present for now.

---------

Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2024-10-23 10:29:41 +02:00
Ryan McKinleyandJean-Philippe Quémémer 3457f219be Storage: Add blob storage interfaces (#90932)
Co-authored-by: Jean-Philippe Quémémer <jeanphilippe.quemener@grafana.com>
2024-10-17 12:18:29 +02:00
Prem Saraswat e2672021bc [unified-storage/apistore] Fix GuranteedUpdate skipping updates when tryUpdate is passed (#94557)
`GuranteedUpdate` method of `apistore.Storage` had a bug, where it would
errorneously conclude that the object is unchanged, in case a
`tryUpdate` function is passed that modifies the existing object itself
(as it is the case in many core types in K8s upstream).

The modified `existingObj` was compared with `updatedObj`, which would
essentially be same and this lead to the update being skipped.

This patch fixes this by always passing a copy of the `existingObj`.

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
2024-10-11 17:15:43 +05:30
Georges Chaudy 03f55e5288 Unistore : Ensure Watch works in HA mode (#94120)
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097)"

This reverts commit 7c3fc2f261.

* make previous_resource_version nullable

* handle nil case
2024-10-07 10:01:53 +02:00
Georges Chaudy 7c3fc2f261 Revert "Unistore : Ensure Watch works in HA mode." (#94097)
Revert "Unistore : Ensure Watch works in HA mode. (#93428)"

This reverts commit 0a26c9e9ae.
2024-10-01 18:45:47 +00:00
Georges ChaudyandDiego Augusto Molina 0a26c9e9ae Unistore : Ensure Watch works in HA mode. (#93428)
* Replace Watch with WatchNext

* remove watchset

* fix previous page and closing the channel

* Remove the broadcaster cache to prevent dupplicated events

* add watch bookmark

* add watch bookmark

* cleanup comments

* disable the tests for bookmarks for now

* Ensure we send previosu events

* lint

* re-introduce the cache

* load from cache

* disabling legacy test

* disabling legacy test

* Update pkg/storage/unified/resource/server.go

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>

* Could not read previous events

* add proper migration

* Add previous_resource_version to both history and resource

* First event should have an RV of 2 and not 1

* Test both storage backends

* fix the inital RV for the sql backend

* ensure graceful stop of the stream decoder

* gocyclo

---------

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-09-30 13:14:07 +02:00
Ryan McKinley 542105b680 ResourceClient: Exercise resource client in k8s apis tests (#93473) 2024-09-19 17:16:48 +03:00
Todd Treece b4126d3bce Chore: Update k8s.io dependencies (#91692) 2024-08-08 18:39:44 +03:00
Ryan McKinley da65825c5f Storage: Consolidate error handling (#91167) 2024-07-30 10:16:16 +00:00
Ryan McKinley 6e39f24588 Storage: Update storage.Interface for backend (#90382) 2024-07-18 07:47:47 -07:00