Commit Graph

35 Commits

Author SHA1 Message Date
Ryan McKinley 17e4a3b386 Playlists: Remove kubernetesPlaylists flag (#104171) 2025-04-22 10:39:40 +02:00
Mariell Hoversholm e1e1d3fd9f Fix: Prints should always include new lines (#102795)
* CI: Allow Bench conversion to fail

We shouldn't mark PRs and commits as X if they fail to convert logs with Bench.

* Fix: Prints should always include new lines

* fix: remove unused import
2025-03-27 12:27:53 +01:00
Ryan McKinley 2e2b5942c8 K8s/Unified: Consolidate generation logic in apistore client (#102260) 2025-03-21 10:45:25 +02:00
Karl Persson 6eeb28e312 Authn: use authenticator for grpc (#99573)
* Remove usage of grpc-authenticator

* Cleanup client construction code
2025-02-17 10:58:59 +01:00
Ryan McKinley 4e703576b0 K8s/OpenAPI: Render openapi into a static file (#99561) 2025-01-28 10:30:53 +03:00
Ryan McKinley d8b6ded101 K8s: use deprecatedInternalID label rather than SQL annotation (#99263) 2025-01-21 12:38:14 +03:00
Ryan McKinley b506fcb11c K8s: Fix the gitVersion response (required for kubectl) (#98571) 2025-01-07 11:43:19 +02:00
Leonor Oliveira a03652494c Dual Writer simplification (#93852)
* All objects should have an UID

* Now with a different error message

* Simplify create on DW 2: use the same object to write to both storages

* Run only one test

* Add check for status code

* Add name if it's not present in mode2

* Populate UID in legacy

* Remove logs and commented code

* Change dualwriter1

* Remove commented code

* Fix list test

* remove get on update from dualwriter 2

* Get object before updating. Better var renaming

* Finish rebasing

* Comment test

* Uncomment tests

* Update legacy first. Add preconditions

* Remove preconditions

* Fix update test

* copy RV from unified to legacy objects

* revert changes to playlist xorm store

* Improve logging. Add go routines for mode3

* Add tests for async funcs in mode3

* Lint

* Lint

* Lint. Start to fix tests

* Fix watcher tests

* Fix store tests

* Fiinish fixing watcher tests

* Fix server tests

* add name check

* Update pkg/apiserver/rest/dualwriter_mode1.go

Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>

* All objects should have an UID

* Now with a different error message

* Simplify create on DW 2: use the same object to write to both storages

* Run only one test

* Add check for status code

* Add name if it's not present in mode2

* Populate UID in legacy

* Remove logs and commented code

* Change dualwriter1

* Remove commented code

* Fix list test

* remove get on update from dualwriter 2

* Get object before updating. Better var renaming

* Finish rebasing

* Comment test

* Uncomment tests

* Fix update test

* revert changes to playlist xorm store

* Improve logging. Add go routines for mode3

* Lint

* Fix watcher tests

* Fiinish fixing watcher tests

* Add mode 5 with etcd test case. Add early check to fail on populated RV in payload

* we can't set RV to the found object when updating

* Lint

* Don't fail on update playlists

* Name should not be different when updating and it should be not empty on creating

* Fix tests

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* Lint

* Fix mode 5 tests

* Lint

* Add generateName condition on every mode. Fix tests

* Lint

* Add condition on where name or generate name have to be set

* Fix test

* Lint

* Fix folders test

* We dont need to send name for mode1

* Fail if UID is not present

* Remove change from not running test

* Remove unused line

* Lint

* Update pkg/storage/unified/apistore/store.go

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* Improve error message

* Fix broken watcher test

* Fail on name mismatch on update

* Remove log

* Make sure UIDs match on create in both stores

* Lint

* Write first to unified storage

* Remove uid setting

* Remove RV only in mode2

* Fix test. Remove log line

* test

* No need to asser on RV in mode3

* Remove RV check due to race condition

* Update dualwriter.go

Co-authored-by: Georges Chaudy <chaudyg@gmail.com>

* Update pkg/storage/unified/client.go

* remove unused parameter

* log an error for object is missing UID instead of returning an error

* remove obj.SetResourceVersion("")

* log an error for object is missing UID instead of returning an error

* FInalise merge

* Move RV check to where it was

* Remove name check

* Remove server check for backwards compatibility

* Remove unused fn

* Move test checks for another PR

* Dont commit go work sum changes

* Only log error if RV is present for now.

---------

Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
2024-10-23 10:29:41 +02:00
Todd Treece 177965704d Storage: Test mode 5 (#93714) 2024-09-25 08:29:17 -04:00
Todd Treece d1ffcc22d9 Playlists: Migrate to App SDK codegen (#93246) 2024-09-13 16:27:40 -04:00
Ryan McKinley 7efadb0a00 Testing: Integration tests for unified-grpc start a local gRPC server (#93201) 2024-09-11 11:50:14 +03:00
Claudiu Dragalina-Paraipan 085aaec958 Fix playlist tests (#92837) 2024-09-05 15:41:28 +03:00
Leonor Oliveira 2e451b2ed7 Use dw dynamic config (#91882)
* Remove kubernetesPlaylists feature_toggle

* Remove unified_storage_mode

* Remove double import

* Read from config instead from feature_toggle

* cover scenario for when unified storage is not defined

* Be temporarily retro compatible with previous feature toggle

* Properly read unified_storage section

* [WIP] Read new format of config

* Fix test

* Fix other tests

* Generate feature flags file

* Use <group>.<resource> schema

* Use <group>.resource format on the FE as well

* Hide UniStore config from Frontend

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* unwanted changes

* Use feature toggles in the FE. Enforce FTs are present before enabling dual writing
Co-authored-by: Ryan McKinley <ryantxu@users.noreply.github.com>

* use kubernetes playlists feature toggle on the FE

* Remove unwanted code

* Remove configs from the FE

* Remove commented code

* Add more explicit example

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Maicon Costa <maiconscosta@gmail.com>
2024-08-30 11:59:42 +02:00
Ryan McKinley 2136fd9a92 Storage: Remove unified storage feature flag (#92192)
remove unified storage flag
2024-08-21 19:28:30 +03:00
Leonor Oliveira d52d04b6d8 Revert "Use dw dynamic config" (#91846)
Revert "Use dw dynamic config (#91222)"

This reverts commit 0258842f87.
2024-08-13 17:24:26 +02:00
Leonor Oliveira 0258842f87 Use dw dynamic config (#91222)
* Remove kubernetesPlaylists feature_toggle

* Remove unified_storage_mode

* Remove double import

* Regenerate feature-toggles

* Read from config instead from feature_toggle

* cover scenario for when unified storage is not defined
2024-08-13 10:03:28 +02:00
Leonor Oliveira 67b74e1e8a Dual writer: mode 3 (#90045)
* Dual writer: mode 3

* Add integration tests for playlits in mode 3

* Remove todo

* Update pkg/apiserver/rest/dualwriter_mode3.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Admin: Fixes an issue where user accounts could not be enabled (#88117)

Fix: unable to enable user

* [REVIEW] FInish mode 3 and add tests

* Improve logging

* Update dependencies

* Update pkg/apiserver/rest/dualwriter_mode3_test.go

Co-authored-by: maicon <maiconscosta@gmail.com>

* remove test assertion

* Use mode log when dual writer is initiated

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: gonvee <gonvee@qq.com>
Co-authored-by: maicon <maiconscosta@gmail.com>
2024-07-24 09:28:14 +02:00
Ryan McKinley 079f0715aa ResourceServer: Add resource server protobuf and wrapper (#90007) 2024-07-09 15:08:13 -07:00
Ryan McKinley de06762852 K8s: Add basic peakq test w/ resource (#90026) 2024-07-04 01:26:33 +03:00
Ryan McKinley 9b7f9ae22e K8s: Rename origin.key to origin.hash (#89337) 2024-06-18 22:27:16 +03:00
Leonor Oliveira b30c81b1ad Add and fix tests for playlists in mode1 (#88543)
* Add and fix tests for playlists in mode1

* Make etcd tests pass mode1 for now

* Fix mode1 and add more tests for playlists in mode 1

* Remove repeated test

* Fix test setup
2024-06-10 16:11:01 +02:00
Arati R 36f42853dd Storage: Read desired mode from config instead of feature flags (#88353)
* Read desired mode from config
* Update playlist integration tests
* Add mode 1 playlist integration tests
* Add mode 0 dual writing to playlist integration tests
* Add documentation for the different dual writing modes
2024-05-31 19:29:59 +02:00
Arati R 4867fd3069 Storage: Test mode 2 dual writer in playlist tests (#87938)
Add mode 2 feature flag to playlist tests
2024-05-16 10:39:30 +02:00
Leonor Oliveira ee2f6a7b49 Force interface implementation also on legacy storage (#87414)
* Force interface implementation also on legacy storage

* Add DeleteCollection to folders and dashboards

* Fix integration tests

* Fix tests
2024-05-07 15:02:30 +02:00
Ryan McKinley 74115f1f08 Chore: fix apiserver integration tests (#83724)
Co-authored-by: Todd Treece <todd.treece@grafana.com>
2024-02-29 14:58:49 -08:00
Ryan McKinley 2b355ff280 K8s: Remove grafanaAPIServer feature toggle (#81030) 2024-01-23 08:27:28 -08:00
Dan Cech c4c9bfaf2e Storage: Unified Storage based on Entity API (#71977)
* first round of entityapi updates

- quote column names and clean up insert/update queries
- replace grn with guid
- streamline table structure

fixes

streamline entity history

move EntitySummary into proto

remove EntitySummary

add guid to json

fix tests

change DB_Uuid to DB_NVarchar

fix folder test

convert interface to any

more cleanup

start entity store under grafana-apiserver dskit target

CRUD working, kind of

rough cut of wiring entity api to kube-apiserver

fake grafana user in context

add key to entity

list working

revert unnecessary changes

move entity storage files to their own package, clean up

use accessor to read/write grafana annotations

implement separate Create and Update functions

* go mod tidy

* switch from Kind to resource

* basic grpc storage server

* basic support for grpc entity store

* don't connect to database unless it's needed, pass user identity over grpc

* support getting user from k8s context, fix some mysql issues

* assign owner to snowflake dependency

* switch from ulid to uuid for guids

* cleanup, rename Search to List

* remove entityListResult

* EntityAPI: remove extra user abstraction (#79033)

* remove extra user abstraction

* add test stub (but

* move grpc context setup into client wrapper, fix lint issue

* remove unused constants

* remove custom json stuff

* basic list filtering, add todo

* change target to storage-server, allow entityStore flag in prod mode

* fix issue with Update

* EntityAPI: make test work, need to resolve expected differences (#79123)

* make test work, need to resolve expected differences

* remove the fields not supported by legacy

* sanitize out the bits legacy does not support

* sanitize out the bits legacy does not support

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* update feature toggle generated files

* remove unused http headers

* update feature flag strategy

* devmode

* update readme

* spelling

* readme

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-06 15:21:21 -05:00
Ryan McKinley 49fc8214a0 K8s: Add etcd tests for dual write (local) (#78161) 2023-11-17 14:20:54 -05:00
Ryan McKinley 2b1e731c15 K8s/Playlist: Test dual write path (#77604) 2023-11-15 00:07:32 +02:00
Ryan McKinley 1be1432926 K8s: Add subresource to the example apiserver (#78030) 2023-11-14 05:51:58 +02:00
Ryan McKinley 4b5310dae7 Playlists: Merge the frontend vs backend k8s feature flags (#77875) 2023-11-08 21:14:05 +02:00
Ryan McKinley 549787d4f9 Playlist: Implement the entire API with k8s client (#77596) 2023-11-03 09:25:29 -07:00
Ryan McKinley e3641d925c K8s/Playlist: Support full CRUD from k8s to existing storage (#75709) 2023-11-01 12:32:24 -07:00
Ryan McKinley 9b472b3726 K8s: Use client-go to test legacy playlist changes (#77245) 2023-10-27 06:59:49 -07:00
Ryan McKinley c122ffc72b K8s: Add integration test infra, and fix authz patterns (#77218) 2023-10-26 20:02:33 +03:00