Commit Graph
85 Commits
Author SHA1 Message Date
Ryan McKinley 5441e4c752 Storage: create a client with access to all interfaces, not just ResourceStore (#92967) 2024-09-05 12:52:30 +03:00
2e451b2ed7 Use dw dynamic config (#91882)
* Remove kubernetesPlaylists feature_toggle

* Remove unified_storage_mode

* Remove double import

* Read from config instead from feature_toggle

* cover scenario for when unified storage is not defined

* Be temporarily retro compatible with previous feature toggle

* Properly read unified_storage section

* [WIP] Read new format of config

* Fix test

* Fix other tests

* Generate feature flags file

* Use <group>.<resource> schema

* Use <group>.resource format on the FE as well

* Hide UniStore config from Frontend

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>

* unwanted changes

* Use feature toggles in the FE. Enforce FTs are present before enabling dual writing
Co-authored-by: Ryan McKinley <ryantxu@users.noreply.github.com>

* use kubernetes playlists feature toggle on the FE

* Remove unwanted code

* Remove configs from the FE

* Remove commented code

* Add more explicit example

---------

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Maicon Costa <maiconscosta@gmail.com>
2024-08-30 11:59:42 +02:00
de2c9a06bf Unified Storage: First iteration Dual Write Syncer (#89809)
* Unified Storage: First iteration Dual Write Syncer

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Leonor Oliveira <9090754+leonorfmartins@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
2024-08-27 13:31:40 -03:00
Ryan McKinley 2136fd9a92 Storage: Remove unified storage feature flag (#92192)
remove unified storage flag
2024-08-21 19:28:30 +03:00
Todd Treece 1b0e6b5c09 K8s: Data Plane Aggregator (#91228) 2024-08-13 22:28:17 +03:00
Todd Treece b4126d3bce Chore: Update k8s.io dependencies (#91692) 2024-08-08 18:39:44 +03:00
owensmallwood 2387798719 Unified Storage: Propagate traces from the apiserver grpc client (#91226) 2024-08-06 09:52:11 -06:00
Ryan McKinley be7b1ce2df Chore: Replace appcontext.User(ctx) with identity.GetRequester(ctx) (#91030) 2024-07-26 16:39:23 +03:00
Georges ChaudyandRyan McKinley 4baca6947d ResourceServer: make the resource store the default unified storage backend (#90899)
* make the resource store the default unified storage backend

* add integration tests

* fix test non passing

* Update pkg/storage/unified/sql/test/integration_test.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* lint

* fix tests

* fix no rows

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-07-25 18:17:39 +02:00
Georges Chaudy 08c611c68b ResourceServer: Resource store sql backend (#90170) 2024-07-18 08:03:18 -07:00
Ryan McKinley 6e39f24588 Storage: Update storage.Interface for backend (#90382) 2024-07-18 07:47:47 -07:00
Ryan McKinley d2bc4f3255 DualWrite: Cleanup and centralize the dual write creation (#90013) 2024-07-11 23:23:31 +03:00
Ryan McKinley 079f0715aa ResourceServer: Add resource server protobuf and wrapper (#90007) 2024-07-09 15:08:13 -07:00
Todd Treece 9c6d3590a2 Chore: Fix apiserver circular dependencies (#89918) 2024-07-01 11:42:34 -04:00
Ryan McKinley 27e800768e K8s: Improve identity mapping setup (#89450) 2024-06-20 17:53:07 +03:00
Leonor Oliveira fd44f2ee4f Init dualwriter metrics (#89003)
* Pass prometheus registerer to the dual writer

* Fix tests

* Remove unused var

* Fix tests

* Uncomment test

* Remove leading line

* Fix tests. Reuse registerer if there's already one

* Lint

* Improve double registering logic

* Rebase main
2024-06-14 11:01:49 +02:00
Diego Augusto Molina 6fcd7d9e03 Unified Storage: Fix Create, Update and Delete wrt Resource Versions (#88183)
* add sqltemplate utilities, improve tests and documentation

* bunch of things

* remove unnecessary message

* add queries

* add queries

* add queries

* add folders support

* fix diff

* fix linters

* fix diff

* fix linters

* fix linters

* fix typo

* fix linters

* fix linters

* fix linters

* several fixes

* several fixes

* temporarily disable k8s integration tests for Entity Server

* postpone some tests

* postpone documentation changes

* Fix bug in create

* improve error reporting

* fix PostgeSQL parameters

* fix MySQL sqlmode

* fix MySQL-5.7

* reduce but document the number of database connection options

* remove unused code and improve docs
2024-06-05 14:23:32 -03:00
Arati R 36f42853dd Storage: Read desired mode from config instead of feature flags (#88353)
* Read desired mode from config
* Update playlist integration tests
* Add mode 1 playlist integration tests
* Add mode 0 dual writing to playlist integration tests
* Add documentation for the different dual writing modes
2024-05-31 19:29:59 +02:00
Giuseppe Guerra 2b54ce6c51 Chore: Bump google.golang.org/grpc to 1.64.0 (#88219)
* Bump google.golang.org/grpc to 1.64.0

* Address grpc.Dial deprecation notices

* go mod tidy

* Update go.work.sum

* go work sync
2024-05-27 12:44:39 +02:00
owensmallwood 557c3a9cdd Unified Storage: Adds metrics for database query counts (#87781) 2024-05-14 10:17:38 -06:00
jackyin ba8b4bde3a fix ipv6 startup fail #81870 (#87108)
* fix ipv6 startup fail #81870

* ipv6 startup fail

* ipv6 startup fail
2024-05-06 14:17:03 -04:00
Georges Chaudy dd0e646e92 k8s: remove dependency on app_mode development (#85474)
* k8s: remove dependency on app_mode development

* remove dev conditions on storage server

* remove logic to drop entity tables
2024-05-03 11:28:15 +03:00
owensmallwood 8c8885ef23 Storage Api: Adds traces (#85391)
- adds traces and improved logging to the unified storage server
- add a configurable logger to the gRPC server service
2024-04-16 08:30:51 -06:00
Charandas b6a020148b K8s: disallow MT storage functionality for Aggregator builders (#84408) 2024-03-14 01:54:30 +02:00
Georges Chaudy 15e358e3b9 k8s: add support for configuring the gRPC server address (#84006)
* k8s: add support for configuring the gRPC server address
2024-03-11 11:23:03 +02:00
CharandasandTodd Treece e916372249 K8s: bug fixes for file storage to allow for watcher initialization on startup (#83873)
---------

Co-authored-by: Todd Treece <todd.treece@grafana.com>
2024-03-05 10:34:47 -08:00
Dan Cech 9fa9eaab44 Storage: Support get with resourceversion (#83849)
support getting old resourceversion, return explicit resource version in list
2024-03-05 10:57:32 -05:00
Charandas b87ec69431 K8s: add a remote services file config option to specify aggregation config (#83646) 2024-02-29 17:29:05 -08:00
Todd Treece e5a26a3f7c K8s: Add apimachinery and apiserver packages (#83190) 2024-02-23 15:15:43 -05:00
Todd Treece f593161ef6 K8s: Set X-Remote- headers for SignedInUser (#82543) 2024-02-15 12:29:36 -05:00
Todd TreeceandCharandas Batra d6e6298103 K8s: Add Aggregation to Backend Service (#81591)
Co-authored-by: Charandas Batra <charandas.batra@grafana.com>
2024-02-12 22:59:35 +02:00
Dan Cech 1f1461734c Storage: Add support for sortBy selector (#80680)
* add support for sortBy field selector

* use label selectors instead of field selectors

* set entity_labels on create & update

* make entity server integration tests work

* test fixes

* be more consistent with handling of empty body, meta or status

* workaround for database is locked errors during migration

* fix double import of sqlite3

* rename functions and tidy up

* refactor update

* disable integration tests until we can fix the database locking issue
2024-02-07 15:05:10 -05:00
Dan Cech 156d7ae194 use in-process grpc client instead of wrapping server interface (#81926)
* use in-process grpc client instead of wrapping server interface

* comment out jwt token checks until we're ready to validate the token
2024-02-07 13:17:02 -05:00
Ryan McKinley 651faff08a Chore: go mod tidy after apiserver refactor (#81824) 2024-02-02 23:03:19 +02:00
Todd Treece 67b6be5515 K8s: Refactor config/options for aggregation (#81739) 2024-02-01 17:27:30 -05:00