Commit Graph

30 Commits

Author SHA1 Message Date
grafana-delivery-bot[bot]
f6606c558e [v11.0.x] Storage: GuaranteedUpdate fix & other improvements (#86323)
Storage: GuaranteedUpdate fix & other improvements (#85206)

make GuaranteedUpdate work when ignoring not found errors, increase poll frequency, fix Delete

(cherry picked from commit ef26fe95dc)

Co-authored-by: Dan Cech <dcech@grafana.com>
2024-04-16 14:42:56 +01:00
Dan Cech
89f3b70e17 Storage: Add support for listing resource history (#84331)
* add support for listing resource history

* make watch handle custom label selectors properly

* fix tests

* Apply suggestions from code review

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>

* properly handle special characters in json label matcher

* tidy up

---------

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-03-15 19:17:54 -04:00
Dan Cech
0fa0cede75 Storage: streamline context handling (#84319)
streamline context handling
2024-03-14 17:12:20 -04:00
Dan Cech
1ffd1cc8f4 Storage: Support listing deleted entities (#84043)
* support listing deleted entities

* fold listDeleted into List
2024-03-11 13:59:54 -04:00
Dan Cech
7e4badff1d Storage: Use our own key format and support unnamespaced objects (#83929)
* use our own key format and support unnamespaced objects

* fix tests
2024-03-05 16:31:39 -05:00
Dan Cech
9fa9eaab44 Storage: Support get with resourceversion (#83849)
support getting old resourceversion, return explicit resource version in list
2024-03-05 10:57:32 -05:00
Dan Cech
7b4925ea37 Storage: Watch support (#82282)
* initial naive implementation

* Update pkg/services/store/entity/sqlstash/sql_storage_server.go

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>

* tidy up

* add action column, batch watch events

* initial implementation of broadcast-based watcher

* fix up watch init

* remove batching, it just adds needless complexity

* use StreamWatcher

* make broadcaster generic

* add circular buffer to replay recent events to new watchers

* loop within poll until all events are read

* add index on entity_history.resource_version to support poller

* increment r.Since when we send events to consumer

* switch broadcaster and cache to use channels instead of mutexes

* cleanup

---------

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
2024-03-05 10:14:38 -05:00
Dan Cech
790e1feb93 Chore: Update test database initialization (#81673)
* streamline initialization of test databases, support on-disk sqlite test db

* clean up test databases

* introduce testsuite helper

* use testsuite everywhere we use a test db

* update documentation

* improve error handling

* disable entity integration test until we can figure out locking error
2024-02-09 09:35:39 -05:00
Dan Cech
1f1461734c Storage: Add support for sortBy selector (#80680)
* add support for sortBy field selector

* use label selectors instead of field selectors

* set entity_labels on create & update

* make entity server integration tests work

* test fixes

* be more consistent with handling of empty body, meta or status

* workaround for database is locked errors during migration

* fix double import of sqlite3

* rename functions and tidy up

* refactor update

* disable integration tests until we can fix the database locking issue
2024-02-07 15:05:10 -05:00
Ryan McKinley
e041055012 K8s/Folders: Rename api group to singular (#81443) 2024-01-30 08:50:37 -08:00
Arati R
77775d548a Chore: Add some tests for the Create method (#81364)
* Add test for create method
    Co-authored-by: Tania B <10127682+undef1nd@users.noreply.github.com>
* Change structure of entity package to break import cycle
* Update wire file

---------

Co-authored-by: Tania B <10127682+undef1nd@users.noreply.github.com>
2024-01-30 14:15:12 +01:00
Arati R
ca9d147a44 Give dialects control over how insert and update queries are performed (#79946)
* Refactor insert, update
* Add separate insert, update methods
* Refactor Insert, Update signatures
2024-01-11 19:55:45 +01:00
Ryan McKinley
d195c3807e Storage: Use int64 for resourceVersion (#79735) 2023-12-20 11:26:35 -08:00
Ryan McKinley
67bbdd7c05 K8s/Folders: Add folders api service (with legacy storage) (#79413) 2023-12-20 20:28:56 +02:00
Dan Cech
d146f4cbc7 Storage: rename uid to name and name to title (#79418)
rename uid to name and name to title
2023-12-12 15:14:55 -05:00
Dan Cech
9f6144059a Storage: Switch from tenant to namespace & remove GRN (#79250)
* remove GRN and switch tenant to namespace

* clean up remaining references

* simplify and remove inconsistency in With* parameters

* parse listing keys so we can use db index

* bump the schema version

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-11 12:26:05 -05:00
Dan Cech
c4c9bfaf2e Storage: Unified Storage based on Entity API (#71977)
* first round of entityapi updates

- quote column names and clean up insert/update queries
- replace grn with guid
- streamline table structure

fixes

streamline entity history

move EntitySummary into proto

remove EntitySummary

add guid to json

fix tests

change DB_Uuid to DB_NVarchar

fix folder test

convert interface to any

more cleanup

start entity store under grafana-apiserver dskit target

CRUD working, kind of

rough cut of wiring entity api to kube-apiserver

fake grafana user in context

add key to entity

list working

revert unnecessary changes

move entity storage files to their own package, clean up

use accessor to read/write grafana annotations

implement separate Create and Update functions

* go mod tidy

* switch from Kind to resource

* basic grpc storage server

* basic support for grpc entity store

* don't connect to database unless it's needed, pass user identity over grpc

* support getting user from k8s context, fix some mysql issues

* assign owner to snowflake dependency

* switch from ulid to uuid for guids

* cleanup, rename Search to List

* remove entityListResult

* EntityAPI: remove extra user abstraction (#79033)

* remove extra user abstraction

* add test stub (but

* move grpc context setup into client wrapper, fix lint issue

* remove unused constants

* remove custom json stuff

* basic list filtering, add todo

* change target to storage-server, allow entityStore flag in prod mode

* fix issue with Update

* EntityAPI: make test work, need to resolve expected differences (#79123)

* make test work, need to resolve expected differences

* remove the fields not supported by legacy

* sanitize out the bits legacy does not support

* sanitize out the bits legacy does not support

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* update feature toggle generated files

* remove unused http headers

* update feature flag strategy

* devmode

* update readme

* spelling

* readme

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-06 15:21:21 -05:00
Dan Cech
67b2972052 Chore: add/update sqlstore-related helper functions (#77408)
* add/update sqlstore-related helper functions

* add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns

* remove old log line
2023-11-03 10:30:52 -04:00
Dan Cech
85a207fceb Chore: Replace entity GRN with infra/grn GRN (#74198)
replace entity GRN with infra/grn GRN
2023-08-31 15:43:35 -04:00
Ryan McKinley
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Ryan McKinley
bb9ae04bd8 Chore: cleanup entity api from CodeQL scan (#64277)
cleanup
2023-03-07 07:54:25 -08:00
Emil Tullstedt
3abaf32cf2 Chore: Upgrade golangci-lint to v1.51.2 (#63630)
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-02-23 15:10:03 +01:00
Ryan McKinley
46c828c2d8 EntityAPI: Rename references from kind to family (#62044) 2023-01-25 23:42:04 +00:00
Kristin Laemmert
dd147a3c31 chore: move entity models into entity store service (#62145) 2023-01-25 12:43:22 -05:00
Ryan McKinley
624e5dbed2 EntityAPI: Save nested summary info in the SQL database (#61732) 2023-01-21 00:00:17 +00:00
Ryan McKinley
0c20fe0ac9 EntityAPI: Include folder support and watch API stubs (#61338) 2023-01-13 00:39:36 +00:00
Ryan McKinley
d036225f7b Chore: get user from appcontext rather than store utility (#59926)
user from appcontext
2022-12-07 07:38:23 -08:00
Artur Wierzbicki
9e349bf9b0 EntityStore: support filtering by labels (#59905)
* label filtering

* filtering in sql

* filtering in sql group by

* label is unique - no need for distinct

* capitalize

* fix capitalization
2022-12-06 23:21:12 +04:00
Ryan McKinley
fb98a97efa EntityStore: Move slug+folder to summary metadata (#59620) 2022-12-01 14:45:08 -05:00
Ryan McKinley
5dc7b01f9d EntityStore: Rename ObjectStore to EntityStore (part 1) (#59613) 2022-11-30 13:42:42 -08:00