* adds metric for watch latency
* registers storage metrics when creating a new ResourceServer
* defines the latency (in milliseconds) as the diff between now and the RV. Still need to wait until PR for switching RV to millisecond timestamp is rolled out.
* should be micro seconds not milli
* for watch latency, use diff between now and resource version and convert to seconds
* fix typo
* Transforms raw US resource into an intermediate IndexableResource and indexes that. Pulls index mapping code out into different file. For now, we will hardcode which spec fields are indexed, per resource.
* Fixes a few bugs with field casing and timestamps not being formatted right (or not existing).
* adds readme section for using search with US
* extracts to function to transform from search hit to IndexedResource
* get folders when building index
`GuranteedUpdate` method of `apistore.Storage` had a bug, where it would
errorneously conclude that the object is unchanged, in case a
`tryUpdate` function is passed that modifies the existing object itself
(as it is the case in many core types in K8s upstream).
The modified `existingObj` was compared with `updatedObj`, which would
essentially be same and this lead to the update being skipped.
This patch fixes this by always passing a copy of the `existingObj`.
Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
* Use epoch with microsecond resolution as RV
* fix backend tests
* Add solution for when the clock goes back
* Add solution for when the clock goes back
* generate mocks
* go lint
* remove comment
* Use Greatest instead of max in msyql and postgres
* update tests
* Update pkg/storage/unified/sql/sqltemplate/dialect_sqlite.go
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* cast to bigint
* add additional round trip
* increment the RV using 2 sql round trips instead of 3
* cleanup comments
* cast unix timestamp to integer
* fix postgres query
* remove old increment test data
* remove greatest
* cast unix_timestamp to signed
* Use statement_timestamp instead of clock_timestamp
---------
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* resource-api: Loosen name validation to match K8s requirements
This patch modifies some of the requirements for name validation of
objects in Resource API to match Kubernetes.
The limit we have on characters in name is 64, but some resources allow
upto 253 characters. Similarly we also include `:` in the regex, as many
objects in default K8s setup use it in the name (the group
`system:masters` for example)
Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
* Update the name column length in migrator and update e2e test to verify
---------
Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
* adds Filter gRPC and make protobuf
* adds route for querying the filter gRPC
* wires up Filter gRPC call
* [WIP] index from start
* renames gRPC endpoint to "Search"
* adds /apis/search route into k8s routes. Hacky for now.
* updates readme - wrong casing
* adds feature toggle for unified storage search
* hides US search behind feature flag. Clean up print statements.
* removes indexer - will be added in another PR
* Search: Add API Builder
* adds required method
* implementing UpdateAPIGroupInfo (WIP)
* adds groupversion
* commenting out for now
* remove unneeded code from experimenting and update register.go to match interface required
* list resources and load into index
* pass context
* namespaces search route
* lint
* watch
* add todo
* add todo
* merge
* cleanup
* add todo
* gen protobuf
* lint; fix migration issue
* Updates index mapping function to map unified storage object Value
* Changes Index() to pointer receiver - fixes panic
* add delete
* cleanup
* gets search/browse functioning. Results show up as base64 encoded. Still a WIP.
* Doesnt json re-encode gRPC response in search handler
* add kind to SearchRequest proto
* Updates query interface to be more generic. Make proto. Parses query params in api server.
* make protobuf
* removes unused method and imports
* Returns all indexed fields in search results. Adds pagination support (limit + offset).
* remove comment
* remove unused struct
* gets tenant in search k8s api handler
* adds hardcoded spec field mappings - starting with playlists
* adds all spec fields to search results
* moved helper function for field mappings into index
* only includes allowed spec fields in search results
* cleans up error handling
* removes debug log
---------
Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
* Add parents field to folder DTO
* Allow subfolder creation when folder flag is enabled
* Update UnstructuredToLegacyFolder
* Include parents field when creating folder
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097)"
This reverts commit 7c3fc2f261.
* make previous_resource_version nullable
* handle nil case
* Reuse MySQL and Postgres Grafana config instead of the object
- Only reuse the Grafana DB object for SQLite. Support for SQLite will be added in a different PR
- Fail when reusing the Grafana DB object if it is using DB instrumentation
- In the case that we have to reuse a Grafana DB with its instrumentation, fail with an error that describes a workaround
- Add regression tests to reproduce incident 2144
* remove temp file
* fix linter
* fix linter x2
* fix linter x3
* adds Filter gRPC and make protobuf
* adds route for querying the filter gRPC
* wires up Filter gRPC call
* [WIP] index from start
* renames gRPC endpoint to "Search"
* adds /apis/search route into k8s routes. Hacky for now.
* updates readme - wrong casing
* adds feature toggle for unified storage search
* hides US search behind feature flag. Clean up print statements.
* removes indexer - will be added in another PR
* Search: Add API Builder
* adds required method
* implementing UpdateAPIGroupInfo (WIP)
* adds groupversion
* commenting out for now
* remove unneeded code from experimenting and update register.go to match interface required
* namespaces search route
---------
Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
* Replace Watch with WatchNext
* remove watchset
* fix previous page and closing the channel
* Remove the broadcaster cache to prevent dupplicated events
* add watch bookmark
* add watch bookmark
* cleanup comments
* disable the tests for bookmarks for now
* Ensure we send previosu events
* lint
* re-introduce the cache
* load from cache
* disabling legacy test
* disabling legacy test
* Update pkg/storage/unified/resource/server.go
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* Could not read previous events
* add proper migration
* Add previous_resource_version to both history and resource
* First event should have an RV of 2 and not 1
* Test both storage backends
* fix the inital RV for the sql backend
* ensure graceful stop of the stream decoder
* gocyclo
---------
Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
* Add authlib gRPC authenticators for in-proc mode
* implement `StaticRequester` signing in the unified resource client
- [x] when the `claims.AuthInfo` value type is `identity.StaticRequester`, and there's no ID token set, create an internal token and sign it with symmetrical key. This is a workaround for `go-jose` not offering the possibility to create an unsigned token.
- [x] update `IDClaimsWrapper` to support the scenario above
- [x] Switch to using `claims.From()` in `dashboardSqlAccess.SaveDashboard()`
---------
Co-authored-by: gamab <gabriel.mabille@grafana.com>
owensmallwood
Scott Lepper
Diego Augusto Molina
Ryan McKinley
Prem Saraswat
Georges Chaudy
Arati R.
Matheus Macabu
Todd Treece
Claudiu Dragalina-Paraipan
Andres Martinez Gotor
ismail simsek
Dave Henderson
Santiago
Marcus Efraimsson
maicon
Charandas