Commit Graph

48 Commits

Author SHA1 Message Date
Serge Zaitsev
cdd7a2cfd2 Chore: Disable CGo in tests (#108764)
* make cgo optional for sqlite

* update go.mod; check error code differently

* reduce api surface even more

* move test errors into sqlite package

* CGO_ENABLED=0 in unit tests

* disable for enterprise, too

* add driver name constant

* remove unused constants

* make test an integration one

* try integration tests without cgo

* implement error codes for modernc sqlite driver

* typo fix

* missing return

* use error pointer as an interface

* alias the driver

* update workspace, check for test errors too

* check error properly

* add missing driver after rebase

* fix missing import after rebase

* debugging, lets try again

* properly parse options, revert many previous changes

* remove another log

* better url parsing

* revert test rename, leave it for later

* revert reusedSession in unistore

* revert more code

* remove driver name

* revert formatting

* add integration test without cgo for sqlite

* remove tracing and logging

* bring driver alias back

* fix type

* wrong package
2025-09-02 17:24:30 +02:00
Mustafa Sencer Özcan
84a4ed612b fix: re-order db initialization logic for resource server (#108059) 2025-07-14 10:25:42 +00:00
Jean-Philippe Quéméner
602b7826c4 refactor(unified-storage): remove resource_server config option (#107649) 2025-07-11 15:33:51 +02:00
Ryan McKinley
3f502f305d Chore: Update mocks with recent mockery (#107816) 2025-07-09 09:15:34 +02:00
Peter Štibraný
041c343a86 Unified storage: Respect GF_DATABASE_URL override (#105331)
* Database for unified storage resources now reuses DB code that respects URL override.

Access instrument_queries via section getter.
2025-07-03 09:57:40 +00:00
Peter Štibraný
e076c74869 sqltemplate, dbimpl: Remove single-method function types (#107525)
* Remove dbProviderFunc function.

This removes one extra indirection that made the code bit more difficult to navigate.

* Remove indirection function types implementing single-method interfaces.

This streamlines the code and makes it bit easier to navigate.

* Update pkg/storage/unified/sql/sqltemplate/dialect_mysql.go

Co-authored-by: Mustafa Sencer Özcan <32759850+mustafasencer@users.noreply.github.com>

---------

Co-authored-by: Mustafa Sencer Özcan <32759850+mustafasencer@users.noreply.github.com>
2025-07-03 10:38:12 +02:00
Mustafa Sencer Özcan
3674d952a6 fix: use database section for resource server db engine configuration (#107495) 2025-07-02 09:00:51 +00:00
Jean-Philippe Quéméner
9a92900ef3 fix(unified-storage): remove 's' in config parameter (#107457) 2025-07-01 14:06:21 +02:00
Jean-Philippe Quéméner
d76e55371a fix(unified-storage): use the provided connection config parameters (#107455)
* fix(unified-storage): use the provided connection config parameters

* extend tests

* make update-workspace
2025-07-01 11:25:16 +00:00
Jean-Philippe Quéméner
392c1a71c9 Storage: Take migration_locking setting into account (#105938) 2025-05-23 14:11:21 +02:00
Peter Štibraný
1ffe653156 Revert change of order of checks when setting up resource DB (#105514)
* Revert change of order in newResourceDBProvider

* Ignore error in provisioningSecrets when working with wrong DB.

* Update workspace.

* Add owner.
2025-05-16 11:35:36 +03:00
Peter Štibraný
7268d6c1bc admin api: Fix reencryption of private_key in signing_key table, introduce test for reencryption (#105010)
* Add integration test for reencrypting secrets.
Rotate keys before reencryption.
Test for setting table from enterprise repo.
2025-05-15 09:51:25 +02:00
Serge Zaitsev
694b9dfe50 Chore: Replace xorm.io/xorm imports (#104458)
* replace xorm.io/xorm imports

* replace xorm from other go.mod files

* clean up workspace

* nolint does not make sense anymore as it is not a module

* try if nolint directive helps

* use nolint:all for xorm

* add more nolints

* try to skip xorm in linter config

* exclude xorm differently

* retrigger ci
2025-05-02 17:13:01 +02:00
Leonor Oliveira
c63cfc11e2 [Chore] Mark CodeQL warning as safe (#101365)
* Mark CodeQL warning as safe

* Temporarily ignore file
2025-04-07 11:52:09 +02:00
Ryan McKinley
9c49c601f1 Unistore: Add generation column (#102222) 2025-04-01 20:38:23 +03:00
Mustafa Sencer Özcan
a4e8bd16de fix: add index for resource history get with trash on (#102894) 2025-03-26 15:52:13 +01:00
Georges Chaudy
4927376f32 unistore: use the same connection string as grafana (#102387) 2025-03-19 13:56:54 +01:00
linoman
b7a0aeeb0d SCIM: Disable auto assign organization if the user has been provisioned (#101307)
* Add isProvisioned field to model

* Add new isProvisioned column to migration

* Disable auto assignment to organization if the user is provisioned

* add annotation to user model

* add annotation to user models

* Remove IsProvisioned field from Identity

* Move new field assignenment and add default value

* Update annotations for user query results

* Remove isProvisioned from identity

* Add new column to test

* Resolve user from identity at SyncOrgHook
2025-03-03 17:51:23 +01:00
Jean-Philippe Quéméner
c522a5b13b fix(unified-storage): proper setup TLS in new db_engine for MySQL (#100686) 2025-02-14 16:23:25 +01:00
Ryan McKinley
9d30911107 SQL/Storage: Use xorm code migration for DeletionMarker (#99353) 2025-01-22 14:07:08 +03:00
Ryan McKinley
429da7fd68 SQL/Blob: Add support for blob storage to SQL backend (#98192) 2025-01-08 22:08:10 +02:00
owensmallwood
9642eedce6 Unified Storage: Use tls preferred when grafana db using ssl (#97378)
When unified storage is using the grafana db (default case for on-prem), if the db is using ssl then set tls=preferred. This fixes an on-prem bug in 11.3.1.
2024-12-03 23:56:13 +02:00
Georges Chaudy
4f8ab73a8c unistore: Add support for verify-full in postgres (#96825)
Add support for verify-full in postgres
2024-11-21 12:31:33 +00:00
Georges Chaudy
f4e91da6b1 Unistore: fix fodler migration (#96299)
configure default migration for resource folder
2024-11-12 16:06:42 +02:00
Georges Chaudy
c3571752b6 Unitstore: Fix the Folder migration (#96078)
* UniStore: Add folder to the SQL backend

This reverts commit aaa4f4a23e.

* fix folder migration
2024-11-12 12:52:04 +01:00
Leonor Oliveira
aaa4f4a23e Revert "UniStore: Add folder to the SQL backend" (#95966)
Revert "UniStore: Add folder to the SQL backend (#95671)"

This reverts commit f075662696.
2024-11-06 17:04:57 +02:00
Georges Chaudy
f075662696 UniStore: Add folder to the SQL backend (#95671)
* Add folder to the SQL backend

* remove unused key

* fix tests

* Return folders from watch
2024-11-05 10:37:23 +01:00
Georges Chaudy
76b43267c8 Unified Storage: Use ssl_mode instead of sslmode (#95658)
change unistore to use ssl_mode instead of sslmode
2024-10-31 16:10:47 +01:00
Scott Lepper
337f2a0a8b [search] add index; increase page size (#95515)
[search] add index; increase page size
2024-10-28 15:48:49 -04:00
owensmallwood
2ffb88b0ee Unified Storage: Add index for poller query (#95293)
add index for poller query
2024-10-23 15:40:56 -06:00
Diego Augusto Molina
3e1f5559a6 Unistore Chore: Add database-level observability (#92266)
* add testing harness

* fix mockery and linters

* WIP

* wip

* fix transactions

* fix transaction tracing; add tracing by default

* rename package

* move WithTx to simplify logic of DB implementations

* fix potential issue with context deadline

* add db instrumentation to dbutil

* add otel tests

* improve naming

* minor fix in semantics and add comprehensive OTel testing

* fix naming

* instrument resourceVersionAtomicInc

* provide a default testing tracer

* fix docs

* fix typo in docs

* add semconv for k8s
2024-10-18 00:32:08 -03:00
Diego Augusto Molina
cf08f6762d Unistore Chore: Add OTEL testing harness (#94835)
* add testing harness

* fix mockery and linters

* fix data race in tests

* fix data race in tests

* reduce cardinality of data
2024-10-17 08:41:06 -03:00
Diego Augusto Molina
44282134da Unistore Chore: Make it easier to implement DB interface (#94680)
make it easier to implement DB interface
2024-10-16 13:40:35 -03:00
Prem Saraswat
5c03c14b25 resource-api: Loosen name validation to match K8s requirements (#93404)
* resource-api: Loosen name validation to match K8s requirements

This patch modifies some of the requirements for name validation of
objects in Resource API to match Kubernetes.

The limit we have on characters in name is 64, but some resources allow
upto 253 characters. Similarly we also include `:` in the regex, as many
objects in default K8s setup use it in the name (the group
`system:masters` for example)

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>

* Update the name column length in migrator and update e2e test to verify

---------

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
2024-10-10 11:03:18 -04:00
Diego Augusto Molina
4a800eda9f Unistore Chore: add inocuous preparative changes for otel-based db observability (#94473)
add inocuous preparative changes for otel tracing
2024-10-09 20:32:09 +00:00
Georges Chaudy
03f55e5288 Unistore : Ensure Watch works in HA mode (#94120)
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097)"

This reverts commit 7c3fc2f261.

* make previous_resource_version nullable

* handle nil case
2024-10-07 10:01:53 +02:00
Diego Augusto Molina
7d32d5eff4 Unistore: Reuse MySQL and Postgres Grafana core config instead of the object (#94223)
* Reuse MySQL and Postgres Grafana config instead of the object

- Only reuse the Grafana DB object for SQLite. Support for SQLite will be added in a different PR
- Fail when reusing the Grafana DB object if it is using DB instrumentation
- In the case that we have to reuse a Grafana DB with its instrumentation, fail with an error that describes a workaround
- Add regression tests to reproduce incident 2144

* remove temp file

* fix linter

* fix linter x2

* fix linter x3
2024-10-04 12:07:20 +00:00
Georges Chaudy
7c3fc2f261 Revert "Unistore : Ensure Watch works in HA mode." (#94097)
Revert "Unistore : Ensure Watch works in HA mode. (#93428)"

This reverts commit 0a26c9e9ae.
2024-10-01 18:45:47 +00:00
Georges Chaudy
0a26c9e9ae Unistore : Ensure Watch works in HA mode. (#93428)
* Replace Watch with WatchNext

* remove watchset

* fix previous page and closing the channel

* Remove the broadcaster cache to prevent dupplicated events

* add watch bookmark

* add watch bookmark

* cleanup comments

* disable the tests for bookmarks for now

* Ensure we send previosu events

* lint

* re-introduce the cache

* load from cache

* disabling legacy test

* disabling legacy test

* Update pkg/storage/unified/resource/server.go

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>

* Could not read previous events

* add proper migration

* Add previous_resource_version to both history and resource

* First event should have an RV of 2 and not 1

* Test both storage backends

* fix the inital RV for the sql backend

* ensure graceful stop of the stream decoder

* gocyclo

---------

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-09-30 13:14:07 +02:00
Ryan McKinley
c59dddf7af MySQL: Add parseTime=true to SQL connections (#92469) 2024-08-27 14:16:04 +03:00
Ryan McKinley
2136fd9a92 Storage: Remove unified storage feature flag (#92192)
remove unified storage flag
2024-08-21 19:28:30 +03:00
Diego Augusto Molina
9a65d0a977 Storage: remove sqlstore.WrapDatabaseDriverWithHooks (#92111)
remove sqlstore.WrapDatabaseDriverWithHooks from Unified Storage after related incident
2024-08-21 10:31:34 -03:00
owensmallwood
1cb3470e3e Unified Storage: Wrapping the mysql driver with hooks causes issues (#92099)
dont wrap the mysql driver with hooks. Its causing the error 'begin tx: sql: driver does not support non-default isolation level'.
2024-08-19 15:09:25 -06:00
owensmallwood
28f2a4078d Unified Storage: Records grafana database metrics (#91932)
* records grafana database metrics for unified storage

* update type to tracing.Tracer

* use nil for tracer in tests
2024-08-15 11:13:36 -06:00
Ryan McKinley
160fe2a3a4 Storage: Remove entity store (#91233) 2024-07-31 09:25:39 +03:00
owensmallwood
2362e79e4e Unified Storage: Configure client to use tls=preferred (#91162)
* optionally set tls=preferred in the mysql connection string from env var
2024-07-30 11:24:28 -06:00
Diego Augusto Molina
399d77a0fd Resource server improvements and fixes (#90715)
* cleanup dependencies and improve list method
* Improve Resource Server API, remove unnecessary dependencies
* Reduce the API footprint of ResourceDBInterface and its implementation
* Improve LifecycleHooks to use context
* Improve testing
* reduce API size and improve code
* sqltemplate: add DialectForDriver func and improve naming
* improve lifecycle API
* many small fixes after adding more tests
2024-07-22 20:08:30 +03:00
Georges Chaudy
08c611c68b ResourceServer: Resource store sql backend (#90170) 2024-07-18 08:03:18 -07:00