Compare commits

..

4 Commits

Author SHA1 Message Date
Will Assis
861af005e0 unified-storage: fix listModifiedSince in storage_backend.go and enable its tests for both badger and sqlkv 2026-01-14 19:24:15 -03:00
Will Assis
14a05137e1 fmt 2026-01-14 15:43:30 -03:00
Will Assis
cfe86378a1 dont run tests on sqlite (yet) 2026-01-14 15:41:51 -03:00
Will Assis
f7d7e09626 unified-storage: sqlkv enable more tests 2026-01-14 15:25:42 -03:00
26 changed files with 201 additions and 690 deletions

View File

@@ -4,8 +4,7 @@ comments: |
This file is used in the following visualizations: candlestick, heatmap, state timeline, status history, time series.
---
You can pan the panel time range left and right, and zoom it and in and out.
This, in turn, changes the dashboard time range.
You can zoom the panel time range in and out, which in turn, changes the dashboard time range.
**Zoom in** - Click and drag on the panel to zoom in on a particular time range.
@@ -17,9 +16,4 @@ For example, if the original time range is from 9:00 to 9:59, the time range cha
- Next range: 8:30 - 10:29
- Next range: 7:30 - 11:29
**Pan** - Click and drag the x-axis area of the panel to pan the time range.
The time range shifts by the distance you drag.
For example, if the original time range is from 9:00 to 9:59 and you drag 30 minutes to the right, the time range changes to 9:30 to 10:29.
For screen recordings showing these interactions, refer to the [Panel overview documentation](https://grafana.com/docs/grafana/<GRAFANA_VERSION>/visualizations/panels-visualizations/panel-overview/#pan-and-zoom-panel-time-range).
For screen recordings showing these interactions, refer to the [Panel overview documentation](https://grafana.com/docs/grafana/<GRAFANA_VERSION>/visualizations/panels-visualizations/panel-overview/#zoom-panel-time-range).

View File

@@ -72,8 +72,6 @@ Each panel needs at least one query to display a visualization.
## Create a dashboard
{{< docs/list >}}
{{< shared id="create-dashboard" >}}
To create a dashboard, follow these steps:
1. Click **Dashboards** in the main menu.

View File

@@ -317,16 +317,13 @@ Click the **Copy time range to clipboard** icon to copy the current time range t
You can also copy and paste a time range using the keyboard shortcuts `t+c` and `t+v` respectively.
#### Zoom out
#### Zoom out (Cmd+Z or Ctrl+Z)
- Click the **Zoom out** icon to view a larger time range in the dashboard or panel visualizations
- Double click on the panel graph area (time series family visualizations only)
- Type the `t-` keyboard shortcut
Click the **Zoom out** icon to view a larger time range in the dashboard or panel visualization.
#### Zoom in
#### Zoom in (only applicable to graph visualizations)
- Click and drag horizontally in the panel graph area to select a time range (time series family visualizations only)
- Type the `t+` keyboard shortcut
Click and drag to select the time range in the visualization that you want to view.
#### Refresh dashboard

View File

@@ -175,10 +175,9 @@ By hovering over a panel with the mouse you can use some shortcuts that will tar
- `pl`: Hide or show legend
- `pr`: Remove Panel
## Pan and zoom panel time range
## Zoom panel time range
You can pan the panel time range left and right, and zoom it and in and out.
This, in turn, changes the dashboard time range.
You can zoom the panel time range in and out, which in turn, changes the dashboard time range.
This feature is supported for the following visualizations:
@@ -192,7 +191,7 @@ This feature is supported for the following visualizations:
Click and drag on the panel to zoom in on a particular time range.
The following screen recordings show this interaction in the time series and candlestick visualizations:
The following screen recordings show this interaction in the time series and x visualizations:
Time series
@@ -212,7 +211,7 @@ For example, if the original time range is from 9:00 to 9:59, the time range cha
- Next range: 8:30 - 10:29
- Next range: 7:30 - 11:29
The following screen recordings demonstrate the preceding example in the time series and heatmap visualizations:
The following screen recordings demonstrate the preceding example in the time series and x visualizations:
Time series
@@ -222,19 +221,6 @@ Heatmap
{{< video-embed src="/media/docs/grafana/panels-visualizations/recording-heatmap-panel-time-zoom-out-mouse.mp4" >}}
### Pan
Click and drag the x-axis area of the panel to pan the time range.
The time range shifts by the distance you drag.
For example, if the original time range is from 9:00 to 9:59 and you drag 30 minutes to the right, the time range changes to 9:30 to 10:29.
The following screen recordings show this interaction in the time series visualization:
Time series
{{< video-embed src="/media/docs/grafana/panels-visualizations/recording-ts-time-pan-mouse.mp4" >}}
## Add a panel
To add a panel in a new dashboard click **+ Add visualization** in the middle of the dashboard:

View File

@@ -92,9 +92,9 @@ The data is converted as follows:
{{< figure src="/media/docs/grafana/panels-visualizations/screenshot-candles-volume-v11.6.png" max-width="750px" alt="A candlestick visualization showing the price movements of specific asset." >}}
## Pan and zoom panel time range
## Zoom panel time range
{{< docs/shared lookup="visualizations/panel-pan-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
{{< docs/shared lookup="visualizations/panel-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
## Configuration options

View File

@@ -79,9 +79,9 @@ The data is converted as follows:
{{< figure src="/static/img/docs/heatmap-panel/heatmap.png" max-width="1025px" alt="A heatmap visualization showing the random walk distribution over time" >}}
## Pan and zoom panel time range
## Zoom panel time range
{{< docs/shared lookup="visualizations/panel-pan-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
{{< docs/shared lookup="visualizations/panel-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
## Configuration options

View File

@@ -93,9 +93,9 @@ You can also create a state timeline visualization using time series data. To do
![State timeline with time series](/media/docs/grafana/panels-visualizations/screenshot-state-timeline-time-series-v11.4.png)
## Pan and zoom panel time range
## Zoom panel time range
{{< docs/shared lookup="visualizations/panel-pan-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
{{< docs/shared lookup="visualizations/panel-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
## Configuration options

View File

@@ -85,9 +85,9 @@ The data is converted as follows:
{{< figure src="/static/img/docs/status-history-panel/status_history.png" max-width="1025px" alt="A status history panel with two time columns showing the status of two servers" >}}
## Pan and zoom panel time range
## Zoom panel time range
{{< docs/shared lookup="visualizations/panel-pan-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
{{< docs/shared lookup="visualizations/panel-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
## Configuration options

View File

@@ -167,9 +167,9 @@ The following example shows three series: Min, Max, and Value. The Min and Max s
{{< docs/shared lookup="visualizations/multiple-y-axes.md" source="grafana" version="<GRAFANA_VERSION>" leveloffset="+2" >}}
## Pan and zoom panel time range
## Zoom panel time range
{{< docs/shared lookup="visualizations/panel-pan-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
{{< docs/shared lookup="visualizations/panel-zoom.md" source="grafana" version="<GRAFANA_VERSION>" >}}
## Configuration options

View File

@@ -76,27 +76,21 @@ func (hs *HTTPServer) CreateDashboardSnapshot(c *contextmodel.ReqContext) {
return
}
cfg := snapshot.SnapshotSharingOptions{
// Do not check permissions when the instance snapshot public mode is enabled
if !hs.Cfg.SnapshotPublicMode {
evaluator := ac.EvalAll(ac.EvalPermission(dashboards.ActionSnapshotsCreate), ac.EvalPermission(dashboards.ActionDashboardsRead, dashboards.ScopeDashboardsProvider.GetResourceScopeUID(cmd.Dashboard.GetNestedString("uid"))))
if canSave, err := hs.AccessControl.Evaluate(c.Req.Context(), c.SignedInUser, evaluator); err != nil || !canSave {
c.JsonApiErr(http.StatusForbidden, "forbidden", err)
return
}
}
dashboardsnapshots.CreateDashboardSnapshot(c, snapshot.SnapshotSharingOptions{
SnapshotsEnabled: hs.Cfg.SnapshotEnabled,
ExternalEnabled: hs.Cfg.ExternalEnabled,
ExternalSnapshotName: hs.Cfg.ExternalSnapshotName,
ExternalSnapshotURL: hs.Cfg.ExternalSnapshotUrl,
}
if hs.Cfg.SnapshotPublicMode {
// Public mode: no user or dashboard validation needed
dashboardsnapshots.CreateDashboardSnapshotPublic(c, cfg, cmd, hs.dashboardsnapshotsService)
return
}
// Regular mode: check permissions
evaluator := ac.EvalAll(ac.EvalPermission(dashboards.ActionSnapshotsCreate), ac.EvalPermission(dashboards.ActionDashboardsRead, dashboards.ScopeDashboardsProvider.GetResourceScopeUID(cmd.Dashboard.GetNestedString("uid"))))
if canSave, err := hs.AccessControl.Evaluate(c.Req.Context(), c.SignedInUser, evaluator); err != nil || !canSave {
c.JsonApiErr(http.StatusForbidden, "forbidden", err)
return
}
dashboardsnapshots.CreateDashboardSnapshot(c, cfg, cmd, hs.dashboardsnapshotsService)
}, cmd, hs.dashboardsnapshotsService)
}
// GET /api/snapshots/:key
@@ -219,6 +213,13 @@ func (hs *HTTPServer) DeleteDashboardSnapshot(c *contextmodel.ReqContext) respon
return response.Error(http.StatusUnauthorized, "OrgID mismatch", nil)
}
if queryResult.External {
err := dashboardsnapshots.DeleteExternalDashboardSnapshot(queryResult.ExternalDeleteURL)
if err != nil {
return response.Error(http.StatusInternalServerError, "Failed to delete external dashboard", err)
}
}
// Dashboard can be empty (creation error or external snapshot). This means that the mustInt here returns a 0,
// which before RBAC would result in a dashboard which has no ACL. A dashboard without an ACL would fallback
// to the users org role, which for editors and admins would essentially always be allowed here. With RBAC,
@@ -238,13 +239,6 @@ func (hs *HTTPServer) DeleteDashboardSnapshot(c *contextmodel.ReqContext) respon
}
}
if queryResult.External {
err := dashboardsnapshots.DeleteExternalDashboardSnapshot(queryResult.ExternalDeleteURL)
if err != nil {
return response.Error(http.StatusInternalServerError, "Failed to delete external dashboard", err)
}
}
cmd := &dashboardsnapshots.DeleteDashboardSnapshotCommand{DeleteKey: queryResult.DeleteKey}
if err := hs.dashboardsnapshotsService.DeleteDashboardSnapshot(c.Req.Context(), cmd); err != nil {

View File

@@ -32,8 +32,6 @@ import (
var (
logger = glog.New("data-proxy-log")
client = newHTTPClient()
errPluginProxyRouteAccessDenied = errors.New("plugin proxy route access denied")
)
type DataSourceProxy struct {
@@ -310,21 +308,12 @@ func (proxy *DataSourceProxy) validateRequest() error {
if err != nil {
return err
}
// issues/116273: When we have an empty input route (or input that becomes relative to "."), we do not want it
// to be ".". This is because the `CleanRelativePath` function will never return "./" prefixes, and as such,
// the common prefix we need is an empty string.
if r1 == "." && proxy.proxyPath != "." {
r1 = ""
}
if r2 == "." && route.Path != "." {
r2 = ""
}
if !strings.HasPrefix(r1, r2) {
continue
}
if !proxy.hasAccessToRoute(route) {
return errPluginProxyRouteAccessDenied
return errors.New("plugin proxy route access denied")
}
proxy.matchedRoute = route

View File

@@ -673,94 +673,6 @@ func TestIntegrationDataSourceProxy_routeRule(t *testing.T) {
runDatasourceAuthTest(t, secretsService, secretsStore, cfg, test)
}
})
t.Run("Regression of 116273: Fallback routes should apply fallback route roles", func(t *testing.T) {
for _, tc := range []struct {
InputPath string
ConfigurationPath string
ExpectError bool
}{
{
InputPath: "api/v2/leak-ur-secrets",
ConfigurationPath: "",
ExpectError: true,
},
{
InputPath: "",
ConfigurationPath: "",
ExpectError: true,
},
{
InputPath: ".",
ConfigurationPath: ".",
ExpectError: true,
},
{
InputPath: "",
ConfigurationPath: ".",
ExpectError: false,
},
{
InputPath: "api",
ConfigurationPath: ".",
ExpectError: false,
},
} {
orEmptyStr := func(s string) string {
if s == "" {
return "<empty>"
}
return s
}
t.Run(
fmt.Sprintf("with inputPath=%s, configurationPath=%s, expectError=%v",
orEmptyStr(tc.InputPath), orEmptyStr(tc.ConfigurationPath), tc.ExpectError),
func(t *testing.T) {
ds := &datasources.DataSource{
UID: "dsUID",
JsonData: simplejson.New(),
}
routes := []*plugins.Route{
{
Path: tc.ConfigurationPath,
ReqRole: org.RoleAdmin,
Method: "GET",
},
{
Path: tc.ConfigurationPath,
ReqRole: org.RoleAdmin,
Method: "POST",
},
{
Path: tc.ConfigurationPath,
ReqRole: org.RoleAdmin,
Method: "PUT",
},
{
Path: tc.ConfigurationPath,
ReqRole: org.RoleAdmin,
Method: "DELETE",
},
}
req, err := http.NewRequestWithContext(t.Context(), "GET", "http://localhost/"+tc.InputPath, nil)
require.NoError(t, err, "failed to create HTTP request")
ctx := &contextmodel.ReqContext{
Context: &web.Context{Req: req},
SignedInUser: &user.SignedInUser{OrgRole: org.RoleViewer},
}
proxy, err := setupDSProxyTest(t, ctx, ds, routes, tc.InputPath)
require.NoError(t, err, "failed to setup proxy test")
err = proxy.validateRequest()
if tc.ExpectError {
require.ErrorIs(t, err, errPluginProxyRouteAccessDenied, "request was not denied due to access denied?")
} else {
require.NoError(t, err, "request was unexpectedly denied access")
}
},
)
}
})
}
// test DataSourceProxy request handling.

View File

@@ -36,9 +36,6 @@ var client = &http.Client{
Transport: &http.Transport{Proxy: http.ProxyFromEnvironment},
}
// CreateDashboardSnapshot creates a snapshot when running Grafana in regular mode.
// It validates the user and dashboard exist before creating the snapshot.
// This mode supports both local and external snapshots.
func CreateDashboardSnapshot(c *contextmodel.ReqContext, cfg snapshot.SnapshotSharingOptions, cmd CreateDashboardSnapshotCommand, svc Service) {
if !cfg.SnapshotsEnabled {
c.JsonApiErr(http.StatusForbidden, "Dashboard Snapshots are disabled", nil)
@@ -46,7 +43,6 @@ func CreateDashboardSnapshot(c *contextmodel.ReqContext, cfg snapshot.SnapshotSh
}
uid := cmd.Dashboard.GetNestedString("uid")
user, err := identity.GetRequester(c.Req.Context())
if err != nil {
c.JsonApiErr(http.StatusBadRequest, "missing user in context", nil)
@@ -63,18 +59,21 @@ func CreateDashboardSnapshot(c *contextmodel.ReqContext, cfg snapshot.SnapshotSh
return
}
cmd.ExternalURL = ""
cmd.OrgID = user.GetOrgID()
cmd.UserID, _ = identity.UserIdentifier(user.GetID())
if cmd.Name == "" {
cmd.Name = "Unnamed snapshot"
}
var snapshotURL string
var snapshotUrl string
cmd.ExternalURL = ""
cmd.OrgID = user.GetOrgID()
cmd.UserID, _ = identity.UserIdentifier(user.GetID())
originalDashboardURL, err := createOriginalDashboardURL(&cmd)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Invalid app URL", err)
return
}
if cmd.External {
// Handle external snapshot creation
if !cfg.ExternalEnabled {
c.JsonApiErr(http.StatusForbidden, "External dashboard creation is disabled", nil)
return
@@ -86,83 +85,40 @@ func CreateDashboardSnapshot(c *contextmodel.ReqContext, cfg snapshot.SnapshotSh
return
}
snapshotUrl = resp.Url
cmd.Key = resp.Key
cmd.DeleteKey = resp.DeleteKey
cmd.ExternalURL = resp.Url
cmd.ExternalDeleteURL = resp.DeleteUrl
cmd.Dashboard = &common.Unstructured{}
snapshotURL = resp.Url
metrics.MApiDashboardSnapshotExternal.Inc()
} else {
// Handle local snapshot creation
originalDashboardURL, err := createOriginalDashboardURL(&cmd)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Invalid app URL", err)
return
cmd.Dashboard.SetNestedField(originalDashboardURL, "snapshot", "originalUrl")
if cmd.Key == "" {
var err error
cmd.Key, err = util.GetRandomString(32)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Could not generate random string", err)
return
}
}
snapshotURL, err = prepareLocalSnapshot(&cmd, originalDashboardURL)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Could not generate random string", err)
return
if cmd.DeleteKey == "" {
var err error
cmd.DeleteKey, err = util.GetRandomString(32)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Could not generate random string", err)
return
}
}
snapshotUrl = setting.ToAbsUrl("dashboard/snapshot/" + cmd.Key)
metrics.MApiDashboardSnapshotCreate.Inc()
}
saveAndRespond(c, svc, cmd, snapshotURL)
}
// CreateDashboardSnapshotPublic creates a snapshot when running Grafana in public mode.
// In public mode, there is no user or dashboard information to validate.
// Only local snapshots are supported (external snapshots are not available).
func CreateDashboardSnapshotPublic(c *contextmodel.ReqContext, cfg snapshot.SnapshotSharingOptions, cmd CreateDashboardSnapshotCommand, svc Service) {
if !cfg.SnapshotsEnabled {
c.JsonApiErr(http.StatusForbidden, "Dashboard Snapshots are disabled", nil)
return
}
if cmd.Name == "" {
cmd.Name = "Unnamed snapshot"
}
snapshotURL, err := prepareLocalSnapshot(&cmd, "")
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Could not generate random string", err)
return
}
metrics.MApiDashboardSnapshotCreate.Inc()
saveAndRespond(c, svc, cmd, snapshotURL)
}
// prepareLocalSnapshot prepares the command for a local snapshot and returns the snapshot URL.
func prepareLocalSnapshot(cmd *CreateDashboardSnapshotCommand, originalDashboardURL string) (string, error) {
cmd.Dashboard.SetNestedField(originalDashboardURL, "snapshot", "originalUrl")
if cmd.Key == "" {
key, err := util.GetRandomString(32)
if err != nil {
return "", err
}
cmd.Key = key
}
if cmd.DeleteKey == "" {
deleteKey, err := util.GetRandomString(32)
if err != nil {
return "", err
}
cmd.DeleteKey = deleteKey
}
return setting.ToAbsUrl("dashboard/snapshot/" + cmd.Key), nil
}
// saveAndRespond saves the snapshot and sends the response.
func saveAndRespond(c *contextmodel.ReqContext, svc Service, cmd CreateDashboardSnapshotCommand, snapshotURL string) {
result, err := svc.CreateDashboardSnapshot(c.Req.Context(), &cmd)
if err != nil {
c.JsonApiErr(http.StatusInternalServerError, "Failed to create snapshot", err)
@@ -172,7 +128,7 @@ func saveAndRespond(c *contextmodel.ReqContext, svc Service, cmd CreateDashboard
c.JSON(http.StatusOK, snapshot.DashboardCreateResponse{
Key: result.Key,
DeleteKey: result.DeleteKey,
URL: snapshotURL,
URL: snapshotUrl,
DeleteURL: setting.ToAbsUrl("api/snapshots-delete/" + result.DeleteKey),
})
}

View File

@@ -20,30 +20,40 @@ import (
"github.com/grafana/grafana/pkg/web"
)
func createTestDashboard(t *testing.T) *common.Unstructured {
t.Helper()
dashboard := &common.Unstructured{}
dashboardData := map[string]any{
"uid": "test-dashboard-uid",
"id": 123,
func TestCreateDashboardSnapshot_DashboardNotFound(t *testing.T) {
mockService := &MockService{}
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
ExternalEnabled: false,
}
dashboardBytes, _ := json.Marshal(dashboardData)
_ = json.Unmarshal(dashboardBytes, dashboard)
return dashboard
}
func createTestUser() *user.SignedInUser {
return &user.SignedInUser{
testUser := &user.SignedInUser{
UserID: 1,
OrgID: 1,
Login: "testuser",
Name: "Test User",
Email: "test@example.com",
}
}
dashboard := &common.Unstructured{}
dashboardData := map[string]interface{}{
"uid": "test-dashboard-uid",
"id": 123,
}
dashboardBytes, _ := json.Marshal(dashboardData)
_ = json.Unmarshal(dashboardBytes, dashboard)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test Snapshot",
},
}
mockService.On("ValidateDashboardExists", mock.Anything, int64(1), "test-dashboard-uid").
Return(dashboards.ErrDashboardNotFound)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
req = req.WithContext(identity.WithRequester(req.Context(), testUser))
func createReqContext(t *testing.T, req *http.Request, testUser *user.SignedInUser) (*contextmodel.ReqContext, *httptest.ResponseRecorder) {
t.Helper()
recorder := httptest.NewRecorder()
ctx := &contextmodel.ReqContext{
Context: &web.Context{
@@ -53,319 +63,13 @@ func createReqContext(t *testing.T, req *http.Request, testUser *user.SignedInUs
SignedInUser: testUser,
Logger: log.NewNopLogger(),
}
return ctx, recorder
}
// TestCreateDashboardSnapshot tests snapshot creation in regular mode (non-public instance).
// These tests cover scenarios when Grafana is running as a regular server with user authentication.
func TestCreateDashboardSnapshot(t *testing.T) {
t.Run("should return error when dashboard not found", func(t *testing.T) {
mockService := &MockService{}
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
ExternalEnabled: false,
}
testUser := createTestUser()
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test Snapshot",
},
}
mockService.On("ValidateDashboardExists", mock.Anything, int64(1), "test-dashboard-uid").
Return(dashboards.ErrDashboardNotFound)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
req = req.WithContext(identity.WithRequester(req.Context(), testUser))
ctx, recorder := createReqContext(t, req, testUser)
CreateDashboardSnapshot(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusBadRequest, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "Dashboard not found", response["message"])
})
t.Run("should create external snapshot when external is enabled", func(t *testing.T) {
externalServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, "/api/snapshots", r.URL.Path)
assert.Equal(t, "POST", r.Method)
response := map[string]any{
"key": "external-key",
"deleteKey": "external-delete-key",
"url": "https://external.example.com/dashboard/snapshot/external-key",
"deleteUrl": "https://external.example.com/api/snapshots-delete/external-delete-key",
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(response)
}))
defer externalServer.Close()
mockService := NewMockService(t)
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
ExternalEnabled: true,
ExternalSnapshotURL: externalServer.URL,
}
testUser := createTestUser()
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test External Snapshot",
External: true,
},
}
mockService.On("ValidateDashboardExists", mock.Anything, int64(1), "test-dashboard-uid").
Return(nil)
mockService.On("CreateDashboardSnapshot", mock.Anything, mock.Anything).
Return(&DashboardSnapshot{
Key: "external-key",
DeleteKey: "external-delete-key",
}, nil)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
req = req.WithContext(identity.WithRequester(req.Context(), testUser))
ctx, recorder := createReqContext(t, req, testUser)
CreateDashboardSnapshot(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusOK, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "external-key", response["key"])
assert.Equal(t, "external-delete-key", response["deleteKey"])
assert.Equal(t, "https://external.example.com/dashboard/snapshot/external-key", response["url"])
})
t.Run("should return forbidden when external is disabled", func(t *testing.T) {
mockService := NewMockService(t)
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
ExternalEnabled: false,
}
testUser := createTestUser()
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test External Snapshot",
External: true,
},
}
mockService.On("ValidateDashboardExists", mock.Anything, int64(1), "test-dashboard-uid").
Return(nil)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
req = req.WithContext(identity.WithRequester(req.Context(), testUser))
ctx, recorder := createReqContext(t, req, testUser)
CreateDashboardSnapshot(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusForbidden, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "External dashboard creation is disabled", response["message"])
})
t.Run("should create local snapshot", func(t *testing.T) {
mockService := NewMockService(t)
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
}
testUser := createTestUser()
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test Local Snapshot",
},
Key: "local-key",
DeleteKey: "local-delete-key",
}
mockService.On("ValidateDashboardExists", mock.Anything, int64(1), "test-dashboard-uid").
Return(nil)
mockService.On("CreateDashboardSnapshot", mock.Anything, mock.Anything).
Return(&DashboardSnapshot{
Key: "local-key",
DeleteKey: "local-delete-key",
}, nil)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
req = req.WithContext(identity.WithRequester(req.Context(), testUser))
ctx, recorder := createReqContext(t, req, testUser)
CreateDashboardSnapshot(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusOK, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "local-key", response["key"])
assert.Equal(t, "local-delete-key", response["deleteKey"])
assert.Contains(t, response["url"], "dashboard/snapshot/local-key")
assert.Contains(t, response["deleteUrl"], "api/snapshots-delete/local-delete-key")
})
}
// TestCreateDashboardSnapshotPublic tests snapshot creation in public mode.
// These tests cover scenarios when Grafana is running as a public snapshot server
// where no user authentication or dashboard validation is required.
func TestCreateDashboardSnapshotPublic(t *testing.T) {
t.Run("should create local snapshot without user context", func(t *testing.T) {
mockService := NewMockService(t)
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: true,
}
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test Snapshot",
},
Key: "test-key",
DeleteKey: "test-delete-key",
}
mockService.On("CreateDashboardSnapshot", mock.Anything, mock.Anything).
Return(&DashboardSnapshot{
Key: "test-key",
DeleteKey: "test-delete-key",
}, nil)
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
recorder := httptest.NewRecorder()
ctx := &contextmodel.ReqContext{
Context: &web.Context{
Req: req,
Resp: web.NewResponseWriter("POST", recorder),
},
Logger: log.NewNopLogger(),
}
CreateDashboardSnapshotPublic(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusOK, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "test-key", response["key"])
assert.Equal(t, "test-delete-key", response["deleteKey"])
assert.Contains(t, response["url"], "dashboard/snapshot/test-key")
assert.Contains(t, response["deleteUrl"], "api/snapshots-delete/test-delete-key")
})
t.Run("should return forbidden when snapshots are disabled", func(t *testing.T) {
mockService := NewMockService(t)
cfg := snapshot.SnapshotSharingOptions{
SnapshotsEnabled: false,
}
dashboard := createTestDashboard(t)
cmd := CreateDashboardSnapshotCommand{
DashboardCreateCommand: snapshot.DashboardCreateCommand{
Dashboard: dashboard,
Name: "Test Snapshot",
},
}
req, _ := http.NewRequest("POST", "/api/snapshots", nil)
recorder := httptest.NewRecorder()
ctx := &contextmodel.ReqContext{
Context: &web.Context{
Req: req,
Resp: web.NewResponseWriter("POST", recorder),
},
Logger: log.NewNopLogger(),
}
CreateDashboardSnapshotPublic(ctx, cfg, cmd, mockService)
assert.Equal(t, http.StatusForbidden, recorder.Code)
var response map[string]any
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "Dashboard Snapshots are disabled", response["message"])
})
}
// TestDeleteExternalDashboardSnapshot tests deletion of external snapshots.
// This function is called in public mode and doesn't require user context.
func TestDeleteExternalDashboardSnapshot(t *testing.T) {
t.Run("should return nil on successful deletion", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, "GET", r.Method)
w.WriteHeader(http.StatusOK)
}))
defer server.Close()
err := DeleteExternalDashboardSnapshot(server.URL)
assert.NoError(t, err)
})
t.Run("should gracefully handle already deleted snapshot", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
response := map[string]any{
"message": "Failed to get dashboard snapshot",
}
_ = json.NewEncoder(w).Encode(response)
}))
defer server.Close()
err := DeleteExternalDashboardSnapshot(server.URL)
assert.NoError(t, err)
})
t.Run("should return error on unexpected status code", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer server.Close()
err := DeleteExternalDashboardSnapshot(server.URL)
assert.Error(t, err)
assert.Contains(t, err.Error(), "unexpected response when deleting external snapshot")
assert.Contains(t, err.Error(), "404")
})
t.Run("should return error on 500 with different message", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
response := map[string]any{
"message": "Some other error",
}
_ = json.NewEncoder(w).Encode(response)
}))
defer server.Close()
err := DeleteExternalDashboardSnapshot(server.URL)
assert.Error(t, err)
assert.Contains(t, err.Error(), "500")
})
CreateDashboardSnapshot(ctx, cfg, cmd, mockService)
mockService.AssertExpectations(t)
assert.Equal(t, http.StatusBadRequest, recorder.Code)
var response map[string]interface{}
err := json.Unmarshal(recorder.Body.Bytes(), &response)
require.NoError(t, err)
assert.Equal(t, "Dashboard not found", response["message"])
}

View File

@@ -14,7 +14,6 @@ import (
"github.com/grafana/grafana/pkg/apimachinery/validation"
"github.com/grafana/grafana/pkg/storage/unified/sql/db"
"github.com/grafana/grafana/pkg/storage/unified/sql/dbutil"
"github.com/grafana/grafana/pkg/storage/unified/sql/rvmanager"
"github.com/grafana/grafana/pkg/storage/unified/sql/sqltemplate"
gocache "github.com/patrickmn/go-cache"
)
@@ -869,18 +868,10 @@ func (d *dataStore) applyBackwardsCompatibleChanges(ctx context.Context, tx db.T
if key.Action == DataActionDeleted {
generation = 0
}
// In compatibility mode, the previous RV, when available, is saved as a microsecond
// timestamp, as is done in the SQL backend.
previousRV := event.PreviousRV
if event.PreviousRV > 0 && isSnowflake(event.PreviousRV) {
previousRV = rvmanager.RVFromSnowflake(event.PreviousRV)
}
_, err := dbutil.Exec(ctx, tx, sqlKVUpdateLegacyResourceHistory, sqlKVLegacyUpdateHistoryRequest{
SQLTemplate: sqltemplate.New(kv.dialect),
GUID: key.GUID,
PreviousRV: previousRV,
PreviousRV: event.PreviousRV,
Generation: generation,
})
@@ -909,7 +900,7 @@ func (d *dataStore) applyBackwardsCompatibleChanges(ctx context.Context, tx db.T
Name: key.Name,
Action: action,
Folder: key.Folder,
PreviousRV: previousRV,
PreviousRV: event.PreviousRV,
})
if err != nil {
@@ -925,7 +916,7 @@ func (d *dataStore) applyBackwardsCompatibleChanges(ctx context.Context, tx db.T
Name: key.Name,
Action: action,
Folder: key.Folder,
PreviousRV: previousRV,
PreviousRV: event.PreviousRV,
})
if err != nil {
@@ -947,15 +938,3 @@ func (d *dataStore) applyBackwardsCompatibleChanges(ctx context.Context, tx db.T
return nil
}
// isSnowflake returns whether the argument passed is a snowflake ID (new) or a microsecond timestamp (old).
// We try to interpret the number as a microsecond timestamp first. If it represents a time in the past,
// it is considered a microsecond timestamp. Snowflake IDs are much larger integers and would lead
// to dates in the future if interpreted as a microsecond timestamp.
func isSnowflake(rv int64) bool {
ts := time.UnixMicro(rv)
oneHourFromNow := time.Now().Add(time.Hour)
isMicroSecRV := ts.Before(oneHourFromNow)
return !isMicroSecRV
}

View File

@@ -184,9 +184,9 @@ func (n *eventStore) Get(ctx context.Context, key EventKey) (Event, error) {
}
// ListSince returns a sequence of events since the given resource version.
func (n *eventStore) ListKeysSince(ctx context.Context, sinceRV int64) iter.Seq2[string, error] {
func (n *eventStore) ListKeysSince(ctx context.Context, sinceRV int64, sortOrder SortOrder) iter.Seq2[string, error] {
opts := ListOptions{
Sort: SortOrderAsc,
Sort: sortOrder,
StartKey: fmt.Sprintf("%d", sinceRV),
}
return func(yield func(string, error) bool) {
@@ -202,9 +202,9 @@ func (n *eventStore) ListKeysSince(ctx context.Context, sinceRV int64) iter.Seq2
}
}
func (n *eventStore) ListSince(ctx context.Context, sinceRV int64) iter.Seq2[Event, error] {
func (n *eventStore) ListSince(ctx context.Context, sinceRV int64, sortOrder SortOrder) iter.Seq2[Event, error] {
return func(yield func(Event, error) bool) {
for evtKey, err := range n.ListKeysSince(ctx, sinceRV) {
for evtKey, err := range n.ListKeysSince(ctx, sinceRV, sortOrder) {
if err != nil {
yield(Event{}, err)
return

View File

@@ -369,7 +369,7 @@ func testEventStoreListKeysSince(t *testing.T, ctx context.Context, store *event
// List events since RV 1500 (should get events with RV 2000 and 3000)
retrievedEvents := make([]string, 0, 2)
for eventKey, err := range store.ListKeysSince(ctx, 1500) {
for eventKey, err := range store.ListKeysSince(ctx, 1500, SortOrderAsc) {
require.NoError(t, err)
retrievedEvents = append(retrievedEvents, eventKey)
}
@@ -429,7 +429,7 @@ func testEventStoreListSince(t *testing.T, ctx context.Context, store *eventStor
// List events since RV 1500 (should get events with RV 2000 and 3000)
retrievedEvents := make([]Event, 0, 2)
for event, err := range store.ListSince(ctx, 1500) {
for event, err := range store.ListSince(ctx, 1500, SortOrderAsc) {
require.NoError(t, err)
retrievedEvents = append(retrievedEvents, event)
}
@@ -453,7 +453,7 @@ func TestEventStore_ListSince_Empty(t *testing.T) {
func testEventStoreListSinceEmpty(t *testing.T, ctx context.Context, store *eventStore) {
// List events when store is empty
retrievedEvents := make([]Event, 0)
for event, err := range store.ListSince(ctx, 0) {
for event, err := range store.ListSince(ctx, 0, SortOrderAsc) {
require.NoError(t, err)
retrievedEvents = append(retrievedEvents, event)
}
@@ -825,7 +825,7 @@ func testListKeysSinceWithSnowflakeTime(t *testing.T, ctx context.Context, store
// List events since 90 minutes ago using subtractDurationFromSnowflake
sinceRV := subtractDurationFromSnowflake(snowflakeFromTime(now), 90*time.Minute)
retrievedEvents := make([]string, 0)
for eventKey, err := range store.ListKeysSince(ctx, sinceRV) {
for eventKey, err := range store.ListKeysSince(ctx, sinceRV, SortOrderAsc) {
require.NoError(t, err)
retrievedEvents = append(retrievedEvents, eventKey)
}
@@ -842,7 +842,7 @@ func testListKeysSinceWithSnowflakeTime(t *testing.T, ctx context.Context, store
// List events since 30 minutes ago using subtractDurationFromSnowflake
sinceRV = subtractDurationFromSnowflake(snowflakeFromTime(now), 30*time.Minute)
retrievedEvents = make([]string, 0)
for eventKey, err := range store.ListKeysSince(ctx, sinceRV) {
for eventKey, err := range store.ListKeysSince(ctx, sinceRV, SortOrderAsc) {
require.NoError(t, err)
retrievedEvents = append(retrievedEvents, eventKey)
}

View File

@@ -119,7 +119,7 @@ func (n *pollingNotifier) Watch(ctx context.Context, opts watchOptions) <-chan E
return
case <-time.After(currentInterval):
foundEvents := false
for evt, err := range n.eventStore.ListSince(ctx, subtractDurationFromSnowflake(lastRV, opts.LookbackPeriod)) {
for evt, err := range n.eventStore.ListSince(ctx, subtractDurationFromSnowflake(lastRV, opts.LookbackPeriod), SortOrderAsc) {
if err != nil {
n.log.Error("Failed to list events since", "error", err)
continue

View File

@@ -456,27 +456,33 @@ func testNotifierWatchMultipleEvents(t *testing.T, ctx context.Context, notifier
},
}
errCh := make(chan error)
go func() {
for _, event := range testEvents {
errCh <- eventStore.Save(ctx, event)
err := eventStore.Save(ctx, event)
require.NoError(t, err)
}
}()
// Receive events
receivedEvents := make([]string, 0, len(testEvents))
for len(receivedEvents) != len(testEvents) {
receivedEvents := make([]Event, 0, len(testEvents))
for i := 0; i < len(testEvents); i++ {
select {
case event := <-events:
receivedEvents = append(receivedEvents, event.Name)
case err := <-errCh:
require.NoError(t, err)
receivedEvents = append(receivedEvents, event)
case <-time.After(1 * time.Second):
t.Fatalf("Timed out waiting for event %d", len(receivedEvents)+1)
t.Fatalf("Timed out waiting for event %d", i+1)
}
}
// Verify all events were received
assert.Len(t, receivedEvents, len(testEvents))
// Verify the events match and ordered by resource version
receivedNames := make([]string, len(receivedEvents))
for i, event := range receivedEvents {
receivedNames[i] = event.Name
}
expectedNames := []string{"test-resource-1", "test-resource-2", "test-resource-3"}
assert.ElementsMatch(t, expectedNames, receivedEvents)
assert.ElementsMatch(t, expectedNames, receivedNames)
}

View File

@@ -473,6 +473,8 @@ func (k *sqlKV) Delete(ctx context.Context, section string, key string) error {
return ErrNotFound
}
// TODO reflect change to resource table
return nil
}

View File

@@ -347,7 +347,7 @@ func (k *kvStorageBackend) WriteEvent(ctx context.Context, event WriteEvent) (in
return 0, fmt.Errorf("failed to write data: %w", err)
}
rv = rvmanager.SnowflakeFromRV(rv)
rv = rvmanager.SnowflakeFromRv(rv)
dataKey.ResourceVersion = rv
} else {
err := k.dataStore.Save(ctx, dataKey, bytes.NewReader(event.Value))
@@ -801,8 +801,20 @@ func (k *kvStorageBackend) ListModifiedSince(ctx context.Context, key Namespaced
}
}
// Generate a new resource version for the list
listRV := k.snowflake.Generate().Int64()
latestEvent, err := k.eventStore.LastEventKey(ctx)
if err != nil {
if errors.Is(err, ErrNotFound) {
return sinceRv, func(yield func(*ModifiedResource, error) bool) { /* nothing to return */ }
}
return 0, func(yield func(*ModifiedResource, error) bool) {
yield(nil, fmt.Errorf("error trying to retrieve last event key: %s", err))
}
}
if latestEvent.ResourceVersion == sinceRv {
return sinceRv, func(yield func(*ModifiedResource, error) bool) { /* nothing to return */ }
}
// Check if sinceRv is older than 1 hour
sinceRvTimestamp := snowflake.ID(sinceRv).Time()
@@ -811,11 +823,11 @@ func (k *kvStorageBackend) ListModifiedSince(ctx context.Context, key Namespaced
if sinceRvAge > time.Hour {
k.log.Debug("ListModifiedSince using data store", "sinceRv", sinceRv, "sinceRvAge", sinceRvAge)
return listRV, k.listModifiedSinceDataStore(ctx, key, sinceRv)
return latestEvent.ResourceVersion, k.listModifiedSinceDataStore(ctx, key, sinceRv)
}
k.log.Debug("ListModifiedSince using event store", "sinceRv", sinceRv, "sinceRvAge", sinceRvAge)
return listRV, k.listModifiedSinceEventStore(ctx, key, sinceRv)
return latestEvent.ResourceVersion, k.listModifiedSinceEventStore(ctx, key, sinceRv)
}
func convertEventType(action DataAction) resourcepb.WatchEvent_Type {
@@ -916,9 +928,9 @@ func (k *kvStorageBackend) listModifiedSinceDataStore(ctx context.Context, key N
func (k *kvStorageBackend) listModifiedSinceEventStore(ctx context.Context, key NamespacedResource, sinceRv int64) iter.Seq2[*ModifiedResource, error] {
return func(yield func(*ModifiedResource, error) bool) {
// store all events ordered by RV for the given tenant here
eventKeys := make([]EventKey, 0)
for evtKeyStr, err := range k.eventStore.ListKeysSince(ctx, subtractDurationFromSnowflake(sinceRv, defaultLookbackPeriod)) {
// we only care about the latest revision of every resource in the list
seen := make(map[string]struct{})
for evtKeyStr, err := range k.eventStore.ListKeysSince(ctx, subtractDurationFromSnowflake(sinceRv, defaultLookbackPeriod), SortOrderDesc) {
if err != nil {
yield(&ModifiedResource{}, err)
return
@@ -938,18 +950,11 @@ func (k *kvStorageBackend) listModifiedSinceEventStore(ctx context.Context, key
continue
}
eventKeys = append(eventKeys, evtKey)
}
// we only care about the latest revision of every resource in the list
seen := make(map[string]struct{})
for i := len(eventKeys) - 1; i >= 0; i -= 1 {
evtKey := eventKeys[i]
if _, ok := seen[evtKey.Name]; ok {
continue
}
seen[evtKey.Name] = struct{}{}
seen[evtKey.Name] = struct{}{}
value, err := k.getValueFromDataStore(ctx, DataKey(evtKey))
if err != nil {
yield(&ModifiedResource{}, err)
@@ -1307,7 +1312,7 @@ func (b *kvStorageBackend) ProcessBulk(ctx context.Context, setting BulkSettings
if setting.RebuildCollection {
for _, key := range setting.Collection {
events := make([]string, 0)
for evtKeyStr, err := range b.eventStore.ListKeysSince(ctx, 1) {
for evtKeyStr, err := range b.eventStore.ListKeysSince(ctx, 1, SortOrderAsc) {
if err != nil {
b.log.Error("failed to list event: %s", err)
return rsp

View File

@@ -307,7 +307,7 @@ func (m *ResourceVersionManager) execBatch(ctx context.Context, group, resource
// Allocate the RVs
for i, guid := range guids {
guidToRV[guid] = rv
guidToSnowflakeRV[guid] = SnowflakeFromRV(rv)
guidToSnowflakeRV[guid] = SnowflakeFromRv(rv)
rvs[i] = rv
rv++
}
@@ -364,20 +364,12 @@ func (m *ResourceVersionManager) execBatch(ctx context.Context, group, resource
}
}
// takes a unix microsecond RV and transforms into a snowflake format. The timestamp is converted from microsecond to
// takes a unix microsecond rv and transforms into a snowflake format. The timestamp is converted from microsecond to
// millisecond (the integer division) and the remainder is saved in the stepbits section. machine id is always 0
func SnowflakeFromRV(rv int64) int64 {
func SnowflakeFromRv(rv int64) int64 {
return (((rv / 1000) - snowflake.Epoch) << (snowflake.NodeBits + snowflake.StepBits)) + (rv % 1000)
}
// It is generally not possible to convert from a snowflakeID to a microsecond RV due to the loss in precision
// (snowflake ID stores timestamp in milliseconds). However, this implementation stores the microsecond fraction
// in the step bits (see SnowflakeFromRV), allowing us to compute the microsecond timestamp.
func RVFromSnowflake(snowflakeID int64) int64 {
microSecFraction := snowflakeID & ((1 << snowflake.StepBits) - 1)
return ((snowflakeID>>(snowflake.NodeBits+snowflake.StepBits))+snowflake.Epoch)*1000 + microSecFraction
}
// helper utility to compare two RVs. The first RV must be in snowflake format. Will convert rv2 to snowflake and retry
// if comparison fails
func IsRvEqual(rv1, rv2 int64) bool {
@@ -385,7 +377,7 @@ func IsRvEqual(rv1, rv2 int64) bool {
return true
}
return rv1 == SnowflakeFromRV(rv2)
return rv1 == SnowflakeFromRv(rv2)
}
// Lock locks the resource version for the given key

View File

@@ -63,13 +63,3 @@ func TestResourceVersionManager(t *testing.T) {
require.Equal(t, rv, int64(200))
})
}
func TestSnowflakeFromRVRoundtrips(t *testing.T) {
// 2026-01-12 19:33:58.806211 +0000 UTC
offset := int64(1768246438806211) // in microseconds
for n := range int64(100) {
ts := offset + n
require.Equal(t, ts, RVFromSnowflake(SnowflakeFromRV(ts)))
}
}

View File

@@ -23,6 +23,7 @@ import (
"github.com/grafana/authlib/types"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/infra/db"
"github.com/grafana/grafana/pkg/storage/unified/resource"
"github.com/grafana/grafana/pkg/storage/unified/resourcepb"
sqldb "github.com/grafana/grafana/pkg/storage/unified/sql/db"
@@ -99,6 +100,10 @@ func RunStorageBackendTest(t *testing.T, newBackend NewBackendFunc, opts *TestOp
}
t.Run(tc.name, func(t *testing.T) {
if db.IsTestDbSQLite() {
t.Skip("Skipping tests on sqlite until channel notifier is implemented")
}
tc.fn(t, newBackend(context.Background()), opts.NSPrefix)
})
}
@@ -550,7 +555,7 @@ func runTestIntegrationBackendListModifiedSince(t *testing.T, backend resource.S
Resource: "resource",
}
latestRv, seq := backend.ListModifiedSince(ctx, key, rvCreated)
require.Greater(t, latestRv, rvCreated)
require.Equal(t, latestRv, rvDeleted)
counter := 0
for res, err := range seq {
@@ -624,11 +629,11 @@ func runTestIntegrationBackendListModifiedSince(t *testing.T, backend resource.S
rvCreated3, _ := writeEvent(ctx, backend, "bItem", resourcepb.WatchEvent_ADDED, WithNamespace(ns))
latestRv, seq := backend.ListModifiedSince(ctx, key, rvCreated1-1)
require.Greater(t, latestRv, rvCreated3)
require.Equal(t, latestRv, rvCreated3)
counter := 0
names := []string{"aItem", "bItem", "cItem"}
rvs := []int64{rvCreated2, rvCreated3, rvCreated1}
names := []string{"bItem", "aItem", "cItem"}
rvs := []int64{rvCreated3, rvCreated2, rvCreated1}
for res, err := range seq {
require.NoError(t, err)
require.Equal(t, key.Namespace, res.Key.Namespace)
@@ -1166,7 +1171,7 @@ func runTestIntegrationBackendCreateNewResource(t *testing.T, backend resource.S
}))
server := newServer(t, backend)
ns := nsPrefix + "-create-resource"
ns := nsPrefix + "-create-rsrce" // create-resource
ctx = request.WithNamespace(ctx, ns)
request := &resourcepb.CreateRequest{
@@ -1607,7 +1612,7 @@ func (s *sliceBulkRequestIterator) RollbackRequested() bool {
func runTestIntegrationBackendOptimisticLocking(t *testing.T, backend resource.StorageBackend, nsPrefix string) {
ctx := testutil.NewTestContext(t, time.Now().Add(30*time.Second))
ns := nsPrefix + "-optimistic-locking"
ns := nsPrefix + "-optimis-lock" // optimistic-locking. need to cut down on characters to not exceed namespace character limit (40)
t.Run("concurrent updates with same RV - only one succeeds", func(t *testing.T) {
// Create initial resource with rv0 (no previous RV)

View File

@@ -36,6 +36,10 @@ func NewTestSqlKvBackend(t *testing.T, ctx context.Context, withRvManager bool)
KvStore: kv,
}
if db.DriverName() == "sqlite3" {
kvOpts.UseChannelNotifier = true
}
if withRvManager {
dialect := sqltemplate.DialectForDriver(db.DriverName())
rvManager, err := rvmanager.NewResourceVersionManager(rvmanager.ResourceManagerOptions{
@@ -200,7 +204,7 @@ func verifyKeyPath(t *testing.T, db sqldb.DB, ctx context.Context, key *resource
var keyPathRV int64
if isSqlBackend {
// Convert microsecond RV to snowflake for key_path construction
keyPathRV = rvmanager.SnowflakeFromRV(resourceVersion)
keyPathRV = rvmanager.SnowflakeFromRv(resourceVersion)
} else {
// KV backend already provides snowflake RV
keyPathRV = resourceVersion
@@ -434,6 +438,9 @@ func verifyResourceHistoryTable(t *testing.T, db sqldb.DB, namespace string, res
rows, err := db.QueryContext(ctx, query, namespace)
require.NoError(t, err)
defer func() {
_ = rows.Close()
}()
var records []ResourceHistoryRecord
for rows.Next() {
@@ -457,34 +464,33 @@ func verifyResourceHistoryTable(t *testing.T, db sqldb.DB, namespace string, res
for resourceIdx, res := range resources {
// Check create record (action=1, generation=1)
createRecord := records[recordIndex]
verifyResourceHistoryRecord(t, createRecord, namespace, res, resourceIdx, 1, 0, 1, resourceVersions[resourceIdx][0])
verifyResourceHistoryRecord(t, createRecord, res, resourceIdx, 1, 0, 1, resourceVersions[resourceIdx][0])
recordIndex++
}
for resourceIdx, res := range resources {
// Check update record (action=2, generation=2)
updateRecord := records[recordIndex]
verifyResourceHistoryRecord(t, updateRecord, namespace, res, resourceIdx, 2, resourceVersions[resourceIdx][0], 2, resourceVersions[resourceIdx][1])
verifyResourceHistoryRecord(t, updateRecord, res, resourceIdx, 2, resourceVersions[resourceIdx][0], 2, resourceVersions[resourceIdx][1])
recordIndex++
}
for resourceIdx, res := range resources[:2] {
// Check delete record (action=3, generation=0) - only first 2 resources were deleted
deleteRecord := records[recordIndex]
verifyResourceHistoryRecord(t, deleteRecord, namespace, res, resourceIdx, 3, resourceVersions[resourceIdx][1], 0, resourceVersions[resourceIdx][2])
verifyResourceHistoryRecord(t, deleteRecord, res, resourceIdx, 3, resourceVersions[resourceIdx][1], 0, resourceVersions[resourceIdx][2])
recordIndex++
}
}
// verifyResourceHistoryRecord validates a single resource_history record
func verifyResourceHistoryRecord(t *testing.T, record ResourceHistoryRecord, namespace string, expectedRes struct{ name, folder string }, resourceIdx, expectedAction int, expectedPrevRV int64, expectedGeneration int, expectedRV int64) {
func verifyResourceHistoryRecord(t *testing.T, record ResourceHistoryRecord, expectedRes struct{ name, folder string }, resourceIdx, expectedAction int, expectedPrevRV int64, expectedGeneration int, expectedRV int64) {
// Validate GUID (should be non-empty)
require.NotEmpty(t, record.GUID, "GUID should not be empty")
// Validate group/resource/namespace/name
require.Equal(t, "playlist.grafana.app", record.Group)
require.Equal(t, "playlists", record.Resource)
require.Equal(t, namespace, record.Namespace)
require.Equal(t, expectedRes.name, record.Name)
// Validate value contains expected JSON - server modifies/formats the JSON differently for different operations
@@ -511,12 +517,8 @@ func verifyResourceHistoryRecord(t *testing.T, record ResourceHistoryRecord, nam
// For KV backend operations, expectedPrevRV is now in snowflake format (returned by KV backend)
// but resource_history table stores microsecond RV, so we need to use IsRvEqual for comparison
if strings.Contains(record.Namespace, "-kv") {
if expectedPrevRV == 0 {
require.Zero(t, record.PreviousResourceVersion)
} else {
require.Equal(t, expectedPrevRV, rvmanager.SnowflakeFromRV(record.PreviousResourceVersion),
"Previous resource version should match (KV backend snowflake format)")
}
require.True(t, rvmanager.IsRvEqual(expectedPrevRV, record.PreviousResourceVersion),
"Previous resource version should match (KV backend snowflake format)")
} else {
require.Equal(t, expectedPrevRV, record.PreviousResourceVersion)
}
@@ -548,6 +550,9 @@ func verifyResourceTable(t *testing.T, db sqldb.DB, namespace string, resources
rows, err := db.QueryContext(ctx, query, namespace)
require.NoError(t, err)
defer func() {
_ = rows.Close()
}()
var records []ResourceRecord
for rows.Next() {
@@ -611,6 +616,9 @@ func verifyResourceVersionTable(t *testing.T, db sqldb.DB, namespace string, res
// Check that we have exactly one entry for playlist.grafana.app/playlists
rows, err := db.QueryContext(ctx, query, "playlist.grafana.app", "playlists")
require.NoError(t, err)
defer func() {
_ = rows.Close()
}()
var records []ResourceVersionRecord
for rows.Next() {
@@ -645,7 +653,7 @@ func verifyResourceVersionTable(t *testing.T, db sqldb.DB, namespace string, res
isKvBackend := strings.Contains(namespace, "-kv")
recordResourceVersion := record.ResourceVersion
if isKvBackend {
recordResourceVersion = rvmanager.SnowflakeFromRV(record.ResourceVersion)
recordResourceVersion = rvmanager.SnowflakeFromRv(record.ResourceVersion)
}
require.Less(t, recordResourceVersion, int64(9223372036854775807), "resource_version should be reasonable")
@@ -837,20 +845,24 @@ func runMixedConcurrentOperations(t *testing.T, sqlServer, kvServer resource.Res
}
// SQL backend operations
wg.Go(func() {
wg.Add(1)
go func() {
defer wg.Done()
<-startBarrier // Wait for signal to start
if err := runBackendOperationsWithCounts(ctx, sqlServer, namespace+"-sql", "sql", opCounts); err != nil {
errors <- fmt.Errorf("SQL backend operations failed: %w", err)
}
})
}()
// KV backend operations
wg.Go(func() {
wg.Add(1)
go func() {
defer wg.Done()
<-startBarrier // Wait for signal to start
if err := runBackendOperationsWithCounts(ctx, kvServer, namespace+"-kv", "kv", opCounts); err != nil {
errors <- fmt.Errorf("KV backend operations failed: %w", err)
}
})
}()
// Start both goroutines simultaneously
close(startBarrier)

View File

@@ -30,7 +30,6 @@ func TestBadgerKVStorageBackend(t *testing.T) {
SkipTests: map[string]bool{
// TODO: fix these tests and remove this skip
TestBlobSupport: true,
TestListModifiedSince: true,
// Badger does not support bulk import yet.
TestGetResourceLastImportTime: true,
},
@@ -41,17 +40,8 @@ func TestIntegrationSQLKVStorageBackend(t *testing.T) {
testutil.SkipIntegrationTestInShortMode(t)
skipTests := map[string]bool{
TestWatchWriteEvents: true,
TestList: true,
TestBlobSupport: true,
TestGetResourceStats: true,
TestListHistory: true,
TestListHistoryErrorReporting: true,
TestListModifiedSince: true,
TestListTrash: true,
TestCreateNewResource: true,
TestGetResourceLastImportTime: true,
TestOptimisticLocking: true,
}
t.Run("Without RvManager", func(t *testing.T) {
@@ -59,7 +49,7 @@ func TestIntegrationSQLKVStorageBackend(t *testing.T) {
backend, _ := NewTestSqlKvBackend(t, ctx, false)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-test",
NSPrefix: "sqlkvstoragetest",
SkipTests: skipTests,
})
})
@@ -69,7 +59,7 @@ func TestIntegrationSQLKVStorageBackend(t *testing.T) {
backend, _ := NewTestSqlKvBackend(t, ctx, true)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-withrvmanager-test",
NSPrefix: "sqlkvstoragetest-rvmanager",
SkipTests: skipTests,
})
})