Provisioning: Remove migration from legacy storage (#112505)

* Deprecate Legacy Storage Migration in Backend

* Change the messaging around legacy storage

* Disable cards to connect

* Commit import changes

* Block repository creation if resources are in legacy storage

* Update error message

* Prettify

* chore: uncomment unified migration

* chore: adapt and fix tests

* Remove legacy storage migration from frontend

* Refactor provisioning job options by removing legacy storage and history fields

- Removed the `History` field from `MigrateJobOptions` and related references in the codebase.
- Eliminated the `LegacyStorage` field from `RepositoryViewList` and its associated comments.
- Updated tests and generated OpenAPI schema to reflect these changes.
- Simplified the `MigrationWorker` by removing dependencies on legacy storage checks.

* Refactor OpenAPI schema and tests to remove deprecated fields

- Removed the `history` field from `MigrateJobOptions` and updated the OpenAPI schema accordingly.
- Eliminated the `legacyStorage` field from `RepositoryViewList` and its associated comments in the schema.
- Updated integration tests to reflect the removal of these fields.

* Fix typescript errors

* Refactor provisioning code to remove legacy storage dependencies

- Eliminated references to `dualwrite.Service` and related legacy storage checks across multiple files.
- Updated `APIBuilder`, `RepositoryController`, and `SyncWorker` to streamline resource handling without legacy storage considerations.
- Adjusted tests to reflect the removal of legacy storage mocks and dependencies, ensuring cleaner and more maintainable code.

* Fix unit tests

* Remove more references to legacy

* Enhance provisioning wizard with migration options

- Added a checkbox for migrating existing resources in the BootstrapStep component.
- Updated the form context to track the new migration option.
- Adjusted the SynchronizeStep and useCreateSyncJob hook to incorporate the migration logic.
- Enhanced localization with new descriptions and labels for migration features.

* Remove unused variable and dualwrite reference in provisioning code

- Eliminated an unused variable declaration in `provisioning_manifest.go`.
- Removed the `nil` reference for dualwrite in `repo_operator.go`, aligning with the standalone operator's assumption of unified storage.

* Update go.mod and go.sum to include new dependencies

- Added `github.com/grafana/grafana-app-sdk` version `0.48.5` and several indirect dependencies including `github.com/getkin/kin-openapi`, `github.com/hashicorp/errwrap`, and others.
- Updated `go.sum` to reflect the new dependencies and their respective versions.

* Refactor provisioning components for improved readability

- Simplified the import statement in HomePage.tsx by removing unnecessary line breaks.
- Consolidated props in the SynchronizeStep component for cleaner code.
- Enhanced the layout of the ProvisioningWizard component by streamlining the rendering of the SynchronizeStep.

* Deprecate MigrationWorker and clean up related comments

- Removed the deprecated MigrationWorker implementation and its associated comments from the provisioning code.
- This change reflects the ongoing effort to eliminate legacy components and improve code maintainability.

* Fix linting issues

* Add explicit comment

* Update useResourceStats hook in BootstrapStep component to accept selected target

- Modified the BootstrapStep component to pass the selected target to the useResourceStats hook.
- Updated related tests to reflect the change in expected arguments for the useResourceStats hook.

* fix(provisioning): Update migrate tests to match export-then-sync behavior for all repository types

Updates test expectations for folder-type repositories to match the
implementation changes where both folder and instance repository types
now run export followed by sync. Only the namespace cleaner is skipped
for folder-type repositories.

Changes:
- Update "should run export and sync for folder-type repositories" test to include export mocks
- Update "should fail when sync job fails for folder-type repositories" test to include export mocks
- Rename test to clarify that both export and sync run for folder types
- Add proper mock expectations for SetMessage, StrictMaxErrors, Process, and ResetResults

All migrate package tests now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Update provisioning wizard text and improve resource counting display

- Enhanced descriptions for migrating existing resources to clarify that unmanaged resources will also be included.
- Refactored BootstrapStepResourceCounting component to simplify the rendering logic and ensure both external storage and unmanaged resources are displayed correctly.
- Updated alert messages in SynchronizeStep to reflect accurate information regarding resource management during migration.
- Adjusted localization strings for consistency with the new descriptions.

* Update provisioning wizard alert messages for clarity and accuracy

- Revised alert points to indicate that resources can still be modified during migration, with a note on potential export issues.
- Clarified that resources will be marked as managed post-provisioning and that dashboards remain accessible throughout the process.

* Fix issue with trigger wrong type of job

* Fix export failure when folder already exists in repository

When exporting resources to a repository, if a folder already exists,
the Read() method would fail with "path component is empty" error.

This occurred because:
1. Folders are identified by trailing slash (e.g., "Legacy Folder/")
2. The Read() method passes this path directly to GetTreeByPath()
3. GetTreeByPath() splits the path by "/" creating empty components
4. This causes the "path component is empty" error

The fix strips the trailing slash before calling GetTreeByPath() to
avoid empty path components, while still using the trailing slash
convention to identify directories.

The Create() method already handles this correctly by appending
".keep" to directory paths, which is why the first export succeeded
but subsequent exports failed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix folder tree not updated when folder already exists in repository

When exporting resources and a folder already exists in the repository,
the folder was not being added to the FolderManager's tree. This caused
subsequent dashboard exports to fail with "folder NOT found in tree".

The fix adds the folder to fm.tree even when it already exists in the
repository, ensuring all folders are available for resource lookups.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Revert "Merge remote-tracking branch 'origin/uncomment-unified-migration-code' into cleanup/deprecate-legacy-storage-migration-in-provisioning"

This reverts commit 6440fae342, reversing
changes made to ec39fb04f2.

* fix: handle empty folder titles in path construction

- Skip folders with empty titles in dirPath to avoid empty path components
- Skip folders with empty paths before checking if they exist in repository
- Fix unit tests to properly check useResourceStats hook calls with type annotations

* Update workspace

* Fix BootstrapStep tests after reverting unified migration merge

Updated test expectations to match the current component behavior where
resource counts are displayed for both instance and folder sync options.

- Changed 'Empty' count expectation from 3 to 4 (2 cards × 2 counts each)
- Changed '7 resources' test to use findAllByText instead of findByText
  since the count appears in multiple cards

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Remove bubbletee deps

* Fix workspace

* provisioning: update error message to reference enableMigration config

Update the error message when provisioning cannot be used due to
incompatible data format to instruct users to enable data migration
for folders and dashboards using the enableMigration configuration
introduced in PR #114857.

Also update the test helper to include EnableMigration: true for both
dashboards and folders to match the new configuration pattern.

* provisioning: add comment explaining Mode5 and EnableMigration requirement

Add a comment in the integration test helper explaining that Provisioning
requires Mode5 (unified storage) and EnableMigration (data migration) as
it expects resources to be fully migrated to unified storage.

* Remove migrate resources checkbox from folder type provisioning wizard

- Remove checkbox UI for migrating existing resources in folder type
- Remove migrateExistingResources from migration logic
- Simplify migration to only use requiresMigration flag
- Remove unused translation keys
- Update i18n strings

* Fix linting

* Remove unnecessary React Fragment wrapper in BootstrapStep

* Address comments

---------

Co-authored-by: Rafael Paulovic <rafael.paulovic@grafana.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Roberto Jiménez Sánchez
2025-12-17 17:22:17 +01:00
committed by GitHub
co-authored by Rafael Paulovic Claude Sonnet 4.5
parent 60298fb02a
commit 7e45a300b9
69 changed files with 392 additions and 4472 deletions
@@ -197,6 +197,8 @@ func (fm *FolderManager) EnsureFolderTreeExists(ctx context.Context, ref, path s
if err != nil && (!errors.Is(err, repository.ErrFileNotFound) && !apierrors.IsNotFound(err)) {
return fn(folder, false, fmt.Errorf("check if folder exists before writing: %w", err))
} else if err == nil {
// Folder already exists in repository, add it to tree so resources can find it
fm.tree.Add(folder, parent)
return fn(folder, false, nil)
}
@@ -4,15 +4,9 @@ import (
"context"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
dashboard "github.com/grafana/grafana/apps/dashboard/pkg/apis/dashboard/v0alpha1"
folders "github.com/grafana/grafana/apps/folder/pkg/apis/folder/v1beta1"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/dashboard/legacy"
"github.com/grafana/grafana/pkg/storage/legacysql/dualwrite"
"github.com/grafana/grafana/pkg/storage/unified/migrations"
"github.com/grafana/grafana/pkg/storage/unified/resource"
"github.com/grafana/grafana/pkg/storage/unified/resourcepb"
)
@@ -31,25 +25,16 @@ type ResourceStore interface {
}
type ResourceListerFromSearch struct {
store ResourceStore
migrator migrations.UnifiedMigrator
storageStatus dualwrite.Service
store ResourceStore
}
func NewResourceLister(store ResourceStore) ResourceLister {
return &ResourceListerFromSearch{store: store}
}
// FIXME: the logic about migration and storage should probably be separated from this
func NewResourceListerForMigrations(
store ResourceStore,
migrator migrations.UnifiedMigrator,
storageStatus dualwrite.Service,
) ResourceLister {
func NewResourceListerForMigrations(store ResourceStore) ResourceLister {
return &ResourceListerFromSearch{
store: store,
migrator: migrator,
storageStatus: storageStatus,
store: store,
}
}
@@ -133,37 +118,6 @@ func (o *ResourceListerFromSearch) Stats(ctx context.Context, namespace, reposit
return stats, nil
}
// Get the stats based on what a migration could support
if o.storageStatus != nil && o.migrator != nil && dualwrite.IsReadingLegacyDashboardsAndFolders(ctx, o.storageStatus) {
rsp, err := o.migrator.Migrate(ctx, legacy.MigrateOptions{
Namespace: namespace,
Resources: []schema.GroupResource{{
Group: dashboard.GROUP, Resource: dashboard.DASHBOARD_RESOURCE,
}, {
Group: folders.GROUP, Resource: folders.RESOURCE,
}},
WithHistory: false,
OnlyCount: true,
})
if err != nil {
return nil, err
}
for _, v := range rsp.Summary {
stats.Instance = append(stats.Instance, provisioning.ResourceCount{
Group: v.Group,
Resource: v.Resource,
Count: v.Count,
})
// Everything is unmanaged in legacy storage
stats.Unmanaged = append(stats.Unmanaged, provisioning.ResourceCount{
Group: v.Group,
Resource: v.Resource,
Count: v.Count,
})
}
return stats, nil
}
// Get full instance stats
info, err := o.store.GetStats(ctx, &resourcepb.ResourceStatsRequest{
Namespace: namespace,
@@ -180,7 +180,12 @@ func (r *ResourcesManager) WriteResourceFileFromObject(ctx context.Context, obj
var ok bool
fid, ok = r.folders.Tree().DirPath(folder, rootFolder)
if !ok {
return "", fmt.Errorf("folder %s NOT found in tree with root: %s", folder, rootFolder)
// HACK: this is a hack to get the folder path without the root folder
// TODO: should we build the tree in a different way?
fid, ok = r.folders.Tree().DirPath(folder, "")
if !ok {
return "", fmt.Errorf("folder %s NOT found in tree", folder)
}
}
}
@@ -1,33 +0,0 @@
package signature
import (
"context"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/pkg/apimachinery/utils"
)
type grafanaSigner struct{}
// FIXME: where should we use this default signature?
// NewGrafanaSigner returns a Signer that uses the grafana user as the author
func NewGrafanaSigner() Signer {
return &grafanaSigner{}
}
func (s *grafanaSigner) Sign(ctx context.Context, item utils.GrafanaMetaAccessor) (context.Context, error) {
sig := repository.CommitSignature{
Name: "grafana",
// TODO: should we add email?
// Email: "grafana@grafana.com",
}
t, err := item.GetUpdatedTimestamp()
if err == nil && t != nil {
sig.When = *t
} else {
sig.When = item.GetCreationTimestamp().Time
}
return repository.WithAuthorSignature(ctx, sig), nil
}
@@ -1,90 +0,0 @@
package signature
import (
"context"
"errors"
"testing"
"time"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/pkg/apimachinery/utils"
)
func TestNewGrafanaSigner(t *testing.T) {
signer := NewGrafanaSigner()
require.NotNil(t, signer, "signer should not be nil")
require.IsType(t, &grafanaSigner{}, signer, "signer should be of type *grafanaSigner")
}
func TestGrafanaSigner_Sign(t *testing.T) {
tests := []struct {
name string
creationTimestamp time.Time
updateTimestampErr error
updatedTimestamp *time.Time
expectedTime time.Time
setupMocks func(meta *utils.MockGrafanaMetaAccessor)
}{
{
name: "should use creation timestamp when no update timestamp",
creationTimestamp: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
updatedTimestamp: ptr(time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)),
updateTimestampErr: errors.New("failed"),
expectedTime: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
setupMocks: func(meta *utils.MockGrafanaMetaAccessor) {
meta.On("GetCreationTimestamp").Return(metav1.Time{Time: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)})
},
},
{
name: "should use creation timestamp when update timestamp is nil",
creationTimestamp: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
updatedTimestamp: nil,
expectedTime: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
setupMocks: func(meta *utils.MockGrafanaMetaAccessor) {
meta.On("GetCreationTimestamp").Return(metav1.Time{Time: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)})
},
},
{
name: "should use update timestamp when available",
creationTimestamp: time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC),
updatedTimestamp: ptr(time.Date(2024, 1, 2, 0, 0, 0, 0, time.UTC)),
expectedTime: time.Date(2024, 1, 2, 0, 0, 0, 0, time.UTC),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
meta := utils.NewMockGrafanaMetaAccessor(t)
var updatedTime *time.Time
if tt.updatedTimestamp != nil {
updatedTime = tt.updatedTimestamp
}
meta.On("GetUpdatedTimestamp").Return(updatedTime, tt.updateTimestampErr)
if tt.setupMocks != nil {
tt.setupMocks(meta)
}
signer := NewGrafanaSigner()
ctx := context.Background()
signedCtx, err := signer.Sign(ctx, meta)
require.NoError(t, err)
// Verify the signature in the context
sig := repository.GetAuthorSignature(signedCtx)
require.NotNil(t, sig, "signature should be present in context")
require.Equal(t, "grafana", sig.Name)
require.Equal(t, tt.expectedTime, sig.When)
meta.AssertExpectations(t)
})
}
}
func ptr[T any](v T) *T {
return &v
}
@@ -1,52 +0,0 @@
package signature
import (
"context"
"fmt"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/resources"
)
//go:generate mockery --name Signer --structname MockSigner --inpackage --filename signer_mock.go --with-expecter
type Signer interface {
Sign(ctx context.Context, item utils.GrafanaMetaAccessor) (context.Context, error)
}
type SignOptions struct {
Namespace string
History bool
}
// SignerFactory is a factory for creating Signers
//
//go:generate mockery --name SignerFactory --structname MockSignerFactory --inpackage --filename signature_factory_mock.go --with-expecter
type SignerFactory interface {
New(ctx context.Context, opts SignOptions) (Signer, error)
}
type signerFactory struct {
clients resources.ClientFactory
}
func NewSignerFactory(clients resources.ClientFactory) SignerFactory {
return &signerFactory{clients}
}
func (f *signerFactory) New(ctx context.Context, opts SignOptions) (Signer, error) {
if !opts.History {
return NewGrafanaSigner(), nil
}
clients, err := f.clients.Clients(ctx, opts.Namespace)
if err != nil {
return nil, fmt.Errorf("get clients: %w", err)
}
userClient, err := clients.User(ctx)
if err != nil {
return nil, fmt.Errorf("get user client: %w", err)
}
return NewLoadUsersOnceSigner(userClient), nil
}
@@ -1,95 +0,0 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package signature
import (
context "context"
mock "github.com/stretchr/testify/mock"
)
// MockSignerFactory is an autogenerated mock type for the SignerFactory type
type MockSignerFactory struct {
mock.Mock
}
type MockSignerFactory_Expecter struct {
mock *mock.Mock
}
func (_m *MockSignerFactory) EXPECT() *MockSignerFactory_Expecter {
return &MockSignerFactory_Expecter{mock: &_m.Mock}
}
// New provides a mock function with given fields: ctx, opts
func (_m *MockSignerFactory) New(ctx context.Context, opts SignOptions) (Signer, error) {
ret := _m.Called(ctx, opts)
if len(ret) == 0 {
panic("no return value specified for New")
}
var r0 Signer
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, SignOptions) (Signer, error)); ok {
return rf(ctx, opts)
}
if rf, ok := ret.Get(0).(func(context.Context, SignOptions) Signer); ok {
r0 = rf(ctx, opts)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(Signer)
}
}
if rf, ok := ret.Get(1).(func(context.Context, SignOptions) error); ok {
r1 = rf(ctx, opts)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockSignerFactory_New_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'New'
type MockSignerFactory_New_Call struct {
*mock.Call
}
// New is a helper method to define mock.On call
// - ctx context.Context
// - opts SignOptions
func (_e *MockSignerFactory_Expecter) New(ctx interface{}, opts interface{}) *MockSignerFactory_New_Call {
return &MockSignerFactory_New_Call{Call: _e.mock.On("New", ctx, opts)}
}
func (_c *MockSignerFactory_New_Call) Run(run func(ctx context.Context, opts SignOptions)) *MockSignerFactory_New_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(SignOptions))
})
return _c
}
func (_c *MockSignerFactory_New_Call) Return(_a0 Signer, _a1 error) *MockSignerFactory_New_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockSignerFactory_New_Call) RunAndReturn(run func(context.Context, SignOptions) (Signer, error)) *MockSignerFactory_New_Call {
_c.Call.Return(run)
return _c
}
// NewMockSignerFactory creates a new instance of MockSignerFactory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockSignerFactory(t interface {
mock.TestingT
Cleanup(func())
}) *MockSignerFactory {
mock := &MockSignerFactory{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -1,92 +0,0 @@
package signature
import (
"context"
"fmt"
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/resources"
)
func TestSignerFactory_New(t *testing.T) {
tests := []struct {
name string
opts SignOptions
setupMocks func(t *testing.T, clients *resources.MockClientFactory)
expectedType interface{}
expectedError string
}{
{
name: "should return grafana signer when history is false",
opts: SignOptions{
History: false,
},
setupMocks: func(t *testing.T, clients *resources.MockClientFactory) {
// No mocks needed as we shouldn't call any clients
},
expectedType: &grafanaSigner{},
},
{
name: "should return load users once signer when history is true",
opts: SignOptions{
History: true,
Namespace: "test-ns",
},
setupMocks: func(t *testing.T, clients *resources.MockClientFactory) {
mockResourceClients := resources.NewMockResourceClients(t)
clients.On("Clients", context.Background(), "test-ns").Return(mockResourceClients, nil)
mockResourceClients.On("User", mock.Anything).Return(nil, nil)
},
expectedType: &loadUsersOnceSigner{},
},
{
name: "should return error when clients factory fails",
opts: SignOptions{
History: true,
Namespace: "test-ns",
},
setupMocks: func(t *testing.T, clients *resources.MockClientFactory) {
clients.On("Clients", context.Background(), "test-ns").Return(nil, fmt.Errorf("clients error"))
},
expectedError: "get clients: clients error",
},
{
name: "should return error when user client fails",
opts: SignOptions{
History: true,
Namespace: "test-ns",
},
setupMocks: func(t *testing.T, clients *resources.MockClientFactory) {
mockResourceClients := resources.NewMockResourceClients(t)
clients.On("Clients", context.Background(), "test-ns").Return(mockResourceClients, nil)
mockResourceClients.On("User", mock.Anything).Return(nil, fmt.Errorf("user client error"))
},
expectedError: "get user client: user client error",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockClients := resources.NewMockClientFactory(t)
tt.setupMocks(t, mockClients)
factory := NewSignerFactory(mockClients)
signer, err := factory.New(context.Background(), tt.opts)
if tt.expectedError != "" {
require.Error(t, err)
require.EqualError(t, err, tt.expectedError)
require.Nil(t, signer)
} else {
require.NoError(t, err)
require.NotNil(t, signer)
require.IsType(t, tt.expectedType, signer, "signer should be of expected type")
}
mockClients.AssertExpectations(t)
})
}
}
@@ -1,96 +0,0 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package signature
import (
context "context"
utils "github.com/grafana/grafana/pkg/apimachinery/utils"
mock "github.com/stretchr/testify/mock"
)
// MockSigner is an autogenerated mock type for the Signer type
type MockSigner struct {
mock.Mock
}
type MockSigner_Expecter struct {
mock *mock.Mock
}
func (_m *MockSigner) EXPECT() *MockSigner_Expecter {
return &MockSigner_Expecter{mock: &_m.Mock}
}
// Sign provides a mock function with given fields: ctx, item
func (_m *MockSigner) Sign(ctx context.Context, item utils.GrafanaMetaAccessor) (context.Context, error) {
ret := _m.Called(ctx, item)
if len(ret) == 0 {
panic("no return value specified for Sign")
}
var r0 context.Context
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, utils.GrafanaMetaAccessor) (context.Context, error)); ok {
return rf(ctx, item)
}
if rf, ok := ret.Get(0).(func(context.Context, utils.GrafanaMetaAccessor) context.Context); ok {
r0 = rf(ctx, item)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(context.Context)
}
}
if rf, ok := ret.Get(1).(func(context.Context, utils.GrafanaMetaAccessor) error); ok {
r1 = rf(ctx, item)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockSigner_Sign_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Sign'
type MockSigner_Sign_Call struct {
*mock.Call
}
// Sign is a helper method to define mock.On call
// - ctx context.Context
// - item utils.GrafanaMetaAccessor
func (_e *MockSigner_Expecter) Sign(ctx interface{}, item interface{}) *MockSigner_Sign_Call {
return &MockSigner_Sign_Call{Call: _e.mock.On("Sign", ctx, item)}
}
func (_c *MockSigner_Sign_Call) Run(run func(ctx context.Context, item utils.GrafanaMetaAccessor)) *MockSigner_Sign_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(utils.GrafanaMetaAccessor))
})
return _c
}
func (_c *MockSigner_Sign_Call) Return(_a0 context.Context, _a1 error) *MockSigner_Sign_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockSigner_Sign_Call) RunAndReturn(run func(context.Context, utils.GrafanaMetaAccessor) (context.Context, error)) *MockSigner_Sign_Call {
_c.Call.Return(run)
return _c
}
// NewMockSigner creates a new instance of MockSigner. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockSigner(t interface {
mock.TestingT
Cleanup(func())
}) *MockSigner {
mock := &MockSigner{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -1,115 +0,0 @@
package signature
import (
"context"
"errors"
"fmt"
"strings"
"sync"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/client-go/dynamic"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/resources"
)
const maxUsers = 10000
type loadUsersOnceSigner struct {
signatures map[string]repository.CommitSignature
client dynamic.ResourceInterface
once sync.Once
onceErr error
}
// NewLoadUsersOnceSigner returns a Signer that loads the signatures from users
// it will only load the signatures once and cache them
// if the user is not found, it will use the grafana user as the author
func NewLoadUsersOnceSigner(client dynamic.ResourceInterface) Signer {
return &loadUsersOnceSigner{
client: client,
once: sync.Once{},
signatures: map[string]repository.CommitSignature{},
}
}
func (s *loadUsersOnceSigner) Sign(ctx context.Context, item utils.GrafanaMetaAccessor) (context.Context, error) {
if s.onceErr != nil {
return ctx, fmt.Errorf("load signatures: %w", s.onceErr)
}
var err error
s.once.Do(func() {
s.signatures, err = s.load(ctx, s.client)
s.onceErr = err
})
if err != nil {
return ctx, fmt.Errorf("load signatures: %w", err)
}
id := item.GetUpdatedBy()
if id == "" {
id = item.GetCreatedBy()
}
if id == "" {
id = "grafana"
}
sig := s.signatures[id] // lookup
if sig.Name == "" && sig.Email == "" {
sig.Name = id
}
t, err := item.GetUpdatedTimestamp()
if err == nil && t != nil {
sig.When = *t
} else {
sig.When = item.GetCreationTimestamp().Time
}
return repository.WithAuthorSignature(ctx, sig), nil
}
func (s *loadUsersOnceSigner) load(ctx context.Context, client dynamic.ResourceInterface) (map[string]repository.CommitSignature, error) {
userInfo := make(map[string]repository.CommitSignature)
var count int
err := resources.ForEach(ctx, client, func(item *unstructured.Unstructured) error {
count++
if count > maxUsers {
return errors.New("too many users")
}
sig := repository.CommitSignature{}
// FIXME: should we improve logging here?
var (
ok bool
err error
)
sig.Name, ok, err = unstructured.NestedString(item.Object, "spec", "login")
if !ok || err != nil {
return nil
}
sig.Email, ok, err = unstructured.NestedString(item.Object, "spec", "email")
if !ok || err != nil {
return nil
}
if sig.Name == sig.Email {
if sig.Name == "" {
sig.Name = item.GetName()
} else if strings.Contains(sig.Email, "@") {
sig.Email = "" // don't use the same value for name+email
}
}
userInfo["user:"+item.GetName()] = sig
return nil
})
if err != nil {
return nil, err
}
return userInfo, nil
}
@@ -1,357 +0,0 @@
package signature
import (
"context"
"errors"
"fmt"
"testing"
"time"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/client-go/dynamic"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/pkg/apimachinery/utils"
)
// mockDynamicInterface implements a simplified version of the dynamic.ResourceInterface
type mockDynamicInterface struct {
dynamic.ResourceInterface
items []unstructured.Unstructured
err error
}
func (m *mockDynamicInterface) List(ctx context.Context, opts metav1.ListOptions) (*unstructured.UnstructuredList, error) {
if m.err != nil {
return nil, m.err
}
return &unstructured.UnstructuredList{
Items: m.items,
}, nil
}
type mockGrafanaMetaAccessor struct {
utils.GrafanaMetaAccessor
createdBy string
updatedBy string
creationTimestamp time.Time
updatedTimestamp *time.Time
updatedTimestampErr error
}
func (m *mockGrafanaMetaAccessor) GetCreatedBy() string {
return m.createdBy
}
func (m *mockGrafanaMetaAccessor) GetUpdatedBy() string {
return m.updatedBy
}
func (m *mockGrafanaMetaAccessor) GetCreationTimestamp() metav1.Time {
return metav1.Time{Time: m.creationTimestamp}
}
func (m *mockGrafanaMetaAccessor) GetUpdatedTimestamp() (*time.Time, error) {
if m.updatedTimestampErr != nil {
return nil, m.updatedTimestampErr
}
return m.updatedTimestamp, nil
}
func TestLoadUsersOnceSigner_Sign(t *testing.T) {
baseTime := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
updateTime := time.Date(2024, 1, 2, 0, 0, 0, 0, time.UTC)
tests := []struct {
name string
items []unstructured.Unstructured
meta *mockGrafanaMetaAccessor
clientErr error
expectedSig repository.CommitSignature
expectedError string
}{
{
name: "should sign with user info when user exists",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "johndoe",
"email": "john@example.com",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
updatedTimestamp: &updateTime,
},
expectedSig: repository.CommitSignature{
Name: "johndoe",
Email: "john@example.com",
When: updateTime,
},
},
{
name: "should fallback to created by when updated by is empty",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "johndoe",
"email": "john@example.com",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
createdBy: "user:user1",
creationTimestamp: baseTime,
},
expectedSig: repository.CommitSignature{
Name: "johndoe",
Email: "john@example.com",
When: baseTime,
},
},
{
name: "should use grafana when no user info available",
meta: &mockGrafanaMetaAccessor{
creationTimestamp: baseTime,
},
expectedSig: repository.CommitSignature{
Name: "grafana",
When: baseTime,
},
},
{
name: "should handle user with same login and email",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "john@example.com",
"email": "john@example.com",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
updatedTimestamp: &updateTime,
},
expectedSig: repository.CommitSignature{
Name: "john@example.com",
Email: "",
When: updateTime,
},
},
{
name: "should handle empty login and email",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "",
"email": "",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
updatedTimestamp: &updateTime,
},
expectedSig: repository.CommitSignature{
Name: "user1",
Email: "",
When: updateTime,
},
},
{
name: "should handle empty email",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "johndoe",
"email": "",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
updatedTimestamp: &updateTime,
},
expectedSig: repository.CommitSignature{
Name: "johndoe",
Email: "",
When: updateTime,
},
},
{
name: "should fail when too many users",
items: func() []unstructured.Unstructured {
items := make([]unstructured.Unstructured, maxUsers+1)
for i := 0; i < maxUsers+1; i++ {
items[i] = unstructured.Unstructured{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "johndoe",
"email": "john@example.com",
},
},
}
}
return items
}(),
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
},
expectedError: "load signatures: too many users",
},
{
name: "should handle missing user fields gracefully",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
// missing login and email
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
},
expectedSig: repository.CommitSignature{
Name: "user:user1",
When: baseTime,
},
},
{
name: "should use creation timestamp when update timestamp has error",
items: []unstructured.Unstructured{
{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
"name": "user1",
},
"spec": map[string]interface{}{
"login": "johndoe",
"email": "john@example.com",
},
},
},
},
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
updatedTimestampErr: errors.New("update timestamp error"),
},
expectedSig: repository.CommitSignature{
Name: "johndoe",
Email: "john@example.com",
When: baseTime,
},
},
{
name: "should fail when listing users fails",
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
},
clientErr: fmt.Errorf("failed to list users"),
expectedError: "load signatures: error executing list: failed to list users",
},
{
name: "should handle empty user list",
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
},
items: []unstructured.Unstructured{},
expectedSig: repository.CommitSignature{
Name: "user:user1",
When: baseTime,
},
},
{
name: "should handle multiple calls with error",
meta: &mockGrafanaMetaAccessor{
updatedBy: "user:user1",
creationTimestamp: baseTime,
},
clientErr: fmt.Errorf("failed to list users"),
expectedError: "load signatures: error executing list: failed to list users",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client := &mockDynamicInterface{
items: tt.items,
err: tt.clientErr,
}
signer := NewLoadUsersOnceSigner(client)
ctx := context.Background()
signedCtx, err := signer.Sign(ctx, tt.meta)
if tt.expectedError != "" {
require.Error(t, err)
require.Contains(t, err.Error(), tt.expectedError)
// Test that subsequent calls also fail with the same error
_, err2 := signer.Sign(ctx, tt.meta)
require.Error(t, err2)
require.Contains(t, err2.Error(), tt.expectedError)
return
}
require.NoError(t, err)
sig := repository.GetAuthorSignature(signedCtx)
require.NotNil(t, sig)
require.Equal(t, tt.expectedSig.Name, sig.Name)
require.Equal(t, tt.expectedSig.Email, sig.Email)
require.Equal(t, tt.expectedSig.When, sig.When)
// Test that subsequent calls use cached data
signedCtx2, err := signer.Sign(ctx, tt.meta)
require.NoError(t, err)
sig2 := repository.GetAuthorSignature(signedCtx2)
require.Equal(t, sig, sig2)
})
}
}