grafana/apps/dashboard/pkg/migration

Dashboard migrations

This document describes the Grafana dashboard migration system, focusing on conversion-level practices including metrics, logging, and testing infrastructure for API version conversions. For schema version migration implementation details, see the SchemaVersion Migration Guide.

Table of Contents

• Overview
  • Conversion Flow
  • v0 to v1 Conversion
  • v1 to v2 Conversion
  • v2 to v0/v1 Conversion
• Conversion Matrix
• API Versions
• Schema Versions
• Testing
  • Backend conversion tests
  • Backend and frontend conversion parity tests
• Monitoring Migrations
  • Metrics
    • Dashboard conversion success metric
    • Dashboard conversion failure metric
  • Error Types
  • Logging
    • Log structure
    • Log levels
  • Implementation Details
    • Automatic instrumentation
    • Error handling
  • Registration
    • Metrics registration
    • Available metrics
• Related Documentation

Overview

The Grafana dashboard migration system operates across three main conversion layers:

Conversion Flow: v0 → v1 → v2

v0alpha1 (Legacy JSON) → v1beta1 (Migrated JSON) → v2alpha1/v2beta1 (Structured)

v0 to v1 Conversion:

• All schema migrations (v0-v42) are executed in the backend
• Ports the logic from DashboardMigrator and implements built-in plugin migrations for panel plugins since backend cannot load plugins
• Transforms legacy JSON dashboards to migrated JSON format
• Handles backward compatibility for older dashboard formats
• See SchemaVersion Migration Guide for detailed instructions on creating new schema migrations

v1 to v2 Conversion:

• API version conversions between different Kubernetes API versions
• Transforms JSON dashboards to structured dashboard format
• v2 schema is the stable, typed schema with proper type definitions
• Handles modern dashboard features and Kubernetes-native storage
• See V2 to V1 Layout Conversion for details on how V2 layouts are converted back to V1 panel arrays

v2 to v0/v1 Conversion:

• Converts structured v2 dashboards back to JSON format (v0alpha1 or v1beta1)
• Chains through intermediate versions: v2 → v1beta1 → v0alpha1
• v0alpha1 and v1beta1 share the same spec structure (only API version differs)
• Enables backward compatibility when storing v2 dashboards in legacy format

Conversion Matrix

The system supports conversions between all dashboard API versions:

From ↓ / To →	v0alpha1	v1beta1	v2alpha1	v2beta1
v0alpha1	✓	✓	✓	✓
v1beta1	✓	✓	✓	✓
v2alpha1	✓	✓	✓	✓
v2beta1	✓	✓	✓	✓

Each conversion path is automatically instrumented with metrics and logging.

API Versions

The supported dashboard API versions are:

• dashboard.grafana.app/v0alpha1 - Legacy JSON dashboard format
• dashboard.grafana.app/v1beta1 - Migrated JSON dashboard format
• dashboard.grafana.app/v2alpha1 - New structured dashboard format
• dashboard.grafana.app/v2beta1 - Enhanced structured dashboard format

Schema Versions

Schema versions (v13-v42) apply only to v0alpha1 and v1beta1 dashboards:

• Minimum supported version: v13
• Latest version: v42
• Migration path: Sequential (v13→v14→v15...→v42)

For detailed information about creating schema version migrations, see the SchemaVersion Migration Guide.

Testing

The implementation includes comprehensive test coverage for conversion-level operations:

• Backend conversion tests: API version conversions with metrics validation
• Frontend tests: TypeScript conversion tests
• Integration tests: End-to-end conversion validation
• Metrics tests: Prometheus counter validation

Backend conversion tests

The backend conversion tests validate API version conversions and metrics instrumentation:

• API conversion tests: Test conversions between v0alpha1, v1beta1, v2alpha1, v2beta1
• Metrics validation: Tests verify that conversion metrics are properly recorded
• Error handling: Tests validate error classification and logging
• Performance: Tests ensure conversion operations are efficient

Test execution:

# All backend conversion tests
go test ./apps/dashboard/pkg/migration/conversion/... -v

# Metrics validation tests
go test ./apps/dashboard/pkg/migration/... -run TestSchemaMigrationMetrics

Backend and frontend conversion parity tests

These tests ensure that backend (Go) and frontend (TypeScript) conversions produce identical outputs. This is critical because:

• Dual implementation: Both backend and frontend implement dashboard version conversions
• Consistency requirement: Users should see the same dashboard regardless of which path is used
• API flexibility: The API may return dashboards in different versions depending on context

Why normalize through Scene?

Both backend and frontend outputs are passed through the same Scene load/save cycle before comparison. This normalization:

• Eliminates differences from default values added by Scene
• Handles field ordering variations
• Simulates the real-world flow: dashboard loaded → edited → saved

Test locations:

Test File	Purpose
public/app/features/dashboard-scene/serialization/transformSaveModelV1ToV2.test.ts	v1beta1 → v2beta1 conversion parity
public/app/features/dashboard-scene/serialization/transformSaveModelV2ToV1.test.ts	v2beta1 → v1beta1/v0alpha1 conversion parity
public/app/features/dashboard/state/DashboardMigratorToBackend.test.ts	Schema version migration parity

Test execution:

# V1 to V2 conversion parity tests
yarn test transformSaveModelV1ToV2.test.ts

# V2 to V1 conversion parity tests
yarn test transformSaveModelV2ToV1.test.ts

# Schema migration parity tests  
yarn test DashboardMigratorToBackend.test.ts

Test approach (v1 → v2):

• Backend path: v1beta1 → Go conversion → v2beta1 → Scene → normalized output
• Frontend path: v1beta1 → Scene → v2beta1 → Scene → normalized output
• Test data: Uses files from apps/dashboard/pkg/migration/conversion/testdata/ and migrated dashboards

Test approach (v2 → v1/v0):

• Backend path: v2beta1 → Go conversion → v1beta1/v0alpha1 → Scene → normalized output
• Frontend path: v2beta1 → Scene → v1beta1 → Scene → normalized output
• Target versions: Tests both v0alpha1 and v1beta1 (they share the same spec structure)
• Test data: Uses files from apps/dashboard/pkg/migration/conversion/testdata/

For schema version migration testing details, see the SchemaVersion Migration Guide.

Monitoring Migrations

The dashboard migration system provides comprehensive observability through metrics, logging, and error classification to monitor conversion operations.

Metrics

The dashboard migration system now provides comprehensive observability through:

• Prometheus metrics for tracking conversion success/failure rates and performance
• Structured logging for debugging and monitoring conversion operations
• Automatic instrumentation via wrapper functions that eliminate code duplication
• Error classification to distinguish between different types of migration failures

1. Dashboard conversion success metric

Metric Name: grafana_dashboard_migration_conversion_success_total

Type: Counter

Description: Total number of successful dashboard conversions

Labels:

• source_version_api - Source API version (e.g., "dashboard.grafana.app/v0alpha1")
• target_version_api - Target API version (e.g., "dashboard.grafana.app/v1beta1")
• source_schema_version - Source schema version (e.g., "16") - only for v0/v1 dashboards
• target_schema_version - Target schema version (e.g., "41") - only for v0/v1 dashboards

Example:

grafana_dashboard_migration_conversion_success_total{
  source_version_api="dashboard.grafana.app/v0alpha1",
  target_version_api="dashboard.grafana.app/v1beta1",
  source_schema_version="16",
  target_schema_version="41"
} 1250

2. Dashboard conversion failure metric

Metric Name: grafana_dashboard_migration_conversion_failure_total

Type: Counter

Description: Total number of failed dashboard conversions

Labels:

• source_version_api - Source API version
• target_version_api - Target API version
• source_schema_version - Source schema version (only for v0/v1 dashboards)
• target_schema_version - Target schema version (only for v0/v1 dashboards)
• error_type - Classification of the error (see Error Types section)

Example:

grafana_dashboard_migration_conversion_failure_total{
  source_version_api="dashboard.grafana.app/v0alpha1",
  target_version_api="dashboard.grafana.app/v1beta1",
  source_schema_version="14",
  target_schema_version="41",
  error_type="schema_version_migration_error"
} 42

Error Types

The error_type label classifies failures into four categories:

1. conversion_error

• General conversion failures not related to schema migration
• API-level conversion issues
• Programming errors in conversion functions

2. schema_version_migration_error

• Failures during individual schema version migrations (v14→v15, v15→v16, etc.)
• Schema-specific transformation errors
• Data format incompatibilities

3. schema_minimum_version_error

• Dashboards with schema versions below the minimum supported version (< v13)
• These are logged as warnings rather than errors
• Indicates dashboards that cannot be migrated automatically

4. conversion_data_loss_error

• Data loss detected during conversion
• Automatically checks that panels, queries, annotations, and links are preserved
• Triggered when target has fewer items than source
• Includes detailed loss metrics in logs (see Data Loss Detection)

Logging

Log structure

All migration logs use structured logging with consistent field names:

Base Fields (always present):

• sourceVersionAPI - Source API version
• targetVersionAPI - Target API version
• dashboardUID - Unique identifier of the dashboard being converted

Schema Version Fields (v0/v1 dashboards only):

• sourceSchemaVersion - Source schema version number
• targetSchemaVersion - Target schema version number
• erroredSchemaVersionFunc - Name of the schema migration function that failed (on error)

Error Fields (failures only):

• errorType - Same classification as metrics error_type label
• erroredConversionFunc - Name of the conversion function that failed
• error - The actual error message

Data Loss Fields (conversion_data_loss_error only):

• panelsLost - Number of panels lost
• queriesLost - Number of queries lost
• annotationsLost - Number of annotations lost
• linksLost - Number of links lost
• variablesLost - Number of template variables lost

Log levels

Success (DEBUG level)

{
  "level": "debug",
  "msg": "Dashboard conversion succeeded",
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "dashboardUID": "abc123",
  "sourceSchemaVersion": 16,
  "targetSchemaVersion": 41
}

Conversion/Migration Error (ERROR level)

{
  "level": "error", 
  "msg": "Dashboard conversion failed",
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "erroredConversionFunc": "Convert_V0_to_V1",
  "dashboardUID": "abc123",
  "sourceSchemaVersion": 16,
  "targetSchemaVersion": 41,
  "erroredSchemaVersionFunc": "V24",
  "errorType": "schema_version_migration_error",
  "error": "migration failed: table panel plugin not found"
}

Minimum Version Error (WARN level)

{
  "level": "warn",
  "msg": "Dashboard conversion failed", 
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "erroredConversionFunc": "Convert_V0_to_V1",
  "dashboardUID": "def456",
  "sourceSchemaVersion": 10,
  "targetSchemaVersion": 41,
  "erroredSchemaVersionFunc": "",
  "errorType": "schema_minimum_version_error",
  "error": "dashboard schema version 10 cannot be migrated"
}

Data Loss Error (ERROR level)

{
  "level": "error",
  "msg": "Dashboard conversion failed",
  "sourceVersionAPI": "dashboard.grafana.app/v1beta1",
  "targetVersionAPI": "dashboard.grafana.app/v2alpha1",
  "erroredConversionFunc": "V1beta1_to_V2alpha1",
  "dashboardUID": "abc123",
  "sourceSchemaVersion": 42,
  "targetSchemaVersion": 42,
  "panelsLost": 0,
  "queriesLost": 2,
  "annotationsLost": 0,
  "linksLost": 0,
  "variablesLost": 0,
  "errorType": "conversion_data_loss_error",
  "error": "data loss detected: query count decreased from 7 to 5"
}

Data Loss Detection

Automatic Runtime Checks:

Every conversion automatically detects data loss by comparing:

• Panel count - Visualization panels (regular + library panels)
• Query count - Data source queries (excludes invalid row panel queries)
• Annotation count - Dashboard-level annotations
• Link count - Navigation links
• Variable count - Template variables (from templating.list in v0/v1, variables in v2)

Detection Logic:

• ✅ Allows additions: Default annotations, enriched data
• ❌ Detects losses: Any decrease in counts triggers conversion_data_loss_error

Testing:

Run comprehensive data loss tests on all conversion test files:

# Test all conversions for data loss
go test ./apps/dashboard/pkg/migration/conversion/... -run TestDataLossDetectionOnAllInputFiles -v

# Test shows detailed panel/query analysis when loss is detected

Implementation: See conversion/conversion_data_loss_detection.go and conversion/README.md for details.

Implementation Details

Automatic instrumentation

All dashboard conversions are automatically instrumented via the withConversionMetrics wrapper function:

// All conversion functions are wrapped automatically
// Includes metrics, logging, and data loss detection
s.AddConversionFunc((*dashv0.Dashboard)(nil), (*dashv1.Dashboard)(nil),
    withConversionMetrics(dashv0.APIVERSION, dashv1.APIVERSION, func(a, b interface{}, scope conversion.Scope) error {
        return Convert_V0_to_V1(a.(*dashv0.Dashboard), b.(*dashv1.Dashboard), scope)
    }))

Error handling

Custom error types provide structured error information:

// Schema migration errors
type MigrationError struct {
    msg            string
    targetVersion  int
    currentVersion int
    functionName   string
}

// API conversion errors  
type ConversionError struct {
    msg               string
    functionName      string
    currentAPIVersion string
    targetAPIVersion  string
}

// Data loss errors are detected when dashboard components (panels, queries, annotations, links, variables) 
// are lost during conversion
type ConversionDataLossError struct {
    functionName     string  // Function where data loss was detected (e.g., "V1_to_V2alpha1")
    message          string  // Detailed error message with loss statistics
    sourceAPIVersion string  // Source API version (e.g., "dashboard.grafana.app/v1beta1")
    targetAPIVersion string  // Target API version (e.g., "dashboard.grafana.app/v2alpha1")
}

Registration

Metrics registration

Metrics must be registered with Prometheus during service initialization:

import "github.com/grafana/grafana/apps/dashboard/pkg/migration"

// Register metrics with Prometheus
migration.RegisterMetrics(prometheusRegistry)

Available metrics

The following metrics are available after registration:

// Success counter
migration.MDashboardConversionSuccessTotal

// Failure counter  
migration.MDashboardConversionFailureTotal

Related Documentation

• Schema Migration Guide - Complete guide for creating new dashboard schema migrations
• V2 to V1 Layout Conversion - How V2 layouts (GridLayout, RowsLayout, TabsLayout, AutoGridLayout) are converted to V1 panel arrays
• PR #110178 - Dashboard migration: Add missing metrics registration