Dashboard migration: Add missing metrics registration (#110178)
This commit is contained in:
Dominik Prokop
@@ -0,0 +1,297 @@
|
||||
# Dashboard migrations
|
||||
|
||||
This document describes the Grafana dashboard migration system, including metrics, logging, and testing infrastructure for dashboard schema migrations and API version conversions.
|
||||
|
||||
## Overview
|
||||
|
||||
## Metrics
|
||||
|
||||
The dashboard migration system now provides comprehensive observability through:
|
||||
- **Prometheus metrics** for tracking conversion success/failure rates and performance
|
||||
- **Structured logging** for debugging and monitoring conversion operations
|
||||
- **Automatic instrumentation** via wrapper functions that eliminate code duplication
|
||||
- **Error classification** to distinguish between different types of migration failures
|
||||
|
||||
### 1. Dashboard conversion success metric
|
||||
|
||||
**Metric Name:** `grafana_dashboard_migration_conversion_success_total`
|
||||
|
||||
**Type:** Counter
|
||||
|
||||
**Description:** Total number of successful dashboard conversions
|
||||
|
||||
**Labels:**
|
||||
- `source_version_api` - Source API version (e.g., "dashboard.grafana.app/v0alpha1")
|
||||
- `target_version_api` - Target API version (e.g., "dashboard.grafana.app/v1beta1")
|
||||
- `source_schema_version` - Source schema version (e.g., "16") - only for v0/v1 dashboards
|
||||
- `target_schema_version` - Target schema version (e.g., "41") - only for v0/v1 dashboards
|
||||
|
||||
**Example:**
|
||||
```prometheus
|
||||
grafana_dashboard_migration_conversion_success_total{
|
||||
source_version_api="dashboard.grafana.app/v0alpha1",
|
||||
target_version_api="dashboard.grafana.app/v1beta1",
|
||||
source_schema_version="16",
|
||||
target_schema_version="41"
|
||||
} 1250
|
||||
```
|
||||
|
||||
### 2. Dashboard conversion failure metric
|
||||
|
||||
**Metric Name:** `grafana_dashboard_migration_conversion_failure_total`
|
||||
|
||||
**Type:** Counter
|
||||
|
||||
**Description:** Total number of failed dashboard conversions
|
||||
|
||||
**Labels:**
|
||||
- `source_version_api` - Source API version
|
||||
- `target_version_api` - Target API version
|
||||
- `source_schema_version` - Source schema version (only for v0/v1 dashboards)
|
||||
- `target_schema_version` - Target schema version (only for v0/v1 dashboards)
|
||||
- `error_type` - Classification of the error (see Error Types section)
|
||||
|
||||
**Example:**
|
||||
```prometheus
|
||||
grafana_dashboard_migration_conversion_failure_total{
|
||||
source_version_api="dashboard.grafana.app/v0alpha1",
|
||||
target_version_api="dashboard.grafana.app/v1beta1",
|
||||
source_schema_version="14",
|
||||
target_schema_version="41",
|
||||
error_type="schema_version_migration_error"
|
||||
} 42
|
||||
```
|
||||
|
||||
## Error types
|
||||
|
||||
The `error_type` label classifies failures into three categories:
|
||||
|
||||
### 1. `conversion_error`
|
||||
- General conversion failures not related to schema migration
|
||||
- API-level conversion issues
|
||||
- Programming errors in conversion functions
|
||||
|
||||
### 2. `schema_version_migration_error`
|
||||
- Failures during individual schema version migrations (v14→v15, v15→v16, etc.)
|
||||
- Schema-specific transformation errors
|
||||
- Data format incompatibilities
|
||||
|
||||
### 3. `schema_minimum_version_error`
|
||||
- Dashboards with schema versions below the minimum supported version (< v13)
|
||||
- These are logged as warnings rather than errors
|
||||
- Indicates dashboards that cannot be migrated automatically
|
||||
|
||||
## Logging
|
||||
|
||||
### Log structure
|
||||
|
||||
All migration logs use structured logging with consistent field names:
|
||||
|
||||
**Base Fields (always present):**
|
||||
- `sourceVersionAPI` - Source API version
|
||||
- `targetVersionAPI` - Target API version
|
||||
- `dashboardUID` - Unique identifier of the dashboard being converted
|
||||
|
||||
**Schema Version Fields (v0/v1 dashboards only):**
|
||||
- `sourceSchemaVersion` - Source schema version number
|
||||
- `targetSchemaVersion` - Target schema version number
|
||||
- `erroredSchemaVersionFunc` - Name of the schema migration function that failed (on error)
|
||||
|
||||
**Error Fields (failures only):**
|
||||
- `errorType` - Same classification as metrics error_type label
|
||||
- `erroredConversionFunc` - Name of the conversion function that failed
|
||||
- `error` - The actual error message
|
||||
|
||||
### Log levels
|
||||
|
||||
#### Success (DEBUG level)
|
||||
```json
|
||||
{
|
||||
"level": "debug",
|
||||
"msg": "Dashboard conversion succeeded",
|
||||
"sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
|
||||
"targetVersionAPI": "dashboard.grafana.app/v1beta1",
|
||||
"dashboardUID": "abc123",
|
||||
"sourceSchemaVersion": 16,
|
||||
"targetSchemaVersion": 41
|
||||
}
|
||||
```
|
||||
|
||||
#### Conversion/Migration Error (ERROR level)
|
||||
```json
|
||||
{
|
||||
"level": "error",
|
||||
"msg": "Dashboard conversion failed",
|
||||
"sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
|
||||
"targetVersionAPI": "dashboard.grafana.app/v1beta1",
|
||||
"erroredConversionFunc": "Convert_V0_to_V1",
|
||||
"dashboardUID": "abc123",
|
||||
"sourceSchemaVersion": 16,
|
||||
"targetSchemaVersion": 41,
|
||||
"erroredSchemaVersionFunc": "V24",
|
||||
"errorType": "schema_version_migration_error",
|
||||
"error": "migration failed: table panel plugin not found"
|
||||
}
|
||||
```
|
||||
|
||||
#### Minimum Version Error (WARN level)
|
||||
```json
|
||||
{
|
||||
"level": "warn",
|
||||
"msg": "Dashboard conversion failed",
|
||||
"sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
|
||||
"targetVersionAPI": "dashboard.grafana.app/v1beta1",
|
||||
"erroredConversionFunc": "Convert_V0_to_V1",
|
||||
"dashboardUID": "def456",
|
||||
"sourceSchemaVersion": 10,
|
||||
"targetSchemaVersion": 41,
|
||||
"erroredSchemaVersionFunc": "",
|
||||
"errorType": "schema_minimum_version_error",
|
||||
"error": "dashboard schema version 10 cannot be migrated"
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation details
|
||||
|
||||
### Automatic instrumentation
|
||||
|
||||
All dashboard conversions are automatically instrumented via the `withConversionMetrics` wrapper function:
|
||||
|
||||
```go
|
||||
// All conversion functions are wrapped automatically
|
||||
s.AddConversionFunc((*dashv0.Dashboard)(nil), (*dashv1.Dashboard)(nil),
|
||||
withConversionMetrics(dashv0.APIVERSION, dashv1.APIVERSION, func(a, b interface{}, scope conversion.Scope) error {
|
||||
return Convert_V0_to_V1(a.(*dashv0.Dashboard), b.(*dashv1.Dashboard), scope)
|
||||
}))
|
||||
```
|
||||
|
||||
### Error handling
|
||||
|
||||
Custom error types provide structured error information:
|
||||
|
||||
```go
|
||||
// Schema migration errors
|
||||
type MigrationError struct {
|
||||
msg string
|
||||
targetVersion int
|
||||
currentVersion int
|
||||
functionName string
|
||||
}
|
||||
|
||||
// API conversion errors
|
||||
type ConversionError struct {
|
||||
msg string
|
||||
functionName string
|
||||
currentAPIVersion string
|
||||
targetAPIVersion string
|
||||
}
|
||||
```
|
||||
|
||||
## Registration
|
||||
|
||||
### Metrics registration
|
||||
|
||||
Metrics must be registered with Prometheus during service initialization:
|
||||
|
||||
```go
|
||||
import "github.com/grafana/grafana/apps/dashboard/pkg/migration"
|
||||
|
||||
// Register metrics with Prometheus
|
||||
migration.RegisterMetrics(prometheusRegistry)
|
||||
```
|
||||
|
||||
### Available metrics
|
||||
|
||||
The following metrics are available after registration:
|
||||
|
||||
```go
|
||||
// Success counter
|
||||
migration.MDashboardConversionSuccessTotal
|
||||
|
||||
// Failure counter
|
||||
migration.MDashboardConversionFailureTotal
|
||||
```
|
||||
|
||||
## Conversion matrix
|
||||
|
||||
The system supports conversions between all dashboard API versions:
|
||||
|
||||
| From ↓ / To → | v0alpha1 | v1beta1 | v2alpha1 | v2beta1 |
|
||||
|---------------|----------|---------|----------|---------|
|
||||
| **v0alpha1** | ✓ | ✓ | ✓ | ✓ |
|
||||
| **v1beta1** | ✓ | ✓ | ✓ | ✓ |
|
||||
| **v2alpha1** | ✓ | ✓ | ✓ | ✓ |
|
||||
| **v2beta1** | ✓ | ✓ | ✓ | ✓ |
|
||||
|
||||
Each conversion path is automatically instrumented with metrics and logging.
|
||||
|
||||
## API versions
|
||||
|
||||
The supported dashboard API versions are:
|
||||
|
||||
- `dashboard.grafana.app/v0alpha1` - Legacy JSON dashboard format
|
||||
- `dashboard.grafana.app/v1beta1` - Migrated JSON dashboard format
|
||||
- `dashboard.grafana.app/v2alpha1` - New structured dashboard format
|
||||
- `dashboard.grafana.app/v2beta1` - Enhanced structured dashboard format
|
||||
|
||||
## Schema versions
|
||||
|
||||
Schema versions (v13-v41) apply only to v0alpha1 and v1beta1 dashboards:
|
||||
|
||||
- **Minimum supported version**: v13
|
||||
- **Latest version**: v41
|
||||
- **Migration path**: Sequential (v13→v14→v15...→v41)
|
||||
|
||||
## Migration testing
|
||||
|
||||
The implementation includes comprehensive test coverage:
|
||||
|
||||
- **Backend tests**: Go migration tests with metrics validation
|
||||
- **Frontend tests**: TypeScript conversion tests
|
||||
- **Integration tests**: End-to-end conversion validation
|
||||
- **Metrics tests**: Prometheus counter validation
|
||||
|
||||
### Backend migration tests
|
||||
|
||||
The backend migration tests validate schema version migrations and API conversions:
|
||||
|
||||
- **Schema migration tests**: Test individual schema version upgrades (v14→v15, v15→v16, etc.)
|
||||
- **Conversion tests**: Test API version conversions with automatic metrics instrumentation
|
||||
- **Test data**: Uses curated test files from `testdata/input/` covering schema versions 14-41
|
||||
- **Metrics validation**: Tests verify that conversion metrics are properly recorded
|
||||
|
||||
**Test execution:**
|
||||
```bash
|
||||
# All backend migration tests
|
||||
go test ./apps/dashboard/pkg/migration/... -v
|
||||
|
||||
# Schema migration tests only
|
||||
go test ./apps/dashboard/pkg/migration/ -v
|
||||
|
||||
# API conversion tests with metrics
|
||||
go test ./apps/dashboard/pkg/migration/conversion/... -v
|
||||
```
|
||||
|
||||
### Frontend migration comparison tests
|
||||
|
||||
The frontend migration comparison tests validate that backend and frontend migration logic produce consistent results:
|
||||
|
||||
- **Test methodology**: Compares backend vs frontend migration outputs through DashboardModel integration
|
||||
- **Dataset coverage**: Tests run against 42 curated test files spanning schema versions 14-41
|
||||
- **Test location**: `public/app/features/dashboard/state/DashboardMigratorToBackend.test.ts`
|
||||
- **Test data**: Located in `apps/dashboard/pkg/migration/testdata/input/` and `testdata/output/`
|
||||
|
||||
**Test execution:**
|
||||
```bash
|
||||
# Frontend migration comparison tests
|
||||
yarn test DashboardMigratorToBackend.test.ts
|
||||
```
|
||||
|
||||
**Test approach:**
|
||||
- **Frontend path**: `jsonInput → DashboardModel → DashboardMigrator → getSaveModelClone()`
|
||||
- **Backend path**: `jsonInput → Backend Migration → backendOutput → DashboardModel → getSaveModelClone()`
|
||||
- **Comparison**: Direct comparison of final migrated states from both paths
|
||||
|
||||
## Related documentation
|
||||
|
||||
- [PR #110178 - Dashboard migration: Add missing metrics registration](https://github.com/grafana/grafana/pull/110178)
|
||||
Reference in New Issue
Block a user