Schema: convert dashboards from v1beta1 to v2beta1 (#109037)

- Implement full conversion pipeline from v1beta1 → v2beta1
- Ensure frontend–backend parity for all dashboard serialization paths
- Add automatic data loss detection for conversions (panels, queries, annotations, links, variables)
- Extract atomic conversion functions for v0 → v1beta1 → v2alpha1 → v2beta1
- Introduce conversion metrics and detailed logging for loss tracking
- Normalize datasource resolution, defaults, and annotation processing
- Improve panel layout serialization and y-coordinate normalization
- Fix inconsistencies in nested panels and collapsed row behavior
- Refine variable handling:
  - Filter refId from variable query specs
  - Default variable refresh to 'never' (matches frontend)
  - Fix constant and interval variable handling for missing queries
- Unify schema defaults (enable, hide, iconColor, editable, liveNow)
- Fix pluginId usage (UID vs type) and datasource references
- Fix metrics.go bug swallowing errors (return nil → return err)
- Add tests for version-specific conversion error handling
- Add data loss detection tests using source/target version comparison
- Clean up lint issues, legacy code, and redundant files
- Update OpenAPI snapshots and migrated dashboards
- Improve backend migrator to reuse datasource provider and match frontend logic

Co-authored-by: Haris Rozajac <haris.rozajac12@gmail.com>
Co-authored-by: Oscar Kilhed <oscar.kilhed@grafana.com>
Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

apps/dashboard/pkg/migration/README.md

@@ -198,7 +198,7 @@ grafana_dashboard_migration_conversion_failure_total{
 
 ### Error Types
 
-The `error_type` label classifies failures into three categories:
+The `error_type` label classifies failures into four categories:
 
 #### 1. `conversion_error`
 - General conversion failures not related to schema migration
@@ -215,6 +215,12 @@ The `error_type` label classifies failures into three categories:
 - These are logged as warnings rather than errors
 - Indicates dashboards that cannot be migrated automatically
 
+#### 4. `conversion_data_loss_error`
+- Data loss detected during conversion
+- Automatically checks that panels, queries, annotations, and links are preserved
+- Triggered when target has fewer items than source
+- Includes detailed loss metrics in logs (see [Data Loss Detection](#data-loss-detection))
+
 ### Logging
 
 #### Log structure
@@ -236,6 +242,13 @@ All migration logs use structured logging with consistent field names:
 - `erroredConversionFunc` - Name of the conversion function that failed
 - `error` - The actual error message
 
+**Data Loss Fields (conversion_data_loss_error only):**
+- `panelsLost` - Number of panels lost
+- `queriesLost` - Number of queries lost
+- `annotationsLost` - Number of annotations lost
+- `linksLost` - Number of links lost
+- `variablesLost` - Number of template variables lost
+
 #### Log levels
 
 ##### Success (DEBUG level)
@@ -285,6 +298,55 @@ All migration logs use structured logging with consistent field names:
 }
 ```
 
+##### Data Loss Error (ERROR level)
+```json
+{
+  "level": "error",
+  "msg": "Dashboard conversion failed",
+  "sourceVersionAPI": "dashboard.grafana.app/v1beta1",
+  "targetVersionAPI": "dashboard.grafana.app/v2alpha1",
+  "erroredConversionFunc": "V1beta1_to_V2alpha1",
+  "dashboardUID": "abc123",
+  "sourceSchemaVersion": 42,
+  "targetSchemaVersion": 42,
+  "panelsLost": 0,
+  "queriesLost": 2,
+  "annotationsLost": 0,
+  "linksLost": 0,
+  "variablesLost": 0,
+  "errorType": "conversion_data_loss_error",
+  "error": "data loss detected: query count decreased from 7 to 5"
+}
+```
+
+### Data Loss Detection
+
+**Automatic Runtime Checks:**
+
+Every conversion automatically detects data loss by comparing:
+- **Panel count** - Visualization panels (regular + library panels)
+- **Query count** - Data source queries (excludes invalid row panel queries)
+- **Annotation count** - Dashboard-level annotations
+- **Link count** - Navigation links
+- **Variable count** - Template variables (from `templating.list` in v0/v1, `variables` in v2)
+
+**Detection Logic:**
+- ✅ **Allows additions**: Default annotations, enriched data
+- ❌ **Detects losses**: Any decrease in counts triggers `conversion_data_loss_error`
+
+**Testing:**
+
+Run comprehensive data loss tests on all conversion test files:
+
+```bash
+# Test all conversions for data loss
+go test ./apps/dashboard/pkg/migration/conversion/... -run TestDataLossDetectionOnAllInputFiles -v
+
+# Test shows detailed panel/query analysis when loss is detected
+```
+
+**Implementation:** See `conversion/conversion_data_loss_detection.go` and `conversion/README.md` for details.
+
 ### Implementation Details
 
 #### Automatic instrumentation
@@ -293,6 +355,7 @@ All dashboard conversions are automatically instrumented via the `withConversion
 
 ```go
 // All conversion functions are wrapped automatically
+// Includes metrics, logging, and data loss detection
 s.AddConversionFunc((*dashv0.Dashboard)(nil), (*dashv1.Dashboard)(nil),
     withConversionMetrics(dashv0.APIVERSION, dashv1.APIVERSION, func(a, b interface{}, scope conversion.Scope) error {
         return Convert_V0_to_V1(a.(*dashv0.Dashboard), b.(*dashv1.Dashboard), scope)
@@ -319,6 +382,15 @@ type ConversionError struct {
     currentAPIVersion string
     targetAPIVersion  string
 }
+
+// Data loss errors are detected when dashboard components (panels, queries, annotations, links, variables) 
+// are lost during conversion
+type ConversionDataLossError struct {
+    functionName     string  // Function where data loss was detected (e.g., "V1_to_V2alpha1")
+    message          string  // Detailed error message with loss statistics
+    sourceAPIVersion string  // Source API version (e.g., "dashboard.grafana.app/v1beta1")
+    targetAPIVersion string  // Target API version (e.g., "dashboard.grafana.app/v2alpha1")
+}
 ```
 
 ### Registration