CI Migration: Backport pkg/build (daggerbuild) and GHAs to release-12.0.3 (#107997)
* CI: move grafana-build into pkg/build (#105640) * move grafana-build into pkg/build * Update go.mod/go.sum files after cherry-pick build - Updates from workspace sync after cherry-picking daggerbuild package - Resolves dependency conflicts from pkg/build integration - Required for successful build after CI migration backport * CI Migration: Replace .github with main branch version - Complete replacement of .github directory from main branch - Includes updated workflows, actions, and configuration files - Ensures all CI improvements from main are included - Next step: fix branch-specific issues in separate commits * Fix branch triggers: Update workflows to target release branches - Update 8 workflows to trigger on release-* instead of main - Ensures workflows run on pull requests to release branches - Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator * Fix branch triggers: Update workflows to target both main and release branches - CORRECTED: Include both main and release-* branch patterns - Ensures workflows run on both main branch (original functionality) and release branches (needed for backport) - Previous commit incorrectly removed main branch support - Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator * CI Migration: Update .gitignore to allow OSS wire file to be committed - Remove **/wire_gen.go (allow pkg/server/wire_gen.go to be committed) - Add /pkg/server/enterprise_wire_gen.go (keep enterprise wire file ignored) - This enables the new committed wire files architecture for CI * CI Migration: Add enhanced wire tool with automatic build constraints - Backport enhanced wire tool from main branch with -gen_tags flag - This enables automatic regeneration with proper constraints to prevent conflicts - Wire file contains all required functions: Initialize, InitializeDocumentBuilders, etc. - Solves the missing wire functions issue causing CI failures - Enhanced tool ensures build constraints persist on regeneration * CI Migration: Update Makefile to use enhanced wire tool with build constraints - Update gen-go target to use -gen_tags flag for automatic build constraints - Ensures future engineers get proper build constraints when running 'make gen-go' - Matches main branch approach for consistent wire file generation - Removes dependency on WIRE_TAGS variable and GO_RACE_FLAG for simplicity * Add gen-enterprise-go target to Makefile for release-12.0.3 - Backports gen-enterprise-go target from main branch - Enables enterprise wire file generation in release branches - Part of CI migration backport wire infrastructure fixes - Resolves CI issues where enterprise wire functions were missing * Fix wire tool golden file format for Drone CI compatibility - Update all 43 golden test files to match main branch format - Change go:generate command from github.com/google/wire/cmd/wire to ./pkg/build/wire/cmd/wire/main.go - Remove legacy '// +build !wireinject' constraints - Fixes Drone CI test failures that were blocking OSS PR merge - All wire tests now pass successfully This aligns the backport branch with the golden file format updates that were made in main branch when the wire tool was enhanced. * Update swagger specs to include enterprise APIs - Regenerated api-enterprise-spec.json with enterprise API definitions - Updated api-merged.json with merged OSS and enterprise specs - Regenerated openapi3.json with complete API specifications - Fixes enterprise CI swagger generation validation failures - Enterprise APIs now properly detected and documented * Fix OSS integration tests missing enterprise build tags in Drone CI - Add -tags=enterprise to all integration test steps in lib.star - Fixes test-backend-integration, postgres-integration-tests, and mysql-integration-tests - Regenerate .drone.yml with proper enterprise build tags - Resolves 'server could not find the requested resource' errors for enterprise APIs - Ensures enterprise APIs (querylibrary, reporting, banners, scim) are registered during tests - Aligns OSS Drone CI with GitHub Actions behavior and enterprise repository * Fix CODEOWNERS for release-12.0.3 compatibility - Remove 21 entries that reference files/directories not present in release branch - Resolves File Exist Checker failures in codeowners-validator workflow - Maintains team ownership assignments from main branch for existing files - Lines removed: .air.toml, apps/secret/, apps/iam/, e2e-playwright/, packages/grafana-alerting/, etc. * baldm0mma/ update releasefinder * Add i18n-extract script for release branch compatibility - Add 'i18n-extract': 'make i18n-extract' to package.json scripts - Resolves i18n-verify workflow failure that expects this script - The make target already exists and handles OSS + enterprise i18n extraction - Maintains i18n verification functionality from main branch workflows - Simple safe addition that calls existing make infrastructure * Phase 4: Fix GitHub Actions workflow branch triggers - Add release-*.*.* pattern to 8 workflows missing it - Fix duplicate release branch patterns in 3 workflows - Ensure consistent branch trigger format across all workflows - Critical workflows now properly trigger on release branches Fixes workflows: actionlint, backend-code-checks, go-lint, reject-gh-secrets, run-schema-v2-e2e, shellcheck, swagger-gen, trivy-scan * Fix: Backport E2E runner infrastructure for Enterprise CI - Add e2e/main.go and e2e/internal/ directory from main branch - Add urfave/cli/v3 dependency required by E2E runner - Fixes Enterprise CI failure: 'Build E2E test runner' and 'Build & package Grafana for e2e tests' - Root cause: E2E runner infrastructure was added to main after release-12.0.3 branch creation - Solution: Backport missing E2E runner files to enable Enterprise CI completion Resolves: no Go files in /opt/actions-runner/_work/grafana-enterprise/grafana-enterprise/grafana/e2e * Fix: Update go.mod dependency classification for urfave/cli/v3 - Change urfave/cli/v3 from indirect to direct dependency - Fixes Go Workspace Check failure in CI - Required after adding E2E runner infrastructure that directly imports urfave/cli/v3 - Resolves: make update-workspace corrects dependency classification * Fix: Add team ownership for urfave/cli/v3 dependency - Assign @grafana/grafana-backend-group as owner for github.com/urfave/cli/v3@v3.3.8 - Follows existing pattern: urfave/cli v1 and v2 also owned by grafana-backend-group - Resolves Backend Code Checks / Validate Backend Configs CI failure - Required for E2E runner infrastructure dependency ownership compliance Fixes: modowners check requiring team assignment for newly added dependencies * Fix: Revert experimental E2E playwright infrastructure to stable version - Revert pr-e2e-tests.yml to stable Cypress-based E2E testing - Remove experimental storybook-verification-playwright.yml workflow - Revert run-dashboard-search-e2e.yml and release-pr.yml to use e2e/test-plugins/ - Keep stable E2E runner infrastructure (e2e/main.go + e2e/internal/) - Remove experimental playwright features per team recommendation Team feedback: Playwright tests are experimental and shouldn't be backported to stable release branches * Fix: Complete cleanup of experimental playwright dependencies - Revert package.json to stable version (remove e2e-playwright scripts and path references) - Revert playwright.config.ts to stable plugin-e2e configuration - Remove all experimental playwright infrastructure dependencies - Ensure clean stable E2E testing environment All experimental features removed per team recommendation for stable release branches * Fix: Restore working package.json configuration - Revert package.json to version 12.0.3 (working release branch version) - Fix workspace dependency resolution issues caused by incorrect revert to main branch version - FE tests, betterer, and linting should now work correctly Issue was caused by reverting package.json to main branch (12.1.0-pre) instead of keeping the working release branch configuration (12.0.3) * CI: mirror some CI dependencies (#106148) * mirror some CI dependencies * remove -v from go build * Fix: Backport missing e2e/run-suite script for daggerbuild E2E system The daggerbuild E2E system (used by OSS workflows) expects ./e2e/run-suite to exist, but we only backported the new E2E runner infrastructure. This script is needed for: - OSS workflow: 'go run ./pkg/build/e2e --suite=dashboards-suite' - Legacy Cypress configuration with video support - Integration between daggerbuild and existing Cypress test suites Resolves E2E test failures in OSS workflows where videos directory cannot be found because run-suite script was missing. * Fix: Resolve daggerbuild E2E path doubling issue - Extract just suite name from full path using filepath.Base() - Prevents doubled paths like './e2e/e2e/dashboards-suite/videos' - Resolves 'no spec files found' and 'no such file or directory' errors - GitHub Actions passes full paths like 'e2e/dashboards-suite' but run-suite script expects just 'dashboards-suite' * Infrastructure: Wholesale copy pkg/build/ from main COMPLETE DAGGERBUILD SYSTEM UPDATE: - Modern E2E system with CLI framework and --flags support - New accessibility testing system (a11y/) - New Playwright E2E testing system (e2e-playwright/) - External infrastructure improvements (musl.cc → dl.grafana.com/ci) - Updated daggerbuild core components with latest fixes - Updated Go dependencies and wire modules ARCHITECTURAL COMPATIBILITY VERIFIED: - Modern pkg/build calls: ./e2e-runner cypress --start-grafana=false --cypress-video - Our e2e runner supports: All required parameters including --suite, --env flags - GitHub Actions workflows: Pass compatible arguments - Binary builds: Successfully creates working e2e-runner RESOLVES ISSUES: - Fixes --flags parameter compatibility with GitHub Actions - Includes all follow-up infrastructure improvements - Provides complete, tested system with modern CLI framework - Eliminates external dependency failures (musl.cc timeouts) - Removes need for path doubling workarounds (modern system handles full paths correctly) Replaces incremental cherry-picking with complete, tested system from main. * Dependencies: Update Go modules after pkg/build wholesale copy DEPENDENCY UPDATES: - Updated go.work.sum with new dependencies from modern pkg/build system - Updated all workspace module dependencies (go.mod/go.sum files) - Removed pkg/build/cmd/enterprise.go (enterprise-only file, gets copied during enterprise development) ENTERPRISE INTEGRATION FIX: - enterprise.go file doesn't belong in OSS repository - Gets copied from grafana-enterprise during development mode - Main branch doesn't have this file, explaining module compatibility All Go modules now properly resolved and compatible with modern pkg/build architecture. * Dependencies: Update workspace Go module checksums after pkg/build wholesale copy - Synchronizes all go.sum files across workspace modules - Adds missing .mod.h1 entries that were required after infrastructure update - Resolves Go Workspace Check CI failures - Updates 22 modules: .citools/, apps/, pkg/ subdirectories plus go.work.sum Addresses CI error: 'Please run make update-workspace and commit the changes' * Infrastructure: Revert to pre-Playwright E2E system for release branch compatibility Strategic combination of modern daggerbuild with stable E2E infrastructure: **Modern Daggerbuild (from main):** - Latest CLI framework (github.com/urfave/cli/v3) - External infrastructure fixes (musl.cc → dl.grafana.com/ci) - Complete pkg/build/ system with all enhancements - Updated dependencies and architecture improvements **Stable E2E Infrastructure (pre-Playwright):** - pkg/build/e2e/service.go: Reverted to version before commitb6580ccb10- pkg/build/a11y/: Reverted to stable version compatible with existing infrastructure - Removed experimental pkg/build/e2e-playwright/ system - Removed experimental e2e-playwright/ test infrastructure **Why This Approach:** - GitHub Actions workflows call 'go run ./pkg/build/e2e' (modern system) - Modern system now expects only standard e2e/ directory (compatible) - Release branches have standard e2e/ infrastructure (dashboards-suite, various-suite, etc.) - Eliminates dependency on experimental Playwright infrastructure **Key Dependencies Met:** - e2e/pa11yci.conf.js: Added for a11y test compatibility - /src/e2e directory: Standard directory structure (no e2e-playwright needed) - Existing test plugins: Uses stable e2e/test-plugins/ **Result:** Best of both worlds - modern, reliable daggerbuild infrastructure with proven E2E testing that works across all release branches without requiring experimental dependencies. * Fix: Remove unused embed.go file causing golangci-lint failure - Removes pkg/build/daggerbuild/msi/embed.go with unused embed.FS variable - MSI build process uses directory mounting instead of embedded filesystem - Resolves 'var resources is unused (unused)' linting error * Fix: Update Node.js version to match working E2E configuration - Changes from v22.11.0 to v22.16.0 to match commit17952d45e4- Working commit had all E2E tests passing with same daggerbuild infrastructure - Node.js version differences can affect frontend builds and UI rendering - Ensures consistent environment between working reference and current branch * Fix: Use conditional container base for E2E compatibility - Alpine for OSS tests (better old arch dashboardScene=false compatibility) - Ubuntu for Enterprise tests (when image renderer needed) - Restores original release-12.0.3 Alpine behavior for OSS - Should resolve old arch E2E test failures * Fix: Go fmt formatting for conditional container logic * Revert: Remove unnecessary Alpine/Ubuntu conditional logic - Real issue was wrong environment variable name (dashboardScene vs DISABLE_SCENES) - Keep Ubuntu base (original main branch behavior) for simplicity - Container base was not the root cause of old arch failures * Experiment: Copy scripts/ from pre-Playwright commit (ea0ddb3fc9) - Copied scripts directory from commitea0ddb3fc9(just before Playwright migration) - This overwrites some CI migration fixes but testing if it's actually needed - Can revert back tob9b4602dbdif CI breaks Test commit to see real impact on GitHub Actions workflows. * trigger ci 01 * Fix: Copy .drone.star from main to match scripts structure - Resolves Starlark evaluation error for missing integration_test_pipelines - Aligns .drone.star with main branch scripts structure - Same fix applied to both OSS and Enterprise repositories * make drone * Fix ARM/v7 Docker build failures by using dagger run for cross-compilation - Change artifacts_cmd() in scripts/drone/steps/rgm.star to use 'dagger run go run' instead of 'go run' - Add --max-execution-steps 100000 to make drone target in Makefile to properly regenerate .drone.yml - This fixes CGO cross-compilation issues with SQLite for ARM/v7 Docker builds - Matches the approach already used successfully in Enterprise builds * Fix Docker base image references in rgm build steps - Change from symbolic names (alpine-base, ubuntu-base) to actual Docker images - alpine-base → alpine:3.21.3 - ubuntu-base → ubuntu:22.04 - Add ubuntu and alpine parameters to rgm_artifacts_step and rgm_build_docker_step functions - Fixes Docker pull errors: 'repository does not exist or may require authorization' - Aligns OSS configuration with Enterprise approach --------- Co-authored-by: Kevin Minehart <5140827+kminehart@users.noreply.github.com>
This commit is contained in:
Jev Forsberg
co-authored by
Kevin Minehart
Kevin Minehart
parent
aee6b20c7f
commit
dc6bd2a20a
@@ -2,10 +2,12 @@ package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
@@ -26,6 +28,23 @@ import (
|
||||
|
||||
const grafanaAPI = "https://grafana.com/api"
|
||||
|
||||
var httpClient = http.Client{
|
||||
Transport: &http.Transport{
|
||||
Proxy: http.ProxyFromEnvironment,
|
||||
DialContext: func(dialer *net.Dialer) func(context.Context, string, string) (net.Conn, error) {
|
||||
return dialer.DialContext
|
||||
}(&net.Dialer{
|
||||
Timeout: 30 * time.Second,
|
||||
KeepAlive: 30 * time.Second,
|
||||
}),
|
||||
ForceAttemptHTTP2: true,
|
||||
MaxIdleConns: 100,
|
||||
IdleConnTimeout: 90 * time.Second,
|
||||
TLSHandshakeTimeout: 10 * time.Second,
|
||||
ExpectContinueTimeout: 1 * time.Second,
|
||||
},
|
||||
}
|
||||
|
||||
// GrafanaCom implements the sub-command "grafana-com".
|
||||
func GrafanaCom(c *cli.Context) error {
|
||||
bucketStr := c.String("src-bucket")
|
||||
@@ -330,7 +349,7 @@ func postRequest(cfg packaging.PublishConfig, pth string, body any, descr string
|
||||
return nil
|
||||
}
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
resp, err := httpClient.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed posting to %s (%s): %s", u, descr, err)
|
||||
}
|
||||
|
||||
+131
-135
@@ -4,8 +4,8 @@ import (
|
||||
"log"
|
||||
"os"
|
||||
|
||||
"github.com/grafana/grafana/pkg/build"
|
||||
"github.com/grafana/grafana/pkg/build/cmd/util"
|
||||
"github.com/grafana/grafana/pkg/build/daggerbuild/cmd"
|
||||
"github.com/urfave/cli/v2"
|
||||
)
|
||||
|
||||
@@ -17,15 +17,131 @@ func registerAppCommand(c *cli.Command) {
|
||||
}
|
||||
|
||||
func main() {
|
||||
app := cli.NewApp()
|
||||
app.Commands = cli.Commands{
|
||||
// TODO change the registerer if the user is running using a JSON file etc
|
||||
for k, v := range cmd.Artifacts {
|
||||
if err := cmd.GlobalCLI.Register(k, v); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
app := cmd.GlobalCLI.App()
|
||||
artifactsCommand := cmd.GlobalCLI.ArtifactsCommand()
|
||||
artifactsCommand.Subcommands = cli.Commands{
|
||||
{
|
||||
Name: "build",
|
||||
Action: build.RunCmdCLI,
|
||||
Name: "storybook",
|
||||
Usage: "[ARCHIVED] Publish Grafana storybook",
|
||||
Action: PublishStorybookAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&cli.StringFlag{
|
||||
Name: "storybook-bucket",
|
||||
Value: "grafana-storybook",
|
||||
Usage: "Google Cloud Storage bucket for storybooks",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "static-assets",
|
||||
Usage: "[ARCHIVED] Publish Grafana static assets",
|
||||
Action: PublishStaticAssetsAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&securityFlag,
|
||||
&securityDestBucketFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&destFlag,
|
||||
&cli.StringFlag{
|
||||
Name: "static-assets-bucket",
|
||||
Value: "grafana-static-assets",
|
||||
Usage: "Google Cloud Storage bucket for static assets",
|
||||
},
|
||||
&cli.StringSliceFlag{
|
||||
Name: "static-asset-editions",
|
||||
Usage: "All the editions of the static assets (or $STATIC_ASSET_EDITIONS)",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "packages",
|
||||
Usage: "[ARCHIVED] Publish Grafana packages",
|
||||
Action: PublishArtifactsAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&securityFlag,
|
||||
&securityDestBucketFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&destFlag,
|
||||
&cli.StringSliceFlag{
|
||||
Name: "artifacts-editions",
|
||||
Value: cli.NewStringSlice("oss", "enterprise", "enterprise2"),
|
||||
Usage: "Editions for which the artifacts should be delivered (oss,enterprise,enterprise2), (or $ARTIFACTS_EDITIONS)",
|
||||
},
|
||||
&cli.StringFlag{
|
||||
Name: "enterprise2-dest-bucket",
|
||||
Value: "grafana-downloads-enterprise2",
|
||||
Usage: "Google Cloud Storage bucket for published packages",
|
||||
},
|
||||
&cli.StringFlag{
|
||||
Name: "enterprise2-security-prefix",
|
||||
Usage: "Bucket path prefix for enterprise2 security releases (or $ENTERPRISE2_SECURITY_PREFIX)",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "docker",
|
||||
Usage: "[ARCHIVED] Handle Grafana Docker images",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "fetch",
|
||||
Usage: "Fetch Grafana Docker images",
|
||||
ArgsUsage: "[version]",
|
||||
Action: util.MaxArgCountWrapper(1, FetchImages),
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "npm",
|
||||
Usage: "[ARCHIVED] Handle Grafana npm packages",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "release",
|
||||
Usage: "Release npm packages",
|
||||
ArgsUsage: "[version]",
|
||||
Action: NpmReleaseAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "store",
|
||||
Usage: "Store npm packages tarball",
|
||||
Action: NpmStoreAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "retrieve",
|
||||
Usage: "Retrieve npm packages tarball",
|
||||
Action: NpmRetrieveAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
app.Commands = append(app.Commands, []*cli.Command{
|
||||
{
|
||||
Name: "e2e-tests",
|
||||
Usage: "Run Grafana e2e tests",
|
||||
Usage: "[ARCHIVED] Run Grafana e2e tests",
|
||||
Action: EndToEndTests,
|
||||
Flags: []cli.Flag{
|
||||
&triesFlag,
|
||||
@@ -50,14 +166,9 @@ func main() {
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "whatsnew-checker",
|
||||
Usage: "Checks whatsNewUrl in package.json for differences between the tag and the docs version",
|
||||
Action: WhatsNewChecker,
|
||||
},
|
||||
{
|
||||
Name: "upload-cdn",
|
||||
Usage: "Upload public/* to a cdn bucket",
|
||||
Usage: "[ARCHIVED] Upload public/* to a cdn bucket",
|
||||
Action: UploadCDN,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
@@ -65,18 +176,18 @@ func main() {
|
||||
},
|
||||
{
|
||||
Name: "publish-metrics",
|
||||
Usage: "Publish a set of metrics from stdin",
|
||||
Usage: "[ARCHIVED] Publish a set of metrics from stdin",
|
||||
ArgsUsage: "<api-key>",
|
||||
Action: util.MaxArgCountWrapper(1, PublishMetrics),
|
||||
},
|
||||
{
|
||||
Name: "verify-drone",
|
||||
Usage: "Verify Drone configuration",
|
||||
Usage: "[ARCHIVED] Verify Drone configuration",
|
||||
Action: VerifyDrone,
|
||||
},
|
||||
{
|
||||
Name: "store-storybook",
|
||||
Usage: "Stores storybook to GCS buckets",
|
||||
Usage: "[ARCHIVED] Stores storybook to GCS buckets",
|
||||
Action: StoreStorybook,
|
||||
Flags: []cli.Flag{
|
||||
&cli.StringFlag{
|
||||
@@ -87,12 +198,12 @@ func main() {
|
||||
},
|
||||
{
|
||||
Name: "verify-storybook",
|
||||
Usage: "Integrity check for storybook build",
|
||||
Usage: "[ARCHIVED] Integrity check for storybook build",
|
||||
Action: VerifyStorybook,
|
||||
},
|
||||
{
|
||||
Name: "upload-packages",
|
||||
Usage: "Upload Grafana packages",
|
||||
Usage: "[ARCHIVED] Upload Grafana packages",
|
||||
Action: UploadPackages,
|
||||
Flags: []cli.Flag{
|
||||
&jobsFlag,
|
||||
@@ -103,125 +214,10 @@ func main() {
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "artifacts",
|
||||
Usage: "Handle Grafana artifacts",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "storybook",
|
||||
Usage: "Publish Grafana storybook",
|
||||
Action: PublishStorybookAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&cli.StringFlag{
|
||||
Name: "storybook-bucket",
|
||||
Value: "grafana-storybook",
|
||||
Usage: "Google Cloud Storage bucket for storybooks",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "static-assets",
|
||||
Usage: "Publish Grafana static assets",
|
||||
Action: PublishStaticAssetsAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&securityFlag,
|
||||
&securityDestBucketFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&destFlag,
|
||||
&cli.StringFlag{
|
||||
Name: "static-assets-bucket",
|
||||
Value: "grafana-static-assets",
|
||||
Usage: "Google Cloud Storage bucket for static assets",
|
||||
},
|
||||
&cli.StringSliceFlag{
|
||||
Name: "static-asset-editions",
|
||||
Usage: "All the editions of the static assets (or $STATIC_ASSET_EDITIONS)",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "packages",
|
||||
Usage: "Publish Grafana packages",
|
||||
Action: PublishArtifactsAction,
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
&securityFlag,
|
||||
&securityDestBucketFlag,
|
||||
&tagFlag,
|
||||
&srcFlag,
|
||||
&destFlag,
|
||||
&cli.StringSliceFlag{
|
||||
Name: "artifacts-editions",
|
||||
Value: cli.NewStringSlice("oss", "enterprise", "enterprise2"),
|
||||
Usage: "Editions for which the artifacts should be delivered (oss,enterprise,enterprise2), (or $ARTIFACTS_EDITIONS)",
|
||||
},
|
||||
&cli.StringFlag{
|
||||
Name: "enterprise2-dest-bucket",
|
||||
Value: "grafana-downloads-enterprise2",
|
||||
Usage: "Google Cloud Storage bucket for published packages",
|
||||
},
|
||||
&cli.StringFlag{
|
||||
Name: "enterprise2-security-prefix",
|
||||
Usage: "Bucket path prefix for enterprise2 security releases (or $ENTERPRISE2_SECURITY_PREFIX)",
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "docker",
|
||||
Usage: "Handle Grafana Docker images",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "fetch",
|
||||
Usage: "Fetch Grafana Docker images",
|
||||
ArgsUsage: "[version]",
|
||||
Action: util.MaxArgCountWrapper(1, FetchImages),
|
||||
Flags: []cli.Flag{
|
||||
&editionFlag,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "npm",
|
||||
Usage: "Handle Grafana npm packages",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "release",
|
||||
Usage: "Release npm packages",
|
||||
ArgsUsage: "[version]",
|
||||
Action: NpmReleaseAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "store",
|
||||
Usage: "Store npm packages tarball",
|
||||
Action: NpmStoreAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "retrieve",
|
||||
Usage: "Retrieve npm packages tarball",
|
||||
Action: NpmRetrieveAction,
|
||||
Flags: []cli.Flag{
|
||||
&tagFlag,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
artifactsCommand,
|
||||
{
|
||||
Name: "publish",
|
||||
Usage: "Publish packages to Grafana com and repositories",
|
||||
Usage: "[ARCHIVED] Publish packages to Grafana com and repositories",
|
||||
Subcommands: cli.Commands{
|
||||
{
|
||||
Name: "grafana-com",
|
||||
@@ -292,7 +288,7 @@ func main() {
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
}...)
|
||||
|
||||
app.Commands = append(app.Commands, additionalCommands...)
|
||||
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/grafana/grafana/pkg/build/config"
|
||||
"github.com/urfave/cli/v2"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
const GrafanaDir = "."
|
||||
|
||||
var whatsNewRegex = regexp.MustCompile(`^.*whats-new-in-(v\d*-[\d+]*)`)
|
||||
|
||||
type PackageJSON struct {
|
||||
Grafana Grafana `json:"grafana"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
|
||||
type Grafana struct {
|
||||
WhatsNewUrl string `json:"whatsNewUrl"`
|
||||
}
|
||||
|
||||
func WhatsNewChecker(c *cli.Context) error {
|
||||
metadata, err := config.GenerateMetadata(c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if metadata.ReleaseMode.IsTest {
|
||||
fmt.Println("test mode, skipping check")
|
||||
return nil
|
||||
}
|
||||
if metadata.ReleaseMode.Mode != config.TagMode {
|
||||
return fmt.Errorf("non-tag pipeline, exiting")
|
||||
}
|
||||
|
||||
tag := fmt.Sprintf("v%s", metadata.GrafanaVersion)
|
||||
|
||||
if !semver.IsValid(tag) {
|
||||
return fmt.Errorf("non-semver compatible version %s, exiting", tag)
|
||||
}
|
||||
|
||||
majorMinorDigits := strings.Replace(semver.MajorMinor(tag), ".", "-", 1)
|
||||
|
||||
pkgJSONPath := filepath.Join(GrafanaDir, "package.json")
|
||||
//nolint:gosec
|
||||
pkgJSONB, err := os.ReadFile(pkgJSONPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read %q: %w", pkgJSONPath, err)
|
||||
}
|
||||
|
||||
var pkgObj PackageJSON
|
||||
if err := json.Unmarshal(pkgJSONB, &pkgObj); err != nil {
|
||||
return fmt.Errorf("failed decoding %q: %w", pkgJSONPath, err)
|
||||
}
|
||||
|
||||
whatsNewSplit := whatsNewRegex.FindStringSubmatch(pkgObj.Grafana.WhatsNewUrl)
|
||||
whatsNewVersion := whatsNewSplit[1]
|
||||
|
||||
if whatsNewVersion != majorMinorDigits {
|
||||
return fmt.Errorf("whatsNewUrl in package.json needs to be updated to %s/", strings.Replace(whatsNewSplit[0], whatsNewVersion, majorMinorDigits, 1))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -1,83 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/grafana/grafana/pkg/build/config"
|
||||
"github.com/stretchr/testify/require"
|
||||
"github.com/urfave/cli/v2"
|
||||
)
|
||||
|
||||
const (
|
||||
DroneBuildEvent = "DRONE_BUILD_EVENT"
|
||||
DroneTag = "DRONE_TAG"
|
||||
DroneSemverPrerelease = "DRONE_SEMVER_PRERELEASE"
|
||||
)
|
||||
|
||||
const whatsNewUrl = "https://grafana.com/docs/grafana/next/whatsnew/whats-new-in-"
|
||||
|
||||
func TestWhatsNewChecker(t *testing.T) {
|
||||
tests := []struct {
|
||||
envMap map[string]string
|
||||
packageJsonVersion string
|
||||
name string
|
||||
wantErr bool
|
||||
errMsg string
|
||||
}{
|
||||
{envMap: map[string]string{DroneBuildEvent: config.PullRequest}, packageJsonVersion: "", name: "non-tag event", wantErr: true, errMsg: "non-tag pipeline, exiting"},
|
||||
{envMap: map[string]string{DroneBuildEvent: config.Tag, DroneTag: "abcd123"}, packageJsonVersion: "", name: "non-semver compatible", wantErr: true, errMsg: "non-semver compatible version vabcd123, exiting"},
|
||||
{envMap: map[string]string{DroneBuildEvent: config.Tag, DroneTag: "v0.0.0", DroneSemverPrerelease: "test"}, packageJsonVersion: "v10-0", name: "skip check for test tags", wantErr: false},
|
||||
{envMap: map[string]string{DroneBuildEvent: config.Tag, DroneTag: "v10.0.0"}, packageJsonVersion: "v10-0", name: "package.json version matches tag", wantErr: false},
|
||||
{envMap: map[string]string{DroneBuildEvent: config.Tag, DroneTag: "v10.0.0"}, packageJsonVersion: "v9-5", name: "package.json doesn't match tag", wantErr: true, errMsg: "whatsNewUrl in package.json needs to be updated to https://grafana.com/docs/grafana/next/whatsnew/whats-new-in-v10-0/"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
app := cli.NewApp()
|
||||
app.Version = "1.0.0"
|
||||
context := cli.NewContext(app, &flag.FlagSet{}, nil)
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
setUpEnv(t, tt.envMap)
|
||||
err := createTempPackageJson(t, tt.packageJsonVersion)
|
||||
require.NoError(t, err)
|
||||
|
||||
err = WhatsNewChecker(context)
|
||||
if tt.wantErr {
|
||||
require.Error(t, err)
|
||||
require.Equal(t, tt.errMsg, err.Error())
|
||||
} else {
|
||||
require.NoError(t, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func setUpEnv(t *testing.T, envMap map[string]string) {
|
||||
t.Helper()
|
||||
|
||||
os.Clearenv()
|
||||
t.Setenv("DRONE_BUILD_NUMBER", "12345")
|
||||
t.Setenv("DRONE_COMMIT", "abcd12345")
|
||||
for k, v := range envMap {
|
||||
t.Setenv(k, v)
|
||||
}
|
||||
}
|
||||
|
||||
func createTempPackageJson(t *testing.T, version string) error {
|
||||
t.Helper()
|
||||
|
||||
grafanaData := Grafana{WhatsNewUrl: fmt.Sprintf("%s%s/", whatsNewUrl, version)}
|
||||
data := PackageJSON{Grafana: grafanaData, Version: "1.2.3"}
|
||||
file, _ := json.MarshalIndent(data, "", " ")
|
||||
|
||||
err := os.WriteFile("package.json", file, 0644)
|
||||
require.NoError(t, err)
|
||||
|
||||
t.Cleanup(func() {
|
||||
err := os.RemoveAll("package.json")
|
||||
require.NoError(t, err)
|
||||
})
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user