* CI: move grafana-build into pkg/build (#105640)
* move grafana-build into pkg/build
* Update go.mod/go.sum files after cherry-pick build
- Updates from workspace sync after cherry-picking daggerbuild package
- Resolves dependency conflicts from pkg/build integration
- Required for successful build after CI migration backport
* CI Migration: Replace .github with main branch version
- Complete replacement of .github directory from main branch
- Includes updated workflows, actions, and configuration files
- Ensures all CI improvements from main are included
- Next step: fix branch-specific issues in separate commits
* Fix branch triggers: Update workflows to target release branches
- Update 8 workflows to trigger on release-* instead of main
- Ensures workflows run on pull requests to release branches
- Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator
* Fix branch triggers: Update workflows to target both main and release branches
- CORRECTED: Include both main and release-* branch patterns
- Ensures workflows run on both main branch (original functionality) and release branches (needed for backport)
- Previous commit incorrectly removed main branch support
- Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator
* CI Migration: Update .gitignore to allow OSS wire file to be committed
- Remove **/wire_gen.go (allow pkg/server/wire_gen.go to be committed)
- Add /pkg/server/enterprise_wire_gen.go (keep enterprise wire file ignored)
- This enables the new committed wire files architecture for CI
* CI Migration: Add enhanced wire tool with automatic build constraints
- Backport enhanced wire tool from main branch with -gen_tags flag
- This enables automatic regeneration with proper constraints to prevent conflicts
- Wire file contains all required functions: Initialize, InitializeDocumentBuilders, etc.
- Solves the missing wire functions issue causing CI failures
- Enhanced tool ensures build constraints persist on regeneration
* CI Migration: Update Makefile to use enhanced wire tool with build constraints
- Update gen-go target to use -gen_tags flag for automatic build constraints
- Ensures future engineers get proper build constraints when running 'make gen-go'
- Matches main branch approach for consistent wire file generation
- Removes dependency on WIRE_TAGS variable and GO_RACE_FLAG for simplicity
* Add gen-enterprise-go target to Makefile for release-12.0.3
- Backports gen-enterprise-go target from main branch
- Enables enterprise wire file generation in release branches
- Part of CI migration backport wire infrastructure fixes
- Resolves CI issues where enterprise wire functions were missing
* Fix wire tool golden file format for Drone CI compatibility
- Update all 43 golden test files to match main branch format
- Change go:generate command from github.com/google/wire/cmd/wire to ./pkg/build/wire/cmd/wire/main.go
- Remove legacy '// +build !wireinject' constraints
- Fixes Drone CI test failures that were blocking OSS PR merge
- All wire tests now pass successfully
This aligns the backport branch with the golden file format updates
that were made in main branch when the wire tool was enhanced.
* Update swagger specs to include enterprise APIs
- Regenerated api-enterprise-spec.json with enterprise API definitions
- Updated api-merged.json with merged OSS and enterprise specs
- Regenerated openapi3.json with complete API specifications
- Fixes enterprise CI swagger generation validation failures
- Enterprise APIs now properly detected and documented
* Fix OSS integration tests missing enterprise build tags in Drone CI
- Add -tags=enterprise to all integration test steps in lib.star
- Fixes test-backend-integration, postgres-integration-tests, and mysql-integration-tests
- Regenerate .drone.yml with proper enterprise build tags
- Resolves 'server could not find the requested resource' errors for enterprise APIs
- Ensures enterprise APIs (querylibrary, reporting, banners, scim) are registered during tests
- Aligns OSS Drone CI with GitHub Actions behavior and enterprise repository
* Fix CODEOWNERS for release-12.0.3 compatibility
- Remove 21 entries that reference files/directories not present in release branch
- Resolves File Exist Checker failures in codeowners-validator workflow
- Maintains team ownership assignments from main branch for existing files
- Lines removed: .air.toml, apps/secret/, apps/iam/, e2e-playwright/, packages/grafana-alerting/, etc.
* baldm0mma/ update releasefinder
* Add i18n-extract script for release branch compatibility
- Add 'i18n-extract': 'make i18n-extract' to package.json scripts
- Resolves i18n-verify workflow failure that expects this script
- The make target already exists and handles OSS + enterprise i18n extraction
- Maintains i18n verification functionality from main branch workflows
- Simple safe addition that calls existing make infrastructure
* Phase 4: Fix GitHub Actions workflow branch triggers
- Add release-*.*.* pattern to 8 workflows missing it
- Fix duplicate release branch patterns in 3 workflows
- Ensure consistent branch trigger format across all workflows
- Critical workflows now properly trigger on release branches
Fixes workflows: actionlint, backend-code-checks, go-lint, reject-gh-secrets,
run-schema-v2-e2e, shellcheck, swagger-gen, trivy-scan
* Fix: Backport E2E runner infrastructure for Enterprise CI
- Add e2e/main.go and e2e/internal/ directory from main branch
- Add urfave/cli/v3 dependency required by E2E runner
- Fixes Enterprise CI failure: 'Build E2E test runner' and 'Build & package Grafana for e2e tests'
- Root cause: E2E runner infrastructure was added to main after release-12.0.3 branch creation
- Solution: Backport missing E2E runner files to enable Enterprise CI completion
Resolves: no Go files in /opt/actions-runner/_work/grafana-enterprise/grafana-enterprise/grafana/e2e
* Fix: Update go.mod dependency classification for urfave/cli/v3
- Change urfave/cli/v3 from indirect to direct dependency
- Fixes Go Workspace Check failure in CI
- Required after adding E2E runner infrastructure that directly imports urfave/cli/v3
- Resolves: make update-workspace corrects dependency classification
* Fix: Add team ownership for urfave/cli/v3 dependency
- Assign @grafana/grafana-backend-group as owner for github.com/urfave/cli/v3@v3.3.8
- Follows existing pattern: urfave/cli v1 and v2 also owned by grafana-backend-group
- Resolves Backend Code Checks / Validate Backend Configs CI failure
- Required for E2E runner infrastructure dependency ownership compliance
Fixes: modowners check requiring team assignment for newly added dependencies
* Fix: Revert experimental E2E playwright infrastructure to stable version
- Revert pr-e2e-tests.yml to stable Cypress-based E2E testing
- Remove experimental storybook-verification-playwright.yml workflow
- Revert run-dashboard-search-e2e.yml and release-pr.yml to use e2e/test-plugins/
- Keep stable E2E runner infrastructure (e2e/main.go + e2e/internal/)
- Remove experimental playwright features per team recommendation
Team feedback: Playwright tests are experimental and shouldn't be backported to stable release branches
* Fix: Complete cleanup of experimental playwright dependencies
- Revert package.json to stable version (remove e2e-playwright scripts and path references)
- Revert playwright.config.ts to stable plugin-e2e configuration
- Remove all experimental playwright infrastructure dependencies
- Ensure clean stable E2E testing environment
All experimental features removed per team recommendation for stable release branches
* Fix: Restore working package.json configuration
- Revert package.json to version 12.0.3 (working release branch version)
- Fix workspace dependency resolution issues caused by incorrect revert to main branch version
- FE tests, betterer, and linting should now work correctly
Issue was caused by reverting package.json to main branch (12.1.0-pre) instead of keeping
the working release branch configuration (12.0.3)
* CI: mirror some CI dependencies (#106148)
* mirror some CI dependencies
* remove -v from go build
* Fix: Backport missing e2e/run-suite script for daggerbuild E2E system
The daggerbuild E2E system (used by OSS workflows) expects ./e2e/run-suite
to exist, but we only backported the new E2E runner infrastructure.
This script is needed for:
- OSS workflow: 'go run ./pkg/build/e2e --suite=dashboards-suite'
- Legacy Cypress configuration with video support
- Integration between daggerbuild and existing Cypress test suites
Resolves E2E test failures in OSS workflows where videos directory
cannot be found because run-suite script was missing.
* Fix: Resolve daggerbuild E2E path doubling issue
- Extract just suite name from full path using filepath.Base()
- Prevents doubled paths like './e2e/e2e/dashboards-suite/videos'
- Resolves 'no spec files found' and 'no such file or directory' errors
- GitHub Actions passes full paths like 'e2e/dashboards-suite' but run-suite script expects just 'dashboards-suite'
* Infrastructure: Wholesale copy pkg/build/ from main
COMPLETE DAGGERBUILD SYSTEM UPDATE:
- Modern E2E system with CLI framework and --flags support
- New accessibility testing system (a11y/)
- New Playwright E2E testing system (e2e-playwright/)
- External infrastructure improvements (musl.cc → dl.grafana.com/ci)
- Updated daggerbuild core components with latest fixes
- Updated Go dependencies and wire modules
ARCHITECTURAL COMPATIBILITY VERIFIED:
- Modern pkg/build calls: ./e2e-runner cypress --start-grafana=false --cypress-video
- Our e2e runner supports: All required parameters including --suite, --env flags
- GitHub Actions workflows: Pass compatible arguments
- Binary builds: Successfully creates working e2e-runner
RESOLVES ISSUES:
- Fixes --flags parameter compatibility with GitHub Actions
- Includes all follow-up infrastructure improvements
- Provides complete, tested system with modern CLI framework
- Eliminates external dependency failures (musl.cc timeouts)
- Removes need for path doubling workarounds (modern system handles full paths correctly)
Replaces incremental cherry-picking with complete, tested system from main.
* Dependencies: Update Go modules after pkg/build wholesale copy
DEPENDENCY UPDATES:
- Updated go.work.sum with new dependencies from modern pkg/build system
- Updated all workspace module dependencies (go.mod/go.sum files)
- Removed pkg/build/cmd/enterprise.go (enterprise-only file, gets copied during enterprise development)
ENTERPRISE INTEGRATION FIX:
- enterprise.go file doesn't belong in OSS repository
- Gets copied from grafana-enterprise during development mode
- Main branch doesn't have this file, explaining module compatibility
All Go modules now properly resolved and compatible with modern pkg/build architecture.
* Dependencies: Update workspace Go module checksums after pkg/build wholesale copy
- Synchronizes all go.sum files across workspace modules
- Adds missing .mod.h1 entries that were required after infrastructure update
- Resolves Go Workspace Check CI failures
- Updates 22 modules: .citools/, apps/, pkg/ subdirectories plus go.work.sum
Addresses CI error: 'Please run make update-workspace and commit the changes'
* Infrastructure: Revert to pre-Playwright E2E system for release branch compatibility
Strategic combination of modern daggerbuild with stable E2E infrastructure:
**Modern Daggerbuild (from main):**
- Latest CLI framework (github.com/urfave/cli/v3)
- External infrastructure fixes (musl.cc → dl.grafana.com/ci)
- Complete pkg/build/ system with all enhancements
- Updated dependencies and architecture improvements
**Stable E2E Infrastructure (pre-Playwright):**
- pkg/build/e2e/service.go: Reverted to version before commit b6580ccb10
- pkg/build/a11y/: Reverted to stable version compatible with existing infrastructure
- Removed experimental pkg/build/e2e-playwright/ system
- Removed experimental e2e-playwright/ test infrastructure
**Why This Approach:**
- GitHub Actions workflows call 'go run ./pkg/build/e2e' (modern system)
- Modern system now expects only standard e2e/ directory (compatible)
- Release branches have standard e2e/ infrastructure (dashboards-suite, various-suite, etc.)
- Eliminates dependency on experimental Playwright infrastructure
**Key Dependencies Met:**
- e2e/pa11yci.conf.js: Added for a11y test compatibility
- /src/e2e directory: Standard directory structure (no e2e-playwright needed)
- Existing test plugins: Uses stable e2e/test-plugins/
**Result:**
Best of both worlds - modern, reliable daggerbuild infrastructure with
proven E2E testing that works across all release branches without
requiring experimental dependencies.
* Fix: Remove unused embed.go file causing golangci-lint failure
- Removes pkg/build/daggerbuild/msi/embed.go with unused embed.FS variable
- MSI build process uses directory mounting instead of embedded filesystem
- Resolves 'var resources is unused (unused)' linting error
* Fix: Update Node.js version to match working E2E configuration
- Changes from v22.11.0 to v22.16.0 to match commit 17952d45e4
- Working commit had all E2E tests passing with same daggerbuild infrastructure
- Node.js version differences can affect frontend builds and UI rendering
- Ensures consistent environment between working reference and current branch
* Fix: Use conditional container base for E2E compatibility
- Alpine for OSS tests (better old arch dashboardScene=false compatibility)
- Ubuntu for Enterprise tests (when image renderer needed)
- Restores original release-12.0.3 Alpine behavior for OSS
- Should resolve old arch E2E test failures
* Fix: Go fmt formatting for conditional container logic
* Revert: Remove unnecessary Alpine/Ubuntu conditional logic
- Real issue was wrong environment variable name (dashboardScene vs DISABLE_SCENES)
- Keep Ubuntu base (original main branch behavior) for simplicity
- Container base was not the root cause of old arch failures
* Experiment: Copy scripts/ from pre-Playwright commit (ea0ddb3fc9)
- Copied scripts directory from commit ea0ddb3fc9 (just before Playwright migration)
- This overwrites some CI migration fixes but testing if it's actually needed
- Can revert back to b9b4602dbd if CI breaks
Test commit to see real impact on GitHub Actions workflows.
* trigger ci 01
* Fix: Copy .drone.star from main to match scripts structure
- Resolves Starlark evaluation error for missing integration_test_pipelines
- Aligns .drone.star with main branch scripts structure
- Same fix applied to both OSS and Enterprise repositories
* make drone
* Fix ARM/v7 Docker build failures by using dagger run for cross-compilation
- Change artifacts_cmd() in scripts/drone/steps/rgm.star to use 'dagger run go run' instead of 'go run'
- Add --max-execution-steps 100000 to make drone target in Makefile to properly regenerate .drone.yml
- This fixes CGO cross-compilation issues with SQLite for ARM/v7 Docker builds
- Matches the approach already used successfully in Enterprise builds
* Fix Docker base image references in rgm build steps
- Change from symbolic names (alpine-base, ubuntu-base) to actual Docker images
- alpine-base → alpine:3.21.3
- ubuntu-base → ubuntu:22.04
- Add ubuntu and alpine parameters to rgm_artifacts_step and rgm_build_docker_step functions
- Fixes Docker pull errors: 'repository does not exist or may require authorization'
- Aligns OSS configuration with Enterprise approach
---------
Co-authored-by: Kevin Minehart <5140827+kminehart@users.noreply.github.com>
* Remove the unused `release_publisher` script.
* Remove the "whats new check" in Drone.
* Automatically set the What's New URL in releases based on the tagged version.
* bumping grafanas go version to 1.21.5
* adding how to upgrade go doc
* removing the whole ci-build and corresponding pipelines
* clarifying that we have to run make drone
* fixing starlark linter and removing unused code
* Adding note about enterprise
* trying to change the underyling image to avoid musl issues
* only need to golang one
* chore(node): update node version to 20
* chore(node): bump to 20.8.0, update drone variables
* chore(drone): update yaml config
* chore(yarn): bump yarn to latest to fix building binaries ERR_IMPORT_ASSERTION_TYPE_MISSING errors
* test(grafana-data): update whitespace in time formats for node 20 (18.13+)
* chore(node): move to LTS 20.8.1
* chore(node): bump to LTS 20.9.0
* CI: Add Windows backend tests in more places
* CI: Add promotion pipeline for publishing new windows-test-images
* CI: Ignore windows backend-test failures for now
* CI: Fix linting issue in ci_images.star file
* Add verify-starlark build action that returns an error for starlark files with lint
Relies on `buildifier` tool.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add verify_starlark_step to PR pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Manually fetch buildifier in curl_image until a new build_image is created
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Format with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove all unused variables retaining one unused function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use snake_case for variable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Replace deprecated dictionary concatenation with .update() method
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Start adding docstrings for all modules and functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Attempt to document the behavior of the init_enterprise_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_backend pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise_downstream_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the pipeline utility function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_steps
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise2_pipelines function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add tags table for Starlark files.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_frontend
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document windows function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add docstrings to verifystarlark functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Refactor error handling to be more clear and document complex behavior
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Split errors into execution errors and verification errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document all other library functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add local variables to TAGS
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add blank line between all Args and Returns sections
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix new linting errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Lint new Starlark files
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Correct buildifier binary mv
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the need to set nofile ulimit to at least 2048
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Update build-container to include buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Ensure buildifier binary is executable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix valid content test
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simply return execution error
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Only check files rather than fixing them
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use updated build-container with executable buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Test that context cancellation stops execution
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simplify error handling
Return execution errors that short circuit WalkDir rather than
separately tracking that error.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove fetching of buildifier binary now that it is in the build-container
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use build image in verify-starlark step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use semver tag
The image is the same but uses a semver tag to make it clearer that
this is a forward upgrade from the old version.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use node 18 image with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
---------
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* apt-transport-https is no longer needed (apt has builtin HTTPS support)
apt-transport-https is a virtual package since Ubuntu Bionic as apt
natively supports HTTPS repos since then, see:
https://packages.ubuntu.com/bionic/apt-transport-https
Signed-off-by: Simon Deziel <simon@sdeziel.info>
* Stop pulling software-properties-common (unused)
software-properties-common was used for the add-apt-repository
command that is no longer used since commit 26cf3d9c66
Signed-off-by: Simon Deziel <simon@sdeziel.info>
* Remove makefile (it's not used anymore)
* Directly download and install nodejs package
The Packages file for deb.nodesource.com/node_16.x doesn't list older
versions, so only the most recent version of nodejs is available. We
don't want to require every update to the build container to have to
update nodejs if a never version is available, so I manually download
the deb and install it with dpkg.
* Update Go version
* Update lib.star
* Generate new .drone.yml
* Update Yarn to v3.2.0
* Update sdks as well
* Update nodejs to latest
* Updates
* Update grabpl to 2.9.23
Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
