CI Migration: Backport pkg/build (daggerbuild) and GHAs to release-12.0.3 (#107997)
* CI: move grafana-build into pkg/build (#105640) * move grafana-build into pkg/build * Update go.mod/go.sum files after cherry-pick build - Updates from workspace sync after cherry-picking daggerbuild package - Resolves dependency conflicts from pkg/build integration - Required for successful build after CI migration backport * CI Migration: Replace .github with main branch version - Complete replacement of .github directory from main branch - Includes updated workflows, actions, and configuration files - Ensures all CI improvements from main are included - Next step: fix branch-specific issues in separate commits * Fix branch triggers: Update workflows to target release branches - Update 8 workflows to trigger on release-* instead of main - Ensures workflows run on pull requests to release branches - Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator * Fix branch triggers: Update workflows to target both main and release branches - CORRECTED: Include both main and release-* branch patterns - Ensures workflows run on both main branch (original functionality) and release branches (needed for backport) - Previous commit incorrectly removed main branch support - Workflows updated: documentation-ci, pr-codeql-analysis-*, pr-dependabot-update-go-workspace, pr-go-workspace-check, pr-k8s-codegen-check, verify-kinds, codeowners-validator * CI Migration: Update .gitignore to allow OSS wire file to be committed - Remove **/wire_gen.go (allow pkg/server/wire_gen.go to be committed) - Add /pkg/server/enterprise_wire_gen.go (keep enterprise wire file ignored) - This enables the new committed wire files architecture for CI * CI Migration: Add enhanced wire tool with automatic build constraints - Backport enhanced wire tool from main branch with -gen_tags flag - This enables automatic regeneration with proper constraints to prevent conflicts - Wire file contains all required functions: Initialize, InitializeDocumentBuilders, etc. - Solves the missing wire functions issue causing CI failures - Enhanced tool ensures build constraints persist on regeneration * CI Migration: Update Makefile to use enhanced wire tool with build constraints - Update gen-go target to use -gen_tags flag for automatic build constraints - Ensures future engineers get proper build constraints when running 'make gen-go' - Matches main branch approach for consistent wire file generation - Removes dependency on WIRE_TAGS variable and GO_RACE_FLAG for simplicity * Add gen-enterprise-go target to Makefile for release-12.0.3 - Backports gen-enterprise-go target from main branch - Enables enterprise wire file generation in release branches - Part of CI migration backport wire infrastructure fixes - Resolves CI issues where enterprise wire functions were missing * Fix wire tool golden file format for Drone CI compatibility - Update all 43 golden test files to match main branch format - Change go:generate command from github.com/google/wire/cmd/wire to ./pkg/build/wire/cmd/wire/main.go - Remove legacy '// +build !wireinject' constraints - Fixes Drone CI test failures that were blocking OSS PR merge - All wire tests now pass successfully This aligns the backport branch with the golden file format updates that were made in main branch when the wire tool was enhanced. * Update swagger specs to include enterprise APIs - Regenerated api-enterprise-spec.json with enterprise API definitions - Updated api-merged.json with merged OSS and enterprise specs - Regenerated openapi3.json with complete API specifications - Fixes enterprise CI swagger generation validation failures - Enterprise APIs now properly detected and documented * Fix OSS integration tests missing enterprise build tags in Drone CI - Add -tags=enterprise to all integration test steps in lib.star - Fixes test-backend-integration, postgres-integration-tests, and mysql-integration-tests - Regenerate .drone.yml with proper enterprise build tags - Resolves 'server could not find the requested resource' errors for enterprise APIs - Ensures enterprise APIs (querylibrary, reporting, banners, scim) are registered during tests - Aligns OSS Drone CI with GitHub Actions behavior and enterprise repository * Fix CODEOWNERS for release-12.0.3 compatibility - Remove 21 entries that reference files/directories not present in release branch - Resolves File Exist Checker failures in codeowners-validator workflow - Maintains team ownership assignments from main branch for existing files - Lines removed: .air.toml, apps/secret/, apps/iam/, e2e-playwright/, packages/grafana-alerting/, etc. * baldm0mma/ update releasefinder * Add i18n-extract script for release branch compatibility - Add 'i18n-extract': 'make i18n-extract' to package.json scripts - Resolves i18n-verify workflow failure that expects this script - The make target already exists and handles OSS + enterprise i18n extraction - Maintains i18n verification functionality from main branch workflows - Simple safe addition that calls existing make infrastructure * Phase 4: Fix GitHub Actions workflow branch triggers - Add release-*.*.* pattern to 8 workflows missing it - Fix duplicate release branch patterns in 3 workflows - Ensure consistent branch trigger format across all workflows - Critical workflows now properly trigger on release branches Fixes workflows: actionlint, backend-code-checks, go-lint, reject-gh-secrets, run-schema-v2-e2e, shellcheck, swagger-gen, trivy-scan * Fix: Backport E2E runner infrastructure for Enterprise CI - Add e2e/main.go and e2e/internal/ directory from main branch - Add urfave/cli/v3 dependency required by E2E runner - Fixes Enterprise CI failure: 'Build E2E test runner' and 'Build & package Grafana for e2e tests' - Root cause: E2E runner infrastructure was added to main after release-12.0.3 branch creation - Solution: Backport missing E2E runner files to enable Enterprise CI completion Resolves: no Go files in /opt/actions-runner/_work/grafana-enterprise/grafana-enterprise/grafana/e2e * Fix: Update go.mod dependency classification for urfave/cli/v3 - Change urfave/cli/v3 from indirect to direct dependency - Fixes Go Workspace Check failure in CI - Required after adding E2E runner infrastructure that directly imports urfave/cli/v3 - Resolves: make update-workspace corrects dependency classification * Fix: Add team ownership for urfave/cli/v3 dependency - Assign @grafana/grafana-backend-group as owner for github.com/urfave/cli/v3@v3.3.8 - Follows existing pattern: urfave/cli v1 and v2 also owned by grafana-backend-group - Resolves Backend Code Checks / Validate Backend Configs CI failure - Required for E2E runner infrastructure dependency ownership compliance Fixes: modowners check requiring team assignment for newly added dependencies * Fix: Revert experimental E2E playwright infrastructure to stable version - Revert pr-e2e-tests.yml to stable Cypress-based E2E testing - Remove experimental storybook-verification-playwright.yml workflow - Revert run-dashboard-search-e2e.yml and release-pr.yml to use e2e/test-plugins/ - Keep stable E2E runner infrastructure (e2e/main.go + e2e/internal/) - Remove experimental playwright features per team recommendation Team feedback: Playwright tests are experimental and shouldn't be backported to stable release branches * Fix: Complete cleanup of experimental playwright dependencies - Revert package.json to stable version (remove e2e-playwright scripts and path references) - Revert playwright.config.ts to stable plugin-e2e configuration - Remove all experimental playwright infrastructure dependencies - Ensure clean stable E2E testing environment All experimental features removed per team recommendation for stable release branches * Fix: Restore working package.json configuration - Revert package.json to version 12.0.3 (working release branch version) - Fix workspace dependency resolution issues caused by incorrect revert to main branch version - FE tests, betterer, and linting should now work correctly Issue was caused by reverting package.json to main branch (12.1.0-pre) instead of keeping the working release branch configuration (12.0.3) * CI: mirror some CI dependencies (#106148) * mirror some CI dependencies * remove -v from go build * Fix: Backport missing e2e/run-suite script for daggerbuild E2E system The daggerbuild E2E system (used by OSS workflows) expects ./e2e/run-suite to exist, but we only backported the new E2E runner infrastructure. This script is needed for: - OSS workflow: 'go run ./pkg/build/e2e --suite=dashboards-suite' - Legacy Cypress configuration with video support - Integration between daggerbuild and existing Cypress test suites Resolves E2E test failures in OSS workflows where videos directory cannot be found because run-suite script was missing. * Fix: Resolve daggerbuild E2E path doubling issue - Extract just suite name from full path using filepath.Base() - Prevents doubled paths like './e2e/e2e/dashboards-suite/videos' - Resolves 'no spec files found' and 'no such file or directory' errors - GitHub Actions passes full paths like 'e2e/dashboards-suite' but run-suite script expects just 'dashboards-suite' * Infrastructure: Wholesale copy pkg/build/ from main COMPLETE DAGGERBUILD SYSTEM UPDATE: - Modern E2E system with CLI framework and --flags support - New accessibility testing system (a11y/) - New Playwright E2E testing system (e2e-playwright/) - External infrastructure improvements (musl.cc → dl.grafana.com/ci) - Updated daggerbuild core components with latest fixes - Updated Go dependencies and wire modules ARCHITECTURAL COMPATIBILITY VERIFIED: - Modern pkg/build calls: ./e2e-runner cypress --start-grafana=false --cypress-video - Our e2e runner supports: All required parameters including --suite, --env flags - GitHub Actions workflows: Pass compatible arguments - Binary builds: Successfully creates working e2e-runner RESOLVES ISSUES: - Fixes --flags parameter compatibility with GitHub Actions - Includes all follow-up infrastructure improvements - Provides complete, tested system with modern CLI framework - Eliminates external dependency failures (musl.cc timeouts) - Removes need for path doubling workarounds (modern system handles full paths correctly) Replaces incremental cherry-picking with complete, tested system from main. * Dependencies: Update Go modules after pkg/build wholesale copy DEPENDENCY UPDATES: - Updated go.work.sum with new dependencies from modern pkg/build system - Updated all workspace module dependencies (go.mod/go.sum files) - Removed pkg/build/cmd/enterprise.go (enterprise-only file, gets copied during enterprise development) ENTERPRISE INTEGRATION FIX: - enterprise.go file doesn't belong in OSS repository - Gets copied from grafana-enterprise during development mode - Main branch doesn't have this file, explaining module compatibility All Go modules now properly resolved and compatible with modern pkg/build architecture. * Dependencies: Update workspace Go module checksums after pkg/build wholesale copy - Synchronizes all go.sum files across workspace modules - Adds missing .mod.h1 entries that were required after infrastructure update - Resolves Go Workspace Check CI failures - Updates 22 modules: .citools/, apps/, pkg/ subdirectories plus go.work.sum Addresses CI error: 'Please run make update-workspace and commit the changes' * Infrastructure: Revert to pre-Playwright E2E system for release branch compatibility Strategic combination of modern daggerbuild with stable E2E infrastructure: **Modern Daggerbuild (from main):** - Latest CLI framework (github.com/urfave/cli/v3) - External infrastructure fixes (musl.cc → dl.grafana.com/ci) - Complete pkg/build/ system with all enhancements - Updated dependencies and architecture improvements **Stable E2E Infrastructure (pre-Playwright):** - pkg/build/e2e/service.go: Reverted to version before commitb6580ccb10- pkg/build/a11y/: Reverted to stable version compatible with existing infrastructure - Removed experimental pkg/build/e2e-playwright/ system - Removed experimental e2e-playwright/ test infrastructure **Why This Approach:** - GitHub Actions workflows call 'go run ./pkg/build/e2e' (modern system) - Modern system now expects only standard e2e/ directory (compatible) - Release branches have standard e2e/ infrastructure (dashboards-suite, various-suite, etc.) - Eliminates dependency on experimental Playwright infrastructure **Key Dependencies Met:** - e2e/pa11yci.conf.js: Added for a11y test compatibility - /src/e2e directory: Standard directory structure (no e2e-playwright needed) - Existing test plugins: Uses stable e2e/test-plugins/ **Result:** Best of both worlds - modern, reliable daggerbuild infrastructure with proven E2E testing that works across all release branches without requiring experimental dependencies. * Fix: Remove unused embed.go file causing golangci-lint failure - Removes pkg/build/daggerbuild/msi/embed.go with unused embed.FS variable - MSI build process uses directory mounting instead of embedded filesystem - Resolves 'var resources is unused (unused)' linting error * Fix: Update Node.js version to match working E2E configuration - Changes from v22.11.0 to v22.16.0 to match commit17952d45e4- Working commit had all E2E tests passing with same daggerbuild infrastructure - Node.js version differences can affect frontend builds and UI rendering - Ensures consistent environment between working reference and current branch * Fix: Use conditional container base for E2E compatibility - Alpine for OSS tests (better old arch dashboardScene=false compatibility) - Ubuntu for Enterprise tests (when image renderer needed) - Restores original release-12.0.3 Alpine behavior for OSS - Should resolve old arch E2E test failures * Fix: Go fmt formatting for conditional container logic * Revert: Remove unnecessary Alpine/Ubuntu conditional logic - Real issue was wrong environment variable name (dashboardScene vs DISABLE_SCENES) - Keep Ubuntu base (original main branch behavior) for simplicity - Container base was not the root cause of old arch failures * Experiment: Copy scripts/ from pre-Playwright commit (ea0ddb3fc9) - Copied scripts directory from commitea0ddb3fc9(just before Playwright migration) - This overwrites some CI migration fixes but testing if it's actually needed - Can revert back tob9b4602dbdif CI breaks Test commit to see real impact on GitHub Actions workflows. * trigger ci 01 * Fix: Copy .drone.star from main to match scripts structure - Resolves Starlark evaluation error for missing integration_test_pipelines - Aligns .drone.star with main branch scripts structure - Same fix applied to both OSS and Enterprise repositories * make drone * Fix ARM/v7 Docker build failures by using dagger run for cross-compilation - Change artifacts_cmd() in scripts/drone/steps/rgm.star to use 'dagger run go run' instead of 'go run' - Add --max-execution-steps 100000 to make drone target in Makefile to properly regenerate .drone.yml - This fixes CGO cross-compilation issues with SQLite for ARM/v7 Docker builds - Matches the approach already used successfully in Enterprise builds * Fix Docker base image references in rgm build steps - Change from symbolic names (alpine-base, ubuntu-base) to actual Docker images - alpine-base → alpine:3.21.3 - ubuntu-base → ubuntu:22.04 - Add ubuntu and alpine parameters to rgm_artifacts_step and rgm_build_docker_step functions - Fixes Docker pull errors: 'repository does not exist or may require authorization' - Aligns OSS configuration with Enterprise approach --------- Co-authored-by: Kevin Minehart <5140827+kminehart@users.noreply.github.com>
This commit is contained in:
Jev ForsbergandKevin Minehart authored and 
GitHub committed
1 parent
aee6b20c7f
commit
dc6bd2a20a
432 files changed
+24660
-16787
No files matched your search
@@ -0,0 +1,152 @@
|
||||
name: Build and Package Grafana Enterprise / Pro
|
||||
description: Creates Grafana artifacts using Dagger & `pkg/build/daggerbuild`
|
||||
inputs:
|
||||
artifacts:
|
||||
description: |
|
||||
Comma-delimited list of artifacts to build and package.
|
||||
Artifacts follow a specific format of `{package-type}:{grafana-edition}:{architecture}`.
|
||||
Not every combination of `package-type`, `grafana-edition`, and `architecture` are supported.
|
||||
Examples:
|
||||
* `grafana:linux/amd64:targz`, `grafana:linux/amd64:deb`
|
||||
* `enterprise:linux/arm64:rpm, enterprise:linux/amd64:docker`
|
||||
* `pro:docker:llinux/amd64`
|
||||
required: true
|
||||
type: string
|
||||
grafana-path:
|
||||
description: Path to a clone of the 'grafana' repo
|
||||
default: grafana
|
||||
type: string
|
||||
grafana-enterprise-path:
|
||||
description: Path to a clone of the 'grafana-enterprise' repo
|
||||
default: grafana-enterprise
|
||||
type: string
|
||||
github-token:
|
||||
type: string
|
||||
required: true
|
||||
version:
|
||||
type: string
|
||||
description: The version to embed in the grafana binary, example `v1.2.3`. If not provided, then the value in Grafana's package.json will be used
|
||||
required: true
|
||||
build-id:
|
||||
type: string
|
||||
description: an identifier number which can be traced back to the workflow run.
|
||||
default: ${{github.run_id}}
|
||||
required: false
|
||||
patches-repo:
|
||||
type: string
|
||||
description: Repository to load for patches repo. If empty, patches won't be applied. Must be an HTTPS git URL.
|
||||
required: false
|
||||
default: ""
|
||||
patches-ref:
|
||||
type: string
|
||||
description: git ref in the patches repo to check out.
|
||||
required: false
|
||||
default: main
|
||||
patches-path:
|
||||
type: string
|
||||
description: Path in the repository where `.patch` files can be found.
|
||||
required: false
|
||||
default: main
|
||||
checksum:
|
||||
type: boolean
|
||||
description: If true, then checksums will be produced for each file (with a '.sha256' extension)
|
||||
required: false
|
||||
default: false
|
||||
verify:
|
||||
type: boolean
|
||||
description: If true, then the e2e smoke tests will run to verify the produced artifacts (--verify)
|
||||
required: false
|
||||
default: false
|
||||
output:
|
||||
type: string
|
||||
description: Filename to redirect stdout to. Contains list of packages that were produced
|
||||
default: packages.txt
|
||||
required: false
|
||||
docker-tag-format:
|
||||
type: string
|
||||
default: "{{ .version }}-{{ .arch }}"
|
||||
description: Go template of Docker image tag
|
||||
required: false
|
||||
docker-tag-format-ubuntu:
|
||||
type: string
|
||||
default: "{{ .version }}-ubuntu-{{ .arch }}"
|
||||
description: Go template of Docker image tag
|
||||
required: false
|
||||
docker-org:
|
||||
type: string
|
||||
description: Docker org of produced images
|
||||
default: grafana
|
||||
required: false
|
||||
docker-registry:
|
||||
type: string
|
||||
description: Docker registry of produced images
|
||||
default: docker.io
|
||||
required: false
|
||||
ubuntu-base:
|
||||
type: string
|
||||
default: 'ubuntu:22.04'
|
||||
required: false
|
||||
alpine-base:
|
||||
type: string
|
||||
default: 'alpine:3.22'
|
||||
required: false
|
||||
outputs:
|
||||
dist-dir:
|
||||
description: Directory where artifacts are placed
|
||||
value: ${{ steps.output.outputs.dist_dir }}
|
||||
file:
|
||||
description: Path to file containing list of artifacts produced
|
||||
value: ${{ steps.output.outputs.file }}
|
||||
grafana-commit:
|
||||
description: Commit hash of the HEAD of the grafana repository used to build grafana.
|
||||
value: ${{ steps.output.outputs.grafana_commit }}
|
||||
enterprise-commit:
|
||||
description: Commit hash of the HEAD of the grafana-enterprise repository used to build grafana.
|
||||
value: ${{ steps.output.outputs.enterprise_commit }}
|
||||
version:
|
||||
description: The `grafana` version that was embedded in the binary
|
||||
value: ${{ steps.output.outputs.version }}
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- shell: bash
|
||||
run: | # zizmor: ignore[github-env]
|
||||
echo "GRAFANA_PATH=${{ github.workspace }}/${GRAFANA_DIR}" >> "$GITHUB_ENV"
|
||||
echo "ENTERPRISE_PATH=${{ github.workspace }}/${ENTERPRISE_DIR}" >> "$GITHUB_ENV"
|
||||
env:
|
||||
GB_PATH: ${{ inputs.path }}
|
||||
GRAFANA_DIR: ${{ inputs.grafana-path }}
|
||||
ENTERPRISE_DIR: ${{ inputs.enterprise-path }}
|
||||
- name: Build Grafana Enterprise packages
|
||||
uses: dagger/dagger-for-github@e47aba410ef9bb9ed81a4d2a97df31061e5e842e
|
||||
env:
|
||||
VERSION: ${{ inputs.version }}
|
||||
ARTIFACTS: ${{ inputs.artifacts }}
|
||||
GITHUB_TOKEN: ${{ inputs.github-token }}
|
||||
PATCHES_REPO: ${{ inputs.patches-repo }}
|
||||
PATCHES_REF: ${{ inputs.patches-ref }}
|
||||
PATCHES_PATH: ${{ inputs.patches-path }}
|
||||
BUILD_ID: ${{ inputs.build-id }}
|
||||
OUTFILE: ${{ inputs.output }}
|
||||
DOCKER_ORG: ${{ inputs.docker-org }}
|
||||
DOCKER_REGISTRY: ${{ inputs.docker-registry }}
|
||||
TAG_FORMAT: ${{ inputs.docker-tag-format }}
|
||||
UBUNTU_TAG_FORMAT: ${{ inputs.docker-tag-format-ubuntu }}
|
||||
CHECKSUM: ${{ inputs.checksum }}
|
||||
VERIFY: ${{ inputs.verify }}
|
||||
ALPINE_BASE: ${{ inputs.alpine-base }}
|
||||
UBUNTU_BASE: ${{ inputs.ubuntu-base }}
|
||||
with:
|
||||
verb: run
|
||||
dagger-flags: --verbose=0
|
||||
args: go run -C ${GRAFANA_PATH} ./pkg/build/cmd artifacts --artifacts ${ARTIFACTS} --grafana-dir=${GRAFANA_PATH} --alpine-base=${ALPINE_BASE} --ubuntu-base=${UBUNTU_BASE} --enterprise-dir=${ENTERPRISE_PATH} --version=${VERSION} --patches-repo=${PATCHES_REPO} --patches-ref=${PATCHES_REF} --patches-path=${PATCHES_PATH} --build-id=${BUILD_ID} --tag-format="${TAG_FORMAT}" --ubuntu-tag-format="${UBUNTU_TAG_FORMAT}" --org=${DOCKER_ORG} --registry=${DOCKER_REGISTRY} --checksum=${CHECKSUM} --verify=${VERIFY} > $OUTFILE
|
||||
- id: output
|
||||
shell: bash
|
||||
env:
|
||||
OUTFILE: ${{ inputs.output }}
|
||||
run: |
|
||||
echo "dist_dir=dist" | tee -a $GITHUB_OUTPUT
|
||||
echo "file=${OUTFILE}" | tee -a $GITHUB_OUTPUT
|
||||
echo "grafana_commit=$(git -C ${GRAFANA_PATH} rev-parse HEAD)" | tee -a $GITHUB_OUTPUT
|
||||
echo "enterprise_commit=$(git -C ${ENTERPRISE_PATH} rev-parse HEAD)" | tee -a $GITHUB_OUTPUT
|
||||
echo "version=$(cat ${GRAFANA_BUILD_PATH}/dist/VERSION)" | tee -a $GITHUB_OUTPUT
|
||||
@@ -0,0 +1,141 @@
|
||||
name: Detect changed files
|
||||
description: Detects whether any matching files have changed in the current PR
|
||||
inputs:
|
||||
self:
|
||||
description: The path to the calling workflow (e.g. .github/workflows/backend-unit-tests.yml). It is regarded as any category.
|
||||
required: true
|
||||
outputs:
|
||||
self:
|
||||
description: Whether the calling workflow has changed in any way
|
||||
value: ${{ steps.changed-files.outputs.self_any_changed || 'true' }}
|
||||
backend:
|
||||
description: Whether the backend or self have changed in any way
|
||||
value: ${{ steps.changed-files.outputs.backend_any_changed || 'true' }}
|
||||
frontend:
|
||||
description: Whether the frontend or self has changed in any way
|
||||
value: ${{ steps.changed-files.outputs.frontend_any_changed || 'true' }}
|
||||
e2e:
|
||||
description: Whether the e2e tests or self have changed in any way
|
||||
value: ${{ steps.changed-files.outputs.e2e_any_changed == 'true' ||
|
||||
steps.changed-files.outputs.backend_any_changed == 'true' ||
|
||||
steps.changed-files.outputs.frontend_any_changed == 'true' || 'true' }}
|
||||
dev-tooling:
|
||||
description: Whether the dev tooling or self have changed in any way
|
||||
value: ${{ steps.changed-files.outputs.dev_tooling_any_changed || 'true' }}
|
||||
docs:
|
||||
description: Whether the docs or self have changed in any way
|
||||
value: ${{ steps.changed-files.outputs.docs_any_changed || 'true' }}
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
# Assumption: We've done a checkout with the actions/checkout action.
|
||||
# It must persist credentials to allow the changed-files action to get more history.
|
||||
- name: Detect changes
|
||||
id: changed-files
|
||||
if: github.event_name == 'pull_request'
|
||||
uses: tj-actions/changed-files@ed68ef82c095e0d48ec87eccea555d944a631a4c # v46
|
||||
with:
|
||||
files_yaml: |
|
||||
self:
|
||||
- '.github/actions/change-detection/**'
|
||||
- '${{ inputs.self }}'
|
||||
backend:
|
||||
- '!*.md'
|
||||
- '!docs/**'
|
||||
- '!.github/**'
|
||||
- '.github/actions/setup-enterprise/**'
|
||||
- '.github/actions/checkout/**'
|
||||
- '**/go.mod'
|
||||
- '**/go.sum'
|
||||
- '**.go'
|
||||
- 'pkg/**'
|
||||
- '!pkg/**.md'
|
||||
- 'apps/**'
|
||||
- '!apps/**.md'
|
||||
- 'build.sh'
|
||||
- '.github/actions/change-detection/**'
|
||||
- '**.cue'
|
||||
- 'devenv/docker/blocks/*_tests/**'
|
||||
- 'kindsv2/**'
|
||||
- '${{ inputs.self }}'
|
||||
frontend:
|
||||
- '.github/actions/setup-enterprise/**'
|
||||
- '.github/actions/checkout/**'
|
||||
- 'public/**'
|
||||
- '**.js'
|
||||
- '**.jsx'
|
||||
- '**.ts'
|
||||
- '**.tsx'
|
||||
- '**.css'
|
||||
- '**.mjs'
|
||||
- 'yarn.lock'
|
||||
- 'package.json'
|
||||
- '!**.md'
|
||||
- '.github/actions/change-detection/**'
|
||||
- '**.cue'
|
||||
- '.prettier*'
|
||||
- '.betterer*'
|
||||
- '.yarnrc.yml'
|
||||
- 'eslint.config.js'
|
||||
- 'jest.config.js'
|
||||
- 'nx.json'
|
||||
- 'tsconfig.json'
|
||||
- '.yarn/**'
|
||||
- '${{ inputs.self }}'
|
||||
e2e:
|
||||
- 'e2e/**'
|
||||
- '.github/actions/setup-enterprise/**'
|
||||
- '.github/actions/checkout/**'
|
||||
- 'emails/**'
|
||||
- 'pkg/**'
|
||||
- 'proto/**'
|
||||
- '**/Makefile'
|
||||
- 'scripts/**'
|
||||
- '!scripts/drone/**'
|
||||
- '!**.md'
|
||||
- '.github/actions/change-detection/**'
|
||||
- '**.cue'
|
||||
- 'conf/**'
|
||||
- 'cypress.config.js'
|
||||
- '${{ inputs.self }}'
|
||||
dev_tooling:
|
||||
- '.github/actions/setup-enterprise/**'
|
||||
- '.github/actions/checkout/**'
|
||||
- '**.sh'
|
||||
- '.trivyignore'
|
||||
- '.prettierrc.js'
|
||||
- '**/Makefile'
|
||||
- 'proto/**.yaml'
|
||||
- 'pkg/build/**'
|
||||
- 'pkg/wire/**'
|
||||
- 'scripts/**'
|
||||
- '!**.md'
|
||||
- '.citools/**'
|
||||
- '.bingo/**'
|
||||
- '.github/actions/change-detection/**'
|
||||
- '${{ inputs.self }}'
|
||||
docs:
|
||||
- 'contribute/**'
|
||||
- 'docs/**'
|
||||
- '**.md'
|
||||
- 'LICENSE'
|
||||
- '.vale.ini'
|
||||
- '.github/actions/change-detection/**'
|
||||
- '${{ inputs.self }}'
|
||||
- name: Print all change groups
|
||||
shell: bash
|
||||
run: |
|
||||
echo "Self: ${{ steps.changed-files.outputs.self_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.self_all_changed_files }}"
|
||||
echo "Backend: ${{ steps.changed-files.outputs.backend_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.backend_all_changed_files }}"
|
||||
echo "Frontend: ${{ steps.changed-files.outputs.frontend_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.frontend_all_changed_files }}"
|
||||
echo "E2E: ${{ steps.changed-files.outputs.e2e_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.e2e_all_changed_files }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.backend_all_changed_files }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.frontend_all_changed_files }}"
|
||||
echo "Dev Tooling: ${{ steps.changed-files.outputs.dev_tooling_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.dev_tooling_all_changed_files }}"
|
||||
echo "Docs: ${{ steps.changed-files.outputs.docs_any_changed || 'true' }}"
|
||||
echo " --> ${{ steps.changed-files.outputs.docs_all_changed_files }}"
|
||||
@@ -1,6 +1,7 @@
|
||||
import { appendFileSync, writeFileSync } from 'fs';
|
||||
import { exec as execCallback } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
import {appendFileSync, writeFileSync} from 'fs';
|
||||
import {exec as execCallback} from 'node:child_process';
|
||||
import {promisify} from 'node:util';
|
||||
import {findPreviousVersion, semverParse} from "./semver.js";
|
||||
|
||||
//
|
||||
// Github Action core utils: logging (notice + debug log levels), must escape
|
||||
@@ -9,35 +10,6 @@ import { promisify } from 'node:util';
|
||||
const escapeData = (s) => s.replace(/%/g, '%25').replace(/\r/g, '%0D').replace(/\n/g, '%0A');
|
||||
const LOG = (msg) => console.log(`::notice::${escapeData(msg)}`);
|
||||
|
||||
//
|
||||
// Semver utils: parse, compare, sort etc (using official regexp)
|
||||
// https://regex101.com/r/Ly7O1x/3/
|
||||
//
|
||||
const semverRegExp =
|
||||
/^v?(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$/;
|
||||
|
||||
const semverParse = (tag) => {
|
||||
const m = tag.match(semverRegExp);
|
||||
if (!m) {
|
||||
return;
|
||||
}
|
||||
const [_, major, minor, patch, prerelease] = m;
|
||||
return [+major, +minor, +patch, prerelease, tag];
|
||||
};
|
||||
|
||||
// semverCompare takes two parsed semver tags and comparest them more or less
|
||||
// according to the semver specs
|
||||
const semverCompare = (a, b) => {
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (a[i] !== b[i]) {
|
||||
return a[i] < b[i] ? 1 : -1;
|
||||
}
|
||||
}
|
||||
if (a[3] !== b[3]) {
|
||||
return a[3] < b[3] ? 1 : -1;
|
||||
}
|
||||
return 0;
|
||||
};
|
||||
|
||||
// Using `git tag -l` output find the tag (version) that goes semantically
|
||||
// right before the given version. This might not work correctly with some
|
||||
@@ -45,29 +17,32 @@ const semverCompare = (a, b) => {
|
||||
// into this action explicitly to avoid this step.
|
||||
const getPreviousVersion = async (version) => {
|
||||
const exec = promisify(execCallback);
|
||||
const { stdout } = await exec('git tag -l');
|
||||
const prev = stdout
|
||||
const {stdout} = await exec('git for-each-ref --sort=-creatordate --format \'%(refname:short)\' refs/tags');
|
||||
|
||||
const parsedTags = stdout
|
||||
.split('\n')
|
||||
.map(semverParse)
|
||||
.filter((tag) => tag)
|
||||
.sort(semverCompare)
|
||||
.find((tag) => semverCompare(tag, semverParse(version)) > 0);
|
||||
.filter(Boolean);
|
||||
|
||||
const parsedVersion = semverParse(version);
|
||||
const prev = findPreviousVersion(parsedTags, parsedVersion);
|
||||
if (!prev) {
|
||||
throw `Could not find previous git tag for ${version}`;
|
||||
}
|
||||
return prev[4];
|
||||
return prev[5];
|
||||
};
|
||||
|
||||
|
||||
// A helper for Github GraphQL API endpoint
|
||||
const graphql = async (ghtoken, query, variables) => {
|
||||
const { env } = process;
|
||||
const {env} = process;
|
||||
const results = await fetch('https://api.github.com/graphql', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Authorization: `Bearer ${ghtoken}`,
|
||||
},
|
||||
body: JSON.stringify({ query, variables }),
|
||||
body: JSON.stringify({query, variables}),
|
||||
});
|
||||
|
||||
const res = await results.json();
|
||||
@@ -100,7 +75,7 @@ const getCommitishDate = async (name, owner, target) => {
|
||||
}
|
||||
}
|
||||
`,
|
||||
{ name, owner, target }
|
||||
{name, owner, target}
|
||||
);
|
||||
return result.repository.object.committedDate;
|
||||
};
|
||||
@@ -160,7 +135,7 @@ const getHistory = async (name, owner, from, to) => {
|
||||
|
||||
let cursor;
|
||||
let nodes = [];
|
||||
for (;;) {
|
||||
for (; ;) {
|
||||
const result = await graphql(ghtoken, query, {
|
||||
name,
|
||||
owner,
|
||||
@@ -170,7 +145,7 @@ const getHistory = async (name, owner, from, to) => {
|
||||
});
|
||||
LOG(`GraphQL: ${JSON.stringify(result)}`);
|
||||
nodes = [...nodes, ...result.repository.ref.compare.commits.nodes];
|
||||
const { hasNextPage, endCursor } = result.repository.ref.compare.commits.pageInfo;
|
||||
const {hasNextPage, endCursor} = result.repository.ref.compare.commits.pageInfo;
|
||||
if (!hasNextPage) {
|
||||
break;
|
||||
}
|
||||
@@ -186,7 +161,7 @@ const getHistory = async (name, owner, from, to) => {
|
||||
// PR grouping relies on Github labels only, not on the PR contents.
|
||||
const getChangeLogItems = async (name, owner, from, to) => {
|
||||
// check if a node contains a certain label
|
||||
const hasLabel = ({ labels }, label) => labels.nodes.some(({ name }) => name === label);
|
||||
const hasLabel = ({labels}, label) => labels.nodes.some(({name}) => name === label);
|
||||
// get all the PRs between the two "commitish" items
|
||||
const history = await getHistory(name, owner, from, to);
|
||||
|
||||
@@ -197,17 +172,17 @@ const getChangeLogItems = async (name, owner, from, to) => {
|
||||
return [];
|
||||
}
|
||||
const item = changes[0];
|
||||
const { number, url, labels } = item;
|
||||
const {number, url, labels} = item;
|
||||
const title = item.title.replace(/^\[[^\]]+\]:?\s*/, '');
|
||||
// for changelog PRs try to find a suitable category.
|
||||
// Note that we can not detect "deprecation notices" like that
|
||||
// as there is no suitable label yet.
|
||||
const isBug = /fix/i.test(title) || hasLabel({ labels }, 'type/bug');
|
||||
const isBreaking = hasLabel({ labels }, 'breaking change');
|
||||
const isBug = /fix/i.test(title) || hasLabel({labels}, 'type/bug');
|
||||
const isBreaking = hasLabel({labels}, 'breaking change');
|
||||
const isPlugin =
|
||||
hasLabel({ labels }, 'area/grafana/ui') ||
|
||||
hasLabel({ labels }, 'area/grafana/toolkit') ||
|
||||
hasLabel({ labels }, 'area/grafana/runtime');
|
||||
hasLabel({labels}, 'area/grafana/ui') ||
|
||||
hasLabel({labels}, 'area/grafana/toolkit') ||
|
||||
hasLabel({labels}, 'area/grafana/runtime');
|
||||
const author = item.commits.nodes[0].commit.author.user?.login;
|
||||
return {
|
||||
repo: name,
|
||||
@@ -227,7 +202,7 @@ const getChangeLogItems = async (name, owner, from, to) => {
|
||||
// ======================================================
|
||||
|
||||
LOG(`Changelog action started`);
|
||||
|
||||
console.log(process.argv);
|
||||
const ghtoken = process.env.GITHUB_TOKEN || process.env.INPUT_GITHUB_TOKEN;
|
||||
if (!ghtoken) {
|
||||
throw 'GITHUB_TOKEN is not set and "github_token" input is empty';
|
||||
@@ -286,15 +261,15 @@ const markdown = (changelog) => {
|
||||
: `### ${title}
|
||||
|
||||
${items
|
||||
.map(
|
||||
(item) =>
|
||||
`- ${item.title.replace(/^([^:]*:)/gm, '**$1**')} ${
|
||||
item.repo === 'grafana-enterprise'
|
||||
? '(Enterprise)'
|
||||
: `${pullRequestLink(item.number)}${item.author ? ', ' + userLink(item.author) : ''}`
|
||||
}`
|
||||
)
|
||||
.join('\n')}
|
||||
.map(
|
||||
(item) =>
|
||||
`- ${item.title.replace(/^([^:]*:)/gm, '**$1**')} ${
|
||||
item.repo === 'grafana-enterprise'
|
||||
? '(Enterprise)'
|
||||
: `${pullRequestLink(item.number)}${item.author ? ', ' + userLink(item.author) : ''}`
|
||||
}`
|
||||
)
|
||||
.join('\n')}
|
||||
`;
|
||||
|
||||
// Render all present sections for the given changelog
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
//
|
||||
// Semver utils: parse, compare, sort etc (using official regexp)
|
||||
// https://regex101.com/r/Ly7O1x/3/
|
||||
//
|
||||
const semverRegExp =
|
||||
/^v?(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$/;
|
||||
|
||||
export function semverParse(tag) {
|
||||
const m = tag.match(semverRegExp);
|
||||
if (!m) {
|
||||
return;
|
||||
}
|
||||
const [_, major, minor, patch, prerelease, build] = m;
|
||||
return [+major, +minor, +patch, prerelease, build, tag];
|
||||
};
|
||||
|
||||
// semverCompare takes two parsed semver tags and comparest them more or less
|
||||
// according to the semver specs
|
||||
export function semverCompare(a, b) {
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (a[i] !== b[i]) {
|
||||
return a[i] < b[i] ? 1 : -1;
|
||||
}
|
||||
}
|
||||
if (a[3] !== b[3]) {
|
||||
return a[3] < b[3] ? 1 : -1;
|
||||
}
|
||||
return 0;
|
||||
};
|
||||
|
||||
|
||||
// Finds the highest version that is lower than the target version.
|
||||
//
|
||||
// This function relies on the following invariant: versions are sorted by the release date.
|
||||
// It will produce wrong result if invariant doesn't hold.
|
||||
export const findPreviousVersion = (versionByDate, target) => {
|
||||
let prev = null;
|
||||
|
||||
for (let i = 0; i < versionByDate.length; i++) {
|
||||
const version = versionByDate[i];
|
||||
|
||||
// version is greater than the target
|
||||
if (semverCompare(target, version) > 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// we came across the target version, all versions seen previously have greater release date.
|
||||
if (semverCompare(target, version) === 0 && target[4] === version[4]) {
|
||||
prev = null;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (prev == null) {
|
||||
prev = version;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (semverCompare(prev, version) > 0) {
|
||||
prev = version;
|
||||
}
|
||||
}
|
||||
|
||||
return prev;
|
||||
};
|
||||
|
||||
|
||||
const versionsByDate = [
|
||||
"v10.4.19", "v12.0.1", "v11.6.2", "v11.5.5", "v11.4.5", "v11.3.7", "v11.2.10", "v12.0.0+security-01", "v11.2.9+security-01", "v11.3.6+security-01",
|
||||
"v11.6.1+security-01", "v11.4.4+security-01", "v11.5.4+security-01", "v10.4.18+security-01", "v12.0.0", "v11.6.1",
|
||||
"v11.5.4", "v11.4.4", "v11.3.6", "v11.2.9", "v10.4.18", "v11.6.0+security-01", "v11.5.3+security-01", "v11.4.3+security-01",
|
||||
"v11.3.5+security-01", "v11.2.8+security-01", "v10.4.17+security-01", "v11.2.8", "v11.6.0", "v11.5.2", "v11.4.2",
|
||||
"v11.3.4", "v11.2.7", "v11.1.12", "v11.0.11", "v10.4.16", "v11.5.1", "v11.5.0", "v11.3.3", "v11.1.11", "v11.2.6",
|
||||
"v11.0.10", "v10.4.15", "v11.4.1", "v11.4.0", "v11.3.2", "v11.2.5", "v11.1.10", "v11.0.9", "v10.4.14", "v11.3.1",
|
||||
"v11.2.4", "v11.1.9", "v11.0.8", "v10.4.13", "v11.0.2", "v10.4.6", "v10.3.8", "v10.2.9", "v11.1.0", "v11.0.1",
|
||||
"v10.4.5", "v10.3.7", "v10.2.8", "v9.5.20", "v10.4.4", "v9.5.19", "v10.1.10", "v10.2.7", "v10.3.6", "v10.4.3",
|
||||
"v11.0.0", "v10.4.2", "v11.0.0-preview", "v10.1.9", "v10.0.13", "v9.2.0", "v9.1.8",
|
||||
].map(semverParse);
|
||||
|
||||
function test(version, expected) {
|
||||
const v1 = semverParse(version);
|
||||
const prev = findPreviousVersion(versionsByDate, v1);
|
||||
|
||||
const failureMessage = `FAIILED. Expected ${expected}, but was ${prev[5]}`;
|
||||
|
||||
console.log(`Test ${version}, ${prev[5] === expected ? 'PASSED' : failureMessage}`);
|
||||
}
|
||||
|
||||
test("v11.5.4+security-01", "v11.5.4");
|
||||
test("v11.5.4", "v11.5.3+security-01");
|
||||
test("v12.0.0", "v11.6.1");
|
||||
test("v12.0.0+security-01", "v12.0.0");
|
||||
test("v11.0.0", "v11.0.0-preview");
|
||||
@@ -0,0 +1,48 @@
|
||||
name: Check jobs results
|
||||
description: Checks if any jobs have failed and exits with error if failures are found. Use to check the results of matrix test runs.
|
||||
inputs:
|
||||
needs:
|
||||
description: JSON string containing the needs context from the workflow
|
||||
required: true
|
||||
failure-message:
|
||||
description: Custom message to display when failures are found
|
||||
required: false
|
||||
default: "One or more jobs have failed"
|
||||
success-message:
|
||||
description: Custom message to display when all jobs pass
|
||||
required: false
|
||||
default: "All jobs passed successfully"
|
||||
outputs:
|
||||
any-failed:
|
||||
description: Whether any jobs failed
|
||||
value: ${{ steps.check-jobs.outputs.any-failed }}
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Check test suites
|
||||
id: check-jobs
|
||||
shell: bash
|
||||
env:
|
||||
NEEDS: ${{ inputs.needs }}
|
||||
FAILURE_MSG: ${{ inputs.failure-message }}
|
||||
SUCCESS_MSG: ${{ inputs.success-message }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
# Print the needs context, debugging
|
||||
echo "$NEEDS" | jq
|
||||
|
||||
# Extract failures
|
||||
FAILURES="$(echo "$NEEDS" | jq 'with_entries(select(.value.result == "failure")) | map_values(.result)')"
|
||||
|
||||
# Check if there are any failures
|
||||
if [ "$(echo "$FAILURES" | jq '. | length')" != "0" ]; then
|
||||
echo "❌ $FAILURE_MSG"
|
||||
echo "Failed suites:"
|
||||
echo "$FAILURES" | jq -r 'to_entries[] | "- \(.key): \(.value)"'
|
||||
echo "any-failed=true" >> "$GITHUB_OUTPUT"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✅ $SUCCESS_MSG"
|
||||
@@ -34,9 +34,9 @@ runs:
|
||||
GH_TOKEN: ${{ steps.generate_token.outputs.token }}
|
||||
run: |
|
||||
git clone https://x-access-token:${GH_TOKEN}@github.com/grafana/grafana-enterprise.git ../grafana-enterprise;
|
||||
|
||||
|
||||
cd ../grafana-enterprise
|
||||
|
||||
|
||||
if git checkout ${GITHUB_HEAD_REF}; then
|
||||
echo "checked out ${GITHUB_HEAD_REF}"
|
||||
elif git checkout ${GITHUB_BASE_REF}; then
|
||||
@@ -44,5 +44,5 @@ runs:
|
||||
else
|
||||
git checkout main
|
||||
fi
|
||||
|
||||
./build.sh
|
||||
|
||||
QUIET=1 ./build.sh
|
||||
@@ -1,50 +0,0 @@
|
||||
name: 'Go Coverage Processor'
|
||||
description: 'Process Go test coverage files and generate reports'
|
||||
|
||||
inputs:
|
||||
test-type:
|
||||
description: 'Type of test (e.g., be-unit, be-integration)'
|
||||
required: true
|
||||
type: string
|
||||
coverage-file:
|
||||
description: 'Path to the Go coverage file (.cov)'
|
||||
required: true
|
||||
type: string
|
||||
codecov-token:
|
||||
description: 'Token for CodeCov (required for CodeCov reporting)'
|
||||
required: false
|
||||
default: ''
|
||||
codecov-flag:
|
||||
description: 'Flag to categorize the upload to CodeCov'
|
||||
required: false
|
||||
default: ''
|
||||
codecov-name:
|
||||
description: 'Custom name for the upload to CodeCov'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Process Go coverage output
|
||||
shell: bash
|
||||
env:
|
||||
COVERAGE_FILE: ${{ inputs.coverage-file }}
|
||||
run: |
|
||||
# Ensure valid coverage file even if empty
|
||||
if [ ! -s "$COVERAGE_FILE" ]; then
|
||||
echo "Coverage file is empty, creating a minimal valid file"
|
||||
echo "mode: set" > "$COVERAGE_FILE"
|
||||
fi
|
||||
|
||||
- name: Report coverage to CodeCov
|
||||
uses: codecov/codecov-action@ad3126e916f78f00edff4ed0317cf185271ccc2d # v5
|
||||
if: inputs.codecov-token != ''
|
||||
with:
|
||||
files: ${{ inputs.coverage-file }}
|
||||
flags: ${{ inputs.codecov-flag || inputs.test-type }}
|
||||
name: ${{ inputs.codecov-name || inputs.test-type }}
|
||||
slug: grafana/grafana
|
||||
# This URL doesn't use the Google auth, but is much more locked down. As such, it requires OIDC or a CodeCov-provided token to do anything.
|
||||
url: https://codecov-webhook.grafana-dev.net
|
||||
token: ${{ inputs.codecov-token }}
|
||||
Reference in new issue
Block a user