CI: Lint starlark files with buildifier (#59157)
* Add verify-starlark build action that returns an error for starlark files with lint Relies on `buildifier` tool. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add verify_starlark_step to PR pipeline Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Manually fetch buildifier in curl_image until a new build_image is created Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Format with buildifier Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Remove all unused variables retaining one unused function Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use snake_case for variable Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Replace deprecated dictionary concatenation with .update() method Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Start adding docstrings for all modules and functions Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16 Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Attempt to document the behavior of the init_enterprise_step Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document test_backend pipeline Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document enterprise_downstream_step Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document the pipeline utility function Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document publish_images_step Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document publish_images_steps Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document enterprise2_pipelines function Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add tags table for Starlark files. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document test_frontend Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document windows function Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add docstrings to verifystarlark functions Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Refactor error handling to be more clear and document complex behavior Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Split errors into execution errors and verification errors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document all other library functions Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add local variables to TAGS Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add blank line between all Args and Returns sections Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix new linting errors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Lint new Starlark files Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Correct buildifier binary mv Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Document the need to set nofile ulimit to at least 2048 Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update build-container to include buildifier Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Ensure buildifier binary is executable Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix valid content test Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Simply return execution error Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Only check files rather than fixing them Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use updated build-container with executable buildifier Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Test that context cancellation stops execution Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Simplify error handling Return execution errors that short circuit WalkDir rather than separately tracking that error. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Remove fetching of buildifier binary now that it is in the build-container Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use build image in verify-starlark step Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use semver tag The image is the same but uses a semver tag to make it clearer that this is a forward upgrade from the old version. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use node 18 image with buildifier Signed-off-by: Jack Baldry <jack.baldry@grafana.com> --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
This commit is contained in:
Jack Baldry
GitHub
@@ -1,143 +1,155 @@
|
||||
load(
|
||||
'scripts/drone/utils/utils.star',
|
||||
'pipeline',
|
||||
)
|
||||
"""
|
||||
This module returns all pipelines used in the event of a pull request.
|
||||
It also includes a function generating a PR trigger from a list of included and excluded paths.
|
||||
"""
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/test_frontend.star',
|
||||
'test_frontend',
|
||||
"scripts/drone/pipelines/test_frontend.star",
|
||||
"test_frontend",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/test_backend.star',
|
||||
'test_backend',
|
||||
"scripts/drone/pipelines/test_backend.star",
|
||||
"test_backend",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/integration_tests.star',
|
||||
'integration_tests',
|
||||
"scripts/drone/pipelines/integration_tests.star",
|
||||
"integration_tests",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/build.star',
|
||||
'build_e2e',
|
||||
"scripts/drone/pipelines/build.star",
|
||||
"build_e2e",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/verify_drone.star',
|
||||
'verify_drone',
|
||||
"scripts/drone/pipelines/verify_drone.star",
|
||||
"verify_drone",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/docs.star',
|
||||
'docs_pipelines',
|
||||
'trigger_docs_pr',
|
||||
"scripts/drone/pipelines/verify_starlark.star",
|
||||
"verify_starlark",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/shellcheck.star',
|
||||
'shellcheck_pipeline',
|
||||
"scripts/drone/pipelines/docs.star",
|
||||
"docs_pipelines",
|
||||
"trigger_docs_pr",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/lint_backend.star',
|
||||
'lint_backend_pipeline',
|
||||
"scripts/drone/pipelines/shellcheck.star",
|
||||
"shellcheck_pipeline",
|
||||
)
|
||||
|
||||
load(
|
||||
'scripts/drone/pipelines/lint_frontend.star',
|
||||
'lint_frontend_pipeline',
|
||||
"scripts/drone/pipelines/lint_backend.star",
|
||||
"lint_backend_pipeline",
|
||||
)
|
||||
load(
|
||||
"scripts/drone/pipelines/lint_frontend.star",
|
||||
"lint_frontend_pipeline",
|
||||
)
|
||||
|
||||
ver_mode = 'pr'
|
||||
ver_mode = "pr"
|
||||
trigger = {
|
||||
'event': [
|
||||
'pull_request',
|
||||
"event": [
|
||||
"pull_request",
|
||||
],
|
||||
'paths': {
|
||||
'exclude': [
|
||||
'*.md',
|
||||
'docs/**',
|
||||
'latest.json',
|
||||
"paths": {
|
||||
"exclude": [
|
||||
"*.md",
|
||||
"docs/**",
|
||||
"latest.json",
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def pr_pipelines():
|
||||
return [
|
||||
verify_drone(
|
||||
get_pr_trigger(
|
||||
include_paths=['scripts/drone/**', '.drone.yml', '.drone.star']
|
||||
include_paths = ["scripts/drone/**", ".drone.yml", ".drone.star"],
|
||||
),
|
||||
ver_mode,
|
||||
),
|
||||
verify_starlark(
|
||||
get_pr_trigger(
|
||||
include_paths = ["scripts/drone/**", ".drone.star"],
|
||||
),
|
||||
ver_mode,
|
||||
),
|
||||
test_frontend(
|
||||
get_pr_trigger(
|
||||
exclude_paths=['pkg/**', 'packaging/**', 'go.sum', 'go.mod']
|
||||
exclude_paths = ["pkg/**", "packaging/**", "go.sum", "go.mod"],
|
||||
),
|
||||
ver_mode,
|
||||
committish='${DRONE_COMMIT}',
|
||||
),
|
||||
lint_frontend_pipeline(
|
||||
get_pr_trigger(
|
||||
exclude_paths=['pkg/**', 'packaging/**', 'go.sum', 'go.mod']
|
||||
exclude_paths = ["pkg/**", "packaging/**", "go.sum", "go.mod"],
|
||||
),
|
||||
ver_mode,
|
||||
),
|
||||
test_backend(
|
||||
get_pr_trigger(
|
||||
include_paths=[
|
||||
'pkg/**',
|
||||
'packaging/**',
|
||||
'.drone.yml',
|
||||
'conf/**',
|
||||
'go.sum',
|
||||
'go.mod',
|
||||
'public/app/plugins/**/plugin.json',
|
||||
'devenv/**',
|
||||
]
|
||||
include_paths = [
|
||||
"pkg/**",
|
||||
"packaging/**",
|
||||
".drone.yml",
|
||||
"conf/**",
|
||||
"go.sum",
|
||||
"go.mod",
|
||||
"public/app/plugins/**/plugin.json",
|
||||
"devenv/**",
|
||||
],
|
||||
),
|
||||
ver_mode,
|
||||
committish='${DRONE_COMMIT}',
|
||||
),
|
||||
lint_backend_pipeline(
|
||||
get_pr_trigger(
|
||||
include_paths=[
|
||||
'pkg/**',
|
||||
'packaging/**',
|
||||
'conf/**',
|
||||
'go.sum',
|
||||
'go.mod',
|
||||
'public/app/plugins/**/plugin.json',
|
||||
'devenv/**',
|
||||
'.bingo/**',
|
||||
]
|
||||
include_paths = [
|
||||
"pkg/**",
|
||||
"packaging/**",
|
||||
"conf/**",
|
||||
"go.sum",
|
||||
"go.mod",
|
||||
"public/app/plugins/**/plugin.json",
|
||||
"devenv/**",
|
||||
".bingo/**",
|
||||
],
|
||||
),
|
||||
ver_mode,
|
||||
),
|
||||
build_e2e(trigger, ver_mode),
|
||||
integration_tests(
|
||||
get_pr_trigger(
|
||||
include_paths=[
|
||||
'pkg/**',
|
||||
'packaging/**',
|
||||
'.drone.yml',
|
||||
'conf/**',
|
||||
'go.sum',
|
||||
'go.mod',
|
||||
'public/app/plugins/**/plugin.json',
|
||||
]
|
||||
include_paths = [
|
||||
"pkg/**",
|
||||
"packaging/**",
|
||||
".drone.yml",
|
||||
"conf/**",
|
||||
"go.sum",
|
||||
"go.mod",
|
||||
"public/app/plugins/**/plugin.json",
|
||||
],
|
||||
),
|
||||
prefix=ver_mode,
|
||||
prefix = ver_mode,
|
||||
),
|
||||
docs_pipelines(ver_mode, trigger_docs_pr()),
|
||||
shellcheck_pipeline(),
|
||||
]
|
||||
|
||||
def get_pr_trigger(include_paths = None, exclude_paths = None):
|
||||
"""Generates a trigger filter from the lists of included and excluded path patterns.
|
||||
|
||||
def get_pr_trigger(include_paths=None, exclude_paths=None):
|
||||
paths_ex = ['docs/**', '*.md']
|
||||
This function is primarily intended to generate a trigger for code changes
|
||||
as the patterns 'docs/**' and '*.md' are always excluded.
|
||||
|
||||
Args:
|
||||
include_paths: a list of path patterns using the same syntax as gitignore.
|
||||
Changes affecting files matching these path patterns trigger the pipeline.
|
||||
exclude_paths: a list of path patterns using the same syntax as gitignore.
|
||||
Changes affecting files matching these path patterns do not trigger the pipeline.
|
||||
|
||||
Returns:
|
||||
Drone trigger.
|
||||
"""
|
||||
paths_ex = ["docs/**", "*.md"]
|
||||
paths_in = []
|
||||
if include_paths:
|
||||
for path in include_paths:
|
||||
@@ -146,11 +158,11 @@ def get_pr_trigger(include_paths=None, exclude_paths=None):
|
||||
for path in exclude_paths:
|
||||
paths_ex.extend([path])
|
||||
return {
|
||||
'event': [
|
||||
'pull_request',
|
||||
"event": [
|
||||
"pull_request",
|
||||
],
|
||||
'paths': {
|
||||
'exclude': paths_ex,
|
||||
'include': paths_in,
|
||||
"paths": {
|
||||
"exclude": paths_ex,
|
||||
"include": paths_in,
|
||||
},
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user