Compare commits

..

26 Commits

Author SHA1 Message Date
Roberto Jimenez Sanchez f49b4df5ff fix: correct import path for FolderReadmeContent 2026-01-13 13:31:23 +01:00
Roberto Jimenez Sanchez 907bea63f2 Provisioning: Add README as a dedicated tab for provisioned folders
Instead of showing README inline on the Dashboards tab, this adds a
dedicated README tab as the first tab for Git Sync provisioned folders.

Changes:
- Add getReadmeTabID and README tab to folder nav model
- Create BrowseFolderReadmePage component
- Add /dashboards/f/:uid/:slug/readme route
- Update FolderReadmeContent to show appropriate empty states
- Remove inline FolderReadme from BrowseDashboardsPage
2026-01-13 13:18:32 +01:00
Roberto Jimenez Sanchez dca2fd6b8d Provisioning: Add README.md support for Git Sync folders
This change enables fetching and rendering README.md files from Git Sync
provisioned folders in the UI. When a user opens a provisioned folder,
the frontend now fetches the README.md from the repository and renders
it as markdown.

Backend changes:
- Add IsReadablePath() to allow .md files for read-only operations
- Add IsRawFile() to detect raw content files (not k8s resources)
- Add handleGetRawFile() to return raw file content without parsing
- Return raw file content in resource.file.content field

Frontend changes:
- Add FolderReadme component to fetch and render README.md
- Integrate component into BrowseDashboardsPage

Testing:
- Unit tests for path validation and raw file reading
- Integration tests for README.md API operations
2026-01-13 12:52:06 +01:00
Oscar Kilhed 91ab753368 Dynamic Dashboards: Fix navigation to repeated panels and update outline when lazy items repeat (#116030)
Dashboard Outline: Fix navigation to repeated panels and lazy-loaded repeats

- Remove cursor: not-allowed styling from repeated panels in outline
- Add RepeatsUpdatedEvent to notify when panel repeats are populated
- Subscribe to RepeatsUpdatedEvent in DashboardEditPane to refresh outline
- Remove memoization from visibleChildren to ensure outline updates on re-render
2026-01-13 08:43:50 +01:00
Alex Khomenko 250ca7985f Provisioning: Add Connections page (#116060)
* Provisioning: Add connections page

* Provisioning: Add connections form

* Provisioning: Add connections form

* Update fields

* Fix generated name

* Update connection name

* Add edit page

* error handling

* Form validation

* Add Connections button

* Cleanup

* Extract ConnectionFormData type

* Add list test and separate empty states

* Add form test

* Update tests

* i18n

* Cleanup

* Use SecretTextArea from grafana-ui

* Fix breadcrumbs

* tweaks

* Add missing URL

* Switch to ShowConfirmModalEvent

* i18n

* redirect to list on success

* add timeout

* Fix tags invalidation
2026-01-13 08:25:40 +02:00
Hugo Häggmark b57ed32484 chore: remove app/core/config barrel files (#116068) 2026-01-13 06:23:21 +01:00
Galen Kistler d0217588a3 LogsDrilldown: Remove exploreLogsLimitedTimeRange flag (#116177)
chore: remove flag
2026-01-12 22:43:01 +00:00
Denis Vodopianov ce9ab6a89a Add non-boolean feature flags support to the StaticProvider (#115085)
* initial commit

* add support of integerts

* finialise the static provider

* minor refactoring

* the rest

* revert:  the rest

* add new thiongs

* more tests added

* add ff parsing tests to check if types are handled correctly

* update tests according to recent changes

* address golint issues

* Update pkg/setting/setting_feature_toggles.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* fix rebase issues

* addressing review comments

* add test cases for enterprise

* handle enterprise cases

* minor refactoring to make api a bit easier to debug

* make test names a bit more precise

* fix linter

* add openfeature sdk to goleak ignore in testutil

* Remove only boolean check in ff gen tests

* add non-boolean types top the doc in default.ini and doc string in FeatureFlag type

* apply remarks, add docs to sample.ini

* reflect changes in feature flags in the public grafana configuration doc

* fix doc formatting

* apply suggestions to the doc file

---------

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
2026-01-12 22:53:23 +01:00
Will Assis 8c8efd2494 unified-storage: skip sqlkv/sqlbackend compatibility tests in sqlite (#116164) 2026-01-12 16:31:29 -05:00
Will Assis 69ccfd6bfc unified-storage: fix sharedwithme search not returning folders (#116089)
* unified-storage: fix dashboard sharedwithme search not returning folders shared with the user
2026-01-12 15:33:34 -05:00
Nick Richmond 53aa5e8f7f MetricsDrilldown: Remove exploreMetricsRelatedLogs feature toggle (#116090)
chore: remove unused exploreMetricsRelatedLogs feature toggle
2026-01-12 12:52:40 -05:00
Ida Štambuk 69bf3068b3 Dashboards: Never show scopes variables (#116132) 2026-01-12 18:52:23 +01:00
Will Assis 1263a3d364 unified-storage: HappyPath and notifier tests + couple of bugfixes (#116087)
* unified-storage: couple of bugfixes and enable HappyPath and notifier sqlkv tests
2026-01-12 12:17:41 -05:00
Daniele Stefano Ferru e4b79e2fc8 Provisioning: Add Validation and Mutation for Connection resource (#115596)
* WIP: mutator added, start working on validator

* first validator iteration

* second validator iteration

* wip: working on integration tests

* re-working mutation and validation, using Connection interface

* fixing some rebase things

* fixing integration tests

* formatting

* fixing unit tests

* k8s codegen

* linting

* moving tests which are available only for enterprise

* addressing comments: using repo config for connections, updating tests

* addressing comments: adding some more info in the app and installation

* fixing app data

* addressing comments: updating connection implementation

* addressing comments

* formatting

* fixing tests
2026-01-12 17:52:00 +01:00
Haris Rozajac 0d1ec94548 Dashboard Schema V2: Activate dashboard ds queries (#116085)
* support dashboard ds

* add test for getPanelDataSource
2026-01-12 08:15:10 -07:00
Jack Westbrook 23a51ec9c5 CI: Fix frontend package validation (#116104)
* ci(frontend-lint): add frontend package change detection and add validate packed packages lint step

* ci(change-detection): add validate-npm-packages.sh to frontend-packages list

* ci(gh-workflows): add actions globs to frontend-packages detection

* ci(gh-workflows): fix typo - > _

* ci(frontend-lint): add missing needs

* chore(i18n): fix publint erroring for custom condition pointing to .cjs file

* ci(validate-npm-packages): make profile node16 default

* chore(validate-npm-packages): remove shellcheck disable comment
2026-01-12 16:08:32 +01:00
Dafydd 51dcdd3499 Datasources: Experimental API group names use full plugin IDs (#112961) 2026-01-12 15:01:40 +00:00
Larissa Wandzura 880bc23c85 Docs: Added a troubleshooting guide for MSSQL, plus some updates (#116088)
* added troubleshooting guide

* cleaned up the intro doc

* cleaned up the before you begin section in the configure doc

* changed a note to a tip

* changed to troubleshooting for cohesion

* final edits

* minor clean up item

* added note about Kerberos not being supported in Cloud

* punctuation fixes
2026-01-12 14:57:45 +00:00
Yunwen Zheng 6dc604c2ea RecentlyViewedDashboards: Add instrumentations (#116036)
RecentlyViewedDashboards: Add instrumentation
2026-01-12 09:56:15 -05:00
Matias Chomicki 77c500dc01 logsExploreTableDefaultVisualization: remove feature flag (#116127) 2026-01-12 14:46:47 +00:00
Kristina Demeshchik bec4d225b3 FieldConfig: Fix multiple value mappings in field overrides being overwritten (#116027)
* multiple value mappingg overrides

* add comment for clarity

* remove extra check
2026-01-12 09:29:16 -05:00
Ivana Huckova b91ca14f48 Icons: Add brain icon (#116023)
* Icons: Add brain icon

* lint

* Add brain to cached icons
2026-01-12 13:38:18 +01:00
Matias Chomicki 2aedbdb76f processing: support duplicated keys when parsing json logs (#116116)
* processing: support duplicated keys when parsing json logs

* Add regression test

* prettier
2026-01-12 13:37:17 +01:00
Rafael Bortolon Paulovic 0d7f46c08a chore(unified): remove unifiedStorageSearch feature toggle (#116109) 2026-01-12 13:22:48 +01:00
Ryan McKinley 1b52718c23 Search: include panel titles and types in index (#115742) 2026-01-12 13:21:03 +01:00
Naimesh Patel e61e406440 Explore: Add keyboard shortcut to run queries (#111675) (#115811)
* Explore: add keyboard shortcut to run queries (#111675)

* Update mock

* Fix linting

---------

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
2026-01-12 13:19:47 +01:00
256 changed files with 9008 additions and 3254 deletions
@@ -14,6 +14,9 @@ outputs:
frontend:
description: Whether the frontend or self has changed in any way
value: ${{ steps.changed-files.outputs.frontend_any_changed || 'true' }}
frontend-packages:
description: Whether any frontend packages have changed
value: ${{ steps.changed-files.outputs.frontend_packages_any_changed || 'true' }}
e2e:
description: Whether the e2e tests or self have changed in any way
value: ${{ steps.changed-files.outputs.e2e_any_changed == 'true' ||
@@ -97,6 +100,12 @@ runs:
- '.yarn/**'
- 'apps/dashboard/pkg/migration/**'
- '${{ inputs.self }}'
frontend_packages:
- '.github/actions/checkout/**'
- '.github/actions/change-detection/**'
- 'packages/**'
- './scripts/validate-npm-packages.sh'
- '${{ inputs.self }}'
e2e:
- 'e2e/**'
- 'e2e-playwright/**'
@@ -153,6 +162,8 @@ runs:
echo " --> ${{ steps.changed-files.outputs.backend_all_changed_files }}"
echo "Frontend: ${{ steps.changed-files.outputs.frontend_any_changed || 'true' }}"
echo " --> ${{ steps.changed-files.outputs.frontend_all_changed_files }}"
echo "Frontend packages: ${{ steps.changed-files.outputs.frontend_packages_any_changed || 'true' }}"
echo " --> ${{ steps.changed-files.outputs.frontend_packages_all_changed_files }}"
echo "E2E: ${{ steps.changed-files.outputs.e2e_any_changed || 'true' }}"
echo " --> ${{ steps.changed-files.outputs.e2e_all_changed_files }}"
echo " --> ${{ steps.changed-files.outputs.backend_all_changed_files }}"
+2 -2
View File
@@ -4,8 +4,8 @@ description: Sets up a node.js environment with presets for the Grafana reposito
runs:
using: "composite"
steps:
- uses: actions/setup-node@v4
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
cache-dependency-path: 'yarn.lock'
+36 -30
View File
@@ -17,6 +17,7 @@ jobs:
outputs:
changed: ${{ steps.detect-changes.outputs.frontend }}
prettier: ${{ steps.detect-changes.outputs.frontend == 'true' || steps.detect-changes.outputs.docs == 'true' }}
changed-frontend-packages: ${{ steps.detect-changes.outputs.frontend-packages }}
steps:
- uses: actions/checkout@v5
with:
@@ -42,11 +43,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- run: yarn install --immutable --check-cache
- run: yarn run prettier:check
- run: yarn run lint
@@ -63,11 +61,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- name: Setup Enterprise
uses: ./.github/actions/setup-enterprise
with:
@@ -89,11 +84,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- run: yarn install --immutable --check-cache
- run: yarn run typecheck
lint-frontend-typecheck-enterprise:
@@ -109,11 +101,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- name: Setup Enterprise
uses: ./.github/actions/setup-enterprise
with:
@@ -133,11 +122,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- run: yarn install --immutable --check-cache
- name: Generate API clients
run: |
@@ -164,11 +150,8 @@ jobs:
- uses: actions/checkout@v5
with:
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'yarn'
cache-dependency-path: 'yarn.lock'
- name: Setup Node
uses: ./.github/actions/setup-node
- name: Setup Enterprise
uses: ./.github/actions/setup-enterprise
with:
@@ -187,3 +170,26 @@ jobs:
echo "${uncommited_error_message}"
exit 1
fi
lint-frontend-packed-packages:
needs: detect-changes
permissions:
contents: read
id-token: write
if: github.event_name == 'pull_request' && needs.detect-changes.outputs.changed-frontend-packages == 'true'
name: Verify packed frontend packages
runs-on: ubuntu-latest
steps:
- name: Checkout build commit
uses: actions/checkout@v5
with:
persist-credentials: false
- name: Setup Node
uses: ./.github/actions/setup-node
- name: Install dependencies
run: yarn install --immutable
- name: Build and pack packages
run: |
yarn run packages:build
yarn run packages:pack
- name: Validate packages
run: ./scripts/validate-npm-packages.sh
@@ -852,6 +852,194 @@
}
}
}
},
"panel-7": {
"kind": "Panel",
"spec": {
"id": 7,
"title": "Single Dashboard DS Query",
"description": "Panel with a single -- Dashboard -- datasource query",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "stat",
"spec": {
"pluginVersion": "12.1.0-pre",
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
},
"panel-8": {
"kind": "Panel",
"spec": {
"id": 8,
"title": "Multiple Dashboard DS Queries",
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "A",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 2,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "B",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 3,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "C",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "stat",
"spec": {
"pluginVersion": "12.1.0-pre",
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
}
},
"layout": {
@@ -914,6 +1102,24 @@
"name": "panel-6"
}
}
},
{
"kind": "AutoGridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-7"
}
}
},
{
"kind": "AutoGridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-8"
}
}
}
]
}
@@ -879,6 +879,200 @@
}
}
}
},
"panel-7": {
"kind": "Panel",
"spec": {
"id": 7,
"title": "Single Dashboard DS Query",
"description": "Panel with a single -- Dashboard -- datasource query",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "stat",
"version": "12.1.0-pre",
"spec": {
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
},
"panel-8": {
"kind": "Panel",
"spec": {
"id": 8,
"title": "Multiple Dashboard DS Queries",
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"refId": "A",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 2,
"withTransforms": true
}
},
"refId": "B",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 3,
"withTransforms": true
}
},
"refId": "C",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "stat",
"version": "12.1.0-pre",
"spec": {
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
}
},
"layout": {
@@ -973,6 +1167,32 @@
"name": "panel-6"
}
}
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 0,
"y": 6,
"width": 8,
"height": 3,
"element": {
"kind": "ElementReference",
"name": "panel-7"
}
}
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 8,
"y": 6,
"width": 8,
"height": 3,
"element": {
"kind": "ElementReference",
"name": "panel-8"
}
}
}
]
}
@@ -711,6 +711,146 @@
],
"title": "Mixed DS WITHOUT REFS",
"type": "timeseries"
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"description": "Panel with a single -- Dashboard -- datasource query",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 9,
"w": 8,
"x": 0,
"y": 18
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
}
],
"title": "Single Dashboard DS Query",
"type": "stat"
},
{
"datasource": {
"type": "mixed",
"uid": "-- Mixed --"
},
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 9,
"w": 8,
"x": 8,
"y": 18
},
"id": 8,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 2,
"refId": "B",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 3,
"refId": "C",
"withTransforms": true
}
],
"title": "Multiple Dashboard DS Queries",
"type": "stat"
}
],
"preload": false,
@@ -711,6 +711,146 @@
],
"title": "Mixed DS WITHOUT REFS",
"type": "timeseries"
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"description": "Panel with a single -- Dashboard -- datasource query",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 9,
"w": 8,
"x": 0,
"y": 18
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
}
],
"title": "Single Dashboard DS Query",
"type": "stat"
},
{
"datasource": {
"type": "mixed",
"uid": "-- Mixed --"
},
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 9,
"w": 8,
"x": 8,
"y": 18
},
"id": 8,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 2,
"refId": "B",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 3,
"refId": "C",
"withTransforms": true
}
],
"title": "Multiple Dashboard DS Queries",
"type": "stat"
}
],
"preload": false,
@@ -879,6 +879,200 @@
}
}
}
},
"panel-7": {
"kind": "Panel",
"spec": {
"id": 7,
"title": "Single Dashboard DS Query",
"description": "Panel with a single -- Dashboard -- datasource query",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "stat",
"version": "12.1.0-pre",
"spec": {
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
},
"panel-8": {
"kind": "Panel",
"spec": {
"id": 8,
"title": "Multiple Dashboard DS Queries",
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"refId": "A",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 2,
"withTransforms": true
}
},
"refId": "B",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "datasource",
"version": "v0",
"datasource": {
"name": "-- Dashboard --"
},
"spec": {
"panelId": 3,
"withTransforms": true
}
},
"refId": "C",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "stat",
"version": "12.1.0-pre",
"spec": {
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
}
},
"layout": {
@@ -941,6 +1135,24 @@
"name": "panel-6"
}
}
},
{
"kind": "AutoGridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-7"
}
}
},
{
"kind": "AutoGridLayoutItem",
"spec": {
"element": {
"kind": "ElementReference",
"name": "panel-8"
}
}
}
]
}
@@ -711,6 +711,146 @@
],
"title": "Mixed DS WITHOUT REFS",
"type": "timeseries"
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"description": "Panel with a single -- Dashboard -- datasource query",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 3,
"w": 8,
"x": 0,
"y": 6
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
}
],
"title": "Single Dashboard DS Query",
"type": "stat"
},
{
"datasource": {
"type": "mixed",
"uid": "-- Mixed --"
},
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 3,
"w": 8,
"x": 8,
"y": 6
},
"id": 8,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 2,
"refId": "B",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 3,
"refId": "C",
"withTransforms": true
}
],
"title": "Multiple Dashboard DS Queries",
"type": "stat"
}
],
"preload": false,
@@ -711,6 +711,146 @@
],
"title": "Mixed DS WITHOUT REFS",
"type": "timeseries"
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"description": "Panel with a single -- Dashboard -- datasource query",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 3,
"w": 8,
"x": 0,
"y": 6
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
}
],
"title": "Single Dashboard DS Query",
"type": "stat"
},
{
"datasource": {
"type": "mixed",
"uid": "-- Mixed --"
},
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": 0
}
]
}
}
},
"gridPos": {
"h": 3,
"w": 8,
"x": 8,
"y": 6
},
"id": 8,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"pluginVersion": "12.1.0-pre",
"targets": [
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 1,
"refId": "A",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 2,
"refId": "B",
"withTransforms": true
},
{
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"panelId": 3,
"refId": "C",
"withTransforms": true
}
],
"title": "Multiple Dashboard DS Queries",
"type": "stat"
}
],
"preload": false,
@@ -852,6 +852,194 @@
}
}
}
},
"panel-7": {
"kind": "Panel",
"spec": {
"id": 7,
"title": "Single Dashboard DS Query",
"description": "Panel with a single -- Dashboard -- datasource query",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "stat",
"spec": {
"pluginVersion": "12.1.0-pre",
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
},
"panel-8": {
"kind": "Panel",
"spec": {
"id": 8,
"title": "Multiple Dashboard DS Queries",
"description": "Panel with multiple -- Dashboard -- datasource queries (should be mixed)",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 1,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "A",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 2,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "B",
"hidden": false
}
},
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "datasource",
"spec": {
"panelId": 3,
"withTransforms": true
}
},
"datasource": {
"type": "datasource",
"uid": "-- Dashboard --"
},
"refId": "C",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "stat",
"spec": {
"pluginVersion": "12.1.0-pre",
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"percentChangeColorMode": "standard",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"showPercentChange": false,
"textMode": "auto",
"wideLayout": true
},
"fieldConfig": {
"defaults": {
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": 0,
"color": "green"
}
]
},
"color": {
"mode": "thresholds"
}
},
"overrides": []
}
}
}
}
}
},
"layout": {
@@ -946,6 +1134,32 @@
"name": "panel-6"
}
}
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 0,
"y": 6,
"width": 8,
"height": 3,
"element": {
"kind": "ElementReference",
"name": "panel-7"
}
}
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 8,
"y": 6,
"width": 8,
"height": 3,
"element": {
"kind": "ElementReference",
"name": "panel-8"
}
}
}
]
}
@@ -1195,16 +1195,36 @@ func getDataSourceForQuery(explicitDS *dashv2alpha1.DashboardDataSourceRef, quer
// getPanelDatasource determines the panel-level datasource for V1.
// Returns:
// - Mixed datasource reference if queries use different datasources
// - Mixed datasource reference if multiple queries use Dashboard datasource (they fetch from different panels)
// - Dashboard datasource reference if a single query uses Dashboard datasource
// - First query's datasource if all queries use the same datasource
// - nil if no queries exist
// Compares based on V2 input without runtime resolution:
// - If query has explicit datasource.uid → use that UID and type
// - Else → use query.Kind as type (empty UID)
func getPanelDatasource(queries []dashv2alpha1.DashboardPanelQueryKind) map[string]interface{} {
const sharedDashboardQuery = "-- Dashboard --"
if len(queries) == 0 {
return nil
}
// Count how many queries use Dashboard datasource
// Multiple dashboard queries need mixed mode because they fetch from different panels
// which may have different underlying datasources
dashboardDsQueryCount := 0
for _, query := range queries {
if query.Spec.Datasource != nil && query.Spec.Datasource.Uid != nil && *query.Spec.Datasource.Uid == sharedDashboardQuery {
dashboardDsQueryCount++
}
}
if dashboardDsQueryCount > 1 {
return map[string]interface{}{
"type": "mixed",
"uid": "-- Mixed --",
}
}
var firstUID, firstType string
var hasFirst bool
@@ -1239,6 +1259,16 @@ func getPanelDatasource(queries []dashv2alpha1.DashboardPanelQueryKind) map[stri
}
}
// Handle case when a single query uses Dashboard datasource.
// This is needed for the frontend to properly activate and fetch data from source panels.
// See DashboardDatasourceBehaviour.tsx for more details.
if firstUID == sharedDashboardQuery {
return map[string]interface{}{
"type": "datasource",
"uid": sharedDashboardQuery,
}
}
// Not mixed - return the first query's datasource so the panel has a datasource set.
// This is required because the frontend's legacy PanelModel.PanelQueryRunner.run uses panel.datasource
// to resolve the datasource, and if undefined, it falls back to the default datasource
@@ -32,7 +32,7 @@ type ConnectionSecure struct {
// Token is the reference of the token used to act as the Connection.
// This value is stored securely and cannot be read back
Token common.InlineSecureValue `json:"webhook,omitzero,omitempty"`
Token common.InlineSecureValue `json:"token,omitzero,omitempty"`
}
func (v ConnectionSecure) IsZero() bool {
@@ -320,7 +320,7 @@ func schema_pkg_apis_provisioning_v0alpha1_ConnectionSecure(ref common.Reference
Ref: ref("github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1.InlineSecureValue"),
},
},
"webhook": {
"token": {
SchemaProps: spec.SchemaProps{
Description: "Token is the reference of the token used to act as the Connection. This value is stored securely and cannot be read back",
Default: map[string]interface{}{},
@@ -22,7 +22,6 @@ API rule violation: list_type_missing,github.com/grafana/grafana/apps/provisioni
API rule violation: list_type_missing,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,ResourceList,Items
API rule violation: list_type_missing,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,TestResults,Errors
API rule violation: list_type_missing,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,WebhookStatus,SubscribedEvents
API rule violation: names_match,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,ConnectionSecure,Token
API rule violation: names_match,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,ConnectionSpec,GitHub
API rule violation: names_match,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,JobSpec,PullRequest
API rule violation: names_match,github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1,JobStatus,URLs
@@ -0,0 +1,16 @@
package connection
import (
"context"
)
//go:generate mockery --name Connection --structname MockConnection --inpackage --filename connection_mock.go --with-expecter
type Connection interface {
// Validate ensures the resource _looks_ correct.
// It should be called before trying to upsert a resource into the Kubernetes API server.
// This is not an indication that the connection information works, just that they are reasonably configured.
Validate(ctx context.Context) error
// Mutate performs in place mutation of the underneath resource.
Mutate(context.Context) error
}
@@ -0,0 +1,128 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package connection
import (
context "context"
mock "github.com/stretchr/testify/mock"
)
// MockConnection is an autogenerated mock type for the Connection type
type MockConnection struct {
mock.Mock
}
type MockConnection_Expecter struct {
mock *mock.Mock
}
func (_m *MockConnection) EXPECT() *MockConnection_Expecter {
return &MockConnection_Expecter{mock: &_m.Mock}
}
// Mutate provides a mock function with given fields: _a0
func (_m *MockConnection) Mutate(_a0 context.Context) error {
ret := _m.Called(_a0)
if len(ret) == 0 {
panic("no return value specified for Mutate")
}
var r0 error
if rf, ok := ret.Get(0).(func(context.Context) error); ok {
r0 = rf(_a0)
} else {
r0 = ret.Error(0)
}
return r0
}
// MockConnection_Mutate_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Mutate'
type MockConnection_Mutate_Call struct {
*mock.Call
}
// Mutate is a helper method to define mock.On call
// - _a0 context.Context
func (_e *MockConnection_Expecter) Mutate(_a0 interface{}) *MockConnection_Mutate_Call {
return &MockConnection_Mutate_Call{Call: _e.mock.On("Mutate", _a0)}
}
func (_c *MockConnection_Mutate_Call) Run(run func(_a0 context.Context)) *MockConnection_Mutate_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context))
})
return _c
}
func (_c *MockConnection_Mutate_Call) Return(_a0 error) *MockConnection_Mutate_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *MockConnection_Mutate_Call) RunAndReturn(run func(context.Context) error) *MockConnection_Mutate_Call {
_c.Call.Return(run)
return _c
}
// Validate provides a mock function with given fields: ctx
func (_m *MockConnection) Validate(ctx context.Context) error {
ret := _m.Called(ctx)
if len(ret) == 0 {
panic("no return value specified for Validate")
}
var r0 error
if rf, ok := ret.Get(0).(func(context.Context) error); ok {
r0 = rf(ctx)
} else {
r0 = ret.Error(0)
}
return r0
}
// MockConnection_Validate_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Validate'
type MockConnection_Validate_Call struct {
*mock.Call
}
// Validate is a helper method to define mock.On call
// - ctx context.Context
func (_e *MockConnection_Expecter) Validate(ctx interface{}) *MockConnection_Validate_Call {
return &MockConnection_Validate_Call{Call: _e.mock.On("Validate", ctx)}
}
func (_c *MockConnection_Validate_Call) Run(run func(ctx context.Context)) *MockConnection_Validate_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context))
})
return _c
}
func (_c *MockConnection_Validate_Call) Return(_a0 error) *MockConnection_Validate_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *MockConnection_Validate_Call) RunAndReturn(run func(context.Context) error) *MockConnection_Validate_Call {
_c.Call.Return(run)
return _c
}
// NewMockConnection creates a new instance of MockConnection. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockConnection(t interface {
mock.TestingT
Cleanup(func())
}) *MockConnection {
mock := &MockConnection{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -0,0 +1,141 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package connection
import (
context "context"
v0alpha1 "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
mock "github.com/stretchr/testify/mock"
)
// MockExtra is an autogenerated mock type for the Extra type
type MockExtra struct {
mock.Mock
}
type MockExtra_Expecter struct {
mock *mock.Mock
}
func (_m *MockExtra) EXPECT() *MockExtra_Expecter {
return &MockExtra_Expecter{mock: &_m.Mock}
}
// Build provides a mock function with given fields: ctx, r
func (_m *MockExtra) Build(ctx context.Context, r *v0alpha1.Connection) (Connection, error) {
ret := _m.Called(ctx, r)
if len(ret) == 0 {
panic("no return value specified for Build")
}
var r0 Connection
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, *v0alpha1.Connection) (Connection, error)); ok {
return rf(ctx, r)
}
if rf, ok := ret.Get(0).(func(context.Context, *v0alpha1.Connection) Connection); ok {
r0 = rf(ctx, r)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(Connection)
}
}
if rf, ok := ret.Get(1).(func(context.Context, *v0alpha1.Connection) error); ok {
r1 = rf(ctx, r)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockExtra_Build_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Build'
type MockExtra_Build_Call struct {
*mock.Call
}
// Build is a helper method to define mock.On call
// - ctx context.Context
// - r *v0alpha1.Connection
func (_e *MockExtra_Expecter) Build(ctx interface{}, r interface{}) *MockExtra_Build_Call {
return &MockExtra_Build_Call{Call: _e.mock.On("Build", ctx, r)}
}
func (_c *MockExtra_Build_Call) Run(run func(ctx context.Context, r *v0alpha1.Connection)) *MockExtra_Build_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(*v0alpha1.Connection))
})
return _c
}
func (_c *MockExtra_Build_Call) Return(_a0 Connection, _a1 error) *MockExtra_Build_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockExtra_Build_Call) RunAndReturn(run func(context.Context, *v0alpha1.Connection) (Connection, error)) *MockExtra_Build_Call {
_c.Call.Return(run)
return _c
}
// Type provides a mock function with no fields
func (_m *MockExtra) Type() v0alpha1.ConnectionType {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Type")
}
var r0 v0alpha1.ConnectionType
if rf, ok := ret.Get(0).(func() v0alpha1.ConnectionType); ok {
r0 = rf()
} else {
r0 = ret.Get(0).(v0alpha1.ConnectionType)
}
return r0
}
// MockExtra_Type_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Type'
type MockExtra_Type_Call struct {
*mock.Call
}
// Type is a helper method to define mock.On call
func (_e *MockExtra_Expecter) Type() *MockExtra_Type_Call {
return &MockExtra_Type_Call{Call: _e.mock.On("Type")}
}
func (_c *MockExtra_Type_Call) Run(run func()) *MockExtra_Type_Call {
_c.Call.Run(func(args mock.Arguments) {
run()
})
return _c
}
func (_c *MockExtra_Type_Call) Return(_a0 v0alpha1.ConnectionType) *MockExtra_Type_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *MockExtra_Type_Call) RunAndReturn(run func() v0alpha1.ConnectionType) *MockExtra_Type_Call {
_c.Call.Return(run)
return _c
}
// NewMockExtra creates a new instance of MockExtra. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockExtra(t interface {
mock.TestingT
Cleanup(func())
}) *MockExtra {
mock := &MockExtra{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -0,0 +1,75 @@
package connection
import (
"context"
"fmt"
"sort"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
)
//go:generate mockery --name=Extra --structname=MockExtra --inpackage --filename=extra_mock.go --with-expecter
type Extra interface {
Type() provisioning.ConnectionType
Build(ctx context.Context, r *provisioning.Connection) (Connection, error)
}
//go:generate mockery --name=Factory --structname=MockFactory --inpackage --filename=factory_mock.go --with-expecter
type Factory interface {
Types() []provisioning.ConnectionType
Build(ctx context.Context, r *provisioning.Connection) (Connection, error)
}
type factory struct {
extras map[provisioning.ConnectionType]Extra
enabled map[provisioning.ConnectionType]struct{}
}
func ProvideFactory(enabled map[provisioning.ConnectionType]struct{}, extras []Extra) (Factory, error) {
f := &factory{
enabled: enabled,
extras: make(map[provisioning.ConnectionType]Extra, len(extras)),
}
for _, e := range extras {
if _, exists := f.extras[e.Type()]; exists {
return nil, fmt.Errorf("connection type %q is already registered", e.Type())
}
f.extras[e.Type()] = e
}
return f, nil
}
func (f *factory) Types() []provisioning.ConnectionType {
var types []provisioning.ConnectionType
for t := range f.enabled {
if _, exists := f.extras[t]; exists {
types = append(types, t)
}
}
sort.Slice(types, func(i, j int) bool {
return string(types[i]) < string(types[j])
})
return types
}
func (f *factory) Build(ctx context.Context, c *provisioning.Connection) (Connection, error) {
for _, e := range f.extras {
if e.Type() == c.Spec.Type {
if _, enabled := f.enabled[e.Type()]; !enabled {
return nil, fmt.Errorf("connection type %q is not enabled", e.Type())
}
return e.Build(ctx, c)
}
}
return nil, fmt.Errorf("connection type %q is not supported", c.Spec.Type)
}
var (
_ Factory = (*factory)(nil)
)
@@ -0,0 +1,143 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package connection
import (
context "context"
v0alpha1 "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
mock "github.com/stretchr/testify/mock"
)
// MockFactory is an autogenerated mock type for the Factory type
type MockFactory struct {
mock.Mock
}
type MockFactory_Expecter struct {
mock *mock.Mock
}
func (_m *MockFactory) EXPECT() *MockFactory_Expecter {
return &MockFactory_Expecter{mock: &_m.Mock}
}
// Build provides a mock function with given fields: ctx, r
func (_m *MockFactory) Build(ctx context.Context, r *v0alpha1.Connection) (Connection, error) {
ret := _m.Called(ctx, r)
if len(ret) == 0 {
panic("no return value specified for Build")
}
var r0 Connection
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, *v0alpha1.Connection) (Connection, error)); ok {
return rf(ctx, r)
}
if rf, ok := ret.Get(0).(func(context.Context, *v0alpha1.Connection) Connection); ok {
r0 = rf(ctx, r)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(Connection)
}
}
if rf, ok := ret.Get(1).(func(context.Context, *v0alpha1.Connection) error); ok {
r1 = rf(ctx, r)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockFactory_Build_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Build'
type MockFactory_Build_Call struct {
*mock.Call
}
// Build is a helper method to define mock.On call
// - ctx context.Context
// - r *v0alpha1.Connection
func (_e *MockFactory_Expecter) Build(ctx interface{}, r interface{}) *MockFactory_Build_Call {
return &MockFactory_Build_Call{Call: _e.mock.On("Build", ctx, r)}
}
func (_c *MockFactory_Build_Call) Run(run func(ctx context.Context, r *v0alpha1.Connection)) *MockFactory_Build_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(*v0alpha1.Connection))
})
return _c
}
func (_c *MockFactory_Build_Call) Return(_a0 Connection, _a1 error) *MockFactory_Build_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockFactory_Build_Call) RunAndReturn(run func(context.Context, *v0alpha1.Connection) (Connection, error)) *MockFactory_Build_Call {
_c.Call.Return(run)
return _c
}
// Types provides a mock function with no fields
func (_m *MockFactory) Types() []v0alpha1.ConnectionType {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Types")
}
var r0 []v0alpha1.ConnectionType
if rf, ok := ret.Get(0).(func() []v0alpha1.ConnectionType); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]v0alpha1.ConnectionType)
}
}
return r0
}
// MockFactory_Types_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Types'
type MockFactory_Types_Call struct {
*mock.Call
}
// Types is a helper method to define mock.On call
func (_e *MockFactory_Expecter) Types() *MockFactory_Types_Call {
return &MockFactory_Types_Call{Call: _e.mock.On("Types")}
}
func (_c *MockFactory_Types_Call) Run(run func()) *MockFactory_Types_Call {
_c.Call.Run(func(args mock.Arguments) {
run()
})
return _c
}
func (_c *MockFactory_Types_Call) Return(_a0 []v0alpha1.ConnectionType) *MockFactory_Types_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *MockFactory_Types_Call) RunAndReturn(run func() []v0alpha1.ConnectionType) *MockFactory_Types_Call {
_c.Call.Return(run)
return _c
}
// NewMockFactory creates a new instance of MockFactory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockFactory(t interface {
mock.TestingT
Cleanup(func())
}) *MockFactory {
mock := &MockFactory{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -0,0 +1,309 @@
package connection
import (
"context"
"errors"
"testing"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestProvideFactory(t *testing.T) {
t.Run("should create factory with valid extras", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GitlabConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
provisioning.GitlabConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.NoError(t, err)
require.NotNil(t, factory)
})
t.Run("should create factory with empty extras", func(t *testing.T) {
enabled := map[provisioning.ConnectionType]struct{}{}
factory, err := ProvideFactory(enabled, []Extra{})
require.NoError(t, err)
require.NotNil(t, factory)
})
t.Run("should create factory with nil enabled map", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
factory, err := ProvideFactory(nil, []Extra{extra1})
require.NoError(t, err)
require.NotNil(t, factory)
})
t.Run("should return error when duplicate repository types", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GithubConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.Error(t, err)
assert.Nil(t, factory)
assert.Contains(t, err.Error(), "connection type \"github\" is already registered")
})
}
func TestFactory_Types(t *testing.T) {
t.Run("should return only enabled types that have extras", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GitlabConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
provisioning.GitlabConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.NoError(t, err)
types := factory.Types()
assert.Len(t, types, 2)
assert.Contains(t, types, provisioning.GithubConnectionType)
assert.Contains(t, types, provisioning.GitlabConnectionType)
})
t.Run("should return sorted list of types", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GitlabConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GithubConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
provisioning.GitlabConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.NoError(t, err)
types := factory.Types()
assert.Len(t, types, 2)
// github should come before gitlab alphabetically
assert.Equal(t, provisioning.GithubConnectionType, types[0])
assert.Equal(t, provisioning.GitlabConnectionType, types[1])
})
t.Run("should return empty list when no types are enabled", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{}
factory, err := ProvideFactory(enabled, []Extra{extra1})
require.NoError(t, err)
types := factory.Types()
assert.Empty(t, types)
})
t.Run("should not return types that are enabled but have no extras", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
provisioning.GitlabConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1})
require.NoError(t, err)
types := factory.Types()
assert.Len(t, types, 1)
assert.Contains(t, types, provisioning.GithubConnectionType)
assert.NotContains(t, types, provisioning.GitlabConnectionType)
})
t.Run("should not return types that have extras but are not enabled", func(t *testing.T) {
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GitlabConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.NoError(t, err)
types := factory.Types()
assert.Len(t, types, 1)
assert.Contains(t, types, provisioning.GithubConnectionType)
assert.NotContains(t, types, provisioning.GitlabConnectionType)
})
t.Run("should return empty list when no extras are provided", func(t *testing.T) {
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{})
require.NoError(t, err)
types := factory.Types()
assert.Empty(t, types)
})
}
func TestFactory_Build(t *testing.T) {
t.Run("should successfully build connection when type is enabled and has extra", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
},
}
mockConnection := NewMockConnection(t)
extra := NewMockExtra(t)
extra.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra.EXPECT().Build(ctx, conn).Return(mockConnection, nil)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra})
require.NoError(t, err)
result, err := factory.Build(ctx, conn)
require.NoError(t, err)
assert.Equal(t, mockConnection, result)
})
t.Run("should return error when type is not enabled", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
},
}
extra := NewMockExtra(t)
extra.EXPECT().Type().Return(provisioning.GitlabConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra})
require.NoError(t, err)
result, err := factory.Build(ctx, conn)
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "connection type \"gitlab\" is not enabled")
})
t.Run("should return error when type is not supported", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
},
}
extra := NewMockExtra(t)
extra.EXPECT().Type().Return(provisioning.GithubConnectionType)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra})
require.NoError(t, err)
result, err := factory.Build(ctx, conn)
require.Error(t, err)
assert.Nil(t, result)
assert.Contains(t, err.Error(), "connection type \"gitlab\" is not supported")
})
t.Run("should pass through errors from extra.Build()", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
},
}
expectedErr := errors.New("build error")
extra := NewMockExtra(t)
extra.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra.EXPECT().Build(ctx, conn).Return(nil, expectedErr)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra})
require.NoError(t, err)
result, err := factory.Build(ctx, conn)
require.Error(t, err)
assert.Nil(t, result)
assert.Equal(t, expectedErr, err)
})
t.Run("should build with multiple extras registered", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
},
}
mockConnection := NewMockConnection(t)
extra1 := NewMockExtra(t)
extra1.EXPECT().Type().Return(provisioning.GithubConnectionType)
extra2 := NewMockExtra(t)
extra2.EXPECT().Type().Return(provisioning.GitlabConnectionType)
extra2.EXPECT().Build(ctx, conn).Return(mockConnection, nil)
enabled := map[provisioning.ConnectionType]struct{}{
provisioning.GithubConnectionType: {},
provisioning.GitlabConnectionType: {},
}
factory, err := ProvideFactory(enabled, []Extra{extra1, extra2})
require.NoError(t, err)
result, err := factory.Build(ctx, conn)
require.NoError(t, err)
assert.Equal(t, mockConnection, result)
})
}
@@ -0,0 +1,93 @@
package github
import (
"context"
"errors"
"fmt"
"net/http"
"strconv"
"github.com/google/go-github/v70/github"
apierrors "k8s.io/apimachinery/pkg/api/errors"
)
// API errors that we need to convey after parsing real GH errors (or faking them).
var (
//lint:ignore ST1005 this is not punctuation
ErrServiceUnavailable = apierrors.NewServiceUnavailable("github is unavailable")
)
//go:generate mockery --name Client --structname MockClient --inpackage --filename client_mock.go --with-expecter
type Client interface {
// Apps and installations
GetApp(ctx context.Context) (App, error)
GetAppInstallation(ctx context.Context, installationID string) (AppInstallation, error)
}
// App represents a Github App.
type App struct {
// ID represents the GH app ID.
ID int64
// Slug represents the GH app slug.
Slug string
// Owner represents the GH account/org owning the app
Owner string
}
// AppInstallation represents a Github App Installation.
type AppInstallation struct {
// ID represents the GH installation ID.
ID int64
// Whether the installation is enabled or not.
Enabled bool
}
type githubClient struct {
gh *github.Client
}
func NewClient(client *github.Client) Client {
return &githubClient{client}
}
// GetApp gets the app by using the given token.
func (r *githubClient) GetApp(ctx context.Context) (App, error) {
app, _, err := r.gh.Apps.Get(ctx, "")
if err != nil {
var ghErr *github.ErrorResponse
if errors.As(err, &ghErr) && ghErr.Response.StatusCode == http.StatusServiceUnavailable {
return App{}, ErrServiceUnavailable
}
return App{}, err
}
// TODO(ferruvich): do we need any other info?
return App{
ID: app.GetID(),
Slug: app.GetSlug(),
Owner: app.GetOwner().GetLogin(),
}, nil
}
// GetAppInstallation gets the installation of the app related to the given token.
func (r *githubClient) GetAppInstallation(ctx context.Context, installationID string) (AppInstallation, error) {
id, err := strconv.Atoi(installationID)
if err != nil {
return AppInstallation{}, fmt.Errorf("invalid installation ID: %s", installationID)
}
installation, _, err := r.gh.Apps.GetInstallation(ctx, int64(id))
if err != nil {
var ghErr *github.ErrorResponse
if errors.As(err, &ghErr) && ghErr.Response.StatusCode == http.StatusServiceUnavailable {
return AppInstallation{}, ErrServiceUnavailable
}
return AppInstallation{}, err
}
// TODO(ferruvich): do we need any other info?
return AppInstallation{
ID: installation.GetID(),
Enabled: installation.GetSuspendedAt().IsZero(),
}, nil
}
@@ -0,0 +1,149 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package github
import (
context "context"
mock "github.com/stretchr/testify/mock"
)
// MockClient is an autogenerated mock type for the Client type
type MockClient struct {
mock.Mock
}
type MockClient_Expecter struct {
mock *mock.Mock
}
func (_m *MockClient) EXPECT() *MockClient_Expecter {
return &MockClient_Expecter{mock: &_m.Mock}
}
// GetApp provides a mock function with given fields: ctx
func (_m *MockClient) GetApp(ctx context.Context) (App, error) {
ret := _m.Called(ctx)
if len(ret) == 0 {
panic("no return value specified for GetApp")
}
var r0 App
var r1 error
if rf, ok := ret.Get(0).(func(context.Context) (App, error)); ok {
return rf(ctx)
}
if rf, ok := ret.Get(0).(func(context.Context) App); ok {
r0 = rf(ctx)
} else {
r0 = ret.Get(0).(App)
}
if rf, ok := ret.Get(1).(func(context.Context) error); ok {
r1 = rf(ctx)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockClient_GetApp_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetApp'
type MockClient_GetApp_Call struct {
*mock.Call
}
// GetApp is a helper method to define mock.On call
// - ctx context.Context
func (_e *MockClient_Expecter) GetApp(ctx interface{}) *MockClient_GetApp_Call {
return &MockClient_GetApp_Call{Call: _e.mock.On("GetApp", ctx)}
}
func (_c *MockClient_GetApp_Call) Run(run func(ctx context.Context)) *MockClient_GetApp_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context))
})
return _c
}
func (_c *MockClient_GetApp_Call) Return(_a0 App, _a1 error) *MockClient_GetApp_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockClient_GetApp_Call) RunAndReturn(run func(context.Context) (App, error)) *MockClient_GetApp_Call {
_c.Call.Return(run)
return _c
}
// GetAppInstallation provides a mock function with given fields: ctx, installationID
func (_m *MockClient) GetAppInstallation(ctx context.Context, installationID string) (AppInstallation, error) {
ret := _m.Called(ctx, installationID)
if len(ret) == 0 {
panic("no return value specified for GetAppInstallation")
}
var r0 AppInstallation
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, string) (AppInstallation, error)); ok {
return rf(ctx, installationID)
}
if rf, ok := ret.Get(0).(func(context.Context, string) AppInstallation); ok {
r0 = rf(ctx, installationID)
} else {
r0 = ret.Get(0).(AppInstallation)
}
if rf, ok := ret.Get(1).(func(context.Context, string) error); ok {
r1 = rf(ctx, installationID)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// MockClient_GetAppInstallation_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetAppInstallation'
type MockClient_GetAppInstallation_Call struct {
*mock.Call
}
// GetAppInstallation is a helper method to define mock.On call
// - ctx context.Context
// - installationID string
func (_e *MockClient_Expecter) GetAppInstallation(ctx interface{}, installationID interface{}) *MockClient_GetAppInstallation_Call {
return &MockClient_GetAppInstallation_Call{Call: _e.mock.On("GetAppInstallation", ctx, installationID)}
}
func (_c *MockClient_GetAppInstallation_Call) Run(run func(ctx context.Context, installationID string)) *MockClient_GetAppInstallation_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(string))
})
return _c
}
func (_c *MockClient_GetAppInstallation_Call) Return(_a0 AppInstallation, _a1 error) *MockClient_GetAppInstallation_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *MockClient_GetAppInstallation_Call) RunAndReturn(run func(context.Context, string) (AppInstallation, error)) *MockClient_GetAppInstallation_Call {
_c.Call.Return(run)
return _c
}
// NewMockClient creates a new instance of MockClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockClient(t interface {
mock.TestingT
Cleanup(func())
}) *MockClient {
mock := &MockClient{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -0,0 +1,297 @@
package github_test
import (
"context"
"encoding/json"
"net/http"
"testing"
"time"
"github.com/google/go-github/v70/github"
conngh "github.com/grafana/grafana/apps/provisioning/pkg/connection/github"
mockhub "github.com/migueleliasweb/go-github-mock/src/mock"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGithubClient_GetApp(t *testing.T) {
tests := []struct {
name string
mockHandler *http.Client
token string
wantApp conngh.App
wantErr error
}{
{
name: "get app successfully",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetApp,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
app := &github.App{
ID: github.Ptr(int64(12345)),
Slug: github.Ptr("my-test-app"),
Owner: &github.User{
Login: github.Ptr("grafana"),
},
}
w.WriteHeader(http.StatusOK)
require.NoError(t, json.NewEncoder(w).Encode(app))
}),
),
),
token: "test-token",
wantApp: conngh.App{
ID: 12345,
Slug: "my-test-app",
Owner: "grafana",
},
wantErr: nil,
},
{
name: "service unavailable",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetApp,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusServiceUnavailable)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusServiceUnavailable,
},
Message: "Service unavailable",
}))
}),
),
),
token: "test-token",
wantApp: conngh.App{},
wantErr: conngh.ErrServiceUnavailable,
},
{
name: "other error",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetApp,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusInternalServerError,
},
Message: "Internal server error",
}))
}),
),
),
token: "test-token",
wantApp: conngh.App{},
wantErr: &github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusInternalServerError,
},
Message: "Internal server error",
},
},
{
name: "unauthorized error",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetApp,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusUnauthorized)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusUnauthorized,
},
Message: "Bad credentials",
}))
}),
),
),
token: "invalid-token",
wantApp: conngh.App{},
wantErr: &github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusUnauthorized,
},
Message: "Bad credentials",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Create a mock client
ghClient := github.NewClient(tt.mockHandler)
client := conngh.NewClient(ghClient)
// Call the method being tested
app, err := client.GetApp(context.Background())
// Check the error
if tt.wantErr != nil {
assert.Error(t, err)
assert.Equal(t, tt.wantApp, app)
} else {
assert.NoError(t, err)
assert.Equal(t, tt.wantApp, app)
}
})
}
}
func TestGithubClient_GetAppInstallation(t *testing.T) {
tests := []struct {
name string
mockHandler *http.Client
appToken string
installationID string
wantInstallation conngh.AppInstallation
wantErr bool
errContains string
}{
{
name: "get disabled app installation successfully",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetAppInstallationsByInstallationId,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
installation := &github.Installation{
ID: github.Ptr(int64(67890)),
SuspendedAt: github.Ptr(github.Timestamp{Time: time.Now()}),
}
w.WriteHeader(http.StatusOK)
require.NoError(t, json.NewEncoder(w).Encode(installation))
}),
),
),
appToken: "test-app-token",
installationID: "67890",
wantInstallation: conngh.AppInstallation{
ID: 67890,
Enabled: false,
},
wantErr: false,
},
{
name: "get enabled app installation successfully",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetAppInstallationsByInstallationId,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
installation := &github.Installation{
ID: github.Ptr(int64(67890)),
SuspendedAt: nil,
}
w.WriteHeader(http.StatusOK)
require.NoError(t, json.NewEncoder(w).Encode(installation))
}),
),
),
appToken: "test-app-token",
installationID: "67890",
wantInstallation: conngh.AppInstallation{
ID: 67890,
Enabled: true,
},
wantErr: false,
},
{
name: "invalid installation ID",
mockHandler: mockhub.NewMockedHTTPClient(),
appToken: "test-app-token",
installationID: "not-a-number",
wantInstallation: conngh.AppInstallation{},
wantErr: true,
errContains: "invalid installation ID",
},
{
name: "service unavailable",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetAppInstallationsByInstallationId,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusServiceUnavailable)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusServiceUnavailable,
},
Message: "Service unavailable",
}))
}),
),
),
appToken: "test-app-token",
installationID: "67890",
wantInstallation: conngh.AppInstallation{},
wantErr: true,
},
{
name: "installation not found",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetAppInstallationsByInstallationId,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusNotFound,
},
Message: "Not Found",
}))
}),
),
),
appToken: "test-app-token",
installationID: "99999",
wantInstallation: conngh.AppInstallation{},
wantErr: true,
},
{
name: "other error",
mockHandler: mockhub.NewMockedHTTPClient(
mockhub.WithRequestMatchHandler(
mockhub.GetAppInstallationsByInstallationId,
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
require.NoError(t, json.NewEncoder(w).Encode(github.ErrorResponse{
Response: &http.Response{
StatusCode: http.StatusInternalServerError,
},
Message: "Internal server error",
}))
}),
),
),
appToken: "test-app-token",
installationID: "67890",
wantInstallation: conngh.AppInstallation{},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Create a mock client
ghClient := github.NewClient(tt.mockHandler)
client := conngh.NewClient(ghClient)
// Call the method being tested
installation, err := client.GetAppInstallation(context.Background(), tt.installationID)
// Check the error
if tt.wantErr {
assert.Error(t, err)
if tt.errContains != "" {
assert.Contains(t, err.Error(), tt.errContains)
}
} else {
assert.NoError(t, err)
}
// Check the result
assert.Equal(t, tt.wantInstallation, installation)
})
}
}
@@ -0,0 +1,192 @@
package github
import (
"context"
"encoding/base64"
"errors"
"fmt"
"time"
"github.com/golang-jwt/jwt/v4"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/util/validation/field"
)
//go:generate mockery --name GithubFactory --structname MockGithubFactory --inpackage --filename factory_mock.go --with-expecter
type GithubFactory interface {
New(ctx context.Context, ghToken common.RawSecureValue) Client
}
type Connection struct {
obj *provisioning.Connection
ghFactory GithubFactory
}
func NewConnection(
obj *provisioning.Connection,
factory GithubFactory,
) Connection {
return Connection{
obj: obj,
ghFactory: factory,
}
}
const (
//TODO(ferruvich): these probably need to be setup in API configuration.
githubInstallationURL = "https://github.com/settings/installations"
jwtExpirationMinutes = 10 // GitHub Apps JWT tokens expire in 10 minutes maximum
)
// Mutate performs in place mutation of the underneath resource.
func (c *Connection) Mutate(_ context.Context) error {
// Do nothing in case spec.Github is nil.
// If this field is required, we should fail at validation time.
if c.obj.Spec.GitHub == nil {
return nil
}
c.obj.Spec.URL = fmt.Sprintf("%s/%s", githubInstallationURL, c.obj.Spec.GitHub.InstallationID)
// Generate JWT token if private key is being provided.
// Same as for the spec.Github, if such a field is required, Validation will take care of that.
if !c.obj.Secure.PrivateKey.Create.IsZero() {
token, err := generateToken(c.obj.Spec.GitHub.AppID, c.obj.Secure.PrivateKey.Create)
if err != nil {
return fmt.Errorf("failed to generate JWT token: %w", err)
}
// Store the generated token
c.obj.Secure.Token = common.InlineSecureValue{Create: token}
}
return nil
}
// Token generates and returns the Connection token.
func generateToken(appID string, privateKey common.RawSecureValue) (common.RawSecureValue, error) {
// Decode base64-encoded private key
privateKeyPEM, err := base64.StdEncoding.DecodeString(string(privateKey))
if err != nil {
return "", fmt.Errorf("failed to decode base64 private key: %w", err)
}
// Parse the private key
key, err := jwt.ParseRSAPrivateKeyFromPEM(privateKeyPEM)
if err != nil {
return "", fmt.Errorf("failed to parse private key: %w", err)
}
// Create the JWT token
now := time.Now()
claims := jwt.RegisteredClaims{
IssuedAt: jwt.NewNumericDate(now),
ExpiresAt: jwt.NewNumericDate(now.Add(time.Duration(jwtExpirationMinutes) * time.Minute)),
Issuer: appID,
}
token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
signedToken, err := token.SignedString(key)
if err != nil {
return "", fmt.Errorf("failed to sign JWT token: %w", err)
}
return common.RawSecureValue(signedToken), nil
}
// Validate ensures the resource _looks_ correct.
func (c *Connection) Validate(ctx context.Context) error {
list := field.ErrorList{}
if c.obj.Spec.Type != provisioning.GithubConnectionType {
list = append(list, field.Invalid(field.NewPath("spec", "type"), c.obj.Spec.Type, "invalid connection type"))
// Doesn't make much sense to continue validating a connection which is not a Github one.
return toError(c.obj.GetName(), list)
}
if c.obj.Spec.GitHub == nil {
list = append(
list, field.Required(field.NewPath("spec", "github"), "github info must be specified for GitHub connection"),
)
// Doesn't make much sense to continue validating a connection with no information.
return toError(c.obj.GetName(), list)
}
if c.obj.Secure.PrivateKey.IsZero() {
list = append(list, field.Required(field.NewPath("secure", "privateKey"), "privateKey must be specified for GitHub connection"))
}
if c.obj.Secure.Token.IsZero() {
list = append(list, field.Required(field.NewPath("secure", "token"), "token must be specified for GitHub connection"))
}
if !c.obj.Secure.ClientSecret.IsZero() {
list = append(list, field.Forbidden(field.NewPath("secure", "clientSecret"), "clientSecret is forbidden in GitHub connection"))
}
// Validate GitHub configuration fields
if c.obj.Spec.GitHub.AppID == "" {
list = append(list, field.Required(field.NewPath("spec", "github", "appID"), "appID must be specified for GitHub connection"))
}
if c.obj.Spec.GitHub.InstallationID == "" {
list = append(list, field.Required(field.NewPath("spec", "github", "installationID"), "installationID must be specified for GitHub connection"))
}
// In case we have any error above, we don't go forward with the validation, and return the errors.
if len(list) > 0 {
return toError(c.obj.GetName(), list)
}
// Validating app content via GH API
if err := c.validateAppAndInstallation(ctx); err != nil {
list = append(list, err)
}
return toError(c.obj.GetName(), list)
}
// validateAppAndInstallation validates the appID and installationID against the given github token.
func (c *Connection) validateAppAndInstallation(ctx context.Context) *field.Error {
ghClient := c.ghFactory.New(ctx, c.obj.Secure.Token.Create)
app, err := ghClient.GetApp(ctx)
if err != nil {
if errors.Is(err, ErrServiceUnavailable) {
return field.InternalError(field.NewPath("spec", "token"), ErrServiceUnavailable)
}
return field.Invalid(field.NewPath("spec", "token"), "[REDACTED]", "invalid token")
}
if fmt.Sprintf("%d", app.ID) != c.obj.Spec.GitHub.AppID {
return field.Invalid(field.NewPath("spec", "appID"), c.obj.Spec.GitHub.AppID, "appID mismatch")
}
_, err = ghClient.GetAppInstallation(ctx, c.obj.Spec.GitHub.InstallationID)
if err != nil {
if errors.Is(err, ErrServiceUnavailable) {
return field.InternalError(field.NewPath("spec", "token"), ErrServiceUnavailable)
}
return field.Invalid(field.NewPath("spec", "installationID"), c.obj.Spec.GitHub.InstallationID, "invalid installation ID")
}
return nil
}
// toError converts a field.ErrorList to an error, returning nil if the list is empty
func toError(name string, list field.ErrorList) error {
if len(list) == 0 {
return nil
}
return apierrors.NewInvalid(
provisioning.ConnectionResourceInfo.GroupVersionKind().GroupKind(),
name,
list,
)
}
var (
_ connection.Connection = (*Connection)(nil)
)
@@ -0,0 +1,434 @@
package github
import (
"context"
"encoding/base64"
"testing"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
//nolint:gosec // Test RSA private key (generated for testing purposes only)
const testPrivateKeyPEM = `-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAoInVbLY9io2Q/wHvUIXlEHg2Qyvd8eRzBAVEJ92DS6fx9H10
06V0VRm78S0MXyo6i+n8ZAbZ0/R+GWpP2Ephxm0Gs2zo+iO2mpB19xQFI4o6ZTOw
b2WyjSaa2Vr4oyDkqti6AvfjW4VUAu932e08GkgwmmQSHXj7FX2CMWjgUwTTcuaX
65SHNKLNYLUP0HTumLzoZeqDTdoMMpKNdgH9Avr4/8vkVJ0mD6rqvxnw3JHsseNO
WdQTxf2aApBNHIIKxWZ2i/ZmjLNey7kltgjEquGiBdJvip3fHhH5XHdkrXcjRtnw
OJDnDmi5lQwv5yUBOSkbvbXRv/L/m0YLoD/fbwIDAQABAoIBAFfl//hM8/cnuesV
+R1Con/ZAgTXQOdPqPXbmEyniVrkMqMmCdBUOBTcST4s5yg36+RtkeaGpb/ajyyF
PAB2AYDucwvMpudGpJWOYTiOOp4R8hU1LvZfXVrRd1lo6NgQi4NLtNUpOtACeVQ+
H4Yv0YemXQ47mnuOoRNMK/u3q5NoIdSahWptXBgUno8KklNpUrH3IYWaUxfBzDN3
2xsVRTn2SfTSyoDmTDdTgptJONmoK1/sV7UsgWksdFc6XyYhsFAZgOGEJrBABRvF
546dyQ0cWxuPyVXpM7CN3tqC5ssvLjElg3LicK1V6gnjpdRnnvX88d1Eh3Uc/9IM
OZInT2ECgYEA6W8sQXTWinyEwl8SDKKMbB2ApIghAcFgdRxprZE4WFxjsYNCNL70
dnSB7MRuzmxf5W77cV0N7JhH66N8HvY6Xq9olrpQ5dNttR4w8Pyv3wavDe8x7seL
5L2Xtbu7ihDr8Dk27MjiBSin3IxhBP5CJS910+pR6LrAWtEuU+FzFfECgYEAsA6y
qxHhCMXlTnauXhsnmPd1g61q7chW8kLQFYtHMLlQlgjHTW7irDZ9cPbPYDNjwRLO
7KLorcpv2NKe7rqq2ZyCm6hf1b9WnlQjo3dLpNWMu6fhy/smK8MgbRqcWpX+oTKF
79mK6hbY7o6eBzsQHBl7Z+LBNuwYmp9qOodPa18CgYEArv6ipKdcNhFGzRfMRiCN
OHederp6VACNuP2F05IsNUF9kxOdTEFirnKE++P+VU01TqA2azOhPp6iO+ohIGzi
MR06QNSH1OL9OWvasK4dggpWrRGF00VQgDgJRTnpS4WH+lxJ6pRlrAxgWpv6F24s
VAgSQr1Ejj2B+hMasdMvHWECgYBJ4uE4yhgXBnZlp4kmFV9Y4wF+cZkekaVrpn6N
jBYkbKFVVfnOlWqru3KJpgsB5I9IyAvvY68iwIKQDFSG+/AXw4dMrC0MF3DSoZ0T
TU2Br92QI7SvVod+djV1lGVp3ukt3XY4YqPZ+hywgUnw3uiz4j3YK2HLGup4ec6r
IX5DIQKBgHRLzvT3zqtlR1Oh0vv098clLwt+pGzXOxzJpxioOa5UqK13xIpFXbcg
iWUVh5YXCcuqaICUv4RLIEac5xQitk9Is/9IhP0NJ/81rHniosvdSpCeFXzxTImS
B8Uc0WUgheB4+yVKGnYpYaSOgFFI5+1BYUva/wDHLy2pWHz39Usb
-----END RSA PRIVATE KEY-----`
func TestConnection_Mutate(t *testing.T) {
t.Run("should add URL to Github connection", func(t *testing.T) {
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
},
}
mockFactory := NewMockGithubFactory(t)
conn := NewConnection(c, mockFactory)
require.NoError(t, conn.Mutate(context.Background()))
assert.Equal(t, "https://github.com/settings/installations/456", c.Spec.URL)
})
t.Run("should generate JWT token when private key is provided", func(t *testing.T) {
privateKeyBase64 := base64.StdEncoding.EncodeToString([]byte(testPrivateKeyPEM))
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue(privateKeyBase64),
},
},
}
mockFactory := NewMockGithubFactory(t)
conn := NewConnection(c, mockFactory)
require.NoError(t, conn.Mutate(context.Background()))
assert.Equal(t, "https://github.com/settings/installations/456", c.Spec.URL)
assert.False(t, c.Secure.Token.Create.IsZero(), "JWT token should be generated")
})
t.Run("should do nothing when GitHub config is nil", func(t *testing.T) {
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
Gitlab: &provisioning.GitlabConnectionConfig{
ClientID: "clientID",
},
},
}
mockFactory := NewMockGithubFactory(t)
conn := NewConnection(c, mockFactory)
require.NoError(t, conn.Mutate(context.Background()))
})
t.Run("should fail when private key is not base64", func(t *testing.T) {
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("invalid-key"),
},
},
}
mockFactory := NewMockGithubFactory(t)
conn := NewConnection(c, mockFactory)
err := conn.Mutate(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to generate JWT token")
assert.Contains(t, err.Error(), "failed to decode base64 private key")
})
t.Run("should fail when private key is invalid", func(t *testing.T) {
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue(base64.StdEncoding.EncodeToString([]byte("invalid-key"))),
},
},
}
mockFactory := NewMockGithubFactory(t)
conn := NewConnection(c, mockFactory)
err := conn.Mutate(context.Background())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to generate JWT token")
assert.Contains(t, err.Error(), "failed to parse private key")
})
}
func TestConnection_Validate(t *testing.T) {
tests := []struct {
name string
connection *provisioning.Connection
setupMock func(*MockGithubFactory)
wantErr bool
errMsgContains []string
}{
{
name: "invalid type returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: "invalid",
},
},
wantErr: true,
errMsgContains: []string{"spec.type"},
},
{
name: "github type without github config returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
},
},
wantErr: true,
errMsgContains: []string{"spec.github"},
},
{
name: "github type without private key returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
},
wantErr: true,
errMsgContains: []string{"secure.privateKey"},
},
{
name: "github type without token returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
},
},
wantErr: true,
errMsgContains: []string{"secure.token"},
},
{
name: "github type with client secret returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
ClientSecret: common.InlineSecureValue{
Create: common.NewSecretValue("test-client-secret"),
},
},
},
wantErr: true,
errMsgContains: []string{"secure.clientSecret"},
},
{
name: "github type without appID returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
Token: common.InlineSecureValue{
Create: common.NewSecretValue("test-token"),
},
},
},
wantErr: true,
errMsgContains: []string{"spec.github.appID"},
},
{
name: "github type without installationID returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
Token: common.InlineSecureValue{
Name: "test-token",
},
},
},
wantErr: true,
errMsgContains: []string{"spec.github.installationID"},
},
{
name: "github type with valid config is valid",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
Token: common.InlineSecureValue{
Create: common.NewSecretValue("test-token"),
},
},
},
wantErr: false,
setupMock: func(mockFactory *MockGithubFactory) {
mockClient := NewMockClient(t)
mockFactory.EXPECT().New(mock.Anything, common.RawSecureValue("test-token")).Return(mockClient)
mockClient.EXPECT().GetApp(mock.Anything).Return(App{ID: 123, Slug: "test-app"}, nil)
mockClient.EXPECT().GetAppInstallation(mock.Anything, "456").Return(AppInstallation{ID: 456}, nil)
},
},
{
name: "problem getting app returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
Token: common.InlineSecureValue{
Create: common.NewSecretValue("test-token"),
},
},
},
wantErr: true,
errMsgContains: []string{"spec.token", "[REDACTED]"},
setupMock: func(mockFactory *MockGithubFactory) {
mockClient := NewMockClient(t)
mockFactory.EXPECT().New(mock.Anything, common.RawSecureValue("test-token")).Return(mockClient)
mockClient.EXPECT().GetApp(mock.Anything).Return(App{}, assert.AnError)
},
},
{
name: "mismatched app ID returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
Token: common.InlineSecureValue{
Create: common.NewSecretValue("test-token"),
},
},
},
wantErr: true,
errMsgContains: []string{"spec.appID"},
setupMock: func(mockFactory *MockGithubFactory) {
mockClient := NewMockClient(t)
mockFactory.EXPECT().New(mock.Anything, common.RawSecureValue("test-token")).Return(mockClient)
mockClient.EXPECT().GetApp(mock.Anything).Return(App{ID: 444, Slug: "test-app"}, nil)
},
},
{
name: "problem when getting installation returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
Token: common.InlineSecureValue{
Create: common.NewSecretValue("test-token"),
},
},
},
wantErr: true,
errMsgContains: []string{"spec.installationID", "456"},
setupMock: func(mockFactory *MockGithubFactory) {
mockClient := NewMockClient(t)
mockFactory.EXPECT().New(mock.Anything, common.RawSecureValue("test-token")).Return(mockClient)
mockClient.EXPECT().GetApp(mock.Anything).Return(App{ID: 123, Slug: "test-app"}, nil)
mockClient.EXPECT().GetAppInstallation(mock.Anything, "456").Return(AppInstallation{}, assert.AnError)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockFactory := NewMockGithubFactory(t)
if tt.setupMock != nil {
tt.setupMock(mockFactory)
}
conn := NewConnection(tt.connection, mockFactory)
err := conn.Validate(context.Background())
if tt.wantErr {
assert.Error(t, err)
for _, msg := range tt.errMsgContains {
assert.Contains(t, err.Error(), msg)
}
} else {
assert.NoError(t, err)
}
})
}
}
@@ -0,0 +1,36 @@
package github
import (
"context"
"fmt"
"github.com/grafana/grafana-app-sdk/logging"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
)
type extra struct {
factory GithubFactory
}
func (e *extra) Type() provisioning.ConnectionType {
return provisioning.GithubConnectionType
}
func (e *extra) Build(ctx context.Context, connection *provisioning.Connection) (connection.Connection, error) {
logger := logging.FromContext(ctx)
if connection == nil || connection.Spec.GitHub == nil {
logger.Error("connection is nil or github info is nil")
return nil, fmt.Errorf("invalid github connection")
}
c := NewConnection(connection, e.factory)
return &c, nil
}
func Extra(factory GithubFactory) connection.Extra {
return &extra{
factory: factory,
}
}
@@ -0,0 +1,126 @@
package github_test
import (
"context"
"testing"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection/github"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestExtra_Type(t *testing.T) {
t.Run("should return GithubConnectionType", func(t *testing.T) {
mockFactory := github.NewMockGithubFactory(t)
e := github.Extra(mockFactory)
result := e.Type()
assert.Equal(t, provisioning.GithubConnectionType, result)
})
}
func TestExtra_Build(t *testing.T) {
t.Run("should successfully build connection", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Create: common.NewSecretValue("test-private-key"),
},
},
}
mockFactory := github.NewMockGithubFactory(t)
e := github.Extra(mockFactory)
result, err := e.Build(ctx, conn)
require.NoError(t, err)
require.NotNil(t, result)
})
t.Run("should handle different connection configurations", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "another-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "789",
InstallationID: "101112",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "existing-private-key",
},
Token: common.InlineSecureValue{
Name: "existing-token",
},
},
}
mockFactory := github.NewMockGithubFactory(t)
e := github.Extra(mockFactory)
result, err := e.Build(ctx, conn)
require.NoError(t, err)
require.NotNil(t, result)
})
t.Run("should build connection with background context", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
}
mockFactory := github.NewMockGithubFactory(t)
e := github.Extra(mockFactory)
result, err := e.Build(ctx, conn)
require.NoError(t, err)
require.NotNil(t, result)
})
t.Run("should always pass empty token to factory.New", func(t *testing.T) {
ctx := context.Background()
conn := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
Token: common.InlineSecureValue{
Create: common.NewSecretValue("some-token"),
},
},
}
mockFactory := github.NewMockGithubFactory(t)
e := github.Extra(mockFactory)
result, err := e.Build(ctx, conn)
require.NoError(t, err)
require.NotNil(t, result)
})
}
@@ -0,0 +1,39 @@
package github
import (
"context"
"net/http"
"github.com/google/go-github/v70/github"
"golang.org/x/oauth2"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
)
// Factory creates new GitHub clients.
// It exists only for the ability to test the code easily.
type Factory struct {
// Client allows overriding the client to use in the GH client returned. It exists primarily for testing.
// FIXME: we should replace in this way. We should add some options pattern for the factory.
Client *http.Client
}
func ProvideFactory() GithubFactory {
return &Factory{}
}
func (r *Factory) New(ctx context.Context, ghToken common.RawSecureValue) Client {
if r.Client != nil {
return NewClient(github.NewClient(r.Client))
}
if !ghToken.IsZero() {
tokenSrc := oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: string(ghToken)},
)
tokenClient := oauth2.NewClient(ctx, tokenSrc)
return NewClient(github.NewClient(tokenClient))
}
return NewClient(github.NewClient(&http.Client{}))
}
@@ -0,0 +1,86 @@
// Code generated by mockery v2.53.4. DO NOT EDIT.
package github
import (
context "context"
v0alpha1 "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
mock "github.com/stretchr/testify/mock"
)
// MockGithubFactory is an autogenerated mock type for the GithubFactory type
type MockGithubFactory struct {
mock.Mock
}
type MockGithubFactory_Expecter struct {
mock *mock.Mock
}
func (_m *MockGithubFactory) EXPECT() *MockGithubFactory_Expecter {
return &MockGithubFactory_Expecter{mock: &_m.Mock}
}
// New provides a mock function with given fields: ctx, ghToken
func (_m *MockGithubFactory) New(ctx context.Context, ghToken v0alpha1.RawSecureValue) Client {
ret := _m.Called(ctx, ghToken)
if len(ret) == 0 {
panic("no return value specified for New")
}
var r0 Client
if rf, ok := ret.Get(0).(func(context.Context, v0alpha1.RawSecureValue) Client); ok {
r0 = rf(ctx, ghToken)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(Client)
}
}
return r0
}
// MockGithubFactory_New_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'New'
type MockGithubFactory_New_Call struct {
*mock.Call
}
// New is a helper method to define mock.On call
// - ctx context.Context
// - ghToken v0alpha1.RawSecureValue
func (_e *MockGithubFactory_Expecter) New(ctx interface{}, ghToken interface{}) *MockGithubFactory_New_Call {
return &MockGithubFactory_New_Call{Call: _e.mock.On("New", ctx, ghToken)}
}
func (_c *MockGithubFactory_New_Call) Run(run func(ctx context.Context, ghToken v0alpha1.RawSecureValue)) *MockGithubFactory_New_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(v0alpha1.RawSecureValue))
})
return _c
}
func (_c *MockGithubFactory_New_Call) Return(_a0 Client) *MockGithubFactory_New_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *MockGithubFactory_New_Call) RunAndReturn(run func(context.Context, v0alpha1.RawSecureValue) Client) *MockGithubFactory_New_Call {
_c.Call.Return(run)
return _c
}
// NewMockGithubFactory creates a new instance of MockGithubFactory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockGithubFactory(t interface {
mock.TestingT
Cleanup(func())
}) *MockGithubFactory {
mock := &MockGithubFactory{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
@@ -1,28 +0,0 @@
package connection
import (
"fmt"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
)
const (
githubInstallationURL = "https://github.com/settings/installations"
)
func MutateConnection(connection *provisioning.Connection) error {
switch connection.Spec.Type {
case provisioning.GithubConnectionType:
// Do nothing in case spec.Github is nil.
// If this field is required, we should fail at validation time.
if connection.Spec.GitHub == nil {
return nil
}
connection.Spec.URL = fmt.Sprintf("%s/%s", githubInstallationURL, connection.Spec.GitHub.InstallationID)
return nil
default:
// TODO: we need to setup the URL for bitbucket and gitlab.
return nil
}
}
@@ -1,35 +0,0 @@
package connection_test
import (
"testing"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestMutateConnection(t *testing.T) {
t.Run("should add URL to Github connection", func(t *testing.T) {
c := &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
},
}
require.NoError(t, connection.MutateConnection(c))
assert.Equal(t, "https://github.com/settings/installations/456", c.Spec.URL)
})
}
@@ -1,104 +0,0 @@
package connection
import (
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/util/validation/field"
)
func ValidateConnection(connection *provisioning.Connection) error {
list := field.ErrorList{}
if connection.Spec.Type == "" {
list = append(list, field.Required(field.NewPath("spec", "type"), "type must be specified"))
}
switch connection.Spec.Type {
case provisioning.GithubConnectionType:
list = append(list, validateGithubConnection(connection)...)
case provisioning.BitbucketConnectionType:
list = append(list, validateBitbucketConnection(connection)...)
case provisioning.GitlabConnectionType:
list = append(list, validateGitlabConnection(connection)...)
default:
list = append(
list, field.NotSupported(
field.NewPath("spec", "type"),
connection.Spec.Type,
[]provisioning.ConnectionType{
provisioning.GithubConnectionType,
provisioning.BitbucketConnectionType,
provisioning.GitlabConnectionType,
}),
)
}
return toError(connection.GetName(), list)
}
func validateGithubConnection(connection *provisioning.Connection) field.ErrorList {
list := field.ErrorList{}
if connection.Spec.GitHub == nil {
list = append(
list, field.Required(field.NewPath("spec", "github"), "github info must be specified for GitHub connection"),
)
}
if connection.Secure.PrivateKey.IsZero() {
list = append(list, field.Required(field.NewPath("secure", "privateKey"), "privateKey must be specified for GitHub connection"))
}
if !connection.Secure.ClientSecret.IsZero() {
list = append(list, field.Forbidden(field.NewPath("secure", "clientSecret"), "clientSecret is forbidden in GitHub connection"))
}
return list
}
func validateBitbucketConnection(connection *provisioning.Connection) field.ErrorList {
list := field.ErrorList{}
if connection.Spec.Bitbucket == nil {
list = append(
list, field.Required(field.NewPath("spec", "bitbucket"), "bitbucket info must be specified in Bitbucket connection"),
)
}
if connection.Secure.ClientSecret.IsZero() {
list = append(list, field.Required(field.NewPath("secure", "clientSecret"), "clientSecret must be specified for Bitbucket connection"))
}
if !connection.Secure.PrivateKey.IsZero() {
list = append(list, field.Forbidden(field.NewPath("secure", "privateKey"), "privateKey is forbidden in Bitbucket connection"))
}
return list
}
func validateGitlabConnection(connection *provisioning.Connection) field.ErrorList {
list := field.ErrorList{}
if connection.Spec.Gitlab == nil {
list = append(
list, field.Required(field.NewPath("spec", "gitlab"), "gitlab info must be specified in Gitlab connection"),
)
}
if connection.Secure.ClientSecret.IsZero() {
list = append(list, field.Required(field.NewPath("secure", "clientSecret"), "clientSecret must be specified for Gitlab connection"))
}
if !connection.Secure.PrivateKey.IsZero() {
list = append(list, field.Forbidden(field.NewPath("secure", "privateKey"), "privateKey is forbidden in Gitlab connection"))
}
return list
}
// toError converts a field.ErrorList to an error, returning nil if the list is empty
func toError(name string, list field.ErrorList) error {
if len(list) == 0 {
return nil
}
return apierrors.NewInvalid(
provisioning.ConnectionResourceInfo.GroupVersionKind().GroupKind(),
name,
list,
)
}
@@ -1,253 +0,0 @@
package connection_test
import (
"testing"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestValidateConnection(t *testing.T) {
tests := []struct {
name string
connection *provisioning.Connection
wantErr bool
errMsg string
}{
{
name: "empty type returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{},
},
wantErr: true,
errMsg: "spec.type",
},
{
name: "invalid type returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: "invalid",
},
},
wantErr: true,
errMsg: "spec.type",
},
{
name: "github type without github config returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
},
},
wantErr: true,
errMsg: "spec.github",
},
{
name: "github type without private key returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
},
wantErr: true,
errMsg: "secure.privateKey",
},
{
name: "github type with client secret returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
ClientSecret: common.InlineSecureValue{
Name: "test-client-secret",
},
},
},
wantErr: true,
errMsg: "secure.clientSecret",
},
{
name: "github type with github config is valid",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GithubConnectionType,
GitHub: &provisioning.GitHubConnectionConfig{
AppID: "123",
InstallationID: "456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
},
},
wantErr: false,
},
{
name: "bitbucket type without bitbucket config returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.BitbucketConnectionType,
},
},
wantErr: true,
errMsg: "spec.bitbucket",
},
{
name: "bitbucket type without client secret returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.BitbucketConnectionType,
Bitbucket: &provisioning.BitbucketConnectionConfig{
ClientID: "client-123",
},
},
},
wantErr: true,
errMsg: "secure.clientSecret",
},
{
name: "bitbucket type with private key returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.BitbucketConnectionType,
Bitbucket: &provisioning.BitbucketConnectionConfig{
ClientID: "client-123",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
ClientSecret: common.InlineSecureValue{
Name: "test-client-secret",
},
},
},
wantErr: true,
errMsg: "secure.privateKey",
},
{
name: "bitbucket type with bitbucket config is valid",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.BitbucketConnectionType,
Bitbucket: &provisioning.BitbucketConnectionConfig{
ClientID: "client-123",
},
},
Secure: provisioning.ConnectionSecure{
ClientSecret: common.InlineSecureValue{
Name: "test-client-secret",
},
},
},
wantErr: false,
},
{
name: "gitlab type without gitlab config returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
},
},
wantErr: true,
errMsg: "spec.gitlab",
},
{
name: "gitlab type without client secret returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
Gitlab: &provisioning.GitlabConnectionConfig{
ClientID: "client-456",
},
},
},
wantErr: true,
errMsg: "secure.clientSecret",
},
{
name: "gitlab type with private key returns error",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
Gitlab: &provisioning.GitlabConnectionConfig{
ClientID: "client-456",
},
},
Secure: provisioning.ConnectionSecure{
PrivateKey: common.InlineSecureValue{
Name: "test-private-key",
},
ClientSecret: common.InlineSecureValue{
Name: "test-client-secret",
},
},
},
wantErr: true,
errMsg: "secure.privateKey",
},
{
name: "gitlab type with gitlab config is valid",
connection: &provisioning.Connection{
ObjectMeta: metav1.ObjectMeta{Name: "test-connection"},
Spec: provisioning.ConnectionSpec{
Type: provisioning.GitlabConnectionType,
Gitlab: &provisioning.GitlabConnectionConfig{
ClientID: "client-456",
},
},
Secure: provisioning.ConnectionSecure{
ClientSecret: common.InlineSecureValue{
Name: "test-client-secret",
},
},
},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := connection.ValidateConnection(tt.connection)
if tt.wantErr {
assert.Error(t, err)
if tt.errMsg != "" {
assert.Contains(t, err.Error(), tt.errMsg)
}
} else {
assert.NoError(t, err)
}
})
}
}
@@ -13,7 +13,7 @@ import (
type ConnectionSecureApplyConfiguration struct {
PrivateKey *commonv0alpha1.InlineSecureValue `json:"privateKey,omitempty"`
ClientSecret *commonv0alpha1.InlineSecureValue `json:"clientSecret,omitempty"`
Token *commonv0alpha1.InlineSecureValue `json:"webhook,omitempty"`
Token *commonv0alpha1.InlineSecureValue `json:"token,omitempty"`
}
// ConnectionSecureApplyConfiguration constructs a declarative configuration of the ConnectionSecure type for use with
+7 -1
View File
@@ -336,7 +336,7 @@ rudderstack_data_plane_url =
rudderstack_sdk_url =
# Rudderstack v3 SDK, optional, defaults to false. If set, Rudderstack v3 SDK will be used instead of v1
rudderstack_v3_sdk_url =
rudderstack_v3_sdk_url =
# Rudderstack Config url, optional, used by Rudderstack SDK to fetch source config
rudderstack_config_url =
@@ -2079,8 +2079,14 @@ enable =
# To enable features by default, set `Expression: "true"` in:
# https://github.com/grafana/grafana/blob/main/pkg/services/featuremgmt/registry.go
# The feature_toggles section supports feature flags of a number of types,
# including boolean, string, integer, float, and structured values, following the OpenFeature specification.
#
# feature1 = true
# feature2 = false
# feature3 = "foobar"
# feature4 = 1.5
# feature5 = { "foo": "bar" }
[feature_toggles.openfeature]
# This is EXPERIMENTAL. Please, do not use this section
+8 -3
View File
@@ -323,7 +323,7 @@
;rudderstack_sdk_url =
# Rudderstack v3 SDK, optional, defaults to false. If set, Rudderstack v3 SDK will be used instead of v1
;rudderstack_v3_sdk_url =
;rudderstack_v3_sdk_url =
# Rudderstack Config url, optional, used by Rudderstack SDK to fetch source config
;rudderstack_config_url =
@@ -1913,7 +1913,7 @@ default_datasource_uid =
# client_queue_max_size is the maximum size in bytes of the client queue
# for Live connections. Defaults to 4MB.
;client_queue_max_size =
;client_queue_max_size =
#################################### Grafana Image Renderer Plugin ##########################
[plugin.grafana-image-renderer]
@@ -1996,9 +1996,14 @@ default_datasource_uid =
;enable = feature1,feature2
# The feature_toggles section supports feature flags of a number of types,
# including boolean, string, integer, float, and structured values, following the OpenFeature specification.
;feature1 = true
;feature2 = false
;feature3 = "foobar"
;feature4 = 1.5
;feature5 = { "foo": "bar" }
[date_formats]
# For information on what formatting patterns that are supported https://momentjs.com/docs/#/displaying/
+46 -3
View File
@@ -99,12 +99,27 @@ refs:
destination: /docs/grafana/<GRAFANA_VERSION>/administration/data-source-management/#query-and-resource-caching
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/administration/data-source-management/#query-and-resource-caching
mssql-troubleshoot:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/troubleshooting/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/troubleshooting/
postgres:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/postgres/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/postgres/
mysql:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mysql/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mysql/
---
# Microsoft SQL Server (MSSQL) data source
Grafana ships with built-in support for Microsoft SQL Server (MSSQL).
You can query and visualize data from any Microsoft SQL Server 2005 or newer, including the Microsoft Azure SQL Database.
You can query and visualize data from any Microsoft SQL Server 2005 or newer, including Microsoft Azure SQL Database.
Use this data source to create dashboards, explore SQL data, and monitor MSSQL-based workloads in real time.
@@ -113,10 +128,33 @@ The following documentation helps you get started working with the Microsoft SQL
- [Configure the Microsoft SQL Server data source](ref:configure-mssql-data-source)
- [Microsoft SQL Server query editor](ref:mssql-query-editor)
- [Microsoft SQL Server template variables](ref:mssql-template-variables)
- [Troubleshoot Microsoft SQL Server data source issues](ref:mssql-troubleshoot)
## Get the most out of the data source
## Supported versions
After installing and configuring the Microsoft SQL Server data source, you can:
This data source supports the following Microsoft SQL Server versions:
- Microsoft SQL Server 2005 and newer
- Microsoft Azure SQL Database
- Azure SQL Managed Instance
Grafana recommends using the latest available service pack for your SQL Server version for optimal compatibility.
## Key capabilities
The Microsoft SQL Server data source supports:
- **Time series queries:** Visualize metrics over time using the built-in time grouping macros.
- **Table queries:** Display query results in table format for any valid SQL query.
- **Template variables:** Create dynamic dashboards with variable-driven queries.
- **Annotations:** Overlay events from SQL Server on your dashboard graphs.
- **Alerting:** Create alerts based on SQL Server query results.
- **Stored procedures:** Execute stored procedures and visualize results.
- **Macros:** Simplify queries with built-in macros for time filtering and grouping.
## Additional resources
After configuring the Microsoft SQL Server data source, you can:
- Create a wide variety of [visualizations](ref:visualizations)
- Configure and use [templates and variables](ref:variables)
@@ -124,3 +162,8 @@ After installing and configuring the Microsoft SQL Server data source, you can:
- Add [annotations](ref:annotate-visualizations)
- Set up [alerting](ref:alerting)
- Optimize performance with [query caching](ref:query-caching)
## Related data sources
- [PostgreSQL](ref:postgres) - For PostgreSQL databases.
- [MySQL](ref:mysql) - For MySQL and MariaDB databases.
@@ -89,6 +89,26 @@ refs:
destination: /docs/grafana/<GRAFANA_VERSION>/setup-grafana/configure-access/configure-authentication/azuread/#enable-azure-ad-oauth-in-grafana
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/setup-grafana/configure-access/configure-authentication/azuread/#enable-azure-ad-oauth-in-grafana
mssql-query-editor:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/query-editor/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/query-editor/
mssql-template-variables:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/template-variables/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/template-variables/
alerting:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/alerting-and-irm/alerting/
mssql-troubleshoot:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/troubleshooting/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/troubleshooting/
---
# Configure the Microsoft SQL Server data source
@@ -97,13 +117,28 @@ This document provides instructions for configuring the Microsoft SQL Server dat
## Before you begin
- Grafana comes with a built-in MSSQL data source plugin, eliminating the need to install a plugin.
Before configuring the Microsoft SQL Server data source, ensure you have the following:
- You must have the `Organization administrator` role to configure the MSSQL data source. Organization administrators can also [configure the data source via YAML](#provision-the-data-source) with the Grafana provisioning system.
- **Grafana permissions:** You must have the `Organization administrator` role to configure data sources. Organization administrators can also [configure the data source via YAML](#provision-the-data-source) with the Grafana provisioning system.
- Familiarize yourself with your MSSQL security configuration and gather any necessary security certificates and client keys.
- **A running SQL Server instance:** Microsoft SQL Server 2005 or newer, Azure SQL Database, or Azure SQL Managed Instance.
- Verify that data from MSSQL is being written to your Grafana instance.
- **Network access:** Grafana must be able to reach your SQL Server. The default port is `1433`.
- **Authentication credentials:** Depending on your authentication method, you need one of:
- SQL Server login credentials (username and password).
- Windows/Kerberos credentials and configuration (not supported in Grafana Cloud).
- Azure Entra ID app registration or managed identity.
- **Security certificates:** If using encrypted connections, gather any necessary TLS/SSL certificates.
{{< admonition type="note" >}}
Grafana ships with a built-in Microsoft SQL Server data source plugin. No additional installation is required.
{{< /admonition >}}
{{< admonition type="tip" >}}
**Grafana Cloud users:** If your SQL Server is in a private network, you can configure [Private data source connect](ref:private-data-source-connect) to establish connectivity.
{{< /admonition >}}
## Add the MSSQL data source
@@ -382,3 +417,48 @@ datasources:
secureJsonData:
password: 'Password!'
```
### Configure with Terraform
You can configure the Microsoft SQL Server data source using [Terraform](https://www.terraform.io/) with the [Grafana Terraform provider](https://registry.terraform.io/providers/grafana/grafana/latest/docs).
For more information about provisioning resources with Terraform, refer to the [Grafana as code using Terraform](https://grafana.com/docs/grafana-cloud/developer-resources/infrastructure-as-code/terraform/) documentation.
#### Terraform example
The following example creates a basic Microsoft SQL Server data source:
```hcl
resource "grafana_data_source" "mssql" {
name = "MSSQL"
type = "mssql"
url = "localhost:1433"
user = "grafana"
json_data_encoded = jsonencode({
database = "grafana"
maxOpenConns = 100
maxIdleConns = 100
maxIdleConnsAuto = true
connMaxLifetime = 14400
connectionTimeout = 0
encrypt = "false"
})
secure_json_data_encoded = jsonencode({
password = "Password!"
})
}
```
For all available configuration options, refer to the [Grafana provider data source resource documentation](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source).
## Next steps
After configuring your Microsoft SQL Server data source, you can:
- [Write queries](ref:mssql-query-editor) using the query editor to explore and visualize your data
- [Create template variables](ref:mssql-template-variables) to build dynamic, reusable dashboards
- [Add annotations](ref:annotate-visualizations) to overlay SQL Server events on your graphs
- [Set up alerting](ref:alerting) to create alert rules based on your SQL Server data
- [Troubleshoot issues](ref:mssql-troubleshoot) if you encounter problems with your data source
@@ -0,0 +1,333 @@
---
description: Troubleshoot common problems with the Microsoft SQL Server data source in Grafana
keywords:
- grafana
- MSSQL
- Microsoft
- SQL
- troubleshooting
- errors
labels:
products:
- cloud
- enterprise
- oss
menuTitle: Troubleshooting
title: Troubleshoot Microsoft SQL Server data source issues
weight: 400
refs:
configure-mssql-data-source:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/configure/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/configure/
mssql-query-editor:
- pattern: /docs/grafana/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/query-editor/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana/<GRAFANA_VERSION>/datasources/mssql/query-editor/
private-data-source-connect:
- pattern: /docs/grafana/
destination: /docs/grafana-cloud/connect-externally-hosted/private-data-source-connect/
- pattern: /docs/grafana-cloud/
destination: /docs/grafana-cloud/connect-externally-hosted/private-data-source-connect/
---
# Troubleshoot Microsoft SQL Server data source issues
This document provides solutions to common issues you may encounter when configuring or using the Microsoft SQL Server (MSSQL) data source in Grafana.
## Connection errors
These errors occur when Grafana cannot establish or maintain a connection to the Microsoft SQL Server.
### Unable to connect to the server
**Error message:** "Unable to open tcp connection" or "dial tcp: connection refused"
**Cause:** Grafana cannot establish a network connection to the SQL Server.
**Solution:**
1. Verify that the SQL Server is running and accessible.
1. Check that the host and port are correct in the data source configuration. The default SQL Server port is `1433`.
1. Ensure there are no firewall rules blocking the connection between Grafana and SQL Server.
1. Verify that SQL Server is configured to allow remote connections.
1. For Grafana Cloud, ensure you have configured [Private data source connect](ref:private-data-source-connect) if your SQL Server instance is not publicly accessible.
### Connection timeout
**Error message:** "Connection timed out" or "I/O timeout"
**Cause:** The connection to SQL Server timed out before receiving a response.
**Solution:**
1. Check the network latency between Grafana and SQL Server.
1. Verify that SQL Server is not overloaded or experiencing performance issues.
1. Increase the **Connection timeout** setting in the data source configuration under **Additional settings**.
1. Check if any network devices (load balancers, proxies) are timing out the connection.
### Encryption-related connection failures
**Error message:** "TLS handshake failed" or "certificate verify failed"
**Cause:** There is a mismatch between the encryption settings in Grafana and what the SQL Server supports or requires.
**Solution:**
1. For older versions of SQL Server (2008, 2008R2), set the **Encrypt** option to **Disable** or **False** in the data source configuration.
1. Verify that the SQL Server has a valid SSL certificate if encryption is enabled.
1. Check that the certificate is trusted by the Grafana server.
1. Ensure you're using the latest available service pack for your SQL Server version for optimal compatibility.
### Named instance connection issues
**Error message:** "Cannot connect to named instance" or connection fails when using instance name
**Cause:** Grafana cannot resolve the SQL Server named instance.
**Solution:**
1. Use the format `hostname\instancename` or `hostname\instancename,port` in the **Host** field.
1. Verify that the SQL Server Browser service is running on the SQL Server machine.
1. If the Browser service is unavailable, specify the port number directly: `hostname,port`.
1. Check that UDP port 1434 is open if using the SQL Server Browser service.
## Authentication errors
These errors occur when there are issues with authentication credentials or permissions.
### Login failed for user
**Error message:** "Login failed for user 'username'" or "Authentication failed"
**Cause:** The authentication credentials are invalid or the user doesn't have permission to access the database.
**Solution:**
1. Verify that the username and password are correct.
1. Check that the user exists in SQL Server and is enabled.
1. Ensure the user has access to the specified database.
1. For Windows Authentication, verify that the credentials are in the correct format (`DOMAIN\User`).
1. Check that the SQL Server authentication mode allows the type of login you're using (SQL Server Authentication, Windows Authentication, or Mixed Mode).
### Access denied to database
**Error message:** "Cannot open database 'dbname' requested by the login"
**Cause:** The authenticated user doesn't have permission to access the specified database.
**Solution:**
1. Verify that the database name is correct in the data source configuration.
1. Ensure the user is mapped to the database with appropriate permissions.
1. Grant at least `SELECT` permission on the required tables:
```sql
USE [your_database]
GRANT SELECT ON dbo.YourTable TO [your_user]
```
1. Check that the user doesn't have any conflicting permissions from the public role.
### Windows Authentication (Kerberos) issues
**Error message:** "Kerberos authentication failed" or "Cannot initialize Kerberos"
**Cause:** Kerberos configuration is incorrect or incomplete.
**Solution:**
1. Verify that the Kerberos configuration file (`krb5.conf`) path is correct in the data source settings.
1. For keytab authentication, ensure the keytab file exists and is readable by Grafana.
1. Check that the realm and KDC settings are correct in the Kerberos configuration.
1. Verify DNS is correctly resolving the KDC servers.
1. Ensure the service principal name (SPN) is registered for the SQL Server instance.
{{< admonition type="note" >}}
Kerberos authentication is not supported in Grafana Cloud.
{{< /admonition >}}
### Azure Entra ID authentication errors
**Error message:** "AADSTS error codes" or "Azure AD authentication failed"
**Cause:** Azure Entra ID (formerly Azure AD) authentication is misconfigured.
**Solution:**
1. For **App Registration** authentication:
- Verify the tenant ID, client ID, and client secret are correct.
- Ensure the app registration has been added as a user in the Azure SQL database.
- Check that the client secret hasn't expired.
1. For **Managed Identity** authentication:
- Verify `managed_identity_enabled = true` is set in the Grafana server configuration.
- Ensure the managed identity has been added to the Azure SQL database.
- Confirm the Azure resource hosting Grafana has managed identity enabled.
1. For **Current User** authentication:
- Ensure `user_identity_enabled = true` is set in the Grafana server configuration.
- Verify the app registration is configured to issue both Access Tokens and ID Tokens.
- Check that the required API permissions are configured (`user_impersonation` for Azure SQL).
For detailed Azure authentication configuration, refer to [Configure the Microsoft SQL Server data source](ref:configure-mssql-data-source).
## Query errors
These errors occur when there are issues with query syntax or configuration.
### Time column not found or invalid
**Error message:** "Could not find time column" or time series visualization shows no data
**Cause:** The query doesn't return a properly formatted `time` column for time series visualization.
**Solution:**
1. Ensure your query includes a column named `time` when using the **Time series** format.
1. Use the `$__time()` macro to rename your date column: `$__time(your_date_column)`.
1. Verify the time column is of a valid SQL date/time type (`datetime`, `datetime2`, `date`) or contains Unix epoch values.
1. Ensure the result set is sorted by the time column using `ORDER BY`.
### Macro expansion errors
**Error message:** "Error parsing query" or macros appear unexpanded in the query
**Cause:** Grafana macros are being used incorrectly.
**Solution:**
1. Verify macro syntax: use `$__timeFilter(column)` not `$_timeFilter(column)`.
1. Macros don't work inside stored procedures—use explicit date parameters instead.
1. Check that the column name passed to macros exists in your table.
1. View the expanded query by clicking **Generated SQL** after running the query to debug macro expansion.
### Timezone and time shift issues
**Cause:** Time series data appears shifted or doesn't align with expected times.
**Solution:**
1. Store timestamps in UTC in your database to avoid timezone issues.
1. Time macros (`$__time`, `$__timeFilter`, etc.) always expand to UTC values.
1. If your timestamps are stored in local time, convert them to UTC in your query:
```sql
SELECT
your_datetime_column AT TIME ZONE 'Your Local Timezone' AT TIME ZONE 'UTC' AS time,
value
FROM your_table
```
1. Don't pass timezone parameters to time macros—they're not supported.
### Query returns too many rows
**Error message:** "Result set too large" or browser becomes unresponsive
**Cause:** The query returns more data than can be efficiently processed.
**Solution:**
1. Add time filters using `$__timeFilter(column)` to limit data to the dashboard time range.
1. Use aggregations (`AVG`, `SUM`, `COUNT`) with `GROUP BY` instead of returning raw rows.
1. Add a `TOP` clause to limit results: `SELECT TOP 1000 ...`.
1. Use the `$__timeGroup()` macro to aggregate data into time intervals.
### Stored procedure returns no data
**Cause:** Stored procedure output isn't being captured correctly.
**Solution:**
1. Ensure the stored procedure uses `SELECT` statements, not just variable assignments.
1. Remove `SET NOCOUNT ON` if present, or ensure it's followed by a `SELECT` statement.
1. Verify the stored procedure parameters are being passed correctly.
1. Test the stored procedure directly in SQL Server Management Studio with the same parameters.
For more information on using stored procedures, refer to the [query editor documentation](ref:mssql-query-editor).
## Performance issues
These issues relate to slow queries or high resource usage.
### Slow query execution
**Cause:** Queries take a long time to execute.
**Solution:**
1. Reduce the dashboard time range to limit data volume.
1. Add indexes to columns used in `WHERE` clauses and time filters.
1. Use aggregations instead of returning individual rows.
1. Increase the **Min time interval** setting to reduce the number of data points.
1. Review the query execution plan in SQL Server Management Studio to identify bottlenecks.
### Connection pool exhaustion
**Error message:** "Too many connections" or "Connection pool exhausted"
**Cause:** Too many concurrent connections to the database.
**Solution:**
1. Increase the **Max open** connection limit in the data source configuration.
1. Enable **Auto max idle** to automatically manage idle connections.
1. Reduce the number of panels querying the same data source simultaneously.
1. Check for long-running queries that might be holding connections.
## Other common issues
The following issues don't produce specific error messages but are commonly encountered.
### System databases appear in queries
**Cause:** Queries accidentally access system databases.
**Solution:**
1. The query editor automatically excludes `tempdb`, `model`, `msdb`, and `master` from the database dropdown.
1. Always specify the database in your data source configuration to restrict access.
1. Ensure the database user only has permissions on the intended database.
### Template variable queries fail
**Cause:** Variable queries return unexpected results or errors.
**Solution:**
1. Verify the variable query syntax is valid SQL that returns a single column.
1. Check that the data source connection is working.
1. Ensure the user has permission to access the tables referenced in the variable query.
1. Test the query in the query editor before using it as a variable query.
### Data appears incorrect or misaligned
**Cause:** Data formatting or type conversion issues.
**Solution:**
1. Use explicit column aliases to ensure consistent naming: `SELECT value AS metric`.
1. Verify numeric columns are actually numeric types, not strings.
1. Check for `NULL` values that might affect aggregations.
1. Use the `FILL` option in `$__timeGroup()` macro to handle missing data points.
## Get additional help
If you continue to experience issues after following this troubleshooting guide:
1. Check the [Grafana community forums](https://community.grafana.com/) for similar issues.
1. Review the [Grafana GitHub issues](https://github.com/grafana/grafana/issues) for known bugs.
1. Enable debug logging in Grafana to capture detailed error information.
1. Check SQL Server logs for additional error details.
1. Contact Grafana Support if you're an Enterprise or Cloud customer.
When reporting issues, include:
- Grafana version
- SQL Server version
- Error messages (redact sensitive information)
- Steps to reproduce
- Relevant query examples (redact sensitive data)
@@ -2836,9 +2836,11 @@ For more information about Grafana Enterprise, refer to [Grafana Enterprise](../
Keys of features to enable, separated by space.
#### `FEATURE_TOGGLE_NAME = false`
#### `FEATURE_NAME = <value>`
Some feature toggles for stable features are on by default. Use this setting to disable an on-by-default feature toggle with the name FEATURE_TOGGLE_NAME, for example, `exploreMixedDatasource = false`.
Use a key-value pair to set feature flag values explicitly, overriding any default values. A few different types are supported, following the OpenFeature specification. See the defaults.ini file for more details.
For example, to disable an on-by-default feature toggle named `exploreMixedDatasource`, specify `exploreMixedDatasource = false`.
<hr>
+5 -5
View File
@@ -1156,11 +1156,6 @@
"count": 2
}
},
"public/app/core/config.ts": {
"no-barrel-files/no-barrel-files": {
"count": 2
}
},
"public/app/core/navigation/types.ts": {
"@typescript-eslint/no-explicit-any": {
"count": 1
@@ -1801,6 +1796,11 @@
"count": 1
}
},
"public/app/features/dashboard-scene/panel-edit/PanelDataPane/PanelDataPane.tsx": {
"@typescript-eslint/consistent-type-assertions": {
"count": 1
}
},
"public/app/features/dashboard-scene/panel-edit/PanelOptionsPane.test.tsx": {
"@typescript-eslint/no-explicit-any": {
"count": 1
@@ -246,6 +246,8 @@ const injectedRtkApi = api
facetLimit: queryArg.facetLimit,
tags: queryArg.tags,
libraryPanel: queryArg.libraryPanel,
panelType: queryArg.panelType,
dataSourceType: queryArg.dataSourceType,
permission: queryArg.permission,
sort: queryArg.sort,
limit: queryArg.limit,
@@ -674,6 +676,10 @@ export type SearchDashboardsAndFoldersApiArg = {
tags?: string[];
/** find dashboards that reference a given libraryPanel */
libraryPanel?: string;
/** find dashboards using panels of a given plugin type */
panelType?: string;
/** find dashboards using datasources of a given plugin type */
dataSourceType?: string;
/** permission needed for the resource (view, edit, admin) */
permission?: 'view' | 'edit' | 'admin';
/** sortable field */
@@ -1452,7 +1452,7 @@ export type ConnectionSecure = {
/** PrivateKey is the reference to the private key used for GitHub App authentication. This value is stored securely and cannot be read back */
privateKey?: InlineSecureValue;
/** Token is the reference of the token used to act as the Connection. This value is stored securely and cannot be read back */
webhook?: InlineSecureValue;
token?: InlineSecureValue;
};
export type BitbucketConnectionConfig = {
/** App client ID */
@@ -9,6 +9,7 @@ import { FieldColorModeId } from '../types/fieldColor';
import { FieldConfigPropertyItem, FieldConfigSource } from '../types/fieldOverrides';
import { InterpolateFunction } from '../types/panel';
import { ThresholdsMode } from '../types/thresholds';
import { MappingType } from '../types/valueMapping';
import { Registry } from '../utils/Registry';
import { locationUtil } from '../utils/location';
import { mockStandardProperties } from '../utils/tests/mockStandardProperties';
@@ -999,6 +1000,45 @@ describe('setDynamicConfigValue', () => {
expect(config.custom.property3).toEqual({});
expect(config.displayName).toBeUndefined();
});
it('works correctly with multiple value mappings in the same override', () => {
const config: FieldConfig = {
mappings: [{ type: MappingType.ValueToText, options: { existing: { text: 'existing' } } }],
};
setDynamicConfigValue(
config,
{
id: 'mappings',
value: [{ type: MappingType.ValueToText, options: { first: { text: 'first' } } }],
},
{
fieldConfigRegistry: customFieldRegistry,
data: [],
field: { type: FieldType.number } as Field,
dataFrameIndex: 0,
}
);
setDynamicConfigValue(
config,
{
id: 'mappings',
value: [{ type: MappingType.ValueToText, options: { second: { text: 'second' } } }],
},
{
fieldConfigRegistry: customFieldRegistry,
data: [],
field: { type: FieldType.number } as Field,
dataFrameIndex: 0,
}
);
expect(config.mappings).toHaveLength(3);
expect(config.mappings![0]).toEqual({ type: MappingType.ValueToText, options: { existing: { text: 'existing' } } });
expect(config.mappings![1]).toEqual({ type: MappingType.ValueToText, options: { first: { text: 'first' } } });
expect(config.mappings![2]).toEqual({ type: MappingType.ValueToText, options: { second: { text: 'second' } } });
});
});
describe('getLinksSupplier', () => {
@@ -341,7 +341,7 @@ export function setDynamicConfigValue(config: FieldConfig, value: DynamicConfigV
return;
}
const val = item.process(value.value, context, item.settings);
let val = item.process(value.value, context, item.settings);
const remove = val === undefined || val === null;
@@ -352,6 +352,15 @@ export function setDynamicConfigValue(config: FieldConfig, value: DynamicConfigV
unset(config, item.path);
}
} else {
// Merge arrays (e.g. mappings) when multiple overrides target the same field
if (Array.isArray(val)) {
const existingValue = item.isCustom ? get(config.custom, item.path) : get(config, item.path);
if (Array.isArray(existingValue)) {
val = [...existingValue, ...val];
}
}
if (item.isCustom) {
if (!config.custom) {
config.custom = {};
-20
View File
@@ -332,10 +332,6 @@ export interface FeatureToggles {
*/
alertingUIUseFullyCompatBackendFilters?: boolean;
/**
* Enables creating alert rules from a panel using a drawer UI
*/
createAlertRuleFromPanel?: boolean;
/**
* Enable Grafana to have a remote Alertmanager instance as the primary Alertmanager.
*/
alertmanagerRemotePrimary?: boolean;
@@ -531,10 +527,6 @@ export interface FeatureToggles {
*/
dashboardTemplates?: boolean;
/**
* Sets the logs table as default visualisation in logs explore
*/
logsExploreTableDefaultVisualization?: boolean;
/**
* Enables the new alert list view design
*/
alertingListViewV2?: boolean;
@@ -630,10 +622,6 @@ export interface FeatureToggles {
*/
exploreLogsAggregatedMetrics?: boolean;
/**
* Used in Logs Drilldown to limit the time range
*/
exploreLogsLimitedTimeRange?: boolean;
/**
* Enables the gRPC client to authenticate with the App Platform by using ID & access tokens
*/
appPlatformGrpcClientAuth?: boolean;
@@ -661,10 +649,6 @@ export interface FeatureToggles {
*/
rolePickerDrawer?: boolean;
/**
* Enable unified storage search
*/
unifiedStorageSearch?: boolean;
/**
* Enable sprinkles on unified storage search
*/
unifiedStorageSearchSprinkles?: boolean;
@@ -707,10 +691,6 @@ export interface FeatureToggles {
*/
passwordlessMagicLinkAuthentication?: boolean;
/**
* Display Related Logs in Grafana Metrics Drilldown
*/
exploreMetricsRelatedLogs?: boolean;
/**
* Adds support for quotes and special characters in label values for Prometheus queries
*/
prometheusSpecialCharsInLabelValues?: boolean;
+1
View File
@@ -52,6 +52,7 @@ export const availableIconsIndex = {
bookmark: true,
'book-open': true,
'brackets-curly': true,
brain: true,
'browser-alt': true,
bug: true,
building: true,
-1
View File
@@ -29,7 +29,6 @@
"@grafana-app/source": "./src/internal/index.ts"
},
"./eslint-plugin": {
"@grafana-app/source": "./src/eslint/index.cjs",
"types": "./src/eslint/index.d.ts",
"default": "./src/eslint/index.cjs"
}
@@ -14,6 +14,8 @@ export type Props = React.ComponentProps<typeof TextArea> & {
isConfigured: boolean;
/** Called when the user clicks on the "Reset" button in order to clear the secret */
onReset: () => void;
/** If true, the text area will grow to fill available width. */
grow?: boolean;
};
export const CONFIGURED_TEXT = 'configured';
@@ -35,11 +37,11 @@ const getStyles = (theme: GrafanaTheme2) => {
*
* https://developers.grafana.com/ui/latest/index.html?path=/docs/inputs-secrettextarea--docs
*/
export const SecretTextArea = ({ isConfigured, onReset, ...props }: Props) => {
export const SecretTextArea = ({ isConfigured, onReset, grow, ...props }: Props) => {
const styles = useStyles2(getStyles);
return (
<Stack>
<Box>
<Box grow={grow ? 1 : undefined}>
{!isConfigured && <TextArea {...props} />}
{isConfigured && (
<TextArea
+57 -19
View File
@@ -142,6 +142,24 @@ func (s *SearchHandler) GetAPIRoutes(defs map[string]common.OpenAPIDefinition) *
Schema: spec.StringProperty(),
},
},
{
ParameterProps: spec3.ParameterProps{
Name: "panelType",
In: "query",
Description: "find dashboards using panels of a given plugin type",
Required: false,
Schema: spec.StringProperty(),
},
},
{
ParameterProps: spec3.ParameterProps{
Name: "dataSourceType",
In: "query",
Description: "find dashboards using datasources of a given plugin type",
Required: false,
Schema: spec.StringProperty(),
},
},
{
ParameterProps: spec3.ParameterProps{
Name: "permission",
@@ -430,14 +448,11 @@ func convertHttpSearchRequestToResourceSearchRequest(queryParams url.Values, use
}
}
// The facet term fields
// Apply facet terms
if facets, ok := queryParams["facet"]; ok {
if queryParams.Has("facetLimit") {
if parsed, err := strconv.Atoi(queryParams.Get("facetLimit")); err == nil && parsed > 0 {
facetLimit = parsed
if facetLimit > 1000 {
facetLimit = 1000
}
facetLimit = min(parsed, 1000)
}
}
searchRequest.Facet = make(map[string]*resourcepb.ResourceSearchRequest_Facet)
@@ -449,21 +464,35 @@ func convertHttpSearchRequestToResourceSearchRequest(queryParams url.Values, use
}
}
// The tags filter
if tags, ok := queryParams["tag"]; ok {
if v, ok := queryParams["tag"]; ok {
searchRequest.Options.Fields = append(searchRequest.Options.Fields, &resourcepb.Requirement{
Key: "tags",
Operator: "=",
Values: tags,
Values: v,
})
}
// The libraryPanel filter
if libraryPanel, ok := queryParams["libraryPanel"]; ok {
if v, ok := queryParams["panelType"]; ok {
searchRequest.Options.Fields = append(searchRequest.Options.Fields, &resourcepb.Requirement{
Key: resource.SEARCH_FIELD_PREFIX + builders.DASHBOARD_PANEL_TYPES,
Operator: "=",
Values: v,
})
}
if v, ok := queryParams["dataSourceType"]; ok {
searchRequest.Options.Fields = append(searchRequest.Options.Fields, &resourcepb.Requirement{
Key: resource.SEARCH_FIELD_PREFIX + builders.DASHBOARD_DS_TYPES,
Operator: "=",
Values: v,
})
}
if v, ok := queryParams["libraryPanel"]; ok {
searchRequest.Options.Fields = append(searchRequest.Options.Fields, &resourcepb.Requirement{
Key: builders.DASHBOARD_LIBRARY_PANEL_REFERENCE,
Operator: "=",
Values: libraryPanel,
Values: v,
})
}
@@ -523,6 +552,7 @@ func (s *SearchHandler) getDashboardsUIDsSharedWithUser(ctx context.Context, use
// gets dashboards that the user was granted read access to
permissions := user.GetPermissions()
dashboardPermissions := permissions[dashboards.ActionDashboardsRead]
folderPermissions := permissions[dashboards.ActionFoldersRead]
dashboardUids := make([]string, 0)
sharedDashboards := make([]string, 0)
@@ -533,6 +563,13 @@ func (s *SearchHandler) getDashboardsUIDsSharedWithUser(ctx context.Context, use
}
}
}
for _, folderPermission := range folderPermissions {
if folderUid, found := strings.CutPrefix(folderPermission, dashboards.ScopeFoldersPrefix); found {
if !slices.Contains(dashboardUids, folderUid) && folderUid != foldermodel.SharedWithMeFolderUID && folderUid != foldermodel.GeneralFolderUID {
dashboardUids = append(dashboardUids, folderUid)
}
}
}
if len(dashboardUids) == 0 {
return sharedDashboards, nil
@@ -543,9 +580,15 @@ func (s *SearchHandler) getDashboardsUIDsSharedWithUser(ctx context.Context, use
return sharedDashboards, err
}
folderKey, err := asResourceKey(user.GetNamespace(), folders.RESOURCE)
if err != nil {
return sharedDashboards, err
}
dashboardSearchRequest := &resourcepb.ResourceSearchRequest{
Fields: []string{"folder"},
Limit: int64(len(dashboardUids)),
Federated: []*resourcepb.ResourceKey{folderKey},
Fields: []string{"folder"},
Limit: int64(len(dashboardUids)),
Options: &resourcepb.ListOptions{
Key: key,
Fields: []*resourcepb.Requirement{{
@@ -581,12 +624,6 @@ func (s *SearchHandler) getDashboardsUIDsSharedWithUser(ctx context.Context, use
}
}
// only folders the user has access to will be returned here
folderKey, err := asResourceKey(user.GetNamespace(), folders.RESOURCE)
if err != nil {
return sharedDashboards, err
}
folderSearchRequest := &resourcepb.ResourceSearchRequest{
Fields: []string{"folder"},
Limit: int64(len(allFolders)),
@@ -599,6 +636,7 @@ func (s *SearchHandler) getDashboardsUIDsSharedWithUser(ctx context.Context, use
}},
},
}
// only folders the user has access to will be returned here
foldersResult, err := s.client.Search(ctx, folderSearchRequest)
if err != nil {
return sharedDashboards, err
+27 -3
View File
@@ -507,6 +507,15 @@ func TestSearchHandlerSharedDashboards(t *testing.T) {
[]byte("publicfolder"), // folder uid
},
},
{
Key: &resourcepb.ResourceKey{
Name: "sharedfolder",
Resource: "folder",
},
Cells: [][]byte{
[]byte("privatefolder"), // folder uid
},
},
},
},
}
@@ -550,6 +559,15 @@ func TestSearchHandlerSharedDashboards(t *testing.T) {
[]byte("privatefolder"), // folder uid
},
},
{
Key: &resourcepb.ResourceKey{
Name: "sharedfolder",
Resource: "folder",
},
Cells: [][]byte{
[]byte("privatefolder"), // folder uid
},
},
},
},
}
@@ -571,6 +589,7 @@ func TestSearchHandlerSharedDashboards(t *testing.T) {
allPermissions := make(map[int64]map[string][]string)
permissions := make(map[string][]string)
permissions[dashboards.ActionDashboardsRead] = []string{"dashboards:uid:dashboardinroot", "dashboards:uid:dashboardinprivatefolder", "dashboards:uid:dashboardinpublicfolder"}
permissions[dashboards.ActionFoldersRead] = []string{"folders:uid:sharedfolder"}
allPermissions[1] = permissions
// "Permissions" is where we store the uid of dashboards shared with the user
req = req.WithContext(identity.WithRequester(req.Context(), &user.SignedInUser{Namespace: "test", OrgID: 1, Permissions: allPermissions}))
@@ -581,14 +600,19 @@ func TestSearchHandlerSharedDashboards(t *testing.T) {
// first call gets all dashboards user has permission for
firstCall := mockClient.MockCalls[0]
assert.Equal(t, firstCall.Options.Fields[0].Values, []string{"dashboardinroot", "dashboardinprivatefolder", "dashboardinpublicfolder"})
assert.Equal(t, firstCall.Options.Fields[0].Values, []string{"dashboardinroot", "dashboardinprivatefolder", "dashboardinpublicfolder", "sharedfolder"})
// verify federated field is set to include folders
assert.NotNil(t, firstCall.Federated)
assert.Equal(t, 1, len(firstCall.Federated))
assert.Equal(t, "folder.grafana.app", firstCall.Federated[0].Group)
assert.Equal(t, "folders", firstCall.Federated[0].Resource)
// second call gets folders associated with the previous dashboards
secondCall := mockClient.MockCalls[1]
assert.Equal(t, secondCall.Options.Fields[0].Values, []string{"privatefolder", "publicfolder"})
// lastly, search ONLY for dashboards user has permission to read that are within folders the user does NOT have
// lastly, search ONLY for dashboards and folders user has permission to read that are within folders the user does NOT have
// permission to read
thirdCall := mockClient.MockCalls[2]
assert.Equal(t, thirdCall.Options.Fields[0].Values, []string{"dashboardinprivatefolder"})
assert.Equal(t, thirdCall.Options.Fields[0].Values, []string{"dashboardinprivatefolder", "sharedfolder"})
resp := rr.Result()
defer func() {
@@ -71,7 +71,6 @@ type cachingDatasourceProvider struct {
}
func (q *cachingDatasourceProvider) GetDatasourceProvider(pluginJson plugins.JSONData) PluginDatasourceProvider {
group, _ := plugins.GetDatasourceGroupNameFromPluginID(pluginJson.ID)
return &scopedDatasourceProvider{
plugin: pluginJson,
dsService: q.dsService,
@@ -81,7 +80,7 @@ func (q *cachingDatasourceProvider) GetDatasourceProvider(pluginJson plugins.JSO
mapper: q.converter.mapper,
plugin: pluginJson.ID,
alias: pluginJson.AliasIDs,
group: group,
group: pluginJson.ID,
},
}
}
+24 -19
View File
@@ -37,6 +37,11 @@ var (
_ builder.APIGroupBuilder = (*DataSourceAPIBuilder)(nil)
)
type DataSourceAPIBuilderConfig struct {
LoadQueryTypes bool
UseDualWriter bool
}
// DataSourceAPIBuilder is used just so wire has something unique to return
type DataSourceAPIBuilder struct {
datasourceResourceInfo utils.ResourceInfo
@@ -46,7 +51,7 @@ type DataSourceAPIBuilder struct {
contextProvider PluginContextWrapper
accessControl accesscontrol.AccessControl
queryTypes *queryV0.QueryTypeDefinitionList
configCrudUseNewApis bool
cfg DataSourceAPIBuilderConfig
dataSourceCRUDMetric *prometheus.HistogramVec
}
@@ -89,20 +94,24 @@ func RegisterAPIService(
return nil, fmt.Errorf("plugin client is not a PluginClient: %T", pluginClient)
}
groupName := pluginJSON.ID + ".datasource.grafana.app"
builder, err = NewDataSourceAPIBuilder(
groupName,
pluginJSON,
client,
datasources.GetDatasourceProvider(pluginJSON),
contextProvider,
accessControl,
//nolint:staticcheck // not yet migrated to OpenFeature
features.IsEnabledGlobally(featuremgmt.FlagDatasourceQueryTypes),
//nolint:staticcheck // not yet migrated to OpenFeature
features.IsEnabledGlobally(featuremgmt.FlagQueryServiceWithConnections),
DataSourceAPIBuilderConfig{
//nolint:staticcheck // not yet migrated to OpenFeature
LoadQueryTypes: features.IsEnabledGlobally(featuremgmt.FlagDatasourceQueryTypes),
UseDualWriter: false,
},
)
if err != nil {
return nil, err
}
builder.SetDataSourceCRUDMetrics(dataSourceCRUDMetric)
apiRegistrar.RegisterAPI(builder)
@@ -120,31 +129,27 @@ type PluginClient interface {
}
func NewDataSourceAPIBuilder(
groupName string,
plugin plugins.JSONData,
client PluginClient,
datasources PluginDatasourceProvider,
contextProvider PluginContextWrapper,
accessControl accesscontrol.AccessControl,
loadQueryTypes bool,
configCrudUseNewApis bool,
cfg DataSourceAPIBuilderConfig,
) (*DataSourceAPIBuilder, error) {
group, err := plugins.GetDatasourceGroupNameFromPluginID(plugin.ID)
if err != nil {
return nil, err
}
builder := &DataSourceAPIBuilder{
datasourceResourceInfo: datasourceV0.DataSourceResourceInfo.WithGroupAndShortName(group, plugin.ID),
datasourceResourceInfo: datasourceV0.DataSourceResourceInfo.WithGroupAndShortName(groupName, plugin.ID),
pluginJSON: plugin,
client: client,
datasources: datasources,
contextProvider: contextProvider,
accessControl: accessControl,
configCrudUseNewApis: configCrudUseNewApis,
cfg: cfg,
}
if loadQueryTypes {
var err error
if cfg.LoadQueryTypes {
// In the future, this will somehow come from the plugin
builder.queryTypes, err = getHardcodedQueryTypes(group)
builder.queryTypes, err = getHardcodedQueryTypes(groupName)
}
return builder, err
}
@@ -154,9 +159,9 @@ func getHardcodedQueryTypes(group string) (*queryV0.QueryTypeDefinitionList, err
var err error
var raw json.RawMessage
switch group {
case "testdata.datasource.grafana.app":
case "testdata.datasource.grafana.app", "grafana-testdata-datasource":
raw, err = kinds.QueryTypeDefinitionListJSON()
case "prometheus.datasource.grafana.app":
case "prometheus.datasource.grafana.app", "prometheus":
raw, err = models.QueryTypeDefinitionListJSON()
}
if err != nil {
@@ -233,7 +238,7 @@ func (b *DataSourceAPIBuilder) UpdateAPIGroupInfo(apiGroupInfo *genericapiserver
storage["connections"] = &noopREST{} // hidden from openapi
storage["connections/query"] = storage[ds.StoragePath("query")] // deprecated in openapi
if b.configCrudUseNewApis {
if b.cfg.UseDualWriter {
legacyStore := &legacyStorage{
datasources: b.datasources,
resourceInfo: &ds,
@@ -2,6 +2,8 @@ package extras
import (
apisprovisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
ghconnection "github.com/grafana/grafana/apps/provisioning/pkg/connection/github"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/apps/provisioning/pkg/repository/git"
"github.com/grafana/grafana/apps/provisioning/pkg/repository/github"
@@ -42,6 +44,15 @@ func ProvideProvisioningOSSRepositoryExtras(
}
}
func ProvideProvisioningOSSConnectionExtras(
_ *setting.Cfg,
ghFactory ghconnection.GithubFactory,
) []connection.Extra {
return []connection.Extra{
ghconnection.Extra(ghFactory),
}
}
func ProvideExtraWorkers(pullRequestWorker *pullrequest.PullRequestWorker) []jobs.Worker {
return []jobs.Worker{pullRequestWorker}
}
@@ -54,3 +65,12 @@ func ProvideFactoryFromConfig(cfg *setting.Cfg, extras []repository.Extra) (repo
return repository.ProvideFactory(enabledTypes, extras)
}
func ProvideConnectionFactoryFromConfig(cfg *setting.Cfg, extras []connection.Extra) (connection.Factory, error) {
enabledTypes := make(map[apisprovisioning.ConnectionType]struct{}, len(cfg.ProvisioningRepositoryTypes))
for _, e := range cfg.ProvisioningRepositoryTypes {
enabledTypes[apisprovisioning.ConnectionType(e)] = struct{}{}
}
return connection.ProvideFactory(enabledTypes, extras)
}
+50 -6
View File
@@ -16,6 +16,7 @@ import (
"github.com/grafana/grafana/apps/provisioning/pkg/auth"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/apps/provisioning/pkg/safepath"
"github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/resources"
)
@@ -121,7 +122,7 @@ func (c *filesConnector) handleRequest(ctx context.Context, name string, r *http
return
}
obj, err := c.handleMethodRequest(ctx, r, opts, isDir, dualReadWriter)
obj, err := c.handleMethodRequest(ctx, r, name, opts, isDir, dualReadWriter, readWriter)
if err != nil {
logger.Debug("got an error after processing request", "error", err)
respondWithError(responder, err)
@@ -175,8 +176,16 @@ func (c *filesConnector) parseRequestOptions(r *http.Request, name string, repo
}
opts.Path = path
if err := resources.IsPathSupported(opts.Path); err != nil {
return opts, err
// For GET requests, allow read-only files like .md in addition to resource files
// For write operations, only allow resource files (yml, yaml, json)
if r.Method == http.MethodGet {
if err := resources.IsReadablePath(opts.Path); err != nil {
return opts, err
}
} else {
if err := resources.IsPathSupported(opts.Path); err != nil {
return opts, err
}
}
return opts, nil
@@ -199,10 +208,10 @@ func (c *filesConnector) handleDirectoryListing(ctx context.Context, name string
}
// handleMethodRequest routes the request to the appropriate handler based on HTTP method.
func (c *filesConnector) handleMethodRequest(ctx context.Context, r *http.Request, opts resources.DualWriteOptions, isDir bool, dualReadWriter *resources.DualReadWriter) (*provisioning.ResourceWrapper, error) {
func (c *filesConnector) handleMethodRequest(ctx context.Context, r *http.Request, repoName string, opts resources.DualWriteOptions, isDir bool, dualReadWriter *resources.DualReadWriter, readWriter repository.ReaderWriter) (*provisioning.ResourceWrapper, error) {
switch r.Method {
case http.MethodGet:
return c.handleGet(ctx, opts, dualReadWriter)
return c.handleGet(ctx, repoName, opts, dualReadWriter, readWriter)
case http.MethodPost:
return c.handlePost(ctx, r, opts, isDir, dualReadWriter)
case http.MethodPut:
@@ -214,7 +223,12 @@ func (c *filesConnector) handleMethodRequest(ctx context.Context, r *http.Reques
}
}
func (c *filesConnector) handleGet(ctx context.Context, opts resources.DualWriteOptions, dualReadWriter *resources.DualReadWriter) (*provisioning.ResourceWrapper, error) {
func (c *filesConnector) handleGet(ctx context.Context, repoName string, opts resources.DualWriteOptions, dualReadWriter *resources.DualReadWriter, readWriter repository.ReaderWriter) (*provisioning.ResourceWrapper, error) {
// For raw files (like .md), read directly without parsing as k8s resource
if resources.IsRawFile(opts.Path) {
return c.handleGetRawFile(ctx, repoName, opts, readWriter)
}
resource, err := dualReadWriter.Read(ctx, opts.Path, opts.Ref)
if err != nil {
return nil, err
@@ -222,6 +236,36 @@ func (c *filesConnector) handleGet(ctx context.Context, opts resources.DualWrite
return resource.AsResourceWrapper(), nil
}
// handleGetRawFile reads a raw file (like README.md) and returns it without parsing.
func (c *filesConnector) handleGetRawFile(ctx context.Context, repoName string, opts resources.DualWriteOptions, readWriter repository.ReaderWriter) (*provisioning.ResourceWrapper, error) {
// Authorize the read operation
if err := c.authorizeListFiles(ctx, repoName); err != nil {
return nil, err
}
info, err := readWriter.Read(ctx, opts.Path, opts.Ref)
if err != nil {
if err == repository.ErrFileNotFound {
return nil, apierrors.NewNotFound(provisioning.RepositoryResourceInfo.GroupResource(), opts.Path)
}
return nil, fmt.Errorf("read raw file: %w", err)
}
// Return the raw content in the ResourceWrapper
return &provisioning.ResourceWrapper{
Path: info.Path,
Ref: info.Ref,
Hash: info.Hash,
Resource: provisioning.ResourceObjects{
File: v0alpha1.Unstructured{
Object: map[string]any{
"content": string(info.Data),
},
},
},
}, nil
}
func (c *filesConnector) handlePost(ctx context.Context, r *http.Request, opts resources.DualWriteOptions, isDir bool, dualReadWriter *resources.DualReadWriter) (*provisioning.ResourceWrapper, error) {
// Check if this is a move operation (originalPath query parameter is present)
if opts.OriginalPath != "" {
@@ -0,0 +1,234 @@
package provisioning
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/auth"
"github.com/grafana/grafana/apps/provisioning/pkg/repository"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/resources"
)
func TestParseRequestOptionsPathValidation(t *testing.T) {
tests := []struct {
name string
method string
path string
wantErr bool
errContains string
}{
{
name: "GET json file allowed",
method: http.MethodGet,
path: "dashboard.json",
},
{
name: "GET yaml file allowed",
method: http.MethodGet,
path: "dashboard.yaml",
},
{
name: "GET yml file allowed",
method: http.MethodGet,
path: "dashboard.yml",
},
{
name: "GET markdown file allowed",
method: http.MethodGet,
path: "README.md",
},
{
name: "GET nested markdown file allowed",
method: http.MethodGet,
path: "folder/subfolder/README.md",
},
{
name: "GET txt file not allowed",
method: http.MethodGet,
path: "file.txt",
wantErr: true,
errContains: "unsupported file extension",
},
{
name: "POST json file allowed",
method: http.MethodPost,
path: "dashboard.json",
},
{
name: "POST markdown file not allowed",
method: http.MethodPost,
path: "README.md",
wantErr: true,
errContains: "unsupported file extension",
},
{
name: "PUT markdown file not allowed",
method: http.MethodPut,
path: "README.md",
wantErr: true,
errContains: "unsupported file extension",
},
{
name: "DELETE markdown file not allowed",
method: http.MethodDelete,
path: "README.md",
wantErr: true,
errContains: "unsupported file extension",
},
{
name: "GET directory allowed",
method: http.MethodGet,
path: "dashboards/",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockRepo := repository.NewMockRepository(t)
mockRepo.On("Config").Return(&provisioning.Repository{
Spec: provisioning.RepositorySpec{
Title: "test-repo",
},
}).Maybe()
connector := &filesConnector{}
r := httptest.NewRequest(tt.method, "/test-repo/files/"+tt.path, nil)
opts, err := connector.parseRequestOptions(r, "test-repo", mockRepo)
if tt.wantErr {
require.Error(t, err)
require.Contains(t, err.Error(), tt.errContains)
} else {
require.NoError(t, err)
require.Equal(t, tt.path, opts.Path)
}
})
}
}
func TestHandleGetRawFile(t *testing.T) {
tests := []struct {
name string
path string
fileContent string
readError error
wantErr bool
errContains string
expectedResult string
}{
{
name: "successful readme read",
path: "README.md",
fileContent: "# Hello World\n\nThis is a test.",
expectedResult: "# Hello World\n\nThis is a test.",
},
{
name: "nested readme read",
path: "folder/README.md",
fileContent: "# Folder Readme",
expectedResult: "# Folder Readme",
},
{
name: "file not found",
path: "README.md",
readError: repository.ErrFileNotFound,
wantErr: true,
errContains: "not found",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockReadWriter := repository.NewMockReaderWriter(t)
mockAccess := auth.NewMockAccessChecker(t)
// Setup auth mock - always allow
mockAccess.EXPECT().Check(mock.Anything, mock.Anything, mock.Anything).Return(nil).Maybe()
// Setup Read mock
if tt.readError != nil {
mockReadWriter.EXPECT().Read(mock.Anything, tt.path, "").Return(nil, tt.readError)
} else {
mockReadWriter.EXPECT().Read(mock.Anything, tt.path, "").Return(&repository.FileInfo{
Path: tt.path,
Data: []byte(tt.fileContent),
Ref: "main",
Hash: "abc123",
}, nil)
}
connector := &filesConnector{
access: mockAccess,
}
opts := resources.DualWriteOptions{
Path: tt.path,
Ref: "",
}
result, err := connector.handleGetRawFile(context.Background(), "test-repo", opts, mockReadWriter)
if tt.wantErr {
require.Error(t, err)
require.Contains(t, err.Error(), tt.errContains)
} else {
require.NoError(t, err)
require.NotNil(t, result)
require.Equal(t, tt.path, result.Path)
// Check that content is in the response
content, ok := result.Resource.File.Object["content"]
require.True(t, ok, "content field should exist")
require.Equal(t, tt.expectedResult, content)
}
})
}
}
func TestIsRawFileIntegration(t *testing.T) {
tests := []struct {
name string
path string
expected bool
}{
{
name: "README.md is raw",
path: "README.md",
expected: true,
},
{
name: "nested README.md is raw",
path: "folder/subfolder/README.md",
expected: true,
},
{
name: "dashboard.json is not raw",
path: "dashboard.json",
expected: false,
},
{
name: "dashboard.yaml is not raw",
path: "dashboard.yaml",
expected: false,
},
{
name: "directory is not raw",
path: "folder/",
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := resources.IsRawFile(tt.path)
require.Equal(t, tt.expected, result)
})
}
}
+68 -23
View File
@@ -30,7 +30,7 @@ import (
provisioning "github.com/grafana/grafana/apps/provisioning/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/apps/provisioning/pkg/auth"
connectionvalidation "github.com/grafana/grafana/apps/provisioning/pkg/connection"
"github.com/grafana/grafana/apps/provisioning/pkg/connection"
appcontroller "github.com/grafana/grafana/apps/provisioning/pkg/controller"
clientset "github.com/grafana/grafana/apps/provisioning/pkg/generated/clientset/versioned"
client "github.com/grafana/grafana/apps/provisioning/pkg/generated/clientset/versioned/typed/provisioning/v0alpha1"
@@ -105,20 +105,21 @@ type APIBuilder struct {
jobs.Queue
jobs.Store
}
jobHistoryConfig *JobHistoryConfig
jobHistoryLoki *jobs.LokiJobHistory
resourceLister resources.ResourceLister
dashboardAccess legacy.MigrationDashboardAccessor
unified resource.ResourceClient
repoFactory repository.Factory
client client.ProvisioningV0alpha1Interface
access auth.AccessChecker
accessWithAdmin auth.AccessChecker
accessWithEditor auth.AccessChecker
accessWithViewer auth.AccessChecker
statusPatcher *appcontroller.RepositoryStatusPatcher
healthChecker *controller.HealthChecker
validator repository.RepositoryValidator
jobHistoryConfig *JobHistoryConfig
jobHistoryLoki *jobs.LokiJobHistory
resourceLister resources.ResourceLister
dashboardAccess legacy.MigrationDashboardAccessor
unified resource.ResourceClient
repoFactory repository.Factory
connectionFactory connection.Factory
client client.ProvisioningV0alpha1Interface
access auth.AccessChecker
accessWithAdmin auth.AccessChecker
accessWithEditor auth.AccessChecker
accessWithViewer auth.AccessChecker
statusPatcher *appcontroller.RepositoryStatusPatcher
healthChecker *controller.HealthChecker
repoValidator repository.RepositoryValidator
// Extras provides additional functionality to the API.
extras []Extra
extraWorkers []jobs.Worker
@@ -133,6 +134,7 @@ type APIBuilder struct {
func NewAPIBuilder(
onlyApiServer bool,
repoFactory repository.Factory,
connectionFactory connection.Factory,
features featuremgmt.FeatureToggles,
unified resource.ResourceClient,
configProvider apiserver.RestConfigProvider,
@@ -176,6 +178,7 @@ func NewAPIBuilder(
usageStats: usageStats,
features: features,
repoFactory: repoFactory,
connectionFactory: connectionFactory,
clients: clients,
parsers: parsers,
repositoryResources: resources.NewRepositoryResourcesFactory(parsers, clients, resourceLister),
@@ -192,7 +195,7 @@ func NewAPIBuilder(
allowedTargets: allowedTargets,
allowImageRendering: allowImageRendering,
registry: registry,
validator: repository.NewValidator(minSyncInterval, allowedTargets, allowImageRendering),
repoValidator: repository.NewValidator(minSyncInterval, allowedTargets, allowImageRendering),
useExclusivelyAccessCheckerForAuthz: useExclusivelyAccessCheckerForAuthz,
}
@@ -253,6 +256,7 @@ func RegisterAPIService(
extraBuilders []ExtraBuilder,
extraWorkers []jobs.Worker,
repoFactory repository.Factory,
connectionFactory connection.Factory,
) (*APIBuilder, error) {
//nolint:staticcheck // not yet migrated to OpenFeature
if !features.IsEnabledGlobally(featuremgmt.FlagProvisioning) {
@@ -271,6 +275,7 @@ func RegisterAPIService(
builder := NewAPIBuilder(
cfg.DisableControllers,
repoFactory,
connectionFactory,
features,
client,
configProvider,
@@ -641,7 +646,7 @@ func (b *APIBuilder) UpdateAPIGroupInfo(apiGroupInfo *genericapiserver.APIGroupI
storage[provisioning.ConnectionResourceInfo.StoragePath("repositories")] = NewConnectionRepositoriesConnector()
// TODO: Add some logic so that the connectors can registered themselves and we don't have logic all over the place
storage[provisioning.RepositoryResourceInfo.StoragePath("test")] = NewTestConnector(b, repository.NewRepositoryTesterWithExistingChecker(repository.NewSimpleRepositoryTester(b.validator), b.VerifyAgainstExistingRepositories))
storage[provisioning.RepositoryResourceInfo.StoragePath("test")] = NewTestConnector(b, repository.NewRepositoryTesterWithExistingChecker(repository.NewSimpleRepositoryTester(b.repoValidator), b.VerifyAgainstExistingRepositories))
storage[provisioning.RepositoryResourceInfo.StoragePath("files")] = NewFilesConnector(b, b.parsers, b.clients, b.accessWithAdmin)
storage[provisioning.RepositoryResourceInfo.StoragePath("refs")] = NewRefsConnector(b)
storage[provisioning.RepositoryResourceInfo.StoragePath("resources")] = &listConnector{
@@ -682,10 +687,15 @@ func (b *APIBuilder) Mutate(ctx context.Context, a admission.Attributes, o admis
if ok {
return nil
}
// TODO: complete this as part of https://github.com/grafana/git-ui-sync-project/issues/700
c, ok := obj.(*provisioning.Connection)
if ok {
return connectionvalidation.MutateConnection(c)
conn, err := b.asConnection(ctx, c, nil)
if err != nil {
return err
}
return conn.Mutate(ctx)
}
r, ok := obj.(*provisioning.Repository)
@@ -736,9 +746,15 @@ func (b *APIBuilder) Validate(ctx context.Context, a admission.Attributes, o adm
return nil
}
connection, ok := obj.(*provisioning.Connection)
// Validate connections
c, ok := obj.(*provisioning.Connection)
if ok {
return connectionvalidation.ValidateConnection(connection)
conn, err := b.asConnection(ctx, c, a.GetOldObject())
if err != nil {
return err
}
return conn.Validate(ctx)
}
// Validate Jobs
@@ -758,7 +774,7 @@ func (b *APIBuilder) Validate(ctx context.Context, a admission.Attributes, o adm
// the only time to add configuration checks here is if you need to compare
// the incoming change to the current configuration
isCreate := a.GetOperation() == admission.Create
list := b.validator.ValidateRepository(repo, isCreate)
list := b.repoValidator.ValidateRepository(repo, isCreate)
cfg := repo.Config()
if a.GetOperation() == admission.Update {
@@ -831,7 +847,7 @@ func (b *APIBuilder) GetPostStartHooks() (map[string]genericapiserver.PostStartH
}
b.statusPatcher = appcontroller.NewRepositoryStatusPatcher(b.GetClient())
b.healthChecker = controller.NewHealthChecker(b.statusPatcher, b.registry, repository.NewSimpleRepositoryTester(b.validator))
b.healthChecker = controller.NewHealthChecker(b.statusPatcher, b.registry, repository.NewSimpleRepositoryTester(b.repoValidator))
// if running solely CRUD, skip the rest of the setup
if b.onlyApiServer {
@@ -1449,6 +1465,35 @@ func (b *APIBuilder) asRepository(ctx context.Context, obj runtime.Object, old r
return b.repoFactory.Build(ctx, r)
}
func (b *APIBuilder) asConnection(ctx context.Context, obj runtime.Object, old runtime.Object) (connection.Connection, error) {
if obj == nil {
return nil, fmt.Errorf("missing connection object")
}
c, ok := obj.(*provisioning.Connection)
if !ok {
return nil, fmt.Errorf("expected connection object")
}
// Copy previous values if they exist
if old != nil {
o, ok := old.(*provisioning.Connection)
if ok && !o.Secure.IsZero() {
if c.Secure.PrivateKey.IsZero() {
c.Secure.PrivateKey = o.Secure.PrivateKey
}
if c.Secure.Token.IsZero() {
c.Secure.Token = o.Secure.Token
}
if c.Secure.ClientSecret.IsZero() {
c.Secure.ClientSecret = o.Secure.ClientSecret
}
}
}
return b.connectionFactory.Build(ctx, c)
}
func getJSONResponse(ref string) *spec3.Responses {
return &spec3.Responses{
ResponsesProps: spec3.ResponsesProps{
@@ -28,7 +28,7 @@ func TestAPIBuilderValidate(t *testing.T) {
repoFactory: factory,
allowedTargets: []v0alpha1.SyncTargetType{v0alpha1.SyncTargetTypeFolder},
allowImageRendering: false,
validator: validator,
repoValidator: validator,
}
t.Run("min sync interval is less than 10 seconds", func(t *testing.T) {
@@ -3,6 +3,7 @@ package resources
import (
"errors"
"path"
"strings"
"github.com/grafana/grafana/apps/provisioning/pkg/safepath"
)
@@ -15,9 +16,65 @@ var (
const maxPathDepth = 8
// IsPathSupported checks if the file path is supported by the provisioning API.
// it also validates if the path is safe and if the file extension is supported.
// resourceExtensions are file extensions that contain k8s resources and can be parsed
var resourceExtensions = map[string]bool{
".yml": true,
".yaml": true,
".json": true,
}
// readOnlyExtensions are file extensions that can be read as raw content (read-only)
var readOnlyExtensions = map[string]bool{
".md": true,
}
// IsPathSupported checks if the file path is supported by the provisioning API for write operations.
// It validates if the path is safe and if the file extension is a resource type (yml, yaml, json).
func IsPathSupported(filePath string) error {
if err := validatePathBasics(filePath); err != nil {
return err
}
// Only check file extension if it's not a folder path
if !safepath.IsDir(filePath) {
ext := path.Ext(filePath)
if !resourceExtensions[ext] {
return ErrUnsupportedFileExtension
}
}
return nil
}
// IsReadablePath checks if the file path is supported for read operations.
// This includes resource files (yml, yaml, json) and read-only files (md).
func IsReadablePath(filePath string) error {
if err := validatePathBasics(filePath); err != nil {
return err
}
// Only check file extension if it's not a folder path
if !safepath.IsDir(filePath) {
ext := path.Ext(filePath)
if !resourceExtensions[ext] && !readOnlyExtensions[ext] {
return ErrUnsupportedFileExtension
}
}
return nil
}
// IsRawFile checks if the file path is a read-only raw file (not a k8s resource).
func IsRawFile(filePath string) bool {
if safepath.IsDir(filePath) {
return false
}
ext := strings.ToLower(path.Ext(filePath))
return readOnlyExtensions[ext]
}
// validatePathBasics performs common path validation checks.
func validatePathBasics(filePath string) error {
// Validate the path for any traversal attempts first
if err := safepath.IsSafe(filePath); err != nil {
return err
@@ -31,12 +88,5 @@ func IsPathSupported(filePath string) error {
return ErrNotRelative
}
// Only check file extension if it's not a folder path
if !safepath.IsDir(filePath) {
if ext := path.Ext(filePath); ext != ".yml" && ext != ".yaml" && ext != ".json" {
return ErrUnsupportedFileExtension
}
}
return nil
}
@@ -38,6 +38,11 @@ func TestIsPathSupported(t *testing.T) {
path: "dashboards/my-dashboard.txt",
expectedErr: ErrUnsupportedFileExtension,
},
{
name: "markdown file not supported for write",
path: "dashboards/README.md",
expectedErr: ErrUnsupportedFileExtension,
},
{
name: "path traversal attempt",
path: "../dashboards/my-dashboard.yaml",
@@ -70,3 +75,127 @@ func TestIsPathSupported(t *testing.T) {
})
}
}
func TestIsReadablePath(t *testing.T) {
tests := []struct {
name string
path string
expectedErr error
}{
{
name: "valid yaml file",
path: "dashboards/my-dashboard.yaml",
},
{
name: "valid yml file",
path: "dashboards/my-dashboard.yml",
},
{
name: "valid json file",
path: "dashboards/my-dashboard.json",
},
{
name: "valid markdown file",
path: "dashboards/README.md",
},
{
name: "markdown file in nested path",
path: "dashboards/folder1/folder2/README.md",
},
{
name: "valid directory path",
path: "dashboards/folder1/",
},
{
name: "unsupported file extension",
path: "dashboards/my-dashboard.txt",
expectedErr: ErrUnsupportedFileExtension,
},
{
name: "path traversal attempt",
path: "../dashboards/README.md",
expectedErr: safepath.ErrPathTraversalAttempt,
},
{
name: "path too deep",
path: "level1/level2/level3/level4/level5/level6/level7/level8/level9/README.md",
expectedErr: ErrPathTooDeep,
},
{
name: "absolute path",
path: "/etc/dashboards/README.md",
expectedErr: ErrNotRelative,
},
{
name: "empty directory path",
path: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := IsReadablePath(tt.path)
if tt.expectedErr != nil {
require.ErrorIs(t, err, tt.expectedErr)
} else {
require.NoError(t, err)
}
})
}
}
func TestIsRawFile(t *testing.T) {
tests := []struct {
name string
path string
expected bool
}{
{
name: "markdown file is raw",
path: "README.md",
expected: true,
},
{
name: "nested markdown file is raw",
path: "dashboards/folder/README.md",
expected: true,
},
{
name: "uppercase MD extension",
path: "README.MD",
expected: true,
},
{
name: "yaml file is not raw",
path: "dashboard.yaml",
expected: false,
},
{
name: "json file is not raw",
path: "dashboard.json",
expected: false,
},
{
name: "yml file is not raw",
path: "dashboard.yml",
expected: false,
},
{
name: "directory is not raw",
path: "dashboards/",
expected: false,
},
{
name: "empty path is not raw",
path: "",
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := IsRawFile(tt.path)
require.Equal(t, tt.expected, result)
})
}
}
+1
View File
@@ -44,6 +44,7 @@ var provisioningExtras = wire.NewSet(
pullrequest.ProvidePullRequestWorker,
webhooks.ProvideWebhooksWithImages,
extras.ProvideFactoryFromConfig,
extras.ProvideConnectionFactoryFromConfig,
extras.ProvideProvisioningExtraAPIs,
extras.ProvideExtraWorkers,
)
+33 -29
View File
@@ -3,6 +3,7 @@ package server
import (
"github.com/stretchr/testify/mock"
githubconnection "github.com/grafana/grafana/apps/provisioning/pkg/connection/github"
"github.com/grafana/grafana/apps/provisioning/pkg/repository/github"
"github.com/grafana/grafana/apps/secret/pkg/decrypt"
"github.com/grafana/grafana/pkg/infra/db"
@@ -34,24 +35,26 @@ func ProvideTestEnv(
featureMgmt featuremgmt.FeatureToggles,
resourceClient resource.ResourceClient,
idService auth.IDService,
githubFactory *github.Factory,
githubRepoFactory *github.Factory,
githubConnectionFactory githubconnection.GithubFactory,
decryptService decrypt.DecryptService,
) (*TestEnv, error) {
return &TestEnv{
TestingT: testingT,
Server: server,
SQLStore: db,
Cfg: cfg,
NotificationService: ns,
GRPCServer: grpcServer,
PluginRegistry: pluginRegistry,
HTTPClientProvider: httpClientProvider,
OAuthTokenService: oAuthTokenService,
FeatureToggles: featureMgmt,
ResourceClient: resourceClient,
IDService: idService,
GitHubFactory: githubFactory,
DecryptService: decryptService,
TestingT: testingT,
Server: server,
SQLStore: db,
Cfg: cfg,
NotificationService: ns,
GRPCServer: grpcServer,
PluginRegistry: pluginRegistry,
HTTPClientProvider: httpClientProvider,
OAuthTokenService: oAuthTokenService,
FeatureToggles: featureMgmt,
ResourceClient: resourceClient,
IDService: idService,
GithubRepoFactory: githubRepoFactory,
GithubConnectionFactory: githubConnectionFactory,
DecryptService: decryptService,
}, nil
}
@@ -60,18 +63,19 @@ type TestEnv struct {
mock.TestingT
Cleanup(func())
}
Server *Server
SQLStore db.DB
Cfg *setting.Cfg
NotificationService *notifications.NotificationServiceMock
GRPCServer grpcserver.Provider
PluginRegistry registry.Service
HTTPClientProvider httpclient.Provider
OAuthTokenService *oauthtokentest.Service
RequestMiddleware web.Middleware
FeatureToggles featuremgmt.FeatureToggles
ResourceClient resource.ResourceClient
IDService auth.IDService
GitHubFactory *github.Factory
DecryptService decrypt.DecryptService
Server *Server
SQLStore db.DB
Cfg *setting.Cfg
NotificationService *notifications.NotificationServiceMock
GRPCServer grpcserver.Provider
PluginRegistry registry.Service
HTTPClientProvider httpclient.Provider
OAuthTokenService *oauthtokentest.Service
RequestMiddleware web.Middleware
FeatureToggles featuremgmt.FeatureToggles
ResourceClient resource.ResourceClient
IDService auth.IDService
GithubRepoFactory *github.Factory
GithubConnectionFactory githubconnection.GithubFactory
DecryptService decrypt.DecryptService
}
+2
View File
@@ -15,6 +15,7 @@ import (
"go.opentelemetry.io/otel/trace"
sdkhttpclient "github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
ghconnection "github.com/grafana/grafana/apps/provisioning/pkg/connection/github"
"github.com/grafana/grafana/apps/provisioning/pkg/repository/github"
"github.com/grafana/grafana/pkg/api"
"github.com/grafana/grafana/pkg/api/avatar"
@@ -297,6 +298,7 @@ var wireBasicSet = wire.NewSet(
notifications.ProvideService,
notifications.ProvideSmtpService,
github.ProvideFactory,
ghconnection.ProvideFactory,
tracing.ProvideService,
tracing.ProvideTracingConfig,
wire.Bind(new(tracing.Tracer), new(*tracing.TracingService)),
+17 -4
View File
File diff suppressed because one or more lines are too long
+1
View File
@@ -72,6 +72,7 @@ import (
var provisioningExtras = wire.NewSet(
extras.ProvideProvisioningOSSRepositoryExtras,
extras.ProvideProvisioningOSSConnectionExtras,
)
var configProviderExtras = wire.NewSet(
+5 -1
View File
@@ -133,7 +133,11 @@ type FeatureFlag struct {
Stage FeatureFlagStage `json:"stage,omitempty"`
Owner codeowner `json:"-"` // Owner person or team that owns this feature flag
// CEL-GO expression. Using the value "true" will mean this is on by default
// Expression defined by the feature_toggles configuration.
// Supports multiple types including boolean, string, integer, float,
// and structured values following the OpenFeature specification.
// Using the value "true" means the feature flag is enabled by default,
// Using the value "1.0" means the default value of the feature flag is 1.0
Expression string `json:"expression,omitempty"`
// Special behavior properties
+4 -3
View File
@@ -8,6 +8,7 @@ import (
clientauthmiddleware "github.com/grafana/grafana/pkg/clientauth/middleware"
"github.com/grafana/grafana/pkg/setting"
"github.com/open-feature/go-sdk/openfeature/memprovider"
sdkhttpclient "github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
"github.com/open-feature/go-sdk/openfeature"
@@ -26,7 +27,7 @@ type OpenFeatureConfig struct {
// HTTPClient is a pre-configured HTTP client (optional, used by features-service + OFREP providers)
HTTPClient *http.Client
// StaticFlags are the feature flags to use with static provider
StaticFlags map[string]bool
StaticFlags map[string]memprovider.InMemoryFlag
// TargetingKey is used for evaluation context
TargetingKey string
// ContextAttrs are additional attributes for evaluation context
@@ -100,7 +101,7 @@ func InitOpenFeatureWithCfg(cfg *setting.Cfg) error {
func createProvider(
providerType string,
u *url.URL,
staticFlags map[string]bool,
staticFlags map[string]memprovider.InMemoryFlag,
httpClient *http.Client,
) (openfeature.FeatureProvider, error) {
if providerType == setting.FeaturesServiceProviderType || providerType == setting.OFREPProviderType {
@@ -117,7 +118,7 @@ func createProvider(
}
}
return newStaticProvider(staticFlags)
return newStaticProvider(staticFlags, standardFeatureFlags)
}
func createHTTPClient(m *clientauthmiddleware.TokenExchangeMiddleware) (*http.Client, error) {
-36
View File
@@ -534,13 +534,6 @@ var (
Owner: grafanaAlertingSquad,
HideFromDocs: true,
},
{
Name: "createAlertRuleFromPanel",
Description: "Enables creating alert rules from a panel using a drawer UI",
Stage: FeatureStageExperimental,
Owner: grafanaAlertingSquad,
FrontendOnly: true,
},
{
Name: "alertmanagerRemotePrimary",
Description: "Enable Grafana to have a remote Alertmanager instance as the primary Alertmanager.",
@@ -879,13 +872,6 @@ var (
Owner: grafanaSharingSquad,
FrontendOnly: false,
},
{
Name: "logsExploreTableDefaultVisualization",
Description: "Sets the logs table as default visualisation in logs explore",
Stage: FeatureStageExperimental,
Owner: grafanaObservabilityLogsSquad,
FrontendOnly: true,
},
{
Name: "alertingListViewV2",
Description: "Enables the new alert list view design",
@@ -1045,13 +1031,6 @@ var (
FrontendOnly: true,
Owner: grafanaObservabilityLogsSquad,
},
{
Name: "exploreLogsLimitedTimeRange",
Description: "Used in Logs Drilldown to limit the time range",
Stage: FeatureStageExperimental,
FrontendOnly: true,
Owner: grafanaObservabilityLogsSquad,
},
{
Name: "appPlatformGrpcClientAuth",
Description: "Enables the gRPC client to authenticate with the App Platform by using ID & access tokens",
@@ -1094,13 +1073,6 @@ var (
Stage: FeatureStageExperimental,
Owner: identityAccessTeam,
},
{
Name: "unifiedStorageSearch",
Description: "Enable unified storage search",
Stage: FeatureStageExperimental,
Owner: grafanaSearchAndStorageSquad,
HideFromDocs: true,
},
{
Name: "unifiedStorageSearchSprinkles",
Description: "Enable sprinkles on unified storage search",
@@ -1169,14 +1141,6 @@ var (
Owner: identityAccessTeam,
HideFromDocs: true,
},
{
Name: "exploreMetricsRelatedLogs",
Description: "Display Related Logs in Grafana Metrics Drilldown",
Stage: FeatureStageExperimental,
Owner: grafanaObservabilityMetricsSquad,
FrontendOnly: true,
HideFromDocs: false,
},
{
Name: "prometheusSpecialCharsInLabelValues",
Description: "Adds support for quotes and special characters in label values for Prometheus queries",
+2 -1
View File
@@ -47,7 +47,8 @@ func ProvideManagerService(cfg *setting.Cfg) (*FeatureManager, error) {
}
mgmt.warnings[key] = "unknown flag in config"
}
mgmt.startup[key] = val
mgmt.startup[key] = val.Variants[val.DefaultVariant] == true
}
// update the values
+1 -1
View File
@@ -29,7 +29,7 @@ func CreateStaticEvaluator(cfg *setting.Cfg) (StaticFlagEvaluator, error) {
return nil, fmt.Errorf("failed to read feature flags from config: %w", err)
}
staticProvider, err := newStaticProvider(staticFlags)
staticProvider, err := newStaticProvider(staticFlags, standardFeatureFlags)
if err != nil {
return nil, fmt.Errorf("failed to create static provider: %w", err)
}
+18 -29
View File
@@ -1,8 +1,13 @@
package featuremgmt
import (
"fmt"
"maps"
"github.com/open-feature/go-sdk/openfeature"
"github.com/open-feature/go-sdk/openfeature/memprovider"
"github.com/grafana/grafana/pkg/setting"
)
// inMemoryBulkProvider is a wrapper around memprovider.InMemoryProvider that
@@ -28,37 +33,21 @@ func (p *inMemoryBulkProvider) ListFlags() ([]string, error) {
return keys, nil
}
func newStaticProvider(confFlags map[string]bool) (openfeature.FeatureProvider, error) {
flags := make(map[string]memprovider.InMemoryFlag, len(standardFeatureFlags))
func newStaticProvider(confFlags map[string]memprovider.InMemoryFlag, standardFlags []FeatureFlag) (openfeature.FeatureProvider, error) {
flags := make(map[string]memprovider.InMemoryFlag, len(standardFlags))
// Parse and add standard flags
for _, flag := range standardFlags {
inMemFlag, err := setting.ParseFlag(flag.Name, flag.Expression)
if err != nil {
return nil, fmt.Errorf("failed to parse flag %s: %w", flag.Name, err)
}
flags[flag.Name] = inMemFlag
}
// Add flags from config.ini file
for name, value := range confFlags {
flags[name] = createInMemoryFlag(name, value)
}
// Add standard flags
for _, flag := range standardFeatureFlags {
if _, exists := flags[flag.Name]; !exists {
enabled := flag.Expression == "true"
flags[flag.Name] = createInMemoryFlag(flag.Name, enabled)
}
}
maps.Copy(flags, confFlags)
return newInMemoryBulkProvider(flags), nil
}
func createInMemoryFlag(name string, enabled bool) memprovider.InMemoryFlag {
variant := "disabled"
if enabled {
variant = "enabled"
}
return memprovider.InMemoryFlag{
Key: name,
DefaultVariant: variant,
Variants: map[string]interface{}{
"enabled": true,
"disabled": false,
},
}
}
@@ -5,6 +5,7 @@ import (
"testing"
"github.com/grafana/grafana/pkg/setting"
"github.com/open-feature/go-sdk/openfeature/memprovider"
"github.com/open-feature/go-sdk/openfeature"
"github.com/stretchr/testify/assert"
@@ -93,3 +94,144 @@ ABCD = true
enabledFeatureManager := mgr.GetEnabled(ctx)
assert.Equal(t, openFeatureEnabledFlags, enabledFeatureManager)
}
func Test_StaticProvider_TypedFlags(t *testing.T) {
tests := []struct {
flags FeatureFlag
defaultValue any
expectedValue any
}{
{
flags: FeatureFlag{
Name: "Flag",
Expression: "true",
},
defaultValue: false,
expectedValue: true,
},
{
flags: FeatureFlag{
Name: "Flag",
Expression: "1.0",
},
defaultValue: 0.0,
expectedValue: 1.0,
},
{
flags: FeatureFlag{
Name: "Flag",
Expression: "blue",
},
defaultValue: "red",
expectedValue: "blue",
},
{
flags: FeatureFlag{
Name: "Flag",
Expression: "1",
},
defaultValue: int64(0),
expectedValue: int64(1),
},
{
flags: FeatureFlag{
Name: "Flag",
Expression: `{ "foo": "bar" }`,
},
expectedValue: map[string]any{"foo": "bar"},
},
}
for _, tt := range tests {
provider, err := newStaticProvider(nil, []FeatureFlag{tt.flags})
assert.NoError(t, err)
var result any
switch tt.expectedValue.(type) {
case bool:
result = provider.BooleanEvaluation(t.Context(), tt.flags.Name, tt.defaultValue.(bool), openfeature.FlattenedContext{}).Value
case float64:
result = provider.FloatEvaluation(t.Context(), tt.flags.Name, tt.defaultValue.(float64), openfeature.FlattenedContext{}).Value
case string:
result = provider.StringEvaluation(t.Context(), tt.flags.Name, tt.defaultValue.(string), openfeature.FlattenedContext{}).Value
case int64:
result = provider.IntEvaluation(t.Context(), tt.flags.Name, tt.defaultValue.(int64), openfeature.FlattenedContext{}).Value
case map[string]any:
result = provider.ObjectEvaluation(t.Context(), tt.flags.Name, tt.defaultValue, openfeature.FlattenedContext{}).Value
}
assert.Equal(t, tt.expectedValue, result)
}
}
func Test_StaticProvider_ConfigOverride(t *testing.T) {
tests := []struct {
name string
originalValue string
configValue any
}{
{
name: "bool",
originalValue: "false",
configValue: true,
},
{
name: "int",
originalValue: "0",
configValue: int64(1),
},
{
name: "float",
originalValue: "0.0",
configValue: 1.0,
},
{
name: "string",
originalValue: "foo",
configValue: "bar",
},
{
name: "structure",
originalValue: "{}",
configValue: make(map[string]any),
},
}
for _, tt := range tests {
configFlags, standardFlags := makeFlags(tt)
provider, err := newStaticProvider(configFlags, standardFlags)
assert.NoError(t, err)
var result any
switch tt.configValue.(type) {
case bool:
result = provider.BooleanEvaluation(t.Context(), tt.name, false, openfeature.FlattenedContext{}).Value
case float64:
result = provider.FloatEvaluation(t.Context(), tt.name, 0.0, openfeature.FlattenedContext{}).Value
case string:
result = provider.StringEvaluation(t.Context(), tt.name, "foo", openfeature.FlattenedContext{}).Value
case int64:
result = provider.IntEvaluation(t.Context(), tt.name, 1, openfeature.FlattenedContext{}).Value
case map[string]any:
result = provider.ObjectEvaluation(t.Context(), tt.name, make(map[string]any), openfeature.FlattenedContext{}).Value
}
assert.Equal(t, tt.configValue, result)
}
}
func makeFlags(tt struct {
name string
originalValue string
configValue any
}) (map[string]memprovider.InMemoryFlag, []FeatureFlag) {
orig := FeatureFlag{
Name: tt.name,
Expression: tt.originalValue,
}
config := map[string]memprovider.InMemoryFlag{
tt.name: setting.NewInMemoryFlag(tt.name, tt.configValue),
}
return config, []FeatureFlag{orig}
}
-5
View File
@@ -74,7 +74,6 @@ alertmanagerRemoteSecondary,experimental,@grafana/alerting-squad,false,false,fal
alertingProvenanceLockWrites,experimental,@grafana/alerting-squad,false,false,false
alertingUIUseBackendFilters,experimental,@grafana/alerting-squad,false,false,false
alertingUIUseFullyCompatBackendFilters,experimental,@grafana/alerting-squad,false,false,false
createAlertRuleFromPanel,experimental,@grafana/alerting-squad,false,false,true
alertmanagerRemotePrimary,experimental,@grafana/alerting-squad,false,false,false
annotationPermissionUpdate,GA,@grafana/identity-access-team,false,false,false
dashboardSceneForViewers,GA,@grafana/dashboards-squad,false,false,true
@@ -121,7 +120,6 @@ queryLibrary,preview,@grafana/sharing-squad,false,false,false
dashboardLibrary,experimental,@grafana/sharing-squad,false,false,false
suggestedDashboards,experimental,@grafana/sharing-squad,false,false,false
dashboardTemplates,preview,@grafana/sharing-squad,false,false,false
logsExploreTableDefaultVisualization,experimental,@grafana/observability-logs,false,false,true
alertingListViewV2,privatePreview,@grafana/alerting-squad,false,false,true
alertingSavedSearches,experimental,@grafana/alerting-squad,false,false,true
alertingDisableSendAlertsExternal,experimental,@grafana/alerting-squad,false,false,false
@@ -144,14 +142,12 @@ vizActionsAuth,preview,@grafana/dataviz-squad,false,false,true
alertingPrometheusRulesPrimary,experimental,@grafana/alerting-squad,false,false,true
exploreLogsShardSplitting,experimental,@grafana/observability-logs,false,false,true
exploreLogsAggregatedMetrics,experimental,@grafana/observability-logs,false,false,true
exploreLogsLimitedTimeRange,experimental,@grafana/observability-logs,false,false,true
appPlatformGrpcClientAuth,experimental,@grafana/identity-access-team,false,false,false
groupAttributeSync,privatePreview,@grafana/identity-access-team,false,false,false
alertingQueryAndExpressionsStepMode,GA,@grafana/alerting-squad,false,false,true
improvedExternalSessionHandling,GA,@grafana/identity-access-team,false,false,false
useSessionStorageForRedirection,GA,@grafana/identity-access-team,false,false,false
rolePickerDrawer,experimental,@grafana/identity-access-team,false,false,false
unifiedStorageSearch,experimental,@grafana/search-and-storage,false,false,false
unifiedStorageSearchSprinkles,experimental,@grafana/search-and-storage,false,false,false
managedDualWriter,experimental,@grafana/search-and-storage,false,false,false
pluginsSriChecks,GA,@grafana/plugins-platform-backend,false,false,false
@@ -162,7 +158,6 @@ newTimeRangeZoomShortcuts,experimental,@grafana/dataviz-squad,false,false,true
azureMonitorDisableLogLimit,GA,@grafana/partner-datasources,false,false,false
playlistsReconciler,experimental,@grafana/grafana-app-platform-squad,false,true,false
passwordlessMagicLinkAuthentication,experimental,@grafana/identity-access-team,false,false,false
exploreMetricsRelatedLogs,experimental,@grafana/observability-metrics,false,false,true
prometheusSpecialCharsInLabelValues,experimental,@grafana/oss-big-tent,false,false,true
enableExtensionsAdminPage,experimental,@grafana/plugins-platform-backend,false,true,false
enableSCIM,preview,@grafana/identity-access-team,false,false,false
1 Name Stage Owner requiresDevMode RequiresRestart FrontendOnly
74 alertingProvenanceLockWrites experimental @grafana/alerting-squad false false false
75 alertingUIUseBackendFilters experimental @grafana/alerting-squad false false false
76 alertingUIUseFullyCompatBackendFilters experimental @grafana/alerting-squad false false false
createAlertRuleFromPanel experimental @grafana/alerting-squad false false true
77 alertmanagerRemotePrimary experimental @grafana/alerting-squad false false false
78 annotationPermissionUpdate GA @grafana/identity-access-team false false false
79 dashboardSceneForViewers GA @grafana/dashboards-squad false false true
120 dashboardLibrary experimental @grafana/sharing-squad false false false
121 suggestedDashboards experimental @grafana/sharing-squad false false false
122 dashboardTemplates preview @grafana/sharing-squad false false false
logsExploreTableDefaultVisualization experimental @grafana/observability-logs false false true
123 alertingListViewV2 privatePreview @grafana/alerting-squad false false true
124 alertingSavedSearches experimental @grafana/alerting-squad false false true
125 alertingDisableSendAlertsExternal experimental @grafana/alerting-squad false false false
142 alertingPrometheusRulesPrimary experimental @grafana/alerting-squad false false true
143 exploreLogsShardSplitting experimental @grafana/observability-logs false false true
144 exploreLogsAggregatedMetrics experimental @grafana/observability-logs false false true
exploreLogsLimitedTimeRange experimental @grafana/observability-logs false false true
145 appPlatformGrpcClientAuth experimental @grafana/identity-access-team false false false
146 groupAttributeSync privatePreview @grafana/identity-access-team false false false
147 alertingQueryAndExpressionsStepMode GA @grafana/alerting-squad false false true
148 improvedExternalSessionHandling GA @grafana/identity-access-team false false false
149 useSessionStorageForRedirection GA @grafana/identity-access-team false false false
150 rolePickerDrawer experimental @grafana/identity-access-team false false false
unifiedStorageSearch experimental @grafana/search-and-storage false false false
151 unifiedStorageSearchSprinkles experimental @grafana/search-and-storage false false false
152 managedDualWriter experimental @grafana/search-and-storage false false false
153 pluginsSriChecks GA @grafana/plugins-platform-backend false false false
158 azureMonitorDisableLogLimit GA @grafana/partner-datasources false false false
159 playlistsReconciler experimental @grafana/grafana-app-platform-squad false true false
160 passwordlessMagicLinkAuthentication experimental @grafana/identity-access-team false false false
exploreMetricsRelatedLogs experimental @grafana/observability-metrics false false true
161 prometheusSpecialCharsInLabelValues experimental @grafana/oss-big-tent false false true
162 enableExtensionsAdminPage experimental @grafana/plugins-platform-backend false true false
163 enableSCIM preview @grafana/identity-access-team false false false
-4
View File
@@ -455,10 +455,6 @@ const (
// Enables the new role picker drawer design
FlagRolePickerDrawer = "rolePickerDrawer"
// FlagUnifiedStorageSearch
// Enable unified storage search
FlagUnifiedStorageSearch = "unifiedStorageSearch"
// FlagUnifiedStorageSearchSprinkles
// Enable sprinkles on unified storage search
FlagUnifiedStorageSearchSprinkles = "unifiedStorageSearchSprinkles"
+8 -17
View File
@@ -962,19 +962,6 @@
"hideFromDocs": true
}
},
{
"metadata": {
"name": "createAlertRuleFromPanel",
"resourceVersion": "1763546460188",
"creationTimestamp": "2025-10-29T09:52:06Z"
},
"spec": {
"description": "Enables creating alert rules from a panel using a drawer UI",
"stage": "experimental",
"codeowner": "@grafana/alerting-squad",
"frontend": true
}
},
{
"metadata": {
"name": "dashboardDisableSchemaValidationV1",
@@ -1395,7 +1382,8 @@
"metadata": {
"name": "exploreLogsLimitedTimeRange",
"resourceVersion": "1764664939750",
"creationTimestamp": "2024-08-29T13:55:59Z"
"creationTimestamp": "2024-08-29T13:55:59Z",
"deletionTimestamp": "2026-01-12T22:18:14Z"
},
"spec": {
"description": "Used in Logs Drilldown to limit the time range",
@@ -1421,7 +1409,8 @@
"metadata": {
"name": "exploreMetricsRelatedLogs",
"resourceVersion": "1764664939750",
"creationTimestamp": "2024-11-05T16:28:43Z"
"creationTimestamp": "2024-11-05T16:28:43Z",
"deletionTimestamp": "2026-01-09T22:14:53Z"
},
"spec": {
"description": "Display Related Logs in Grafana Metrics Drilldown",
@@ -2259,7 +2248,8 @@
"metadata": {
"name": "logsExploreTableDefaultVisualization",
"resourceVersion": "1764664939750",
"creationTimestamp": "2024-05-02T15:28:15Z"
"creationTimestamp": "2024-05-02T15:28:15Z",
"deletionTimestamp": "2026-01-12T14:11:46Z"
},
"spec": {
"description": "Sets the logs table as default visualisation in logs explore",
@@ -3710,7 +3700,8 @@
"metadata": {
"name": "unifiedStorageSearch",
"resourceVersion": "1764664939750",
"creationTimestamp": "2024-09-30T19:46:14Z"
"creationTimestamp": "2024-09-30T19:46:14Z",
"deletionTimestamp": "2026-01-12T10:02:12Z"
},
"spec": {
"description": "Enable unified storage search",
@@ -190,9 +190,6 @@ func verifyFlagsConfiguration(t *testing.T) {
if flag.Stage == FeatureStageGeneralAvailability && flag.Expression == "" {
t.Errorf("GA features must be explicitly enabled or disabled, please add the `Expression` property for %s", flag.Name)
}
if flag.Expression != "" && flag.Expression != "true" && flag.Expression != "false" {
t.Errorf("the `Expression` property for %s is incorrect. valid values are: `true`, `false` or empty string for default", flag.Name)
}
// Check camel case names
if flag.Name != strcase.ToLowerCamel(flag.Name) && !legacyNames[flag.Name] {
invalidNames = append(invalidNames, flag.Name)
+10 -1
View File
@@ -100,6 +100,9 @@ func (d *DsLookup) ByRef(ref *DataSourceRef) *DataSourceRef {
if ref == nil {
return d.defaultDS
}
if ref.UID == "default" && ref.Type == "" {
return d.defaultDS
}
key := ""
if ref.UID != "" {
@@ -117,7 +120,13 @@ func (d *DsLookup) ByRef(ref *DataSourceRef) *DataSourceRef {
return ds
}
return d.byName[key]
ds, ok = d.byName[key]
if ok {
return ds
}
// With nothing was found (or configured), use the original reference
return ref
}
func (d *DsLookup) ByType(dsType string) []DataSourceRef {
@@ -4,8 +4,8 @@
"tags": null,
"datasource": [
{
"uid": "default.uid",
"type": "default.type"
"uid": "000000001",
"type": "graphite"
}
],
"panels": [
@@ -16,8 +16,8 @@
"libraryPanel": "dfkljg98345dkf",
"datasource": [
{
"uid": "default.uid",
"type": "default.type"
"uid": "000000001",
"type": "graphite"
}
]
}
@@ -1,5 +1,7 @@
package dashboard
import "iter"
type PanelSummaryInfo struct {
ID int64 `json:"id"`
Title string `json:"title"`
@@ -30,3 +32,20 @@ type DashboardSummaryInfo struct {
Refresh string `json:"refresh,omitempty"`
ReadOnly bool `json:"readOnly,omitempty"` // editable = false
}
func (d *DashboardSummaryInfo) PanelIterator() iter.Seq[PanelSummaryInfo] {
return func(yield func(PanelSummaryInfo) bool) {
for _, p := range d.Panels {
if len(p.Collapsed) > 0 {
for _, c := range p.Collapsed {
if !yield(c) { // NOTE, rows can only be one level deep!
return
}
}
}
if !yield(p) {
return
}
}
}
}
+5 -2
View File
@@ -10,6 +10,7 @@ import (
"testing"
"github.com/open-feature/go-sdk/openfeature"
"github.com/open-feature/go-sdk/openfeature/memprovider"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/log"
@@ -378,8 +379,10 @@ func setupOpenFeatureProvider(t *testing.T, flagValue bool) {
err := featuremgmt.InitOpenFeature(featuremgmt.OpenFeatureConfig{
ProviderType: setting.StaticProviderType,
StaticFlags: map[string]bool{
featuremgmt.FlagPluginsAutoUpdate: flagValue,
StaticFlags: map[string]memprovider.InMemoryFlag{
featuremgmt.FlagPluginsAutoUpdate: {
Key: featuremgmt.FlagPluginsAutoUpdate, Variants: map[string]any{"": flagValue},
},
},
})
require.NoError(t, err)
+75 -5
View File
@@ -1,13 +1,20 @@
package setting
import (
"encoding/json"
"math"
"strconv"
"gopkg.in/ini.v1"
"github.com/open-feature/go-sdk/openfeature/memprovider"
"github.com/grafana/grafana/pkg/util"
)
// DefaultVariantName a placeholder name for config-based Feature Flags
const DefaultVariantName = "default"
// Deprecated: should use `featuremgmt.FeatureToggles`
func (cfg *Cfg) readFeatureToggles(iniFile *ini.File) error {
section := iniFile.Section("feature_toggles")
@@ -15,18 +22,27 @@ func (cfg *Cfg) readFeatureToggles(iniFile *ini.File) error {
if err != nil {
return err
}
// TODO IsFeatureToggleEnabled has been deprecated for 2 years now, we should remove this function completely
// nolint:staticcheck
cfg.IsFeatureToggleEnabled = func(key string) bool { return toggles[key] }
cfg.IsFeatureToggleEnabled = func(key string) bool {
toggle, ok := toggles[key]
if !ok {
return false
}
value, ok := toggle.Variants[toggle.DefaultVariant].(bool)
return value && ok
}
return nil
}
func ReadFeatureTogglesFromInitFile(featureTogglesSection *ini.Section) (map[string]bool, error) {
featureToggles := make(map[string]bool, 10)
func ReadFeatureTogglesFromInitFile(featureTogglesSection *ini.Section) (map[string]memprovider.InMemoryFlag, error) {
featureToggles := make(map[string]memprovider.InMemoryFlag, 10)
// parse the comma separated list in `enable`.
featuresTogglesStr := valueAsString(featureTogglesSection, "enable", "")
for _, feature := range util.SplitString(featuresTogglesStr) {
featureToggles[feature] = true
featureToggles[feature] = memprovider.InMemoryFlag{Key: feature, DefaultVariant: DefaultVariantName, Variants: map[string]any{DefaultVariantName: true}}
}
// read all other settings under [feature_toggles]. If a toggle is
@@ -36,7 +52,7 @@ func ReadFeatureTogglesFromInitFile(featureTogglesSection *ini.Section) (map[str
continue
}
b, err := strconv.ParseBool(v.Value())
b, err := ParseFlag(v.Name(), v.Value())
if err != nil {
return featureToggles, err
}
@@ -45,3 +61,57 @@ func ReadFeatureTogglesFromInitFile(featureTogglesSection *ini.Section) (map[str
}
return featureToggles, nil
}
func ParseFlag(name, value string) (memprovider.InMemoryFlag, error) {
var structure map[string]any
if integer, err := strconv.Atoi(value); err == nil {
return NewInMemoryFlag(name, integer), nil
}
if float, err := strconv.ParseFloat(value, 64); err == nil {
return NewInMemoryFlag(name, float), nil
}
if err := json.Unmarshal([]byte(value), &structure); err == nil {
return NewInMemoryFlag(name, structure), nil
}
if boolean, err := strconv.ParseBool(value); err == nil {
return NewInMemoryFlag(name, boolean), nil
}
return NewInMemoryFlag(name, value), nil
}
func NewInMemoryFlag(name string, value any) memprovider.InMemoryFlag {
return memprovider.InMemoryFlag{Key: name, DefaultVariant: DefaultVariantName, Variants: map[string]any{DefaultVariantName: value}}
}
func AsStringMap(m map[string]memprovider.InMemoryFlag) map[string]string {
var res = map[string]string{}
for k, v := range m {
res[k] = serializeFlagValue(v)
}
return res
}
func serializeFlagValue(flag memprovider.InMemoryFlag) string {
value := flag.Variants[flag.DefaultVariant]
switch castedValue := value.(type) {
case bool:
return strconv.FormatBool(castedValue)
case int64:
return strconv.FormatInt(castedValue, 10)
case float64:
// handle cases with a single or no zeros after the decimal point
if math.Trunc(castedValue) == castedValue {
return strconv.FormatFloat(castedValue, 'f', 1, 64)
}
return strconv.FormatFloat(castedValue, 'g', -1, 64)
case string:
return castedValue
default:
val, _ := json.Marshal(value)
return string(val)
}
}
+54 -23
View File
@@ -1,9 +1,11 @@
package setting
import (
"strconv"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/open-feature/go-sdk/openfeature/memprovider"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gopkg.in/ini.v1"
)
@@ -12,17 +14,16 @@ func TestFeatureToggles(t *testing.T) {
testCases := []struct {
name string
conf map[string]string
err error
expectedToggles map[string]bool
expectedToggles map[string]memprovider.InMemoryFlag
}{
{
name: "can parse feature toggles passed in the `enable` array",
conf: map[string]string{
"enable": "feature1,feature2",
},
expectedToggles: map[string]bool{
"feature1": true,
"feature2": true,
expectedToggles: map[string]memprovider.InMemoryFlag{
"feature1": NewInMemoryFlag("feature1", true),
"feature2": NewInMemoryFlag("feature2", true),
},
},
{
@@ -31,10 +32,10 @@ func TestFeatureToggles(t *testing.T) {
"enable": "feature1,feature2",
"feature3": "true",
},
expectedToggles: map[string]bool{
"feature1": true,
"feature2": true,
"feature3": true,
expectedToggles: map[string]memprovider.InMemoryFlag{
"feature1": NewInMemoryFlag("feature1", true),
"feature2": NewInMemoryFlag("feature2", true),
"feature3": NewInMemoryFlag("feature3", true),
},
},
{
@@ -43,19 +44,26 @@ func TestFeatureToggles(t *testing.T) {
"enable": "feature1,feature2",
"feature2": "false",
},
expectedToggles: map[string]bool{
"feature1": true,
"feature2": false,
expectedToggles: map[string]memprovider.InMemoryFlag{
"feature1": NewInMemoryFlag("feature1", true),
"feature2": NewInMemoryFlag("feature2", false),
},
},
{
name: "invalid boolean value should return syntax error",
name: "feature flags of different types are handled correctly",
conf: map[string]string{
"enable": "feature1,feature2",
"feature2": "invalid",
"feature1": "1", "feature2": "1.0",
"feature3": `{"foo":"bar"}`, "feature4": "bar",
"feature5": "t", "feature6": "T",
},
expectedToggles: map[string]memprovider.InMemoryFlag{
"feature1": NewInMemoryFlag("feature1", 1),
"feature2": NewInMemoryFlag("feature2", 1.0),
"feature3": NewInMemoryFlag("feature3", map[string]any{"foo": "bar"}),
"feature4": NewInMemoryFlag("feature4", "bar"),
"feature5": NewInMemoryFlag("feature5", true),
"feature6": NewInMemoryFlag("feature6", true),
},
expectedToggles: map[string]bool{},
err: strconv.ErrSyntax,
},
}
@@ -69,12 +77,35 @@ func TestFeatureToggles(t *testing.T) {
}
featureToggles, err := ReadFeatureTogglesFromInitFile(toggles)
require.ErrorIs(t, err, tc.err)
require.NoError(t, err)
if err == nil {
for k, v := range featureToggles {
require.Equal(t, tc.expectedToggles[k], v, tc.name)
}
for k, v := range featureToggles {
toggle := tc.expectedToggles[k]
require.Equal(t, toggle, v, tc.name)
}
}
}
func TestFlagValueSerialization(t *testing.T) {
testCases := []memprovider.InMemoryFlag{
NewInMemoryFlag("int", 1),
NewInMemoryFlag("1.0f", 1.0),
NewInMemoryFlag("1.01f", 1.01),
NewInMemoryFlag("1.10f", 1.10),
NewInMemoryFlag("struct", map[string]any{"foo": "bar"}),
NewInMemoryFlag("string", "bar"),
NewInMemoryFlag("true", true),
NewInMemoryFlag("false", false),
}
for _, tt := range testCases {
asStringMap := AsStringMap(map[string]memprovider.InMemoryFlag{tt.Key: tt})
deserialized, err := ParseFlag(tt.Key, asStringMap[tt.Key])
assert.NoError(t, err)
if diff := cmp.Diff(tt, deserialized); diff != "" {
t.Errorf("(-want, +got) = %v", diff)
}
}
}
-1
View File
@@ -236,7 +236,6 @@ kubernetesDashboards = true
kubernetesFolders = true
unifiedStorage = true
unifiedStorageHistoryPruner = true
unifiedStorageSearch = true
unifiedStorageSearchPermissionFiltering = false
unifiedStorageSearchSprinkles = false
+4 -4
View File
@@ -78,13 +78,13 @@ func (n *notifier) Watch(ctx context.Context, opts watchOptions) <-chan Event {
cache := gocache.New(cacheTTL, cacheCleanupInterval)
events := make(chan Event, opts.BufferSize)
initialRV, err := n.lastEventResourceVersion(ctx)
lastRV, err := n.lastEventResourceVersion(ctx)
if errors.Is(err, ErrNotFound) {
initialRV = snowflakeFromTime(time.Now()) // No events yet, start from the beginning
lastRV = 0 // No events yet, start from the beginning
} else if err != nil {
n.log.Error("Failed to get last event resource version", "error", err)
}
lastRV := initialRV + 1 // We want to start watching from the next event
lastRV = lastRV + 1 // We want to start watching from the next event
go func() {
defer close(events)
@@ -110,7 +110,7 @@ func (n *notifier) Watch(ctx context.Context, opts watchOptions) <-chan Event {
}
// Skip old events lower than the requested resource version
if evt.ResourceVersion <= initialRV {
if evt.ResourceVersion < lastRV {
continue
}
+7 -15
View File
@@ -25,7 +25,6 @@ func setupTestNotifier(t *testing.T) (*notifier, *eventStore) {
return notifier, eventStore
}
// nolint:unused
func setupTestNotifierSqlKv(t *testing.T) (*notifier, *eventStore) {
dbstore := db.InitTestDB(t)
eDB, err := dbimpl.ProvideResourceDB(dbstore, setting.NewCfg(), nil)
@@ -60,8 +59,7 @@ func runNotifierTestWith(t *testing.T, storeName string, newStoreFn func(*testin
func TestNotifier_lastEventResourceVersion(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierLastEventResourceVersion)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierLastEventResourceVersion)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierLastEventResourceVersion)
}
func testNotifierLastEventResourceVersion(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -112,8 +110,7 @@ func testNotifierLastEventResourceVersion(t *testing.T, ctx context.Context, not
func TestNotifier_cachekey(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierCachekey)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierCachekey)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierCachekey)
}
func testNotifierCachekey(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -167,8 +164,7 @@ func testNotifierCachekey(t *testing.T, ctx context.Context, notifier *notifier,
func TestNotifier_Watch_NoEvents(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierWatchNoEvents)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchNoEvents)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchNoEvents)
}
func testNotifierWatchNoEvents(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -209,8 +205,7 @@ func testNotifierWatchNoEvents(t *testing.T, ctx context.Context, notifier *noti
func TestNotifier_Watch_WithExistingEvents(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierWatchWithExistingEvents)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchWithExistingEvents)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchWithExistingEvents)
}
func testNotifierWatchWithExistingEvents(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -284,8 +279,7 @@ func testNotifierWatchWithExistingEvents(t *testing.T, ctx context.Context, noti
func TestNotifier_Watch_EventDeduplication(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierWatchEventDeduplication)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchEventDeduplication)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchEventDeduplication)
}
func testNotifierWatchEventDeduplication(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -351,8 +345,7 @@ func testNotifierWatchEventDeduplication(t *testing.T, ctx context.Context, noti
func TestNotifier_Watch_ContextCancellation(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierWatchContextCancellation)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchContextCancellation)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchContextCancellation)
}
func testNotifierWatchContextCancellation(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
@@ -398,8 +391,7 @@ func testNotifierWatchContextCancellation(t *testing.T, ctx context.Context, not
func TestNotifier_Watch_MultipleEvents(t *testing.T) {
runNotifierTestWith(t, "badger", setupTestNotifier, testNotifierWatchMultipleEvents)
// enable this when sqlkv is ready
// runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchMultipleEvents)
runNotifierTestWith(t, "sqlkv", setupTestNotifierSqlKv, testNotifierWatchMultipleEvents)
}
func testNotifierWatchMultipleEvents(t *testing.T, ctx context.Context, notifier *notifier, eventStore *eventStore) {
+1 -1
View File
@@ -863,7 +863,7 @@ func newRebuildRequest(key NamespacedResource, minBuildTime, lastImportTime time
func (s *searchSupport) getOrCreateIndex(ctx context.Context, stats *SearchStats, key NamespacedResource, reason string) (ResourceIndex, error) {
if s == nil || s.search == nil {
return nil, fmt.Errorf("search is not configured properly (missing unifiedStorageSearch feature toggle?)")
return nil, fmt.Errorf("search is not configured properly (missing enable_search config?)")
}
ctx, span := tracer.Start(ctx, "resource.searchSupport.getOrCreateIndex")
@@ -346,7 +346,8 @@ func (k *kvStorageBackend) WriteEvent(ctx context.Context, event WriteEvent) (in
return 0, fmt.Errorf("failed to write data: %w", err)
}
dataKey.ResourceVersion = rvmanager.SnowflakeFromRv(rv)
rv = rvmanager.SnowflakeFromRv(rv)
dataKey.ResourceVersion = rv
} else {
err := k.dataStore.Save(ctx, dataKey, bytes.NewReader(event.Value))
if err != nil {
+4 -2
View File
@@ -1253,21 +1253,23 @@ func (b *bleveIndex) toBleveSearchRequest(ctx context.Context, req *resourcepb.R
queryExact.SetField(resource.SEARCH_FIELD_TITLE)
queryExact.Analyzer = keyword.Name // don't analyze the query input - treat it as a single token
queryExact.Operator = query.MatchQueryOperatorAnd // This doesn't make a difference for keyword analyzer, we add it just to be explicit.
searchQuery := bleve.NewDisjunctionQuery(queryExact)
// Query 2: Phrase query with standard analyzer
queryPhrase := bleve.NewMatchPhraseQuery(req.Query)
queryPhrase.SetBoost(5.0)
queryPhrase.SetField(resource.SEARCH_FIELD_TITLE)
queryPhrase.Analyzer = standard.Name
searchQuery.AddQuery(queryPhrase)
// Query 3: Match query with standard analyzer
queryAnalyzed := bleve.NewMatchQuery(removeSmallTerms(req.Query))
queryAnalyzed.SetField(resource.SEARCH_FIELD_TITLE)
queryAnalyzed.SetBoost(2.0)
queryAnalyzed.Analyzer = standard.Name
queryAnalyzed.Operator = query.MatchQueryOperatorAnd // Make sure all terms from the query are matched
searchQuery.AddQuery(queryAnalyzed)
// At least one of the queries must match
searchQuery := bleve.NewDisjunctionQuery(queryExact, queryAnalyzed, queryPhrase)
queries = append(queries, searchQuery)
}
-1
View File
@@ -23,7 +23,6 @@ import (
"go.uber.org/goleak"
authlib "github.com/grafana/authlib/types"
"github.com/grafana/grafana/pkg/apimachinery/identity"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/infra/log"
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"slices"
"sort"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -18,6 +19,7 @@ import (
const DASHBOARD_SCHEMA_VERSION = "schema_version"
const DASHBOARD_LINK_COUNT = "link_count"
const DASHBOARD_PANEL_TYPES = "panel_types"
const DASHBOARD_PANEL_TITLE = "panel_title"
const DASHBOARD_DS_TYPES = "ds_types"
const DASHBOARD_TRANSFORMATIONS = "transformation"
const DASHBOARD_LIBRARY_PANEL_REFERENCE = "reference.LibraryPanel"
@@ -53,11 +55,21 @@ func DashboardBuilder(namespaced resource.NamespacedDocumentSupplier) (resource.
Type: resourcepb.ResourceTableColumnDefinition_INT32,
Description: "How many links appear on the page",
},
{
Name: DASHBOARD_PANEL_TITLE,
Type: resourcepb.ResourceTableColumnDefinition_STRING,
IsArray: true,
Description: "The panel title text",
Properties: &resourcepb.ResourceTableColumnDefinition_Properties{
Filterable: false, // full text
FreeText: true,
},
},
{
Name: DASHBOARD_PANEL_TYPES,
Type: resourcepb.ResourceTableColumnDefinition_STRING,
IsArray: true,
Description: "How many links appear on the page",
Description: "The panel types used in this dashboard",
Properties: &resourcepb.ResourceTableColumnDefinition_Properties{
Filterable: true,
},
@@ -269,14 +281,22 @@ func (s *DashboardDocumentBuilder) BuildDocument(ctx context.Context, key *resou
doc.Description = summary.Description
doc.Tags = summary.Tags
panelTitles := []string{}
panelTypes := []string{}
transformations := []string{}
dsTypes := []string{}
for _, p := range summary.Panels {
if p.Type != "" {
for p := range summary.PanelIterator() {
switch p.Type {
case "": // ignore
case "row": // row should map to a layout type when we support v2 constructs
default:
panelTypes = append(panelTypes, p.Type)
}
if len(p.Title) > 0 {
panelTitles = append(panelTitles, p.Title)
}
if len(p.Transformer) > 0 {
transformations = append(transformations, p.Transformer...)
}
@@ -309,17 +329,20 @@ func (s *DashboardDocumentBuilder) BuildDocument(ctx context.Context, key *resou
resource.SEARCH_FIELD_LEGACY_ID: summary.ID,
}
if len(panelTitles) > 0 {
doc.Fields[DASHBOARD_PANEL_TITLE] = panelTitles
}
if len(panelTypes) > 0 {
sort.Strings(panelTypes)
doc.Fields[DASHBOARD_PANEL_TYPES] = panelTypes
doc.Fields[DASHBOARD_PANEL_TYPES] = slices.Compact(panelTypes) // distinct values
}
if len(dsTypes) > 0 {
sort.Strings(dsTypes)
doc.Fields[DASHBOARD_DS_TYPES] = dsTypes
doc.Fields[DASHBOARD_DS_TYPES] = slices.Compact(dsTypes) // distinct values
}
if len(transformations) > 0 {
sort.Strings(transformations)
doc.Fields[DASHBOARD_TRANSFORMATIONS] = transformations
doc.Fields[DASHBOARD_TRANSFORMATIONS] = slices.Compact(transformations) // distinct values
}
for k, v := range s.Stats[summary.UID] {
@@ -32,10 +32,16 @@
"errors_last_7_days": 1,
"grafana.app/deprecatedInternalID": 141,
"link_count": 0,
"panel_title": [
"green pie",
"red pie",
"blue pie",
"collapsed row"
],
"panel_types": [
"barchart",
"graph",
"row"
"pie"
],
"schema_version": 38
},
@@ -46,6 +52,12 @@
"kind": "DataSource",
"name": "DSUID"
},
{
"relation": "depends-on",
"group": "dashboards.grafana.app",
"kind": "LibraryPanel",
"name": "l3d2s634-fdgf-75u4-3fg3-67j966ii7jur"
},
{
"relation": "depends-on",
"group": "dashboards.grafana.app",
@@ -67,7 +67,7 @@
"name": "red pie",
"uid": "e1d5f519-dabd-47c6-9ad7-83d181ce1cee"
},
"title": "green pie"
"title": "red pie"
},
{
"id": 7,
@@ -78,6 +78,14 @@
"id": 8,
"type": "graph"
},
{
"id": 20,
"type": "graph"
},
{
"id": 30,
"type": "graph"
},
{
"collapsed": true,
"gridPos": {
@@ -101,6 +109,10 @@
"uid": "l3d2s634-fdgf-75u4-3fg3-67j966ii7jur"
},
"title": "blue pie"
},
{
"id": 40,
"type": "pie"
}
],
"title": "collapsed row",
+1 -1
View File
@@ -19,7 +19,7 @@ func NewSearchOptions(
ownsIndexFn func(key resource.NamespacedResource) (bool, error),
) (resource.SearchOptions, error) {
//nolint:staticcheck // not yet migrated to OpenFeature
if cfg.EnableSearch || features.IsEnabledGlobally(featuremgmt.FlagUnifiedStorageSearch) || features.IsEnabledGlobally(featuremgmt.FlagProvisioning) {
if cfg.EnableSearch || features.IsEnabledGlobally(featuremgmt.FlagProvisioning) {
root := cfg.IndexPath
if root == "" {
root = filepath.Join(cfg.DataPath, "unified-search", "bleve")
+11 -1
View File
@@ -71,11 +71,18 @@
"description": "How many links appear on the page",
"priority": 0
},
{
"name": "panel_title",
"type": "string",
"format": "",
"description": "The panel title text",
"priority": 0
},
{
"name": "panel_types",
"type": "string",
"format": "",
"description": "How many links appear on the page",
"description": "The panel types used in this dashboard",
"priority": 0
},
{
@@ -214,6 +221,7 @@
null,
null,
null,
null,
null
],
"object": {
@@ -239,6 +247,7 @@
"repo",
null,
null,
null,
[
"timeseries"
],
@@ -282,6 +291,7 @@
"repo",
null,
null,
null,
[
"timeseries",
"table"
@@ -9,7 +9,6 @@ import (
"testing"
"time"
"github.com/bwmarrin/snowflake"
"github.com/stretchr/testify/require"
claims "github.com/grafana/authlib/types"
@@ -82,6 +81,12 @@ func RunSQLStorageBackendCompatibilityTest(t *testing.T, newSqlBackend, newKvBac
kvbackend, db := newKvBackend(t.Context())
sqlbackend, _ := newSqlBackend(t.Context())
// Skip on SQLite due to concurrency limitations
if db.DriverName() == "sqlite3" {
t.Skip("Skipping concurrent operations stress test on SQLite")
}
tc.fn(t, sqlbackend, kvbackend, opts.NSPrefix, db)
})
}
@@ -187,13 +192,30 @@ func runKeyPathTest(t *testing.T, backend resource.StorageBackend, nsPrefix stri
// verifyKeyPath is a helper function to verify key_path generation
func verifyKeyPath(t *testing.T, db sqldb.DB, ctx context.Context, key *resourcepb.ResourceKey, action string, resourceVersion int64, expectedFolder string) {
// For SQL backend (namespace contains "-sql"), resourceVersion is in microsecond format
// but key_path stores snowflake RV, so convert to snowflake
// For KV backend (namespace contains "-kv"), resourceVersion is already in snowflake format
isSqlBackend := strings.Contains(key.Namespace, "-sql")
var keyPathRV int64
if isSqlBackend {
// Convert microsecond RV to snowflake for key_path construction
keyPathRV = rvmanager.SnowflakeFromRv(resourceVersion)
} else {
// KV backend already provides snowflake RV
keyPathRV = resourceVersion
}
// Build the expected key_path using DataKey format: unified/data/group/resource/namespace/name/resourceVersion~action~folder
expectedKeyPath := fmt.Sprintf("unified/data/%s/%s/%s/%s/%d~%s~%s", key.Group, key.Resource, key.Namespace, key.Name, keyPathRV, action, expectedFolder)
var query string
if db.DriverName() == "postgres" {
query = "SELECT key_path, resource_version, action, folder FROM resource_history WHERE namespace = $1 AND name = $2 AND resource_version = $3"
query = "SELECT key_path, resource_version, action, folder FROM resource_history WHERE key_path = $1"
} else {
query = "SELECT key_path, resource_version, action, folder FROM resource_history WHERE namespace = ? AND name = ? AND resource_version = ?"
query = "SELECT key_path, resource_version, action, folder FROM resource_history WHERE key_path = ?"
}
rows, err := db.QueryContext(ctx, query, key.Namespace, key.Name, resourceVersion)
rows, err := db.QueryContext(ctx, query, expectedKeyPath)
require.NoError(t, err)
require.True(t, rows.Next(), "Resource not found in resource_history table - both SQL and KV backends should write to this table")
@@ -220,10 +242,6 @@ func verifyKeyPath(t *testing.T, db sqldb.DB, ctx context.Context, key *resource
// Verify action suffix
require.Contains(t, keyPath, fmt.Sprintf("~%s~", action))
// Verify snowflake calculation
expectedSnowflake := (((resourceVersion / 1000) - snowflake.Epoch) << (snowflake.NodeBits + snowflake.StepBits)) + (resourceVersion % 1000)
require.Contains(t, keyPath, fmt.Sprintf("/%d~", expectedSnowflake), "actual RV: %d", actualRV)
// Verify folder if specified
if expectedFolder != "" {
require.Equal(t, expectedFolder, actualFolder)
@@ -492,10 +510,10 @@ func verifyResourceHistoryRecord(t *testing.T, record ResourceHistoryRecord, exp
}
// Validate previous_resource_version
// For KV backend operations, resource versions are stored as snowflake format
// but expectedPrevRV is in microsecond format, so we need to use IsRvEqual for comparison
// For KV backend operations, expectedPrevRV is now in snowflake format (returned by KV backend)
// but resource_history table stores microsecond RV, so we need to use IsRvEqual for comparison
if strings.Contains(record.Namespace, "-kv") {
require.True(t, rvmanager.IsRvEqual(record.PreviousResourceVersion, expectedPrevRV),
require.True(t, rvmanager.IsRvEqual(expectedPrevRV, record.PreviousResourceVersion),
"Previous resource version should match (KV backend snowflake format)")
} else {
require.Equal(t, expectedPrevRV, record.PreviousResourceVersion)
@@ -505,9 +523,10 @@ func verifyResourceHistoryRecord(t *testing.T, record ResourceHistoryRecord, exp
require.Equal(t, expectedGeneration, record.Generation)
// Validate resource_version
// For KV backend operations, resource versions are stored as snowflake format
// For KV backend operations, expectedRV is now in snowflake format (returned by KV backend)
// but resource_history table stores microsecond RV, so we need to use IsRvEqual for comparison
if strings.Contains(record.Namespace, "-kv") {
require.True(t, rvmanager.IsRvEqual(record.ResourceVersion, expectedRV),
require.True(t, rvmanager.IsRvEqual(expectedRV, record.ResourceVersion),
"Resource version should match (KV backend snowflake format)")
} else {
require.Equal(t, expectedRV, record.ResourceVersion)
@@ -574,7 +593,7 @@ func verifyResourceTable(t *testing.T, db sqldb.DB, namespace string, resources
// Resource version should match the expected version for test-resource-3 (updated version)
expectedRV := resourceVersions[2][1] // test-resource-3's update version
if strings.Contains(namespace, "-kv") {
require.True(t, rvmanager.IsRvEqual(record.ResourceVersion, expectedRV),
require.True(t, rvmanager.IsRvEqual(expectedRV, record.ResourceVersion),
"Resource version should match (KV backend snowflake format)")
} else {
require.Equal(t, expectedRV, record.ResourceVersion)
@@ -625,9 +644,16 @@ func verifyResourceVersionTable(t *testing.T, db sqldb.DB, namespace string, res
// The resource_version table should contain the latest RV for the group+resource
// It might be slightly higher due to RV manager operations, so check it's at least our max
require.GreaterOrEqual(t, record.ResourceVersion, maxRV, "resource_version should be at least the latest RV we tracked")
// But it shouldn't be too much higher (within a reasonable range)
require.LessOrEqual(t, record.ResourceVersion, maxRV+100, "resource_version shouldn't be much higher than expected")
// For KV backend, maxRV is in snowflake format but record.ResourceVersion is in microsecond format
// Use IsRvEqual for proper comparison between different RV formats
isKvBackend := strings.Contains(namespace, "-kv")
recordResourceVersion := record.ResourceVersion
if isKvBackend {
recordResourceVersion = rvmanager.SnowflakeFromRv(record.ResourceVersion)
}
require.Less(t, recordResourceVersion, int64(9223372036854775807), "resource_version should be reasonable")
require.Greater(t, recordResourceVersion, maxRV, "resource_version should be at least the latest RV we tracked")
}
// runTestCrossBackendConsistency tests basic consistency between SQL and KV backends (lightweight)
@@ -666,11 +692,6 @@ func runTestCrossBackendConsistency(t *testing.T, sqlBackend, kvBackend resource
// runTestConcurrentOperationsStress tests heavy concurrent operations between SQL and KV backends
func runTestConcurrentOperationsStress(t *testing.T, sqlBackend, kvBackend resource.StorageBackend, nsPrefix string, db sqldb.DB) {
// Skip on SQLite due to concurrency limitations
if db.DriverName() == "sqlite3" {
t.Skip("Skipping concurrent operations stress test on SQLite")
}
ctx := testutil.NewDefaultTestContext(t)
// Create storage servers from both backends
@@ -38,7 +38,6 @@ func TestBadgerKVStorageBackend(t *testing.T) {
func TestSQLKVStorageBackend(t *testing.T) {
skipTests := map[string]bool{
TestHappyPath: true,
TestWatchWriteEvents: true,
TestList: true,
TestBlobSupport: true,
@@ -51,21 +50,24 @@ func TestSQLKVStorageBackend(t *testing.T) {
TestGetResourceLastImportTime: true,
TestOptimisticLocking: true,
}
// without RvManager
RunStorageBackendTest(t, func(ctx context.Context) resource.StorageBackend {
backend, _ := NewTestSqlKvBackend(t, ctx, false)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-test",
SkipTests: skipTests,
t.Run("Without RvManager", func(t *testing.T) {
RunStorageBackendTest(t, func(ctx context.Context) resource.StorageBackend {
backend, _ := NewTestSqlKvBackend(t, ctx, false)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-test",
SkipTests: skipTests,
})
})
// with RvManager
RunStorageBackendTest(t, func(ctx context.Context) resource.StorageBackend {
backend, _ := NewTestSqlKvBackend(t, ctx, true)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-withrvmanager-test",
SkipTests: skipTests,
t.Run("With RvManager", func(t *testing.T) {
RunStorageBackendTest(t, func(ctx context.Context) resource.StorageBackend {
backend, _ := NewTestSqlKvBackend(t, ctx, true)
return backend
}, &TestOptions{
NSPrefix: "sqlkvstorage-withrvmanager-test",
SkipTests: skipTests,
})
})
}
+168
View File
@@ -4,10 +4,15 @@ import (
"context"
"encoding/json"
"fmt"
"io/fs"
"math"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/client-go/dynamic"
@@ -16,12 +21,167 @@ import (
dashboardV0 "github.com/grafana/grafana/apps/dashboard/pkg/apis/dashboard/v0alpha1"
"github.com/grafana/grafana/pkg/apimachinery/identity"
"github.com/grafana/grafana/pkg/apiserver/rest"
"github.com/grafana/grafana/pkg/components/simplejson"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/tests/apis"
"github.com/grafana/grafana/pkg/tests/testinfra"
"github.com/grafana/grafana/pkg/util/testutil"
)
func TestIntegrationSearchDevDashboards(t *testing.T) {
testutil.SkipIntegrationTestInShortMode(t)
ctx := context.Background()
helper := apis.NewK8sTestHelper(t, testinfra.GrafanaOpts{
DisableDataMigrations: true,
AppModeProduction: true,
DisableAnonymous: true,
APIServerStorageType: "unified",
UnifiedStorageConfig: map[string]setting.UnifiedStorageConfig{
"dashboards.dashboard.grafana.app": {DualWriterMode: rest.Mode5},
"folders.folder.grafana.app": {DualWriterMode: rest.Mode5},
},
UnifiedStorageEnableSearch: true,
})
defer helper.Shutdown()
// Create devenv dashboards from legacy API
cfg := dynamic.ConfigFor(helper.Org1.Admin.NewRestConfig())
cfg.GroupVersion = &dashboardV0.GroupVersion
adminClient, err := k8srest.RESTClientFor(cfg)
require.NoError(t, err)
adminClient.Get()
fileCount := 0
devenv := "../../../../devenv/dev-dashboards/panel-timeseries"
err = filepath.WalkDir(devenv, func(p string, d fs.DirEntry, e error) error {
require.NoError(t, err)
if d.IsDir() || filepath.Ext(d.Name()) != ".json" {
return nil
}
// use the filename as UID
uid := strings.TrimSuffix(d.Name(), ".json")
if len(uid) > 40 {
uid = uid[:40] // avoid uid too long, max 40 characters
}
// nolint:gosec
data, err := os.ReadFile(p)
require.NoError(t, err)
cmd := dashboards.SaveDashboardCommand{
Dashboard: &simplejson.Json{},
Overwrite: true,
}
err = cmd.Dashboard.FromDB(data)
require.NoError(t, err)
cmd.Dashboard.Set("id", nil)
cmd.Dashboard.Set("uid", uid)
data, err = json.Marshal(cmd)
require.NoError(t, err)
var statusCode int
result := adminClient.Post().AbsPath("api", "dashboards", "db").
Body(data).
SetHeader("Content-type", "application/json").
Do(ctx).
StatusCode(&statusCode)
require.NoError(t, result.Error(), "file: [%d] %s [status:%d]", fileCount, d.Name(), statusCode)
require.Equal(t, int(http.StatusOK), statusCode)
fileCount++
return nil
})
require.NoError(t, err)
require.Equal(t, 16, fileCount, "file count from %s", devenv)
// Helper to call search
callSearch := func(user apis.User, params string) dashboardV0.SearchResults {
require.NotNil(t, user)
ns := user.Identity.GetNamespace()
cfg := dynamic.ConfigFor(user.NewRestConfig())
cfg.GroupVersion = &dashboardV0.GroupVersion
restClient, err := k8srest.RESTClientFor(cfg)
require.NoError(t, err)
var statusCode int
req := restClient.Get().AbsPath("apis", "dashboard.grafana.app", "v0alpha1", "namespaces", ns, "search").
Param("limit", "1000").
Param("type", "dashboard") // Only search dashboards
for kv := range strings.SplitSeq(params, "&") {
if kv == "" {
continue
}
parts := strings.SplitN(kv, "=", 2)
if len(parts) == 2 {
req = req.Param(parts[0], parts[1])
}
}
res := req.Do(ctx).StatusCode(&statusCode)
require.NoError(t, res.Error())
require.Equal(t, int(http.StatusOK), statusCode)
var sr dashboardV0.SearchResults
raw, err := res.Raw()
require.NoError(t, err)
require.NoError(t, json.Unmarshal(raw, &sr))
// Normalize scores and query cost for snapshot comparison
sr.QueryCost = 0 // this depends on the hardware
sr.MaxScore = roundTo(sr.MaxScore, 3)
for i := range sr.Hits {
sr.Hits[i].Score = roundTo(sr.Hits[i].Score, 3) // 0.6250571494814442 -> 0.625
}
return sr
}
// Compare a results to snapshots
testCases := []struct {
name string
user apis.User
params string
}{
{
name: "all",
user: helper.Org1.Admin,
params: "", // only dashboards
},
{
name: "simple-query",
user: helper.Org1.Admin,
params: "query=stacking",
},
{
name: "with-text-panel",
user: helper.Org1.Admin,
params: "field=panel_types&panelType=text",
},
}
for i, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
res := callSearch(tc.user, tc.params)
jj, err := json.MarshalIndent(res, "", " ")
require.NoError(t, err)
fname := fmt.Sprintf("testdata/searchV0/t%02d-%s.json", i, tc.name)
// nolint:gosec
snapshot, err := os.ReadFile(fname)
if err != nil {
assert.Failf(t, "Failed to read snapshot", "file: %s", fname)
err = os.WriteFile(fname, jj, 0o644)
require.NoErrorf(t, err, "Failed to write snapshot file %s", fname)
return
}
if !assert.JSONEq(t, string(snapshot), string(jj)) {
err = os.WriteFile(fname, jj, 0o644)
require.NoErrorf(t, err, "Failed to write snapshot file %s", fname)
}
})
}
}
func TestIntegrationSearchPermissionFiltering(t *testing.T) {
testutil.SkipIntegrationTestInShortMode(t)
@@ -285,3 +445,11 @@ func setFolderPermissions(t *testing.T, helper *apis.K8sTestHelper, actingUser a
require.Equal(t, http.StatusOK, resp.Response.StatusCode, "Failed to set permissions for folder %s", folderUID)
}
// roundTo rounds a float64 to a specified number of decimal places.
func roundTo(n float64, decimals uint32) float64 {
// Calculate the power of 10 for the desired number of decimals
scale := math.Pow(10, float64(decimals))
// Multiply, round to the nearest integer, and then divide back
return math.Round(n*scale) / scale
}
+165
View File
@@ -0,0 +1,165 @@
{
"totalHits": 16,
"hits": [
{
"resource": "dashboards",
"name": "timeseries",
"title": "Panel Tests - Graph NG",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-by-value-color-schemes",
"title": "Panel Tests - Graph NG - By value color schemes",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-nulls",
"title": "Panel Tests - Graph NG - Discrete panels",
"tags": [
"gdev",
"panel-tests",
"graph-ng",
"timeseries",
"trend",
"state-timeline",
"transform"
]
},
{
"resource": "dashboards",
"name": "timeseries-gradient-area",
"title": "Panel Tests - Graph NG - Gradient Area Fills",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-soft-limits",
"title": "Panel Tests - Graph NG - softMin/softMax",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-yaxis-ticks",
"title": "Panel Tests - Graph NG - Y axis ticks",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-hue-gradients",
"title": "Panel Tests - GraphNG - Hue Gradients",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-time",
"title": "Panel Tests - GraphNG - Time Axis",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-thresholds",
"title": "Panel Tests - GraphNG Thresholds",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-shared-tooltip-cursor-positio",
"title": "Panel Tests - shared tooltips cursor positioning",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-bars-high-density",
"title": "Panel Tests - TimeSeries - bars high density (stroke + fill)",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-out-of-rage",
"title": "Panel Tests - Timeseries - Out of range",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-stacking",
"title": "Panel Tests - TimeSeries - stacking",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-formats",
"title": "Panel Tests - Timeseries - Supported input formats"
},
{
"resource": "dashboards",
"name": "timeseries-stacking2",
"title": "TimeSeries \u0026 BarChart Stacking",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
},
{
"resource": "dashboards",
"name": "timeseries-y-ticks-zero-decimals",
"title": "Zero Decimals Y Ticks",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
]
}
],
"maxScore": 1
}
@@ -0,0 +1,28 @@
{
"totalHits": 2,
"hits": [
{
"resource": "dashboards",
"name": "timeseries-stacking",
"title": "Panel Tests - TimeSeries - stacking",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
],
"score": 0.658
},
{
"resource": "dashboards",
"name": "timeseries-stacking2",
"title": "TimeSeries \u0026 BarChart Stacking",
"tags": [
"gdev",
"panel-tests",
"graph-ng"
],
"score": 0.625
}
],
"maxScore": 0.658
}

Some files were not shown because too many files have changed in this diff Show More