public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Scopes: Script for setting up gdev scope resources (#113448)

Browse Source 
* Script for setting up gdev scope objects

* Script for setting up gdev scope objects

* Format

* Update codeowners

* Do a feature flag check

* Formatting

* Remove FF check, because creation is explicit anyways

* Formatting
This commit is contained in:
Tobias Skarhed
2025-11-07 10:56:16 +01:00
committed by GitHub
parent 942b847952
commit 36e28963d3
5 changed files with 674 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/CODEOWNERS vendored
View File

@@ -227,6 +227,7 @@
/devenv/datasources.yaml @grafana/grafana-backend-group
/devenv/datasources_docker.yaml @grafana/grafana-backend-group
/devenv/dev-dashboards-without-uid/ @grafana/dashboards-squad
/devenv/scopes/ @grafana/grafana-operator-experience-squad
/devenv/dev-dashboards/annotations @grafana/dataviz-squad
/devenv/dev-dashboards/migrations @grafana/dataviz-squad

140
devenv/scopes/README.md Normal file
View File

@@ -0,0 +1,140 @@
# Scopes Provisioning Script
This script generates Scopes, ScopeNodes, and ScopeNavigations for Grafana development environments.
## Usage
### Create resources
```bash
# From devenv directory
./setup.sh scopes
# Or run directly
cd scopes
go run scopes.go
```
### Delete all gdev-prefixed resources
```bash
# From devenv directory
./setup.sh undev
# Or run directly
cd scopes
go run scopes.go -clean
```
**Note about caching**: The `/find/scope_navigations` endpoint used by the UI caches ScopeNavigation results for 15 minutes. After running cleanup, deleted resources may still appear in the UI until the cache expires. The resources are actually deleted (you can verify by checking the `/scopenavigations` list endpoint), but the UI will refresh after ~15 minutes or after restarting Grafana.
Doing an `Empty Cache and Hard Reload` will also help.
## Configuration
The script reads from `scopes-config.yaml` by default. You can specify a different config file:
```bash
go run scopes.go -config=my-config.yaml
```
### Configuration Format
The configuration file uses YAML format with a natural tree structure. The indentation itself represents the hierarchy:
- **scopes**: Map of scope definitions (key is the scope name)
- **tree**: Tree structure of scope nodes where the YAML structure defines parent-child relationships
- **navigations**: Map of scope navigations linking URLs to scopes (key is the navigation name)
Example:
```yaml
scopes:
app1:
title: Application 1
filters:
- key: app
operator: equals
value: app1
tree:
environments:
title: Environments
nodeType: container
children:
production:
title: Production
nodeType: container
children:
app1-prod:
title: Application 1
nodeType: leaf
linkId: app1
linkType: scope
navigations:
# Link to a dashboard
app1-nav:
url: /d/86Js1xRmk
scope: app1
# Link to another dashboard
app2-nav:
url: /d/GlAqcPgmz
scope: app2
# Custom URLs
explore-nav:
url: /explore
scope: app1
```
### Tree Structure
The tree structure uses YAML's natural indentation to represent hierarchy:
- **Key**: Unique identifier for the node (will be prefixed with "gdev-")
- **title**: Display title
- **nodeType**: Either "container" (can have children) or "leaf" (selectable scope)
- **linkId**: References a scope name (if nodeType is "leaf")
- **linkType**: Usually "scope"
- **children**: Map of child nodes (nested structure follows YAML indentation)
### Node Types
- **container**: A category/grouping node that can contain other nodes
- **leaf**: A selectable node that links to a scope
### Navigations
Navigations link URLs to scopes. The `url` field should contain the full URL path (e.g., `/d/abc123` for dashboards or `/explore` for other pages).
To find dashboard UIDs from gdev dashboards:
```bash
# Find UIDs of all gdev dashboards
find devenv/dev-dashboards -name "*.json" -exec sh -c 'echo "{}:" && jq -r ".uid // .dashboard.uid // \"NO_UID\"" {}' \;
# Or for a specific dashboard
jq -r ".uid // .dashboard.uid" devenv/dev-dashboards/all-panels.json
```
## Environment Variables
- `GRAFANA_URL`: Grafana URL (default: http://localhost:3000)
- `GRAFANA_NAMESPACE`: Namespace (default: default)
- `GRAFANA_USER`: Grafana username (default: admin)
- `GRAFANA_PASSWORD`: Grafana password (default: admin)
## Command Line Flags
- `-url`: Grafana URL
- `-namespace`: Namespace
- `-config`: Config file path (default: scopes-config.yaml)
- `-user`: Grafana username
- `-password`: Grafana password
- `-clean`: Delete all gdev-prefixed resources
## Prefix
All resources are automatically prefixed with "gdev-" to avoid conflicts with production data.

84
devenv/scopes/scopes-config.yaml Normal file
View File

@@ -0,0 +1,84 @@
scopes:
app1:
title: Application 1
filters:
- key: app
operator: equals
value: app1
app2:
title: Application 2
filters:
- key: app
operator: equals
value: app2
cluster1:
title: Cluster 1
filters:
- key: cluster
operator: equals
value: cluster1
tree:
gdev-scopes:
title: gdev-scopes
nodeType: container
children:
production:
title: Production
nodeType: container
children:
app1-prod:
title: Application 1
nodeType: leaf
linkId: app1
linkType: scope
app2-prod:
title: Application 2
nodeType: leaf
linkId: app2
linkType: scope
test-cases:
title: Test cases
nodeType: container
disableMultiSelect: true
children:
test-case-1:
title: Test case 1
nodeType: leaf
linkId: test-case-1
linkType: scope
test-case-2:
title: Test case 2
nodeType: leaf
linkId: test-case-2
linkType: scope
clusters:
title: Clusters
nodeType: container
linkId: cluster1
linkType: scope
children:
cluster1-node:
title: Cluster 1
nodeType: leaf
linkId: cluster1
linkType: scope
navigations:
# Example: Link to a dashboard
app1-nav:
url: /d/86Js1xRmk
scope: app1
# Example: Link to a dashboard with full URL (already has /d/)
app2-nav:
url: /d/GlAqcPgmz
scope: app2
# Example: Custom URL path
custom-nav:
url: /explore
scope: app1

433
devenv/scopes/scopes.go Normal file
View File

@@ -0,0 +1,433 @@
//go:build ignore
// +build ignore
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"strings"
"gopkg.in/yaml.v3"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/grafana/grafana/apps/scope/pkg/apis/scope/v0alpha1"
)
const (
prefix = "gdev"
apiVersion = "scope.grafana.app/v0alpha1"
defaultURL = "http://localhost:3000"
defaultUser = "admin"
)
var (
grafanaURL = flag.String("url", getEnv("GRAFANA_URL", defaultURL), "Grafana URL")
namespace = flag.String("namespace", getEnv("GRAFANA_NAMESPACE", "default"), "Namespace")
configFile = flag.String("config", "scopes-config.yaml", "Config file path")
user = flag.String("user", getEnv("GRAFANA_USER", defaultUser), "Grafana username")
password = flag.String("password", getEnv("GRAFANA_PASSWORD", "admin"), "Grafana password")
cleanupFlag = flag.Bool("clean", false, "Delete all gdev-prefixed resources")
)
func getEnv(key, defaultValue string) string {
if value := os.Getenv(key); value != "" {
return value
}
return defaultValue
}
type Config struct {
Scopes map[string]ScopeConfig `yaml:"scopes"`
Tree map[string]TreeNode `yaml:"tree"`
Navigations map[string]NavigationConfig `yaml:"navigations"`
}
// ScopeConfig is used for YAML parsing - converts to v0alpha1.ScopeSpec
type ScopeConfig struct {
Title string `yaml:"title"`
Filters []ScopeFilterConfig `yaml:"filters"`
}
// ScopeFilterConfig is used for YAML parsing - converts to v0alpha1.ScopeFilter
type ScopeFilterConfig struct {
Key string `yaml:"key"`
Value string `yaml:"value"`
Values []string `yaml:"values,omitempty"`
Operator string `yaml:"operator"`
}
// TreeNode is used for YAML parsing - converts to v0alpha1.ScopeNodeSpec
type TreeNode struct {
Title string `yaml:"title"`
NodeType string `yaml:"nodeType"`
LinkID string `yaml:"linkId,omitempty"`
LinkType string `yaml:"linkType,omitempty"`
Children map[string]TreeNode `yaml:"children,omitempty"`
}
type NavigationConfig struct {
URL string `yaml:"url"` // URL path (e.g., /d/abc123 or /explore)
Scope string `yaml:"scope"`
}
// Helper function to convert ScopeFilterConfig to v0alpha1.ScopeFilter
func convertFilter(cfg ScopeFilterConfig) v0alpha1.ScopeFilter {
filter := v0alpha1.ScopeFilter{
Key: cfg.Key,
Value: cfg.Value,
Values: cfg.Values,
Operator: v0alpha1.FilterOperator(cfg.Operator),
}
return filter
}
// Helper function to convert ScopeConfig to v0alpha1.ScopeSpec
func convertScopeSpec(cfg ScopeConfig) v0alpha1.ScopeSpec {
filters := make([]v0alpha1.ScopeFilter, len(cfg.Filters))
for i, f := range cfg.Filters {
filters[i] = convertFilter(f)
}
return v0alpha1.ScopeSpec{
Title: cfg.Title,
Filters: filters,
}
}
type Client struct {
baseURL string
namespace string
httpClient *http.Client
auth string
}
func NewClient(baseURL, namespace, user, password string) *Client {
return &Client{
baseURL: baseURL,
namespace: namespace,
httpClient: &http.Client{},
auth: basicAuth(user, password),
}
}
func basicAuth(username, password string) string {
return fmt.Sprintf("%s:%s", username, password)
}
func (c *Client) makeRequest(method, endpoint string, body []byte) error {
url := fmt.Sprintf("%s/apis/%s/namespaces/%s%s", c.baseURL, apiVersion, c.namespace, endpoint)
var req *http.Request
var err error
if body != nil {
req, err = http.NewRequest(method, url, bytes.NewBuffer(body))
} else {
req, err = http.NewRequest(method, url, nil)
}
if err != nil {
return fmt.Errorf("failed to create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.SetBasicAuth(strings.Split(c.auth, ":")[0], strings.Split(c.auth, ":")[1])
resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("request failed: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
bodyBytes, _ := io.ReadAll(resp.Body)
// For DELETE requests, 404 is acceptable (resource already deleted)
if resp.StatusCode == 404 {
return nil
}
return fmt.Errorf("API request failed: HTTP %d - %s", resp.StatusCode, string(bodyBytes))
}
return nil
}
func (c *Client) createScope(name string, cfg ScopeConfig) error {
prefixedName := prefix + "-" + name
spec := convertScopeSpec(cfg)
resource := v0alpha1.Scope{
TypeMeta: metav1.TypeMeta{
APIVersion: apiVersion,
Kind: "Scope",
},
ObjectMeta: metav1.ObjectMeta{
Name: prefixedName,
},
Spec: spec,
}
body, err := json.Marshal(resource)
if err != nil {
return fmt.Errorf("failed to marshal scope: %w", err)
}
fmt.Printf("✓ Creating scope: %s\n", prefixedName)
return c.makeRequest("POST", "/scopes", body)
}
func (c *Client) createScopeNode(name string, node TreeNode, parentName string) error {
prefixedName := prefix + "-" + name
prefixedParent := ""
prefixedLinkID := ""
if parentName != "" {
prefixedParent = prefix + "-" + parentName
}
if node.LinkID != "" {
prefixedLinkID = prefix + "-" + node.LinkID
}
nodeType := v0alpha1.NodeType(node.NodeType)
if nodeType == "" {
nodeType = v0alpha1.NodeTypeContainer
}
linkType := v0alpha1.LinkType(node.LinkType)
if linkType == "" {
linkType = v0alpha1.LinkTypeScope
}
spec := v0alpha1.ScopeNodeSpec{
Title: node.Title,
NodeType: nodeType,
DisableMultiSelect: false,
}
if prefixedParent != "" {
spec.ParentName = prefixedParent
}
if prefixedLinkID != "" {
spec.LinkID = prefixedLinkID
spec.LinkType = linkType
}
resource := v0alpha1.ScopeNode{
TypeMeta: metav1.TypeMeta{
APIVersion: apiVersion,
Kind: "ScopeNode",
},
ObjectMeta: metav1.ObjectMeta{
Name: prefixedName,
},
Spec: spec,
}
body, err := json.Marshal(resource)
if err != nil {
return fmt.Errorf("failed to marshal scope node: %w", err)
}
fmt.Printf("✓ Creating scope node: %s\n", prefixedName)
return c.makeRequest("POST", "/scopenodes", body)
}
func (c *Client) createScopeNavigation(name string, nav NavigationConfig) error {
prefixedName := prefix + "-" + name
prefixedScope := prefix + "-" + nav.Scope
if nav.URL == "" {
return fmt.Errorf("navigation %s must have 'url' specified", name)
}
spec := v0alpha1.ScopeNavigationSpec{
URL: nav.URL,
Scope: prefixedScope,
}
resource := v0alpha1.ScopeNavigation{
TypeMeta: metav1.TypeMeta{
APIVersion: apiVersion,
Kind: "ScopeNavigation",
},
ObjectMeta: metav1.ObjectMeta{
Name: prefixedName,
},
Spec: spec,
}
body, err := json.Marshal(resource)
if err != nil {
return fmt.Errorf("failed to marshal scope navigation: %w", err)
}
fmt.Printf("✓ Creating scope navigation: %s\n", prefixedName)
return c.makeRequest("POST", "/scopenavigations", body)
}
func (c *Client) createTreeNodes(children map[string]TreeNode, parentName string) error {
for name, node := range children {
// Build full node name by appending to parent name
// This makes it easy to see the tree path from the node name
fullNodeName := name
if parentName != "" {
fullNodeName = parentName + "-" + name
}
// parentName here is the full parent name (already includes full path)
err := c.createScopeNode(fullNodeName, node, parentName)
if err != nil {
return err
}
if len(node.Children) > 0 {
// Pass fullNodeName as parent for children (will be prefixed with "gdev-" in createScopeNode)
if err := c.createTreeNodes(node.Children, fullNodeName); err != nil {
return err
}
}
}
return nil
}
func (c *Client) deleteResources() {
fmt.Println("Deleting all gdev-prefixed resources...")
// Delete scopes (silently handle errors if endpoints aren't available)
c.deleteResourceType("/scopes", "scope")
// Delete scope nodes
c.deleteResourceType("/scopenodes", "scope node")
// Delete scope navigations
c.deleteResourceType("/scopenavigations", "scope navigation")
fmt.Println("✓ Cleanup complete")
}
func (c *Client) deleteResourceType(endpoint, resourceType string) {
url := fmt.Sprintf("%s/apis/%s/namespaces/%s%s", c.baseURL, apiVersion, c.namespace, endpoint)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
// Silently skip if we can't create request
return
}
req.Header.Set("Content-Type", "application/json")
req.SetBasicAuth(strings.Split(c.auth, ":")[0], strings.Split(c.auth, ":")[1])
resp, err := c.httpClient.Do(req)
if err != nil {
// Silently skip if endpoint isn't available
return
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
// Silently skip if endpoint returns error (might not be available)
return
}
var listResponse struct {
Items []struct {
Metadata struct {
Name string `json:"name"`
} `json:"metadata"`
} `json:"items"`
}
bodyBytes, _ := io.ReadAll(resp.Body)
if err := json.Unmarshal(bodyBytes, &listResponse); err != nil {
// Silently skip if we can't decode response
return
}
if len(listResponse.Items) == 0 {
return
}
deletedCount := 0
for _, item := range listResponse.Items {
if strings.HasPrefix(item.Metadata.Name, prefix+"-") {
fmt.Printf(" Deleting %s: %s\n", resourceType, item.Metadata.Name)
deleteURL := fmt.Sprintf("%s/%s", endpoint, item.Metadata.Name)
if err := c.makeRequest("DELETE", deleteURL, nil); err != nil {
// Silently skip deletion errors
} else {
deletedCount++
}
}
}
}
func main() {
flag.Parse()
client := NewClient(*grafanaURL, *namespace, *user, *password)
if *cleanupFlag {
// Cleanup should be silent if endpoints aren't available
client.deleteResources()
return
}
configData, err := os.ReadFile(*configFile)
if err != nil {
fmt.Fprintf(os.Stderr, "Error reading config file: %v\n", err)
os.Exit(1)
}
var config Config
if err := yaml.Unmarshal(configData, &config); err != nil {
fmt.Fprintf(os.Stderr, "Error parsing config file: %v\n", err)
os.Exit(1)
}
fmt.Printf("Loading configuration from: %s\n", *configFile)
fmt.Printf("Grafana URL: %s\n", *grafanaURL)
fmt.Printf("Namespace: %s\n", *namespace)
fmt.Printf("Prefix: %s\n\n", prefix)
// Create scopes
fmt.Println("Creating scopes...")
for name, scope := range config.Scopes {
if err := client.createScope(name, scope); err != nil {
fmt.Fprintf(os.Stderr, "Error creating scope %s: %v\n", name, err)
os.Exit(1)
}
}
fmt.Println()
// Create scope nodes (tree structure)
if len(config.Tree) > 0 {
fmt.Println("Creating scope nodes...")
if err := client.createTreeNodes(config.Tree, ""); err != nil {
fmt.Fprintf(os.Stderr, "Error creating scope nodes: %v\n", err)
os.Exit(1)
}
fmt.Println()
}
// Create scope navigations
if len(config.Navigations) > 0 {
fmt.Println("Creating scope navigations...")
for name, nav := range config.Navigations {
if err := client.createScopeNavigation(name, nav); err != nil {
fmt.Fprintf(os.Stderr, "Error creating scope navigation %s: %v\n", name, err)
os.Exit(1)
}
}
fmt.Println()
}
fmt.Println("✓ All resources created successfully!")
}

16
devenv/setup.sh
View File

@@ -19,6 +19,13 @@ bulkFolders() {
ln -s -f ../../../devenv/bulk-folders/bulk-folders.yaml ../conf/provisioning/dashboards/bulk-folders.yaml
}
scopes() {
echo -e "\xE2\x9C\x94 Setting up scopes, scope nodes, and scope navigations"
cd scopes
go run scopes.go
cd ..
}
requiresJsonnet() {
if ! type "jsonnet" > /dev/null; then
echo "you need you install jsonnet to run this script"
@@ -49,6 +56,12 @@ undev() {
rm -rf bulk-folders/Bulk\ Folder*
echo -e " \xE2\x9C\x94 Reverting bulk-folders provisioning"
# Removing scopes, scope nodes, and scope navigations
cd scopes
go run scopes.go -clean
cd ..
echo -e " \xE2\x9C\x94 Deleting scopes, scope nodes, and scope navigations"
# Removing the symlinks
rm -f ../conf/provisioning/dashboards/custom.yaml
rm -f ../conf/provisioning/dashboards/bulk-folders.yaml
@@ -63,6 +76,7 @@ usage() {
echo " bulk-dashboards - provision 400 dashboards"
echo " bulk-folders [folders] [dashboards] - provision many folders with dashboards"
echo " bulk-folders - provision 200 folders with 3 dashboards in each"
echo " scopes - provision scopes, scope nodes, and scope navigations"
echo " no args - provision core datasources and dev dashboards"
echo " undev - removes any provisioning done by the setup.sh"
}
@@ -80,6 +94,8 @@ main() {
bulkDashboard
elif [[ $cmd == "bulk-folders" ]]; then
bulkFolders "$arg1"
elif [[ $cmd == "scopes" ]]; then
scopes
elif [[ $cmd == "undev" ]]; then
undev
else