Compare commits

...

11 Commits

Author SHA1 Message Date
Larissa Wandzura e83c88bf75 made updates based on feedback 2026-01-09 09:48:21 -06:00
Larissa Wandzura e81769147f fixed linter gripes 2025-12-12 14:47:11 -06:00
Larissa Wandzura 70e49947a1 created the troubleshooting guide 2025-12-12 14:23:52 -06:00
colin-stuart 5156177079 SCIM: show error if SCIM-provisioned user attempts login with non-SAML auth module (#115271) 2025-12-12 13:51:37 -06:00
Paul Marbach 4817ecf6a3 Sparkline: Guess decimals rather than going with 0 (#115246)
* Sparkline: Guess decimals rather than going with 0

* Update packages/grafana-ui/src/components/Sparkline/utils.test.ts
2025-12-12 13:59:54 -05:00
Renato Costa c73cab8eef chore: add cleanup task for duplicated provisioned dashboards (#115103)
* chore: add cleanup task for duplicated provisioned dashboards
2025-12-12 13:56:47 -05:00
Adela Almasan a37ebf609e VizSuggestions: Fix unique key warning (#115112) 2025-12-12 12:25:03 -06:00
Kristina Demeshchik b29e8ccb45 Dashboards: Generate default tab title when converting rows with empty titles to tabs (#115256)
Generate default title for empty row titles
2025-12-12 13:14:56 -05:00
Matias Chomicki 644f7b7001 Infinite scroll: Fix interaction with client-side filter (#115243)
Infinite scroll: fix interaction with client-side filter
2025-12-12 18:59:49 +01:00
Alexander Zobnin 629570926d Zanzana: Fix resource translation for dashboards (#115077) 2025-12-12 11:05:10 -06:00
Will Assis 1b59c82b74 Revert "Unified-storage: sql backend key path backfill (#115033)" (#115257)
This reverts commit b2dd095bd8.
2025-12-12 17:00:08 +00:00
25 changed files with 1221 additions and 178 deletions
+2 -1
View File
@@ -520,7 +520,7 @@ i18next.config.ts @grafana/grafana-frontend-platform
/e2e-playwright/various-suite/solo-route.spec.ts @grafana/dashboards-squad
/e2e-playwright/various-suite/trace-view-scrolling.spec.ts @grafana/observability-traces-and-profiling
/e2e-playwright/various-suite/verify-i18n.spec.ts @grafana/grafana-frontend-platform
/e2e-playwright/various-suite/visualization-suggestions.spec.ts @grafana/dashboards-squad
/e2e-playwright/various-suite/visualization-suggestions.spec.ts @grafana/dataviz-squad
/e2e-playwright/various-suite/perf-test.spec.ts @grafana/grafana-frontend-platform
# Packages
@@ -956,6 +956,7 @@ playwright.storybook.config.ts @grafana/grafana-frontend-platform
/public/app/features/notifications/ @grafana/grafana-search-navigate-organise
/public/app/features/org/ @grafana/grafana-search-navigate-organise
/public/app/features/panel/ @grafana/dashboards-squad
/public/app/features/panel/components/VizTypePicker/VisualizationSuggestions.tsx @grafana/dataviz-squad
/public/app/features/panel/suggestions/ @grafana/dataviz-squad
/public/app/features/playlist/ @grafana/dashboards-squad
/public/app/features/plugins/ @grafana/plugins-platform-frontend
@@ -113,6 +113,7 @@ The following documentation will help you get started working with Prometheus an
- [Configure the Prometheus data source](ref:configure-prometheus-data-source)
- [Prometheus query editor](query-editor/)
- [Template variables](template-variables/)
- [Troubleshooting](troubleshooting/)
## Exemplars
@@ -0,0 +1,313 @@
---
aliases:
- ../../data-sources/prometheus/troubleshooting/
description: Troubleshooting the Prometheus data source in Grafana
keywords:
- grafana
- prometheus
- troubleshooting
- errors
- promql
labels:
products:
- cloud
- enterprise
- oss
menuTitle: Troubleshooting
title: Troubleshoot Prometheus data source issues
weight: 600
---
# Troubleshoot Prometheus data source issues
This document provides troubleshooting information for common errors you may encounter when using the Prometheus data source in Grafana.
## Connection errors
The following errors occur when Grafana cannot establish or maintain a connection to Prometheus.
### Failed to connect to Prometheus
**Error message:** "There was an error returned querying the Prometheus API"
**Cause:** Grafana cannot establish a network connection to the Prometheus server.
**Solution:**
1. Verify that the Prometheus server URL is correct in the data source configuration.
1. Check that Prometheus is running and accessible from the Grafana server.
1. Ensure the URL includes the protocol (`http://` or `https://`).
1. Verify the port is correct (the Prometheus default port is `9090`).
1. Ensure there are no firewall rules blocking the connection.
1. If Grafana and Prometheus are running in separate containers, use the container IP address or hostname instead of `localhost`.
1. For Grafana Cloud, ensure you have configured [Private data source connect](https://grafana.com/docs/grafana-cloud/connect-externally-hosted/private-data-source-connect/) if your Prometheus instance is not publicly accessible.
### Request timed out
**Error message:** "context deadline exceeded" or "request timeout"
**Cause:** The connection to Prometheus timed out before receiving a response.
**Solution:**
1. Check the network latency between Grafana and Prometheus.
1. Verify that Prometheus is not overloaded or experiencing performance issues.
1. Increase the **Query timeout** setting in the data source configuration under **Interval behavior**.
1. Check the [Grafana server timeout configuration](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana#timeout) for server-level timeout settings.
1. Reduce the time range or complexity of your query.
1. Check if any network devices (load balancers, proxies) are timing out the connection.
### Failed to parse data source URL
**Error message:** "Failed to parse data source URL"
**Cause:** The URL entered in the data source configuration is not valid.
**Solution:**
1. Verify the URL format is correct (for example, `http://localhost:9090` or `https://prometheus.example.com:9090`).
1. Ensure the URL includes the protocol (`http://` or `https://`).
1. Remove any trailing slashes or invalid characters from the URL.
## Authentication errors
The following errors occur when there are issues with authentication credentials or permissions.
### Unauthorized (401)
**Error message:** "401 Unauthorized" or "Authorization failed"
**Cause:** The authentication credentials are invalid or missing.
**Solution:**
1. Verify that the username and password are correct if using basic authentication.
1. Check that the authentication method selected matches your Prometheus configuration.
1. If using a reverse proxy with authentication, verify the credentials are correct.
1. For AWS SigV4 authentication, verify the IAM credentials and permissions. Alternatively, consider using the [Amazon Managed Service for Prometheus data source](https://grafana.com/grafana/plugins/grafana-amazonprometheus-datasource/) for simplified AWS authentication.
### Forbidden (403)
**Error message:** "403 Forbidden" or "Access denied"
**Cause:** The authenticated user does not have permission to access the requested resource.
**Solution:**
1. Verify the user has read access to the Prometheus API.
1. Check Prometheus security settings and access control configuration.
1. If using a reverse proxy, verify the proxy is not blocking the request.
1. For AWS Managed Prometheus, verify the IAM policy grants the required permissions. Alternatively, consider using the [Amazon Managed Service for Prometheus data source](https://grafana.com/grafana/plugins/grafana-amazonprometheus-datasource/) for simplified AWS authentication.
## Query errors
The following errors occur when there are issues with PromQL syntax or query execution.
### Query syntax error
**Error message:** "parse error: unexpected character" or "bad_data: 1:X: parse error"
**Cause:** The PromQL query contains invalid syntax.
**Alternative cause:** A proxy between Grafana and Prometheus requires authentication. When proxy authentication fails, the proxy redirects the request to an HTML authentication page. Grafana cannot parse the HTML response, which results in a parse error. This appears to be a query issue but is actually a proxy authentication issue.
**Solution:**
1. Check your query syntax for typos or invalid characters.
1. Verify that metric names and label names are valid identifiers.
1. Ensure string values in label matchers are enclosed in quotes.
1. Use the Prometheus expression browser to test your query directly.
1. Refer to the [Prometheus querying documentation](https://prometheus.io/docs/prometheus/latest/querying/basics/) for syntax guidance.
1. If you have a proxy between Grafana and Prometheus, verify that proxy authentication is correctly configured. Check your proxy logs for authentication failures or redirects.
### Query returns no data for a metric
**Symptom:** The query returns no data and the visualization is empty.
**Cause:** The specified metric does not exist in Prometheus, or there is no data for the selected time range.
**Solution:**
1. Verify the metric name is spelled correctly.
1. Check that the metric is being scraped by Prometheus.
1. Use the Prometheus API to browse available metrics at `/api/v1/label/__name__/values`.
1. Use the [target metadata API](https://prometheus.io/docs/prometheus/latest/querying/api#querying-target-metadata) to verify which metrics a target exposes.
1. Verify the time range includes data for the metric.
### Query timeout limit exceeded
**Error message:** "query timed out in expression evaluation" or "query processing would load too many samples"
**Cause:** The query took longer than the configured timeout limit or would return too many samples.
**Solution:**
1. Reduce the time range of your query.
1. Add more specific label filters to limit the data scanned.
1. Increase the **Query timeout** setting in the data source configuration.
1. Use aggregation functions like `sum()`, `avg()`, or `rate()` to reduce the number of time series.
1. Increase the `query.timeout` or `query.max-samples` settings in Prometheus if you have admin access.
### Too many time series
**Error message:** "exceeded maximum resolution of 11,000 points per timeseries" or "maximum number of series limit exceeded"
**Cause:** The query is returning more time series or data points than the configured limits allow.
**Solution:**
1. Reduce the time range of your query.
1. Add label filters to limit the number of time series returned.
1. Increase the **Min interval** or **Resolution** in the query options to reduce the number of data points.
1. Use aggregation functions to combine time series.
1. Adjust the **Series limit** setting in the data source configuration under **Other settings**.
### Invalid function or aggregation
**Error message:** "unknown function" or "parse error: unexpected aggregation"
**Cause:** The query uses an invalid or unsupported PromQL function.
**Solution:**
1. Verify the function name is spelled correctly and is a valid PromQL function.
1. Check that you are using the correct syntax for the function.
1. Ensure your Prometheus version supports the function you are using.
1. Refer to the [PromQL functions documentation](https://prometheus.io/docs/prometheus/latest/querying/functions/) for available functions.
## Configuration errors
The following errors occur when the data source is not configured correctly.
### Invalid Prometheus type
**Error message:** Unexpected behavior when querying metrics or labels
**Cause:** The **Prometheus type** setting does not match your actual Prometheus-compatible database.
**Solution:**
1. Open the data source configuration in Grafana.
1. Under **Performance**, select the correct **Prometheus type** (Prometheus, Cortex, Mimir, or Thanos).
1. Different database types support different APIs, so setting this incorrectly may cause unexpected behavior.
### Scrape interval mismatch
**Error message:** Data appears sparse or aggregated incorrectly
**Cause:** The **Scrape interval** setting in Grafana does not match the actual scrape interval in Prometheus.
**Solution:**
1. Check your Prometheus configuration file for the `scrape_interval` setting.
1. Update the **Scrape interval** in the Grafana data source configuration under **Interval behavior** to match.
1. If the Grafana interval is higher than the Prometheus interval, you may see less data points than expected.
## TLS and certificate errors
The following errors occur when there are issues with TLS configuration.
### Certificate verification failed
**Error message:** "x509: certificate signed by unknown authority" or "certificate verify failed"
**Cause:** Grafana cannot verify the TLS certificate presented by Prometheus.
**Solution:**
1. If using a self-signed certificate, enable **Add self-signed certificate** in the TLS settings and add your CA certificate.
1. Verify the certificate chain is complete and valid.
1. Ensure the certificate has not expired.
1. As a temporary workaround for testing, enable **Skip TLS verify** (not recommended for production).
### TLS handshake error
**Error message:** "TLS: handshake failure" or "connection reset"
**Cause:** The TLS handshake between Grafana and Prometheus failed.
**Solution:**
1. Verify that Prometheus is configured to use TLS.
1. Check that the TLS version and cipher suites are compatible.
1. If using client certificates, ensure they are correctly configured in the **TLS client authentication** section.
1. Verify the server name matches the certificate's Common Name or Subject Alternative Name.
## Other common issues
The following issues don't produce specific error messages but are commonly encountered.
### Empty query results
**Cause:** The query returns no data.
**Solution:**
1. Verify the time range includes data in Prometheus.
1. Check that the metric and label names are correct.
1. Test the query directly in the Prometheus expression browser.
1. Ensure label filters are not excluding all data.
1. For rate or increase functions, ensure the time range is at least twice the scrape interval.
### Slow query performance
**Cause:** Queries take a long time to execute.
**Solution:**
1. Reduce the time range of your query.
1. Add more specific label filters to limit the data scanned.
1. Increase the **Min interval** in the query options.
1. Check Prometheus server performance and resource utilization.
1. Enable **Disable metrics lookup** in the data source configuration for large Prometheus instances.
1. Enable **Incremental querying (beta)** to cache query results.
1. Consider using recording rules to pre-aggregate frequently queried data.
### Data appears delayed or missing recent points
**Cause:** The visualization doesn't show the most recent data.
**Solution:**
1. Check the dashboard time range and refresh settings.
1. Verify the **Scrape interval** is configured correctly.
1. Ensure Prometheus has finished scraping the target.
1. Check for clock synchronization issues between Grafana and Prometheus.
1. For `rate()` and similar functions, remember that they need at least two data points to calculate.
### Exemplars not showing
**Cause:** Exemplar data is not appearing in visualizations.
**Solution:**
1. Verify that exemplars are enabled in the data source configuration under **Exemplars**.
1. Check that your Prometheus version supports exemplars (2.26+).
1. Ensure your instrumented application is sending exemplar data.
1. Verify the tracing data source is correctly configured for the exemplar link.
1. Enable the **Exemplars** toggle in the query editor.
### Alerting rules not visible
**Cause:** Prometheus alerting rules are not appearing in the Grafana Alerting UI.
**Solution:**
1. Verify that **Manage alerts via Alerting UI** is enabled in the data source configuration.
1. Check that Prometheus has alerting rules configured.
1. Ensure Grafana can access the Prometheus rules API endpoint.
1. Note that for Prometheus (unlike Mimir), the Alerting UI only supports viewing existing rules, not creating new ones.
## Get additional help
If you continue to experience issues after following this troubleshooting guide:
1. Check the [Prometheus documentation](https://prometheus.io/docs/) for API and PromQL guidance.
1. Review the [Grafana community forums](https://community.grafana.com/) for similar issues.
1. Contact Grafana Support if you are a Cloud Pro, Cloud Contracted, or Enterprise user.
1. When reporting issues, include:
- Grafana version
- Prometheus version and type (Prometheus, Mimir, Cortex, Thanos)
- Error messages (redact sensitive information)
- Steps to reproduce
- Relevant configuration such as data source settings, query timeout, and TLS settings (redact tokens, passwords, and other credentials)
@@ -119,7 +119,14 @@ describe('Get y range', () => {
values: [2, 1.999999999999999, 2.000000000000001, 2, 2],
type: FieldType.number,
config: {},
state: { range: { min: 1.999999999999999, max: 2.000000000000001, delta: 0 } },
state: { range: { min: 1.9999999999999999999, max: 2.000000000000000001, delta: 0 } },
};
const decimalsNotCloseYField: Field = {
name: 'y',
values: [2, 0.0094, 0.0053, 0.0078, 0.0061],
type: FieldType.number,
config: {},
state: { range: { min: 0.0053, max: 0.0094, delta: 0.0041 } },
};
const xField: Field = {
name: 'x',
@@ -183,6 +190,11 @@ describe('Get y range', () => {
field: decimalsCloseYField,
expected: [2, 4],
},
{
description: 'decimal values which are not close to equal should not be rounded out',
field: decimalsNotCloseYField,
expected: [0.0053, 0.0094],
},
])(`should return correct range for $description`, ({ field, expected }) => {
const actual = getYRange(getAlignedFrame(field));
expect(actual).toEqual(expected);
@@ -8,6 +8,7 @@ import {
FieldType,
getFieldColorModeForField,
GrafanaTheme2,
guessDecimals,
isLikelyAscendingVector,
nullToValue,
roundDecimals,
@@ -76,8 +77,6 @@ export function getYRange(alignedFrame: DataFrame): Range.MinMax {
min = Math.min(min!, field.config.min ?? Infinity);
max = Math.max(max!, field.config.max ?? -Infinity);
// console.log({ min, max });
// if noValue is set, ensure that it is included in the range as well
const noValue = +field.config?.noValue!;
if (!Number.isNaN(noValue)) {
@@ -85,9 +84,11 @@ export function getYRange(alignedFrame: DataFrame): Range.MinMax {
max = Math.max(max, noValue);
}
const decimals = field.config.decimals ?? Math.max(guessDecimals(min), guessDecimals(max));
// call roundDecimals to mirror what is going to eventually happen in uplot
let roundedMin = roundDecimals(min, field.config.decimals ?? 0);
let roundedMax = roundDecimals(max, field.config.decimals ?? 0);
let roundedMin = roundDecimals(min, decimals);
let roundedMax = roundDecimals(max, decimals);
// if the rounded min and max are different,
// we can return the real min and max.
@@ -102,11 +103,9 @@ export function getYRange(alignedFrame: DataFrame): Range.MinMax {
roundedMax = 1;
} else if (roundedMin < 0) {
// both are negative
// max = 0;
roundedMin *= 2;
} else {
// both are positive
// min = 0;
roundedMax *= 2;
}
@@ -77,6 +77,10 @@ var (
"user.sync.user-externalUID-mismatch",
errutil.WithPublicMessage("User externalUID mismatch"),
)
errSCIMAuthModuleMismatch = errutil.Unauthorized(
"user.sync.scim-auth-module-mismatch",
errutil.WithPublicMessage("User was provisioned via SCIM and must login via SAML"),
)
)
var (
@@ -308,6 +312,21 @@ func (s *UserSync) SyncUserHook(ctx context.Context, id *authn.Identity, _ *auth
// just try to fetch the user one more to make the other request work.
if errors.Is(err, user.ErrUserAlreadyExists) {
usr, _, err = s.getUser(ctx, id)
// Check if this is a SCIM-provisioned user trying to login via an auth module that is not SAML or GCOM
if err == nil && usr != nil && usr.IsProvisioned && id.AuthenticatedBy != login.GrafanaComAuthModule {
_, authErr := s.authInfoService.GetAuthInfo(ctx, &login.GetAuthInfoQuery{
UserId: usr.ID,
AuthModule: id.AuthenticatedBy,
})
if errors.Is(authErr, user.ErrUserNotFound) {
s.log.FromContext(ctx).Error("SCIM-provisioned user attempted login via non-SAML auth module",
"user_id", usr.ID,
"attempted_module", id.AuthenticatedBy,
)
return errSCIMAuthModuleMismatch.Errorf("user was provisioned via SCIM but attempted login via %s", id.AuthenticatedBy)
}
}
}
if err != nil {
@@ -1926,3 +1926,100 @@ func TestUserSync_SCIMLoginUsageStatSet(t *testing.T) {
finalCount := finalStats["stats.features.scim.has_successful_login.count"].(int)
require.Equal(t, int(1), finalCount)
}
func TestUserSync_SyncUserHook_SCIMAuthModuleMismatch(t *testing.T) {
userSrv := usertest.NewMockService(t)
authInfoSrv := authinfotest.NewMockAuthInfoService(t)
userSrv.On("GetByEmail", mock.Anything, mock.Anything).Return(nil, user.ErrUserNotFound).Once()
userSrv.On("Create", mock.Anything, mock.Anything).Return(nil, user.ErrUserAlreadyExists).Once()
userSrv.On("GetByEmail", mock.Anything, mock.Anything).Return(&user.User{
ID: 1,
Email: "test@test.com",
IsProvisioned: true,
}, nil).Once()
authInfoSrv.On("GetAuthInfo", mock.Anything, mock.MatchedBy(func(q *login.GetAuthInfoQuery) bool {
return q.AuthModule == "oauth_azuread"
})).Return(nil, user.ErrUserNotFound).Once()
s := ProvideUserSync(
userSrv,
authinfoimpl.ProvideOSSUserProtectionService(),
authInfoSrv,
&quotatest.FakeQuotaService{},
tracing.NewNoopTracerService(),
featuremgmt.WithFeatures(),
setting.NewCfg(),
nil,
)
email := "test@test.com"
err := s.SyncUserHook(context.Background(), &authn.Identity{
AuthenticatedBy: "oauth_azuread",
ClientParams: authn.ClientParams{
SyncUser: true,
AllowSignUp: true,
LookUpParams: login.UserLookupParams{
Email: &email,
},
},
}, nil)
require.Error(t, err)
assert.ErrorIs(t, err, errSCIMAuthModuleMismatch)
assert.Contains(t, err.Error(), "SCIM")
assert.Contains(t, err.Error(), "oauth_azuread")
}
func TestUserSync_SyncUserHook_SCIMUserAllowsGCOMLogin(t *testing.T) {
userSrv := usertest.NewMockService(t)
authInfoSrv := authinfotest.NewMockAuthInfoService(t)
authInfoSrv.On("GetAuthInfo", mock.Anything, mock.MatchedBy(func(q *login.GetAuthInfoQuery) bool {
return q.AuthModule == login.GrafanaComAuthModule && q.AuthId == "gcom-user-123"
})).Return(nil, user.ErrUserNotFound).Once()
userSrv.On("GetByEmail", mock.Anything, mock.Anything).Return(nil, user.ErrUserNotFound).Once()
userSrv.On("Create", mock.Anything, mock.Anything).Return(nil, user.ErrUserAlreadyExists).Once()
authInfoSrv.On("GetAuthInfo", mock.Anything, mock.MatchedBy(func(q *login.GetAuthInfoQuery) bool {
return q.AuthModule == login.GrafanaComAuthModule && q.AuthId == "gcom-user-123"
})).Return(nil, user.ErrUserNotFound).Once()
userSrv.On("GetByEmail", mock.Anything, mock.Anything).Return(&user.User{
ID: 1,
Email: "test@test.com",
IsProvisioned: true,
}, nil).Once()
s := ProvideUserSync(
userSrv,
authinfoimpl.ProvideOSSUserProtectionService(),
authInfoSrv,
&quotatest.FakeQuotaService{},
tracing.NewNoopTracerService(),
featuremgmt.WithFeatures(),
setting.NewCfg(),
nil,
)
email := "test@test.com"
err := s.SyncUserHook(context.Background(), &authn.Identity{
AuthenticatedBy: login.GrafanaComAuthModule,
AuthID: "gcom-user-123",
ClientParams: authn.ClientParams{
SyncUser: true,
AllowSignUp: true,
LookUpParams: login.UserLookupParams{
Email: &email,
},
},
}, nil)
require.NoError(t, err)
}
+31 -1
View File
@@ -4,8 +4,12 @@ import (
"google.golang.org/protobuf/types/known/structpb"
authzv1 "github.com/grafana/authlib/authz/proto/v1"
dashboardV1 "github.com/grafana/grafana/apps/dashboard/pkg/apis/dashboard/v1beta1"
folders "github.com/grafana/grafana/apps/folder/pkg/apis/folder/v1beta1"
iamv0alpha1 "github.com/grafana/grafana/apps/iam/pkg/apis/iam/v0alpha1"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/services/accesscontrol"
authzextv1 "github.com/grafana/grafana/pkg/services/authz/proto/v1"
)
@@ -44,7 +48,8 @@ func getTypeInfo(group, resource string) (typeInfo, bool) {
func NewResourceInfoFromCheck(r *authzv1.CheckRequest) ResourceInfo {
typ, relations := getTypeAndRelations(r.GetGroup(), r.GetResource())
return newResource(
resource := newResource(
typ,
r.GetGroup(),
r.GetResource(),
@@ -53,6 +58,19 @@ func NewResourceInfoFromCheck(r *authzv1.CheckRequest) ResourceInfo {
r.GetSubresource(),
relations,
)
// Special case for creating folders and resources in the root folder
if r.GetVerb() == utils.VerbCreate {
if resource.IsFolderResource() && resource.name == "" {
resource.name = accesscontrol.GeneralFolderUID
} else if resource.HasFolderSupport() && resource.folder == "" {
resource.folder = accesscontrol.GeneralFolderUID
}
return resource
}
return resource
}
func NewResourceInfoFromBatchItem(i *authzextv1.BatchCheckItem) ResourceInfo {
@@ -164,3 +182,15 @@ func (r ResourceInfo) IsValidRelation(relation string) bool {
func (r ResourceInfo) HasSubresource() bool {
return r.subresource != ""
}
var resourcesWithFolderSupport = map[string]bool{
dashboardV1.DashboardResourceInfo.GroupResource().Group: true,
}
func (r ResourceInfo) HasFolderSupport() bool {
return resourcesWithFolderSupport[r.group]
}
func (r ResourceInfo) IsFolderResource() bool {
return r.group == folders.FolderResourceInfo.GroupResource().Group
}
@@ -228,6 +228,9 @@ func TranslateToResourceTuple(subject string, action, kind, name string) (*openf
}
if name == "*" {
if m.group != "" && m.resource != "" {
return NewGroupResourceTuple(subject, m.relation, m.group, m.resource, m.subresource), true
}
return NewGroupResourceTuple(subject, m.relation, translation.group, translation.resource, m.subresource), true
}
@@ -0,0 +1,89 @@
package common
import (
"testing"
openfgav1 "github.com/openfga/api/proto/openfga/v1"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/types/known/structpb"
)
type translationTestCase struct {
testName string
subject string
action string
kind string
name string
expected *openfgav1.TupleKey
}
func TestTranslateToResourceTuple(t *testing.T) {
tests := []translationTestCase{
{
testName: "dashboards:read in folders",
subject: "user:1",
action: "dashboards:read",
kind: "folders",
name: "*",
expected: &openfgav1.TupleKey{
User: "user:1",
Relation: "get",
Object: "group_resource:dashboard.grafana.app/dashboards",
},
},
{
testName: "dashboards:read for all dashboards",
subject: "user:1",
action: "dashboards:read",
kind: "dashboards",
name: "*",
expected: &openfgav1.TupleKey{
User: "user:1",
Relation: "get",
Object: "group_resource:dashboard.grafana.app/dashboards",
},
},
{
testName: "dashboards:read for general folder",
subject: "user:1",
action: "dashboards:read",
kind: "folders",
name: "general",
expected: &openfgav1.TupleKey{
User: "user:1",
Relation: "resource_get",
Object: "folder:general",
Condition: &openfgav1.RelationshipCondition{
Name: "subresource_filter",
Context: &structpb.Struct{
Fields: map[string]*structpb.Value{
"subresources": structpb.NewListValue(&structpb.ListValue{
Values: []*structpb.Value{structpb.NewStringValue("dashboard.grafana.app/dashboards")},
}),
},
},
},
},
},
{
testName: "folders:read",
subject: "user:1",
action: "folders:read",
kind: "folders",
name: "*",
expected: &openfgav1.TupleKey{
User: "user:1",
Relation: "get",
Object: "group_resource:folder.grafana.app/folders",
},
},
}
for _, test := range tests {
t.Run(test.testName, func(t *testing.T) {
tuple, ok := TranslateToResourceTuple(test.subject, test.action, test.kind, test.name)
require.True(t, ok)
require.EqualExportedValues(t, test.expected, tuple)
})
}
}
@@ -212,4 +212,16 @@ func testCheck(t *testing.T, server *Server) {
require.NoError(t, err)
assert.True(t, res.GetAllowed(), "user should be able to view dashboards in folder 6")
})
t.Run("user:18 should be able to create folder in root folder", func(t *testing.T) {
res, err := server.Check(newContextWithNamespace(), newReq("user:18", utils.VerbCreate, folderGroup, folderResource, "", "", ""))
require.NoError(t, err)
assert.Equal(t, true, res.GetAllowed())
})
t.Run("user:18 should be able to create dashboard in root folder", func(t *testing.T) {
res, err := server.Check(newContextWithNamespace(), newReq("user:18", utils.VerbCreate, dashboardGroup, dashboardResource, "", "", ""))
require.NoError(t, err)
assert.Equal(t, true, res.GetAllowed())
})
}
@@ -71,6 +71,8 @@ func setup(t *testing.T, srv *Server) *Server {
common.NewTypedResourceTuple("user:15", common.RelationGet, common.TypeUser, userGroup, userResource, statusSubresource, "1"),
common.NewTypedResourceTuple("user:16", common.RelationGet, common.TypeServiceAccount, serviceAccountGroup, serviceAccountResource, statusSubresource, "1"),
common.NewFolderTuple("user:17", common.RelationSetView, "4"),
common.NewFolderTuple("user:18", common.RelationCreate, "general"),
common.NewFolderResourceTuple("user:18", common.RelationCreate, dashboardGroup, dashboardResource, "", "general"),
}
return setupOpenFGADatabase(t, srv, tuples)
+10 -1
View File
@@ -304,8 +304,15 @@ type DeleteDashboardCommand struct {
RemovePermissions bool
}
type ProvisioningConfig struct {
Name string
OrgID int64
Folder string
AllowUIUpdates bool
}
type DeleteOrphanedProvisionedDashboardsCommand struct {
ReaderNames []string
Config []ProvisioningConfig
}
type DashboardProvisioningSearchResults struct {
@@ -405,6 +412,8 @@ type DashboardSearchProjection struct {
FolderTitle string
SortMeta int64
Tags []string
ManagedBy utils.ManagerKind
ManagerId string
Deleted *time.Time
}
@@ -877,24 +877,32 @@ func (dr *DashboardServiceImpl) waitForSearchQuery(ctx context.Context, query *d
}
func (dr *DashboardServiceImpl) DeleteOrphanedProvisionedDashboards(ctx context.Context, cmd *dashboards.DeleteOrphanedProvisionedDashboardsCommand) error {
// cleanup duplicate provisioned dashboards first (this will have the same name and external_id)
// note: only works in modes 1-3
if err := dr.DeleteDuplicateProvisionedDashboards(ctx); err != nil {
dr.log.Error("Failed to delete duplicate provisioned dashboards", "error", err)
}
// check each org for orphaned provisioned dashboards
orgs, err := dr.orgService.Search(ctx, &org.SearchOrgsQuery{})
if err != nil {
return err
}
orgIDs := make([]int64, 0, len(orgs))
for _, org := range orgs {
orgIDs = append(orgIDs, org.ID)
}
if err := dr.DeleteDuplicateProvisionedDashboards(ctx, orgIDs, cmd.Config); err != nil {
dr.log.Error("Failed to delete duplicate provisioned dashboards", "error", err)
}
currentNames := make([]string, 0, len(cmd.Config))
for _, cfg := range cmd.Config {
currentNames = append(currentNames, cfg.Name)
}
for _, org := range orgs {
ctx, _ := identity.WithServiceIdentity(ctx, org.ID)
// find all dashboards in the org that have a file repo set that is not in the given readers list
foundDashs, err := dr.searchProvisionedDashboardsThroughK8s(ctx, &dashboards.FindPersistedDashboardsQuery{
ManagedBy: utils.ManagerKindClassicFP, //nolint:staticcheck
ManagerIdentityNotIn: cmd.ReaderNames,
ManagerIdentityNotIn: currentNames,
OrgId: org.ID,
})
if err != nil {
@@ -921,7 +929,129 @@ func (dr *DashboardServiceImpl) DeleteOrphanedProvisionedDashboards(ctx context.
return nil
}
func (dr *DashboardServiceImpl) DeleteDuplicateProvisionedDashboards(ctx context.Context) error {
// searchExistingProvisionedData fetches provisioned data for the purposes of
// duplication cleanup. Returns the set of folder UIDs for folders with the
// given title, and the set of resources contained in those folders.
func (dr *DashboardServiceImpl) searchExistingProvisionedData(
ctx context.Context, orgID int64, folderTitle string,
) ([]string, []dashboards.DashboardSearchProjection, error) {
ctx, user := identity.WithServiceIdentity(ctx, orgID)
cmd := folder.SearchFoldersQuery{
OrgID: orgID,
SignedInUser: user,
Title: folderTitle,
TitleExactMatch: true,
}
searchResults, err := dr.folderService.SearchFolders(ctx, cmd)
if err != nil {
return nil, nil, fmt.Errorf("checking if provisioning reset is required: %w", err)
}
var matchingFolders []string //nolint:prealloc
for _, result := range searchResults {
f, err := dr.folderService.Get(ctx, &folder.GetFolderQuery{
OrgID: orgID,
UID: &result.UID,
SignedInUser: user,
})
if err != nil {
return nil, nil, err
}
// We are only interested in folders at the top-level of the folder hierarchy.
// Cleanup is not performed for provisioned folders that were moved to
// a different location.
if f.ParentUID != "" {
continue
}
matchingFolders = append(matchingFolders, f.UID)
}
if len(matchingFolders) == 0 {
// If there are no folders with the same title as the provisioned folder we
// are looking for, there is nothing to be cleaned up.
return nil, nil, nil
}
resources, err := dr.FindDashboards(ctx, &dashboards.FindPersistedDashboardsQuery{
OrgId: orgID,
SignedInUser: user,
FolderUIDs: matchingFolders,
})
if err != nil {
return nil, nil, err
}
return matchingFolders, resources, nil
}
// maybeResetProvisioning will check for duplicated provisioned dashboards in the database. These duplications
// happen when multiple provisioned dashboards of the same title are found, or multiple provisioned
// folders are found. In this case, provisioned resources are deleted, allowing the provisioning
// process to start from scratch after this function returns.
func (dr *DashboardServiceImpl) maybeResetProvisioning(ctx context.Context, orgs []int64, configs []dashboards.ProvisioningConfig) {
if skipReason := canBeAutomaticallyCleanedUp(configs); skipReason != "" {
dr.log.Info("not eligible for automated cleanup", "reason", skipReason)
return
}
folderTitle := configs[0].Folder
provisionedNames := map[string]bool{}
for _, c := range configs {
provisionedNames[c.Name] = true
}
for _, orgID := range orgs {
ctx, user := identity.WithServiceIdentity(ctx, orgID)
provFolders, resources, err := dr.searchExistingProvisionedData(ctx, orgID, folderTitle)
if err != nil {
dr.log.Error("failed to search for provisioned data for cleanup", "org", orgID, "error", err)
continue
}
steps, err := cleanupSteps(provFolders, resources, provisionedNames)
if err != nil {
dr.log.Warn("not possible to perform automated duplicate cleanup", "org", orgID, "error", err)
continue
}
for _, step := range steps {
var err error
switch step.Type {
case searchstore.TypeDashboard:
err = dr.deleteDashboard(ctx, 0, step.UID, orgID, false)
case searchstore.TypeFolder:
err = dr.folderService.Delete(ctx, &folder.DeleteFolderCommand{
OrgID: orgID,
SignedInUser: user,
UID: step.UID,
})
}
if err == nil {
dr.log.Info("deleted duplicated provisioned resource",
"type", step.Type, "uid", step.UID,
)
} else {
dr.log.Error("failed to delete duplicated provisioned resource",
"type", step.Type, "uid", step.UID, "error", err,
)
}
}
}
}
func (dr *DashboardServiceImpl) DeleteDuplicateProvisionedDashboards(ctx context.Context, orgs []int64, configs []dashboards.ProvisioningConfig) error {
// Start from scratch if duplications that cannot be fixed by the logic
// below are found in the database.
dr.maybeResetProvisioning(ctx, orgs, configs)
// cleanup duplicate provisioned dashboards (i.e., with the same name and external_id).
// Note: only works in modes 1-3. This logic can be removed once mode5 is
// enabled everywhere.
duplicates, err := dr.dashboardStore.GetDuplicateProvisionedDashboards(ctx)
if err != nil {
return err
@@ -1511,6 +1641,8 @@ func (dr *DashboardServiceImpl) FindDashboards(ctx context.Context, query *dashb
FolderTitle: folderTitle,
FolderID: folderID,
FolderSlug: slugify.Slugify(folderTitle),
ManagedBy: hit.ManagedBy.Kind,
ManagerId: hit.ManagedBy.ID,
Tags: hit.Tags,
}
@@ -779,7 +779,7 @@ func TestDeleteOrphanedProvisionedDashboards(t *testing.T) {
}, nil).Twice()
err := service.DeleteOrphanedProvisionedDashboards(context.Background(), &dashboards.DeleteOrphanedProvisionedDashboardsCommand{
ReaderNames: []string{"test"},
Config: []dashboards.ProvisioningConfig{{Name: "test"}},
})
require.NoError(t, err)
k8sCliMock.AssertExpectations(t)
@@ -874,7 +874,7 @@ func TestDeleteOrphanedProvisionedDashboards(t *testing.T) {
}, nil).Once()
err := singleOrgService.DeleteOrphanedProvisionedDashboards(ctx, &dashboards.DeleteOrphanedProvisionedDashboardsCommand{
ReaderNames: []string{"test"},
Config: []dashboards.ProvisioningConfig{{Name: "test"}},
})
require.NoError(t, err)
k8sCliMock.AssertExpectations(t)
@@ -906,7 +906,7 @@ func TestDeleteOrphanedProvisionedDashboards(t *testing.T) {
}, nil)
err := singleOrgService.DeleteOrphanedProvisionedDashboards(ctx, &dashboards.DeleteOrphanedProvisionedDashboardsCommand{
ReaderNames: []string{"test"},
Config: []dashboards.ProvisioningConfig{{Name: "test"}},
})
require.NoError(t, err)
k8sCliMock.AssertExpectations(t)
@@ -0,0 +1,107 @@
package service
import (
"errors"
"fmt"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/services/sqlstore/searchstore"
)
// canBeAutomaticallyCleanedUp determines whether this instance can be automatically cleaned up
// if duplicated provisioned resources are found. To ensure the process does not delete
// resources it shouldn't, automatic cleanups only happen if all provisioned dashboards
// are stored in the same folder (by title), and no dashboards allow UI updates.
func canBeAutomaticallyCleanedUp(configs []dashboards.ProvisioningConfig) string {
if len(configs) == 0 {
return "no provisioned dashboards"
}
folderTitle := configs[0].Folder
if len(folderTitle) == 0 {
return fmt.Sprintf("dashboard has no folder: %s", configs[0].Name)
}
for _, cfg := range configs {
if cfg.AllowUIUpdates {
return "contains dashboards with allowUiUpdates"
}
if cfg.Folder != folderTitle {
return "dashboards provisioned across multiple folders"
}
}
return ""
}
type deleteProvisionedResource struct {
Type string
UID string
}
// cleanupSteps computes the sequence of steps to be performed in order to cleanup the
// provisioning resources and allow the process to start from scratch when duplication
// is detected. The sequence of steps will dictate the order in which dashboards and folders
// are to be deleted.
func cleanupSteps(provFolders []string, resources []dashboards.DashboardSearchProjection, configDashboards map[string]bool) ([]deleteProvisionedResource, error) {
var hasDuplicatedProvisionedDashboard bool
var hasUserCreatedResource bool
var uniqueNames = map[string]struct{}{}
var deleteProvisionedDashboards []deleteProvisionedResource //nolint:prealloc
for _, r := range resources {
// nolint:staticcheck
if r.IsFolder || r.ManagedBy != utils.ManagerKindClassicFP {
hasUserCreatedResource = true
continue
}
// Only delete dashboards if they are included in the provisioning configuration
// for this instance.
if !configDashboards[r.ManagerId] {
continue
}
if _, exists := uniqueNames[r.ManagerId]; exists {
hasDuplicatedProvisionedDashboard = true
}
uniqueNames[r.ManagerId] = struct{}{}
deleteProvisionedDashboards = append(deleteProvisionedDashboards, deleteProvisionedResource{
Type: searchstore.TypeDashboard,
UID: r.UID,
})
}
if len(provFolders) == 0 {
// When there are no provisioned folders, there is nothing to do.
return nil, nil
} else if len(provFolders) == 1 {
// If only one folder was found, keep it and delete the provisioned dashboards if
// duplication was found.
if hasDuplicatedProvisionedDashboard {
return deleteProvisionedDashboards, nil
}
} else {
// If multiple folders were found *and* a user-created resource exists in
// one of them, bail, as we wouldn't be able to delete one of the duplicated folders.
if hasUserCreatedResource {
return nil, errors.New("multiple provisioning folders exist with at least one user-created resource")
}
// Delete provisioned dashboards first, and then the folders.
steps := deleteProvisionedDashboards
for _, uid := range provFolders {
steps = append(steps, deleteProvisionedResource{
Type: searchstore.TypeFolder,
UID: uid,
})
}
return steps, nil
}
return nil, nil
}
@@ -0,0 +1,279 @@
package service
import (
"testing"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/services/sqlstore/searchstore"
"github.com/stretchr/testify/require"
)
func Test_canBeAutomaticallyCleanedUp(t *testing.T) {
testCases := []struct {
name string
configs []dashboards.ProvisioningConfig
expectedSkip string
}{
{
name: "no dashboards defined in the configuration",
configs: []dashboards.ProvisioningConfig{},
expectedSkip: "no provisioned dashboards",
},
{
name: "first defined dashboard has no folder defined",
configs: []dashboards.ProvisioningConfig{
{Name: "1", Folder: ""},
{Folder: "f1"},
},
expectedSkip: "dashboard has no folder: 1",
},
{
name: "one of the provisioned dashboards has no folder defined",
configs: []dashboards.ProvisioningConfig{
{Name: "1", Folder: "f1"},
{Name: "2", Folder: "f1"},
{Name: "3", Folder: ""},
{Name: "4", Folder: "f1"},
},
expectedSkip: "dashboards provisioned across multiple folders",
},
{
name: "one of the provisioned dashboards allows UI updates",
configs: []dashboards.ProvisioningConfig{
{Name: "1", Folder: "f1"},
{Name: "2", Folder: "f1", AllowUIUpdates: true},
{Name: "3", Folder: "f1"},
{Name: "4", Folder: "f1"},
},
expectedSkip: "contains dashboards with allowUiUpdates",
},
{
name: "one of the provisioned dashboards is in a different folder",
configs: []dashboards.ProvisioningConfig{
{Name: "1", Folder: "f1"},
{Name: "2", Folder: "f1"},
{Name: "3", Folder: "f1"},
{Name: "4", Folder: "different"},
},
expectedSkip: "dashboards provisioned across multiple folders",
},
{
name: "can be skipped when all conditions are met",
configs: []dashboards.ProvisioningConfig{
{Name: "1", Folder: "f1"},
{Name: "2", Folder: "f1"},
{Name: "3", Folder: "f1"},
{Name: "4", Folder: "f1"},
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
require.Equal(t, tc.expectedSkip, canBeAutomaticallyCleanedUp(tc.configs))
})
}
}
func Test_cleanupSteps(t *testing.T) {
isDashboard, isFolder := false, true
fromUser := func(uid, name string, isFolder bool) dashboards.DashboardSearchProjection {
return dashboards.DashboardSearchProjection{
UID: uid,
ManagerId: name,
IsFolder: isFolder,
}
}
provisioned := func(uid, name string, isFolder bool) dashboards.DashboardSearchProjection {
dashboard := fromUser(uid, name, isFolder)
dashboard.ManagedBy = utils.ManagerKindClassicFP //nolint:staticcheck
return dashboard
}
testCases := []struct {
name string
provisionedFolders []string
provisionedResources []dashboards.DashboardSearchProjection
configDashboards []string
expectedSteps []deleteProvisionedResource
expectedErr string
}{
{
name: "no provisioned folders, nothing to do",
provisionedFolders: []string{},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
},
},
{
name: "multiple folders, a user-created dashboard in one of them",
provisionedFolders: []string{"folder1", "folder2"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
fromUser("d3", "User1", isDashboard),
provisioned("d4", "Provisioned3", isDashboard),
},
expectedErr: "multiple provisioning folders exist with at least one user-created resource",
},
{
name: "multiple folders, a user-created folder in one of them",
provisionedFolders: []string{"folder1", "folder2"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned3", isDashboard),
fromUser("f1", "UserFolder1", isFolder),
},
expectedErr: "multiple provisioning folders exist with at least one user-created resource",
},
{
name: "single folder, some dashboards duplicated",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
// Provisioned1 is duplicated.
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned1", isDashboard),
provisioned("d4", "Provisioned3", isDashboard),
},
expectedSteps: []deleteProvisionedResource{
{Type: searchstore.TypeDashboard, UID: "d1"},
{Type: searchstore.TypeDashboard, UID: "d2"},
{Type: searchstore.TypeDashboard, UID: "d3"},
{Type: searchstore.TypeDashboard, UID: "d4"},
},
},
{
name: "single folder, duplicated dashboards, user-created dashboards are ignored",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
// Provisioned1 is duplicated.
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
fromUser("d3", "User1", isDashboard),
provisioned("d4", "Provisioned3", isDashboard),
provisioned("d5", "Provisioned1", isDashboard),
},
// User dashboard (d3) is not deleted.
expectedSteps: []deleteProvisionedResource{
{Type: searchstore.TypeDashboard, UID: "d1"},
{Type: searchstore.TypeDashboard, UID: "d2"},
{Type: searchstore.TypeDashboard, UID: "d4"},
{Type: searchstore.TypeDashboard, UID: "d5"},
},
},
{
name: "single folder, duplicated dashboards, user-created folders are ignored",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3"},
provisionedResources: []dashboards.DashboardSearchProjection{
// Provisioned1 is duplicated.
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned3", isDashboard),
provisioned("d4", "Provisioned1", isDashboard),
fromUser("f1", "UserFolder1", isFolder),
},
// User folder (f1) is not deleted.
expectedSteps: []deleteProvisionedResource{
{Type: searchstore.TypeDashboard, UID: "d1"},
{Type: searchstore.TypeDashboard, UID: "d2"},
{Type: searchstore.TypeDashboard, UID: "d3"},
{Type: searchstore.TypeDashboard, UID: "d4"},
},
},
{
name: "multiple folders, only provisioned dashboards",
provisionedFolders: []string{"folder1", "folder2"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned3", isDashboard),
provisioned("d4", "Provisioned4", isDashboard),
},
// Delete all dashboards, then all folders.
expectedSteps: []deleteProvisionedResource{
{Type: searchstore.TypeDashboard, UID: "d1"},
{Type: searchstore.TypeDashboard, UID: "d2"},
{Type: searchstore.TypeDashboard, UID: "d3"},
{Type: searchstore.TypeDashboard, UID: "d4"},
{Type: searchstore.TypeFolder, UID: "folder1"},
{Type: searchstore.TypeFolder, UID: "folder2"},
},
},
{
name: "single folder, only deletes dashboards defined in the config file",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned1", isDashboard),
provisioned("d4", "Provisioned4", isDashboard),
provisioned("d5", "Provisioned4", isDashboard),
},
// Delete duplicated dashboards, but keep Provisioned4, since it's not in the config file.
expectedSteps: []deleteProvisionedResource{
{Type: searchstore.TypeDashboard, UID: "d1"},
{Type: searchstore.TypeDashboard, UID: "d2"},
{Type: searchstore.TypeDashboard, UID: "d3"},
},
},
{
name: "single folder, no duplicated dashboards",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
provisioned("d3", "Provisioned3", isDashboard),
provisioned("d4", "Provisioned4", isDashboard),
},
expectedSteps: nil, // no duplicates, nothing to do
},
{
name: "single folder, no duplicated dashboards, multiple user-created resources",
provisionedFolders: []string{"folder1"},
configDashboards: []string{"Provisioned1", "Provisioned2", "Provisioned3", "Provisioned4"},
provisionedResources: []dashboards.DashboardSearchProjection{
provisioned("d1", "Provisioned1", isDashboard),
provisioned("d2", "Provisioned2", isDashboard),
fromUser("f1", "UserFolder1", isFolder),
provisioned("d3", "Provisioned3", isDashboard),
fromUser("d4", "User1", isDashboard),
provisioned("d5", "Provisioned4", isDashboard),
fromUser("d6", "User2", isDashboard),
fromUser("f2", "UserFolder2", isFolder),
},
expectedSteps: nil, // no duplicates in the provisioned set, nothing to do
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
provisionedSet := make(map[string]bool)
for _, name := range tc.configDashboards {
provisionedSet[name] = true
}
steps, err := cleanupSteps(tc.provisionedFolders, tc.provisionedResources, provisionedSet)
if tc.expectedErr == "" {
require.NoError(t, err)
require.Equal(t, tc.expectedSteps, steps)
} else {
require.Error(t, err)
require.Equal(t, tc.expectedErr, err.Error())
}
})
}
}
@@ -202,6 +202,11 @@ func (s *Service) searchFoldersFromApiServer(ctx context.Context, query folder.S
if query.Title != "" {
// allow wildcard search
request.Query = "*" + strings.ToLower(query.Title) + "*"
// or perform exact match if requested
if query.TitleExactMatch {
request.Query = query.Title
}
// if using query, you need to specify the fields you want
request.Fields = dashboardsearch.IncludeFields
}
+7 -6
View File
@@ -224,12 +224,13 @@ type GetFoldersQuery struct {
}
type SearchFoldersQuery struct {
OrgID int64
UIDs []string
IDs []int64
Title string
Limit int64
SignedInUser identity.Requester `json:"-"`
OrgID int64
UIDs []string
IDs []int64
Title string
TitleExactMatch bool
Limit int64
SignedInUser identity.Requester `json:"-"`
}
// GetParentsQuery captures the information required by the folder service to
@@ -153,13 +153,20 @@ func (provider *Provisioner) Provision(ctx context.Context) error {
// CleanUpOrphanedDashboards deletes provisioned dashboards missing a linked reader.
func (provider *Provisioner) CleanUpOrphanedDashboards(ctx context.Context) {
currentReaders := make([]string, len(provider.fileReaders))
configs := make([]dashboards.ProvisioningConfig, len(provider.fileReaders))
for index, reader := range provider.fileReaders {
currentReaders[index] = reader.Cfg.Name
configs[index] = dashboards.ProvisioningConfig{
Name: reader.Cfg.Name,
OrgID: reader.Cfg.OrgID,
Folder: reader.Cfg.Folder,
AllowUIUpdates: reader.Cfg.AllowUIUpdates,
}
}
if err := provider.provisioner.DeleteOrphanedProvisionedDashboards(ctx, &dashboards.DeleteOrphanedProvisionedDashboardsCommand{ReaderNames: currentReaders}); err != nil {
if err := provider.provisioner.DeleteOrphanedProvisionedDashboards(
ctx, &dashboards.DeleteOrphanedProvisionedDashboardsCommand{Config: configs},
); err != nil {
provider.log.Warn("Failed to delete orphaned provisioned dashboards", "err", err)
}
}
@@ -2,11 +2,8 @@ package migrations
import (
"fmt"
"strings"
"github.com/bwmarrin/snowflake"
"github.com/grafana/grafana/pkg/services/sqlstore/migrator"
"github.com/grafana/grafana/pkg/util/xorm"
)
func initResourceTables(mg *migrator.Migrator) string {
@@ -207,142 +204,5 @@ func initResourceTables(mg *migrator.Migrator) string {
Name: "IDX_resource_history_key_path",
}))
mg.AddMigration("resource_history key_path backfill", &ResourceHistoryKeyPathBackfillMigration{})
return marker
}
type ResourceHistoryKeyPathBackfillMigration struct {
migrator.MigrationBase
}
func (m *ResourceHistoryKeyPathBackfillMigration) SQL(_ migrator.Dialect) string {
return "resource_history key_path backfill code migration"
}
func (m *ResourceHistoryKeyPathBackfillMigration) Exec(sess *xorm.Session, mg *migrator.Migrator) error {
rows, err := getResourceHistoryRows(sess, mg, resourceHistoryRow{})
if err != nil {
return err
}
for len(rows) > 0 {
if err := updateResourceHistoryKeyPath(sess, rows); err != nil {
return err
}
rows, err = getResourceHistoryRows(sess, mg, rows[len(rows)-1])
if err != nil {
return err
}
}
return nil
}
func updateResourceHistoryKeyPath(sess *xorm.Session, rows []resourceHistoryRow) error {
if len(rows) == 0 {
return nil
}
updates := []resourceHistoryRow{}
for _, row := range rows {
if row.KeyPath == "" {
row.KeyPath = parseKeyPath(row)
updates = append(updates, row)
}
}
if len(updates) == 0 {
return nil
}
guids := ""
setCases := "CASE"
for _, row := range updates {
guids += fmt.Sprintf("'%s',", row.GUID)
setCases += fmt.Sprintf(" WHEN guid = '%s' THEN '%s'", row.GUID, row.KeyPath)
}
guids = strings.TrimRight(guids, ",")
setCases += " ELSE key_path END "
// the query will look like this
// UPDATE resource_history
// SET key_path = CASE
// WHEN guid = '1402de51-669b-4206-8a6c-005a00eee6e3' then 'unified/data/folder.grafana.app/folders/default/cf6lylpvls000c/1998492888241012800~created~'
// WHEN guid = '8842cc56-f22b-45e1-82b1-99759cd443b3' then 'unified/data/dashboard.grafana.app/dashboards/default/adzvfhp/1998492902577144677~created~cf6lylpvls000c'
// ELSE key_path END
// WHERE guid IN ('1402de51-669b-4206-8a6c-005a00eee6e3', '8842cc56-f22b-45e1-82b1-99759cd443b3')
// AND key_path = '';
sql := fmt.Sprintf(`
UPDATE resource_history
SET key_path = %s
WHERE guid IN (%s)
AND key_path = '';
`, setCases, guids)
if _, err := sess.Exec(sql); err != nil {
return err
}
return nil
}
func parseKeyPath(row resourceHistoryRow) string {
var action string
switch row.Action {
case 1:
action = "created"
case 2:
action = "updated"
case 3:
action = "deleted"
}
return fmt.Sprintf("unified/data/%s/%s/%s/%s/%d~%s~%s", row.Group, row.Resource, row.Namespace, row.Name, snowflakeFromRv(row.ResourceVersion), action, row.Folder)
}
func snowflakeFromRv(rv int64) int64 {
return (((rv / 1000) - snowflake.Epoch) << (snowflake.NodeBits + snowflake.StepBits)) + (rv % 1000)
}
type resourceHistoryRow struct {
GUID string `xorm:"guid"`
Group string `xorm:"group"`
Resource string `xorm:"resource"`
Namespace string `xorm:"namespace"`
Name string `xorm:"name"`
ResourceVersion int64 `xorm:"resource_version"`
Action int64 `xorm:"action"`
Folder string `xorm:"folder"`
KeyPath string `xorm:"key_path"`
}
func getResourceHistoryRows(sess *xorm.Session, mg *migrator.Migrator, continueRow resourceHistoryRow) ([]resourceHistoryRow, error) {
var rows []resourceHistoryRow
cols := fmt.Sprintf(
"%s, %s, %s, %s, %s, %s, %s, %s, %s",
mg.Dialect.Quote("guid"),
mg.Dialect.Quote("group"),
mg.Dialect.Quote("resource"),
mg.Dialect.Quote("namespace"),
mg.Dialect.Quote("name"),
mg.Dialect.Quote("resource_version"),
mg.Dialect.Quote("action"),
mg.Dialect.Quote("folder"),
mg.Dialect.Quote("key_path"))
sql := fmt.Sprintf(`
SELECT %s
FROM resource_history
WHERE (resource_version > %d OR (resource_version = %d AND guid > '%s'))
AND key_path = ''
ORDER BY resource_version ASC, guid ASC
LIMIT 1000;
`, cols, continueRow.ResourceVersion, continueRow.ResourceVersion, continueRow.GUID)
if err := sess.SQL(sql).Find(&rows); err != nil {
return nil, err
}
return rows, nil
}
@@ -262,4 +262,57 @@ describe('TabsLayoutManager', () => {
expect(manager.getVizPanels().length).toBe(1);
});
});
describe('createFromLayout', () => {
it('should convert rows with titles to tabs', () => {
const rowsLayout = new RowsLayoutManager({
rows: [new RowItem({ title: 'Row 1' }), new RowItem({ title: 'Row 2' })],
});
const tabsManager = TabsLayoutManager.createFromLayout(rowsLayout);
expect(tabsManager.state.tabs).toHaveLength(2);
expect(tabsManager.state.tabs[0].state.title).toBe('Row 1');
expect(tabsManager.state.tabs[1].state.title).toBe('Row 2');
});
it('should use default title when row has empty title', () => {
const rowsLayout = new RowsLayoutManager({
rows: [new RowItem({ title: '' })],
});
const tabsManager = TabsLayoutManager.createFromLayout(rowsLayout);
expect(tabsManager.state.tabs).toHaveLength(1);
expect(tabsManager.state.tabs[0].state.title).toBe('New tab');
});
it('should generate unique titles for multiple rows with empty titles', () => {
const rowsLayout = new RowsLayoutManager({
rows: [new RowItem({ title: '' }), new RowItem({ title: '' }), new RowItem({ title: '' })],
});
const tabsManager = TabsLayoutManager.createFromLayout(rowsLayout);
expect(tabsManager.state.tabs).toHaveLength(3);
expect(tabsManager.state.tabs[0].state.title).toBe('New tab');
expect(tabsManager.state.tabs[1].state.title).toBe('New tab 1');
expect(tabsManager.state.tabs[2].state.title).toBe('New tab 2');
});
it('should generate unique titles when mixing empty and existing titles', () => {
const rowsLayout = new RowsLayoutManager({
rows: [
new RowItem({ title: 'New row' }), // existing title that matches default
new RowItem({ title: '' }), // empty, should get unique title
],
});
const tabsManager = TabsLayoutManager.createFromLayout(rowsLayout);
expect(tabsManager.state.tabs).toHaveLength(2);
expect(tabsManager.state.tabs[0].state.title).toBe('New row');
expect(tabsManager.state.tabs[1].state.title).toBe('New tab');
});
});
});
@@ -410,6 +410,10 @@ export class TabsLayoutManager extends SceneObjectBase<TabsLayoutManagerState> i
let tabs: TabItem[] = [];
if (layout instanceof RowsLayoutManager) {
const existingNames = new Set(
layout.state.rows.map((row) => row.state.title).filter((title): title is string => !!title)
);
for (const row of layout.state.rows) {
if (row.state.repeatSourceKey) {
continue;
@@ -420,10 +424,14 @@ export class TabsLayoutManager extends SceneObjectBase<TabsLayoutManagerState> i
// We need to clear the target since we don't want to point the original row anymore (if it was set)
conditionalRendering?.setTarget(undefined);
const newTitle =
row.state.title || generateUniqueTitle(t('dashboard.tabs-layout.tab.new', 'New tab'), existingNames);
existingNames.add(newTitle);
tabs.push(
new TabItem({
layout: row.state.layout.clone(),
title: row.state.title,
title: newTitle,
conditionalRendering,
repeatByVariable: row.state.repeatByVariable,
})
@@ -256,7 +256,11 @@ export const InfiniteScroll = ({
if (props.visibleStartIndex === 0) {
noScrollRef.current = scrollElement.scrollHeight <= scrollElement.clientHeight;
}
if (noScrollRef.current || infiniteLoaderState === 'loading' || infiniteLoaderState === 'out-of-bounds') {
if (noScrollRef.current) {
setInfiniteLoaderState('idle');
return;
}
if (infiniteLoaderState === 'loading' || infiniteLoaderState === 'out-of-bounds') {
return;
}
const lastLogIndex = logs.length - 1;
@@ -267,7 +271,7 @@ export const InfiniteScroll = ({
setInfiniteLoaderState('idle');
}
},
[infiniteLoaderState, logs.length, scrollElement]
[infiniteLoaderState, logs, scrollElement]
);
const getItemKey = useCallback((index: number) => (logs[index] ? logs[index].uid : index.toString()), [logs]);
@@ -1,5 +1,5 @@
import { css } from '@emotion/css';
import { useState, useEffect, useCallback, useMemo } from 'react';
import { Fragment, useState, useEffect, useCallback, useMemo } from 'react';
import { useAsync, useMeasure } from 'react-use';
import {
@@ -133,9 +133,9 @@ export function VisualizationSuggestions({ onChange, data, panel }: Props) {
return (
<div className={styles.grid}>
{isNewVizSuggestionsEnabled
? suggestionsByVizType.map(([vizType, vizTypeSuggestions]) => (
<>
<div className={styles.vizTypeHeader} key={vizType?.id || 'unknown-viz-type'}>
? suggestionsByVizType.map(([vizType, vizTypeSuggestions], groupIndex) => (
<Fragment key={vizType?.id || `unknown-viz-type-${groupIndex}`}>
<div className={styles.vizTypeHeader}>
<Text variant="body" weight="medium">
{vizType?.info && <img className={styles.vizTypeLogo} src={vizType.info.logos.small} alt="" />}
{vizType?.name || t('panel.visualization-suggestions.unknown-viz-type', 'Unknown visualization type')}
@@ -190,7 +190,7 @@ export function VisualizationSuggestions({ onChange, data, panel }: Props) {
</div>
);
})}
</>
</Fragment>
))
: suggestions?.map((suggestion, index) => (
<div key={suggestion.hash} className={styles.cardContainer} ref={index === 0 ? firstCardRef : undefined}>