You can include additional connection properties (e.g., `ApplicationIntent`) by separating them with semicolons (`;`). | +| **Database** | Sets the name of the MSSQL database to connect to. | + +**TLS/SSL Auth:** + +Encrypt - Determines whether or to which extent a secure SSL TCP/IP connection will be negotiated with the server. + +| Encrypt Setting | Description | +| --------------- | ------------------------------------------------------------------------------------------------ | +| **Disable** | Data sent between the client and server is **not encrypted**. | +| **False** | The default setting. Only the login packet is encrypted; **all other data is sent unencrypted**. | +| **True** | **All data** sent between the client and server is **encrypted**. | + +{{< admonition type="note" >}} +If you're using an older version of Microsoft SQL Server like 2008 and 2008R2, you may need to disable encryption to be able to connect. +{{< /admonition >}} + +**Authentication:** + +| Authentication Type | Description | Credentials / Fields | +| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| **SQL Server Authentication** | Default method to connect to MSSQL. Use a SQL Server or Windows login in `DOMAIN\User` format. | - **Username**: SQL Server username
- **Password**: SQL Server password | +| **Windows Authentication**
(Integrated Security) | Uses the logged-in Windows user's credentials via single sign-on. Available only when SQL Server allows Windows Authentication. | No input required; uses the logged-in Windows user's credentials | +| **Windows AD**
(Username/Password) | Authenticates a domain user with their Active Directory username and password. | - **Username**: `user@example.com`
- **Password**: Active Directory password | +| **Windows AD**
(Keytab) | Authenticates a domain user using a keytab file. | - **Username**: `user@example.com`
- **Keytab file path**: Path to your keytab file | +| **Windows AD**
(Credential Cache) | Uses a Kerberos credential cache already loaded in memory (e.g., from a prior `kinit` command). No file needed. | - **Credential cache path**: Path to in-memory credential (e.g., `/tmp/krb5cc_1000`) | +| **Windows AD**
(Credential Cache File) | Authenticates a domain user using a credential cache file (`.ccache`). | - **Username**: `user@example.com`
- **Credential cache file path**: e.g., `/home/grot/cache.json` | + +**Additional settings:** + +Additional settings are optional settings you configure for more control over your data source. This includes connection limits, connection timeout, group-by time interval, and Secure Socks Proxy. + +**Connection limits**: + +| Setting | Description | +| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Max open** | The maximum number of open connections to the database. If set to `0`, there is no limit. If `max open` is greater than `0` and less than `max idle`, `max idle` is adjusted to match. | +| **Auto max idle** | When enabled, automatically sets `max idle` to match `max open`. If `max open` isn’t set, it defaults to `100`. | +| **Max idle** | The maximum number of idle connections in the pool. If `max open` is set and is lower than `max idle`, then `max idle` is reduced to match. If set to `0`, no idle connections are retained. | +| **Max lifetime** | The maximum time (in seconds) a connection can be reused before being closed and replaced. If set to `0`, connections are reused indefinitely. | + +**Connection details:** + +| **Setting** | **Description** | +| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Min time interval** | Specifies the lower bound for the auto-generated `GROUP BY` time interval. Grafana recommends matching this value to your data's write frequency—for example, `1m` if data is written every minute. Refer to [Min time interval](#min-time-interval) for details. | +| **Connection timeout** | Specifies the maximum number of seconds to wait when attempting to connect to the database before timing out. A value of `0` (the default) disables the timeout. | + +**Windows ADS Advanced Settings** + +| Setting | Description | Default | +| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | +| **UDP Preference Limit** | Defines the maximum packet size (in bytes) that Kerberos libraries will attempt to send over UDP before retrying with TCP. A value of `1` forces all communication to use TCP. | `1` (always use TCP) | +| **DNS Lookup KDC** | Controls whether DNS `SRV` records are used to locate [Key Distribution Centers (KDCs)](https://web.mit.edu/kerberos/krb5-latest/doc/admin/realm_config.html#key-distribution-centers) and other servers for the realm. | `true` | +| **krb5 config file path** | Specifies the path to the Kerberos configuration file used by the [MIT krb5 package](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html). | `/etc/krb5.conf` | + +**Private data source connect** - _Only for Grafana Cloud users._ + +Private data source connect, or PDC, allows you to establish a private, secured connection between a Grafana Cloud instance, or stack, and data sources secured within a private network. Click the drop-down to locate the URL for PDC. For more information regarding Grafana PDC refer to [Private data source connect (PDC)](ref:private-data-source-connect) and [Configure Grafana private data source connect (PDC)](ref:configure-pdc) for instructions on setting up a PDC connection. + +Click **Manage private data source connect** to open your PDC connection page and view your configuration details. + +After configuring your MSSQL data source options, click **Save & test** at the bottom to test the connection. You should see a confirmation dialog box that says: + +**Database Connection OK** + +### Min time interval + +The **Min time interval** setting defines a lower limit for the [`$__interval`](ref:add-template-variables-interval) and [`$__interval_ms`][add-template-variables-interval_ms] variables. + +This value _must_ be formatted as a number followed by a valid time identifier: + +| Identifier | Description | +| ---------- | ----------- | +| `y` | year | +| `M` | month | +| `w` | week | +| `d` | day | +| `h` | hour | +| `m` | minute | +| `s` | second | +| `ms` | millisecond | + +Grafana recommends setting this value to match your Microsoft SQL Server write frequency. +For example, use `1m` if Microsoft SQL Server writes data every minute. + +You can also override this setting in a dashboard panel under its data source options. + +### Database user permissions + +When adding a data source, ensure the database user you specify has only SELECT permissions on the relevant database and tables. Grafana does not validate the safety of queries, which means they can include potentially harmful SQL statements, such as `USE otherdb`; or `DROP TABLE user;`, which could get executed. To minimize this risk, Grafana strongly recommends creating a dedicated MySQL user with restricted permissions. + +```sql +CREATE USER grafanareader WITH PASSWORD 'password' +GRANT SELECT ON dbo.YourTable3 TO grafanareader +``` + +Also, ensure that the user doesn't have any unwanted privileges from the public role. + +### Diagnose connection issues + +If you use older versions of Microsoft SQL Server, such as 2008 and 2008R2, you might need to disable encryption before you can connect the data source. + +Grafana recommends that you use the latest available service pack for optimal compatibility. + +### Provision the data source + +You can define and configure the data source in YAML files as part of the Grafana provisioning system. For more information about provisioning, and for available configuration options, refer to [Provision Grafana](ref:provision-grafana). + +#### Provisioning example + +```yaml +apiVersion: 1 + +datasources: + - name: MSSQL + type: mssql + url: localhost:1433 + user: grafana + jsonData: + database: grafana + maxOpenConns: 100 + maxIdleConns: 100 + maxIdleConnsAuto: true + connMaxLifetime: 14400 + connectionTimeout: 0 + encrypt: 'false' + secureJsonData: + password: 'Password!' +``` diff --git a/docs/sources/datasources/mssql/query-editor/index.md b/docs/sources/datasources/mssql/query-editor/index.md index d582f7be4b3..6a6c00c526c 100644 --- a/docs/sources/datasources/mssql/query-editor/index.md +++ b/docs/sources/datasources/mssql/query-editor/index.md @@ -39,64 +39,94 @@ refs: destination: /docs/grafana/
For example, _DATEDIFF(second, '1970-01-01', dateColumn) AS time_ | -| `$__timeFilter(dateColumn)` | A time range filter using the specified column name.
For example, _dateColumn BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:06:17Z'_ | -| `$__timeFrom()` | The start of the currently active time selection. For example, _'2017-04-21T05:01:17Z'_ | -| `$__timeTo()` | The end of the currently active time selection. For example, _'2017-04-21T05:06:17Z'_ | -| `$__timeGroup(dateColumn,'5m'[, fillvalue])` | An expression usable in GROUP BY clause. Providing a _fillValue_ of _NULL_ or _floating value_ will automatically fill empty series in timerange with that value.
For example, _CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) as bigint)\*300_. | -| `$__timeGroup(dateColumn,'5m', 0)` | Same as above but with a fill parameter so missing points in that series will be added by grafana and 0 will be used as value. | -| `$__timeGroup(dateColumn,'5m', NULL)` | Same as above but NULL will be used as value for missing points. | -| `$__timeGroup(dateColumn,'5m', previous)` | Same as above but the previous value in that series will be used as fill value if no value has been seen yet NULL will be used. | -| `$__timeGroupAlias(dateColumn,'5m')` | Same as `$__timeGroup` but with an added column alias. | -| `$__unixEpochFilter(dateColumn)` | A time range filter using the specified column name with times represented as Unix timestamp. For example, _dateColumn > 1494410783 AND dateColumn < 1494497183_ | -| `$__unixEpochFrom()` | The start of the currently active time selection as Unix timestamp. For example, _1494410783_ | -| `$__unixEpochTo()` | The end of the currently active time selection as Unix timestamp. For example, _1494497183_ | -| `$__unixEpochNanoFilter(dateColumn)` | A time range filter using the specified column name with times represented as nanosecond timestamp. For example, _dateColumn > 1494410783152415214 AND dateColumn < 1494497183142514872_ | -| `$__unixEpochNanoFrom()` | The start of the currently active time selection as nanosecond timestamp. For example, _1494410783152415214_ | -| `$__unixEpochNanoTo()` | The end of the currently active time selection as nanosecond timestamp. For example, _1494497183142514872_ | -| `$__unixEpochGroup(dateColumn,'5m', [fillmode])` | Same as `$__timeGroup` but for times stored as Unix timestamp. | -| `$__unixEpochGroupAlias(dateColumn,'5m', [fillmode])` | Same as above but also adds a column alias. | +Use macros in the `SELECT` clause to simplify the creation of time series queries. +From the **Data operations** drop-down, choose a macro such as `$\_\_timeGroup` or `$\_\_timeGroupAlias`. Then, select a time column from the **Column** drop-down and a time interval from the **Interval** drop-down. This generates a time-series query based on your selected time grouping. + +| **Macro** | **Description** | +| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `$__time(dateColumn)` | Renames the specified column to `_time`.
Example: `dateColumn AS time` | +| `$__timeEpoch(dateColumn)` | Converts a `DATETIME` column to a Unix timestamp and renames it to `_time`.
Example: `DATEDIFF(second, '1970-01-01', dateColumn) AS time` | +| `$__timeFilter(dateColumn)` | Adds a time range filter for the specified column.
Example: `dateColumn BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:06:17Z'` | +| `$__timeFrom()` | Returns the start of the current time range.
Example: `'2017-04-21T05:01:17Z'` | +| `$__timeTo()` | Returns the end of the current time range.
Example: `'2017-04-21T05:06:17Z'` | +| `$__timeGroup(dateColumn, '5m'[, fillValue])` | Groups the specified time column into intervals (e.g., 5 minutes). Optionally fills gaps with a value like `0`, `NULL`, or `previous`.
Example: `CAST(ROUND(DATEDIFF(second, '1970-01-01', time_column)/300.0, 0) AS bigint) * 300` | +| `$__timeGroup(dateColumn, '5m', 0)` | Same as above, with `0` used to fill missing data points. | +| `$__timeGroup(dateColumn, '5m', NULL)` | Same as above, with `NULL` used for missing data points. | +| `$__timeGroup(dateColumn, '5m', previous)` | Same as above, using the previous value to fill gaps. If no previous value exists, `NULL` is used. | +| `$__timeGroupAlias(dateColumn, '5m')` | Same as `$__timeGroup`, but also adds an alias to the resulting column. | +| `$__unixEpochFilter(dateColumn)` | Adds a time range filter using Unix timestamps.
Example: `dateColumn > 1494410783 AND dateColumn < 1494497183` | +| `$__unixEpochFrom()` | Returns the start of the current time range as a Unix timestamp.
Example: `1494410783` | +| `$__unixEpochTo()` | Returns the end of the current time range as a Unix timestamp.
Example: `1494497183` | +| `$__unixEpochNanoFilter(dateColumn)` | Adds a time range filter using nanosecond-precision Unix timestamps.
Example: `dateColumn > 1494410783152415214 AND dateColumn < 1494497183142514872` | +| `$__unixEpochNanoFrom()` | Returns the start of the current time range as a nanosecond Unix timestamp.
Example: `1494410783152415214` | +| `$__unixEpochNanoTo()` | Returns the end of the current time range as a nanosecond Unix timestamp.
Example: `1494497183142514872` | +| `$__unixEpochGroup(dateColumn, '5m', [fillMode])` | Same as `$__timeGroup`, but for Unix timestamps. Optional `fillMode` controls how to handle missing points. | +| `$__unixEpochGroupAlias(dateColumn, '5m', [fillMode])` | Same as above, but adds an alias to the grouped column. | ### View the interpolated query -The query editor also includes a link named **Generated SQL** that appears after running a query while in panel edit mode. -To display the raw interpolated SQL string that the data source executed, click on this link. +The query editor includes a **Generated SQL** link that appears after you run a query while editing a panel. Click this link to view the raw interpolated SQL that Grafana executed, including any macros that were expanded during query processing. -## Use table queries +## Table queries -If the **Format** query option is set to **Table** for a [Table panel](ref:table), you can enter any type of SQL query. -The Table panel then displays the query results with whatever columns and rows are returned. +To create a Table query, set the **Format** option in the query editor to [**Table**](ref:table). This allows you to write any valid SQL query, and the Table panel will display the results using the returned columns and rows. -**Example database table:** +**Example:** ```sql CREATE TABLE [event] ( @@ -220,43 +203,43 @@ SELECT GETDATE(), CAST(GETDATE() AS DATETIME2), CAST(GETDATE() AS SMALLDATETIME), CAST(GETDATE() AS DATE), CAST(GETDATE() AS TIME), SWITCHOFFSET(CAST(GETDATE() AS DATETIMEOFFSET), '-07:00') ``` -Query editor with example query: - -{{< figure src="/static/img/docs/v51/mssql_table_query.png" max-width="500px" class="docs-image--no-shadow" >}} - -The query: +**Example query with output:** ```sql SELECT * FROM [mssql_types] ``` -To control the name of the Table panel columns, use the standard `AS` SQL column selection syntax. +{{< figure src="/static/img/docs/v51/mssql_table_query.png" max-width="500px" class="docs-image--no-shadow" >}} -For example: +Use the keyword `AS` to define an alias in your query to rename a column or table. + +**Example query with output:** ```sql SELECT - c_bit as [column1], c_tinyint as [column2] + c_bit AS [column1], c_tinyint AS [column2] FROM [mssql_types] ``` -The resulting table panel: - {{< figure src="/static/img/docs/v51/mssql_table_result.png" max-width="1489px" class="docs-image--no-shadow" >}} -## Use time series queries +## Time series queries {{< admonition type="note" >}} Store timestamps in UTC to avoid issues with time shifts in Grafana when using non-UTC timezones. {{< /admonition >}} -If you set the **Format** setting in the query editor to **Time series**, then the query must have a column named `time` that returns either a SQL datetime or any numeric datatype representing Unix epoch in seconds. -Result sets of time series queries must also be sorted by time for panels to properly visualize the result. +To create a time series query, set the **Format** option in the query editor to **Time series**. The query must include a column named `time`, which should contain either a SQL `datetime` value or a numeric value representing Unix epoch time in seconds. The result set must be sorted by the `time` column for panels to visualize the data correctly. -A time series query result is returned in a [wide data frame format](https://grafana.com/developers/plugin-tools/key-concepts/data-frames#wide-format). -Any column except time or of type string transforms into value fields in the data frame query result. -Any string column transforms into field labels in the data frame query result. +A time series query returns results[wide data frame format](https://grafana.com/developers/plugin-tools/key-concepts/data-frames#wide-format). + +- Any column except `time` or of the type `string` transforms into value fields in the data frame query result. +- Any string column transforms into field labels in the data frame query result. + +You can enable macro support in the `SELECT` clause to create time series queries more easily. Use the **Data operations** drop-down to choose a macro such as `$\_\_timeGroup` or `$\_\_timeGroupAlias`, then select a time column from the Column drop-down and a time interval from the Interval drop-down. This generates a time-series query based on your selected time grouping. + +{{< docs/shared source="grafana" lookup="datasources/sql-query-builder-macros.md" version="
- **Linear** - Points are joined by straight lines.
- **Smooth** - Points are joined by curved lines that smooths transitions between points.
- **Step before** - The line is displayed as steps between points. Points are rendered at the end of the step.
- **Step after** - The line is displayed as steps between points. Points are rendered at the beginning of the step.
- **None** - No gradient fill. This is the default setting.
- **Opacity** - An opacity gradient where the opacity of the fill increases as y-axis values increase.
- **Hue** - A subtle gradient that's based on the hue of the series color.
- **Solid**
- **Dash** - Select the length and gap for the line dashes. Default dash spacing is 10, 10.
- **Dots** - Select the gap for the dot spacing. Default dot spacing is 0, 10.
- **Never** - Time series data points with gaps in the data are never connected.
- **Always** - Time series data points with gaps in the data are always connected.
- **Threshold** - Specify a threshold above which gaps in the data are no longer connected. This can be useful when the connected gaps in the data are of a known size or within a known range, and gaps outside this range should no longer be connected.
- **Auto** - Grafana determines a point's visibility based on the density of the data. If the density is low, then points appear.
- **Always** - Show the points regardless of how dense the dataset is.
- **Never** - Don't show points.
- **Linear** - Points are joined by straight lines.
- **Smooth** - Points are joined by curved lines that smooths transitions between points.
- **Step before** - The line is displayed as steps between points. Points are rendered at the end of the step.
- **Step after** - The line is displayed as steps between points. Points are rendered at the beginning of the step.
- **None** - No gradient fill. This is the default setting.
- **Opacity** - An opacity gradient where the opacity of the fill increases as y-axis values increase.
- **Hue** - A subtle gradient that's based on the hue of the series color.
- **Solid**
- **Dash** - Select the length and gap for the line dashes. Default dash spacing is 10, 10.
- **Dots** - Select the gap for the dot spacing. Default dot spacing is 0, 10.
- **Never** - Time series data points with gaps in the data are never connected.
- **Always** - Time series data points with gaps in the data are always connected.
- **Threshold** - Specify a threshold above which gaps in the data are no longer connected. This can be useful when the connected gaps in the data are of a known size or within a known range, and gaps outside this range should no longer be connected.
- **Auto** - Grafana determines a point's visibility based on the density of the data. If the density is low, then points appear.
- **Always** - Show the points regardless of how dense the dataset is.
- **Never** - Don't show points.
- onChangeFullMetaSearch()}
- />
-
- {placeholders.metadataSearchSwitch}
-
- onChangeIncludeNullMetadata()}
- />
-
- {placeholders.includeNullMetadata}
-
- onChangeDisableTextWrap()} />
-
-
-
- Disable text wrap
-
-
-
- onChangeUseBackend()}
- />
-
-
-
-
- );
-}
-
-function getStyles(theme: GrafanaTheme2) {
- return {
- settingsIcon: css({
- color: theme.colors.text.secondary,
- }),
- selectItem: css({
- display: 'flex',
- flexDirection: 'row',
- alignItems: 'center',
- padding: '4px 0',
- }),
- selectItemLabel: css({
- margin: `0 0 0 ${theme.spacing(1)}`,
- alignSelf: 'center',
- color: theme.colors.text.secondary,
- fontSize: '12px',
- }),
- };
-}
diff --git a/packages/grafana-prometheus/src/querybuilder/components/metrics-modal/MetricsModal.test.tsx b/packages/grafana-prometheus/src/querybuilder/components/metrics-modal/MetricsModal.test.tsx
index c85089657cb..f57fbb59bd6 100644
--- a/packages/grafana-prometheus/src/querybuilder/components/metrics-modal/MetricsModal.test.tsx
+++ b/packages/grafana-prometheus/src/querybuilder/components/metrics-modal/MetricsModal.test.tsx
@@ -12,7 +12,7 @@ import { PromOptions } from '../../../types';
import { PromVisualQuery } from '../../types';
import { MetricsModal } from './MetricsModal';
-import { metricsModaltestIds } from './shared/testIds';
+import { metricsModaltestIds } from './testIds';
// don't care about interaction tracking in our unit tests
jest.mock('@grafana/runtime', () => ({
@@ -117,28 +117,6 @@ describe('MetricsModal', () => {
});
});
- it('shows results metrics per page chosen by the user', async () => {
- setup(defaultQuery, listOfMetrics);
- const resultsPerPageInput = screen.getByTestId(metricsModaltestIds.resultsPerPage);
- await userEvent.type(resultsPerPageInput, '12');
- const metricInsideRange = screen.getByText('j');
- expect(metricInsideRange).toBeInTheDocument();
- });
-
- it('paginates lots of metrics and does not run out of memory', async () => {
- const lotsOfMetrics: string[] = [...Array(100000).keys()].map((i) => '' + i);
- setup(defaultQuery, lotsOfMetrics);
- await waitFor(() => {
- // doesn't break on loading
- expect(screen.getByText('0')).toBeInTheDocument();
- });
- const resultsPerPageInput = screen.getByTestId(metricsModaltestIds.resultsPerPage);
- // doesn't break on changing results per page
- await userEvent.type(resultsPerPageInput, '11');
- const metricInsideRange = screen.getByText('9');
- expect(metricInsideRange).toBeInTheDocument();
- });
-
// Fuzzy search
it('searches and filter by metric name with a fuzzy search', async () => {
// search for a_bucket by name
@@ -171,14 +149,6 @@ describe('MetricsModal', () => {
expect(metricABucket).toBeInTheDocument();
});
- const showSettingsButton = screen.getByTestId(metricsModaltestIds.showAdditionalSettings);
- expect(showSettingsButton).toBeInTheDocument();
- await userEvent.click(showSettingsButton);
-
- const metadataSwitch = screen.getByTestId(metricsModaltestIds.searchWithMetadata);
- expect(metadataSwitch).toBeInTheDocument();
- await userEvent.click(metadataSwitch);
-
const searchMetric = screen.getByTestId(metricsModaltestIds.searchMetric);
expect(searchMetric).toBeInTheDocument();
await userEvent.type(searchMetric, 'functions');
@@ -248,6 +218,34 @@ function createDatasource(withLabels?: boolean) {
const languageProvider = new EmptyLanguageProviderMock() as unknown as PrometheusLanguageProviderInterface;
// display different results if their labels are selected in the PromVisualQuery
+ if (withLabels) {
+ languageProvider.queryMetricsMetadata = jest.fn().mockResolvedValue({
+ 'with-labels': {
+ type: 'with-labels-type',
+ help: 'with-labels-help',
+ },
+ });
+ } else {
+ // all metrics - create metadata for all metrics in listOfMetrics
+ const mockMetadata: Record{placeholders.setUseBackend}
-
- dispatch(setSelectedTypes(v))}
- />
-
- )}
-
-
-
-
-
-
-
-
+
+
+
@@ -298,61 +123,23 @@ export const MetricsModal = (props: MetricsModalProps) => {
)}
- {state.metrics && (
-
- {
- const page = val ?? 1;
- dispatch(setPageNum(page));
- }}
+ currentPage={pagination.pageNum > pagination.totalPageNum ? 1 : pagination.pageNum}
+ numberOfPages={pagination.totalPageNum}
+ onNavigate={(val: number) => setPagination({ ...pagination, pageNum: val ?? 1 })}
/>
-
-
- Showing {'{{num}}'} of {'{{count}}'} results
-
-
-
-
frontend
;
+ };
+
+ const { getByTestId } = render(
+ |
|
- {state.hasMetadata && (
- <>
-
- |
-
- |
-
- )}
- + |
+ |
+
+ |
<>
- {metrics.length > 0 &&
- metrics.map((metric: MetricData, idx: number) => {
+ {slicedMetrics.length > 0 &&
+ slicedMetrics.map((metric: MetricData, idx: number) => {
return (
-
|---|---|
|
|
- {state.hasMetadata && metaRows(metric)}
- - - | + {metaRows(metric)}
+ {
+ clickedOutside = true;
+ }}
+ >
+ Click Outside
+
+
+
+ );
+
+ const button = screen.getByText('Click me');
+ await userEvent.click(button);
+ expect(clickedOutside).toBe(true);
+ });
+});
diff --git a/packages/grafana-ui/src/components/ClickOutsideWrapper/ClickOutsideWrapper.tsx b/packages/grafana-ui/src/components/ClickOutsideWrapper/ClickOutsideWrapper.tsx
index 015aada0881..4a3b9e9b88a 100644
--- a/packages/grafana-ui/src/components/ClickOutsideWrapper/ClickOutsideWrapper.tsx
+++ b/packages/grafana-ui/src/components/ClickOutsideWrapper/ClickOutsideWrapper.tsx
@@ -1,62 +1,51 @@
-import { PureComponent, createRef } from 'react';
+import { useCallback, useEffect, useRef } from 'react';
import * as React from 'react';
export interface Props {
- /**
- * Callback to trigger when clicking outside of current element occurs.
- */
+ /** Callback to trigger when clicking outside of current element occurs. */
onClick: () => void;
- /**
- * Runs the 'onClick' function when pressing a key outside of the current element. Defaults to true.
- */
- includeButtonPress: boolean;
+ /** Runs the 'onClick' function when pressing a key outside of the current element. Defaults to true. */
+ includeButtonPress?: boolean;
/** Object to attach the click event listener to. */
- parent: Window | Document;
- /**
- * https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener. Defaults to false.
- */
+ parent?: Window | Document;
+ /** https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener. Defaults to false. */
useCapture?: boolean;
+ children: React.ReactNode;
}
-interface State {
- hasEventListener: boolean;
-}
+export function ClickOutsideWrapper({
+ includeButtonPress = true,
+ parent = window,
+ useCapture = false,
+ onClick,
+ children,
+}: Props) {
+ const wrapperRef = useRef{this.props.children}
;
- }
+ return {children}
;
}
diff --git a/packages/grafana-ui/src/components/Table/TableNG/Cells/PillCell.test.tsx b/packages/grafana-ui/src/components/Table/TableNG/Cells/PillCell.test.tsx
index b7249e88b2b..55b9d3e19ba 100644
--- a/packages/grafana-ui/src/components/Table/TableNG/Cells/PillCell.test.tsx
+++ b/packages/grafana-ui/src/components/Table/TableNG/Cells/PillCell.test.tsx
@@ -1,54 +1,18 @@
import { render, RenderResult } from '@testing-library/react';
-import { DataFrame, Field, FieldType, GrafanaTheme2, MappingType, createTheme } from '@grafana/data';
-import { TableCellDisplayMode, TablePillCellOptions } from '@grafana/schema';
+import { Field, FieldType, MappingType, createTheme } from '@grafana/data';
-import { mockThemeContext } from '../../../../themes/ThemeContext';
-
-import { PillCell, getStyles } from './PillCell';
+import { PillCell } from './PillCell';
describe('PillCell', () => {
- let pillClass: string;
- let restoreThemeContext: () => void;
+ const theme = createTheme();
- beforeEach(() => {
- pillClass = getStyles(createTheme()).pill;
- restoreThemeContext = mockThemeContext(createTheme());
- });
-
- afterEach(() => {
- restoreThemeContext();
- });
-
- const mockCellOptions: TablePillCellOptions = {
- type: TableCellDisplayMode.Pill,
- };
-
- const mockField: Field = {
+ const fieldWithValues = (values: unknown[]): Field => ({
name: 'test',
type: FieldType.string,
- values: [],
+ values: values,
config: {},
- };
-
- const mockFrame: DataFrame = {
- name: 'test',
- fields: [mockField],
- length: 1,
- };
-
- const defaultProps = {
- field: mockField,
- justifyContent: 'flex-start' as const,
- cellOptions: mockCellOptions,
- rowIdx: 0,
- frame: mockFrame,
- height: 30,
- width: 100,
- theme: {} as GrafanaTheme2,
- cellInspect: false,
- showFilters: false,
- };
+ });
const ser = new XMLSerializer();
@@ -60,88 +24,84 @@ describe('PillCell', () => {
// one class for lightTextPill, darkTextPill
describe('Color by hash (classic palette)', () => {
- const props = { ...defaultProps };
-
it('single value', () => {
expectHTML(
- render(
- {items.map(({ label, value, color, colorIndicator, colorPlacement, isActive, lineStyle }, i) => (
+ {items.map(({ label, value, color, colorIndicator, colorPlacement, isActive, lineStyle, isHiddenFromViz }, i) => (
{
marginRight?: string;
isPinned: boolean;
showValueScroll?: boolean;
+ isHiddenFromViz?: boolean;
}
enum LabelValueTypes {
@@ -41,6 +42,7 @@ export const VizTooltipRow = ({
isPinned,
lineStyle,
showValueScroll,
+ isHiddenFromViz,
}: VizTooltipRowProps) => {
const styles = useStyles2(getStyles, justify, marginRight);
@@ -132,7 +134,12 @@ export const VizTooltipRow = ({
{(color || label) && (
References
{' '}
- ({data.length})
+ ', () => {
- const props = {
- compact: false,
- data: warnings,
- highContrast: false,
- isOpen: false,
- label: 'le-label',
- onToggle: jest.fn(),
- };
-
- it('renders without exploding', () => {
- render()
export type KeyValuesTableProps = {
data: TraceKeyValuePair[];
linksGetter?: ((pairs: TraceKeyValuePair[], index: number) => KeyValuesTableLink[]) | TNil;
+ onlyValues?: boolean;
};
export default function KeyValuesTable(props: KeyValuesTableProps) {
- const { data, linksGetter } = props;
+ const { data, linksGetter, onlyValues } = props;
const styles = useStyles2(getStyles);
return (
{data.map((row, i) => {
- const markup = {
- __html: jsonMarkup(parseIfComplexJson(row.value)),
- };
+ let markup = { __html: '' };
+ if (row.type === 'code') {
+ markup = {
+ __html: `
{color && colorPlacement === ColorPlacement.first && (
- / format", ownerReference.APIVersion)
+ }
+
+ ownerReferenceAPIGroup = toNullString(&gv.Group)
+ ownerReferenceAPIVersion = toNullString(&gv.Version)
+ ownerReferenceKind = toNullString(&ownerReference.Kind)
+ ownerReferenceName = toNullString(&ownerReference.Name)
+ }
+
return &secureValueDB{
- GUID: string(sv.UID),
- Name: sv.Name,
- Namespace: sv.Namespace,
- Annotations: annotations,
- Labels: labels,
- Created: meta.GetCreationTimestamp().UnixMilli(),
- CreatedBy: meta.GetCreatedBy(),
- Updated: updatedTimestamp,
- UpdatedBy: meta.GetUpdatedBy(),
+ GUID: string(sv.UID),
+ Name: sv.Name,
+ Namespace: sv.Namespace,
+ Annotations: annotations,
+ Labels: labels,
+ Created: meta.GetCreationTimestamp().UnixMilli(),
+ CreatedBy: meta.GetCreatedBy(),
+ Updated: updatedTimestamp,
+ UpdatedBy: meta.GetUpdatedBy(),
+ OwnerReferenceAPIGroup: ownerReferenceAPIGroup,
+ OwnerReferenceAPIVersion: ownerReferenceAPIVersion,
+ OwnerReferenceKind: ownerReferenceKind,
+ OwnerReferenceName: ownerReferenceName,
Version: sv.Status.Version,
diff --git a/pkg/storage/secret/metadata/secure_value_store.go b/pkg/storage/secret/metadata/secure_value_store.go
index 3e539b83b3e..5782ee9a3e4 100644
--- a/pkg/storage/secret/metadata/secure_value_store.go
+++ b/pkg/storage/secret/metadata/secure_value_store.go
@@ -3,18 +3,21 @@ package metadata
import (
"context"
"fmt"
+ "strconv"
"time"
"github.com/prometheus/client_golang/prometheus"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
+ "github.com/grafana/grafana-app-sdk/logging"
secretv1beta1 "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
"github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
"github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
"github.com/grafana/grafana/pkg/storage/secret/metadata/metrics"
"github.com/grafana/grafana/pkg/storage/unified/sql"
"github.com/grafana/grafana/pkg/storage/unified/sql/sqltemplate"
+ "go.opentelemetry.io/otel/codes"
)
var _ contracts.SecureValueMetadataStorage = (*secureValueMetadataStorage)(nil)
@@ -40,16 +43,39 @@ type secureValueMetadataStorage struct {
tracer trace.Tracer
}
-func (s *secureValueMetadataStorage) Create(ctx context.Context, sv *secretv1beta1.SecureValue, actorUID string) (*secretv1beta1.SecureValue, error) {
+func (s *secureValueMetadataStorage) Create(ctx context.Context, sv *secretv1beta1.SecureValue, actorUID string) (_ *secretv1beta1.SecureValue, svmCreateErr error) {
start := time.Now()
+ name := sv.GetName()
+ namespace := sv.GetNamespace()
ctx, span := s.tracer.Start(ctx, "SecureValueMetadataStorage.Create", trace.WithAttributes(
- attribute.String("name", sv.GetName()),
- attribute.String("namespace", sv.GetNamespace()),
+ attribute.String("name", name),
+ attribute.String("namespace", namespace),
attribute.String("actorUID", actorUID),
))
defer span.End()
- // Set inside of the transaction callback
+ defer func() {
+ args := []any{
+ "name", name,
+ "namespace", namespace,
+ "actorUID", actorUID,
+ }
+
+ success := svmCreateErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueMetadataStorage.Create failed")
+ span.RecordError(svmCreateErr)
+ args = append(args, "error", svmCreateErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueMetadataStorage.Create", args...)
+
+ s.metrics.SecureValueMetadataCreateDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueMetadataCreateCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
+ // Set inside the transaction callback
var row *secureValueDB
err := s.db.Transaction(ctx, func(ctx context.Context) error {
@@ -145,9 +171,6 @@ func (s *secureValueMetadataStorage) Create(ctx context.Context, sv *secretv1bet
return nil, fmt.Errorf("convert to kubernetes object: %w", err)
}
- s.metrics.SecureValueMetadataCreateDuration.Observe(time.Since(start).Seconds())
- s.metrics.SecureValueMetadataCreateCount.Inc()
-
return createdSecureValue, nil
}
@@ -220,7 +243,9 @@ func (s *secureValueMetadataStorage) readActiveVersion(ctx context.Context, name
&secureValue.Annotations, &secureValue.Labels,
&secureValue.Created, &secureValue.CreatedBy,
&secureValue.Updated, &secureValue.UpdatedBy,
- &secureValue.Description, &secureValue.Keeper, &secureValue.Decrypters, &secureValue.Ref, &secureValue.ExternalID, &secureValue.Active, &secureValue.Version); err != nil {
+ &secureValue.Description, &secureValue.Keeper, &secureValue.Decrypters, &secureValue.Ref, &secureValue.ExternalID, &secureValue.Active, &secureValue.Version,
+ &secureValue.OwnerReferenceAPIGroup, &secureValue.OwnerReferenceAPIVersion, &secureValue.OwnerReferenceKind, &secureValue.OwnerReferenceName,
+ ); err != nil {
return secureValueDB{}, fmt.Errorf("failed to scan secure value row: %w", err)
}
@@ -230,7 +255,7 @@ func (s *secureValueMetadataStorage) readActiveVersion(ctx context.Context, name
return secureValue, nil
}
-func (s *secureValueMetadataStorage) Read(ctx context.Context, namespace xkube.Namespace, name string, opts contracts.ReadOpts) (*secretv1beta1.SecureValue, error) {
+func (s *secureValueMetadataStorage) Read(ctx context.Context, namespace xkube.Namespace, name string, opts contracts.ReadOpts) (_ *secretv1beta1.SecureValue, readErr error) {
start := time.Now()
ctx, span := s.tracer.Start(ctx, "SecureValueMetadataStorage.Read", trace.WithAttributes(
attribute.String("name", name),
@@ -239,6 +264,13 @@ func (s *secureValueMetadataStorage) Read(ctx context.Context, namespace xkube.N
))
defer span.End()
+ defer func() {
+ logging.FromContext(ctx).Info("SecureValueMetadataStorage.Read", "namespace", namespace, "name", name, "success", readErr == nil, "error", readErr)
+
+ s.metrics.SecureValueMetadataGetDuration.Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueMetadataGetCount.Inc()
+ }()
+
secureValue, err := s.readActiveVersion(ctx, namespace, name, opts)
if err != nil {
return nil, err
@@ -249,9 +281,6 @@ func (s *secureValueMetadataStorage) Read(ctx context.Context, namespace xkube.N
return nil, fmt.Errorf("convert to kubernetes object: %w", err)
}
- s.metrics.SecureValueMetadataGetDuration.Observe(time.Since(start).Seconds())
- s.metrics.SecureValueMetadataGetCount.Inc()
-
return secureValueKub, nil
}
@@ -293,6 +322,7 @@ func (s *secureValueMetadataStorage) List(ctx context.Context, namespace xkube.N
&row.Updated, &row.UpdatedBy,
&row.Description, &row.Keeper, &row.Decrypters,
&row.Ref, &row.ExternalID, &row.Version, &row.Active,
+ &row.OwnerReferenceAPIGroup, &row.OwnerReferenceAPIVersion, &row.OwnerReferenceKind, &row.OwnerReferenceName,
)
if err != nil {
diff --git a/pkg/storage/secret/metadata/secure_value_test.go b/pkg/storage/secret/metadata/secure_value_test.go
index 9400735cdd1..ce1513d95cf 100644
--- a/pkg/storage/secret/metadata/secure_value_test.go
+++ b/pkg/storage/secret/metadata/secure_value_test.go
@@ -403,7 +403,7 @@ func TestStateMachine(t *testing.T) {
},
"decrypt": func(t *rapid.T) {
input := decryptGen.Draw(t, "decryptInput")
- authCtx := testutils.CreateServiceAuthContext(t.Context(), input.decrypter, []string{fmt.Sprintf("secret.grafana.app/securevalues/%+v:decrypt", input.name)})
+ authCtx := testutils.CreateServiceAuthContext(t.Context(), input.decrypter, input.namespace, []string{fmt.Sprintf("secret.grafana.app/securevalues/%+v:decrypt", input.name)})
modelResult, modelErr := model.decrypt(input.decrypter, input.namespace, input.name)
result, err := sut.DecryptService.Decrypt(authCtx, input.namespace, input.name)
if err != nil || modelErr != nil {
@@ -440,7 +440,7 @@ func TestSecureValueServiceExampleBased(t *testing.T) {
require.NoError(t, err)
require.Equal(t, sv.Status.Version, deletedSv.Status.Version)
- authCtx := testutils.CreateServiceAuthContext(t.Context(), sv.Spec.Decrypters[0], []string{fmt.Sprintf("secret.grafana.app/securevalues/%+v:decrypt", sv.Name)})
+ authCtx := testutils.CreateServiceAuthContext(t.Context(), sv.Spec.Decrypters[0], sv.Namespace, []string{fmt.Sprintf("secret.grafana.app/securevalues/%+v:decrypt", sv.Name)})
result, err := sut.DecryptService.Decrypt(authCtx, sv.Namespace, sv.Name)
require.NoError(t, err)
require.Equal(t, 1, len(result))
diff --git a/pkg/storage/secret/metadata/testdata/mysql--secure_value_create-create-not-null.sql b/pkg/storage/secret/metadata/testdata/mysql--secure_value_create-create-not-null.sql
index aadea1d47a9..b629d32fc52 100755
--- a/pkg/storage/secret/metadata/testdata/mysql--secure_value_create-create-not-null.sql
+++ b/pkg/storage/secret/metadata/testdata/mysql--secure_value_create-create-not-null.sql
@@ -14,6 +14,10 @@ INSERT INTO `secret_secure_value` (
`keeper`,
`decrypters`,
`ref`,
+ `owner_reference_api_group`,
+ `owner_reference_api_version`,
+ `owner_reference_kind`,
+ `owner_reference_name`,
`external_id`
) VALUES (
'abc',
@@ -31,5 +35,9 @@ INSERT INTO `secret_secure_value` (
'keeper_test',
'decrypters_test',
'ref_test',
+ 'prometheus.datasource.grafana.app',
+ 'v0alpha1',
+ 'DataSource',
+ 'prom-config',
'extId'
);
diff --git a/pkg/storage/secret/metadata/testdata/mysql--secure_value_list-list.sql b/pkg/storage/secret/metadata/testdata/mysql--secure_value_list-list.sql
index d73828bfae8..5faf0cca659 100755
--- a/pkg/storage/secret/metadata/testdata/mysql--secure_value_list-list.sql
+++ b/pkg/storage/secret/metadata/testdata/mysql--secure_value_list-list.sql
@@ -14,7 +14,11 @@ SELECT
`ref`,
`external_id`,
`version`,
- `active`
+ `active`,
+ `owner_reference_api_group`,
+ `owner_reference_api_version`,
+ `owner_reference_kind`,
+ `owner_reference_name`
FROM
`secret_secure_value`
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read-for-update.sql b/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read-for-update.sql
index 4d8525dd0a0..f48f1a1d703 100755
--- a/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read-for-update.sql
+++ b/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read-for-update.sql
@@ -14,7 +14,11 @@ SELECT
`ref`,
`external_id`,
`active`,
- `version`
+ `version`,
+ `owner_reference_api_group`,
+ `owner_reference_api_version`,
+ `owner_reference_kind`,
+ `owner_reference_name`
FROM
`secret_secure_value`
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read.sql b/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read.sql
index c42fd0037c5..3dfc6f1e0b9 100755
--- a/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read.sql
+++ b/pkg/storage/secret/metadata/testdata/mysql--secure_value_read-read.sql
@@ -14,7 +14,11 @@ SELECT
`ref`,
`external_id`,
`active`,
- `version`
+ `version`,
+ `owner_reference_api_group`,
+ `owner_reference_api_version`,
+ `owner_reference_kind`,
+ `owner_reference_name`
FROM
`secret_secure_value`
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/postgres--secure_value_create-create-not-null.sql b/pkg/storage/secret/metadata/testdata/postgres--secure_value_create-create-not-null.sql
index cf43e0c130c..bbf94f89fcc 100755
--- a/pkg/storage/secret/metadata/testdata/postgres--secure_value_create-create-not-null.sql
+++ b/pkg/storage/secret/metadata/testdata/postgres--secure_value_create-create-not-null.sql
@@ -14,6 +14,10 @@ INSERT INTO "secret_secure_value" (
"keeper",
"decrypters",
"ref",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name",
"external_id"
) VALUES (
'abc',
@@ -31,5 +35,9 @@ INSERT INTO "secret_secure_value" (
'keeper_test',
'decrypters_test',
'ref_test',
+ 'prometheus.datasource.grafana.app',
+ 'v0alpha1',
+ 'DataSource',
+ 'prom-config',
'extId'
);
diff --git a/pkg/storage/secret/metadata/testdata/postgres--secure_value_list-list.sql b/pkg/storage/secret/metadata/testdata/postgres--secure_value_list-list.sql
index 40ee42ebcf7..0095a993c35 100755
--- a/pkg/storage/secret/metadata/testdata/postgres--secure_value_list-list.sql
+++ b/pkg/storage/secret/metadata/testdata/postgres--secure_value_list-list.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"version",
- "active"
+ "active",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read-for-update.sql b/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read-for-update.sql
index cce38e6db7a..162d97d042c 100755
--- a/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read-for-update.sql
+++ b/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read-for-update.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"active",
- "version"
+ "version",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read.sql b/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read.sql
index 2d16c211a85..86aab54919d 100755
--- a/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read.sql
+++ b/pkg/storage/secret/metadata/testdata/postgres--secure_value_read-read.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"active",
- "version"
+ "version",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_create-create-not-null.sql b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_create-create-not-null.sql
index cf43e0c130c..bbf94f89fcc 100755
--- a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_create-create-not-null.sql
+++ b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_create-create-not-null.sql
@@ -14,6 +14,10 @@ INSERT INTO "secret_secure_value" (
"keeper",
"decrypters",
"ref",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name",
"external_id"
) VALUES (
'abc',
@@ -31,5 +35,9 @@ INSERT INTO "secret_secure_value" (
'keeper_test',
'decrypters_test',
'ref_test',
+ 'prometheus.datasource.grafana.app',
+ 'v0alpha1',
+ 'DataSource',
+ 'prom-config',
'extId'
);
diff --git a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_list-list.sql b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_list-list.sql
index 40ee42ebcf7..0095a993c35 100755
--- a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_list-list.sql
+++ b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_list-list.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"version",
- "active"
+ "active",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read-for-update.sql b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read-for-update.sql
index 2d16c211a85..86aab54919d 100755
--- a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read-for-update.sql
+++ b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read-for-update.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"active",
- "version"
+ "version",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read.sql b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read.sql
index 2d16c211a85..86aab54919d 100755
--- a/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read.sql
+++ b/pkg/storage/secret/metadata/testdata/sqlite--secure_value_read-read.sql
@@ -14,7 +14,11 @@ SELECT
"ref",
"external_id",
"active",
- "version"
+ "version",
+ "owner_reference_api_group",
+ "owner_reference_api_version",
+ "owner_reference_kind",
+ "owner_reference_name"
FROM
"secret_secure_value"
WHERE
diff --git a/pkg/storage/secret/migrator/migrator.go b/pkg/storage/secret/migrator/migrator.go
index 0b7790a875f..a208497c65a 100644
--- a/pkg/storage/secret/migrator/migrator.go
+++ b/pkg/storage/secret/migrator/migrator.go
@@ -152,4 +152,33 @@ func (*SecretDB) AddMigration(mg *migrator.Migrator) {
Cols: []string{"namespace", "label", "active"},
Type: migrator.IndexType,
}))
+
+ // Owner Reference columns
+ mg.AddMigration("add owner_reference_api_group column to "+TableNameSecureValue, migrator.NewAddColumnMigration(secureValueTable, &migrator.Column{
+ Name: "owner_reference_api_group",
+ Type: migrator.DB_NVarchar,
+ Length: 253, // Limit enforced by K8s.
+ Nullable: true,
+ }))
+
+ mg.AddMigration("add owner_reference_api_version column to "+TableNameSecureValue, migrator.NewAddColumnMigration(secureValueTable, &migrator.Column{
+ Name: "owner_reference_api_version",
+ Type: migrator.DB_NVarchar,
+ Length: 253, // Limit enforced by K8s.
+ Nullable: true,
+ }))
+
+ mg.AddMigration("add owner_reference_kind column to "+TableNameSecureValue, migrator.NewAddColumnMigration(secureValueTable, &migrator.Column{
+ Name: "owner_reference_kind",
+ Type: migrator.DB_NVarchar,
+ Length: 253, // Limit enforced by K8s.
+ Nullable: true,
+ }))
+
+ mg.AddMigration("add owner_reference_name column to "+TableNameSecureValue, migrator.NewAddColumnMigration(secureValueTable, &migrator.Column{
+ Name: "owner_reference_name",
+ Type: migrator.DB_NVarchar,
+ Length: 253, // Limit enforced by K8s.
+ Nullable: true,
+ }))
}
diff --git a/pkg/storage/unified/README.md b/pkg/storage/unified/README.md
index 03301cf12bc..dd53fa622b0 100644
--- a/pkg/storage/unified/README.md
+++ b/pkg/storage/unified/README.md
@@ -777,3 +777,512 @@ The dual writer system provides metrics for monitoring:
- `dual_writer_mode_transitions_total`: Counter of mode transitions
Use these metrics to monitor the health of your migration and identify any issues with the dual writer system.
+
+---
+
+## Unified Search System
+
+The Unified Search system provides a scalable, distributed search capability for Grafana's Unified Storage. It uses a ring-based architecture to distribute search requests across multiple search server instances, with namespace-based sharding for optimal performance and data distribution.
+
+### System Architecture
+
+The search system provides both unified and legacy search capabilities, with routing based on dual writer mode configuration:
+
+```mermaid
+graph TB
+ subgraph "Request Sources"
+ A[Grafana UI
User Search] + B[Dashboard Service
Resource Searches] + C[Folder Service
Resource Searches] + D[Alerting Service
Resource Searches] + E[Provisioning Service
Resource Searches] + F[API Endpoints
Search Operations] + end + + subgraph "API Gateway Layer" + G[Grafana API Server
Search Endpoint] + H[Search Client
Dual Writer Aware] + end + + subgraph "Routing Decision" + I{Dual Writer Mode
Check} + end + + subgraph "Unified Search Path (Mode 3+)" + J[Search Distributor
Ring-based Routing] + K[Ring
Consistent Hashing] + L[Search API Server 1
Namespace Sharding
+ Embedded Bleve Backend] + M[Search API Server 2
Namespace Sharding
+ Embedded Bleve Backend] + N[Search API Server 3
Namespace Sharding
+ Embedded Bleve Backend] + O[Unified Storage
K8s-style Resources] + end + + subgraph "Legacy Search Path (Mode 0-2)" + Q[Legacy Search Service
SQL-based Search] + R[Legacy Storage
Traditional Tables] + S[Shadow Traffic
Mode 1-2 + Flag Enabled] + end + + A --> G + B --> H + C --> H + D --> H + E --> H + F --> G + G --> H + H --> I + + I -->|Mode 3+| J + I -->|Mode 0-2| Q + + J --> K + K --> L + K --> M + K --> N + L -.->|Just-in-Time
Indexing| O + M -.->|Just-in-Time
Indexing| O + N -.->|Just-in-Time
Indexing| O + + Q --> R + Q -.->|Shadow Traffic
Mode 1-2 + Flag| S + S -.-> J +``` + +### Search Backend Routing + +The search client routes requests based on the dual writer mode configuration for each resource type: + +#### Dual Writer Mode → Backend Routing +- **Mode 0-2**: Route to **Legacy Search** + - Mode 1-2: Shadow traffic to Unified Search (if `unifiedStorageSearchDualReaderEnabled` is enabled) + - Mode 0: No shadow traffic +- **Mode 3+**: Route to **Unified Search** + - No shadow traffic needed (unified is primary) + +### Feature Flags + +Unified Search requires several feature flags to be enabled depending on the desired functionality: + +#### Prerequisites (Required for Unified Storage) + +| Feature Flag | Purpose | Stage | Required For | +|--------------|---------|-------|--------------| +| `grafanaAPIServerWithExperimentalAPIs` | Allow experimental API groups | Development | Access to v0alpha1 APIs (including search) | + +#### Unified Search Specific Flags + +| Feature Flag | Purpose | Stage | Required For | +|--------------|---------|-------|--------------| +| `unifiedStorageSearch` | Core search functionality | Experimental | Search API servers, indexing | +| `unifiedStorageSearchUI` | Frontend search interface | Experimental | Grafana UI search | +| `unifiedStorageSearchPermissionFiltering` | User permission filtering | GA | Access control in search results | +| `unifiedStorageSearchSprinkles` | Usage insights integration | Experimental | Dashboard usage sorting (Enterprise) | +| `unifiedStorageSearchDualReaderEnabled` | Shadow traffic to unified search | Experimental | Shadow traffic during migration | + +#### Basic Configuration +```ini +[feature_toggles] +; Prerequisites for unified storage (required) +grafanaAPIServerWithExperimentalAPIs = true + +; Core search functionality (required) +unifiedStorageSearch = true + +; Enable search UI (required for frontend) +unifiedStorageSearchUI = true + +; Enable permission filtering (recommended) +unifiedStorageSearchPermissionFiltering = true + +; Enable shadow traffic during migration (optional) +unifiedStorageSearchDualReaderEnabled = true + +; Enable usage insights sorting (Enterprise only) +unifiedStorageSearchSprinkles = true +``` + +### Request Flow Diagrams + +#### Search Request Flow with Dual Writer Mode Routing + +Search requests originate from multiple sources, and the search client routes based on dual writer mode configuration: + +```mermaid +flowchart TD + A[Search Request] --> B{Dual Writer Mode} + B -->|Mode 3+| C[Unified Search] + B -->|Mode 0-2| D[Legacy Search] + C --> E[Return Results] + D --> E +``` + +#### Search Request Flow with Shadow Traffic + +When `unifiedStorageSearchDualReaderEnabled` is enabled and resource is in dual writer Mode 1-2 (legacy primary), shadow traffic is generated: + +```mermaid +flowchart TD + A[Search Request] --> B{Shadow Traffic Enabled?} + B -->|Yes| C[Primary: Legacy Search] + B -->|No| D[Single Search Path] + C --> E[Background: Unified Search] + C --> F[Return Legacy Results] + E --> G[Log Results for Comparison] + D --> H[Return Results] +``` + +### Distributor Architecture + +The Search Distributor acts as a smart proxy that routes search requests to the appropriate search API server based on namespace hashing: + +```mermaid +flowchart TD + A[Incoming Search Request] --> B[Hash Namespace] + B --> C[Select Random Instance] + C --> D[Proxy Request] + D --> E[Return Response] +``` + +#### Key Features: +- **Namespace-based routing**: Each request is routed based on the target namespace +- **Load balancing**: Random selection among available replicas for the namespace +- **Health awareness**: Only routes to `ACTIVE` ring instances +- **Connection pooling**: Reuses gRPC connections for efficiency +- **Proxy headers**: Adds metadata for debugging and tracing + +### Ring Architecture + +The hash ring provides consistent, distributed assignment of namespaces to search API servers: + +```mermaid +flowchart TD + A[Namespace] --> B[Hash Function] + B --> C[Ring Position] + C --> D[Assigned Instance] + D --> E[Search Processing] +``` + +#### Ring Properties: +- **Consistent hashing**: Uses FNV32 hash function for namespace distribution +- **128 tokens per instance**: Provides good distribution across the ring +- **Replication factor**: Configurable redundancy (default based on cluster size) +- **State management**: Instances transition through JOINING → ACTIVE → LEAVING +- **Automatic rebalancing**: Ring adjusts when instances join/leave + +### Namespace-Based Sharding + +Unified Search uses namespace-based sharding to distribute search indexes across multiple search API servers: + +```mermaid +flowchart LR + A[Namespaces] --> B[Hash Ring] + B --> C[Search Server 1] + B --> D[Search Server 2] + B --> E[Search Server 3] + C --> F[Indexes for Assigned Namespaces] + D --> G[Indexes for Assigned Namespaces] + E --> H[Indexes for Assigned Namespaces] +``` + +#### Sharding Benefits: +1. **Horizontal scalability**: Add more search servers to handle more namespaces +2. **Resource isolation**: Each namespace's index is independent +3. **Parallel processing**: Multiple searches can run simultaneously across different servers +4. **Fault tolerance**: Namespace availability depends only on its assigned server(s) + +#### Sharding Algorithm: +```go +func getSearchServer(namespace string) string { + hash := fnv.New32a() + hash.Write([]byte(namespace)) + + // Get replication set from ring + replicationSet := ring.GetWithOptions( + hash.Sum32(), + searchRingRead, + ring.WithReplicationFactor(ring.ReplicationFactor()) + ) + + // Random load balancing within replication set + instance := replicationSet.Instances[rand.Intn(len(replicationSet.Instances))] + return instance.Id +} +``` + +### Search Index Management + +Each search API server contains an embedded Bleve search engine that manages indexes for its assigned namespaces: + +```mermaid +flowchart TD + A[Search Request] --> B{Index Ready?} + B -->|Yes| C[Query Index] + B -->|No| D[Build Index] + D --> E[Fetch Resources] + E --> F[Create Index] + F --> C + C --> G[Return Results] + + H[Resource Changes] --> I[Update Index] + I --> F +``` + +#### Index Architecture Details: + +**Embedded Bleve Backend**: Each Search API Server contains its own Bleve search engine instance, not a shared external service. + +**Just-in-Time Indexing**: When a search request arrives for a namespace that doesn't have an index (or has an outdated index): +1. The Search API Server fetches all resources for that namespace from Unified Storage +2. Builds search documents in memory +3. Creates either a memory-based or disk-based Bleve index depending on size +4. Executes the search query against the newly built index +5. Returns results to the user + +**Index Storage Strategy**: +- **Memory indexes**: For small datasets (< `index_file_threshold` documents) +- **Disk indexes**: For large datasets (≥ `index_file_threshold` documents) +- Indexes are stored per Search API Server instance, not globally shared + +**Background Updates**: In addition to just-in-time indexing, Search API Servers also maintain indexes through background watch events for incremental updates. + +#### Index Configuration: +```ini +[unified_storage] +; Path for disk-based search indexes +index_path = /var/lib/grafana/unified-search/bleve + +; Threshold for file-based vs memory indexes +index_file_threshold = 1000 + +; Maximum batch size for indexing +index_max_batch_size = 100 + +; Number of worker threads for indexing +index_workers = 4 + +; Cache TTL for indexes +index_cache_ttl = 1h + +; Periodic rebuild interval (for usage insights) +index_rebuild_interval = 24h + +; Minimum resource count required to build an index (default: 1) +; If a namespace has fewer resources than this threshold, no index will be created +index_min_count = 1 + +; Maximum resource count before creating an empty index (default: 0 = no limit) +; When exceeded, creates an empty index instead of indexing all resources for performance +index_max_count = 0 +``` + +### Search Request Sources + +Unified Search serves multiple types of consumers within the Grafana ecosystem: + +#### 1. User-Initiated Searches +- **Source**: Grafana UI search interface +- **Purpose**: Interactive dashboard and folder discovery +- **Characteristics**: Real-time, user-facing, latency-sensitive +- **Endpoint**: `/api/v1/search` (legacy search UI) or `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search` (when `unifiedStorageSearchUI` is enabled) + +#### 2. Internal Service Searches + +Internal services use different search backends depending on dual writer mode configuration: + +- **Dashboard Service**: + - Find related dashboards based on tags, folders, or content + - Discover dashboards for playlist creation + - Validate dashboard references during operations + - **Backend**: Depends on dashboard dual writer mode (Legacy for Mode 0-2, Unified for Mode 3+) + +- **Folder Service**: + - Retrieve folder contents and nested structures + - Resolve folder hierarchy relationships + - Check folder permissions and accessibility + - **Backend**: Depends on folder dual writer mode (Legacy for Mode 0-2, Unified for Mode 3+) + +- **Alerting Service**: + - Discover dashboards and panels for alert rule creation + - Find existing alert rules across namespaces + - Resolve dashboard/panel references in alert definitions + - **Backend**: Mixed - dashboard searches use dashboard dual writer mode, alert rule searches typically use legacy + +- **Provisioning Service**: + - Check for existing resources before provisioning + - Validate resource uniqueness and naming conflicts + - Discover resources for bulk operations + - **Backend**: Depends on each resource type's dual writer mode configuration + +- **API Services**: + - Backend support for various API endpoints + - Resource validation and dependency checking + - **Backend**: Routes based on resource type's dual writer mode + +#### 3. Search Operation Types + +Unified Search supports multiple types of search operations: + +##### Resource Search +- **Purpose**: Find resources (dashboards, folders, etc.) by content +- **Endpoint**: `/api/v1/search` (legacy) or `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search` (when `unifiedStorageSearchUI` is enabled) +- **Additional endpoint**: `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search/sortable` for retrieving sortable fields +- **Features**: Full-text search, filtering, sorting + +**Sortable Fields:** + +The `/search/sortable` endpoint currently returns a limited static list: +```json +{ + "fields": [ + {"field": "title", "display": "Title (A-Z)", "type": "string"}, + {"field": "-title", "display": "Title (Z-A)", "type": "string"} + ] +} +``` + +However, the search backend actually supports sorting by many more fields: + +**Standard Fields:** +- `title` - Resource display name (uses `title_phrase` for exact sorting) +- `name` - Kubernetes resource name +- `description` - Resource description +- `folder` - Parent folder name +- `created` - Creation timestamp (int64) +- `updated` - Last update timestamp (int64) +- `createdBy` - Creator user ID +- `updatedBy` - Last updater user ID +- `tags` - Resource tags (array) +- `rv` - Resource version (int64) + +**Dashboard-Specific Fields** (require `fields.` prefix): +- `fields.schema_version` - Dashboard schema version +- `fields.link_count` - Number of dashboard links +- `fields.panel_types` - Panel types used in dashboard +- `fields.ds_types` - Data source types used +- `fields.transformation` - Transformations used + +**Usage Insights Fields** (Enterprise only, require `fields.` prefix): +- `fields.views_total` - Total dashboard views +- `fields.views_last_1_days` / `fields.views_last_7_days` / `fields.views_last_30_days` - Recent views +- `fields.views_today` - Today's views +- `fields.queries_total` - Total queries executed +- `fields.queries_last_1_days` / `fields.queries_last_7_days` / `fields.queries_last_30_days` - Recent queries +- `fields.queries_today` - Today's queries +- `fields.errors_total` - Total errors +- `fields.errors_last_1_days` / `fields.errors_last_7_days` / `fields.errors_last_30_days` - Recent errors +- `fields.errors_today` - Today's errors + +**Usage Examples:** +```bash +# Sort by title (ascending) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=title + +# Sort by creation date (descending) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=-created + +# Sort by usage insights (Enterprise) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=-fields.views_total +``` + +*Note: There's currently a discrepancy between the limited fields exposed by `/search/sortable` and the full range of fields actually supported by the search backend.* + +##### Federated Search +- **Purpose**: Search across multiple resource types simultaneously +- **Features**: Cross-resource queries, unified result ranking, combined sorting and faceting +- **Implementation**: Uses Bleve IndexAlias to combine multiple indexes for unified searching +- **Default behavior**: When no type is specified, automatically federates dashboards and folders + +**How Federated Search Works:** + +Federated search is implemented using Bleve's IndexAlias feature, which allows searching across multiple indexes as if they were a single unified index. This enables: + +1. **Cross-resource queries**: Search for content across dashboards, folders, and other resource types +2. **Unified sorting**: Results from different resource types are merged and sorted together +3. **Combined faceting**: Aggregate facet statistics across all federated resource types +4. **Permission filtering**: Respects user permissions for each resource type independently + +**API Usage Examples:** + +**1. Default Federation (Dashboards + Folders):** +```bash +# When no type is specified, automatically searches dashboards and folders +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?query=my-search +``` + +**2. Single Resource Type Search:** +```bash +# Search only folders (despite the "dashboard" API group, type parameter controls what's searched) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=folders&query=my-search + +# Search only dashboards +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=dashboards&query=my-search +``` + +**3. Explicit Two-Type Federation:** +```bash +# Search dashboards (primary) with folders federated +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=dashboards&type=folders&query=my-search +``` + +**4. Protocol Buffer Request Structure:** +```protobuf +message ResourceSearchRequest { + ListOptions options = 1; // Primary resource type to search + repeated ResourceKey federated = 2; // Additional resource types to federate + string query = 3; // Search query applied across all types + // ... other fields +} +``` + +**Example gRPC/Protocol Buffer Usage:** +```go +searchRequest := &resourcepb.ResourceSearchRequest{ + Options: &resourcepb.ListOptions{ + Key: dashboardKey, // Primary: search dashboards + }, + Federated: []*resourcepb.ResourceKey{ + folderKey, // Also search folders + }, + Query: "monitoring", + Limit: 50, + SortBy: []*resourcepb.ResourceSearchRequest_Sort{ + {Field: "title", Desc: false}, // Sort combined results by title + }, +} +``` + +**5. Unified Results:** +Federated search returns a single result set containing resources from all specified types, with: +- **Unified ranking**: All results scored and ranked together +- **Cross-type sorting**: Resources from different types sorted by common fields (title, tags, etc.) +- **Resource type identification**: Each result includes metadata indicating its resource type +- **Permission-aware filtering**: Only returns resources the user has permission to see + +**Limitations:** +- Federation only works across resource types with **common fields** (title, tags, folder, etc.) +- All federated indexes must be of the same search backend type (currently Bleve) +- Currently supports up to 2 resource types in federation via the API endpoint +- **Architectural note**: The search endpoint is under `dashboard.grafana.app` but can search any resource type via the `type` parameter - this is a design choice where the "dashboard search" has evolved into a generic search endpoint + +##### Managed Objects +- **Purpose**: Administrative queries for resource management +- **Operations**: Count, list, statistics + +##### Stats and Monitoring +- **Purpose**: Index health and performance metrics +- **Metrics**: Document counts, index sizes, search latency + + +### Monitoring and Observability + +Key metrics for monitoring Unified Search: + +- `unified_search_requests_total`: Search request counts by type and status +- `unified_search_request_duration_seconds`: Search request latency +- `unified_search_index_size_bytes`: Size of search indexes +- `unified_search_documents_total`: Number of indexed documents +- `unified_search_indexing_duration_seconds`: Time to build/update indexes +- `unified_search_shadow_requests_total`: Shadow traffic request counts +- `unified_search_ring_members`: Number of active search server instances + + diff --git a/pkg/storage/unified/resource/bleve_index_metrics.go b/pkg/storage/unified/resource/bleve_index_metrics.go index 73a18d4fabe..3f130eb0b1b 100644 --- a/pkg/storage/unified/resource/bleve_index_metrics.go +++ b/pkg/storage/unified/resource/bleve_index_metrics.go @@ -9,11 +9,14 @@ import ( ) type BleveIndexMetrics struct { - IndexLatency *prometheus.HistogramVec - IndexSize prometheus.Gauge - IndexedKinds *prometheus.GaugeVec - IndexCreationTime *prometheus.HistogramVec - OpenIndexes *prometheus.GaugeVec + IndexLatency *prometheus.HistogramVec + IndexSize prometheus.Gauge + IndexedKinds *prometheus.GaugeVec + IndexCreationTime *prometheus.HistogramVec + OpenIndexes *prometheus.GaugeVec + IndexBuilds *prometheus.CounterVec + IndexBuildFailures prometheus.Counter + IndexBuildSkipped prometheus.Counter } var IndexCreationBuckets = []float64{1, 5, 10, 25, 50, 75, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000} @@ -37,8 +40,8 @@ func ProvideIndexMetrics(reg prometheus.Registerer) *BleveIndexMetrics { Help: "Number of indexed documents by kind", }, []string{"kind"}), IndexCreationTime: promauto.With(reg).NewHistogramVec(prometheus.HistogramOpts{ - Name: "index_server_index_creation_time_seconds", - Help: "Time (in seconds) it takes until index is created", + Name: "index_server_index_build_time_seconds", + Help: "Time it takes to successfully build an index. Failed or skipped builds are not counted.", Buckets: IndexCreationBuckets, NativeHistogramBucketFactor: 1.1, // enable native histograms NativeHistogramMaxBucketNumber: 160, @@ -48,11 +51,22 @@ func ProvideIndexMetrics(reg prometheus.Registerer) *BleveIndexMetrics { Name: "index_server_open_indexes", Help: "Number of open indexes per storage type. An open index corresponds to single resource group.", }, []string{"index_storage"}), // index_storage is either "file" or "memory" + IndexBuilds: promauto.With(reg).NewCounterVec(prometheus.CounterOpts{ + Name: "index_server_index_build_total", + Help: "Number of times index build was attempted due to specific reason", + }, []string{"reason"}), + IndexBuildFailures: promauto.With(reg).NewCounter(prometheus.CounterOpts{ + Name: "index_server_index_build_failures_total", + Help: "Number of times index build failed", + }), + IndexBuildSkipped: promauto.With(reg).NewCounter(prometheus.CounterOpts{ + Name: "index_server_index_build_skipped_total", + Help: "Number of times index build has been skipped due to existing valid index being found on disk", + }), } // Initialize labels. m.OpenIndexes.WithLabelValues("file").Set(0) m.OpenIndexes.WithLabelValues("memory").Set(0) - return m } diff --git a/pkg/storage/unified/resource/bulk.go b/pkg/storage/unified/resource/bulk.go index f1dbb3131c2..c3f2e61e290 100644 --- a/pkg/storage/unified/resource/bulk.go +++ b/pkg/storage/unified/resource/bulk.go @@ -244,7 +244,7 @@ func (s *server) BulkProcess(stream resourcepb.BulkStore_BulkProcessServer) erro Namespace: summary.Namespace, Group: summary.Group, Resource: summary.Resource, - }, summary.Count, summary.ResourceVersion) + }, summary.Count, summary.ResourceVersion, "rebuildAfterBatchLoad") if err != nil { s.log.Warn("error building search index after batch load", "err", err) rsp.Error = &resourcepb.ErrorResult{ diff --git a/pkg/storage/unified/resource/search.go b/pkg/storage/unified/resource/search.go index bda857993ba..4752fa35cec 100644 --- a/pkg/storage/unified/resource/search.go +++ b/pkg/storage/unified/resource/search.go @@ -90,7 +90,7 @@ type SearchBackend interface { // Depending on the size, the backend may choose different options (eg: memory vs disk). // The last known resource version can be used to detect that nothing has changed, and existing on-disk index can be reused. // The builder will write all documents before returning. - BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, nonStandardFields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) + BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, nonStandardFields SearchableDocumentFields, indexBuildReason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) // TotalDocs returns the total number of documents across all indexes. TotalDocs() int64 @@ -196,7 +196,7 @@ func (s *searchSupport) ListManagedObjects(ctx context.Context, req *resourcepb. Namespace: req.Namespace, Group: info.Group, Resource: info.Resource, - }) + }, "listManagedObjects") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -237,7 +237,7 @@ func (s *searchSupport) CountManagedObjects(ctx context.Context, req *resourcepb Namespace: req.Namespace, Group: info.Group, Resource: info.Resource, - }) + }, "countManagedObjects") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -282,7 +282,7 @@ func (s *searchSupport) Search(ctx context.Context, req *resourcepb.ResourceSear Namespace: req.Options.Key.Namespace, Resource: req.Options.Key.Resource, } - idx, err := s.getOrCreateIndex(ctx, nsr) + idx, err := s.getOrCreateIndex(ctx, nsr, "search") if err != nil { return &resourcepb.ResourceSearchResponse{ Error: AsErrorResult(err), @@ -294,7 +294,7 @@ func (s *searchSupport) Search(ctx context.Context, req *resourcepb.ResourceSear for i, f := range req.Federated { nsr.Group = f.Group nsr.Resource = f.Resource - federate[i], err = s.getOrCreateIndex(ctx, nsr) + federate[i], err = s.getOrCreateIndex(ctx, nsr, "federatedSearch") if err != nil { return &resourcepb.ResourceSearchResponse{ Error: AsErrorResult(err), @@ -323,7 +323,7 @@ func (s *searchSupport) GetStats(ctx context.Context, req *resourcepb.ResourceSt Namespace: req.Namespace, Group: parts[0], Resource: parts[1], - }) + }, "getStats") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -367,7 +367,7 @@ func (s *searchSupport) GetStats(ctx context.Context, req *resourcepb.ResourceSt Namespace: req.Namespace, Group: stat.Group, Resource: stat.Resource, - }) + }, "getStats") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -449,8 +449,12 @@ func (s *searchSupport) buildIndexes(ctx context.Context, rebuild bool) (int, er return err } - s.log.Debug("building index", "namespace", info.Namespace, "group", info.Group, "resource", info.Resource) - _, _, err := s.build(ctx, info.NamespacedResource, info.Count, info.ResourceVersion) + s.log.Debug("building index", "namespace", info.Namespace, "group", info.Group, "resource", info.Resource, "rebuild", rebuild) + reason := "init" + if rebuild { + reason = "rebuild" + } + _, _, err := s.build(ctx, info.NamespacedResource, info.Count, info.ResourceVersion, reason) return err }) } @@ -504,9 +508,6 @@ func (s *searchSupport) init(ctx context.Context) error { end := time.Now().Unix() s.log.Info("search index initialized", "duration_secs", end-start, "total_docs", s.search.TotalDocs()) - if s.indexMetrics != nil { - s.indexMetrics.IndexCreationTime.WithLabelValues().Observe(float64(end - start)) - } return nil } @@ -537,7 +538,7 @@ func (s *searchSupport) dispatchEvent(ctx context.Context, evt *WrittenEvent) { Group: evt.Key.Group, Resource: evt.Key.Resource, } - index, err := s.getOrCreateIndex(ctx, nsr) + index, err := s.getOrCreateIndex(ctx, nsr, "dispatchEvent") if err != nil { s.log.Warn("error getting index for watch event", "error", err) span.RecordError(err) @@ -622,15 +623,10 @@ func (s *searchSupport) rebuildDashboardIndexes(ctx context.Context) error { "duration", duration, "rebuilt_indexes", totalBatchesIndexed, "total_docs", s.search.TotalDocs()) - - if s.indexMetrics != nil { - s.indexMetrics.IndexCreationTime.WithLabelValues().Observe(duration.Seconds()) - } - return nil } -func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedResource) (ResourceIndex, error) { +func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedResource, reason string) (ResourceIndex, error) { if s == nil || s.search == nil { return nil, fmt.Errorf("search is not configured properly (missing unifiedStorageSearch feature toggle?)") } @@ -648,6 +644,10 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } ch := s.buildIndex.DoChan(key.String(), func() (interface{}, error) { + // We want to finish building of the index even if original context is canceled. + // We reuse original context without cancel to keep the tracing spans correct. + ctx := context.WithoutCancel(ctx) + // Recheck if some other goroutine managed to build an index in the meantime. // (That is, it finished running this function and stored the index into the cache) idx, err := s.search.GetIndex(ctx, key) @@ -672,7 +672,7 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } } - idx, _, err = s.build(ctx, key, size, rv) + idx, _, err = s.build(ctx, key, size, rv, reason) if err != nil { return nil, fmt.Errorf("error building search index, %w", err) } @@ -693,10 +693,18 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } } -func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size int64, rv int64) (ResourceIndex, int64, error) { +func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size int64, rv int64, indexBuildReason string) (ResourceIndex, int64, error) { ctx, span := s.tracer.Start(ctx, tracingPrexfixSearch+"Build") defer span.End() + span.SetAttributes( + attribute.String("namespace", nsr.Namespace), + attribute.String("group", nsr.Group), + attribute.String("resource", nsr.Resource), + attribute.Int64("size", size), + attribute.Int64("rv", rv), + ) + logger := s.log.With("namespace", nsr.Namespace, "group", nsr.Group, "resource", nsr.Resource) builder, err := s.builders.get(ctx, nsr) @@ -705,7 +713,10 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size } fields := s.builders.GetFields(nsr) - index, err := s.search.BuildIndex(ctx, nsr, size, rv, fields, func(index ResourceIndex) (int64, error) { + index, err := s.search.BuildIndex(ctx, nsr, size, rv, fields, indexBuildReason, func(index ResourceIndex) (int64, error) { + span := trace.SpanFromContext(ctx) + span.AddEvent("building index", trace.WithAttributes(attribute.Int64("size", size), attribute.Int64("rv", rv), attribute.String("reason", indexBuildReason))) + rv, err = s.storage.ListIterator(ctx, &resourcepb.ListRequest{ Limit: 1000000000000, // big number Options: &resourcepb.ListOptions{ @@ -734,9 +745,11 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size Name: iter.Name(), } + span.AddEvent("building document", trace.WithAttributes(attribute.String("name", iter.Name()))) // Convert it to an indexable document doc, err := builder.BuildDocument(ctx, key, iter.ResourceVersion(), iter.Value()) if err != nil { + span.RecordError(err) logger.Error("error building search document", "key", SearchID(key), "err", err) continue } @@ -749,6 +762,7 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size // When we reach the batch size, perform bulk index and reset the batch. if len(items) >= maxBatchSize { + span.AddEvent("bulk indexing", trace.WithAttributes(attribute.Int("count", len(items)))) if err = index.BulkIndex(&BulkIndexRequest{ Items: items, }); err != nil { @@ -762,6 +776,7 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size // Index any remaining items in the final batch. if len(items) > 0 { + span.AddEvent("bulk indexing", trace.WithAttributes(attribute.Int("count", len(items)))) if err = index.BulkIndex(&BulkIndexRequest{ Items: items, }); err != nil { @@ -799,7 +814,7 @@ func (s *searchSupport) buildEmptyIndex(ctx context.Context, nsr NamespacedResou s.log.Debug("Building empty index", "namespace", nsr.Namespace, "group", nsr.Group, "resource", nsr.Resource, "rv", rv) // Build an empty index by passing a builder function that doesn't add any documents - return s.search.BuildIndex(ctx, nsr, 0, rv, fields, func(index ResourceIndex) (int64, error) { + return s.search.BuildIndex(ctx, nsr, 0, rv, fields, "empty", func(index ResourceIndex) (int64, error) { // Return the resource version without adding any documents to the index return rv, nil }) diff --git a/pkg/storage/unified/resource/search_test.go b/pkg/storage/unified/resource/search_test.go index 07a9a464ae1..677e24b8b8b 100644 --- a/pkg/storage/unified/resource/search_test.go +++ b/pkg/storage/unified/resource/search_test.go @@ -121,7 +121,7 @@ func (m *mockSearchBackend) GetIndex(ctx context.Context, key NamespacedResource return nil, nil } -func (m *mockSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { +func (m *mockSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, reason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { index := &MockResourceIndex{} index.On("BulkIndex", mock.Anything).Return(nil).Maybe() index.On("DocCount", mock.Anything, mock.Anything).Return(int64(0), nil).Maybe() @@ -317,7 +317,7 @@ func TestSearchGetOrCreateIndex(t *testing.T) { go func() { defer wg.Done() <-start - _, _ = support.getOrCreateIndex(context.Background(), NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}) + _, _ = support.getOrCreateIndex(context.Background(), NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}, "test") }() } @@ -340,7 +340,7 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { {NamespacedResource: NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}, Count: 50, ResourceVersion: 11111111}, }, } - search := &slowSearchBackend{ + search := &slowSearchBackendWithCache{ mockSearchBackend: mockSearchBackend{}, } supplier := &TestDocumentBuilderSupplier{ @@ -362,10 +362,12 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { require.NoError(t, err) require.NotNil(t, support) + key := NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"} + ctx, cancel := context.WithTimeout(context.Background(), 1*time.Millisecond) defer cancel() - _, err = support.getOrCreateIndex(ctx, NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}) + _, err = support.getOrCreateIndex(ctx, key, "test") // Make sure we get context deadline error require.ErrorIs(t, err, context.DeadlineExceeded) @@ -373,16 +375,53 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { search.wg.Wait() require.NotEmpty(t, search.buildIndexCalls) + + // Wait until new index is put into cache. + require.Eventually(t, func() bool { + idx, err := support.search.GetIndex(ctx, key) + return err == nil && idx != nil + }, 1*time.Second, 100*time.Millisecond, "Indexing finishes despite context cancellation") + + // Second call to getOrCreateIndex returns index immediately, even if context is canceled, as the index is now ready and cached. + _, err = support.getOrCreateIndex(ctx, key, "test") + require.NoError(t, err) } -type slowSearchBackend struct { +type slowSearchBackendWithCache struct { mockSearchBackend wg sync.WaitGroup + + mu sync.Mutex + cache map[NamespacedResource]ResourceIndex } -func (m *slowSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { +func (m *slowSearchBackendWithCache) GetIndex(ctx context.Context, key NamespacedResource) (ResourceIndex, error) { + m.mu.Lock() + defer m.mu.Unlock() + return m.cache[key], nil +} + +func (m *slowSearchBackendWithCache) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, reason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { m.wg.Add(1) defer m.wg.Done() + time.Sleep(1 * time.Second) - return m.mockSearchBackend.BuildIndex(ctx, key, size, resourceVersion, fields, builder) + + // Simulate erroring out when context is cancelled. + if ctx.Err() != nil { + return nil, ctx.Err() + } + idx, err := m.mockSearchBackend.BuildIndex(ctx, key, size, resourceVersion, fields, reason, builder) + if err != nil { + return nil, err + } + + m.mu.Lock() + defer m.mu.Unlock() + + if m.cache == nil { + m.cache = make(map[NamespacedResource]ResourceIndex) + } + m.cache[key] = idx + return idx, nil } diff --git a/pkg/storage/unified/search/bleve.go b/pkg/storage/unified/search/bleve.go index 141e02b4030..2626ab36f1d 100644 --- a/pkg/storage/unified/search/bleve.go +++ b/pkg/storage/unified/search/bleve.go @@ -23,6 +23,7 @@ import ( "github.com/blevesearch/bleve/v2/search/query" bleveSearch "github.com/blevesearch/bleve/v2/search/searcher" index "github.com/blevesearch/bleve_index_api" + "go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/trace" "k8s.io/apimachinery/pkg/selection" @@ -197,11 +198,21 @@ func (b *bleveBackend) BuildIndex( size int64, resourceVersion int64, fields resource.SearchableDocumentFields, + indexBuildReason string, builder func(index resource.ResourceIndex) (int64, error), ) (resource.ResourceIndex, error) { _, span := b.tracer.Start(ctx, tracingPrexfixBleve+"BuildIndex") defer span.End() + span.SetAttributes( + attribute.String("namespace", key.Namespace), + attribute.String("group", key.Group), + attribute.String("resource", key.Resource), + attribute.Int64("size", size), + attribute.Int64("rv", resourceVersion), + attribute.String("reason", indexBuildReason), + ) + mapper, err := GetBleveMappings(fields) if err != nil { return nil, err @@ -214,7 +225,7 @@ func (b *bleveBackend) BuildIndex( return nil, err } - logWithDetails := b.log.With("namespace", key.Namespace, "group", key.Group, "resource", key.Resource, "size", size, "rv", resourceVersion) + logWithDetails := b.log.With("namespace", key.Namespace, "group", key.Group, "resource", key.Resource, "size", size, "rv", resourceVersion, "reason", indexBuildReason) // Close the newly created/opened index by default. closeIndex := true @@ -306,14 +317,29 @@ func (b *bleveBackend) BuildIndex( } if build { + if b.indexMetrics != nil { + b.indexMetrics.IndexBuilds.WithLabelValues(indexBuildReason).Inc() + } + start := time.Now() _, err = builder(idx) if err != nil { logWithDetails.Error("Failed to build index", "err", err) + if b.indexMetrics != nil { + b.indexMetrics.IndexBuildFailures.Inc() + } return nil, fmt.Errorf("failed to build index: %w", err) } elapsed := time.Since(start) logWithDetails.Info("Finished building index", "elapsed", elapsed) + if b.indexMetrics != nil { + b.indexMetrics.IndexCreationTime.WithLabelValues().Observe(elapsed.Seconds()) + } + } else { + logWithDetails.Info("Skipping index build, using existing index") + if b.indexMetrics != nil { + b.indexMetrics.IndexBuildSkipped.Inc() + } } // Set expiration after building the index. Only expire in-memory indexes. diff --git a/pkg/storage/unified/search/bleve_search_test.go b/pkg/storage/unified/search/bleve_search_test.go index a795b4c0f8d..0680026e2af 100644 --- a/pkg/storage/unified/search/bleve_search_test.go +++ b/pkg/storage/unified/search/bleve_search_test.go @@ -553,7 +553,7 @@ func newTestDashboardsIndex(t TB, threshold int64, size int64, batchSize int64, Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, size, rv, info.Fields, writer) + }, size, rv, info.Fields, "test", writer) require.NoError(t, err) return index, tmpdir diff --git a/pkg/storage/unified/search/bleve_test.go b/pkg/storage/unified/search/bleve_test.go index 5a4c16be2b3..821ce6de0c0 100644 --- a/pkg/storage/unified/search/bleve_test.go +++ b/pkg/storage/unified/search/bleve_test.go @@ -71,7 +71,7 @@ func TestBleveBackend(t *testing.T) { Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, 2, rv, info.Fields, func(index resource.ResourceIndex) (int64, error) { + }, 2, rv, info.Fields, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -352,7 +352,7 @@ func TestBleveBackend(t *testing.T) { Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, 2, rv, fields, func(index resource.ResourceIndex) (int64, error) { + }, 2, rv, fields, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -766,7 +766,7 @@ func TestBleveInMemoryIndexExpiration(t *testing.T) { Resource: "resource", } - builtIndex, err := backend.BuildIndex(context.Background(), ns, 1 /* below FileThreshold */, 100, nil, indexTestDocs(ns, 1)) + builtIndex, err := backend.BuildIndex(context.Background(), ns, 1 /* below FileThreshold */, 100, nil, "test", indexTestDocs(ns, 1)) require.NoError(t, err) // Wait for index expiration, which is 1ns @@ -798,7 +798,7 @@ func TestBleveFileIndexExpiration(t *testing.T) { } // size=100 is above FileThreshold, this will be file-based index - builtIndex, err := backend.BuildIndex(context.Background(), ns, 100, 100, nil, indexTestDocs(ns, 1)) + builtIndex, err := backend.BuildIndex(context.Background(), ns, 100, 100, nil, "test", indexTestDocs(ns, 1)) require.NoError(t, err) // Wait for index expiration, which is 1ns @@ -830,7 +830,7 @@ func TestFileIndexIsReusedOnSameSizeAndRV(t *testing.T) { tmpDir := t.TempDir() backend1, reg1 := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) // Verify one open index. @@ -853,7 +853,7 @@ func TestFileIndexIsReusedOnSameSizeAndRV(t *testing.T) { // We open new backend using same directory, and run indexing with same size (10) and RV (100). This should reuse existing index, and skip indexing. backend2, reg2 := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, indexTestDocs(ns, 1000)) + idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, "test", indexTestDocs(ns, 1000)) require.NoError(t, err) // Verify that we're reusing existing index and there is only 10 documents in it, not 1000. @@ -879,13 +879,13 @@ func TestFileIndexIsNotReusedOnDifferentSize(t *testing.T) { tmpDir := t.TempDir() backend1, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) backend1.closeAllIndexes() // We open new backend using same directory, but with different size. Index should be rebuilt. backend2, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 100, 100, nil, indexTestDocs(ns, 100)) + idx, err := backend2.BuildIndex(context.Background(), ns, 100, 100, nil, "test", indexTestDocs(ns, 100)) require.NoError(t, err) // Verify that index has updated number of documents. @@ -904,13 +904,13 @@ func TestFileIndexIsNotReusedOnDifferentRV(t *testing.T) { tmpDir := t.TempDir() backend1, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) backend1.closeAllIndexes() // We open new backend using same directory, but with different RV. Index should be rebuilt. backend2, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 999999, nil, indexTestDocs(ns, 100)) + idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 999999, nil, "test", indexTestDocs(ns, 100)) require.NoError(t, err) // Verify that index has updated number of documents. @@ -942,7 +942,7 @@ func TestRebuildingIndexClosesPreviousCachedIndex(t *testing.T) { if testCase.firstInMemory { firstSize = 1 } - firstIndex, err := backend.BuildIndex(context.Background(), ns, int64(firstSize), 100, nil, indexTestDocs(ns, firstSize)) + firstIndex, err := backend.BuildIndex(context.Background(), ns, int64(firstSize), 100, nil, "test", indexTestDocs(ns, firstSize)) require.NoError(t, err) openInMemoryIndexes := 0 @@ -952,7 +952,7 @@ func TestRebuildingIndexClosesPreviousCachedIndex(t *testing.T) { secondSize = 1 openInMemoryIndexes = 1 } - secondIndex, err := backend.BuildIndex(context.Background(), ns, int64(secondSize), 100, nil, indexTestDocs(ns, secondSize)) + secondIndex, err := backend.BuildIndex(context.Background(), ns, int64(secondSize), 100, nil, "test", indexTestDocs(ns, secondSize)) require.NoError(t, err) // Verify that first and second index are different, and first one is now closed. @@ -1050,12 +1050,12 @@ func testBleveIndexWithFailures(t *testing.T, fileBased bool) { // size=100 is above FileThreshold (5), make it a file-based index. size = 100 } - _, err := backend.BuildIndex(context.Background(), ns, size, 100, nil, func(index resource.ResourceIndex) (int64, error) { + _, err := backend.BuildIndex(context.Background(), ns, size, 100, nil, "test", func(index resource.ResourceIndex) (int64, error) { return 0, fmt.Errorf("fail") }) require.Error(t, err) // Even though previous build of the index failed, new building of the index should work. - _, err = backend.BuildIndex(context.Background(), ns, size, 100, nil, indexTestDocs(ns, int(size))) + _, err = backend.BuildIndex(context.Background(), ns, size, 100, nil, "test", indexTestDocs(ns, int(size))) require.NoError(t, err) } diff --git a/pkg/storage/unified/sql/backend.go b/pkg/storage/unified/sql/backend.go index a23cf9469ea..7733de8ff91 100644 --- a/pkg/storage/unified/sql/backend.go +++ b/pkg/storage/unified/sql/backend.go @@ -10,9 +10,9 @@ import ( "time" "github.com/go-sql-driver/mysql" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/jackc/pgx/v5/pgconn" "github.com/lib/pq" - "github.com/mattn/go-sqlite3" "github.com/prometheus/client_golang/prometheus" "go.opentelemetry.io/otel/trace" "go.opentelemetry.io/otel/trace/noop" @@ -389,9 +389,8 @@ func (b *backend) create(ctx context.Context, event resource.WriteEvent) (int64, // IsRowAlreadyExistsError checks if the error is the result of the row inserted already existing. func IsRowAlreadyExistsError(err error) bool { - var sqlite sqlite3.Error - if errors.As(err, &sqlite) { - return sqlite.ExtendedCode == sqlite3.ErrConstraintUnique + if sqlite.IsUniqueConstraintViolation(err) { + return true } var pg *pgconn.PgError diff --git a/pkg/storage/unified/sql/backend_test.go b/pkg/storage/unified/sql/backend_test.go index 92989e8454a..534b393065e 100644 --- a/pkg/storage/unified/sql/backend_test.go +++ b/pkg/storage/unified/sql/backend_test.go @@ -8,7 +8,6 @@ import ( "testing" "github.com/DATA-DOG/go-sqlmock" - "github.com/mattn/go-sqlite3" "github.com/stretchr/testify/require" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" @@ -18,6 +17,7 @@ import ( "github.com/grafana/grafana/pkg/storage/unified/resourcepb" "github.com/grafana/grafana/pkg/storage/unified/sql/db/dbimpl" "github.com/grafana/grafana/pkg/storage/unified/sql/test" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/grafana/grafana/pkg/util/testutil" ) @@ -242,7 +242,7 @@ func TestBackend_create(t *testing.T) { ) b.SQLMock.ExpectCommit() b.SQLMock.ExpectBegin() - b.SQLMock.ExpectExec("insert resource").WillReturnError(sqlite3.Error{Code: sqlite3.ErrConstraint, ExtendedCode: sqlite3.ErrConstraintUnique}) + b.SQLMock.ExpectExec("insert resource").WillReturnError(sqlite.TestErrUniqueConstraintViolation) b.SQLMock.ExpectRollback() // First we insert the resource successfully. This is what the happy path test does as well. diff --git a/pkg/storage/unified/testing/benchmark.go b/pkg/storage/unified/testing/benchmark.go index 8503b1dd2bd..dcdf113c212 100644 --- a/pkg/storage/unified/testing/benchmark.go +++ b/pkg/storage/unified/testing/benchmark.go @@ -215,7 +215,7 @@ func runSearchBackendBenchmarkWriteThroughput(ctx context.Context, backend resou // Build initial index size := int64(10000) // force the index to be on disk - index, err := backend.BuildIndex(ctx, nr, size, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, nr, size, 0, nil, "benchmark", func(index resource.ResourceIndex) (int64, error) { return 0, nil }) if err != nil { diff --git a/pkg/storage/unified/testing/search_backend.go b/pkg/storage/unified/testing/search_backend.go index 57d887682c6..52af572b01a 100644 --- a/pkg/storage/unified/testing/search_backend.go +++ b/pkg/storage/unified/testing/search_backend.go @@ -64,7 +64,7 @@ func runTestSearchBackendBuildIndex(t *testing.T, backend resource.SearchBackend require.Nil(t, index) // Build the index - index, err = backend.BuildIndex(ctx, ns, 0, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err = backend.BuildIndex(ctx, ns, 0, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { // Write a test document err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ @@ -111,7 +111,7 @@ func runTestResourceIndex(t *testing.T, backend resource.SearchBackend, nsPrefix } // Build initial index with some test documents - index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -235,7 +235,7 @@ func runTestResourceIndex(t *testing.T, backend resource.SearchBackend, nsPrefix t.Run("Search by LibraryPanel reference", func(t *testing.T) { // Build index with dashboards that have LibraryPanel references - index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { diff --git a/pkg/tests/api/shorturl/short_url_test.go b/pkg/tests/api/shorturl/short_url_test.go index e598c36ea7a..a938b861281 100644 --- a/pkg/tests/api/shorturl/short_url_test.go +++ b/pkg/tests/api/shorturl/short_url_test.go @@ -7,8 +7,10 @@ import ( "fmt" "io" "net/http" + "net/url" "testing" + "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "github.com/grafana/grafana/pkg/infra/db" @@ -31,10 +33,12 @@ func TestMain(m *testing.M) { func TestShortURL(t *testing.T) { dir, path := testinfra.CreateGrafDir(t, testinfra.GrafanaOpts{ AppModeProduction: true, + DisableAnonymous: true, }) grafanaListedAddr, env := testinfra.StartGrafanaEnv(t, dir, path) + // Test that the endpoint is accessible with authentication. username, password := "viewer", "viewer" createUser(t, env.SQLStore, env.Cfg, user.CreateUserCommand{ DefaultOrgRole: string(org.RoleEditor), @@ -50,7 +54,7 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, http.StatusOK, res.StatusCode) + assert.Equal(t, http.StatusOK, res.StatusCode) bodyRaw, err := io.ReadAll(res.Body) require.NoError(t, err) @@ -67,8 +71,8 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, "http://localhost:3000/explore", res.Header.Get("Location")) - require.Equal(t, http.StatusFound, res.StatusCode) + assert.Equal(t, "http://localhost:3000/explore", res.Header.Get("Location")) + assert.Equal(t, http.StatusFound, res.StatusCode) // If the go-to does not exist, it should redirect to the home page and return 308. res, err = c.get("/goto/DoesNotExist") @@ -76,8 +80,28 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, "http://localhost:3000/", res.Header.Get("Location")) - require.Equal(t, http.StatusPermanentRedirect, res.StatusCode) + assert.Equal(t, "http://localhost:3000/", res.Header.Get("Location")) + assert.Equal(t, http.StatusPermanentRedirect, res.StatusCode) + + // Create a client that does not have authentication. + notLoggedInClient := client(grafanaListedAddr, "", "") + // Test that the short-urls endpoint is not accessible without authentication. + res, err = notLoggedInClient.post("/api/short-urls", bytes.NewReader([]byte(`{"path":"explore"}`))) + require.NoError(t, err) + assert.Equal(t, http.StatusUnauthorized, res.StatusCode) + defer func() { + _ = res.Body.Close() + }() + + // If the user is not logged in, it should redirect to the login page and return 302. + res, err = notLoggedInClient.get(fmt.Sprintf("/goto/%s", resParsed.UID)) + require.NoError(t, err) + defer func() { + _ = res.Body.Close() + }() + expectedRedirect := "/login?redirectTo=" + url.QueryEscape("/goto/"+resParsed.UID) + assert.Equal(t, expectedRedirect, res.Header.Get("Location")) + assert.Equal(t, http.StatusFound, res.StatusCode) } func createUser(t *testing.T, db db.DB, cfg *setting.Cfg, cmd user.CreateUserCommand) int64 { diff --git a/pkg/tests/apis/provisioning/helper_test.go b/pkg/tests/apis/provisioning/helper_test.go index 2d5585b819f..03ca9342f31 100644 --- a/pkg/tests/apis/provisioning/helper_test.go +++ b/pkg/tests/apis/provisioning/helper_test.go @@ -112,11 +112,11 @@ func (h *provisioningTestHelper) AwaitJobSuccess(t *testing.T, ctx context.Conte require.NoError(t, err) require.NotNil(t, result) + errors := mustNestedStringSlice(result.Object, "status", "errors") + require.Empty(t, errors, "historic job '%s' has errors: %v", job.GetName(), errors) state := mustNestedString(result.Object, "status", "state") require.Equal(t, string(provisioning.JobStateSuccess), state, "historic job '%s' was not successful", job.GetName()) - errors := mustNestedStringSlice(result.Object, "status", "errors") - require.Empty(t, errors, "historic job '%s' has errors: %v", job.GetName(), errors) }, time.Second*10, time.Millisecond*25) { // We also want to add the job details to the error when it fails. job, err := h.Jobs.Resource.Get(ctx, job.GetName(), metav1.GetOptions{}) @@ -163,6 +163,50 @@ func (h *provisioningTestHelper) AwaitJobs(t *testing.T, repoName string) { } } +// AwaitJobsWithStates waits for all jobs for a repository to complete and accepts multiple valid end states +func (h *provisioningTestHelper) AwaitJobsWithStates(t *testing.T, repoName string, acceptedStates []string) { + t.Helper() + + // First, we wait for all jobs for the repository to disappear (i.e. complete/fail). + require.EventuallyWithT(t, func(collect *assert.CollectT) { + list, err := h.Jobs.Resource.List(context.Background(), metav1.ListOptions{}) + if assert.NoError(collect, err, "failed to list active jobs") { + for _, elem := range list.Items { + repo, _, err := unstructured.NestedString(elem.Object, "spec", "repository") + require.NoError(t, err) + if repo == repoName { + collect.Errorf("there are still remaining jobs for %s: %+v", repoName, elem) + return + } + } + } + }, time.Second*10, time.Millisecond*25, "job queue must be empty") + + // Then, as all jobs are now historic jobs, we make sure they are in an accepted state. + result, err := h.Repositories.Resource.Get(context.Background(), repoName, metav1.GetOptions{}, "jobs") + require.NoError(t, err, "failed to list historic jobs") + + list, err := result.ToList() + require.NoError(t, err, "results should be a list") + require.NotEmpty(t, list.Items, "expect at least one job") + + for _, elem := range list.Items { + require.Equal(t, repoName, elem.GetLabels()[jobs.LabelRepository], "should have repo label") + + state := mustNestedString(elem.Object, "status", "state") + + // Check if state is in accepted states + found := false + for _, acceptedState := range acceptedStates { + if state == acceptedState { + found = true + break + } + } + require.True(t, found, "job %s completed with unexpected state %s (expected one of %v): %+v", elem.GetName(), state, acceptedStates, elem.Object) + } +} + // RenderObject reads the filePath and renders it as a template with the given values. // The template is expected to be a YAML or JSON file. // diff --git a/pkg/tests/apis/provisioning/provisioning_test.go b/pkg/tests/apis/provisioning/provisioning_test.go index cdf210e2133..f9a5c40a14e 100644 --- a/pkg/tests/apis/provisioning/provisioning_test.go +++ b/pkg/tests/apis/provisioning/provisioning_test.go @@ -161,14 +161,21 @@ func TestIntegrationProvisioning_CreatingAndGetting(t *testing.T) { // Viewer can see settings listing t.Run("viewer has access to list", func(t *testing.T) { settings := &provisioning.RepositoryViewList{} - rsp := helper.ViewerREST.Get(). - Namespace("default"). - Suffix("settings"). - Do(context.Background()) - require.NoError(t, rsp.Error()) - err := rsp.Into(settings) - require.NoError(t, err) - require.Len(t, settings.Items, len(inputFiles)) + // Wait for unified storage to make the data available + require.Eventually(t, func() bool { + rsp := helper.ViewerREST.Get(). + Namespace("default"). + Suffix("settings"). + Do(context.Background()) + if rsp.Error() != nil { + return false + } + err := rsp.Into(settings) + if err != nil { + return false + } + return len(settings.Items) == len(inputFiles) + }, time.Second*10, time.Millisecond*100, "Expected settings to have len(inputFiles) items") // FIXME: this should be an enterprise integration test if extensions.IsEnterprise { @@ -1825,8 +1832,10 @@ func TestIntegrationProvisioning_MoveResources(t *testing.T) { // Verify dashboard still exists in Grafana with same content but may have updated path references helper.SyncAndWait(t, repo, nil) - _, err = helper.DashboardsV1.Resource.Get(ctx, allPanelsUID, metav1.GetOptions{}) - require.NoError(t, err, "dashboard should still exist in Grafana after move") + require.Eventually(t, func() bool { + _, err = helper.DashboardsV1.Resource.Get(ctx, allPanelsUID, metav1.GetOptions{}) + return err == nil + }, 10*time.Second, 100*time.Millisecond, "dashboard should still exist in Grafana after move") // Using Eventually to account for potential delays in dashboards APIs. }) t.Run("move file to nested path without ref", func(t *testing.T) { @@ -2107,3 +2116,196 @@ func TestIntegrationProvisioning_MoveResources(t *testing.T) { }, time.Second*10, time.Millisecond*100, "Expected move job to handle non-existent resource") }) } + +func TestIntegrationProvisioning_SecondRepositoryOnlyExportsNewDashboards(t *testing.T) { + if testing.Short() { + t.Skip("skipping integration test") + } + + helper := runGrafana(t) + ctx := context.Background() + + // Create some unmanaged dashboards directly in Grafana first + dashboard1 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v1.yaml") + dashboard1Obj, err := helper.DashboardsV1.Resource.Create(ctx, dashboard1, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create first dashboard") + dashboard1Name := dashboard1Obj.GetName() + + dashboard2 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v2beta1.yaml") + dashboard2Obj, err := helper.DashboardsV2beta1.Resource.Create(ctx, dashboard2, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create second dashboard") + dashboard2Name := dashboard2Obj.GetName() + + // Create the first repository with sync enabled + const repo1 = "first-repository" + repo1Path := filepath.Join(helper.ProvisioningPath, repo1) + err = os.MkdirAll(repo1Path, 0750) + require.NoError(t, err, "should be able to create repository path") + + createBody1 := helper.RenderObject(t, "testdata/local-write.json.tmpl", map[string]any{ + "Name": repo1, + "SyncEnabled": true, + "SyncTarget": "folder", + "Path": repo1Path, + }) + _, err = helper.Repositories.Resource.Create(ctx, createBody1, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create first repository") + + // Print file tree before export + printFileTree(t, helper.ProvisioningPath) + + // Initial export + result := helper.AdminREST.Post(). + Namespace("default"). + Resource("repositories"). + Name(repo1). + SubResource("jobs"). + SetHeader("Content-Type", "application/json"). + Body(asJSON(&provisioning.JobSpec{ + Push: &provisioning.ExportJobOptions{ + Folder: "", // export entire instance + Path: "", // no prefix necessary for testing + }, + })). + Do(ctx) + require.NoError(t, result.Error(), "should be able to create export job for first repo") + helper.AwaitJobsWithStates(t, repo1, []string{"success"}) + // Wait for first repository to sync + helper.SyncAndWait(t, repo1, nil) + + printFileTree(t, helper.ProvisioningPath) + // Verify that the first repository has claimed ownership of the dashboards + managedDash1, err := helper.DashboardsV1.Resource.Get(ctx, dashboard1Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, managedDash1.GetAnnotations()[utils.AnnoKeyManagerIdentity], "dashboard1 should be managed by first repo") + + managedDash2, err := helper.DashboardsV2beta1.Resource.Get(ctx, dashboard2Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, managedDash2.GetAnnotations()[utils.AnnoKeyManagerIdentity], "dashboard2 should be managed by first repo") + + // Create second repository - enable sync and set different target + + const repo2 = "second-repository" + repo2Path := filepath.Join(helper.ProvisioningPath, repo2) + err = os.MkdirAll(repo2Path, 0750) + require.NoError(t, err, "should be able to create seconrd repository path") + + printFileTree(t, helper.ProvisioningPath) + + createBody2 := helper.RenderObject(t, "testdata/local-write.json.tmpl", map[string]any{ + "Name": repo2, + "SyncEnabled": true, + "SyncTarget": "folder", + "Path": repo2Path, + }) + + _, err = helper.Repositories.Resource.Create(ctx, createBody2, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create second repository") + + // Wait for second repository to sync + helper.SyncAndWait(t, repo2, nil) + + // Validate that folders for both repositories exist + folders, err := helper.Folders.Resource.List(ctx, metav1.ListOptions{}) + require.NoError(t, err, "should be able to list folders") + + var repo1FolderFound, repo2FolderFound bool + for _, folder := range folders.Items { + if folder.GetName() == repo1 { + repo1FolderFound = true + } + if folder.GetName() == repo2 { + repo2FolderFound = true + } + } + require.True(t, repo1FolderFound, "folder for first repository %s should exist after sync", repo1) + require.True(t, repo2FolderFound, "folder for second repository %s should exist after sync", repo2) + + // Create a third dashboard that won't be claimed by the first repo + dashboard3 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v0.yaml") + dashboard3Obj, err := helper.DashboardsV0.Resource.Create(ctx, dashboard3, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create third dashboard") + dashboard3Name := dashboard3Obj.GetName() + + // Verify dashboard3 is not managed by anyone initially + unmanagedDash3, err := helper.DashboardsV0.Resource.Get(ctx, dashboard3Name, metav1.GetOptions{}) + require.NoError(t, err) + manager, found := unmanagedDash3.GetAnnotations()[utils.AnnoKeyManagerIdentity] + require.True(t, !found || manager == "", "dashboard3 should not be managed initially") + + printFileTree(t, helper.ProvisioningPath) + // Count files in first repo before second export + files1Before, err := countFilesInDir(repo1Path) + require.NoError(t, err) + + // Export from second repository - this should only export the unmanaged dashboard3 + result = helper.AdminREST.Post(). + Namespace("default"). + Resource("repositories"). + Name(repo2). + SubResource("jobs"). + SetHeader("Content-Type", "application/json"). + Body(asJSON(&provisioning.JobSpec{ + Push: &provisioning.ExportJobOptions{ + Folder: "", // export entire instance + Path: "", // no prefix necessary for testing + }, + })). + Do(ctx) + require.NoError(t, result.Error(), "should be able to create export job for second repo") + + // Wait for second repository export to complete + helper.AwaitJobsWithStates(t, repo2, []string{"success"}) + + // Wait for second repository to sync + helper.SyncAndWait(t, repo1, nil) + helper.SyncAndWait(t, repo2, nil) + + printFileTree(t, helper.ProvisioningPath) + files1After, err := countFilesInDir(repo1Path) + require.NoError(t, err) + + actualNewFiles := files1After - files1Before + require.Equal(t, 0, actualNewFiles, + "second repository should skip managed dashboards and had folder issues with unmanaged dashboard (expected %d new files, got %d)", + 0, actualNewFiles) + + // Verify files in the second repository + files2After, err := countFilesInDir(repo2Path) + require.NoError(t, err) + require.Equal(t, 1, files2After, + "second repository should only export the unmanaged dashboard (expected %d new files, got %d)", + 1, files2After) + + // Verify dashboard1 and dashboard2 are still managed by repo1 (unchanged) + stillManagedDash1, err := helper.DashboardsV1.Resource.Get(ctx, dashboard1Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, stillManagedDash1.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard1 should still be managed by first repo") + + stillManagedDash2, err := helper.DashboardsV2beta1.Resource.Get(ctx, dashboard2Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, stillManagedDash2.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard2 should still be managed by first repo") + + // Verify dashboard3 is now managed by repo2 + stillManagedDash3, err := helper.DashboardsV0.Resource.Get(ctx, dashboard3Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo2, stillManagedDash3.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard3 should now be managed by second repo") +} + +// Helper function to count files in a directory recursively +func countFilesInDir(rootPath string) (int, error) { + count := 0 + err := filepath.WalkDir(rootPath, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if !d.IsDir() { + count++ + } + return nil + }) + return count, err +} diff --git a/pkg/tests/testinfra/testinfra.go b/pkg/tests/testinfra/testinfra.go index eb9b460c6b8..56f9f3000e2 100644 --- a/pkg/tests/testinfra/testinfra.go +++ b/pkg/tests/testinfra/testinfra.go @@ -13,6 +13,7 @@ import ( "time" "github.com/grafana/grafana/pkg/services/featuremgmt" + "github.com/grafana/grafana/pkg/services/sqlstore/sqlutil" "github.com/prometheus/client_golang/prometheus" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -85,6 +86,20 @@ func StartGrafanaEnv(t *testing.T, grafDir, cfgPath string) (string, *server.Tes err = featuremgmt.InitOpenFeatureWithCfg(cfg) require.NoError(t, err) + + // Use proper database type based on the environment variable GRAFANA_TEST_DB in tests + testDB, err := sqlutil.GetTestDB(sqlutil.GetTestDBType()) + require.NoError(t, err) + t.Cleanup(testDB.Cleanup) + + dbCfg := cfg.Raw.Section("database") + dbCfg.Key("type").SetValue(testDB.DriverName) + dbCfg.Key("host").SetValue(testDB.Host) + dbCfg.Key("port").SetValue(testDB.Port) + dbCfg.Key("user").SetValue(testDB.User) + dbCfg.Key("password").SetValue(testDB.Password) + dbCfg.Key("name").SetValue(testDB.Database) + env, err := server.InitializeForTest(t, t, cfg, serverOpts, apiServerOpts) require.NoError(t, err) diff --git a/pkg/tsdb/elasticsearch/healthcheck.go b/pkg/tsdb/elasticsearch/healthcheck.go index a3e04aafb6f..928945691de 100644 --- a/pkg/tsdb/elasticsearch/healthcheck.go +++ b/pkg/tsdb/elasticsearch/healthcheck.go @@ -191,11 +191,15 @@ func validateIndex(ctx context.Context, ds *es.DatasourceInfo) (message string, return "Failed to unmarshal field capabilities response", "error" } if fieldCaps["error"] != nil { - if errorMessage, ok := fieldCaps["error"].(map[string]any)["reason"].(string); ok { - return fmt.Sprintf("Error validating index: %s", errorMessage), "warning" - } else { + errorMap, ok := fieldCaps["error"].(map[string]any) + if !ok { return "Error validating index", "warning" } + errorMessage, ok := errorMap["reason"].(string) + if !ok { + return "Error validating index", "warning" + } + return fmt.Sprintf("Error validating index: %s", errorMessage), "warning" } fields, ok := fieldCaps["fields"].(map[string]any) diff --git a/pkg/tsdb/elasticsearch/healthcheck_test.go b/pkg/tsdb/elasticsearch/healthcheck_test.go index b3f6dc97c93..48fd00e8adc 100644 --- a/pkg/tsdb/elasticsearch/healthcheck_test.go +++ b/pkg/tsdb/elasticsearch/healthcheck_test.go @@ -58,6 +58,16 @@ func Test_validateIndex_Warning_ErrorValidatingIndex(t *testing.T) { assert.Equal(t, "Elasticsearch data source is healthy. Warning: Error validating index: index_not_found", res.Message) } +func Test_validateIndex_Warning_ErrorValidatingIndex2(t *testing.T) { + service := GetMockService(http.StatusOK, "200 OK", `{"status":"green"}`, `{"error":"not a map"}`) + res, _ := service.CheckHealth(mockedCfg, &backend.CheckHealthRequest{ + PluginContext: backend.PluginContext{}, + Headers: nil, + }) + assert.Equal(t, backend.HealthStatusOk, res.Status) + assert.Equal(t, "Elasticsearch data source is healthy. Warning: Error validating index", res.Message) +} + func Test_validateIndex_Warning_WrongTimestampType(t *testing.T) { service := GetMockService(http.StatusOK, "200 OK", `{"status":"green"}`, `{"fields":{"timestamp":{"float":{"metadata_field":true}}}}`) res, _ := service.CheckHealth(mockedCfg, &backend.CheckHealthRequest{ diff --git a/pkg/tsdb/influxdb/flux/flux.go b/pkg/tsdb/influxdb/flux/flux.go index cab5c0d1067..6643f064910 100644 --- a/pkg/tsdb/influxdb/flux/flux.go +++ b/pkg/tsdb/influxdb/flux/flux.go @@ -27,9 +27,8 @@ func Query(ctx context.Context, dsInfo *models.DatasourceInfo, tsdbQuery backend } defer r.client.Close() - timeRange := tsdbQuery.Queries[0].TimeRange for _, query := range tsdbQuery.Queries { - qm, err := getQueryModel(query, timeRange, dsInfo) + qm, err := getQueryModel(query, query.TimeRange, dsInfo) if err != nil { tRes.Responses[query.RefID] = backend.DataResponse{ Error: err, diff --git a/pkg/tsdb/jaeger/client.go b/pkg/tsdb/jaeger/client.go index 653c0463892..d858aa48f56 100644 --- a/pkg/tsdb/jaeger/client.go +++ b/pkg/tsdb/jaeger/client.go @@ -115,11 +115,15 @@ func (j *JaegerClient) Operations(s string) ([]string, error) { } func (j *JaegerClient) Search(query *JaegerQuery, start, end int64) ([]TraceResponse, error) { - jaegerURL, err := url.Parse(j.url) + u, err := url.JoinPath(j.url, "/api/traces") if err != nil { - return []TraceResponse{}, fmt.Errorf("failed to parse Jaeger URL: %w", err) + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to join url path: %w", err)) + } + + jaegerURL, err := url.Parse(u) + if err != nil { + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to parse Jaeger URL: %w", err)) } - jaegerURL.Path = "/api/traces" var queryTags string if query.Tags != "" { @@ -135,7 +139,7 @@ func (j *JaegerClient) Search(query *JaegerQuery, start, end int64) ([]TraceResp marshaledTags, err := json.Marshal(tagMap) if err != nil { - return []TraceResponse{}, fmt.Errorf("failed to convert tags to JSON: %w", err) + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to convert tags to JSON: %w", err)) } queryTags = string(marshaledTags) diff --git a/pkg/tsdb/jaeger/client_test.go b/pkg/tsdb/jaeger/client_test.go index afabdc4bccc..eceb103ab82 100644 --- a/pkg/tsdb/jaeger/client_test.go +++ b/pkg/tsdb/jaeger/client_test.go @@ -186,6 +186,19 @@ func TestJaegerClient_Search(t *testing.T) { expectError bool expectedError error }{ + { + name: "Preserves base path in Jaeger URL", + query: &JaegerQuery{ + Service: "test-service", + }, + start: 1735689600000000, + end: 1738368000000000, + mockResponse: `{"data":[{"traceID":"test-trace-id"}]}`, + mockStatusCode: http.StatusOK, + expectedURL: "/abc/api/traces?end=1738368000000000&service=test-service&start=1735689600000000", + expectError: false, + expectedError: nil, + }, { name: "Successful search with all parameters", query: &JaegerQuery{ @@ -245,6 +258,11 @@ func TestJaegerClient_Search(t *testing.T) { settings := backend.DataSourceInstanceSettings{ URL: server.URL, } + + if tt.name == "Preserves base path in Jaeger URL" { + settings.URL = server.URL + "/abc" + } + client, err := New(server.Client(), log.NewNullLogger(), settings) assert.NoError(t, err) traces, err := client.Search(tt.query, tt.start, tt.end) diff --git a/pkg/util/sqlite/sqlite_cgo.go b/pkg/util/sqlite/sqlite_cgo.go new file mode 100644 index 00000000000..de18511741d --- /dev/null +++ b/pkg/util/sqlite/sqlite_cgo.go @@ -0,0 +1,46 @@ +//go:build cgo + +package sqlite + +import ( + "errors" + + "github.com/mattn/go-sqlite3" +) + +type Driver = sqlite3.SQLiteDriver + +// The errors below are used in tests to simulate specific SQLite errors. It's a temporary solution +// until we rewrite the tests not to depend on the sqlite3 package internals directly. +var ( + TestErrUniqueConstraintViolation = sqlite3.Error{Code: sqlite3.ErrConstraint, ExtendedCode: sqlite3.ErrConstraintUnique} + TestErrBusy = sqlite3.Error{Code: sqlite3.ErrBusy} + TestErrLocked = sqlite3.Error{Code: sqlite3.ErrLocked} +) + +func IsBusyOrLocked(err error) bool { + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.Code == sqlite3.ErrLocked || sqliteErr.Code == sqlite3.ErrBusy + } + return false +} + +func IsUniqueConstraintViolation(err error) bool { + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.ExtendedCode == sqlite3.ErrConstraintUnique || sqliteErr.ExtendedCode == sqlite3.ErrConstraintPrimaryKey + } + return false +} + +func ErrorMessage(err error) string { + if err == nil { + return "" + } + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.Error() + } + return err.Error() +} diff --git a/pkg/util/sqlite/sqlite_nocgo.go b/pkg/util/sqlite/sqlite_nocgo.go new file mode 100644 index 00000000000..e3b9a0429d3 --- /dev/null +++ b/pkg/util/sqlite/sqlite_nocgo.go @@ -0,0 +1,24 @@ +//go:build !cgo + +package sqlite + +import "modernc.org/sqlite" + +// +// FIXME (@zserge) +// +// This non-CGo "implementation" is merely a stub to make Grafana compile without CGo. +// Any attempts to actually use this driver are likely to fail at runtime in the most brutal ways. +// + +type Driver = sqlite.Driver + +func IsBusyOrLocked(err error) bool { + return false // FIXME +} +func IsUniqueConstraintViolation(err error) bool { + return false // FIXME +} +func ErrorMessage(err error) string { + return "" // FIXME +} diff --git a/pkg/util/xorm/dialect_sqlite3.go b/pkg/util/xorm/dialect_sqlite3.go index 381a38bb490..4c235ee1c90 100644 --- a/pkg/util/xorm/dialect_sqlite3.go +++ b/pkg/util/xorm/dialect_sqlite3.go @@ -11,8 +11,8 @@ import ( "regexp" "strings" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/grafana/grafana/pkg/util/xorm/core" - sqlite "github.com/mattn/go-sqlite3" ) var ( @@ -476,11 +476,7 @@ func (db *sqlite3) Filters() []core.Filter { } func (db *sqlite3) RetryOnError(err error) bool { - var sqlError sqlite.Error - if errors.As(err, &sqlError) && (sqlError.Code == sqlite.ErrLocked || sqlError.Code == sqlite.ErrBusy) { - return true - } - return false + return sqlite.IsBusyOrLocked(err) } type sqlite3Driver struct { diff --git a/pkg/util/xorm/xorm_test.go b/pkg/util/xorm/xorm_test.go index f2939fbf9a8..281282a1ef8 100644 --- a/pkg/util/xorm/xorm_test.go +++ b/pkg/util/xorm/xorm_test.go @@ -4,7 +4,7 @@ import ( "encoding/json" "testing" - _ "github.com/mattn/go-sqlite3" + _ "github.com/grafana/grafana/pkg/util/sqlite" "github.com/stretchr/testify/require" ) diff --git a/playwright.config.ts b/playwright.config.ts index a3388e20c5e..e1a6b654e73 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -195,6 +195,15 @@ export default defineConfig({
},
dependencies: ['authenticate'],
},
+ {
+ name: 'canvas',
+ testDir: path.join(testDirRoot, '/canvas'),
+ use: {
+ ...devices['Desktop Chrome'],
+ storageState: 'playwright/.auth/admin.json',
+ },
+ dependencies: ['authenticate'],
+ },
{
name: 'zipkin',
testDir: path.join(pluginDirRoot, '/zipkin'),
diff --git a/public/api-enterprise-spec.json b/public/api-enterprise-spec.json
index 6d5885e3bdd..699a3535e54 100644
--- a/public/api-enterprise-spec.json
+++ b/public/api-enterprise-spec.json
@@ -8321,7 +8321,7 @@
"type": "object",
"properties": {
"Object": {
- "description": "Object is a JSON compatible map with string, float, int, bool, []interface{},\nor map[string]interface{} children.",
+ "description": "Object is a JSON compatible map with string, float, int, bool, []any,\nor map[string]any children.",
"type": "object",
"additionalProperties": {}
}
diff --git a/public/api-merged.json b/public/api-merged.json
index 2f2eed4fca5..53f61f1eff1 100644
--- a/public/api-merged.json
+++ b/public/api-merged.json
@@ -2188,6 +2188,64 @@
}
}
},
+ "/anonymous/devices": {
+ "get": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "devices"
+ ],
+ "summary": "Lists all devices within the last 30 days",
+ "operationId": "listDevices",
+ "responses": {
+ "200": {
+ "$ref": "#/responses/devicesResponse"
+ },
+ "401": {
+ "$ref": "#/responses/unauthorisedError"
+ },
+ "403": {
+ "$ref": "#/responses/forbiddenError"
+ },
+ "404": {
+ "$ref": "#/responses/notFoundError"
+ },
+ "500": {
+ "$ref": "#/responses/internalServerError"
+ }
+ }
+ }
+ },
+ "/anonymous/search": {
+ "get": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "devices"
+ ],
+ "summary": "Lists all devices within the last 30 days",
+ "operationId": "SearchDevices",
+ "responses": {
+ "200": {
+ "$ref": "#/responses/devicesSearchResponse"
+ },
+ "401": {
+ "$ref": "#/responses/unauthorisedError"
+ },
+ "403": {
+ "$ref": "#/responses/forbiddenError"
+ },
+ "404": {
+ "$ref": "#/responses/notFoundError"
+ },
+ "500": {
+ "$ref": "#/responses/internalServerError"
+ }
+ }
+ }
+ },
"/cloudmigration/migration": {
"get": {
"tags": [
@@ -2671,6 +2729,586 @@
}
}
},
+ "/convert/api/prom/rules": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
+ "operationId": "RouteConvertPrometheusCortexGetRules",
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "post": {
+ "consumes": [
+ "application/json",
+ "application/yaml"
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
+ "operationId": "RouteConvertPrometheusCortexPostRuleGroups",
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
+ "/convert/api/prom/rules/{NamespaceTitle}": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
+ "operationId": "RouteConvertPrometheusCortexGetNamespace",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "post": {
+ "description": "If the group already exists and was not imported from a Prometheus-compatible source initially,\nit will not be replaced and an error will be returned.",
+ "consumes": [
+ "application/yaml"
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
+ "operationId": "RouteConvertPrometheusCortexPostRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-datasource-uid",
+ "in": "header"
+ },
+ {
+ "type": "boolean",
+ "name": "x-grafana-alerting-recording-rules-paused",
+ "in": "header"
+ },
+ {
+ "type": "boolean",
+ "name": "x-grafana-alerting-alert-rules-paused",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-target-datasource-uid",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-folder-uid",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-notification-receiver",
+ "in": "header"
+ },
+ {
+ "name": "Body",
+ "in": "body",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "x-raw-request": "true"
+ },
+ "delete": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
+ "operationId": "RouteConvertPrometheusCortexDeleteNamespace",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
+ "/convert/api/prom/rules/{NamespaceTitle}/{Group}": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
+ "operationId": "RouteConvertPrometheusCortexGetRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "Group",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusRuleGroup",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "delete": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
+ "operationId": "RouteConvertPrometheusCortexDeleteRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "Group",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
+ "/convert/prometheus/config/v1/rules": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
+ "operationId": "RouteConvertPrometheusGetRules",
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "post": {
+ "consumes": [
+ "application/json",
+ "application/yaml"
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
+ "operationId": "RouteConvertPrometheusPostRuleGroups",
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
+ "/convert/prometheus/config/v1/rules/{NamespaceTitle}": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
+ "operationId": "RouteConvertPrometheusGetNamespace",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "post": {
+ "description": "If the group already exists and was not imported from a Prometheus-compatible source initially,\nit will not be replaced and an error will be returned.",
+ "consumes": [
+ "application/yaml"
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
+ "operationId": "RouteConvertPrometheusPostRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-datasource-uid",
+ "in": "header"
+ },
+ {
+ "type": "boolean",
+ "name": "x-grafana-alerting-recording-rules-paused",
+ "in": "header"
+ },
+ {
+ "type": "boolean",
+ "name": "x-grafana-alerting-alert-rules-paused",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-target-datasource-uid",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-folder-uid",
+ "in": "header"
+ },
+ {
+ "type": "string",
+ "name": "x-grafana-alerting-notification-receiver",
+ "in": "header"
+ },
+ {
+ "name": "Body",
+ "in": "body",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "x-raw-request": "true"
+ },
+ "delete": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
+ "operationId": "RouteConvertPrometheusDeleteNamespace",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
+ "/convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group}": {
+ "get": {
+ "produces": [
+ "application/yaml"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
+ "operationId": "RouteConvertPrometheusGetRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "Group",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusRuleGroup",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ }
+ },
+ "delete": {
+ "produces": [
+ "application/json"
+ ],
+ "tags": [
+ "convert_prometheus"
+ ],
+ "summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
+ "operationId": "RouteConvertPrometheusDeleteRuleGroup",
+ "parameters": [
+ {
+ "type": "string",
+ "name": "NamespaceTitle",
+ "in": "path",
+ "required": true
+ },
+ {
+ "type": "string",
+ "name": "Group",
+ "in": "path",
+ "required": true
+ }
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ }
+ }
+ },
"/dashboard/snapshots": {
"get": {
"tags": [
@@ -8672,33 +9310,6 @@
"$ref": "#/responses/internalServerError"
}
}
- },
- "post": {
- "produces": [
- "application/json"
- ],
- "tags": [
- "devices"
- ],
- "summary": "Lists all devices within the last 30 days",
- "operationId": "SearchDevices",
- "responses": {
- "200": {
- "$ref": "#/responses/devicesSearchResponse"
- },
- "401": {
- "$ref": "#/responses/unauthorisedError"
- },
- "403": {
- "$ref": "#/responses/forbiddenError"
- },
- "404": {
- "$ref": "#/responses/notFoundError"
- },
- "500": {
- "$ref": "#/responses/internalServerError"
- }
- }
}
},
"/search/sorting": {
@@ -9223,35 +9834,6 @@
}
}
},
- "/stats": {
- "get": {
- "produces": [
- "application/json"
- ],
- "tags": [
- "devices"
- ],
- "summary": "Lists all devices within the last 30 days",
- "operationId": "listDevices",
- "responses": {
- "200": {
- "$ref": "#/responses/devicesResponse"
- },
- "401": {
- "$ref": "#/responses/unauthorisedError"
- },
- "403": {
- "$ref": "#/responses/forbiddenError"
- },
- "404": {
- "$ref": "#/responses/notFoundError"
- },
- "500": {
- "$ref": "#/responses/internalServerError"
- }
- }
- }
- },
"/teams": {
"post": {
"tags": [
diff --git a/public/app/api/clients/dashboard/v0alpha1/baseAPI.ts b/public/app/api/clients/dashboard/v0alpha1/baseAPI.ts
new file mode 100644
index 00000000000..dca53563fa0
--- /dev/null
+++ b/public/app/api/clients/dashboard/v0alpha1/baseAPI.ts
@@ -0,0 +1,14 @@
+import { createApi } from '@reduxjs/toolkit/query/react';
+
+import { createBaseQuery } from 'app/api/createBaseQuery';
+import { getAPIBaseURL } from 'app/api/utils';
+
+export const BASE_URL = getAPIBaseURL('dashboard.grafana.app', 'v0alpha1');
+
+export const api = createApi({
+ reducerPath: 'dashboardAPIv0alpha1',
+ baseQuery: createBaseQuery({
+ baseURL: BASE_URL,
+ }),
+ endpoints: () => ({}),
+});
diff --git a/public/app/api/clients/dashboard/v0alpha1/endpoints.gen.ts b/public/app/api/clients/dashboard/v0alpha1/endpoints.gen.ts
new file mode 100644
index 00000000000..8730157dedf
--- /dev/null
+++ b/public/app/api/clients/dashboard/v0alpha1/endpoints.gen.ts
@@ -0,0 +1,53 @@
+import { api } from './baseAPI';
+export const addTagTypes = ['Search'] as const;
+const injectedRtkApi = api
+ .enhanceEndpoints({
+ addTagTypes,
+ })
+ .injectEndpoints({
+ endpoints: (build) => ({
+ getSearch: build.query({
+ query: (queryArg) => ({
+ url: `/search`,
+ params: {
+ query: queryArg.query,
+ folder: queryArg.folder,
+ sort: queryArg.sort,
+ },
+ }),
+ providesTags: ['Search'],
+ }),
+ }),
+ overrideExisting: false,
+ });
+export { injectedRtkApi as generatedAPI };
+export type GetSearchApiResponse = /** status 200 undefined */ {
+ /** APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources */
+ apiVersion?: string;
+ /** Facet results */
+ facets?: {
+ [key: string]: any;
+ };
+ /** The dashboard body (unstructured for now) */
+ hits: any[];
+ /** Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds */
+ kind?: string;
+ /** Max score */
+ maxScore?: number;
+ /** Where the query started from */
+ offset?: number;
+ /** Cost of running the query */
+ queryCost?: number;
+ /** How are the results sorted */
+ sortBy?: any;
+ /** The number of matching results */
+ totalHits: number;
+};
+export type GetSearchApiArg = {
+ /** user query string */
+ query?: string;
+ /** search/list within a folder (not recursive) */
+ folder?: string;
+ /** sortable field */
+ sort?: string;
+};
diff --git a/public/app/api/clients/dashboard/v0alpha1/index.ts b/public/app/api/clients/dashboard/v0alpha1/index.ts
new file mode 100644
index 00000000000..f61b46bc1f9
--- /dev/null
+++ b/public/app/api/clients/dashboard/v0alpha1/index.ts
@@ -0,0 +1,27 @@
+import { generatedAPI, GetSearchApiArg } from './endpoints.gen';
+
+type OverrideGetSearchRequestOptions = GetSearchApiArg & {
+ type: string;
+};
+
+export const dashboardAPIv0alpha1 = generatedAPI.enhanceEndpoints({
+ addTagTypes: ['Folder', 'Dashboard'],
+ endpoints: {
+ getSearch: (endpointDefinition) => {
+ const originalQuery = endpointDefinition.query;
+ endpointDefinition.providesTags = ['Search', 'Folder', 'Dashboard'];
+ if (originalQuery) {
+ // TODO: Remove once API spec is updated with `type`
+ endpointDefinition.query = (requestOptions: OverrideGetSearchRequestOptions) => ({
+ ...originalQuery(requestOptions),
+ params: {
+ ...requestOptions,
+ type: requestOptions.type,
+ },
+ });
+ }
+ },
+ },
+});
+
+export const { useGetSearchQuery } = dashboardAPIv0alpha1;
diff --git a/public/app/app.ts b/public/app/app.ts
index 0e3ce53e238..5c48558c30c 100644
--- a/public/app/app.ts
+++ b/public/app/app.ts
@@ -50,7 +50,7 @@ import {
setPanelRenderer,
setPluginPage,
} from '@grafana/runtime/internal';
-import { loadResources as loadScenesResources } from '@grafana/scenes';
+import { loadResources as loadScenesResources, sceneUtils } from '@grafana/scenes';
import config, { updateConfig } from 'app/core/config';
import { getStandardTransformers } from 'app/features/transformers/standardTransformers';
@@ -82,6 +82,7 @@ import { initAlerting } from './features/alerting/unified/initAlerting';
import { initAuthConfig } from './features/auth-config';
import { getTimeSrv } from './features/dashboard/services/TimeSrv';
import { EmbeddedDashboardLazy } from './features/dashboard-scene/embedding/EmbeddedDashboardLazy';
+import { DashboardLevelTimeMacro } from './features/dashboard-scene/scene/DashboardLevelTimeMacro';
import { initGrafanaLive } from './features/live';
import { PanelDataErrorView } from './features/panel/components/PanelDataErrorView';
import { PanelRenderer } from './features/panel/components/PanelRenderer';
@@ -284,6 +285,11 @@ export class GrafanaApp {
initializeCrashDetection();
}
+ if (config.featureToggles.dashboardLevelTimeMacros) {
+ sceneUtils.registerVariableMacro('__from', DashboardLevelTimeMacro, true);
+ sceneUtils.registerVariableMacro('__to', DashboardLevelTimeMacro, true);
+ }
+
const root = createRoot(document.getElementById('reactRoot')!);
root.render(
createElement(AppWrapper, {
diff --git a/public/app/core/components/NestedFolderPicker/NestedFolderPicker.test.tsx b/public/app/core/components/NestedFolderPicker/NestedFolderPicker.test.tsx
index 226583c54e1..38674034a0d 100644
--- a/public/app/core/components/NestedFolderPicker/NestedFolderPicker.test.tsx
+++ b/public/app/core/components/NestedFolderPicker/NestedFolderPicker.test.tsx
@@ -1,89 +1,31 @@
-import { fireEvent, render as rtlRender, screen } from '@testing-library/react';
-import userEvent from '@testing-library/user-event';
-import { HttpResponse, http } from 'msw';
-import { SetupServer, setupServer } from 'msw/node';
-import { TestProvider } from 'test/helpers/TestProvider';
+import { fireEvent, render, screen } from 'test/test-utils';
-import { config } from '@grafana/runtime';
+import { config, setBackendSrv } from '@grafana/runtime';
+import { setupMockServer } from '@grafana/test-utils/server';
+import { getFolderFixtures } from '@grafana/test-utils/unstable';
import { backendSrv } from 'app/core/services/backend_srv';
-import {
- treeViewersCanEdit,
- wellFormedTree,
-} from '../../../features/browse-dashboards/fixtures/dashboardsTreeItem.fixture';
-
import { NestedFolderPicker } from './NestedFolderPicker';
-const [mockTree, { folderA, folderB, folderC, folderA_folderA, folderA_folderB }] = wellFormedTree();
-const [mockTreeThatViewersCanEdit /* shares folders with wellFormedTree */] = treeViewersCanEdit();
+const [_, { folderA, folderB, folderC, folderA_folderA, folderA_folderB, folderA_folderC }] = getFolderFixtures();
-jest.mock('@grafana/runtime', () => ({
- ...jest.requireActual('@grafana/runtime'),
- getBackendSrv: () => backendSrv,
-}));
-
-function render(...[ui, options]: Parameters) {
- rtlRender({ui} , options);
-}
+setupMockServer();
+setBackendSrv(backendSrv);
describe('NestedFolderPicker', () => {
const mockOnChange = jest.fn();
const originalScrollIntoView = window.HTMLElement.prototype.scrollIntoView;
- let server: SetupServer;
beforeAll(() => {
window.HTMLElement.prototype.scrollIntoView = function () {};
-
- server = setupServer(
- http.get('/api/folders/:uid', () => {
- return HttpResponse.json({
- title: folderA.item.title,
- uid: folderA.item.uid,
- });
- }),
-
- http.get('/apis/provisioning.grafana.app/v0alpha1/namespaces/default/settings', () => {
- return HttpResponse.json({
- items: [],
- });
- }),
-
- http.get('/api/folders', ({ request }) => {
- const url = new URL(request.url);
- const parentUid = url.searchParams.get('parentUid') ?? undefined;
- const permission = url.searchParams.get('permission');
-
- const limit = parseInt(url.searchParams.get('limit') ?? '1000', 10);
- const page = parseInt(url.searchParams.get('page') ?? '1', 10);
-
- const tree = permission === 'Edit' ? mockTreeThatViewersCanEdit : mockTree;
-
- // reconstruct a folder API response from the flat tree fixture
- const folders = tree
- .filter((v) => v.item.kind === 'folder' && v.item.parentUID === parentUid)
- .map((folder) => {
- return {
- uid: folder.item.uid,
- title: folder.item.kind === 'folder' ? folder.item.title : "invalid - this shouldn't happen",
- };
- })
- .slice(limit * (page - 1), limit * page);
-
- return HttpResponse.json(folders);
- })
- );
-
- server.listen();
});
afterAll(() => {
- server.close();
window.HTMLElement.prototype.scrollIntoView = originalScrollIntoView;
});
afterEach(() => {
jest.resetAllMocks();
- server.resetHandlers();
});
it('renders a button with the correct label when no folder is selected', async () => {
@@ -92,18 +34,18 @@ describe('NestedFolderPicker', () => {
});
it('renders a button with the correct label when a folder is selected', async () => {
- render({children} ;
};
-// Mock the legacy API client
-jest.mock('app/features/browse-dashboards/api/browseDashboardsAPI', () => {
- const PAGE_SIZE = 10;
- return {
- PAGE_SIZE,
- browseDashboardsAPI: {
- endpoints: {
- listFolders: {
- select: jest.fn(() => () => legacyResponse),
- initiate: jest.fn(() => ({
- arg: { parentUid: undefined, page: 1, limit: PAGE_SIZE, permission: 'Edit' },
- unsubscribe: jest.fn(),
- })),
- },
- },
- },
- };
-});
-
-const appPlatfromResponse = {
- status: 'fulfilled',
- originalArgs: { name: 'general' },
- data: {
- items: [
- {
- metadata: { name: 'app1', annotations: {} },
- spec: { title: 'AppPlatform Folder' },
- },
- ],
- },
-};
-
-// Mock the appPlatform API client
-jest.mock('app/api/clients/folder/v1beta1', () => ({
- folderAPIv1beta1: {
- endpoints: {
- getFolderChildren: {
- select: jest.fn(() => () => appPlatfromResponse),
- initiate: jest.fn((arg: unknown) => ({
- arg,
- unsubscribe: jest.fn(),
- })),
- },
- },
- },
-}));
-
-// Mock getPaginationPlaceholders to return empty array for simplicity
-jest.mock('app/features/browse-dashboards/state/utils', () => ({
- getPaginationPlaceholders: jest.fn((): DashboardsTreeItem[] => []),
-}));
-
-// Mock useDispatch and useSelector to just pass through
-jest.mock('app/types/store', () => {
- const mod = jest.requireActual('app/types/store');
- return {
- ...mod,
- useDispatch: () => (val: unknown) => val,
- useSelector: (selector: Function) => selector(),
- };
-});
describe('useFoldersQuery', () => {
let configBackup: GrafanaConfig;
@@ -89,28 +33,40 @@ describe('useFoldersQuery', () => {
runtime.config.featureToggles = configBackup.featureToggles;
});
- it('returns data using legacy api', () => {
- runtime.config.featureToggles.foldersAppPlatformAPI = false;
- const items = testFn();
- expect((items[1].item as DashboardViewItem).title).toBe('Legacy Folder');
- });
+ describe.each([
+ // foldersAppPlatformAPI enabled
+ true,
+ // foldersAppPlatformAPI disabled
+ false,
+ ])('foldersAppPlatformAPI feature toggle set to %s', (featureToggleState) => {
+ it('returns data using legacy api', async () => {
+ runtime.config.featureToggles.foldersAppPlatformAPI = featureToggleState;
+ const [_dashboardsContainer, ...items] = await testFn();
- it('returns appPlatform hook result when foldersAppPlatformAPI is on', () => {
- runtime.config.featureToggles.foldersAppPlatformAPI = true;
- const items = testFn();
- expect((items[1].item as DashboardViewItem).title).toBe('AppPlatform Folder');
+ const sortedItemTitles = items.map((item) => (item.item as DashboardViewItem).title).sort();
+ const expectedTitles = [folderA.item.title, folderB.item.title, folderC.item.title].sort();
+
+ expect(sortedItemTitles).toEqual(expectedTitles);
+ });
});
});
-function testFn() {
- const { result } = renderHook(() => useFoldersQuery(true, {}));
+async function testFn() {
+ const { result } = renderHook(() => useFoldersQuery(true, {}), { wrapper });
- expect(result.current.items).toEqual([getRootFolderItem()]);
+ expect(result.current.items[0]).toEqual(getRootFolderItem());
expect(result.current.isLoading).toBe(false);
+
act(() => {
result.current.requestNextPage(undefined);
});
- expect(result.current.items.length).toBe(2);
+ expect(result.current.isLoading).toBe(true);
+
+ await waitFor(() => {
+ const withoutPaginationPlaceholders = result.current.items.filter((item) => item.item.kind !== 'ui');
+ return expect(withoutPaginationPlaceholders.length).toBeGreaterThan(1);
+ });
+
return result.current.items;
}
diff --git a/public/app/core/components/NestedFolderPicker/useFoldersQueryAppPlatform.ts b/public/app/core/components/NestedFolderPicker/useFoldersQueryAppPlatform.ts
index 4828417c10c..b877c69aba0 100644
--- a/public/app/core/components/NestedFolderPicker/useFoldersQueryAppPlatform.ts
+++ b/public/app/core/components/NestedFolderPicker/useFoldersQueryAppPlatform.ts
@@ -2,7 +2,7 @@ import { createSelector } from '@reduxjs/toolkit';
import { QueryStatus } from '@reduxjs/toolkit/query';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
-import { folderAPIv1beta1 } from 'app/api/clients/folder/v1beta1';
+import { dashboardAPIv0alpha1 } from 'app/api/clients/dashboard/v0alpha1';
import { DashboardViewItemWithUIItems, DashboardsTreeItem } from 'app/features/browse-dashboards/types';
import { useDispatch, useSelector } from 'app/types/store';
@@ -12,7 +12,7 @@ import { getPaginationPlaceholders } from '../../../features/browse-dashboards/s
import { getRootFolderItem } from './utils';
-type GetFolderChildrenQuery = ReturnType>;
+type GetFolderChildrenQuery = ReturnType>;
type GetFolderChildrenRequest = {
unsubscribe: () => void;
};
@@ -32,9 +32,9 @@ export function useFoldersQueryAppPlatform(isBrowsing: boolean, openFolders: Rec
const requestsRef = useRef([]);
// Keep a list of selectors for dynamic state selection
- const [selectors, setSelectors] = useState<
- Array>
- >([]);
+ const [selectors, setSelectors] = useState>>(
+ []
+ );
// This is an aggregated dynamic selector of all the selectors for all the request issued while loading the folder
// tree and returns the whole tree that was loaded so far.
@@ -50,7 +50,7 @@ export function useFoldersQueryAppPlatform(isBrowsing: boolean, openFolders: Rec
isLoading = true;
}
- const parentName = response.originalArgs?.name;
+ const parentName = response.originalArgs?.folder;
if (parentName) {
responseByParent[parentName] = response;
}
@@ -77,13 +77,13 @@ export function useFoldersQueryAppPlatform(isBrowsing: boolean, openFolders: Rec
return;
}
- const args = { name: finalParentUid };
+ const args = { folder: finalParentUid, type: 'folder' };
// Make a request
- const subscription = dispatch(folderAPIv1beta1.endpoints.getFolderChildren.initiate(args));
+ const subscription = dispatch(dashboardAPIv0alpha1.endpoints.getSearch.initiate(args));
// Add selector for the response to the list so we can then have an aggregated selector for all the folders
- const selector = folderAPIv1beta1.endpoints.getFolderChildren.select(args);
+ const selector = dashboardAPIv0alpha1.endpoints.getSearch.select(args);
setSelectors((selectors) => selectors.concat(selector));
// the subscriptions are saved in a ref so they can be unsubscribed on unmount
@@ -113,18 +113,18 @@ export function useFoldersQueryAppPlatform(isBrowsing: boolean, openFolders: Rec
response: GetFolderChildrenQuery | undefined,
level: number
): Array> {
- let folders = response?.data?.items ? [...response.data.items] : [];
- folders.sort((a, b) => collator.compare(a.spec.title, b.spec.title));
+ let folders = response?.data?.hits ? [...response.data.hits] : [];
+ folders.sort((a, b) => collator.compare(a.title, b.title));
const list = folders.flatMap((item) => {
- const name = item.metadata.name!;
+ const name = item.name;
const folderIsOpen = openFolders[name];
const flatItem: DashboardsTreeItem = {
isOpen: Boolean(folderIsOpen),
level: level,
item: {
kind: 'folder' as const,
- title: item.spec.title,
+ title: item.title,
// We use resource name as UID because well, not sure what metadata.uid would be used for now as you cannot
// query by it.
uid: name,
diff --git a/public/app/core/icons/cached.json b/public/app/core/icons/cached.json
index ce86bef4eae..810e2676755 100644
--- a/public/app/core/icons/cached.json
+++ b/public/app/core/icons/cached.json
@@ -74,6 +74,8 @@
"unicons/file-alt",
"unicons/file-blank",
"unicons/filter",
+ "unicons/filter-plus",
+ "unicons/filter-minus",
"unicons/folder",
"unicons/folder-open",
"unicons/folder-plus",
diff --git a/public/app/core/reducers/root.ts b/public/app/core/reducers/root.ts
index 67078fdbd27..bc296ee14b6 100644
--- a/public/app/core/reducers/root.ts
+++ b/public/app/core/reducers/root.ts
@@ -2,6 +2,7 @@ import { ReducersMapObject } from '@reduxjs/toolkit';
import { AnyAction, combineReducers } from 'redux';
import { alertingAPI as alertingPackageAPI } from '@grafana/alerting/unstable';
+import { dashboardAPIv0alpha1 } from 'app/api/clients/dashboard/v0alpha1';
import sharedReducers from 'app/core/reducers';
import ldapReducers from 'app/features/admin/state/reducers';
import alertingReducers from 'app/features/alerting/state/reducers';
@@ -71,6 +72,7 @@ const rootReducers = {
[provisioningAPIv0alpha1.reducerPath]: provisioningAPIv0alpha1.reducer,
[folderAPIv1beta1.reducerPath]: folderAPIv1beta1.reducer,
[advisorAPIv0alpha1.reducerPath]: advisorAPIv0alpha1.reducer,
+ [dashboardAPIv0alpha1.reducerPath]: dashboardAPIv0alpha1.reducer,
// PLOP_INJECT_REDUCER
// Used by the API client generator
};
diff --git a/public/app/features/alerting/unified/components/contact-points/ContactPointHeader.tsx b/public/app/features/alerting/unified/components/contact-points/ContactPointHeader.tsx
index 40a7afeddeb..1825b5e40a0 100644
--- a/public/app/features/alerting/unified/components/contact-points/ContactPointHeader.tsx
+++ b/public/app/features/alerting/unified/components/contact-points/ContactPointHeader.tsx
@@ -68,14 +68,14 @@ export const ContactPointHeader = ({ contactPoint, onDelete }: ContactPointHeade
*/
const isReferencedByAnything = usingK8sApi ? Boolean(numberOfPolicies || numberOfRules) : policies.length > 0;
/** Does the current user have permissions to edit the contact point? */
- const hasAbilityToEdit = canEditEntity(contactPoint) || editAllowed;
+ const hasAbilityToEdit = usingK8sApi ? canEditEntity(contactPoint) : editAllowed;
/** Can the contact point actually be edited via the UI? */
const contactPointIsEditable = !provisioned;
/** Given the alertmanager, the user's permissions, and the state of the contact point - can it actually be edited? */
const canEdit = editSupported && hasAbilityToEdit && contactPointIsEditable;
/** Does the current user have permissions to delete the contact point? */
- const hasAbilityToDelete = canDeleteEntity(contactPoint) || deleteAllowed;
+ const hasAbilityToDelete = usingK8sApi ? canDeleteEntity(contactPoint) : deleteAllowed;
/** Can the contact point actually be deleted, regardless of permissions? i.e. ensuring it isn't provisioned and isn't referenced elsewhere */
const contactPointIsDeleteable = !provisioned && !numberOfPoliciesPreventingDeletion && !numberOfRules;
/** Given the alertmanager, the user's permissions, and the state of the contact point - can it actually be deleted? */
diff --git a/public/app/features/alerting/unified/components/contact-points/ContactPoints.test.tsx b/public/app/features/alerting/unified/components/contact-points/ContactPoints.test.tsx
index 79cec523d69..db39d474957 100644
--- a/public/app/features/alerting/unified/components/contact-points/ContactPoints.test.tsx
+++ b/public/app/features/alerting/unified/components/contact-points/ContactPoints.test.tsx
@@ -198,12 +198,37 @@ describe('contact points', () => {
const unusedBadge = screen.getAllByLabelText('unused');
expect(unusedBadge).toHaveLength(4);
- const viewProvisioned = screen.getByTestId('view-action');
- expect(viewProvisioned).toBeInTheDocument();
- expect(viewProvisioned).toBeEnabled();
+ // Two contact points should have view buttons: grafana-default-email (cannot be edited) and provisioned-contact-point (provisioned)
+ const viewButtons = screen.getAllByRole('link', { name: /^view$/i });
+ expect(viewButtons).toHaveLength(2);
+
+ // Check view buttons by their href to verify which contact points they belong to
+ // The url is the same but the form should be readonly
+ expect(viewButtons[0]).toHaveAttribute('href', '/alerting/notifications/receivers/grafana-default-email/edit');
+ expect(viewButtons[1]).toHaveAttribute(
+ 'href',
+ '/alerting/notifications/receivers/provisioned-contact-point/edit'
+ );
+
+ viewButtons.forEach((button) => {
+ expect(button).toBeEnabled();
+ });
+
+ // Three contact points should have edit buttons: lotsa-emails, Slack with multiple channels, OnCall Contact point
+ const editButtons = screen.getAllByRole('link', { name: /^edit$/i });
+ expect(editButtons).toHaveLength(3);
+
+ // Check edit buttons by their href to verify which contact points they belong to
+ expect(editButtons[0]).toHaveAttribute('href', '/alerting/notifications/receivers/lotsa-emails/edit');
+ expect(editButtons[1]).toHaveAttribute(
+ 'href',
+ '/alerting/notifications/receivers/OnCall%20Conctact%20point/edit'
+ );
+ expect(editButtons[2]).toHaveAttribute(
+ 'href',
+ '/alerting/notifications/receivers/Slack%20with%20multiple%20channels/edit'
+ );
- const editButtons = screen.getAllByTestId('edit-action');
- expect(editButtons).toHaveLength(4);
editButtons.forEach((button) => {
expect(button).toBeEnabled();
});
@@ -227,11 +252,11 @@ describe('contact points', () => {
expect(screen.getByRole('link', { name: 'add contact point' })).toHaveAttribute('aria-disabled', 'true');
// edit permission is based on API response - we should have 3 buttons
- const editButtons = await screen.findAllByTestId('edit-action');
+ const editButtons = await screen.findAllByRole('link', { name: /^edit$/i });
expect(editButtons).toHaveLength(3);
// there should be view buttons though - one for provisioned, and one for the un-editable contact point
- const viewButtons = screen.getAllByTestId('view-action');
+ const viewButtons = screen.getAllByRole('link', { name: /^view$/i });
expect(viewButtons).toHaveLength(2);
// check buttons in Notification Templates
@@ -329,7 +354,18 @@ describe('contact points', () => {
},
];
- const { user } = renderWithProvider(
-
);
}
+ if (querySourceUIDs.length > 0) {
+ metadata.push(
-
);
}
+ if (querySourceUIDs.length > 0) {
+ metadata.push(
-
+ {dataSources.map((dataSource) => {
+ return (
+
+
+ );
+ })}
+
+ );
+});
+
function RuleLabels({ labels }: { labels: Labels }) {
const styles = useStyles2(getStyles);
@@ -368,38 +425,6 @@ export const UnknownRuleListItem = ({ ruleName, groupIdentifier, ruleDefinition
);
};
-interface RuleLocationProps {
- namespace: string;
- group: string;
- rulesSource?: RulesSourceIdentifier;
- application?: RulesSourceApplication;
-}
-
-// @TODO make the datasource / namespace / group click-able to allow further filtering of the list
-export const RuleLocation = ({ namespace, group, rulesSource, application }: RuleLocationProps) => {
- const isGrafanaApp = application === 'grafana';
- const isDataSourceApp = !!rulesSource && !!application && !isGrafanaApp;
-
- return (
-
- {isGrafanaApp &&
-
-
- )}
-
-
- {namespace}
-
-
- );
-};
-
const getStyles = (theme: GrafanaTheme2) => ({
alertListItemContainer: css({
position: 'relative',
@@ -426,3 +451,33 @@ export type RuleListItemCommonProps = Pick<
AlertRuleListItemProps,
Extract
>;
+
+interface DataSourceLogoProps {
+ dataSource: DataSourceInstanceSettings;
+}
+
+const DataSourceLogo = forwardRef(({ dataSource }, ref) => {
+ const styles = useStyles2(dataSourceLogoStyles);
+
+ return (
+ 
+ );
+});
+
+const dataSourceLogoStyles = (theme: GrafanaTheme2) => ({
+ logo: css({
+ height: '14px',
+ width: '14px',
+ borderRadius: theme.shape.radius.default,
+ }),
+ filter: css({
+ filter: `invert(${theme.isLight ? 1 : 0})`,
+ }),
+});
diff --git a/public/app/features/alerting/unified/rule-list/components/ListItem.tsx b/public/app/features/alerting/unified/rule-list/components/ListItem.tsx
index ce3ea7f6fb6..0e15e5fb696 100644
--- a/public/app/features/alerting/unified/rule-list/components/ListItem.tsx
+++ b/public/app/features/alerting/unified/rule-list/components/ListItem.tsx
@@ -39,7 +39,7 @@ export const ListItem = (props: ListItemProps) => {
{/* metadata */}
-
+
{meta?.map((item, index) => (
{index > 0 &&
- {'·'}
+ {'|'}
);
diff --git a/public/app/features/alerting/unified/rule-list/components/RuleLocation.tsx b/public/app/features/alerting/unified/rule-list/components/RuleLocation.tsx
new file mode 100644
index 00000000000..b0d4b31de5d
--- /dev/null
+++ b/public/app/features/alerting/unified/rule-list/components/RuleLocation.tsx
@@ -0,0 +1,43 @@
+import { Icon, Stack, TextLink, Tooltip } from '@grafana/ui';
+import { RulesSourceIdentifier } from 'app/types/unified-alerting';
+import { RulesSourceApplication } from 'app/types/unified-alerting-dto';
+
+import { DataSourceIcon } from './Namespace';
+
+interface RuleLocationProps {
+ namespace: string;
+ group: string;
+ groupUrl?: string;
+ rulesSource?: RulesSourceIdentifier;
+ application?: RulesSourceApplication;
+}
+
+export function RuleLocation({ namespace, group, groupUrl, rulesSource, application }: RuleLocationProps) {
+ const isGrafanaApp = application === 'grafana';
+ const isDataSourceApp = !!rulesSource && !!application && !isGrafanaApp;
+
+ return (
+
+ {isGrafanaApp &&
+
+
+ )}
+
+
+ {namespace}
+
+ {group}
+
+ ) : (
+ group
+ )}
+
+
+ );
+}
diff --git a/public/app/features/alerting/unified/rule-list/hooks/filters.ts b/public/app/features/alerting/unified/rule-list/hooks/filters.ts
index 3602081e7c0..2e9535dd97f 100644
--- a/public/app/features/alerting/unified/rule-list/hooks/filters.ts
+++ b/public/app/features/alerting/unified/rule-list/hooks/filters.ts
@@ -22,12 +22,10 @@ export function groupFilter(
const { name, file } = group;
const { namespace, groupName } = filterState;
- // Use fuzzy search for namespace
if (namespace && !fuzzyMatches(file, namespace)) {
return false;
}
- // Use fuzzy search for group name
if (groupName && !fuzzyMatches(name, groupName)) {
return false;
}
@@ -41,17 +39,17 @@ export function groupFilter(
export function ruleFilter(rule: PromRuleDTO, filterState: RulesFilter) {
const { name, labels = {}, health, type } = rule;
- // Free form words filter (uses fuzzy matching for each word)
- if (filterState.freeFormWords.length > 0 && !filterState.freeFormWords.some((word) => fuzzyMatches(name, word))) {
- return false;
+ if (filterState.freeFormWords.length > 0) {
+ const nameMatches = fuzzyMatches(name, filterState.freeFormWords.join(' '));
+ if (!nameMatches) {
+ return false;
+ }
}
- // Rule name filter (uses fuzzy matching)
if (filterState.ruleName && !fuzzyMatches(name, filterState.ruleName)) {
return false;
}
- // Labels filter
if (filterState.labels.length > 0) {
const matchers = compact(filterState.labels.map(looseParseMatcher));
const doRuleLabelsMatchQuery = matchers.length > 0 && labelsMatchMatchers(labels, matchers);
@@ -68,12 +66,10 @@ export function ruleFilter(rule: PromRuleDTO, filterState: RulesFilter) {
}
}
- // Rule type filter
if (filterState.ruleType && type !== filterState.ruleType) {
return false;
}
- // Rule state filter (for alerting rules only)
if (filterState.ruleState) {
if (!prometheusRuleType.alertingRule(rule)) {
return false;
@@ -83,7 +79,6 @@ export function ruleFilter(rule: PromRuleDTO, filterState: RulesFilter) {
}
}
- // Rule health filter
if (filterState.ruleHealth && health !== filterState.ruleHealth) {
return false;
}
@@ -102,7 +97,6 @@ export function ruleFilter(rule: PromRuleDTO, filterState: RulesFilter) {
}
}
- // Dashboard UID filter
if (filterState.dashboardUid) {
if (!prometheusRuleType.alertingRule(rule)) {
return false;
diff --git a/public/app/features/browse-dashboards/BrowseDashboardsPage.test.tsx b/public/app/features/browse-dashboards/BrowseDashboardsPage.test.tsx
index 4a2c9f4a82b..a9a0dda6274 100644
--- a/public/app/features/browse-dashboards/BrowseDashboardsPage.test.tsx
+++ b/public/app/features/browse-dashboards/BrowseDashboardsPage.test.tsx
@@ -1,7 +1,6 @@
import { render as rtlRender, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { HttpResponse, http } from 'msw';
-import { setupServer, SetupServer } from 'msw/node';
import { ComponentProps } from 'react';
import * as React from 'react';
import { useParams } from 'react-router-dom-v5-compat';
@@ -9,13 +8,16 @@ import AutoSizer from 'react-virtualized-auto-sizer';
import { TestProvider } from 'test/helpers/TestProvider';
import { selectors } from '@grafana/e2e-selectors';
+import server, { setupMockServer } from '@grafana/test-utils/server';
+import { getFolderFixtures } from '@grafana/test-utils/unstable';
import { contextSrv } from 'app/core/core';
import { backendSrv } from 'app/core/services/backend_srv';
import BrowseDashboardsPage from './BrowseDashboardsPage';
-import { wellFormedTree } from './fixtures/dashboardsTreeItem.fixture';
import * as permissions from './permissions';
-const [mockTree, { dashbdD, folderA, folderA_folderA }] = wellFormedTree();
+
+setupMockServer();
+const [mockTree, { dashbdD, folderA, folderA_folderA }] = getFolderFixtures();
jest.mock('@grafana/runtime', () => ({
...jest.requireActual('@grafana/runtime'),
@@ -111,7 +113,6 @@ jest.mock('app/features/browse-dashboards/api/services', () => {
});
describe('browse-dashboards BrowseDashboardsPage', () => {
- let server: SetupServer;
const mockPermissions = {
canCreateDashboards: true,
canEditDashboards: true,
@@ -123,33 +124,14 @@ describe('browse-dashboards BrowseDashboardsPage', () => {
canDeleteDashboards: true,
};
- beforeAll(() => {
- server = setupServer(
- http.get('/api/folders/:uid', () => {
- return HttpResponse.json({
- title: folderA.item.title,
- uid: folderA.item.uid,
- });
- }),
- http.get('/api/search', () => {
- return HttpResponse.json({});
- }),
+ beforeEach(() => {
+ server.use(
http.get('/api/search/sorting', () => {
return HttpResponse.json({
sortOptions: [],
});
- }),
- http.get('/apis/provisioning.grafana.app/v0alpha1/namespaces/default/settings', () => {
- return HttpResponse.json({
- items: [],
- });
})
);
- server.listen();
- });
-
- afterAll(() => {
- server.close();
});
beforeEach(() => {
@@ -170,7 +152,6 @@ describe('browse-dashboards BrowseDashboardsPage', () => {
canDeleteDashboards: true,
});
jest.restoreAllMocks();
- server.resetHandlers();
});
describe('at the root level', () => {
diff --git a/public/app/features/browse-dashboards/BrowseFolderLibraryPanelsPage.test.tsx b/public/app/features/browse-dashboards/BrowseFolderLibraryPanelsPage.test.tsx
index c60a8a8de56..9b96316add6 100644
--- a/public/app/features/browse-dashboards/BrowseFolderLibraryPanelsPage.test.tsx
+++ b/public/app/features/browse-dashboards/BrowseFolderLibraryPanelsPage.test.tsx
@@ -1,9 +1,9 @@
-import { render as rtlRender, screen } from '@testing-library/react';
import { http, HttpResponse } from 'msw';
-import { SetupServer, setupServer } from 'msw/node';
import { useParams } from 'react-router-dom-v5-compat';
-import { TestProvider } from 'test/helpers/TestProvider';
+import { render, screen } from 'test/test-utils';
+import server, { setupMockServer } from '@grafana/test-utils/server';
+import { getFolderFixtures } from '@grafana/test-utils/unstable';
import { contextSrv } from 'app/core/core';
import { backendSrv } from 'app/core/services/backend_srv';
@@ -11,10 +11,7 @@ import BrowseFolderLibraryPanelsPage from './BrowseFolderLibraryPanelsPage';
import { getLibraryElementsResponse } from './fixtures/libraryElements.fixture';
import * as permissions from './permissions';
-function render(...[ui, options]: Parameters) {
- rtlRender({ui} , options);
-}
-
+setupMockServer();
jest.mock('@grafana/runtime', () => ({
...jest.requireActual('@grafana/runtime'),
getBackendSrv: () => backendSrv,
@@ -28,15 +25,15 @@ jest.mock('react-router-dom-v5-compat', () => ({
useParams: jest.fn(),
}));
-const mockFolderName = 'myFolder';
-const mockFolderUid = '12345';
+const [_, { folderA }] = getFolderFixtures();
+const mockFolderName = folderA.item.title;
+const mockFolderUid = folderA.item.uid;
const mockLibraryElementsResponse = getLibraryElementsResponse(1, {
folderUid: mockFolderUid,
});
describe('browse-dashboards BrowseFolderLibraryPanelsPage', () => {
(useParams as jest.Mock).mockReturnValue({ uid: mockFolderUid });
- let server: SetupServer;
const mockPermissions = {
canCreateDashboards: true,
canEditDashboards: true,
@@ -48,14 +45,8 @@ describe('browse-dashboards BrowseFolderLibraryPanelsPage', () => {
canDeleteDashboards: true,
};
- beforeAll(() => {
- server = setupServer(
- http.get('/api/folders/:uid', () => {
- return HttpResponse.json({
- title: mockFolderName,
- uid: mockFolderUid,
- });
- }),
+ beforeEach(() => {
+ server.use(
http.get('/api/library-elements', () => {
return HttpResponse.json({
result: mockLibraryElementsResponse,
@@ -65,11 +56,6 @@ describe('browse-dashboards BrowseFolderLibraryPanelsPage', () => {
return HttpResponse.json({});
})
);
- server.listen();
- });
-
- afterAll(() => {
- server.close();
});
beforeEach(() => {
@@ -79,7 +65,6 @@ describe('browse-dashboards BrowseFolderLibraryPanelsPage', () => {
afterEach(() => {
jest.restoreAllMocks();
- server.resetHandlers();
});
it('displays the folder title', async () => {
diff --git a/public/app/features/browse-dashboards/components/BrowseActions/MoveModal.test.tsx b/public/app/features/browse-dashboards/components/BrowseActions/MoveModal.test.tsx
index fb1a160d8b1..c941cd76150 100644
--- a/public/app/features/browse-dashboards/components/BrowseActions/MoveModal.test.tsx
+++ b/public/app/features/browse-dashboards/components/BrowseActions/MoveModal.test.tsx
@@ -1,38 +1,26 @@
-import userEvent from '@testing-library/user-event';
import { HttpResponse, http } from 'msw';
-import { SetupServer, setupServer } from 'msw/node';
import { render, screen } from 'test/test-utils';
+import { setBackendSrv } from '@grafana/runtime';
+import server, { setupMockServer } from '@grafana/test-utils/server';
+import { getFolderFixtures } from '@grafana/test-utils/unstable';
import { backendSrv } from 'app/core/services/backend_srv';
-import { treeViewersCanEdit, wellFormedTree } from '../../fixtures/dashboardsTreeItem.fixture';
-
import { MoveModal, Props } from './MoveModal';
-const [mockTree, { folderA }] = wellFormedTree();
-const [mockTreeThatViewersCanEdit /* shares folders with wellFormedTree */] = treeViewersCanEdit();
+const [_, { folderA }] = getFolderFixtures();
-jest.mock('@grafana/runtime', () => ({
- ...jest.requireActual('@grafana/runtime'),
- getBackendSrv: () => backendSrv,
-}));
+setBackendSrv(backendSrv);
+setupMockServer();
describe('browse-dashboards MoveModal', () => {
const mockOnDismiss = jest.fn();
const mockOnConfirm = jest.fn();
let props: Props;
- let server: SetupServer;
window.HTMLElement.prototype.scrollIntoView = () => {};
- beforeAll(() => {
- server = setupServer(
- http.get('/api/folders/:uid', () => {
- return HttpResponse.json({
- title: folderA.item.title,
- uid: folderA.item.uid,
- });
- }),
-
+ beforeEach(() => {
+ server.use(
http.get('/api/folders/:uid/counts', () => {
return HttpResponse.json({
folder: 1,
@@ -40,43 +28,9 @@ describe('browse-dashboards MoveModal', () => {
librarypanel: 3,
alertrule: 4,
});
- }),
-
- http.get('/apis/provisioning.grafana.app/v0alpha1/namespaces/default/settings', () => {
- return HttpResponse.json({
- items: [],
- });
- }),
-
- http.get('/api/folders', ({ request }) => {
- const url = new URL(request.url);
- const parentUid = url.searchParams.get('parentUid') ?? undefined;
- const permission = url.searchParams.get('permission');
-
- const limit = parseInt(url.searchParams.get('limit') ?? '1000', 10);
- const page = parseInt(url.searchParams.get('page') ?? '1', 10);
-
- const tree = permission === 'Edit' ? mockTreeThatViewersCanEdit : mockTree;
-
- // reconstruct a folder API response from the flat tree fixture
- const folders = tree
- .filter((v) => v.item.kind === 'folder' && v.item.parentUID === parentUid)
- .map((folder) => {
- return {
- uid: folder.item.uid,
- title: folder.item.kind === 'folder' ? folder.item.title : "invalid - this shouldn't happen",
- };
- })
- .slice(limit * (page - 1), limit * page);
-
- return HttpResponse.json(folders);
})
);
- server.listen();
- });
-
- beforeEach(() => {
props = {
isOpen: true,
onConfirm: mockOnConfirm,
@@ -90,10 +44,6 @@ describe('browse-dashboards MoveModal', () => {
};
});
- afterAll(() => {
- server.close();
- });
-
it('renders a dialog with the correct title', async () => {
render() {
rtlRender({ui} , options);
diff --git a/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.test.tsx b/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.test.tsx
index 00687677e94..a3ddd150785 100644
--- a/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.test.tsx
+++ b/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.test.tsx
@@ -49,6 +49,26 @@ jest.mock('app/features/dashboard-scene/components/Provisioned/ResourceEditFormS
ResourceEditFormSharedFields: () => ,
}));
+const MOCK_DATA = {
+ repository: {
+ name: 'test-repo',
+ namespace: 'default',
+ title: 'Test Repository',
+ type: 'git',
+ },
+ resource: {
+ type: {
+ kind: 'Folder',
+ },
+ upsert: {
+ apiVersion: 'v1',
+ kind: 'Folder',
+ metadata: { name: 'test-folder', uid: 'test-folder-uid' },
+ spec: { title: 'Test Folder' },
+ },
+ },
+};
+
const mockUseDeleteRepositoryFilesMutation = useDeleteRepositoryFilesWithPathMutation as jest.MockedFunction<
typeof useDeleteRepositoryFilesWithPathMutation
>;
@@ -267,7 +287,13 @@ describe('DeleteProvisionedFolderForm', () => {
describe('success handling', () => {
it('should navigate to parent folder on successful write workflow', async () => {
- const successState = { isLoading: false, isSuccess: true, isError: false, error: null };
+ const successState = {
+ isLoading: false,
+ isSuccess: true,
+ isError: false,
+ error: null,
+ data: MOCK_DATA,
+ };
setup({}, defaultHookData, successState);
await waitFor(() => {
@@ -277,7 +303,13 @@ describe('DeleteProvisionedFolderForm', () => {
it('should navigate to dashboards root when parent folder has no parentUid', async () => {
const folderWithoutParent = { ...mockParentFolder, parentUid: undefined };
- const successState = { isLoading: false, isSuccess: true, isError: false, error: null };
+ const successState = {
+ isLoading: false,
+ isSuccess: true,
+ isError: false,
+ error: null,
+ data: MOCK_DATA,
+ };
setup({ parentFolder: folderWithoutParent }, defaultHookData, successState);
await waitFor(() => {
@@ -292,13 +324,19 @@ describe('DeleteProvisionedFolderForm', () => {
isSuccess: true,
isError: false,
error: null,
- data: { urls: { newPullRequestURL: 'https://github.com/test/repo/pull/new' } },
+ data: {
+ ...MOCK_DATA,
+ ref: 'feature-branch',
+ path: 'folders/test-folder.json',
+ urls: { newPullRequestURL: 'https://github.com/test/repo/pull/new' },
+ },
};
const { mockNavigate } = setup({}, { ...defaultHookData, initialValues: branchFormData }, successState);
await waitFor(() => {
const expectedParams = new URLSearchParams();
expectedParams.set('new_pull_request_url', 'https://github.com/test/repo/pull/new');
+ expectedParams.set('repo_type', 'git');
const expectedUrl = `/dashboards?${expectedParams.toString()}`;
expect(mockNavigate).toHaveBeenCalledWith(expectedUrl);
diff --git a/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.tsx b/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.tsx
index 70638377a3a..f6fb26101f8 100644
--- a/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.tsx
+++ b/public/app/features/browse-dashboards/components/DeleteProvisionedFolderForm.tsx
@@ -1,4 +1,3 @@
-import { useEffect } from 'react';
import { FormProvider, useForm } from 'react-hook-form';
import { useNavigate } from 'react-router-dom-v5-compat';
@@ -12,6 +11,10 @@ import { AnnoKeySourcePath } from 'app/features/apiserver/types';
import { ResourceEditFormSharedFields } from 'app/features/dashboard-scene/components/Provisioned/ResourceEditFormSharedFields';
import { BaseProvisionedFormData } from 'app/features/dashboard-scene/saving/shared';
import { buildResourceBranchRedirectUrl } from 'app/features/dashboard-scene/settings/utils';
+import {
+ useProvisionedRequestHandler,
+ ProvisionedOperationInfo,
+} from 'app/features/dashboard-scene/utils/useProvisionedRequestHandler';
import { FolderDTO } from 'app/types/folders';
import { useProvisionedFolderFormData } from '../hooks/useProvisionedFolderFormData';
@@ -57,50 +60,51 @@ function FormContent({ initialValues, parentFolder, repository, workflowOptions,
});
};
- // TODO: move to a hook if this useEffect shared mostly the same logic as in NewProvisionedFolderForm
- useEffect(() => {
- if (request.isSuccess && repository) {
- const prUrl = request.data?.urls?.newPullRequestURL;
- if (workflow === 'branch' && prUrl) {
- const url = buildResourceBranchRedirectUrl({
- paramName: 'new_pull_request_url',
- paramValue: prUrl,
- repoType: request.data?.repository?.type,
- });
- navigate(url);
- return;
- }
-
- if (workflow === 'write') {
- getAppEvents().publish({
- type: AppEvents.alertSuccess.name,
- payload: [
- t(
- 'browse-dashboards.delete-provisioned-folder-form.alert-folder-deleted-successfully',
- 'Folder deleted successfully'
- ),
- ],
- });
- // Navigate back to parent folder if it exists, otherwise go to dashboards root
- if (parentFolder?.parentUid) {
- window.location.href = getFolderURL(parentFolder.parentUid);
- } else {
- window.location.href = '/dashboards';
- }
- }
- }
-
- if (request.isError) {
- getAppEvents().publish({
- type: AppEvents.alertError.name,
- payload: [
- t('browse-dashboards.delete-provisioned-folder-form.api-error', 'Failed to delete folder'),
- request.error,
- ],
+ const onBranchSuccess = ({ urls }: { urls?: Record }, info: ProvisionedOperationInfo) => {
+ const prUrl = urls?.newPullRequestURL;
+ if (prUrl) {
+ const url = buildResourceBranchRedirectUrl({
+ paramName: 'new_pull_request_url',
+ paramValue: prUrl,
+ repoType: info.repoType,
});
- return;
+ navigate(url);
}
- }, [request, repository, workflow, parentFolder, navigate]);
+ };
+
+ const onWriteSuccess = () => {
+ // Navigate back to parent folder if it exists, otherwise go to dashboards root
+ if (parentFolder?.parentUid) {
+ window.location.href = getFolderURL(parentFolder.parentUid);
+ } else {
+ window.location.href = '/dashboards';
+ }
+ };
+
+ const onError = (error: unknown) => {
+ getAppEvents().publish({
+ type: AppEvents.alertError.name,
+ payload: [t('browse-dashboards.delete-provisioned-folder-form.api-error', 'Failed to delete folder'), error],
+ });
+ };
+
+ // Use the repository-type and resource-type aware provisioned request handler
+ useProvisionedRequestHandler({
+ request,
+ workflow,
+ successMessage: t(
+ 'browse-dashboards.delete-provisioned-folder-form.success-message',
+ 'Folder deleted successfully'
+ ),
+ resourceType: 'folder',
+ repository,
+ handlers: {
+ onDismiss,
+ onBranchSuccess,
+ onWriteSuccess,
+ onError,
+ },
+ });
return (
diff --git a/public/app/features/browse-dashboards/components/NewProvisionedFolderForm.tsx b/public/app/features/browse-dashboards/components/NewProvisionedFolderForm.tsx
index a453490e07c..797f4bb98ba 100644
--- a/public/app/features/browse-dashboards/components/NewProvisionedFolderForm.tsx
+++ b/public/app/features/browse-dashboards/components/NewProvisionedFolderForm.tsx
@@ -1,5 +1,4 @@
import { css } from '@emotion/css';
-import { useEffect } from 'react';
import { FormProvider, useForm } from 'react-hook-form';
import { useNavigate } from 'react-router-dom-v5-compat';
@@ -13,6 +12,10 @@ import { AnnoKeySourcePath, Resource } from 'app/features/apiserver/types';
import { ResourceEditFormSharedFields } from 'app/features/dashboard-scene/components/Provisioned/ResourceEditFormSharedFields';
import { BaseProvisionedFormData } from 'app/features/dashboard-scene/saving/shared';
import { buildResourceBranchRedirectUrl } from 'app/features/dashboard-scene/settings/utils';
+import {
+ useProvisionedRequestHandler,
+ ProvisionedOperationInfo,
+} from 'app/features/dashboard-scene/utils/useProvisionedRequestHandler';
import { PROVISIONING_URL } from 'app/features/provisioning/constants';
import { usePullRequestParam } from 'app/features/provisioning/hooks/usePullRequestParam';
import { FolderDTO } from 'app/types/folders';
@@ -44,58 +47,60 @@ function FormContent({ initialValues, repository, workflowOptions, folder, onDis
});
const { handleSubmit, watch, register, formState } = methods;
- const [workflow, ref, title] = watch(['workflow', 'ref', 'title']);
+ const [workflow, title] = watch(['workflow', 'title']);
- // TODO: replace with useProvisionedRequestHandler hook
- useEffect(() => {
- const appEvents = getAppEvents();
- if (request.isSuccess && repository) {
- onDismiss?.();
-
- appEvents.publish({
- type: AppEvents.alertSuccess.name,
- payload: [
- t(
- 'browse-dashboards.new-provisioned-folder-form.alert-folder-created-successfully',
- 'Folder created successfully'
- ),
- ],
+ const onBranchSuccess = ({ urls }: { urls?: Record }, info: ProvisionedOperationInfo) => {
+ const prUrl = urls?.newPullRequestURL;
+ if (prUrl) {
+ const url = buildResourceBranchRedirectUrl({
+ paramName: 'new_pull_request_url',
+ paramValue: prUrl,
+ repoType: info.repoType,
});
+ navigate(url);
+ }
+ };
- const prUrl = request.data?.urls?.newPullRequestURL;
- if (workflow === 'branch' && prUrl) {
- const url = buildResourceBranchRedirectUrl({
- paramName: 'new_pull_request_url',
- paramValue: prUrl,
- repoType: request.data?.repository?.type,
- });
- navigate(url);
- return;
- }
-
- // TODO: Update when the upsert type is fixed
- // eslint-disable-next-line @typescript-eslint/consistent-type-assertions
- const folder = request.data.resource?.upsert as Resource;
- if (folder?.metadata?.name) {
- navigate(`/dashboards/f/${folder?.metadata?.name}/`);
- return;
- }
+ const onWriteSuccess = (resource: Resource) => {
+ // Navigation for new folders (resource-specific concern)
+ if (resource?.metadata?.name) {
+ navigate(`/dashboards/f/${resource.metadata.name}/`);
+ return;
+ }
+ // Fallback to provisioning URL
+ if (repository?.name && request.data?.path) {
let url = `${PROVISIONING_URL}/${repository.name}/file/${request.data.path}`;
if (request.data.ref?.length) {
url += '?ref=' + request.data.ref;
}
navigate(url);
- } else if (request.isError) {
- appEvents.publish({
- type: AppEvents.alertError.name,
- payload: [
- t('browse-dashboards.new-provisioned-folder-form.alert-error-creating-folder', 'Error creating folder'),
- request.error,
- ],
- });
}
- }, [request.isSuccess, request.isError, request.error, ref, request.data, workflow, navigate, repository, onDismiss]);
+ };
+
+ const onError = (error: unknown) => {
+ getAppEvents().publish({
+ type: AppEvents.alertError.name,
+ payload: [
+ t('browse-dashboards.new-provisioned-folder-form.alert-error-creating-folder', 'Error creating folder'),
+ error,
+ ],
+ });
+ };
+
+ // Use the repository-type and resource-type aware provisioned request handler
+ useProvisionedRequestHandler({
+ request,
+ workflow,
+ repository,
+ resourceType: 'folder',
+ handlers: {
+ onDismiss,
+ onBranchSuccess,
+ onWriteSuccess: (_, resource) => onWriteSuccess(resource),
+ onError,
+ },
+ });
const doSave = async ({ ref, title, workflow, comment }: BaseProvisionedFormData) => {
const repoName = repository?.name;
diff --git a/public/app/features/browse-dashboards/fixtures/dashboardsTreeItem.fixture.ts b/public/app/features/browse-dashboards/fixtures/dashboardsTreeItem.fixture.ts
index 4ac6515c726..7b5d17a2825 100644
--- a/public/app/features/browse-dashboards/fixtures/dashboardsTreeItem.fixture.ts
+++ b/public/app/features/browse-dashboards/fixtures/dashboardsTreeItem.fixture.ts
@@ -1,5 +1,6 @@
import { Chance } from 'chance';
+import { getFolderFixtures } from '@grafana/test-utils/unstable';
import { DashboardViewItem } from 'app/features/search/types';
import { DashboardsTreeItem, UIDashboardViewItem } from '../types';
@@ -74,7 +75,7 @@ export function sharedWithMeFolder(seed = 1): DashboardsTreeItem {
- const onZoom = (zoomPanPinchRef: ReactZoomPanPinchRef) => {
- const scale = zoomPanPinchRef.state.scale;
- scene.scale = scale;
-
- if (scene.shouldInfinitePan) {
- const isScaleZoomedOut = scale < 1;
-
- if (isScaleZoomedOut) {
- scene.updateSize(scene.width / scale, scene.height / scale);
- scene.panel.forceUpdate();
- }
- }
- };
-
- const onZoomStop = (zoomPanPinchRef: ReactZoomPanPinchRef) => {
- const scale = zoomPanPinchRef.state.scale;
- scene.scale = scale;
- updateMoveable(scale);
- };
-
- const onTransformed = (
- _: ReactZoomPanPinchRef,
- state: {
- scale: number;
- positionX: number;
- positionY: number;
- }
- ) => {
- const scale = state.scale;
- scene.scale = scale;
- updateMoveable(scale);
- };
-
- const updateMoveable = (scale: number) => {
- if (scene.moveable && scale > 0) {
- scene.moveable.zoom = 1 / scale;
- if (scale === 1) {
- scene.moveable.snappable = true;
- } else {
- scene.moveable.snappable = false;
- }
- }
- };
-
- const onPanning = (_: ReactZoomPanPinchRef, event: MouseEvent | TouchEvent) => {
- if (scene.shouldInfinitePan && event instanceof MouseEvent) {
- // Get deltaX and deltaY from pan event and add it to current canvas dimensions
- let deltaX = event.movementX;
- let deltaY = event.movementY;
- if (deltaX > 0) {
- deltaX = 0;
- }
- if (deltaY > 0) {
- deltaY = 0;
- }
-
- // TODO: Consider bounding to the scene elements instead of allowing "infinite" panning
- // TODO: Consider making scene grow in all directions vs just down to the right / bottom
- scene.updateSize(scene.width - deltaX, scene.height - deltaY);
- scene.panel.forceUpdate();
- }
- };
-
- const onSceneContainerMouseDown = (e: React.MouseEvent) => {
- // If pan and zoom is disabled or context menu is visible, don't pan
- if ((!scene.shouldPanZoom || scene.contextMenuVisible) && (e.button === 1 || (e.button === 2 && e.ctrlKey))) {
- e.preventDefault();
- e.stopPropagation();
- }
-
- // If context menu is hidden, ignore left mouse or non-ctrl right mouse for pan
- if (!scene.contextMenuVisible && !scene.isPanelEditing && e.button === 2 && !e.ctrlKey) {
- e.preventDefault();
- e.stopPropagation();
- }
- };
-
- // Set panel content overflow to hidden to prevent canvas content from overflowing
- scene.div?.parentElement?.parentElement?.parentElement?.parentElement?.setAttribute('style', `overflow: hidden`);
-
- return (
-
-
- {/* The ();
- transformComponentRef: RefObject | undefined;
constructor(
- cfg: CanvasFrameOptions,
- enableEditing: boolean,
- showAdvancedTypes: boolean,
- panZoom: boolean,
- infinitePan: boolean,
+ options: Options,
public onSave: (cfg: CanvasFrameOptions) => void,
panel: CanvasPanel
) {
- this.root = this.load(cfg, enableEditing, showAdvancedTypes, panZoom, infinitePan);
+ // TODO: Will need to update this approach for dashboard scenes
+ // migration (new dashboard edit experience)
+ const dashboard = getDashboardSrv().getCurrent();
+ const enableEditing = options.inlineEditing && dashboard?.editable;
+
+ this.root = this.load(options, enableEditing);
this.subscription = this.editModeEnabled.subscribe((open) => {
if (!this.moveable || !this.isEditingEnabled) {
@@ -124,8 +125,7 @@ export class Scene {
});
this.panel = panel;
- this.connections = new Connections(this);
- this.transformComponentRef = createRef();
+ this.connections = config.featureToggles.canvasPanelPanZoom ? new Connections2(this) : new Connections(this);
}
getNextElementName = (isFrame = false) => {
@@ -147,15 +147,12 @@ export class Scene {
return !this.byName.has(v);
};
- load(
- cfg: CanvasFrameOptions,
- enableEditing: boolean,
- showAdvancedTypes: boolean,
- panZoom: boolean,
- infinitePan: boolean
- ) {
+ load(options: Options, enableEditing: boolean) {
+ const { root, showAdvancedTypes, panZoom, zoomToContent, tooltip } = options;
+ const tooltipMode = tooltip?.mode ?? TooltipDisplayMode.Single;
+
this.root = new RootElement(
- cfg ?? {
+ root ?? {
type: 'frame',
elements: [DEFAULT_CANVAS_ELEMENT_CONFIG],
},
@@ -166,17 +163,39 @@ export class Scene {
this.isEditingEnabled = enableEditing;
this.shouldShowAdvancedTypes = showAdvancedTypes;
this.shouldPanZoom = panZoom;
- this.shouldInfinitePan = infinitePan;
+ this.zoomToContent = zoomToContent;
+ this.tooltipMode = tooltipMode;
setTimeout(() => {
- if (this.div) {
- // If editing is enabled, clear selecto instance
- const destroySelecto = enableEditing;
- initMoveable(destroySelecto, enableEditing, this);
- this.currentLayer = this.root;
- this.selection.next([]);
- this.connections.select(undefined);
- this.connections.updateState();
+ if (config.featureToggles.canvasPanelPanZoom) {
+ if (this.viewportDiv && this.viewerDiv) {
+ if (!this.shouldPanZoom) {
+ this.scale = 1;
+ this.scrollLeft = 0;
+ this.scrollTop = 0;
+ }
+
+ // If editing is enabled, clear selecto instance
+ const destroySelecto = enableEditing;
+ initMoveable(destroySelecto, enableEditing, this);
+ this.currentLayer = this.root;
+ this.selection.next([]);
+ this.connections.select(undefined);
+ this.connections.updateState();
+ // update initial connections svg size
+ this.updateConnectionsSize();
+ this.fitContent(this, zoomToContent);
+ }
+ } else {
+ if (this.div) {
+ // If editing is enabled, clear selecto instance
+ const destroySelecto = enableEditing;
+ initMoveable(destroySelecto, enableEditing, this);
+ this.currentLayer = this.root;
+ this.selection.next([]);
+ this.connections.select(undefined);
+ this.connections.updateState();
+ }
}
});
return this.root;
@@ -204,12 +223,53 @@ export class Scene {
if (this.selecto?.getSelectedTargets().length) {
this.clearCurrentSelection();
}
+
+ if (config.featureToggles.canvasPanelPanZoom) {
+ this.updateConnectionsSize();
+ this.fitContent(this, this.zoomToContent!);
+
+ // TODO: This is a workaround to apply styles to the elements after the size update.
+ // It's a good to go approach used by movable creator, but maybe we can find a better way.
+ this.root.elements.forEach((el) => {
+ el.applyLayoutStylesToDiv(false);
+ });
+ // TODO: This is a workaround to apply styles to the elements after the size update.
+ // Remove this after dealing with the connection anchors stacking context issue.
+ if (this.connections.connectionAnchorDiv) {
+ this.connections.connectionAnchorDiv.style.display = 'none';
+ }
+ }
+ }
+
+ updateConnectionsSize() {
+ const svgConnections = this.connections.connectionsSVG;
+
+ if (svgConnections) {
+ const scale = this.infiniteViewer!.getZoom();
+ // NOTE: sometimes getScrollLeft and getScrollTop return NaN,
+ // so we use || 0 to ensure we have a valid number
+ const left = this.infiniteViewer!.getScrollLeft() || 0;
+ const top = this.infiniteViewer!.getScrollTop() || 0;
+ const width = this.width;
+ const height = this.height;
+
+ svgConnections.style.left = `${left}px`;
+ svgConnections.style.top = `${top}px`;
+ svgConnections.style.width = `${width / scale}px`;
+ svgConnections.style.height = `${height / scale}px`;
+
+ svgConnections.setAttribute('viewBox', `${left} ${top} ${width / scale} ${height / scale}`);
+ }
}
clearCurrentSelection(skipNextSelectionBroadcast = false) {
this.skipNextSelectionBroadcast = skipNextSelectionBroadcast;
let event: MouseEvent = new MouseEvent('click');
- this.selecto?.clickTarget(event, this.div);
+ if (config.featureToggles.canvasPanelPanZoom) {
+ this.selecto?.clickTarget(event, this.viewportDiv);
+ } else {
+ this.selecto?.clickTarget(event, this.div);
+ }
}
save = (updateMoveable = false) => {
@@ -217,8 +277,15 @@ export class Scene {
if (updateMoveable) {
setTimeout(() => {
- if (this.div) {
- initMoveable(true, this.isEditingEnabled, this);
+ if (config.featureToggles.canvasPanelPanZoom) {
+ if (this.viewportDiv && this.viewerDiv) {
+ initMoveable(true, this.isEditingEnabled, this);
+ this.updateConnectionsSize();
+ }
+ } else {
+ if (this.div) {
+ initMoveable(true, this.isEditingEnabled, this);
+ }
}
});
}
@@ -244,6 +311,14 @@ export class Scene {
this.div = sceneContainer;
};
+ setViewerRef = (viewerContainer: HTMLDivElement) => {
+ this.viewerDiv = viewerContainer;
+ };
+
+ setViewportRef = (viewportContainer: HTMLDivElement) => {
+ this.viewportDiv = viewportContainer;
+ };
+
select = (selection: SelectionParams) => {
if (this.selecto) {
this.selecto.setSelectedTargets(selection.targets);
@@ -282,15 +357,27 @@ export class Scene {
}
};
- render() {
- const hasDataLinks = this.tooltip?.element?.getLinks && this.tooltip.element.getLinks({}).length > 0;
- const hasActions = this.tooltip?.element?.options.actions && this.tooltip.element.options.actions.length > 0;
+ fitContent = (scene: Scene, zoomToContent: boolean) => {
+ const { root, viewerDiv, infiniteViewer } = scene;
+ if (zoomToContent && root.div && infiniteViewer && viewerDiv) {
+ const dimentions = calculateZoomToFitScale(Array.from(root.div.children), viewerDiv);
+ const { scale, centerX, centerY } = dimentions;
+ infiniteViewer.setZoom(scale);
+ infiniteViewer.scrollTo(centerX, centerY);
+ }
+ };
- const isTooltipValid = hasDataLinks || hasActions || this.tooltip?.element?.data?.field;
- const canShowElementTooltip = !this.isEditingEnabled && isTooltipValid;
+ render() {
+ const hasDataLinks = this.tooltipPayload?.element?.getLinks && this.tooltipPayload.element.getLinks({}).length > 0;
+ const hasActions =
+ this.tooltipPayload?.element?.options.actions && this.tooltipPayload.element.options.actions.length > 0;
+
+ const isTooltipValid = hasDataLinks || hasActions || this.tooltipPayload?.element?.data?.field;
+ const isTooltipEnabled = this.tooltipMode !== TooltipDisplayMode.None;
+ const canShowElementTooltip = !this.isEditingEnabled && isTooltipValid && isTooltipEnabled;
const sceneDiv = (
- {sceneDiv}
+
const queryController = sceneGraph.getQueryController(dashboard);
trackDashboardSceneLoaded(dashboard, measure?.duration);
- queryController?.startProfile('DashboardScene');
+ queryController?.startProfile('dashboard_view');
if (options.route !== DashboardRoutes.New) {
emitDashboardViewEvent({
diff --git a/public/app/features/dashboard-scene/panel-edit/PanelDataPane/PanelDataQueriesTab.tsx b/public/app/features/dashboard-scene/panel-edit/PanelDataPane/PanelDataQueriesTab.tsx
index ce2784758be..668b8612851 100644
--- a/public/app/features/dashboard-scene/panel-edit/PanelDataPane/PanelDataQueriesTab.tsx
+++ b/public/app/features/dashboard-scene/panel-edit/PanelDataPane/PanelDataQueriesTab.tsx
@@ -376,6 +376,7 @@ export function PanelDataQueriesTabRendered({ model }: SceneComponentProps
@@ -393,7 +394,9 @@ export function PanelDataQueriesTabRendered({ model }: SceneComponentProps
- openQueryLibraryDrawer(getDatasourceNames(datasource, queries), onSelectQueryFromLibrary)
+ openQueryLibraryDrawer(getDatasourceNames(datasource, queries), onSelectQueryFromLibrary, {
+ context: CoreApp.PanelEditor,
+ })
}
variant="secondary"
data-testid={selectors.components.QueryTab.addQueryFromLibrary}
diff --git a/public/app/features/dashboard-scene/panel-edit/PanelEditorRenderer.tsx b/public/app/features/dashboard-scene/panel-edit/PanelEditorRenderer.tsx
index 6071c79f41a..9aee611d4f0 100644
--- a/public/app/features/dashboard-scene/panel-edit/PanelEditorRenderer.tsx
+++ b/public/app/features/dashboard-scene/panel-edit/PanelEditorRenderer.tsx
@@ -195,6 +195,7 @@ function getStyles(theme: GrafanaTheme2) {
position: 'absolute',
width: '100%',
height: '100%',
+ overflow: 'unset',
}),
body: css({
label: 'body',
diff --git a/public/app/features/dashboard-scene/saving/DashboardSceneChangeTracker.ts b/public/app/features/dashboard-scene/saving/DashboardSceneChangeTracker.ts
index 6fc70618106..5b02068f80c 100644
--- a/public/app/features/dashboard-scene/saving/DashboardSceneChangeTracker.ts
+++ b/public/app/features/dashboard-scene/saving/DashboardSceneChangeTracker.ts
@@ -46,20 +46,24 @@ export class DashboardSceneChangeTracker {
}
static isUpdatingPersistedState({ payload }: SceneObjectStateChangedEvent) {
+ const partialUpdateKeys = Object.keys(payload.partialUpdate);
+
// If there are no changes in the state, the check is not needed
- if (Object.keys(payload.partialUpdate).length === 0) {
+ if (partialUpdateKeys.length === 0) {
return false;
}
- // Any change in the panel should trigger a change detection
+ // Any change in the grid item should trigger a change detection
// The PanelTimeRange includes the overrides configuration
- if (
- payload.changedObject instanceof VizPanel ||
- payload.changedObject instanceof DashboardGridItem ||
- payload.changedObject instanceof PanelTimeRange
- ) {
+ if (payload.changedObject instanceof DashboardGridItem || payload.changedObject instanceof PanelTimeRange) {
return true;
}
+ // Panels contain a _renderCounter state prop which should not be marked as a change
+ if (payload.changedObject instanceof VizPanel) {
+ if (partialUpdateKeys.length > 1 || partialUpdateKeys[0] !== '_renderCounter') {
+ return true;
+ }
+ }
// SceneQueryRunner includes the DS configuration
if (payload.changedObject instanceof SceneQueryRunner) {
if (!Object.prototype.hasOwnProperty.call(payload.partialUpdate, 'data')) {
diff --git a/public/app/features/dashboard-scene/saving/provisioned/SaveProvisionedDashboardForm.tsx b/public/app/features/dashboard-scene/saving/provisioned/SaveProvisionedDashboardForm.tsx
index 4387af82439..260c491bb57 100644
--- a/public/app/features/dashboard-scene/saving/provisioned/SaveProvisionedDashboardForm.tsx
+++ b/public/app/features/dashboard-scene/saving/provisioned/SaveProvisionedDashboardForm.tsx
@@ -18,7 +18,7 @@ import { useCreateOrUpdateRepositoryFile } from 'app/features/provisioning/hooks
import { ResourceEditFormSharedFields } from '../../components/Provisioned/ResourceEditFormSharedFields';
import { buildResourceBranchRedirectUrl } from '../../settings/utils';
import { getDashboardUrl } from '../../utils/getDashboardUrl';
-import { useProvisionedRequestHandler } from '../../utils/useProvisionedRequestHandler';
+import { ProvisionedOperationInfo, useProvisionedRequestHandler } from '../../utils/useProvisionedRequestHandler';
import { SaveDashboardFormCommonOptions } from '../SaveDashboardForm';
import { ProvisionedDashboardFormData } from '../shared';
@@ -60,25 +60,15 @@ export function SaveProvisionedDashboardForm({
reset(defaultValues);
}, [defaultValues, reset]);
- const onRequestError = (error: unknown) => {
+ const onRequestError = (error: unknown, info: ProvisionedOperationInfo) => {
appEvents.publish({
type: AppEvents.alertError.name,
payload: [t('dashboard-scene.save-provisioned-dashboard-form.api-error', 'Error saving dashboard'), error],
});
};
- const onWriteSuccess = () => {
- panelEditor?.onDiscard();
- drawer.onClose();
- locationService.partial({
- viewPanel: null,
- editPanel: null,
- });
- };
-
- const onNewDashboardSuccess = (upsert: Resource) => {
- panelEditor?.onDiscard();
- drawer.onClose();
+ const handleNewDashboard = (upsert: Resource) => {
+ // Navigation for new dashboards
const url = locationUtil.assureBaseUrl(
getDashboardUrl({
uid: upsert.metadata.name,
@@ -86,34 +76,50 @@ export function SaveProvisionedDashboardForm({
currentQueryParams: window.location.search,
})
);
-
navigate(url);
};
- const onBranchSuccess = (ref: string, path: string) => {
+ const onWriteSuccess = (_: ProvisionedOperationInfo, upsert: Resource) => {
+ if (isNew && upsert?.metadata.name) {
+ handleNewDashboard(upsert);
+ } else {
+ locationService.partial({
+ viewPanel: null,
+ editPanel: null,
+ });
+ }
+ };
+
+ const onBranchSuccess = (ref: string, path: string, info: ProvisionedOperationInfo, upsert: Resource) => {
+ if (isNew && upsert?.metadata?.name) {
+ handleNewDashboard(upsert);
+ } else {
+ const url = buildResourceBranchRedirectUrl({
+ baseUrl: `${PROVISIONING_URL}/${defaultValues.repo}/dashboard/preview/${path}`,
+ paramName: 'ref',
+ paramValue: ref,
+ repoType: info.repoType,
+ });
+ navigate(url);
+ }
+ };
+
+ const onDismiss = () => {
+ dashboard.setState({ isDirty: false });
panelEditor?.onDiscard();
drawer.onClose();
-
- const url = buildResourceBranchRedirectUrl({
- baseUrl: `${PROVISIONING_URL}/${defaultValues.repo}/dashboard/preview/${path}`,
- paramName: 'ref',
- paramValue: ref,
- repoType: request.data?.repository?.type,
- });
- navigate(url);
};
- useProvisionedRequestHandler({
- dashboard,
+ useProvisionedRequestHandler({
request,
workflow,
+ resourceType: 'dashboard',
handlers: {
- onBranchSuccess: ({ ref, path }) => onBranchSuccess(ref, path),
+ onBranchSuccess: ({ ref, path }, info, resource) => onBranchSuccess(ref, path, info, resource),
onWriteSuccess,
- onNewDashboardSuccess,
onError: onRequestError,
+ onDismiss,
},
- isNew,
});
// Submit handler for saving the form data
diff --git a/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.test.ts b/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.test.ts
new file mode 100644
index 00000000000..d55c4940f52
--- /dev/null
+++ b/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.test.ts
@@ -0,0 +1,74 @@
+import { getPanelPlugin } from '@grafana/data/test';
+import { setPluginImportUtils } from '@grafana/runtime';
+import { SceneTimeRange, sceneGraph, sceneUtils, VizPanel } from '@grafana/scenes';
+
+import { activateFullSceneTree } from '../utils/test-utils';
+
+import { DashboardLevelTimeMacro } from './DashboardLevelTimeMacro';
+import { DashboardScene } from './DashboardScene';
+import { PanelTimeRange } from './PanelTimeRange';
+import { AutoGridItem } from './layout-auto-grid/AutoGridItem';
+import { AutoGridLayout } from './layout-auto-grid/AutoGridLayout';
+import {
+ AutoGridLayoutManager,
+ getAutoRowsTemplate,
+ getTemplateColumnsTemplate,
+} from './layout-auto-grid/AutoGridLayoutManager';
+
+jest.mock('@grafana/runtime', () => ({
+ ...jest.requireActual('@grafana/runtime'),
+ getPluginLinkExtensions: jest.fn().mockReturnValue({ extensions: [] }),
+}));
+
+setPluginImportUtils({
+ importPanelPlugin: (id: string) => Promise.resolve(getPanelPlugin({})),
+ getPanelPluginFromCache: (id: string) => undefined,
+});
+
+describe('dashboardLevelTimeMacros', () => {
+ it('Can use use $__from and $__to', async () => {
+ const panel = new VizPanel({
+ $timeRange: new PanelTimeRange({ timeShift: '1h' }),
+ title: 'Test Panel',
+ key: 'panel-1',
+ pluginId: 'timeseries',
+ });
+
+ const scene = new DashboardScene({
+ $timeRange: new SceneTimeRange({ from: '2023-05-23T06:09:57.073Z', to: '2023-05-23T07:09:57.073Z' }),
+ body: new AutoGridLayoutManager({
+ maxColumnCount: 12,
+ columnWidth: 100,
+ rowHeight: 100,
+ fillScreen: true,
+ layout: new AutoGridLayout({
+ isDraggable: true,
+ templateColumns: getTemplateColumnsTemplate(12, 100),
+ autoRows: getAutoRowsTemplate(100, true),
+ children: [
+ new AutoGridItem({
+ body: panel,
+ }),
+ ],
+ }),
+ }),
+ });
+
+ activateFullSceneTree(scene);
+
+ // Wait for the scene to be activated
+ await new Promise((resolve) => setTimeout(resolve, 100));
+
+ expect(sceneGraph.interpolate(scene, '$__from')).toBe('1684822197073'); // Dashboard level time range
+ expect(sceneGraph.interpolate(scene, '$__to')).toBe('1684825797073'); // Dashboard level time range
+
+ expect(sceneGraph.interpolate(panel, '$__from')).toBe('1684818597073'); // Time shifted by 1h
+ expect(sceneGraph.interpolate(panel, '$__to')).toBe('1684822197073'); // Time shifted by 1h
+
+ sceneUtils.registerVariableMacro('__from', DashboardLevelTimeMacro, true);
+ sceneUtils.registerVariableMacro('__to', DashboardLevelTimeMacro, true);
+
+ expect(sceneGraph.interpolate(panel, '$__from')).toBe('1684822197073'); // Dashboard level time range even when panel is time shifted
+ expect(sceneGraph.interpolate(panel, '$__to')).toBe('1684825797073'); // Dashboard level time range even when panel is time shifted
+ });
+});
diff --git a/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.ts b/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.ts
new file mode 100644
index 00000000000..e6d0fc032fb
--- /dev/null
+++ b/public/app/features/dashboard-scene/scene/DashboardLevelTimeMacro.ts
@@ -0,0 +1,33 @@
+import { dateTimeFormat } from '@grafana/data';
+import { FormatVariable, sceneGraph, SceneObject } from '@grafana/scenes';
+
+/**
+ * This macro is used to support the old __to and __from macros that always used the dashboard level time range.
+ **/
+export class DashboardLevelTimeMacro implements FormatVariable {
+ public state: { name: string; type: string };
+ private _sceneObject: SceneObject;
+
+ public constructor(name: string, sceneObject: SceneObject) {
+ this.state = { name: name, type: 'time_macro' };
+ this._sceneObject = sceneObject.getRoot();
+ }
+
+ public getValue() {
+ const timeRange = sceneGraph.getTimeRange(this._sceneObject);
+ if (this.state.name === '__from') {
+ return timeRange.state.value.from.valueOf();
+ } else {
+ return timeRange.state.value.to.valueOf();
+ }
+ }
+
+ public getValueText?(): string {
+ const timeRange = sceneGraph.getTimeRange(this._sceneObject);
+ if (this.state.name === '__from') {
+ return dateTimeFormat(timeRange.state.value.from, { timeZone: timeRange.getTimeZone() });
+ } else {
+ return dateTimeFormat(timeRange.state.value.to, { timeZone: timeRange.getTimeZone() });
+ }
+ }
+}
diff --git a/public/app/features/dashboard-scene/scene/DashboardScene.tsx b/public/app/features/dashboard-scene/scene/DashboardScene.tsx
index 662d92a32df..48581e303ae 100644
--- a/public/app/features/dashboard-scene/scene/DashboardScene.tsx
+++ b/public/app/features/dashboard-scene/scene/DashboardScene.tsx
@@ -88,6 +88,7 @@ import { DashboardGridItem } from './layout-default/DashboardGridItem';
import { DefaultGridLayoutManager } from './layout-default/DefaultGridLayoutManager';
import { addNewRowTo } from './layouts-shared/addNew';
import { clearClipboard } from './layouts-shared/paste';
+import { getIsLazy } from './layouts-shared/utils';
import { DashboardLayoutManager } from './types/DashboardLayoutManager';
import { LayoutParent } from './types/LayoutParent';
@@ -198,7 +199,7 @@ export class DashboardScene extends SceneObjectBase impleme
meta: {},
editable: true,
$timeRange: state.$timeRange ?? new SceneTimeRange({}),
- body: state.body ?? DefaultGridLayoutManager.fromVizPanels(),
+ body: state.body ?? DefaultGridLayoutManager.fromVizPanels([], getIsLazy(state.preload)),
links: state.links ?? [],
...state,
editPane: new DashboardEditPane(),
diff --git a/public/app/features/dashboard-scene/scene/DashboardSceneUrlSync.ts b/public/app/features/dashboard-scene/scene/DashboardSceneUrlSync.ts
index 3d9acceed0b..e19a7331d5c 100644
--- a/public/app/features/dashboard-scene/scene/DashboardSceneUrlSync.ts
+++ b/public/app/features/dashboard-scene/scene/DashboardSceneUrlSync.ts
@@ -208,8 +208,11 @@ export class DashboardSceneUrlSync implements SceneObjectUrlSyncHandler {
if (panel) {
this._viewEventSub?.unsubscribe();
this._scene.setState({ viewPanelScene: new ViewPanelScene({ panelRef: panel.getRef() }) });
+ this._viewEventSub = undefined;
}
});
+
+ this._scene.state.body.activateRepeaters?.();
}
}
diff --git a/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayout.tsx b/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayout.tsx
index be19f886d83..b96550cfa22 100644
--- a/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayout.tsx
+++ b/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayout.tsx
@@ -22,9 +22,6 @@ export interface AutoGridLayoutState extends SceneObjectState, AutoGridLayoutOpt
*/
md?: AutoGridLayoutOptions;
- /** True when the items should be lazy loaded */
- isLazy?: boolean;
-
/** True when the items should be draggable */
isDraggable?: boolean;
diff --git a/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayoutRenderer.tsx b/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayoutRenderer.tsx
index 7f66994354d..2a44564b536 100644
--- a/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayoutRenderer.tsx
+++ b/public/app/features/dashboard-scene/scene/layout-auto-grid/AutoGridLayoutRenderer.tsx
@@ -1,4 +1,5 @@
import { css, cx } from '@emotion/css';
+import { useMemo } from 'react';
import { GrafanaTheme2 } from '@grafana/data';
import { LazyLoader, SceneComponentProps, sceneGraph } from '@grafana/scenes';
@@ -8,18 +9,21 @@ import { useHasClonedParents } from '../../utils/clone';
import { useDashboardState } from '../../utils/utils';
import { CanvasGridAddActions } from '../layouts-shared/CanvasGridAddActions';
import { dashboardCanvasAddButtonHoverStyles } from '../layouts-shared/styles';
+import { getIsLazy } from '../layouts-shared/utils';
import { AutoGridLayout, AutoGridLayoutState } from './AutoGridLayout';
import { AutoGridLayoutManager } from './AutoGridLayoutManager';
export function AutoGridLayoutRenderer({ model }: SceneComponentProps) {
- const { children, isHidden, isLazy } = model.useState();
+ const { children, isHidden } = model.useState();
const hasClonedParents = useHasClonedParents(model);
const styles = useStyles2(getStyles, model.state);
- const { layoutOrchestrator, isEditing } = useDashboardState(model);
+ const { layoutOrchestrator, isEditing, preload } = useDashboardState(model);
const layoutManager = sceneGraph.getAncestor(model, AutoGridLayoutManager);
const { fillScreen } = layoutManager.useState();
+ const isLazy = useMemo(() => getIsLazy(preload), [preload]);
+
if (isHidden || !layoutOrchestrator) {
return null;
}
diff --git a/public/app/features/dashboard-scene/scene/layout-default/DefaultGridLayoutManager.tsx b/public/app/features/dashboard-scene/scene/layout-default/DefaultGridLayoutManager.tsx
index b85cb2bcf4b..76fffddd0a5 100644
--- a/public/app/features/dashboard-scene/scene/layout-default/DefaultGridLayoutManager.tsx
+++ b/public/app/features/dashboard-scene/scene/layout-default/DefaultGridLayoutManager.tsx
@@ -46,6 +46,7 @@ import { AutoGridItem } from '../layout-auto-grid/AutoGridItem';
import { CanvasGridAddActions } from '../layouts-shared/CanvasGridAddActions';
import { clearClipboard, getDashboardGridItemFromClipboard } from '../layouts-shared/paste';
import { dashboardCanvasAddButtonHoverStyles } from '../layouts-shared/styles';
+import { getIsLazy } from '../layouts-shared/utils';
import { DashboardLayoutManager } from '../types/DashboardLayoutManager';
import { LayoutRegistryItem } from '../types/LayoutRegistryItem';
@@ -558,10 +559,11 @@ export class DefaultGridLayoutManager
public static createFromLayout(currentLayout: DashboardLayoutManager): DefaultGridLayoutManager {
const panels = currentLayout.getVizPanels();
- return DefaultGridLayoutManager.fromVizPanels(panels);
+ const isLazy = getIsLazy(getDashboardSceneFor(currentLayout).state.preload)!;
+ return DefaultGridLayoutManager.fromVizPanels(panels, isLazy);
}
- public static fromVizPanels(panels: VizPanel[] = []): DefaultGridLayoutManager {
+ public static fromVizPanels(panels: VizPanel[] = [], isLazy?: boolean | undefined): DefaultGridLayoutManager {
const children: DashboardGridItem[] = [];
const panelHeight = 10;
const panelWidth = GRID_COLUMN_COUNT / 3;
@@ -599,6 +601,7 @@ export class DefaultGridLayoutManager
children: children,
isDraggable: true,
isResizable: true,
+ isLazy,
}),
});
}
@@ -606,7 +609,8 @@ export class DefaultGridLayoutManager
public static fromGridItems(
gridItems: SceneGridItemLike[],
isDraggable?: boolean,
- isResizable?: boolean
+ isResizable?: boolean,
+ isLazy?: boolean | undefined
): DefaultGridLayoutManager {
const children = gridItems.reduce((acc, gridItem) => {
gridItem.clearParent();
@@ -620,6 +624,7 @@ export class DefaultGridLayoutManager
children,
isDraggable,
isResizable,
+ isLazy,
}),
});
}
diff --git a/public/app/features/dashboard-scene/scene/layout-rows/RowItemRenderer.tsx b/public/app/features/dashboard-scene/scene/layout-rows/RowItemRenderer.tsx
index b03b8b26e26..affa68d54ea 100644
--- a/public/app/features/dashboard-scene/scene/layout-rows/RowItemRenderer.tsx
+++ b/public/app/features/dashboard-scene/scene/layout-rows/RowItemRenderer.tsx
@@ -83,6 +83,7 @@ export function RowItemRenderer({ model }: SceneComponentProps) {
setTimeout(() => onSelect?.(evt));
}}
+ data-testid={selectors.components.DashboardRow.wrapper(title!)}
{...dragProvided.draggableProps}
>
{(!isHeaderHidden || isEditing) && (
diff --git a/public/app/features/dashboard-scene/scene/layout-rows/RowsLayoutManager.tsx b/public/app/features/dashboard-scene/scene/layout-rows/RowsLayoutManager.tsx
index 34b2a9e3a1a..1b04a9d6c23 100644
--- a/public/app/features/dashboard-scene/scene/layout-rows/RowsLayoutManager.tsx
+++ b/public/app/features/dashboard-scene/scene/layout-rows/RowsLayoutManager.tsx
@@ -258,7 +258,8 @@ export class RowsLayoutManager extends SceneObjectBase i
layout: DefaultGridLayoutManager.fromGridItems(
rowConfig.children,
rowConfig.isDraggable ?? layout.state.grid.state.isDraggable,
- rowConfig.isResizable ?? layout.state.grid.state.isResizable
+ rowConfig.isResizable ?? layout.state.grid.state.isResizable,
+ layout.state.grid.state.isLazy
),
})
);
diff --git a/public/app/features/dashboard-scene/scene/layouts-shared/utils.ts b/public/app/features/dashboard-scene/scene/layouts-shared/utils.ts
index 41402564081..eac9ff86333 100644
--- a/public/app/features/dashboard-scene/scene/layouts-shared/utils.ts
+++ b/public/app/features/dashboard-scene/scene/layouts-shared/utils.ts
@@ -1,6 +1,7 @@
import { useEffect, useRef } from 'react';
import { SceneObject } from '@grafana/scenes';
+import { contextSrv } from 'app/core/core';
import { DashboardLayoutManager, isDashboardLayoutManager } from '../types/DashboardLayoutManager';
import { isLayoutParent } from '../types/LayoutParent';
@@ -75,3 +76,8 @@ export function ungroupLayout(layout: DashboardLayoutManager, innerLayout: Dashb
layoutParent.switchLayout(innerLayout);
}
}
+
+export function getIsLazy(preload: boolean | undefined): boolean {
+ // We don't want to lazy load panels in the case of image renderer
+ return !(preload || (contextSrv.user && contextSrv.user.authenticatedBy === 'render'));
+}
diff --git a/public/app/features/dashboard-scene/serialization/layoutSerializers/DefaultGridLayoutSerializer.ts b/public/app/features/dashboard-scene/serialization/layoutSerializers/DefaultGridLayoutSerializer.ts
index 8688d1cbb08..e19059547b7 100644
--- a/public/app/features/dashboard-scene/serialization/layoutSerializers/DefaultGridLayoutSerializer.ts
+++ b/public/app/features/dashboard-scene/serialization/layoutSerializers/DefaultGridLayoutSerializer.ts
@@ -9,10 +9,10 @@ import {
PanelKind,
LibraryPanelKind,
} from '@grafana/schema/dist/esm/schema/dashboard/v2';
-import { contextSrv } from 'app/core/core';
import { DashboardGridItem } from '../../scene/layout-default/DashboardGridItem';
import { DefaultGridLayoutManager } from '../../scene/layout-default/DefaultGridLayoutManager';
+import { getIsLazy } from '../../scene/layouts-shared/utils';
import { dashboardSceneGraph } from '../../utils/dashboardSceneGraph';
import { calculateGridItemDimensions, isLibraryPanel } from '../../utils/utils';
@@ -41,7 +41,7 @@ export function deserializeDefaultGridLayout(
}
return new DefaultGridLayoutManager({
grid: new SceneGridLayout({
- isLazy: !(preload || contextSrv.user.authenticatedBy === 'render'),
+ isLazy: getIsLazy(preload),
children: createSceneGridLayoutForItems(layout, elements, panelIdGenerator),
}),
});
diff --git a/public/app/features/dashboard-scene/serialization/transformSaveModelSchemaV2ToScene.ts b/public/app/features/dashboard-scene/serialization/transformSaveModelSchemaV2ToScene.ts
index 986e0018329..2b6dc7d5864 100644
--- a/public/app/features/dashboard-scene/serialization/transformSaveModelSchemaV2ToScene.ts
+++ b/public/app/features/dashboard-scene/serialization/transformSaveModelSchemaV2ToScene.ts
@@ -51,6 +51,10 @@ import {
DeprecatedInternalId,
} from 'app/features/apiserver/types';
import { DashboardWithAccessInfo } from 'app/features/dashboard/api/types';
+import {
+ getDashboardInteractionCallback,
+ getDashboardSceneProfiler,
+} from 'app/features/dashboard/services/DashboardProfiler';
import { DashboardMeta } from 'app/types/dashboard';
import { addPanelsOnLoadBehavior } from '../addToDashboard/addPanelsOnLoadBehavior';
@@ -157,6 +161,15 @@ export function transformSaveModelSchemaV2ToScene(dto: DashboardWithAccessInfo uid === '*' || uid === metadata.name) !== -1,
+ onProfileComplete: getDashboardInteractionCallback(metadata.name, dashboard.title),
+ },
+ getDashboardSceneProfiler()
+ );
+
const dashboardScene = new DashboardScene(
{
description: dashboard.description,
@@ -184,7 +197,7 @@ export function transformSaveModelSchemaV2ToScene(dto: DashboardWithAccessInfo uid === '*' || uid === oldModel.uid) !== -1,
+ onProfileComplete: getDashboardInteractionCallback(oldModel.uid, oldModel.title),
+ },
+ getDashboardSceneProfiler()
+ );
+
const behaviorList: SceneObjectState['$behaviors'] = [
new behaviors.CursorSync({
sync: oldModel.graphTooltip,
}),
- new behaviors.SceneQueryController({
- enableProfiling:
- config.dashboardPerformanceMetrics.findIndex((uid) => uid === '*' || uid === oldModel.uid) !== -1,
- onProfileComplete: getDashboardInteractionCallback(oldModel.uid, oldModel.title),
- }),
+ queryController,
registerDashboardMacro,
registerPanelInteractionsReporter,
new behaviors.LiveNowTimer({ enabled: oldModel.liveNow }),
@@ -318,7 +326,7 @@ export function createDashboardSceneFromDashboardModel(oldModel: DashboardModel,
} else {
body = new DefaultGridLayoutManager({
grid: new SceneGridLayout({
- isLazy: !(dto.preload || contextSrv.user.authenticatedBy === 'render'),
+ isLazy: getIsLazy(dto.preload),
children: createSceneObjectsForPanels(oldModel.panels),
}),
});
@@ -499,40 +507,3 @@ export const convertOldSnapshotToScenesSnapshot = (panel: PanelModel) => {
panel.snapshotData = [];
}
};
-
-function getDashboardInteractionCallback(uid: string, title: string) {
- return (e: SceneInteractionProfileEvent) => {
- let interactionType = '';
-
- if (e.origin === 'SceneTimeRange') {
- interactionType = 'time-range-change';
- } else if (e.origin === 'SceneRefreshPicker') {
- interactionType = 'refresh';
- } else if (e.origin === 'DashboardScene') {
- interactionType = 'view';
- } else if (e.origin.indexOf('Variable') > -1) {
- interactionType = 'variable-change';
- }
- reportInteraction('dashboard-render', {
- interactionType,
- duration: e.duration,
- networkDuration: e.networkDuration,
- totalJSHeapSize: e.totalJSHeapSize,
- usedJSHeapSize: e.usedJSHeapSize,
- jsHeapSizeLimit: e.jsHeapSizeLimit,
- });
-
- logMeasurement(
- `dashboard.${interactionType}`,
- {
- duration: e.duration,
- networkDuration: e.networkDuration,
- totalJSHeapSize: e.totalJSHeapSize,
- usedJSHeapSize: e.usedJSHeapSize,
- jsHeapSizeLimit: e.jsHeapSizeLimit,
- timeSinceBoot: performance.measure('time_since_boot', 'frontend_boot_js_done_time_seconds').duration,
- },
- { dashboard: uid, title: title }
- );
- };
-}
diff --git a/public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx b/public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx
index 53bb342cd21..9d101b2f8c5 100644
--- a/public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx
+++ b/public/app/features/dashboard-scene/settings/DeleteProvisionedDashboardForm.tsx
@@ -11,7 +11,7 @@ import { PROVISIONING_URL } from 'app/features/provisioning/constants';
import { ResourceEditFormSharedFields } from '../components/Provisioned/ResourceEditFormSharedFields';
import { ProvisionedDashboardFormData } from '../saving/shared';
import { DashboardScene } from '../scene/DashboardScene';
-import { useProvisionedRequestHandler } from '../utils/useProvisionedRequestHandler';
+import { useProvisionedRequestHandler, ProvisionedOperationInfo } from '../utils/useProvisionedRequestHandler';
import { buildResourceBranchRedirectUrl } from './utils';
@@ -67,7 +67,7 @@ export function DeleteProvisionedDashboardForm({
const navigate = useNavigate();
- const onRequestError = (error: unknown) => {
+ const onError = (error: unknown) => {
getAppEvents().publish({
type: AppEvents.alertError.name,
payload: [t('dashboard-scene.delete-provisioned-dashboard-form.api-error', 'Failed to delete dashboard'), error],
@@ -75,32 +75,36 @@ export function DeleteProvisionedDashboardForm({
};
const onWriteSuccess = () => {
+ dashboard.setState({ isDirty: false });
panelEditor?.onDiscard();
- onDismiss();
// TODO reset search state instead
window.location.href = '/dashboards';
};
- const onBranchSuccess = (path: string, urls?: Record) => {
+ const onBranchSuccess = (path: string, info: ProvisionedOperationInfo, urls?: Record) => {
panelEditor?.onDiscard();
- onDismiss();
const url = buildResourceBranchRedirectUrl({
baseUrl: `${PROVISIONING_URL}/${defaultValues.repo}/dashboard/preview/${path}`,
paramName: 'pull_request_url',
paramValue: urls?.newPullRequestURL,
- repoType: request.data?.repository?.type,
+ repoType: info.repoType,
});
navigate(url);
};
useProvisionedRequestHandler({
- dashboard,
request,
workflow,
+ resourceType: 'dashboard',
+ successMessage: t(
+ 'dashboard-scene.delete-provisioned-dashboard-form.success-message',
+ 'Dashboard deleted successfully'
+ ),
handlers: {
- onBranchSuccess: ({ path, urls }) => onBranchSuccess(path, urls),
+ onDismiss,
+ onBranchSuccess: ({ path, urls }, info) => onBranchSuccess(path, info, urls),
onWriteSuccess,
- onError: onRequestError,
+ onError,
},
});
diff --git a/public/app/features/dashboard-scene/settings/MoveProvisionedDashboardForm.tsx b/public/app/features/dashboard-scene/settings/MoveProvisionedDashboardForm.tsx
index 8d431f3f6a1..2ad4f936309 100644
--- a/public/app/features/dashboard-scene/settings/MoveProvisionedDashboardForm.tsx
+++ b/public/app/features/dashboard-scene/settings/MoveProvisionedDashboardForm.tsx
@@ -18,7 +18,7 @@ import { getTargetFolderPathInRepo } from 'app/features/browse-dashboards/compon
import { ResourceEditFormSharedFields } from '../components/Provisioned/ResourceEditFormSharedFields';
import { ProvisionedDashboardFormData } from '../saving/shared';
import { DashboardScene } from '../scene/DashboardScene';
-import { useProvisionedRequestHandler } from '../utils/useProvisionedRequestHandler';
+import { useProvisionedRequestHandler, ProvisionedOperationInfo } from '../utils/useProvisionedRequestHandler';
import { buildResourceBranchRedirectUrl } from './utils';
@@ -118,6 +118,7 @@ export function MoveProvisionedDashboardForm({
};
const onWriteSuccess = () => {
+ dashboard.setState({ isDirty: false });
panelEditor?.onDiscard();
if (targetFolderUID && targetFolderTitle) {
onSuccess(targetFolderUID, targetFolderTitle);
@@ -125,23 +126,40 @@ export function MoveProvisionedDashboardForm({
navigate('/dashboards');
};
- const onBranchSuccess = () => {
+ const onBranchSuccess = (info: ProvisionedOperationInfo) => {
+ dashboard.setState({ isDirty: false });
panelEditor?.onDiscard();
const url = buildResourceBranchRedirectUrl({
paramName: 'new_pull_request_url',
paramValue: moveRequest?.data?.urls?.newPullRequestURL,
- repoType: moveRequest?.data?.repository?.type,
+ repoType: info.repoType,
});
navigate(url);
};
+ const onError = (error: unknown) => {
+ getAppEvents().publish({
+ type: AppEvents.alertError.name,
+ payload: [
+ t('dashboard-scene.move-provisioned-dashboard-form.alert-error-moving-dashboard', 'Error moving dashboard'),
+ error,
+ ],
+ });
+ };
+
useProvisionedRequestHandler({
- dashboard,
request: moveRequest,
workflow,
+ successMessage: t(
+ 'dashboard-scene.move-provisioned-dashboard-form.success-message',
+ 'Dashboard moved successfully'
+ ),
+ resourceType: 'dashboard',
handlers: {
- onBranchSuccess,
+ onBranchSuccess: (_, info) => onBranchSuccess(info),
onWriteSuccess,
+ onDismiss,
+ onError,
},
});
diff --git a/public/app/features/dashboard-scene/sharing/ExportButton/ExportMenu.tsx b/public/app/features/dashboard-scene/sharing/ExportButton/ExportMenu.tsx
index d5524108b07..c0099c17f2f 100644
--- a/public/app/features/dashboard-scene/sharing/ExportButton/ExportMenu.tsx
+++ b/public/app/features/dashboard-scene/sharing/ExportButton/ExportMenu.tsx
@@ -52,6 +52,7 @@ export default function ExportMenu({ dashboard }: { dashboard: DashboardScene })
menuItems.push({
shareId: shareDashboardType.image,
+ testId: newExportButtonSelector.exportAsImage,
icon: 'camera',
label: t('share-dashboard.menu.export-image-title', 'Export as image'),
renderCondition: Boolean(config.featureToggles.sharingDashboardImage),
diff --git a/public/app/features/dashboard-scene/sharing/ExportButton/utils.test.ts b/public/app/features/dashboard-scene/sharing/ExportButton/utils.test.ts
index 1c4ad371d0a..fa13471055a 100644
--- a/public/app/features/dashboard-scene/sharing/ExportButton/utils.test.ts
+++ b/public/app/features/dashboard-scene/sharing/ExportButton/utils.test.ts
@@ -96,6 +96,13 @@ describe('Dashboard Export Image Utils', () => {
const fetchMock = jest.fn().mockReturnValue(of({ ok: true, data: mockBlob }));
(getBackendSrv as jest.Mock).mockReturnValue({ fetch: fetchMock });
+ // Mock window.innerWidth
+ Object.defineProperty(window, 'innerWidth', {
+ writable: true,
+ configurable: true,
+ value: 1280,
+ });
+
const dashboard = {
state: {
uid: 'test-uid',
@@ -119,7 +126,7 @@ describe('Dashboard Export Image Utils', () => {
absolute: true,
updateQuery: {
height: -1,
- width: 1000,
+ width: 1280,
scale: 2,
kiosk: true,
hideNav: true,
@@ -128,5 +135,46 @@ describe('Dashboard Export Image Utils', () => {
},
});
});
+
+ it('should fallback to config width when window.innerWidth is not available', async () => {
+ config.rendererAvailable = true;
+ config.rendererDefaultImageWidth = 1500;
+ const mockBlob = new Blob(['test'], { type: 'image/png' });
+ const fetchMock = jest.fn().mockReturnValue(of({ ok: true, data: mockBlob }));
+ (getBackendSrv as jest.Mock).mockReturnValue({ fetch: fetchMock });
+
+ // Ensure window.innerWidth is undefined
+ Object.defineProperty(window, 'innerWidth', {
+ writable: true,
+ configurable: true,
+ value: undefined,
+ });
+
+ const dashboard = {
+ state: {
+ uid: 'test-uid',
+ },
+ } as DashboardScene;
+
+ const result = await generateDashboardImage({ dashboard, scale: 1 });
+
+ expect(result.error).toBeUndefined();
+ expect(result.blob).toBe(mockBlob);
+ expect(getDashboardUrl).toHaveBeenCalledWith({
+ uid: 'test-uid',
+ currentQueryParams: '',
+ render: true,
+ absolute: true,
+ updateQuery: {
+ height: -1,
+ width: 1500, // Should use config value
+ scale: 1,
+ kiosk: true,
+ hideNav: true,
+ orgId: '1',
+ fullPageImage: true,
+ },
+ });
+ });
});
});
diff --git a/public/app/features/dashboard-scene/sharing/ExportButton/utils.ts b/public/app/features/dashboard-scene/sharing/ExportButton/utils.ts
index 64d84893b45..f9fb9e5f04b 100644
--- a/public/app/features/dashboard-scene/sharing/ExportButton/utils.ts
+++ b/public/app/features/dashboard-scene/sharing/ExportButton/utils.ts
@@ -28,7 +28,7 @@ export interface ImageGenerationResult {
*/
export async function generateDashboardImage({
dashboard,
- scale = config.rendererDefaultImageScale || 1,
+ scale = config.rendererDefaultImageScale || 2,
}: ImageGenerationOptions): Promise {
try {
// Check if renderer plugin is available
@@ -46,7 +46,7 @@ export async function generateDashboardImage({
absolute: true,
updateQuery: {
height: -1, // image renderer will scroll through the dashboard and set the appropriate height
- width: config.rendererDefaultImageWidth || 1000,
+ width: window.innerWidth || config.rendererDefaultImageWidth || 1000,
scale,
kiosk: true,
hideNav: true,
diff --git a/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.test.ts b/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.test.ts
index 0587691ace0..d8c4f8a0a96 100644
--- a/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.test.ts
+++ b/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.test.ts
@@ -3,18 +3,10 @@ import { renderHook } from '@testing-library/react';
import { AppEvents } from '@grafana/data';
import { getAppEvents } from '@grafana/runtime';
import { Dashboard } from '@grafana/schema';
-import {
- DeleteRepositoryFilesWithPathApiResponse,
- GetRepositoryFilesWithPathApiResponse,
- ResourceWrapper,
-} from 'app/api/clients/provisioning/v0alpha1';
-import { Resource } from 'app/features/apiserver/types';
+import { ResourceWrapper } from 'app/api/clients/provisioning/v0alpha1';
-import { DashboardScene } from '../scene/DashboardScene';
+import { useProvisionedRequestHandler, RequestHandlers } from './useProvisionedRequestHandler';
-import { useProvisionedRequestHandler } from './useProvisionedRequestHandler';
-
-// Mock dependencies
jest.mock('@grafana/runtime', () => ({
getAppEvents: jest.fn(),
}));
@@ -30,9 +22,9 @@ describe('useProvisionedRequestHandler', () => {
jest.clearAllMocks();
});
- describe('when request has an error', () => {
- it('should call onError handler', () => {
- const { request, handlers, dashboard } = setup({
+ describe('error handling', () => {
+ it('should call onError handler with correct parameters', () => {
+ const { request, handlers } = setup({
requestOverrides: {
isError: true,
isSuccess: false,
@@ -42,264 +34,127 @@ describe('useProvisionedRequestHandler', () => {
renderHook(() =>
useProvisionedRequestHandler({
- dashboard,
request,
+ repository: {
+ type: 'github',
+ name: 'test-repo',
+ target: 'folder',
+ title: 'Test Repository',
+ workflows: [],
+ },
+ resourceType: 'dashboard',
handlers,
})
);
- expect(handlers.onError).toHaveBeenCalledWith(new Error('Test error'));
+ expect(handlers.onError).toHaveBeenCalledWith(
+ new Error('Test error'),
+ expect.objectContaining({
+ resourceType: 'dashboard',
+ repoType: 'github',
+ })
+ );
expect(handlers.onBranchSuccess).not.toHaveBeenCalled();
expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
- expect(handlers.onNewDashboardSuccess).not.toHaveBeenCalled();
});
});
- describe('when request is successful', () => {
- it('should set dashboard isDirty to false', () => {
- const { request, handlers, dashboard } = setup({
+ describe('success handling', () => {
+ it('should publish success event and call onDismiss', () => {
+ const { request, handlers, mockPublish } = setup({
requestOverrides: {
isError: false,
isSuccess: true,
- data: {
- ref: 'main',
- path: '/path/to/dashboard',
- },
- },
- workflowOverride: 'branch',
- });
-
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- workflow: 'branch',
- handlers,
- })
- );
-
- expect(dashboard.setState).toHaveBeenCalledWith({ isDirty: false });
- });
-
- it('should publish success event', () => {
- const { request, handlers, dashboard, mockPublish } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {},
+ data: createMockResourceWrapper(),
},
});
renderHook(() =>
useProvisionedRequestHandler({
- dashboard,
request,
+ resourceType: 'dashboard',
handlers,
})
);
expect(mockPublish).toHaveBeenCalledWith({
type: AppEvents.alertSuccess.name,
- payload: ['Dashboard changes saved successfully'],
+ payload: ['Dashboard saved successfully'],
});
+ expect(handlers.onDismiss).toHaveBeenCalled();
});
- describe('branch workflow', () => {
- it('should call onBranchSuccess when workflow is branch and data has ref and path', () => {
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {
- ref: 'feature-branch',
- path: '/path/to/dashboard.json',
- urls: { compareURL: 'http://example.com/edit' },
- },
- },
- workflowOverride: 'branch',
- });
+ it('should call onBranchSuccess for branch workflow', () => {
+ const { request, handlers } = setup({
+ requestOverrides: {
+ isError: false,
+ isSuccess: true,
+ data: createMockResourceWrapper({
+ ref: 'feature-branch',
+ path: '/path/to/dashboard.json',
+ urls: { compareURL: 'http://example.com/edit' },
+ }),
+ },
+ });
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- workflow: 'branch',
- handlers,
- })
- );
+ renderHook(() =>
+ useProvisionedRequestHandler({
+ request,
+ workflow: 'branch',
+ resourceType: 'dashboard',
+ handlers,
+ })
+ );
- expect(handlers.onBranchSuccess).toHaveBeenCalledWith({
+ expect(handlers.onBranchSuccess).toHaveBeenCalledWith(
+ {
ref: 'feature-branch',
path: '/path/to/dashboard.json',
urls: { compareURL: 'http://example.com/edit' },
- });
- expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
- });
-
- it('should not call onBranchSuccess when ref is missing', () => {
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {
- path: '/path/to/dashboard.json',
- },
- },
- workflowOverride: 'branch',
- });
-
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- workflow: 'branch',
- handlers,
- })
- );
-
- expect(handlers.onBranchSuccess).not.toHaveBeenCalled();
- expect(handlers.onWriteSuccess).toHaveBeenCalled();
- });
+ },
+ expect.objectContaining({
+ resourceType: 'dashboard',
+ repoType: 'git',
+ workflow: 'branch',
+ }),
+ expect.any(Object)
+ );
+ expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
});
- describe('new dashboard flow', () => {
- it('should call onNewDashboardSuccess when isNew is true and resource.upsert exists', () => {
- const mockUpsertResource = {
- metadata: {
- name: 'test-dashboard',
- uid: 'test-uid',
- resourceVersion: '1',
- creationTimestamp: new Date().toISOString(),
- },
- spec: { title: 'Test Dashboard' } as Dashboard,
- apiVersion: 'v1',
- kind: 'Dashboard',
- };
-
- const mockResource = {
- metadata: {
- name: 'test-dashboard',
- uid: 'test-uid',
- resourceVersion: '1',
- creationTimestamp: new Date().toISOString(),
- },
- spec: { title: 'Test Dashboard' } as Dashboard,
- apiVersion: 'v1',
- kind: 'Dashboard',
- upsert: mockUpsertResource,
- } as Resource & { upsert: Resource };
-
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {
- repository: 'test-repo',
- resource: mockResource,
- } as unknown as ProvisionedRequestData,
- },
- });
-
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- handlers,
- isNew: true,
- })
- );
-
- expect(handlers.onNewDashboardSuccess).toHaveBeenCalledWith(mockResource.upsert);
- expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
+ it('should call onWriteSuccess for write workflow', () => {
+ const { request, handlers } = setup({
+ requestOverrides: {
+ isError: false,
+ isSuccess: true,
+ data: createMockResourceWrapper(),
+ },
});
- it('should not call onNewDashboardSuccess when isNew is false', () => {
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {
- repository: 'test-repo',
- resource: {
- upsert: {
- apiVersion: 'v1',
- kind: 'Dashboard',
- metadata: { name: 'test-dashboard' },
- spec: { title: 'Test Dashboard' } as Dashboard,
- },
- metadata: { name: 'test-dashboard' },
- spec: { title: 'Test Dashboard' } as Dashboard,
- apiVersion: 'v1',
- kind: 'Dashboard',
- } as unknown as Resource,
- } as unknown as ProvisionedRequestData,
- },
- });
+ renderHook(() =>
+ useProvisionedRequestHandler({
+ request,
+ workflow: 'write',
+ resourceType: 'dashboard',
+ handlers,
+ })
+ );
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- handlers,
- isNew: false,
- })
- );
-
- expect(handlers.onNewDashboardSuccess).not.toHaveBeenCalled();
- expect(handlers.onWriteSuccess).toHaveBeenCalled();
- });
-
- it('should not call onNewDashboardSuccess when resource.upsert is missing', () => {
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {
- resource: {},
- } as ResourceWrapper,
- },
- });
-
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- handlers,
- isNew: true,
- })
- );
-
- expect(handlers.onNewDashboardSuccess).not.toHaveBeenCalled();
- expect(handlers.onWriteSuccess).toHaveBeenCalled();
- });
- });
-
- describe('write workflow', () => {
- it('should call onWriteSuccess as fallback', () => {
- const { request, handlers, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- data: {} as GetRepositoryFilesWithPathApiResponse,
- },
- });
-
- renderHook(() =>
- useProvisionedRequestHandler({
- dashboard,
- request,
- handlers,
- })
- );
-
- expect(handlers.onWriteSuccess).toHaveBeenCalled();
- });
+ expect(handlers.onWriteSuccess).toHaveBeenCalledWith(
+ expect.objectContaining({
+ resourceType: 'dashboard',
+ repoType: 'git',
+ workflow: 'write',
+ }),
+ expect.any(Object)
+ );
+ expect(handlers.onDismiss).toHaveBeenCalled();
});
});
- describe('when request is neither error nor success', () => {
- it('should not call any handlers', () => {
- const { request, handlers, dashboard, mockPublish } = setup({
+ describe('edge cases', () => {
+ it('should not call any handlers when request is loading', () => {
+ const { request, handlers, mockPublish } = setup({
requestOverrides: {
isError: false,
isSuccess: false,
@@ -309,7 +164,6 @@ describe('useProvisionedRequestHandler', () => {
renderHook(() =>
useProvisionedRequestHandler({
- dashboard,
request,
handlers,
})
@@ -318,15 +172,11 @@ describe('useProvisionedRequestHandler', () => {
expect(handlers.onError).not.toHaveBeenCalled();
expect(handlers.onBranchSuccess).not.toHaveBeenCalled();
expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
- expect(handlers.onNewDashboardSuccess).not.toHaveBeenCalled();
- expect(dashboard.setState).not.toHaveBeenCalled();
expect(mockPublish).not.toHaveBeenCalled();
});
- });
- describe('when request success but no data', () => {
- it('should not call any handlers when data is undefined', () => {
- const { request, handlers, dashboard, mockPublish } = setup({
+ it('should not call handlers when success but no data', () => {
+ const { request, handlers, mockPublish } = setup({
requestOverrides: {
isError: false,
isSuccess: true,
@@ -336,7 +186,6 @@ describe('useProvisionedRequestHandler', () => {
renderHook(() =>
useProvisionedRequestHandler({
- dashboard,
request,
handlers,
})
@@ -344,25 +193,18 @@ describe('useProvisionedRequestHandler', () => {
expect(handlers.onWriteSuccess).not.toHaveBeenCalled();
expect(handlers.onBranchSuccess).not.toHaveBeenCalled();
- expect(dashboard.setState).not.toHaveBeenCalled();
expect(mockPublish).not.toHaveBeenCalled();
});
- });
- describe('optional handlers', () => {
it('should not throw when optional handlers are not provided', () => {
- const { request, dashboard } = setup({
- requestOverrides: {
- isError: false,
- isSuccess: true,
- },
+ const { request } = setup({
+ requestOverrides: { isError: false, isSuccess: true },
handlersOverrides: {},
});
expect(() => {
renderHook(() =>
useProvisionedRequestHandler({
- dashboard,
request,
handlers: {},
})
@@ -372,62 +214,69 @@ describe('useProvisionedRequestHandler', () => {
});
});
-type ProvisionedRequestData = DeleteRepositoryFilesWithPathApiResponse | GetRepositoryFilesWithPathApiResponse;
+// Helper function to create a properly structured mock ResourceWrapper
+function createMockResourceWrapper(overrides: Partial = {}): ResourceWrapper {
+ return {
+ repository: {
+ name: 'test-repo',
+ namespace: 'default',
+ title: 'Test Repository',
+ type: 'git',
+ },
+ resource: {
+ type: {
+ kind: 'Dashboard',
+ },
+ upsert: {
+ apiVersion: 'v1',
+ kind: 'Dashboard',
+ metadata: { name: 'test-dashboard', uid: 'test-uid' },
+ spec: { title: 'Test Dashboard' },
+ },
+ },
+ ...overrides,
+ };
+}
function setup({
requestOverrides = {},
handlersOverrides = {},
- workflowOverride,
}: {
requestOverrides?: Partial<{
isError: boolean;
isSuccess: boolean;
isLoading?: boolean;
error?: unknown;
- data?: Partial;
+ data?: ResourceWrapper;
}>;
- handlersOverrides?: Partial<{
- onBranchSuccess?: jest.Mock;
- onWriteSuccess?: jest.Mock;
- onNewDashboardSuccess?: jest.Mock;
- onError?: jest.Mock;
- }>;
- workflowOverride?: string;
+ handlersOverrides?: Partial>;
} = {}) {
const mockPublish = jest.fn();
- const mockSetState = jest.fn();
mockGetAppEvents.mockReturnValue({
publish: mockPublish,
} as unknown as ReturnType);
- const dashboard = {
- setState: mockSetState,
- } as unknown as DashboardScene;
-
const request = {
isError: false,
isSuccess: false,
isLoading: false,
error: undefined,
data: undefined,
- ...(requestOverrides as ResourceWrapper),
+ ...requestOverrides,
};
- const handlers = {
+ const handlers: RequestHandlers = {
onError: jest.fn(),
onBranchSuccess: jest.fn(),
onWriteSuccess: jest.fn(),
- onNewDashboardSuccess: jest.fn(),
+ onDismiss: jest.fn(),
...handlersOverrides,
};
return {
- dashboard,
request,
handlers,
mockPublish,
- mockSetState,
- workflow: workflowOverride,
};
}
diff --git a/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.ts b/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.ts
index d5b9ca4ba7b..45fb3f18b79 100644
--- a/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.ts
+++ b/public/app/features/dashboard-scene/utils/useProvisionedRequestHandler.ts
@@ -3,20 +3,32 @@ import { useEffect } from 'react';
import { AppEvents } from '@grafana/data';
import { t } from '@grafana/i18n';
import { getAppEvents } from '@grafana/runtime';
-import { Dashboard } from '@grafana/schema';
import {
DeleteRepositoryFilesWithPathApiResponse,
GetRepositoryFilesWithPathApiResponse,
+ RepositoryView,
} from 'app/api/clients/provisioning/v0alpha1';
import { Resource } from 'app/features/apiserver/types';
+import { RepoType } from 'app/features/provisioning/Wizard/types';
-import { DashboardScene } from '../scene/DashboardScene';
+type ResourceType = 'dashboard' | 'folder'; // Add more as needed, e.g., 'alert', etc.
-interface RequestHandlers {
- onBranchSuccess?: (data: { ref: string; path: string; urls?: Record }) => void;
- onWriteSuccess?: () => void;
- onNewDashboardSuccess?: (resource: Resource) => void;
- onError?: (error: unknown) => void;
+// Information object that gets passed to all handlers
+interface ProvisionedOperationInfo {
+ repoType: RepoType;
+ resourceType?: ResourceType;
+ workflow?: string;
+}
+
+interface RequestHandlers {
+ onBranchSuccess?: (
+ data: { ref: string; path: string; urls?: Record },
+ info: ProvisionedOperationInfo,
+ resource: Resource
+ ) => void;
+ onWriteSuccess?: (info: ProvisionedOperationInfo, resource: Resource) => void;
+ onError?: (error: unknown, info: ProvisionedOperationInfo) => void;
+ onDismiss?: () => void;
}
interface ProvisionedRequest {
@@ -27,51 +39,85 @@ interface ProvisionedRequest {
data?: DeleteRepositoryFilesWithPathApiResponse | GetRepositoryFilesWithPathApiResponse;
}
-// This hook handles save new dashboard, edit existing dashboard, and delete dashboard response logic for provisioned dashboards.
-export function useProvisionedRequestHandler({
- dashboard,
+// Resource-specific configuration for different resource types
+interface ResourceConfig {
+ defaultSuccessMessage: string;
+ supportedWorkflows: string[];
+}
+
+/**
+ * Generic hook for handling provisioned resource operations across any resource type and repository provider.
+ *
+ * This hook is intentionally decoupled from specific components (like DashboardScene) to promote reusability.
+ * Components are responsible for their own state management through specific workflow handlers.
+ */
+export function useProvisionedRequestHandler({
request,
workflow,
handlers,
- isNew,
+ successMessage,
+ repository,
+ resourceType,
}: {
- dashboard: DashboardScene;
request: ProvisionedRequest;
workflow?: string;
- handlers: RequestHandlers;
- isNew?: boolean;
+ handlers: RequestHandlers;
+ successMessage?: string;
+ repository?: RepositoryView;
+ resourceType?: ResourceType;
}) {
useEffect(() => {
+ const repoType = repository?.type || 'git';
+ const info: ProvisionedOperationInfo = {
+ repoType,
+ resourceType,
+ workflow,
+ };
+
if (request.isError) {
- handlers.onError?.(request.error);
+ handlers.onError?.(request.error, info);
return;
}
if (request.isSuccess && request.data) {
- dashboard.setState({ isDirty: false });
const { ref, path, urls, resource } = request.data;
+ // eslint-disable-next-line @typescript-eslint/consistent-type-assertions
+ const resourceData = resource.upsert as Resource;
- // Branch workflow
- if (workflow === 'branch' && ref && path) {
- handlers.onBranchSuccess?.({ ref, path, urls });
- return;
- }
-
- // Success message (could be configurable)
+ // Success message
+ const message = successMessage || getContextualSuccessMessage(info);
getAppEvents().publish({
type: AppEvents.alertSuccess.name,
- payload: [t('dashboard-scene.edit-provisioned-dashboard-form.success', 'Dashboard changes saved successfully')],
+ payload: [message],
});
- // New dashboard flow
- if (isNew && resource?.upsert && handlers.onNewDashboardSuccess) {
- // eslint-disable-next-line @typescript-eslint/consistent-type-assertions
- handlers.onNewDashboardSuccess(resource.upsert as Resource);
- return;
+ // Branch workflow
+ if (workflow === 'branch' && handlers.onBranchSuccess && ref && path) {
+ const branchData = { ref, path, urls };
+ handlers.onBranchSuccess?.(branchData, info, resourceData);
}
// Write workflow
- handlers.onWriteSuccess?.();
+ if (workflow === 'write' && handlers.onWriteSuccess) {
+ handlers.onWriteSuccess(info, resourceData);
+ }
+
+ handlers.onDismiss?.();
}
- }, [request, workflow, handlers, isNew, dashboard]);
+ }, [request, workflow, handlers, successMessage, repository, resourceType]);
}
+
+function getContextualSuccessMessage(info: ProvisionedOperationInfo): string {
+ const { resourceType } = info;
+
+ switch (resourceType) {
+ case 'dashboard':
+ return t('provisioned-resource-request-handler-dashboard', 'Dashboard saved successfully');
+ case 'folder':
+ return t('provisioned-resource-request-handler-folder', 'Folder created successfully');
+ default:
+ return t('provisioned-resource-request-handler', 'Resource saved successfully');
+ }
+}
+
+export type { ResourceType, ProvisionedOperationInfo, RequestHandlers, ResourceConfig };
diff --git a/public/app/features/dashboard/dashgrid/SeriesVisibilityConfigFactory.ts b/public/app/features/dashboard/dashgrid/SeriesVisibilityConfigFactory.ts
index a03cc64bb5a..412d37d4bfc 100644
--- a/public/app/features/dashboard/dashgrid/SeriesVisibilityConfigFactory.ts
+++ b/public/app/features/dashboard/dashgrid/SeriesVisibilityConfigFactory.ts
@@ -97,7 +97,7 @@ function createOverride(
value: {
viz: true,
legend: false,
- tooltip: false,
+ tooltip: true,
},
};
@@ -118,7 +118,7 @@ function createOverride(
value: {
viz: true,
legend: false,
- tooltip: false,
+ tooltip: true,
},
},
],
diff --git a/public/app/features/dashboard/services/DashboardProfiler.ts b/public/app/features/dashboard/services/DashboardProfiler.ts
new file mode 100644
index 00000000000..aa4ce4d2caa
--- /dev/null
+++ b/public/app/features/dashboard/services/DashboardProfiler.ts
@@ -0,0 +1,34 @@
+import { logMeasurement, reportInteraction } from '@grafana/runtime';
+import { SceneInteractionProfileEvent, SceneRenderProfiler } from '@grafana/scenes';
+
+let dashboardSceneProfiler: SceneRenderProfiler | undefined;
+
+export function getDashboardSceneProfiler() {
+ if (!dashboardSceneProfiler) {
+ dashboardSceneProfiler = new SceneRenderProfiler();
+ }
+ return dashboardSceneProfiler;
+}
+
+export function getDashboardInteractionCallback(uid: string, title: string) {
+ return (e: SceneInteractionProfileEvent) => {
+ const payload = {
+ duration: e.duration,
+ networkDuration: e.networkDuration,
+ startTs: e.startTs,
+ endTs: e.endTs,
+ totalJSHeapSize: e.totalJSHeapSize,
+ usedJSHeapSize: e.usedJSHeapSize,
+ jsHeapSizeLimit: e.jsHeapSizeLimit,
+ timeSinceBoot: performance.measure('time_since_boot', 'frontend_boot_js_done_time_seconds').duration,
+ };
+
+ reportInteraction('dashboard_render', {
+ interactionType: e.origin,
+ uid,
+ ...payload,
+ });
+
+ logMeasurement(`dashboard_render`, payload, { interactionType: e.origin, dashboard: uid, title: title });
+ };
+}
diff --git a/public/app/features/dashboard/services/dashboard-render-performance-profiling.md b/public/app/features/dashboard/services/dashboard-render-performance-profiling.md
new file mode 100644
index 00000000000..574e4f7eb90
--- /dev/null
+++ b/public/app/features/dashboard/services/dashboard-render-performance-profiling.md
@@ -0,0 +1,199 @@
+# Grafana Dashboard Render Performance Metrics
+
+This documentation describes the dashboard render performance metrics exposed from Grafana's frontend.
+
+## Overview
+
+The exposed dashboard performance metrics feature provides comprehensive tracking and profiling of dashboard interactions, allowing administrators and developers to analyze dashboard render performance, user interactions, and identify performance bottlenecks.
+
+## Configuration
+
+### Enabling Performance Metrics
+
+Dashboard performance metrics are configured in the Grafana configuration file (`grafana.ini`) under the `[dashboards]` section:
+
+```ini
+[dashboards]
+# Dashboards UIDs to report performance metrics for. * can be used to report metrics for all dashboards
+dashboard_performance_metrics = *
+```
+
+**Configuration Options:**
+
+- **`*`** - Enable profiling on all dashboards
+- **``** - Enable profiling on specific dashboards only
+- **`""` (empty)** - Disable performance metrics (default)
+
+**Examples:**
+
+```ini
+# Enable for all dashboards
+dashboard_performance_metrics = *
+
+# Enable for specific dashboards
+dashboard_performance_metrics = dashboard-uid-1,dashboard-uid-2,dashboard-uid-3
+
+# Disable performance metrics
+dashboard_performance_metrics =
+```
+
+## Tracked Interactions
+
+The system tracks various dashboard interaction types automatically using the [`@grafana/scenes`](https://github.com/grafana/scenes) library. Each interaction is captured with a specific origin identifier that describes the type of user action performed. In Grafana, these interaction events are then reported as `dashboard_render` events with interaction type information included.
+
+### Core Performance-Tracked Interactions
+
+The following dashboard interaction types are tracked for dashboard render performance profiling:
+
+| Interaction Type | Trigger | When Measured |
+| ------------------------ | -------------------------- | -------------------------------------------------------- |
+| `dashboard_view` | Dashboard view | When user loads or navigates to a dashboard |
+| `refresh` | Manual/Auto refresh | When user clicks refresh button or auto-refresh triggers |
+| `time_range_change` | Time picker changes | When user changes time range in time picker |
+| `filter_added` | Ad-hoc filter addition | When user adds a new filter to the dashboard |
+| `filter_removed` | Ad-hoc filter removal | When user removes a filter from the dashboard |
+| `filter_changed` | Ad-hoc filter modification | When user changes filter values or operators |
+| `filter_restored` | Ad-hoc filter restoration | When user restores a previously applied filter |
+| `variable_value_changed` | Variable value changes | When user changes dashboard variable values |
+| `scopes_changed` | Scopes modifications | When user modifies dashboard scopes |
+
+The interactions mentioned above are reported to Echo service as well as sent to [Faro](https://grafana.com/docs/grafana-cloud/monitor-applications/frontend-observability/) as `dashboard_render` measurements:
+
+```ts
+const payload = {
+ duration: e.duration,
+ networkDuration: e.networkDuration,
+ startTs: e.startTs,
+ endTs: e.endTs,
+ totalJSHeapSize: e.totalJSHeapSize,
+ usedJSHeapSize: e.usedJSHeapSize,
+ jsHeapSizeLimit: e.jsHeapSizeLimit,
+ timeSinceBoot: performance.measure('time_since_boot', 'frontend_boot_js_done_time_seconds').duration,
+};
+
+reportInteraction('dashboard_render', {
+ interactionType: e.origin,
+ uid,
+ ...payload,
+});
+
+logMeasurement(`dashboard_render`, payload, { interactionType: e.origin, dashboard: uid, title: title });
+```
+
+### Interaction Origin Mapping
+
+The profiling system uses profiler event's `origin` directly as the `interactionType`, providing direct mapping between user actions and performance measurements.
+
+## Profiling Implementation
+
+### Profile Data Structure
+
+Each interaction profile event captures:
+
+```typescript
+interface SceneInteractionProfileEvent {
+ origin: string; // Interaction type
+ duration: number; // Total interaction duration
+ networkDuration: number; // Network requests duration
+ totalJSHeapSize: number; // JavaScript heap size metrics
+ usedJSHeapSize: number; // Used JavaScript heap size
+ jsHeapSizeLimit: number; // JavaScript heap size limit
+ startTs: number; // Profile start timestamp
+ endTs: number; // Profile end timestamp
+}
+```
+
+### Collected Metrics
+
+For each tracked interaction, the system collects:
+
+- **Dashboard Metadata**: UID, title
+- **Performance Metrics**: Duration, network duration
+- **Memory Metrics**: JavaScript heap usage statistics
+- **Timing Information**: Time since boot, profile start and end timestamps
+- **Interaction Context**: Type of user interaction
+
+## Debugging and Development
+
+### Enable Profiler Debug Logging
+
+To observe profiling events in the browser console:
+
+```javascript
+localStorage.setItem('grafana.debug.scenes', 'true');
+```
+
+#### Console Output
+
+When debug logging is enabled, you'll see console logs for each profiling event:
+
+```
+SceneRenderProfiler: Profile started: {origin: , crumbs: Array(0)}
+... // intermediate steps adding profile crumbs
+SceneRenderProfiler: Stopped recording, total measured time (network included): 2123
+```
+
+### Enable Echo Service Debug Logging
+
+To observe Echo events in the browser console:
+
+```javascript
+_debug.echo.enable();
+```
+
+#### Console Output
+
+When Echo debug logging is enabled, you'll see console logs for each profiling event captured by Echo service:
+
+```
+[EchoSrv: interaction event]: {interactionName: 'dashboard_render', properties: {…}, meta: {…}}
+```
+
+### Browser Performance Profiler
+
+Dashboard interactions can be recorded in the browser's performance profiler, where they appear as:
+
+```
+Dashboard Interaction
+```
+
+## Analytics Integration
+
+### Interaction Reporting
+
+Performance data is integrated with Grafana's analytics system through:
+
+- **`reportInteraction`**: Reports interaction events to Echo service with performance data
+- **`logMeasurement`**: Records Faro's performance measurements with metadata
+
+### Data Collection
+
+The system reports the following data for each interaction:
+
+```typescript
+{
+ interactionType: string, // Type of interaction
+ uid: string, // Dashboard UID
+ duration: number, // Total duration
+ networkDuration: number, // Network time
+ startTs: number, // Profile start timestamp
+ endTs: number, // Profile end timestamp
+ totalJSHeapSize: number, // Memory metrics
+ usedJSHeapSize: number,
+ jsHeapSizeLimit: number,
+ timeSinceBoot: number // Time since frontend boot
+}
+```
+
+## Implementation Details
+
+The profiler is integrated into dashboard creation paths and uses a singleton pattern to share profiler instances across dashboard reloads. The performance tracking is implemented using the `SceneRenderProfiler` from the `@grafana/scenes` library.
+
+## Related Documentation
+
+- [PR #858 - Add SceneRenderProfiler to scenes](https://github.com/grafana/scenes/pull/858)
+- [PR #99629 - Dashboard render performance metrics](https://github.com/grafana/grafana/pull/99629)
+- [PR #108658 - Dashboard: Tweak interaction tracking](https://github.com/grafana/grafana/pull/108658)
+- [PR #1195 - Enhance SceneRenderProfiler with additional interaction tracking](https://github.com/grafana/scenes/pull/1195)
+- [PR #1198 - Make SceneRenderProfiler optional and injectable](https://github.com/grafana/scenes/pull/1198)
+- [PR #1199 - SceneRenderProfiler: add start and end timestamps to profile events](https://github.com/grafana/scenes/pull/1199)
diff --git a/public/app/features/dashboard/state/DashboardMigrator.ts b/public/app/features/dashboard/state/DashboardMigrator.ts
index da9a51195f1..6b82290c993 100644
--- a/public/app/features/dashboard/state/DashboardMigrator.ts
+++ b/public/app/features/dashboard/state/DashboardMigrator.ts
@@ -927,6 +927,10 @@ export class DashboardMigrator {
}
}
+ if (oldVersion < 42) {
+ panelUpgrades.push(migrateHideFromFunctionality);
+ }
+
/**
* -==- Add migration here -==-
* Your migration should go below the previous
@@ -1480,3 +1484,23 @@ function ensureXAxisVisibility(panel: PanelModel) {
return panel;
}
+
+function migrateHideFromFunctionality(panel: PanelModel) {
+ // migrate overrides with hideFrom.viz = true to also set tooltip = true
+ // this includes the __systemRef override
+ if (panel.fieldConfig && panel.fieldConfig.overrides) {
+ panel.fieldConfig.overrides = panel.fieldConfig.overrides.map((override) => {
+ if (override.properties) {
+ override.properties = override.properties.map((property) => {
+ if (property.id === 'custom.hideFrom' && property.value?.viz === true) {
+ property.value.tooltip = true;
+ }
+ return property;
+ });
+ }
+ return override;
+ });
+ }
+
+ return panel;
+}
diff --git a/public/app/features/explore/Logs/Logs.tsx b/public/app/features/explore/Logs/Logs.tsx
index a6d4040a446..fbac705c277 100644
--- a/public/app/features/explore/Logs/Logs.tsx
+++ b/public/app/features/explore/Logs/Logs.tsx
@@ -52,6 +52,7 @@ import { ControlledLogRows } from 'app/features/logs/components/ControlledLogRow
import { InfiniteScroll } from 'app/features/logs/components/InfiniteScroll';
import { LogRows } from 'app/features/logs/components/LogRows';
import { LogRowContextModal } from 'app/features/logs/components/log-context/LogRowContextModal';
+import { LogLineContext } from 'app/features/logs/components/panel/LogLineContext';
import { LogList, LogListControlOptions } from 'app/features/logs/components/panel/LogList';
import { isDedupStrategy, isLogsSortOrder } from 'app/features/logs/components/panel/LogListContext';
import { LogLevelColor, dedupLogRows } from 'app/features/logs/logsModel';
@@ -767,7 +768,7 @@ const UnthemedLogs: React.FunctionComponent = (props: Props) => {
return (
<>
- {getRowContext && contextRow && (
+ {(!config.featureToggles.newLogsPanel || !config.featureToggles.newLogContext) && getRowContext && contextRow && (
= (props: Props) => {
timeZone={timeZone}
/>
)}
+ {config.featureToggles.newLogsPanel && config.featureToggles.newLogContext && getRowContext && contextRow && (
+ getRowContext(row, contextRow, options)}
+ getRowContextQuery={getRowContextQuery}
+ getLogRowContextUi={getLogRowContextUi}
+ logOptionsStorageKey={SETTING_KEY_ROOT}
+ timeZone={timeZone}
+ displayedFields={displayedFields}
+ onClickShowField={showField}
+ onClickHideField={hideField}
+ />
+ )}
void;
closeDrawer: () => void;
isDrawerOpen: boolean;
@@ -32,7 +37,7 @@ export type QueryLibraryContextType = {
* Opens a modal for adding a query to the library.
* @param query Query to be saved
* @param options.onSave Callback that will be called after the query is saved.
- * @param options.context Used for tracking. Should identify the context this is called from, like 'explore' or
+ * @param options.context Used for rendering QueryEditor. Should identify the context this is called from, like 'explore' or
* 'dashboard'.
* @param options.title Default title for the modal, can be overridden by the query title.
*/
@@ -46,8 +51,9 @@ export type QueryLibraryContextType = {
* Returns a predefined small button that can be used to save a query to the library.
* @param query
*/
- renderSaveQueryButton: (query: DataQuery) => ReactNode;
+ renderSaveQueryButton: (query: DataQuery, app?: CoreApp) => ReactNode;
queryLibraryEnabled: boolean;
+ context: string;
};
export const QueryLibraryContext = createContext({
@@ -63,6 +69,7 @@ export const QueryLibraryContext = createContext({
},
queryLibraryEnabled: false,
+ context: 'unknown',
});
export function useQueryLibraryContext() {
diff --git a/public/app/features/explore/QueryLibrary/mocks.tsx b/public/app/features/explore/QueryLibrary/mocks.tsx
index 9126fdabe42..e59732870a9 100644
--- a/public/app/features/explore/QueryLibrary/mocks.tsx
+++ b/public/app/features/explore/QueryLibrary/mocks.tsx
@@ -17,6 +17,7 @@ export function QueryLibraryContextProviderMock(props: PropsWithChildren)
closeAddQueryModal: jest.fn(),
renderSaveQueryButton: jest.fn(),
queryLibraryEnabled: Boolean(props.queryLibraryEnabled),
+ context: 'explore',
}}
>
{props.children}
diff --git a/public/app/features/explore/RichHistory/RichHistoryAddToLibrary.tsx b/public/app/features/explore/RichHistory/RichHistoryAddToLibrary.tsx
index 6c70ac1bf89..e99c0e6cebc 100644
--- a/public/app/features/explore/RichHistory/RichHistoryAddToLibrary.tsx
+++ b/public/app/features/explore/RichHistory/RichHistoryAddToLibrary.tsx
@@ -22,7 +22,7 @@ export const RichHistoryAddToLibrary = ({ query }: Props) => {
variant="secondary"
aria-label={buttonLabel}
onClick={() => {
- openAddQueryModal(query, { onSave: () => setHasBeenSaved(true), context: 'richHistory' });
+ openAddQueryModal(query, { onSave: () => setHasBeenSaved(true), context: 'rich-history' });
}}
>
{buttonLabel}
diff --git a/public/app/features/explore/SecondaryActions.tsx b/public/app/features/explore/SecondaryActions.tsx
index cd9d25b860b..b5b30aaae1d 100644
--- a/public/app/features/explore/SecondaryActions.tsx
+++ b/public/app/features/explore/SecondaryActions.tsx
@@ -1,6 +1,6 @@
import { css } from '@emotion/css';
-import { GrafanaTheme2 } from '@grafana/data';
+import { CoreApp, GrafanaTheme2 } from '@grafana/data';
import { selectors } from '@grafana/e2e-selectors';
import { Trans, t } from '@grafana/i18n';
import { ToolbarButton, useTheme2 } from '@grafana/ui';
@@ -76,7 +76,11 @@ export function SecondaryActions({
data-testid={selectors.pages.Explore.General.addFromQueryLibrary}
aria-label={t('explore.secondary-actions.add-from-query-library', 'Add query from library')}
variant="canvas"
- onClick={() => openQueryLibraryDrawer(activeDatasources, onSelectQueryFromLibrary)}
+ onClick={() =>
+ openQueryLibraryDrawer(activeDatasources, onSelectQueryFromLibrary, {
+ context: CoreApp.Explore,
+ })
+ }
icon="plus"
>
Add query from library
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.test.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.test.tsx
index 597b6d125c6..908126b649b 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.test.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.test.tsx
@@ -96,7 +96,7 @@ describe('AccordianKeyValues test', () => {
it('renders the summary instead of the table when it is not expanded', () => {
setupAccordian({ isOpen: false } as AccordianKeyValuesProps);
- expect(screen.getByRole('switch', { name: 'test accordian: span.kind client omg mos-def' })).toBeInTheDocument();
+ expect(screen.getByRole('switch', { name: 'test accordian span.kind client omg mos-def' })).toBeInTheDocument();
expect(screen.queryByRole('table')).not.toBeInTheDocument();
expect(screen.queryAllByRole('cell')).toHaveLength(0);
});
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.tsx
index d8d621cf218..93822458ac7 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianKeyValues.tsx
@@ -17,7 +17,7 @@ import cx from 'classnames';
import * as React from 'react';
import { GrafanaTheme2, TraceKeyValuePair } from '@grafana/data';
-import { Icon, useStyles2 } from '@grafana/ui';
+import { Counter, Icon, useStyles2 } from '@grafana/ui';
import { autoColor } from '../../Theme';
import TNil from '../../types/TNil';
@@ -43,6 +43,10 @@ export const getStyles = (theme: GrafanaTheme2) => {
background: autoColor(theme, '#e8e8e8'),
},
}),
+ headerLabel: css({
+ width: '120px',
+ display: 'inline-block',
+ }),
headerEmpty: css({
label: 'headerEmpty',
background: 'none',
@@ -87,6 +91,9 @@ export type AccordianKeyValuesProps = {
logName?: string;
highContrast?: boolean;
interactive?: boolean;
+ onlyValues?: boolean;
+ showSummary?: boolean;
+ showCountBadge?: boolean;
isOpen: boolean;
label: string | React.ReactNode;
linksGetter?: ((pairs: TraceKeyValuePair[], index: number) => KeyValuesTableLink[]) | TNil;
@@ -127,6 +134,9 @@ export default function AccordianKeyValues({
isOpen,
label,
linksGetter,
+ onlyValues = false,
+ showSummary = true,
+ showCountBadge = false,
onToggle = null,
}: AccordianKeyValuesProps) {
const isEmpty = (!Array.isArray(data) || !data.length) && !logName;
@@ -148,7 +158,7 @@ export default function AccordianKeyValues({
};
}
- const showDataSummaryFields = data.length > 0 && !isOpen;
+ const showDataSummaryFields = showSummary && data.length > 0 && !isOpen;
return (
);
}
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.test.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.test.tsx
index 18085f3fdcb..19ff85b2dd7 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.test.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.test.tsx
@@ -103,10 +103,10 @@ describe('AccordianLogs tests', () => {
setup({ isOpen: true, openedItems: new Set() } as AccordianLogsProps);
expect(
screen.getByRole('switch', {
- name: '15μs (foo event name) : message oh the next log message more stuff',
+ name: '15μs (foo event name) message oh the next log message more stuff',
})
).toBeInTheDocument();
- expect(screen.getByRole('switch', { name: '5μs: message oh the log message something else' })).toBeInTheDocument();
+ expect(screen.getByRole('switch', { name: '5μs message oh the log message something else' })).toBeInTheDocument();
});
it('renders event name and duration when events list is open', () => {
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.tsx
index 5584760b04b..b708b7f3ec8 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianLogs.tsx
@@ -32,14 +32,12 @@ const getStyles = (theme: GrafanaTheme2) => {
AccordianLogs: css({
label: 'AccordianLogs',
position: 'relative',
- marginBottom: '0.25rem',
}),
AccordianLogsHeader: css({
label: 'AccordianLogsHeader',
color: 'inherit',
display: 'flex',
alignItems: 'center',
- padding: '0.25rem 0.1em',
'&:hover': {
background: autoColor(theme, '#e8e8e8'),
},
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.test.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.test.tsx
index d7296055bfc..14fa5187396 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.test.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.test.tsx
@@ -75,7 +75,7 @@ describe('AccordianReferences tests', () => {
it('renders the correct number of references', () => {
setup();
- expect(screen.getByRole('switch', { name: 'References (3)' })).toBeInTheDocument();
+ expect(screen.getByRole('switch', { name: 'References 3' })).toBeInTheDocument();
});
it('content doesnt show when not expanded', () => {
@@ -88,7 +88,7 @@ describe('AccordianReferences tests', () => {
it('renders the content when it is expanded', () => {
setup({ isOpen: true } as AccordianReferencesProps);
- expect(screen.getByRole('switch', { name: 'References (3)' })).toBeInTheDocument();
+ expect(screen.getByRole('switch', { name: 'References 3' })).toBeInTheDocument();
expect(screen.getAllByRole('link', { name: /^service\d\sop\d/ })).toHaveLength(2);
expect(screen.getByRole('link', { name: /^View\sLinked/ })).toBeInTheDocument();
});
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.tsx
index de60443707e..10c31897ec6 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/AccordianReferences.tsx
@@ -17,7 +17,7 @@ import * as React from 'react';
import { Field, GrafanaTheme2, LinkModel } from '@grafana/data';
import { Trans, t } from '@grafana/i18n';
-import { Icon, useStyles2 } from '@grafana/ui';
+import { Counter, Icon, useStyles2 } from '@grafana/ui';
import { autoColor } from '../../Theme';
import { TraceSpanReference } from '../../types/trace';
@@ -36,16 +36,14 @@ const getStyles = (theme: GrafanaTheme2) => ({
}),
AccordianReferences: css({
label: 'AccordianReferences',
- border: `1px solid ${autoColor(theme, '#d8d8d8')}`,
position: 'relative',
marginBottom: '0.25rem',
}),
AccordianReferencesHeader: css({
label: 'AccordianReferencesHeader',
- background: autoColor(theme, '#e4e4e4'),
color: 'inherit',
display: 'block',
- padding: '0.25rem 0.5rem',
+ padding: '0.25rem 0',
'&:hover': {
background: autoColor(theme, '#dadada'),
},
@@ -223,7 +221,7 @@ const AccordianReferences = ({
{label}
diff --git a/packages/grafana-ui/src/components/VizTooltip/types.ts b/packages/grafana-ui/src/components/VizTooltip/types.ts
index de4fbae048f..bd4f66488fa 100644
--- a/packages/grafana-ui/src/components/VizTooltip/types.ts
+++ b/packages/grafana-ui/src/components/VizTooltip/types.ts
@@ -27,6 +27,7 @@ export interface VizTooltipItem {
colorPlacement?: ColorPlacement;
isActive?: boolean;
lineStyle?: LineStyle;
+ isHiddenFromViz?: boolean;
// internal/tmp for sorting
numeric?: number;
diff --git a/packages/grafana-ui/src/components/VizTooltip/utils.ts b/packages/grafana-ui/src/components/VizTooltip/utils.ts
index e192340974f..d14d636caa5 100644
--- a/packages/grafana-ui/src/components/VizTooltip/utils.ts
+++ b/packages/grafana-ui/src/components/VizTooltip/utils.ts
@@ -47,6 +47,8 @@ export const calculateTooltipPosition = (
export const getColorIndicatorClass = (colorIndicator: string, styles: ColorIndicatorStyles) => {
switch (colorIndicator) {
+ case ColorIndicator.series:
+ return styles.series;
case ColorIndicator.value:
return styles.value;
case ColorIndicator.hexagon:
@@ -80,7 +82,8 @@ export const getContentItems = (
mode: TooltipDisplayMode,
sortOrder: SortOrder,
fieldFilter = (field: Field) => true,
- hideZeros = false
+ hideZeros = false,
+ _restFields?: Field[]
): VizTooltipItem[] => {
let rows: VizTooltipItem[] = [];
@@ -93,8 +96,7 @@ export const getContentItems = (
field === xField ||
field.type === FieldType.time ||
!fieldFilter(field) ||
- field.config.custom?.hideFrom?.tooltip ||
- field.config.custom?.hideFrom?.viz
+ field.config.custom?.hideFrom?.tooltip
) {
continue;
}
@@ -130,15 +132,7 @@ export const getContentItems = (
? Number.MIN_SAFE_INTEGER
: Number.MAX_SAFE_INTEGER;
- const colorMode = getFieldColorModeForField(field);
-
- let colorIndicator = ColorIndicator.series;
- let colorPlacement = ColorPlacement.first;
-
- if (colorMode.isByValue) {
- colorIndicator = ColorIndicator.value;
- colorPlacement = ColorPlacement.trailing;
- }
+ const { colorIndicator, colorPlacement } = getIndicatorAndPlacement(field);
rows.push({
label: field.state?.displayName ?? field.name,
@@ -152,6 +146,23 @@ export const getContentItems = (
});
}
+ _restFields?.forEach((field) => {
+ if (!field.config.custom?.hideFrom?.tooltip) {
+ const { colorIndicator, colorPlacement } = getIndicatorAndPlacement(field);
+ const display = field.display!(field.values[dataIdxs[0]!]);
+
+ rows.push({
+ label: field.state?.displayName ?? field.name,
+ value: formattedValueToString(display),
+ color: FALLBACK_COLOR,
+ colorIndicator,
+ colorPlacement,
+ lineStyle: field.config.custom?.lineStyle,
+ isHiddenFromViz: true,
+ });
+ }
+ });
+
if (sortOrder !== SortOrder.None && rows.length > 1) {
const cmp = allNumeric ? numberCmp : stringCmp;
const mult = sortOrder === SortOrder.Descending ? -1 : 1;
@@ -160,3 +171,17 @@ export const getContentItems = (
return rows;
};
+
+const getIndicatorAndPlacement = (field: Field) => {
+ const colorMode = getFieldColorModeForField(field);
+
+ let colorIndicator = ColorIndicator.series;
+ let colorPlacement = ColorPlacement.first;
+
+ if (colorMode.isByValue) {
+ colorIndicator = ColorIndicator.value;
+ colorPlacement = ColorPlacement.trailing;
+ }
+
+ return { colorIndicator, colorPlacement };
+};
diff --git a/packages/grafana-ui/src/index.ts b/packages/grafana-ui/src/index.ts
index db367a8790e..d35561e598f 100644
--- a/packages/grafana-ui/src/index.ts
+++ b/packages/grafana-ui/src/index.ts
@@ -358,6 +358,7 @@ export { type UPlotConfigPrepFn } from './components/uPlot/config/UPlotConfigBui
export * from './components/PanelChrome/types';
export { Label as BrowserLabel } from './components/BrowserLabel/Label';
export { PanelContainer } from './components/PanelContainer/PanelContainer';
+export { VariablesInputModal } from './components/Actions/VariablesInputModal';
// -----------------------------------------------------
// Graveyard: exported, but no longer used internally
diff --git a/pkg/api/api.go b/pkg/api/api.go
index 12de28eaa90..1561c28ea21 100644
--- a/pkg/api/api.go
+++ b/pkg/api/api.go
@@ -176,7 +176,6 @@ func (hs *HTTPServer) registerRoutes() {
r.Get("/import/dashboard", reqSignedIn, hs.Index)
r.Get("/dashboards/", reqSignedIn, hs.Index)
r.Get("/dashboards/*", reqSignedIn, hs.Index)
- r.Get("/goto/:uid", reqSignedIn, hs.redirectFromShortURL, hs.Index)
if hs.Cfg.PublicDashboardsEnabled {
// list public dashboards
@@ -264,6 +263,9 @@ func (hs *HTTPServer) registerRoutes() {
providerParam := ac.Parameter(":provider")
r.Get("/admin/authentication/:provider", authorize(ac.EvalPermission(ac.ActionSettingsRead, ac.ScopeSettingsOAuth(providerParam))), hs.Index)
+ // ShortURL API
+ hs.registerShortURLAPI(r)
+
// authed api
r.Group("/api", func(apiRoute routing.RouteRegister) {
// user (signed in)
@@ -549,9 +551,6 @@ func (hs *HTTPServer) registerRoutes() {
// Some channels may have info
liveRoute.Get("/info/*", routing.Wrap(hs.Live.HandleInfoHTTP))
}, requestmeta.SetSLOGroup(requestmeta.SLOGroupNone))
-
- // short urls
- apiRoute.Post("/short-urls", routing.Wrap(hs.createShortURL))
}, reqSignedIn)
// admin api
diff --git a/pkg/api/short_url.go b/pkg/api/short_url.go
index 22cb57a31f8..9be4bdab028 100644
--- a/pkg/api/short_url.go
+++ b/pkg/api/short_url.go
@@ -7,6 +7,8 @@ import (
"github.com/grafana/grafana/pkg/api/dtos"
"github.com/grafana/grafana/pkg/api/response"
+ "github.com/grafana/grafana/pkg/api/routing"
+ "github.com/grafana/grafana/pkg/middleware"
contextmodel "github.com/grafana/grafana/pkg/services/contexthandler/model"
"github.com/grafana/grafana/pkg/services/shorturls"
"github.com/grafana/grafana/pkg/setting"
@@ -14,6 +16,12 @@ import (
"github.com/grafana/grafana/pkg/web"
)
+func (hs *HTTPServer) registerShortURLAPI(apiRoute routing.RouteRegister) {
+ reqSignedIn := middleware.ReqSignedIn
+ apiRoute.Post("/api/short-urls", reqSignedIn, hs.createShortURL)
+ apiRoute.Get("/goto/:uid", reqSignedIn, hs.redirectFromShortURL, hs.Index)
+}
+
// createShortURL handles requests to create short URLs.
func (hs *HTTPServer) createShortURL(c *contextmodel.ReqContext) response.Response {
cmd := dtos.CreateShortURLCmd{}
diff --git a/pkg/apimachinery/identity/static.go b/pkg/apimachinery/identity/static.go
index 6c0473ae0a9..afe505e261e 100644
--- a/pkg/apimachinery/identity/static.go
+++ b/pkg/apimachinery/identity/static.go
@@ -106,7 +106,7 @@ func (u *StaticRequester) GetExtra() map[string][]string {
}
result := map[string][]string{}
- if u.AccessTokenClaims.Rest.ServiceIdentity != "" {
+ if u.AccessTokenClaims != nil && u.AccessTokenClaims.Rest.ServiceIdentity != "" {
result[authnlib.ServiceIdentityKey] = []string{u.AccessTokenClaims.Rest.ServiceIdentity}
}
return result
diff --git a/pkg/apiserver/go.mod b/pkg/apiserver/go.mod
index 24e2c11626c..4efad4e627d 100644
--- a/pkg/apiserver/go.mod
+++ b/pkg/apiserver/go.mod
@@ -5,7 +5,7 @@ go 1.24.5
require (
github.com/google/go-cmp v0.7.0
github.com/grafana/authlib/types v0.0.0-20250710201142-9542f2f28d43
- github.com/grafana/grafana-app-sdk/logging v0.40.0
+ github.com/grafana/grafana-app-sdk/logging v0.40.1
github.com/grafana/grafana/pkg/apimachinery v0.0.0-20250514132646-acbc7b54ed9e
github.com/prometheus/client_golang v1.22.0
github.com/stretchr/testify v1.10.0
diff --git a/pkg/apiserver/go.sum b/pkg/apiserver/go.sum
index 5547406bf98..dc2e70d63a1 100644
--- a/pkg/apiserver/go.sum
+++ b/pkg/apiserver/go.sum
@@ -84,8 +84,8 @@ github.com/grafana/authlib/types v0.0.0-20250710201142-9542f2f28d43 h1:NlkGMnVi/
github.com/grafana/authlib/types v0.0.0-20250710201142-9542f2f28d43/go.mod h1:qeWYbnWzaYGl88JlL9+DsP1GT2Cudm58rLtx13fKZdw=
github.com/grafana/dskit v0.0.0-20250611075409-46f51e1ce914 h1:qcSGhr691f1mmPHwg2svGyO40Ex92G02aOyHzP6XHCE=
github.com/grafana/dskit v0.0.0-20250611075409-46f51e1ce914/go.mod h1:OiN4P4aC6LwLzLbEupH3Ue83VfQoNMfG48rsna8jI/E=
-github.com/grafana/grafana-app-sdk/logging v0.40.0 h1:3LHA0pFM9mGGFyq6qoHvJOGEe0H0OVhzzlOKd4Ehu+E=
-github.com/grafana/grafana-app-sdk/logging v0.40.0/go.mod h1:otUD9XpJD7A5sCLb8mcs9hIXGdeV6lnhzVwe747g4RU=
+github.com/grafana/grafana-app-sdk/logging v0.40.1 h1:ru+GqbaQk6jthA5l2Yo1WI/JbNXKNQmLiqNrxz7HGP4=
+github.com/grafana/grafana-app-sdk/logging v0.40.1/go.mod h1:otUD9XpJD7A5sCLb8mcs9hIXGdeV6lnhzVwe747g4RU=
github.com/grafana/grafana/pkg/apimachinery v0.0.0-20250514132646-acbc7b54ed9e h1:BTKk7LHuG1kmAkucwTA7DuMbKpKvJTKrGdBmUNO4dfQ=
github.com/grafana/grafana/pkg/apimachinery v0.0.0-20250514132646-acbc7b54ed9e/go.mod h1:IA4SOwun8QyST9c5UNs/fN37XL6boXXDvRYFcFwbipg=
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 h1:UH//fgunKIs4JdUbpDl1VZCDaL56wXCB/5+wF6uHfaI=
diff --git a/pkg/cmd/grafana-cli/commands/commands.go b/pkg/cmd/grafana-cli/commands/commands.go
index f55209ac1d6..ae7f5f93f67 100644
--- a/pkg/cmd/grafana-cli/commands/commands.go
+++ b/pkg/cmd/grafana-cli/commands/commands.go
@@ -7,6 +7,7 @@ import (
"github.com/urfave/cli/v2"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/commands/datamigrations"
+ "github.com/grafana/grafana/pkg/cmd/grafana-cli/commands/secretsconsolidation"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/commands/secretsmigrations"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
@@ -184,6 +185,17 @@ var adminCommands = []*cli.Command{
},
},
},
+ {
+ Name: "secrets-consolidation",
+ Usage: "Runs an operation that re-encrypts all encrypted values in your database with new data keys",
+ Subcommands: []*cli.Command{
+ {
+ Name: "consolidate",
+ Usage: "Re-encrypts all encrypted values with new data keys and deletes the old deactivated data keys. Returns ok unless there is an error. Safe to execute multiple times.",
+ Action: runRunnerCommand(secretsconsolidation.ConsolidateSecrets),
+ },
+ },
+ },
}
var Commands = []*cli.Command{
diff --git a/pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage.go b/pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage.go
index 8e46423be87..89053e6b932 100644
--- a/pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage.go
+++ b/pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage.go
@@ -63,6 +63,12 @@ func ToUnifiedStorage(c utils.CommandLine, cfg *setting.Cfg, sqlStore db.DB) err
},
}
+ featureManager, err := featuremgmt.ProvideManagerService(cfg)
+ if err != nil {
+ return err
+ }
+ featureToggles := featuremgmt.ProvideToggles(featureManager)
+
provisioning, err := newStubProvisioning(cfg.ProvisioningPath)
if err != nil {
return err
@@ -76,9 +82,10 @@ func ToUnifiedStorage(c utils.CommandLine, cfg *setting.Cfg, sqlStore db.DB) err
nil, // no librarypanels.Service
sort.ProvideService(),
acimpl.ProvideAccessControl(featuremgmt.WithFeatures()),
+ featureToggles,
)
- client, err := newUnifiedClient(cfg, sqlStore)
+ client, err := newUnifiedClient(cfg, sqlStore, featureToggles)
if err != nil {
return err
}
@@ -215,12 +222,7 @@ func promptYesNo(prompt string) (bool, error) {
}
}
-func newUnifiedClient(cfg *setting.Cfg, sqlStore db.DB) (resource.ResourceClient, error) {
- featureManager, err := featuremgmt.ProvideManagerService(cfg)
- if err != nil {
- return nil, err
- }
- featureToggles := featuremgmt.ProvideToggles(featureManager)
+func newUnifiedClient(cfg *setting.Cfg, sqlStore db.DB, featureToggles featuremgmt.FeatureToggles) (resource.ResourceClient, error) {
return unified.ProvideUnifiedStorageClient(&unified.Options{
Cfg: cfg,
Features: featureToggles,
diff --git a/pkg/cmd/grafana-cli/commands/secretsconsolidation/secretsconsolidation.go b/pkg/cmd/grafana-cli/commands/secretsconsolidation/secretsconsolidation.go
new file mode 100644
index 00000000000..9be8e5f5a2a
--- /dev/null
+++ b/pkg/cmd/grafana-cli/commands/secretsconsolidation/secretsconsolidation.go
@@ -0,0 +1,13 @@
+package secretsconsolidation
+
+import (
+ "context"
+
+ "github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
+ "github.com/grafana/grafana/pkg/server"
+)
+
+func ConsolidateSecrets(_ utils.CommandLine, runner server.Runner) error {
+ err := runner.SecretsConsolidationService.Consolidate(context.Background())
+ return err
+}
diff --git a/pkg/registry/apis/dashboard/legacy/migrate.go b/pkg/registry/apis/dashboard/legacy/migrate.go
index 3da3a01aab8..73d01caaade 100644
--- a/pkg/registry/apis/dashboard/legacy/migrate.go
+++ b/pkg/registry/apis/dashboard/legacy/migrate.go
@@ -16,6 +16,7 @@ import (
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/infra/db"
"github.com/grafana/grafana/pkg/services/accesscontrol"
+ "github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/librarypanels"
"github.com/grafana/grafana/pkg/services/provisioning"
"github.com/grafana/grafana/pkg/services/search/sort"
@@ -51,9 +52,10 @@ func ProvideLegacyMigrator(
provisioning provisioning.ProvisioningService, // only needed for dashboard settings
libraryPanelSvc librarypanels.Service,
accessControl accesscontrol.AccessControl,
+ features featuremgmt.FeatureToggles,
) LegacyMigrator {
dbp := legacysql.NewDatabaseProvider(sql)
- return NewDashboardAccess(dbp, authlib.OrgNamespaceFormatter, nil, provisioning, libraryPanelSvc, sort.ProvideService(), accessControl)
+ return NewDashboardAccess(dbp, authlib.OrgNamespaceFormatter, nil, provisioning, libraryPanelSvc, sort.ProvideService(), accessControl, features)
}
type BlobStoreInfo struct {
@@ -309,11 +311,12 @@ func (a *dashboardSqlAccess) migrateDashboards(ctx context.Context, orgId int64,
for _, row := range rows.rejected {
id := row.Dash.Labels[utils.LabelKeyDeprecatedInternalID]
a.log.Warn("rejected dashboard",
+ "namespace", opts.Namespace,
"dashboard", row.Dash.Name,
"uid", row.Dash.UID,
"id", id,
+ "version", row.Dash.Generation,
"stackId", opts.StackID,
- "namespace", opts.Namespace,
)
opts.Progress(-2, fmt.Sprintf("rejected: id:%s, uid:%s", id, row.Dash.Name))
}
diff --git a/pkg/registry/apis/dashboard/legacy/query_dashboards.sql b/pkg/registry/apis/dashboard/legacy/query_dashboards.sql
index 75439c60d03..e8503170a2c 100644
--- a/pkg/registry/apis/dashboard/legacy/query_dashboards.sql
+++ b/pkg/registry/apis/dashboard/legacy/query_dashboards.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/sql_dashboards.go b/pkg/registry/apis/dashboard/legacy/sql_dashboards.go
index 0771f0fc3ee..f019a39a799 100644
--- a/pkg/registry/apis/dashboard/legacy/sql_dashboards.go
+++ b/pkg/registry/apis/dashboard/legacy/sql_dashboards.go
@@ -29,6 +29,7 @@ import (
"github.com/grafana/grafana/pkg/services/apiserver/endpoints/request"
gapiutil "github.com/grafana/grafana/pkg/services/apiserver/utils"
"github.com/grafana/grafana/pkg/services/dashboards"
+ "github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/libraryelements"
"github.com/grafana/grafana/pkg/services/librarypanels"
"github.com/grafana/grafana/pkg/services/provisioning"
@@ -62,6 +63,8 @@ type dashboardSqlAccess struct {
namespacer request.NamespaceMapper
provisioning provisioning.ProvisioningService
+ invalidDashboardParseFallbackEnabled bool
+
// Use for writing (not reading)
dashStore dashboards.Store
dashboardSearchClient legacysearcher.DashboardSearchClient
@@ -82,17 +85,19 @@ func NewDashboardAccess(sql legacysql.LegacyDatabaseProvider,
libraryPanelSvc librarypanels.Service,
sorter sort.Service,
accessControl accesscontrol.AccessControl,
+ features featuremgmt.FeatureToggles,
) DashboardAccess {
dashboardSearchClient := legacysearcher.NewDashboardSearchClient(dashStore, sorter)
return &dashboardSqlAccess{
- sql: sql,
- namespacer: namespacer,
- dashStore: dashStore,
- provisioning: provisioning,
- dashboardSearchClient: *dashboardSearchClient,
- libraryPanelSvc: libraryPanelSvc,
- accessControl: accessControl,
- log: log.New("dashboard.legacysql"),
+ sql: sql,
+ namespacer: namespacer,
+ dashStore: dashStore,
+ provisioning: provisioning,
+ dashboardSearchClient: *dashboardSearchClient,
+ libraryPanelSvc: libraryPanelSvc,
+ accessControl: accessControl,
+ log: log.New("dashboard.legacysql"),
+ invalidDashboardParseFallbackEnabled: features.IsEnabled(context.Background(), featuremgmt.FlagScanRowInvalidDashboardParseFallbackEnabled),
}
}
@@ -176,7 +181,7 @@ func (r *rowsWrapper) Next() bool {
r.row, err = r.a.scanRow(r.rows, r.history)
if err != nil {
r.a.log.Error("error scanning dashboard", "error", err)
- if len(r.rejected) > 0 || r.row == nil {
+ if len(r.rejected) > 100 || r.row == nil {
r.err = fmt.Errorf("too many rejected rows (%d) %w", len(r.rejected), err)
return false
}
@@ -228,6 +233,51 @@ func (r *rowsWrapper) Value() []byte {
return b
}
+func generateFallbackDashboard(data []byte, title, uid string) ([]byte, error) {
+ generatedDashboard := map[string]interface{}{
+ "editable": true,
+ "id": 1,
+ "panels": []map[string]interface{}{
+ {
+ "description": "The JSON is invalid. You can import it again after fixing it.",
+ "gridPos": map[string]interface{}{"h": 8, "w": 24, "x": 0, "y": 0},
+ "id": 1,
+ "options": map[string]interface{}{
+ "code": map[string]interface{}{"language": "plaintext", "showLineNumbers": false, "showMiniMap": false},
+ "content": string(data),
+ "mode": "code",
+ },
+ "title": "Invalid dashboard",
+ "type": "text",
+ },
+ },
+ "schemaVersion": 41,
+ "title": title,
+ "uid": uid,
+ "version": 3,
+ }
+ return json.Marshal(generatedDashboard)
+}
+
+func (a *dashboardSqlAccess) parseDashboard(dash *dashboardV1.Dashboard, data []byte, id int64, title string) error {
+ if err := dash.Spec.UnmarshalJSON(data); err != nil {
+ a.log.Warn("error unmarshalling dashboard spec. Generating fallback dashboard data", "error", err, "uid", dash.UID, "name", dash.Name)
+ dash.Spec = *dashboardV0.NewDashboardSpec()
+
+ dashboardData, err := generateFallbackDashboard(data, title, string(dash.UID))
+ if err != nil {
+ a.log.Warn("error generating fallback dashboard data", "error", err, "uid", dash.UID, "name", dash.Name)
+ return err
+ }
+
+ if err = dash.Spec.UnmarshalJSON(dashboardData); err != nil {
+ a.log.Warn("error unmarshalling fallback dashboard data", "error", err, "uid", dash.UID, "name", dash.Name)
+ return err
+ }
+ }
+ return nil
+}
+
func (a *dashboardSqlAccess) scanRow(rows *sql.Rows, history bool) (*dashboardRow, error) {
dash := &dashboardV1.Dashboard{
TypeMeta: dashboardV1.DashboardResourceInfo.TypeMeta(),
@@ -238,6 +288,7 @@ func (a *dashboardSqlAccess) scanRow(rows *sql.Rows, history bool) (*dashboardRo
var dashboard_id int64
var orgId int64
var folder_uid sql.NullString
+ var title string
var updated time.Time
var updatedBy sql.NullString
var updatedByID sql.NullInt64
@@ -257,7 +308,7 @@ func (a *dashboardSqlAccess) scanRow(rows *sql.Rows, history bool) (*dashboardRo
var data []byte // the dashboard JSON
var version int64
- err := rows.Scan(&orgId, &dashboard_id, &dash.Name, &folder_uid,
+ err := rows.Scan(&orgId, &dashboard_id, &dash.Name, &title, &folder_uid,
&deleted, &plugin_id,
&origin_name, &origin_path, &origin_hash, &origin_ts,
&created, &createdBy, &createdByID,
@@ -286,6 +337,7 @@ func (a *dashboardSqlAccess) scanRow(rows *sql.Rows, history bool) (*dashboardRo
dash.SetCreationTimestamp(metav1.NewTime(created))
meta, err := utils.MetaAccessor(dash)
if err != nil {
+ a.log.Debug("failed to get meta accessor for dashboard", "error", err, "uid", dash.UID, "name", dash.Name, "version", version)
return nil, err
}
meta.SetUpdatedTimestamp(&updated)
@@ -331,9 +383,14 @@ func (a *dashboardSqlAccess) scanRow(rows *sql.Rows, history bool) (*dashboardRo
}
if len(data) > 0 {
- err = dash.Spec.UnmarshalJSON(data)
- if err != nil {
- return row, fmt.Errorf("JSON unmarshal error for: %s // %w", dash.Name, err)
+ if a.invalidDashboardParseFallbackEnabled {
+ if err := a.parseDashboard(dash, data, dashboard_id, title); err != nil {
+ return row, err
+ }
+ } else {
+ if err := dash.Spec.UnmarshalJSON(data); err != nil {
+ return row, fmt.Errorf("JSON unmarshal error for: %s // %w", dash.Name, err)
+ }
}
}
// Ignore any saved values for id/version/uid
diff --git a/pkg/registry/apis/dashboard/legacy/sql_dashboards_test.go b/pkg/registry/apis/dashboard/legacy/sql_dashboards_test.go
index 41da615b6ff..da8d2bfe25d 100644
--- a/pkg/registry/apis/dashboard/legacy/sql_dashboards_test.go
+++ b/pkg/registry/apis/dashboard/legacy/sql_dashboards_test.go
@@ -32,13 +32,15 @@ func TestScanRow(t *testing.T) {
provisioner := provisioning.NewProvisioningServiceMock(context.Background())
provisioner.GetDashboardProvisionerResolvedPathFunc = func(name string) string { return "provisioner" }
store := &dashboardSqlAccess{
- namespacer: func(_ int64) string { return "default" },
- provisioning: provisioner,
- log: log.New("test"),
+ namespacer: func(_ int64) string { return "default" },
+ provisioning: provisioner,
+ log: log.New("test"),
+ invalidDashboardParseFallbackEnabled: false,
}
- columns := []string{"orgId", "dashboard_id", "name", "folder_uid", "deleted", "plugin_id", "origin_name", "origin_path", "origin_hash", "origin_ts", "created", "createdBy", "createdByID", "updated", "updatedBy", "updatedByID", "version", "message", "data", "api_version"}
+ columns := []string{"orgId", "dashboard_id", "name", "title", "folder_uid", "deleted", "plugin_id", "origin_name", "origin_path", "origin_hash", "origin_ts", "created", "createdBy", "createdByID", "updated", "updatedBy", "updatedByID", "version", "message", "data", "api_version"}
id := int64(100)
+ uid := "someuid"
title := "Test Dashboard"
folderUID := "folder123"
timestamp := time.Now()
@@ -49,7 +51,7 @@ func TestScanRow(t *testing.T) {
updatedUser := "updator"
t.Run("Should scan a valid row correctly", func(t *testing.T) {
- rows := sqlmock.NewRows(columns).AddRow(1, id, title, folderUID, nil, "", "", "", "", 0, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
+ rows := sqlmock.NewRows(columns).AddRow(1, id, uid, title, folderUID, nil, "", "", "", "", 0, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
mock.ExpectQuery("SELECT *").WillReturnRows(rows)
resultRows, err := mockDB.Query("SELECT *")
require.NoError(t, err)
@@ -59,7 +61,7 @@ func TestScanRow(t *testing.T) {
row, err := store.scanRow(resultRows, false)
require.NoError(t, err)
require.NotNil(t, row)
- require.Equal(t, "Test Dashboard", row.Dash.Name)
+ require.Equal(t, uid, row.Dash.Name)
require.Equal(t, version, row.RV) // rv should be the dashboard version
require.Equal(t, common.Unstructured{
Object: map[string]interface{}{"key": "value"},
@@ -80,7 +82,7 @@ func TestScanRow(t *testing.T) {
})
t.Run("File provisioned dashboard should have annotations", func(t *testing.T) {
- rows := sqlmock.NewRows(columns).AddRow(1, id, title, folderUID, nil, "", "provisioner", pathToFile, "hashing", 100000, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
+ rows := sqlmock.NewRows(columns).AddRow(1, id, uid, title, folderUID, nil, "", "provisioner", pathToFile, "hashing", 100000, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
mock.ExpectQuery("SELECT *").WillReturnRows(rows)
resultRows, err := mockDB.Query("SELECT *")
require.NoError(t, err)
@@ -108,7 +110,7 @@ func TestScanRow(t *testing.T) {
})
t.Run("Plugin provisioned dashboard should have annotations", func(t *testing.T) {
- rows := sqlmock.NewRows(columns).AddRow(1, id, title, folderUID, nil, "slo", "", "", "", 0, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
+ rows := sqlmock.NewRows(columns).AddRow(1, id, uid, title, folderUID, nil, "slo", "", "", "", 0, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, []byte(`{"key": "value"}`), "vXyz")
mock.ExpectQuery("SELECT *").WillReturnRows(rows)
resultRows, err := mockDB.Query("SELECT *")
require.NoError(t, err)
@@ -144,7 +146,7 @@ func TestScanRow(t *testing.T) {
// In migration scenario, COALESCE functions return dashboard table values
// when dashboard_version values are NULL, ensuring all dashboards are migrated
rows := sqlmock.NewRows(columns).AddRow(
- 1, id, title, folderUID, nil, "", // basic dashboard fields
+ 1, id, uid, title, folderUID, nil, "", // basic dashboard fields
"", "", "", 0, // origin fields
timestamp, createdUser, 0, // created fields
// These represent COALESCED values from dashboard table (not version table)
@@ -163,7 +165,8 @@ func TestScanRow(t *testing.T) {
require.NotNil(t, row)
// Verify migration scenario works correctly with fallback data
- require.Equal(t, title, row.Dash.Name)
+ require.Equal(t, uid, row.Dash.Name)
+ require.Equal(t, "Migrated Dashboard", row.Dash.Spec.Object["title"])
require.Equal(t, migrationVersion, row.RV) // Should use COALESCEd dashboard table version
require.Equal(t, common.Unstructured{
Object: map[string]interface{}{
@@ -187,6 +190,72 @@ func TestScanRow(t *testing.T) {
require.Equal(t, folderUID, meta.GetFolder())
require.Equal(t, "dashboard.grafana.app/"+migrationAPIVersion, row.Dash.APIVersion)
})
+
+ t.Run("should follow dashboard template when failing to unmarshal dashboard if feature flag X is enabled", func(t *testing.T) {
+ // row with bad data
+ badData := []byte(`{"rows":[{"panels":[{"targets":[{"refId":"A","target":"aliasSub(alias, '^(.{27}).+', '\1...')"}]}]}]}`)
+ rows := sqlmock.NewRows(columns).AddRow(1, id, uid, title, folderUID, nil, "", "", "", "", 0, timestamp, createdUser, 0, timestamp, updatedUser, 0, version, message, badData, "vXyz")
+ mock.ExpectQuery("SELECT *").WillReturnRows(rows)
+ resultRows, err := mockDB.Query("SELECT *")
+ require.NoError(t, err)
+ defer resultRows.Close() // nolint:errcheck
+ resultRows.Next()
+
+ row, err := store.scanRow(resultRows, false)
+ require.Error(t, err, "JSON unmarshal error for: Test Dashboard // invalid character '1' in string escape code")
+ require.NotNil(t, row)
+ // correctly scans these
+ require.Equal(t, uid, row.Dash.Name)
+ require.Equal(t, version, row.RV)
+ require.Equal(t, "default", row.Dash.Namespace)
+ require.Equal(t, &continueToken{orgId: int64(1), id: id}, row.token)
+
+ // failure case: does NOT parse the dashboard itself
+ require.Equal(t, common.Unstructured{
+ Object: nil,
+ }, row.Dash.Spec)
+
+ // store with feature flag enabled
+ store = &dashboardSqlAccess{
+ namespacer: func(_ int64) string { return "default" },
+ provisioning: provisioner,
+ log: log.New("test"),
+ invalidDashboardParseFallbackEnabled: true,
+ }
+
+ row, err = store.scanRow(resultRows, false)
+ require.NoError(t, err)
+ require.NotNil(t, row)
+ require.Equal(t, uid, row.Dash.Name)
+ require.Equal(t, version, row.RV)
+ require.Equal(t, "default", row.Dash.Namespace)
+ require.Equal(t, &continueToken{orgId: int64(1), id: id}, row.token)
+
+ // instead of failing, create dummy dashboard with broken json inlined in text panel
+ require.Equal(t, title, row.Dash.Spec.Object["title"])
+ panels, exists := row.Dash.Spec.Object["panels"]
+ require.True(t, exists, "panels property should exist")
+
+ panelsSlice, ok := panels.([]interface{})
+ require.True(t, ok, "panels should be a slice")
+ require.Len(t, panelsSlice, 1, "panels should have exactly one element")
+
+ panel, ok := panelsSlice[0].(map[string]interface{})
+ require.True(t, ok, "panel should be a map")
+
+ options, exists := panel["options"]
+ require.True(t, exists, "panel should have options property")
+
+ optionsMap, ok := options.(map[string]interface{})
+ require.True(t, ok, "options should be a map")
+
+ content, exists := optionsMap["content"]
+ require.True(t, exists, "options should have content property")
+
+ contentStr, ok := content.(string)
+ require.True(t, ok, "content should be a string")
+ require.Equal(t, string(badData), contentStr, "content should match bad json data")
+ })
}
func TestBuildSaveDashboardCommand(t *testing.T) {
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard.sql
index e3a945a53db..63f1231205e 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard_next_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard_next_page.sql
index 862725168b0..0992bbd3ce2 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard_next_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-dashboard_next_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-export_with_history.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-export_with_history.sql
index fd54e6dc949..1d57a86d450 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-export_with_history.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-export_with_history.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-folders.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-folders.sql
index 5935eb3422c..38cf8f56bea 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-folders.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-folders.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid.sql
index ec0a40a5934..5c1a6974590 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_at_version.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_at_version.sql
index e04b1ff7430..bf85c6ba3d5 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_at_version.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_at_version.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_second_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_second_page.sql
index ec0a40a5934..5c1a6974590 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_second_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-history_uid_second_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-migration_with_fallback.sql b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-migration_with_fallback.sql
index ba38c048136..8c3801ba771 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-migration_with_fallback.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/mysql--query_dashboards-migration_with_fallback.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard.sql
index 552a486f8cd..c04123f90ab 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard_next_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard_next_page.sql
index fb09d57d08d..5916a9b315f 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard_next_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-dashboard_next_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-export_with_history.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-export_with_history.sql
index 4d0affb337f..5eec54770c7 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-export_with_history.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-export_with_history.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-folders.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-folders.sql
index b994d617708..5c9cca1e30e 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-folders.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-folders.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid.sql
index 9bcbb168149..876fca02fe3 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_at_version.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_at_version.sql
index 61311c5c99b..0fa23f2db08 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_at_version.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_at_version.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_second_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_second_page.sql
index 9bcbb168149..876fca02fe3 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_second_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-history_uid_second_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-migration_with_fallback.sql b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-migration_with_fallback.sql
index 364fbebf417..fee2c28d525 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-migration_with_fallback.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/postgres--query_dashboards-migration_with_fallback.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard.sql
index 552a486f8cd..c04123f90ab 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard_next_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard_next_page.sql
index fb09d57d08d..5916a9b315f 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard_next_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-dashboard_next_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-export_with_history.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-export_with_history.sql
index 4d0affb337f..5eec54770c7 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-export_with_history.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-export_with_history.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-folders.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-folders.sql
index b994d617708..5c9cca1e30e 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-folders.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-folders.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid.sql
index 9bcbb168149..876fca02fe3 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_at_version.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_at_version.sql
index 61311c5c99b..0fa23f2db08 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_at_version.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_at_version.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_second_page.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_second_page.sql
index 9bcbb168149..876fca02fe3 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_second_page.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-history_uid_second_page.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-migration_with_fallback.sql b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-migration_with_fallback.sql
index 364fbebf417..fee2c28d525 100755
--- a/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-migration_with_fallback.sql
+++ b/pkg/registry/apis/dashboard/legacy/testdata/sqlite--query_dashboards-migration_with_fallback.sql
@@ -2,6 +2,7 @@ SELECT
dashboard.org_id,
dashboard.id,
dashboard.uid,
+ dashboard.title,
dashboard.folder_uid,
dashboard.deleted,
plugin_id,
diff --git a/pkg/registry/apis/dashboard/register.go b/pkg/registry/apis/dashboard/register.go
index 19108edadf8..75e2ed5b518 100644
--- a/pkg/registry/apis/dashboard/register.go
+++ b/pkg/registry/apis/dashboard/register.go
@@ -143,7 +143,7 @@ func RegisterAPIService(
folderClient: folderClient,
legacy: &DashboardStorage{
- Access: legacy.NewDashboardAccess(dbp, namespacer, dashStore, provisioning, libraryPanelSvc, sorter, accessControl),
+ Access: legacy.NewDashboardAccess(dbp, namespacer, dashStore, provisioning, libraryPanelSvc, sorter, accessControl, features),
DashboardService: dashboardService,
},
reg: reg,
diff --git a/pkg/registry/apis/provisioning/jobs/export/folders.go b/pkg/registry/apis/provisioning/jobs/export/folders.go
index afc7db05a87..c8b108d6023 100644
--- a/pkg/registry/apis/provisioning/jobs/export/folders.go
+++ b/pkg/registry/apis/provisioning/jobs/export/folders.go
@@ -32,8 +32,9 @@ func ExportFolders(ctx context.Context, repoName string, options provisioning.Ex
}
manager, _ := meta.GetManagerProperties()
- if manager.Identity == repoName {
- return nil // skip it... already in tree?
+ // Skip if already managed by any manager (repository, file provisioning, etc.)
+ if manager.Identity != "" {
+ return nil
}
return tree.AddUnstructured(item)
diff --git a/pkg/registry/apis/provisioning/jobs/export/folders_test.go b/pkg/registry/apis/provisioning/jobs/export/folders_test.go
index bec0214d7ef..49c07a64347 100644
--- a/pkg/registry/apis/provisioning/jobs/export/folders_test.go
+++ b/pkg/registry/apis/provisioning/jobs/export/folders_test.go
@@ -298,38 +298,21 @@ func TestExportFolders(t *testing.T) {
progress.On("SetMessage", mock.Anything, "read folder tree from API server").Return()
progress.On("SetMessage", mock.Anything, "write folders to repository").Return()
progress.On("Record", mock.Anything, mock.MatchedBy(func(result jobs.JobResourceResult) bool {
- return result.Name == "parent-uid" && result.Action == repository.FileActionCreated
+ return result.Name == "parent-folder" && result.Action == repository.FileActionCreated
})).Return()
progress.On("Record", mock.Anything, mock.MatchedBy(func(result jobs.JobResourceResult) bool {
- return result.Name == "child-uid" && result.Action == repository.FileActionCreated
+ return result.Name == "child-folder" && result.Action == repository.FileActionCreated
})).Return()
progress.On("TooManyErrors").Return(nil)
progress.On("TooManyErrors").Return(nil)
},
setupResources: func(repoResources *resources.MockRepositoryResources) {
repoResources.On("EnsureFolderTreeExists", mock.Anything, "feature/branch", "grafana", mock.MatchedBy(func(tree resources.FolderTree) bool {
- expectedFolders := []resources.Folder{
- {ID: "parent-folder", Path: "parent-folder"},
- {ID: "child-folder", Path: "parent-folder/child-folder"},
- }
-
- if tree.Count() != len(expectedFolders) {
- return false
- }
-
- for _, folder := range expectedFolders {
- dir, ok := tree.DirPath(folder.ID, "")
- if !ok || dir.Path != folder.Path {
- return false
- }
- }
-
- return true
+ return tree.Count() == 2
}), mock.MatchedBy(func(fn func(folder resources.Folder, created bool, err error) error) bool {
- // Parent folder should be processed first
- require.NoError(t, fn(resources.Folder{ID: "parent-uid", Path: "grafana/parent-folder"}, true, nil))
- // Then child folder with nested path
- require.NoError(t, fn(resources.Folder{ID: "child-uid", Path: "grafana/parent-folder/child-folder"}, true, nil))
+ require.NoError(t, fn(resources.Folder{ID: "parent-folder", Path: "grafana/parent-folder"}, true, nil))
+ require.NoError(t, fn(resources.Folder{ID: "child-folder", Path: "grafana/parent-folder/child-folder"}, true, nil))
+
return true
})).Return(nil)
},
@@ -380,7 +363,7 @@ func TestExportFolders(t *testing.T) {
}
func TestFolderMetaAccessor(t *testing.T) {
- t.Run("should export folders from another manager", func(t *testing.T) {
+ t.Run("should skip folders from another manager", func(t *testing.T) {
obj := &unstructured.Unstructured{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
@@ -405,21 +388,12 @@ func TestFolderMetaAccessor(t *testing.T) {
mockRepoResources := resources.NewMockRepositoryResources(t)
mockRepoResources.On("EnsureFolderTreeExists", mock.Anything, "feature/branch", "grafana", mock.MatchedBy(func(tree resources.FolderTree) bool {
- return tree.Count() == 1
- }), mock.MatchedBy(func(fn func(folder resources.Folder, created bool, err error) error) bool {
- require.NoError(t, fn(resources.Folder{ID: "test-folder-uid", Path: "grafana/test-folder"}, true, nil))
- return true
- })).Return(nil)
+ return tree.Count() == 0 // Should be 0 since folder is managed by other manager
+ }), mock.Anything).Return(nil)
progress := jobs.NewMockJobProgressRecorder(t)
- progress.On("SetMessage", mock.Anything, mock.Anything).Return()
- progress.On("Record", mock.Anything, mock.MatchedBy(func(result jobs.JobResourceResult) bool {
- return result.Action == repository.FileActionCreated &&
- result.Name == "test-folder-uid" &&
- result.Error == nil &&
- result.Path == "grafana/test-folder"
- })).Return()
- progress.On("TooManyErrors").Return(nil)
+ progress.On("SetMessage", mock.Anything, mock.Anything).Return().Twice()
+ // No Record calls expected since folder should be skipped
err = ExportFolders(context.Background(), "test-repo", v0alpha1.ExportJobOptions{
Path: "grafana",
Branch: "feature/branch",
@@ -430,7 +404,7 @@ func TestFolderMetaAccessor(t *testing.T) {
mockRepoResources.AssertExpectations(t)
progress.AssertExpectations(t)
})
- t.Run("should skip if repo is the manager", func(t *testing.T) {
+ t.Run("should skip if current repo is the manager", func(t *testing.T) {
obj := &unstructured.Unstructured{
Object: map[string]interface{}{
"metadata": map[string]interface{}{
@@ -492,6 +466,45 @@ func TestFolderMetaAccessor(t *testing.T) {
mockRepoResources.AssertExpectations(t)
progress.AssertExpectations(t)
})
+ t.Run("should skip if managed by any other manager", func(t *testing.T) {
+ obj := &unstructured.Unstructured{
+ Object: map[string]interface{}{
+ "metadata": map[string]interface{}{
+ "name": "test-folder",
+ "annotations": map[string]interface{}{
+ "folder.grafana.app/uid": "test-folder-uid",
+ },
+ },
+ },
+ }
+ meta, err := utils.MetaAccessor(obj)
+ require.NoError(t, err)
+ meta.SetManagerProperties(utils.ManagerProperties{
+ Kind: utils.ManagerKindTerraform,
+ Identity: "terraform-provisioning",
+ AllowsEdits: false,
+ Suspended: false,
+ })
+ fakeFolderClient := &mockDynamicInterface{
+ items: []unstructured.Unstructured{*obj},
+ }
+
+ mockRepoResources := resources.NewMockRepositoryResources(t)
+ progress := jobs.NewMockJobProgressRecorder(t)
+ progress.On("SetMessage", mock.Anything, mock.Anything).Return().Twice()
+ mockRepoResources.On("EnsureFolderTreeExists", mock.Anything, "feature/branch", "grafana", mock.MatchedBy(func(tree resources.FolderTree) bool {
+ return tree.Count() == 0 // Should be empty since folder was skipped
+ }), mock.Anything).Return(nil)
+
+ err = ExportFolders(context.Background(), "test-repo", v0alpha1.ExportJobOptions{
+ Path: "grafana",
+ Branch: "feature/branch",
+ }, fakeFolderClient, mockRepoResources, progress)
+
+ require.NoError(t, err)
+ mockRepoResources.AssertExpectations(t)
+ progress.AssertExpectations(t)
+ })
}
// mockDynamicInterface implements a simplified version of the dynamic.ResourceInterface
diff --git a/pkg/registry/apis/provisioning/jobs/export/resources.go b/pkg/registry/apis/provisioning/jobs/export/resources.go
index b986392c3d6..54805784cee 100644
--- a/pkg/registry/apis/provisioning/jobs/export/resources.go
+++ b/pkg/registry/apis/provisioning/jobs/export/resources.go
@@ -10,6 +10,7 @@ import (
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/client-go/dynamic"
+ "github.com/grafana/grafana/pkg/apimachinery/utils"
provisioning "github.com/grafana/grafana/pkg/apis/provisioning/v0alpha1"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/jobs"
"github.com/grafana/grafana/pkg/registry/apis/provisioning/repository"
@@ -103,6 +104,23 @@ func exportResource(ctx context.Context,
Action: repository.FileActionCreated,
}
+ // Check if resource is already managed by a repository
+ meta, err := utils.MetaAccessor(item)
+ if err != nil {
+ result.Action = repository.FileActionIgnored
+ result.Error = fmt.Errorf("extract meta accessor: %w", err)
+ progress.Record(ctx, result)
+ return nil
+ }
+
+ manager, _ := meta.GetManagerProperties()
+ // Skip if already managed by any manager (repository, file provisioning, etc.)
+ if manager.Identity != "" {
+ result.Action = repository.FileActionIgnored
+ progress.Record(ctx, result)
+ return nil
+ }
+
if shim != nil {
item, err = shim(ctx, item)
}
diff --git a/pkg/registry/apis/provisioning/jobs/export/resources_test.go b/pkg/registry/apis/provisioning/jobs/export/resources_test.go
index db8209cf414..e77ae6105ed 100644
--- a/pkg/registry/apis/provisioning/jobs/export/resources_test.go
+++ b/pkg/registry/apis/provisioning/jobs/export/resources_test.go
@@ -5,6 +5,7 @@ import (
"fmt"
"testing"
+ "github.com/grafana/grafana/pkg/apimachinery/utils"
mock "github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -532,3 +533,37 @@ func TestExportResources_Dashboards_V2beta1_ClientError(t *testing.T) {
err := runExportTest(t, mockItems, setupProgress, setupResources)
require.NoError(t, err)
}
+
+func TestExportResources_Dashboards_SkipsManagedResources(t *testing.T) {
+ // Create a dashboard managed by file provisioning
+ dashboard := createDashboardObject("managed-dashboard")
+
+ // Add manager metadata using utils package
+ meta, err := utils.MetaAccessor(&dashboard)
+ require.NoError(t, err)
+ meta.SetManagerProperties(utils.ManagerProperties{
+ Kind: utils.ManagerKindTerraform,
+ Identity: "terraform-provisioning",
+ AllowsEdits: false,
+ Suspended: false,
+ })
+
+ mockItems := []unstructured.Unstructured{dashboard}
+
+ setupProgress := func(progress *jobs.MockJobProgressRecorder) {
+ progress.On("SetMessage", mock.Anything, "start resource export").Return()
+ progress.On("SetMessage", mock.Anything, "export dashboards").Return()
+ progress.On("Record", mock.Anything, mock.MatchedBy(func(result jobs.JobResourceResult) bool {
+ return result.Name == "managed-dashboard" && result.Action == repository.FileActionIgnored
+ })).Return()
+ progress.On("TooManyErrors").Return(nil).Maybe()
+ }
+
+ setupResources := func(repoResources *resources.MockRepositoryResources, resourceClients *resources.MockResourceClients, mockClient *mockDynamicInterface, gvk schema.GroupVersionKind) {
+ resourceClients.On("ForResource", resources.DashboardResource).Return(mockClient, gvk, nil)
+ // No WriteResourceFileFromObject call expected since resource should be skipped
+ }
+
+ err = runExportTest(t, mockItems, setupProgress, setupResources)
+ require.NoError(t, err)
+}
diff --git a/pkg/registry/apis/provisioning/repository/git/repository.go b/pkg/registry/apis/provisioning/repository/git/repository.go
index 5bcdb12cd94..7ca9ed8998e 100644
--- a/pkg/registry/apis/provisioning/repository/git/repository.go
+++ b/pkg/registry/apis/provisioning/repository/git/repository.go
@@ -790,6 +790,10 @@ func (r *gitRepository) createSignature(ctx context.Context) (nanogit.Author, na
func (r *gitRepository) commit(ctx context.Context, writer nanogit.StagedWriter, comment string) error {
author, committer := r.createSignature(ctx)
if _, err := writer.Commit(ctx, comment, author, committer); err != nil {
+ if errors.Is(err, nanogit.ErrNothingToCommit) {
+ return repository.ErrNothingToCommit
+ }
+
return fmt.Errorf("commit changes: %w", err)
}
return nil
diff --git a/pkg/registry/apis/provisioning/repository/git/staged.go b/pkg/registry/apis/provisioning/repository/git/staged.go
index a3f79567029..f55eb709341 100644
--- a/pkg/registry/apis/provisioning/repository/git/staged.go
+++ b/pkg/registry/apis/provisioning/repository/git/staged.go
@@ -219,12 +219,24 @@ func (r *stagedGitRepository) Push(ctx context.Context) error {
if message == "" {
message = "Staged changes"
}
+
if err := r.commit(ctx, r.writer, message); err != nil {
return err
}
}
- return r.writer.Push(ctx)
+ err := r.writer.Push(ctx)
+ if err != nil {
+ // Convert nanogit-specific errors to repository-level errors to avoid leaky abstraction
+ if errors.Is(err, nanogit.ErrNothingToPush) {
+ return repository.ErrNothingToPush
+ }
+ if errors.Is(err, nanogit.ErrNothingToCommit) {
+ return repository.ErrNothingToCommit
+ }
+ return err
+ }
+ return nil
}
func (r *stagedGitRepository) Remove(ctx context.Context) error {
diff --git a/pkg/registry/apis/provisioning/repository/git/staged_test.go b/pkg/registry/apis/provisioning/repository/git/staged_test.go
index 4501a087e7a..77c1d8b5edb 100644
--- a/pkg/registry/apis/provisioning/repository/git/staged_test.go
+++ b/pkg/registry/apis/provisioning/repository/git/staged_test.go
@@ -3,6 +3,7 @@ package git
import (
"context"
"errors"
+ "fmt"
"strings"
"testing"
"time"
@@ -995,6 +996,54 @@ func TestStagedGitRepository_Push(t *testing.T) {
expectPushCalls: 1,
expectCommitCalls: 1,
},
+ {
+ name: "returns repository ErrNothingToPush when nanogit returns ErrNothingToPush",
+ opts: repository.StageOptions{},
+ setupMock: func(mockWriter *mocks.FakeStagedWriter) {
+ mockWriter.PushReturns(nanogit.ErrNothingToPush)
+ },
+ wantError: repository.ErrNothingToPush,
+ expectPushCalls: 1,
+ expectCommitCalls: 0,
+ },
+ {
+ name: "returns repository ErrNothingToCommit when nanogit returns ErrNothingToCommit",
+ opts: repository.StageOptions{
+ Mode: repository.StageModeCommitOnlyOnce,
+ },
+ setupMock: func(mockWriter *mocks.FakeStagedWriter) {
+ mockWriter.CommitReturns(nil, nanogit.ErrNothingToCommit)
+ },
+ wantError: repository.ErrNothingToCommit,
+ expectPushCalls: 0,
+ expectCommitCalls: 1,
+ },
+ {
+ name: "returns repository ErrNothingToPush when nanogit returns wrapped ErrNothingToPush",
+ opts: repository.StageOptions{},
+ setupMock: func(mockWriter *mocks.FakeStagedWriter) {
+ // Use fmt.Errorf with %w to create a wrapped error that errors.Is can detect
+ wrappedErr := fmt.Errorf("git operation failed: %w", nanogit.ErrNothingToPush)
+ mockWriter.PushReturns(wrappedErr)
+ },
+ wantError: repository.ErrNothingToPush,
+ expectPushCalls: 1,
+ expectCommitCalls: 0,
+ },
+ {
+ name: "returns repository ErrNothingToCommit when nanogit returns wrapped ErrNothingToCommit",
+ opts: repository.StageOptions{
+ Mode: repository.StageModeCommitOnlyOnce,
+ },
+ setupMock: func(mockWriter *mocks.FakeStagedWriter) {
+ // Use fmt.Errorf with %w to create a wrapped error that errors.Is can detect
+ wrappedErr := fmt.Errorf("git operation failed: %w", nanogit.ErrNothingToCommit)
+ mockWriter.CommitReturns(nil, wrappedErr)
+ },
+ wantError: repository.ErrNothingToCommit,
+ expectPushCalls: 0,
+ expectCommitCalls: 1,
+ },
}
for _, tt := range tests {
@@ -1007,7 +1056,12 @@ func TestStagedGitRepository_Push(t *testing.T) {
err := stagedRepo.Push(context.Background())
if tt.wantError != nil {
- require.EqualError(t, err, tt.wantError.Error())
+ // For nanogit error conversion tests, use ErrorIs to verify type conversion
+ if errors.Is(tt.wantError, repository.ErrNothingToPush) || errors.Is(tt.wantError, repository.ErrNothingToCommit) {
+ require.ErrorIs(t, err, tt.wantError)
+ } else {
+ require.EqualError(t, err, tt.wantError.Error())
+ }
} else {
require.NoError(t, err)
}
diff --git a/pkg/registry/apis/provisioning/repository/staged.go b/pkg/registry/apis/provisioning/repository/staged.go
index ca5b59c72c9..1861d66336c 100644
--- a/pkg/registry/apis/provisioning/repository/staged.go
+++ b/pkg/registry/apis/provisioning/repository/staged.go
@@ -7,9 +7,14 @@ import (
"time"
"github.com/grafana/grafana-app-sdk/logging"
- "github.com/grafana/nanogit"
)
+// ErrNothingToPush indicates that there are no changes to push to the remote repository
+var ErrNothingToPush = errors.New("nothing to push")
+
+// ErrNothingToCommit indicates that there are no changes to commit
+var ErrNothingToCommit = errors.New("nothing to commit")
+
//go:generate mockery --name WrapWithStageFn --structname MockWrapWithStageFn --inpackage --filename mock_wrap_with_stage_fn.go --with-expecter
type WrapWithStageFn func(ctx context.Context, repo Repository, stageOptions StageOptions, fn func(repo Repository, staged bool) error) error
@@ -83,7 +88,7 @@ func WrapWithStageAndPushIfPossible(
}
if err = staged.Push(ctx); err != nil {
- if errors.Is(err, nanogit.ErrNothingToPush) {
+ if errors.Is(err, ErrNothingToPush) || errors.Is(err, ErrNothingToCommit) {
return nil // OK, already pushed
}
return fmt.Errorf("wrapped push error: %w", err)
diff --git a/pkg/registry/apis/provisioning/repository/staged_test.go b/pkg/registry/apis/provisioning/repository/staged_test.go
index 3efe8ebbd5e..c640e2cd0db 100644
--- a/pkg/registry/apis/provisioning/repository/staged_test.go
+++ b/pkg/registry/apis/provisioning/repository/staged_test.go
@@ -3,6 +3,7 @@ package repository
import (
"context"
"errors"
+ "fmt"
"testing"
"github.com/stretchr/testify/mock"
@@ -127,6 +128,84 @@ func TestWrapWithStageAndPushIfPossible(t *testing.T) {
return nil
},
},
+ {
+ name: "nothing to push - should not error",
+ setupMocks: func(t *testing.T) *mockStagedRepo {
+ mockRepo := NewMockStageableRepository(t)
+ mockStaged := NewMockStagedRepository(t)
+
+ mockRepo.EXPECT().Stage(mock.Anything, StageOptions{}).Return(mockStaged, nil)
+ mockStaged.EXPECT().Push(mock.Anything).Return(ErrNothingToPush)
+ mockStaged.EXPECT().Remove(mock.Anything).Return(nil)
+
+ return &mockStagedRepo{
+ MockStageableRepository: mockRepo,
+ MockStagedRepository: mockStaged,
+ }
+ },
+ operation: func(repo Repository, staged bool) error {
+ return nil
+ },
+ },
+ {
+ name: "nothing to commit - should not error",
+ setupMocks: func(t *testing.T) *mockStagedRepo {
+ mockRepo := NewMockStageableRepository(t)
+ mockStaged := NewMockStagedRepository(t)
+
+ mockRepo.EXPECT().Stage(mock.Anything, StageOptions{}).Return(mockStaged, nil)
+ mockStaged.EXPECT().Push(mock.Anything).Return(ErrNothingToCommit)
+ mockStaged.EXPECT().Remove(mock.Anything).Return(nil)
+
+ return &mockStagedRepo{
+ MockStageableRepository: mockRepo,
+ MockStagedRepository: mockStaged,
+ }
+ },
+ operation: func(repo Repository, staged bool) error {
+ return nil
+ },
+ },
+ {
+ name: "wrapped nothing to push error - should not error",
+ setupMocks: func(t *testing.T) *mockStagedRepo {
+ mockRepo := NewMockStageableRepository(t)
+ mockStaged := NewMockStagedRepository(t)
+
+ wrappedErr := fmt.Errorf("some wrapper: %w", ErrNothingToPush)
+ mockRepo.EXPECT().Stage(mock.Anything, StageOptions{}).Return(mockStaged, nil)
+ mockStaged.EXPECT().Push(mock.Anything).Return(wrappedErr)
+ mockStaged.EXPECT().Remove(mock.Anything).Return(nil)
+
+ return &mockStagedRepo{
+ MockStageableRepository: mockRepo,
+ MockStagedRepository: mockStaged,
+ }
+ },
+ operation: func(repo Repository, staged bool) error {
+ return nil
+ },
+ },
+ {
+ name: "wrapped nothing to commit error - should not error",
+ setupMocks: func(t *testing.T) *mockStagedRepo {
+ mockRepo := NewMockStageableRepository(t)
+ mockStaged := NewMockStagedRepository(t)
+
+ wrappedErr := fmt.Errorf("some wrapper: %w", ErrNothingToCommit)
+ mockRepo.EXPECT().Stage(mock.Anything, StageOptions{}).Return(mockStaged, nil)
+ mockStaged.EXPECT().Push(mock.Anything).Return(wrappedErr)
+ mockStaged.EXPECT().Remove(mock.Anything).Return(nil)
+
+ return &mockStagedRepo{
+ MockStageableRepository: mockRepo,
+ MockStagedRepository: mockStaged,
+ }
+ },
+ operation: func(repo Repository, staged bool) error {
+ return nil
+ },
+ },
}
for _, tt := range tests {
diff --git a/pkg/registry/apis/provisioning/resources/resources.go b/pkg/registry/apis/provisioning/resources/resources.go
index 28dd06ea43a..ec5942c21ec 100644
--- a/pkg/registry/apis/provisioning/resources/resources.go
+++ b/pkg/registry/apis/provisioning/resources/resources.go
@@ -92,19 +92,29 @@ func (r *ResourcesManager) WriteResourceFileFromObject(ctx context.Context, obj
if title == "" {
title = name
}
- folder := meta.GetFolder()
+ folder := meta.GetFolder()
// Get the absolute path of the folder
rootFolder := RootFolder(r.repo.Config())
- fid, ok := r.folders.Tree().DirPath(folder, rootFolder)
- if !ok {
- return "", fmt.Errorf("folder not found in tree: %s", folder)
+
+ // If no folder is specified in the file, set it to the root to ensure everything is written under it
+ var fid Folder
+ if folder == "" {
+ fid = Folder{ID: rootFolder}
+ meta.SetFolder(rootFolder) // Set the folder in the metadata to the root folder
+ } else {
+ var ok bool
+ fid, ok = r.folders.Tree().DirPath(folder, rootFolder)
+ if !ok {
+ return "", fmt.Errorf("folder %s NOT found in tree with root: %s", folder, rootFolder)
+ }
}
fileName := slugify.Slugify(title) + ".json"
if fid.Path != "" {
fileName = safepath.Join(fid.Path, fileName)
}
+
if options.Path != "" {
fileName = safepath.Join(options.Path, fileName)
}
diff --git a/pkg/registry/apis/query/query.go b/pkg/registry/apis/query/query.go
index 66a1db38eae..a70aa657765 100644
--- a/pkg/registry/apis/query/query.go
+++ b/pkg/registry/apis/query/query.go
@@ -3,9 +3,11 @@ package query
import (
"context"
"encoding/json"
+ "errors"
"net/http"
"slices"
"strconv"
+ "strings"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana-plugin-sdk-go/experimental/apis/data/v0alpha1"
@@ -108,6 +110,7 @@ func (r *queryREST) NewConnectOptions() (runtime.Object, bool, string) {
return nil, false, "" // true means you can use the trailing path as a variable
}
+// called by mt query service and also when queryServiceFromUI is enabled, can be both mt and st
func (r *queryREST) Connect(connectCtx context.Context, name string, _ runtime.Object, incomingResponder rest.Responder) (http.Handler, error) {
// See: /pkg/services/apiserver/builder/helper.go#L34
// The name is set with a rewriter hack
@@ -175,7 +178,7 @@ func (r *queryREST) Connect(connectCtx context.Context, name string, _ runtime.O
return
}
- qdr, err := handleQuery(ctx, *raw, *b, httpreq, *responder)
+ qdr, err := handleQuery(ctx, *raw, *b, httpreq, *responder, connectLogger)
if err != nil {
b.log.Error("execute error", "http code", query.GetResponseCode(qdr), "err", err)
@@ -186,9 +189,25 @@ func (r *queryREST) Connect(connectCtx context.Context, name string, _ runtime.O
})
return
} else {
- // return the error to the client, will send all non k8s errors as a k8 unexpected error
- b.log.Error("hit unexpected error while executing query, this will show as an unhandled k8s status error", "err", err)
- responder.Error(err)
+ var errorDataResponse backend.DataResponse
+ if errors.Is(err, service.ErrInvalidDatasourceID) || errors.Is(err, service.ErrNoQueriesFound) || errors.Is(err, service.ErrMissingDataSourceInfo) || errors.Is(err, service.ErrQueryParamMismatch) || errors.Is(err, service.ErrDuplicateRefId) {
+ errorDataResponse = backend.ErrDataResponseWithSource(backend.StatusBadRequest, backend.ErrorSourceDownstream, err.Error())
+ } else if strings.Contains(err.Error(), "expression request error") {
+ b.log.Error("Error calling TransformData in an expression", "err", err)
+ errorDataResponse = backend.ErrDataResponseWithSource(backend.StatusBadRequest, backend.ErrorSourceDownstream, err.Error())
+ } else {
+ b.log.Error("unknown error, treated as a 500", "err", err)
+ responder.Error(err)
+ return
+ }
+ qdr = &backend.QueryDataResponse{
+ Responses: map[string]backend.DataResponse{
+ "A": errorDataResponse,
+ },
+ }
+ responder.Object(query.GetResponseCode(qdr), &query.QueryDataResponse{
+ QueryDataResponse: *qdr,
+ })
return
}
}
@@ -199,7 +218,7 @@ func (r *queryREST) Connect(connectCtx context.Context, name string, _ runtime.O
}), nil
}
-func handleQuery(ctx context.Context, raw query.QueryDataRequest, b QueryAPIBuilder, httpreq *http.Request, responder responderWrapper) (*backend.QueryDataResponse, error) {
+func handleQuery(ctx context.Context, raw query.QueryDataRequest, b QueryAPIBuilder, httpreq *http.Request, responder responderWrapper, connectLogger log.Logger) (*backend.QueryDataResponse, error) {
var jsonQueries = make([]*simplejson.Json, 0, len(raw.Queries))
for _, query := range raw.Queries {
jsonBytes, err := json.Marshal(query)
@@ -214,6 +233,7 @@ func handleQuery(ctx context.Context, raw query.QueryDataRequest, b QueryAPIBuil
jsonQueries = append(jsonQueries, sjQuery)
}
+
mReq := dtos.MetricRequest{
From: raw.From,
To: raw.To,
@@ -228,17 +248,19 @@ func handleQuery(ctx context.Context, raw query.QueryDataRequest, b QueryAPIBuil
instanceConfig, err := b.clientSupplier.GetInstanceConfigurationSettings(ctx)
if err != nil {
- b.log.Error("failed to get instance configuration settings", "err", err)
+ connectLogger.Error("failed to get instance configuration settings", "err", err)
responder.Error(err)
return nil, err
}
+ dsQuerierLoggerWithSlug := connectLogger.New("slug", instanceConfig.Options["slug"], "ruleuid", headers["X-Rule-Uid"])
+
mtDsClientBuilder := mtdsclient.NewMtDatasourceClientBuilderWithClientSupplier(
b.clientSupplier,
ctx,
headers,
instanceConfig,
- b.log,
+ dsQuerierLoggerWithSlug,
)
exprService := expr.ProvideService(
@@ -256,7 +278,7 @@ func handleQuery(ctx context.Context, raw query.QueryDataRequest, b QueryAPIBuil
mtDsClientBuilder,
)
- qdr, err := service.QueryData(ctx, b.log, cache, exprService, mReq, mtDsClientBuilder, headers)
+ qdr, err := service.QueryData(ctx, dsQuerierLoggerWithSlug, cache, exprService, mReq, mtDsClientBuilder, headers)
if err != nil {
return qdr, err
diff --git a/pkg/registry/apis/secret/contracts/decrypt.go b/pkg/registry/apis/secret/contracts/decrypt.go
index 32f9f7f6c1d..2c5403c790d 100644
--- a/pkg/registry/apis/secret/contracts/decrypt.go
+++ b/pkg/registry/apis/secret/contracts/decrypt.go
@@ -27,7 +27,7 @@ type DecryptAuthorizer interface {
Authorize(ctx context.Context, secureValueName string, secureValueDecrypters []string) (identity string, allowed bool)
}
-// DecryptService is the inferface for the decrypt service.
+// DecryptService is the interface for the decrypt service.
type DecryptService interface {
Decrypt(ctx context.Context, namespace string, names ...string) (map[string]DecryptResult, error)
Close() error
diff --git a/pkg/registry/apis/secret/contracts/encryption.go b/pkg/registry/apis/secret/contracts/encryption.go
index f0596de9034..f24fe73a544 100644
--- a/pkg/registry/apis/secret/contracts/encryption.go
+++ b/pkg/registry/apis/secret/contracts/encryption.go
@@ -38,3 +38,7 @@ type GlobalEncryptedValueStorage interface {
ListAll(ctx context.Context, opts ListOpts, untilTime *int64) ([]*EncryptedValue, error)
CountAll(ctx context.Context, untilTime *int64) (int64, error)
}
+
+type ConsolidationService interface {
+ Consolidate(ctx context.Context) error
+}
diff --git a/pkg/registry/apis/secret/contracts/inline.go b/pkg/registry/apis/secret/contracts/inline.go
new file mode 100644
index 00000000000..f2f3de71568
--- /dev/null
+++ b/pkg/registry/apis/secret/contracts/inline.go
@@ -0,0 +1,19 @@
+package contracts
+
+import (
+ "context"
+
+ common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
+)
+
+type InlineSecureValueSupport interface {
+ // Check that the request user can reference secure value names in the context of a given resource (owner)
+ CanReference(ctx context.Context, owner common.ObjectReference, names ...string) error
+
+ // CreateInline creates a secret that is owned by the referenced object
+ // returns the name of the created secret or an error
+ CreateInline(ctx context.Context, owner common.ObjectReference, value common.RawSecureValue) (string, error)
+
+ // DeleteWhenOwnedByResource removes secrets if and only if they are owned by a referenced object
+ DeleteWhenOwnedByResource(ctx context.Context, owner common.ObjectReference, name string) error
+}
diff --git a/pkg/registry/apis/secret/service/consolidation.go b/pkg/registry/apis/secret/service/consolidation.go
new file mode 100644
index 00000000000..b0baea659c4
--- /dev/null
+++ b/pkg/registry/apis/secret/service/consolidation.go
@@ -0,0 +1,87 @@
+package service
+
+import (
+ "context"
+ "fmt"
+
+ "github.com/grafana/grafana-app-sdk/logging"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
+ otelcodes "go.opentelemetry.io/otel/codes"
+ "go.opentelemetry.io/otel/trace"
+)
+
+type ConsolidationService struct {
+ tracer trace.Tracer
+ globalDataKeyStore contracts.GlobalDataKeyStorage
+ encryptedValueStore contracts.EncryptedValueStorage
+ globalEncryptedValueStore contracts.GlobalEncryptedValueStorage
+ encryptionManager contracts.EncryptionManager
+}
+
+func ProvideConsolidationService(
+ tracer trace.Tracer,
+ globalDataKeyStore contracts.GlobalDataKeyStorage,
+ encryptedValueStore contracts.EncryptedValueStorage,
+ globalEncryptedValueStore contracts.GlobalEncryptedValueStorage,
+ encryptionManager contracts.EncryptionManager,
+) contracts.ConsolidationService {
+ return &ConsolidationService{
+ tracer: tracer,
+ globalDataKeyStore: globalDataKeyStore,
+ encryptedValueStore: encryptedValueStore,
+ globalEncryptedValueStore: globalEncryptedValueStore,
+ encryptionManager: encryptionManager,
+ }
+}
+
+func (s *ConsolidationService) Consolidate(ctx context.Context) (err error) {
+ ctx, span := s.tracer.Start(ctx, "ConsolidationService.Consolidate")
+ defer span.End()
+
+ defer func() {
+ if err != nil {
+ span.SetStatus(otelcodes.Error, err.Error())
+ span.RecordError(err)
+ }
+ }()
+
+ // Disable all active data keys.
+ // This will ensure that no new data can be encrypted with the old keys.
+ err = s.globalDataKeyStore.DisableAllDataKeys(ctx)
+ if err != nil {
+ return fmt.Errorf("disabling all data keys: %w", err)
+ }
+
+ // List all encrypted values.
+ encryptedValues, err := s.globalEncryptedValueStore.ListAll(ctx, contracts.ListOpts{}, nil)
+ if err != nil {
+ return fmt.Errorf("listing all encrypted values: %w", err)
+ }
+
+ for _, ev := range encryptedValues {
+ // Decrypt the value using its old data key.
+ decryptedValue, err := s.encryptionManager.Decrypt(ctx, ev.Namespace, ev.EncryptedData)
+ if err != nil {
+ logging.FromContext(ctx).Error("Failed to decrypt value", "namespace", ev.Namespace, "name", ev.Name, "error", err)
+ continue
+ }
+
+ // Re-encrypt the value using a new data key.
+ reEncryptedValue, err := s.encryptionManager.Encrypt(ctx, ev.Namespace, decryptedValue)
+ if err != nil {
+ logging.FromContext(ctx).Error("Failed to re-encrypt value", "namespace", ev.Namespace, "name", ev.Name, "error", err)
+ continue
+ }
+
+ // Update the encrypted value in the store.
+ err = s.encryptedValueStore.Update(ctx, ev.Namespace, ev.Name, ev.Version, reEncryptedValue)
+ if err != nil {
+ logging.FromContext(ctx).Error("Failed to update encrypted value", "namespace", ev.Namespace, "name", ev.Name, "error", err)
+ continue
+ }
+ }
+
+ // TODO: After all values are re-encrypted, we can safely remove the old data keys.
+
+ return nil
+}
diff --git a/pkg/registry/apis/secret/service/consolidation_test.go b/pkg/registry/apis/secret/service/consolidation_test.go
new file mode 100644
index 00000000000..57ac94253c9
--- /dev/null
+++ b/pkg/registry/apis/secret/service/consolidation_test.go
@@ -0,0 +1,281 @@
+package service_test
+
+import (
+ "context"
+ "testing"
+
+ "github.com/grafana/authlib/authn"
+ "github.com/grafana/authlib/types"
+ secretv1beta1 "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
+ "github.com/grafana/grafana/pkg/apimachinery/identity"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/service"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/testutils"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
+ "github.com/stretchr/testify/require"
+ "go.opentelemetry.io/otel/trace/noop"
+ metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+ "k8s.io/utils/ptr"
+)
+
+// mockGlobalEncryptedValueStorage wraps the real storage and allows injecting behavior during ListAll
+type mockGlobalEncryptedValueStorage struct {
+ real contracts.GlobalEncryptedValueStorage
+ sut *testutils.Sut
+ ctx context.Context
+ onListAll func()
+}
+
+func (m *mockGlobalEncryptedValueStorage) ListAll(ctx context.Context, opts contracts.ListOpts, untilTime *int64) ([]*contracts.EncryptedValue, error) {
+ if m.onListAll != nil {
+ m.onListAll()
+ }
+ return m.real.ListAll(ctx, opts, untilTime)
+}
+
+func (m *mockGlobalEncryptedValueStorage) CountAll(ctx context.Context, untilTime *int64) (int64, error) {
+ return m.real.CountAll(ctx, untilTime)
+}
+
+func TestConsolidation(t *testing.T) {
+ t.Parallel()
+
+ t.Run("consolidation re-encrypts values but preserves decrypted content", func(t *testing.T) {
+ t.Parallel()
+ sut := testutils.Setup(t)
+
+ ctx := context.Background()
+ createAuthContext := func(ctx context.Context, namespace string, identityType types.IdentityType) context.Context {
+ return types.WithAuthInfo(ctx, &identity.StaticRequester{
+ Type: identityType,
+ Namespace: namespace,
+ AccessTokenClaims: &authn.Claims[authn.AccessTokenClaims]{
+ Rest: authn.AccessTokenClaims{
+ Permissions: []string{"secret.grafana.app/securevalues:decrypt"},
+ ServiceIdentity: "decrypter1",
+ },
+ },
+ })
+ }
+
+ // Create several secure values in different namespaces
+ testCases := []struct {
+ name string
+ namespace string
+ value string
+ }{
+ {"test-secret-1", "namespace1", "test-value-1"},
+ {"test-secret-2", "namespace1", "test-value-2"},
+ {"test-secret-3", "namespace2", "test-value-3"},
+ {"test-secret-4", "namespace2", "test-value-4"},
+ }
+
+ var originalDecryptedValues []string
+ var originalEncryptedData [][]byte
+
+ // Create secure values and store their original decrypted values and encrypted data
+ for _, tc := range testCases {
+ sv := &secretv1beta1.SecureValue{
+ ObjectMeta: metav1.ObjectMeta{
+ Name: tc.name,
+ Namespace: tc.namespace,
+ },
+ Spec: secretv1beta1.SecureValueSpec{
+ Value: ptr.To(secretv1beta1.NewExposedSecureValue(tc.value)),
+ Decrypters: []string{"decrypter1"},
+ },
+ }
+
+ createdSv, err := sut.CreateSv(ctx, testutils.CreateSvWithSv(sv))
+ require.NoError(t, err)
+ require.NotNil(t, createdSv)
+
+ // Store the original decrypted data and encrypted data
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ originalDecryptedValues = append(originalDecryptedValues, decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ require.NotNil(t, encryptedValue)
+ originalEncryptedData = append(originalEncryptedData, encryptedValue.EncryptedData)
+ }
+
+ // Run consolidation
+ err := sut.ConsolidationService.Consolidate(ctx)
+ require.NoError(t, err)
+
+ for i, tc := range testCases {
+ // Verify that the decrypted values are still the same
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ require.Equal(t, originalDecryptedValues[i], decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ // Verify that the encrypted data has changed (indicating re-encryption)
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ require.NotEqual(t, originalEncryptedData[i], encryptedValue.EncryptedData)
+ }
+ })
+
+ t.Run("consolidation handles secrets created during the process", func(t *testing.T) {
+ t.Parallel()
+ sut := testutils.Setup(t)
+
+ ctx := context.Background()
+ createAuthContext := func(ctx context.Context, namespace string, identityType types.IdentityType) context.Context {
+ return types.WithAuthInfo(ctx, &identity.StaticRequester{
+ Type: identityType,
+ Namespace: namespace,
+ AccessTokenClaims: &authn.Claims[authn.AccessTokenClaims]{
+ Rest: authn.AccessTokenClaims{
+ Permissions: []string{"secret.grafana.app/securevalues:decrypt"},
+ ServiceIdentity: "decrypter1",
+ },
+ },
+ })
+ }
+
+ // Create initial secure values
+ initialSecrets := []struct {
+ name string
+ namespace string
+ value string
+ }{
+ {"initial-secret-1", "namespace1", "initial-value-1"},
+ {"initial-secret-2", "namespace2", "initial-value-2"},
+ }
+
+ var initialDecryptedValues []string
+ var initialEncryptedData [][]byte
+
+ for _, tc := range initialSecrets {
+ sv := &secretv1beta1.SecureValue{
+ ObjectMeta: metav1.ObjectMeta{
+ Name: tc.name,
+ Namespace: tc.namespace,
+ },
+ Spec: secretv1beta1.SecureValueSpec{
+ Value: ptr.To(secretv1beta1.NewExposedSecureValue(tc.value)),
+ Decrypters: []string{"decrypter1"},
+ },
+ }
+
+ _, err := sut.CreateSv(ctx, testutils.CreateSvWithSv(sv))
+ require.NoError(t, err)
+
+ // Store original decrypted values and encrypted data
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ initialDecryptedValues = append(initialDecryptedValues, decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ initialEncryptedData = append(initialEncryptedData, encryptedValue.EncryptedData)
+ }
+
+ // Secrets to be created during consolidation (after data keys are disabled)
+ var newSecretDecryptedValues []string
+ var newSecretEncryptedData [][]byte
+
+ // Create a mock GlobalEncryptedValueStorage that will create new secrets when ListAll is called
+ mockStorage := &mockGlobalEncryptedValueStorage{
+ real: sut.GlobalEncryptedValueStorage,
+ sut: &sut,
+ ctx: ctx,
+ onListAll: func() {
+ // This function is called during consolidation, after data keys are disabled
+ // but before the re-encryption loop begins
+ newSecrets := []struct {
+ name string
+ namespace string
+ value string
+ desc string
+ }{
+ {"new-secret-1", "namespace1", "new-value-1", "New secret created during consolidation"},
+ {"new-secret-2", "namespace3", "new-value-2", "Another new secret during consolidation"},
+ }
+
+ for _, tc := range newSecrets {
+ sv := &secretv1beta1.SecureValue{
+ ObjectMeta: metav1.ObjectMeta{
+ Name: tc.name,
+ Namespace: tc.namespace,
+ },
+ Spec: secretv1beta1.SecureValueSpec{
+ Description: tc.desc,
+ Value: ptr.To(secretv1beta1.NewExposedSecureValue(tc.value)),
+ Decrypters: []string{"decrypter1"},
+ },
+ }
+
+ _, err := sut.CreateSv(ctx, testutils.CreateSvWithSv(sv))
+ require.NoError(t, err)
+
+ // Store their decrypted values and original encrypted data
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ newSecretDecryptedValues = append(newSecretDecryptedValues, decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ newSecretEncryptedData = append(newSecretEncryptedData, encryptedValue.EncryptedData)
+ }
+ },
+ }
+
+ // Create a custom consolidation service that uses the mocked storage
+ tracer := noop.NewTracerProvider().Tracer("test")
+ customConsolidationService := service.ProvideConsolidationService(
+ tracer,
+ sut.GlobalDataKeyStore,
+ sut.EncryptedValueStorage,
+ mockStorage,
+ sut.EncryptionManager,
+ )
+
+ // Run consolidation
+ err := customConsolidationService.Consolidate(ctx)
+ require.NoError(t, err)
+
+ for i, tc := range initialSecrets {
+ // Verify that all initial secrets still decrypt to the same values
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ require.Equal(t, initialDecryptedValues[i], decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ // Verify that the encrypted data has changed (indicating re-encryption)
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ require.NotEqual(t, initialEncryptedData[i], encryptedValue.EncryptedData)
+ }
+
+ // Verify that the new secrets (created during consolidation) also decrypt correctly
+ // These secrets should have been re-encrypted as well during the consolidation process
+ newSecrets := []struct {
+ name string
+ namespace string
+ }{
+ {"new-secret-1", "namespace1"},
+ {"new-secret-2", "namespace3"},
+ }
+
+ for i, tc := range newSecrets {
+ authCtx := createAuthContext(ctx, tc.namespace, types.TypeAccessPolicy)
+ decryptedValue, err := sut.DecryptStorage.Decrypt(authCtx, xkube.Namespace(tc.namespace), tc.name)
+ require.NoError(t, err)
+ require.Equal(t, newSecretDecryptedValues[i], decryptedValue.DangerouslyExposeAndConsumeValue())
+
+ // Verify that the encrypted data has changed from what it was when first created
+ // (indicating it was re-encrypted during consolidation)
+ encryptedValue, err := sut.EncryptedValueStorage.Get(ctx, tc.namespace, tc.name, 1)
+ require.NoError(t, err)
+ require.NotEqual(t, newSecretEncryptedData[i], encryptedValue.EncryptedData)
+ }
+ })
+}
diff --git a/pkg/registry/apis/secret/service/inline_secure_value.go b/pkg/registry/apis/secret/service/inline_secure_value.go
new file mode 100644
index 00000000000..665bc622f54
--- /dev/null
+++ b/pkg/registry/apis/secret/service/inline_secure_value.go
@@ -0,0 +1,256 @@
+package service
+
+import (
+ "context"
+ "errors"
+ "fmt"
+
+ "github.com/grafana/authlib/authn"
+ authlib "github.com/grafana/authlib/types"
+ "go.opentelemetry.io/otel/attribute"
+ "go.opentelemetry.io/otel/trace"
+ metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+ "k8s.io/apimachinery/pkg/runtime/schema"
+
+ secretv1beta1 "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
+ common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
+ "github.com/grafana/grafana/pkg/apimachinery/utils"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
+ "github.com/grafana/grafana/pkg/util"
+)
+
+type inlineSecureValueService struct {
+ tracer trace.Tracer
+ secureValueService contracts.SecureValueService
+ accessChecker authlib.AccessChecker
+}
+
+func ProvideInlineSecureValueService(
+ tracer trace.Tracer,
+ secureValueService contracts.SecureValueService,
+ accessClient authlib.AccessClient,
+) contracts.InlineSecureValueSupport {
+ return &inlineSecureValueService{
+ tracer: tracer,
+ secureValueService: secureValueService,
+ accessChecker: accessClient,
+ }
+}
+
+func (s *inlineSecureValueService) CanReference(ctx context.Context, owner common.ObjectReference, names ...string) error {
+ ctx, span := s.tracer.Start(ctx, "InlineSecureValueService.CanReference", trace.WithAttributes(
+ attribute.String("owner.namespace", owner.Namespace),
+ attribute.String("owner.apiGroup", owner.APIGroup),
+ attribute.String("owner.apiVersion", owner.APIVersion),
+ attribute.String("owner.kind", owner.Kind),
+ attribute.String("owner.name", owner.Name),
+ attribute.StringSlice("secureValueNames", names),
+ ))
+ defer span.End()
+
+ authInfo, ok := authlib.AuthInfoFrom(ctx)
+ if !ok {
+ return fmt.Errorf("missing auth info in context")
+ }
+
+ if owner.Namespace == "" || !authlib.NamespaceMatches(authInfo.GetNamespace(), owner.Namespace) {
+ return fmt.Errorf("owner namespace %s does not match auth info namespace %s", owner.Namespace, authInfo.GetNamespace())
+ }
+
+ if owner.APIGroup == "" || owner.APIVersion == "" || owner.Kind == "" || owner.Name == "" {
+ return fmt.Errorf("owner reference must have a valid API group, API version, kind and name")
+ }
+
+ if len(names) == 0 {
+ return fmt.Errorf("no inline secure values provided")
+ }
+
+ for _, name := range names {
+ if name == "" {
+ return fmt.Errorf("empty secure value name")
+ }
+
+ owned, err := s.isSecureValueOwnedByResource(ctx, owner, name)
+ if err != nil {
+ return err
+ }
+
+ if !owned {
+ if err := s.canIdentityReadSecureValue(ctx, xkube.Namespace(owner.Namespace), name); err != nil {
+ return err
+ }
+ }
+ }
+
+ return nil
+}
+
+func (s *inlineSecureValueService) isSecureValueOwnedByResource(ctx context.Context, owner common.ObjectReference, name string) (bool, error) {
+ sv, err := s.secureValueService.Read(ctx, xkube.Namespace(owner.Namespace), name)
+ if err != nil {
+ if errors.Is(err, contracts.ErrSecureValueNotFound) {
+ return false, err
+ }
+
+ return false, fmt.Errorf("error reading secure value %s: %w", name, err)
+ }
+
+ secureValueOwners := sv.GetOwnerReferences()
+ if len(secureValueOwners) > 1 {
+ return false, fmt.Errorf("bug found: secure value %s with multiple owners, expected only one", name)
+ }
+
+ if len(secureValueOwners) == 1 {
+ actualOwner := secureValueOwners[0]
+
+ gv, err := schema.ParseGroupVersion(actualOwner.APIVersion)
+ if err != nil {
+ return false, fmt.Errorf("bug found: secure value %s should have valid group version here: %w", name, err)
+ }
+ if gv.Group == "" {
+ return false, fmt.Errorf("bug found: secure value %s should have a non-empty group in the owner reference", name)
+ }
+
+ sameOwner := owner.APIGroup == gv.Group && owner.Kind == actualOwner.Kind && owner.Name == actualOwner.Name
+ if sameOwner {
+ return true, nil // The secure value is owned by the same owner reference, pass!
+ }
+
+ return false, fmt.Errorf("secure value %s is not owned by %v but by %v", name, owner, actualOwner)
+ }
+
+ // not owned
+ return false, nil
+}
+
+func (s *inlineSecureValueService) canIdentityReadSecureValue(ctx context.Context, namespace xkube.Namespace, name string) error {
+ authInfo, ok := authlib.AuthInfoFrom(ctx)
+ if !ok {
+ return fmt.Errorf("missing auth info in context")
+ }
+
+ // If the secure value is shared, we always need a user/svc account in the context.
+ if authInfo.GetIdentityType() != authlib.TypeUser && authInfo.GetIdentityType() != authlib.TypeServiceAccount {
+ return fmt.Errorf("identity type %s not allowed, expected either %s or %s", authInfo.GetIdentityType(), authlib.TypeUser, authlib.TypeServiceAccount)
+ }
+
+ resp, err := s.accessChecker.Check(ctx, authInfo, authlib.CheckRequest{
+ Verb: utils.VerbGet,
+ Group: secretv1beta1.APIGroup,
+ Resource: secretv1beta1.SecureValuesResourceInfo.GroupResource().Resource,
+ Namespace: namespace.String(),
+ Name: name,
+ })
+ if err != nil {
+ return fmt.Errorf("checking access for secure value %s: %w", name, err)
+ }
+
+ if !resp.Allowed {
+ return fmt.Errorf("identity is not allowed to reference secure value %s", name)
+ }
+
+ return nil
+}
+
+func (s *inlineSecureValueService) CreateInline(ctx context.Context, owner common.ObjectReference, value common.RawSecureValue) (string, error) {
+ ctx, span := s.tracer.Start(ctx, "InlineSecureValueService.CreateInline", trace.WithAttributes(
+ attribute.String("owner.namespace", owner.Namespace),
+ attribute.String("owner.apiGroup", owner.APIGroup),
+ attribute.String("owner.apiVersion", owner.APIVersion),
+ attribute.String("owner.kind", owner.Kind),
+ attribute.String("owner.name", owner.Name),
+ ))
+ defer span.End()
+
+ authInfo, ok := authlib.AuthInfoFrom(ctx)
+ if !ok {
+ return "", fmt.Errorf("missing auth info in context")
+ }
+
+ if authInfo.GetIdentityType() != authlib.TypeUser && authInfo.GetIdentityType() != authlib.TypeServiceAccount {
+ return "", fmt.Errorf("identity type %s not allowed, expected either %s or %s", authInfo.GetIdentityType(), authlib.TypeUser, authlib.TypeServiceAccount)
+ }
+
+ serviceIdentityList, ok := authInfo.GetExtra()[authn.ServiceIdentityKey]
+ if !ok || len(serviceIdentityList) != 1 {
+ return "", fmt.Errorf("expected exactly one service identity, found %d", len(serviceIdentityList))
+ }
+ serviceIdentity := serviceIdentityList[0]
+
+ if owner.Namespace == "" || !authlib.NamespaceMatches(authInfo.GetNamespace(), owner.Namespace) {
+ return "", fmt.Errorf("owner namespace %s does not match auth info namespace %s", owner.Namespace, authInfo.GetNamespace())
+ }
+
+ if owner.APIGroup == "" || owner.APIVersion == "" || owner.Kind == "" || owner.Name == "" {
+ return "", fmt.Errorf("owner reference must have a valid API group, API version, kind and name")
+ }
+
+ if value.IsZero() {
+ return "", fmt.Errorf("trying to create an inline secure value with empty value")
+ }
+
+ // TODO(2025-07-31): when we migrate to using the common type, we don't need this conversion.
+ secret := secretv1beta1.ExposedSecureValue(value)
+
+ spec := &secretv1beta1.SecureValue{
+ ObjectMeta: metav1.ObjectMeta{
+ Name: "sv-" + util.GenerateShortUID(),
+ Namespace: owner.Namespace,
+ OwnerReferences: []metav1.OwnerReference{owner.ToOwnerReference()},
+ },
+ Spec: secretv1beta1.SecureValueSpec{
+ Description: fmt.Sprintf("Inline secure value for %s/%s in %s/%s", owner.Kind, owner.Name, owner.APIVersion, owner.APIVersion),
+ Value: &secret,
+ Decrypters: []string{
+ serviceIdentity,
+ },
+ },
+ }
+
+ createdSv, err := s.secureValueService.Create(ctx, spec, authInfo.GetUID())
+ if err != nil {
+ return "", fmt.Errorf("error creating secure value %s for owner %v: %w", spec.Name, owner, err)
+ }
+
+ return createdSv.GetName(), nil
+}
+
+func (s *inlineSecureValueService) DeleteWhenOwnedByResource(ctx context.Context, owner common.ObjectReference, name string) error {
+ ctx, span := s.tracer.Start(ctx, "InlineSecureValueService.DeleteWhenOwnedByResource", trace.WithAttributes(
+ attribute.String("owner.namespace", owner.Namespace),
+ attribute.String("owner.apiGroup", owner.APIGroup),
+ attribute.String("owner.apiVersion", owner.APIVersion),
+ attribute.String("owner.kind", owner.Kind),
+ attribute.String("owner.name", owner.Name),
+ attribute.String("secureValue.name", name),
+ ))
+ defer span.End()
+
+ authInfo, ok := authlib.AuthInfoFrom(ctx)
+ if !ok {
+ return fmt.Errorf("missing auth info in context")
+ }
+
+ if owner.Namespace == "" || !authlib.NamespaceMatches(authInfo.GetNamespace(), owner.Namespace) {
+ return fmt.Errorf("owner namespace %s does not match auth info namespace %s", owner.Namespace, authInfo.GetNamespace())
+ }
+
+ if owner.APIGroup == "" || owner.APIVersion == "" || owner.Kind == "" || owner.Name == "" {
+ return fmt.Errorf("owner reference must have a valid API group, API version, kind and name")
+ }
+
+ owned, err := s.isSecureValueOwnedByResource(ctx, owner, name)
+ if err != nil {
+ return fmt.Errorf("error checking if secure value %s is owned by %v: %w", name, owner, err)
+ }
+
+ if owned {
+ if _, err := s.secureValueService.Delete(ctx, xkube.Namespace(owner.Namespace), name); err != nil {
+ return fmt.Errorf("error deleting secure value %s for owner %v: %w", name, owner, err)
+ }
+ }
+
+ // if it is not owned, this is a no-op
+ return nil
+}
diff --git a/pkg/registry/apis/secret/service/inline_secure_value_test.go b/pkg/registry/apis/secret/service/inline_secure_value_test.go
new file mode 100644
index 00000000000..3c8742bffb3
--- /dev/null
+++ b/pkg/registry/apis/secret/service/inline_secure_value_test.go
@@ -0,0 +1,512 @@
+package service_test
+
+import (
+ "testing"
+
+ common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
+ "github.com/grafana/grafana/pkg/apimachinery/identity"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/service"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/testutils"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
+ "github.com/stretchr/testify/require"
+ "go.opentelemetry.io/otel/trace/noop"
+ metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+func TestIntegration_InlineSecureValue_CanReference(t *testing.T) {
+ t.Parallel()
+
+ tracer := noop.NewTracerProvider().Tracer("test")
+
+ defaultNs := "org-1234"
+ owner := common.ObjectReference{
+ APIGroup: "prometheus.datasource.grafana.app",
+ APIVersion: "v1alpha1",
+ Kind: "DataSourceConfig",
+ Name: "test-datasource",
+ Namespace: defaultNs,
+ }
+
+ t.Run("happy path with owned and shared secure values", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ createdSv1, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ cfg.Sv.OwnerReferences = []metav1.OwnerReference{owner.ToOwnerReference()}
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv1)
+
+ sv2 := "test-secure-value-2"
+ createdSv2, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv2
+ cfg.Sv.Namespace = defaultNs
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv2)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{
+ "securevalues:read": {"securevalues:uid:" + sv2},
+ })
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, tu.AccessClient)
+
+ err = svc.CanReference(ctx, owner, sv1, sv2)
+ require.NoError(t, err)
+ })
+
+ t.Run("when the auth info is missing it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+ err := svc.CanReference(t.Context(), common.ObjectReference{})
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace does not match auth info namespace it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ reqNs := "org-2345"
+ ctx := testutils.CreateUserAuthContext(t.Context(), reqNs, map[string][]string{})
+
+ err := svc.CanReference(ctx, owner)
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace is empty it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ err := svc.CanReference(ctx, common.ObjectReference{})
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner reference has empty fields it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ owner := common.ObjectReference{
+ Namespace: defaultNs,
+ }
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ err := svc.CanReference(ctx, owner)
+ require.Error(t, err)
+
+ owner.APIGroup = "prometheus.datasource.grafana.app"
+ require.Error(t, svc.CanReference(ctx, owner))
+ owner.APIGroup = ""
+
+ owner.APIVersion = "v1alpha1"
+ require.Error(t, svc.CanReference(ctx, owner))
+ owner.APIVersion = ""
+
+ owner.Kind = "DataSourceConfig"
+ require.Error(t, svc.CanReference(ctx, owner))
+ owner.Kind = ""
+
+ owner.Name = "test-datasource"
+ require.Error(t, svc.CanReference(ctx, owner))
+ owner.Name = ""
+ })
+
+ t.Run("when no secure values are provided it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ err := svc.CanReference(ctx, owner)
+ require.Error(t, err)
+ })
+
+ t.Run("when the secure value does not exist, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ err := svc.CanReference(ctx, owner, "non-existent-sv")
+ require.Error(t, err)
+ })
+
+ t.Run("when the secure value is owned by a different resource, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ differentOwner := common.ObjectReference{
+ APIGroup: "prometheus.datasource.grafana.app",
+ APIVersion: "v1alpha1",
+ Kind: "DataSourceConfig",
+ Name: "different-datasource",
+ Namespace: defaultNs,
+ }
+
+ sv1 := "test-secure-value-1"
+ createdSv1, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ cfg.Sv.OwnerReferences = []metav1.OwnerReference{differentOwner.ToOwnerReference()}
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv1)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ err = svc.CanReference(ctx, owner, sv1)
+ require.Error(t, err)
+ })
+
+ t.Run("when the request identity is not a user nor a service account, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ _, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ })
+ require.NoError(t, err)
+
+ ctx := identity.WithServiceIdentityContext(t.Context(), 1234)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ err = svc.CanReference(ctx, owner, sv1)
+ require.Error(t, err)
+ })
+
+ t.Run("when the identity does not have permissions to read the secure value, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ _, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ })
+ require.NoError(t, err)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, tu.AccessClient)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{
+ "securevalues:read": {"securevalues:uid:another-sv"}, // can read, but another resource!
+ })
+
+ err = svc.CanReference(ctx, owner, sv1)
+ require.Error(t, err)
+
+ ctx = testutils.CreateUserAuthContext(t.Context(), defaultNs, nil)
+
+ err = svc.CanReference(ctx, owner, sv1)
+ require.Error(t, err)
+ })
+}
+
+func TestIntegration_InlineSecureValue_CreateInline(t *testing.T) {
+ t.Parallel()
+
+ tracer := noop.NewTracerProvider().Tracer("test")
+
+ defaultNs := "org-1234"
+ owner := common.ObjectReference{
+ APIGroup: "prometheus.datasource.grafana.app",
+ APIVersion: "v1alpha1",
+ Kind: "DataSourceConfig",
+ Name: "test-datasource",
+ Namespace: defaultNs,
+ }
+
+ t.Run("happy path creates an inline secure value", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ secret := common.NewSecretValue("test-value")
+
+ serviceIdentity := "service-identity"
+
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), serviceIdentity, owner.Namespace, nil, nil)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ createdName, err := svc.CreateInline(createAuthCtx, owner, secret)
+ require.NoError(t, err)
+ require.NotEmpty(t, createdName)
+
+ decryptAuthCtx := testutils.CreateServiceAuthContext(t.Context(), serviceIdentity, owner.Namespace, []string{"secret.grafana.app/securevalues:decrypt"})
+
+ decryptedValues, err := tu.DecryptService.Decrypt(decryptAuthCtx, owner.Namespace, createdName)
+ require.NoError(t, err)
+
+ decryptedResult, ok := decryptedValues[createdName]
+ require.True(t, ok)
+ require.Equal(t, decryptedResult.Value().DangerouslyExposeAndConsumeValue(), secret.DangerouslyExposeAndConsumeValue())
+ })
+
+ t.Run("when the auth info is missing it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+ _, err := svc.CreateInline(t.Context(), common.ObjectReference{}, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the request identity is not a user nor a service account, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ createAuthCtx := testutils.CreateServiceAuthContext(t.Context(), "service-identity", defaultNs, nil)
+
+ _, err := svc.CreateInline(createAuthCtx, common.ObjectReference{}, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace does not match auth info namespace it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ reqNs := "org-2345"
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), "service-identity", reqNs, nil, nil)
+
+ _, err := svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace is empty it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), "service-identity", defaultNs, nil, nil)
+
+ _, err := svc.CreateInline(createAuthCtx, common.ObjectReference{}, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner reference has empty fields it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ owner := common.ObjectReference{
+ Namespace: defaultNs,
+ }
+
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), "service-identity", defaultNs, nil, nil)
+
+ _, err := svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+
+ owner.APIGroup = "prometheus.datasource.grafana.app"
+ _, err = svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+
+ owner.APIVersion = "v1alpha1"
+ _, err = svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+
+ owner.Kind = "DataSourceConfig"
+ _, err = svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+ owner.Kind = ""
+
+ owner.Name = "test-datasource"
+ _, err = svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when an empty secret is provided it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), "service-identity", defaultNs, nil, nil)
+
+ _, err := svc.CreateInline(createAuthCtx, owner, "")
+ require.Error(t, err)
+ })
+}
+
+func TestIntegration_InlineSecureValue_DeleteWhenOwnedByResource(t *testing.T) {
+ t.Parallel()
+
+ tracer := noop.NewTracerProvider().Tracer("test")
+
+ defaultNs := "org-1234"
+ owner := common.ObjectReference{
+ APIGroup: "prometheus.datasource.grafana.app",
+ APIVersion: "v1alpha1",
+ Kind: "DataSourceConfig",
+ Name: "test-datasource",
+ Namespace: defaultNs,
+ }
+
+ t.Run("happy path deletes an owned secure value", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ createdSv1, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ cfg.Sv.OwnerReferences = []metav1.OwnerReference{owner.ToOwnerReference()}
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv1)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ ctx := testutils.CreateServiceAuthContext(t.Context(), "", defaultNs, nil)
+
+ err = svc.DeleteWhenOwnedByResource(ctx, owner, sv1)
+ require.NoError(t, err)
+
+ // make sure it got deleted
+ sv, err := tu.SecureValueService.Read(ctx, xkube.Namespace(owner.Namespace), sv1)
+ require.ErrorIs(t, err, contracts.ErrSecureValueNotFound)
+ require.Nil(t, sv)
+ })
+
+ t.Run("when the auth info is missing it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+ err := svc.DeleteWhenOwnedByResource(t.Context(), common.ObjectReference{}, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace does not match auth info namespace it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ reqNs := "org-2345"
+ ctx := testutils.CreateUserAuthContext(t.Context(), reqNs, map[string][]string{})
+
+ err := svc.DeleteWhenOwnedByResource(ctx, owner, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner namespace is empty it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ ctx := testutils.CreateUserAuthContext(t.Context(), defaultNs, map[string][]string{})
+
+ err := svc.DeleteWhenOwnedByResource(ctx, common.ObjectReference{}, "")
+ require.Error(t, err)
+ })
+
+ t.Run("when the owner reference has empty fields it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ svc := service.ProvideInlineSecureValueService(tracer, nil, nil)
+
+ owner := common.ObjectReference{
+ Namespace: defaultNs,
+ }
+
+ createAuthCtx := testutils.CreateOBOAuthContext(t.Context(), "service-identity", defaultNs, nil, nil)
+
+ require.Error(t, svc.DeleteWhenOwnedByResource(createAuthCtx, owner, ""))
+
+ owner.APIGroup = "prometheus.datasource.grafana.app"
+ require.Error(t, svc.DeleteWhenOwnedByResource(createAuthCtx, owner, ""))
+
+ owner.APIVersion = "v1alpha1"
+ require.Error(t, svc.DeleteWhenOwnedByResource(createAuthCtx, owner, ""))
+
+ owner.Kind = "DataSourceConfig"
+ require.Error(t, svc.DeleteWhenOwnedByResource(createAuthCtx, owner, ""))
+ owner.Kind = ""
+
+ owner.Name = "test-datasource"
+ require.Error(t, svc.DeleteWhenOwnedByResource(createAuthCtx, owner, ""))
+ })
+
+ t.Run("when the secure value exists but the owner does not match, it returns an error", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ createdSv1, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ cfg.Sv.OwnerReferences = []metav1.OwnerReference{
+ {
+ APIVersion: "another.example.com/v0alpha1",
+ Kind: "another-kind",
+ Name: "another-name",
+ },
+ }
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv1)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ ctx := testutils.CreateServiceAuthContext(t.Context(), "", defaultNs, nil)
+
+ err = svc.DeleteWhenOwnedByResource(ctx, owner, sv1)
+ require.Error(t, err)
+
+ // make sure it still exists
+ sv, err := tu.SecureValueService.Read(ctx, xkube.Namespace(owner.Namespace), sv1)
+ require.NoError(t, err)
+ require.NotNil(t, sv)
+ require.Equal(t, sv1, sv.GetName())
+ })
+
+ t.Run("when the secure value exists but it is shared (no owner), it does not return an error (noop)", func(t *testing.T) {
+ t.Parallel()
+
+ tu := testutils.Setup(t)
+
+ sv1 := "test-secure-value-1"
+ createdSv1, err := tu.CreateSv(t.Context(), func(cfg *testutils.CreateSvConfig) {
+ cfg.Sv.Name = sv1
+ cfg.Sv.Namespace = defaultNs
+ })
+ require.NoError(t, err)
+ require.NotNil(t, createdSv1)
+
+ svc := service.ProvideInlineSecureValueService(tracer, tu.SecureValueService, nil)
+
+ ctx := testutils.CreateServiceAuthContext(t.Context(), "", defaultNs, nil)
+
+ err = svc.DeleteWhenOwnedByResource(ctx, owner, sv1)
+ require.NoError(t, err)
+
+ // make sure it still exists
+ sv, err := tu.SecureValueService.Read(ctx, xkube.Namespace(owner.Namespace), sv1)
+ require.NoError(t, err)
+ require.NotNil(t, sv)
+ require.Equal(t, sv1, sv.GetName())
+ })
+}
diff --git a/pkg/registry/apis/secret/service/metrics/metrics.go b/pkg/registry/apis/secret/service/metrics/metrics.go
new file mode 100644
index 00000000000..b179ead20b7
--- /dev/null
+++ b/pkg/registry/apis/secret/service/metrics/metrics.go
@@ -0,0 +1,128 @@
+package metrics
+
+import (
+ "sync"
+
+ "github.com/prometheus/client_golang/prometheus"
+)
+
+const (
+ namespace = "grafana_secrets_manager"
+ subsystem = "service"
+)
+
+// SecureValueServiceMetrics is a struct that contains all the metrics for SecureValue.
+type SecureValueServiceMetrics struct {
+ SecureValueCreateDuration *prometheus.HistogramVec
+ SecureValueCreateCount *prometheus.CounterVec
+ SecureValueUpdateDuration *prometheus.HistogramVec
+ SecureValueUpdateCount *prometheus.CounterVec
+ SecureValueReadDuration *prometheus.HistogramVec
+ SecureValueReadCount *prometheus.CounterVec
+ SecureValueListDuration *prometheus.HistogramVec
+ SecureValueListCount *prometheus.CounterVec
+ SecureValueDeleteDuration *prometheus.HistogramVec
+ SecureValueDeleteCount *prometheus.CounterVec
+}
+
+func newSecureValueServiceMetrics() *SecureValueServiceMetrics {
+ return &SecureValueServiceMetrics{
+ SecureValueCreateDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_create_duration_seconds",
+ Help: "Duration of Secure Value create operations",
+ Buckets: prometheus.DefBuckets,
+ }, []string{"success"}),
+ SecureValueCreateCount: prometheus.NewCounterVec(prometheus.CounterOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_create_count",
+ Help: "Count of Secure Value create operations",
+ }, []string{"success"}),
+ SecureValueReadDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_read_duration_seconds",
+ Help: "Duration of Secure Value read operations",
+ Buckets: prometheus.DefBuckets,
+ }, []string{"success"}),
+ SecureValueReadCount: prometheus.NewCounterVec(prometheus.CounterOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_read_count",
+ Help: "Count of Secure Value read operations",
+ }, []string{"success"}),
+ SecureValueUpdateDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_update_duration_seconds",
+ Help: "Duration of Secure Value update operations",
+ Buckets: prometheus.DefBuckets,
+ }, []string{"success"}),
+ SecureValueUpdateCount: prometheus.NewCounterVec(prometheus.CounterOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_update_count",
+ Help: "Count of Secure Value update operations",
+ }, []string{"success"}),
+ SecureValueListDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_list_duration_seconds",
+ Help: "Duration of Secure Value list operations",
+ Buckets: prometheus.DefBuckets,
+ }, []string{"success"}),
+ SecureValueListCount: prometheus.NewCounterVec(prometheus.CounterOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_list_count",
+ Help: "Count of Secure Value list operations",
+ }, []string{"success"}),
+ SecureValueDeleteDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_delete_duration_seconds",
+ Help: "Duration of Secure Value delete operations",
+ Buckets: prometheus.DefBuckets,
+ }, []string{"success"}),
+ SecureValueDeleteCount: prometheus.NewCounterVec(prometheus.CounterOpts{
+ Namespace: namespace,
+ Subsystem: subsystem,
+ Name: "secure_value_delete_count",
+ Help: "Count of Secure Value delete operations",
+ }, []string{"success"}),
+ }
+}
+
+var (
+ initOnce sync.Once
+ metricsInstance *SecureValueServiceMetrics
+)
+
+func NewSecureValueServiceMetrics(reg prometheus.Registerer) *SecureValueServiceMetrics {
+ initOnce.Do(func() {
+ m := newSecureValueServiceMetrics()
+
+ if reg != nil {
+ reg.MustRegister(
+ m.SecureValueCreateDuration,
+ m.SecureValueCreateCount,
+ m.SecureValueReadDuration,
+ m.SecureValueReadCount,
+ m.SecureValueUpdateDuration,
+ m.SecureValueUpdateCount,
+ m.SecureValueListDuration,
+ m.SecureValueListCount,
+ m.SecureValueDeleteDuration,
+ m.SecureValueDeleteCount,
+ )
+ }
+ metricsInstance = m
+ })
+ return metricsInstance
+}
+
+func NewTestMetrics() *SecureValueServiceMetrics {
+ return newSecureValueServiceMetrics()
+}
diff --git a/pkg/registry/apis/secret/service/secure_value.go b/pkg/registry/apis/secret/service/secure_value.go
index f9bafad9467..fe32208562d 100644
--- a/pkg/registry/apis/secret/service/secure_value.go
+++ b/pkg/registry/apis/secret/service/secure_value.go
@@ -3,6 +3,8 @@ package service
import (
"context"
"fmt"
+ "strconv"
+ "time"
claims "github.com/grafana/authlib/types"
"go.opentelemetry.io/otel/attribute"
@@ -12,9 +14,14 @@ import (
secretv1beta1 "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
+ "github.com/grafana/grafana/pkg/registry/apis/secret/service/metrics"
"github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
+ "github.com/prometheus/client_golang/prometheus"
+ "go.opentelemetry.io/otel/codes"
)
+var _ contracts.SecureValueService = (*SecureValueService)(nil)
+
type SecureValueService struct {
tracer trace.Tracer
accessClient claims.AccessClient
@@ -22,6 +29,7 @@ type SecureValueService struct {
secureValueMetadataStorage contracts.SecureValueMetadataStorage
keeperMetadataStorage contracts.KeeperMetadataStorage
keeperService contracts.KeeperService
+ metrics *metrics.SecureValueServiceMetrics
}
func ProvideSecureValueService(
@@ -31,6 +39,7 @@ func ProvideSecureValueService(
secureValueMetadataStorage contracts.SecureValueMetadataStorage,
keeperMetadataStorage contracts.KeeperMetadataStorage,
keeperService contracts.KeeperService,
+ reg prometheus.Registerer,
) contracts.SecureValueService {
return &SecureValueService{
tracer: tracer,
@@ -39,27 +48,77 @@ func ProvideSecureValueService(
secureValueMetadataStorage: secureValueMetadataStorage,
keeperMetadataStorage: keeperMetadataStorage,
keeperService: keeperService,
+ metrics: metrics.NewSecureValueServiceMetrics(reg),
}
}
-func (s *SecureValueService) Create(ctx context.Context, sv *secretv1beta1.SecureValue, actorUID string) (*secretv1beta1.SecureValue, error) {
+func (s *SecureValueService) Create(ctx context.Context, sv *secretv1beta1.SecureValue, actorUID string) (_ *secretv1beta1.SecureValue, createErr error) {
+ start := time.Now()
+ name, namespace := sv.GetName(), sv.GetNamespace()
ctx, span := s.tracer.Start(ctx, "SecureValueService.Create", trace.WithAttributes(
- attribute.String("name", sv.GetName()),
- attribute.String("namespace", sv.GetNamespace()),
+ attribute.String("name", name),
+ attribute.String("namespace", namespace),
attribute.String("actor", actorUID),
))
defer span.End()
+
+ defer func() {
+ args := []any{
+ "name", name,
+ "namespace", namespace,
+ "actorUID", actorUID,
+ }
+
+ success := createErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueService.Create failed")
+ span.RecordError(createErr)
+ args = append(args, "error", createErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueService.Create finished", args...)
+
+ s.metrics.SecureValueCreateDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueCreateCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
return s.createNewVersion(ctx, sv, actorUID)
}
-func (s *SecureValueService) Update(ctx context.Context, newSecureValue *secretv1beta1.SecureValue, actorUID string) (*secretv1beta1.SecureValue, bool, error) {
+func (s *SecureValueService) Update(ctx context.Context, newSecureValue *secretv1beta1.SecureValue, actorUID string) (_ *secretv1beta1.SecureValue, sync bool, updateErr error) {
+ start := time.Now()
+ name, namespace := newSecureValue.GetName(), newSecureValue.GetNamespace()
+
ctx, span := s.tracer.Start(ctx, "SecureValueService.Update", trace.WithAttributes(
- attribute.String("name", newSecureValue.GetName()),
- attribute.String("namespace", newSecureValue.GetNamespace()),
+ attribute.String("name", name),
+ attribute.String("namespace", namespace),
attribute.String("actor", actorUID),
))
defer span.End()
+ defer func() {
+ args := []any{
+ "name", name,
+ "namespace", namespace,
+ "actorUID", actorUID,
+ "sync", sync,
+ }
+
+ success := updateErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueService.Update failed")
+ span.RecordError(updateErr)
+ args = append(args, "error", updateErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueService.Update finished", args...)
+
+ s.metrics.SecureValueUpdateDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueUpdateCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
if newSecureValue.Spec.Value == nil {
currentVersion, err := s.secureValueMetadataStorage.Read(ctx, xkube.Namespace(newSecureValue.Namespace), newSecureValue.Name, contracts.ReadOpts{})
if err != nil {
@@ -136,22 +195,66 @@ func (s *SecureValueService) createNewVersion(ctx context.Context, sv *secretv1b
return createdSv, nil
}
-func (s *SecureValueService) Read(ctx context.Context, namespace xkube.Namespace, name string) (*secretv1beta1.SecureValue, error) {
+func (s *SecureValueService) Read(ctx context.Context, namespace xkube.Namespace, name string) (_ *secretv1beta1.SecureValue, readErr error) {
+ start := time.Now()
+
ctx, span := s.tracer.Start(ctx, "SecureValueService.Read", trace.WithAttributes(
attribute.String("name", name),
attribute.String("namespace", namespace.String()),
))
+
+ defer func() {
+ args := []any{
+ "name", name,
+ "namespace", namespace,
+ }
+
+ success := readErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueService.Read failed")
+ span.RecordError(readErr)
+ args = append(args, "error", readErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueService.Read finished", args...)
+
+ s.metrics.SecureValueReadDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueReadCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
defer span.End()
return s.secureValueMetadataStorage.Read(ctx, namespace, name, contracts.ReadOpts{ForUpdate: false})
}
-func (s *SecureValueService) List(ctx context.Context, namespace xkube.Namespace) (*secretv1beta1.SecureValueList, error) {
+func (s *SecureValueService) List(ctx context.Context, namespace xkube.Namespace) (_ *secretv1beta1.SecureValueList, listErr error) {
+ start := time.Now()
+
ctx, span := s.tracer.Start(ctx, "SecureValueService.List", trace.WithAttributes(
attribute.String("namespace", namespace.String()),
))
defer span.End()
+ defer func() {
+ args := []any{
+ "namespace", namespace,
+ }
+
+ success := listErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueService.List failed")
+ span.RecordError(listErr)
+ args = append(args, "error", listErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueService.List finished", args...)
+
+ s.metrics.SecureValueListDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueListCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
user, ok := claims.AuthInfoFrom(ctx)
if !ok {
return nil, fmt.Errorf("missing auth info in context")
@@ -188,13 +291,35 @@ func (s *SecureValueService) List(ctx context.Context, namespace xkube.Namespace
}, nil
}
-func (s *SecureValueService) Delete(ctx context.Context, namespace xkube.Namespace, name string) (*secretv1beta1.SecureValue, error) {
+func (s *SecureValueService) Delete(ctx context.Context, namespace xkube.Namespace, name string) (_ *secretv1beta1.SecureValue, deleteErr error) {
+ start := time.Now()
+
ctx, span := s.tracer.Start(ctx, "SecureValueService.Delete", trace.WithAttributes(
attribute.String("name", name),
attribute.String("namespace", namespace.String()),
))
defer span.End()
+ defer func() {
+ args := []any{
+ "name", name,
+ "namespace", namespace,
+ }
+
+ success := deleteErr == nil
+ args = append(args, "success", success)
+ if !success {
+ span.SetStatus(codes.Error, "SecureValueService.Delete failed")
+ span.RecordError(deleteErr)
+ args = append(args, "error", deleteErr)
+ }
+
+ logging.FromContext(ctx).Info("SecureValueService.Delete finished", args...)
+
+ s.metrics.SecureValueDeleteDuration.WithLabelValues(strconv.FormatBool(success)).Observe(time.Since(start).Seconds())
+ s.metrics.SecureValueDeleteCount.WithLabelValues(strconv.FormatBool(success)).Inc()
+ }()
+
// TODO: does this need to be for update?
sv, err := s.secureValueMetadataStorage.Read(ctx, namespace, name, contracts.ReadOpts{ForUpdate: true})
if err != nil {
diff --git a/pkg/registry/apis/secret/testutils/testutils.go b/pkg/registry/apis/secret/testutils/testutils.go
index f2d24b05905..3e742439bb0 100644
--- a/pkg/registry/apis/secret/testutils/testutils.go
+++ b/pkg/registry/apis/secret/testutils/testutils.go
@@ -87,6 +87,9 @@ func Setup(t *testing.T, opts ...func(*SetupConfig)) Sut {
store, err := encryptionstorage.ProvideDataKeyStorage(database, tracer, nil)
require.NoError(t, err)
+ globalDataKeyStore, err := encryptionstorage.ProvideGlobalDataKeyStorage(database, tracer, nil)
+ require.NoError(t, err)
+
usageStats := &usagestats.UsageStatsMock{T: t}
enc, err := cipher.ProvideAESGCMCipherService(tracer, usageStats)
@@ -120,7 +123,7 @@ func Setup(t *testing.T, opts ...func(*SetupConfig)) Sut {
keeperService = setupCfg.KeeperService
}
- secureValueService := service.ProvideSecureValueService(tracer, accessClient, database, secureValueMetadataStorage, keeperMetadataStorage, keeperService)
+ secureValueService := service.ProvideSecureValueService(tracer, accessClient, database, secureValueMetadataStorage, keeperMetadataStorage, keeperService, nil)
decryptAuthorizer := decrypt.ProvideDecryptAuthorizer(tracer)
@@ -135,6 +138,8 @@ func Setup(t *testing.T, opts ...func(*SetupConfig)) Sut {
decryptService, err := decrypt.ProvideDecryptService(testCfg, tracer, decryptStorage)
require.NoError(t, err)
+ consolidationService := service.ProvideConsolidationService(tracer, globalDataKeyStore, encryptedValueStorage, globalEncryptedValueStorage, encryptionManager)
+
return Sut{
SecureValueService: secureValueService,
SecureValueMetadataStorage: secureValueMetadataStorage,
@@ -145,6 +150,9 @@ func Setup(t *testing.T, opts ...func(*SetupConfig)) Sut {
SQLKeeper: sqlKeeper,
Database: database,
AccessClient: accessClient,
+ ConsolidationService: consolidationService,
+ EncryptionManager: encryptionManager,
+ GlobalDataKeyStore: globalDataKeyStore,
}
}
@@ -158,6 +166,9 @@ type Sut struct {
SQLKeeper *sqlkeeper.SQLKeeper
Database *database.Database
AccessClient types.AccessClient
+ ConsolidationService contracts.ConsolidationService
+ EncryptionManager contracts.EncryptionManager
+ GlobalDataKeyStore contracts.GlobalDataKeyStorage
}
type CreateSvConfig struct {
@@ -233,8 +244,9 @@ func CreateUserAuthContext(ctx context.Context, namespace string, permissions ma
return types.WithAuthInfo(ctx, requester)
}
-func CreateServiceAuthContext(ctx context.Context, serviceIdentity string, permissions []string) context.Context {
+func CreateServiceAuthContext(ctx context.Context, serviceIdentity string, namespace string, permissions []string) context.Context {
requester := &identity.StaticRequester{
+ Namespace: namespace,
AccessTokenClaims: &authn.Claims[authn.AccessTokenClaims]{
Rest: authn.AccessTokenClaims{
Permissions: permissions,
@@ -245,3 +257,32 @@ func CreateServiceAuthContext(ctx context.Context, serviceIdentity string, permi
return types.WithAuthInfo(ctx, requester)
}
+
+// CreateOBOAuthContext emulates a context where the request is made on-behalf-of (OBO) a user, with an access token.
+func CreateOBOAuthContext(
+ ctx context.Context,
+ serviceIdentity string,
+ namespace string,
+ userPermissions map[string][]string,
+ delegatedPermissions []string,
+) context.Context {
+ requester := &identity.StaticRequester{
+ Namespace: namespace,
+ Type: types.TypeUser,
+ UserID: 1,
+ Permissions: map[int64]map[string][]string{
+ 1: userPermissions,
+ },
+ AccessTokenClaims: &authn.Claims[authn.AccessTokenClaims]{
+ Rest: authn.AccessTokenClaims{
+ ServiceIdentity: serviceIdentity,
+ DelegatedPermissions: delegatedPermissions,
+ Actor: &authn.ActorClaims{
+ Subject: "user:1",
+ },
+ },
+ },
+ }
+
+ return types.WithAuthInfo(ctx, requester)
+}
diff --git a/pkg/registry/apps/playlist/register.go b/pkg/registry/apps/playlist/register.go
index 4374c0265dd..123974c2dcc 100644
--- a/pkg/registry/apps/playlist/register.go
+++ b/pkg/registry/apps/playlist/register.go
@@ -26,7 +26,6 @@ import (
var (
_ appsdkapiserver.AppInstaller = (*PlaylistAppInstaller)(nil)
_ appinstaller.LegacyStorageProvider = (*PlaylistAppInstaller)(nil)
- _ appinstaller.APIEnablementProvider = (*PlaylistAppInstaller)(nil)
)
type PlaylistAppInstaller struct {
@@ -102,10 +101,3 @@ func (p *PlaylistAppInstaller) GetLegacyStorage(requested schema.GroupVersionRes
)
return legacyStore
}
-
-// GetAllowedV0Alpha1Resources returns the list of resources that are allowed to be accessed in v0alpha1.
-func (p *PlaylistAppInstaller) GetAllowedV0Alpha1Resources() []string {
- return []string{
- playlistv0alpha1.PlaylistKind().Plural(),
- }
-}
diff --git a/pkg/server/runner.go b/pkg/server/runner.go
index ae9ccf28469..cc31af9711b 100644
--- a/pkg/server/runner.go
+++ b/pkg/server/runner.go
@@ -8,32 +8,36 @@ import (
"github.com/grafana/grafana/pkg/services/secrets/manager"
"github.com/grafana/grafana/pkg/services/user"
"github.com/grafana/grafana/pkg/setting"
+
+ "github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
)
type Runner struct {
- Cfg *setting.Cfg
- SQLStore db.DB
- SettingsProvider setting.Provider
- Features featuremgmt.FeatureToggles
- EncryptionService encryption.Internal
- SecretsService *manager.SecretsService
- SecretsMigrator secrets.Migrator
- UserService user.Service
+ Cfg *setting.Cfg
+ SQLStore db.DB
+ SettingsProvider setting.Provider
+ Features featuremgmt.FeatureToggles
+ EncryptionService encryption.Internal
+ SecretsService *manager.SecretsService
+ SecretsMigrator secrets.Migrator
+ UserService user.Service
+ SecretsConsolidationService contracts.ConsolidationService
}
func NewRunner(cfg *setting.Cfg, sqlStore db.DB, settingsProvider setting.Provider,
encryptionService encryption.Internal, features featuremgmt.FeatureToggles,
secretsService *manager.SecretsService, secretsMigrator secrets.Migrator,
- userService user.Service,
+ userService user.Service, secretsConsolidationService contracts.ConsolidationService,
) Runner {
return Runner{
- Cfg: cfg,
- SQLStore: sqlStore,
- SettingsProvider: settingsProvider,
- EncryptionService: encryptionService,
- SecretsService: secretsService,
- SecretsMigrator: secretsMigrator,
- Features: features,
- UserService: userService,
+ Cfg: cfg,
+ SQLStore: sqlStore,
+ SettingsProvider: settingsProvider,
+ EncryptionService: encryptionService,
+ SecretsService: secretsService,
+ SecretsMigrator: secretsMigrator,
+ Features: features,
+ UserService: userService,
+ SecretsConsolidationService: secretsConsolidationService,
}
}
diff --git a/pkg/server/wire.go b/pkg/server/wire.go
index 741ec53ede8..a56c1bff1a6 100644
--- a/pkg/server/wire.go
+++ b/pkg/server/wire.go
@@ -429,7 +429,9 @@ var wireBasicSet = wire.NewSet(
secretdecrypt.ProvideDecryptAuthorizer,
secretdecrypt.ProvideDecryptService,
secretencryption.ProvideDataKeyStorage,
+ secretencryption.ProvideGlobalDataKeyStorage,
secretencryption.ProvideEncryptedValueStorage,
+ secretencryption.ProvideGlobalEncryptedValueStorage,
secretsecurevalueservice.ProvideSecureValueService,
secretvalidator.ProvideKeeperValidator,
secretvalidator.ProvideSecureValueValidator,
diff --git a/pkg/server/wire_gen.go b/pkg/server/wire_gen.go
index b398b85d5a9..cf76c869512 100644
--- a/pkg/server/wire_gen.go
+++ b/pkg/server/wire_gen.go
@@ -746,7 +746,7 @@ func Initialize(cfg *setting.Cfg, opts Options, apiOpts api.ServerOptions) (*Ser
}
userStorageAPIBuilder := userstorage.RegisterAPIService(featureToggles, apiserverService, registerer)
factory := github.ProvideFactory()
- legacyMigrator := legacy.ProvideLegacyMigrator(sqlStore, provisioningServiceImpl, libraryPanelService, accessControl)
+ legacyMigrator := legacy.ProvideLegacyMigrator(sqlStore, provisioningServiceImpl, libraryPanelService, accessControl, featureToggles)
databaseDatabase := database5.ProvideDatabase(sqlStore, tracer)
secureValueMetadataStorage, err := metadata.ProvideSecureValueMetadataStorage(databaseDatabase, tracer, registerer)
if err != nil {
@@ -780,7 +780,7 @@ func Initialize(cfg *setting.Cfg, opts Options, apiOpts api.ServerOptions) (*Ser
if err != nil {
return nil, err
}
- secureValueService := service12.ProvideSecureValueService(tracer, accessClient, databaseDatabase, secureValueMetadataStorage, keeperMetadataStorage, ossKeeperService)
+ secureValueService := service12.ProvideSecureValueService(tracer, accessClient, databaseDatabase, secureValueMetadataStorage, keeperMetadataStorage, ossKeeperService, registerer)
secureValueValidator := validator3.ProvideSecureValueValidator()
secureValueClient := secret.ProvideSecureValueClient(secureValueService, secureValueValidator, accessClient)
decryptAuthorizer := decrypt.ProvideDecryptAuthorizer(tracer)
@@ -1307,7 +1307,7 @@ func InitializeForTest(t sqlutil.ITestDB, testingT interface {
}
userStorageAPIBuilder := userstorage.RegisterAPIService(featureToggles, apiserverService, registerer)
factory := github.ProvideFactory()
- legacyMigrator := legacy.ProvideLegacyMigrator(sqlStore, provisioningServiceImpl, libraryPanelService, accessControl)
+ legacyMigrator := legacy.ProvideLegacyMigrator(sqlStore, provisioningServiceImpl, libraryPanelService, accessControl, featureToggles)
databaseDatabase := database5.ProvideDatabase(sqlStore, tracer)
secureValueMetadataStorage, err := metadata.ProvideSecureValueMetadataStorage(databaseDatabase, tracer, registerer)
if err != nil {
@@ -1341,7 +1341,7 @@ func InitializeForTest(t sqlutil.ITestDB, testingT interface {
if err != nil {
return nil, err
}
- secureValueService := service12.ProvideSecureValueService(tracer, accessClient, databaseDatabase, secureValueMetadataStorage, keeperMetadataStorage, ossKeeperService)
+ secureValueService := service12.ProvideSecureValueService(tracer, accessClient, databaseDatabase, secureValueMetadataStorage, keeperMetadataStorage, ossKeeperService, registerer)
secureValueValidator := validator3.ProvideSecureValueValidator()
secureValueClient := secret.ProvideSecureValueClient(secureValueService, secureValueValidator, accessClient)
decryptAuthorizer := decrypt.ProvideDecryptAuthorizer(tracer)
@@ -1453,7 +1453,39 @@ func InitializeForCLI(cfg *setting.Cfg) (Runner, error) {
if err != nil {
return Runner{}, err
}
- runner := NewRunner(cfg, sqlStore, ossImpl, serviceService, featureToggles, secretsService, secretsMigrator, userService)
+ tracer := otelTracer()
+ databaseDatabase := database5.ProvideDatabase(sqlStore, tracer)
+ registerer := metrics.ProvideRegisterer()
+ globalDataKeyStorage, err := encryption.ProvideGlobalDataKeyStorage(databaseDatabase, tracer, registerer)
+ if err != nil {
+ return Runner{}, err
+ }
+ encryptedValueStorage, err := encryption.ProvideEncryptedValueStorage(databaseDatabase, tracer)
+ if err != nil {
+ return Runner{}, err
+ }
+ globalEncryptedValueStorage, err := encryption.ProvideGlobalEncryptedValueStorage(databaseDatabase, tracer)
+ if err != nil {
+ return Runner{}, err
+ }
+ dataKeyStorage, err := encryption.ProvideDataKeyStorage(databaseDatabase, tracer, registerer)
+ if err != nil {
+ return Runner{}, err
+ }
+ cipher, err := service11.ProvideAESGCMCipherService(tracer, usageStats)
+ if err != nil {
+ return Runner{}, err
+ }
+ providerConfig, err := kmsproviders.ProvideOSSKMSProviders(cfg, cipher)
+ if err != nil {
+ return Runner{}, err
+ }
+ encryptionManager, err := manager4.ProvideEncryptionManager(tracer, dataKeyStorage, usageStats, cipher, providerConfig)
+ if err != nil {
+ return Runner{}, err
+ }
+ consolidationService := service12.ProvideConsolidationService(tracer, globalDataKeyStorage, encryptedValueStorage, globalEncryptedValueStorage, encryptionManager)
+ runner := NewRunner(cfg, sqlStore, ossImpl, serviceService, featureToggles, secretsService, secretsMigrator, userService, consolidationService)
return runner, nil
}
@@ -1540,7 +1572,7 @@ var withOTelSet = wire.NewSet(
otelTracer, grpcserver.ProvideService, interceptors.ProvideAuthenticator,
)
-var wireBasicSet = wire.NewSet(annotationsimpl.ProvideService, wire.Bind(new(annotations.Repository), new(*annotationsimpl.RepositoryImpl)), New, api.ProvideHTTPServer, query.ProvideService, wire.Bind(new(query.Service), new(*query.ServiceImpl)), bus.ProvideBus, wire.Bind(new(bus.Bus), new(*bus.InProcBus)), rendering.ProvideService, wire.Bind(new(rendering.Service), new(*rendering.RenderingService)), routing.ProvideRegister, wire.Bind(new(routing.RouteRegister), new(*routing.RouteRegisterImpl)), hooks.ProvideService, kvstore.ProvideService, localcache.ProvideService, bundleregistry.ProvideService, wire.Bind(new(supportbundles.Service), new(*bundleregistry.Service)), updatemanager.ProvideGrafanaService, updatemanager.ProvidePluginsService, service.ProvideService, wire.Bind(new(usagestats.Service), new(*service.UsageStats)), validator2.ProvideService, legacy.ProvideLegacyMigrator, pluginsintegration.WireSet, dashboards.ProvideFileStoreManager, wire.Bind(new(dashboards.FileStore), new(*dashboards.FileStoreManager)), cloudwatch.ProvideService, cloudmonitoring.ProvideService, azuremonitor.ProvideService, postgres.ProvideService, mysql.ProvideService, mssql.ProvideService, store.ProvideEntityEventsService, dualwrite.ProvideService, httpclientprovider.New, wire.Bind(new(httpclient.Provider), new(*httpclient2.Provider)), serverlock.ProvideService, annotationsimpl.ProvideCleanupService, wire.Bind(new(annotations.Cleaner), new(*annotationsimpl.CleanupServiceImpl)), cleanup.ProvideService, shorturlimpl.ProvideService, wire.Bind(new(shorturls.Service), new(*shorturlimpl.ShortURLService)), queryhistory.ProvideService, wire.Bind(new(queryhistory.Service), new(*queryhistory.QueryHistoryService)), correlations.ProvideService, wire.Bind(new(correlations.Service), new(*correlations.CorrelationsService)), quotaimpl.ProvideService, remotecache.ProvideService, wire.Bind(new(remotecache.CacheStorage), new(*remotecache.RemoteCache)), authinfoimpl.ProvideService, wire.Bind(new(login.AuthInfoService), new(*authinfoimpl.Service)), authinfoimpl.ProvideStore, datasourceproxy.ProvideService, sort.ProvideService, search2.ProvideService, searchV2.ProvideService, searchV2.ProvideSearchHTTPService, store.ProvideService, store.ProvideSystemUsersService, live.ProvideService, pushhttp.ProvideService, contexthandler.ProvideService, service10.ProvideService, wire.Bind(new(service10.LDAP), new(*service10.LDAPImpl)), jwt.ProvideService, wire.Bind(new(jwt.JWTService), new(*jwt.AuthService)), store2.ProvideDBStore, image.ProvideDeleteExpiredService, ngalert.ProvideService, librarypanels.ProvideService, wire.Bind(new(librarypanels.Service), new(*librarypanels.LibraryPanelService)), libraryelements.ProvideService, wire.Bind(new(libraryelements.Service), new(*libraryelements.LibraryElementService)), notifications.ProvideService, notifications.ProvideSmtpService, github.ProvideFactory, tracing.ProvideService, tracing.ProvideTracingConfig, wire.Bind(new(tracing.Tracer), new(*tracing.TracingService)), withOTelSet, testdatasource.ProvideService, api4.ProvideService, opentsdb.ProvideService, socialimpl.ProvideService, influxdb.ProvideService, wire.Bind(new(social.Service), new(*socialimpl.SocialService)), tempo.ProvideService, loki.ProvideService, graphite.ProvideService, prometheus.ProvideService, elasticsearch.ProvideService, pyroscope.ProvideService, parca.ProvideService, zipkin.ProvideService, jaeger.ProvideService, service7.ProvideCacheService, wire.Bind(new(datasources.CacheService), new(*service7.CacheServiceImpl)), service2.ProvideEncryptionService, wire.Bind(new(encryption2.Internal), new(*service2.Service)), manager.ProvideSecretsService, wire.Bind(new(secrets2.Service), new(*manager.SecretsService)), database.ProvideSecretsStore, wire.Bind(new(secrets2.Store), new(*database.SecretsStoreImpl)), grafanads.ProvideService, wire.Bind(new(dashboardsnapshots.Store), new(*database4.DashboardSnapshotStore)), database4.ProvideStore, wire.Bind(new(dashboardsnapshots.Service), new(*service8.ServiceImpl)), service8.ProvideService, service7.ProvideService, wire.Bind(new(datasources.DataSourceService), new(*service7.Service)), service7.ProvideLegacyDataSourceLookup, retriever.ProvideService, wire.Bind(new(serviceaccounts.ServiceAccountRetriever), new(*retriever.Service)), ossaccesscontrol.ProvideServiceAccountPermissions, wire.Bind(new(accesscontrol.ServiceAccountPermissionsService), new(*ossaccesscontrol.ServiceAccountPermissionsService)), manager2.ProvideServiceAccountsService, proxy.ProvideServiceAccountsProxy, wire.Bind(new(serviceaccounts.Service), new(*proxy.ServiceAccountsProxy)), mtdsclient.NewNullMTDatasourceClientBuilder, expr.ProvideService, featuremgmt.ProvideManagerService, featuremgmt.ProvideToggles, service5.ProvideDashboardServiceImpl, wire.Bind(new(dashboards2.PermissionsRegistrationService), new(*service5.DashboardServiceImpl)), service5.ProvideDashboardService, service5.ProvideDashboardProvisioningService, service5.ProvideDashboardPluginService, database2.ProvideDashboardStore, folderimpl.ProvideService, wire.Bind(new(folder.Service), new(*folderimpl.Service)), folderimpl.ProvideStore, wire.Bind(new(folder.Store), new(*folderimpl.FolderStoreImpl)), folderimpl.ProvideDashboardFolderStore, wire.Bind(new(folder.FolderStore), new(*folderimpl.DashboardFolderStoreImpl)), service9.ProvideService, wire.Bind(new(dashboardimport.Service), new(*service9.ImportDashboardService)), service6.ProvideService, wire.Bind(new(plugindashboards.Service), new(*service6.Service)), service6.ProvideDashboardUpdater, sanitizer.ProvideService, kvstore2.ProvideService, avatar.ProvideAvatarCacheServer, statscollector.ProvideService, csrf.ProvideCSRFFilter, wire.Bind(new(csrf.Service), new(*csrf.CSRF)), ossaccesscontrol.ProvideTeamPermissions, wire.Bind(new(accesscontrol.TeamPermissionsService), new(*ossaccesscontrol.TeamPermissionsService)), ossaccesscontrol.ProvideFolderPermissions, wire.Bind(new(accesscontrol.FolderPermissionsService), new(*ossaccesscontrol.FolderPermissionsService)), ossaccesscontrol.ProvideDashboardPermissions, wire.Bind(new(accesscontrol.DashboardPermissionsService), new(*ossaccesscontrol.DashboardPermissionsService)), ossaccesscontrol.ProvideReceiverPermissionsService, wire.Bind(new(accesscontrol.ReceiverPermissionsService), new(*ossaccesscontrol.ReceiverPermissionsService)), starimpl.ProvideService, playlistimpl.ProvideService, apikeyimpl.ProvideService, dashverimpl.ProvideService, service3.ProvideService, wire.Bind(new(publicdashboards.Service), new(*service3.PublicDashboardServiceImpl)), database3.ProvideStore, wire.Bind(new(publicdashboards.Store), new(*database3.PublicDashboardStoreImpl)), metric.ProvideService, api2.ProvideApi, api3.ProvideApi, userimpl.ProvideService, orgimpl.ProvideService, orgimpl.ProvideDeletionService, statsimpl.ProvideService, grpccontext.ProvideContextHandler, grpcserver.ProvideHealthService, grpcserver.ProvideReflectionService, resolver.ProvideEntityReferenceResolver, teamimpl.ProvideService, teamapi.ProvideTeamAPI, tempuserimpl.ProvideService, loginattemptimpl.ProvideService, wire.Bind(new(loginattempt.Service), new(*loginattemptimpl.Service)), migrations2.ProvideDataSourceMigrationService, migrations2.ProvideSecretMigrationProvider, wire.Bind(new(migrations2.SecretMigrationProvider), new(*migrations2.SecretMigrationProviderImpl)), resourcepermissions.NewActionSetService, wire.Bind(new(accesscontrol.ActionResolver), new(resourcepermissions.ActionSetService)), wire.Bind(new(pluginaccesscontrol.ActionSetRegistry), new(resourcepermissions.ActionSetService)), permreg.ProvidePermissionRegistry, acimpl.ProvideAccessControl, dualwrite2.ProvideZanzanaReconciler, navtreeimpl.ProvideService, wire.Bind(new(accesscontrol.AccessControl), new(*acimpl.AccessControl)), wire.Bind(new(notifications.TempUserStore), new(tempuser.Service)), tagimpl.ProvideService, wire.Bind(new(tag.Service), new(*tagimpl.Service)), authnimpl.ProvideService, authnimpl.ProvideIdentitySynchronizer, authnimpl.ProvideAuthnService, authnimpl.ProvideAuthnServiceAuthenticateOnly, authnimpl.ProvideRegistration, supportbundlesimpl.ProvideService, extsvcaccounts.ProvideExtSvcAccountsService, wire.Bind(new(serviceaccounts.ExtSvcAccountsService), new(*extsvcaccounts.ExtSvcAccountsService)), registry2.ProvideExtSvcRegistry, wire.Bind(new(extsvcauth.ExternalServiceRegistry), new(*registry2.Registry)), anonstore.ProvideAnonDBStore, wire.Bind(new(anonstore.AnonStore), new(*anonstore.AnonDBStore)), loggermw.Provide, slogadapter.Provide, signingkeysimpl.ProvideEmbeddedSigningKeysService, wire.Bind(new(signingkeys.Service), new(*signingkeysimpl.Service)), ssosettingsimpl.ProvideService, wire.Bind(new(ssosettings.Service), new(*ssosettingsimpl.Service)), idimpl.ProvideService, wire.Bind(new(auth.IDService), new(*idimpl.Service)), cloudmigrationimpl.ProvideService, userimpl.ProvideVerifier, connectors.ProvideOrgRoleMapper, wire.Bind(new(user.Verifier), new(*userimpl.Verifier)), authz.WireSet, metadata.ProvideSecureValueMetadataStorage, metadata.ProvideKeeperMetadataStorage, metadata.ProvideDecryptStorage, decrypt.ProvideDecryptAuthorizer, decrypt.ProvideDecryptService, encryption.ProvideDataKeyStorage, encryption.ProvideEncryptedValueStorage, service12.ProvideSecureValueService, validator3.ProvideKeeperValidator, validator3.ProvideSecureValueValidator, migrator2.NewWithEngine, database5.ProvideDatabase, wire.Bind(new(contracts.Database), new(*database5.Database)), manager4.ProvideEncryptionManager, service11.ProvideAESGCMCipherService, resource.ProvideStorageMetrics, resource.ProvideIndexMetrics, apiserver.WireSet, apiregistry.WireSet, appregistry.WireSet)
+var wireBasicSet = wire.NewSet(annotationsimpl.ProvideService, wire.Bind(new(annotations.Repository), new(*annotationsimpl.RepositoryImpl)), New, api.ProvideHTTPServer, query.ProvideService, wire.Bind(new(query.Service), new(*query.ServiceImpl)), bus.ProvideBus, wire.Bind(new(bus.Bus), new(*bus.InProcBus)), rendering.ProvideService, wire.Bind(new(rendering.Service), new(*rendering.RenderingService)), routing.ProvideRegister, wire.Bind(new(routing.RouteRegister), new(*routing.RouteRegisterImpl)), hooks.ProvideService, kvstore.ProvideService, localcache.ProvideService, bundleregistry.ProvideService, wire.Bind(new(supportbundles.Service), new(*bundleregistry.Service)), updatemanager.ProvideGrafanaService, updatemanager.ProvidePluginsService, service.ProvideService, wire.Bind(new(usagestats.Service), new(*service.UsageStats)), validator2.ProvideService, legacy.ProvideLegacyMigrator, pluginsintegration.WireSet, dashboards.ProvideFileStoreManager, wire.Bind(new(dashboards.FileStore), new(*dashboards.FileStoreManager)), cloudwatch.ProvideService, cloudmonitoring.ProvideService, azuremonitor.ProvideService, postgres.ProvideService, mysql.ProvideService, mssql.ProvideService, store.ProvideEntityEventsService, dualwrite.ProvideService, httpclientprovider.New, wire.Bind(new(httpclient.Provider), new(*httpclient2.Provider)), serverlock.ProvideService, annotationsimpl.ProvideCleanupService, wire.Bind(new(annotations.Cleaner), new(*annotationsimpl.CleanupServiceImpl)), cleanup.ProvideService, shorturlimpl.ProvideService, wire.Bind(new(shorturls.Service), new(*shorturlimpl.ShortURLService)), queryhistory.ProvideService, wire.Bind(new(queryhistory.Service), new(*queryhistory.QueryHistoryService)), correlations.ProvideService, wire.Bind(new(correlations.Service), new(*correlations.CorrelationsService)), quotaimpl.ProvideService, remotecache.ProvideService, wire.Bind(new(remotecache.CacheStorage), new(*remotecache.RemoteCache)), authinfoimpl.ProvideService, wire.Bind(new(login.AuthInfoService), new(*authinfoimpl.Service)), authinfoimpl.ProvideStore, datasourceproxy.ProvideService, sort.ProvideService, search2.ProvideService, searchV2.ProvideService, searchV2.ProvideSearchHTTPService, store.ProvideService, store.ProvideSystemUsersService, live.ProvideService, pushhttp.ProvideService, contexthandler.ProvideService, service10.ProvideService, wire.Bind(new(service10.LDAP), new(*service10.LDAPImpl)), jwt.ProvideService, wire.Bind(new(jwt.JWTService), new(*jwt.AuthService)), store2.ProvideDBStore, image.ProvideDeleteExpiredService, ngalert.ProvideService, librarypanels.ProvideService, wire.Bind(new(librarypanels.Service), new(*librarypanels.LibraryPanelService)), libraryelements.ProvideService, wire.Bind(new(libraryelements.Service), new(*libraryelements.LibraryElementService)), notifications.ProvideService, notifications.ProvideSmtpService, github.ProvideFactory, tracing.ProvideService, tracing.ProvideTracingConfig, wire.Bind(new(tracing.Tracer), new(*tracing.TracingService)), withOTelSet, testdatasource.ProvideService, api4.ProvideService, opentsdb.ProvideService, socialimpl.ProvideService, influxdb.ProvideService, wire.Bind(new(social.Service), new(*socialimpl.SocialService)), tempo.ProvideService, loki.ProvideService, graphite.ProvideService, prometheus.ProvideService, elasticsearch.ProvideService, pyroscope.ProvideService, parca.ProvideService, zipkin.ProvideService, jaeger.ProvideService, service7.ProvideCacheService, wire.Bind(new(datasources.CacheService), new(*service7.CacheServiceImpl)), service2.ProvideEncryptionService, wire.Bind(new(encryption2.Internal), new(*service2.Service)), manager.ProvideSecretsService, wire.Bind(new(secrets2.Service), new(*manager.SecretsService)), database.ProvideSecretsStore, wire.Bind(new(secrets2.Store), new(*database.SecretsStoreImpl)), grafanads.ProvideService, wire.Bind(new(dashboardsnapshots.Store), new(*database4.DashboardSnapshotStore)), database4.ProvideStore, wire.Bind(new(dashboardsnapshots.Service), new(*service8.ServiceImpl)), service8.ProvideService, service7.ProvideService, wire.Bind(new(datasources.DataSourceService), new(*service7.Service)), service7.ProvideLegacyDataSourceLookup, retriever.ProvideService, wire.Bind(new(serviceaccounts.ServiceAccountRetriever), new(*retriever.Service)), ossaccesscontrol.ProvideServiceAccountPermissions, wire.Bind(new(accesscontrol.ServiceAccountPermissionsService), new(*ossaccesscontrol.ServiceAccountPermissionsService)), manager2.ProvideServiceAccountsService, proxy.ProvideServiceAccountsProxy, wire.Bind(new(serviceaccounts.Service), new(*proxy.ServiceAccountsProxy)), mtdsclient.NewNullMTDatasourceClientBuilder, expr.ProvideService, featuremgmt.ProvideManagerService, featuremgmt.ProvideToggles, service5.ProvideDashboardServiceImpl, wire.Bind(new(dashboards2.PermissionsRegistrationService), new(*service5.DashboardServiceImpl)), service5.ProvideDashboardService, service5.ProvideDashboardProvisioningService, service5.ProvideDashboardPluginService, database2.ProvideDashboardStore, folderimpl.ProvideService, wire.Bind(new(folder.Service), new(*folderimpl.Service)), folderimpl.ProvideStore, wire.Bind(new(folder.Store), new(*folderimpl.FolderStoreImpl)), folderimpl.ProvideDashboardFolderStore, wire.Bind(new(folder.FolderStore), new(*folderimpl.DashboardFolderStoreImpl)), service9.ProvideService, wire.Bind(new(dashboardimport.Service), new(*service9.ImportDashboardService)), service6.ProvideService, wire.Bind(new(plugindashboards.Service), new(*service6.Service)), service6.ProvideDashboardUpdater, sanitizer.ProvideService, kvstore2.ProvideService, avatar.ProvideAvatarCacheServer, statscollector.ProvideService, csrf.ProvideCSRFFilter, wire.Bind(new(csrf.Service), new(*csrf.CSRF)), ossaccesscontrol.ProvideTeamPermissions, wire.Bind(new(accesscontrol.TeamPermissionsService), new(*ossaccesscontrol.TeamPermissionsService)), ossaccesscontrol.ProvideFolderPermissions, wire.Bind(new(accesscontrol.FolderPermissionsService), new(*ossaccesscontrol.FolderPermissionsService)), ossaccesscontrol.ProvideDashboardPermissions, wire.Bind(new(accesscontrol.DashboardPermissionsService), new(*ossaccesscontrol.DashboardPermissionsService)), ossaccesscontrol.ProvideReceiverPermissionsService, wire.Bind(new(accesscontrol.ReceiverPermissionsService), new(*ossaccesscontrol.ReceiverPermissionsService)), starimpl.ProvideService, playlistimpl.ProvideService, apikeyimpl.ProvideService, dashverimpl.ProvideService, service3.ProvideService, wire.Bind(new(publicdashboards.Service), new(*service3.PublicDashboardServiceImpl)), database3.ProvideStore, wire.Bind(new(publicdashboards.Store), new(*database3.PublicDashboardStoreImpl)), metric.ProvideService, api2.ProvideApi, api3.ProvideApi, userimpl.ProvideService, orgimpl.ProvideService, orgimpl.ProvideDeletionService, statsimpl.ProvideService, grpccontext.ProvideContextHandler, grpcserver.ProvideHealthService, grpcserver.ProvideReflectionService, resolver.ProvideEntityReferenceResolver, teamimpl.ProvideService, teamapi.ProvideTeamAPI, tempuserimpl.ProvideService, loginattemptimpl.ProvideService, wire.Bind(new(loginattempt.Service), new(*loginattemptimpl.Service)), migrations2.ProvideDataSourceMigrationService, migrations2.ProvideSecretMigrationProvider, wire.Bind(new(migrations2.SecretMigrationProvider), new(*migrations2.SecretMigrationProviderImpl)), resourcepermissions.NewActionSetService, wire.Bind(new(accesscontrol.ActionResolver), new(resourcepermissions.ActionSetService)), wire.Bind(new(pluginaccesscontrol.ActionSetRegistry), new(resourcepermissions.ActionSetService)), permreg.ProvidePermissionRegistry, acimpl.ProvideAccessControl, dualwrite2.ProvideZanzanaReconciler, navtreeimpl.ProvideService, wire.Bind(new(accesscontrol.AccessControl), new(*acimpl.AccessControl)), wire.Bind(new(notifications.TempUserStore), new(tempuser.Service)), tagimpl.ProvideService, wire.Bind(new(tag.Service), new(*tagimpl.Service)), authnimpl.ProvideService, authnimpl.ProvideIdentitySynchronizer, authnimpl.ProvideAuthnService, authnimpl.ProvideAuthnServiceAuthenticateOnly, authnimpl.ProvideRegistration, supportbundlesimpl.ProvideService, extsvcaccounts.ProvideExtSvcAccountsService, wire.Bind(new(serviceaccounts.ExtSvcAccountsService), new(*extsvcaccounts.ExtSvcAccountsService)), registry2.ProvideExtSvcRegistry, wire.Bind(new(extsvcauth.ExternalServiceRegistry), new(*registry2.Registry)), anonstore.ProvideAnonDBStore, wire.Bind(new(anonstore.AnonStore), new(*anonstore.AnonDBStore)), loggermw.Provide, slogadapter.Provide, signingkeysimpl.ProvideEmbeddedSigningKeysService, wire.Bind(new(signingkeys.Service), new(*signingkeysimpl.Service)), ssosettingsimpl.ProvideService, wire.Bind(new(ssosettings.Service), new(*ssosettingsimpl.Service)), idimpl.ProvideService, wire.Bind(new(auth.IDService), new(*idimpl.Service)), cloudmigrationimpl.ProvideService, userimpl.ProvideVerifier, connectors.ProvideOrgRoleMapper, wire.Bind(new(user.Verifier), new(*userimpl.Verifier)), authz.WireSet, metadata.ProvideSecureValueMetadataStorage, metadata.ProvideKeeperMetadataStorage, metadata.ProvideDecryptStorage, decrypt.ProvideDecryptAuthorizer, decrypt.ProvideDecryptService, encryption.ProvideDataKeyStorage, encryption.ProvideGlobalDataKeyStorage, encryption.ProvideEncryptedValueStorage, encryption.ProvideGlobalEncryptedValueStorage, service12.ProvideSecureValueService, validator3.ProvideKeeperValidator, validator3.ProvideSecureValueValidator, migrator2.NewWithEngine, database5.ProvideDatabase, wire.Bind(new(contracts.Database), new(*database5.Database)), manager4.ProvideEncryptionManager, service11.ProvideAESGCMCipherService, resource.ProvideStorageMetrics, resource.ProvideIndexMetrics, apiserver.WireSet, apiregistry.WireSet, appregistry.WireSet)
var wireSet = wire.NewSet(
wireBasicSet, metrics.WireSet, sqlstore.ProvideService, metrics2.ProvideService, wire.Bind(new(notifications.Service), new(*notifications.NotificationService)), wire.Bind(new(notifications.WebhookSender), new(*notifications.NotificationService)), wire.Bind(new(notifications.EmailSender), new(*notifications.NotificationService)), wire.Bind(new(db.DB), new(*sqlstore.SQLStore)), prefimpl.ProvideService, oauthtoken.ProvideService, wire.Bind(new(oauthtoken.OAuthTokenService), new(*oauthtoken.Service)), wire.Bind(new(cleanup.AlertRuleService), new(*store2.DBstore)),
diff --git a/pkg/server/wireexts_oss.go b/pkg/server/wireexts_oss.go
index bb715ed7dea..61a05716b47 100644
--- a/pkg/server/wireexts_oss.go
+++ b/pkg/server/wireexts_oss.go
@@ -19,6 +19,7 @@ import (
"github.com/grafana/grafana/pkg/registry/apis/secret/contracts"
gsmKMSProviders "github.com/grafana/grafana/pkg/registry/apis/secret/encryption/kmsproviders"
"github.com/grafana/grafana/pkg/registry/apis/secret/secretkeeper"
+ secretService "github.com/grafana/grafana/pkg/registry/apis/secret/service"
"github.com/grafana/grafana/pkg/registry/backgroundsvcs"
"github.com/grafana/grafana/pkg/registry/usagestatssvcs"
"github.com/grafana/grafana/pkg/services/accesscontrol"
@@ -108,6 +109,7 @@ var wireExtsBasicSet = wire.NewSet(
wire.Bind(new(kmsproviders.Service), new(osskmsproviders.Service)),
secretkeeper.ProvideService,
wire.Bind(new(contracts.KeeperService), new(*secretkeeper.OSSKeeperService)),
+ secretService.ProvideConsolidationService,
ldap.ProvideGroupsService,
wire.Bind(new(ldap.Groups), new(*ldap.OSSGroups)),
guardian.ProvideGuardian,
diff --git a/pkg/services/anonymous/anonimpl/api/api.go b/pkg/services/anonymous/anonimpl/api/api.go
index 95774e1682b..b7e4e091ed2 100644
--- a/pkg/services/anonymous/anonimpl/api/api.go
+++ b/pkg/services/anonymous/anonimpl/api/api.go
@@ -55,7 +55,7 @@ func (api *AnonDeviceServiceAPI) RegisterAPIEndpoints() {
})
}
-// swagger:route GET /stats devices listDevices
+// swagger:route GET /anonymous/devices devices listDevices
//
// # Lists all devices within the last 30 days
//
@@ -91,7 +91,7 @@ func (api *AnonDeviceServiceAPI) ListDevices(c *contextmodel.ReqContext) respons
return response.JSON(http.StatusOK, resDevices)
}
-// swagger:route POST /search devices SearchDevices
+// swagger:route GET /anonymous/search devices SearchDevices
//
// # Lists all devices within the last 30 days
//
diff --git a/pkg/services/apiserver/appinstaller/installer.go b/pkg/services/apiserver/appinstaller/installer.go
index d80e865b5f0..9817f7513bc 100644
--- a/pkg/services/apiserver/appinstaller/installer.go
+++ b/pkg/services/apiserver/appinstaller/installer.go
@@ -8,17 +8,19 @@ import (
appsdkapiserver "github.com/grafana/grafana-app-sdk/k8s/apiserver"
"github.com/grafana/grafana-app-sdk/logging"
- grafanarest "github.com/grafana/grafana/pkg/apiserver/rest"
- "github.com/grafana/grafana/pkg/services/apiserver/builder"
- "github.com/grafana/grafana/pkg/services/apiserver/endpoints/request"
- grafanaapiserveroptions "github.com/grafana/grafana/pkg/services/apiserver/options"
"github.com/grafana/grafana/pkg/storage/legacysql/dualwrite"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apiserver/pkg/authorization/authorizer"
"k8s.io/apiserver/pkg/registry/generic"
genericapiserver "k8s.io/apiserver/pkg/server"
+ serverstore "k8s.io/apiserver/pkg/server/storage"
"k8s.io/kube-openapi/pkg/common"
+
+ grafanarest "github.com/grafana/grafana/pkg/apiserver/rest"
+ "github.com/grafana/grafana/pkg/services/apiserver/builder"
+ "github.com/grafana/grafana/pkg/services/apiserver/endpoints/request"
+ grafanaapiserveroptions "github.com/grafana/grafana/pkg/services/apiserver/options"
)
type LegacyStorageGetterFunc func(schema.GroupVersionResource) grafanarest.Storage
@@ -31,13 +33,6 @@ type AuthorizerProvider interface {
GetAuthorizer() authorizer.Authorizer
}
-type APIEnablementProvider interface {
- // Do not implement this unless you have special circumstances! This is a list of resources that are allowed to be accessed in v0alpha1,
- // to prevent accidental exposure of experimental APIs. While developing, use the feature flag `grafanaAPIServerWithExperimentalAPIs`.
- // And then, when you're ready to expose this to the end user, go to v1beta1 instead.
- GetAllowedV0Alpha1Resources() []string
-}
-
type AppInstallerConfig struct {
CustomConfig any
AllowedV0Alpha1Resources []string
@@ -132,9 +127,9 @@ func InstallAPIs(
dualWriteService dualwrite.Service,
dualWriterMetrics *grafanarest.DualWriterMetrics,
builderMetrics *builder.BuilderMetrics,
+ apiResourceConfig *serverstore.ResourceConfig,
) error {
logger := logging.FromContext(ctx)
-
for _, installer := range appInstallers {
logger.Debug("Installing APIs for app installer", "app", installer.ManifestData().AppName)
wrapper := &serverWrapper{
@@ -149,6 +144,7 @@ func InstallAPIs(
dualWriteService: dualWriteService,
dualWriterMetrics: dualWriterMetrics,
builderMetrics: builderMetrics,
+ apiResourceConfig: apiResourceConfig,
}
if err := installer.InstallAPIs(wrapper, restOpsGetter); err != nil {
return fmt.Errorf("failed to install APIs for app %s: %w", installer.ManifestData().AppName, err)
diff --git a/pkg/services/apiserver/appinstaller/resourceconfig.go b/pkg/services/apiserver/appinstaller/resourceconfig.go
new file mode 100644
index 00000000000..c2e45eb74e9
--- /dev/null
+++ b/pkg/services/apiserver/appinstaller/resourceconfig.go
@@ -0,0 +1,32 @@
+package appinstaller
+
+import (
+ appsdkapiserver "github.com/grafana/grafana-app-sdk/k8s/apiserver"
+ "k8s.io/apimachinery/pkg/runtime/schema"
+ serverstorage "k8s.io/apiserver/pkg/server/storage"
+)
+
+func NewAPIResourceConfig(installers []appsdkapiserver.AppInstaller) *serverstorage.ResourceConfig {
+ ret := serverstorage.NewResourceConfig()
+ enable := []schema.GroupVersion{}
+ disable := []schema.GroupVersion{}
+
+ for _, installer := range installers {
+ for _, version := range installer.ManifestData().Versions {
+ gv := schema.GroupVersion{
+ Group: installer.ManifestData().Group,
+ Version: version.Name,
+ }
+ if version.Served {
+ enable = append(enable, gv)
+ } else {
+ disable = append(disable, gv)
+ }
+ }
+ }
+
+ ret.EnableVersions(enable...)
+ ret.DisableVersions(disable...)
+
+ return ret
+}
diff --git a/pkg/services/apiserver/appinstaller/server.go b/pkg/services/apiserver/appinstaller/server.go
index 723c0753605..20792eeab0d 100644
--- a/pkg/services/apiserver/appinstaller/server.go
+++ b/pkg/services/apiserver/appinstaller/server.go
@@ -8,7 +8,9 @@ import (
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apiserver/pkg/registry/generic"
genericregistry "k8s.io/apiserver/pkg/registry/generic/registry"
+ genericrest "k8s.io/apiserver/pkg/registry/rest"
genericapiserver "k8s.io/apiserver/pkg/server"
+ serverstorage "k8s.io/apiserver/pkg/server/storage"
appsdkapiserver "github.com/grafana/grafana-app-sdk/k8s/apiserver"
"github.com/grafana/grafana-app-sdk/logging"
@@ -34,21 +36,14 @@ type serverWrapper struct {
dualWriteService dualwrite.Service
dualWriterMetrics *grafanarest.DualWriterMetrics
builderMetrics *builder.BuilderMetrics
+ apiResourceConfig *serverstorage.ResourceConfig
}
func (s *serverWrapper) InstallAPIGroup(apiGroupInfo *genericapiserver.APIGroupInfo) error {
log := logging.FromContext(s.ctx)
- legacyProvider, ok := s.installer.(LegacyStorageProvider)
- if !ok {
- return s.GenericAPIServer.InstallAPIGroup(apiGroupInfo)
- }
for v, storageMap := range apiGroupInfo.VersionedResourcesStorageMap {
for storagePath, restStorage := range storageMap {
- genericStorage, ok := restStorage.(*genericregistry.Store)
- if !ok {
- log.Error("Expected generic registry store", "storagePath", storagePath, "version", v)
- continue
- }
+ legacyProvider, dualWriteSupported := s.installer.(LegacyStorageProvider)
resource, err := getResourceFromStoragePath(storagePath)
if err != nil {
return err
@@ -57,29 +52,34 @@ func (s *serverWrapper) InstallAPIGroup(apiGroupInfo *genericapiserver.APIGroupI
Group: s.installer.ManifestData().Group,
Resource: resource,
}
- genericStorage.KeyRootFunc = grafanaregistry.KeyRootFunc(gr)
- genericStorage.KeyFunc = grafanaregistry.NamespaceKeyFunc(gr)
- genericStorage.UpdateStrategy = &updateStrategyWrapper{
- RESTUpdateStrategy: genericStorage.UpdateStrategy,
+ gvr := gr.WithVersion(v)
+ if s.apiResourceConfig != nil && !s.apiResourceConfig.ResourceEnabled(gvr) {
+ log.Debug("Skipping storage for disabled resource", "gvr", gvr.String(), "storagePath", storagePath)
+ delete(apiGroupInfo.VersionedResourcesStorageMap[v], storagePath)
+ continue
}
-
- dw, err := NewDualWriter(
- s.ctx,
- gr,
- s.storageOpts,
- legacyProvider.GetLegacyStorage(gr.WithVersion(v)),
- grafanarest.Storage(genericStorage),
- s.kvStore,
- s.lock,
- s.namespaceMapper,
- s.dualWriteService,
- s.dualWriterMetrics,
- s.builderMetrics,
- )
- if err != nil {
- return err
+ storage := s.configureStorage(gr, dualWriteSupported, restStorage)
+ if unifiedStorage, ok := storage.(grafanarest.Storage); ok && dualWriteSupported {
+ log.Debug("Configuring dual writer for storage", "resource", gr.String(), "version", v, "storagePath", storagePath)
+ dw, err := NewDualWriter(
+ s.ctx,
+ gr,
+ s.storageOpts,
+ legacyProvider.GetLegacyStorage(gr.WithVersion(v)),
+ unifiedStorage,
+ s.kvStore,
+ s.lock,
+ s.namespaceMapper,
+ s.dualWriteService,
+ s.dualWriterMetrics,
+ s.builderMetrics,
+ )
+ if err != nil {
+ return err
+ }
+ storage = dw
}
- apiGroupInfo.VersionedResourcesStorageMap[v][storagePath] = dw
+ apiGroupInfo.VersionedResourcesStorageMap[v][storagePath] = storage
}
}
@@ -93,3 +93,27 @@ func getResourceFromStoragePath(storagePath string) (string, error) {
}
return parts[0], nil
}
+
+func (s *serverWrapper) configureStorage(gr schema.GroupResource, dualWriteSupported bool, storage genericrest.Storage) genericrest.Storage {
+ if gs, ok := storage.(*genericregistry.Store); ok {
+ // if dual write is supported, we need to modify the update strategy
+ // this is not needed for the status store
+ if dualWriteSupported {
+ gs.UpdateStrategy = &updateStrategyWrapper{
+ RESTUpdateStrategy: gs.UpdateStrategy,
+ }
+ }
+ gs.KeyFunc = grafanaregistry.NamespaceKeyFunc(gr)
+ gs.KeyRootFunc = grafanaregistry.KeyRootFunc(gr)
+ return gs
+ }
+
+ // if the storage is a status store, we need to extract the underlying generic registry store
+ if statusStore, ok := storage.(*appsdkapiserver.StatusREST); ok {
+ statusStore.Store.KeyFunc = grafanaregistry.NamespaceKeyFunc(gr)
+ statusStore.Store.KeyRootFunc = grafanaregistry.KeyRootFunc(gr)
+ return statusStore
+ }
+
+ return storage
+}
diff --git a/pkg/services/apiserver/builder/openapi.go b/pkg/services/apiserver/builder/openapi.go
index 9489ac38b95..cde3548a136 100644
--- a/pkg/services/apiserver/builder/openapi.go
+++ b/pkg/services/apiserver/builder/openapi.go
@@ -1,21 +1,42 @@
package builder
import (
+ "bytes"
+ "encoding/json"
"maps"
"strings"
+ "sync"
+ apiequality "k8s.io/apimachinery/pkg/api/equality"
"k8s.io/apimachinery/pkg/runtime/schema"
openapi "k8s.io/kube-openapi/pkg/common"
"k8s.io/kube-openapi/pkg/spec3"
spec "k8s.io/kube-openapi/pkg/validation/spec"
+ "github.com/grafana/grafana-app-sdk/logging"
data "github.com/grafana/grafana-plugin-sdk-go/experimental/apis/data/v0alpha1"
secret "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
common "github.com/grafana/grafana/pkg/apimachinery/apis/common/v0alpha1"
)
+var (
+ equalityInit sync.Once
+)
+
// This should eventually live in grafana-app-sdk
func GetOpenAPIDefinitions(builders []APIGroupBuilder, additionalGetters ...openapi.GetOpenAPIDefinitions) openapi.GetOpenAPIDefinitions {
+ equalityInit.Do(func() {
+ // DataQuery has private variables, so it needs an explicit equality helper
+ err := apiequality.Semantic.AddFunc(
+ func(a, b data.DataQuery) bool {
+ aa, _ := json.Marshal(a)
+ bb, _ := json.Marshal(b)
+ return bytes.Equal(aa, bb)
+ },
+ )
+ logging.DefaultLogger.Error("error initializing DataQuery apiequality", "err", err)
+ })
+
return func(ref openapi.ReferenceCallback) map[string]openapi.OpenAPIDefinition {
defs := common.GetOpenAPIDefinitions(ref) // common grafana apis
maps.Copy(defs, data.GetOpenAPIDefinitions(ref))
diff --git a/pkg/services/apiserver/config.go b/pkg/services/apiserver/config.go
index 8cf5789c777..dc2234e6e56 100644
--- a/pkg/services/apiserver/config.go
+++ b/pkg/services/apiserver/config.go
@@ -39,6 +39,13 @@ func applyGrafanaConfig(cfg *setting.Cfg, features featuremgmt.FeatureToggles, o
apiserverCfg := cfg.SectionWithEnvOverrides("grafana-apiserver")
+ runtimeConfig := apiserverCfg.Key("runtime_config").String()
+ if runtimeConfig != "" {
+ if err := o.APIEnablementOptions.RuntimeConfig.Set(runtimeConfig); err != nil {
+ return fmt.Errorf("failed to set runtime config: %w", err)
+ }
+ }
+
o.RecommendedOptions.Etcd.StorageConfig.Transport.ServerList = apiserverCfg.Key("etcd_servers").Strings(",")
o.RecommendedOptions.SecureServing.BindAddress = ip
diff --git a/pkg/services/apiserver/options/options.go b/pkg/services/apiserver/options/options.go
index a9209cdbc00..721f1ce570b 100644
--- a/pkg/services/apiserver/options/options.go
+++ b/pkg/services/apiserver/options/options.go
@@ -21,6 +21,7 @@ const defaultEtcdPathPrefix = "/registry/grafana.app"
type Options struct {
RecommendedOptions *genericoptions.RecommendedOptions
+ APIEnablementOptions *genericoptions.APIEnablementOptions
GrafanaAggregatorOptions *GrafanaAggregatorOptions
StorageOptions *StorageOptions
ExtraOptions *ExtraOptions
@@ -30,6 +31,7 @@ type Options struct {
func NewOptions(codec runtime.Codec) *Options {
return &Options{
RecommendedOptions: NewRecommendedOptions(codec),
+ APIEnablementOptions: genericoptions.NewAPIEnablementOptions(),
GrafanaAggregatorOptions: NewGrafanaAggregatorOptions(),
StorageOptions: NewStorageOptions(),
ExtraOptions: NewExtraOptions(),
@@ -38,6 +40,7 @@ func NewOptions(codec runtime.Codec) *Options {
func (o *Options) AddFlags(fs *pflag.FlagSet) {
o.RecommendedOptions.AddFlags(fs)
+ o.APIEnablementOptions.AddFlags(fs)
o.GrafanaAggregatorOptions.AddFlags(fs)
o.StorageOptions.AddFlags(fs)
o.ExtraOptions.AddFlags(fs)
diff --git a/pkg/services/apiserver/service.go b/pkg/services/apiserver/service.go
index 58bf5ad4ac7..be580025823 100644
--- a/pkg/services/apiserver/service.go
+++ b/pkg/services/apiserver/service.go
@@ -304,11 +304,19 @@ func (s *service) start(ctx context.Context) error {
return errs[0]
}
+ if errs := o.APIEnablementOptions.Validate(s.scheme); len(errs) != 0 {
+ return errs[0]
+ }
+
serverConfig := genericapiserver.NewRecommendedConfig(s.codecs)
if err := o.ApplyTo(serverConfig); err != nil {
return err
}
+ if err := o.APIEnablementOptions.ApplyTo(&serverConfig.Config, appinstaller.NewAPIResourceConfig(s.appInstallers), s.scheme); err != nil {
+ return err
+ }
+
serverConfig.Authorization.Authorizer = s.authorizer
serverConfig.Authentication.Authenticator = authenticator.NewAuthenticator(serverConfig.Authentication.Authenticator)
serverConfig.TracerProvider = s.tracing.GetTracerProvider()
@@ -395,6 +403,7 @@ func (s *service) start(ctx context.Context) error {
s.storageStatus,
s.dualWriterMetrics,
s.builderMetrics,
+ serverConfig.MergedResourceConfig,
); err != nil {
return err
}
diff --git a/pkg/services/dashboards/service/dashboard_service.go b/pkg/services/dashboards/service/dashboard_service.go
index 043438db828..1eb611fa670 100644
--- a/pkg/services/dashboards/service/dashboard_service.go
+++ b/pkg/services/dashboards/service/dashboard_service.go
@@ -474,40 +474,36 @@ func (dr *DashboardServiceImpl) Count(ctx context.Context, scopeParams *quota.Sc
}
func (dr *DashboardServiceImpl) GetDashboardsByLibraryPanelUID(ctx context.Context, libraryPanelUID string, orgID int64) ([]*dashboards.DashboardRef, error) {
- if dr.features.IsEnabledGlobally(featuremgmt.FlagKubernetesLibraryPanelConnections) {
- res, err := dr.k8sclient.Search(ctx, orgID, &resourcepb.ResourceSearchRequest{
- Options: &resourcepb.ListOptions{
- Fields: []*resourcepb.Requirement{
- {
- Key: search.DASHBOARD_LIBRARY_PANEL_REFERENCE,
- Operator: string(selection.Equals),
- Values: []string{libraryPanelUID},
- },
+ res, err := dr.k8sclient.Search(ctx, orgID, &resourcepb.ResourceSearchRequest{
+ Options: &resourcepb.ListOptions{
+ Fields: []*resourcepb.Requirement{
+ {
+ Key: search.DASHBOARD_LIBRARY_PANEL_REFERENCE,
+ Operator: string(selection.Equals),
+ Values: []string{libraryPanelUID},
},
},
- Limit: listAllDashboardsLimit,
- })
- if err != nil {
- return nil, err
- }
-
- results, err := dashboardsearch.ParseResults(res, 0)
- if err != nil {
- return nil, err
- }
-
- dashes := make([]*dashboards.DashboardRef, 0, len(results.Hits))
- for _, row := range results.Hits {
- dashes = append(dashes, &dashboards.DashboardRef{
- UID: row.Name,
- FolderUID: row.Folder,
- ID: row.Field.GetNestedInt64(resource.SEARCH_FIELD_LEGACY_ID), // nolint:staticcheck
- })
- }
- return dashes, nil
+ },
+ Limit: listAllDashboardsLimit,
+ })
+ if err != nil {
+ return nil, err
}
- return dr.dashboardStore.GetDashboardsByLibraryPanelUID(ctx, libraryPanelUID, orgID)
+ results, err := dashboardsearch.ParseResults(res, 0)
+ if err != nil {
+ return nil, err
+ }
+
+ dashes := make([]*dashboards.DashboardRef, 0, len(results.Hits))
+ for _, row := range results.Hits {
+ dashes = append(dashes, &dashboards.DashboardRef{
+ UID: row.Name,
+ FolderUID: row.Folder,
+ ID: row.Field.GetNestedInt64(resource.SEARCH_FIELD_LEGACY_ID), // nolint:staticcheck
+ })
+ }
+ return dashes, nil
}
func (dr *DashboardServiceImpl) CountDashboardsInOrg(ctx context.Context, orgID int64) (int64, error) {
diff --git a/pkg/services/dashboards/service/dashboard_service_test.go b/pkg/services/dashboards/service/dashboard_service_test.go
index 95e5a0a9023..1cda81c5288 100644
--- a/pkg/services/dashboards/service/dashboard_service_test.go
+++ b/pkg/services/dashboards/service/dashboard_service_test.go
@@ -2493,7 +2493,7 @@ func TestGetDashboardsByLibraryPanelUID(t *testing.T) {
dashboardStore: &fakeStore,
folderService: folderSvc,
ac: actest.FakeAccessControl{ExpectedEvaluate: true},
- features: featuremgmt.WithFeatures(featuremgmt.FlagKubernetesLibraryPanelConnections),
+ features: featuremgmt.WithFeatures(),
publicDashboardService: fakePublicDashboardService,
k8sclient: k8sCliMock,
}
diff --git a/pkg/services/dashboards/store_mock.go b/pkg/services/dashboards/store_mock.go
index 9185887aec9..74259652fbf 100644
--- a/pkg/services/dashboards/store_mock.go
+++ b/pkg/services/dashboards/store_mock.go
@@ -90,7 +90,6 @@ func (_m *FakeDashboardStore) CountInOrg(ctx context.Context, orgID int64, isFol
return r0, r1
}
-
// DeleteDashboard provides a mock function with given fields: ctx, cmd
func (_m *FakeDashboardStore) DeleteDashboard(ctx context.Context, cmd *DeleteDashboardCommand) error {
ret := _m.Called(ctx, cmd)
@@ -127,7 +126,6 @@ func (_m *FakeDashboardStore) DeleteDashboardsInFolders(ctx context.Context, req
return r0
}
-
// FindDashboards provides a mock function with given fields: ctx, query
func (_m *FakeDashboardStore) FindDashboards(ctx context.Context, query *FindPersistedDashboardsQuery) ([]DashboardSearchProjection, error) {
ret := _m.Called(ctx, query)
diff --git a/pkg/services/featuremgmt/registry.go b/pkg/services/featuremgmt/registry.go
index 0ac3c4d9ce2..7b74ad685e8 100644
--- a/pkg/services/featuremgmt/registry.go
+++ b/pkg/services/featuremgmt/registry.go
@@ -469,13 +469,6 @@ var (
Owner: grafanaAppPlatformSquad,
RequiresRestart: true, // changes the API routing
},
- {
- Name: "kubernetesLibraryPanelConnections",
- Description: "Routes library panel connections requests from /api to using search",
- Stage: FeatureStageExperimental,
- Owner: grafanaAppPlatformSquad,
- RequiresRestart: true, // changes the API routing
- },
{
Name: "kubernetesDashboards",
Description: "Use the kubernetes API in the frontend for dashboards",
@@ -501,6 +494,12 @@ var (
Stage: FeatureStageExperimental,
Owner: grafanaAppPlatformSquad,
},
+ {
+ Name: "scanRowInvalidDashboardParseFallbackEnabled",
+ Description: "Enable fallback parsing behavior when scan row encounters invalid dashboard JSON",
+ Stage: FeatureStageExperimental,
+ Owner: grafanaSearchAndStorageSquad,
+ },
{
Name: "datasourceQueryTypes",
Description: "Show query type endpoints in datasource API servers (currently hardcoded for testdata, expressions, and prometheus)",
@@ -770,6 +769,16 @@ var (
HideFromAdminPage: true,
Expression: "false",
},
+ {
+ Name: "useScopeSingleNodeEndpoint",
+ Description: "Use the single node endpoint for the scope api. This is used to fetch the scope parent node.",
+ Stage: FeatureStageExperimental,
+ Owner: grafanaOperatorExperienceSquad,
+ Expression: "false",
+ FrontendOnly: true,
+ HideFromDocs: true,
+ HideFromAdminPage: true,
+ },
{
Name: "promQLScope",
Description: "In-development feature that will allow injection of labels into prometheus queries.",
@@ -1858,6 +1867,13 @@ var (
Owner: grafanaDataProSquad,
FrontendOnly: true,
},
+ {
+ Name: "dashboardLevelTimeMacros",
+ Description: "Supports __from and __to macros that always use the dashboard level time range",
+ Stage: FeatureStageExperimental,
+ Owner: grafanaDashboardsSquad,
+ FrontendOnly: true,
+ },
{
Name: "alertmanagerRemoteSecondaryWithRemoteState",
Description: "Starts Grafana in remote secondary mode pulling the latest state from the remote Alertmanager to avoid duplicate notifications.",
@@ -1874,6 +1890,13 @@ var (
Owner: grafanaDataProSquad,
FrontendOnly: true,
},
+ {
+ Name: "newLogContext",
+ Description: "New Log Context component",
+ Stage: FeatureStageExperimental,
+ Owner: grafanaObservabilityLogsSquad,
+ FrontendOnly: true,
+ },
}
)
diff --git a/pkg/services/featuremgmt/toggles_gen.csv b/pkg/services/featuremgmt/toggles_gen.csv
index 7b5c27e5123..6e5c55cfd35 100644
--- a/pkg/services/featuremgmt/toggles_gen.csv
+++ b/pkg/services/featuremgmt/toggles_gen.csv
@@ -60,11 +60,11 @@ disableClassicHTTPHistogram,experimental,@grafana/grafana-backend-services-squad
formatString,GA,@grafana/dataviz-squad,false,false,true
kubernetesSnapshots,experimental,@grafana/grafana-app-platform-squad,false,true,false
kubernetesLibraryPanels,experimental,@grafana/grafana-app-platform-squad,false,true,false
-kubernetesLibraryPanelConnections,experimental,@grafana/grafana-app-platform-squad,false,true,false
kubernetesDashboards,experimental,@grafana/grafana-app-platform-squad,false,false,true
dashboardDisableSchemaValidationV1,experimental,@grafana/grafana-app-platform-squad,false,false,false
dashboardDisableSchemaValidationV2,experimental,@grafana/grafana-app-platform-squad,false,false,false
dashboardSchemaValidationLogging,experimental,@grafana/grafana-app-platform-squad,false,false,false
+scanRowInvalidDashboardParseFallbackEnabled,experimental,@grafana/search-and-storage,false,false,false
datasourceQueryTypes,experimental,@grafana/grafana-app-platform-squad,false,true,false
queryService,experimental,@grafana/grafana-datasources-core-services,false,true,false
queryServiceRewrite,experimental,@grafana/grafana-datasources-core-services,false,true,false
@@ -101,6 +101,7 @@ secretsManagementAppPlatform,experimental,@grafana/grafana-operator-experience-s
alertingSaveStatePeriodic,privatePreview,@grafana/alerting-squad,false,false,false
alertingSaveStateCompressed,preview,@grafana/alerting-squad,false,false,false
scopeApi,experimental,@grafana/grafana-app-platform-squad,false,false,false
+useScopeSingleNodeEndpoint,experimental,@grafana/grafana-operator-experience-squad,false,false,true
promQLScope,GA,@grafana/oss-big-tent,false,false,false
logQLScope,privatePreview,@grafana/observability-logs,false,false,false
sqlExpressions,privatePreview,@grafana/grafana-datasources-core-services,false,false,false
@@ -240,5 +241,7 @@ alertingNotificationHistory,experimental,@grafana/alerting-squad,false,false,fal
pluginAssetProvider,experimental,@grafana/plugins-platform-backend,false,true,false
unifiedStorageSearchDualReaderEnabled,experimental,@grafana/search-and-storage,false,false,false
dashboardDsAdHocFiltering,experimental,@grafana/datapro,false,false,true
+dashboardLevelTimeMacros,experimental,@grafana/dashboards-squad,false,false,true
alertmanagerRemoteSecondaryWithRemoteState,experimental,@grafana/alerting-squad,false,false,false
adhocFiltersInTooltips,experimental,@grafana/datapro,false,false,true
+newLogContext,experimental,@grafana/observability-logs,false,false,true
diff --git a/pkg/services/featuremgmt/toggles_gen.go b/pkg/services/featuremgmt/toggles_gen.go
index b1cc2fb7f13..0b00cf45ac1 100644
--- a/pkg/services/featuremgmt/toggles_gen.go
+++ b/pkg/services/featuremgmt/toggles_gen.go
@@ -251,10 +251,6 @@ const (
// Routes library panel requests from /api to the /apis endpoint
FlagKubernetesLibraryPanels = "kubernetesLibraryPanels"
- // FlagKubernetesLibraryPanelConnections
- // Routes library panel connections requests from /api to using search
- FlagKubernetesLibraryPanelConnections = "kubernetesLibraryPanelConnections"
-
// FlagKubernetesDashboards
// Use the kubernetes API in the frontend for dashboards
FlagKubernetesDashboards = "kubernetesDashboards"
@@ -271,6 +267,10 @@ const (
// Log schema validation errors so they can be analyzed later
FlagDashboardSchemaValidationLogging = "dashboardSchemaValidationLogging"
+ // FlagScanRowInvalidDashboardParseFallbackEnabled
+ // Enable fallback parsing behavior when scan row encounters invalid dashboard JSON
+ FlagScanRowInvalidDashboardParseFallbackEnabled = "scanRowInvalidDashboardParseFallbackEnabled"
+
// FlagDatasourceQueryTypes
// Show query type endpoints in datasource API servers (currently hardcoded for testdata, expressions, and prometheus)
FlagDatasourceQueryTypes = "datasourceQueryTypes"
@@ -415,6 +415,10 @@ const (
// In-development feature flag for the scope api using the app platform.
FlagScopeApi = "scopeApi"
+ // FlagUseScopeSingleNodeEndpoint
+ // Use the single node endpoint for the scope api. This is used to fetch the scope parent node.
+ FlagUseScopeSingleNodeEndpoint = "useScopeSingleNodeEndpoint"
+
// FlagPromQLScope
// In-development feature that will allow injection of labels into prometheus queries.
FlagPromQLScope = "promQLScope"
@@ -971,6 +975,10 @@ const (
// Enables adhoc filtering support for the dashboard datasource
FlagDashboardDsAdHocFiltering = "dashboardDsAdHocFiltering"
+ // FlagDashboardLevelTimeMacros
+ // Supports __from and __to macros that always use the dashboard level time range
+ FlagDashboardLevelTimeMacros = "dashboardLevelTimeMacros"
+
// FlagAlertmanagerRemoteSecondaryWithRemoteState
// Starts Grafana in remote secondary mode pulling the latest state from the remote Alertmanager to avoid duplicate notifications.
FlagAlertmanagerRemoteSecondaryWithRemoteState = "alertmanagerRemoteSecondaryWithRemoteState"
@@ -978,4 +986,8 @@ const (
// FlagAdhocFiltersInTooltips
// Enable adhoc filter buttons in visualization tooltips
FlagAdhocFiltersInTooltips = "adhocFiltersInTooltips"
+
+ // FlagNewLogContext
+ // New Log Context component
+ FlagNewLogContext = "newLogContext"
)
diff --git a/pkg/services/featuremgmt/toggles_gen.json b/pkg/services/featuremgmt/toggles_gen.json
index 048f5512581..e47603d79d4 100644
--- a/pkg/services/featuremgmt/toggles_gen.json
+++ b/pkg/services/featuremgmt/toggles_gen.json
@@ -836,6 +836,19 @@
"frontend": true
}
},
+ {
+ "metadata": {
+ "name": "dashboardLevelTimeMacros",
+ "resourceVersion": "1753435849295",
+ "creationTimestamp": "2025-07-25T09:30:49Z"
+ },
+ "spec": {
+ "description": "Supports __from and __to macros that always use the dashboard level time range",
+ "stage": "experimental",
+ "codeowner": "@grafana/dashboards-squad",
+ "frontend": true
+ }
+ },
{
"metadata": {
"name": "dashboardNewLayouts",
@@ -1756,19 +1769,6 @@
"hideFromAdminPage": true
}
},
- {
- "metadata": {
- "name": "kubernetesLibraryPanelConnections",
- "resourceVersion": "1753448760331",
- "creationTimestamp": "2025-07-25T13:06:00Z"
- },
- "spec": {
- "description": "Routes library panel connections requests from /api to using search",
- "stage": "experimental",
- "codeowner": "@grafana/grafana-app-platform-squad",
- "requiresRestart": true
- }
- },
{
"metadata": {
"name": "kubernetesLibraryPanels",
@@ -2150,6 +2150,19 @@
"expression": "false"
}
},
+ {
+ "metadata": {
+ "name": "newLogContext",
+ "resourceVersion": "1754044501326",
+ "creationTimestamp": "2025-08-01T10:35:01Z"
+ },
+ "spec": {
+ "description": "New Log Context component",
+ "stage": "experimental",
+ "codeowner": "@grafana/observability-logs",
+ "frontend": true
+ }
+ },
{
"metadata": {
"name": "newLogsPanel",
@@ -2760,6 +2773,18 @@
"codeowner": "@grafana/identity-access-team"
}
},
+ {
+ "metadata": {
+ "name": "scanRowInvalidDashboardParseFallbackEnabled",
+ "resourceVersion": "1753730899886",
+ "creationTimestamp": "2025-07-28T19:28:19Z"
+ },
+ "spec": {
+ "description": "Enable fallback parsing behavior when scan row encounters invalid dashboard JSON",
+ "stage": "experimental",
+ "codeowner": "@grafana/search-and-storage"
+ }
+ },
{
"metadata": {
"name": "scopeApi",
@@ -3230,6 +3255,22 @@
"hideFromDocs": true
}
},
+ {
+ "metadata": {
+ "name": "useScopeSingleNodeEndpoint",
+ "resourceVersion": "1753960766702",
+ "creationTimestamp": "2025-07-31T11:19:26Z"
+ },
+ "spec": {
+ "description": "Use the single node endpoint for the scope api. This is used to fetch the scope parent node.",
+ "stage": "experimental",
+ "codeowner": "@grafana/grafana-operator-experience-squad",
+ "frontend": true,
+ "hideFromAdminPage": true,
+ "hideFromDocs": true,
+ "expression": "false"
+ }
+ },
{
"metadata": {
"name": "useScopesNavigationEndpoint",
diff --git a/pkg/services/frontend/index.html b/pkg/services/frontend/index.html
index 2ece6114fa5..f582df4604e 100644
--- a/pkg/services/frontend/index.html
+++ b/pkg/services/frontend/index.html
@@ -86,7 +86,7 @@
.fs-hidden {
display: none;
}
-
+
.fs-spinner {
animation: spin 1500ms linear infinite;
width: 32px;
@@ -100,7 +100,7 @@
.fs-spinner-arc {
stroke: #F55F3E;
}
-
+
.fs-loader-text {
opacity: 0;
font-size: 16px;
@@ -199,7 +199,7 @@
async function fetchBootData() {
const resp = await fetch("/bootdata");
const textResponse = await resp.text();
-
+
let rawBootData;
try {
rawBootData = JSON.parse(textResponse);
@@ -211,7 +211,7 @@
if (resp.status === 503 && rawBootData.code === 'Loading') {
return;
}
-
+
if (!resp.ok) {
throw new Error("Unexpected response body: " + textResponse);
}
@@ -259,21 +259,20 @@
const cssLink = document.createElement("link");
cssLink.rel = 'stylesheet';
- let theme = window.grafanaBootData.user.theme;
+ const theme = window.grafanaBootData.user.theme;
if (theme === "system") {
const darkQuery = window.matchMedia("(prefers-color-scheme: dark)");
- theme = darkQuery.matches ? 'dark' : 'light';
- }
- if (theme === "light") {
- document.body.classList.add("theme-light");
- cssLink.href = window.grafanaBootData.assets.light;
- window.grafanaBootData.user.lightTheme = true;
- } else if (theme === "dark") {
- document.body.classList.add("theme-dark");
- cssLink.href = window.grafanaBootData.assets.dark;
- window.grafanaBootData.user.lightTheme = false;
+ if (darkQuery.matches) {
+ document.body.classList.add("theme-dark");
+ window.grafanaBootData.user.lightTheme = false;
+ } else {
+ document.body.classList.add("theme-light");
+ window.grafanaBootData.user.lightTheme = true;
+ }
}
+ const isLightTheme = window.grafanaBootData.user.lightTheme;
+ cssLink.href = window.grafanaBootData.assets[isLightTheme ? 'light' : 'dark'];
document.head.appendChild(cssLink);
}
diff --git a/pkg/services/navtree/models.go b/pkg/services/navtree/models.go
index 5514705ed9a..250b3bf8717 100644
--- a/pkg/services/navtree/models.go
+++ b/pkg/services/navtree/models.go
@@ -17,6 +17,7 @@ const (
WeightDashboard
WeightExplore
WeightDrilldown
+ WeightAssistant
WeightAlerting
WeightAlertsAndIncidents
WeightAIAndML
diff --git a/pkg/services/navtree/navtreeimpl/applinks.go b/pkg/services/navtree/navtreeimpl/applinks.go
index 42770b6ee96..b4571c88a69 100644
--- a/pkg/services/navtree/navtreeimpl/applinks.go
+++ b/pkg/services/navtree/navtreeimpl/applinks.go
@@ -324,7 +324,8 @@ func (s *ServiceImpl) readNavigationSettings() {
"grafana-irm-app": {SectionID: navtree.NavIDAlertsAndIncidents, SortWeight: 3, Text: "IRM"},
"grafana-oncall-app": {SectionID: navtree.NavIDAlertsAndIncidents, SortWeight: 4, Text: "OnCall"},
"grafana-incident-app": {SectionID: navtree.NavIDAlertsAndIncidents, SortWeight: 5, Text: "Incident"},
- "grafana-ml-app": {SectionID: navtree.NavIDRoot, SortWeight: navtree.WeightAIAndML, Text: "AI & machine learning", SubTitle: "Explore AI and machine learning features", Icon: "gf-ml-alt"},
+ "grafana-assistant-app": {SectionID: navtree.NavIDRoot, SortWeight: navtree.WeightAssistant, Text: "Assistant", SubTitle: "AI-powered assistant for Grafana", Icon: "ai-sparkle", IsNew: true},
+ "grafana-ml-app": {SectionID: navtree.NavIDRoot, SortWeight: navtree.WeightAIAndML, Text: "Machine Learning", SubTitle: "Explore AI and machine learning features", Icon: "gf-ml-alt"},
"grafana-slo-app": {SectionID: navtree.NavIDAlertsAndIncidents, SortWeight: 7},
"grafana-cloud-link-app": {SectionID: navtree.NavIDCfgPlugins, SortWeight: 3},
"grafana-costmanagementui-app": {SectionID: navtree.NavIDCfg, Text: "Cost management"},
diff --git a/pkg/services/ngalert/api/tooling/api.json b/pkg/services/ngalert/api/tooling/api.json
index bf2a859fd61..7eae1a55974 100644
--- a/pkg/services/ngalert/api/tooling/api.json
+++ b/pkg/services/ngalert/api/tooling/api.json
@@ -5523,6 +5523,586 @@
"version": "1.1.0"
},
"paths": {
+ "/convert/api/prom/rules": {
+ "get": {
+ "operationId": "RouteConvertPrometheusCortexGetRules",
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "post": {
+ "consumes": [
+ "application/json",
+ "application/yaml"
+ ],
+ "operationId": "RouteConvertPrometheusCortexPostRuleGroups",
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ }
+ },
+ "/convert/api/prom/rules/{NamespaceTitle}": {
+ "delete": {
+ "operationId": "RouteConvertPrometheusCortexDeleteNamespace",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "get": {
+ "operationId": "RouteConvertPrometheusCortexGetNamespace",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "post": {
+ "consumes": [
+ "application/yaml"
+ ],
+ "description": "If the group already exists and was not imported from a Prometheus-compatible source initially,\nit will not be replaced and an error will be returned.",
+ "operationId": "RouteConvertPrometheusCortexPostRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-datasource-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-recording-rules-paused",
+ "type": "boolean"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-alert-rules-paused",
+ "type": "boolean"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-target-datasource-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-folder-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-notification-receiver",
+ "type": "string"
+ },
+ {
+ "in": "body",
+ "name": "Body",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
+ "tags": [
+ "convert_prometheus"
+ ],
+ "x-raw-request": "true"
+ }
+ },
+ "/convert/api/prom/rules/{NamespaceTitle}/{Group}": {
+ "delete": {
+ "operationId": "RouteConvertPrometheusCortexDeleteRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "path",
+ "name": "Group",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "get": {
+ "operationId": "RouteConvertPrometheusCortexGetRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "path",
+ "name": "Group",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusRuleGroup",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ }
+ },
+ "/convert/prometheus/config/v1/rules": {
+ "get": {
+ "operationId": "RouteConvertPrometheusGetRules",
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "post": {
+ "consumes": [
+ "application/json",
+ "application/yaml"
+ ],
+ "operationId": "RouteConvertPrometheusPostRuleGroups",
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ }
+ },
+ "/convert/prometheus/config/v1/rules/{NamespaceTitle}": {
+ "delete": {
+ "operationId": "RouteConvertPrometheusDeleteNamespace",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "get": {
+ "operationId": "RouteConvertPrometheusGetNamespace",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusNamespace",
+ "schema": {
+ "$ref": "#/definitions/PrometheusNamespace"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "post": {
+ "consumes": [
+ "application/yaml"
+ ],
+ "description": "If the group already exists and was not imported from a Prometheus-compatible source initially,\nit will not be replaced and an error will be returned.",
+ "operationId": "RouteConvertPrometheusPostRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-datasource-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-recording-rules-paused",
+ "type": "boolean"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-alert-rules-paused",
+ "type": "boolean"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-target-datasource-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-folder-uid",
+ "type": "string"
+ },
+ {
+ "in": "header",
+ "name": "x-grafana-alerting-notification-receiver",
+ "type": "string"
+ },
+ {
+ "in": "body",
+ "name": "Body",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
+ "tags": [
+ "convert_prometheus"
+ ],
+ "x-raw-request": "true"
+ }
+ },
+ "/convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group}": {
+ "delete": {
+ "operationId": "RouteConvertPrometheusDeleteRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "path",
+ "name": "Group",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/json"
+ ],
+ "responses": {
+ "202": {
+ "description": "ConvertPrometheusResponse",
+ "schema": {
+ "$ref": "#/definitions/ConvertPrometheusResponse"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ }
+ },
+ "summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ },
+ "get": {
+ "operationId": "RouteConvertPrometheusGetRuleGroup",
+ "parameters": [
+ {
+ "in": "path",
+ "name": "NamespaceTitle",
+ "required": true,
+ "type": "string"
+ },
+ {
+ "in": "path",
+ "name": "Group",
+ "required": true,
+ "type": "string"
+ }
+ ],
+ "produces": [
+ "application/yaml"
+ ],
+ "responses": {
+ "200": {
+ "description": "PrometheusRuleGroup",
+ "schema": {
+ "$ref": "#/definitions/PrometheusRuleGroup"
+ }
+ },
+ "403": {
+ "description": "ForbiddenError",
+ "schema": {
+ "$ref": "#/definitions/ForbiddenError"
+ }
+ },
+ "404": {
+ "description": "NotFound",
+ "schema": {
+ "$ref": "#/definitions/NotFound"
+ }
+ }
+ },
+ "summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
+ "tags": [
+ "convert_prometheus"
+ ]
+ }
+ },
"/v1/provisioning/alert-rules": {
"get": {
"operationId": "RouteGetAlertRules",
diff --git a/pkg/services/ngalert/api/tooling/definitions/convert_prometheus_api.go b/pkg/services/ngalert/api/tooling/definitions/convert_prometheus_api.go
index 98b17d19ed3..b3c2db6fe7a 100644
--- a/pkg/services/ngalert/api/tooling/definitions/convert_prometheus_api.go
+++ b/pkg/services/ngalert/api/tooling/definitions/convert_prometheus_api.go
@@ -5,7 +5,7 @@ import (
)
// Route for mimirtool
-// swagger:route GET /convert/prometheus/config/v1/rules convert_prometheus RouteConvertPrometheusGetRules
+// swagger:route GET /convert/prometheus/config/v1/rules convert_prometheus stable RouteConvertPrometheusGetRules
//
// Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.
//
@@ -18,7 +18,7 @@ import (
// 404: NotFound
// Route for cortextool
-// swagger:route GET /convert/api/prom/rules convert_prometheus RouteConvertPrometheusCortexGetRules
+// swagger:route GET /convert/api/prom/rules convert_prometheus stable RouteConvertPrometheusCortexGetRules
//
// Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.
//
@@ -31,7 +31,7 @@ import (
// 404: NotFound
// Route for mimirtool
-// swagger:route GET /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusGetNamespace
+// swagger:route GET /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusGetNamespace
//
// Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).
//
@@ -44,7 +44,7 @@ import (
// 404: NotFound
// Route for cortextool
-// swagger:route GET /convert/api/prom/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusCortexGetNamespace
+// swagger:route GET /convert/api/prom/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusCortexGetNamespace
//
// Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).
//
@@ -57,7 +57,7 @@ import (
// 404: NotFound
// Route for mimirtool
-// swagger:route GET /convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group} convert_prometheus RouteConvertPrometheusGetRuleGroup
+// swagger:route GET /convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group} convert_prometheus stable RouteConvertPrometheusGetRuleGroup
//
// Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.
//
@@ -70,7 +70,7 @@ import (
// 404: NotFound
// Route for cortextool
-// swagger:route GET /convert/api/prom/rules/{NamespaceTitle}/{Group} convert_prometheus RouteConvertPrometheusCortexGetRuleGroup
+// swagger:route GET /convert/api/prom/rules/{NamespaceTitle}/{Group} convert_prometheus stable RouteConvertPrometheusCortexGetRuleGroup
//
// Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.
//
@@ -82,7 +82,7 @@ import (
// 403: ForbiddenError
// 404: NotFound
-// swagger:route POST /convert/prometheus/config/v1/rules convert_prometheus RouteConvertPrometheusPostRuleGroups
+// swagger:route POST /convert/prometheus/config/v1/rules convert_prometheus stable RouteConvertPrometheusPostRuleGroups
//
// Converts the submitted rule groups into Grafana-Managed Rules.
//
@@ -97,7 +97,7 @@ import (
// 202: ConvertPrometheusResponse
// 403: ForbiddenError
-// swagger:route POST /convert/api/prom/rules convert_prometheus RouteConvertPrometheusCortexPostRuleGroups
+// swagger:route POST /convert/api/prom/rules convert_prometheus stable RouteConvertPrometheusCortexPostRuleGroups
//
// Converts the submitted rule groups into Grafana-Managed Rules.
//
@@ -113,7 +113,7 @@ import (
// 403: ForbiddenError
// Route for mimirtool
-// swagger:route POST /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusPostRuleGroup
+// swagger:route POST /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusPostRuleGroup
//
// Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.
// If the group already exists and was not imported from a Prometheus-compatible source initially,
@@ -133,7 +133,7 @@ import (
// x-raw-request: true
// Route for cortextool
-// swagger:route POST /convert/api/prom/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusCortexPostRuleGroup
+// swagger:route POST /convert/api/prom/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusCortexPostRuleGroup
//
// Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.
// If the group already exists and was not imported from a Prometheus-compatible source initially,
@@ -153,7 +153,7 @@ import (
// x-raw-request: true
// Route for mimirtool
-// swagger:route DELETE /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusDeleteNamespace
+// swagger:route DELETE /convert/prometheus/config/v1/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusDeleteNamespace
//
// Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.
//
@@ -165,7 +165,7 @@ import (
// 403: ForbiddenError
// Route for cortextool
-// swagger:route DELETE /convert/api/prom/rules/{NamespaceTitle} convert_prometheus RouteConvertPrometheusCortexDeleteNamespace
+// swagger:route DELETE /convert/api/prom/rules/{NamespaceTitle} convert_prometheus stable RouteConvertPrometheusCortexDeleteNamespace
//
// Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.
//
@@ -177,7 +177,7 @@ import (
// 403: ForbiddenError
// Route for mimirtool
-// swagger:route DELETE /convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group} convert_prometheus RouteConvertPrometheusDeleteRuleGroup
+// swagger:route DELETE /convert/prometheus/config/v1/rules/{NamespaceTitle}/{Group} convert_prometheus stable RouteConvertPrometheusDeleteRuleGroup
//
// Deletes a specific rule group if it was imported from a Prometheus-compatible source.
//
@@ -189,7 +189,7 @@ import (
// 403: ForbiddenError
// Route for cortextool
-// swagger:route DELETE /convert/api/prom/rules/{NamespaceTitle}/{Group} convert_prometheus RouteConvertPrometheusCortexDeleteRuleGroup
+// swagger:route DELETE /convert/api/prom/rules/{NamespaceTitle}/{Group} convert_prometheus stable RouteConvertPrometheusCortexDeleteRuleGroup
//
// Deletes a specific rule group if it was imported from a Prometheus-compatible source.
//
diff --git a/pkg/services/ngalert/api/tooling/spec.json b/pkg/services/ngalert/api/tooling/spec.json
index 1b2744b976d..0617865b8d2 100644
--- a/pkg/services/ngalert/api/tooling/spec.json
+++ b/pkg/services/ngalert/api/tooling/spec.json
@@ -1076,7 +1076,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
"operationId": "RouteConvertPrometheusCortexGetRules",
@@ -1110,7 +1111,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
"operationId": "RouteConvertPrometheusCortexPostRuleGroups",
@@ -1136,7 +1138,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
"operationId": "RouteConvertPrometheusCortexGetNamespace",
@@ -1178,7 +1181,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
"operationId": "RouteConvertPrometheusCortexPostRuleGroup",
@@ -1248,7 +1252,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
"operationId": "RouteConvertPrometheusCortexDeleteNamespace",
@@ -1282,7 +1287,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
"operationId": "RouteConvertPrometheusCortexGetRuleGroup",
@@ -1326,7 +1332,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
"operationId": "RouteConvertPrometheusCortexDeleteRuleGroup",
@@ -1483,7 +1490,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets all Grafana-managed alert rules that were imported from Prometheus-compatible sources, grouped by namespace.",
"operationId": "RouteConvertPrometheusGetRules",
@@ -1517,7 +1525,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Converts the submitted rule groups into Grafana-Managed Rules.",
"operationId": "RouteConvertPrometheusPostRuleGroups",
@@ -1543,7 +1552,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets Grafana-managed alert rules that were imported from Prometheus-compatible sources for a specified namespace (folder).",
"operationId": "RouteConvertPrometheusGetNamespace",
@@ -1585,7 +1595,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Converts a Prometheus rule group into a Grafana rule group and creates or updates it within the specified namespace.",
"operationId": "RouteConvertPrometheusPostRuleGroup",
@@ -1655,7 +1666,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Deletes all rule groups that were imported from Prometheus-compatible sources within the specified namespace.",
"operationId": "RouteConvertPrometheusDeleteNamespace",
@@ -1689,7 +1701,8 @@
"application/yaml"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Gets a single rule group in Prometheus-compatible format if it was imported from a Prometheus-compatible source.",
"operationId": "RouteConvertPrometheusGetRuleGroup",
@@ -1733,7 +1746,8 @@
"application/json"
],
"tags": [
- "convert_prometheus"
+ "convert_prometheus",
+ "stable"
],
"summary": "Deletes a specific rule group if it was imported from a Prometheus-compatible source.",
"operationId": "RouteConvertPrometheusDeleteRuleGroup",
diff --git a/pkg/services/ngalert/ngalert.go b/pkg/services/ngalert/ngalert.go
index 3861fa60c54..e794460fc5e 100644
--- a/pkg/services/ngalert/ngalert.go
+++ b/pkg/services/ngalert/ngalert.go
@@ -225,7 +225,6 @@ func (ng *AlertNG) init() error {
if remotePrimary {
ng.Log.Debug("Starting Grafana with remote primary mode enabled")
m.Info.WithLabelValues(metrics.ModeRemotePrimary).Set(1)
- ng.Cfg.UnifiedAlerting.SkipClustering = true
// This function will be used by the MOA to create new Alertmanagers.
override = notifier.WithAlertmanagerOverride(func(factoryFn notifier.OrgAlertmanagerFactory) notifier.OrgAlertmanagerFactory {
return func(ctx context.Context, orgID int64) (notifier.Alertmanager, error) {
diff --git a/pkg/services/ngalert/notifier/multiorg_alertmanager.go b/pkg/services/ngalert/notifier/multiorg_alertmanager.go
index 4ce738c08a9..4aa0151d18f 100644
--- a/pkg/services/ngalert/notifier/multiorg_alertmanager.go
+++ b/pkg/services/ngalert/notifier/multiorg_alertmanager.go
@@ -161,12 +161,8 @@ func NewMultiOrgAlertmanager(
peer: &NilPeer{},
}
- if cfg.UnifiedAlerting.SkipClustering {
- l.Info("Skipping setting up clustering for MOA")
- } else {
- if err := moa.setupClustering(cfg); err != nil {
- return nil, err
- }
+ if err := moa.setupClustering(cfg); err != nil {
+ return nil, err
}
// Set up the default per tenant Alertmanager factory.
diff --git a/pkg/services/ngalert/remote/alertmanager.go b/pkg/services/ngalert/remote/alertmanager.go
index 7c1e1dd4060..8329c7bb8de 100644
--- a/pkg/services/ngalert/remote/alertmanager.go
+++ b/pkg/services/ngalert/remote/alertmanager.go
@@ -2,10 +2,10 @@ package remote
import (
"context"
- "crypto/md5"
"encoding/base64"
"encoding/json"
"fmt"
+ "hash/fnv"
"net/http"
"net/url"
"strings"
@@ -16,12 +16,12 @@ import (
"github.com/grafana/alerting/definition"
alertingModels "github.com/grafana/alerting/models"
alertingNotify "github.com/grafana/alerting/notify"
+ "github.com/grafana/alerting/utils/hash"
amalert "github.com/prometheus/alertmanager/api/v2/client/alert"
amalertgroup "github.com/prometheus/alertmanager/api/v2/client/alertgroup"
amgeneral "github.com/prometheus/alertmanager/api/v2/client/general"
amsilence "github.com/prometheus/alertmanager/api/v2/client/silence"
"github.com/prometheus/client_golang/prometheus"
-
"gopkg.in/yaml.v3"
"github.com/grafana/grafana/pkg/infra/log"
@@ -73,6 +73,9 @@ type Alertmanager struct {
amClient *remoteClient.Alertmanager
mimirClient remoteClient.MimirClient
+
+ promoteConfig bool
+ externalURL string
}
type AlertmanagerConfig struct {
@@ -127,13 +130,10 @@ func NewAlertmanager(ctx context.Context, cfg AlertmanagerConfig, store stateSto
logger := log.New("ngalert.remote.alertmanager")
mcCfg := &remoteClient.Config{
- Logger: logger,
- Password: cfg.BasicAuthPassword,
- TenantID: cfg.TenantID,
- URL: u,
- PromoteConfig: cfg.PromoteConfig,
- ExternalURL: cfg.ExternalURL,
- Smtp: cfg.SmtpConfig,
+ Logger: logger,
+ Password: cfg.BasicAuthPassword,
+ TenantID: cfg.TenantID,
+ URL: u,
}
mc, err := remoteClient.New(mcCfg, metrics, tracer)
if err != nil {
@@ -188,7 +188,10 @@ func NewAlertmanager(ctx context.Context, cfg AlertmanagerConfig, store stateSto
syncInterval: cfg.SyncInterval,
tenantID: cfg.TenantID,
url: cfg.URL,
- smtp: cfg.SmtpConfig,
+
+ externalURL: cfg.ExternalURL,
+ promoteConfig: cfg.PromoteConfig,
+ smtp: cfg.SmtpConfig,
}
// Parse the default configuration once and remember its hash so we can compare it later.
@@ -196,15 +199,11 @@ func NewAlertmanager(ctx context.Context, cfg AlertmanagerConfig, store stateSto
// (grouping, group timing, time intervals etc) changes the autogenerated configuration.
// The `default` flag is sent to the remote Alertmanager for informational purposes, so we can tolerate this.
err = func() error {
- defaultCfg, err := am.buildConfiguration(ctx, []byte(cfg.DefaultConfig))
+ defaultCfg, err := am.buildConfiguration(ctx, []byte(cfg.DefaultConfig), 0)
if err != nil {
return fmt.Errorf("unable to build default configuration: %w", err)
}
- rawDefaultCfg, err := json.Marshal(defaultCfg)
- if err != nil {
- return fmt.Errorf("unable to marshal default configuration: %w", err)
- }
- am.defaultConfigHash = fmt.Sprintf("%x", md5.Sum(rawDefaultCfg))
+ am.defaultConfigHash = defaultCfg.Hash
return nil
}()
if err != nil {
@@ -265,22 +264,16 @@ func (am *Alertmanager) checkReadiness(ctx context.Context) error {
// CompareAndSendConfiguration checks whether a given configuration is being used by the remote Alertmanager.
// If not, it sends the configuration to the remote Alertmanager.
func (am *Alertmanager) CompareAndSendConfiguration(ctx context.Context, config *models.AlertConfiguration) error {
- payload, err := am.buildConfiguration(ctx, []byte(config.AlertmanagerConfiguration))
+ payload, err := am.buildConfiguration(ctx, []byte(config.AlertmanagerConfiguration), config.CreatedAt)
if err != nil {
return fmt.Errorf("unable to build configuration: %w", err)
}
- rawPayload, err := json.Marshal(payload)
- if err != nil {
- return fmt.Errorf("unable to marshal decrypted configuration: %w", err)
- }
- configHash := fmt.Sprintf("%x", md5.Sum(rawPayload))
-
// Send the configuration only if we need to.
- if !am.shouldSendConfig(ctx, configHash) {
+ if !am.shouldSendConfig(ctx, payload.Hash) {
return nil
}
- return am.sendConfiguration(ctx, payload, configHash, config.CreatedAt, am.isDefaultConfiguration(configHash))
+ return am.sendConfiguration(ctx, payload)
}
func (am *Alertmanager) isDefaultConfiguration(configHash string) bool {
@@ -303,31 +296,31 @@ func decrypter(ctx context.Context, crypto Crypto) models.DecryptFn {
// buildConfiguration takes a raw Alertmanager configuration and returns a config that the remote Alertmanager can use.
// It parses the initial configuration, adds auto-generated routes, decrypts receivers, and merges the extra configs.
-func (am *Alertmanager) buildConfiguration(ctx context.Context, raw []byte) (remoteClient.GrafanaAlertmanagerConfig, error) {
+func (am *Alertmanager) buildConfiguration(ctx context.Context, raw []byte, createdAtEpoch int64) (remoteClient.UserGrafanaConfig, error) {
c, err := notifier.Load(raw)
if err != nil {
- return remoteClient.GrafanaAlertmanagerConfig{}, err
+ return remoteClient.UserGrafanaConfig{}, err
}
// Add auto-generated routes and decrypt before comparing.
if err := am.autogenFn(ctx, am.log, am.orgID, &c.AlertmanagerConfig, true); err != nil {
- return remoteClient.GrafanaAlertmanagerConfig{}, err
+ return remoteClient.UserGrafanaConfig{}, err
}
// Decrypt the receivers in the configuration.
decryptedReceivers, err := legacy_storage.DecryptedReceivers(c.AlertmanagerConfig.Receivers, decrypter(ctx, am.crypto))
if err != nil {
- return remoteClient.GrafanaAlertmanagerConfig{}, fmt.Errorf("unable to decrypt receivers: %w", err)
+ return remoteClient.UserGrafanaConfig{}, fmt.Errorf("unable to decrypt receivers: %w", err)
}
c.AlertmanagerConfig.Receivers = decryptedReceivers
if err := am.crypto.DecryptExtraConfigs(ctx, c); err != nil {
- return remoteClient.GrafanaAlertmanagerConfig{}, fmt.Errorf("unable to decrypt extra configs: %w", err)
+ return remoteClient.UserGrafanaConfig{}, fmt.Errorf("unable to decrypt extra configs: %w", err)
}
mergeResult, err := c.GetMergedAlertmanagerConfig()
if err != nil {
- return remoteClient.GrafanaAlertmanagerConfig{}, fmt.Errorf("unable to get merged Alertmanager configuration: %w", err)
+ return remoteClient.UserGrafanaConfig{}, fmt.Errorf("unable to get merged Alertmanager configuration: %w", err)
}
var templates []definition.PostableApiTemplate
@@ -335,22 +328,31 @@ func (am *Alertmanager) buildConfiguration(ctx context.Context, raw []byte) (rem
templates = definition.TemplatesMapToPostableAPITemplates(c.ExtraConfigs[0].TemplateFiles, definition.MimirTemplateKind)
}
- return remoteClient.GrafanaAlertmanagerConfig{
- TemplateFiles: c.TemplateFiles,
- AlertmanagerConfig: mergeResult.Config,
- Templates: templates,
- }, nil
+ payload := remoteClient.UserGrafanaConfig{
+ GrafanaAlertmanagerConfig: remoteClient.GrafanaAlertmanagerConfig{
+ TemplateFiles: c.TemplateFiles,
+ AlertmanagerConfig: mergeResult.Config,
+ Templates: templates,
+ },
+ CreatedAt: createdAtEpoch,
+ Promoted: am.promoteConfig,
+ ExternalURL: am.externalURL,
+ SmtpConfig: am.smtp,
+ }
+
+ cfgHash, err := calculateUserGrafanaConfigHash(payload)
+ if err != nil {
+ am.log.Error("Unable to calculate hash of the configuration. Using the empty string", "error", err)
+ cfgHash = ""
+ }
+ payload.Hash = cfgHash
+ payload.Default = am.isDefaultConfiguration(cfgHash)
+ return payload, nil
}
-func (am *Alertmanager) sendConfiguration(ctx context.Context, cfg remoteClient.GrafanaAlertmanagerConfig, hash string, createdAt int64, isDefault bool) error {
+func (am *Alertmanager) sendConfiguration(ctx context.Context, cfg remoteClient.UserGrafanaConfig) error {
am.metrics.ConfigSyncsTotal.Inc()
- if err := am.mimirClient.CreateGrafanaAlertmanagerConfig(
- ctx,
- cfg,
- hash,
- createdAt,
- isDefault,
- ); err != nil {
+ if err := am.mimirClient.CreateGrafanaAlertmanagerConfig(ctx, &cfg); err != nil {
am.metrics.ConfigSyncErrorsTotal.Inc()
return err
}
@@ -422,40 +424,25 @@ func (am *Alertmanager) SaveAndApplyConfig(ctx context.Context, cfg *apimodels.P
return err
}
- payload, err := am.buildConfiguration(ctx, rawCopy)
+ payload, err := am.buildConfiguration(ctx, rawCopy, time.Now().Unix())
if err != nil {
return fmt.Errorf("unable to build configuration: %w", err)
}
- rawCfg, err := json.Marshal(payload)
- if err != nil {
- return err
- }
- hash := fmt.Sprintf("%x", md5.Sum(rawCfg))
-
- return am.sendConfiguration(ctx, payload, hash, time.Now().Unix(), false)
+ return am.sendConfiguration(ctx, payload)
}
// SaveAndApplyDefaultConfig sends the default Grafana Alertmanager configuration to the remote Alertmanager.
func (am *Alertmanager) SaveAndApplyDefaultConfig(ctx context.Context) error {
am.log.Debug("Sending default configuration to a remote Alertmanager", "url", am.url)
- payload, err := am.buildConfiguration(ctx, []byte(am.defaultConfig))
+ payload, err := am.buildConfiguration(ctx, []byte(am.defaultConfig), time.Now().Unix())
if err != nil {
return fmt.Errorf("unable to build default configuration: %w", err)
}
-
- rawCfg, err := json.Marshal(payload)
- if err != nil {
- return err
- }
- hash := fmt.Sprintf("%x", md5.Sum(rawCfg))
-
+ payload.Default = true // override default status
return am.sendConfiguration(
ctx,
payload,
- hash,
- time.Now().Unix(),
- true,
)
}
@@ -696,37 +683,29 @@ func (am *Alertmanager) getFullState(ctx context.Context) (string, error) {
// shouldSendConfig compares the remote Alertmanager configuration with our local one.
// It returns true if the configurations are different.
func (am *Alertmanager) shouldSendConfig(ctx context.Context, hash string) bool {
+ if hash == "" { // empty hash means that something went wrong while calculating it. In this case, always send the config.
+ return true
+ }
rc, err := am.mimirClient.GetGrafanaAlertmanagerConfig(ctx)
if err != nil {
// Log the error and return true so we try to upload our config anyway.
am.log.Warn("Unable to get the remote Alertmanager configuration for comparison, sending the configuration without comparing", "err", err)
return true
}
-
- if rc.Promoted != am.mimirClient.ShouldPromoteConfig() {
+ if rc.Hash != hash {
+ am.log.Debug("Hash of the remote Alertmanager configuration is different, sending the configuration", "remote", rc.Hash, "local", hash)
return true
}
+ return false
+}
- // Compare SMTP configs.
- if rc.SmtpConfig.EhloIdentity != am.smtp.EhloIdentity ||
- rc.SmtpConfig.Password != am.smtp.Password ||
- rc.SmtpConfig.FromAddress != am.smtp.FromAddress ||
- rc.SmtpConfig.FromName != am.smtp.FromName ||
- rc.SmtpConfig.Host != am.smtp.Host ||
- rc.SmtpConfig.SkipVerify != am.smtp.SkipVerify ||
- rc.SmtpConfig.StartTLSPolicy != am.smtp.StartTLSPolicy ||
- len(rc.SmtpConfig.StaticHeaders) != len(am.smtp.StaticHeaders) ||
- rc.SmtpConfig.User != am.smtp.User {
- am.log.Debug("SMTP config is different, sending the configuration to the remote Alertmanager")
- return true
- }
+func calculateUserGrafanaConfigHash(config remoteClient.UserGrafanaConfig) (string, error) {
+ // Ignore some fields when calculating the hash. Make sure the original struct is not modified after that.
+ config.Default = false
+ config.CreatedAt = 0 // ignore createdAt to support comparison with hash of default config
+ config.Hash = ""
- for k, v := range rc.SmtpConfig.StaticHeaders {
- if value, ok := am.smtp.StaticHeaders[k]; !ok || v != value {
- am.log.Debug("SMTP static headers are different, sending the configuration to the remote Alertmanager")
- return true
- }
- }
-
- return rc.Hash != hash
+ hasher := fnv.New64a()
+ hash.DeepHashObject(hasher, &config)
+ return fmt.Sprintf("%x", hasher.Sum64()), nil
}
diff --git a/pkg/services/ngalert/remote/alertmanager_test.go b/pkg/services/ngalert/remote/alertmanager_test.go
index 6ce6abb8994..10ea5e45eaf 100644
--- a/pkg/services/ngalert/remote/alertmanager_test.go
+++ b/pkg/services/ngalert/remote/alertmanager_test.go
@@ -19,10 +19,13 @@ import (
"time"
"github.com/go-openapi/strfmt"
+ "github.com/google/go-cmp/cmp"
+ "github.com/google/go-cmp/cmp/cmpopts"
amv2 "github.com/prometheus/alertmanager/api/v2/models"
"github.com/prometheus/alertmanager/config"
"github.com/prometheus/alertmanager/pkg/labels"
"github.com/prometheus/client_golang/prometheus"
+ common_config "github.com/prometheus/common/config"
"github.com/stretchr/testify/require"
alertingClusterPB "github.com/grafana/alerting/cluster/clusterpb"
@@ -501,15 +504,6 @@ func TestCompareAndSendConfiguration(t *testing.T) {
AlertmanagerConfig: testAutogenRoutes.AlertmanagerConfig,
}
- // Calculate hashes for expected configurations
- cfgWithDecryptedSecretBytes, err := json.Marshal(cfgWithDecryptedSecret)
- require.NoError(t, err)
- cfgWithDecryptedSecretHash := fmt.Sprintf("%x", md5.Sum(cfgWithDecryptedSecretBytes))
-
- cfgWithAutogenRoutesBytes, err := json.Marshal(cfgWithAutogenRoutes)
- require.NoError(t, err)
- cfgWithAutogenRoutesHash := fmt.Sprintf("%x", md5.Sum(cfgWithAutogenRoutesBytes))
-
cfgWithExtraUnmergedBytes, err := testData.ReadFile(path.Join("test-data", "config-with-extra.json"))
require.NoError(t, err)
cfgWithExtraUnmerged, err := notifier.Load(cfgWithExtraUnmergedBytes)
@@ -521,9 +515,6 @@ func TestCompareAndSendConfiguration(t *testing.T) {
AlertmanagerConfig: r.Config,
Templates: definition.TemplatesMapToPostableAPITemplates(cfgWithExtraUnmerged.ExtraConfigs[0].TemplateFiles, definition.MimirTemplateKind),
}
- cfgWithExtraMergedBytes, err := json.Marshal(cfgWithExtraMerged)
- require.NoError(t, err)
- cfgWithExtraMergedHash := fmt.Sprintf("%x", md5.Sum(cfgWithExtraMergedBytes))
tests := []struct {
name string
@@ -566,7 +557,6 @@ func TestCompareAndSendConfiguration(t *testing.T) {
NoopAutogenFn,
&client.UserGrafanaConfig{
GrafanaAlertmanagerConfig: cfgWithDecryptedSecret,
- Hash: cfgWithDecryptedSecretHash,
},
nil,
},
@@ -576,7 +566,6 @@ func TestCompareAndSendConfiguration(t *testing.T) {
testAutogenFn,
&client.UserGrafanaConfig{
GrafanaAlertmanagerConfig: cfgWithAutogenRoutes,
- Hash: cfgWithAutogenRoutesHash,
},
nil,
},
@@ -586,7 +575,6 @@ func TestCompareAndSendConfiguration(t *testing.T) {
autogenFn: NoopAutogenFn,
expCfg: &client.UserGrafanaConfig{
GrafanaAlertmanagerConfig: cfgWithExtraMerged,
- Hash: cfgWithExtraMergedHash,
},
},
}
@@ -614,9 +602,26 @@ func TestCompareAndSendConfiguration(t *testing.T) {
err = am.CompareAndSendConfiguration(ctx, &cfg)
if len(test.expErrContains) == 0 {
require.NoError(tt, err)
- rawCfg, err := json.Marshal(test.expCfg)
+
+ var gotCfg client.UserGrafanaConfig
+ require.NoError(tt, json.Unmarshal([]byte(got), &gotCfg))
+
+ require.NotEmpty(tt, gotCfg.Hash)
+ require.Empty(tt, cmp.Diff(test.expCfg, &gotCfg,
+ cmpopts.IgnoreFields(client.UserGrafanaConfig{}, "Hash"), // do not compare hashes because the config is processed slightly different: empty maps are nils.
+ cmpopts.EquateEmpty(),
+ cmpopts.IgnoreUnexported(
+ time.Location{},
+ labels.Matcher{},
+ common_config.ProxyConfig{})))
+
+ got1 := got
+ got = ""
+ err = am.CompareAndSendConfiguration(ctx, &cfg)
require.NoError(tt, err)
- require.JSONEq(tt, string(rawCfg), got)
+
+ got2 := got
+ require.Equalf(tt, got1, got2, "Configuration is not idempotent")
return
}
for _, expErr := range test.expErrContains {
@@ -815,12 +820,7 @@ receivers:
require.NotNil(t, extraReceiver)
require.Len(t, extraReceiver.EmailConfigs, 1)
require.Equal(t, "alerts@grafana.com", extraReceiver.EmailConfigs[0].To)
-
- // Verify the config hash
- expectedConfigBytes, err := json.Marshal(configSent.GrafanaAlertmanagerConfig)
- require.NoError(t, err)
- expectedHash := fmt.Sprintf("%x", md5.Sum(expectedConfigBytes))
- require.Equal(t, expectedHash, configSent.Hash)
+ require.NotEmpty(t, configSent.Hash)
}
func TestCompareAndSendConfigurationWithExtraConfigs(t *testing.T) {
@@ -934,10 +934,7 @@ receivers:
require.True(t, found)
// Verify the config hash
- expectedConfigBytes, err := json.Marshal(configSent.GrafanaAlertmanagerConfig)
- require.NoError(t, err)
- expectedHash := fmt.Sprintf("%x", md5.Sum(expectedConfigBytes))
- require.Equal(t, expectedHash, configSent.Hash)
+ require.NotEmpty(t, configSent.Hash)
}
func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
@@ -961,11 +958,10 @@ func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
DefaultConfig: defaultGrafanaConfig,
}
- testConfigHash := fmt.Sprintf("%x", md5.Sum([]byte(testGrafanaConfig)))
testConfigCreatedAt := time.Now().Unix()
testConfig := &ngmodels.AlertConfiguration{
AlertmanagerConfiguration: testGrafanaConfig,
- ConfigurationHash: testConfigHash,
+ ConfigurationHash: "",
ConfigurationVersion: "v2",
CreatedAt: testConfigCreatedAt,
OrgID: 1,
@@ -1012,7 +1008,6 @@ func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
rawCfg, err := json.Marshal(config.GrafanaAlertmanagerConfig)
require.NoError(t, err)
require.JSONEq(t, testGrafanaConfig, string(rawCfg))
- require.Equal(t, testConfigHash, config.Hash)
require.Equal(t, testConfigCreatedAt, config.CreatedAt)
require.Equal(t, testConfig.Default, config.Default)
@@ -1038,7 +1033,6 @@ func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
rawCfg, err := json.Marshal(config.GrafanaAlertmanagerConfig)
require.NoError(t, err)
require.JSONEq(t, testGrafanaConfig, string(rawCfg))
- require.Equal(t, testConfigHash, config.Hash)
require.Equal(t, testConfigCreatedAt, config.CreatedAt)
require.False(t, config.Default)
@@ -1085,9 +1079,6 @@ func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
require.JSONEq(t, testGrafanaConfigWithSecret, string(got))
- // Verify that the hash is calculated from the final configuration, including simplified routing
- expectedHash := fmt.Sprintf("%x", md5.Sum(got))
- require.Equal(t, expectedHash, config.Hash, "Hash should be calculated from the final processed configuration")
require.False(t, config.Default)
// An error while adding auto-generated rutes should be returned.
@@ -1114,7 +1105,6 @@ func TestIntegrationRemoteAlertmanagerConfiguration(t *testing.T) {
require.NoError(t, err)
require.JSONEq(t, string(want), string(got))
- require.Equal(t, fmt.Sprintf("%x", md5.Sum(want)), config.Hash)
require.True(t, config.Default)
// An error while adding auto-generated rutes should be returned.
diff --git a/pkg/services/ngalert/remote/client/alertmanager_configuration.go b/pkg/services/ngalert/remote/client/alertmanager_configuration.go
index b812c146687..a53132a8812 100644
--- a/pkg/services/ngalert/remote/client/alertmanager_configuration.go
+++ b/pkg/services/ngalert/remote/client/alertmanager_configuration.go
@@ -39,10 +39,6 @@ type UserGrafanaConfig struct {
SmtpConfig SmtpConfig `json:"smtp_config"`
}
-func (mc *Mimir) ShouldPromoteConfig() bool {
- return mc.promoteConfig
-}
-
func (mc *Mimir) GetGrafanaAlertmanagerConfig(ctx context.Context) (*UserGrafanaConfig, error) {
gc := &UserGrafanaConfig{}
response := successResponse{
@@ -62,16 +58,8 @@ func (mc *Mimir) GetGrafanaAlertmanagerConfig(ctx context.Context) (*UserGrafana
return gc, nil
}
-func (mc *Mimir) CreateGrafanaAlertmanagerConfig(ctx context.Context, cfg GrafanaAlertmanagerConfig, hash string, createdAt int64, isDefault bool) error {
- payload, err := definition.MarshalJSONWithSecrets(&UserGrafanaConfig{
- GrafanaAlertmanagerConfig: cfg,
- Hash: hash,
- CreatedAt: createdAt,
- Default: isDefault,
- Promoted: mc.promoteConfig,
- ExternalURL: mc.externalURL,
- SmtpConfig: mc.smtpConfig,
- })
+func (mc *Mimir) CreateGrafanaAlertmanagerConfig(ctx context.Context, cfg *UserGrafanaConfig) error {
+ payload, err := definition.MarshalJSONWithSecrets(cfg)
if err != nil {
return err
}
diff --git a/pkg/services/ngalert/remote/client/mimir.go b/pkg/services/ngalert/remote/client/mimir.go
index 1533e24c208..d8f9a51f327 100644
--- a/pkg/services/ngalert/remote/client/mimir.go
+++ b/pkg/services/ngalert/remote/client/mimir.go
@@ -30,26 +30,21 @@ type MimirClient interface {
DeleteGrafanaAlertmanagerState(ctx context.Context) error
GetGrafanaAlertmanagerConfig(ctx context.Context) (*UserGrafanaConfig, error)
- CreateGrafanaAlertmanagerConfig(ctx context.Context, configuration GrafanaAlertmanagerConfig, hash string, createdAt int64, isDefault bool) error
+ CreateGrafanaAlertmanagerConfig(ctx context.Context, config *UserGrafanaConfig) error
DeleteGrafanaAlertmanagerConfig(ctx context.Context) error
TestTemplate(ctx context.Context, c alertingNotify.TestTemplatesConfigBodyParams) (*alertingNotify.TestTemplatesResults, error)
TestReceivers(ctx context.Context, c alertingNotify.TestReceiversConfigBodyParams) (*alertingNotify.TestReceiversResult, int, error)
- ShouldPromoteConfig() bool
-
// Mimir implements an extended version of the receivers API under a different path.
GetReceivers(ctx context.Context) ([]apimodels.Receiver, error)
}
type Mimir struct {
- client client.Requester
- endpoint *url.URL
- logger log.Logger
- metrics *metrics.RemoteAlertmanager
- promoteConfig bool
- externalURL string
- smtpConfig SmtpConfig
+ client client.Requester
+ endpoint *url.URL
+ logger log.Logger
+ metrics *metrics.RemoteAlertmanager
}
type SmtpConfig struct {
@@ -69,10 +64,7 @@ type Config struct {
TenantID string
Password string
- Logger log.Logger
- PromoteConfig bool
- ExternalURL string
- Smtp SmtpConfig
+ Logger log.Logger
}
// successResponse represents a successful response from the Mimir API.
@@ -110,13 +102,10 @@ func New(cfg *Config, metrics *metrics.RemoteAlertmanager, tracer tracing.Tracer
trc := client.NewTracedClient(tc, tracer, "remote.alertmanager.client")
return &Mimir{
- endpoint: cfg.URL,
- client: trc,
- logger: cfg.Logger,
- metrics: metrics,
- promoteConfig: cfg.PromoteConfig,
- externalURL: cfg.ExternalURL,
- smtpConfig: cfg.Smtp,
+ endpoint: cfg.URL,
+ client: trc,
+ logger: cfg.Logger,
+ metrics: metrics,
}, nil
}
diff --git a/pkg/services/query/query_test.go b/pkg/services/query/query_test.go
index 8b0d17a381c..66c4ff17f85 100644
--- a/pkg/services/query/query_test.go
+++ b/pkg/services/query/query_test.go
@@ -78,6 +78,38 @@ func TestIntegrationParseMetricRequest(t *testing.T) {
assert.Len(t, parsedReq.getFlattenedQueries(), 2)
})
+ t.Run("Test a simple single datasource query with missing time range", func(t *testing.T) {
+ tc := setup(t, false, nil)
+ mr := metricRequestWithQueries(t, `{
+ "refId": "A",
+ "datasource": {
+ "uid": "gIEkMvIVz",
+ "type": "postgres"
+ }
+ }`, `{
+ "refId": "B",
+ "datasource": {
+ "uid": "gIEkMvIVz",
+ "type": "postgres"
+ }
+ }`)
+ mr.From = ""
+ mr.To = ""
+ parsedReq, err := tc.queryService.parseMetricRequest(context.Background(), tc.signedInUser, true, mr)
+ require.NoError(t, err)
+ require.NotNil(t, parsedReq)
+ assert.False(t, parsedReq.hasExpression)
+ assert.Len(t, parsedReq.parsedQueries, 1)
+ assert.Contains(t, parsedReq.parsedQueries, "gIEkMvIVz")
+ queries := parsedReq.getFlattenedQueries()
+ assert.Len(t, queries, 2)
+
+ for _, q := range queries {
+ require.Equal(t, int64(0), q.query.TimeRange.From.UnixMilli())
+ require.Equal(t, int64(0), q.query.TimeRange.To.UnixMilli())
+ }
+ })
+
t.Run("Test a single datasource query with expressions", func(t *testing.T) {
tc := setup(t, false, nil)
mr := metricRequestWithQueries(t, `{
diff --git a/pkg/services/sqlstore/database_wrapper.go b/pkg/services/sqlstore/database_wrapper.go
index f691cb0c5a3..6a033b3b248 100644
--- a/pkg/services/sqlstore/database_wrapper.go
+++ b/pkg/services/sqlstore/database_wrapper.go
@@ -10,9 +10,9 @@ import (
"github.com/gchaincl/sqlhooks"
"github.com/go-sql-driver/mysql"
+ "github.com/grafana/grafana/pkg/util/sqlite"
"github.com/grafana/grafana/pkg/util/xorm/core"
"github.com/lib/pq"
- "github.com/mattn/go-sqlite3"
"github.com/prometheus/client_golang/prometheus"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
@@ -42,7 +42,7 @@ func init() {
// database queries. It also registers the metrics.
func WrapDatabaseDriverWithHooks(dbType string, tracer tracing.Tracer) string {
drivers := map[string]driver.Driver{
- migrator.SQLite: &sqlite3.SQLiteDriver{},
+ migrator.SQLite: &sqlite.Driver{},
migrator.MySQL: &mysql.MySQLDriver{},
migrator.Postgres: &pq.Driver{},
}
diff --git a/pkg/services/sqlstore/migrator/migrator.go b/pkg/services/sqlstore/migrator/migrator.go
index d8e55c4ee71..75ebf8e540b 100644
--- a/pkg/services/sqlstore/migrator/migrator.go
+++ b/pkg/services/sqlstore/migrator/migrator.go
@@ -2,15 +2,14 @@ package migrator
import (
"context"
- "errors"
"fmt"
"strings"
"time"
_ "github.com/go-sql-driver/mysql"
"github.com/golang-migrate/migrate/v4/database"
+ "github.com/grafana/grafana/pkg/util/sqlite"
_ "github.com/lib/pq"
- "github.com/mattn/go-sqlite3"
"github.com/prometheus/client_golang/prometheus"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
@@ -326,7 +325,7 @@ func (mg *Migrator) doMigration(ctx context.Context, m Migration) error {
err := mg.exec(ctx, m, sess)
// if we get an sqlite busy/locked error, sleep 100ms and try again
cnt := 0
- for cnt < 3 && (errors.Is(err, sqlite3.ErrLocked) || errors.Is(err, sqlite3.ErrBusy)) {
+ for cnt < 3 && sqlite.IsBusyOrLocked(err) {
cnt++
logger.Debug("Database locked, sleeping then retrying", "error", err, "sql", sql)
span.AddEvent("Database locked, sleeping then retrying",
diff --git a/pkg/services/sqlstore/migrator/sqlite_dialect.go b/pkg/services/sqlstore/migrator/sqlite_dialect.go
index 6c0e77972d8..d4302c18181 100644
--- a/pkg/services/sqlstore/migrator/sqlite_dialect.go
+++ b/pkg/services/sqlstore/migrator/sqlite_dialect.go
@@ -1,12 +1,10 @@
package migrator
import (
- "errors"
"fmt"
"strings"
- "github.com/mattn/go-sqlite3"
-
+ "github.com/grafana/grafana/pkg/util/sqlite"
"github.com/grafana/grafana/pkg/util/xorm"
)
@@ -139,27 +137,12 @@ func (db *SQLite3) TruncateDBTables(engine *xorm.Engine) error {
return nil
}
-func (db *SQLite3) isThisError(err error, errcode int) bool {
- var driverErr sqlite3.Error
- if errors.As(err, &driverErr) {
- if int(driverErr.ExtendedCode) == errcode {
- return true
- }
- }
-
- return false
-}
-
func (db *SQLite3) ErrorMessage(err error) string {
- var driverErr sqlite3.Error
- if errors.As(err, &driverErr) {
- return driverErr.Error()
- }
- return ""
+ return sqlite.ErrorMessage(err)
}
func (db *SQLite3) IsUniqueConstraintViolation(err error) bool {
- return db.isThisError(err, int(sqlite3.ErrConstraintUnique)) || db.isThisError(err, int(sqlite3.ErrConstraintPrimaryKey))
+ return sqlite.IsUniqueConstraintViolation(err)
}
func (db *SQLite3) IsDeadlock(err error) bool {
diff --git a/pkg/services/sqlstore/session_test.go b/pkg/services/sqlstore/session_test.go
index a30f11e6d11..0f2d0629a47 100644
--- a/pkg/services/sqlstore/session_test.go
+++ b/pkg/services/sqlstore/session_test.go
@@ -6,10 +6,10 @@ import (
"fmt"
"testing"
- "github.com/mattn/go-sqlite3"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/services/sqlstore/migrator"
+ "github.com/grafana/grafana/pkg/util/sqlite"
)
func TestIntegration_RetryingDisabled(t *testing.T) {
@@ -144,7 +144,7 @@ func getRetryErrors(t *testing.T, store *SQLStore) []error {
var retryErrors []error
switch store.GetDialect().DriverName() {
case migrator.SQLite:
- retryErrors = []error{sqlite3.Error{Code: sqlite3.ErrBusy}, sqlite3.Error{Code: sqlite3.ErrLocked}}
+ retryErrors = []error{sqlite.TestErrBusy, sqlite.TestErrLocked}
}
if len(retryErrors) == 0 {
diff --git a/pkg/services/sqlstore/sqlutil/sqlutil.go b/pkg/services/sqlstore/sqlutil/sqlutil.go
index 8506acdab79..7d3d09a5ff3 100644
--- a/pkg/services/sqlstore/sqlutil/sqlutil.go
+++ b/pkg/services/sqlstore/sqlutil/sqlutil.go
@@ -22,6 +22,11 @@ type TestDB struct {
DriverName string
ConnStr string
Path string
+ Host string
+ Port string
+ User string
+ Password string
+ Database string
Cleanup func()
}
@@ -132,6 +137,11 @@ func mySQLTestDB() (*TestDB, error) {
return &TestDB{
DriverName: "mysql",
ConnStr: conn_str,
+ Host: host,
+ Port: port,
+ User: "grafana",
+ Password: "password",
+ Database: "grafana_tests",
Cleanup: func() {},
}, nil
}
@@ -149,6 +159,11 @@ func postgresTestDB() (*TestDB, error) {
return &TestDB{
DriverName: "postgres",
ConnStr: connStr,
+ Host: host,
+ Port: port,
+ User: "grafanatest",
+ Password: "grafanatest",
+ Database: "grafanatest",
Cleanup: func() {},
}, nil
}
diff --git a/pkg/setting/setting_unified_alerting.go b/pkg/setting/setting_unified_alerting.go
index 665c751532f..0187652cb0e 100644
--- a/pkg/setting/setting_unified_alerting.go
+++ b/pkg/setting/setting_unified_alerting.go
@@ -121,7 +121,6 @@ type UnifiedAlertingSettings struct {
DefaultRuleEvaluationInterval time.Duration
Screenshots UnifiedAlertingScreenshotSettings
ReservedLabels UnifiedAlertingReservedLabelSettings
- SkipClustering bool
StateHistory UnifiedAlertingStateHistorySettings
NotificationHistory UnifiedAlertingNotificationHistorySettings
RemoteAlertmanager RemoteAlertmanagerSettings
diff --git a/pkg/storage/secret/metadata/data/secure_value_create.sql b/pkg/storage/secret/metadata/data/secure_value_create.sql
index a120feaa5e5..66d404f5bd2 100644
--- a/pkg/storage/secret/metadata/data/secure_value_create.sql
+++ b/pkg/storage/secret/metadata/data/secure_value_create.sql
@@ -20,6 +20,18 @@ INSERT INTO {{ .Ident "secret_secure_value" }} (
{{ if .Row.Ref.Valid }}
{{ .Ident "ref" }},
{{ end }}
+ {{ if .Row.OwnerReferenceAPIGroup.Valid }}
+ {{ .Ident "owner_reference_api_group" }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceAPIVersion.Valid }}
+ {{ .Ident "owner_reference_api_version" }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceKind.Valid }}
+ {{ .Ident "owner_reference_kind" }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceName.Valid }}
+ {{ .Ident "owner_reference_name" }},
+ {{ end }}
{{ .Ident "external_id" }}
) VALUES (
{{ .Arg .Row.GUID }},
@@ -43,5 +55,17 @@ INSERT INTO {{ .Ident "secret_secure_value" }} (
{{ if .Row.Ref.Valid }}
{{ .Arg .Row.Ref.String }},
{{ end }}
+ {{ if .Row.OwnerReferenceAPIGroup.Valid }}
+ {{ .Arg .Row.OwnerReferenceAPIGroup.String }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceAPIVersion.Valid }}
+ {{ .Arg .Row.OwnerReferenceAPIVersion.String }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceKind.Valid }}
+ {{ .Arg .Row.OwnerReferenceKind.String }},
+ {{ end }}
+ {{ if .Row.OwnerReferenceName.Valid }}
+ {{ .Arg .Row.OwnerReferenceName.String }},
+ {{ end }}
{{ .Arg .Row.ExternalID }}
);
\ No newline at end of file
diff --git a/pkg/storage/secret/metadata/data/secure_value_list.sql b/pkg/storage/secret/metadata/data/secure_value_list.sql
index 9c0c604b72d..5d2ecf51e34 100644
--- a/pkg/storage/secret/metadata/data/secure_value_list.sql
+++ b/pkg/storage/secret/metadata/data/secure_value_list.sql
@@ -14,7 +14,11 @@ SELECT
{{ .Ident "ref" }},
{{ .Ident "external_id" }},
{{ .Ident "version" }},
- {{ .Ident "active" }}
+ {{ .Ident "active" }},
+ {{ .Ident "owner_reference_api_group" }},
+ {{ .Ident "owner_reference_api_version" }},
+ {{ .Ident "owner_reference_kind" }},
+ {{ .Ident "owner_reference_name" }}
FROM
{{ .Ident "secret_secure_value" }}
WHERE
diff --git a/pkg/storage/secret/metadata/data/secure_value_read.sql b/pkg/storage/secret/metadata/data/secure_value_read.sql
index b90d54b4a5f..4f6e0ae0707 100644
--- a/pkg/storage/secret/metadata/data/secure_value_read.sql
+++ b/pkg/storage/secret/metadata/data/secure_value_read.sql
@@ -14,7 +14,11 @@ SELECT
{{ .Ident "ref" }},
{{ .Ident "external_id" }},
{{ .Ident "active" }},
- {{ .Ident "version" }}
+ {{ .Ident "version" }},
+ {{ .Ident "owner_reference_api_group" }},
+ {{ .Ident "owner_reference_api_version" }},
+ {{ .Ident "owner_reference_kind" }},
+ {{ .Ident "owner_reference_name" }}
FROM
{{ .Ident "secret_secure_value" }}
WHERE
diff --git a/pkg/storage/secret/metadata/decrypt_store_test.go b/pkg/storage/secret/metadata/decrypt_store_test.go
index 03d0daace36..3c35a443794 100644
--- a/pkg/storage/secret/metadata/decrypt_store_test.go
+++ b/pkg/storage/secret/metadata/decrypt_store_test.go
@@ -292,8 +292,13 @@ func TestIntegrationDecrypt(t *testing.T) {
require.NotEmpty(t, exposed)
require.Equal(t, "value", exposed.DangerouslyExposeAndConsumeValue())
- require.Len(t, fakeLogger.InfoArgs, 1)
- args := fakeLogger.InfoArgs[0]
+ require.Len(t, fakeLogger.InfoMsgs, 2)
+ require.Equal(t, fakeLogger.InfoMsgs[0], "SecureValueMetadataStorage.Read")
+ require.Equal(t, fakeLogger.InfoMsgs[1], "Secrets Audit Log")
+
+ require.Len(t, fakeLogger.InfoArgs, 2)
+ // we only want to check the audit log args
+ args := fakeLogger.InfoArgs[1]
require.Contains(t, args, "grafana_decrypter_identity")
require.Contains(t, args, "decrypter_identity")
for i, arg := range args {
diff --git a/pkg/storage/secret/metadata/metrics/metrics.go b/pkg/storage/secret/metadata/metrics/metrics.go
index aea3817f12c..f9baba1de64 100644
--- a/pkg/storage/secret/metadata/metrics/metrics.go
+++ b/pkg/storage/secret/metadata/metrics/metrics.go
@@ -25,12 +25,8 @@ type StorageMetrics struct {
KeeperMetadataListCount prometheus.Counter
KeeperMetadataGetKeeperConfigDuration prometheus.Histogram
- SecureValueMetadataCreateDuration prometheus.Histogram
- SecureValueMetadataCreateCount prometheus.Counter
- SecureValueMetadataUpdateDuration prometheus.Histogram
- SecureValueMetadataUpdateCount prometheus.Counter
- SecureValueMetadataDeleteDuration prometheus.Histogram
- SecureValueMetadataDeleteCount prometheus.Counter
+ SecureValueMetadataCreateDuration *prometheus.HistogramVec
+ SecureValueMetadataCreateCount *prometheus.CounterVec
SecureValueMetadataGetDuration prometheus.Histogram
SecureValueMetadataGetCount prometheus.Counter
SecureValueMetadataListDuration prometheus.Histogram
@@ -119,45 +115,19 @@ func newStorageMetrics() *StorageMetrics {
}),
// Secure value metrics
- SecureValueMetadataCreateDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
+ SecureValueMetadataCreateDuration: prometheus.NewHistogramVec(prometheus.HistogramOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "secure_value_metadata_create_duration_seconds",
Help: "Duration of secure value metadata create operations",
Buckets: prometheus.DefBuckets,
- }),
- SecureValueMetadataCreateCount: prometheus.NewCounter(prometheus.CounterOpts{
+ }, []string{"successful"}),
+ SecureValueMetadataCreateCount: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "secure_value_metadata_create_count",
Help: "Count of secure value metadata create operations",
- }),
- SecureValueMetadataUpdateDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
- Namespace: namespace,
- Subsystem: subsystem,
- Name: "secure_value_metadata_update_duration_seconds",
- Help: "Duration of secure value metadata update operations",
- Buckets: prometheus.DefBuckets,
- }),
- SecureValueMetadataUpdateCount: prometheus.NewCounter(prometheus.CounterOpts{
- Namespace: namespace,
- Subsystem: subsystem,
- Name: "secure_value_metadata_update_count",
- Help: "Count of secure value metadata update operations",
- }),
- SecureValueMetadataDeleteDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
- Namespace: namespace,
- Subsystem: subsystem,
- Name: "secure_value_metadata_delete_duration_seconds",
- Help: "Duration of secure value metadata delete operations",
- Buckets: prometheus.DefBuckets,
- }),
- SecureValueMetadataDeleteCount: prometheus.NewCounter(prometheus.CounterOpts{
- Namespace: namespace,
- Subsystem: subsystem,
- Name: "secure_value_metadata_delete_count",
- Help: "Count of secure value metadata delete operations",
- }),
+ }, []string{"successful"}),
SecureValueMetadataGetDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
Namespace: namespace,
Subsystem: subsystem,
@@ -241,10 +211,6 @@ func NewStorageMetrics(reg prometheus.Registerer) *StorageMetrics {
m.KeeperMetadataGetKeeperConfigDuration,
m.SecureValueMetadataCreateDuration,
m.SecureValueMetadataCreateCount,
- m.SecureValueMetadataUpdateDuration,
- m.SecureValueMetadataUpdateCount,
- m.SecureValueMetadataDeleteDuration,
- m.SecureValueMetadataDeleteCount,
m.SecureValueMetadataGetDuration,
m.SecureValueMetadataGetCount,
m.SecureValueMetadataListDuration,
diff --git a/pkg/storage/secret/metadata/query_test.go b/pkg/storage/secret/metadata/query_test.go
index 915fd261ec2..2b0c232f2c1 100644
--- a/pkg/storage/secret/metadata/query_test.go
+++ b/pkg/storage/secret/metadata/query_test.go
@@ -177,21 +177,25 @@ func TestSecureValueQueries(t *testing.T) {
Data: &createSecureValue{
SQLTemplate: mocks.NewTestingSQLTemplate(),
Row: &secureValueDB{
- GUID: "abc",
- Name: "name",
- Namespace: "ns",
- Annotations: `{"x":"XXXX"}`,
- Labels: `{"a":"AAA", "b", "BBBB"}`,
- Created: 1234,
- CreatedBy: "user:ryan",
- Updated: 5678,
- UpdatedBy: "user:cameron",
- Version: 1,
- Description: "description",
- Keeper: toNullString(nil),
- Decrypters: toNullString(nil),
- Ref: toNullString(nil),
- ExternalID: "extId",
+ GUID: "abc",
+ Name: "name",
+ Namespace: "ns",
+ Annotations: `{"x":"XXXX"}`,
+ Labels: `{"a":"AAA", "b", "BBBB"}`,
+ Created: 1234,
+ CreatedBy: "user:ryan",
+ Updated: 5678,
+ UpdatedBy: "user:cameron",
+ Version: 1,
+ Description: "description",
+ Keeper: toNullString(nil),
+ Decrypters: toNullString(nil),
+ Ref: toNullString(nil),
+ ExternalID: "extId",
+ OwnerReferenceAPIGroup: toNullString(nil),
+ OwnerReferenceAPIVersion: toNullString(nil),
+ OwnerReferenceKind: toNullString(nil),
+ OwnerReferenceName: toNullString(nil),
},
},
},
@@ -200,21 +204,25 @@ func TestSecureValueQueries(t *testing.T) {
Data: &createSecureValue{
SQLTemplate: mocks.NewTestingSQLTemplate(),
Row: &secureValueDB{
- GUID: "abc",
- Name: "name",
- Namespace: "ns",
- Annotations: `{"x":"XXXX"}`,
- Labels: `{"a":"AAA", "b", "BBBB"}`,
- Created: 1234,
- CreatedBy: "user:ryan",
- Updated: 5678,
- UpdatedBy: "user:cameron",
- Version: 1,
- Description: "description",
- Keeper: toNullString(ptr.To("keeper_test")),
- Decrypters: toNullString(ptr.To("decrypters_test")),
- Ref: toNullString(ptr.To("ref_test")),
- ExternalID: "extId",
+ GUID: "abc",
+ Name: "name",
+ Namespace: "ns",
+ Annotations: `{"x":"XXXX"}`,
+ Labels: `{"a":"AAA", "b", "BBBB"}`,
+ Created: 1234,
+ CreatedBy: "user:ryan",
+ Updated: 5678,
+ UpdatedBy: "user:cameron",
+ Version: 1,
+ Description: "description",
+ Keeper: toNullString(ptr.To("keeper_test")),
+ Decrypters: toNullString(ptr.To("decrypters_test")),
+ Ref: toNullString(ptr.To("ref_test")),
+ ExternalID: "extId",
+ OwnerReferenceAPIGroup: toNullString(ptr.To("prometheus.datasource.grafana.app")),
+ OwnerReferenceAPIVersion: toNullString(ptr.To("v0alpha1")),
+ OwnerReferenceKind: toNullString(ptr.To("DataSource")),
+ OwnerReferenceName: toNullString(ptr.To("prom-config")),
},
},
},
diff --git a/pkg/storage/secret/metadata/secure_value_model.go b/pkg/storage/secret/metadata/secure_value_model.go
index 6e5c8511784..c64411bf19b 100644
--- a/pkg/storage/secret/metadata/secure_value_model.go
+++ b/pkg/storage/secret/metadata/secure_value_model.go
@@ -10,22 +10,26 @@ import (
secretv1beta1 "github.com/grafana/grafana/apps/secret/pkg/apis/secret/v1beta1"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/registry/apis/secret/xkube"
- "github.com/grafana/grafana/pkg/storage/secret/migrator"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+ "k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/types"
)
type secureValueDB struct {
// Kubernetes Metadata
- GUID string
- Name string
- Namespace string
- Annotations string // map[string]string
- Labels string // map[string]string
- Created int64
- CreatedBy string
- Updated int64
- UpdatedBy string
+ GUID string
+ Name string
+ Namespace string
+ Annotations string // map[string]string
+ Labels string // map[string]string
+ Created int64
+ CreatedBy string
+ Updated int64
+ UpdatedBy string
+ OwnerReferenceAPIGroup sql.NullString
+ OwnerReferenceAPIVersion sql.NullString
+ OwnerReferenceKind sql.NullString
+ OwnerReferenceName sql.NullString
// Kubernetes Status
Active bool
@@ -39,10 +43,6 @@ type secureValueDB struct {
ExternalID string
}
-func (*secureValueDB) TableName() string {
- return migrator.TableNameSecureValue
-}
-
// toKubernetes maps a DB row into a Kubernetes resource (metadata + spec).
func (sv *secureValueDB) toKubernetes() (*secretv1beta1.SecureValue, error) {
annotations := make(map[string]string, 0)
@@ -85,8 +85,6 @@ func (sv *secureValueDB) toKubernetes() (*secretv1beta1.SecureValue, error) {
resource.Spec.Ref = &sv.Ref.String
}
- resource.Status.ExternalID = sv.ExternalID
-
// Set all meta fields here for consistency.
meta, err := utils.MetaAccessor(resource)
if err != nil {
@@ -106,6 +104,20 @@ func (sv *secureValueDB) toKubernetes() (*secretv1beta1.SecureValue, error) {
meta.SetUpdatedTimestamp(&updated)
meta.SetResourceVersionInt64(sv.Updated)
+ hasOwnerReference := sv.OwnerReferenceAPIGroup.Valid && sv.OwnerReferenceAPIGroup.String != "" &&
+ sv.OwnerReferenceAPIVersion.Valid && sv.OwnerReferenceAPIVersion.String != "" &&
+ sv.OwnerReferenceKind.Valid && sv.OwnerReferenceKind.String != "" &&
+ sv.OwnerReferenceName.Valid && sv.OwnerReferenceName.String != ""
+ if hasOwnerReference {
+ meta.SetOwnerReferences([]metav1.OwnerReference{
+ {
+ APIVersion: schema.GroupVersion{Group: sv.OwnerReferenceAPIGroup.String, Version: sv.OwnerReferenceAPIVersion.String}.String(),
+ Kind: sv.OwnerReferenceKind.String,
+ Name: sv.OwnerReferenceName.String,
+ },
+ })
+ }
+
return resource, nil
}
@@ -179,16 +191,48 @@ func toRow(sv *secretv1beta1.SecureValue, externalID string) (*secureValueDB, er
return nil, fmt.Errorf("failed to get resource version: %w", err)
}
+ var (
+ ownerReferenceAPIGroup sql.NullString
+ ownerReferenceAPIVersion sql.NullString
+ ownerReferenceKind sql.NullString
+ ownerReferenceName sql.NullString
+ )
+
+ ownerReferences := meta.GetOwnerReferences()
+ if len(ownerReferences) > 1 {
+ return nil, fmt.Errorf("only one owner reference is supported, found %d", len(ownerReferences))
+ }
+ if len(ownerReferences) == 1 {
+ ownerReference := ownerReferences[0]
+
+ gv, err := schema.ParseGroupVersion(ownerReference.APIVersion)
+ if err != nil {
+ return nil, fmt.Errorf("failed to parse owner reference API version %s: %w", ownerReference.APIVersion, err)
+ }
+ if gv.Group == "" {
+ return nil, fmt.Errorf("malformed api version %s requires User Search] + B[Dashboard Service
Resource Searches] + C[Folder Service
Resource Searches] + D[Alerting Service
Resource Searches] + E[Provisioning Service
Resource Searches] + F[API Endpoints
Search Operations] + end + + subgraph "API Gateway Layer" + G[Grafana API Server
Search Endpoint] + H[Search Client
Dual Writer Aware] + end + + subgraph "Routing Decision" + I{Dual Writer Mode
Check} + end + + subgraph "Unified Search Path (Mode 3+)" + J[Search Distributor
Ring-based Routing] + K[Ring
Consistent Hashing] + L[Search API Server 1
Namespace Sharding
+ Embedded Bleve Backend] + M[Search API Server 2
Namespace Sharding
+ Embedded Bleve Backend] + N[Search API Server 3
Namespace Sharding
+ Embedded Bleve Backend] + O[Unified Storage
K8s-style Resources] + end + + subgraph "Legacy Search Path (Mode 0-2)" + Q[Legacy Search Service
SQL-based Search] + R[Legacy Storage
Traditional Tables] + S[Shadow Traffic
Mode 1-2 + Flag Enabled] + end + + A --> G + B --> H + C --> H + D --> H + E --> H + F --> G + G --> H + H --> I + + I -->|Mode 3+| J + I -->|Mode 0-2| Q + + J --> K + K --> L + K --> M + K --> N + L -.->|Just-in-Time
Indexing| O + M -.->|Just-in-Time
Indexing| O + N -.->|Just-in-Time
Indexing| O + + Q --> R + Q -.->|Shadow Traffic
Mode 1-2 + Flag| S + S -.-> J +``` + +### Search Backend Routing + +The search client routes requests based on the dual writer mode configuration for each resource type: + +#### Dual Writer Mode → Backend Routing +- **Mode 0-2**: Route to **Legacy Search** + - Mode 1-2: Shadow traffic to Unified Search (if `unifiedStorageSearchDualReaderEnabled` is enabled) + - Mode 0: No shadow traffic +- **Mode 3+**: Route to **Unified Search** + - No shadow traffic needed (unified is primary) + +### Feature Flags + +Unified Search requires several feature flags to be enabled depending on the desired functionality: + +#### Prerequisites (Required for Unified Storage) + +| Feature Flag | Purpose | Stage | Required For | +|--------------|---------|-------|--------------| +| `grafanaAPIServerWithExperimentalAPIs` | Allow experimental API groups | Development | Access to v0alpha1 APIs (including search) | + +#### Unified Search Specific Flags + +| Feature Flag | Purpose | Stage | Required For | +|--------------|---------|-------|--------------| +| `unifiedStorageSearch` | Core search functionality | Experimental | Search API servers, indexing | +| `unifiedStorageSearchUI` | Frontend search interface | Experimental | Grafana UI search | +| `unifiedStorageSearchPermissionFiltering` | User permission filtering | GA | Access control in search results | +| `unifiedStorageSearchSprinkles` | Usage insights integration | Experimental | Dashboard usage sorting (Enterprise) | +| `unifiedStorageSearchDualReaderEnabled` | Shadow traffic to unified search | Experimental | Shadow traffic during migration | + +#### Basic Configuration +```ini +[feature_toggles] +; Prerequisites for unified storage (required) +grafanaAPIServerWithExperimentalAPIs = true + +; Core search functionality (required) +unifiedStorageSearch = true + +; Enable search UI (required for frontend) +unifiedStorageSearchUI = true + +; Enable permission filtering (recommended) +unifiedStorageSearchPermissionFiltering = true + +; Enable shadow traffic during migration (optional) +unifiedStorageSearchDualReaderEnabled = true + +; Enable usage insights sorting (Enterprise only) +unifiedStorageSearchSprinkles = true +``` + +### Request Flow Diagrams + +#### Search Request Flow with Dual Writer Mode Routing + +Search requests originate from multiple sources, and the search client routes based on dual writer mode configuration: + +```mermaid +flowchart TD + A[Search Request] --> B{Dual Writer Mode} + B -->|Mode 3+| C[Unified Search] + B -->|Mode 0-2| D[Legacy Search] + C --> E[Return Results] + D --> E +``` + +#### Search Request Flow with Shadow Traffic + +When `unifiedStorageSearchDualReaderEnabled` is enabled and resource is in dual writer Mode 1-2 (legacy primary), shadow traffic is generated: + +```mermaid +flowchart TD + A[Search Request] --> B{Shadow Traffic Enabled?} + B -->|Yes| C[Primary: Legacy Search] + B -->|No| D[Single Search Path] + C --> E[Background: Unified Search] + C --> F[Return Legacy Results] + E --> G[Log Results for Comparison] + D --> H[Return Results] +``` + +### Distributor Architecture + +The Search Distributor acts as a smart proxy that routes search requests to the appropriate search API server based on namespace hashing: + +```mermaid +flowchart TD + A[Incoming Search Request] --> B[Hash Namespace] + B --> C[Select Random Instance] + C --> D[Proxy Request] + D --> E[Return Response] +``` + +#### Key Features: +- **Namespace-based routing**: Each request is routed based on the target namespace +- **Load balancing**: Random selection among available replicas for the namespace +- **Health awareness**: Only routes to `ACTIVE` ring instances +- **Connection pooling**: Reuses gRPC connections for efficiency +- **Proxy headers**: Adds metadata for debugging and tracing + +### Ring Architecture + +The hash ring provides consistent, distributed assignment of namespaces to search API servers: + +```mermaid +flowchart TD + A[Namespace] --> B[Hash Function] + B --> C[Ring Position] + C --> D[Assigned Instance] + D --> E[Search Processing] +``` + +#### Ring Properties: +- **Consistent hashing**: Uses FNV32 hash function for namespace distribution +- **128 tokens per instance**: Provides good distribution across the ring +- **Replication factor**: Configurable redundancy (default based on cluster size) +- **State management**: Instances transition through JOINING → ACTIVE → LEAVING +- **Automatic rebalancing**: Ring adjusts when instances join/leave + +### Namespace-Based Sharding + +Unified Search uses namespace-based sharding to distribute search indexes across multiple search API servers: + +```mermaid +flowchart LR + A[Namespaces] --> B[Hash Ring] + B --> C[Search Server 1] + B --> D[Search Server 2] + B --> E[Search Server 3] + C --> F[Indexes for Assigned Namespaces] + D --> G[Indexes for Assigned Namespaces] + E --> H[Indexes for Assigned Namespaces] +``` + +#### Sharding Benefits: +1. **Horizontal scalability**: Add more search servers to handle more namespaces +2. **Resource isolation**: Each namespace's index is independent +3. **Parallel processing**: Multiple searches can run simultaneously across different servers +4. **Fault tolerance**: Namespace availability depends only on its assigned server(s) + +#### Sharding Algorithm: +```go +func getSearchServer(namespace string) string { + hash := fnv.New32a() + hash.Write([]byte(namespace)) + + // Get replication set from ring + replicationSet := ring.GetWithOptions( + hash.Sum32(), + searchRingRead, + ring.WithReplicationFactor(ring.ReplicationFactor()) + ) + + // Random load balancing within replication set + instance := replicationSet.Instances[rand.Intn(len(replicationSet.Instances))] + return instance.Id +} +``` + +### Search Index Management + +Each search API server contains an embedded Bleve search engine that manages indexes for its assigned namespaces: + +```mermaid +flowchart TD + A[Search Request] --> B{Index Ready?} + B -->|Yes| C[Query Index] + B -->|No| D[Build Index] + D --> E[Fetch Resources] + E --> F[Create Index] + F --> C + C --> G[Return Results] + + H[Resource Changes] --> I[Update Index] + I --> F +``` + +#### Index Architecture Details: + +**Embedded Bleve Backend**: Each Search API Server contains its own Bleve search engine instance, not a shared external service. + +**Just-in-Time Indexing**: When a search request arrives for a namespace that doesn't have an index (or has an outdated index): +1. The Search API Server fetches all resources for that namespace from Unified Storage +2. Builds search documents in memory +3. Creates either a memory-based or disk-based Bleve index depending on size +4. Executes the search query against the newly built index +5. Returns results to the user + +**Index Storage Strategy**: +- **Memory indexes**: For small datasets (< `index_file_threshold` documents) +- **Disk indexes**: For large datasets (≥ `index_file_threshold` documents) +- Indexes are stored per Search API Server instance, not globally shared + +**Background Updates**: In addition to just-in-time indexing, Search API Servers also maintain indexes through background watch events for incremental updates. + +#### Index Configuration: +```ini +[unified_storage] +; Path for disk-based search indexes +index_path = /var/lib/grafana/unified-search/bleve + +; Threshold for file-based vs memory indexes +index_file_threshold = 1000 + +; Maximum batch size for indexing +index_max_batch_size = 100 + +; Number of worker threads for indexing +index_workers = 4 + +; Cache TTL for indexes +index_cache_ttl = 1h + +; Periodic rebuild interval (for usage insights) +index_rebuild_interval = 24h + +; Minimum resource count required to build an index (default: 1) +; If a namespace has fewer resources than this threshold, no index will be created +index_min_count = 1 + +; Maximum resource count before creating an empty index (default: 0 = no limit) +; When exceeded, creates an empty index instead of indexing all resources for performance +index_max_count = 0 +``` + +### Search Request Sources + +Unified Search serves multiple types of consumers within the Grafana ecosystem: + +#### 1. User-Initiated Searches +- **Source**: Grafana UI search interface +- **Purpose**: Interactive dashboard and folder discovery +- **Characteristics**: Real-time, user-facing, latency-sensitive +- **Endpoint**: `/api/v1/search` (legacy search UI) or `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search` (when `unifiedStorageSearchUI` is enabled) + +#### 2. Internal Service Searches + +Internal services use different search backends depending on dual writer mode configuration: + +- **Dashboard Service**: + - Find related dashboards based on tags, folders, or content + - Discover dashboards for playlist creation + - Validate dashboard references during operations + - **Backend**: Depends on dashboard dual writer mode (Legacy for Mode 0-2, Unified for Mode 3+) + +- **Folder Service**: + - Retrieve folder contents and nested structures + - Resolve folder hierarchy relationships + - Check folder permissions and accessibility + - **Backend**: Depends on folder dual writer mode (Legacy for Mode 0-2, Unified for Mode 3+) + +- **Alerting Service**: + - Discover dashboards and panels for alert rule creation + - Find existing alert rules across namespaces + - Resolve dashboard/panel references in alert definitions + - **Backend**: Mixed - dashboard searches use dashboard dual writer mode, alert rule searches typically use legacy + +- **Provisioning Service**: + - Check for existing resources before provisioning + - Validate resource uniqueness and naming conflicts + - Discover resources for bulk operations + - **Backend**: Depends on each resource type's dual writer mode configuration + +- **API Services**: + - Backend support for various API endpoints + - Resource validation and dependency checking + - **Backend**: Routes based on resource type's dual writer mode + +#### 3. Search Operation Types + +Unified Search supports multiple types of search operations: + +##### Resource Search +- **Purpose**: Find resources (dashboards, folders, etc.) by content +- **Endpoint**: `/api/v1/search` (legacy) or `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search` (when `unifiedStorageSearchUI` is enabled) +- **Additional endpoint**: `/apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search/sortable` for retrieving sortable fields +- **Features**: Full-text search, filtering, sorting + +**Sortable Fields:** + +The `/search/sortable` endpoint currently returns a limited static list: +```json +{ + "fields": [ + {"field": "title", "display": "Title (A-Z)", "type": "string"}, + {"field": "-title", "display": "Title (Z-A)", "type": "string"} + ] +} +``` + +However, the search backend actually supports sorting by many more fields: + +**Standard Fields:** +- `title` - Resource display name (uses `title_phrase` for exact sorting) +- `name` - Kubernetes resource name +- `description` - Resource description +- `folder` - Parent folder name +- `created` - Creation timestamp (int64) +- `updated` - Last update timestamp (int64) +- `createdBy` - Creator user ID +- `updatedBy` - Last updater user ID +- `tags` - Resource tags (array) +- `rv` - Resource version (int64) + +**Dashboard-Specific Fields** (require `fields.` prefix): +- `fields.schema_version` - Dashboard schema version +- `fields.link_count` - Number of dashboard links +- `fields.panel_types` - Panel types used in dashboard +- `fields.ds_types` - Data source types used +- `fields.transformation` - Transformations used + +**Usage Insights Fields** (Enterprise only, require `fields.` prefix): +- `fields.views_total` - Total dashboard views +- `fields.views_last_1_days` / `fields.views_last_7_days` / `fields.views_last_30_days` - Recent views +- `fields.views_today` - Today's views +- `fields.queries_total` - Total queries executed +- `fields.queries_last_1_days` / `fields.queries_last_7_days` / `fields.queries_last_30_days` - Recent queries +- `fields.queries_today` - Today's queries +- `fields.errors_total` - Total errors +- `fields.errors_last_1_days` / `fields.errors_last_7_days` / `fields.errors_last_30_days` - Recent errors +- `fields.errors_today` - Today's errors + +**Usage Examples:** +```bash +# Sort by title (ascending) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=title + +# Sort by creation date (descending) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=-created + +# Sort by usage insights (Enterprise) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?sortBy=-fields.views_total +``` + +*Note: There's currently a discrepancy between the limited fields exposed by `/search/sortable` and the full range of fields actually supported by the search backend.* + +##### Federated Search +- **Purpose**: Search across multiple resource types simultaneously +- **Features**: Cross-resource queries, unified result ranking, combined sorting and faceting +- **Implementation**: Uses Bleve IndexAlias to combine multiple indexes for unified searching +- **Default behavior**: When no type is specified, automatically federates dashboards and folders + +**How Federated Search Works:** + +Federated search is implemented using Bleve's IndexAlias feature, which allows searching across multiple indexes as if they were a single unified index. This enables: + +1. **Cross-resource queries**: Search for content across dashboards, folders, and other resource types +2. **Unified sorting**: Results from different resource types are merged and sorted together +3. **Combined faceting**: Aggregate facet statistics across all federated resource types +4. **Permission filtering**: Respects user permissions for each resource type independently + +**API Usage Examples:** + +**1. Default Federation (Dashboards + Folders):** +```bash +# When no type is specified, automatically searches dashboards and folders +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?query=my-search +``` + +**2. Single Resource Type Search:** +```bash +# Search only folders (despite the "dashboard" API group, type parameter controls what's searched) +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=folders&query=my-search + +# Search only dashboards +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=dashboards&query=my-search +``` + +**3. Explicit Two-Type Federation:** +```bash +# Search dashboards (primary) with folders federated +GET /apis/dashboard.grafana.app/v0alpha1/namespaces/{namespace}/search?type=dashboards&type=folders&query=my-search +``` + +**4. Protocol Buffer Request Structure:** +```protobuf +message ResourceSearchRequest { + ListOptions options = 1; // Primary resource type to search + repeated ResourceKey federated = 2; // Additional resource types to federate + string query = 3; // Search query applied across all types + // ... other fields +} +``` + +**Example gRPC/Protocol Buffer Usage:** +```go +searchRequest := &resourcepb.ResourceSearchRequest{ + Options: &resourcepb.ListOptions{ + Key: dashboardKey, // Primary: search dashboards + }, + Federated: []*resourcepb.ResourceKey{ + folderKey, // Also search folders + }, + Query: "monitoring", + Limit: 50, + SortBy: []*resourcepb.ResourceSearchRequest_Sort{ + {Field: "title", Desc: false}, // Sort combined results by title + }, +} +``` + +**5. Unified Results:** +Federated search returns a single result set containing resources from all specified types, with: +- **Unified ranking**: All results scored and ranked together +- **Cross-type sorting**: Resources from different types sorted by common fields (title, tags, etc.) +- **Resource type identification**: Each result includes metadata indicating its resource type +- **Permission-aware filtering**: Only returns resources the user has permission to see + +**Limitations:** +- Federation only works across resource types with **common fields** (title, tags, folder, etc.) +- All federated indexes must be of the same search backend type (currently Bleve) +- Currently supports up to 2 resource types in federation via the API endpoint +- **Architectural note**: The search endpoint is under `dashboard.grafana.app` but can search any resource type via the `type` parameter - this is a design choice where the "dashboard search" has evolved into a generic search endpoint + +##### Managed Objects +- **Purpose**: Administrative queries for resource management +- **Operations**: Count, list, statistics + +##### Stats and Monitoring +- **Purpose**: Index health and performance metrics +- **Metrics**: Document counts, index sizes, search latency + + +### Monitoring and Observability + +Key metrics for monitoring Unified Search: + +- `unified_search_requests_total`: Search request counts by type and status +- `unified_search_request_duration_seconds`: Search request latency +- `unified_search_index_size_bytes`: Size of search indexes +- `unified_search_documents_total`: Number of indexed documents +- `unified_search_indexing_duration_seconds`: Time to build/update indexes +- `unified_search_shadow_requests_total`: Shadow traffic request counts +- `unified_search_ring_members`: Number of active search server instances + + diff --git a/pkg/storage/unified/resource/bleve_index_metrics.go b/pkg/storage/unified/resource/bleve_index_metrics.go index 73a18d4fabe..3f130eb0b1b 100644 --- a/pkg/storage/unified/resource/bleve_index_metrics.go +++ b/pkg/storage/unified/resource/bleve_index_metrics.go @@ -9,11 +9,14 @@ import ( ) type BleveIndexMetrics struct { - IndexLatency *prometheus.HistogramVec - IndexSize prometheus.Gauge - IndexedKinds *prometheus.GaugeVec - IndexCreationTime *prometheus.HistogramVec - OpenIndexes *prometheus.GaugeVec + IndexLatency *prometheus.HistogramVec + IndexSize prometheus.Gauge + IndexedKinds *prometheus.GaugeVec + IndexCreationTime *prometheus.HistogramVec + OpenIndexes *prometheus.GaugeVec + IndexBuilds *prometheus.CounterVec + IndexBuildFailures prometheus.Counter + IndexBuildSkipped prometheus.Counter } var IndexCreationBuckets = []float64{1, 5, 10, 25, 50, 75, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000} @@ -37,8 +40,8 @@ func ProvideIndexMetrics(reg prometheus.Registerer) *BleveIndexMetrics { Help: "Number of indexed documents by kind", }, []string{"kind"}), IndexCreationTime: promauto.With(reg).NewHistogramVec(prometheus.HistogramOpts{ - Name: "index_server_index_creation_time_seconds", - Help: "Time (in seconds) it takes until index is created", + Name: "index_server_index_build_time_seconds", + Help: "Time it takes to successfully build an index. Failed or skipped builds are not counted.", Buckets: IndexCreationBuckets, NativeHistogramBucketFactor: 1.1, // enable native histograms NativeHistogramMaxBucketNumber: 160, @@ -48,11 +51,22 @@ func ProvideIndexMetrics(reg prometheus.Registerer) *BleveIndexMetrics { Name: "index_server_open_indexes", Help: "Number of open indexes per storage type. An open index corresponds to single resource group.", }, []string{"index_storage"}), // index_storage is either "file" or "memory" + IndexBuilds: promauto.With(reg).NewCounterVec(prometheus.CounterOpts{ + Name: "index_server_index_build_total", + Help: "Number of times index build was attempted due to specific reason", + }, []string{"reason"}), + IndexBuildFailures: promauto.With(reg).NewCounter(prometheus.CounterOpts{ + Name: "index_server_index_build_failures_total", + Help: "Number of times index build failed", + }), + IndexBuildSkipped: promauto.With(reg).NewCounter(prometheus.CounterOpts{ + Name: "index_server_index_build_skipped_total", + Help: "Number of times index build has been skipped due to existing valid index being found on disk", + }), } // Initialize labels. m.OpenIndexes.WithLabelValues("file").Set(0) m.OpenIndexes.WithLabelValues("memory").Set(0) - return m } diff --git a/pkg/storage/unified/resource/bulk.go b/pkg/storage/unified/resource/bulk.go index f1dbb3131c2..c3f2e61e290 100644 --- a/pkg/storage/unified/resource/bulk.go +++ b/pkg/storage/unified/resource/bulk.go @@ -244,7 +244,7 @@ func (s *server) BulkProcess(stream resourcepb.BulkStore_BulkProcessServer) erro Namespace: summary.Namespace, Group: summary.Group, Resource: summary.Resource, - }, summary.Count, summary.ResourceVersion) + }, summary.Count, summary.ResourceVersion, "rebuildAfterBatchLoad") if err != nil { s.log.Warn("error building search index after batch load", "err", err) rsp.Error = &resourcepb.ErrorResult{ diff --git a/pkg/storage/unified/resource/search.go b/pkg/storage/unified/resource/search.go index bda857993ba..4752fa35cec 100644 --- a/pkg/storage/unified/resource/search.go +++ b/pkg/storage/unified/resource/search.go @@ -90,7 +90,7 @@ type SearchBackend interface { // Depending on the size, the backend may choose different options (eg: memory vs disk). // The last known resource version can be used to detect that nothing has changed, and existing on-disk index can be reused. // The builder will write all documents before returning. - BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, nonStandardFields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) + BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, nonStandardFields SearchableDocumentFields, indexBuildReason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) // TotalDocs returns the total number of documents across all indexes. TotalDocs() int64 @@ -196,7 +196,7 @@ func (s *searchSupport) ListManagedObjects(ctx context.Context, req *resourcepb. Namespace: req.Namespace, Group: info.Group, Resource: info.Resource, - }) + }, "listManagedObjects") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -237,7 +237,7 @@ func (s *searchSupport) CountManagedObjects(ctx context.Context, req *resourcepb Namespace: req.Namespace, Group: info.Group, Resource: info.Resource, - }) + }, "countManagedObjects") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -282,7 +282,7 @@ func (s *searchSupport) Search(ctx context.Context, req *resourcepb.ResourceSear Namespace: req.Options.Key.Namespace, Resource: req.Options.Key.Resource, } - idx, err := s.getOrCreateIndex(ctx, nsr) + idx, err := s.getOrCreateIndex(ctx, nsr, "search") if err != nil { return &resourcepb.ResourceSearchResponse{ Error: AsErrorResult(err), @@ -294,7 +294,7 @@ func (s *searchSupport) Search(ctx context.Context, req *resourcepb.ResourceSear for i, f := range req.Federated { nsr.Group = f.Group nsr.Resource = f.Resource - federate[i], err = s.getOrCreateIndex(ctx, nsr) + federate[i], err = s.getOrCreateIndex(ctx, nsr, "federatedSearch") if err != nil { return &resourcepb.ResourceSearchResponse{ Error: AsErrorResult(err), @@ -323,7 +323,7 @@ func (s *searchSupport) GetStats(ctx context.Context, req *resourcepb.ResourceSt Namespace: req.Namespace, Group: parts[0], Resource: parts[1], - }) + }, "getStats") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -367,7 +367,7 @@ func (s *searchSupport) GetStats(ctx context.Context, req *resourcepb.ResourceSt Namespace: req.Namespace, Group: stat.Group, Resource: stat.Resource, - }) + }, "getStats") if err != nil { rsp.Error = AsErrorResult(err) return rsp, nil @@ -449,8 +449,12 @@ func (s *searchSupport) buildIndexes(ctx context.Context, rebuild bool) (int, er return err } - s.log.Debug("building index", "namespace", info.Namespace, "group", info.Group, "resource", info.Resource) - _, _, err := s.build(ctx, info.NamespacedResource, info.Count, info.ResourceVersion) + s.log.Debug("building index", "namespace", info.Namespace, "group", info.Group, "resource", info.Resource, "rebuild", rebuild) + reason := "init" + if rebuild { + reason = "rebuild" + } + _, _, err := s.build(ctx, info.NamespacedResource, info.Count, info.ResourceVersion, reason) return err }) } @@ -504,9 +508,6 @@ func (s *searchSupport) init(ctx context.Context) error { end := time.Now().Unix() s.log.Info("search index initialized", "duration_secs", end-start, "total_docs", s.search.TotalDocs()) - if s.indexMetrics != nil { - s.indexMetrics.IndexCreationTime.WithLabelValues().Observe(float64(end - start)) - } return nil } @@ -537,7 +538,7 @@ func (s *searchSupport) dispatchEvent(ctx context.Context, evt *WrittenEvent) { Group: evt.Key.Group, Resource: evt.Key.Resource, } - index, err := s.getOrCreateIndex(ctx, nsr) + index, err := s.getOrCreateIndex(ctx, nsr, "dispatchEvent") if err != nil { s.log.Warn("error getting index for watch event", "error", err) span.RecordError(err) @@ -622,15 +623,10 @@ func (s *searchSupport) rebuildDashboardIndexes(ctx context.Context) error { "duration", duration, "rebuilt_indexes", totalBatchesIndexed, "total_docs", s.search.TotalDocs()) - - if s.indexMetrics != nil { - s.indexMetrics.IndexCreationTime.WithLabelValues().Observe(duration.Seconds()) - } - return nil } -func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedResource) (ResourceIndex, error) { +func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedResource, reason string) (ResourceIndex, error) { if s == nil || s.search == nil { return nil, fmt.Errorf("search is not configured properly (missing unifiedStorageSearch feature toggle?)") } @@ -648,6 +644,10 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } ch := s.buildIndex.DoChan(key.String(), func() (interface{}, error) { + // We want to finish building of the index even if original context is canceled. + // We reuse original context without cancel to keep the tracing spans correct. + ctx := context.WithoutCancel(ctx) + // Recheck if some other goroutine managed to build an index in the meantime. // (That is, it finished running this function and stored the index into the cache) idx, err := s.search.GetIndex(ctx, key) @@ -672,7 +672,7 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } } - idx, _, err = s.build(ctx, key, size, rv) + idx, _, err = s.build(ctx, key, size, rv, reason) if err != nil { return nil, fmt.Errorf("error building search index, %w", err) } @@ -693,10 +693,18 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso } } -func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size int64, rv int64) (ResourceIndex, int64, error) { +func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size int64, rv int64, indexBuildReason string) (ResourceIndex, int64, error) { ctx, span := s.tracer.Start(ctx, tracingPrexfixSearch+"Build") defer span.End() + span.SetAttributes( + attribute.String("namespace", nsr.Namespace), + attribute.String("group", nsr.Group), + attribute.String("resource", nsr.Resource), + attribute.Int64("size", size), + attribute.Int64("rv", rv), + ) + logger := s.log.With("namespace", nsr.Namespace, "group", nsr.Group, "resource", nsr.Resource) builder, err := s.builders.get(ctx, nsr) @@ -705,7 +713,10 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size } fields := s.builders.GetFields(nsr) - index, err := s.search.BuildIndex(ctx, nsr, size, rv, fields, func(index ResourceIndex) (int64, error) { + index, err := s.search.BuildIndex(ctx, nsr, size, rv, fields, indexBuildReason, func(index ResourceIndex) (int64, error) { + span := trace.SpanFromContext(ctx) + span.AddEvent("building index", trace.WithAttributes(attribute.Int64("size", size), attribute.Int64("rv", rv), attribute.String("reason", indexBuildReason))) + rv, err = s.storage.ListIterator(ctx, &resourcepb.ListRequest{ Limit: 1000000000000, // big number Options: &resourcepb.ListOptions{ @@ -734,9 +745,11 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size Name: iter.Name(), } + span.AddEvent("building document", trace.WithAttributes(attribute.String("name", iter.Name()))) // Convert it to an indexable document doc, err := builder.BuildDocument(ctx, key, iter.ResourceVersion(), iter.Value()) if err != nil { + span.RecordError(err) logger.Error("error building search document", "key", SearchID(key), "err", err) continue } @@ -749,6 +762,7 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size // When we reach the batch size, perform bulk index and reset the batch. if len(items) >= maxBatchSize { + span.AddEvent("bulk indexing", trace.WithAttributes(attribute.Int("count", len(items)))) if err = index.BulkIndex(&BulkIndexRequest{ Items: items, }); err != nil { @@ -762,6 +776,7 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size // Index any remaining items in the final batch. if len(items) > 0 { + span.AddEvent("bulk indexing", trace.WithAttributes(attribute.Int("count", len(items)))) if err = index.BulkIndex(&BulkIndexRequest{ Items: items, }); err != nil { @@ -799,7 +814,7 @@ func (s *searchSupport) buildEmptyIndex(ctx context.Context, nsr NamespacedResou s.log.Debug("Building empty index", "namespace", nsr.Namespace, "group", nsr.Group, "resource", nsr.Resource, "rv", rv) // Build an empty index by passing a builder function that doesn't add any documents - return s.search.BuildIndex(ctx, nsr, 0, rv, fields, func(index ResourceIndex) (int64, error) { + return s.search.BuildIndex(ctx, nsr, 0, rv, fields, "empty", func(index ResourceIndex) (int64, error) { // Return the resource version without adding any documents to the index return rv, nil }) diff --git a/pkg/storage/unified/resource/search_test.go b/pkg/storage/unified/resource/search_test.go index 07a9a464ae1..677e24b8b8b 100644 --- a/pkg/storage/unified/resource/search_test.go +++ b/pkg/storage/unified/resource/search_test.go @@ -121,7 +121,7 @@ func (m *mockSearchBackend) GetIndex(ctx context.Context, key NamespacedResource return nil, nil } -func (m *mockSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { +func (m *mockSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, reason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { index := &MockResourceIndex{} index.On("BulkIndex", mock.Anything).Return(nil).Maybe() index.On("DocCount", mock.Anything, mock.Anything).Return(int64(0), nil).Maybe() @@ -317,7 +317,7 @@ func TestSearchGetOrCreateIndex(t *testing.T) { go func() { defer wg.Done() <-start - _, _ = support.getOrCreateIndex(context.Background(), NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}) + _, _ = support.getOrCreateIndex(context.Background(), NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}, "test") }() } @@ -340,7 +340,7 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { {NamespacedResource: NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}, Count: 50, ResourceVersion: 11111111}, }, } - search := &slowSearchBackend{ + search := &slowSearchBackendWithCache{ mockSearchBackend: mockSearchBackend{}, } supplier := &TestDocumentBuilderSupplier{ @@ -362,10 +362,12 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { require.NoError(t, err) require.NotNil(t, support) + key := NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"} + ctx, cancel := context.WithTimeout(context.Background(), 1*time.Millisecond) defer cancel() - _, err = support.getOrCreateIndex(ctx, NamespacedResource{Namespace: "ns", Group: "group", Resource: "resource"}) + _, err = support.getOrCreateIndex(ctx, key, "test") // Make sure we get context deadline error require.ErrorIs(t, err, context.DeadlineExceeded) @@ -373,16 +375,53 @@ func TestSearchGetOrCreateIndexWithCancellation(t *testing.T) { search.wg.Wait() require.NotEmpty(t, search.buildIndexCalls) + + // Wait until new index is put into cache. + require.Eventually(t, func() bool { + idx, err := support.search.GetIndex(ctx, key) + return err == nil && idx != nil + }, 1*time.Second, 100*time.Millisecond, "Indexing finishes despite context cancellation") + + // Second call to getOrCreateIndex returns index immediately, even if context is canceled, as the index is now ready and cached. + _, err = support.getOrCreateIndex(ctx, key, "test") + require.NoError(t, err) } -type slowSearchBackend struct { +type slowSearchBackendWithCache struct { mockSearchBackend wg sync.WaitGroup + + mu sync.Mutex + cache map[NamespacedResource]ResourceIndex } -func (m *slowSearchBackend) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { +func (m *slowSearchBackendWithCache) GetIndex(ctx context.Context, key NamespacedResource) (ResourceIndex, error) { + m.mu.Lock() + defer m.mu.Unlock() + return m.cache[key], nil +} + +func (m *slowSearchBackendWithCache) BuildIndex(ctx context.Context, key NamespacedResource, size int64, resourceVersion int64, fields SearchableDocumentFields, reason string, builder func(index ResourceIndex) (int64, error)) (ResourceIndex, error) { m.wg.Add(1) defer m.wg.Done() + time.Sleep(1 * time.Second) - return m.mockSearchBackend.BuildIndex(ctx, key, size, resourceVersion, fields, builder) + + // Simulate erroring out when context is cancelled. + if ctx.Err() != nil { + return nil, ctx.Err() + } + idx, err := m.mockSearchBackend.BuildIndex(ctx, key, size, resourceVersion, fields, reason, builder) + if err != nil { + return nil, err + } + + m.mu.Lock() + defer m.mu.Unlock() + + if m.cache == nil { + m.cache = make(map[NamespacedResource]ResourceIndex) + } + m.cache[key] = idx + return idx, nil } diff --git a/pkg/storage/unified/search/bleve.go b/pkg/storage/unified/search/bleve.go index 141e02b4030..2626ab36f1d 100644 --- a/pkg/storage/unified/search/bleve.go +++ b/pkg/storage/unified/search/bleve.go @@ -23,6 +23,7 @@ import ( "github.com/blevesearch/bleve/v2/search/query" bleveSearch "github.com/blevesearch/bleve/v2/search/searcher" index "github.com/blevesearch/bleve_index_api" + "go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/trace" "k8s.io/apimachinery/pkg/selection" @@ -197,11 +198,21 @@ func (b *bleveBackend) BuildIndex( size int64, resourceVersion int64, fields resource.SearchableDocumentFields, + indexBuildReason string, builder func(index resource.ResourceIndex) (int64, error), ) (resource.ResourceIndex, error) { _, span := b.tracer.Start(ctx, tracingPrexfixBleve+"BuildIndex") defer span.End() + span.SetAttributes( + attribute.String("namespace", key.Namespace), + attribute.String("group", key.Group), + attribute.String("resource", key.Resource), + attribute.Int64("size", size), + attribute.Int64("rv", resourceVersion), + attribute.String("reason", indexBuildReason), + ) + mapper, err := GetBleveMappings(fields) if err != nil { return nil, err @@ -214,7 +225,7 @@ func (b *bleveBackend) BuildIndex( return nil, err } - logWithDetails := b.log.With("namespace", key.Namespace, "group", key.Group, "resource", key.Resource, "size", size, "rv", resourceVersion) + logWithDetails := b.log.With("namespace", key.Namespace, "group", key.Group, "resource", key.Resource, "size", size, "rv", resourceVersion, "reason", indexBuildReason) // Close the newly created/opened index by default. closeIndex := true @@ -306,14 +317,29 @@ func (b *bleveBackend) BuildIndex( } if build { + if b.indexMetrics != nil { + b.indexMetrics.IndexBuilds.WithLabelValues(indexBuildReason).Inc() + } + start := time.Now() _, err = builder(idx) if err != nil { logWithDetails.Error("Failed to build index", "err", err) + if b.indexMetrics != nil { + b.indexMetrics.IndexBuildFailures.Inc() + } return nil, fmt.Errorf("failed to build index: %w", err) } elapsed := time.Since(start) logWithDetails.Info("Finished building index", "elapsed", elapsed) + if b.indexMetrics != nil { + b.indexMetrics.IndexCreationTime.WithLabelValues().Observe(elapsed.Seconds()) + } + } else { + logWithDetails.Info("Skipping index build, using existing index") + if b.indexMetrics != nil { + b.indexMetrics.IndexBuildSkipped.Inc() + } } // Set expiration after building the index. Only expire in-memory indexes. diff --git a/pkg/storage/unified/search/bleve_search_test.go b/pkg/storage/unified/search/bleve_search_test.go index a795b4c0f8d..0680026e2af 100644 --- a/pkg/storage/unified/search/bleve_search_test.go +++ b/pkg/storage/unified/search/bleve_search_test.go @@ -553,7 +553,7 @@ func newTestDashboardsIndex(t TB, threshold int64, size int64, batchSize int64, Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, size, rv, info.Fields, writer) + }, size, rv, info.Fields, "test", writer) require.NoError(t, err) return index, tmpdir diff --git a/pkg/storage/unified/search/bleve_test.go b/pkg/storage/unified/search/bleve_test.go index 5a4c16be2b3..821ce6de0c0 100644 --- a/pkg/storage/unified/search/bleve_test.go +++ b/pkg/storage/unified/search/bleve_test.go @@ -71,7 +71,7 @@ func TestBleveBackend(t *testing.T) { Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, 2, rv, info.Fields, func(index resource.ResourceIndex) (int64, error) { + }, 2, rv, info.Fields, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -352,7 +352,7 @@ func TestBleveBackend(t *testing.T) { Namespace: key.Namespace, Group: key.Group, Resource: key.Resource, - }, 2, rv, fields, func(index resource.ResourceIndex) (int64, error) { + }, 2, rv, fields, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -766,7 +766,7 @@ func TestBleveInMemoryIndexExpiration(t *testing.T) { Resource: "resource", } - builtIndex, err := backend.BuildIndex(context.Background(), ns, 1 /* below FileThreshold */, 100, nil, indexTestDocs(ns, 1)) + builtIndex, err := backend.BuildIndex(context.Background(), ns, 1 /* below FileThreshold */, 100, nil, "test", indexTestDocs(ns, 1)) require.NoError(t, err) // Wait for index expiration, which is 1ns @@ -798,7 +798,7 @@ func TestBleveFileIndexExpiration(t *testing.T) { } // size=100 is above FileThreshold, this will be file-based index - builtIndex, err := backend.BuildIndex(context.Background(), ns, 100, 100, nil, indexTestDocs(ns, 1)) + builtIndex, err := backend.BuildIndex(context.Background(), ns, 100, 100, nil, "test", indexTestDocs(ns, 1)) require.NoError(t, err) // Wait for index expiration, which is 1ns @@ -830,7 +830,7 @@ func TestFileIndexIsReusedOnSameSizeAndRV(t *testing.T) { tmpDir := t.TempDir() backend1, reg1 := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) // Verify one open index. @@ -853,7 +853,7 @@ func TestFileIndexIsReusedOnSameSizeAndRV(t *testing.T) { // We open new backend using same directory, and run indexing with same size (10) and RV (100). This should reuse existing index, and skip indexing. backend2, reg2 := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, indexTestDocs(ns, 1000)) + idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 100, nil, "test", indexTestDocs(ns, 1000)) require.NoError(t, err) // Verify that we're reusing existing index and there is only 10 documents in it, not 1000. @@ -879,13 +879,13 @@ func TestFileIndexIsNotReusedOnDifferentSize(t *testing.T) { tmpDir := t.TempDir() backend1, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) backend1.closeAllIndexes() // We open new backend using same directory, but with different size. Index should be rebuilt. backend2, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 100, 100, nil, indexTestDocs(ns, 100)) + idx, err := backend2.BuildIndex(context.Background(), ns, 100, 100, nil, "test", indexTestDocs(ns, 100)) require.NoError(t, err) // Verify that index has updated number of documents. @@ -904,13 +904,13 @@ func TestFileIndexIsNotReusedOnDifferentRV(t *testing.T) { tmpDir := t.TempDir() backend1, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, indexTestDocs(ns, 10)) + _, err := backend1.BuildIndex(context.Background(), ns, 10, 100, nil, "test", indexTestDocs(ns, 10)) require.NoError(t, err) backend1.closeAllIndexes() // We open new backend using same directory, but with different RV. Index should be rebuilt. backend2, _ := setupBleveBackend(t, 5, time.Nanosecond, tmpDir) - idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 999999, nil, indexTestDocs(ns, 100)) + idx, err := backend2.BuildIndex(context.Background(), ns, 10 /* file based */, 999999, nil, "test", indexTestDocs(ns, 100)) require.NoError(t, err) // Verify that index has updated number of documents. @@ -942,7 +942,7 @@ func TestRebuildingIndexClosesPreviousCachedIndex(t *testing.T) { if testCase.firstInMemory { firstSize = 1 } - firstIndex, err := backend.BuildIndex(context.Background(), ns, int64(firstSize), 100, nil, indexTestDocs(ns, firstSize)) + firstIndex, err := backend.BuildIndex(context.Background(), ns, int64(firstSize), 100, nil, "test", indexTestDocs(ns, firstSize)) require.NoError(t, err) openInMemoryIndexes := 0 @@ -952,7 +952,7 @@ func TestRebuildingIndexClosesPreviousCachedIndex(t *testing.T) { secondSize = 1 openInMemoryIndexes = 1 } - secondIndex, err := backend.BuildIndex(context.Background(), ns, int64(secondSize), 100, nil, indexTestDocs(ns, secondSize)) + secondIndex, err := backend.BuildIndex(context.Background(), ns, int64(secondSize), 100, nil, "test", indexTestDocs(ns, secondSize)) require.NoError(t, err) // Verify that first and second index are different, and first one is now closed. @@ -1050,12 +1050,12 @@ func testBleveIndexWithFailures(t *testing.T, fileBased bool) { // size=100 is above FileThreshold (5), make it a file-based index. size = 100 } - _, err := backend.BuildIndex(context.Background(), ns, size, 100, nil, func(index resource.ResourceIndex) (int64, error) { + _, err := backend.BuildIndex(context.Background(), ns, size, 100, nil, "test", func(index resource.ResourceIndex) (int64, error) { return 0, fmt.Errorf("fail") }) require.Error(t, err) // Even though previous build of the index failed, new building of the index should work. - _, err = backend.BuildIndex(context.Background(), ns, size, 100, nil, indexTestDocs(ns, int(size))) + _, err = backend.BuildIndex(context.Background(), ns, size, 100, nil, "test", indexTestDocs(ns, int(size))) require.NoError(t, err) } diff --git a/pkg/storage/unified/sql/backend.go b/pkg/storage/unified/sql/backend.go index a23cf9469ea..7733de8ff91 100644 --- a/pkg/storage/unified/sql/backend.go +++ b/pkg/storage/unified/sql/backend.go @@ -10,9 +10,9 @@ import ( "time" "github.com/go-sql-driver/mysql" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/jackc/pgx/v5/pgconn" "github.com/lib/pq" - "github.com/mattn/go-sqlite3" "github.com/prometheus/client_golang/prometheus" "go.opentelemetry.io/otel/trace" "go.opentelemetry.io/otel/trace/noop" @@ -389,9 +389,8 @@ func (b *backend) create(ctx context.Context, event resource.WriteEvent) (int64, // IsRowAlreadyExistsError checks if the error is the result of the row inserted already existing. func IsRowAlreadyExistsError(err error) bool { - var sqlite sqlite3.Error - if errors.As(err, &sqlite) { - return sqlite.ExtendedCode == sqlite3.ErrConstraintUnique + if sqlite.IsUniqueConstraintViolation(err) { + return true } var pg *pgconn.PgError diff --git a/pkg/storage/unified/sql/backend_test.go b/pkg/storage/unified/sql/backend_test.go index 92989e8454a..534b393065e 100644 --- a/pkg/storage/unified/sql/backend_test.go +++ b/pkg/storage/unified/sql/backend_test.go @@ -8,7 +8,6 @@ import ( "testing" "github.com/DATA-DOG/go-sqlmock" - "github.com/mattn/go-sqlite3" "github.com/stretchr/testify/require" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" @@ -18,6 +17,7 @@ import ( "github.com/grafana/grafana/pkg/storage/unified/resourcepb" "github.com/grafana/grafana/pkg/storage/unified/sql/db/dbimpl" "github.com/grafana/grafana/pkg/storage/unified/sql/test" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/grafana/grafana/pkg/util/testutil" ) @@ -242,7 +242,7 @@ func TestBackend_create(t *testing.T) { ) b.SQLMock.ExpectCommit() b.SQLMock.ExpectBegin() - b.SQLMock.ExpectExec("insert resource").WillReturnError(sqlite3.Error{Code: sqlite3.ErrConstraint, ExtendedCode: sqlite3.ErrConstraintUnique}) + b.SQLMock.ExpectExec("insert resource").WillReturnError(sqlite.TestErrUniqueConstraintViolation) b.SQLMock.ExpectRollback() // First we insert the resource successfully. This is what the happy path test does as well. diff --git a/pkg/storage/unified/testing/benchmark.go b/pkg/storage/unified/testing/benchmark.go index 8503b1dd2bd..dcdf113c212 100644 --- a/pkg/storage/unified/testing/benchmark.go +++ b/pkg/storage/unified/testing/benchmark.go @@ -215,7 +215,7 @@ func runSearchBackendBenchmarkWriteThroughput(ctx context.Context, backend resou // Build initial index size := int64(10000) // force the index to be on disk - index, err := backend.BuildIndex(ctx, nr, size, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, nr, size, 0, nil, "benchmark", func(index resource.ResourceIndex) (int64, error) { return 0, nil }) if err != nil { diff --git a/pkg/storage/unified/testing/search_backend.go b/pkg/storage/unified/testing/search_backend.go index 57d887682c6..52af572b01a 100644 --- a/pkg/storage/unified/testing/search_backend.go +++ b/pkg/storage/unified/testing/search_backend.go @@ -64,7 +64,7 @@ func runTestSearchBackendBuildIndex(t *testing.T, backend resource.SearchBackend require.Nil(t, index) // Build the index - index, err = backend.BuildIndex(ctx, ns, 0, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err = backend.BuildIndex(ctx, ns, 0, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { // Write a test document err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ @@ -111,7 +111,7 @@ func runTestResourceIndex(t *testing.T, backend resource.SearchBackend, nsPrefix } // Build initial index with some test documents - index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { @@ -235,7 +235,7 @@ func runTestResourceIndex(t *testing.T, backend resource.SearchBackend, nsPrefix t.Run("Search by LibraryPanel reference", func(t *testing.T) { // Build index with dashboards that have LibraryPanel references - index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, func(index resource.ResourceIndex) (int64, error) { + index, err := backend.BuildIndex(ctx, ns, 3, 0, nil, "test", func(index resource.ResourceIndex) (int64, error) { err := index.BulkIndex(&resource.BulkIndexRequest{ Items: []*resource.BulkIndexItem{ { diff --git a/pkg/tests/api/shorturl/short_url_test.go b/pkg/tests/api/shorturl/short_url_test.go index e598c36ea7a..a938b861281 100644 --- a/pkg/tests/api/shorturl/short_url_test.go +++ b/pkg/tests/api/shorturl/short_url_test.go @@ -7,8 +7,10 @@ import ( "fmt" "io" "net/http" + "net/url" "testing" + "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "github.com/grafana/grafana/pkg/infra/db" @@ -31,10 +33,12 @@ func TestMain(m *testing.M) { func TestShortURL(t *testing.T) { dir, path := testinfra.CreateGrafDir(t, testinfra.GrafanaOpts{ AppModeProduction: true, + DisableAnonymous: true, }) grafanaListedAddr, env := testinfra.StartGrafanaEnv(t, dir, path) + // Test that the endpoint is accessible with authentication. username, password := "viewer", "viewer" createUser(t, env.SQLStore, env.Cfg, user.CreateUserCommand{ DefaultOrgRole: string(org.RoleEditor), @@ -50,7 +54,7 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, http.StatusOK, res.StatusCode) + assert.Equal(t, http.StatusOK, res.StatusCode) bodyRaw, err := io.ReadAll(res.Body) require.NoError(t, err) @@ -67,8 +71,8 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, "http://localhost:3000/explore", res.Header.Get("Location")) - require.Equal(t, http.StatusFound, res.StatusCode) + assert.Equal(t, "http://localhost:3000/explore", res.Header.Get("Location")) + assert.Equal(t, http.StatusFound, res.StatusCode) // If the go-to does not exist, it should redirect to the home page and return 308. res, err = c.get("/goto/DoesNotExist") @@ -76,8 +80,28 @@ func TestShortURL(t *testing.T) { defer func() { _ = res.Body.Close() }() - require.Equal(t, "http://localhost:3000/", res.Header.Get("Location")) - require.Equal(t, http.StatusPermanentRedirect, res.StatusCode) + assert.Equal(t, "http://localhost:3000/", res.Header.Get("Location")) + assert.Equal(t, http.StatusPermanentRedirect, res.StatusCode) + + // Create a client that does not have authentication. + notLoggedInClient := client(grafanaListedAddr, "", "") + // Test that the short-urls endpoint is not accessible without authentication. + res, err = notLoggedInClient.post("/api/short-urls", bytes.NewReader([]byte(`{"path":"explore"}`))) + require.NoError(t, err) + assert.Equal(t, http.StatusUnauthorized, res.StatusCode) + defer func() { + _ = res.Body.Close() + }() + + // If the user is not logged in, it should redirect to the login page and return 302. + res, err = notLoggedInClient.get(fmt.Sprintf("/goto/%s", resParsed.UID)) + require.NoError(t, err) + defer func() { + _ = res.Body.Close() + }() + expectedRedirect := "/login?redirectTo=" + url.QueryEscape("/goto/"+resParsed.UID) + assert.Equal(t, expectedRedirect, res.Header.Get("Location")) + assert.Equal(t, http.StatusFound, res.StatusCode) } func createUser(t *testing.T, db db.DB, cfg *setting.Cfg, cmd user.CreateUserCommand) int64 { diff --git a/pkg/tests/apis/provisioning/helper_test.go b/pkg/tests/apis/provisioning/helper_test.go index 2d5585b819f..03ca9342f31 100644 --- a/pkg/tests/apis/provisioning/helper_test.go +++ b/pkg/tests/apis/provisioning/helper_test.go @@ -112,11 +112,11 @@ func (h *provisioningTestHelper) AwaitJobSuccess(t *testing.T, ctx context.Conte require.NoError(t, err) require.NotNil(t, result) + errors := mustNestedStringSlice(result.Object, "status", "errors") + require.Empty(t, errors, "historic job '%s' has errors: %v", job.GetName(), errors) state := mustNestedString(result.Object, "status", "state") require.Equal(t, string(provisioning.JobStateSuccess), state, "historic job '%s' was not successful", job.GetName()) - errors := mustNestedStringSlice(result.Object, "status", "errors") - require.Empty(t, errors, "historic job '%s' has errors: %v", job.GetName(), errors) }, time.Second*10, time.Millisecond*25) { // We also want to add the job details to the error when it fails. job, err := h.Jobs.Resource.Get(ctx, job.GetName(), metav1.GetOptions{}) @@ -163,6 +163,50 @@ func (h *provisioningTestHelper) AwaitJobs(t *testing.T, repoName string) { } } +// AwaitJobsWithStates waits for all jobs for a repository to complete and accepts multiple valid end states +func (h *provisioningTestHelper) AwaitJobsWithStates(t *testing.T, repoName string, acceptedStates []string) { + t.Helper() + + // First, we wait for all jobs for the repository to disappear (i.e. complete/fail). + require.EventuallyWithT(t, func(collect *assert.CollectT) { + list, err := h.Jobs.Resource.List(context.Background(), metav1.ListOptions{}) + if assert.NoError(collect, err, "failed to list active jobs") { + for _, elem := range list.Items { + repo, _, err := unstructured.NestedString(elem.Object, "spec", "repository") + require.NoError(t, err) + if repo == repoName { + collect.Errorf("there are still remaining jobs for %s: %+v", repoName, elem) + return + } + } + } + }, time.Second*10, time.Millisecond*25, "job queue must be empty") + + // Then, as all jobs are now historic jobs, we make sure they are in an accepted state. + result, err := h.Repositories.Resource.Get(context.Background(), repoName, metav1.GetOptions{}, "jobs") + require.NoError(t, err, "failed to list historic jobs") + + list, err := result.ToList() + require.NoError(t, err, "results should be a list") + require.NotEmpty(t, list.Items, "expect at least one job") + + for _, elem := range list.Items { + require.Equal(t, repoName, elem.GetLabels()[jobs.LabelRepository], "should have repo label") + + state := mustNestedString(elem.Object, "status", "state") + + // Check if state is in accepted states + found := false + for _, acceptedState := range acceptedStates { + if state == acceptedState { + found = true + break + } + } + require.True(t, found, "job %s completed with unexpected state %s (expected one of %v): %+v", elem.GetName(), state, acceptedStates, elem.Object) + } +} + // RenderObject reads the filePath and renders it as a template with the given values. // The template is expected to be a YAML or JSON file. // diff --git a/pkg/tests/apis/provisioning/provisioning_test.go b/pkg/tests/apis/provisioning/provisioning_test.go index cdf210e2133..f9a5c40a14e 100644 --- a/pkg/tests/apis/provisioning/provisioning_test.go +++ b/pkg/tests/apis/provisioning/provisioning_test.go @@ -161,14 +161,21 @@ func TestIntegrationProvisioning_CreatingAndGetting(t *testing.T) { // Viewer can see settings listing t.Run("viewer has access to list", func(t *testing.T) { settings := &provisioning.RepositoryViewList{} - rsp := helper.ViewerREST.Get(). - Namespace("default"). - Suffix("settings"). - Do(context.Background()) - require.NoError(t, rsp.Error()) - err := rsp.Into(settings) - require.NoError(t, err) - require.Len(t, settings.Items, len(inputFiles)) + // Wait for unified storage to make the data available + require.Eventually(t, func() bool { + rsp := helper.ViewerREST.Get(). + Namespace("default"). + Suffix("settings"). + Do(context.Background()) + if rsp.Error() != nil { + return false + } + err := rsp.Into(settings) + if err != nil { + return false + } + return len(settings.Items) == len(inputFiles) + }, time.Second*10, time.Millisecond*100, "Expected settings to have len(inputFiles) items") // FIXME: this should be an enterprise integration test if extensions.IsEnterprise { @@ -1825,8 +1832,10 @@ func TestIntegrationProvisioning_MoveResources(t *testing.T) { // Verify dashboard still exists in Grafana with same content but may have updated path references helper.SyncAndWait(t, repo, nil) - _, err = helper.DashboardsV1.Resource.Get(ctx, allPanelsUID, metav1.GetOptions{}) - require.NoError(t, err, "dashboard should still exist in Grafana after move") + require.Eventually(t, func() bool { + _, err = helper.DashboardsV1.Resource.Get(ctx, allPanelsUID, metav1.GetOptions{}) + return err == nil + }, 10*time.Second, 100*time.Millisecond, "dashboard should still exist in Grafana after move") // Using Eventually to account for potential delays in dashboards APIs. }) t.Run("move file to nested path without ref", func(t *testing.T) { @@ -2107,3 +2116,196 @@ func TestIntegrationProvisioning_MoveResources(t *testing.T) { }, time.Second*10, time.Millisecond*100, "Expected move job to handle non-existent resource") }) } + +func TestIntegrationProvisioning_SecondRepositoryOnlyExportsNewDashboards(t *testing.T) { + if testing.Short() { + t.Skip("skipping integration test") + } + + helper := runGrafana(t) + ctx := context.Background() + + // Create some unmanaged dashboards directly in Grafana first + dashboard1 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v1.yaml") + dashboard1Obj, err := helper.DashboardsV1.Resource.Create(ctx, dashboard1, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create first dashboard") + dashboard1Name := dashboard1Obj.GetName() + + dashboard2 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v2beta1.yaml") + dashboard2Obj, err := helper.DashboardsV2beta1.Resource.Create(ctx, dashboard2, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create second dashboard") + dashboard2Name := dashboard2Obj.GetName() + + // Create the first repository with sync enabled + const repo1 = "first-repository" + repo1Path := filepath.Join(helper.ProvisioningPath, repo1) + err = os.MkdirAll(repo1Path, 0750) + require.NoError(t, err, "should be able to create repository path") + + createBody1 := helper.RenderObject(t, "testdata/local-write.json.tmpl", map[string]any{ + "Name": repo1, + "SyncEnabled": true, + "SyncTarget": "folder", + "Path": repo1Path, + }) + _, err = helper.Repositories.Resource.Create(ctx, createBody1, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create first repository") + + // Print file tree before export + printFileTree(t, helper.ProvisioningPath) + + // Initial export + result := helper.AdminREST.Post(). + Namespace("default"). + Resource("repositories"). + Name(repo1). + SubResource("jobs"). + SetHeader("Content-Type", "application/json"). + Body(asJSON(&provisioning.JobSpec{ + Push: &provisioning.ExportJobOptions{ + Folder: "", // export entire instance + Path: "", // no prefix necessary for testing + }, + })). + Do(ctx) + require.NoError(t, result.Error(), "should be able to create export job for first repo") + helper.AwaitJobsWithStates(t, repo1, []string{"success"}) + // Wait for first repository to sync + helper.SyncAndWait(t, repo1, nil) + + printFileTree(t, helper.ProvisioningPath) + // Verify that the first repository has claimed ownership of the dashboards + managedDash1, err := helper.DashboardsV1.Resource.Get(ctx, dashboard1Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, managedDash1.GetAnnotations()[utils.AnnoKeyManagerIdentity], "dashboard1 should be managed by first repo") + + managedDash2, err := helper.DashboardsV2beta1.Resource.Get(ctx, dashboard2Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, managedDash2.GetAnnotations()[utils.AnnoKeyManagerIdentity], "dashboard2 should be managed by first repo") + + // Create second repository - enable sync and set different target + + const repo2 = "second-repository" + repo2Path := filepath.Join(helper.ProvisioningPath, repo2) + err = os.MkdirAll(repo2Path, 0750) + require.NoError(t, err, "should be able to create seconrd repository path") + + printFileTree(t, helper.ProvisioningPath) + + createBody2 := helper.RenderObject(t, "testdata/local-write.json.tmpl", map[string]any{ + "Name": repo2, + "SyncEnabled": true, + "SyncTarget": "folder", + "Path": repo2Path, + }) + + _, err = helper.Repositories.Resource.Create(ctx, createBody2, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create second repository") + + // Wait for second repository to sync + helper.SyncAndWait(t, repo2, nil) + + // Validate that folders for both repositories exist + folders, err := helper.Folders.Resource.List(ctx, metav1.ListOptions{}) + require.NoError(t, err, "should be able to list folders") + + var repo1FolderFound, repo2FolderFound bool + for _, folder := range folders.Items { + if folder.GetName() == repo1 { + repo1FolderFound = true + } + if folder.GetName() == repo2 { + repo2FolderFound = true + } + } + require.True(t, repo1FolderFound, "folder for first repository %s should exist after sync", repo1) + require.True(t, repo2FolderFound, "folder for second repository %s should exist after sync", repo2) + + // Create a third dashboard that won't be claimed by the first repo + dashboard3 := helper.LoadYAMLOrJSONFile("exportunifiedtorepository/dashboard-test-v0.yaml") + dashboard3Obj, err := helper.DashboardsV0.Resource.Create(ctx, dashboard3, metav1.CreateOptions{}) + require.NoError(t, err, "should be able to create third dashboard") + dashboard3Name := dashboard3Obj.GetName() + + // Verify dashboard3 is not managed by anyone initially + unmanagedDash3, err := helper.DashboardsV0.Resource.Get(ctx, dashboard3Name, metav1.GetOptions{}) + require.NoError(t, err) + manager, found := unmanagedDash3.GetAnnotations()[utils.AnnoKeyManagerIdentity] + require.True(t, !found || manager == "", "dashboard3 should not be managed initially") + + printFileTree(t, helper.ProvisioningPath) + // Count files in first repo before second export + files1Before, err := countFilesInDir(repo1Path) + require.NoError(t, err) + + // Export from second repository - this should only export the unmanaged dashboard3 + result = helper.AdminREST.Post(). + Namespace("default"). + Resource("repositories"). + Name(repo2). + SubResource("jobs"). + SetHeader("Content-Type", "application/json"). + Body(asJSON(&provisioning.JobSpec{ + Push: &provisioning.ExportJobOptions{ + Folder: "", // export entire instance + Path: "", // no prefix necessary for testing + }, + })). + Do(ctx) + require.NoError(t, result.Error(), "should be able to create export job for second repo") + + // Wait for second repository export to complete + helper.AwaitJobsWithStates(t, repo2, []string{"success"}) + + // Wait for second repository to sync + helper.SyncAndWait(t, repo1, nil) + helper.SyncAndWait(t, repo2, nil) + + printFileTree(t, helper.ProvisioningPath) + files1After, err := countFilesInDir(repo1Path) + require.NoError(t, err) + + actualNewFiles := files1After - files1Before + require.Equal(t, 0, actualNewFiles, + "second repository should skip managed dashboards and had folder issues with unmanaged dashboard (expected %d new files, got %d)", + 0, actualNewFiles) + + // Verify files in the second repository + files2After, err := countFilesInDir(repo2Path) + require.NoError(t, err) + require.Equal(t, 1, files2After, + "second repository should only export the unmanaged dashboard (expected %d new files, got %d)", + 1, files2After) + + // Verify dashboard1 and dashboard2 are still managed by repo1 (unchanged) + stillManagedDash1, err := helper.DashboardsV1.Resource.Get(ctx, dashboard1Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, stillManagedDash1.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard1 should still be managed by first repo") + + stillManagedDash2, err := helper.DashboardsV2beta1.Resource.Get(ctx, dashboard2Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo1, stillManagedDash2.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard2 should still be managed by first repo") + + // Verify dashboard3 is now managed by repo2 + stillManagedDash3, err := helper.DashboardsV0.Resource.Get(ctx, dashboard3Name, metav1.GetOptions{}) + require.NoError(t, err) + require.Equal(t, repo2, stillManagedDash3.GetAnnotations()[utils.AnnoKeyManagerIdentity], + "dashboard3 should now be managed by second repo") +} + +// Helper function to count files in a directory recursively +func countFilesInDir(rootPath string) (int, error) { + count := 0 + err := filepath.WalkDir(rootPath, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if !d.IsDir() { + count++ + } + return nil + }) + return count, err +} diff --git a/pkg/tests/testinfra/testinfra.go b/pkg/tests/testinfra/testinfra.go index eb9b460c6b8..56f9f3000e2 100644 --- a/pkg/tests/testinfra/testinfra.go +++ b/pkg/tests/testinfra/testinfra.go @@ -13,6 +13,7 @@ import ( "time" "github.com/grafana/grafana/pkg/services/featuremgmt" + "github.com/grafana/grafana/pkg/services/sqlstore/sqlutil" "github.com/prometheus/client_golang/prometheus" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -85,6 +86,20 @@ func StartGrafanaEnv(t *testing.T, grafDir, cfgPath string) (string, *server.Tes err = featuremgmt.InitOpenFeatureWithCfg(cfg) require.NoError(t, err) + + // Use proper database type based on the environment variable GRAFANA_TEST_DB in tests + testDB, err := sqlutil.GetTestDB(sqlutil.GetTestDBType()) + require.NoError(t, err) + t.Cleanup(testDB.Cleanup) + + dbCfg := cfg.Raw.Section("database") + dbCfg.Key("type").SetValue(testDB.DriverName) + dbCfg.Key("host").SetValue(testDB.Host) + dbCfg.Key("port").SetValue(testDB.Port) + dbCfg.Key("user").SetValue(testDB.User) + dbCfg.Key("password").SetValue(testDB.Password) + dbCfg.Key("name").SetValue(testDB.Database) + env, err := server.InitializeForTest(t, t, cfg, serverOpts, apiServerOpts) require.NoError(t, err) diff --git a/pkg/tsdb/elasticsearch/healthcheck.go b/pkg/tsdb/elasticsearch/healthcheck.go index a3e04aafb6f..928945691de 100644 --- a/pkg/tsdb/elasticsearch/healthcheck.go +++ b/pkg/tsdb/elasticsearch/healthcheck.go @@ -191,11 +191,15 @@ func validateIndex(ctx context.Context, ds *es.DatasourceInfo) (message string, return "Failed to unmarshal field capabilities response", "error" } if fieldCaps["error"] != nil { - if errorMessage, ok := fieldCaps["error"].(map[string]any)["reason"].(string); ok { - return fmt.Sprintf("Error validating index: %s", errorMessage), "warning" - } else { + errorMap, ok := fieldCaps["error"].(map[string]any) + if !ok { return "Error validating index", "warning" } + errorMessage, ok := errorMap["reason"].(string) + if !ok { + return "Error validating index", "warning" + } + return fmt.Sprintf("Error validating index: %s", errorMessage), "warning" } fields, ok := fieldCaps["fields"].(map[string]any) diff --git a/pkg/tsdb/elasticsearch/healthcheck_test.go b/pkg/tsdb/elasticsearch/healthcheck_test.go index b3f6dc97c93..48fd00e8adc 100644 --- a/pkg/tsdb/elasticsearch/healthcheck_test.go +++ b/pkg/tsdb/elasticsearch/healthcheck_test.go @@ -58,6 +58,16 @@ func Test_validateIndex_Warning_ErrorValidatingIndex(t *testing.T) { assert.Equal(t, "Elasticsearch data source is healthy. Warning: Error validating index: index_not_found", res.Message) } +func Test_validateIndex_Warning_ErrorValidatingIndex2(t *testing.T) { + service := GetMockService(http.StatusOK, "200 OK", `{"status":"green"}`, `{"error":"not a map"}`) + res, _ := service.CheckHealth(mockedCfg, &backend.CheckHealthRequest{ + PluginContext: backend.PluginContext{}, + Headers: nil, + }) + assert.Equal(t, backend.HealthStatusOk, res.Status) + assert.Equal(t, "Elasticsearch data source is healthy. Warning: Error validating index", res.Message) +} + func Test_validateIndex_Warning_WrongTimestampType(t *testing.T) { service := GetMockService(http.StatusOK, "200 OK", `{"status":"green"}`, `{"fields":{"timestamp":{"float":{"metadata_field":true}}}}`) res, _ := service.CheckHealth(mockedCfg, &backend.CheckHealthRequest{ diff --git a/pkg/tsdb/influxdb/flux/flux.go b/pkg/tsdb/influxdb/flux/flux.go index cab5c0d1067..6643f064910 100644 --- a/pkg/tsdb/influxdb/flux/flux.go +++ b/pkg/tsdb/influxdb/flux/flux.go @@ -27,9 +27,8 @@ func Query(ctx context.Context, dsInfo *models.DatasourceInfo, tsdbQuery backend } defer r.client.Close() - timeRange := tsdbQuery.Queries[0].TimeRange for _, query := range tsdbQuery.Queries { - qm, err := getQueryModel(query, timeRange, dsInfo) + qm, err := getQueryModel(query, query.TimeRange, dsInfo) if err != nil { tRes.Responses[query.RefID] = backend.DataResponse{ Error: err, diff --git a/pkg/tsdb/jaeger/client.go b/pkg/tsdb/jaeger/client.go index 653c0463892..d858aa48f56 100644 --- a/pkg/tsdb/jaeger/client.go +++ b/pkg/tsdb/jaeger/client.go @@ -115,11 +115,15 @@ func (j *JaegerClient) Operations(s string) ([]string, error) { } func (j *JaegerClient) Search(query *JaegerQuery, start, end int64) ([]TraceResponse, error) { - jaegerURL, err := url.Parse(j.url) + u, err := url.JoinPath(j.url, "/api/traces") if err != nil { - return []TraceResponse{}, fmt.Errorf("failed to parse Jaeger URL: %w", err) + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to join url path: %w", err)) + } + + jaegerURL, err := url.Parse(u) + if err != nil { + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to parse Jaeger URL: %w", err)) } - jaegerURL.Path = "/api/traces" var queryTags string if query.Tags != "" { @@ -135,7 +139,7 @@ func (j *JaegerClient) Search(query *JaegerQuery, start, end int64) ([]TraceResp marshaledTags, err := json.Marshal(tagMap) if err != nil { - return []TraceResponse{}, fmt.Errorf("failed to convert tags to JSON: %w", err) + return []TraceResponse{}, backend.DownstreamError(fmt.Errorf("failed to convert tags to JSON: %w", err)) } queryTags = string(marshaledTags) diff --git a/pkg/tsdb/jaeger/client_test.go b/pkg/tsdb/jaeger/client_test.go index afabdc4bccc..eceb103ab82 100644 --- a/pkg/tsdb/jaeger/client_test.go +++ b/pkg/tsdb/jaeger/client_test.go @@ -186,6 +186,19 @@ func TestJaegerClient_Search(t *testing.T) { expectError bool expectedError error }{ + { + name: "Preserves base path in Jaeger URL", + query: &JaegerQuery{ + Service: "test-service", + }, + start: 1735689600000000, + end: 1738368000000000, + mockResponse: `{"data":[{"traceID":"test-trace-id"}]}`, + mockStatusCode: http.StatusOK, + expectedURL: "/abc/api/traces?end=1738368000000000&service=test-service&start=1735689600000000", + expectError: false, + expectedError: nil, + }, { name: "Successful search with all parameters", query: &JaegerQuery{ @@ -245,6 +258,11 @@ func TestJaegerClient_Search(t *testing.T) { settings := backend.DataSourceInstanceSettings{ URL: server.URL, } + + if tt.name == "Preserves base path in Jaeger URL" { + settings.URL = server.URL + "/abc" + } + client, err := New(server.Client(), log.NewNullLogger(), settings) assert.NoError(t, err) traces, err := client.Search(tt.query, tt.start, tt.end) diff --git a/pkg/util/sqlite/sqlite_cgo.go b/pkg/util/sqlite/sqlite_cgo.go new file mode 100644 index 00000000000..de18511741d --- /dev/null +++ b/pkg/util/sqlite/sqlite_cgo.go @@ -0,0 +1,46 @@ +//go:build cgo + +package sqlite + +import ( + "errors" + + "github.com/mattn/go-sqlite3" +) + +type Driver = sqlite3.SQLiteDriver + +// The errors below are used in tests to simulate specific SQLite errors. It's a temporary solution +// until we rewrite the tests not to depend on the sqlite3 package internals directly. +var ( + TestErrUniqueConstraintViolation = sqlite3.Error{Code: sqlite3.ErrConstraint, ExtendedCode: sqlite3.ErrConstraintUnique} + TestErrBusy = sqlite3.Error{Code: sqlite3.ErrBusy} + TestErrLocked = sqlite3.Error{Code: sqlite3.ErrLocked} +) + +func IsBusyOrLocked(err error) bool { + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.Code == sqlite3.ErrLocked || sqliteErr.Code == sqlite3.ErrBusy + } + return false +} + +func IsUniqueConstraintViolation(err error) bool { + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.ExtendedCode == sqlite3.ErrConstraintUnique || sqliteErr.ExtendedCode == sqlite3.ErrConstraintPrimaryKey + } + return false +} + +func ErrorMessage(err error) string { + if err == nil { + return "" + } + var sqliteErr sqlite3.Error + if errors.As(err, &sqliteErr) { + return sqliteErr.Error() + } + return err.Error() +} diff --git a/pkg/util/sqlite/sqlite_nocgo.go b/pkg/util/sqlite/sqlite_nocgo.go new file mode 100644 index 00000000000..e3b9a0429d3 --- /dev/null +++ b/pkg/util/sqlite/sqlite_nocgo.go @@ -0,0 +1,24 @@ +//go:build !cgo + +package sqlite + +import "modernc.org/sqlite" + +// +// FIXME (@zserge) +// +// This non-CGo "implementation" is merely a stub to make Grafana compile without CGo. +// Any attempts to actually use this driver are likely to fail at runtime in the most brutal ways. +// + +type Driver = sqlite.Driver + +func IsBusyOrLocked(err error) bool { + return false // FIXME +} +func IsUniqueConstraintViolation(err error) bool { + return false // FIXME +} +func ErrorMessage(err error) string { + return "" // FIXME +} diff --git a/pkg/util/xorm/dialect_sqlite3.go b/pkg/util/xorm/dialect_sqlite3.go index 381a38bb490..4c235ee1c90 100644 --- a/pkg/util/xorm/dialect_sqlite3.go +++ b/pkg/util/xorm/dialect_sqlite3.go @@ -11,8 +11,8 @@ import ( "regexp" "strings" + "github.com/grafana/grafana/pkg/util/sqlite" "github.com/grafana/grafana/pkg/util/xorm/core" - sqlite "github.com/mattn/go-sqlite3" ) var ( @@ -476,11 +476,7 @@ func (db *sqlite3) Filters() []core.Filter { } func (db *sqlite3) RetryOnError(err error) bool { - var sqlError sqlite.Error - if errors.As(err, &sqlError) && (sqlError.Code == sqlite.ErrLocked || sqlError.Code == sqlite.ErrBusy) { - return true - } - return false + return sqlite.IsBusyOrLocked(err) } type sqlite3Driver struct { diff --git a/pkg/util/xorm/xorm_test.go b/pkg/util/xorm/xorm_test.go index f2939fbf9a8..281282a1ef8 100644 --- a/pkg/util/xorm/xorm_test.go +++ b/pkg/util/xorm/xorm_test.go @@ -4,7 +4,7 @@ import ( "encoding/json" "testing" - _ "github.com/mattn/go-sqlite3" + _ "github.com/grafana/grafana/pkg/util/sqlite" "github.com/stretchr/testify/require" ) diff --git a/playwright.config.ts b/playwright.config.ts index a3388e20c5e..e1a6b654e73 100644 --- a/playwright.config.ts +++ b/playwright.config.ts @@ -195,6 +195,15 @@ export default defineConfig
element has child elements that allow for mouse events, so we need to disable the linter rule */}
- {/* eslint-disable-next-line jsx-a11y/no-static-element-interactions */}
- {
- this.showConfirmation = false;
- action.onClick(new MouseEvent('click'));
+ this.showActionConfirmation = false;
+ action.onClick(new MouseEvent('click'), null, this.actionVars);
this.forceUpdate();
}}
onDismiss={() => {
- this.showConfirmation = false;
+ this.showActionConfirmation = false;
this.forceUpdate();
}}
/>
@@ -770,6 +1036,31 @@ export class ElementState implements LayerElement {
);
};
+ renderVariablesInputModal = (action: ActionModel | undefined) => {
+ if (!action || !action.variables || action.variables.length === 0) {
+ return;
+ }
+
+ const onModalContinue = () => {
+ this.showActionVarsModal = false;
+ this.showActionConfirmation = true;
+ this.forceUpdate();
+ };
+
+ return (
+ {
+ this.showActionVarsModal = false;
+ this.forceUpdate();
+ }}
+ onShowConfirm={onModalContinue}
+ />
+ );
+ };
+
render() {
const { item, div } = this;
const scene = this.getScene();
@@ -786,6 +1077,7 @@ export class ElementState implements LayerElement {
onKeyDown={!scene?.isEditingEnabled ? this.onElementKeyDown : undefined}
role="button"
tabIndex={0}
+ style={{ userSelect: 'none' }}
>
- {this.showConfirmation && this.renderActionsConfirmModal(this.getPrimaryAction())}
+ {this.showActionConfirmation && this.renderActionsConfirmModal(this.getPrimaryAction())}
+ {this.showActionVarsModal && this.renderVariablesInputModal(this.getPrimaryAction())}
);
}
diff --git a/public/app/features/canvas/runtime/scene.tsx b/public/app/features/canvas/runtime/scene.tsx
index 5e54afd9be8..7495d74ff88 100644
--- a/public/app/features/canvas/runtime/scene.tsx
+++ b/public/app/features/canvas/runtime/scene.tsx
@@ -1,7 +1,7 @@
import { css } from '@emotion/css';
+import InfiniteViewer from 'infinite-viewer';
import Moveable from 'moveable';
-import { createRef, CSSProperties, RefObject } from 'react';
-import { ReactZoomPanPinchContentRef } from 'react-zoom-pan-pinch';
+import { CSSProperties } from 'react';
import { BehaviorSubject, ReplaySubject, Subject, Subscription } from 'rxjs';
import Selecto from 'selecto';
@@ -13,6 +13,7 @@ import {
ScalarDimensionConfig,
ScaleDimensionConfig,
TextDimensionConfig,
+ TooltipDisplayMode,
} from '@grafana/schema';
import { Portal } from '@grafana/ui';
import { config } from 'app/core/config';
@@ -27,19 +28,20 @@ import {
import { CanvasContextMenu } from 'app/plugins/panel/canvas/components/CanvasContextMenu';
import { CanvasTooltip } from 'app/plugins/panel/canvas/components/CanvasTooltip';
import { Connections } from 'app/plugins/panel/canvas/components/connections/Connections';
+import { Connections2 } from 'app/plugins/panel/canvas/components/connections/Connections2';
+import { Options } from 'app/plugins/panel/canvas/panelcfg.gen';
import { AnchorPoint, CanvasTooltipPayload } from 'app/plugins/panel/canvas/types';
-import { getTransformInstance } from 'app/plugins/panel/canvas/utils';
import appEvents from '../../../core/app_events';
import { CanvasPanel } from '../../../plugins/panel/canvas/CanvasPanel';
+import { getDashboardSrv } from '../../dashboard/services/DashboardSrv';
import { CanvasFrameOptions } from '../frame';
import { DEFAULT_CANVAS_ELEMENT_CONFIG } from '../registry';
-import { SceneTransformWrapper } from './SceneTransformWrapper';
import { ElementState } from './element';
import { FrameState } from './frame';
import { RootElement } from './root';
-import { initMoveable } from './sceneAbleManagement';
+import { initMoveable, calculateZoomToFitScale } from './sceneAbleManagement';
import { findElementByTarget } from './sceneElementManagement';
export interface SelectionParams {
@@ -60,31 +62,30 @@ export class Scene {
width = 0;
height = 0;
scale = 1;
+ scrollLeft = 0;
+ scrollTop = 0;
style: CSSProperties = {};
data?: PanelData;
selecto?: Selecto;
moveable?: Moveable;
+ infiniteViewer?: InfiniteViewer;
div?: HTMLDivElement;
- connections: Connections;
+ viewerDiv?: HTMLDivElement;
+ viewportDiv?: HTMLDivElement;
+ connections: Connections | Connections2;
currentLayer?: FrameState;
isEditingEnabled?: boolean;
shouldShowAdvancedTypes?: boolean;
shouldPanZoom?: boolean;
- shouldInfinitePan?: boolean;
+ zoomToContent?: boolean;
+ tooltipMode?: TooltipDisplayMode;
skipNextSelectionBroadcast = false;
ignoreDataUpdate = false;
panel: CanvasPanel;
contextMenuVisible?: boolean;
+ openContextMenu?: (position: AnchorPoint) => void;
contextMenuOnVisibilityChange = (visible: boolean) => {
this.contextMenuVisible = visible;
- const transformInstance = getTransformInstance(this);
- if (transformInstance) {
- if (visible) {
- transformInstance.setup.disabled = true;
- } else {
- transformInstance.setup.disabled = false;
- }
- }
};
isPanelEditing = locationService.getSearchObject().editPanel !== undefined;
@@ -93,7 +94,7 @@ export class Scene {
setBackgroundCallback?: (anchorPoint: AnchorPoint) => void;
tooltipCallback?: (tooltip: CanvasTooltipPayload | undefined) => void;
- tooltip?: CanvasTooltipPayload;
+ tooltipPayload?: CanvasTooltipPayload;
moveableActionCallback?: (moved: boolean) => void;
@@ -103,18 +104,18 @@ export class Scene {
subscription: Subscription;
targetsToSelect = new Set{sceneDiv}
-
-
- );
-};
diff --git a/public/app/features/canvas/runtime/element.tsx b/public/app/features/canvas/runtime/element.tsx
index 047d42a5e45..a64d004967b 100644
--- a/public/app/features/canvas/runtime/element.tsx
+++ b/public/app/features/canvas/runtime/element.tsx
@@ -10,10 +10,13 @@ import {
ValueLinkConfig,
OneClickMode,
ActionModel,
+ ActionVariableInput,
} from '@grafana/data';
import { t } from '@grafana/i18n';
-import { ConfirmModal } from '@grafana/ui';
+import { TooltipDisplayMode } from '@grafana/schema';
+import { ConfirmModal, VariablesInputModal } from '@grafana/ui';
import { LayerElement } from 'app/core/components/Layers/types';
+import { config } from 'app/core/config';
import { notFoundItem } from 'app/features/canvas/elements/notFound';
import { DimensionContext } from 'app/features/dimensions/context';
import {
@@ -23,7 +26,13 @@ import {
Placement,
VerticalConstraint,
} from 'app/plugins/panel/canvas/panelcfg.gen';
-import { getConnectionsByTarget, getRowIndex, isConnectionTarget } from 'app/plugins/panel/canvas/utils';
+import {
+ applyStyles,
+ getConnectionsByTarget,
+ getRowIndex,
+ isConnectionTarget,
+ removeStyles,
+} from 'app/plugins/panel/canvas/utils';
import { getActions, getActionsDefaultField } from '../../actions/utils';
import { CanvasElementItem, CanvasElementOptions } from '../element';
@@ -58,7 +67,15 @@ export class ElementState implements LayerElement {
// cached for tooltips/mousemove
oneClickMode = OneClickMode.Off;
- showConfirmation = false;
+ showActionConfirmation = false;
+
+ showActionVarsModal = false;
+ actionVars: ActionVariableInput = {};
+
+ setActionVars = (vars: ActionVariableInput) => {
+ this.actionVars = vars;
+ this.forceUpdate();
+ };
constructor(
public item: CanvasElementItem,
@@ -104,6 +121,10 @@ export class ElementState implements LayerElement {
/** Use the configured options to update CSS style properties directly on the wrapper div **/
applyLayoutStylesToDiv(disablePointerEvents?: boolean) {
+ if (config.featureToggles.canvasPanelPanZoom) {
+ this.applyLayoutStylesToDiv2(disablePointerEvents);
+ return;
+ }
if (this.isRoot()) {
// Root supersedes layout engine and is always 100% width + height of panel
return;
@@ -214,34 +235,166 @@ export class ElementState implements LayerElement {
this.sizeStyle = style;
if (this.div) {
- for (const key in this.sizeStyle) {
- // eslint-disable-next-line @typescript-eslint/no-explicit-any, @typescript-eslint/consistent-type-assertions
- this.div.style[key as any] = (this.sizeStyle as any)[key];
- }
+ applyStyles(this.sizeStyle, this.div);
// TODO: This is a hack, we should have a better way to handle this
const elementType = this.options.type;
if (!SVGElements.has(elementType)) {
// apply styles to div if it's not an SVG element
- for (const key in this.dataStyle) {
- // eslint-disable-next-line @typescript-eslint/no-explicit-any, @typescript-eslint/consistent-type-assertions
- this.div.style[key as any] = (this.dataStyle as any)[key];
- }
+ applyStyles(this.dataStyle, this.div);
} else {
// ELEMENT IS SVG
// clean data styles from div if it's an SVG element; SVG elements have their own data styles;
// this is necessary for changing type of element cases;
// wrapper div element (this.div) doesn't re-render (has static `key` property),
// so we have to clean styles manually;
- for (const key in this.dataStyle) {
- // eslint-disable-next-line @typescript-eslint/no-explicit-any, @typescript-eslint/consistent-type-assertions
- this.div.style[key as any] = '';
- }
+ removeStyles(this.dataStyle, this.div);
}
}
}
+ /** Use the configured options to update CSS style properties directly on the wrapper div **/
+ applyLayoutStylesToDiv2(disablePointerEvents?: boolean) {
+ if (this.isRoot()) {
+ // Root supersedes layout engine and is always 100% width + height of panel
+ return;
+ }
+
+ const scene = this.getScene();
+ const { width: sceneWidth, height: sceneHeight } = scene ?? {};
+
+ const { constraint } = this.options;
+ const { vertical, horizontal } = constraint ?? {};
+ const placement: Placement = this.options.placement ?? {};
+
+ const editingEnabled = scene?.isEditingEnabled;
+
+ const style: React.CSSProperties = {
+ cursor: editingEnabled ? 'grab' : 'auto',
+ pointerEvents: disablePointerEvents ? 'none' : 'auto',
+ position: 'absolute',
+ // Minimum element size is 10x10
+ minWidth: '10px',
+ minHeight: '10px',
+ };
+
+ let transformY = '0px';
+ let transformX = '0px';
+
+ switch (vertical) {
+ case VerticalConstraint.Top:
+ placement.top = placement.top ?? 0;
+ placement.height = placement.height ?? 100;
+ transformY = `${placement.top ?? 0}px`;
+ style.height = `${placement.height}px`;
+ delete placement.bottom;
+ break;
+ case VerticalConstraint.Bottom:
+ placement.bottom = placement.bottom ?? 0;
+ placement.height = placement.height ?? 100;
+ transformY = `${sceneHeight! - (placement.bottom ?? 0) - (placement.height ?? 100)}px`;
+ style.height = `${placement.height}px`;
+ delete placement.top;
+ break;
+ case VerticalConstraint.TopBottom:
+ placement.top = placement.top ?? 0;
+ placement.bottom = placement.bottom ?? 0;
+ transformY = `${placement.top ?? 0}px`;
+ style.height = `${sceneHeight! - (placement.top ?? 0) - (placement.bottom ?? 0)}px`;
+ delete placement.height;
+ break;
+ case VerticalConstraint.Center:
+ placement.top = placement.top ?? 0;
+ placement.height = placement.height ?? 100;
+ transformY = `${sceneHeight! / 2 - (placement.top ?? 0) - (placement.height ?? 0) / 2}px`;
+ style.height = `${placement.height}px`;
+ delete placement.bottom;
+ break;
+ case VerticalConstraint.Scale:
+ placement.top = placement.top ?? 0;
+ placement.bottom = placement.bottom ?? 0;
+ transformY = `${(placement.top ?? 0) * (sceneHeight! / 100)}px`;
+ style.height = `${sceneHeight! - (placement.top ?? 0) * (sceneHeight! / 100) - (placement.bottom ?? 0) * (sceneHeight! / 100)}px`;
+ delete placement.height;
+ break;
+ }
+
+ switch (horizontal) {
+ case HorizontalConstraint.Left:
+ placement.left = placement.left ?? 0;
+ placement.width = placement.width ?? 100;
+ transformX = `${placement.left ?? 0}px`;
+ style.width = `${placement.width}px`;
+ delete placement.right;
+ break;
+ case HorizontalConstraint.Right:
+ placement.right = placement.right ?? 0;
+ placement.width = placement.width ?? 100;
+ transformX = `${sceneWidth! - (placement.right ?? 0) - (placement.width ?? 100)}px`;
+ style.width = `${placement.width}px`;
+ delete placement.left;
+ break;
+ case HorizontalConstraint.LeftRight:
+ placement.left = placement.left ?? 0;
+ placement.right = placement.right ?? 0;
+ transformX = `${placement.left ?? 0}px`;
+ style.width = `${sceneWidth! - (placement.left ?? 0) - (placement.right ?? 0)}px`;
+ delete placement.width;
+ break;
+ case HorizontalConstraint.Center:
+ placement.left = placement.left ?? 0;
+ placement.width = placement.width ?? 100;
+ transformX = `${sceneWidth! / 2 - (placement.left ?? 0) - (placement.width ?? 0) / 2}px`;
+ style.width = `${placement.width}px`;
+ delete placement.right;
+ break;
+ case HorizontalConstraint.Scale:
+ placement.left = placement.left ?? 0;
+ placement.right = placement.right ?? 0;
+ transformX = `${(placement.left ?? 0) * (sceneWidth! / 100)}px`;
+ style.width = `${sceneWidth! - (placement.left ?? 0) * (sceneWidth! / 100) - (placement.right ?? 0) * (sceneWidth! / 100)}px`;
+ delete placement.width;
+ break;
+ }
+ this.options.placement = placement;
+ style.transform = `translate(${transformX}, ${transformY}) rotate(${placement.rotation ?? 0}deg)`;
+ this.sizeStyle = style;
+
+ if (this.div) {
+ applyStyles(this.sizeStyle, this.div);
+
+ // TODO: This is a hack, we should have a better way to handle this
+ const elementType = this.options.type;
+ if (!SVGElements.has(elementType)) {
+ // apply styles to div if it's not an SVG element
+ applyStyles(this.dataStyle, this.div);
+ } else {
+ // ELEMENT IS SVG
+ // clean data styles from div if it's an SVG element; SVG elements have their own data styles;
+ // this is necessary for changing type of element cases;
+ // wrapper div element (this.div) doesn't re-render (has static `key` property),
+ // so we have to clean styles manually;
+ removeStyles(this.dataStyle, this.div);
+ }
+ }
+ }
+
+ getTopLeftValues(element: Element) {
+ const style = window.getComputedStyle(element);
+ const matrix = new DOMMatrix(style.transform || '');
+ return {
+ left: matrix.m41,
+ top: matrix.m42,
+ width: style.width ? parseFloat(style.width) : element.clientWidth,
+ height: style.height ? parseFloat(style.height) : element.clientHeight,
+ }; // m41 = translateX, m42 = translateY
+ }
+
setPlacementFromConstraint(elementContainer?: DOMRect, parentContainer?: DOMRect, transformScale = 1) {
+ if (config.featureToggles.canvasPanelPanZoom) {
+ this.setPlacementFromConstraint2(elementContainer, parentContainer, transformScale);
+ return;
+ }
const { constraint } = this.options;
const { vertical, horizontal } = constraint ?? {};
@@ -379,6 +532,101 @@ export class ElementState implements LayerElement {
this.getScene()?.save();
}
+ setPlacementFromConstraint2(elementContainer?: DOMRect, parentContainer?: DOMRect, transformScale = 1) {
+ const scene = this.getScene()!;
+ const { constraint } = this.options;
+ const { vertical, horizontal } = constraint ?? {};
+
+ const elementRect = this.getTopLeftValues(this.div!);
+
+ if (!elementContainer) {
+ elementContainer = this.div && this.div.getBoundingClientRect();
+ }
+ // let parentBorderWidth = 0;
+ if (!parentContainer) {
+ parentContainer = this.div && this.div.parentElement?.getBoundingClientRect();
+ }
+
+ const relativeTop = Math.round(elementRect.top);
+ const relativeBottom = Math.round(scene.height - elementRect.top - elementRect.height);
+ const relativeLeft = Math.round(elementRect.left);
+ const relativeRight = Math.round(scene.width - elementRect.left - elementRect.width);
+
+ const placement: Placement = {};
+
+ const width = elementRect.width;
+ const height = elementRect.height;
+
+ // INFO: calculate it anyway to be able to use it for pan&zoom
+ placement.top = relativeTop;
+ placement.left = relativeLeft;
+
+ switch (vertical) {
+ case VerticalConstraint.Top:
+ placement.top = relativeTop;
+ placement.height = height;
+ break;
+ case VerticalConstraint.Bottom:
+ placement.bottom = relativeBottom;
+ placement.height = height;
+ break;
+ case VerticalConstraint.TopBottom:
+ placement.top = relativeTop;
+ placement.bottom = relativeBottom;
+ break;
+ case VerticalConstraint.Center:
+ const elementCenter = elementContainer ? relativeTop + height / 2 : 0;
+ const parentCenter = scene.height / 2; // Use scene height instead of scaled viewport height
+ const distanceFromCenter = parentCenter - elementCenter;
+ placement.top = distanceFromCenter;
+ placement.height = height;
+ break;
+ case VerticalConstraint.Scale:
+ placement.top = (relativeTop / (parentContainer?.height ?? height)) * 100 * transformScale;
+ placement.bottom = (relativeBottom / (parentContainer?.height ?? height)) * 100 * transformScale;
+ break;
+ }
+
+ switch (horizontal) {
+ case HorizontalConstraint.Left:
+ placement.left = relativeLeft;
+ placement.width = width;
+ break;
+ case HorizontalConstraint.Right:
+ placement.right = relativeRight;
+ placement.width = width;
+ break;
+ case HorizontalConstraint.LeftRight:
+ placement.left = relativeLeft;
+ placement.right = relativeRight;
+ break;
+ case HorizontalConstraint.Center:
+ const elementCenter = elementContainer ? relativeLeft + width / 2 : 0;
+ const parentCenter = scene.width / 2; // Use scene width instead of scaled viewport width
+ const distanceFromCenter = parentCenter - elementCenter;
+ placement.left = distanceFromCenter;
+ placement.width = width;
+ break;
+ case HorizontalConstraint.Scale:
+ placement.left = (relativeLeft / (parentContainer?.width ?? width)) * 100 * transformScale;
+ placement.right = (relativeRight / (parentContainer?.width ?? width)) * 100 * transformScale;
+ break;
+ }
+
+ if (this.options.placement?.rotation) {
+ placement.rotation = this.options.placement.rotation;
+ placement.width = this.options.placement.width;
+ placement.height = this.options.placement.height;
+ }
+
+ this.options.placement = placement;
+
+ this.applyLayoutStylesToDiv();
+ this.revId++;
+
+ this.getScene()?.save();
+ }
+
updateData(ctx: DimensionContext) {
if (this.item.prepareData) {
this.data = this.item.prepareData(ctx, this.options);
@@ -394,6 +642,8 @@ export class ElementState implements LayerElement {
this.oneClickMode = OneClickMode.Link;
} else if (this.options.actions?.some((action) => action.oneClick === true)) {
this.oneClickMode = OneClickMode.Action;
+ } else {
+ this.oneClickMode = OneClickMode.Off;
}
if (frames) {
@@ -564,12 +814,12 @@ export class ElementState implements LayerElement {
// kinda like:
// https://github.com/grafana/grafana-edge-app/blob/main/src/panels/draw/WrapItem.tsx#L44
- applyResize = (event: OnResize, transformScale = 1) => {
+ applyResize = (event: OnResize) => {
const placement = this.options.placement!;
const style = event.target.style;
- let deltaX = event.delta[0] / transformScale;
- let deltaY = event.delta[1] / transformScale;
+ let deltaX = event.delta[0];
+ let deltaY = event.delta[1];
let dirLR = event.direction[0];
let dirTB = event.direction[1];
@@ -590,14 +840,22 @@ export class ElementState implements LayerElement {
} else if (dirLR === -1) {
placement.left! -= deltaX;
placement.width = event.width;
- style.left = `${placement.left}px`;
+ if (config.featureToggles.canvasPanelPanZoom) {
+ style.transform = `translate(${placement.left}px, ${placement.top}px) rotate(${placement.rotation ?? 0}deg)`;
+ } else {
+ style.left = `${placement.left}px`;
+ }
style.width = `${placement.width}px`;
}
if (dirTB === -1) {
placement.top! -= deltaY;
placement.height = event.height;
- style.top = `${placement.top}px`;
+ if (config.featureToggles.canvasPanelPanZoom) {
+ style.transform = `translate(${placement.left}px, ${placement.top}px) rotate(${placement.rotation ?? 0}deg)`;
+ } else {
+ style.top = `${placement.top}px`;
+ }
style.height = `${placement.height}px`;
} else if (dirTB === 1) {
placement.height = event.height;
@@ -608,7 +866,7 @@ export class ElementState implements LayerElement {
handleMouseEnter = (event: React.MouseEvent, isSelected: boolean | undefined) => {
const scene = this.getScene();
- const shouldHandleTooltip = !scene?.isEditingEnabled && !scene?.tooltip?.isOpen;
+ const shouldHandleTooltip = !scene?.isEditingEnabled && !scene?.tooltipPayload?.isOpen;
if (shouldHandleTooltip) {
this.handleTooltip(event);
} else if (!isSelected) {
@@ -675,7 +933,7 @@ export class ElementState implements LayerElement {
handleTooltip = (event: React.MouseEvent) => {
const scene = this.getScene();
- if (scene?.tooltipCallback) {
+ if (scene?.tooltipCallback && scene.tooltipMode !== TooltipDisplayMode.None) {
const rect = this.div?.getBoundingClientRect();
scene.tooltipCallback({
anchorPoint: { x: rect?.right ?? event.pageX, y: rect?.top ?? event.pageY },
@@ -687,7 +945,7 @@ export class ElementState implements LayerElement {
handleMouseLeave = (event: React.MouseEvent) => {
const scene = this.getScene();
- if (scene?.tooltipCallback && !scene?.tooltip?.isOpen) {
+ if (scene?.tooltipCallback && !scene?.tooltipPayload?.isOpen) {
scene.tooltipCallback(undefined);
}
@@ -705,8 +963,16 @@ export class ElementState implements LayerElement {
window.open(primaryDataLink.href, primaryDataLink.target ?? '_self');
}
} else if (this.oneClickMode === OneClickMode.Action) {
- this.showConfirmation = true;
- this.forceUpdate();
+ const primaryAction = this.getPrimaryAction();
+ const actionHasVariables = primaryAction?.variables && primaryAction.variables.length > 0;
+
+ if (actionHasVariables) {
+ this.showActionVarsModal = true;
+ this.forceUpdate();
+ } else {
+ this.showActionConfirmation = true;
+ this.forceUpdate();
+ }
} else {
this.handleTooltip(event);
this.onTooltipCallback();
@@ -725,9 +991,9 @@ export class ElementState implements LayerElement {
onTooltipCallback = () => {
const scene = this.getScene();
- if (scene?.tooltipCallback && scene.tooltip?.anchorPoint) {
+ if (scene?.tooltipCallback && scene.tooltipPayload?.anchorPoint) {
scene.tooltipCallback({
- anchorPoint: { x: scene.tooltip.anchorPoint.x, y: scene.tooltip.anchorPoint.y },
+ anchorPoint: { x: scene.tooltipPayload.anchorPoint.x, y: scene.tooltipPayload.anchorPoint.y },
element: this,
isOpen: true,
});
@@ -748,7 +1014,7 @@ export class ElementState implements LayerElement {
return (
<>
- {this.showConfirmation && action && (
+ {this.showActionConfirmation && action && (
+ <>
{this.connections.render()}
{this.root.render()}
{this.isEditingEnabled && (
@@ -307,13 +394,30 @@ export class Scene {
+
);
return config.featureToggles.canvasPanelPanZoom ? (
-
+
) : (
- sceneDiv
+
+ {sceneDiv}
+
+
+ {sceneDiv}
+
);
}
}
@@ -323,4 +427,16 @@ const getStyles = () => ({
overflow: 'hidden',
position: 'relative',
}),
+ selected: css({
+ zIndex: '999 !important',
+ }),
+ viewer: css({
+ overflow: 'hidden',
+ width: '100%',
+ height: '100%',
+ }),
+ viewport: css({
+ width: '100%',
+ height: '100%',
+ }),
});
diff --git a/public/app/features/canvas/runtime/sceneAbleManagement.ts b/public/app/features/canvas/runtime/sceneAbleManagement.ts
index 0befab40297..9af3e1a74e0 100644
--- a/public/app/features/canvas/runtime/sceneAbleManagement.ts
+++ b/public/app/features/canvas/runtime/sceneAbleManagement.ts
@@ -1,13 +1,14 @@
+import InfiniteViewer from 'infinite-viewer';
import Moveable from 'moveable';
import Selecto from 'selecto';
+import { config } from 'app/core/config';
import { CONNECTION_ANCHOR_DIV_ID } from 'app/plugins/panel/canvas/components/connections/ConnectionAnchors';
import {
CONNECTION_VERTEX_ID,
CONNECTION_VERTEX_ADD_ID,
} from 'app/plugins/panel/canvas/components/connections/Connections';
import { VerticalConstraint, HorizontalConstraint } from 'app/plugins/panel/canvas/panelcfg.gen';
-import { getParent } from 'app/plugins/panel/canvas/utils';
import { dimensionViewable, constraintViewable, settingsViewable } from './ables';
import { ElementState } from './element';
@@ -15,6 +16,8 @@ import { FrameState } from './frame';
import { Scene } from './scene';
import { findElementByTarget } from './sceneElementManagement';
+const ZOOM_RANGE = [0.1, 4]; // Minimum zoom 0.1x (10%), maximum zoom 4x (400%)
+
// Helper function that disables custom able functionality
const disableCustomables = (moveable: Moveable) => {
moveable!.props = {
@@ -95,8 +98,8 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
}
scene.selecto = new Selecto({
- container: scene.div,
- rootContainer: getParent(scene),
+ rootContainer: config.featureToggles.canvasPanelPanZoom ? scene.viewerDiv : scene.div,
+ dragContainer: config.featureToggles.canvasPanelPanZoom ? scene.viewerDiv : scene.div,
selectableTargets: targetElements,
toggleContinueSelect: 'shift',
selectFromInside: false,
@@ -106,7 +109,7 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
const snapDirections = { top: true, left: true, bottom: true, right: true, center: true, middle: true };
const elementSnapDirections = { top: true, left: true, bottom: true, right: true, center: true, middle: true };
- scene.moveable = new Moveable(scene.div!, {
+ scene.moveable = new Moveable(config.featureToggles.canvasPanelPanZoom ? scene.viewerDiv! : scene.div!, {
draggable: allowChanges && !scene.editModeEnabled.getValue(),
resizable: allowChanges,
@@ -137,6 +140,12 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
if (targetedElement) {
targetedElement.applyRotate(event);
+
+ if (config.featureToggles.canvasPanelPanZoom) {
+ if (scene.connections.connectionsNeedUpdate(targetedElement) && scene.moveableActionCallback) {
+ scene.moveableActionCallback(true);
+ }
+ }
}
})
.on('rotateGroup', (e) => {
@@ -221,9 +230,7 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
e.events.forEach((event) => {
const targetedElement = findElementByTarget(event.target, scene.root.elements);
if (targetedElement) {
- if (targetedElement) {
- targetedElement.setPlacementFromConstraint(undefined, undefined, scene.scale);
- }
+ targetedElement.setPlacementFromConstraint(undefined, undefined, scene.scale);
// re-add the selected elements to the snappable guidelines
if (scene.moveable && scene.moveable.elementGuidelines) {
@@ -280,11 +287,23 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
}
}
}
+ // Temporarily set Top-Left constraints on each group element for predictable resizing; restore originals on end.
+ for (let event of e.events) {
+ const targetedElement = findElementByTarget(event.target, scene.root.elements);
+ if (targetedElement) {
+ targetedElement.tempConstraint = { ...targetedElement.options.constraint };
+ targetedElement.options.constraint = {
+ vertical: VerticalConstraint.Top,
+ horizontal: HorizontalConstraint.Left,
+ };
+ targetedElement.setPlacementFromConstraint(undefined, undefined, scene.scale);
+ }
+ }
})
.on('resize', (event) => {
const targetedElement = findElementByTarget(event.target, scene.root.elements);
if (targetedElement) {
- targetedElement.applyResize(event, scene.scale);
+ targetedElement.applyResize(event);
if (scene.connections.connectionsNeedUpdate(targetedElement) && scene.moveableActionCallback) {
scene.moveableActionCallback(true);
@@ -319,7 +338,6 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
targetedElement.options.constraint = targetedElement.tempConstraint;
targetedElement.tempConstraint = undefined;
}
-
targetedElement.setPlacementFromConstraint(undefined, undefined, scene.scale);
// re-add the selected element to the snappable guidelines
@@ -409,4 +427,194 @@ export const initMoveable = (destroySelecto = false, allowChanges = true, scene:
.on('dragEnd', (event) => {
clearTimeout(event.data.timer);
});
+
+ if (config.featureToggles.canvasPanelPanZoom) {
+ /******************/
+ /* infiniteViewer */
+ /******************/
+ scene.infiniteViewer = new InfiniteViewer(scene.viewerDiv!, scene.viewportDiv!, {
+ preventWheelClick: false,
+ useAutoZoom: true,
+ useMouseDrag: false, // `true` blocks metricValue dropdown
+ useWheelScroll: scene.shouldPanZoom,
+ displayHorizontalScroll: false,
+ displayVerticalScroll: false,
+ zoomRange: ZOOM_RANGE,
+ });
+ scene.infiniteViewer.setZoom(scene.scale);
+ scene.infiniteViewer.scrollTo(scene.scrollLeft, scene.scrollTop);
+
+ // Handles context menu activation
+ // Uses openContextMenu with coordinates when available (after CanvasContextMenu mounts), but
+ // uses the basic visibility toggle when openContextMenu isn't ready (as a fallback)
+ const triggerContextMenu = (x: number, y: number) => {
+ if (scene.openContextMenu) {
+ scene.openContextMenu({ x, y });
+ } else {
+ scene.contextMenuOnVisibilityChange(true);
+ }
+ };
+
+ /* ----------------------------- EVENT HANDLERS ----------------------------- */
+ // Helper for panning with mouse drag (middle mouse or Ctrl+right-click)
+ // TODO: It was implemented as a workaround to unblock left click metricsValue dropdown,
+ // but it should be replaced with a more robust solution that doesn't interfere with left click interactions.
+ function startPanning(e: MouseEvent) {
+ e.preventDefault();
+
+ const startX = e.clientX;
+ const startY = e.clientY;
+ const startScrollLeft = scene.infiniteViewer!.getScrollLeft();
+ const startScrollTop = scene.infiniteViewer!.getScrollTop();
+
+ const handleMouseMove = (moveEvent: MouseEvent) => {
+ const deltaX = startX - moveEvent.clientX;
+ const deltaY = startY - moveEvent.clientY;
+ const scaleAdjustedDeltaX = deltaX / scene.scale;
+ const scaleAdjustedDeltaY = deltaY / scene.scale;
+ scene.infiniteViewer!.scrollTo(startScrollLeft + scaleAdjustedDeltaX, startScrollTop + scaleAdjustedDeltaY);
+ moveEvent.preventDefault();
+ };
+
+ const handleMouseUp = () => {
+ document.removeEventListener('mousemove', handleMouseMove);
+ document.removeEventListener('mouseup', handleMouseUp);
+ };
+
+ document.addEventListener('mousemove', handleMouseMove);
+ document.addEventListener('mouseup', handleMouseUp);
+ }
+
+ // Right click
+ scene.viewerDiv!.addEventListener('contextmenu', (e) => {
+ if (e.ctrlKey && e.button === 2 && scene.shouldPanZoom) {
+ // Enable panning with Ctrl+right-click
+ startPanning(e);
+ } else {
+ // Prevent default browser context menu
+ e.preventDefault();
+ triggerContextMenu(e.pageX, e.pageY);
+ }
+ });
+
+ // Enable panning with middle mouse button (wheel button)
+ scene.viewerDiv!.addEventListener('mousedown', (e: MouseEvent) => {
+ if (e.button === 1 && scene.shouldPanZoom) {
+ // Middle mouse button
+ startPanning(e);
+ }
+ });
+
+ // Prevent wheel scrolling when pan/zoom is disabled
+ scene.viewportDiv!.addEventListener(
+ 'wheel',
+ (e) => {
+ if (!scene.shouldPanZoom) {
+ e.stopImmediatePropagation();
+ e.preventDefault();
+ }
+ },
+ { passive: false }
+ );
+
+ // Reset zoom and scroll position on double click
+ scene.viewerDiv!.addEventListener('dblclick', (e: MouseEvent) => {
+ // Only reset if not in edit mode and pan/zoom is enabled
+ if (!scene.editModeEnabled.getValue() && scene.shouldPanZoom && scene.infiniteViewer) {
+ scene.infiniteViewer.setZoom(1);
+ scene.infiniteViewer.scrollTo(0, 0);
+ }
+ });
+
+ // Mouse scroll click
+ // Only allow panning with middle mouse button (button 1)
+ // Left click is reserved for selection/manipulation, right click for context menu
+ scene.infiniteViewer!.on('dragStart', (e) => {
+ if (e.inputEvent.button !== 1) {
+ e.preventDefault();
+ e.preventDrag();
+ }
+ });
+
+ // Scroll
+ scene.infiniteViewer!.on('scroll', () => {
+ // TODO: clear current selection is default behaviour on zoom-in or zoom-out,
+ // but looks like we prevented this event to trigger at some point
+ scene.clearCurrentSelection(true);
+
+ scene.updateConnectionsSize();
+ scene.scale = scene.infiniteViewer!.getZoom();
+
+ scene.scrollLeft = scene.infiniteViewer!.getScrollLeft();
+ scene.scrollTop = scene.infiniteViewer!.getScrollTop();
+ });
+ }
};
+
+// Zoom to content helper functions
+export function calculateZoomToFitScale(elements: Element[], container: HTMLDivElement, paddingRatio = 0.05) {
+ const bounds = calculateGroupBoundingBox(elements);
+ const containerRect = container.getBoundingClientRect();
+ const containerWidth = containerRect.width;
+ const containerHeight = containerRect.height;
+
+ const paddedWidth = containerWidth * (1 - 2 * paddingRatio);
+ const paddedHeight = containerHeight * (1 - 2 * paddingRatio);
+
+ const scaleX = paddedWidth / bounds.width;
+ const scaleY = paddedHeight / bounds.height;
+
+ // Use the smaller one to fit both horizontally and vertically
+ const scale = Math.min(scaleX, scaleY);
+
+ // calculate value to move to center
+ const centerX = (bounds.centerX * scale - containerWidth / 2) / scale;
+ const centerY = (bounds.centerY * scale - containerHeight / 2) / scale;
+
+ return {
+ scale,
+ centerX,
+ centerY,
+ };
+}
+
+export function extractTranslateFromTransform(transform: string) {
+ const matrix = new DOMMatrix(transform);
+ return { x: matrix.m41, y: matrix.m42 }; // m41 = translateX, m42 = translateY
+}
+
+export function calculateGroupBoundingBox(elements: Element[]) {
+ let minX = Infinity,
+ minY = Infinity;
+ let maxX = -Infinity,
+ maxY = -Infinity;
+
+ for (const el of elements) {
+ const style = window.getComputedStyle(el);
+ const { x: tx, y: ty } = extractTranslateFromTransform(style.transform || '');
+
+ const width = parseFloat(style.width);
+ const height = parseFloat(style.height);
+
+ const left = tx;
+ const top = ty;
+ const right = tx + width;
+ const bottom = ty + height;
+
+ minX = Math.min(minX, left);
+ minY = Math.min(minY, top);
+ maxX = Math.max(maxX, right);
+ maxY = Math.max(maxY, bottom);
+ }
+
+ return {
+ left: minX,
+ top: minY,
+ right: maxX,
+ bottom: maxY,
+ width: maxX - minX,
+ height: maxY - minY,
+ centerX: (minX + maxX) / 2,
+ centerY: (minY + maxY) / 2,
+ };
+}
diff --git a/public/app/features/dashboard-scene/pages/DashboardScenePageStateManager.ts b/public/app/features/dashboard-scene/pages/DashboardScenePageStateManager.ts
index d91cf0d2230..63f1aaa0bcd 100644
--- a/public/app/features/dashboard-scene/pages/DashboardScenePageStateManager.ts
+++ b/public/app/features/dashboard-scene/pages/DashboardScenePageStateManager.ts
@@ -267,7 +267,7 @@ abstract class DashboardScenePageStateManagerBase
@@ -161,9 +171,9 @@ export default function AccordianKeyValues({
data-testid="AccordianKeyValues--header"
>
{arrow}
-
+
{label}
- {showDataSummaryFields && ':'}
+ {showCountBadge ?
- {isOpen &&
-
- );
-}
diff --git a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/KeyValuesTable.tsx b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/KeyValuesTable.tsx
index 83632c9351d..a6c79b2d2cf 100644
--- a/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/KeyValuesTable.tsx
+++ b/public/app/features/explore/TraceView/components/TraceTimelineViewer/SpanDetail/KeyValuesTable.tsx
@@ -47,7 +47,7 @@ export const getStyles = (theme: GrafanaTheme2) => {
row: css({
label: 'row',
'& > td': {
- padding: '0.5rem 0.5rem',
+ padding: '0 0.5rem',
height: '30px',
},
'&:nth-child(2n) > td': {
@@ -68,7 +68,6 @@ export const getStyles = (theme: GrafanaTheme2) => {
color: autoColor(theme, '#888'),
whiteSpace: 'pre',
width: '125px',
- verticalAlign: 'top',
}),
copyColumn: css({
label: 'copyColumn',
@@ -118,19 +117,32 @@ export const LinkValue = ({ link, children }: PropsWithChildren
- {arrow}
- {label} ({data.length})
-
- {isOpen && | - {row.key} - | + {!onlyValues && ( ++ {row.key} + | + )}{valueMarkup} |
|