public/grafana
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

merge main

Browse Source
This commit is contained in:
Ryan McKinley committed 2025-02-11 21:01:39 +03:00
2 parents 3509e795b4 a5355fd66c
commit 459240e9a8
69 files changed
+4355 -1147
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/create-security-patch-from-security-mirror.yml
+1
View File
@@ -11,6 +11,7 @@ on:
branches:
- "main"
- "v*.*.*"
- "release-*.*.*"
# This is run before the pull request has been merged, so we'll run against the src branch
jobs:
docs/sources/datasources/prometheus/configure-prometheus-data-source.md
+21 -1
View File
@@ -48,6 +48,8 @@ Grafana ships with built-in support for Prometheus. If you are new to Prometheus
## Configure the data source
{{< shared id="add-prom-data-source" >}}
To add the Prometheus data source, complete the following steps:
1. Click **Connections** in the left-side menu.
@@ -55,6 +57,9 @@ To add the Prometheus data source, complete the following steps:
1. Enter `Prometheus` in the search bar.
1. Select **Prometheus**.
1. Click **Add new data source** in the upper right.
1. Enter a name for the data source.
{{< /shared >}}
You will be taken to the **Settings** tab where you will set up your Prometheus configuration.
@@ -70,7 +75,8 @@ The first option to configure is the name of your connection:
### Connection section
- **Prometheus server URL** - The URL of your Prometheus server. If your Prometheus server is local, use `http://localhost:9090`. If it is on a server within a network, this is the URL with port where you are running Prometheus. Example: `http://prometheus.example.orgname:9090`.
- **Prometheus server URL** - The URL of your Prometheus server. {{< shared id="prom-data-source-url" >}}
If your Prometheus server is local, use `http://localhost:9090`. If it's on a server within a network, this is the URL with the port where you are running Prometheus. Example: `http://prometheus.example.orgname:9090`.
{{< admonition type="note" >}}
@@ -80,6 +86,8 @@ You should use the IP address of the Prometheus container, or the hostname if yo
{{< /admonition >}}
{{< /shared >}}
### Authentication section
There are several authentication methods you can choose in the Authentication section.
@@ -171,3 +179,15 @@ Support for exemplars is available only for the Prometheus data source. If this
- **Label name** - The name of the field in the `labels` object used to obtain the traceID property.
- **Remove exemplar link** - Click to remove existing links.
### Troubleshooting
Refer to the following troubleshooting information, as required.
#### Data doesn't appear in Explore metrics
If metric data doesn't appear in Explore after you've successfully tested a connection to a Prometheus data source, ensure that you've selected the correct data source in the **Data source** drop-down menu.
The following image shows the **Data source** field in Explore metrics.
![Image that shows Prometheus metrics in Explore](/media/docs/grafana/data-sources/prometheus/troubleshoot-connection-1.png)
docs/sources/setup-grafana/installation/debian/index.md
+81 -7
View File
@@ -18,9 +18,9 @@ This topic explains how to install Grafana dependencies, install Grafana on Linu
There are multiple ways to install Grafana: using the Grafana Labs APT repository, by downloading a `.deb` package, or by downloading a binary `.tar.gz` file. Choose only one of the methods below that best suits your needs.
{{% admonition type="note" %}}
{{< admonition type="note" >}}
If you install via the `.deb` package or `.tar.gz` file, then you must manually update Grafana for each new version.
{{% /admonition %}}
{{< /admonition >}}
The following video demonstrates how to install Grafana on Debian and Ubuntu as outlined in this document:
@@ -37,9 +37,9 @@ If you install from the APT repository, Grafana automatically updates when you r
| Grafana OSS | grafana | `https://apt.grafana.com stable main` |
| Grafana OSS (Beta) | grafana | `https://apt.grafana.com beta main` |
{{% admonition type="note" %}}
{{< admonition type="note" >}}
Grafana Enterprise is the recommended and default edition. It is available for free and includes all the features of the OSS edition. You can also upgrade to the [full Enterprise feature set](/products/enterprise/?utm_source=grafana-install-page), which has support for [Enterprise plugins](/grafana/plugins/?enterprise=1&utcm_source=grafana-install-page).
{{% /admonition %}}
{{< /admonition >}}
Complete the following steps to install Grafana from the APT repository:
@@ -89,11 +89,11 @@ Complete the following steps to install Grafana from the APT repository:
sudo apt-get install grafana-enterprise
```
## Install Grafana using a deb package or as a standalone binary
## Install Grafana using a deb package
If you choose not to install Grafana using APT, you can download and install Grafana using the deb package or as a standalone binary.
If you install Grafana manually using the deb package, then you must manually update Grafana for each new version.
Complete the following steps to install Grafana using DEB or the standalone binaries:
Complete the following steps to install Grafana using a deb package:
1. Navigate to the [Grafana download page](/grafana/download).
1. Select the Grafana version you want to install.
@@ -105,6 +105,80 @@ Complete the following steps to install Grafana using DEB or the standalone bina
1. Depending on which system you are running, click the **Linux** or **ARM** tab on the [download page](/grafana/download).
1. Copy and paste the code from the [download page](/grafana/download) into your command line and run.
## Install Grafana as a standalone binary
Complete the following steps to install Grafana using the standalone binaries:
1. Navigate to the [Grafana download page](/grafana/download).
1. Select the Grafana version you want to install.
- The most recent Grafana version is selected by default.
- The **Version** field displays only tagged releases. If you want to install a nightly build, click **Nightly Builds** and then select a version.
1. Select an **Edition**.
- **Enterprise:** This is the recommended version. It is functionally identical to the open source version but includes features you can unlock with a license if you so choose.
- **Open Source:** This version is functionally identical to the Enterprise version, but you will need to download the Enterprise version if you want Enterprise features.
1. Depending on which system you are running, click the **Linux** or **ARM** tab on the [download page](/grafana/download).
1. Copy and paste the code from the [download page](/grafana/download) page into your command line and run.
1. Create a user account for Grafana on your system:
```shell
sudo useradd -r -s /bin/false grafana
```
1. Move the unpacked binary to `/usr/local/grafana`:
```shell
sudo mv <DOWNLOAD PATH> /usr/local/grafana
```
1. Change the owner of `/usr/local/grafana` to Grafana users:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. Create a Grafana server systemd unit file:
```shell
sudo touch /etc/systemd/system/grafana-server.service
```
1. Add the following to the unit file in a text editor of your choice:
```ini
[Unit]
Description=Grafana Server
After=network.target
[Service]
Type=simple
User=grafana
Group=users
ExecStart=/usr/local/grafana/bin/grafana server --config=/usr/local/grafana/conf/grafana.ini --homepath=/usr/local/grafana
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
1. Use the binary to manually start the Grafana server:
```shell
/usr/local/grafana/bin/grafana-server --homepath /usr/local/grafana
```
{{< admonition type="note" >}}
Manually invoking the binary in this step automatically creates the `/usr/local/grafana/data` directory, which needs to be created and configured before the installation can be considered complete.
{{< /admonition >}}
1. Press `CTRL+C` to stop the Grafana server.
1. Change the owner of `/usr/local/grafana` to Grafana users again to apply the ownership to the newly created `/usr/local/grafana/data` directory:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. [Configure the Grafana server to start at boot time using systemd](https://grafana.com/docs/grafana/latest/setup-grafana/start-restart-grafana/#configure-the-grafana-server-to-start-at-boot-using-systemd).
## Uninstall on Debian or Ubuntu
Complete any of the following steps to uninstall Grafana.
docs/sources/setup-grafana/installation/redhat-rhel-fedora/index.md
+67 -5
View File
@@ -32,15 +32,15 @@ If you install from the RPM repository, then Grafana is automatically updated ev
| Grafana OSS | grafana | `https://rpm.grafana.com` |
| Grafana OSS (Beta) | grafana | `https://rpm-beta.grafana.com` |
{{% admonition type="note" %}}
{{< admonition type="note" >}}
Grafana Enterprise is the recommended and default edition. It is available for free and includes all the features of the OSS edition. You can also upgrade to the [full Enterprise feature set](/products/enterprise/?utm_source=grafana-install-page), which has support for [Enterprise plugins](/grafana/plugins/?enterprise=1&utcm_source=grafana-install-page).
{{% /admonition %}}
{{< /admonition >}}
To install Grafana from the RPM repository, complete the following steps:
{{% admonition type="note" %}}
{{< admonition type="note" >}}
If you wish to install beta versions of Grafana, substitute the repository URL for the beta URL listed above.
{{% /admonition %}}
{{< /admonition >}}
1. Import the GPG key:
@@ -96,6 +96,8 @@ If you install Grafana manually using YUM or RPM, then you must manually update
## Install Grafana as a standalone binary
If you install Grafana manually using the standalone binaries, then you must manually update Grafana for each new version.
Complete the following steps to install Grafana using the standalone binaries:
1. Navigate to the [Grafana download page](/grafana/download).
@@ -103,10 +105,70 @@ Complete the following steps to install Grafana using the standalone binaries:
- The most recent Grafana version is selected by default.
- The **Version** field displays only tagged releases. If you want to install a nightly build, click **Nightly Builds** and then select a version.
1. Select an **Edition**.
- **Enterprise:** This is the recommended version. It is functionally identical to the open-source version but includes features you can unlock with a license if you so choose.
- **Enterprise:** This is the recommended version. It is functionally identical to the open source version but includes features you can unlock with a license if you so choose.
- **Open Source:** This version is functionally identical to the Enterprise version, but you will need to download the Enterprise version if you want Enterprise features.
1. Depending on which system you are running, click the **Linux** or **ARM** tab on the [download page](/grafana/download).
1. Copy and paste the code from the [download page](/grafana/download) page into your command line and run.
1. Create a user account for Grafana on your system:
```shell
sudo useradd -r -s /bin/false grafana
```
1. Move the unpacked binary to `/usr/local/grafana`:
```shell
sudo mv <DOWNLOAD PATH> /usr/local/grafana
```
1. Change the owner of `/usr/local/grafana` to Grafana users:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. Create a Grafana server systemd unit file:
```shell
sudo touch /etc/systemd/system/grafana-server.service
```
1. Add the following to the unit file in a text editor of your choice:
```ini
[Unit]
Description=Grafana Server
After=network.target
[Service]
Type=simple
User=grafana
Group=users
ExecStart=/usr/local/grafana/bin/grafana server --config=/usr/local/grafana/conf/grafana.ini --homepath=/usr/local/grafana
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
1. Use the binary to manually start the Grafana server:
```shell
/usr/local/grafana/bin/grafana-server --homepath /usr/local/grafana
```
{{< admonition type="note" >}}
Manually invoking the binary in this step automatically creates the `/usr/local/grafana/data` directory, which needs to be created and configured before the installation can be considered complete.
{{< /admonition >}}
1. Press `CTRL+C` to stop the Grafana server.
1. Change the owner of `/usr/local/grafana` to Grafana users again to apply the ownership to the newly created `/usr/local/grafana/data` directory:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. [Configure the Grafana server to start at boot time using systemd]({{< relref "../../start-restart-grafana#configure-the-grafana-server-to-start-at-boot-using-systemd" >}}).
## Uninstall on RHEL or Fedora
docs/sources/setup-grafana/installation/suse-opensuse/index.md
+65 -3
View File
@@ -30,9 +30,9 @@ If you install from the RPM repository, then Grafana is automatically updated ev
| Grafana Enterprise | grafana-enterprise | `https://rpm.grafana.com` |
| Grafana OSS | grafana | `https://rpm.grafana.com` |
{{% admonition type="note" %}}
{{< admonition type="note" >}}
Grafana Enterprise is the recommended and default edition. It is available for free and includes all the features of the OSS edition. You can also upgrade to the [full Enterprise feature set](/products/enterprise/?utm_source=grafana-install-page), which has support for [Enterprise plugins](/grafana/plugins/?enterprise=1&utcm_source=grafana-install-page).
{{% /admonition %}}
{{< /admonition >}}
To install Grafana using the RPM repository, complete the following steps:
@@ -84,6 +84,8 @@ If you install Grafana manually using RPM, then you must manually update Grafana
## Install Grafana as a standalone binary
If you install Grafana manually using the standalone binaries, then you must manually update Grafana for each new version.
Complete the following steps to install Grafana using the standalone binaries:
1. Navigate to the [Grafana download page](/grafana/download).
@@ -91,10 +93,70 @@ Complete the following steps to install Grafana using the standalone binaries:
- The most recent Grafana version is selected by default.
- The **Version** field displays only tagged releases. If you want to install a nightly build, click **Nightly Builds** and then select a version.
1. Select an **Edition**.
- **Enterprise:** This is the recommended version. It is functionally identical to the open-source version but includes features you can unlock with a license if you so choose.
- **Enterprise:** This is the recommended version. It is functionally identical to the open source version but includes features you can unlock with a license if you so choose.
- **Open Source:** This version is functionally identical to the Enterprise version, but you will need to download the Enterprise version if you want Enterprise features.
1. Depending on which system you are running, click the **Linux** or **ARM** tab on the [download page](/grafana/download).
1. Copy and paste the code from the [download page](/grafana/download) into your command line and run.
1. Create a user account for Grafana on your system:
```shell
sudo useradd -r -s /bin/false grafana
```
1. Move the unpacked binary to `/usr/local/grafana`:
```shell
sudo mv <DOWNLOAD PATH> /usr/local/grafana
```
1. Change the owner of `/usr/local/grafana` to Grafana users:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. Create a Grafana server systemd unit file:
```shell
sudo touch /etc/systemd/system/grafana-server.service
```
1. Add the following to the unit file in a text editor of your choice:
```ini
[Unit]
Description=Grafana Server
After=network.target
[Service]
Type=simple
User=grafana
Group=users
ExecStart=/usr/local/grafana/bin/grafana server --config=/usr/local/grafana/conf/grafana.ini --homepath=/usr/local/grafana
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
1. Use the binary to manually start the Grafana server:
```shell
/usr/local/grafana/bin/grafana-server --homepath /usr/local/grafana
```
{{< admonition type="note" >}}
Manually invoking the binary in this step automatically creates the `/usr/local/grafana/data` directory, which needs to be created and configured before the installation can be considered complete.
{{< /admonition >}}
1. Press `CTRL+C` to stop the Grafana server.
1. Change the owner of `/usr/local/grafana` to Grafana users again to apply the ownership to the newly created `/usr/local/grafana/data` directory:
```shell
sudo chown -R grafana:users /usr/local/grafana
```
1. [Configure the Grafana server to start at boot time using systemd](https://grafana.com/docs/grafana/latest/setup-grafana/start-restart-grafana/#configure-the-grafana-server-to-start-at-boot-using-systemd).
## Uninstall on SUSE or openSUSE
go.mod
+5
View File
@@ -244,6 +244,7 @@ require (
github.com/Azure/go-ntlmssp v0.0.0-20220621081337-cb9428e4ac1e // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.2 // indirect
github.com/FZambia/eagle v0.1.0 // indirect
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/squirrel v1.5.4 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
@@ -254,6 +255,7 @@ require (
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect
github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/apache/thrift v0.21.0 // indirect
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
github.com/apparentlymart/go-textseg/v13 v13.0.0 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
@@ -406,6 +408,7 @@ require (
github.com/jszwedko/go-datemath v0.1.1-0.20230526204004-640a500621d6 // indirect
github.com/jtolds/gls v4.20.0+incompatible // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
@@ -425,6 +428,8 @@ require (
github.com/mdlayher/vsock v1.2.1 // indirect
github.com/mfridman/interpolate v0.0.2 // indirect
github.com/miekg/dns v1.1.62 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
go.sum
+1
View File
@@ -708,6 +708,7 @@ github.com/FZambia/eagle v0.1.0 h1:9gyX6x+xjoIfglgyPTcYm7dvY7FJ93us1QY5De4CyXA=
github.com/FZambia/eagle v0.1.0/go.mod h1:YjGSPVkQTNcVLfzEUQJNgW9ScPR0K4u/Ky0yeFa4oDA=
github.com/HdrHistogram/hdrhistogram-go v1.1.2 h1:5IcZpTvzydCQeHzK4Ef/D5rrSqwxob0t8PQPMybUNFM=
github.com/HdrHistogram/hdrhistogram-go v1.1.2/go.mod h1:yDgFjdqOqDEKOvasDdhWNXYg9BVp4O+o5f6V/ehm6Oo=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c h1:RGWPOewvKIROun94nF7v2cua9qP+thov/7M50KEoeSU=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c/go.mod h1:X0CRv0ky0k6m906ixxpzmDRLvX58TFUKS2eePweuyxk=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
package.json
+3 -3
View File
@@ -171,9 +171,9 @@
"cypress": "13.10.0",
"cypress-file-upload": "5.0.8",
"cypress-recurse": "^1.35.3",
"esbuild": "0.24.2",
"esbuild-loader": "4.2.2",
"esbuild-plugin-browserslist": "^0.15.0",
"esbuild": "0.25.0",
"esbuild-loader": "4.3.0",
"esbuild-plugin-browserslist": "^0.16.0",
"eslint": "9.19.0",
"eslint-config-prettier": "9.1.0",
"eslint-plugin-import": "^2.31.0",
packages/grafana-data/package.json
+2 -2
View File
@@ -70,13 +70,13 @@
"@types/react": "18.3.18",
"@types/react-dom": "18.3.5",
"@types/tinycolor2": "1.4.6",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"react": "18.3.1",
"react-dom": "18.3.1",
"rimraf": "6.0.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"typescript": "5.7.3"
},
packages/grafana-e2e-selectors/package.json
+2 -2
View File
@@ -42,11 +42,11 @@
"@rollup/plugin-node-resolve": "16.0.0",
"@types/node": "22.12.0",
"@types/semver": "7.5.8",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"rimraf": "6.0.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0"
},
"dependencies": {
packages/grafana-flamegraph/package.json
+2 -2
View File
@@ -73,12 +73,12 @@
"@types/react-virtualized-auto-sizer": "1.0.4",
"@types/tinycolor2": "1.4.6",
"babel-jest": "29.7.0",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"jest": "^29.6.4",
"jest-canvas-mock": "2.5.2",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"ts-jest": "29.2.5",
"ts-node": "10.9.2",
packages/grafana-icons/package.json
+2 -2
View File
@@ -48,14 +48,14 @@
"@types/node": "22.12.0",
"@types/react": "18.3.18",
"@types/react-dom": "18.3.5",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"prettier": "3.4.2",
"react": "18.3.1",
"react-dom": "18.3.1",
"rimraf": "6.0.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "8.0.0",
"ts-node": "10.9.2",
"typescript": "5.7.3"
packages/grafana-prometheus/package.json
+2 -2
View File
@@ -106,7 +106,7 @@
"@typescript-eslint/parser": "8.22.0",
"copy-webpack-plugin": "12.0.2",
"css-loader": "7.1.2",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"eslint": "9.19.0",
"eslint-config-prettier": "9.1.0",
"eslint-plugin-import": "^2.31.0",
@@ -128,7 +128,7 @@
"react-select-event": "5.5.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"sass": "1.83.4",
"sass-loader": "16.0.4",
packages/grafana-prometheus/src/components/monaco-query-field/MonacoQueryField.tsx
+5 -23
View File
@@ -1,7 +1,6 @@
// Core Grafana history https://github.com/grafana/grafana/blob/v11.0.0-preview/public/app/plugins/datasource/prometheus/components/monaco-query-field/MonacoQueryField.tsx
import { css } from '@emotion/css';
import { parser } from '@prometheus-io/lezer-promql';
import { debounce } from 'lodash';
import { promLanguageDefinition } from 'monaco-promql';
import { useEffect, useRef } from 'react';
import { useLatest } from 'react-use';
@@ -106,13 +105,12 @@ const MonacoQueryField = (props: Props) => {
// we need only one instance of `overrideServices` during the lifetime of the react component
const overrideServicesRef = useRef(getOverrideServices());
const containerRef = useRef<HTMLDivElement>(null);
const { languageProvider, history, onBlur, onRunQuery, initialValue, placeholder, onChange, datasource } = props;
const { languageProvider, history, onBlur, onRunQuery, initialValue, placeholder, datasource } = props;
const lpRef = useLatest(languageProvider);
const historyRef = useLatest(history);
const onRunQueryRef = useLatest(onRunQuery);
const onBlurRef = useLatest(onBlur);
const onChangeRef = useLatest(onChange);
const autocompleteDisposeFun = useRef<(() => void) | null>(null);
@@ -134,6 +132,8 @@ const MonacoQueryField = (props: Props) => {
ref={containerRef}
>
<ReactMonacoEditor
// see https://github.com/suren-atoyan/monaco-react/issues/365
saveViewState
overrideServices={overrideServicesRef.current}
options={options}
language="promql"
@@ -201,23 +201,6 @@ const MonacoQueryField = (props: Props) => {
editor.onDidContentSizeChange(updateElementHeight);
updateElementHeight();
// Whenever the editor changes, lets save the last value so the next query for this editor will be up-to-date.
// This change is being introduced to fix a bug where you can submit a query via shift+enter:
// If you clicked into another field and haven't un-blurred the active field,
// then the query that is run will be stale, as the reference is only updated
// with the value of the last blurred input.
// This can run quite slowly, so we're debouncing this which should accomplish two things
// 1. Should prevent this function from blocking the current call stack by pushing into the web API callback queue
// 2. Should prevent a bunch of duplicates of this function being called as the user is typing
const updateCurrentEditorValue = debounce(() => {
const editorValue = editor.getValue();
onChangeRef.current(editorValue);
}, lpRef.current.datasource.getDebounceTimeInMilliseconds());
editor.getModel()?.onDidChangeContent(() => {
updateCurrentEditorValue();
});
// handle: shift + enter
// FIXME: maybe move this functionality into CodeEditor?
editor.addCommand(
@@ -235,9 +218,8 @@ const MonacoQueryField = (props: Props) => {
command: null,
});
/* Something in this configuration of monaco doesn't bubble up [mod]+K, which the
command palette uses. Pass the event out of monaco manually
*/
// Something in this configuration of monaco doesn't bubble up [mod]+K,
// which the command palette uses. Pass the event out of monaco manually
editor.addCommand(monaco.KeyMod.CtrlCmd | monaco.KeyCode.KeyK, function () {
global.dispatchEvent(new KeyboardEvent('keydown', { key: 'k', metaKey: true }));
});
packages/grafana-prometheus/src/components/monaco-query-field/MonacoQueryFieldLazy.tsx
-2
View File
@@ -4,8 +4,6 @@ import { Suspense } from 'react';
import MonacoQueryField from './MonacoQueryField';
import { Props } from './MonacoQueryFieldProps';
// const Field = React.lazy(() => import('./MonacoQueryField'));
export const MonacoQueryFieldLazy = (props: Props) => {
return (
<Suspense fallback={null}>
packages/grafana-prometheus/src/components/monaco-query-field/MonacoQueryFieldProps.ts
-2
View File
@@ -16,7 +16,5 @@ export type Props = {
placeholder: string;
onRunQuery: (value: string) => void;
onBlur: (value: string) => void;
// onChange will never initiate a query, it just denotes that a query value has been changed
onChange: (value: string) => void;
datasource: PrometheusDatasource;
};
packages/grafana-prometheus/src/components/monaco-query-field/MonacoQueryFieldWrapper.tsx
+1 -9
View File
@@ -23,13 +23,5 @@ export const MonacoQueryFieldWrapper = (props: Props) => {
onChange(value);
};
/**
* Handles changes without running any queries
* @param value
*/
const handleChange = (value: string) => {
onChange(value);
};
return <MonacoQueryFieldLazy onChange={handleChange} onRunQuery={handleRunQuery} onBlur={handleBlur} {...rest} />;
return <MonacoQueryFieldLazy onRunQuery={handleRunQuery} onBlur={handleBlur} {...rest} />;
};
packages/grafana-runtime/package.json
+2 -2
View File
@@ -62,14 +62,14 @@
"@types/react": "18.3.18",
"@types/react-dom": "18.3.5",
"@types/systemjs": "6.15.1",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"lodash": "4.17.21",
"react": "18.3.1",
"react-dom": "18.3.1",
"rimraf": "6.0.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"rollup-plugin-sourcemaps": "0.6.3",
"typescript": "5.7.3"
packages/grafana-schema/package.json
+2 -2
View File
@@ -38,12 +38,12 @@
"devDependencies": {
"@grafana/tsconfig": "^2.0.0",
"@rollup/plugin-node-resolve": "16.0.0",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"glob": "^11.0.0",
"rimraf": "6.0.1",
"rollup": "^4.22.4",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"typescript": "5.7.3"
},
packages/grafana-ui/package.json
+2 -2
View File
@@ -163,7 +163,7 @@
"core-js": "3.40.0",
"css-loader": "7.1.2",
"csstype": "3.1.3",
"esbuild": "0.24.2",
"esbuild": "0.25.0",
"expose-loader": "5.0.0",
"fs-extra": "^11.2.0",
"mock-raf": "1.0.1",
@@ -175,7 +175,7 @@
"rollup": "^4.22.4",
"rollup-plugin-copy": "3.5.0",
"rollup-plugin-dts": "^6.1.1",
"rollup-plugin-esbuild": "6.1.1",
"rollup-plugin-esbuild": "6.2.0",
"rollup-plugin-node-externals": "^8.0.0",
"rollup-plugin-svg-import": "3.0.0",
"sass-loader": "16.0.4",
pkg/apiserver/rest/dualwriter_mode2.go
+15 -18
View File
@@ -207,28 +207,25 @@ func (d *DualWriterMode2) Delete(ctx context.Context, name string, deleteValidat
log := d.Log.WithValues("name", name, "method", method)
ctx = klog.NewContext(ctx, log)
// We should delete from Unified storage first so we can retry if legacy fails.
startStorage := time.Now()
deletedS, _, err := d.Storage.Delete(ctx, name, deleteValidation, options)
d.recordStorageDuration(err != nil, mode2Str, d.resource, method, startStorage)
if err != nil {
if !apierrors.IsNotFound(err) {
log.WithValues("objectList", deletedS).Error(err, "could not delete from unified storage")
return nil, false, err
}
}
startLegacy := time.Now()
deletedLS, async, err := d.Legacy.Delete(ctx, name, deleteValidation, options)
d.recordLegacyDuration(err != nil, mode2Str, d.resource, method, startLegacy)
// Deleting from legacy should always work in mode two, as legacy is still the primary database and
// needs to have all the data.
if err != nil {
if !apierrors.IsNotFound(err) {
log.WithValues("objectList", deletedLS).Error(err, "could not delete from legacy store")
d.recordLegacyDuration(true, mode2Str, d.resource, method, startLegacy)
return deletedLS, async, err
}
return nil, false, err
}
d.recordLegacyDuration(false, mode2Str, d.resource, method, startLegacy)
startStorage := time.Now()
deletedS, async, err := d.Storage.Delete(ctx, name, deleteValidation, options)
if err != nil {
if !apierrors.IsNotFound(err) {
log.WithValues("objectList", deletedS).Error(err, "could not delete from duplicate storage")
d.recordStorageDuration(true, mode2Str, d.resource, method, startStorage)
}
return deletedS, async, err
}
d.recordStorageDuration(false, mode2Str, d.resource, method, startStorage)
go func() {
areEqual := Compare(deletedS, deletedLS)
pkg/apiserver/rest/dualwriter_mode3.go
+3 -9
View File
@@ -176,28 +176,22 @@ func (d *DualWriterMode3) Delete(ctx context.Context, name string, deleteValidat
// we want to delete from legacy first, otherwise if the delete from unistore was successful,
// but legacy failed, the user would get a failure, but not be able to retry the delete
// as they would not be able to see the object in unistore anymore.
startLegacy := time.Now()
objFromLegacy, asyncLegacy, err := d.Legacy.Delete(ctx, name, deleteValidation, options)
d.recordLegacyDuration(err != nil && !apierrors.IsNotFound(err), mode3Str, d.resource, method, startLegacy)
if err != nil {
if !apierrors.IsNotFound(err) {
log.WithValues("object", objFromLegacy).Error(err, "could not delete from legacy store")
d.recordLegacyDuration(true, mode3Str, d.resource, method, startLegacy)
return objFromLegacy, asyncLegacy, err
}
}
d.recordLegacyDuration(false, mode3Str, d.resource, method, startLegacy)
startStorage := time.Now()
objFromStorage, asyncStorage, err := d.Storage.Delete(ctx, name, deleteValidation, options)
d.recordStorageDuration(err != nil && !apierrors.IsNotFound(err), mode3Str, d.resource, method, startStorage)
if err != nil {
if !apierrors.IsNotFound(err) {
log.WithValues("object", objFromStorage).Error(err, "could not delete from storage")
d.recordStorageDuration(true, mode3Str, d.resource, method, startStorage)
}
return objFromStorage, asyncStorage, err
return nil, false, err
}
d.recordStorageDuration(false, mode3Str, d.resource, method, startStorage)
areEqual := Compare(objFromStorage, objFromLegacy)
d.recordOutcome(mode3Str, name, areEqual, method)
pkg/cmd/grafana-cli/commands/commands.go
+5
View File
@@ -144,6 +144,11 @@ var adminCommands = []*cli.Command{
Usage: "Migrates passwords from unsecured fields to secure_json_data field. Return ok unless there is an error. Safe to execute multiple times.",
Action: runDbCommand(datamigrations.EncryptDatasourcePasswords),
},
{
Name: "to-unified-storage",
Usage: "Migrates classic SQL data into unified storage",
Action: runDbCommand(datamigrations.ToUnifiedStorage),
},
},
},
{
pkg/cmd/grafana-cli/commands/datamigrations/stubs.go
+70
View File
@@ -0,0 +1,70 @@
package datamigrations
import (
"context"
"path/filepath"
"github.com/grafana/grafana/pkg/services/provisioning"
"github.com/grafana/grafana/pkg/services/provisioning/dashboards"
)
var (
_ provisioning.ProvisioningService = (*stubProvisioning)(nil)
)
func newStubProvisioning(path string) (provisioning.ProvisioningService, error) {
cfgs, err := dashboards.ReadDashboardConfig(filepath.Join(path, "dashboards"))
if err != nil {
return nil, err
}
stub := &stubProvisioning{
path: make(map[string]string),
}
for _, cfg := range cfgs {
stub.path[cfg.Name] = cfg.Options["path"].(string)
}
return &stubProvisioning{}, nil
}
type stubProvisioning struct {
path map[string]string // name > options.path
}
// GetAllowUIUpdatesFromConfig implements provisioning.ProvisioningService.
func (s *stubProvisioning) GetAllowUIUpdatesFromConfig(name string) bool {
return false
}
func (s *stubProvisioning) GetDashboardProvisionerResolvedPath(name string) string {
return s.path[name]
}
// ProvisionAlerting implements provisioning.ProvisioningService.
func (s *stubProvisioning) ProvisionAlerting(ctx context.Context) error {
panic("unimplemented")
}
// ProvisionDashboards implements provisioning.ProvisioningService.
func (s *stubProvisioning) ProvisionDashboards(ctx context.Context) error {
panic("unimplemented")
}
// ProvisionDatasources implements provisioning.ProvisioningService.
func (s *stubProvisioning) ProvisionDatasources(ctx context.Context) error {
panic("unimplemented")
}
// ProvisionPlugins implements provisioning.ProvisioningService.
func (s *stubProvisioning) ProvisionPlugins(ctx context.Context) error {
panic("unimplemented")
}
// Run implements provisioning.ProvisioningService.
func (s *stubProvisioning) Run(ctx context.Context) error {
panic("unimplemented")
}
// RunInitProvisioners implements provisioning.ProvisioningService.
func (s *stubProvisioning) RunInitProvisioners(ctx context.Context) error {
panic("unimplemented")
}
pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage.go
+207
View File
@@ -0,0 +1,207 @@
package datamigrations
import (
"context"
"encoding/json"
"fmt"
"os"
"strings"
"time"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/apimachinery/pkg/runtime/schema"
dashboard "github.com/grafana/grafana/pkg/apis/dashboard"
folders "github.com/grafana/grafana/pkg/apis/folder/v0alpha1"
authlib "github.com/grafana/authlib/types"
"github.com/grafana/grafana/pkg/apimachinery/identity"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
"github.com/grafana/grafana/pkg/infra/db"
"github.com/grafana/grafana/pkg/infra/tracing"
"github.com/grafana/grafana/pkg/registry/apis/dashboard/legacy"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/storage/legacysql"
"github.com/grafana/grafana/pkg/storage/unified"
"github.com/grafana/grafana/pkg/storage/unified/parquet"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
// ToUnifiedStorage converts dashboards+folders into unified storage
func ToUnifiedStorage(c utils.CommandLine, cfg *setting.Cfg, sqlStore db.DB) error {
namespace := "default" // TODO... from command line
ns, err := authlib.ParseNamespace(namespace)
if err != nil {
return err
}
ctx := identity.WithServiceIdentityContext(context.Background(), ns.OrgID)
start := time.Now()
last := time.Now()
opts := legacy.MigrateOptions{
Namespace: namespace,
Resources: []schema.GroupResource{
{Group: folders.GROUP, Resource: folders.RESOURCE},
{Group: dashboard.GROUP, Resource: dashboard.DASHBOARD_RESOURCE},
{Group: dashboard.GROUP, Resource: dashboard.LIBRARY_PANEL_RESOURCE},
},
LargeObjects: nil, // TODO... from config
Progress: func(count int, msg string) {
if count < 1 || time.Since(last) > time.Second {
fmt.Printf("[%4d] %s\n", count, msg)
last = time.Now()
}
},
}
provisioning, err := newStubProvisioning(cfg.ProvisioningPath)
if err != nil {
return err
}
migrator := legacy.NewDashboardAccess(
legacysql.NewDatabaseProvider(sqlStore),
authlib.OrgNamespaceFormatter,
nil, provisioning, false,
)
yes, err := promptYesNo(fmt.Sprintf("Count legacy resources for namespace: %s?", opts.Namespace))
if err != nil {
return err
}
if yes {
opts.OnlyCount = true
rsp, err := migrator.Migrate(ctx, opts)
if err != nil {
return err
}
fmt.Printf("Counting DONE: %s\n", time.Since(start))
if rsp != nil {
jj, _ := json.MarshalIndent(rsp, "", " ")
fmt.Printf("%s\n", string(jj))
}
}
opts.OnlyCount = false
opts.WithHistory, err = promptYesNo("Include history in exports?")
if err != nil {
return err
}
yes, err = promptYesNo("Export legacy resources to parquet file?")
if err != nil {
return err
}
if yes {
file, err := os.CreateTemp(cfg.DataPath, "grafana-export-*.parquet")
if err != nil {
return err
}
start = time.Now()
last = time.Now()
opts.Store, err = newParquetClient(file)
if err != nil {
return err
}
rsp, err := migrator.Migrate(ctx, opts)
if err != nil {
return err
}
fmt.Printf("Parquet export DONE: %s\n", time.Since(start))
if rsp != nil {
jj, _ := json.MarshalIndent(rsp, "", " ")
fmt.Printf("%s\n", string(jj))
}
fmt.Printf("File: %s\n", file.Name())
}
yes, err = promptYesNo("Export legacy resources to unified storage?")
if err != nil {
return err
}
if yes {
client, err := newUnifiedClient(cfg, sqlStore)
if err != nil {
return err
}
// Check the stats (eventually compare)
req := &resource.ResourceStatsRequest{
Namespace: opts.Namespace,
}
for _, r := range opts.Resources {
req.Kinds = append(req.Kinds, fmt.Sprintf("%s/%s", r.Group, r.Resource))
}
stats, err := client.GetStats(ctx, req)
if err != nil {
return err
}
if stats != nil {
fmt.Printf("Existing resources in unified storage:\n")
jj, _ := json.MarshalIndent(stats, "", " ")
fmt.Printf("%s\n", string(jj))
}
yes, err = promptYesNo("Would you like to continue? (existing resources will be replaced)")
if err != nil {
return err
}
if yes {
start = time.Now()
last = time.Now()
opts.Store = client
opts.BlobStore = client
rsp, err := migrator.Migrate(ctx, opts)
if err != nil {
return err
}
fmt.Printf("Unified storage export: %s\n", time.Since(start))
if rsp != nil {
jj, _ := json.MarshalIndent(rsp, "", " ")
fmt.Printf("%s\n", string(jj))
}
}
}
return nil
}
func promptYesNo(prompt string) (bool, error) {
line := ""
for {
fmt.Printf("%s (Y/N) >", prompt)
_, err := fmt.Scanln(&line)
if err != nil && err.Error() != "unexpected newline" {
return false, err
}
switch strings.ToLower(line) {
case "y", "yes":
return true, nil
case "n", "no":
return false, nil
}
}
}
func newUnifiedClient(cfg *setting.Cfg, sqlStore db.DB) (resource.ResourceClient, error) {
return unified.ProvideUnifiedStorageClient(cfg,
featuremgmt.WithFeatures(), // none??
sqlStore,
tracing.NewNoopTracerService(),
prometheus.NewPedanticRegistry(),
authlib.FixedAccessClient(true), // always true!
nil, // document supplier
)
}
func newParquetClient(file *os.File) (resource.BatchStoreClient, error) {
writer, err := parquet.NewParquetWriter(file)
if err != nil {
return nil, err
}
client := parquet.NewBatchResourceWriterClient(writer)
return client, nil
}
pkg/cmd/grafana-cli/commands/datamigrations/to_unified_storage_test.go
+29
View File
@@ -0,0 +1,29 @@
package datamigrations
import (
"context"
"fmt"
"testing"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/db"
)
func TestUnifiedStorageCommand(t *testing.T) {
// setup datasources with password, basic_auth and none
store := db.InitTestDB(t)
err := store.WithDbSession(context.Background(), func(sess *db.Session) error {
unistoreMigrationTest(t, sess, store)
return nil
})
require.NoError(t, err)
}
func unistoreMigrationTest(t *testing.T, session *db.Session, sqlstore db.DB) {
// empty stats
t.Run("get stats", func(t *testing.T) {
fmt.Printf("TODO... add folders and check that they migrate\n")
})
}
pkg/expr/sql/db_test.go
+2 -7
View File
@@ -45,11 +45,11 @@ func TestQueryFrames(t *testing.T) {
name: "query all rows from single input frame",
query: `SELECT * FROM inputFrameRefId LIMIT 1;`,
input_frames: []*data.Frame{
setRefID(data.NewFrame(
data.NewFrame(
"",
//nolint:misspell
data.NewField("OSS Projects with Typos", nil, []string{"Garfana", "Pormetheus"}),
), "inputFrameRefId"),
).SetRefID("inputFrameRefId"),
},
expected: data.NewFrame(
"sqlExpressionRefId",
@@ -174,8 +174,3 @@ func TestQueryFramesDateTimeSelect(t *testing.T) {
func p[T any](v T) *T {
return &v
}
func setRefID(f *data.Frame, refID string) *data.Frame {
f.RefID = refID
return f
}
pkg/expr/sql/frame_db_conv.go
-62
View File
@@ -410,65 +410,3 @@ func fieldValFromRowVal(fieldType data.FieldType, val interface{}) (interface{},
return nil, fmt.Errorf("unsupported field type %s for val %v", fieldType, val)
}
}
// Is the field nilAt the index. Can panic if out of range.
// TODO: Maybe this should be a method on data.Field?
func nilAt(field data.Field, at int) bool {
if !field.Nullable() {
return false
}
switch field.Type() {
case data.FieldTypeNullableInt8:
v := field.At(at).(*int8)
return v == nil
case data.FieldTypeNullableUint8:
v := field.At(at).(*uint8)
return v == nil
case data.FieldTypeNullableInt16:
v := field.At(at).(*int16)
return v == nil
case data.FieldTypeNullableUint16:
v := field.At(at).(*uint16)
return v == nil
case data.FieldTypeNullableInt32:
v := field.At(at).(*int32)
return v == nil
case data.FieldTypeNullableUint32:
v := field.At(at).(*uint32)
return v == nil
case data.FieldTypeNullableInt64:
v := field.At(at).(*int64)
return v == nil
case data.FieldTypeNullableUint64:
v := field.At(at).(*uint64)
return v == nil
case data.FieldTypeNullableFloat64:
v := field.At(at).(*float64)
return v == nil
case data.FieldTypeNullableString:
v := field.At(at).(*string)
return v == nil
case data.FieldTypeNullableTime:
v := field.At(at).(*time.Time)
return v == nil
case data.FieldTypeNullableBool:
v := field.At(at).(*bool)
return v == nil
default:
// Either it's not a nullable type or it's unsupported
return false
}
}
pkg/expr/sql/frame_table.go
+1 -1
View File
@@ -85,7 +85,7 @@ func (ri *rowIter) Next(_ *mysql.Context) (mysql.Row, error) {
// the value from each column at the current row index.
row := make(mysql.Row, len(ri.ft.Frame.Fields))
for colIndex, field := range ri.ft.Frame.Fields {
if nilAt(*field, ri.row) {
if field.NilAt(ri.row) {
continue
}
row[colIndex], _ = field.ConcreteAt(ri.row)
pkg/registry/apis/dashboard/legacy/client.go
+6 -1
View File
@@ -87,5 +87,10 @@ func (d *directResourceClient) Update(ctx context.Context, in *resource.UpdateRe
// Watch implements ResourceClient.
func (d *directResourceClient) Watch(ctx context.Context, in *resource.WatchRequest, opts ...grpc.CallOption) (resource.ResourceStore_WatchClient, error) {
return nil, fmt.Errorf("watch not yet supported with direct resource client")
return nil, fmt.Errorf("watch not supported with direct resource client")
}
// BatchProcess implements resource.ResourceClient.
func (d *directResourceClient) BatchProcess(ctx context.Context, opts ...grpc.CallOption) (resource.BatchStore_BatchProcessClient, error) {
return nil, fmt.Errorf("BatchProcess not supported with direct resource client")
}
pkg/registry/apis/dashboard/legacy/migrate.go
+411
View File
@@ -0,0 +1,411 @@
package legacy
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"google.golang.org/grpc/metadata"
"k8s.io/apimachinery/pkg/runtime/schema"
authlib "github.com/grafana/authlib/types"
"github.com/grafana/grafana/pkg/apimachinery/utils"
dashboard "github.com/grafana/grafana/pkg/apis/dashboard"
folders "github.com/grafana/grafana/pkg/apis/folder/v0alpha1"
"github.com/grafana/grafana/pkg/services/sqlstore"
"github.com/grafana/grafana/pkg/storage/unified/apistore"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
type MigrateOptions struct {
Namespace string
Store resource.BatchStoreClient
Writer resource.BatchResourceWriter
LargeObjects apistore.LargeObjectSupport
BlobStore resource.BlobStoreClient
Resources []schema.GroupResource
WithHistory bool // only applies to dashboards
OnlyCount bool // just count the values
Progress func(count int, msg string)
}
// Read from legacy and write into unified storage
type LegacyMigrator interface {
Migrate(ctx context.Context, opts MigrateOptions) (*resource.BatchResponse, error)
}
type BlobStoreInfo struct {
Count int64
Size int64
}
// migrate function -- works for a single kind
type migrator = func(ctx context.Context, orgId int64, opts MigrateOptions, stream resource.BatchStore_BatchProcessClient) (*BlobStoreInfo, error)
func (a *dashboardSqlAccess) Migrate(ctx context.Context, opts MigrateOptions) (*resource.BatchResponse, error) {
info, err := authlib.ParseNamespace(opts.Namespace)
if err != nil {
return nil, err
}
// Migrate everything
if len(opts.Resources) < 1 {
return nil, fmt.Errorf("missing resource selector")
}
migrators := []migrator{}
settings := resource.BatchSettings{
RebuildCollection: true,
SkipValidation: true,
}
for _, res := range opts.Resources {
switch fmt.Sprintf("%s/%s", res.Group, res.Resource) {
case "folder.grafana.app/folders":
migrators = append(migrators, a.migrateFolders)
settings.Collection = append(settings.Collection, &resource.ResourceKey{
Namespace: opts.Namespace,
Group: folders.GROUP,
Resource: folders.RESOURCE,
})
case "dashboard.grafana.app/librarypanels":
migrators = append(migrators, a.migratePanels)
settings.Collection = append(settings.Collection, &resource.ResourceKey{
Namespace: opts.Namespace,
Group: dashboard.GROUP,
Resource: dashboard.LIBRARY_PANEL_RESOURCE,
})
case "dashboard.grafana.app/dashboards":
migrators = append(migrators, a.migrateDashboards)
settings.Collection = append(settings.Collection, &resource.ResourceKey{
Namespace: opts.Namespace,
Group: dashboard.GROUP,
Resource: dashboard.DASHBOARD_RESOURCE,
})
default:
return nil, fmt.Errorf("unsupported resource: %s", res)
}
}
if opts.OnlyCount {
return a.countValues(ctx, opts)
}
ctx = metadata.NewOutgoingContext(ctx, settings.ToMD())
stream, err := opts.Store.BatchProcess(ctx)
if err != nil {
return nil, err
}
// Now run each migration
blobStore := BlobStoreInfo{}
for _, m := range migrators {
blobs, err := m(ctx, info.OrgID, opts, stream)
if err != nil {
return nil, err
}
if blobs != nil {
blobStore.Count += blobs.Count
blobStore.Size += blobs.Size
}
}
fmt.Printf("BLOBS: %+v\n", blobStore)
return stream.CloseAndRecv()
}
func (a *dashboardSqlAccess) countValues(ctx context.Context, opts MigrateOptions) (*resource.BatchResponse, error) {
sql, err := a.sql(ctx)
if err != nil {
return nil, err
}
ns, err := authlib.ParseNamespace(opts.Namespace)
if err != nil {
return nil, err
}
orgId := ns.OrgID
rsp := &resource.BatchResponse{}
err = sql.DB.WithDbSession(ctx, func(sess *sqlstore.DBSession) error {
for _, res := range opts.Resources {
switch fmt.Sprintf("%s/%s", res.Group, res.Resource) {
case "folder.grafana.app/folders":
summary := &resource.BatchResponse_Summary{}
summary.Group = folders.GROUP
summary.Group = folders.RESOURCE
_, err = sess.SQL("SELECT COUNT(*) FROM "+sql.Table("dashboard")+
" WHERE is_folder=FALSE AND org_id=?", orgId).Get(&summary.Count)
rsp.Summary = append(rsp.Summary, summary)
case "dashboard.grafana.app/librarypanels":
summary := &resource.BatchResponse_Summary{}
summary.Group = dashboard.GROUP
summary.Resource = dashboard.LIBRARY_PANEL_RESOURCE
_, err = sess.SQL("SELECT COUNT(*) FROM "+sql.Table("library_element")+
" WHERE org_id=?", orgId).Get(&summary.Count)
rsp.Summary = append(rsp.Summary, summary)
case "dashboard.grafana.app/dashboards":
summary := &resource.BatchResponse_Summary{}
summary.Group = dashboard.GROUP
summary.Resource = dashboard.DASHBOARD_RESOURCE
rsp.Summary = append(rsp.Summary, summary)
_, err = sess.SQL("SELECT COUNT(*) FROM "+sql.Table("dashboard")+
" WHERE is_folder=FALSE AND org_id=?", orgId).Get(&summary.Count)
if err != nil {
return err
}
// Also count history
_, err = sess.SQL(`SELECT COUNT(*)
FROM `+sql.Table("dashboard_version")+` as dv
JOIN `+sql.Table("dashboard")+` as dd
ON dd.id = dv.dashboard_id
WHERE org_id=?`, orgId).Get(&summary.History)
}
if err != nil {
return err
}
}
return nil
})
return rsp, nil
}
func (a *dashboardSqlAccess) migrateDashboards(ctx context.Context, orgId int64, opts MigrateOptions, stream resource.BatchStore_BatchProcessClient) (*BlobStoreInfo, error) {
query := &DashboardQuery{
OrgID: orgId,
Limit: 100000000,
GetHistory: opts.WithHistory, // include history
}
blobs := &BlobStoreInfo{}
sql, err := a.sql(ctx)
if err != nil {
return blobs, err
}
opts.Progress(-1, "migrating dashboards...")
rows, err := a.getRows(ctx, sql, query)
if rows != nil {
defer func() {
_ = rows.Close()
}()
}
if err != nil {
return blobs, err
}
large := opts.LargeObjects
// Now send each dashboard
for i := 1; rows.Next(); i++ {
dash := rows.row.Dash
dash.APIVersion = fmt.Sprintf("%s/v0alpha1", dashboard.GROUP) // << eventually v0
dash.SetNamespace(opts.Namespace)
dash.SetResourceVersion("") // it will be filled in by the backend
body, err := json.Marshal(dash)
if err != nil {
err = fmt.Errorf("error reading json from: %s // %w", rows.row.Dash.Name, err)
return blobs, err
}
req := &resource.BatchRequest{
Key: &resource.ResourceKey{
Namespace: opts.Namespace,
Group: dashboard.GROUP,
Resource: dashboard.DASHBOARD_RESOURCE,
Name: rows.Name(),
},
Value: body,
Folder: rows.row.FolderUID,
Action: resource.BatchRequest_ADDED,
}
if dash.Generation > 1 {
req.Action = resource.BatchRequest_MODIFIED
} else if dash.Generation < 0 {
req.Action = resource.BatchRequest_DELETED
}
// With large object support
if large != nil && len(body) > large.Threshold() {
obj, err := utils.MetaAccessor(dash)
if err != nil {
return blobs, err
}
opts.Progress(i, fmt.Sprintf("[v:%d] %s Large object (%d)", dash.Generation, dash.Name, len(body)))
err = large.Deconstruct(ctx, req.Key, opts.BlobStore, obj, req.Value)
if err != nil {
return blobs, err
}
// The smaller version (most of spec removed)
req.Value, err = json.Marshal(dash)
if err != nil {
return blobs, err
}
blobs.Count++
blobs.Size += int64(len(body))
}
opts.Progress(i, fmt.Sprintf("[v:%2d] %s (size:%d / %d|%d)", dash.Generation, dash.Name, len(req.Value), i, rows.count))
err = stream.Send(req)
if err != nil {
if errors.Is(err, io.EOF) {
opts.Progress(i, fmt.Sprintf("stream EOF/cancelled. index=%d", i))
err = nil
}
return blobs, err
}
}
if len(rows.rejected) > 0 {
for _, row := range rows.rejected {
id := row.Dash.Labels[utils.LabelKeyDeprecatedInternalID]
fmt.Printf("REJECTED: %s / %s\n", id, row.Dash.Name)
opts.Progress(-2, fmt.Sprintf("rejected: id:%s, uid:%s", id, row.Dash.Name))
}
}
if rows.Error() != nil {
return blobs, rows.Error()
}
opts.Progress(-2, fmt.Sprintf("finished dashboards... (%d)", rows.count))
return blobs, err
}
func (a *dashboardSqlAccess) migrateFolders(ctx context.Context, orgId int64, opts MigrateOptions, stream resource.BatchStore_BatchProcessClient) (*BlobStoreInfo, error) {
query := &DashboardQuery{
OrgID: orgId,
Limit: 100000000,
GetFolders: true,
}
sql, err := a.sql(ctx)
if err != nil {
return nil, err
}
opts.Progress(-1, "migrating folders...")
rows, err := a.getRows(ctx, sql, query)
if rows != nil {
defer func() {
_ = rows.Close()
}()
}
if err != nil {
return nil, err
}
// Now send each dashboard
for i := 1; rows.Next(); i++ {
dash := rows.row.Dash
dash.APIVersion = "folder.grafana.app/v0alpha1"
dash.Kind = "Folder"
dash.SetNamespace(opts.Namespace)
dash.SetResourceVersion("") // it will be filled in by the backend
spec := map[string]any{
"title": dash.Spec.Object["title"],
}
description := dash.Spec.Object["description"]
if description != nil {
spec["description"] = description
}
dash.Spec.Object = spec
body, err := json.Marshal(dash)
if err != nil {
return nil, err
}
req := &resource.BatchRequest{
Key: &resource.ResourceKey{
Namespace: opts.Namespace,
Group: "folder.grafana.app",
Resource: "folders",
Name: rows.Name(),
},
Value: body,
Folder: rows.row.FolderUID,
Action: resource.BatchRequest_ADDED,
}
if dash.Generation > 1 {
req.Action = resource.BatchRequest_MODIFIED
} else if dash.Generation < 0 {
req.Action = resource.BatchRequest_DELETED
}
opts.Progress(i, fmt.Sprintf("[v:%d] %s (%d)", dash.Generation, dash.Name, len(req.Value)))
err = stream.Send(req)
if err != nil {
if errors.Is(err, io.EOF) {
err = nil
}
return nil, err
}
}
if rows.Error() != nil {
return nil, rows.Error()
}
opts.Progress(-2, fmt.Sprintf("finished folders... (%d)", rows.count))
return nil, err
}
func (a *dashboardSqlAccess) migratePanels(ctx context.Context, orgId int64, opts MigrateOptions, stream resource.BatchStore_BatchProcessClient) (*BlobStoreInfo, error) {
opts.Progress(-1, "migrating library panels...")
panels, err := a.GetLibraryPanels(ctx, LibraryPanelQuery{
OrgID: orgId,
Limit: 1000000,
})
if err != nil {
return nil, err
}
for i, panel := range panels.Items {
meta, err := utils.MetaAccessor(&panel)
if err != nil {
return nil, err
}
body, err := json.Marshal(panel)
if err != nil {
return nil, err
}
req := &resource.BatchRequest{
Key: &resource.ResourceKey{
Namespace: opts.Namespace,
Group: dashboard.GROUP,
Resource: dashboard.LIBRARY_PANEL_RESOURCE,
Name: panel.Name,
},
Value: body,
Folder: meta.GetFolder(),
Action: resource.BatchRequest_ADDED,
}
if panel.Generation > 1 {
req.Action = resource.BatchRequest_MODIFIED
}
opts.Progress(i, fmt.Sprintf("[v:%d] %s (%d)", i, meta.GetName(), len(req.Value)))
err = stream.Send(req)
if err != nil {
if errors.Is(err, io.EOF) {
err = nil
}
return nil, err
}
}
opts.Progress(-2, fmt.Sprintf("finished panels... (%d)", len(panels.Items)))
return nil, nil
}
pkg/registry/apis/dashboard/legacy/types.go
+1
View File
@@ -49,6 +49,7 @@ type LibraryPanelQuery struct {
type DashboardAccess interface {
resource.StorageBackend
resource.ResourceIndexServer
LegacyMigrator
GetDashboard(ctx context.Context, orgId int64, uid string, version int64) (*dashboard.Dashboard, int64, error)
SaveDashboard(ctx context.Context, orgId int64, dash *dashboard.Dashboard) (*dashboard.Dashboard, bool, error)
pkg/services/folder/folderimpl/folder_unifiedstorage_test.go
+3
View File
@@ -529,6 +529,9 @@ func (r resourceClientMock) List(ctx context.Context, in *resource.ListRequest,
func (r resourceClientMock) Watch(ctx context.Context, in *resource.WatchRequest, opts ...grpc.CallOption) (resource.ResourceStore_WatchClient, error) {
return nil, nil
}
func (r resourceClientMock) BatchProcess(ctx context.Context, opts ...grpc.CallOption) (resource.BatchStore_BatchProcessClient, error) {
return nil, nil
}
func (r resourceClientMock) Search(ctx context.Context, in *resource.ResourceSearchRequest, opts ...grpc.CallOption) (*resource.ResourceSearchResponse, error) {
if len(in.Options.Labels) > 0 &&
in.Options.Labels[0].Key == utils.LabelKeyDeprecatedInternalID &&
pkg/services/ngalert/models/alert_rule.go
+75
View File
@@ -680,6 +680,81 @@ func (alertRule *AlertRule) Type() RuleType {
return RuleTypeAlerting
}
// Copy creates and returns a deep copy of the AlertRule instance, duplicating all fields and nested data structures.
func (alertRule *AlertRule) Copy() *AlertRule {
if alertRule == nil {
return nil
}
result := AlertRule{
ID: alertRule.ID,
OrgID: alertRule.OrgID,
Title: alertRule.Title,
Condition: alertRule.Condition,
Updated: alertRule.Updated,
UpdatedBy: alertRule.UpdatedBy,
IntervalSeconds: alertRule.IntervalSeconds,
Version: alertRule.Version,
UID: alertRule.UID,
NamespaceUID: alertRule.NamespaceUID,
RuleGroup: alertRule.RuleGroup,
RuleGroupIndex: alertRule.RuleGroupIndex,
NoDataState: alertRule.NoDataState,
ExecErrState: alertRule.ExecErrState,
For: alertRule.For,
Record: alertRule.Record,
IsPaused: alertRule.IsPaused,
Metadata: alertRule.Metadata,
}
if alertRule.DashboardUID != nil {
dash := *alertRule.DashboardUID
result.DashboardUID = &dash
}
if alertRule.PanelID != nil {
p := *alertRule.PanelID
result.PanelID = &p
}
for _, d := range alertRule.Data {
q := AlertQuery{
RefID: d.RefID,
QueryType: d.QueryType,
RelativeTimeRange: d.RelativeTimeRange,
DatasourceUID: d.DatasourceUID,
}
q.Model = make([]byte, 0, cap(d.Model))
q.Model = append(q.Model, d.Model...)
result.Data = append(result.Data, q)
}
if alertRule.Annotations != nil {
result.Annotations = make(map[string]string, len(alertRule.Annotations))
for s, s2 := range alertRule.Annotations {
result.Annotations[s] = s2
}
}
if alertRule.Labels != nil {
result.Labels = make(map[string]string, len(alertRule.Labels))
for s, s2 := range alertRule.Labels {
result.Labels[s] = s2
}
}
if alertRule.Record != nil {
result.Record = &Record{
From: alertRule.Record.From,
Metric: alertRule.Record.Metric,
}
}
for _, s := range alertRule.NotificationSettings {
result.NotificationSettings = append(result.NotificationSettings, CopyNotificationSettings(s))
}
return &result
}
func ClearRecordingRuleIgnoredFields(rule *AlertRule) {
rule.NoDataState = ""
rule.ExecErrState = ""
pkg/services/ngalert/models/alert_rule_test.go
+65 -1
View File
@@ -15,6 +15,7 @@ import (
"github.com/prometheus/common/model"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/exp/maps"
"gopkg.in/yaml.v3"
"github.com/grafana/grafana/pkg/util"
@@ -407,7 +408,7 @@ func TestDiff(t *testing.T) {
rule1 := RuleGen.GenerateRef()
rule2 := RuleGen.GenerateRef()
diffs := rule1.Diff(rule2, "Data", "Annotations", "Labels", "NotificationSettings") // these fields will be tested separately
diffs := rule1.Diff(rule2, "Data", "Annotations", "Labels", "NotificationSettings", "Metadata") // these fields will be tested separately
difCnt := 0
if rule1.ID != rule2.ID {
@@ -839,6 +840,24 @@ func TestDiff(t *testing.T) {
})
}
})
t.Run("should detect changes in Metadata", func(t *testing.T) {
rule1 := RuleGen.With(RuleGen.WithMetadata(AlertRuleMetadata{EditorSettings: EditorSettings{
SimplifiedQueryAndExpressionsSection: false,
SimplifiedNotificationsSection: false,
}})).GenerateRef()
rule2 := CopyRule(rule1, RuleGen.WithMetadata(AlertRuleMetadata{EditorSettings: EditorSettings{
SimplifiedQueryAndExpressionsSection: true,
SimplifiedNotificationsSection: true,
}}))
diff := rule1.Diff(rule2)
assert.ElementsMatch(t, []string{
"Metadata.EditorSettings.SimplifiedQueryAndExpressionsSection",
"Metadata.EditorSettings.SimplifiedNotificationsSection",
}, diff.Paths())
})
}
func TestSortByGroupIndex(t *testing.T) {
@@ -919,3 +938,48 @@ func TestAlertRuleGetKeyWithGroup(t *testing.T) {
require.Equal(t, expected, rule.GetKeyWithGroup())
})
}
func TestAlertRuleCopy(t *testing.T) {
for i := 0; i < 100; i++ {
rule := RuleGen.GenerateRef()
copied := rule.Copy()
require.Empty(t, rule.Diff(copied))
}
}
// This test makes sure the default generator
func TestGeneratorFillsAllFields(t *testing.T) {
ignoredFields := map[string]struct{}{
"ID": {},
"IsPaused": {},
"Record": {},
}
tpe := reflect.TypeOf(AlertRule{})
fields := make(map[string]struct{}, tpe.NumField())
for i := 0; i < tpe.NumField(); i++ {
if _, ok := ignoredFields[tpe.Field(i).Name]; ok {
continue
}
fields[tpe.Field(i).Name] = struct{}{}
}
for i := 0; i < 1000; i++ {
rule := RuleGen.Generate()
v := reflect.ValueOf(rule)
for j := 0; j < tpe.NumField(); j++ {
field := tpe.Field(j)
value := v.Field(j)
if !value.IsValid() || value.Kind() == reflect.Ptr && value.IsNil() || value.IsZero() {
continue
}
delete(fields, field.Name)
if len(fields) == 0 {
return
}
}
}
require.FailNow(t, "AlertRule generator does not populate fields", "skipped fields: %v", maps.Keys(fields))
}
pkg/services/ngalert/models/testing.go
+19 -69
View File
@@ -123,6 +123,7 @@ func (g *AlertRuleGenerator) Generate() AlertRule {
Annotations: annotations,
Labels: labels,
NotificationSettings: ns,
Metadata: GenerateMetadata(),
}
for _, mutator := range g.mutators {
@@ -569,6 +570,12 @@ func (a *AlertRuleMutators) WithVersion(version int64) AlertRuleMutator {
}
}
func (a *AlertRuleMutators) WithMetadata(meta AlertRuleMetadata) AlertRuleMutator {
return func(r *AlertRule) {
r.Metadata = meta
}
}
func (g *AlertRuleGenerator) GenerateLabels(min, max int, prefix string) data.Labels {
count := max
if min > max {
@@ -643,79 +650,13 @@ func GenerateGroupKey(orgID int64) AlertRuleGroupKey {
// CopyRule creates a deep copy of AlertRule
func CopyRule(r *AlertRule, mutators ...AlertRuleMutator) *AlertRule {
result := AlertRule{
ID: r.ID,
OrgID: r.OrgID,
Title: r.Title,
Condition: r.Condition,
Updated: r.Updated,
UpdatedBy: r.UpdatedBy,
IntervalSeconds: r.IntervalSeconds,
Version: r.Version,
UID: r.UID,
NamespaceUID: r.NamespaceUID,
RuleGroup: r.RuleGroup,
RuleGroupIndex: r.RuleGroupIndex,
NoDataState: r.NoDataState,
ExecErrState: r.ExecErrState,
For: r.For,
Record: r.Record,
IsPaused: r.IsPaused,
}
if r.DashboardUID != nil {
dash := *r.DashboardUID
result.DashboardUID = &dash
}
if r.PanelID != nil {
p := *r.PanelID
result.PanelID = &p
}
for _, d := range r.Data {
q := AlertQuery{
RefID: d.RefID,
QueryType: d.QueryType,
RelativeTimeRange: d.RelativeTimeRange,
DatasourceUID: d.DatasourceUID,
}
q.Model = make([]byte, 0, cap(d.Model))
q.Model = append(q.Model, d.Model...)
result.Data = append(result.Data, q)
}
if r.Annotations != nil {
result.Annotations = make(map[string]string, len(r.Annotations))
for s, s2 := range r.Annotations {
result.Annotations[s] = s2
}
}
if r.Labels != nil {
result.Labels = make(map[string]string, len(r.Labels))
for s, s2 := range r.Labels {
result.Labels[s] = s2
}
}
if r.Record != nil {
result.Record = &Record{
From: r.Record.From,
Metric: r.Record.Metric,
}
}
for _, s := range r.NotificationSettings {
result.NotificationSettings = append(result.NotificationSettings, CopyNotificationSettings(s))
}
result := r.Copy()
if len(mutators) > 0 {
for _, mutator := range mutators {
mutator(&result)
mutator(result)
}
}
return &result
return result
}
func CreateClassicConditionExpression(refID string, inputRefID string, reducer string, operation string, threshold int) AlertQuery {
@@ -862,6 +803,15 @@ func CreateHysteresisExpression(t *testing.T, refID string, inputRefID string, t
return q
}
func GenerateMetadata() AlertRuleMetadata {
return AlertRuleMetadata{
EditorSettings: EditorSettings{
SimplifiedQueryAndExpressionsSection: rand.Int()%2 == 0,
SimplifiedNotificationsSection: rand.Int()%2 == 0,
},
}
}
type AlertInstanceMutator func(*AlertInstance)
// AlertInstanceGen provides a factory function that generates a random AlertInstance.
pkg/services/ngalert/schedule/schedule_unit_test.go
+2
View File
@@ -784,6 +784,7 @@ func TestSchedule_updateRulesMetrics(t *testing.T) {
alertRuleWithAdvancedSettings := models.RuleGen.With(
models.RuleGen.WithOrgID(firstOrgID),
models.RuleGen.WithEditorSettingsSimplifiedNotificationsSection(false),
models.RuleGen.WithEditorSettingsSimplifiedQueryAndExpressionsSection(false),
).GenerateRef()
@@ -818,6 +819,7 @@ func TestSchedule_updateRulesMetrics(t *testing.T) {
alertRule2 := models.RuleGen.With(
models.RuleGen.WithOrgID(secondOrgID),
models.RuleGen.WithEditorSettingsSimplifiedNotificationsSection(false),
models.RuleGen.WithEditorSettingsSimplifiedQueryAndExpressionsSection(true),
).GenerateRef()
pkg/services/ngalert/store/alert_rule.go
+9 -3
View File
@@ -329,7 +329,13 @@ func (st DBstore) UpdateAlertRules(ctx context.Context, user *ngmodels.UserUID,
v := alertRuleToAlertRuleVersion(converted)
v.Version++
v.ParentVersion = r.Existing.Version
ruleVersions = append(ruleVersions, v)
// check if there is diff between existing and new, and if no, skip saving version.
existingConverted, err := alertRuleFromModelsAlertRule(*r.Existing)
if err != nil || !alertRuleToAlertRuleVersion(existingConverted).EqualSpec(v) {
ruleVersions = append(ruleVersions, v)
}
keys = append(keys, ngmodels.AlertRuleKey{OrgID: r.New.OrgID, UID: r.New.UID})
}
if len(ruleVersions) > 0 {
@@ -984,7 +990,7 @@ func (st DBstore) RenameReceiverInNotificationSettings(ctx context.Context, orgI
continue
}
r := ngmodels.CopyRule(rule)
r := rule.Copy()
for idx := range r.NotificationSettings {
if r.NotificationSettings[idx].Receiver == oldReceiver {
r.NotificationSettings[idx].Receiver = newReceiver
@@ -1059,7 +1065,7 @@ func (st DBstore) RenameTimeIntervalInNotificationSettings(
continue
}
r := ngmodels.CopyRule(rule)
r := rule.Copy()
for idx := range r.NotificationSettings {
for mtIdx := range r.NotificationSettings[idx].MuteTimeIntervals {
if r.NotificationSettings[idx].MuteTimeIntervals[mtIdx] == oldTimeInterval {
pkg/services/ngalert/store/alert_rule_test.go
+36
View File
@@ -73,6 +73,16 @@ func TestIntegrationUpdateAlertRules(t *testing.T) {
require.NoError(t, err)
require.Equal(t, rule.Version+1, dbrule.Version)
t.Run("should create version record", func(t *testing.T) {
var count int64
err = sqlStore.WithDbSession(context.Background(), func(sess *db.Session) error {
count, err = sess.Table(alertRuleVersion{}).Where("rule_uid = ?", rule.UID).Count()
return err
})
require.NoError(t, err)
require.EqualValues(t, 1, count) // only the current version, insert did not create version.
})
})
t.Run("updating record field should increase version", func(t *testing.T) {
@@ -191,6 +201,32 @@ func TestIntegrationUpdateAlertRules(t *testing.T) {
assert.Nil(t, dbrule.UpdatedBy)
})
})
t.Run("should save noop update", func(t *testing.T) {
rule := createRule(t, store, gen)
newRule := models.CopyRule(rule)
err := store.UpdateAlertRules(context.Background(), &usr, []models.UpdateRule{{
Existing: rule,
New: *newRule,
},
})
require.NoError(t, err)
newRule, err = store.GetAlertRuleByUID(context.Background(), &models.GetAlertRuleByUIDQuery{UID: rule.UID})
require.NoError(t, err)
assert.Equal(t, rule.Version+1, newRule.Version)
t.Run("should not create version record", func(t *testing.T) {
var count int64
err = sqlStore.WithDbSession(context.Background(), func(sess *db.Session) error {
count, err = sess.Table(alertRuleVersion{}).Where("rule_uid = ?", rule.UID).Count()
return err
})
require.NoError(t, err)
require.EqualValues(t, 1, count) // only the current version
})
})
}
func TestIntegrationUpdateAlertRulesWithUniqueConstraintViolation(t *testing.T) {
pkg/services/ngalert/store/deltas.go
+1 -1
View File
@@ -175,7 +175,7 @@ func UpdateCalculatedRuleFields(ch *GroupDelta) *GroupDelta {
}
if groupKey != ch.GroupKey {
if rule.RuleGroupIndex != idx {
upd.New = models.CopyRule(rule)
upd.New = rule.Copy()
upd.New.RuleGroupIndex = idx
upd.Diff = rule.Diff(upd.New, AlertRuleFieldsToIgnoreInDiff[:]...)
}
pkg/services/ngalert/store/deltas_test.go
+2 -2
View File
@@ -81,7 +81,7 @@ func TestCalculateChanges(t *testing.T) {
submittedMap := groupByUID(t, rules)
submitted := make([]*models.AlertRuleWithOptionals, 0, len(rules))
for _, rule := range rules {
submitted = append(submitted, &models.AlertRuleWithOptionals{AlertRule: *rule})
submitted = append(submitted, &models.AlertRuleWithOptionals{AlertRule: *rule, HasMetadata: true})
}
fakeStore := fakes.NewRuleStore(t)
@@ -216,7 +216,7 @@ func TestCalculateChanges(t *testing.T) {
submittedMap := groupByUID(t, rules)
submitted := make([]*models.AlertRuleWithOptionals, 0, len(rules))
for _, rule := range rules {
submitted = append(submitted, &models.AlertRuleWithOptionals{AlertRule: *rule})
submitted = append(submitted, &models.AlertRuleWithOptionals{AlertRule: *rule, HasMetadata: true})
}
changes, err := CalculateChanges(context.Background(), fakeStore, groupKey, submitted)
pkg/storage/unified/apistore/go.mod
+6
View File
@@ -47,6 +47,7 @@ require (
github.com/Azure/go-autorest/autorest/to v0.4.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.2 // indirect
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.3.0 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
@@ -56,8 +57,10 @@ require (
github.com/VividCortex/mysqlerr v0.0.0-20170204212430-6c6b55f8796f // indirect
github.com/Yiling-J/theine-go v0.6.0 // indirect
github.com/alecthomas/units v0.0.0-20240927000941-0f3dac36c52b // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/apache/arrow-go/v18 v18.0.1-0.20241212180703-82be143d7c30 // indirect
github.com/apache/thrift v0.21.0 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/at-wat/mqtt-go v0.19.4 // indirect
@@ -217,6 +220,7 @@ require (
github.com/jpillora/backoff v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/jszwedko/go-datemath v0.1.1-0.20230526204004-640a500621d6 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
@@ -237,6 +241,8 @@ require (
github.com/mdlayher/vsock v1.2.1 // indirect
github.com/mfridman/interpolate v0.0.2 // indirect
github.com/miekg/dns v1.1.62 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.1-0.20231216201459-8508981c8b6c // indirect
pkg/storage/unified/apistore/go.sum
+4
View File
@@ -87,6 +87,8 @@ github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/HdrHistogram/hdrhistogram-go v1.1.2 h1:5IcZpTvzydCQeHzK4Ef/D5rrSqwxob0t8PQPMybUNFM=
github.com/HdrHistogram/hdrhistogram-go v1.1.2/go.mod h1:yDgFjdqOqDEKOvasDdhWNXYg9BVp4O+o5f6V/ehm6Oo=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c h1:RGWPOewvKIROun94nF7v2cua9qP+thov/7M50KEoeSU=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c/go.mod h1:X0CRv0ky0k6m906ixxpzmDRLvX58TFUKS2eePweuyxk=
github.com/Masterminds/goutils v1.1.1 h1:5nUrii3FMTL5diU80unEVvNevw1nH4+ZV4DSLVJLSYI=
github.com/Masterminds/goutils v1.1.1/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
@@ -1050,6 +1052,8 @@ github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c/go.mod h1:lB8K/P019DLNhe
github.com/xdg/stringprep v1.0.0/go.mod h1:Jhud4/sHMO4oL310DaZAKk9ZaJ08SJfe+sJh0HrGL1Y=
github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510 h1:S2dVYn90KE98chqDkyE9Z4N61UnQd+KOfgp5Iu53llk=
github.com/xiang90/probing v0.0.0-20221125231312-a49e3df8f510/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
pkg/storage/unified/parquet/README.md
+6
View File
@@ -0,0 +1,6 @@
# Parquet Support
This package implements a limited parquet backend that is currently only useful
as a pass-though buffer while batch writing values.
Eventually this package could evolve into a full storage backend.
pkg/storage/unified/parquet/client.go
+78
View File
@@ -0,0 +1,78 @@
package parquet
import (
"context"
"errors"
"fmt"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
var (
_ resource.BatchStoreClient = (*writerClient)(nil)
_ resource.BatchStore_BatchProcessClient = (*writerClient)(nil)
errUnimplemented = errors.New("not implemented (BatchResourceWriter as BatchStoreClient shim)")
)
type writerClient struct {
writer resource.BatchResourceWriter
ctx context.Context
}
// NewBatchResourceWriterClient wraps a BatchResourceWriter so that it can be used as a ResourceStoreClient
func NewBatchResourceWriterClient(writer resource.BatchResourceWriter) *writerClient {
return &writerClient{writer: writer}
}
// Send implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) Send(req *resource.BatchRequest) error {
return w.writer.Write(w.ctx, req.Key, req.Value)
}
// BatchProcess implements resource.ResourceStoreClient.
func (w *writerClient) BatchProcess(ctx context.Context, opts ...grpc.CallOption) (resource.BatchStore_BatchProcessClient, error) {
if w.ctx != nil {
return nil, fmt.Errorf("only one batch request supported")
}
w.ctx = ctx
return w, nil
}
// CloseAndRecv implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) CloseAndRecv() (*resource.BatchResponse, error) {
return w.writer.CloseWithResults()
}
// CloseSend implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) CloseSend() error {
return w.writer.Close()
}
// Context implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) Context() context.Context {
return w.ctx
}
// Header implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) Header() (metadata.MD, error) {
return nil, errUnimplemented
}
// RecvMsg implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) RecvMsg(m any) error {
return errUnimplemented
}
// SendMsg implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) SendMsg(m any) error {
return errUnimplemented
}
// Trailer implements resource.ResourceStore_BatchProcessClient.
func (w *writerClient) Trailer() metadata.MD {
return nil
}
pkg/storage/unified/parquet/reader.go
+264
View File
@@ -0,0 +1,264 @@
package parquet
import (
"fmt"
"github.com/apache/arrow-go/v18/parquet"
"github.com/apache/arrow-go/v18/parquet/file"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
var (
_ resource.BatchRequestIterator = (*parquetReader)(nil)
)
func NewParquetReader(inputPath string, batchSize int64) (resource.BatchRequestIterator, error) {
return newResourceReader(inputPath, batchSize)
}
type parquetReader struct {
reader *file.Reader
namespace *stringColumn
group *stringColumn
resource *stringColumn
name *stringColumn
value *stringColumn
folder *stringColumn
action *int32Column
columns []columnBuffer
batchSize int64
defLevels []int16
repLevels []int16
// how many we already read
bufferSize int
bufferIndex int
rowGroupIDX int
req *resource.BatchRequest
err error
}
// Next implements resource.BatchRequestIterator.
func (r *parquetReader) Next() bool {
r.req = nil
for r.err == nil && r.reader != nil {
if r.bufferIndex >= r.bufferSize && r.value.reader.HasNext() {
r.bufferIndex = 0
r.err = r.readBatch()
if r.err != nil {
return false
}
r.bufferIndex = r.value.count
}
if r.bufferSize > r.bufferIndex {
i := r.bufferIndex
r.bufferIndex++
r.req = &resource.BatchRequest{
Key: &resource.ResourceKey{
Group: r.group.buffer[i].String(),
Resource: r.resource.buffer[i].String(),
Namespace: r.namespace.buffer[i].String(),
Name: r.name.buffer[i].String(),
},
Action: resource.BatchRequest_Action(r.action.buffer[i]),
Value: r.value.buffer[i].Bytes(),
Folder: r.folder.buffer[i].String(),
}
return true
}
r.rowGroupIDX++
if r.rowGroupIDX >= r.reader.NumRowGroups() {
_ = r.reader.Close()
r.reader = nil
return false
}
r.err = r.open(r.reader.RowGroup(r.rowGroupIDX))
}
return false
}
// Request implements resource.BatchRequestIterator.
func (r *parquetReader) Request() *resource.BatchRequest {
return r.req
}
// RollbackRequested implements resource.BatchRequestIterator.
func (r *parquetReader) RollbackRequested() bool {
return r.err != nil
}
func newResourceReader(inputPath string, batchSize int64) (*parquetReader, error) {
rdr, err := file.OpenParquetFile(inputPath, true)
if err != nil {
return nil, err
}
schema := rdr.MetaData().Schema
makeColumn := func(name string) *stringColumn {
index := schema.ColumnIndexByName(name)
if index < 0 {
err = fmt.Errorf("missing column: %s", name)
}
return &stringColumn{
index: index,
buffer: make([]parquet.ByteArray, batchSize),
}
}
reader := &parquetReader{
reader: rdr,
namespace: makeColumn("namespace"),
group: makeColumn("group"),
resource: makeColumn("resource"),
name: makeColumn("name"),
value: makeColumn("value"),
folder: makeColumn("folder"),
action: &int32Column{
index: schema.ColumnIndexByName("action"),
buffer: make([]int32, batchSize),
},
batchSize: batchSize,
defLevels: make([]int16, batchSize),
repLevels: make([]int16, batchSize),
}
if err != nil {
_ = rdr.Close()
return nil, err
}
reader.columns = []columnBuffer{
reader.namespace,
reader.group,
reader.resource,
reader.name,
reader.action,
reader.value,
}
// Empty file, close and return
if rdr.NumRowGroups() < 1 {
err = rdr.Close()
reader.reader = nil
return reader, err
}
err = reader.open(rdr.RowGroup(0))
if err != nil {
_ = rdr.Close()
return nil, err
}
// get the first batch
err = reader.readBatch()
if err != nil {
_ = rdr.Close()
return nil, err
}
return reader, nil
}
func (r *parquetReader) open(rgr *file.RowGroupReader) error {
for _, c := range r.columns {
err := c.open(rgr)
if err != nil {
return err
}
}
return nil
}
func (r *parquetReader) readBatch() error {
r.bufferIndex = 0
r.bufferSize = 0
for i, c := range r.columns {
count, err := c.batch(r.batchSize, r.defLevels, r.repLevels)
if err != nil {
return err
}
if i > 0 && r.bufferSize != count {
return fmt.Errorf("expecting the same size for all columns")
}
r.bufferSize = count
}
return nil
}
//-------------------------------
// Column support
//-------------------------------
type columnBuffer interface {
open(rgr *file.RowGroupReader) error
batch(batchSize int64, defLevels []int16, repLevels []int16) (int, error)
}
type stringColumn struct {
index int // within the schema
reader *file.ByteArrayColumnChunkReader
buffer []parquet.ByteArray
count int // the active count
}
func (c *stringColumn) open(rgr *file.RowGroupReader) error {
tmp, err := rgr.Column(c.index)
if err != nil {
return err
}
var ok bool
c.reader, ok = tmp.(*file.ByteArrayColumnChunkReader)
if !ok {
return fmt.Errorf("expected resource strings")
}
return nil
}
func (c *stringColumn) batch(batchSize int64, defLevels []int16, repLevels []int16) (int, error) {
_, count, err := c.reader.ReadBatch(batchSize, c.buffer, defLevels, repLevels)
c.count = count
return count, err
}
type int32Column struct {
index int // within the schemna
reader *file.Int32ColumnChunkReader
buffer []int32
count int // the active count
}
func (c *int32Column) open(rgr *file.RowGroupReader) error {
tmp, err := rgr.Column(c.index)
if err != nil {
return err
}
var ok bool
c.reader, ok = tmp.(*file.Int32ColumnChunkReader)
if !ok {
return fmt.Errorf("expected resource strings")
}
return nil
}
func (c *int32Column) batch(batchSize int64, defLevels []int16, repLevels []int16) (int, error) {
_, count, err := c.reader.ReadBatch(batchSize, c.buffer, defLevels, repLevels)
c.count = count
return count, err
}
//-------------------------------
// Column support
//-------------------------------
pkg/storage/unified/parquet/reader_test.go
+125
View File
@@ -0,0 +1,125 @@
package parquet
import (
"context"
"os"
"testing"
"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
func TestParquetWriteThenRead(t *testing.T) {
t.Run("read-write-couple-rows", func(t *testing.T) {
file, err := os.CreateTemp(t.TempDir(), "temp-*.parquet")
require.NoError(t, err)
defer func() { _ = os.Remove(file.Name()) }()
writer, err := NewParquetWriter(file)
require.NoError(t, err)
ctx := context.Background()
require.NoError(t, writer.Write(toKeyAndBytes(ctx, "ggg", "rrr", &unstructured.Unstructured{
Object: map[string]any{
"metadata": map[string]any{
"namespace": "ns",
"name": "aaa",
"resourceVersion": "1234",
"annotations": map[string]string{
utils.AnnoKeyFolder: "xyz",
},
},
"spec": map[string]any{
"hello": "first",
},
},
})))
require.NoError(t, writer.Write(toKeyAndBytes(ctx, "ggg", "rrr", &unstructured.Unstructured{
Object: map[string]any{
"metadata": map[string]any{
"namespace": "ns",
"name": "bbb",
"resourceVersion": "5678",
"generation": -999, // deleted action
},
"spec": map[string]any{
"hello": "second",
},
},
})))
require.NoError(t, writer.Write(toKeyAndBytes(ctx, "ggg", "rrr", &unstructured.Unstructured{
Object: map[string]any{
"metadata": map[string]any{
"namespace": "ns",
"name": "ccc",
"resourceVersion": "789",
"generation": 3, // modified action
},
"spec": map[string]any{
"hello": "thirt",
},
},
})))
res, err := writer.CloseWithResults()
require.NoError(t, err)
require.Equal(t, int64(3), res.Processed)
var keys []string
reader, err := newResourceReader(file.Name(), 20)
require.NoError(t, err)
for reader.Next() {
req := reader.Request()
keys = append(keys, req.Key.SearchID())
}
// Verify that we read all values
require.Equal(t, []string{
"rrr/ns/ggg/aaa",
"rrr/ns/ggg/bbb",
"rrr/ns/ggg/ccc",
}, keys)
})
t.Run("read-write-empty-db", func(t *testing.T) {
file, err := os.CreateTemp(t.TempDir(), "temp-*.parquet")
require.NoError(t, err)
defer func() { _ = os.Remove(file.Name()) }()
writer, err := NewParquetWriter(file)
require.NoError(t, err)
err = writer.Close()
require.NoError(t, err)
var keys []string
reader, err := newResourceReader(file.Name(), 20)
require.NoError(t, err)
for reader.Next() {
req := reader.Request()
keys = append(keys, req.Key.SearchID())
}
require.NoError(t, reader.err)
require.Empty(t, keys)
})
}
func toKeyAndBytes(ctx context.Context, group string, res string, obj *unstructured.Unstructured) (context.Context, *resource.ResourceKey, []byte) {
if obj.GetKind() == "" {
obj.SetKind(res)
}
if obj.GetAPIVersion() == "" {
obj.SetAPIVersion(group + "/vXyz")
}
data, _ := obj.MarshalJSON()
return ctx, &resource.ResourceKey{
Namespace: obj.GetNamespace(),
Resource: res,
Group: group,
Name: obj.GetName(),
}, data
}
pkg/storage/unified/parquet/writer.go
+209
View File
@@ -0,0 +1,209 @@
package parquet
import (
"context"
"io"
"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/memory"
"github.com/apache/arrow-go/v18/parquet"
"github.com/apache/arrow-go/v18/parquet/compress"
"github.com/apache/arrow-go/v18/parquet/pqarrow"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"github.com/grafana/grafana-app-sdk/logging"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/storage/unified/resource"
)
var (
_ resource.BatchResourceWriter = (*parquetWriter)(nil)
)
// Write resources into a parquet file
func NewParquetWriter(f io.Writer) (*parquetWriter, error) {
w := &parquetWriter{
pool: memory.DefaultAllocator,
schema: newSchema(nil),
buffer: 1024 * 10 * 100 * 10, // 10MB
logger: logging.DefaultLogger.With("logger", "parquet.writer"),
rsp: &resource.BatchResponse{},
summary: make(map[string]*resource.BatchResponse_Summary),
}
props := parquet.NewWriterProperties(
parquet.WithCompression(compress.Codecs.Brotli),
)
writer, err := pqarrow.NewFileWriter(w.schema, f, props, pqarrow.DefaultWriterProps())
if err != nil {
return nil, err
}
w.writer = writer
return w, w.init()
}
// ProcessBatch implements resource.BatchProcessingBackend.
func (w *parquetWriter) ProcessBatch(ctx context.Context, setting resource.BatchSettings, iter resource.BatchRequestIterator) *resource.BatchResponse {
defer func() { _ = w.Close() }()
var err error
for iter.Next() {
if iter.RollbackRequested() {
break
}
req := iter.Request()
err = w.Write(ctx, req.Key, req.Value)
if err != nil {
break
}
}
rsp, err := w.CloseWithResults()
if err != nil {
w.logger.Warn("error closing parquet file", "err", err)
}
if rsp == nil {
rsp = &resource.BatchResponse{}
}
if err != nil {
rsp.Error = resource.AsErrorResult(err)
}
return rsp
}
type parquetWriter struct {
pool memory.Allocator
buffer int
wrote int
schema *arrow.Schema
writer *pqarrow.FileWriter
logger logging.Logger
rv *array.Int64Builder
namespace *array.StringBuilder
group *array.StringBuilder
resource *array.StringBuilder
name *array.StringBuilder
folder *array.StringBuilder
action *array.Int8Builder
value *array.StringBuilder
rsp *resource.BatchResponse
summary map[string]*resource.BatchResponse_Summary
}
func (w *parquetWriter) CloseWithResults() (*resource.BatchResponse, error) {
err := w.Close()
return w.rsp, err
}
func (w *parquetWriter) Close() error {
if w.rv.Len() > 0 {
_ = w.flush()
}
w.logger.Info("close")
return w.writer.Close()
}
// writes the current buffer to parquet and re-inits the arrow buffer
func (w *parquetWriter) flush() error {
w.logger.Info("flush", "count", w.rv.Len())
rec := array.NewRecord(w.schema, []arrow.Array{
w.rv.NewArray(),
w.namespace.NewArray(),
w.group.NewArray(),
w.resource.NewArray(),
w.name.NewArray(),
w.folder.NewArray(),
w.action.NewArray(),
w.value.NewArray(),
}, int64(w.rv.Len()))
defer rec.Release()
err := w.writer.Write(rec)
if err != nil {
return err
}
return w.init()
}
func (w *parquetWriter) init() error {
w.rv = array.NewInt64Builder(w.pool)
w.namespace = array.NewStringBuilder(w.pool)
w.group = array.NewStringBuilder(w.pool)
w.resource = array.NewStringBuilder(w.pool)
w.name = array.NewStringBuilder(w.pool)
w.folder = array.NewStringBuilder(w.pool)
w.action = array.NewInt8Builder(w.pool)
w.value = array.NewStringBuilder(w.pool)
w.wrote = 0
return nil
}
func (w *parquetWriter) Write(ctx context.Context, key *resource.ResourceKey, value []byte) error {
w.rsp.Processed++
obj := &unstructured.Unstructured{}
err := obj.UnmarshalJSON(value)
if err != nil {
return err
}
meta, err := utils.MetaAccessor(obj)
if err != nil {
return err
}
rv, _ := meta.GetResourceVersionInt64() // it can be empty
w.rv.Append(rv)
w.namespace.Append(key.Namespace)
w.group.Append(key.Group)
w.resource.Append(key.Resource)
w.name.Append(key.Name)
w.folder.Append(meta.GetFolder())
w.value.Append(string(value))
var action resource.WatchEvent_Type
switch meta.GetGeneration() {
case 0, 1:
action = resource.WatchEvent_ADDED
case utils.DeletedGeneration:
action = resource.WatchEvent_DELETED
default:
action = resource.WatchEvent_MODIFIED
}
w.action.Append(int8(action))
w.wrote = w.wrote + len(value)
if w.wrote > w.buffer {
w.logger.Info("buffer full", "buffer", w.wrote, "max", w.buffer)
return w.flush()
}
summary := w.summary[key.BatchID()]
if summary == nil {
summary = &resource.BatchResponse_Summary{
Namespace: key.Namespace,
Group: key.Group,
Resource: key.Resource,
}
w.summary[key.BatchID()] = summary
w.rsp.Summary = append(w.rsp.Summary, summary)
}
summary.Count++
return nil
}
func newSchema(metadata *arrow.Metadata) *arrow.Schema {
return arrow.NewSchema([]arrow.Field{
{Name: "resource_version", Type: &arrow.Int64Type{}, Nullable: false},
{Name: "group", Type: &arrow.StringType{}, Nullable: false},
{Name: "resource", Type: &arrow.StringType{}, Nullable: false},
{Name: "namespace", Type: &arrow.StringType{}, Nullable: false},
{Name: "name", Type: &arrow.StringType{}, Nullable: false},
{Name: "folder", Type: &arrow.StringType{}, Nullable: false},
{Name: "action", Type: &arrow.Int8Type{}, Nullable: false}, // 1,2,3
{Name: "value", Type: &arrow.StringType{}, Nullable: false},
}, metadata)
}
pkg/storage/unified/resource/batch.go
+297
View File
@@ -0,0 +1,297 @@
package resource
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"google.golang.org/grpc/metadata"
authlib "github.com/grafana/authlib/types"
"github.com/grafana/grafana/pkg/apimachinery/utils"
)
const grpcMetaKeyCollection = "x-gf-batch-collection"
const grpcMetaKeyRebuildCollection = "x-gf-batch-rebuild-collection"
const grpcMetaKeySkipValidation = "x-gf-batch-skip-validation"
func grpcMetaValueIsTrue(vals []string) bool {
return len(vals) == 1 && vals[0] == "true"
}
type BatchRequestIterator interface {
Next() bool
// The next event we should process
Request() *BatchRequest
// Rollback requested
RollbackRequested() bool
}
type BatchProcessingBackend interface {
ProcessBatch(ctx context.Context, setting BatchSettings, iter BatchRequestIterator) *BatchResponse
}
type BatchResourceWriter interface {
io.Closer
Write(ctx context.Context, key *ResourceKey, value []byte) error
// Called when finished writing
CloseWithResults() (*BatchResponse, error)
}
type BatchSettings struct {
// All requests will be within this namespace/group/resource
Collection []*ResourceKey
// The batch will include everything from the collection
// - all existing values will be removed/replaced if the batch completes successfully
RebuildCollection bool
// The byte[] payload and folder has already been validated - no need to decode and verify
SkipValidation bool
}
func (x *BatchSettings) ToMD() metadata.MD {
md := make(metadata.MD)
if len(x.Collection) > 0 {
for _, v := range x.Collection {
md[grpcMetaKeyCollection] = append(md[grpcMetaKeyCollection], v.SearchID())
}
}
if x.RebuildCollection {
md[grpcMetaKeyRebuildCollection] = []string{"true"}
}
if x.SkipValidation {
md[grpcMetaKeySkipValidation] = []string{"true"}
}
return md
}
func NewBatchSettings(md metadata.MD) (BatchSettings, error) {
settings := BatchSettings{}
for k, v := range md {
switch k {
case grpcMetaKeyCollection:
for _, c := range v {
key := &ResourceKey{}
err := key.ReadSearchID(c)
if err != nil {
return settings, fmt.Errorf("error reading collection metadata: %s / %w", c, err)
}
settings.Collection = append(settings.Collection, key)
}
case grpcMetaKeyRebuildCollection:
settings.RebuildCollection = grpcMetaValueIsTrue(v)
case grpcMetaKeySkipValidation:
settings.SkipValidation = grpcMetaValueIsTrue(v)
}
}
return settings, nil
}
// BatchWrite implements ResourceServer.
// All requests must be to the same NAMESPACE/GROUP/RESOURCE
func (s *server) BatchProcess(stream BatchStore_BatchProcessServer) error {
ctx := stream.Context()
user, ok := authlib.AuthInfoFrom(ctx)
if !ok || user == nil {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "no user found in context",
Code: http.StatusUnauthorized,
},
})
}
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "unable to read metadata gRPC request",
Code: http.StatusPreconditionFailed,
},
})
}
runner := &batchRunner{
checker: make(map[string]authlib.ItemChecker), // Can create
stream: stream,
}
settings, err := NewBatchSettings(md)
if err != nil {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "error reading settings",
Reason: err.Error(),
Code: http.StatusPreconditionFailed,
},
})
}
if len(settings.Collection) < 1 {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "Missing target collection(s) in request header",
Code: http.StatusBadRequest,
},
})
}
// HACK!!! always allow everything!!!!!!
access := authlib.FixedAccessClient(true)
if settings.RebuildCollection {
for _, k := range settings.Collection {
// Can we delete the whole collection
rsp, err := access.Check(ctx, user, authlib.CheckRequest{
Namespace: k.Namespace,
Group: k.Group,
Resource: k.Resource,
Verb: utils.VerbDeleteCollection,
})
if err != nil || !rsp.Allowed {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: fmt.Sprintf("Requester must be able to: %s", utils.VerbDeleteCollection),
Code: http.StatusForbidden,
},
})
}
// This will be called for each request -- with the folder ID
runner.checker[k.BatchID()], err = access.Compile(ctx, user, authlib.ListRequest{
Namespace: k.Namespace,
Group: k.Group,
Resource: k.Resource,
Verb: utils.VerbCreate,
})
if err != nil {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "Unable to check `create` permission",
Code: http.StatusForbidden,
},
})
}
}
} else {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "Batch currently only supports RebuildCollection",
Code: http.StatusBadRequest,
},
})
}
backend, ok := s.backend.(BatchProcessingBackend)
if !ok {
return stream.SendAndClose(&BatchResponse{
Error: &ErrorResult{
Message: "The server backend does not support batch processing",
Code: http.StatusNotImplemented,
},
})
}
// BatchProcess requests
rsp := backend.ProcessBatch(ctx, settings, runner)
if rsp == nil {
rsp = &BatchResponse{
Error: &ErrorResult{
Code: http.StatusInternalServerError,
Message: "Nothing returned from process batch",
},
}
}
if runner.err != nil {
rsp.Error = AsErrorResult(runner.err)
}
if rsp.Error == nil && s.search != nil {
// Rebuild any changed indexes
for _, summary := range rsp.Summary {
_, _, err := s.search.build(ctx, NamespacedResource{
Namespace: summary.Namespace,
Group: summary.Group,
Resource: summary.Resource,
}, summary.Count, summary.ResourceVersion)
if err != nil {
s.log.Warn("error building search index after batch load", "err", err)
rsp.Error = &ErrorResult{
Code: http.StatusInternalServerError,
Message: "err building search index: " + summary.Resource,
Reason: err.Error(),
}
}
}
}
return stream.SendAndClose(rsp)
}
var (
_ BatchRequestIterator = (*batchRunner)(nil)
)
type batchRunner struct {
stream BatchStore_BatchProcessServer
rollback bool
request *BatchRequest
err error
checker map[string]authlib.ItemChecker
}
// Next implements BatchRequestIterator.
func (b *batchRunner) Next() bool {
if b.rollback {
return true
}
b.request, b.err = b.stream.Recv()
if errors.Is(b.err, io.EOF) {
b.err = nil
b.rollback = false
b.request = nil
return false
}
if b.err != nil {
b.rollback = true
return true
}
if b.request != nil {
key := b.request.Key
k := key.BatchID()
checker, ok := b.checker[k]
if !ok {
b.err = fmt.Errorf("missing access control for: %s", k)
b.rollback = true
} else if !checker(key.Namespace, key.Name, b.request.Folder) {
b.err = fmt.Errorf("not allowed to create resource")
b.rollback = true
}
return true
}
return false
}
// Request implements BatchRequestIterator.
func (b *batchRunner) Request() *BatchRequest {
if b.rollback {
return nil
}
return b.request
}
// RollbackRequested implements BatchRequestIterator.
func (b *batchRunner) RollbackRequested() bool {
if b.rollback {
b.rollback = false // break iterator
return true
}
return false
}
pkg/storage/unified/resource/client.go
+17 -8
View File
@@ -25,6 +25,7 @@ type ResourceClient interface {
ResourceStoreClient
ResourceIndexClient
RepositoryIndexClient
BatchStoreClient
BlobStoreClient
DiagnosticsClient
}
@@ -34,6 +35,7 @@ type resourceClient struct {
ResourceStoreClient
ResourceIndexClient
RepositoryIndexClient
BatchStoreClient
BlobStoreClient
DiagnosticsClient
}
@@ -44,6 +46,7 @@ func NewLegacyResourceClient(channel *grpc.ClientConn) ResourceClient {
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
RepositoryIndexClient: NewRepositoryIndexClient(cc),
BatchStoreClient: NewBatchStoreClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
}
@@ -59,6 +62,7 @@ func NewLocalResourceClient(server ResourceServer) ResourceClient {
&ResourceIndex_ServiceDesc,
&RepositoryIndex_ServiceDesc,
&BlobStore_ServiceDesc,
&BatchStore_ServiceDesc,
&Diagnostics_ServiceDesc,
} {
channel.RegisterService(
@@ -82,6 +86,7 @@ func NewLocalResourceClient(server ResourceServer) ResourceClient {
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
RepositoryIndexClient: NewRepositoryIndexClient(cc),
BatchStoreClient: NewBatchStoreClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
}
@@ -101,10 +106,12 @@ func NewGRPCResourceClient(tracer tracing.Tracer, conn *grpc.ClientConn) (Resour
cc := grpchan.InterceptClientConn(conn, clientInt.UnaryClientInterceptor, clientInt.StreamClientInterceptor)
return &resourceClient{
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
BatchStoreClient: NewBatchStoreClient(cc),
RepositoryIndexClient: NewRepositoryIndexClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
}, nil
}
@@ -126,10 +133,12 @@ func NewCloudResourceClient(tracer tracing.Tracer, conn *grpc.ClientConn, cfg au
cc := grpchan.InterceptClientConn(conn, clientInt.UnaryClientInterceptor, clientInt.StreamClientInterceptor)
return &resourceClient{
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
ResourceStoreClient: NewResourceStoreClient(cc),
ResourceIndexClient: NewResourceIndexClient(cc),
BlobStoreClient: NewBlobStoreClient(cc),
BatchStoreClient: NewBatchStoreClient(cc),
RepositoryIndexClient: NewRepositoryIndexClient(cc),
DiagnosticsClient: NewDiagnosticsClient(cc),
}, nil
}
pkg/storage/unified/resource/keys.go
+23 -4
View File
@@ -48,24 +48,43 @@ func (x *ResourceKey) SearchID() string {
sb.WriteString(x.Group)
sb.WriteString("/")
sb.WriteString(x.Resource)
sb.WriteString("/")
sb.WriteString(x.Name)
if x.Name != "" {
sb.WriteString("/")
sb.WriteString(x.Name)
}
return sb.String()
}
func (x *ResourceKey) ReadSearchID(v string) error {
parts := strings.Split(v, "/")
if len(parts) != 4 {
if len(parts) < 3 {
return fmt.Errorf("invalid search id (expecting 3 slashes)")
}
x.Namespace = parts[0]
x.Group = parts[1]
x.Resource = parts[2]
x.Name = parts[3]
if len(parts) > 3 {
x.Name = parts[3]
}
if x.Namespace == clusterNamespace {
x.Namespace = ""
}
return nil
}
// The namespace/group/resource
func (x *ResourceKey) BatchID() string {
var sb strings.Builder
if x.Namespace == "" {
sb.WriteString(clusterNamespace)
} else {
sb.WriteString(x.Namespace)
}
sb.WriteString("/")
sb.WriteString(x.Group)
sb.WriteString("/")
sb.WriteString(x.Resource)
return sb.String()
}
pkg/storage/unified/resource/keys_test.go
+8 -1
View File
@@ -33,13 +33,20 @@ func TestSearchIDKeys(t *testing.T) {
Resource: "resource",
Name: "name",
}},
{input: "/group/resource/", // missing name
{input: "/group/resource/",
expected: &ResourceKey{
Namespace: "",
Group: "group",
Resource: "resource",
Name: "",
}},
{input: "default/group/resource",
expected: &ResourceKey{
Namespace: "default",
Group: "group",
Resource: "resource",
Name: "",
}},
{input: "**cluster**/group/resource/aaa", // cluster namespace
expected: &ResourceKey{
Namespace: "",
pkg/storage/unified/resource/resource.pb.go
+1024 -593
View File
File diff suppressed because it is too large. Load diff
pkg/storage/unified/resource/resource.proto
+67 -1
View File
@@ -308,7 +308,7 @@ message WatchEvent {
// Timestamp the event was sent
int64 timestamp = 1;
// Timestamp the event was sent
// The event type
Type type = 2;
// Resource version for the object
@@ -318,6 +318,65 @@ message WatchEvent {
Resource previous = 4;
}
message BatchRequest {
enum Action {
// will be an error
UNKNOWN = 0;
// Matches Watch event enum
ADDED = 1;
MODIFIED = 2;
DELETED = 3;
}
// NOTE everything in the same stream must share the same Namespace/Group/Resource
ResourceKey key = 1;
// Requested action
Action action = 2;
// The resource value
bytes value = 3;
// Hint that a new version will be written on-top of this
string folder = 4;
}
message BatchResponse {
message Summary {
string namespace = 1;
string group = 2;
string resource = 3;
int64 count = 4;
int64 history = 5;
int64 resource_version = 6; // The max saved RV
// The previous count
int64 previous_count = 7;
int64 previous_history = 8;
}
// Collect a few invalid messages
message Rejected {
ResourceKey key = 1;
BatchRequest.Action action = 2;
string error = 3;
}
// Error details
ErrorResult error = 1;
// Total events processed
int64 processed = 2;
// Summary status for the processed values
repeated Summary summary = 3;
// Rejected
repeated Rejected rejected = 4;
}
// Get statistics across multiple resources
// For these queries, we do not need authorization to see the actual values
message ResourceStatsRequest {
@@ -754,6 +813,13 @@ service ResourceStore {
rpc Watch(WatchRequest) returns (stream WatchEvent);
}
service BatchStore {
// Write multiple resources to the same Namespace/Group/Resource
// Events will not be sent until the stream is complete
// Only the *create* permissions is checked
rpc BatchProcess(stream BatchRequest) returns (BatchResponse);
}
// Unlike the ResourceStore, this service can be exposed to clients directly
// It should be implemented with efficient indexes and does not need read-after-write semantics
service ResourceIndex {
pkg/storage/unified/resource/resource_grpc.pb.go
+129
View File
@@ -385,6 +385,135 @@ var ResourceStore_ServiceDesc = grpc.ServiceDesc{
Metadata: "resource.proto",
}
const (
BatchStore_BatchProcess_FullMethodName = "/resource.BatchStore/BatchProcess"
)
// BatchStoreClient is the client API for BatchStore service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type BatchStoreClient interface {
// Write multiple resources to the same Namespace/Group/Resource
// Events will not be sent until the stream is complete
// Only the *create* permissions is checked
BatchProcess(ctx context.Context, opts ...grpc.CallOption) (BatchStore_BatchProcessClient, error)
}
type batchStoreClient struct {
cc grpc.ClientConnInterface
}
func NewBatchStoreClient(cc grpc.ClientConnInterface) BatchStoreClient {
return &batchStoreClient{cc}
}
func (c *batchStoreClient) BatchProcess(ctx context.Context, opts ...grpc.CallOption) (BatchStore_BatchProcessClient, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &BatchStore_ServiceDesc.Streams[0], BatchStore_BatchProcess_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &batchStoreBatchProcessClient{ClientStream: stream}
return x, nil
}
type BatchStore_BatchProcessClient interface {
Send(*BatchRequest) error
CloseAndRecv() (*BatchResponse, error)
grpc.ClientStream
}
type batchStoreBatchProcessClient struct {
grpc.ClientStream
}
func (x *batchStoreBatchProcessClient) Send(m *BatchRequest) error {
return x.ClientStream.SendMsg(m)
}
func (x *batchStoreBatchProcessClient) CloseAndRecv() (*BatchResponse, error) {
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
m := new(BatchResponse)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
// BatchStoreServer is the server API for BatchStore service.
// All implementations should embed UnimplementedBatchStoreServer
// for forward compatibility
type BatchStoreServer interface {
// Write multiple resources to the same Namespace/Group/Resource
// Events will not be sent until the stream is complete
// Only the *create* permissions is checked
BatchProcess(BatchStore_BatchProcessServer) error
}
// UnimplementedBatchStoreServer should be embedded to have forward compatible implementations.
type UnimplementedBatchStoreServer struct {
}
func (UnimplementedBatchStoreServer) BatchProcess(BatchStore_BatchProcessServer) error {
return status.Errorf(codes.Unimplemented, "method BatchProcess not implemented")
}
// UnsafeBatchStoreServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to BatchStoreServer will
// result in compilation errors.
type UnsafeBatchStoreServer interface {
mustEmbedUnimplementedBatchStoreServer()
}
func RegisterBatchStoreServer(s grpc.ServiceRegistrar, srv BatchStoreServer) {
s.RegisterService(&BatchStore_ServiceDesc, srv)
}
func _BatchStore_BatchProcess_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(BatchStoreServer).BatchProcess(&batchStoreBatchProcessServer{ServerStream: stream})
}
type BatchStore_BatchProcessServer interface {
SendAndClose(*BatchResponse) error
Recv() (*BatchRequest, error)
grpc.ServerStream
}
type batchStoreBatchProcessServer struct {
grpc.ServerStream
}
func (x *batchStoreBatchProcessServer) SendAndClose(m *BatchResponse) error {
return x.ServerStream.SendMsg(m)
}
func (x *batchStoreBatchProcessServer) Recv() (*BatchRequest, error) {
m := new(BatchRequest)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
// BatchStore_ServiceDesc is the grpc.ServiceDesc for BatchStore service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var BatchStore_ServiceDesc = grpc.ServiceDesc{
ServiceName: "resource.BatchStore",
HandlerType: (*BatchStoreServer)(nil),
Methods: []grpc.MethodDesc{},
Streams: []grpc.StreamDesc{
{
StreamName: "BatchProcess",
Handler: _BatchStore_BatchProcess_Handler,
ClientStreams: true,
},
},
Metadata: "resource.proto",
}
const (
ResourceIndex_Search_FullMethodName = "/resource.ResourceIndex/Search"
ResourceIndex_GetStats_FullMethodName = "/resource.ResourceIndex/GetStats"
pkg/storage/unified/resource/search.go
+11 -2
View File
@@ -107,6 +107,9 @@ func newSearchSupport(opts SearchOptions, storage StorageBackend, access types.A
if opts.Backend == nil {
return nil, nil
}
if tracer == nil {
return nil, fmt.Errorf("missing tracer")
}
if opts.WorkerThreads < 1 {
opts.WorkerThreads = 1
@@ -384,6 +387,11 @@ func (s *searchSupport) init(ctx context.Context) error {
for {
v := <-events
// Skip events during batch updates
if v.PreviousRV < 0 {
continue
}
s.handleEvent(watchctx, v)
}
}()
@@ -496,7 +504,7 @@ func (s *searchSupport) getOrCreateIndex(ctx context.Context, key NamespacedReso
if idx == nil {
idx, _, err = s.build(ctx, key, 10, 0) // unknown size and RV
if err != nil {
return nil, err
return nil, fmt.Errorf("error building search index, %w", err)
}
if idx == nil {
return nil, fmt.Errorf("nil index after build")
@@ -541,7 +549,8 @@ func (s *searchSupport) build(ctx context.Context, nsr NamespacedResource, size
// Convert it to an indexable document
doc, err := builder.BuildDocument(ctx, key, iter.ResourceVersion(), iter.Value())
if err != nil {
return err
s.log.Error("error building search document", "key", key.SearchID(), "err", err)
continue
}
// And finally write it to the index
pkg/storage/unified/resource/server.go
+9 -2
View File
@@ -26,6 +26,7 @@ import (
// ResourceServer implements all gRPC services
type ResourceServer interface {
ResourceStoreServer
BatchStoreServer
ResourceIndexServer
RepositoryIndexServer
BlobStoreServer
@@ -260,7 +261,7 @@ func NewResourceServer(opts ResourceServerOptions) (ResourceServer, error) {
err := s.Init(ctx)
if err != nil {
s.log.Error("error initializing resource server", "error", err)
s.log.Error("resource server init failed", "error", err)
return nil, err
}
@@ -314,7 +315,7 @@ func (s *server) Init(ctx context.Context) error {
}
if s.initErr != nil {
s.log.Error("error initializing resource server", "error", s.initErr)
s.log.Error("error running resource server init", "error", s.initErr)
}
})
return s.initErr
@@ -921,6 +922,12 @@ func (s *server) initWatcher() error {
for {
// pipe all events
v := <-events
// Skip events during batch updates
if v.PreviousRV < 0 {
continue
}
s.log.Debug("Server. Streaming Event", "type", v.Type, "previousRV", v.PreviousRV, "group", v.Key.Group, "namespace", v.Key.Namespace, "resource", v.Key.Resource, "name", v.Key.Name)
s.mostRecentRV.Store(v.ResourceVersion)
out <- v
pkg/storage/unified/sql/backend.go
+10 -2
View File
@@ -59,6 +59,7 @@ func NewBackend(opts BackendOptions) (Backend, error) {
tracer: opts.Tracer,
dbProvider: opts.DBProvider,
pollingInterval: pollingInterval,
batchLock: &batchLock{running: make(map[string]bool)},
}, nil
}
@@ -77,6 +78,7 @@ type backend struct {
dbProvider db.DBProvider
db db.DB
dialect sqltemplate.Dialect
batchLock *batchLock
// watch streaming
//stream chan *resource.WatchEvent
@@ -701,7 +703,7 @@ func (b *backend) WatchWriteEvents(ctx context.Context) (<-chan *resource.Writte
// Get the latest RV
since, err := b.listLatestRVs(ctx)
if err != nil {
return nil, fmt.Errorf("get the latest resource version: %w", err)
return nil, fmt.Errorf("watch, get latest resource version: %w", err)
}
// Start the poller
stream := make(chan *resource.WrittenEvent)
@@ -713,17 +715,23 @@ func (b *backend) poller(ctx context.Context, since groupResourceRV, stream chan
t := time.NewTicker(b.pollingInterval)
defer close(stream)
defer t.Stop()
isSQLite := b.dialect.DialectName() == "sqlite"
for {
select {
case <-b.done:
return
case <-t.C:
// Block polling duffing import to avoid database locked issues
if isSQLite && b.batchLock.Active() {
continue
}
ctx, span := b.tracer.Start(ctx, tracePrefix+"poller")
// List the latest RVs
grv, err := b.listLatestRVs(ctx)
if err != nil {
b.log.Error("get the latest resource version", "err", err)
b.log.Error("poller get latest resource version", "err", err)
t.Reset(b.pollingInterval)
continue
}
pkg/storage/unified/sql/batch.go
+338
View File
@@ -0,0 +1,338 @@
package sql
import (
"context"
"fmt"
"net/http"
"os"
"sync"
"time"
"github.com/google/uuid"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"github.com/grafana/grafana-app-sdk/logging"
"github.com/grafana/grafana/pkg/apimachinery/utils"
"github.com/grafana/grafana/pkg/storage/unified/parquet"
"github.com/grafana/grafana/pkg/storage/unified/resource"
"github.com/grafana/grafana/pkg/storage/unified/sql/db"
"github.com/grafana/grafana/pkg/storage/unified/sql/dbutil"
"github.com/grafana/grafana/pkg/storage/unified/sql/sqltemplate"
)
var (
_ resource.BatchProcessingBackend = (*backend)(nil)
)
type batchRV struct {
max int64
counter int64
}
func newBatchRV() *batchRV {
t := time.Now().Truncate(time.Second * 10)
return &batchRV{
max: (t.UnixMicro() / 10000000) * 10000000,
counter: 0,
}
}
func (x *batchRV) next(obj metav1.Object) int64 {
ts := obj.GetCreationTimestamp().UnixMicro()
anno := obj.GetAnnotations()
if anno != nil {
v := anno[utils.AnnoKeyUpdatedTimestamp]
t, err := time.Parse(time.RFC3339, v)
if err == nil {
ts = t.UnixMicro()
}
}
if ts > x.max || ts < 10000000 {
ts = x.max
}
x.counter++
return (ts/10000000)*10000000 + x.counter
}
type batchLock struct {
running map[string]bool
mu sync.Mutex
}
func (x *batchLock) Start(keys []*resource.ResourceKey) error {
x.mu.Lock()
defer x.mu.Unlock()
// First verify that it is not already running
ids := make([]string, len(keys))
for i, k := range keys {
id := k.BatchID()
if x.running[id] {
return &apierrors.StatusError{ErrStatus: metav1.Status{
Code: http.StatusPreconditionFailed,
Message: "batch export is already running",
}}
}
ids[i] = id
}
// Then add the keys to the lock
for _, k := range ids {
x.running[k] = true
}
return nil
}
func (x *batchLock) Finish(keys []*resource.ResourceKey) {
x.mu.Lock()
defer x.mu.Unlock()
for _, k := range keys {
delete(x.running, k.BatchID())
}
}
func (x *batchLock) Active() bool {
x.mu.Lock()
defer x.mu.Unlock()
return len(x.running) > 0
}
func (b *backend) ProcessBatch(ctx context.Context, setting resource.BatchSettings, iter resource.BatchRequestIterator) *resource.BatchResponse {
err := b.batchLock.Start(setting.Collection)
if err != nil {
return &resource.BatchResponse{
Error: resource.AsErrorResult(err),
}
}
defer b.batchLock.Finish(setting.Collection)
// We may want to first write parquet, then read parquet
if b.dialect.DialectName() == "sqlite" {
file, err := os.CreateTemp("", "grafana-batch-export-*.parquet")
if err != nil {
return &resource.BatchResponse{
Error: resource.AsErrorResult(err),
}
}
writer, err := parquet.NewParquetWriter(file)
if err != nil {
return &resource.BatchResponse{
Error: resource.AsErrorResult(err),
}
}
// write batch to parquet
rsp := writer.ProcessBatch(ctx, setting, iter)
if rsp.Error != nil {
return rsp
}
b.log.Info("using parquet buffer", "parquet", file)
// Replace the iterator with one from parquet
iter, err = parquet.NewParquetReader(file.Name(), 50)
if err != nil {
return &resource.BatchResponse{
Error: resource.AsErrorResult(err),
}
}
}
return b.processBatch(ctx, setting, iter)
}
// internal batch process
func (b *backend) processBatch(ctx context.Context, setting resource.BatchSettings, iter resource.BatchRequestIterator) *resource.BatchResponse {
rsp := &resource.BatchResponse{}
err := b.db.WithTx(ctx, ReadCommitted, func(ctx context.Context, tx db.Tx) error {
rollbackWithError := func(err error) error {
txerr := tx.Rollback()
if txerr != nil {
b.log.Warn("rollback", "error", txerr)
} else {
b.log.Info("rollback")
}
return err
}
batch := &batchWroker{
ctx: ctx,
tx: tx,
dialect: b.dialect,
logger: logging.FromContext(ctx),
}
// Calculate the RV based on incoming request timestamps
rv := newBatchRV()
summaries := make(map[string]*resource.BatchResponse_Summary, len(setting.Collection)*4)
// First clear everything in the transaction
if setting.RebuildCollection {
for _, key := range setting.Collection {
summary, err := batch.deleteCollection(key)
if err != nil {
return rollbackWithError(err)
}
summaries[key.BatchID()] = summary
rsp.Summary = append(rsp.Summary, summary)
}
}
obj := &unstructured.Unstructured{}
// Write each event into the history
for iter.Next() {
if iter.RollbackRequested() {
return rollbackWithError(nil)
}
req := iter.Request()
if req == nil {
return rollbackWithError(fmt.Errorf("missing request"))
}
rsp.Processed++
if req.Action == resource.BatchRequest_UNKNOWN {
rsp.Rejected = append(rsp.Rejected, &resource.BatchResponse_Rejected{
Key: req.Key,
Action: req.Action,
Error: "unknown action",
})
continue
}
err := obj.UnmarshalJSON(req.Value)
if err != nil {
rsp.Rejected = append(rsp.Rejected, &resource.BatchResponse_Rejected{
Key: req.Key,
Action: req.Action,
Error: "unable to unmarshal json",
})
continue
}
// Write the event to history
if _, err := dbutil.Exec(ctx, tx, sqlResourceHistoryInsert, sqlResourceRequest{
SQLTemplate: sqltemplate.New(b.dialect),
WriteEvent: resource.WriteEvent{
Key: req.Key,
Type: resource.WatchEvent_Type(req.Action),
Value: req.Value,
PreviousRV: -1, // Used for WATCH, but we want to skip watch events
},
Folder: req.Folder,
GUID: uuid.NewString(),
ResourceVersion: rv.next(obj),
}); err != nil {
return rollbackWithError(fmt.Errorf("insert into resource history: %w", err))
}
}
// Now update the resource table from history
for _, key := range setting.Collection {
k := fmt.Sprintf("%s/%s/%s", key.Namespace, key.Group, key.Resource)
summary := summaries[k]
if summary == nil {
return rollbackWithError(fmt.Errorf("missing summary key for: %s", k))
}
err := batch.syncCollection(key, summary)
if err != nil {
return err
}
// Make sure the collection RV is above our last written event
_, err = b.resourceVersionAtomicInc(ctx, tx, key)
if err != nil {
b.log.Warn("error increasing RV", "error", err)
}
}
return nil
})
if err != nil {
rsp.Error = resource.AsErrorResult(err)
}
return rsp
}
type batchWroker struct {
ctx context.Context
tx db.ContextExecer
dialect sqltemplate.Dialect
logger logging.Logger
}
// This will remove everything from the `resource` and `resource_history` table for a given namespace/group/resource
func (w *batchWroker) deleteCollection(key *resource.ResourceKey) (*resource.BatchResponse_Summary, error) {
summary := &resource.BatchResponse_Summary{
Namespace: key.Namespace,
Group: key.Group,
Resource: key.Resource,
}
// First delete history
res, err := dbutil.Exec(w.ctx, w.tx, sqlResourceHistoryDelete, &sqlResourceHistoryDeleteRequest{
SQLTemplate: sqltemplate.New(w.dialect),
Namespace: key.Namespace,
Group: key.Group,
Resource: key.Resource,
})
if err != nil {
return nil, err
}
summary.PreviousHistory, err = res.RowsAffected()
if err != nil {
return nil, err
}
// Next delete the active resource table
res, err = dbutil.Exec(w.ctx, w.tx, sqlResourceDelete, &sqlResourceRequest{
SQLTemplate: sqltemplate.New(w.dialect),
WriteEvent: resource.WriteEvent{
Key: key,
},
})
if err != nil {
return nil, err
}
summary.PreviousCount, err = res.RowsAffected()
return summary, err
}
// Copy the latest value from history into the active resource table
func (w *batchWroker) syncCollection(key *resource.ResourceKey, summary *resource.BatchResponse_Summary) error {
w.logger.Info("synchronize collection", "key", key.BatchID())
_, err := dbutil.Exec(w.ctx, w.tx, sqlResourceInsertFromHistory, &sqlResourceInsertFromHistoryRequest{
SQLTemplate: sqltemplate.New(w.dialect),
Key: key,
})
if err != nil {
return err
}
w.logger.Info("get stats (still in transaction)", "key", key.BatchID())
rows, err := dbutil.QueryRows(w.ctx, w.tx, sqlResourceStats, &sqlStatsRequest{
SQLTemplate: sqltemplate.New(w.dialect),
Namespace: key.Namespace,
Group: key.Group,
Resource: key.Resource,
})
if err != nil {
return err
}
if rows != nil {
defer func() {
_ = rows.Close()
}()
}
if rows.Next() {
row := resource.ResourceStats{}
return rows.Scan(&row.Namespace, &row.Group, &row.Resource,
&summary.Count,
&summary.ResourceVersion)
}
return err
}
pkg/storage/unified/sql/batch_test.go
+24
View File
@@ -0,0 +1,24 @@
package sql
import (
"testing"
"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)
func TestBatch(t *testing.T) {
t.Parallel()
t.Run("rv iterator", func(t *testing.T) {
t.Parallel()
rv := newBatchRV()
v0 := rv.next(&unstructured.Unstructured{})
v1 := rv.next(&unstructured.Unstructured{})
v2 := rv.next(&unstructured.Unstructured{})
require.True(t, v0 > 1000)
require.Equal(t, int64(1), v1-v0)
require.Equal(t, int64(1), v2-v1)
})
}
pkg/storage/unified/sql/data/resource_insert_from_history.sql
+52
View File
@@ -0,0 +1,52 @@
INSERT INTO {{ .Ident "resource" }}
SELECT
kv.{{ .Ident "guid" }},
kv.{{ .Ident "resource_version" }},
kv.{{ .Ident "group" }},
kv.{{ .Ident "resource" }},
kv.{{ .Ident "namespace" }},
kv.{{ .Ident "name" }},
kv.{{ .Ident "value" }},
kv.{{ .Ident "action" }},
kv.{{ .Ident "label_set" }},
kv.{{ .Ident "previous_resource_version" }},
kv.{{ .Ident "folder" }}
FROM {{ .Ident "resource_history" }} AS kv
INNER JOIN (
SELECT {{ .Ident "namespace" }}, {{ .Ident "group" }}, {{ .Ident "resource" }}, {{ .Ident "name" }}, max({{ .Ident "resource_version" }}) AS {{ .Ident "resource_version" }}
FROM {{ .Ident "resource_history" }} AS mkv
WHERE 1 = 1
{{ if .Key.Namespace }}
AND {{ .Ident "namespace" }} = {{ .Arg .Key.Namespace }}
{{ end }}
{{ if .Key.Group }}
AND {{ .Ident "group" }} = {{ .Arg .Key.Group }}
{{ end }}
{{ if .Key.Resource }}
AND {{ .Ident "resource" }} = {{ .Arg .Key.Resource }}
{{ end }}
{{ if .Key.Name }}
AND {{ .Ident "name" }} = {{ .Arg .Key.Name }}
{{ end }}
GROUP BY mkv.{{ .Ident "namespace" }}, mkv.{{ .Ident "group" }}, mkv.{{ .Ident "resource" }}, mkv.{{ .Ident "name" }}
) AS maxkv
ON maxkv.{{ .Ident "resource_version" }} = kv.{{ .Ident "resource_version" }}
AND maxkv.{{ .Ident "namespace" }} = kv.{{ .Ident "namespace" }}
AND maxkv.{{ .Ident "group" }} = kv.{{ .Ident "group" }}
AND maxkv.{{ .Ident "resource" }} = kv.{{ .Ident "resource" }}
AND maxkv.{{ .Ident "name" }} = kv.{{ .Ident "name" }}
WHERE kv.{{ .Ident "action" }} != 3
{{ if .Key.Namespace }}
AND kv.{{ .Ident "namespace" }} = {{ .Arg .Key.Namespace }}
{{ end }}
{{ if .Key.Group }}
AND kv.{{ .Ident "group" }} = {{ .Arg .Key.Group }}
{{ end }}
{{ if .Key.Resource }}
AND kv.{{ .Ident "resource" }} = {{ .Arg .Key.Resource }}
{{ end }}
{{ if .Key.Name }}
AND kv.{{ .Ident "name" }} = {{ .Arg .Key.Name }}
{{ end }}
ORDER BY kv.{{ .Ident "resource_version" }} ASC
;
pkg/storage/unified/sql/queries.go
+13
View File
@@ -44,6 +44,7 @@ var (
sqlResourceHistoryPoll = mustTemplate("resource_history_poll.sql")
sqlResourceHistoryGet = mustTemplate("resource_history_get.sql")
sqlResourceHistoryDelete = mustTemplate("resource_history_delete.sql")
sqlResourceInsertFromHistory = mustTemplate("resource_insert_from_history.sql")
// sqlResourceLabelsInsert = mustTemplate("resource_labels_insert.sql")
sqlResourceVersionGet = mustTemplate("resource_version_get.sql")
@@ -83,6 +84,18 @@ func (r sqlResourceRequest) Validate() error {
return nil // TODO
}
type sqlResourceInsertFromHistoryRequest struct {
sqltemplate.SQLTemplate
Key *resource.ResourceKey
}
func (r sqlResourceInsertFromHistoryRequest) Validate() error {
if r.Key == nil {
return fmt.Errorf("missing key")
}
return nil
}
type sqlStatsRequest struct {
sqltemplate.SQLTemplate
Namespace string
pkg/storage/unified/sql/queries_test.go
+13
View File
@@ -385,5 +385,18 @@ func TestUnifiedStorageQueries(t *testing.T) {
},
},
},
sqlResourceInsertFromHistory: {
{
Name: "update",
Data: &sqlResourceInsertFromHistoryRequest{
SQLTemplate: mocks.NewTestingSQLTemplate(),
Key: &resource.ResourceKey{
Namespace: "default",
Group: "dashboard.grafana.app",
Resource: "dashboards",
},
},
},
},
}})
}
pkg/storage/unified/sql/service.go
+2
View File
@@ -126,7 +126,9 @@ func (s *service) start(ctx context.Context) error {
srv := s.handler.GetServer()
resource.RegisterResourceStoreServer(srv, server)
resource.RegisterBatchStoreServer(srv, server)
resource.RegisterResourceIndexServer(srv, server)
resource.RegisterRepositoryIndexServer(srv, server)
resource.RegisterBlobStoreServer(srv, server)
resource.RegisterDiagnosticsServer(srv, server)
grpc_health_v1.RegisterHealthServer(srv, healthService)
pkg/storage/unified/sql/testdata/mysql--resource_insert_from_history-update.sql
Vendored Executable
+34
View File
@@ -0,0 +1,34 @@
INSERT INTO `resource`
SELECT
kv.`guid`,
kv.`resource_version`,
kv.`group`,
kv.`resource`,
kv.`namespace`,
kv.`name`,
kv.`value`,
kv.`action`,
kv.`label_set`,
kv.`previous_resource_version`,
kv.`folder`
FROM `resource_history` AS kv
INNER JOIN (
SELECT `namespace`, `group`, `resource`, `name`, max(`resource_version`) AS `resource_version`
FROM `resource_history` AS mkv
WHERE 1 = 1
AND `namespace` = 'default'
AND `group` = 'dashboard.grafana.app'
AND `resource` = 'dashboards'
GROUP BY mkv.`namespace`, mkv.`group`, mkv.`resource`, mkv.`name`
) AS maxkv
ON maxkv.`resource_version` = kv.`resource_version`
AND maxkv.`namespace` = kv.`namespace`
AND maxkv.`group` = kv.`group`
AND maxkv.`resource` = kv.`resource`
AND maxkv.`name` = kv.`name`
WHERE kv.`action` != 3
AND kv.`namespace` = 'default'
AND kv.`group` = 'dashboard.grafana.app'
AND kv.`resource` = 'dashboards'
ORDER BY kv.`resource_version` ASC
;
pkg/storage/unified/sql/testdata/postgres--resource_insert_from_history-update.sql
Vendored Executable
+34
View File
@@ -0,0 +1,34 @@
INSERT INTO "resource"
SELECT
kv."guid",
kv."resource_version",
kv."group",
kv."resource",
kv."namespace",
kv."name",
kv."value",
kv."action",
kv."label_set",
kv."previous_resource_version",
kv."folder"
FROM "resource_history" AS kv
INNER JOIN (
SELECT "namespace", "group", "resource", "name", max("resource_version") AS "resource_version"
FROM "resource_history" AS mkv
WHERE 1 = 1
AND "namespace" = 'default'
AND "group" = 'dashboard.grafana.app'
AND "resource" = 'dashboards'
GROUP BY mkv."namespace", mkv."group", mkv."resource", mkv."name"
) AS maxkv
ON maxkv."resource_version" = kv."resource_version"
AND maxkv."namespace" = kv."namespace"
AND maxkv."group" = kv."group"
AND maxkv."resource" = kv."resource"
AND maxkv."name" = kv."name"
WHERE kv."action" != 3
AND kv."namespace" = 'default'
AND kv."group" = 'dashboard.grafana.app'
AND kv."resource" = 'dashboards'
ORDER BY kv."resource_version" ASC
;
pkg/storage/unified/sql/testdata/sqlite--resource_insert_from_history-update.sql
Vendored Executable
+34
View File
@@ -0,0 +1,34 @@
INSERT INTO "resource"
SELECT
kv."guid",
kv."resource_version",
kv."group",
kv."resource",
kv."namespace",
kv."name",
kv."value",
kv."action",
kv."label_set",
kv."previous_resource_version",
kv."folder"
FROM "resource_history" AS kv
INNER JOIN (
SELECT "namespace", "group", "resource", "name", max("resource_version") AS "resource_version"
FROM "resource_history" AS mkv
WHERE 1 = 1
AND "namespace" = 'default'
AND "group" = 'dashboard.grafana.app'
AND "resource" = 'dashboards'
GROUP BY mkv."namespace", mkv."group", mkv."resource", mkv."name"
) AS maxkv
ON maxkv."resource_version" = kv."resource_version"
AND maxkv."namespace" = kv."namespace"
AND maxkv."group" = kv."group"
AND maxkv."resource" = kv."resource"
AND maxkv."name" = kv."name"
WHERE kv."action" != 3
AND kv."namespace" = 'default'
AND kv."group" = 'dashboard.grafana.app'
AND kv."resource" = 'dashboards'
ORDER BY kv."resource_version" ASC
;
public/app/features/gops/configuration-tracker/irmHooks.ts
+2 -2
View File
@@ -235,7 +235,7 @@ export function useGetEssentialsConfiguration(): EssentialsConfigurationData {
},
{
title: 'Respond',
description: 'Configure OnCall and Incident',
description: getIsIrmPluginPresent() ? 'Configure IRM' : 'Configure OnCall and Incident',
steps: getIsIrmPluginPresent()
? [
{
@@ -261,7 +261,7 @@ export function useGetEssentialsConfiguration(): EssentialsConfigurationData {
button: {
type: 'openLink',
urlLink: {
url: `/a/${getIrmIfPresentOrIncidentPluginId()}/integrations/grate.slack`,
url: `/a/${getIrmIfPresentOrIncidentPluginId()}/integrations/apps/grate.irm.slack`,
},
label: 'Connect',
urlLinkOnDone: {
yarn.lock
+294 -287
View File
@@ -2067,13 +2067,6 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/aix-ppc64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/aix-ppc64@npm:0.21.5"
conditions: os=aix & cpu=ppc64
languageName: node
linkType: hard
"@esbuild/aix-ppc64@npm:0.24.2":
version: 0.24.2
resolution: "@esbuild/aix-ppc64@npm:0.24.2"
@@ -2081,10 +2074,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/android-arm64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/android-arm64@npm:0.21.5"
conditions: os=android & cpu=arm64
"@esbuild/aix-ppc64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/aix-ppc64@npm:0.25.0"
conditions: os=aix & cpu=ppc64
languageName: node
linkType: hard
@@ -2095,10 +2088,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/android-arm@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/android-arm@npm:0.21.5"
conditions: os=android & cpu=arm
"@esbuild/android-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/android-arm64@npm:0.25.0"
conditions: os=android & cpu=arm64
languageName: node
linkType: hard
@@ -2109,10 +2102,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/android-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/android-x64@npm:0.21.5"
conditions: os=android & cpu=x64
"@esbuild/android-arm@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/android-arm@npm:0.25.0"
conditions: os=android & cpu=arm
languageName: node
linkType: hard
@@ -2123,10 +2116,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/darwin-arm64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/darwin-arm64@npm:0.21.5"
conditions: os=darwin & cpu=arm64
"@esbuild/android-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/android-x64@npm:0.25.0"
conditions: os=android & cpu=x64
languageName: node
linkType: hard
@@ -2137,10 +2130,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/darwin-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/darwin-x64@npm:0.21.5"
conditions: os=darwin & cpu=x64
"@esbuild/darwin-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/darwin-arm64@npm:0.25.0"
conditions: os=darwin & cpu=arm64
languageName: node
linkType: hard
@@ -2151,10 +2144,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/freebsd-arm64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/freebsd-arm64@npm:0.21.5"
conditions: os=freebsd & cpu=arm64
"@esbuild/darwin-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/darwin-x64@npm:0.25.0"
conditions: os=darwin & cpu=x64
languageName: node
linkType: hard
@@ -2165,10 +2158,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/freebsd-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/freebsd-x64@npm:0.21.5"
conditions: os=freebsd & cpu=x64
"@esbuild/freebsd-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/freebsd-arm64@npm:0.25.0"
conditions: os=freebsd & cpu=arm64
languageName: node
linkType: hard
@@ -2179,10 +2172,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-arm64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-arm64@npm:0.21.5"
conditions: os=linux & cpu=arm64
"@esbuild/freebsd-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/freebsd-x64@npm:0.25.0"
conditions: os=freebsd & cpu=x64
languageName: node
linkType: hard
@@ -2193,10 +2186,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-arm@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-arm@npm:0.21.5"
conditions: os=linux & cpu=arm
"@esbuild/linux-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-arm64@npm:0.25.0"
conditions: os=linux & cpu=arm64
languageName: node
linkType: hard
@@ -2207,10 +2200,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-ia32@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-ia32@npm:0.21.5"
conditions: os=linux & cpu=ia32
"@esbuild/linux-arm@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-arm@npm:0.25.0"
conditions: os=linux & cpu=arm
languageName: node
linkType: hard
@@ -2221,10 +2214,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-loong64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-loong64@npm:0.21.5"
conditions: os=linux & cpu=loong64
"@esbuild/linux-ia32@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-ia32@npm:0.25.0"
conditions: os=linux & cpu=ia32
languageName: node
linkType: hard
@@ -2235,10 +2228,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-mips64el@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-mips64el@npm:0.21.5"
conditions: os=linux & cpu=mips64el
"@esbuild/linux-loong64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-loong64@npm:0.25.0"
conditions: os=linux & cpu=loong64
languageName: node
linkType: hard
@@ -2249,10 +2242,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-ppc64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-ppc64@npm:0.21.5"
conditions: os=linux & cpu=ppc64
"@esbuild/linux-mips64el@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-mips64el@npm:0.25.0"
conditions: os=linux & cpu=mips64el
languageName: node
linkType: hard
@@ -2263,10 +2256,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-riscv64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-riscv64@npm:0.21.5"
conditions: os=linux & cpu=riscv64
"@esbuild/linux-ppc64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-ppc64@npm:0.25.0"
conditions: os=linux & cpu=ppc64
languageName: node
linkType: hard
@@ -2277,10 +2270,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-s390x@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-s390x@npm:0.21.5"
conditions: os=linux & cpu=s390x
"@esbuild/linux-riscv64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-riscv64@npm:0.25.0"
conditions: os=linux & cpu=riscv64
languageName: node
linkType: hard
@@ -2291,10 +2284,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/linux-x64@npm:0.21.5"
conditions: os=linux & cpu=x64
"@esbuild/linux-s390x@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-s390x@npm:0.25.0"
conditions: os=linux & cpu=s390x
languageName: node
linkType: hard
@@ -2305,6 +2298,13 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/linux-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/linux-x64@npm:0.25.0"
conditions: os=linux & cpu=x64
languageName: node
linkType: hard
"@esbuild/netbsd-arm64@npm:0.24.2":
version: 0.24.2
resolution: "@esbuild/netbsd-arm64@npm:0.24.2"
@@ -2312,10 +2312,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/netbsd-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/netbsd-x64@npm:0.21.5"
conditions: os=netbsd & cpu=x64
"@esbuild/netbsd-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/netbsd-arm64@npm:0.25.0"
conditions: os=netbsd & cpu=arm64
languageName: node
linkType: hard
@@ -2326,6 +2326,13 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/netbsd-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/netbsd-x64@npm:0.25.0"
conditions: os=netbsd & cpu=x64
languageName: node
linkType: hard
"@esbuild/openbsd-arm64@npm:0.24.2":
version: 0.24.2
resolution: "@esbuild/openbsd-arm64@npm:0.24.2"
@@ -2333,10 +2340,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/openbsd-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/openbsd-x64@npm:0.21.5"
conditions: os=openbsd & cpu=x64
"@esbuild/openbsd-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/openbsd-arm64@npm:0.25.0"
conditions: os=openbsd & cpu=arm64
languageName: node
linkType: hard
@@ -2347,10 +2354,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/sunos-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/sunos-x64@npm:0.21.5"
conditions: os=sunos & cpu=x64
"@esbuild/openbsd-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/openbsd-x64@npm:0.25.0"
conditions: os=openbsd & cpu=x64
languageName: node
linkType: hard
@@ -2361,10 +2368,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/win32-arm64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/win32-arm64@npm:0.21.5"
conditions: os=win32 & cpu=arm64
"@esbuild/sunos-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/sunos-x64@npm:0.25.0"
conditions: os=sunos & cpu=x64
languageName: node
linkType: hard
@@ -2375,10 +2382,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/win32-ia32@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/win32-ia32@npm:0.21.5"
conditions: os=win32 & cpu=ia32
"@esbuild/win32-arm64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/win32-arm64@npm:0.25.0"
conditions: os=win32 & cpu=arm64
languageName: node
linkType: hard
@@ -2389,10 +2396,10 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/win32-x64@npm:0.21.5":
version: 0.21.5
resolution: "@esbuild/win32-x64@npm:0.21.5"
conditions: os=win32 & cpu=x64
"@esbuild/win32-ia32@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/win32-ia32@npm:0.25.0"
conditions: os=win32 & cpu=ia32
languageName: node
linkType: hard
@@ -2403,6 +2410,13 @@ __metadata:
languageName: node
linkType: hard
"@esbuild/win32-x64@npm:0.25.0":
version: 0.25.0
resolution: "@esbuild/win32-x64@npm:0.25.0"
conditions: os=win32 & cpu=x64
languageName: node
linkType: hard
"@eslint-community/eslint-utils@npm:^4.2.0, @eslint-community/eslint-utils@npm:^4.4.0":
version: 4.4.0
resolution: "@eslint-community/eslint-utils@npm:4.4.0"
@@ -3214,7 +3228,7 @@ __metadata:
d3-interpolate: "npm:3.0.1"
date-fns: "npm:4.1.0"
dompurify: "npm:3.2.4"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
eventemitter3: "npm:5.0.1"
fast_array_intersect: "npm:1.1.0"
history: "npm:4.10.1"
@@ -3231,7 +3245,7 @@ __metadata:
rimraf: "npm:6.0.1"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
rxjs: "npm:7.8.1"
string-hash: "npm:^1.1.3"
@@ -3265,11 +3279,11 @@ __metadata:
"@rollup/plugin-node-resolve": "npm:16.0.0"
"@types/node": "npm:22.12.0"
"@types/semver": "npm:7.5.8"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
rimraf: "npm:6.0.1"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
semver: "npm:7.7.0"
tslib: "npm:2.8.1"
@@ -3311,27 +3325,7 @@ __metadata:
languageName: unknown
linkType: soft
"@grafana/faro-core@npm:^1.12.3":
version: 1.12.3
resolution: "@grafana/faro-core@npm:1.12.3"
dependencies:
"@opentelemetry/api": "npm:^1.9.0"
"@opentelemetry/otlp-transformer": "npm:^0.57.1"
checksum: 10/616592a8e65c72be763f27baa3a44c228fc61aeff9cd6c895dac7b4adc24a836accc4766d574a3b5156896b3068c8d7df51c1d715679f3f1a1feef90dcaf82e3
languageName: node
linkType: hard
"@grafana/faro-core@npm:^1.13.1":
version: 1.13.1
resolution: "@grafana/faro-core@npm:1.13.1"
dependencies:
"@opentelemetry/api": "npm:^1.9.0"
"@opentelemetry/otlp-transformer": "npm:^0.57.1"
checksum: 10/ce3747c476bd2b0f2b07c808c584953de75bf970fc17fbeb27258decf86919a753472cc464a4dccd4e0091003232feec6d3ba07f2e1b7bc38b487d9199abd18f
languageName: node
linkType: hard
"@grafana/faro-core@npm:^1.13.2":
"@grafana/faro-core@npm:^1.13.1, @grafana/faro-core@npm:^1.13.2":
version: 1.13.2
resolution: "@grafana/faro-core@npm:1.13.2"
dependencies:
@@ -3341,7 +3335,7 @@ __metadata:
languageName: node
linkType: hard
"@grafana/faro-web-sdk@npm:^1.13.2":
"@grafana/faro-web-sdk@npm:^1.13.2, @grafana/faro-web-sdk@npm:^1.3.6":
version: 1.13.2
resolution: "@grafana/faro-web-sdk@npm:1.13.2"
dependencies:
@@ -3352,17 +3346,6 @@ __metadata:
languageName: node
linkType: hard
"@grafana/faro-web-sdk@npm:^1.3.6":
version: 1.12.3
resolution: "@grafana/faro-web-sdk@npm:1.12.3"
dependencies:
"@grafana/faro-core": "npm:^1.12.3"
ua-parser-js: "npm:^1.0.32"
web-vitals: "npm:^4.0.1"
checksum: 10/433b118ff5b8bc231e209902c20b8a645efe6efd9ecc4ef57947111fa076842becc9760b5bcc8957c1f04cd20c801241ba9cca0a6d4637b8aa2b034d7a7c1888
languageName: node
linkType: hard
"@grafana/faro-web-tracing@npm:^1.13.2":
version: 1.13.2
resolution: "@grafana/faro-web-tracing@npm:1.13.2"
@@ -3409,7 +3392,7 @@ __metadata:
"@types/tinycolor2": "npm:1.4.6"
babel-jest: "npm:29.7.0"
d3: "npm:^7.8.5"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
jest: "npm:^29.6.4"
jest-canvas-mock: "npm:2.5.2"
lodash: "npm:4.17.21"
@@ -3418,7 +3401,7 @@ __metadata:
react-virtualized-auto-sizer: "npm:1.0.25"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
tinycolor2: "npm:1.6.0"
ts-jest: "npm:29.2.5"
@@ -3676,7 +3659,7 @@ __metadata:
d3: "npm:7.9.0"
date-fns: "npm:4.1.0"
debounce-promise: "npm:3.1.2"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
eslint: "npm:9.19.0"
eslint-config-prettier: "npm:9.1.0"
eslint-plugin-import: "npm:^2.31.0"
@@ -3712,7 +3695,7 @@ __metadata:
react-window: "npm:1.8.11"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
rxjs: "npm:7.8.1"
sass: "npm:1.83.4"
@@ -3755,7 +3738,7 @@ __metadata:
"@types/react": "npm:18.3.18"
"@types/react-dom": "npm:18.3.5"
"@types/systemjs": "npm:6.15.1"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
history: "npm:4.10.1"
lodash: "npm:4.17.21"
react: "npm:18.3.1"
@@ -3764,7 +3747,7 @@ __metadata:
rimraf: "npm:6.0.1"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
rollup-plugin-sourcemaps: "npm:0.6.3"
rxjs: "npm:7.8.1"
@@ -3814,14 +3797,14 @@ __metadata:
"@types/node": "npm:22.12.0"
"@types/react": "npm:18.3.18"
"@types/react-dom": "npm:18.3.5"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
prettier: "npm:3.4.2"
react: "npm:18.3.1"
react-dom: "npm:18.3.1"
rimraf: "npm:6.0.1"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:8.0.0"
ts-node: "npm:10.9.2"
typescript: "npm:5.7.3"
@@ -3888,12 +3871,12 @@ __metadata:
dependencies:
"@grafana/tsconfig": "npm:^2.0.0"
"@rollup/plugin-node-resolve": "npm:16.0.0"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
glob: "npm:^11.0.0"
rimraf: "npm:6.0.1"
rollup: "npm:^4.22.4"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
tslib: "npm:2.8.1"
typescript: "npm:5.7.3"
@@ -4106,7 +4089,7 @@ __metadata:
d3: "npm:7.9.0"
date-fns: "npm:4.1.0"
downshift: "npm:^9.0.6"
esbuild: "npm:0.24.2"
esbuild: "npm:0.25.0"
expose-loader: "npm:5.0.0"
fs-extra: "npm:^11.2.0"
hoist-non-react-statics: "npm:3.3.2"
@@ -4151,7 +4134,7 @@ __metadata:
rollup: "npm:^4.22.4"
rollup-plugin-copy: "npm:3.5.0"
rollup-plugin-dts: "npm:^6.1.1"
rollup-plugin-esbuild: "npm:6.1.1"
rollup-plugin-esbuild: "npm:6.2.0"
rollup-plugin-node-externals: "npm:^8.0.0"
rollup-plugin-svg-import: "npm:3.0.0"
rxjs: "npm:7.8.1"
@@ -7167,7 +7150,7 @@ __metadata:
languageName: node
linkType: hard
"@rollup/pluginutils@npm:^5.0.1, @rollup/pluginutils@npm:^5.0.5, @rollup/pluginutils@npm:^5.1.0":
"@rollup/pluginutils@npm:^5.0.1, @rollup/pluginutils@npm:^5.1.0":
version: 5.1.2
resolution: "@rollup/pluginutils@npm:5.1.2"
dependencies:
@@ -14923,15 +14906,15 @@ __metadata:
languageName: node
linkType: hard
"debug@npm:4, debug@npm:^4.1.0, debug@npm:^4.1.1, debug@npm:^4.3.1, debug@npm:^4.3.2, debug@npm:^4.3.4, debug@npm:^4.3.5, debug@npm:^4.3.6, debug@npm:^4.3.7, debug@npm:~4.3.1, debug@npm:~4.3.2, debug@npm:~4.3.4":
version: 4.3.7
resolution: "debug@npm:4.3.7"
"debug@npm:4, debug@npm:^4.1.0, debug@npm:^4.1.1, debug@npm:^4.3.1, debug@npm:^4.3.2, debug@npm:^4.3.4, debug@npm:^4.3.5, debug@npm:^4.3.6, debug@npm:^4.3.7, debug@npm:^4.4.0":
version: 4.4.0
resolution: "debug@npm:4.4.0"
dependencies:
ms: "npm:^2.1.3"
peerDependenciesMeta:
supports-color:
optional: true
checksum: 10/71168908b9a78227ab29d5d25fe03c5867750e31ce24bf2c44a86efc5af041758bb56569b0a3d48a9b5344c00a24a777e6f4100ed6dfd9534a42c1dde285125a
checksum: 10/1847944c2e3c2c732514b93d11886575625686056cd765336212dc15de2d2b29612b6cd80e1afba767bb8e1803b778caf9973e98169ef1a24a7a7009e1820367
languageName: node
linkType: hard
@@ -14944,6 +14927,18 @@ __metadata:
languageName: node
linkType: hard
"debug@npm:~4.3.1, debug@npm:~4.3.2, debug@npm:~4.3.4":
version: 4.3.7
resolution: "debug@npm:4.3.7"
dependencies:
ms: "npm:^2.1.3"
peerDependenciesMeta:
supports-color:
optional: true
checksum: 10/71168908b9a78227ab29d5d25fe03c5867750e31ce24bf2c44a86efc5af041758bb56569b0a3d48a9b5344c00a24a777e6f4100ed6dfd9534a42c1dde285125a
languageName: node
linkType: hard
"decamelize-keys@npm:^1.1.0":
version: 1.1.1
resolution: "decamelize-keys@npm:1.1.1"
@@ -14961,14 +14956,7 @@ __metadata:
languageName: node
linkType: hard
"decimal.js@npm:10":
version: 10.4.3
resolution: "decimal.js@npm:10.4.3"
checksum: 10/de663a7bc4d368e3877db95fcd5c87b965569b58d16cdc4258c063d231ca7118748738df17cd638f7e9dd0be8e34cec08d7234b20f1f2a756a52fc5a38b188d0
languageName: node
linkType: hard
"decimal.js@npm:^10.4.2":
"decimal.js@npm:10, decimal.js@npm:^10.4.2":
version: 10.5.0
resolution: "decimal.js@npm:10.5.0"
checksum: 10/714d49cf2f2207b268221795ede330e51452b7c451a0c02a770837d2d4faed47d603a729c2aa1d952eb6c4102d999e91c9b952c1aa016db3c5cba9fc8bf4cda2
@@ -15901,10 +15889,10 @@ __metadata:
languageName: node
linkType: hard
"es-module-lexer@npm:^1.2.1, es-module-lexer@npm:^1.3.1, es-module-lexer@npm:^1.5.0, es-module-lexer@npm:^1.5.3":
version: 1.5.4
resolution: "es-module-lexer@npm:1.5.4"
checksum: 10/f29c7c97a58eb17640dcbd71bd6ef754ad4f58f95c3073894573d29dae2cad43ecd2060d97ed5b866dfb7804d5590fb7de1d2c5339a5fceae8bd60b580387fc5
"es-module-lexer@npm:^1.2.1, es-module-lexer@npm:^1.5.0, es-module-lexer@npm:^1.5.3, es-module-lexer@npm:^1.6.0":
version: 1.6.0
resolution: "es-module-lexer@npm:1.6.0"
checksum: 10/807ee7020cc46a9c970c78cad1f2f3fc139877e5ebad7f66dbfbb124d451189ba1c48c1c632bd5f8ce1b8af2caef3fca340ba044a410fa890d17b080a59024bb
languageName: node
linkType: hard
@@ -15975,30 +15963,30 @@ __metadata:
languageName: node
linkType: hard
"esbuild-loader@npm:4.2.2":
version: 4.2.2
resolution: "esbuild-loader@npm:4.2.2"
"esbuild-loader@npm:4.3.0":
version: 4.3.0
resolution: "esbuild-loader@npm:4.3.0"
dependencies:
esbuild: "npm:^0.21.0"
esbuild: "npm:^0.25.0"
get-tsconfig: "npm:^4.7.0"
loader-utils: "npm:^2.0.4"
webpack-sources: "npm:^1.4.3"
peerDependencies:
webpack: ^4.40.0 || ^5.0.0
checksum: 10/235d06c60e26827333c3c66df6e2f13fef31e1b84b8310ba36ebdb6aaa31abdf7628a940326e4923824cdb501c8ce6a6e9129c4e465daaeab112f0b5db3423f4
checksum: 10/451bf9b344419870b6a10a9d80c1e07ccb4e2871867c8e4dbdf7b491171f0de31571933a8a96627034b360627a034e6f43edd004b1f3e3fe291100ded5985fa6
languageName: node
linkType: hard
"esbuild-plugin-browserslist@npm:^0.15.0":
version: 0.15.0
resolution: "esbuild-plugin-browserslist@npm:0.15.0"
"esbuild-plugin-browserslist@npm:^0.16.0":
version: 0.16.0
resolution: "esbuild-plugin-browserslist@npm:0.16.0"
dependencies:
debug: "npm:^4.3.7"
zod: "npm:^3.23.8"
debug: "npm:^4.4.0"
zod: "npm:^3.24.1"
peerDependencies:
browserslist: ^4.21.8
esbuild: ~0.24.0
checksum: 10/b1afe26f5c013a37664ff6ca2f8190d50e36f719afca8aba2c0b18f0fd0f34de3334db1435154a70dbfeb5e91acf732cc20296bb3fd3cbdfe68ae37dadcffe32
esbuild: ~0.25.0
checksum: 10/9f3986901521627270a74500a77a3ddf3599016729de6ebe04e675230188a0e5dd3f5c6a5a1ab781a0b9cb54952df0edb066a3a5737166ef8e8d9db4a2eb44ab
languageName: node
linkType: hard
@@ -16013,7 +16001,93 @@ __metadata:
languageName: node
linkType: hard
"esbuild@npm:0.24.2, esbuild@npm:^0.18.0 || ^0.19.0 || ^0.20.0 || ^0.21.0 || ^0.22.0 || ^0.23.0 || ^0.24.0, esbuild@npm:^0.24.0":
"esbuild@npm:0.25.0, esbuild@npm:^0.25.0":
version: 0.25.0
resolution: "esbuild@npm:0.25.0"
dependencies:
"@esbuild/aix-ppc64": "npm:0.25.0"
"@esbuild/android-arm": "npm:0.25.0"
"@esbuild/android-arm64": "npm:0.25.0"
"@esbuild/android-x64": "npm:0.25.0"
"@esbuild/darwin-arm64": "npm:0.25.0"
"@esbuild/darwin-x64": "npm:0.25.0"
"@esbuild/freebsd-arm64": "npm:0.25.0"
"@esbuild/freebsd-x64": "npm:0.25.0"
"@esbuild/linux-arm": "npm:0.25.0"
"@esbuild/linux-arm64": "npm:0.25.0"
"@esbuild/linux-ia32": "npm:0.25.0"
"@esbuild/linux-loong64": "npm:0.25.0"
"@esbuild/linux-mips64el": "npm:0.25.0"
"@esbuild/linux-ppc64": "npm:0.25.0"
"@esbuild/linux-riscv64": "npm:0.25.0"
"@esbuild/linux-s390x": "npm:0.25.0"
"@esbuild/linux-x64": "npm:0.25.0"
"@esbuild/netbsd-arm64": "npm:0.25.0"
"@esbuild/netbsd-x64": "npm:0.25.0"
"@esbuild/openbsd-arm64": "npm:0.25.0"
"@esbuild/openbsd-x64": "npm:0.25.0"
"@esbuild/sunos-x64": "npm:0.25.0"
"@esbuild/win32-arm64": "npm:0.25.0"
"@esbuild/win32-ia32": "npm:0.25.0"
"@esbuild/win32-x64": "npm:0.25.0"
dependenciesMeta:
"@esbuild/aix-ppc64":
optional: true
"@esbuild/android-arm":
optional: true
"@esbuild/android-arm64":
optional: true
"@esbuild/android-x64":
optional: true
"@esbuild/darwin-arm64":
optional: true
"@esbuild/darwin-x64":
optional: true
"@esbuild/freebsd-arm64":
optional: true
"@esbuild/freebsd-x64":
optional: true
"@esbuild/linux-arm":
optional: true
"@esbuild/linux-arm64":
optional: true
"@esbuild/linux-ia32":
optional: true
"@esbuild/linux-loong64":
optional: true
"@esbuild/linux-mips64el":
optional: true
"@esbuild/linux-ppc64":
optional: true
"@esbuild/linux-riscv64":
optional: true
"@esbuild/linux-s390x":
optional: true
"@esbuild/linux-x64":
optional: true
"@esbuild/netbsd-arm64":
optional: true
"@esbuild/netbsd-x64":
optional: true
"@esbuild/openbsd-arm64":
optional: true
"@esbuild/openbsd-x64":
optional: true
"@esbuild/sunos-x64":
optional: true
"@esbuild/win32-arm64":
optional: true
"@esbuild/win32-ia32":
optional: true
"@esbuild/win32-x64":
optional: true
bin:
esbuild: bin/esbuild
checksum: 10/451daf6a442df29ec5d528587caa4ce783d41ff4acb93252da5a852b8d36c22e9f84d17f6721d4fbef9a1ba9855bc9fe1f167dd732c11665fe53032f2b89f114
languageName: node
linkType: hard
"esbuild@npm:^0.18.0 || ^0.19.0 || ^0.20.0 || ^0.21.0 || ^0.22.0 || ^0.23.0 || ^0.24.0, esbuild@npm:^0.24.0":
version: 0.24.2
resolution: "esbuild@npm:0.24.2"
dependencies:
@@ -16099,86 +16173,6 @@ __metadata:
languageName: node
linkType: hard
"esbuild@npm:^0.21.0":
version: 0.21.5
resolution: "esbuild@npm:0.21.5"
dependencies:
"@esbuild/aix-ppc64": "npm:0.21.5"
"@esbuild/android-arm": "npm:0.21.5"
"@esbuild/android-arm64": "npm:0.21.5"
"@esbuild/android-x64": "npm:0.21.5"
"@esbuild/darwin-arm64": "npm:0.21.5"
"@esbuild/darwin-x64": "npm:0.21.5"
"@esbuild/freebsd-arm64": "npm:0.21.5"
"@esbuild/freebsd-x64": "npm:0.21.5"
"@esbuild/linux-arm": "npm:0.21.5"
"@esbuild/linux-arm64": "npm:0.21.5"
"@esbuild/linux-ia32": "npm:0.21.5"
"@esbuild/linux-loong64": "npm:0.21.5"
"@esbuild/linux-mips64el": "npm:0.21.5"
"@esbuild/linux-ppc64": "npm:0.21.5"
"@esbuild/linux-riscv64": "npm:0.21.5"
"@esbuild/linux-s390x": "npm:0.21.5"
"@esbuild/linux-x64": "npm:0.21.5"
"@esbuild/netbsd-x64": "npm:0.21.5"
"@esbuild/openbsd-x64": "npm:0.21.5"
"@esbuild/sunos-x64": "npm:0.21.5"
"@esbuild/win32-arm64": "npm:0.21.5"
"@esbuild/win32-ia32": "npm:0.21.5"
"@esbuild/win32-x64": "npm:0.21.5"
dependenciesMeta:
"@esbuild/aix-ppc64":
optional: true
"@esbuild/android-arm":
optional: true
"@esbuild/android-arm64":
optional: true
"@esbuild/android-x64":
optional: true
"@esbuild/darwin-arm64":
optional: true
"@esbuild/darwin-x64":
optional: true
"@esbuild/freebsd-arm64":
optional: true
"@esbuild/freebsd-x64":
optional: true
"@esbuild/linux-arm":
optional: true
"@esbuild/linux-arm64":
optional: true
"@esbuild/linux-ia32":
optional: true
"@esbuild/linux-loong64":
optional: true
"@esbuild/linux-mips64el":
optional: true
"@esbuild/linux-ppc64":
optional: true
"@esbuild/linux-riscv64":
optional: true
"@esbuild/linux-s390x":
optional: true
"@esbuild/linux-x64":
optional: true
"@esbuild/netbsd-x64":
optional: true
"@esbuild/openbsd-x64":
optional: true
"@esbuild/sunos-x64":
optional: true
"@esbuild/win32-arm64":
optional: true
"@esbuild/win32-ia32":
optional: true
"@esbuild/win32-x64":
optional: true
bin:
esbuild: bin/esbuild
checksum: 10/d2ff2ca84d30cce8e871517374d6c2290835380dc7cd413b2d49189ed170d45e407be14de2cb4794cf76f75cf89955c4714726ebd3de7444b3046f5cab23ab6b
languageName: node
linkType: hard
"escalade@npm:^3.1.1, escalade@npm:^3.2.0":
version: 3.2.0
resolution: "escalade@npm:3.2.0"
@@ -17381,18 +17375,7 @@ __metadata:
languageName: node
linkType: hard
"form-data@npm:^4.0.0":
version: 4.0.0
resolution: "form-data@npm:4.0.0"
dependencies:
asynckit: "npm:^0.4.0"
combined-stream: "npm:^1.0.8"
mime-types: "npm:^2.1.12"
checksum: 10/7264aa760a8cf09482816d8300f1b6e2423de1b02bba612a136857413fdc96d7178298ced106817655facc6b89036c6e12ae31c9eb5bdc16aabf502ae8a5d805
languageName: node
linkType: hard
"form-data@npm:~4.0.0":
"form-data@npm:^4.0.0, form-data@npm:~4.0.0":
version: 4.0.1
resolution: "form-data@npm:4.0.1"
dependencies:
@@ -17773,12 +17756,12 @@ __metadata:
languageName: node
linkType: hard
"get-tsconfig@npm:^4.7.0, get-tsconfig@npm:^4.7.2":
version: 4.8.1
resolution: "get-tsconfig@npm:4.8.1"
"get-tsconfig@npm:^4.10.0, get-tsconfig@npm:^4.7.0":
version: 4.10.0
resolution: "get-tsconfig@npm:4.10.0"
dependencies:
resolve-pkg-maps: "npm:^1.0.0"
checksum: 10/3fb5a8ad57b9633eaea085d81661e9e5c9f78b35d8f8689eaf8b8b45a2a3ebf3b3422266d4d7df765e308cc1e6231648d114803ab3d018332e29916f2c1de036
checksum: 10/5259b5c99a1957114337d9d0603b4a305ec9e29fa6cac7d2fbf634ba6754a0cc88bfd281a02416ce64e604b637d3cb239185381a79a5842b17fb55c097b38c4b
languageName: node
linkType: hard
@@ -18307,9 +18290,9 @@ __metadata:
date-fns: "npm:4.1.0"
debounce-promise: "npm:3.1.2"
diff: "npm:^7.0.0"
esbuild: "npm:0.24.2"
esbuild-loader: "npm:4.2.2"
esbuild-plugin-browserslist: "npm:^0.15.0"
esbuild: "npm:0.25.0"
esbuild-loader: "npm:4.3.0"
esbuild-plugin-browserslist: "npm:^0.16.0"
eslint: "npm:9.19.0"
eslint-config-prettier: "npm:9.1.0"
eslint-plugin-import: "npm:^2.31.0"
@@ -24543,6 +24526,13 @@ __metadata:
languageName: node
linkType: hard
"pathe@npm:^2.0.2":
version: 2.0.2
resolution: "pathe@npm:2.0.2"
checksum: 10/027dd246720ec6d3b5567e2b0201f1a815b6a69f2912a4dcafed59620afc729af15b4aff4bc780504c88d11dfb081c051e37327b928a093e714c3e09bf35aff3
languageName: node
linkType: hard
"pbf@npm:3.2.1":
version: 3.2.1
resolution: "pbf@npm:3.2.1"
@@ -24607,6 +24597,13 @@ __metadata:
languageName: node
linkType: hard
"picomatch@npm:^4.0.2":
version: 4.0.2
resolution: "picomatch@npm:4.0.2"
checksum: 10/ce617b8da36797d09c0baacb96ca8a44460452c89362d7cb8f70ca46b4158ba8bc3606912de7c818eb4a939f7f9015cef3c766ec8a0c6bfc725fdc078e39c717
languageName: node
linkType: hard
"pify@npm:5.0.0":
version: 5.0.0
resolution: "pify@npm:5.0.0"
@@ -25519,7 +25516,7 @@ __metadata:
languageName: node
linkType: hard
"qs@npm:6.13.0, qs@npm:^6.11.2, qs@npm:^6.4.0":
"qs@npm:6.13.0":
version: 6.13.0
resolution: "qs@npm:6.13.0"
dependencies:
@@ -25528,7 +25525,7 @@ __metadata:
languageName: node
linkType: hard
"qs@npm:6.13.1":
"qs@npm:6.13.1, qs@npm:^6.11.2, qs@npm:^6.4.0":
version: 6.13.1
resolution: "qs@npm:6.13.1"
dependencies:
@@ -27867,18 +27864,18 @@ __metadata:
languageName: node
linkType: hard
"rollup-plugin-esbuild@npm:6.1.1":
version: 6.1.1
resolution: "rollup-plugin-esbuild@npm:6.1.1"
"rollup-plugin-esbuild@npm:6.2.0":
version: 6.2.0
resolution: "rollup-plugin-esbuild@npm:6.2.0"
dependencies:
"@rollup/pluginutils": "npm:^5.0.5"
debug: "npm:^4.3.4"
es-module-lexer: "npm:^1.3.1"
get-tsconfig: "npm:^4.7.2"
debug: "npm:^4.4.0"
es-module-lexer: "npm:^1.6.0"
get-tsconfig: "npm:^4.10.0"
unplugin-utils: "npm:^0.2.3"
peerDependencies:
esbuild: ">=0.18.0"
rollup: ^1.20.0 || ^2.0.0 || ^3.0.0 || ^4.0.0
checksum: 10/bba2d1dfb92a193823ac9dd1cdd44a8fd8cd9f25868e9a22ca077e1b7445feb4eaaf6df051148e367fc902d7d59c9f50efab49086c24c367972f05c86f3a656d
checksum: 10/305563dfa72a843fd7db7b885583f22f62d4d584cbce7e8d80f138692ce67eb34e45dc1227b3420662f22fa75fa603ca09b64e7d1670919275a8577da653fada
languageName: node
linkType: hard
@@ -31129,6 +31126,16 @@ __metadata:
languageName: node
linkType: hard
"unplugin-utils@npm:^0.2.3":
version: 0.2.3
resolution: "unplugin-utils@npm:0.2.3"
dependencies:
pathe: "npm:^2.0.2"
picomatch: "npm:^4.0.2"
checksum: 10/95a18001b4a3aa92e58259162ee70be74f49780bff7aca0272035b206143772d07f3b117922ec9d7caaed8bd24578afcf7baa4330943796ea9f011dcc96af1d9
languageName: node
linkType: hard
"unplugin@npm:^1.3.1":
version: 1.5.0
resolution: "unplugin@npm:1.5.0"
@@ -32498,10 +32505,10 @@ __metadata:
languageName: node
linkType: hard
"zod@npm:^3.23.8":
version: 3.23.8
resolution: "zod@npm:3.23.8"
checksum: 10/846fd73e1af0def79c19d510ea9e4a795544a67d5b34b7e1c4d0425bf6bfd1c719446d94cdfa1721c1987d891321d61f779e8236fde517dc0e524aa851a6eff1
"zod@npm:^3.23.8, zod@npm:^3.24.1":
version: 3.24.1
resolution: "zod@npm:3.24.1"
checksum: 10/54e25956495dec22acb9399c168c6ba657ff279801a7fcd0530c414d867f1dcca279335e160af9b138dd70c332e17d548be4bc4d2f7eaf627dead50d914fec27
languageName: node
linkType: hard