public/grafana
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'v8.4.x' of github.com:grafana/grafana into v8.4.x

Browse Source
This commit is contained in:
Stephanie Closson
2022-02-23 13:53:00 -04:00
parent 53f5c6a44c 1c10ae4106
commit d98982bf02
60 changed files with 1321 additions and 538 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.drone.star
+1
View File
@@ -6,6 +6,7 @@
load('scripts/drone/pipelines/pr.star', 'pr_pipelines')
load('scripts/drone/pipelines/main.star', 'main_pipelines')
load('scripts/drone/pipelines/docs.star', 'docs_pipelines')
load('scripts/drone/pipelines/release.star', 'release_pipelines', 'publish_image_pipelines', 'publish_artifacts_pipelines', 'publish_npm_pipelines', 'publish_packages_pipeline')
load('scripts/drone/version.star', 'version_branch_pipelines')
load('scripts/drone/pipelines/cron.star', 'cronjobs')
.drone.yml
+308 -216
View File
File diff suppressed because it is too large Load Diff
CHANGELOG.md
+15
View File
@@ -1,3 +1,18 @@
<!-- 8.4.1 START -->
# 8.4.1 (2022-02-18)
### Features and enhancements
- **Cloudwatch:** Add support for AWS/PrivateLink\* metrics and dimensions. [#45515](https://github.com/grafana/grafana/pull/45515), [@szymonpk](https://github.com/szymonpk)
- **Configuration:** Add ability to customize okta login button name and icon. [#44079](https://github.com/grafana/grafana/pull/44079), [@DanCech](https://github.com/DanCech)
- **Tempo:** Switch out Select with AsyncSelect component to get loading state in Tempo Search. [#45110](https://github.com/grafana/grafana/pull/45110), [@CatPerry](https://github.com/CatPerry)
### Bug fixes
- **Alerting:** Fix migrations by making send_alerts_to field nullable. [#45572](https://github.com/grafana/grafana/pull/45572), [@santihernandezc](https://github.com/santihernandezc)
<!-- 8.4.1 END -->
<!-- 8.4.0 START -->
# 8.4.0 (2022-02-16)
Dockerfile
+1 -1
View File
@@ -20,7 +20,7 @@ COPY emails emails
ENV NODE_ENV production
RUN yarn build
FROM golang:1.17.6-alpine3.15 as go-builder
FROM golang:1.17.7-alpine3.15 as go-builder
RUN apk add --no-cache gcc g++ make
Dockerfile.ubuntu
+1 -1
View File
@@ -21,7 +21,7 @@ COPY emails emails
ENV NODE_ENV production
RUN yarn build
FROM golang:1.17.6 AS go-builder
FROM golang:1.17.7 AS go-builder
WORKDIR /src/grafana
babel.config.json
+3
View File
@@ -33,6 +33,9 @@
"allowDeclareFields": true
}
],
// added to mitigate https://github.com/babel/babel/issues/14289
// package (and following line) can be removed once the issue is fixed and released
"@babel/plugin-proposal-class-properties",
["@babel/plugin-proposal-object-rest-spread", { "loose": true }],
"@babel/plugin-transform-react-constant-elements",
"@babel/plugin-proposal-nullish-coalescing-operator",
conf/defaults.ini
+3
View File
@@ -400,6 +400,9 @@ oauth_auto_login = false
# OAuth state max age cookie duration in seconds. Defaults to 600 seconds.
oauth_state_cookie_max_age = 600
# Skip forced assignment of OrgID 1 or 'auto_assign_org_id' for social logins
oauth_skip_org_role_update_sync = false
# limit of api_key seconds to live before expiration
api_key_max_seconds_to_live = -1
conf/sample.ini
+3
View File
@@ -394,6 +394,9 @@
# OAuth state max age cookie duration in seconds. Defaults to 600 seconds.
;oauth_state_cookie_max_age = 600
# Skip forced assignment of OrgID 1 or 'auto_assign_org_id' for social logins
;oauth_skip_org_role_update_sync = false
# limit of api_key seconds to live before expiration
;api_key_max_seconds_to_live = -1
docs/sources/administration/configuration.md
+6
View File
@@ -751,6 +751,12 @@ This setting is ignored if multiple OAuth providers are configured. Default is `
How many seconds the OAuth state cookie lives before being deleted. Default is `600` (seconds)
Administrators can increase this if they experience OAuth login state mismatch errors.
### oauth_skip_org_role_update_sync
Skip forced assignment of OrgID `1` or `auto_assign_org_id` for external logins. Default is `false`.
Use this setting to distribute users with external login to multiple organizations.
Otherwise, the users' organization would get reset on every new login, for example, via AzureAD.
### api_key_max_seconds_to_live
Limit of API key seconds to live before expiration. Default is -1 (unlimited).
docs/sources/alerting/unified-alerting/notifications/mute-timings.md
+1 -1
View File
@@ -7,7 +7,7 @@ weight = 450
# Mute timings
A mute timing is a recurring interval of time when no new notifications for a policy are generated or sent. are sent for a policy. Use them to prevent alerts from firing a specific and reoccurring period, for example, a regular maintenance period.
A mute timing is a recurring interval of time when no new notifications for a policy are generated or sent. Use them to prevent alerts from firing a specific and reoccurring period, for example, a regular maintenance period.
Similar to silences, mute timings do not prevent alert rules from being evaluated, nor do they stop alert instances from being shown in the user interface. They only prevent notifications from being created.
docs/sources/datasources/alertmanager.md
+1 -1
View File
@@ -26,7 +26,7 @@ apiVersion: 1
datasources:
- name: Alertmanager
type: alertmanager
url: http://localhost:9090
url: http://localhost:9093
access: proxy
jsonData:
implementation: 'prometheus' # alternatively 'cortex'
docs/sources/datasources/jaeger.md
+10 -6
View File
@@ -32,6 +32,7 @@ This is a configuration for the [trace to logs feature]({{< relref "../explore/t
- **Data source -** Target data source.
- **Tags -** The tags that will be used in the Loki query. Default is `'cluster', 'hostname', 'namespace', 'pod'`.
- **Map tag names -** When enabled, allows configuring how Jaeger tag names map to Loki label names. For example, map `service.name` to `service`.
- **Span start time shift -** Shift in the start time for the Loki query based on the span start time. In order to extend to the past, you need to use a negative value. Use time interval units like 5s, 1m, 3h. The default is 0.
- **Span end time shift -** Shift in the end time for the Loki query based on the span end time. Time units can be used here, for example, 5s, 1m, 3h. The default is 0.
- **Filter by Trace ID -** Toggle to append the trace ID to the Loki query.
@@ -147,12 +148,15 @@ datasources:
tracesToLogs:
# Field with internal link pointing to a Loki data source in Grafana.
# datasourceUid value must match the `datasourceUid` value of the Loki data source.
datasourceUid: loki
tags:
- cluster
- hostname
- namespace
- pod
datasourceUid: 'loki'
tags: ['job', 'instance', 'pod', 'namespace']
mappedTags: [{ key: 'service.name', value: 'service' }]
mapTagNamesEnabled: false
spanStartTimeShift: '1h'
spanEndTimeShift: '1h'
filterByTraceID: false
filterBySpanID: false
lokiSearch: true
secureJsonData:
basicAuthPassword: my_password
```
docs/sources/datasources/tempo.md
+3
View File
@@ -31,6 +31,7 @@ This is a configuration for the [trace to logs feature]({{< relref "../explore/t
- **Data source -** Target data source.
- **Tags -** The tags that will be used in the Loki query. Default is `'cluster', 'hostname', 'namespace', 'pod'`.
- **Map tag names -** When enabled, allows configuring how Tempo tag names map to Loki label names. For example, map `service.name` to `service`.
- **Span start time shift -** A shift in the start time for the Loki query based on the start time for the span. To extend the time to the past, use a negative value. You can use time units, for example, 5s, 1m, 3h. The default is 0.
- **Span end time shift -** Shift in the end time for the Loki query based on the span end time. Time units can be used here, for example, 5s, 1m, 3h. The default is 0.
- **Filter by Trace ID -** Toggle to append the trace ID to the Loki query.
@@ -167,6 +168,8 @@ datasources:
tracesToLogs:
datasourceUid: 'loki'
tags: ['job', 'instance', 'pod', 'namespace']
mappedTags: [{ key: 'service.name', value: 'service' }]
mapTagNamesEnabled: false
spanStartTimeShift: '1h'
spanEndTimeShift: '1h'
filterByTraceID: false
docs/sources/datasources/zipkin.md
+1
View File
@@ -32,6 +32,7 @@ This is a configuration for the [trace to logs feature]({{< relref "../explore/t
- **Data source -** Target data source.
- **Tags -** The tags that will be used in the Loki query. Default is `'cluster', 'hostname', 'namespace', 'pod'`.
- **Map tag names -** When enabled, allows configuring how Zipkin tag names map to Loki label names. For example, map `service.name` to `service`.
- **Span start time shift -** Shift in the start time for the Loki query based on the span start time. In order to extend to the past, you need to use a negative value. Use time interval units like 5s, 1m, 3h. The default is 0.
- **Span end time shift -** Shift in the end time for the Loki query based on the span end time. Time units can be used here, for example, 5s, 1m, 3h. The default is 0.
- **Filter by Trace ID -** Toggle to append the trace ID to the Loki query.
docs/sources/developers/_index.md
+7 -1
View File
@@ -6,4 +6,10 @@ weight = 190
# Developers
This section of the documentation contains pages with resources for Grafana developers.
This section of the documentation contains pages with resources for Grafana developers:
- [Grafana Contribution Guide - learn how you can contribute your skills to make Grafana even better.]({{< relref "contribute.md" >}})
- [How to build a plugin]({{< relref "plugins" >}})
- [Grafana Tutorials - step-by-step guides that help you make the most of Grafana](https://grafana.com/tutorials/)
- [Grafana Community Forums - get technical support for open source Grafana, Loki, and Tempo.](https://community.grafana.com)
- [Grafana design system - library of reusable Grafana components and guidelines that help you with contribution and plugin development](https://developers.grafana.com)
docs/sources/enterprise/enterprise-encryption/_index.md
+4 -4
View File
@@ -16,10 +16,10 @@ You can choose to encrypt secrets stored in the Grafana database using a key fro
Grafana integrates with the following key management systems:
- [AWS KMS]({{< relref "/using-aws-kms-to-encrypt-database-secrets.md" >}})
- [Azure Key Vault]({{< relref "/using-azure-key-vault-to-encrypt-database-secrets.md" >}})
- [Google Cloud KMS]({{< relref "/using-google-cloud-kms-to-encrypt-database-secrets.md" >}})
- [Hashicorp Key Vault]({{< relref "/using-hashicorp-key-vault-to-encrypt-database-secrets.md" >}})
- [AWS KMS]({{< relref "using-aws-kms-to-encrypt-database-secrets.md" >}})
- [Azure Key Vault]({{< relref "using-azure-key-vault-to-encrypt-database-secrets.md" >}})
- [Google Cloud KMS]({{< relref "using-google-cloud-kms-to-encrypt-database-secrets.md" >}})
- [Hashicorp Key Vault]({{< relref "using-hashicorp-key-vault-to-encrypt-database-secrets.md" >}})
Refer to [Database encryption]({{< relref "../../administration/database-encryption.md" >}}) to learn more about how Grafana encrypts secrets in the database.
docs/sources/enterprise/enterprise-encryption/using-hashicorp-key-vault-to-encrypt-database-secrets.md
+2 -2
View File
@@ -5,9 +5,9 @@ keywords = ["grafana", "Hashicorp Vault integration"]
weight = 3
+++
# Using Google Cloud KMS to encrypt database secrets
# Using Hashicorp Vault to encrypt database secrets
You can use an encryption key from Google Cloud Key Management Service to encrypt secrets in the Grafana database.
You can use an encryption key from Hashicorp Vault to encrypt secrets in the Grafana database.
**Prerequisites:**
docs/sources/enterprise/query-caching.md
+13
View File
@@ -71,6 +71,19 @@ To disable query caching for a single data source:
To disable query caching for an entire Grafana instance, set the `enabled` flag to `false` in the [Query caching section of Enterprise Configuration]({{< relref "./enterprise-configuration.md#caching" >}}). You will no longer see the Cache tab on any data sources, and no data source queries will be cached.
## Clear cache
If you experience performance issues or repeated queries become slower to execute, consider clearing your cache.
> **Note:** This action impacts all cache-enabled data sources. If you are using Memcached, the system clears all data from the Memcached instance.
**To clear the cache**:
1. Sign in to Grafana and click **Settings > Data Sources**.
1. Select a data source.
1. Click the **Cache** tab.
1. Click **Clear cache**.
## Sending a request without cache
If a data source query request contains an `X-Cache-Skip` header, then Grafana skips the caching middleware, and does not search the cache for a response. This can be particularly useful when debugging data source queries using cURL.
docs/sources/http_api/folder_dashboard_search.md
+1 -1
View File
@@ -19,7 +19,7 @@ Query parameters:
- **dashboardIds** – List of dashboard id's to search for
- **folderIds** – List of folder id's to search in for dashboards
- **starred** – Flag indicating if only starred Dashboards should be returned
- **limit** – Limit the number of returned results (max 5000)
- **limit** – Limit the number of returned results (max is 5000; default is 1000)
- **page** – Use this parameter to access hits beyond limit. Numbering starts at 1. limit param acts as page size. Only available in Grafana v6.2+.
**Example request for retrieving folders and dashboards of the general folder**:
docs/sources/release-notes/_index.md
+1
View File
@@ -8,6 +8,7 @@ weight = 10000
Here you can find detailed release notes that list everything that is included in every release as well as notices
about deprecations, breaking changes as well as changes that relate to plugin development.
- [Release notes for 8.4.1]({{< relref "release-notes-8-4-1" >}})
- [Release notes for 8.4.0-beta1]({{< relref "release-notes-8-4-0-beta1" >}})
- [Release notes for 8.3.5]({{< relref "release-notes-8-3-5" >}})
- [Release notes for 8.3.4]({{< relref "release-notes-8-3-4" >}})
docs/sources/release-notes/release-notes-8-4-1.md
+18
View File
@@ -0,0 +1,18 @@
+++
title = "Release notes for Grafana 8.4.1"
hide_menu = true
+++
<!-- Auto generated by update changelog github action -->
# Release notes for Grafana 8.4.1
### Features and enhancements
- **Cloudwatch:** Add support for AWS/PrivateLink\* metrics and dimensions. [#45515](https://github.com/grafana/grafana/pull/45515), [@szymonpk](https://github.com/szymonpk)
- **Configuration:** Add ability to customize okta login button name and icon. [#44079](https://github.com/grafana/grafana/pull/44079), [@DanCech](https://github.com/DanCech)
- **Tempo:** Switch out Select with AsyncSelect component to get loading state in Tempo Search. [#45110](https://github.com/grafana/grafana/pull/45110), [@CatPerry](https://github.com/CatPerry)
### Bug fixes
- **Alerting:** Fix migrations by making send_alerts_to field nullable. [#45572](https://github.com/grafana/grafana/pull/45572), [@santihernandezc](https://github.com/santihernandezc)
package.json
+1
View File
@@ -69,6 +69,7 @@
},
"devDependencies": {
"@babel/core": "7.16.7",
"@babel/plugin-proposal-class-properties": "7.16.7",
"@babel/plugin-proposal-nullish-coalescing-operator": "7.16.7",
"@babel/plugin-proposal-object-rest-spread": "7.16.7",
"@babel/plugin-proposal-optional-chaining": "7.16.7",
packages/grafana-ui/src/components/Modal/Modal.test.tsx
+11 -14
View File
@@ -1,27 +1,24 @@
import React from 'react';
import { mount } from 'enzyme';
import { Modal } from './Modal';
import { render, screen } from '@testing-library/react';
describe('Modal', () => {
it('renders without error', () => {
mount(<Modal title={'Some Title'} isOpen={true} />);
});
it('renders nothing by default or when isOpen is false', () => {
const wrapper = mount(<Modal title={'Some Title'} />);
expect(wrapper.html()).toBe(null);
render(<Modal title="Some Title" />);
wrapper.setProps({ ...wrapper.props(), isOpen: false });
expect(wrapper.html()).toBe(null);
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
});
it('renders correct contents', () => {
const wrapper = mount(
<Modal title={'Some Title'} isOpen={true}>
<div id={'modal-content'}>Content</div>
render(
<Modal title="Some Title" isOpen>
<div data-testid="modal-content">Content</div>
</Modal>
);
expect(wrapper.find('div#modal-content').length).toBe(1);
expect(wrapper.contains('Some Title')).toBeTruthy();
expect(screen.getByRole('dialog')).toBeInTheDocument();
expect(screen.getByLabelText('Some Title')).toBeInTheDocument();
expect(screen.getByTestId('modal-content')).toBeInTheDocument();
});
});
packages/grafana-ui/src/components/Modal/Modal.tsx
+27 -32
View File
@@ -1,7 +1,9 @@
import { cx } from '@emotion/css';
import { FocusScope } from '@react-aria/focus';
import { OverlayContainer } from '@react-aria/overlays';
import React, { PropsWithChildren, useCallback, useEffect } from 'react';
import { useDialog } from '@react-aria/dialog';
import { OverlayContainer, useOverlay } from '@react-aria/overlays';
import React, { PropsWithChildren, useRef } from 'react';
import { useTheme2 } from '../../themes';
import { IconName } from '../../types';
@@ -39,33 +41,24 @@ export function Modal(props: PropsWithChildren<Props>) {
closeOnBackdropClick = true,
className,
contentClassName,
onDismiss: propsOnDismiss,
onDismiss,
onClickBackdrop,
trapFocus = true,
} = props;
const theme = useTheme2();
const styles = getModalStyles(theme);
const onDismiss = useCallback(() => {
if (propsOnDismiss) {
propsOnDismiss();
}
}, [propsOnDismiss]);
useEffect(() => {
const onEscKey = (ev: KeyboardEvent) => {
if (ev.key === 'Esc' || ev.key === 'Escape') {
onDismiss();
}
};
if (isOpen && closeOnEscape) {
document.addEventListener('keydown', onEscKey, false);
} else {
document.removeEventListener('keydown', onEscKey, false);
}
return () => {
document.removeEventListener('keydown', onEscKey, false);
};
}, [closeOnEscape, isOpen, onDismiss]);
const ref = useRef<HTMLDivElement>(null);
// Handle interacting outside the dialog and pressing
// the Escape key to close the modal.
const { overlayProps, underlayProps } = useOverlay(
{ isKeyboardDismissDisabled: closeOnEscape, isOpen, onClose: onDismiss },
ref
);
// Get props for the dialog and its title
const { dialogProps, titleProps } = useDialog({}, ref);
if (!isOpen) {
return null;
@@ -78,16 +71,17 @@ export function Modal(props: PropsWithChildren<Props>) {
<div
className={styles.modalBackdrop}
onClick={onClickBackdrop || (closeOnBackdropClick ? onDismiss : undefined)}
{...underlayProps}
/>
<FocusScope contain={trapFocus} autoFocus restoreFocus>
{/*
tabIndex=-1 is needed here to support highlighting text within the modal when using FocusScope
see https://github.com/adobe/react-spectrum/issues/1604#issuecomment-781574668
*/}
<div tabIndex={-1} className={cx(styles.modal, className)}>
<div className={cx(styles.modal, className)} ref={ref} {...overlayProps} {...dialogProps}>
<div className={headerClass}>
{typeof title === 'string' && <DefaultModalHeader {...props} title={title} />}
{typeof title !== 'string' && title}
{typeof title === 'string' && <DefaultModalHeader {...props} title={title} id={titleProps.id} />}
{
// FIXME: custom title components won't get an accessible title.
// Do we really want to support them or shall we just limit this ModalTabsHeader?
typeof title !== 'string' && title
}
<div className={styles.modalHeaderClose}>
<IconButton aria-label="Close dialogue" surface="header" name="times" size="xl" onClick={onDismiss} />
</div>
@@ -130,11 +124,12 @@ function ModalButtonRow({ leftItems, children }: { leftItems?: React.ReactNode;
Modal.ButtonRow = ModalButtonRow;
interface DefaultModalHeaderProps {
id?: string;
title: string;
icon?: IconName;
iconTooltip?: string;
}
function DefaultModalHeader({ icon, iconTooltip, title }: DefaultModalHeaderProps): JSX.Element {
return <ModalHeader icon={icon} iconTooltip={iconTooltip} title={title} />;
function DefaultModalHeader({ icon, iconTooltip, title, id }: DefaultModalHeaderProps): JSX.Element {
return <ModalHeader icon={icon} iconTooltip={iconTooltip} title={title} id={id} />;
}
packages/grafana-ui/src/components/Modal/ModalHeader.tsx
+5 -2
View File
@@ -5,6 +5,7 @@ import { useStyles2 } from '../../themes';
interface Props {
title: string;
id?: string;
/** @deprecated */
icon?: IconName;
/** @deprecated */
@@ -12,12 +13,14 @@ interface Props {
}
/** @internal */
export const ModalHeader: React.FC<Props> = ({ icon, iconTooltip, title, children }) => {
export const ModalHeader: React.FC<Props> = ({ icon, iconTooltip, title, children, id }) => {
const styles = useStyles2(getModalStyles);
return (
<>
<h2 className={styles.modalHeaderTitle}>{title}</h2>
<h2 className={styles.modalHeaderTitle} id={id}>
{title}
</h2>
{children}
</>
);
pkg/api/login_oauth.go
+11 -10
View File
@@ -11,6 +11,8 @@ import (
"net/http"
"net/url"
"golang.org/x/oauth2"
"github.com/grafana/grafana/pkg/api/response"
"github.com/grafana/grafana/pkg/bus"
"github.com/grafana/grafana/pkg/infra/log"
@@ -21,7 +23,6 @@ import (
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/web"
"golang.org/x/oauth2"
)
var (
@@ -132,7 +133,7 @@ func (hs *HTTPServer) OAuthLogin(ctx *models.ReqContext) response.Response {
return nil
}
hashedState := hashStatecode(state, provider.ClientSecret)
hashedState := hs.hashStatecode(state, provider.ClientSecret)
cookies.WriteCookie(ctx.Resp, OauthStateCookieName, hashedState, hs.Cfg.OAuthCookieMaxAge, hs.CookieOptionsFromCfg)
if provider.HostedDomain != "" {
opts = append(opts, oauth2.SetAuthURLParam("hd", provider.HostedDomain))
@@ -155,7 +156,7 @@ func (hs *HTTPServer) OAuthLogin(ctx *models.ReqContext) response.Response {
return nil
}
queryState := hashStatecode(ctx.Query("state"), provider.ClientSecret)
queryState := hs.hashStatecode(ctx.Query("state"), provider.ClientSecret)
oauthLogger.Info("state check", "queryState", queryState, "cookieState", cookieState)
if cookieState != queryState {
hs.handleOAuthLoginError(ctx, loginInfo, LoginError{
@@ -234,7 +235,7 @@ func (hs *HTTPServer) OAuthLogin(ctx *models.ReqContext) response.Response {
return nil
}
loginInfo.ExternalUser = *buildExternalUserInfo(token, userInfo, name)
loginInfo.ExternalUser = *hs.buildExternalUserInfo(token, userInfo, name)
loginInfo.User, err = syncUser(ctx, &loginInfo.ExternalUser, connect)
if err != nil {
hs.handleOAuthLoginErrorWithRedirect(ctx, loginInfo, err)
@@ -265,7 +266,7 @@ func (hs *HTTPServer) OAuthLogin(ctx *models.ReqContext) response.Response {
}
// buildExternalUserInfo returns a ExternalUserInfo struct from OAuth user profile
func buildExternalUserInfo(token *oauth2.Token, userInfo *social.BasicUserInfo, name string) *models.ExternalUserInfo {
func (hs *HTTPServer) buildExternalUserInfo(token *oauth2.Token, userInfo *social.BasicUserInfo, name string) *models.ExternalUserInfo {
oauthLogger.Debug("Building external user info from OAuth user info")
extUser := &models.ExternalUserInfo{
@@ -279,13 +280,13 @@ func buildExternalUserInfo(token *oauth2.Token, userInfo *social.BasicUserInfo,
Groups: userInfo.Groups,
}
if userInfo.Role != "" {
if userInfo.Role != "" && !hs.Cfg.OAuthSkipOrgRoleUpdateSync {
rt := models.RoleType(userInfo.Role)
if rt.IsValid() {
// The user will be assigned a role in either the auto-assigned organization or in the default one
var orgID int64
if setting.AutoAssignOrg && setting.AutoAssignOrgId > 0 {
orgID = int64(setting.AutoAssignOrgId)
if hs.Cfg.AutoAssignOrg && hs.Cfg.AutoAssignOrgId > 0 {
orgID = int64(hs.Cfg.AutoAssignOrgId)
plog.Debug("The user has a role assignment and organization membership is auto-assigned",
"role", userInfo.Role, "orgId", orgID)
} else {
@@ -327,8 +328,8 @@ func syncUser(
return cmd.Result, nil
}
func hashStatecode(code, seed string) string {
hashBytes := sha256.Sum256([]byte(code + setting.SecretKey + seed))
func (hs *HTTPServer) hashStatecode(code, seed string) string {
hashBytes := sha256.Sum256([]byte(code + hs.Cfg.SecretKey + seed))
return hex.EncodeToString(hashBytes[:])
}
pkg/cmd/grafana-cli/commands/secretsmigrations/reencrypt_deks.go
-1
View File
@@ -3,7 +3,6 @@ package secretsmigrations
import (
"context"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/runner"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
"github.com/grafana/grafana/pkg/services/featuremgmt"
pkg/cmd/grafana-cli/commands/secretsmigrations/reencrypt_secrets.go
+145 -60
View File
@@ -4,9 +4,9 @@ import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/runner"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
"github.com/grafana/grafana/pkg/services/featuremgmt"
@@ -17,69 +17,123 @@ import (
"xorm.io/xorm"
)
func (s simpleSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) error {
func (s simpleSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) {
var rows []struct {
Id int
Secret string
Secret []byte
}
if err := sess.Table(s.tableName).Select(fmt.Sprintf("id, %s as secret", s.columnName)).Find(&rows); err != nil {
return err
logger.Warn("Could not find any secret to re-encrypt", "table", s.tableName)
return
}
var anyFailure bool
for _, row := range rows {
if len(row.Secret) == 0 {
continue
}
var (
err error
decoded = []byte(row.Secret)
)
if s.isBase64Encoded {
decoded, err = base64.StdEncoding.DecodeString(row.Secret)
if err != nil {
return err
}
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
decrypted, err := secretsSrv.Decrypt(context.Background(), row.Secret)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encrypted, err := secretsSrv.EncryptWithDBSession(context.Background(), decrypted, secrets.WithoutScope(), sess)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not encrypt secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encoded := string(encrypted)
if s.isBase64Encoded {
encoded = base64.StdEncoding.EncodeToString(encrypted)
}
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ? WHERE id = ?", s.tableName, s.columnName)
if _, err := sess.Exec(updateSQL, encoded, row.Id); err != nil {
return err
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ?, updated = ? WHERE id = ?", s.tableName, s.columnName)
if _, err = sess.Exec(updateSQL, encrypted, nowInUTC(), row.Id); err != nil {
anyFailure = true
logger.Warn("Could not update secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
logger.Infof("Column %s from %s has been re-encrypted successfully\n", s.columnName, s.tableName)
return nil
if anyFailure {
logger.Warn(fmt.Sprintf("Column %s from %s has been re-encrypted with errors", s.columnName, s.tableName))
} else {
logger.Info(fmt.Sprintf("Column %s from %s has been re-encrypted successfully", s.columnName, s.tableName))
}
}
func (s jsonSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) error {
func (s b64Secret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) {
var rows []struct {
Id int
Secret string
}
if err := sess.Table(s.tableName).Select(fmt.Sprintf("id, %s as secret", s.columnName)).Find(&rows); err != nil {
logger.Warn("Could not find any secret to re-encrypt", "table", s.tableName)
return
}
var anyFailure bool
for _, row := range rows {
if len(row.Secret) == 0 {
continue
}
decoded, err := base64.StdEncoding.DecodeString(row.Secret)
if err != nil {
anyFailure = true
logger.Warn("Could not decode base64-encoded secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
if err != nil {
anyFailure = true
logger.Warn("Could not decrypt secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encrypted, err := secretsSrv.EncryptWithDBSession(context.Background(), decrypted, secrets.WithoutScope(), sess)
if err != nil {
anyFailure = true
logger.Warn("Could not encrypt secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encoded := base64.StdEncoding.EncodeToString(encrypted)
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ? WHERE id = ?", s.tableName, s.columnName)
_, err = sess.Exec(updateSQL, encoded, row.Id)
if err != nil {
anyFailure = true
logger.Warn("Could not update secret while re-encrypting it", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
if anyFailure {
logger.Warn(fmt.Sprintf("Column %s from %s has been re-encrypted with errors", s.columnName, s.tableName))
} else {
logger.Info(fmt.Sprintf("Column %s from %s has been re-encrypted successfully", s.columnName, s.tableName))
}
}
func (s jsonSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) {
var rows []struct {
Id int
SecureJsonData map[string][]byte
}
if err := sess.Table(s.tableName).Cols("id", "secure_json_data").Find(&rows); err != nil {
return err
logger.Warn("Could not find any secret to re-encrypt", "table", s.tableName)
return
}
var anyFailure bool
for _, row := range rows {
if len(row.SecureJsonData) == 0 {
continue
@@ -87,29 +141,38 @@ func (s jsonSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Ses
decrypted, err := secretsSrv.DecryptJsonData(context.Background(), row.SecureJsonData)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt secrets while re-encrypting them", "table", s.tableName, "id", row.Id, "error", err)
continue
}
var toUpdate struct {
toUpdate := struct {
SecureJsonData map[string][]byte
}
Updated string
}{Updated: nowInUTC()}
toUpdate.SecureJsonData, err = secretsSrv.EncryptJsonDataWithDBSession(context.Background(), decrypted, secrets.WithoutScope(), sess)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not re-encrypt secrets", "table", s.tableName, "id", row.Id, "error", err)
continue
}
if _, err := sess.Table(s.tableName).Where("id = ?", row.Id).Update(toUpdate); err != nil {
return err
anyFailure = true
logger.Warn("Could not update secrets while re-encrypting them", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
logger.Infof("Secure json data from %s has been re-encrypted successfully\n", s.tableName)
return nil
if anyFailure {
logger.Warn(fmt.Sprintf("Secure json data secrets from %s have been re-encrypted with errors", s.tableName))
} else {
logger.Info(fmt.Sprintf("Secure json data secrets from %s have been re-encrypted successfully", s.tableName))
}
}
func (s alertingSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) error {
func (s alertingSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm.Session) {
var results []struct {
Id int
AlertmanagerConfiguration string
@@ -117,14 +180,19 @@ func (s alertingSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm
selectSQL := "SELECT id, alertmanager_configuration FROM alert_configuration"
if err := sess.SQL(selectSQL).Find(&results); err != nil {
return err
logger.Warn("Could not find any alert_configuration secret to re-encrypt")
return
}
var anyFailure bool
for _, result := range results {
result := result
postableUserConfig, err := notifier.Load([]byte(result.AlertmanagerConfiguration))
if err != nil {
return err
anyFailure = true
logger.Warn("Could not load alert_configuration while re-encrypting it", "id", result.Id, "error", err)
continue
}
for _, receiver := range postableUserConfig.AlertmanagerConfig.Receivers {
@@ -132,17 +200,23 @@ func (s alertingSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm
for k, v := range gmr.SecureSettings {
decoded, err := base64.StdEncoding.DecodeString(v)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decode base64-encoded alert_configuration secret", "id", result.Id, "key", k, "error", err)
continue
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt alert_configuration secret", "id", result.Id, "key", k, "error", err)
continue
}
reencrypted, err := secretsSrv.EncryptWithDBSession(context.Background(), decrypted, secrets.WithoutScope(), sess)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not re-encrypt alert_configuration secret", "id", result.Id, "key", k, "error", err)
continue
}
gmr.SecureSettings[k] = base64.StdEncoding.EncodeToString(reencrypted)
@@ -152,18 +226,24 @@ func (s alertingSecret) reencrypt(secretsSrv *manager.SecretsService, sess *xorm
marshalled, err := json.Marshal(postableUserConfig)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not marshal alert_configuration while re-encrypting it", "id", result.Id, "error", err)
continue
}
result.AlertmanagerConfiguration = string(marshalled)
if _, err := sess.Table("alert_configuration").Where("id = ?", result.Id).Update(&result); err != nil {
return err
anyFailure = true
logger.Warn("Could not update alert_configuration secret while re-encrypting it", "id", result.Id, "error", err)
continue
}
}
logger.Info("Alerting secrets has been re-encrypted successfully\n")
return nil
if anyFailure {
logger.Warn("Alerting configuration secrets have been re-encrypted with errors")
} else {
logger.Info("Alerting configuration secrets have been re-encrypted successfully")
}
}
func ReEncryptSecrets(_ utils.CommandLine, runner runner.Runner) error {
@@ -173,22 +253,27 @@ func ReEncryptSecrets(_ utils.CommandLine, runner runner.Runner) error {
}
toMigrate := []interface {
reencrypt(*manager.SecretsService, *xorm.Session) error
reencrypt(*manager.SecretsService, *xorm.Session)
}{
simpleSecret{tableName: "dashboard_snapshot", columnName: "dashboard_encrypted", isBase64Encoded: false},
simpleSecret{tableName: "user_auth", columnName: "o_auth_access_token", isBase64Encoded: true},
simpleSecret{tableName: "user_auth", columnName: "o_auth_refresh_token", isBase64Encoded: true},
simpleSecret{tableName: "user_auth", columnName: "o_auth_token_type", isBase64Encoded: true},
simpleSecret{tableName: "dashboard_snapshot", columnName: "dashboard_encrypted"},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_access_token"}},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_refresh_token"}},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_token_type"}},
jsonSecret{tableName: "data_source"},
jsonSecret{tableName: "plugin_setting"},
alertingSecret{},
}
return runner.SQLStore.WithDbSession(context.Background(), func(sess *sqlstore.DBSession) error {
for _, m := range toMigrate {
if err := m.reencrypt(runner.SecretsService, sess.Session); err != nil {
return err
return runner.SQLStore.WithTransactionalDbSession(context.Background(), func(sess *sqlstore.DBSession) (err error) {
defer func() {
if r := recover(); r != nil {
err = errors.New(fmt.Sprint(r))
logger.Error("Secrets re-encryption failed, rolling back transaction...", "error", err)
}
}()
for _, m := range toMigrate {
m.reencrypt(runner.SecretsService, sess.Session)
}
return nil
pkg/cmd/grafana-cli/commands/secretsmigrations/rollback_secrets.go
+156 -66
View File
@@ -4,14 +4,13 @@ import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/runner"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/utils"
"github.com/grafana/grafana/pkg/services/encryption"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/ngalert/notifier"
"github.com/grafana/grafana/pkg/services/secrets/manager"
"github.com/grafana/grafana/pkg/services/sqlstore"
@@ -23,14 +22,15 @@ func (s simpleSecret) rollback(
encryptionSrv encryption.Internal,
sess *xorm.Session,
secretKey string,
) error {
) (anyFailure bool) {
var rows []struct {
Id int
Secret string
Secret []byte
}
if err := sess.Table(s.tableName).Select(fmt.Sprintf("id, %s as secret", s.columnName)).Find(&rows); err != nil {
return err
logger.Warn("Could not find any secret to roll back", "table", s.tableName)
return true
}
for _, row := range rows {
@@ -38,42 +38,95 @@ func (s simpleSecret) rollback(
continue
}
var (
err error
decoded = []byte(row.Secret)
)
if s.isBase64Encoded {
decoded, err = base64.StdEncoding.DecodeString(row.Secret)
if err != nil {
return err
}
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
decrypted, err := secretsSrv.Decrypt(context.Background(), row.Secret)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encrypted, err := encryptionSrv.Encrypt(context.Background(), decrypted, secretKey)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not encrypt secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encoded := string(encrypted)
if s.isBase64Encoded {
encoded = base64.StdEncoding.EncodeToString(encrypted)
}
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ? WHERE id = ?", s.tableName, s.columnName)
if _, err := sess.Exec(updateSQL, encoded, row.Id); err != nil {
return err
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ?, updated = ? WHERE id = ?", s.tableName, s.columnName)
if _, err = sess.Exec(updateSQL, encrypted, nowInUTC(), row.Id); err != nil {
anyFailure = true
logger.Warn("Could not update secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
logger.Infof("Column %s from %s have been rolled back successfully\n", s.columnName, s.tableName)
if anyFailure {
logger.Warn(fmt.Sprintf("Column %s from %s has been rolled back with errors", s.columnName, s.tableName))
} else {
logger.Info(fmt.Sprintf("Column %s from %s has been rolled back successfully", s.columnName, s.tableName))
}
return nil
return anyFailure
}
func (s b64Secret) rollback(
secretsSrv *manager.SecretsService,
encryptionSrv encryption.Internal,
sess *xorm.Session,
secretKey string,
) (anyFailure bool) {
var rows []struct {
Id int
Secret string
}
if err := sess.Table(s.tableName).Select(fmt.Sprintf("id, %s as secret", s.columnName)).Find(&rows); err != nil {
logger.Warn("Could not find any secret to roll back", "table", s.tableName)
return true
}
for _, row := range rows {
if len(row.Secret) == 0 {
continue
}
decoded, err := base64.StdEncoding.DecodeString(row.Secret)
if err != nil {
anyFailure = true
logger.Warn("Could not decode base64-encoded secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
if err != nil {
anyFailure = true
logger.Warn("Could not decrypt secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encrypted, err := encryptionSrv.Encrypt(context.Background(), decrypted, secretKey)
if err != nil {
anyFailure = true
logger.Warn("Could not encrypt secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
encoded := base64.StdEncoding.EncodeToString(encrypted)
updateSQL := fmt.Sprintf("UPDATE %s SET %s = ? WHERE id = ?", s.tableName, s.columnName)
if _, err := sess.Exec(updateSQL, encoded, row.Id); err != nil {
anyFailure = true
logger.Warn("Could not update secret while rolling it back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
if anyFailure {
logger.Warn(fmt.Sprintf("Column %s from %s has been rolled back with errors", s.columnName, s.tableName))
} else {
logger.Info(fmt.Sprintf("Column %s from %s has been rolled back successfully", s.columnName, s.tableName))
}
return anyFailure
}
func (s jsonSecret) rollback(
@@ -81,14 +134,15 @@ func (s jsonSecret) rollback(
encryptionSrv encryption.Internal,
sess *xorm.Session,
secretKey string,
) error {
) (anyFailure bool) {
var rows []struct {
Id int
SecureJsonData map[string][]byte
}
if err := sess.Table(s.tableName).Cols("id", "secure_json_data").Find(&rows); err != nil {
return err
logger.Warn("Could not find any secret to roll back", "table", s.tableName)
return true
}
for _, row := range rows {
@@ -98,26 +152,35 @@ func (s jsonSecret) rollback(
decrypted, err := secretsSrv.DecryptJsonData(context.Background(), row.SecureJsonData)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt secrets while rolling them back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
var toUpdate struct {
toUpdate := struct {
SecureJsonData map[string][]byte
}
Updated string
}{Updated: nowInUTC()}
toUpdate.SecureJsonData, err = encryptionSrv.EncryptJsonData(context.Background(), decrypted, secretKey)
if err != nil {
return err
logger.Warn("Could not re-encrypt secrets while rolling them back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
if _, err := sess.Table(s.tableName).Where("id = ?", row.Id).Update(toUpdate); err != nil {
return err
logger.Warn("Could not update secrets while rolling them back", "table", s.tableName, "id", row.Id, "error", err)
continue
}
}
logger.Infof("Secure json data from %s have been rolled back successfully\n", s.tableName)
if anyFailure {
logger.Warn(fmt.Sprintf("Secure json data secrets from %s have been rolled back with errors", s.tableName))
} else {
logger.Info(fmt.Sprintf("Secure json data secrets from %s have been rolled back successfully", s.tableName))
}
return nil
return anyFailure
}
func (s alertingSecret) rollback(
@@ -125,7 +188,7 @@ func (s alertingSecret) rollback(
encryptionSrv encryption.Internal,
sess *xorm.Session,
secretKey string,
) error {
) (anyFailure bool) {
var results []struct {
Id int
AlertmanagerConfiguration string
@@ -133,14 +196,17 @@ func (s alertingSecret) rollback(
selectSQL := "SELECT id, alertmanager_configuration FROM alert_configuration"
if err := sess.SQL(selectSQL).Find(&results); err != nil {
return err
logger.Warn("Could not find any alert_configuration secret to roll back")
return true
}
for _, result := range results {
result := result
postableUserConfig, err := notifier.Load([]byte(result.AlertmanagerConfiguration))
if err != nil {
return err
anyFailure = true
logger.Warn("Could not load configuration (alert_configuration with id: %d) while rolling it back", result.Id, err)
continue
}
for _, receiver := range postableUserConfig.AlertmanagerConfig.Receivers {
@@ -148,17 +214,23 @@ func (s alertingSecret) rollback(
for k, v := range gmr.SecureSettings {
decoded, err := base64.StdEncoding.DecodeString(v)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decode base64-encoded secret (alert_configuration with id: %d, key)", k, result.Id, err)
continue
}
decrypted, err := secretsSrv.Decrypt(context.Background(), decoded)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not decrypt secret (alert_configuration with id: %d, key)", k, result.Id, err)
continue
}
reencrypted, err := encryptionSrv.Encrypt(context.Background(), decrypted, secretKey)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not re-encrypt secret (alert_configuration with id: %d, key)", k, result.Id, err)
continue
}
gmr.SecureSettings[k] = base64.StdEncoding.EncodeToString(reencrypted)
@@ -168,18 +240,26 @@ func (s alertingSecret) rollback(
marshalled, err := json.Marshal(postableUserConfig)
if err != nil {
return err
anyFailure = true
logger.Warn("Could not marshal configuration (alert_configuration with id: %d) while rolling it back", result.Id, err)
continue
}
result.AlertmanagerConfiguration = string(marshalled)
if _, err := sess.Table("alert_configuration").Where("id = ?", result.Id).Update(&result); err != nil {
return err
anyFailure = true
logger.Warn("Could not update secret (alert_configuration with id: %d) while rolling it back", result.Id, err)
continue
}
}
logger.Info("Alerting secrets have rolled re-encrypted successfully\n")
if anyFailure {
logger.Warn("Alerting configuration secrets have been rolled back with errors")
} else {
logger.Info("Alerting configuration secrets have been rolled back successfully")
}
return nil
return anyFailure
}
func RollBackSecrets(_ utils.CommandLine, runner runner.Runner) error {
@@ -188,31 +268,41 @@ func RollBackSecrets(_ utils.CommandLine, runner runner.Runner) error {
return nil
}
toMigrate := []interface {
rollback(*manager.SecretsService, encryption.Internal, *xorm.Session, string) error
toRollback := []interface {
rollback(*manager.SecretsService, encryption.Internal, *xorm.Session, string) bool
}{
simpleSecret{tableName: "dashboard_snapshot", columnName: "dashboard_encrypted", isBase64Encoded: false},
simpleSecret{tableName: "user_auth", columnName: "o_auth_access_token", isBase64Encoded: true},
simpleSecret{tableName: "user_auth", columnName: "o_auth_refresh_token", isBase64Encoded: true},
simpleSecret{tableName: "user_auth", columnName: "o_auth_token_type", isBase64Encoded: true},
simpleSecret{tableName: "dashboard_snapshot", columnName: "dashboard_encrypted"},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_access_token"}},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_refresh_token"}},
b64Secret{simpleSecret{tableName: "user_auth", columnName: "o_auth_token_type"}},
jsonSecret{tableName: "data_source"},
jsonSecret{tableName: "plugin_setting"},
alertingSecret{},
}
return runner.SQLStore.WithDbSession(context.Background(), func(sess *sqlstore.DBSession) error {
for _, m := range toMigrate {
if err := m.rollback(
runner.SecretsService,
runner.EncryptionService,
sess.Session,
runner.Cfg.SecretKey); err != nil {
return err
return runner.SQLStore.WithTransactionalDbSession(context.Background(), func(sess *sqlstore.DBSession) (err error) {
defer func() {
if r := recover(); r != nil {
err = errors.New(fmt.Sprint(r))
logger.Error("Secrets roll back failed, rolling back transaction...", "error", err)
}
}()
var anyFailure bool
for _, r := range toRollback {
if failed := r.rollback(runner.SecretsService, runner.EncryptionService, sess.Session, runner.Cfg.SecretKey); failed {
anyFailure = true
}
}
if _, err := sess.Exec("DELETE FROM data_keys"); err != nil {
logger.Warn("Error while cleaning up data keys table...", "err", err)
if anyFailure {
logger.Warn("Some errors happened, not cleaning up data keys table...")
return nil
}
if _, sqlErr := sess.Exec("DELETE FROM data_keys"); sqlErr != nil {
logger.Warn("Error while cleaning up data keys table...", "error", sqlErr)
}
return nil
pkg/cmd/grafana-cli/commands/secretsmigrations/secretsmigrations.go
+28
View File
@@ -0,0 +1,28 @@
package secretsmigrations
import (
"time"
"github.com/grafana/grafana/pkg/infra/log"
)
type simpleSecret struct {
tableName string
columnName string
}
type b64Secret struct {
simpleSecret
}
type jsonSecret struct {
tableName string
}
type alertingSecret struct{}
func nowInUTC() string {
return time.Now().UTC().Format("2006-01-02 15:04:05")
}
var logger = log.New("secrets.migrations")
pkg/cmd/grafana-cli/commands/secretsmigrations/types.go
-13
View File
@@ -1,13 +0,0 @@
package secretsmigrations
type simpleSecret struct {
tableName string
columnName string
isBase64Encoded bool
}
type jsonSecret struct {
tableName string
}
type alertingSecret struct{}
pkg/services/kmsproviders/kmsproviders.go
+8
View File
@@ -17,3 +17,11 @@ const (
type Service interface {
Provide() (map[secrets.ProviderID]secrets.Provider, error)
}
func NormalizeProviderID(id secrets.ProviderID) secrets.ProviderID {
if id == Legacy {
return Default
}
return id
}
pkg/services/secrets/database/database.go
+30 -5
View File
@@ -6,6 +6,7 @@ import (
"time"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/services/kmsproviders"
"github.com/grafana/grafana/pkg/services/secrets"
"github.com/grafana/grafana/pkg/services/sqlstore"
"xorm.io/xorm"
@@ -100,26 +101,50 @@ func (ss *SecretsStoreImpl) ReEncryptDataKeys(
}
for _, k := range keys {
provider, ok := providers[k.Provider]
provider, ok := providers[kmsproviders.NormalizeProviderID(k.Provider)]
if !ok {
return fmt.Errorf("could not find encryption provider '%s'", k.Provider)
ss.log.Warn(
"Could not find provider to re-encrypt data encryption key",
"key_id", k.Name,
"provider", k.Provider,
)
continue
}
decrypted, err := provider.Decrypt(ctx, k.EncryptedData)
if err != nil {
return err
ss.log.Warn(
"Error while decrypting data encryption key to re-encrypt it",
"key_id", k.Name,
"provider", k.Provider,
"err", err,
)
continue
}
// Updating current data key by re-encrypting it with current provider.
// Accessing the current provider within providers map should be safe.
k.Provider = currProvider
k.Updated = time.Now()
k.EncryptedData, err = providers[currProvider].Encrypt(ctx, decrypted)
if err != nil {
return err
ss.log.Warn(
"Error while re-encrypting data encryption key",
"key_id", k.Name,
"provider", k.Provider,
"err", err,
)
continue
}
if _, err := sess.Table(dataKeysTable).Where("name = ?", k.Name).Update(k); err != nil {
return err
ss.log.Warn(
"Error while re-encrypting data encryption key",
"key_id", k.Name,
"provider", k.Provider,
"err", err,
)
continue
}
}
pkg/services/secrets/manager/manager.go
+2 -10
View File
@@ -48,7 +48,7 @@ func ProvideSecretsService(
logger := log.New("secrets")
enabled := features.IsEnabled(featuremgmt.FlagEnvelopeEncryption)
currentProviderID := normalizeProviderID(secrets.ProviderID(
currentProviderID := kmsproviders.NormalizeProviderID(secrets.ProviderID(
settings.KeyValue("security", "encryption_provider").MustString(kmsproviders.Default),
))
@@ -79,14 +79,6 @@ func ProvideSecretsService(
return s, nil
}
func normalizeProviderID(id secrets.ProviderID) secrets.ProviderID {
if id == kmsproviders.Legacy {
return kmsproviders.Default
}
return id
}
func (s *SecretsService) registerUsageMetrics() {
s.usageStats.RegisterMetricsFunc(func(context.Context) (map[string]interface{}, error) {
usageMetrics := make(map[string]interface{})
@@ -330,7 +322,7 @@ func (s *SecretsService) dataKey(ctx context.Context, name string) ([]byte, erro
}
// 2. decrypt data key
provider, exists := s.providers[normalizeProviderID(dataKey.Provider)]
provider, exists := s.providers[kmsproviders.NormalizeProviderID(dataKey.Provider)]
if !exists {
return nil, fmt.Errorf("could not find encryption provider '%s'", dataKey.Provider)
}
pkg/services/sqlstore/migrations/migrations_test.go
+6 -1
View File
@@ -155,6 +155,11 @@ func TestMigrationLock(t *testing.T) {
func TestMigratorLocking(t *testing.T) {
dbType := getDBType()
testDB := getTestDB(t, dbType)
// skip for SQLite for now since it occasionally fails for not clear reason
// anyway starting migrations concurretly for the same migrator is impossible use case
if dbType == SQLite {
t.Skip()
}
x, err := xorm.NewEngine(testDB.DriverName, testDB.ConnStr)
require.NoError(t, err)
@@ -181,7 +186,7 @@ func TestMigratorLocking(t *testing.T) {
})
}
})
assert.Equal(t, int64(1), errorNum)
assert.Equal(t, int64(1), atomic.LoadInt64(&errorNum))
}
func TestDatabaseLocking(t *testing.T) {
pkg/services/sqlstore/searchstore/builder.go
+21 -5
View File
@@ -115,15 +115,31 @@ func (b *Builder) applyFilters() (ordering string) {
b.params = append(b.params, whereParams...)
}
if len(groups) > 0 {
b.sql.WriteString(fmt.Sprintf(" GROUP BY %s", strings.Join(groups, ", ")))
b.params = append(b.params, groupParams...)
}
if len(orders) < 1 {
orders = append(orders, TitleSorter{}.OrderBy())
}
if len(groups) > 0 {
cols := make([]string, 0, len(orders)+len(groups))
for _, o := range orders {
o := strings.TrimSuffix(o, " DESC")
o = strings.TrimSuffix(o, " ASC")
exists := false
for _, g := range groups {
if g == o {
exists = true
break
}
}
if !exists {
cols = append(cols, o)
}
}
cols = append(cols, groups...)
b.sql.WriteString(fmt.Sprintf(" GROUP BY %s", strings.Join(cols, ", ")))
b.params = append(b.params, groupParams...)
}
orderBy := fmt.Sprintf(" ORDER BY %s", strings.Join(orders, ", "))
b.sql.WriteString(orderBy)
pkg/setting/setting.go
+5 -3
View File
@@ -393,9 +393,10 @@ type Cfg struct {
DefaultTheme string
HomePage string
AutoAssignOrg bool
AutoAssignOrgId int
AutoAssignOrgRole string
AutoAssignOrg bool
AutoAssignOrgId int
AutoAssignOrgRole string
OAuthSkipOrgRoleUpdateSync bool
// ExpressionsEnabled specifies whether expressions are enabled.
ExpressionsEnabled bool
@@ -1256,6 +1257,7 @@ func readAuthSettings(iniFile *ini.File, cfg *Cfg) (err error) {
OAuthAutoLogin = auth.Key("oauth_auto_login").MustBool(false)
cfg.OAuthCookieMaxAge = auth.Key("oauth_state_cookie_max_age").MustInt(600)
SignoutRedirectUrl = valueAsString(auth, "signout_redirect_url", "")
cfg.OAuthSkipOrgRoleUpdateSync = auth.Key("oauth_skip_org_role_update_sync").MustBool(false)
// SigV4
SigV4AuthEnabled = auth.Key("sigv4_auth_enabled").MustBool(false)
pkg/tsdb/cloudmonitoring/test-data/9-series-response-distribution-without-points.json
+17
View File
@@ -0,0 +1,17 @@
{
"timeSeries": [
{
"metric": {
"type": "loadbalancing.googleapis.com\/https\/backend_latencies"
},
"resource": {
"type": "https_lb_rule",
"labels": {
"project_id": "grafana-prod"
}
},
"metricKind": "DELTA",
"valueType": "DISTRIBUTION"
}
]
}
pkg/tsdb/cloudmonitoring/time_series_filter.go
+8 -28
View File
@@ -166,11 +166,10 @@ func (timeSeriesFilter *cloudMonitoringTimeSeriesFilter) parseResponse(queryRes
if len(point.Value.DistributionValue.BucketCounts) == 0 {
continue
}
maxKey := 0
for i := 0; i < len(point.Value.DistributionValue.BucketCounts); i++ {
value, err := strconv.ParseFloat(point.Value.DistributionValue.BucketCounts[i], 64)
if err != nil {
continue
return err
}
if _, ok := buckets[i]; !ok {
// set lower bounds
@@ -193,40 +192,21 @@ func (timeSeriesFilter *cloudMonitoringTimeSeriesFilter) parseResponse(queryRes
valueField,
},
RefID: timeSeriesFilter.RefID,
}
if maxKey < i {
maxKey = i
Meta: &data.FrameMeta{
ExecutedQueryString: executedQueryString,
},
}
}
buckets[i].AppendRow(point.Interval.EndTime, value)
}
for i := 0; i < maxKey; i++ {
if _, ok := buckets[i]; !ok {
bucketBound := calcBucketBound(point.Value.DistributionValue.BucketOptions, i)
additionalLabels := data.Labels{"bucket": bucketBound}
timeField := data.NewField(data.TimeSeriesTimeFieldName, nil, []time.Time{})
valueField := data.NewField(data.TimeSeriesValueFieldName, nil, []float64{})
frameName := formatLegendKeys(series.Metric.Type, defaultMetricName, seriesLabels,
additionalLabels, timeSeriesFilter)
valueField.Name = frameName
valueField.Labels = seriesLabels
setDisplayNameAsFieldName(valueField)
buckets[i] = &data.Frame{
Name: frameName,
RefID: timeSeriesFilter.RefID,
Fields: []*data.Field{
timeField,
valueField,
},
}
}
}
}
for i := 0; i < len(buckets); i++ {
buckets[i].Meta.Custom = customFrameMeta
frames = append(frames, buckets[i])
}
if len(buckets) == 0 {
frames = append(frames, frame)
}
}
if len(response.TimeSeries) > 0 {
dl := timeSeriesFilter.buildDeepLink()
pkg/tsdb/cloudmonitoring/time_series_filter_test.go
+73
View File
@@ -11,6 +11,8 @@ import (
"time"
"github.com/grafana/grafana-plugin-sdk-go/backend"
sdkdata "github.com/grafana/grafana-plugin-sdk-go/data"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -386,6 +388,77 @@ func TestTimeSeriesFilter(t *testing.T) {
require.True(t, ok)
assert.Equal(t, "114250375703598695", labels["resource.label.instance_id"])
})
t.Run("parseResponse successfully parses metadata for distribution valueType", func(t *testing.T) {
t.Run("exponential bounds", func(t *testing.T) {
data, err := loadTestFile("./test-data/3-series-response-distribution-exponential.json")
require.NoError(t, err)
assert.Equal(t, 1, len(data.TimeSeries))
res := &backend.DataResponse{}
require.NoError(t, (&cloudMonitoringTimeSeriesFilter{GroupBys: []string{"test_group_by"}}).parseResponse(res, data, "test_query"))
require.NotNil(t, res.Frames[0].Meta)
assert.Equal(t, sdkdata.FrameMeta{
ExecutedQueryString: "test_query",
Custom: map[string]interface{}{
"groupBys": []string{"test_group_by"},
"alignmentPeriod": "",
"labels": map[string]string{
"resource.label.project_id": "grafana-prod",
"resource.type": "https_lb_rule",
},
"perSeriesAligner": "",
},
}, *res.Frames[0].Meta)
})
t.Run("explicit bounds", func(t *testing.T) {
data, err := loadTestFile("./test-data/4-series-response-distribution-explicit.json")
require.NoError(t, err)
assert.Equal(t, 1, len(data.TimeSeries))
res := &backend.DataResponse{}
require.NoError(t, (&cloudMonitoringTimeSeriesFilter{GroupBys: []string{"test_group_by"}}).parseResponse(res, data, "test_query"))
require.NotNil(t, res.Frames[0].Meta)
assert.Equal(t, sdkdata.FrameMeta{
ExecutedQueryString: "test_query",
Custom: map[string]interface{}{
"groupBys": []string{"test_group_by"},
"alignmentPeriod": "",
"labels": map[string]string{
"resource.label.project_id": "grafana-demo",
"resource.type": "global",
},
"perSeriesAligner": "",
},
}, *res.Frames[0].Meta)
})
t.Run("without series points", func(t *testing.T) {
data, err := loadTestFile("./test-data/3-series-response-distribution-exponential.json")
require.NoError(t, err)
assert.Equal(t, 1, len(data.TimeSeries))
res := &backend.DataResponse{}
require.NoError(t, (&cloudMonitoringTimeSeriesFilter{GroupBys: []string{"test_group_by"}}).parseResponse(res, data, "test_query"))
require.NotNil(t, res.Frames[0].Meta)
assert.Equal(t, sdkdata.FrameMeta{
ExecutedQueryString: "test_query",
Custom: map[string]interface{}{
"groupBys": []string{"test_group_by"},
"alignmentPeriod": "",
"labels": map[string]string{
"resource.label.project_id": "grafana-prod",
"resource.type": "https_lb_rule",
},
"perSeriesAligner": "",
},
}, *res.Frames[0].Meta)
})
})
}
func loadTestFile(path string) (cloudMonitoringResponse, error) {
public/app/core/components/TraceToLogs/KeyValueInput.tsx
+110
View File
@@ -0,0 +1,110 @@
import { css } from '@emotion/css';
import { GrafanaTheme, KeyValue } from '@grafana/data';
import { SegmentInput, useStyles, InlineLabel, Icon } from '@grafana/ui';
import React from 'react';
const EQ_WIDTH = 3; // = 24px in inline label
interface Props {
values: Array<KeyValue<string>>;
onChange: (values: Array<KeyValue<string>>) => void;
id?: string;
keyPlaceholder?: string;
valuePlaceholder?: string;
}
const KeyValueInput = ({
values,
onChange,
id,
keyPlaceholder = 'Key',
valuePlaceholder = 'Value (optional)',
}: Props) => {
const styles = useStyles(getStyles);
return (
<div className={styles.wrapper}>
{values.length ? (
values.map((value, idx) => (
<div className={styles.pair} key={idx}>
<SegmentInput
id={`${id}-key-${idx}`}
placeholder={keyPlaceholder}
value={value.key}
onChange={(e) => {
onChange(
values.map((v, i) => {
if (i === idx) {
v.key = String(e);
}
return v;
})
);
}}
/>
<InlineLabel aria-label="equals" className={styles.operator} width={EQ_WIDTH}>
=
</InlineLabel>
<SegmentInput
id={`${id}-value-${idx}`}
placeholder={valuePlaceholder}
value={value.value}
onChange={(e) => {
onChange(
values.map((v, i) => {
if (i === idx) {
v.value = String(e);
}
return v;
})
);
}}
/>
<button
onClick={() => onChange([...values.slice(0, idx), ...values.slice(idx + 1)])}
className="gf-form-label query-part"
aria-label="Remove tag"
>
<Icon name="times" />
</button>
{idx === values.length - 1 ? (
<button
onClick={() => onChange([...values, { key: '', value: '' }])}
className="gf-form-label query-part"
aria-label="Add tag"
>
<Icon name="plus" />
</button>
) : null}
</div>
))
) : (
<button
onClick={() => onChange([...values, { key: '', value: '' }])}
className="gf-form-label query-part"
aria-label="Add tag"
>
<Icon name="plus" />
</button>
)}
</div>
);
};
export default KeyValueInput;
const getStyles = (theme: GrafanaTheme) => ({
wrapper: css`
display: flex;
flex-direction: column;
gap: ${theme.spacing.xs} 0;
`,
pair: css`
display: flex;
justify-content: start;
align-items: center;
`,
operator: css`
color: ${theme.palette.orange};
`,
});
public/app/core/components/TraceToLogsSettings.tsx → public/app/core/components/TraceToLogs/TraceToLogsSettings.tsx
+58 -8
View File
@@ -3,15 +3,19 @@ import {
DataSourceJsonData,
DataSourcePluginOptionsEditorProps,
GrafanaTheme,
KeyValue,
updateDatasourcePluginJsonDataOption,
} from '@grafana/data';
import { DataSourcePicker } from '@grafana/runtime';
import { InlineField, InlineFieldRow, Input, TagsInput, useStyles, InlineSwitch } from '@grafana/ui';
import React from 'react';
import KeyValueInput from './KeyValueInput';
export interface TraceToLogsOptions {
datasourceUid?: string;
tags?: string[];
mappedTags?: Array<KeyValue<string>>;
mapTagNamesEnabled?: boolean;
spanStartTimeShift?: string;
spanEndTimeShift?: string;
filterByTraceID?: boolean;
@@ -54,19 +58,64 @@ export function TraceToLogsSettings({ options, onOptionsChange }: Props) {
</InlineField>
</InlineFieldRow>
{options.jsonData.tracesToLogs?.mapTagNamesEnabled ? (
<InlineFieldRow>
<InlineField
tooltip="Tags that will be used in the Loki query. Default tags: 'cluster', 'hostname', 'namespace', 'pod'"
label="Tags"
labelWidth={26}
>
<KeyValueInput
keyPlaceholder="Tag"
values={
options.jsonData.tracesToLogs?.mappedTags ??
options.jsonData.tracesToLogs?.tags?.map((tag) => ({ key: tag })) ??
[]
}
onChange={(v) =>
updateDatasourcePluginJsonDataOption({ onOptionsChange, options }, 'tracesToLogs', {
...options.jsonData.tracesToLogs,
mappedTags: v,
})
}
/>
</InlineField>
</InlineFieldRow>
) : (
<InlineFieldRow>
<InlineField
tooltip="Tags that will be used in the Loki query. Default tags: 'cluster', 'hostname', 'namespace', 'pod'"
label="Tags"
labelWidth={26}
>
<TagsInput
tags={options.jsonData.tracesToLogs?.tags}
width={40}
onChange={(tags) =>
updateDatasourcePluginJsonDataOption({ onOptionsChange, options }, 'tracesToLogs', {
...options.jsonData.tracesToLogs,
tags: tags,
})
}
/>
</InlineField>
</InlineFieldRow>
)}
<InlineFieldRow>
<InlineField
tooltip="Tags that will be used in the Loki query. Default tags: 'cluster', 'hostname', 'namespace', 'pod'"
label="Tags"
label="Map tag names"
labelWidth={26}
grow
tooltip="Map trace tag names to log label names. Ex: k8s.pod.name -> pod"
>
<TagsInput
tags={options.jsonData.tracesToLogs?.tags}
width={40}
onChange={(tags) =>
<InlineSwitch
id="mapTagNames"
value={options.jsonData.tracesToLogs?.mapTagNamesEnabled ?? false}
onChange={(event: React.SyntheticEvent<HTMLInputElement>) =>
updateDatasourcePluginJsonDataOption({ onOptionsChange, options }, 'tracesToLogs', {
datasourceUid: options.jsonData.tracesToLogs?.datasourceUid,
tags: tags,
...options.jsonData.tracesToLogs,
mapTagNamesEnabled: event.currentTarget.checked,
})
}
/>
@@ -156,6 +205,7 @@ export function TraceToLogsSettings({ options, onOptionsChange }: Props) {
/>
</InlineField>
</InlineFieldRow>
<InlineFieldRow>
<InlineField label="Loki Search" labelWidth={26} grow tooltip="Use this logs data source to search for traces.">
<InlineSwitch
public/app/core/utils/fetch.ts
+6 -1
View File
@@ -105,7 +105,12 @@ export async function parseResponseBody<T>(
return response.blob() as any;
case 'json':
return response.json();
try {
return await response.json();
} catch (err) {
console.warn(`${response.url} returned an invalid JSON -`, err);
return {} as unknown as T;
}
case 'text':
return response.text() as any;
public/app/features/explore/LogsNavigation.tsx
+2 -2
View File
@@ -81,8 +81,8 @@ function LogsNavigation({
}, [visibleRange, absoluteRange, logsSortOrder, queries, clearCache, addResultsToCache]);
useEffect(() => {
return () => clearCache();
// We can't enforce the eslint rule here because we only want to run when component unmounts.
clearCache();
// We can't enforce the eslint rule here because we only want to run when component is mounted.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
public/app/features/explore/TraceView/TraceView.tsx
+1 -1
View File
@@ -20,7 +20,7 @@ import {
TTraceTimeline,
UIElementsContext,
} from '@jaegertracing/jaeger-ui-components';
import { TraceToLogsData } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsData } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import { getDatasourceSrv } from 'app/features/plugins/datasource_srv';
import { getTimeZone } from 'app/features/profile/state/selectors';
import { StoreState } from 'app/types';
public/app/features/explore/TraceView/createSpanLink.test.ts
+55 -1
View File
@@ -2,7 +2,7 @@ import { DataSourceInstanceSettings, MutableDataFrame } from '@grafana/data';
import { setDataSourceSrv, setTemplateSrv } from '@grafana/runtime';
import { createSpanLinkFactory } from './createSpanLink';
import { TraceSpan } from '@jaegertracing/jaeger-ui-components';
import { TraceToLogsOptions } from '../../../core/components/TraceToLogsSettings';
import { TraceToLogsOptions } from '../../../core/components/TraceToLogs/TraceToLogsSettings';
import { LinkSrv, setLinkSrv } from '../../panel/panellinks/link_srv';
import { TemplateSrv } from '../../templating/template_srv';
@@ -141,6 +141,60 @@ describe('createSpanLinkFactory', () => {
expect(linkDef!.href).toBe('testSpanId');
});
it('handles renamed tags', () => {
const createLink = setupSpanLinkFactory({
mapTagNamesEnabled: true,
mappedTags: [
{ key: 'service.name', value: 'service' },
{ key: 'k8s.pod.name', value: 'pod' },
],
});
expect(createLink).toBeDefined();
const linkDef = createLink!(
createTraceSpan({
process: {
serviceName: 'service',
tags: [
{ key: 'service.name', value: 'serviceName' },
{ key: 'k8s.pod.name', value: 'podName' },
],
},
})
);
expect(linkDef!.href).toBe(
`/explore?left=${encodeURIComponent(
'{"range":{"from":"2020-10-14T01:00:00.000Z","to":"2020-10-14T01:00:01.000Z"},"datasource":"loki1","queries":[{"expr":"{service=\\"serviceName\\", pod=\\"podName\\"}","refId":""}],"panelsState":{}}'
)}`
);
});
it('handles incomplete renamed tags', () => {
const createLink = setupSpanLinkFactory({
mapTagNamesEnabled: true,
mappedTags: [
{ key: 'service.name', value: '' },
{ key: 'k8s.pod.name', value: 'pod' },
],
});
expect(createLink).toBeDefined();
const linkDef = createLink!(
createTraceSpan({
process: {
serviceName: 'service',
tags: [
{ key: 'service.name', value: 'serviceName' },
{ key: 'k8s.pod.name', value: 'podName' },
],
},
})
);
expect(linkDef!.href).toBe(
`/explore?left=${encodeURIComponent(
'{"range":{"from":"2020-10-14T01:00:00.000Z","to":"2020-10-14T01:00:01.000Z"},"datasource":"loki1","queries":[{"expr":"{service.name=\\"serviceName\\", pod=\\"podName\\"}","refId":""}],"panelsState":{}}'
)}`
);
});
});
});
public/app/features/explore/TraceView/createSpanLink.tsx
+17 -5
View File
@@ -3,6 +3,7 @@ import {
DataLink,
dateTime,
Field,
KeyValue,
mapInternalLinkToExplore,
rangeUtil,
SplitOpen,
@@ -11,7 +12,7 @@ import {
import { getTemplateSrv } from '@grafana/runtime';
import { Icon } from '@grafana/ui';
import { SpanLinkDef, SpanLinkFunc, TraceSpan } from '@jaegertracing/jaeger-ui-components';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import { getDatasourceSrv } from 'app/features/plugins/datasource_srv';
import React from 'react';
import { LokiQuery } from '../../../plugins/datasource/loki/types';
@@ -122,11 +123,22 @@ function legacyCreateSpanLinkFactory(splitOpenFn: SplitOpen, traceToLogsOptions?
const defaultKeys = ['cluster', 'hostname', 'namespace', 'pod'];
function getLokiQueryFromSpan(span: TraceSpan, options: TraceToLogsOptions): string {
const { tags: keys, filterByTraceID, filterBySpanID } = options;
const keysToCheck = keys?.length ? keys : defaultKeys;
const { tags: keys, filterByTraceID, filterBySpanID, mapTagNamesEnabled, mappedTags } = options;
// In order, try to use mapped tags -> tags -> default tags
const keysToCheck = mapTagNamesEnabled && mappedTags?.length ? mappedTags : keys?.length ? keys : defaultKeys;
// Build tag portion of query
const tags = [...span.process.tags, ...span.tags].reduce((acc, tag) => {
if (keysToCheck.includes(tag.key)) {
acc.push(`${tag.key}="${tag.value}"`);
if (mapTagNamesEnabled) {
const keyValue = (keysToCheck as KeyValue[]).find((keyValue: KeyValue) => keyValue.key === tag.key);
if (keyValue) {
acc.push(`${keyValue.value ? keyValue.value : keyValue.key}="${tag.value}"`);
}
} else {
if ((keysToCheck as string[]).includes(tag.key)) {
acc.push(`${tag.key}="${tag.value}"`);
}
}
return acc;
}, [] as string[]);
public/app/features/plugins/admin/state/actions.ts
+4
View File
@@ -73,6 +73,8 @@ export const install = createAsyncThunk(
return { id, changes } as Update<CatalogPlugin>;
} catch (e) {
console.error(e);
return thunkApi.rejectWithValue('Unknown error.');
}
}
@@ -90,6 +92,8 @@ export const uninstall = createAsyncThunk(`${STATE_PREFIX}/uninstall`, async (id
changes: { isInstalled: false, installedVersion: undefined },
} as Update<CatalogPlugin>;
} catch (e) {
console.error(e);
return thunkApi.rejectWithValue('Unknown error.');
}
});
public/app/plugins/datasource/grafana/components/AnnotationQueryEditor.tsx
+2 -2
View File
@@ -88,7 +88,7 @@ export default function AnnotationQueryEditor({ query, onChange }: Props) {
onChange={onMaxLimitChange}
/>
</Field>
{type === GrafanaAnnotationType.Tags && tags && (
{type === GrafanaAnnotationType.Tags && (
<>
<Field label="Match any" description={matchTooltipContent}>
<Switch id="grafana-annotations__match-any" value={matchAny} onChange={onMatchAnyChange} />
@@ -100,7 +100,7 @@ export default function AnnotationQueryEditor({ query, onChange }: Props) {
inputId="grafana-annotations__tags"
onChange={onTagsChange}
tagOptions={getAnnotationTags}
tags={tags}
tags={tags ?? []}
/>
</Field>
</>
public/app/plugins/datasource/jaeger/components/ConfigEditor.tsx
+1 -1
View File
@@ -1,7 +1,7 @@
import { DataSourcePluginOptionsEditorProps } from '@grafana/data';
import { DataSourceHttpSettings } from '@grafana/ui';
import { NodeGraphSettings } from 'app/core/components/NodeGraphSettings';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import React from 'react';
export type Props = DataSourcePluginOptionsEditorProps;
public/app/plugins/datasource/tempo/QueryEditor/QueryField.tsx
+1 -1
View File
@@ -12,7 +12,7 @@ import {
Themeable2,
withTheme2,
} from '@grafana/ui';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import React from 'react';
import { LokiQueryField } from '../../loki/components/LokiQueryField';
import { LokiQuery } from '../../loki/types';
public/app/plugins/datasource/tempo/configuration/ConfigEditor.tsx
+1 -1
View File
@@ -1,6 +1,6 @@
import { DataSourcePluginOptionsEditorProps } from '@grafana/data';
import { DataSourceHttpSettings } from '@grafana/ui';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import React from 'react';
import { ServiceGraphSettings } from './ServiceGraphSettings';
import { config } from '@grafana/runtime';
public/app/plugins/datasource/tempo/datasource.ts
+1 -1
View File
@@ -10,7 +10,7 @@ import {
isValidGoDuration,
LoadingState,
} from '@grafana/data';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsOptions } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import { config, BackendSrvRequest, DataSourceWithBackend, getBackendSrv } from '@grafana/runtime';
import { serializeParams } from 'app/core/utils/fetch';
import { getDatasourceSrv } from 'app/features/plugins/datasource_srv';
public/app/plugins/datasource/zipkin/ConfigEditor.tsx
+1 -1
View File
@@ -1,7 +1,7 @@
import { DataSourcePluginOptionsEditorProps } from '@grafana/data';
import { DataSourceHttpSettings } from '@grafana/ui';
import { NodeGraphSettings } from 'app/core/components/NodeGraphSettings';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogsSettings';
import { TraceToLogsSettings } from 'app/core/components/TraceToLogs/TraceToLogsSettings';
import React from 'react';
export type Props = DataSourcePluginOptionsEditorProps;
scripts/build/ci-build/Dockerfile
+1 -1
View File
@@ -110,7 +110,7 @@ RUN rm dockerize-linux-amd64-v${DOCKERIZE_VERSION}.tar.gz
# Use old Debian (this has support into 2022) in order to ensure binary compatibility with older glibc's.
FROM debian:stretch-20210208
ENV GOVERSION=1.17.6 \
ENV GOVERSION=1.17.7 \
PATH=/usr/local/go/bin:$PATH \
GOPATH=/go \
NODEVERSION=16.13.0-1nodesource1 \
scripts/drone/pipelines/docs.star
+52
View File
@@ -0,0 +1,52 @@
load(
'scripts/drone/steps/lib.star',
'initialize_step',
'download_grabpl_step',
'lint_frontend_step',
'codespell_step',
'shellcheck_step',
'build_frontend_step',
'test_frontend_step',
'build_storybook_step',
'build_frontend_docs_step',
'build_docs_website_step',
)
load(
'scripts/drone/services/services.star',
'integration_test_services',
'ldap_service',
)
load(
'scripts/drone/utils/utils.star',
'pipeline',
)
def docs_pipelines(edition, ver_mode, trigger):
steps = [download_grabpl_step()] + initialize_step(edition, platform='linux', ver_mode=ver_mode)
# Insert remaining steps
steps.extend([
build_frontend_docs_step(edition=edition),
build_docs_website_step(),
])
return pipeline(
name='{}-docs'.format(ver_mode), edition=edition, trigger=trigger, services=[], steps=steps,
)
def trigger_docs():
return {
'event': [
'pull_request',
],
'paths': {
'include': [
'docs/**',
'packages/**',
],
},
}
scripts/drone/pipelines/main.star
+6 -2
View File
@@ -18,7 +18,6 @@ load(
'e2e_tests_step',
'e2e_tests_artifacts',
'build_storybook_step',
'build_frontend_docs_step',
'copy_packages_for_docker_step',
'build_docker_images_step',
'publish_images_step',
@@ -55,6 +54,11 @@ load(
'drone_change_template',
)
load(
'scripts/drone/pipelines/docs.star',
'docs_pipelines',
)
ver_mode = 'main'
def get_steps(edition, is_downstream=False):
@@ -110,7 +114,6 @@ def get_steps(edition, is_downstream=False):
store_storybook_step(edition=edition, ver_mode=ver_mode),
test_a11y_frontend_step(ver_mode=ver_mode, edition=edition),
frontend_metrics_step(edition=edition),
build_frontend_docs_step(edition=edition),
copy_packages_for_docker_step(),
build_docker_images_step(edition=edition, ver_mode=ver_mode, publish=False),
build_docker_images_step(edition=edition, ver_mode=ver_mode, ubuntu=True, publish=False),
@@ -171,6 +174,7 @@ def main_pipelines(edition):
integration_test_steps.append(benchmark_ldap_step())
pipelines = [
docs_pipelines(edition, ver_mode, trigger),
pipeline(
name='main-test', edition=edition, trigger=trigger, services=[],
steps=[download_grabpl_step()] + initialize_step(edition, platform='linux', ver_mode=ver_mode) + test_steps,
scripts/drone/pipelines/pr.star
+15 -6
View File
@@ -18,8 +18,6 @@ load(
'e2e_tests_step',
'e2e_tests_artifacts',
'build_storybook_step',
'build_frontend_docs_step',
'build_docs_website_step',
'copy_packages_for_docker_step',
'build_docker_images_step',
'postgres_integration_tests_step',
@@ -47,6 +45,12 @@ load(
'drone_change_template',
)
load(
'scripts/drone/pipelines/docs.star',
'docs_pipelines',
'trigger_docs',
)
ver_mode = 'pr'
def pr_pipelines(edition):
@@ -100,8 +104,6 @@ def pr_pipelines(edition):
e2e_tests_artifacts(edition=edition),
build_storybook_step(edition=edition, ver_mode=ver_mode),
test_a11y_frontend_step(ver_mode=ver_mode, edition=edition),
build_frontend_docs_step(edition=edition),
build_docs_website_step(),
copy_packages_for_docker_step(),
build_docker_images_step(edition=edition, ver_mode=ver_mode, archs=['amd64',]),
])
@@ -116,7 +118,14 @@ def pr_pipelines(edition):
])
trigger = {
'event': ['pull_request',],
'event': [
'pull_request',
],
'paths': {
'exclude': [
'docs/**',
],
},
}
return [
@@ -130,5 +139,5 @@ def pr_pipelines(edition):
name='pr-integration-tests', edition=edition, trigger=trigger, services=services,
steps=[download_grabpl_step()] + integration_test_steps,
volumes=volumes,
),
), docs_pipelines(edition, ver_mode, trigger_docs())
]
scripts/drone/steps/lib.star
+17 -8
View File
@@ -1,7 +1,7 @@
load('scripts/drone/vault.star', 'from_secret', 'github_token', 'pull_secret', 'drone_token', 'prerelease_bucket')
grabpl_version = 'v2.9.2'
build_image = 'grafana/build-container:1.4.9'
grabpl_version = 'v2.9.5'
build_image = 'grafana/build-container:1.5.1'
publish_image = 'grafana/grafana-ci-deploy:1.3.1'
deploy_docker_image = 'us.gcr.io/kubernetes-dev/drone/plugins/deploy-image'
alpine_image = 'alpine:3.15'
@@ -64,12 +64,20 @@ def initialize_step(edition, platform, ver_mode, is_downstream=False, install_de
if ver_mode == 'release':
committish = '${DRONE_TAG}'
source_commit = ' ${DRONE_TAG}'
environment = {
'GITHUB_TOKEN': from_secret(github_token),
}
token = "--github-token $${GITHUB_TOKEN}"
elif ver_mode == 'release-branch':
committish = '${DRONE_BRANCH}'
environment = {}
token = ""
else:
environment = {}
if is_downstream:
source_commit = ' $${SOURCE_COMMIT}'
committish = '${DRONE_COMMIT}'
token = ""
steps = [
identify_runner,
clone_enterprise(committish),
@@ -79,14 +87,12 @@ def initialize_step(edition, platform, ver_mode, is_downstream=False, install_de
'depends_on': [
'clone-enterprise',
],
'environment': {
'GITHUB_TOKEN': from_secret(github_token),
},
'environment': environment,
'commands': [
'mv bin/grabpl /tmp/',
'rmdir bin',
'mv grafana-enterprise /tmp/',
'/tmp/grabpl init-enterprise --github-token $${{GITHUB_TOKEN}} /tmp/grafana-enterprise{}'.format(source_commit),
'/tmp/grabpl init-enterprise {} /tmp/grafana-enterprise{}'.format(token, source_commit),
'mv /tmp/grafana-enterprise/deployment_tools_config.json deployment_tools_config.json',
'mkdir bin',
'mv /tmp/grabpl bin/'
@@ -183,7 +189,7 @@ def enterprise_downstream_step(edition):
'grafana/grafana-enterprise@main',
],
'params': [
'SOURCE_BUILD_NUMBER=${DRONE_BUILD_NUMBER}',
'SOURCE_BUILD_NUMBER=${DRONE_COMMIT}',
'SOURCE_COMMIT=${DRONE_COMMIT}',
],
},
@@ -421,9 +427,12 @@ def build_frontend_docs_step(edition):
'name': 'build-frontend-docs',
'image': build_image,
'depends_on': [
'build-frontend'
'initialize'
],
'commands': [
'yarn packages:build',
'yarn packages:docsExtract',
'yarn packages:docsToMarkdown',
'./scripts/ci-reference-docs-lint.sh ci',
]
}
yarn.lock
+13 -12
View File
@@ -1006,6 +1006,18 @@ __metadata:
languageName: node
linkType: hard
"@babel/plugin-proposal-class-properties@npm:7.16.7, @babel/plugin-proposal-class-properties@npm:^7.16.7":
version: 7.16.7
resolution: "@babel/plugin-proposal-class-properties@npm:7.16.7"
dependencies:
"@babel/helper-create-class-features-plugin": ^7.16.7
"@babel/helper-plugin-utils": ^7.16.7
peerDependencies:
"@babel/core": ^7.0.0-0
checksum: 3977e841e17b45b47be749b9a5b67b9e8b25ff0840f9fdad3f00cbcb35db4f5ff15f074939fe19b01207a29688c432cc2c682351959350834d62920b7881f803
languageName: node
linkType: hard
"@babel/plugin-proposal-class-properties@npm:^7.12.1, @babel/plugin-proposal-class-properties@npm:^7.13.0, @babel/plugin-proposal-class-properties@npm:^7.14.5":
version: 7.14.5
resolution: "@babel/plugin-proposal-class-properties@npm:7.14.5"
@@ -1018,18 +1030,6 @@ __metadata:
languageName: node
linkType: hard
"@babel/plugin-proposal-class-properties@npm:^7.16.7":
version: 7.16.7
resolution: "@babel/plugin-proposal-class-properties@npm:7.16.7"
dependencies:
"@babel/helper-create-class-features-plugin": ^7.16.7
"@babel/helper-plugin-utils": ^7.16.7
peerDependencies:
"@babel/core": ^7.0.0-0
checksum: 3977e841e17b45b47be749b9a5b67b9e8b25ff0840f9fdad3f00cbcb35db4f5ff15f074939fe19b01207a29688c432cc2c682351959350834d62920b7881f803
languageName: node
linkType: hard
"@babel/plugin-proposal-class-static-block@npm:^7.15.4":
version: 7.15.4
resolution: "@babel/plugin-proposal-class-static-block@npm:7.15.4"
@@ -19572,6 +19572,7 @@ __metadata:
resolution: "grafana@workspace:."
dependencies:
"@babel/core": 7.16.7
"@babel/plugin-proposal-class-properties": 7.16.7
"@babel/plugin-proposal-nullish-coalescing-operator": 7.16.7
"@babel/plugin-proposal-object-rest-spread": 7.16.7
"@babel/plugin-proposal-optional-chaining": 7.16.7